Blog

Posted on

How AI Prototyping Can Break Silos in Faith-Tech Products

A year ago, I was in a conference room with a faith-tech startup team watching a product manager and a designer talk past each other. The PM wanted data before committing to any direction. The designer wanted to build something the team could react to. Neither was wrong — but they were stuck, and the feature stalled for weeks because nobody said the obvious thing: “Let’s build a quick prototype and test it.”

That moment crystallized something I’d been seeing across product organizations: silos don’t break down through better meetings. They break down through shared artifacts. When a team can look at the same rough prototype together, the conversation shifts from “whose perspective is right” to “what does this teach us.” AI prototyping tools have made that shift dramatically cheaper and faster — which means the teams that aren’t using them are paying an avoidable coordination tax.

What AI Prototyping Actually Does for a Team

The most valuable thing an AI prototyping tool does is collapse the time between idea and artifact. What used to take a designer a week to wireframe can now take a few hours using tools like Galileo, Uizard, or even well-structured prompting in Figma’s AI features. That speed matters less for the prototype itself and more for what the prototype enables: a shared object the whole team — product, design, engineering, stakeholders — can react to together.

Teresa Torres’ continuous discovery framework makes the argument clearly: prototypes aren’t about showing off ideas. They’re about learning. The faster you can get something in front of a real user — even a rough, unpolished version — the faster you discover whether your assumptions are right. AI prototyping compresses that loop significantly.

In faith-tech contexts specifically, where teams are often small and stakeholders can include everyone from engineering leads to ministry directors, this matters. A prototype that a non-technical stakeholder can react to is worth more than a PRD that only product and engineering understand. It gets everyone speaking the same language before anyone writes a line of code.

The Silo Problem AI Prototyping Actually Solves

Product silos usually persist for one of two reasons: different teams operate from different information, or different teams have different incentives. AI prototyping directly addresses the first. When the same working mockup is in front of a PM, a designer, an engineer, and a ministry stakeholder simultaneously, you rapidly surface where each person’s mental model diverges. That divergence, made visible, is far cheaper to resolve before development than after.

I’ve watched this play out in prayer app development, church management software, and discipleship platforms. In each case, the friction wasn’t disagreement about the product’s mission — it was that nobody had made their assumptions concrete enough for others to react to. A rough AI-generated wireframe of a specific user flow did in one session what three roadmap reviews had failed to do: it gave everyone the same starting point.

The discipline that matters here: keep the prototype simple and provisional. The temptation with good prototyping tools is to overinvest in polish before you’ve learned anything. A polished prototype signals “we’ve decided” when the point is “we’re still figuring this out.” Users interact differently with rough prototypes — they’re more honest, and more willing to suggest changes when the artifact looks unfinished.

Building a Prototyping Habit, Not Just a Prototyping Moment

The real leverage isn’t a single prototyping sprint — it’s making rapid, collaborative prototyping a standard step in every major feature decision. Teams that prototype as a habit stop having the same alignment conversations over and over, because they’ve built a shared practice of making assumptions concrete before debating them.

For faith-tech teams in particular, I’d suggest the following sequencing: before any feature goes on the roadmap, someone on the team produces a one-screen wireframe answering the question “what does success look like for the user here?” That wireframe — rough, provisional, built in an hour — becomes the discussion artifact for the roadmap conversation. It doesn’t answer all the questions. It just makes sure everyone is answering the same questions.

Your Turn: Apply This Today

Whether your team has a formal prototyping practice or none at all, here’s how to start building the habit — starting this week:

  • Identify the next feature your team will debate in roadmap planning. Before the meeting, have one person spend 60 minutes building a rough wireframe using an AI prototyping tool. Bring the wireframe to the meeting instead of a written spec. Watch how the conversation changes.
  • Run a 90-minute silo-breaker session. Put product, design, and at least one engineering rep in a room. Give them a specific user problem, a prototyping tool, and a timer. The output should be a rough, functional prototype — not a polished one. Speed over polish is the rule.
  • Get it in front of three to five real users within 48 hours. Pastors, volunteers, small group leaders — whoever your actual end user is. Your goal is one concrete thing you learned that you didn’t expect. If you can’t identify one, the prototype wasn’t specific enough.
  • Run a 30-minute team debrief after user feedback. What did you learn? What did you assume that turned out to be wrong? What’s the next iteration? Write down three specific things the feedback changed about your direction.
  • Make it a gate, not an option. Decide as a team that any feature above a certain size threshold requires a rough prototype before it can be added to the roadmap. This isn’t about slowing down — it’s about ensuring you’re making decisions from the same picture.
  • Celebrate what the prototype taught you, not just what you shipped. Prototypes that prevent bad features from being built are wins, even if nothing ships. Make sure your team understands that learning is the output — not just working software.

Team alignment and discovery go hand in hand. Opportunity Solution Trees Meet AI Reality at Scale covers how to adapt discovery frameworks when you’re building AI-powered features, and Why Teresa Torres’ Continuous Discovery Cadence Is Breaking Down in the AI Age addresses the structural changes that AI creates for discovery-driven teams.

Struggling with cross-functional alignment or trying to build a more disciplined discovery practice in your faith-tech or product organization? I consult with product leaders on prototyping cadences, team alignment, and the practices that close the gap between roadmap and user reality. Let’s talk.

Posted on

AI Learning for Product Leaders: Stop Chasing Tools, Start Solving Problems

Six months ago, I was coaching a product leader at a faith-tech nonprofit who was drowning in the flood of AI tools hitting his inbox. He’d spent hours watching tutorials, clicking through demos, and trying to “keep up” with every new feature drop. His voice cracked over Zoom as he admitted: “I’m learning a lot, but I’m not getting anywhere.”

He wasn’t lazy. He wasn’t slow. He was caught in the most common trap in AI learning — mistaking tool fluency for strategic capability. He knew how to use the tools. He had no idea which problem to point them at.

Learning AI Without a Purpose Kills Momentum

This pattern shows up everywhere in product organizations right now. Teams attend AI workshops, subscribe to newsletters, build internal sandboxes — and then struggle to translate any of it into shipped product improvements. The bottleneck isn’t knowledge. It’s the absence of a clear problem that the knowledge is meant to serve.

There’s a meaningful distinction between AI literacy and AI effectiveness. Literacy is knowing what the tools can do. Effectiveness is knowing which problem in your specific product context is worth solving with them, and having the judgment to evaluate whether the tool actually helps. Most AI learning programs develop the first. Almost none develop the second.

The result is product leaders who can speak fluently about LLMs, embeddings, and RAG pipelines but can’t answer the question that actually matters for their team: “What is the most important thing AI can do for our users right now?” That gap is expensive. Not just in time and training budget — in team credibility and strategic direction.

The Mental Model That Changes How You Learn

Charlie Munger’s latticework mental model is useful here. His argument was that knowledge organized around real problems is exponentially more useful than knowledge accumulated for its own sake. Every new concept anchors to an existing one, and the whole structure becomes more useful with each addition. Knowledge without a problem to anchor to just drifts.

Applied to AI learning, the implication is direct: start with the problem, not the tool. Before you invest time in learning any AI capability, write down the specific friction point in your product that you’d like it to address. Not “AI could make our onboarding better” — that’s too vague to act on. Something like: “Our support team spends 40% of their time answering questions that could be answered by a well-designed in-product FAQ. Could an AI-assisted feature reduce that?” Now you have a problem worth learning toward.

This reframe does something important: it makes your AI learning time accountable. If you’re three hours into a tutorial and you can’t articulate how it connects to your stated problem, that’s a signal — either the tutorial is the wrong one, or your problem definition isn’t sharp enough yet.

Clear Thinking Beats Endless Learning

The product leaders I’ve seen develop genuine AI effectiveness share a habit: they apply opportunity cost thinking ruthlessly to their learning time. Every hour spent on a tutorial that doesn’t connect to a current product problem is an hour not spent on user interviews, roadmap clarity, or team alignment. The cost is real even when it’s invisible.

This doesn’t mean stopping learning. It means changing the sequence. Identify the problem first. Map the assumptions behind the problem. Then go find the specific AI capability that addresses it — and learn only that, deeply, before moving on. You’ll learn slower in volume and faster in applicability. And applicability is what actually ships.

One practical reframe I use with teams: replace “keep up with AI” as a goal with “improve one specific outcome in the next 30 days using AI.” The first goal is a treadmill. The second is a target. Targets produce movement; treadmills produce fatigue.

Your Turn: Apply This Today

Before you open the next AI newsletter or sit through the next demo, run through this checklist to make sure your AI learning is anchored to something real:

  • Write down your current product’s top three friction points. Not AI opportunities — actual user or team pain points. Any AI learning you do this month should connect directly to one of these three items.
  • Cancel or pause one AI learning subscription that you can’t connect to a current problem. The mental overhead of processing irrelevant information is a real cost. Reduce the noise before adding more signal.
  • If you’re learning an AI tool without a clear problem to solve, pause and redirect that time to user interviews. Thirty minutes of user conversation will point your AI investment better than most tutorials will.
  • Set a 30-minute timer this week to map a mental model for your product’s biggest gap. Start with the end goal (e.g., higher retention, faster activation) and work backward to the root cause. That root cause is where AI learning should be aimed.
  • Test one AI tool or feature against a specific friction point in the next two weeks. Don’t aim for perfection — aim for a measurable result: “Reduce support ticket volume by 10%,” “Cut onboarding time by two steps.” Specificity is what makes the test informative.
  • Share this framing with your team in your next meeting. Walk them through prioritizing mission-relevant learning over tech hype using one real user story. Teams that learn together toward shared problems develop AI effectiveness faster than teams where everyone is self-directing their own curriculum.

If you’re working through where AI fits in your product strategy, Why Your AI Feature Doesn’t Need More Data — It Needs Better Problem Definition addresses how to scope AI features correctly before you write a line of code. And Munger’s Mental Models and AI Product Decisions goes deeper on the latticework framework applied to product leadership.

Working through how to make your team’s AI investment actually move the needle on outcomes that matter? I consult with product leaders on AI strategy, learning frameworks, and the organizational dynamics that determine whether AI capability translates into product impact. Let’s talk.

Posted on

What Retention Data Actually Tells You (And What It Doesn’t)

The dashboard showed 62% 30-day retention. The executive team thought that was good. The product team thought it meant users loved the core experience. The data team thought the metric was clean. Everyone was wrong in the same direction.

They were wrong not because 62% was a bad number — whether it’s good or bad depends entirely on the product category, the user segment, and what “retained” actually means in the calculation. They were wrong because they were treating a lagging indicator as if it explained something, when all it actually did was confirm that something had already happened.

Retention Data Is a Report Card, Not a Diagnosis

This is the foundational misread that causes product teams to misuse retention data: they treat it as causal when it’s correlational, and they treat it as diagnostic when it’s descriptive. Retention tells you that something happened. It doesn’t tell you why. And it certainly doesn’t tell you what to do about it.

When your 30-day retention drops 8 points in a quarter, the number tells you users are leaving faster than they were. It doesn’t tell you whether that’s because of a product change, a cohort quality shift (you acquired different users who were never going to stay), a competitive alternative, a support failure, or a pricing change. You need a different kind of analysis to answer those questions — and most teams don’t do it, because the retention number feels like an answer.

The dangerous consequence: teams fix the wrong thing. They rebuild features, rewrite onboarding, and slash churn in pricing tiers — all without knowing whether any of those things had anything to do with the retention decline. Some of those bets will accidentally work. Most won’t. And the team will draw the wrong lessons from both outcomes.

What Retention Data Actually Tells You

Used correctly, retention data is most valuable for three things: trend detection, cohort comparison, and feature correlation. Each of these is distinct from diagnosis, and each has limits you need to respect.

Trend detection is the most straightforward use. If retention is declining, something changed. If it’s improving, something changed. The number gives you a signal that warrants investigation. It does not give you the investigation’s findings. Too many teams stop at trend detection and start building solutions before they’ve identified the problem.

Cohort comparison is more powerful and more underused. By segmenting retention by acquisition channel, signup period, plan type, or user role, you can identify whether a retention change is universal or concentrated. A 5-point retention decline that’s driven entirely by one acquisition channel is a completely different problem than a 5-point decline that’s uniform across all segments. Cohort comparison gets you from “retention declined” to “retention declined for users who came through paid social but not organic search” — and that’s actually diagnostic.

Feature correlation is the most commonly misused. Teams identify that retained users use feature X more than churned users, and conclude that they should invest in feature X. Sometimes that’s right. Often it’s not — because heavy feature X usage may be a proxy for user sophistication, engagement depth, or use case fit, rather than a direct cause of retention. Correlation between feature usage and retention is a hypothesis, not a finding. You still need to test it.

What Retention Data Doesn’t Tell You

Retention data tells you almost nothing about intent. Users who retain without engaging aren’t loyal — they’re inertial. Users who churn may have gotten exactly what they came for and simply don’t need to come back. Users who return after a lapse may have been re-engaged by a competitor’s failure rather than your improvement.

This is why retention data without qualitative context is incomplete at best and misleading at worst. The number doesn’t contain the story. The story lives in user interviews, support ticket analysis, NPS verbatims, and churn surveys — the qualitative layer that tells you what users were thinking when they left or stayed.

The most important question retention data can’t answer: are you retaining the right users? A product can improve its retention rate while retaining the wrong customers — the ones who require the most support, generate the least revenue, and produce the least word-of-mouth. Retention optimized without an eye on customer quality can actually damage the business while the metric improves.

Building a Retention Intelligence System

The teams that use retention data well have built what I’d call a retention intelligence system — a regular practice of combining quantitative retention metrics with qualitative signals to produce an interpretive picture, not just a number.

It has four components. First, a retention dashboard that breaks down the top-line number by the segments that matter for your business — acquisition channel, plan type, user role, industry, and activation status. Second, a weekly churn review where someone reads and categorizes every churn reason collected that week. Third, a monthly user interview series specifically with churned users — not just retained ones — to understand what didn’t work. Fourth, a feature correlation analysis run quarterly, with explicit hypotheses tested via experiment rather than assumed from correlation.

This system doesn’t replace the retention metric. It gives the metric meaning. And meaning is what product decisions actually require.

Your Turn: Apply This Today

Here’s how to move your team from tracking retention to actually understanding it — without needing to overhaul your entire analytics stack:

  • Add one cohort cut to your retention report this week. Break your retention number down by acquisition channel, plan type, or user role — whichever is most relevant to your current questions. A single segmentation often reveals more than months of top-line trend watching.
  • Write down what you believe is causing your current retention rate. Be specific. List the top three factors you think explain it. Then ask: what evidence do I actually have for each of these? This exercise usually surfaces how much of your current retention narrative is assumption versus finding.
  • Interview five churned users this month. Not NPS detractors — actual churned users. Ask them: “What were you hoping this product would do for you, and what happened instead?” Their answers will tell you things your data never will.
  • Audit your “retained” definition. How are you calculating retention? Are logins enough? Do users have to complete a core action? Are you excluding trials? Different definitions produce dramatically different numbers — and the wrong definition can make a sick product look healthy.
  • Turn your top feature correlation hypothesis into an experiment. If you believe that users who use feature X retain better, design a test that increases feature X exposure for a new cohort and measures retention against a control. Correlation becomes conviction only after you test it.
  • Ask whether you’re retaining the right users. Map your retained users against your ideal customer profile. Are your best-retained users also your highest-value users? If not, your retention optimization may be pointing in the wrong direction — and that’s a strategic conversation worth having before your next sprint.

Retention analysis doesn’t exist in isolation — it’s downstream of activation and upstream of growth. The Activation Problem covers how early user experience shapes the retention curve, and What Nobody Tells You About Product-Led Growth addresses how retention dynamics shift when your growth model depends on users discovering value organically.

Trying to understand why your retention metrics aren’t moving despite product investments? I consult with product leaders on analytics strategy, user research design, and the frameworks that turn data into decisions. Let’s talk.

View all posts