Blog

Last year, I nearly blew a product budget on an AI tool that promised to transform user engagement. I fell for the slick demo, the polished pitch deck, and the assumption that token costs were negligible. Three months in, the monthly AI infrastructure bill had quietly grown to something that required an uncomfortable conversation with finance — and the engagement lift didn’t justify it.

I’m not alone in this. AI cost management has become one of the most common and least-discussed problems in product organizations. Teams scope AI features based on demo performance and initial estimates, then watch costs scale unexpectedly as usage grows. By the time the problem is visible, it’s already a crisis — because it’s now entangled with a shipped feature that users depend on.

Why AI Costs Are Harder to Predict Than Other Infrastructure

Traditional infrastructure costs scale in relatively predictable ways — more users means more compute, and the relationship is roughly linear. AI costs don’t work that way. Token pricing for LLM-based features can vary by orders of magnitude depending on model selection, prompt design, context window usage, and whether you’re doing inference, fine-tuning, or both. A feature that costs $200/month at 1,000 users might cost $40,000/month at 100,000 users — not because of linear scaling, but because of how the feature was architected.

This unpredictability creates a specific trap for product teams: they validate the feature at small scale, ship it, and don’t discover the cost problem until they’re well past the point of easy architectural change. The feature works. Users like it. And it’s quietly becoming the most expensive line item in the infrastructure budget.

In faith-tech and mission-driven organizations especially, where margins are thin and every dollar is tied to a stated mission, this trap is particularly dangerous. The organizations that get this wrong don’t just waste money — they create a credibility problem for AI investment overall, making it harder to fund the next initiative that might actually matter.

The AI Budget Discipline Most Product Teams Skip

The fix is a cost architecture conversation that needs to happen before feature development begins — not after launch. This conversation covers four things: unit economics at scale, model selection tradeoffs, prompt efficiency, and cost monitoring.

Unit economics at scale means estimating the cost per user interaction at 10x and 100x your current scale before you commit to an architecture. If the math doesn’t work at 100x, you either need a different model, a different architecture, or a different feature scope. Finding this out at 1x is cheap. Finding it out at 100x is expensive.

Model selection tradeoffs means being intentional about which model you need for which task. The most capable model is not always the right model. For many product use cases — classification, simple summarization, structured extraction — smaller, cheaper models perform comparably to frontier models at a fraction of the cost. Using GPT-4 class models for tasks that a fine-tuned smaller model could handle is a budget decision masquerading as a technical one.

Prompt efficiency matters because token costs are real. Bloated system prompts, unnecessary context, redundant instructions — these add up at scale. A prompt engineering discipline that optimizes for token efficiency alongside output quality pays dividends as usage grows.

Cost monitoring means treating AI infrastructure costs as a first-class product metric, not just an engineering concern. Product leaders who monitor cost-per-interaction alongside user engagement metrics catch problems early and make better architectural decisions.

Your Turn: Apply This Today

Whether you’re scoping a new AI feature or auditing an existing one, here’s how to build cost discipline into your AI product process:

• Run a cost projection before any AI feature enters development. Estimate the cost per user interaction. Multiply it by your current user base, then by 10x and 100x. If the 100x number is uncomfortable, that’s important information — not a reason to kill the feature, but a reason to make intentional architectural choices now.
• Ask “do we actually need this model?” for every AI implementation. Document the specific capability requirement. Then check whether a cheaper model meets that requirement at acceptable quality. If you haven’t tested a smaller model, you haven’t answered this question.
• Audit your most expensive prompt. Pull the system prompt for your highest-usage AI feature and count the tokens. Look for redundancy, unnecessary context, and instructions that could be simplified. Even a 20% reduction in prompt size compounds significantly at scale.
• Add cost-per-interaction to your product metrics dashboard. It should sit alongside engagement, retention, and error rate. If your team can’t see it, they can’t manage it. Cost visibility changes the conversation about AI feature scope and architecture.
• Set a cost ceiling before launch. Define the maximum acceptable monthly AI infrastructure cost for this feature at your current scale and at 3x scale. Make it explicit in the feature spec. This gives engineering a clear target and surfaces tradeoffs early, when they’re still cheap to resolve.
• Review AI costs quarterly with the same rigor as other infrastructure. AI pricing changes. Model options change. What was the most cost-effective architecture six months ago may not be today. A quarterly review ensures you’re not paying legacy prices for capabilities that have gotten cheaper.

AI budget discipline connects directly to infrastructure strategy. Jensen Huang’s Sovereign AI: Infrastructure Argument for Product Builders covers the build-vs-buy dimension of AI infrastructure decisions. And The Product Leader’s AI Infrastructure Blind Spot addresses the organizational patterns that cause AI infrastructure costs to spiral before anyone catches them.

Managing AI infrastructure costs alongside feature development and trying to make the numbers work? I consult with product leaders on AI strategy, infrastructure decisions, and the budget frameworks that keep AI investment sustainable as products scale. Let’s talk.

I’ve seen AI generate a sermon outline in seconds. I’ve watched it draft a social media calendar in minutes. I’ve used it to analyze engagement data that would have taken a team a week to process manually. And in every one of those cases, the value didn’t come from the AI. It came from a person who knew which problem to point it at, how to evaluate whether the output was any good, and what to do with what it found.

The “AI will replace product managers / pastors / tech leaders” conversation misses the actual dynamic. AI doesn’t replace judgment. It makes judgment more consequential. The leaders who understand this are pulling ahead. The ones waiting to see if they’ll be replaced are falling behind — not to AI, but to the people who learned to work with it.

Automation vs. Augmentation: The Distinction That Matters

The replacement anxiety is driven by a conflation of two very different things: automation and augmentation. Automation replaces a task. Augmentation improves the quality or speed of a decision. Most of what AI does in product and ministry contexts is augmentation, not automation — and augmentation doesn’t reduce the need for the person doing the work. It raises the bar for what that person can accomplish.

When AI drafts a sermon outline, a good pastor isn’t replaced — they now have more time to do the thing AI can’t do: knowing the widow in row three, understanding the weight of what the congregation is carrying this week, and delivering something that actually meets them there. The draft is scaffolding. The sermon is still fully human.

When AI surfaces a pattern in support tickets, a good product manager isn’t replaced — they have faster access to a signal that used to take weeks to surface. The decision about what to do with that signal still requires product judgment, user empathy, and strategic context that no model has. The analysis is faster. The judgment is still theirs.

What AI Actually Replaces

Here’s the honest version: AI does replace some things. It replaces the first draft of documents that don’t require original insight. It replaces basic data aggregation that humans were doing slowly and inconsistently. It replaces repetitive categorization tasks that consumed analyst time without producing analyst thinking. These are real things — and losing them to AI is, on balance, good. They were preventing people from doing their best work.

What AI doesn’t replace: the ability to define which problem is worth solving. The judgment to evaluate whether an AI output is trustworthy in a specific context. The relationships and credibility that make a recommendation worth following. The courage to make a call when the data is ambiguous. The discernment — in ministry contexts especially — to know when the technically correct answer is the wrong pastoral choice.

These are the capabilities that compound with AI rather than competing with it. The more AI handles the mechanical, the more valuable human judgment becomes. Which means the leaders who invest in judgment — in pattern recognition, in mental models, in knowing their users deeply — are the ones whose value increases as AI capabilities improve.

The Trap: Outsourcing Ownership, Not Just Tasks

The real risk isn’t that AI replaces product leaders or ministry leaders. It’s that leaders outsource ownership to AI alongside the tasks. There’s a meaningful difference between “AI drafted this and I refined it” and “AI drafted this and I shipped it.” The first is augmentation. The second is abdication.

I’ve watched this happen in both product organizations and faith-tech teams: leaders hand first drafts to AI and stop interrogating the output. The quality degrades slowly, then quickly. Users notice before leaders do. And when something goes wrong — a feature that misses the user’s actual need, a message that lands badly with the congregation — there’s no accountability structure, because nobody owns the decision.

Agency over automation means staying in the decision loop even when AI is doing the heavy lifting. It means knowing enough about what your AI tools are doing to evaluate their output critically. It means treating AI-assisted work as your work — with your name on it and your judgment applied to it.

Your Turn: Apply This Today

Here’s how to build an “agency over automation” practice that keeps you in the decision loop as AI takes over more of the mechanical work:

• Audit what you’re outsourcing vs. what you’re augmenting. For every AI tool you currently use, ask: am I using this to do my thinking faster, or to avoid thinking? The first is augmentation. The second is a skill you’re letting atrophy. Be honest about which is which.
• Always apply one critical edit to every AI output before it ships. Not to prove you’re involved — to stay in the judgment loop. If you can’t identify something worth changing or adding, the output isn’t being evaluated critically enough. Friction in evaluation is a feature, not a bug.
• Name the judgment call in every AI-assisted decision. “AI surfaced this pattern; I decided to prioritize it because [reason].” This keeps ownership explicit and builds a decision log that’s useful when you need to explain your reasoning later.
• Invest in the capabilities AI doesn’t replace. User empathy. Mental models. Deep domain knowledge. Relationship credibility. These compound with AI rather than competing with it. Wherever you’re currently underinvesting in these, that’s where your development time should go.
• Set one “AI-free” decision this week. Pick a product or strategic question and work through it without prompting a model first. Notice what you know, what you don’t, and where your judgment is weakest. That map is where you need to grow.
• Talk about AI augmentation explicitly with your team. Not as a policy — as a culture conversation. “We use AI to do [tasks] faster. We own the judgment on [decisions]. Here’s the line.” Teams that have this conversation explicitly are more resilient to the drift toward outsourcing that happens when nobody names the boundary.

Agency and ownership in AI contexts are closely related to how you build team culture around AI adoption. Why Agency Trumps Skills for AI-Driven Church Tech Teams covers the ownership dynamic from a team-building angle. And Why AI Decision-Making Needs Human Judgment: The Solomon Test explores where AI judgment breaks down and human judgment becomes non-negotiable.

Thinking through how to keep human judgment at the center of an AI-augmented product or ministry organization? I consult with product leaders and ministry innovators on AI strategy, team culture, and the practices that keep people — not tools — in the decision loop. Let’s talk.