Blog

Dear PM handed the AI mandate at a denomination that still runs on paper calendars and Sunday volunteers,

You walked in thinking your job was to pick the model and set the guardrails. Instead you now decide what counts as acceptable output for the children’s ministry volunteer who has seven minutes between the end of the service and the moment the first kid walks through the door.

That decision sits heavier than any prompt you will write this quarter.

Charlie Munger’s latticework of mental models forces you to run the same choice through second-order effects, inversion, and redundancy before you call it taste. Most PMs stop after the first model. They judge output by how clean it reads on their own screen. The latticework requires you to add the model of the actual handoff: what breaks when the output reaches someone who never chose to be a prompt engineer.

The spec step that collapsed first

The first spec you inherited assumed the AI would handle the full lesson flow. It listed tone, length, and scripture references, then stopped. No one wrote down what happens when the volunteer prints page three and discovers the activity needs scissors the supply closet does not have.

Munger would call this a missing mental model. You cannot judge taste until you invert the flow and ask what the volunteer must now do by hand because the model omitted it. In one rollout the missing model was “print margins for a 1998 copier.” The taste looked polished until the first Sunday the margins cut off the prayer.

Taste without Sunday validation loops

Taste becomes expensive when it never meets the person who will use it. You can iterate prompts for weeks and still ship something that adds friction at 9:15 on Saturday night. The latticework exposes the gap: you are optimizing inside one discipline (model behavior) while ignoring the adjacent discipline (physical volunteer workflow).

The curriculum product I worked on learned this the hard way. Early AI drafts scored high on internal review for warmth and clarity. They failed the moment a volunteer tried to cut the craft in half to fit the table size. The second-order cost was not bad content. It was lost trust the next time leadership asked that same volunteer to try anything new.

The routing decision that actually protects presence

The real product decision is not which model to call. It is which outputs you route to a human before they reach the volunteer. Munger’s redundancy model applies here. You build a quick human checkpoint only on the elements that touch physical handoff: supply lists, timing, and anything that must survive a printer jam.

One team set the rule that any activity requiring more than two physical objects had to pass through a former volunteer before the AI version was approved. The checkpoint added four hours to the cycle. It removed an average of eleven minutes of volunteer rework on Sunday. The math favored the checkpoint once you measured against actual handoff friction instead of token spend.

Your Turn: Apply This Today

• Pull the last three AI-generated specs that went to volunteers and list every physical object or timing assumption each one made.
• Time one volunteer completing the print-and-prep step for each of those three outputs; note the actual minutes against what the spec claimed.
• Mark which of the three outputs required an extra supply run or format fix that the model did not flag.
• Write the missing mental model for each failure in one sentence, using the exact handoff moment as the subject.
• Route the next spec through a single former volunteer before it reaches the model, limiting their review to physical objects and timing only.
• Log the delta in volunteer completion time for the next three Sundays and keep the log in the same place you track token costs.

The PM Role That Stopped Shipping and Started Choosing showed what happens when selection replaces production as the core skill. The Morning the Print Step Broke traced the same friction back to one overlooked handoff.

I consult with product leaders at faith-based organizations on AI mandates, volunteer handoff friction, and taste decisions that survive real Sunday workflows. Let’s talk.

The “fail fast” doctrine from Eric Ries’s Lean Startup gets applied to AI agent workflows with a simple rule: ship the highest-velocity agent path first, then patch whatever breaks. In practice this means routing every ministry content review, volunteer schedule sync, and donor record query through an agent default that optimizes for token count and completion speed. The framework assumes downstream fixes will catch edge cases, yet it never accounts for the irreversible data exposure that happens the moment an agent touches a children’s ministry profile without a human gate.

Teams adopt this because the metrics dashboards reward it. Output per engineer climbs, demo videos look impressive, and leadership sees the 8x claim in real time. What disappears is the single checkpoint that once forced a printed curriculum draft or a volunteer background check summary to sit in front of two people before it left the building. The doctrine does not fail because it is fast; it fails because it treats the review layer as optional overhead rather than the only place where ministry-specific data risks become visible.

This is the foundational misread that causes product teams to delete security reviews without ever logging the deletion. The assumption is that speed and control are trade-offs you can rebalance later. In ministry data environments they are not. Once the agent owns the default path, the control surface shrinks to whatever the model vendor exposes, and the original human checkpoint never returns.

How Agent Defaults Removed the Print-Step Checkpoint

In the old Sermons4Kids workflow, a children’s ministry volunteer printed the lesson, wrote notes in the margin, and handed the marked copy to a coordinator before the next Sunday. That physical handoff was the review layer. It surfaced whether a story contained an age-inappropriate illustration or referenced a child by name in a way that violated the church’s child protection policy.

When the team moved the same content into an agent-orchestrated pipeline, the print step was marked low priority because it added friction to the “publish in under seven minutes” target. The agent now assembles the lesson, formats the handout, and emails it directly to the volunteer list. The only remaining check is whatever safety filter the model provider ships by default. No one on the team sees the output unless a volunteer flags an error after the fact.

The result is not theoretical. One deployment routed a lesson containing a real child’s photo from a past event because the agent pulled from an unsegmented media library. The photo never appeared in any test run because the test prompts used placeholder names. The print-step checkpoint would have caught it in thirty seconds. The agent path did not.

The Infrastructure Control Teams Traded for Speed

Jensen Huang’s sovereign AI argument centers on the idea that organizations must retain ownership of the full stack, from silicon to orchestration, if they want to enforce their own security and policy boundaries. When faith-tech teams adopt agent prioritization, they usually do the opposite. They hand the routing layer to whichever vendor offers the fastest agent SDK and accept the vendor’s data retention defaults as the new baseline.

This trade shows up in the logging configuration. The team disables persistent audit trails for agent decisions because the extra tokens slow the loop. It shows up in the data segmentation rules. Ministry records that once lived in an on-premise instance now flow through the vendor’s context window because the agent needs them to generate personalized follow-up emails. Huang’s point is not that speed is bad; it is that without sovereign control the organization no longer decides what counts as an acceptable risk.

The pattern repeats across teams. A product manager inherits an 8x output target, turns on the agent default, and removes the manual override because it is the only way to hit the number in the current quarter. Six months later the security review that used to run on every new workflow exists only as a quarterly checkbox that no one has time to execute.

Rebuilding a Minimal Override Protocol

The fix is not to abandon agents. It is to restore a single human-controlled gate that the agent cannot bypass by default. The gate does not need to review every output. It needs to review the decision to let the agent touch a new data class.

Start by listing every data type the current agent workflows access. Mark which ones carry ministry-specific risk: children’s names and photos, counseling notes, giving history tied to individuals. For each marked type, require an explicit approval record before the agent is granted access in production. The approval is one sentence and one signature. It lives in the same repo as the prompt templates so it cannot be lost during a refactor.

Next, instrument the agent to surface its intended action before execution when the data type is marked. The surface can be a simple Slack message or an internal ticket. The rule is that the agent waits for the override acknowledgment. If the acknowledgment does not arrive within the defined window, the workflow falls back to the non-agent path. This keeps the speed benefit for unmarked data while restoring the review layer exactly where the risk lives.

Your Turn: Apply This Today

• Open the last agent workflow your team shipped and export the list of data classes it touches; save it as a one-page reversal log in the project repo.
• Mark every children’s ministry or counseling-related field on that page with a red flag and note the date the agent first gained access.
• Write one-sentence override rules for each red-flagged field and commit them next to the prompt files before the end of the week.
• Configure the agent to pause and notify a named reviewer when any red-flagged field appears in a new production run.
• Run the workflow once with the override in place and record whether the reviewer caught anything the agent would have sent uninspected.
• Schedule a ten-minute review of the reversal log in your next roadmap meeting and decide which additional data class needs the same gate.

The Morning the Print Step Broke and The Quality Erosion No 8x Claim Ever Mentions both trace the same pattern of lost checkpoints in AI product work.

I consult with ministry tech product leaders on agent prioritization, data security protocols, and minimal override design. Let’s talk.

An 8x reduction in inference costs showed up in the first quarter after several faith-tech teams shifted to open-weight models. The number came from pooled benchmarks on common English queries, yet the same dashboards revealed that low-volume requests from non-English ministry contexts now routed through multiple fallback paths and produced effective costs two to three times higher than the prior proprietary baseline.

The reversal sits in the routing layer, not the model weights. Once cost became the primary signal for model selection, teams stopped measuring whether a particular workflow ever completed for the users who needed it most.

This pattern repeats because product groups treat the headline metric as stable across contexts. The assumption collapses once routing rules begin prioritizing volume and latency over coverage.

Teresa Torres’s continuous discovery approach names the mistake directly. Teams that stop interviewing users and observing workflows after the initial model switch lose sight of how inference decisions actually shape which ministries can participate. Continuous discovery requires ongoing contact with the real sequence of steps a volunteer or pastor takes, not just the token counts logged in the backend.

The Cost Shift That Teams Celebrated Too Early

Early wins came from high-frequency sermon-prep queries in English. Routing logic sent these to the cheapest available open-weight model and reported the expected savings. The same logic, however, sent queries in Spanish, Swahili, or regional dialects to secondary paths because the primary model lacked sufficient training data.

Those secondary paths often invoked a larger model or an external service, wiping out the per-token advantage. Ministry leaders running children’s curriculum in those languages saw longer wait times and occasional failures without any visible change in the cost dashboard.

The 8x figure therefore described only the subset of traffic that already matched the model’s strengths. Everything else moved into a hidden cost bucket that grew as more teams adopted the routing pattern.

Data Residency Decisions That Now Sit With Product

Routing choices now determine where user prompts are stored and processed. Some open-weight endpoints keep logs in regions that meet basic compliance but lack the data-processing agreements required by certain mission organizations.

Product teams that previously deferred residency questions to infrastructure now face them weekly. A routing rule that favors the lowest-cost provider can send a pastor’s draft sermon outline across borders without explicit consent, creating friction that discovery interviews later surface as reluctance to adopt the tool.

Continuous discovery surfaces these constraints when product managers sit with users while they prepare materials rather than reviewing aggregated logs after the fact. The residency decision stops being an infrastructure detail and becomes a product constraint that affects who can safely use the system.

Tying Routing Logs Back to One Observable Workflow

Most dashboards still separate cost data from workflow completion data. A single children’s ministry volunteer preparing a lesson may trigger three distinct inference calls across two models, yet the system reports only the summed token count.

When logs are joined to the actual sequence—select curriculum, adapt for age group, print handout—the product team sees which routing decisions cause the volunteer to abandon the task. The 8x savings on the first call become irrelevant if the third call fails or delays the print step beyond the seven minutes the volunteer has available.

Reconnecting logs to the observable workflow is the minimum requirement for Torres-style discovery. Without that link, cost metrics continue to celebrate savings that never reach the users whose contexts differ from the benchmark set.

Your Turn: Apply This Today

• Select one agent workflow your team currently runs and list every inference call it triggers in order.
• Tag each call with the exact user context it serves, including language and ministry type.
• Calculate the real cost per completed workflow, not per token, using last week’s actual traffic.
• Identify the single routing rule that most often diverts low-volume contexts to higher-cost or lower-quality paths.
• Schedule three 20-minute calls this week with users who match the diverted contexts and watch them run the workflow live.
• Adjust the routing rule for one context based on what the calls reveal, then measure completion rate and effective cost for the next seven days.

The Quality Erosion No 8x Claim Ever Mentions and The PM Who Inherited the 8x Target both trace how headline metrics drift from actual ministry outcomes once routing and residency choices accumulate.

I consult with product leaders in faith-tech on inference routing decisions, data residency constraints for ministry tools, and continuous discovery practices that reconnect logs to observable workflows. Let’s talk.