Blog

The children’s ministry coordinator leaned over her laptop in the church office, the smell of stale coffee still hanging in the air from the early volunteers. She copied the parent’s question about Emma’s baptism date into the new agent window, added the child’s name and the note about the family traveling from out of state, then pressed send. The interface confirmed the query had been accepted and would route automatically.

Seconds later the reply arrived: a standard calendar link. No mention of the church’s actual baptism schedule, no check against the minimum age requirement the elders had set three months earlier, and no flag for the out-of-state travel that usually triggered a pastoral follow-up call. The coordinator stared at the screen, already knowing she would have to undo the damage in a separate email thread.

That single handoff exposed the gap. Routing logic built for speed and token cost had treated the query as interchangeable with any other scheduling request. Ministry context never entered the calculation.

Cost Savings That Erase Ministry Context

Jensen Huang has argued that organizations need sovereign control over their AI stack precisely because generic infrastructure erases the constraints that matter most in a given domain. When a church team adopts the cheapest available model for every incoming message, the same erasure happens at the routing layer.

The coordinator’s experience is common now. Product teams measure success by reduced API spend and faster average response times. Those numbers improve while actual decision quality declines in ways that only surface days later, when a family shows up on the wrong Sunday or a volunteer has to apologize for an automated reply that ignored a prior pastoral conversation.

The pattern repeats across tools built for ministry. A routing decision optimized for cost treats every parent inquiry as equivalent, whether it concerns a baptism, a special-needs accommodation, or a request to speak with someone who already knows the family’s situation. The savings appear on the dashboard. The relational cost appears in the follow-up meetings the system never sees.

The Last-Mile Check That Models Skip

Cheap routing paths stop at the point of plausible text. They do not reach the moment when a human must confirm that the generated answer respects the local policy that was never written into the prompt. In practice, this means the system can return a calendar link even when the policy requires a phone call first.

Teams that treat routing as a pure optimization problem discover the limitation only after deployment. A model can generate the right format of response while still missing the conditional that matters: this child is under the age threshold, this family needs the senior pastor copied, this date conflicts with the building’s scheduled maintenance. Those conditions live in the heads of two or three staff members, not in the token count.

Huang’s point about sovereignty applies here. The organization that controls its own routing logic can insert the last-mile check before the reply leaves the system. The organization that accepts the default cheapest path cannot, because the decision to skip that check was made upstream by someone optimizing for cost alone.

Owning the Router Without Owning the Outcome

Many teams now describe their routing setup as “set and forget.” They choose a model tier, define a few topic buckets, and let the system decide which requests receive the expensive model and which receive the cheap one. The assumption is that topic detection is sufficient to protect ministry-critical moments.

Topic detection fails when the same words appear in different contexts. A baptism question can be routine or urgent depending on prior conversations that exist only in the church database or in the memory of the children’s director. Routing that ignores those signals produces replies that look correct until a human reads them.

The cost win is real only when the team still owns the final handoff. That requires explicit checkpoints where the system surfaces the draft response along with the missing context it could not retrieve, rather than sending the reply directly. Without those checkpoints, ownership of the router becomes disconnected from ownership of the outcome.

Your Turn: Apply This Today

• Pick one incoming message type your current router handles automatically and add a human review step for any reply that includes a date or eligibility rule before it is sent.
• Log the last ten routed queries that produced follow-up work for staff and note which policy condition the model missed in each case.
• Write the three most common local policy conditions that affect your ministry area as explicit if-then rules and test whether your router can surface them before choosing the model tier.
• Run a one-week experiment where every reply generated by the cheapest model is held for a thirty-second human glance before release, then measure how often the glance changes the output.
• Identify the single staff member who currently catches the most routing errors after the fact and give that person edit rights inside the router configuration this week.
• Replace one generic topic bucket in your router with a rule that requires a named staff member to be copied when the query mentions a child under a set age or a family with prior pastoral contact.

Teams that treat routing as an infrastructure choice rather than a ministry handoff decision keep rediscovering the same gaps. The same pattern shows up in how product teams decide which features receive heavy human oversight and which are allowed to ship on model output alone.

I consult with product leaders and ministry technology teams on routing design, last-mile human checkpoints, and policy-aware automation. Let’s talk.

You’re the product manager who just got handed the AI agent pilot at a denomination that still prints bulletins on Wednesday afternoons. The brief landed in your inbox with three bullets: “explore agents,” “show value by Q3,” and “don’t break anything.” No one mentioned who actually uses the tools or how decisions get made across twenty regions that don’t share a CRM.

That handoff feels familiar because it is. Most pilots arrive this way: clean scope, unclear ownership, and an assumption that the technology will somehow find its own users.

This is the foundational misread that causes product teams to build agents that work in demos but stall in real ministry settings. The mistake isn’t technical. It’s assuming quality and usefulness will travel through existing channels without deliberate design for distribution and trust.

Charlie Munger’s latticework of mental models forces you to hold multiple lenses at once instead of defaulting to the one your team already knows. When you apply the distribution model, the incentives model, and the trust model together, the pilot stops looking like a feature rollout and starts looking like a handoff problem that needs visible human oversight at every layer.

The Bias That Treats Distribution as Someone Else’s Job

Most pilots optimize for the moment the agent produces an output. The team measures token count, accuracy against a test set, or time saved in a controlled workflow. None of those numbers travel outside the product Slack channel.

Faith communities run on relationships that predate any software. A children’s ministry volunteer who prints a lesson on Saturday night will ignore an agent that arrives through an app notification, no matter how accurate the content is. The same volunteer will try an agent if the regional coordinator they already text shares it first.

The bias shows up when the pilot plan ends at “launch.” No one owns the step where a pastor in another time zone decides whether this tool is safe to mention from the platform on Sunday. That decision sits outside the product roadmap until someone forces it onto the latticework.

How the Latticework Exposes Hidden Handoff Risks

Munger’s approach requires naming the second-order effects before they appear in usage data. Here, the second-order effect is the loss of visible human oversight. When an agent drafts a follow-up email to a family that missed three services, the recipient does not know who approved the tone or the timing.

Without an explicit owner for that approval step, trust erodes faster than any accuracy metric can recover. The denomination’s regions already operate with different comfort levels around technology. One region requires every automated message to carry a staff name and phone number. Another region has no policy at all. The agent does not know the difference.

The latticework reveals the risk by forcing the question: which mental model explains why adoption will vary by region even if the agent performs identically? The answer is the trust model, not the capability model. Once that is named, the pilot scope expands to include who will stand behind each output.

Rebuilding the Pilot Around Actual Ministry Workflows

Start by mapping the agent to the moments when a human already makes a judgment call. For the children’s curriculum team, this might be the Saturday night decision about which story to tell. The agent can surface three options with volunteer completion rates from the last quarter, but the final choice stays with the person who knows which kids will be present.

Next, attach distribution channels that already exist instead of creating new ones. The agent should appear inside the weekly email the regional coordinator sends to volunteers, not inside a new dashboard that requires a login. The coordinator’s name stays on the message. The agent supplies the content underneath that name.

Finally, instrument the handoff itself. Track how many outputs receive human review before they reach end users and how many are sent without review. Those two numbers tell you more about long-term adoption than any engagement metric collected inside the agent interface.

Your Turn: Apply This Today

• List the three mental models you will hold for this pilot and write one sentence for each that describes a failure mode you have not yet designed against.
• Identify the single existing channel, email list, group text, or printed packet, through which every region already receives new resources, then route the agent’s first output through that channel only.
• Name the person in each region who must appear as the approver on any automated message and add their review step to the workflow before the next sprint planning.
• Build a simple shared doc that records every agent output sent without human review in the last two weeks and review it with your stakeholder group this Friday.
• Run the current demo script past one volunteer who has never used the pilot and note the first question they ask about who is responsible for the content.
• Adjust the success criteria for the Q3 review to include one distribution number and one oversight number alongside any capability metrics.

This approach follows the same pattern behind the other letters and case studies on this site: distribution and trust decide adoption long before capability does, and the handoff between staff and region is where most pilots quietly succeed or fail.

I consult with product leaders running AI pilots inside denominations and mission agencies on distribution design, oversight workflows, and applying multiple mental models to adoption problems. Let’s talk.

The best benchmarks for ministry AI still require a human gate, even when the model scores 92 on every automated test.

Teams treat higher automated scores as proof that the system understands the real constraints of a children’s ministry volunteer printing materials on Saturday night. In practice those scores only confirm the model stayed inside the distribution of its training data. They reveal nothing about whether the output survives contact with a person who has seven minutes and a printer low on toner.

This is the foundational misread that causes product teams to ship workflows that look efficient in staging and collapse on Sunday. Teresa Torres’s continuous discovery framework exposes the gap: discovery happens through repeated, lightweight contact with users performing their actual jobs, not through static test sets that grow stale the moment the first volunteer deviates from the expected path.

Why 45-Minute Human Benchmarks Beat the Dashboard

Torres emphasizes that discovery is continuous precisely because user context shifts faster than any fixed dataset can track. A dashboard of LLM-as-judge scores creates the opposite condition. It freezes the evaluation criteria at the moment the prompt was written, then rewards the model for staying inside that frame.

A 45-minute human benchmark works differently. One person pulls a real, frozen input set from last month’s curriculum export, runs it through the new model, and scores the output against the single observable behavior that matters: whether the volunteer can complete the activity without opening a second tab or asking for help. The score is binary and tied to time-on-task, not to semantic similarity.

Teams that adopt this pattern quickly notice the dashboard and the human score diverge. The model that aces BLEU, ROUGE, and custom LLM judges still produces instructions that assume the volunteer has already read the full lesson plan. The 45-minute check catches the failure before it reaches the print queue.

The Ministry Failure Mode LLM Judges Miss

LLM judges optimize for coherence and doctrinal alignment because those dimensions are easy to express in a grading rubric. They miss the failure that actually costs ministries users: an output that looks complete but leaves the volunteer without the next physical action.

Consider a children’s ministry workflow that generates small-group discussion questions. An automated judge will rate the questions as age-appropriate and on-topic. The same judge will not notice that the questions require the leader to locate three separate craft supplies that are not listed in the materials section. The volunteer hits that gap at 6:43 p.m. on Saturday, and the activity is abandoned.

Continuous discovery treats this as a signal to update the frozen input set, not as noise to be averaged out. Each new human-scored run adds the exact edge case the previous version missed. Over six weeks, the benchmark grows into a living record of what actually breaks in the field rather than a polished but brittle leaderboard.

Running Your First Frozen-Input Check This Week

The first check does not require new tooling. Export the last three weeks of curriculum PDFs that were actually printed and used. Strip any date-specific or church-specific identifiers so the set can be reused. Load those files into whatever generation system you are evaluating and treat the output as a single frozen artifact.

Run the artifact through one human reviewer who has performed the volunteer role before. Give that person a timer and the exact instructions a real volunteer receives. Record only whether the task finished inside the time budget and whether any required external resource was missing. Do not collect qualitative feedback on this first pass; the binary outcome is the only data that survives contact with next week’s schedule.

Repeat the identical frozen set after each model or prompt change. The moment the human score drops, the change is rejected regardless of what the automated judges report. This single loop implements Torres’s core practice: the product team stays in regular, low-cost contact with the job the user actually performs.

Your Turn: Apply This Today

• Export the last three curriculum PDFs that were actually printed and used in a children’s ministry setting this month and strip identifying details to create your first frozen input set.
• Recruit one former volunteer who still remembers the seven-minute constraint and schedule a single 45-minute session this week to run the frozen set through your current generation workflow.
• Score the output using only two criteria: task completed inside the time limit, and no missing physical materials required.
• Store the scored output and the exact model version or prompt hash so the same input can be re-run after any change.
• Reject the next model or prompt update that drops the human score below the previous run, even if automated metrics improve.
• Repeat the identical frozen-input check once every two weeks for the next six weeks and log the human completion rate alongside dashboard numbers.

Two earlier posts trace the same pattern from different angles: “The Bias I Let Shape Every Eval Until a Human Scored It” and “The 45-Minute Benchmark That Exposed My LLM Judge Habit.”

I consult with product leaders building AI features for ministry and nonprofit tools on human-scored benchmarks, continuous discovery loops, and frozen-input checks for Sunday workflows. Let’s talk.