Blog

The volunteer clicked the default model button at 9:14 p.m. on her laptop. The Spanish-language baptism curriculum loaded, the cursor blinked once, and the answer appeared in clean paragraphs that distinguished immersion from sprinkling without extra cost or visible delay. She copied the text into the class handout, closed the tab, and moved on to printing name tags for the next group. No one else on the team saw the model identifier, the token count, or the routing path that produced the response.

The scene repeated across three other ministry sites that week. Each time the output looked usable, the meter stayed flat, and the decision stayed invisible. The pattern matched what continuous discovery work is meant to surface: the moment when a product choice gets treated as finished instead of observed.

Teresa Torres frames continuous discovery as the practice of keeping direct contact with users and their context so that assumptions do not harden into defaults. When routing logic sits between the user and the model, that contact breaks unless teams deliberately log what the user actually experienced and why the system chose that path.

The Default Path That Hid the Model Choice

The routing layer in the pilot defaulted to the lowest-cost model whenever the prompt length stayed under a certain threshold. The volunteer never knew the threshold existed or that a different model would have handled the theological distinction with different phrasing. The output satisfied the immediate need, so the team recorded the class as complete and moved the file to the shared drive.

Continuous discovery requires seeing the decision point, not just the result. Here the decision point was erased before the volunteer finished her shift. The cost saving appeared real because no one measured the downstream effect on how the material would be explained during the actual class.

Three other pilots followed the same pattern. Each team assumed the default choice was neutral. None captured the model name or the routing rule that produced it. The data that would have shown whether the cheaper model altered meaning simply never existed.

Discovery That Happens After the First Output

Torres emphasizes interviewing users while they still hold the artifact in their hands. In this case the artifact was the printed handout. The volunteer reviewed it at the kitchen table, crossed out one sentence, and rewrote it by hand. That revision moment was the discovery signal. It arrived after the model call had already been logged as successful.

The routing decision needed to be visible at that exact point. Instead the system treated the generation step as the end of the workflow. The human correction happened outside any trace, so the next similar prompt would trigger the same default without any adjustment.

When the checkpoint moves after the output, teams gain the chance to ask what the user changed and why. Without the log, that question never gets asked. The routing layer continues to optimize for cost while the actual meaning drifts in small, unrecorded edits.

Ownership of the Routing Decision

Someone on the product side chose the default model and the threshold that triggered it. That choice sat inside configuration files the ministry volunteers never opened. Continuous discovery treats the person who sets the default as part of the user loop, not outside it.

In the observed pilots, no one owned the routing rule after the initial setup. The volunteer who hit the button carried the outcome, yet she had no channel to report that the model had produced phrasing she needed to correct. The product team measured token spend and uptime, not revision rate on the final handout.

Ownership means attaching the model identifier and the routing condition to the same record the volunteer saves. When that link exists, the next discovery interview can start with the actual decision rather than the polished output. Without it, the routing layer remains a black box that quietly shapes the material delivered to the class.

Your Turn: Apply This Today

• Insert a mandatory log line that records the exact model name and routing rule immediately before any generation call in the Spanish baptism workflow.
• Add a second log line that captures the volunteer’s first edit or comment on the generated text, tied to the same session ID, before the file is saved or printed.
• Expose both log lines in the shared drive folder so the next person opening the handout sees which model answered and what changed.
• Run a 15-minute review of last week’s handouts with the two log lines visible and note any pattern between model choice and required edits.
• Update the agent prompt template to require these two logging steps as non-skippable fields before the workflow can close.
• Schedule one 20-minute interview this week with the volunteer who used the handout, starting from the logged model name rather than the final version.

Two recent posts examine the same tension between automated paths and required human checkpoints: “The Open-Weight Number That Still Required a Human Gate” and “Why Top PMs Still Route Specs Through Sunday Validation.”

I consult with product leaders on AI routing decisions, continuous discovery checkpoints, and ministry tech workflows. Let’s talk.

The “metrics over theater” mandate from Marty Cagan’s Inspired framework still collapses when product teams move from enterprise software into ministry tools. Cagan calls for replacing executive confidence with observable experiments and clear success criteria. That prescription assumes every handoff can be instrumented inside a single codebase or dashboard. Ministry products break the assumption at the point where an AI draft leaves the screen and enters a volunteer workflow that ends in a printed page or a live children’s lesson.

The framework treats the experiment as complete once token counts, latency, and click-through numbers move. It never accounts for the physical or relational transfer that determines whether the output actually gets used. In practice, teams celebrate a 40 percent drop in generation time while the curriculum still sits unopened because the volunteer cannot finish the customization in seven minutes.

This is the foundational misread that causes product teams to ship pilots that look rigorous in review but never survive the first real gate. The misread treats measurement as a software problem rather than a sequence of human transfers that each carry their own failure modes.

Charlie Munger’s latticework of mental models demands that any new tool be checked against multiple independent lenses at once. One model is the incentive model. Another is the handoff model. A third is the completion model. Most AI pilots in faith settings run only the incentive model and stop. They reward speed and cost reduction while ignoring whether the next human in line can still do their job without new friction. The lattice reveals the gap because each model must be satisfied before the pilot earns the label “working.”

The Experiment That Never Reached the Print Step

A recent Sermons4Kids pilot generated full lesson outlines in under two minutes. The internal review celebrated the output quality and the 60 percent reduction in writer hours. No one logged what happened after the file left the platform. Volunteers still had to reformat the material for their specific room size, age mix, and available supplies. The print step added twenty-three minutes of manual work that the experiment never measured.

The same pattern appears when teams test AI-generated discussion questions for small groups. The model produces theologically sound text. The handoff to the group leader who must decide whether the questions fit the actual people in the room remains invisible. When the leader skips the questions, the metric dashboard still shows success because the generation event completed.

Munger would require a second model that tracks whether the printed or displayed artifact survives the next human filter. Without it, the experiment only proves the model can produce text, not that the text can travel the full distance to use.

Metrics That Reward Speed Over Gatekeeping

Internal reviews often track generation cost, model version, and time-to-first-draft. These numbers move quickly and look decisive on a slide. They do not record whether a ministry director rejected the draft because it lacked local application points. The rejection happens outside the tracked system, so the metric stays clean.

When the same team later adds a “human review” checkbox, the checkbox becomes another speed metric. Reviewers learn to clear items fast to keep the dashboard green. The real gatekeeping judgment gets compressed into a binary field that carries no weight in the next sprint planning meeting.

The latticework exposes the missing model here: the gatekeeper model. It asks whether the person who must say yes or no still has enough context and time to exercise judgment. When that model is absent, speed metrics quietly train teams to bypass the very humans the product claims to serve.

The Lattice Model Missing from Most AI Pilots

Most pilots apply only the engineering model of iteration. They ship, measure, adjust, repeat. Munger’s approach adds the second-order effect model and the reputation model in parallel. The second-order effect model asks what behavior the new speed incentive creates among volunteers who now receive faster but less contextual material. The reputation model asks whether pastors will continue to trust the platform once they notice the gap between polished output and usable output.

A pilot that only watches token spend cannot surface either effect. The first sign of trouble appears weeks later when volunteer completion rates drop or when a pastor quietly switches to another source. By then the internal review has already declared the pilot a win.

Teams that run all three models together produce different artifacts. They log the moment the AI file reaches the volunteer inbox, the time until the volunteer either completes or abandons customization, and the explicit reason given for abandonment. Those logs become the actual experiment data rather than the generation metrics that stop at the screen.

Your Turn: Apply This Today

• Pick one current AI pilot and add a single handoff log field that records the exact minute the output left the platform and reached its first human recipient.
• This week, require every pilot review to include the completion timestamp from the volunteer or leader side before any success claim is accepted.
• Create a two-column template that forces the team to list both the generation metric and the handoff metric for the same item before the item can move to the next sprint.
• Schedule one thirty-minute call with a children’s ministry volunteer who used last week’s generated material and ask only what broke during the print or prep step.
• Remove any speed or cost metric from the next internal review slide deck and replace it with the handoff completion rate for the same period.
• Write down the second model you will run alongside the current experiment and commit to sharing its result in the same review that shows the token numbers.

The Trust Step Kent Beck Made Non-Negotiable and The Open-Weight Number That Still Required a Human Gate both trace how unmeasured handoffs quietly undo technical wins. I consult with ministry product leaders on AI pilot design and handoff measurement systems. Let’s talk.

Product teams assume their models will flag their own security failures before launch. That assumption has already removed the review step from three separate agent pilots in ministry tools.

The bias surfaces as a quiet confidence that higher output quality replaces external checks. Teams ship faster because the model sounds coherent and the metrics look clean. What disappears is any process that asks whether the output could cause harm in a real ministry setting.

John Wesley’s three rules expose the flaw. The first rule, do no harm, was never meant to be satisfied by good intentions or polished results. It required an active step to examine the action before it reached people. Modern product defaults treat that step as optional once the model passes internal tests.

This is the foundational misread that causes product teams to treat security as a quality problem instead of a presence problem. When the model generates acceptable text, the team moves on. The absence of a human who must sign off creates exactly the conditions Wesley warned against: harm that looks reasonable until it lands in a local church.

How the Bias Shows Up in Agent Defaults

Agent builders set defaults for speed and coherence. The system prompt rewards fluent answers and penalizes refusals that feel slow. In practice this means the model will generate a children’s ministry outline or a counseling response without checking whether the source material was licensed or the advice fits the age group.

Observed pilots showed the pattern clearly. One team built an agent to draft volunteer schedules. The model produced clean tables and avoided obvious conflicts. No one ran the output against the church’s actual volunteer database because the internal accuracy score was high. The schedule was sent to print before anyone noticed it assigned a registered sex offender to the nursery.

A second pilot used an agent to summarize sermon feedback. The summaries looked useful until a staff member noticed the model had pulled direct quotes from private prayer requests that had been entered in the same form. The team had assumed the model would respect context boundaries because its training data was clean.

These failures did not come from bad models. They came from the decision that the model’s output quality was sufficient proof that no external review was needed.

The Rule That Forces a Human Print Step

Wesley’s first rule demands an explicit pause before action. In product terms that pause must be a required human step attached to any output that touches real people. The print step works because it cannot be faked by the model and because it forces someone to own the result.

Curriculum products built for children’s ministry already use this pattern. Drafts are generated, then forced through a human who must physically approve before the file reaches volunteers. The approval is not a quality gate about writing style. It is a presence gate that asks whether this content belongs in a room with seven-year-olds.

The same gate applies to agent outputs. When an agent produces a counseling script, the print step requires a licensed counselor to initial the page before it is shown to a user. The model can generate the text in seconds. The human step still takes minutes. That minutes-long delay is the feature, not the bug.

Without the print step, the bias reasserts itself. Teams celebrate the speed and quietly drop the review. Within two cycles the review process is treated as legacy rather than required.

Rebuilding the Review Without Slowing Shipping

The review can be rebuilt by separating the fast generation loop from the slow presence loop. Generation stays automated. Presence stays human and mandatory for any output that will reach a ministry context.

One working pattern splits the agent into two stages. The first stage produces a candidate response and tags it with a risk score based on content type. Any response tagged above a low threshold moves to a queue that requires a named reviewer to approve before the response is released. The queue is visible to the whole team so no item sits unclaimed.

A second pattern attaches the human step to the data rather than the text. Before an agent can draw from a church’s internal notes or prayer requests, a steward must mark that dataset as cleared for agent use. The clearance is time-limited and tied to a specific project. This prevents the model from quietly incorporating sensitive material that was never intended for automated systems.

Teams that adopted both patterns shipped at nearly the same cadence as before. The difference was that every high-risk output carried an accountable name attached to it. The model still produced the draft. The human still had to be present.

Your Turn: Apply This Today

• Pick the current agent build and list every output type it can produce that touches children, counseling, or private data.
• Write a one-sentence rule that forces a named human to approve before any of those outputs leaves the system.
• Add a visible queue in the admin panel that shows pending approvals and who is responsible for each one.
• Run the agent on last week’s real data and force yourself to print and initial the three highest-risk responses before they would have shipped.
• Set a calendar reminder for 48 hours after any new dataset is connected to confirm the clearance step was actually completed.
• Document the single slowest review that occurred this week and decide whether that delay protected people or was pure process theater.

The Prioritization That Quietly Dropped Security Reviews and The Trust Step Kent Beck Made Non-Negotiable both trace the same pattern of reviews that disappeared once metrics looked acceptable.

I consult with product leaders building AI agents for ministry contexts on security review gates and human presence requirements. Let’s talk.