A ministry PM walked me through their router logs last week and stopped at one line. A pastoral-care query about suffering had gone to the cheap model, gotten a clean but shallow answer, and moved on without any handoff record.
They had hit their 25 percent savings target. They had also lost the only place they could have seen the model was out of its depth.
The fix wasn’t another model. It was making the moment of handoff itself visible.
The missing record matters because ministry content carries downstream review obligations. A children’s curriculum outline that surfaces an age-inappropriate illustration must be caught before it reaches volunteer printers. Without an explicit handoff log, the reviewer has no starting point to trace which model produced the problematic line.
Munger would call this a failure to apply the mental model of redundancy. Systems that depend on a single fast path need a second, slower path that records the switch. The log does not need to block inference; it only needs to persist the timestamp, model pair, and prompt hash so quality checks can run asynchronously.
Two real ministry workflows where cost wins hid quality drops
One team routed sermon illustration requests through a small model for the first 400 tokens and escalated only when the prompt contained the word “doctrine.” The cheaper path produced vivid stories that later required manual correction for historical accuracy. Because the escalation trigger was keyword-based, a theologically imprecise story about the Reformation slipped through three review cycles before anyone noticed.
A second workflow handled volunteer onboarding scripts. The team saved roughly 22 percent on tokens by keeping the greeting and FAQ sections on the small model. Within two weeks, several churches reported that the script referred to “small group leaders” as paid staff, a distinction that does not exist in most volunteer-run congregations. The error originated in the light model’s training distribution and was never surfaced because no handoff log existed to flag the section for doctrinal review.
Both cases show the same pattern: the cost reduction succeeded on paper while the quality signal moved downstream to the user. The latticework requires the product team to model both the cost variable and the review variable in the same diagram.
Logging tactics that make the handoff visible without adding latency
The minimal viable log records four fields at the moment of escalation: prompt hash, source model, target model, and a one-line reason code. These fields fit in a single database row and write in under 5 milliseconds on standard infrastructure.
A second tactic stores the full prompt only when the reason code indicates doctrinal or safety review. The storage decision itself becomes another model call, but it runs after the user has already received the response. This keeps perceived latency unchanged while preserving the exact context a human reviewer needs later.
Teams that already emit OpenTelemetry traces can add the handoff event as a span attribute rather than a new table. The attribute appears in the same trace that already tracks the agent run, so existing dashboards surface the routing decision without new infrastructure.
Your Turn: Apply This Today
- Pick one agent workflow that already uses model routing and add a single database column to store the handoff timestamp and reason code.
- Instrument the router to write that row before it calls the heavier model, using an async write so inference is not blocked.
- Build a five-minute dashboard query that surfaces any handoff where the reason code is “doctrinal” or “safety” and no human has yet reviewed the output.
- Choose one ministry workflow you observed in the past month where a light-model error reached users; replay the prompt through the new log to confirm the handoff point is now captured.
- Write a one-paragraph runbook that tells the on-call reviewer exactly which fields in the log to check first when a quality ticket arrives.
- Schedule a 30-minute review next week with the person who owns quality metrics to agree on the threshold that will trigger an automatic heavier-model retry instead of a human ticket.
The Routing Step Ministry Teams Still Can’t Automate and The Benchmark Number That Still Needed a Human Gate both explore the same tension between automated efficiency and necessary human oversight. I consult with product leaders on AI routing decisions, handoff logging, and quality checkpoints in faith-tech tools. Let’s talk.