The Routing Layer Ministry Agents Needed Before Compute Ate the Budget

Several widely-circulated AI infrastructure playbooks push a “frontier model default,” the assumption that every production query benefits from the largest available model. That assumption collapses once you measure actual model routing costs against the queries coming from volunteer coordinators at a mid-size ministry organization. Those playbooks treat capability as a one-dimensional scale. They ignore that most ministry interactions are narrow classification or retrieval tasks that smaller open models handle at a fraction of the cost.

This single-default habit produces budgets that grow faster than usage. Teams ship an agent, watch the first invoices, then discover that most of the load never needed the frontier tier at all.

Charlie Munger’s latticework of mental models insists you keep multiple tools sharp and switch between them deliberately. The same principle applies to model routing: you hold several models in inventory, each suited to a distinct slice of work, and you define the switch rules in advance.

The single-model habit that hid real costs

Teams default to the strongest model because the prompt works on the first try. In practice that choice masks the real distribution of work. A children’s ministry volunteer asking for a printable activity sheet triggers the same frontier-tier call as a pastor drafting a full sermon series. Both return usable text, so the cost difference never surfaces in testing.

The hidden expense compounds across volume. In the curriculum-support logs we reviewed for a volunteer-curriculum team we worked with, simple requests outnumbered theological research queries by a wide margin. Running every request through the same model turns a predictable pattern into an unpredictable monthly bill.

Open-model benchmarks run regularly against actual query logs expose the mismatch quickly. A smaller, fine-tuned open model matched frontier accuracy on the large majority of volunteer requests while cutting per-token cost substantially. The remaining harder cases still route upward, but the overall spend drops right away.

How routing exposed the coordination failures

Once multiple models sit in the stack, the routing layer must decide which one owns each incoming request. Without explicit rules, the decision falls to whoever last touched the prompt. One week the agent tries to answer everything locally; the next week someone flips a flag and every request escalates. The inconsistency shows up in both cost spikes and user frustration when answers suddenly lose context.

Munger’s latticework requires the decision criteria to be written down and tested, not left to individual judgment. Ministry teams that adopted a simple decision tree — query length, presence of curriculum keywords, presence of personal data — noticeably cut coordination meetings. The tree lives in the same repo as the agent code so changes require a pull request rather than a Slack thread.

The same tree also surfaces coordination failures between agents. A routing decision that sends a volunteer request to the small model sometimes needs to hand off to a knowledge-graph lookup agent next. Without the handoff rule documented, the small model simply hallucinates the missing data instead of calling the graph.

Building the handoff rule that stuck

The durable rule set three conditions for escalation. First, the query must contain a named person or family identifier. Second, it must reference content older than the current quarter. Third, the small model’s first-pass answer must score below a fixed similarity threshold against the local knowledge base. Any one condition triggers the frontier model; all three must fail to stay on the cheaper tier.

Teams tested the rule for several consecutive weeks on live traffic from a volunteer-facing curriculum platform we advised. The first week the rule was too strict and escalated far more queries than expected. By the third week the threshold settled at a point that escalated a much smaller share of traffic while keeping user-reported accuracy comfortably high. The rule survived because it was measured against real queries rather than synthetic test sets.

The handoff also preserved audit trails. Every escalation logged the three conditions that fired, so product managers could review edge cases without re-running the entire conversation history. That audit layer turned a cost-saving tactic into a repeatable evaluation practice.

Your Turn: Apply This Today

• Export a sample of real queries from your ministry agent logs this week and tag each one with expected model tier.
• Run the same queries against a couple of open-weight models using identical prompts and record accuracy plus cost per query.
• Write a three-condition decision tree in a single markdown file and commit it to the agent repo before Friday.
• Deploy the tree behind a feature flag and route a small slice of live traffic through it for a week.
• Compare the flagged cohort’s spend and accuracy against the control cohort in a shared dashboard.
• Adjust one condition in the tree based on the week’s results and schedule the next test run for the following Monday.

The Token Cost Drop That Only Appeared After We Added the Knowledge Graph and Graph Engineering: Stop Waiting on Agents That Don’t Need Each Other both show how routing decisions only stabilize once the surrounding data layer is explicit.

I consult with ministry product leaders on task routing for agents, open model evaluation, and compute cost control in faith-tech deployments. Let’s talk.