I spent eight months sending every product spec to the largest model we could afford. The assumption was simple: more parameters meant fewer gaps for the volunteer teams who would later implement the feature. Prayer request flows, curriculum upload paths, even the small print-first forms for Sermons4Kids all went through the same pipeline. Output looked finished. Token counts stayed under budget. Launches still slipped because the teams had to rebuild the exact judgment calls the model had flattened. What finally fixed it was a set of routing rules, not a bigger model.
The cost showed up in volunteer completion rates. A children’s ministry coordinator would open the generated story, stare at the polished steps, and realize the model had removed the three places where a tired volunteer needed to decide whether to skip, adapt, or hand off. Those decisions never appeared in the original ticket. They reappeared as rework after the first test with real users.
This pattern is the foundational misread that causes product teams to treat model scale as a substitute for visible ownership. The bigger model does not eliminate judgment work; it buries it inside language that feels authoritative. Teams then discover the missing pieces only when the artifact reaches the people who actually run the program on Sunday morning.
John Wesley’s three rules offer a practical lens here. Do no harm. Do good. Stay in love with the disciplines that keep the work honest. Applied to model routing, the first rule means refusing any output that hides the exact decisions volunteers must still make. The second rule means routing only the portions of work that genuinely reduce repetition without erasing context. The third rule means keeping a running record of what each tier of model forces the team to restore by hand.
When Routing Rules Were Missing for a Prayer Request Flow
The spec described a simple intake form that let parents submit a request and receive a follow-up from a small-group leader within forty-eight hours. Fable 5 produced ten acceptance criteria, four edge-case branches, and a suggested notification schedule. Every sentence read as if someone had already tested it with real families.
The ministry lead opened the ticket and immediately flagged three problems the model had resolved without stating its assumptions. It had decided that every request would route to the same leader pool. It had assumed the parent would receive an automated confirmation that included the leader’s name. It had removed the step where the volunteer could mark a request as “needs pastoral review” without logging a separate ticket.
Those three decisions were the actual product. Restoring them took longer than writing the original spec because the language in the ticket made the choices look settled. The team had to reopen the conversation with the volunteer coordinators to learn what they actually needed to retain control over.
The Rewrite Cost Routing Rules Would Have Caught
Every time we routed a full artifact through the largest model, the rewrite work happened after the ticket reached the volunteer side. The model produced clean flows that assumed perfect information and consistent volunteer availability. Real ministry calendars contain neither.
One Sermons4Kids update required the volunteer to choose between printing a single sheet or a full lesson packet depending on how many kids showed up that week. The model had defaulted to the full packet because that option looked more complete. The volunteer had to add back the decision point and the accompanying print instructions. That single addition required three rounds of internal review because the original ticket no longer contained the rationale.
The token bill recorded only generation time. It never captured the hours spent reintroducing the places where a human had to exercise judgment under time pressure. Those hours appeared later as missed launch dates and lowered volunteer retention metrics.
Routing Rules That Actually Kept the Work Visible
Anthropic’s guidance on choosing the right model for the task reinforces why routing rules beat defaulting to the biggest model every time.
We eventually replaced the single-pipeline approach with three explicit tiers. Tier one handled only repetitive formatting and placeholder text. Tier two generated options with the decision points left explicit and bracketed. Tier three was reserved for narrow slices where the model had already demonstrated consistent accuracy against logged volunteer feedback.
Each tier carried a required log entry: what the model produced and what the team had to restore before the artifact reached the volunteer test. The log lived in the same ticket system the ministry teams used, not in a separate research document.
The discipline forced us to notice when the largest model was smoothing over exactly the friction that made a feature usable by a seven-minute volunteer. It also made the cost of each routing choice visible to the rest of the product group before the next planning cycle.
Your Turn: Apply This Today
- Pick the next three product artifacts on your roadmap and assign each one a model tier before any generation begins.
- For the artifact assigned tier one, log every sentence the model adds that removes a volunteer decision point.
- For the artifact assigned tier two, require the team to restore at least two explicit judgment steps before the ticket moves to volunteer review.
- After the third artifact ships, compare the three logs and mark which tier produced the highest volume of hidden rewrite work.
- Share the marked log with the two volunteer coordinators who will actually use the feature and ask them to confirm the restored steps match what they need.
- Update the routing rules document with the single sentence that best describes when tier three is now off-limits for your current volunteer contexts.
The v1 Where the Model Handed Me the Safe Path and Trust Logs Beat the Next Feature Sprint both trace the same pattern of hidden judgment costs across different product surfaces.
I consult with product leaders and ministry technology teams on explicit model tiering, restoring volunteer judgment points, and keeping rewrite costs visible in the ticket system. Let’s talk.