The volunteer clicked the default model button at 9:14 p.m. on her laptop. The Spanish-language baptism curriculum loaded, the cursor blinked once, and the answer appeared in clean paragraphs that distinguished immersion from sprinkling without extra cost or visible delay. She copied the text into the class handout, closed the tab, and moved on to printing name tags for the next group. No one else on the team saw the model identifier, the token count, or the routing path that produced the response.
The scene repeated across three other ministry sites that week. Each time the output looked usable, the meter stayed flat, and the decision stayed invisible. The pattern matched what continuous discovery work is meant to surface: the moment when a product choice gets treated as finished instead of observed.
Teresa Torres frames continuous discovery as the practice of keeping direct contact with users and their context so that assumptions do not harden into defaults. When routing logic sits between the user and the model, that contact breaks unless teams deliberately log what the user actually experienced and why the system chose that path.
The Default Path That Hid the Model Choice
The routing layer in the pilot defaulted to the lowest-cost model whenever the prompt length stayed under a certain threshold. The volunteer never knew the threshold existed or that a different model would have handled the theological distinction with different phrasing. The output satisfied the immediate need, so the team recorded the class as complete and moved the file to the shared drive.
Continuous discovery requires seeing the decision point, not just the result. Here the decision point was erased before the volunteer finished her shift. The cost saving appeared real because no one measured the downstream effect on how the material would be explained during the actual class.
Three other pilots followed the same pattern. Each team assumed the default choice was neutral. None captured the model name or the routing rule that produced it. The data that would have shown whether the cheaper model altered meaning simply never existed.
Discovery That Happens After the First Output
Torres emphasizes interviewing users while they still hold the artifact in their hands. In this case the artifact was the printed handout. The volunteer reviewed it at the kitchen table, crossed out one sentence, and rewrote it by hand. That revision moment was the discovery signal. It arrived after the model call had already been logged as successful.
The routing decision needed to be visible at that exact point. Instead the system treated the generation step as the end of the workflow. The human correction happened outside any trace, so the next similar prompt would trigger the same default without any adjustment.
When the checkpoint moves after the output, teams gain the chance to ask what the user changed and why. Without the log, that question never gets asked. The routing layer continues to optimize for cost while the actual meaning drifts in small, unrecorded edits.
Ownership of the Routing Decision
Someone on the product side chose the default model and the threshold that triggered it. That choice sat inside configuration files the ministry volunteers never opened. Continuous discovery treats the person who sets the default as part of the user loop, not outside it.
In the observed pilots, no one owned the routing rule after the initial setup. The volunteer who hit the button carried the outcome, yet she had no channel to report that the model had produced phrasing she needed to correct. The product team measured token spend and uptime, not revision rate on the final handout.
Ownership means attaching the model identifier and the routing condition to the same record the volunteer saves. When that link exists, the next discovery interview can start with the actual decision rather than the polished output. Without it, the routing layer remains a black box that quietly shapes the material delivered to the class.
Your Turn: Apply This Today
- Insert a mandatory log line that records the exact model name and routing rule immediately before any generation call in the Spanish baptism workflow.
- Add a second log line that captures the volunteer’s first edit or comment on the generated text, tied to the same session ID, before the file is saved or printed.
- Expose both log lines in the shared drive folder so the next person opening the handout sees which model answered and what changed.
- Run a 15-minute review of last week’s handouts with the two log lines visible and note any pattern between model choice and required edits.
- Update the agent prompt template to require these two logging steps as non-skippable fields before the workflow can close.
- Schedule one 20-minute interview this week with the volunteer who used the handout, starting from the logged model name rather than the final version.
Two recent posts examine the same tension between automated paths and required human checkpoints: “The Open-Weight Number That Still Required a Human Gate” and “Why Top PMs Still Route Specs Through Sunday Validation.”
I consult with product leaders on AI routing decisions, continuous discovery checkpoints, and ministry tech workflows. Let’s talk.