Blog

The real test for any AI tooling budget is whether a volunteer can name one concrete change in their weekly work without being prompted by a product manager. Most line items never clear that bar. They multiply compute costs and internal velocity metrics while leaving the actual user path untouched.

This is the foundational misread that causes product teams to approve tooling spend based on engineering demos rather than observable volunteer behavior. The spend grows because the demo looks impressive, yet the volunteer still prints the same PDF at the same time each week.

John Wesley’s first rule — do no harm — offers the clearest lens here. Applied to tooling, it means refusing any hidden cost that quietly increases friction for the people who never see the roadmap.

The AI Line Item That Grew While Velocity Stayed Flat

One children’s ministry platform added three successive AI code-generation tools over eighteen months. The engineering team reported a 40 percent rise in pull requests merged. The volunteer completion rate on the curriculum builder stayed flat at 62 percent.

The new components arrived as internal micro-features: smarter autocomplete in the lesson editor, auto-tagging for activity files, and a prompt that suggested reorderings of the printed handout. None of these altered the seven-minute window in which a volunteer opens the material on a phone and decides whether to print or abandon it.

Feature velocity rose on the internal dashboard. The number of volunteers who reached the print step did not. The budget line item kept its justification because the demos never required a volunteer to describe the difference.

Wesley’s Do-No-Harm Test on the Hidden Line Item

Wesley’s rule forces a different question: does this spend create new friction that only appears after the contract is signed? In practice this shows up as added login steps, new file formats that require conversion, or prompt interfaces that demand the volunteer rephrase their intent before the system accepts it.

A sermon resource team once introduced an AI summarizer for weekly emails. The tool reduced drafting time for staff but required volunteers to confirm the summary matched their printed notes before forwarding. The confirmation step added forty seconds per email. Across hundreds of weekly users, the cumulative harm exceeded the drafting savings within one quarter.

The rule is simple to apply once named: if the cost cannot be explained to a volunteer in one sentence that ends with a visible action they already perform, the spend fails the test.

The Single Metric That Justifies the Line Item

Research on software ROI, including Harvard Business Review on measuring value, shows that a tool budget survives only when one usage metric ties the line item to real behavior change.

The only metric that held steady across recent quarters was the percentage of volunteers who completed their task without opening a support ticket or re-downloading the same file twice. AI features that improved this number stayed funded. Those that improved only internal velocity metrics were quietly deprioritized in the next planning cycle.

This metric survives because it cannot be gamed by a demo. A volunteer either finishes or does not. The tooling spend must produce a measurable lift here or it is, by definition, a line item no demo can justify.

Teams that adopted this filter reduced their AI tooling stack by roughly one-third while holding volunteer completion rates steady. The remaining spend now maps directly to actions the volunteer can name without prompting.

Your Turn: Apply This Today

• Open the current AI tooling invoice and select the single largest monthly line item.
• Write one sentence that describes the exact volunteer action this item must improve within the next seven days.
• Run a five-user test this week using only that sentence as the success criterion.
• Remove or renegotiate the line item if the test shows no lift in the named action.
• Repeat the same mapping for the next two largest AI-related expenses before the next budget review.
• Document the outcome in the same place the engineering velocity numbers are tracked so the comparison is visible to leadership.

The Compensation Split No Playbook Prepared PMs For and Trust Logs Beat the Next Feature Sprint both trace similar gaps between internal metrics and observable user behavior. The same pattern appears whenever tooling spend outruns the volunteer’s ability to name its effect.

I consult with product leaders and ministry tech teams on AI tooling justification and volunteer outcome mapping. Let’s talk.

You’re the product manager who just inherited the Fable Agent mandate at a denomination you only half-understand, and the brief says to ship something that feels like an AI assistant without breaking any existing volunteer workflows. The leadership team keeps repeating the same line about “safe defaults” and “conservative guardrails,” which sounded reassuring in the first three meetings. Now the first prototype sits in front of you and every suggested prompt returns the same careful, slightly generic answer. Inheriting an agent mandate like this means owning the decisions the model quietly makes for you.

That feeling is the signal. Fable 5’s training pushes it to stay inside a narrow band of acceptable ministry language and task patterns. The model was never rewarded for guessing what a children’s ministry volunteer actually needs when the craft supplies run out at 6:47 p.m. on a Tuesday. Most teams treat that narrowness as a feature. It is the exact place where product edge leaks away.

Charlie Munger’s circle of competence makes the cost visible. He warned that knowing the boundaries of what you understand is more valuable than expanding the territory you claim to know. When the agent’s defaults define the circle, every new workflow gets pulled back inside it. The denomination’s actual edge lives outside that circle, in the messy handoffs between volunteers who have never opened the app before. The product decision is whether to leave the model inside its comfort zone or to force it across the boundary on purpose.

Where the agent mandate stops short on volunteer orchestration

Volunteer orchestration is the workflow most teams test last because it looks simple on a flowchart. Assign curriculum, send reminder, mark complete. Fable 5 handles the first two steps inside its training distribution. It suggests standard email language, attaches the right PDF, and logs the assignment. The moment the volunteer replies with anything outside the script—“my kid has a fever and the printer is jammed”—the agent retreats to a safe, non-committal response or hands the thread back to a human.

I watched this exact failure on a children’s ministry rollout that served two hundred congregations. The agent could schedule the lesson but could not reroute the physical materials when a volunteer was suddenly unavailable. Completion rates dropped because the system kept treating the volunteer as a reliable node instead of a person whose availability changed daily. The safety layer had been tuned to avoid promising anything it could not guarantee, which is reasonable until the guarantee itself becomes the blocker.

The pattern repeats whenever the task requires real-time adaptation rather than pre-approved steps. The model’s circle of competence includes “send the lesson plan.” It does not include “decide whether the substitute volunteer can teach the same material with only a printed outline and ten minutes of prep.” That decision sits outside the training distribution, so the agent never offers it.

The prompt pattern that breaks the agent mandate safety layer

Anthropic’s own core views on AI safety describe why agent systems need human judgment at exactly these boundaries, and the agent mandate makes that responsibility yours.

The safety layer loosens when the prompt forces the model to declare its own uncertainty before it acts. Instead of asking the agent to generate the reminder email, you ask it to list every assumption it is making about the volunteer’s context and then rewrite the email once those assumptions are stated. The extra step is small in tokens but large in effect. It moves the output from the center of the circle toward the edge.

Teams that discover this pattern usually find it by accident. One product lead added the line “state the three facts about this volunteer you are least certain about” to every orchestration prompt. The agent began surfacing gaps the product team had never logged: the volunteer who only checks email on Sunday afternoons, the one whose printer only works at the church building, the substitute who has never taught without a co-leader. Those gaps became the actual product surface. The model was no longer optimizing for safe language; it was optimizing for explicit ignorance.

The same pattern works on curriculum adaptation. Ask the agent to generate the lesson for a mixed-age group, then immediately require it to name the single age range where the activity will fail. That second instruction pulls the output outside the conservative default and into the specific constraints of the room the volunteer actually walks into.

How the agent mandate shifts competence boundaries

Once the agent is allowed to produce the first draft of any artifact, the ownership question changes. The PM who treats the draft as a finished suggestion keeps the circle of competence exactly where Fable 5 drew it. The PM who treats the draft as raw material that must be stress-tested against one real volunteer’s constraints moves the boundary outward. The difference shows up in the revision log, not in the initial output.

At SermonCentral we learned this when we let the agent draft volunteer instructions for a new series. The first versions were polished and complete. They also assumed every volunteer would read the full document before arriving. When we forced the agent to rewrite after a volunteer read the draft aloud in real time and stumbled on the third paragraph, the instructions shortened, the steps reordered, and the completion rate rose. The model had not become more capable; the team had simply stopped accepting output that stayed inside its original circle.

The boundary keeps moving only if the team measures something the model cannot see. Volunteer completion rate after the first contact is one such measure. Time from assignment to first material request is another. Both sit outside the agent’s training data, so they remain useful signals long after the language inside the agent has been optimized.

Your Turn: Apply This Today

• Pick one existing Fable Agent workflow that touches volunteers and add the exact line “state the three facts about this volunteer you are least certain about” to the prompt; run it on the next five real assignments and note which facts the model consistently misses.
• Take the agent’s most recent draft for any volunteer-facing email or instruction set, read it aloud to one actual volunteer within 24 hours, and rewrite only the sentence that caused the first hesitation.
• Log the time between assignment and first material request for the next ten volunteer tasks; compare the numbers against the agent’s predicted timeline and mark every gap larger than two days.
• Replace the default “send reminder” step in one orchestration flow with a prompt that forces the agent to name the single constraint that would make the reminder useless, then adjust the send time accordingly.
• Choose one children’s ministry workflow and require the agent to generate both the main lesson and the single-age-range failure case before any output is shown to a human reviewer.
• At the end of this week, count how many of the six prompts above produced at least one output that the model originally refused or softened; keep only those revised prompts in the active library.

The same boundary problem shows up in how teams decide what counts as finished once AI writes the first draft; that pattern is laid out in The Months I Outsourced My Sense of What Counts as Finished. It also appears in the decision to keep human judgment visible rather than sprinting toward the next automated feature, which Trust Logs Beat the Next Feature Sprint examines in detail.

I consult with product leaders shipping AI features into volunteer and ministry workflows on prompt boundary testing, completion-rate instrumentation, and first-draft ownership. Let’s talk.

The cursor sat on the shared screen while the church tech lead’s finger hovered over the trackpad. One click and the entire volunteer onboarding spec filled the document—every permission tier, every reminder cadence, every fallback for a missed shift. The lines came straight from the model output. They read like something shipped already: clean states, graceful error paths, even a note about mobile push timing that matched the style guide. No one in the room mentioned the actual Saturday line at the welcome desk, the one where parents juggle coffee and name tags while two kids argue over the same blue sticker. The model had quietly chosen the safe path, and no one had noticed.

That paste replaced three sentences the team had written the week before. Those sentences described the moment a new volunteer stands at the table and asks which form to sign first, then waits while the tablet loads the wrong page. The model version never included the wait. It assumed the form would already be open and the signature field highlighted.

The difference showed up later when the same lead tested the flow on a borrowed phone during the actual sign-up hour. The polished steps collapsed at the handoff between printed name tag and digital check-in. The model had optimized for completion rate. It had never seen the table.

Tony Fadell built the first iPod and the early iPhone under the same constraint. He refused to let early usage data or simulated models dictate the v1 shape. Data at that stage only reflected what the team already knew how to measure. Real judgment came from standing in the room with the first users and noticing which physical action they performed before they ever touched the screen. Fadell treated that observation as the only reliable input for the first release.

A model-generated safe path always reads as finished, which is exactly why AI agents skip that observation by default. They optimize for the average documented flow across thousands of prior examples. The result is a v1 that looks complete because every listed edge case carries a solution. The edges that matter locally never appear in the training distribution.

The model removed the awkward handoff because awkwardness does not register as a measurable failure state. In the volunteer flow the handoff lasted eight seconds on average. Those seconds contained the only moment the new person decided whether they belonged in the room. The agent replaced them with an automatic redirect. The redirect worked in staging. It left the actual parent holding a tablet while their child reached for the sticker sheet.

Fadell’s stance makes the cost visible. He would have required the team to keep the handoff in the first version even when every metric suggested removing it. The metric only captured task completion. It missed the decision to return next week. Ministry products live or die on that second decision, not the first tap.

The same pattern appears whenever an agent generates an entire feature branch. The code passes every test the prompt described. It never contains the one condition the builder only notices after watching the real person hesitate.

Re-inserting that condition requires an explicit step the model cannot perform. The builder must name the single lived moment the spec still ignores. That moment becomes the required test before any other acceptance criteria are written. Everything else can stay generated. The one judgment stays manual.

This rule collides with current agent workflows because the safe path already satisfies the prompt. The model will happily generate five alternative versions, each one still missing the same lived detail. Only the builder who stood at the table can supply it. The model has no table.

The practice forces a smaller v1. The smaller surface makes the missing judgment impossible to overlook. It also makes the first user test cheaper. The team ships the awkward handoff on purpose, watches what happens, then decides whether the metric or the moment was right.

The moment the model handed me the safe path

The team had already run the generated flow against their internal checklist. Every required field existed. Every confirmation message matched the tone document. The only remaining task was to map the digital steps onto the physical table that actually existed on Saturday mornings.

When they tried the mapping, the redirect that looked efficient in Figma forced the volunteer to turn their back on the parent for three seconds. That turn broke the single point of eye contact that kept new people from walking away. The model had treated the turn as irrelevant because no ticket described it.

The handoff survived only because one person on the team refused to delete the three original sentences. Those sentences stayed in the spec as a separate acceptance criterion labeled “observed Saturday behavior.” Without that label the redirect would have shipped.

What Fadell’s v1 stance exposes in agent-built flows

Fadell treated early data as a mirror of existing assumptions rather than new information. The same mirror effect appears when an agent builds from prompt alone. The prompt already encodes what the team believes it needs. The generated flow therefore confirms the belief instead of challenging it.

The confirmation shows up most clearly in the edge cases the model adds. They are the edges that appeared in other products. They are rarely the edges that appear at 8:45 a.m. when the children’s check-in line backs up into the parking lot. Those edges only become visible after the builder watches the line form.

Keeping Fadell’s rule means the first version ships with fewer automated steps, not more. The reduced surface leaves room for the observation that the model cannot generate. The observation then becomes the actual scope for iteration two.

Re-inserting the judgment the safe path skips

The judgment usually hides inside a single sentence that describes a physical action rather than a screen state. In the volunteer flow the sentence was: “The parent sets the coffee cup down before they can sign.” That sentence dictated the height of the tablet stand and the size of the signature field. No model output contained it.

The builder must add the sentence after the agent finishes its work. The addition happens in the same document, not in a separate research file. Once the sentence exists, every subsequent change must preserve the action it describes or explicitly replace it with a better observed action.

This single addition prevents the safe path from becoming the only path. It also gives the team a concrete test: does the new volunteer complete the action without turning away? The answer arrives from the table, not from the analytics dashboard.

Your Turn: Apply This Today

• Open the current v1 spec you generated with an agent this week and highlight every paragraph that describes a screen state instead of a physical action the user performs first.
• Replace one of those paragraphs with a single sentence written from direct observation of the real moment the user hesitates.
• Mark that sentence as a non-negotiable acceptance criterion that must pass before any automated edge case is implemented.
• Run the revised spec past the person who will actually use the flow and ask them to change only that one sentence if it no longer matches what they do.
• Delete every generated step that now conflicts with the updated sentence, even if the deletion reduces the visible feature count.
• Ship the smaller version to a single location this week and record whether the observed hesitation disappears or changes form.

The Judgment Call No Dataset Will Make for Your v1 and The Months I Outsourced My Sense of What Counts as Finished both trace the same pattern of handing scope decisions to external sources before the builder has named the one moment that matters.

I consult with product leaders building AI features for ministry tools on v1 scoping, judgment integration, and agent-assisted flows. Let’s talk.