Blog

You’re the product manager who walked into a Monday standup and learned the denomination’s new agent would now own volunteer routing, curriculum reminders, and follow-up texts for parents who miss check-in. No one asked how the agent would know which family just moved or which small group leader quit without notice. They just handed you the mandate and the dashboard.

You know the tool will generate schedules faster than the old spreadsheet ever could. You also know the first time it books two overlapping events on the same night, the phone calls will come to you, not the vendor.

This is the foundational misread that causes product teams to treat agent deployment as a handoff rather than a permanent increase in surface area they must personally monitor. Teresa Torres’s continuous discovery framework makes the cost visible: opportunity and regret both compound when the people closest to the work stop talking to the people who experience the work every week.

The handoff that never stayed handed off

The first agent your team shipped handled basic reminder texts for children’s ministry volunteers. Engineering owned the integration, data owned the success metrics, and you moved on to the next initiative. Within six weeks the agent began texting the wrong shift leaders because a children’s director had updated the volunteer roster in the church management system rather than the spreadsheet the model was trained on.

The failure landed in your inbox because the director knew your name from the original requirements meeting. No one else on the team had that relationship. The handoff had never actually left your desk; it had only left the meeting room.

Continuous discovery insists that the same person who decides what the agent should optimize must also sit with the person whose week gets wrecked when the optimization is wrong. In ministry settings that person is usually a volunteer coordinator working two jobs, not a salaried staff member who can absorb the coordination cost.

Discovery loops that must run on live schedules

Torres’s interview cadence assumes weekly customer contact. When the product is an agent that books rooms and sends texts, the relevant signal arrives on Tuesday at 4:17 p.m. when the youth pastor texts you directly because three families showed up to an event that no longer exists on the calendar.

You cannot batch those moments into a quarterly research sprint. The loop has to close the same day the agent creates the mismatch, or the next week’s schedule will repeat the same error with different families.

Teams that delegate the loop to an analyst or an LLM summary lose the texture. The analyst sees a drop in response rate. You hear that the volunteer who always covers last-minute nursery shifts stopped checking her phone because the agent kept asking her to cover events she had already declined three times. That distinction changes the next prompt and the next guardrail.

The boundary that keeps agents from forking real work

One church ran an agent that could propose small group meeting times based on member availability pulled from multiple calendars. The model suggested a Tuesday night slot that worked for the majority. It did not know the associate pastor’s standing counseling commitment or that the host home had just lost internet. Two groups met anyway and one dissolved after the confusion.

The boundary is not a technical constraint on the model. It is an explicit rule that certain categories of schedule change must route through a human before the agent sends any message. The product manager owns the definition of that category because only the product manager sees both the model’s output volume and the downstream coordination cost when the output is accepted without review.

Continuous discovery treats that boundary as a living artifact. Every time an agent creates a fork that requires manual cleanup, the boundary gets one line tighter or one exception wider. The work is never finished because the agent’s context window keeps expanding.

Your Turn: Apply This Today

• Pick the single agent your team shipped most recently and list every coordination failure reported in the last fourteen days; schedule thirty-minute calls with the two people who felt the impact most directly before Friday.
• Write the exact sentence you will send the next time the agent books overlapping events and save it as a template so you are not composing under pressure at 6 p.m.
• Define one category of change the agent may never execute without human review and add it to the prompt and the monitoring dashboard this week.
• Block two hours on your calendar every Tuesday for the next month labeled only “agent regret review” and refuse to move them for internal meetings.
• Ask the volunteer coordinator at the largest site using the agent what single piece of information the model still lacks and ship the smallest possible data fix that week.
• Document the last three times you personally had to intervene after the agent acted and turn each instance into one new discovery question for the following week’s interviews.

The same pattern appears in both “The Question Every PM Gets After the First Agent Ships” and “The Night the Voice Agent Forked Three Ministry Schedules at Once.” The difference is whether the product manager treats the agent’s output as someone else’s problem to clean up.

I consult with product leaders shipping agents into ministry and nonprofit workflows on continuous discovery loops, regret surface ownership, and live coordination boundaries. Let’s talk.

A 2025 internal audit across 120 mission-driven AI teams showed that 71 percent now publish explicit value statements on their public sites. The same audit revealed that only nine of those teams had logged a single instance where a value statement blocked a planned feature release in the preceding twelve months.

The obvious reading is that values language has finally taken hold in faith-tech and nonprofit AI work. The data points the other direction. Most teams adopted the language after seeing it succeed elsewhere, then treated the statements as external-facing copy rather than hard limits on what ships. The result is a growing stack of polished principle pages that never touch the backlog.

John Wesley’s three simple rules worked because they functioned as live constraints on daily decisions, not as aspirational posters. “Do no harm,” “do good,” and “stay in love with God” were not discussion prompts; they were tests applied to specific choices about buildings, money, and travel. When the test failed, the activity stopped. That same test logic is what most current value statements lack.

Values as product constraints, not slogans

Most church-tech value statements read like they were written by a committee that never had to say no to revenue. They list words such as “stewardship,” “formation,” and “trust” without defining the observable behavior that would violate them. The absence of a violation test turns the statement into marketing.

When I worked on curriculum tools for volunteer teachers, the team adopted a rule that any new feature had to be completable in seven minutes or fewer on a printed page. The rule was not aspirational. It was checked against every prototype. Features that required more time or a screen were cut before they reached users. The constraint changed the product more than any vision slide ever did.

The same pattern appears in AI work. A value that cannot name the exact decision it would kill is not yet a constraint. It remains a slogan until the first time it removes scope or delays a launch.

How Anthropic’s approach differs from typical church tech culture copy

Anthropic’s constitution for Claude was written as a set of principles that raters and engineers must apply to concrete outputs. The principles are scored, not admired. When a generated response fails the test, the model is retrained or the prompt is rewritten. The constraint sits inside the release process, not beside it.

Most church-tech teams I see do the reverse. They publish a values page, then ask the product team to “keep the values in mind” while shipping. The values never enter the ticket template or the model evaluation rubric. The language travels; the test does not.

The difference is not size or funding. It is whether anyone on the team is paid to reject work that fails the stated rule. Without that role, the values remain external.

Where the constraint actually changes shipping decisions

One team building an agent for sermon preparation added a rule that the agent must surface the original source material for every generated illustration and must never produce an illustration that could not be verified in the pastor’s own library. The rule was applied at the prompt layer and again in the review queue. Two planned features were cut because they would have required the agent to generate unverifiable stories. The decision cost a quarter of planned scope, but it kept the product inside the constraint.

Another team working on volunteer scheduling software adopted the rule that no schedule change could be pushed without an explicit human confirmation step from the volunteer coordinator. The rule blocked an otherwise attractive automation that would have auto-swapped shifts based on predicted availability. The automation was never shipped. Retention among coordinators rose because the tool stopped making decisions they could not see or override.

In both cases the constraint was written before the feature was built, not reviewed after. That timing is what separates a working rule from a retrospective justification.

Your Turn: Apply This Today

• Pick one existing value statement your team has published and write the single sentence that would cause a feature to be rejected this quarter.
• Add that sentence to the definition-of-done checklist in your current AI project tracker before the next sprint planning meeting.
• Schedule a 30-minute review with the engineer or PM who owns the next model release and ask them to name the specific output that would fail the sentence.
• Remove or rewrite any value language on your public site that cannot produce a comparable rejection test within one hour of discussion.
• Log the first instance this month where the test removed scope, changed a prompt, or delayed a launch, then note the ticket number and the date.
• Repeat the exercise with one additional value statement next week and compare how many decisions each constraint actually touched.

The Ministry App That Chased Every Model Release and Lost Volunteer Trust shows what happens when copied language never becomes a live test. The Question Every PM Gets After the First Agent Ships tracks the moment teams realize the values they published never reached the release checklist.

I consult with product leaders at mission-driven AI teams on turning published values into enforceable shipping constraints and building review processes that survive model updates. Let’s talk.

A product leader asked me last month whether AI code generation meant their small team could finally skip the second human review on new features. The answer is no. Without that verification step, AI-assisted builds create ownership debt that surfaces exactly when schedules, volunteers, or users depend on the output.

Charlie Munger’s latticework of mental models makes the reason plain. Munger treated every new tool as one more model that must be cross-checked against several others rather than trusted on its own. AI generation supplies one model: statistical likelihood of correct syntax and logic. It supplies none of the models that matter in deployed systems—edge-case user behavior, downstream schedule impact, or the cost of a single wrong assumption reaching a real person. Teams that treat the generated code as the finished article are using only one model and calling the job done.

Verification as the new core engineering skill

The skill that now separates reliable teams from fragile ones is not prompt writing. It is the ability to hold generated output at arm’s length and run it through a second, slower mental model before it touches anything that affects live data.

In practice this looks like a short, repeatable checklist applied to every AI-produced function or workflow. The checklist asks three questions the model cannot answer: What assumption did the generation make about user intent? What happens if that assumption is off by one hour or one volunteer? Who owns the correction if the assumption fails on a Thursday night before a weekend event?

Teams that embed this checklist into the pull-request template see the same pattern. The first few times feel slow. After the third or fourth instance the verification step becomes faster than the original generation because the reviewer already knows the failure modes that matter in their context. The latticework grows stronger with each pass, exactly as Munger described.

Without the checklist the team ships faster for a short window, then spends the next quarter answering support tickets that trace back to one unexamined assumption in a generated script. The net velocity gain disappears once real users are involved.

Why ministry contexts expose the gap faster than startups

Startups can sometimes absorb a broken feature behind a feature flag or a small beta cohort. Ministry tools rarely have that buffer. A single incorrect schedule generated for children’s ministry volunteers reaches dozens of families who have already rearranged their weekend. The cost is paid in trust and in actual attendance the following Sunday.

The same pattern appears in resource platforms that serve volunteer teachers. When AI suggests a lesson outline that misreads the age range or the available materials, the volunteer standing in the classroom at 9:15 a.m. discovers the problem first. There is no second deployment cycle before the damage is done.

Ministry environments also concentrate the downstream effects. One schedule error does not affect a single user; it affects every family that volunteer was meant to serve. The latticework required here must include models of calendar friction, volunteer availability, and parental expectation. Statistical likelihood alone is not enough.

The cost of skipping the second human check

The most expensive failures I have watched did not come from bad prompts. They came from teams that accepted the first clean-looking output and moved it into production because the generation looked complete. One generated scheduling rule assumed volunteers could be reassigned with twenty-four hours’ notice. The assumption was statistically common in the training data. It was operationally false for the actual volunteer pool.

Recovering from that single rule required three weeks of manual overrides, personal phone calls, and rebuilt trust with the volunteer coordinators who had relied on the tool. The generation step had taken minutes. The verification debt took far longer to clear.

Munger’s point was never that new tools are suspect. It was that every new tool adds a model that must be tested against the others already in use. When the second human check is removed, the latticework stays incomplete and the first real-world collision reveals the missing pieces at the worst possible moment.

Your Turn: Apply This Today

• Choose the single AI-assisted workflow your team ships most often this month and write a three-question verification checklist that must be answered in the pull request before merge.
• Run the last three pieces of generated code through that checklist and log the one assumption each piece made that was not visible in the output itself.
• Assign the next generated feature to two people: one to accept the generation, one to run the verification questions and record the time required.
• Compare the verification time against the time spent fixing the last production incident that traced to an unchecked generation; use the comparison to set a minimum review duration for the next sprint.
• Document one concrete failure mode the checklist caught this week and add it to the shared list the team reviews before every new AI-assisted build.
• Schedule a thirty-minute retro in two weeks to drop any checklist item that never surfaces a problem and replace it with the next observed failure pattern.

The verification layer is the difference between shipping quickly and owning the result when real schedules and real volunteers are involved. Two earlier posts that explore the same tension are “The Night the Voice Agent Forked Three Ministry Schedules at Once” and “The Question Every PM Gets After the First Agent Ships.”

I consult with product leaders and ministry technology teams on verification discipline, AI-assisted workflow design, and downstream ownership costs. Let’s talk.