Letter to the PM Who Inherited the Agent Pilot

You’re the product manager who just got handed the AI agent pilot at a denomination that still prints bulletins on Wednesday afternoons. The brief landed in your inbox with three bullets: “explore agents,” “show value by Q3,” and “don’t break anything.” No one mentioned who actually uses the tools or how decisions get made across twenty regions that don’t share a CRM.

That handoff feels familiar because it is. Most pilots arrive this way: clean scope, unclear ownership, and an assumption that the technology will somehow find its own users.

This is the foundational misread that causes product teams to build agents that work in demos but stall in real ministry settings. The mistake isn’t technical. It’s assuming quality and usefulness will travel through existing channels without deliberate design for distribution and trust.

Charlie Munger’s latticework of mental models forces you to hold multiple lenses at once instead of defaulting to the one your team already knows. When you apply the distribution model, the incentives model, and the trust model together, the pilot stops looking like a feature rollout and starts looking like a handoff problem that needs visible human oversight at every layer.

The Bias That Treats Distribution as Someone Else’s Job

Most pilots optimize for the moment the agent produces an output. The team measures token count, accuracy against a test set, or time saved in a controlled workflow. None of those numbers travel outside the product Slack channel.

Faith communities run on relationships that predate any software. A children’s ministry volunteer who prints a lesson on Saturday night will ignore an agent that arrives through an app notification, no matter how accurate the content is. The same volunteer will try an agent if the regional coordinator they already text shares it first.

The bias shows up when the pilot plan ends at “launch.” No one owns the step where a pastor in another time zone decides whether this tool is safe to mention from the platform on Sunday. That decision sits outside the product roadmap until someone forces it onto the latticework.

How the Latticework Exposes Hidden Handoff Risks

Munger’s approach requires naming the second-order effects before they appear in usage data. Here, the second-order effect is the loss of visible human oversight. When an agent drafts a follow-up email to a family that missed three services, the recipient does not know who approved the tone or the timing.

Without an explicit owner for that approval step, trust erodes faster than any accuracy metric can recover. The denomination’s regions already operate with different comfort levels around technology. One region requires every automated message to carry a staff name and phone number. Another region has no policy at all. The agent does not know the difference.

The latticework reveals the risk by forcing the question: which mental model explains why adoption will vary by region even if the agent performs identically? The answer is the trust model, not the capability model. Once that is named, the pilot scope expands to include who will stand behind each output.

Rebuilding the Pilot Around Actual Ministry Workflows

Start by mapping the agent to the moments when a human already makes a judgment call. For the children’s curriculum team, this might be the Saturday night decision about which story to tell. The agent can surface three options with volunteer completion rates from the last quarter, but the final choice stays with the person who knows which kids will be present.

Next, attach distribution channels that already exist instead of creating new ones. The agent should appear inside the weekly email the regional coordinator sends to volunteers, not inside a new dashboard that requires a login. The coordinator’s name stays on the message. The agent supplies the content underneath that name.

Finally, instrument the handoff itself. Track how many outputs receive human review before they reach end users and how many are sent without review. Those two numbers tell you more about long-term adoption than any engagement metric collected inside the agent interface.

Your Turn: Apply This Today

  • List the three mental models you will hold for this pilot and write one sentence for each that describes a failure mode you have not yet designed against.
  • Identify the single existing channel, email list, group text, or printed packet, through which every region already receives new resources, then route the agent’s first output through that channel only.
  • Name the person in each region who must appear as the approver on any automated message and add their review step to the workflow before the next sprint planning.
  • Build a simple shared doc that records every agent output sent without human review in the last two weeks and review it with your stakeholder group this Friday.
  • Run the current demo script past one volunteer who has never used the pilot and note the first question they ask about who is responsible for the content.
  • Adjust the success criteria for the Q3 review to include one distribution number and one oversight number alongside any capability metrics.

This approach follows the same pattern behind the other letters and case studies on this site: distribution and trust decide adoption long before capability does, and the handoff between staff and region is where most pilots quietly succeed or fail.

I consult with product leaders running AI pilots inside denominations and mission agencies on distribution design, oversight workflows, and applying multiple mental models to adoption problems. Let’s talk.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.