I once spent nine months building an AI sermon-prep assistant that scored 92 percent on internal accuracy benchmarks. I assumed the quality metric would carry it across the finish line into volunteer workflows. Three pilots later the tool sat unused in two of the churches and barely touched in the third, while the teams reverted to their old copy-paste routines. The cost was not just the wasted engineering hours; it was the lost credibility with ministry leaders who had cleared their calendars for the pilot and then watched nothing change in their actual week.
That outcome forced me to admit the assumption I had carried into every ministry AI project up to that point: if the model performed well in isolation, the rest would sort itself out. The assumption was convenient because it kept the scope of my work narrow. It also proved expensive each time distribution, positioning, and the handoff to non-technical users were left as afterthoughts.
How the bias showed up in three pilots I ran
The first pilot placed the assistant inside a children’s ministry platform similar to the one I helped shape for Sermons4Kids users. The model could generate story outlines in seconds. Yet the volunteer onboarding flow still required a logged-in account, a desktop browser, and a five-step export to the print queue the volunteers actually used on Sunday morning. After six weeks the completion rate for the AI-generated outlines sat at 11 percent while the older static curriculum pages held steady at 68 percent.
In the second pilot, aimed at small-church pastors, we positioned the tool as an “AI co-preacher” inside an existing sermon resource site. The language in the marketing email framed it as a breakthrough. Pastors who clicked through landed on a clean interface that still demanded they learn a new prompt syntax and then manually reconcile the output with their denominational review process. Two months in, only four of the thirty invited pastors had run the tool more than once.
The third pilot tried to shortcut distribution by embedding the model directly into an existing church management system. The integration passed every technical test. What the logs later showed was that the handoff step—moving an AI-generated pastoral care note from the system into the actual follow-up workflow—still required a separate spreadsheet the staff kept on a shared drive. The model stayed technically sound and practically invisible.
Each time the pattern repeated: strong model performance inside the sandbox, weak or nonexistent movement once the artifact reached the person who had to act on it by nine o’clock Tuesday morning.
Munger’s model that forces the other factors into view
Charlie Munger described a latticework of mental models as the habit of pulling ideas from multiple disciplines so no single narrow frame dominates the decision. The quality-only frame is itself a model—one borrowed from software engineering where a better algorithm often does win. Ministry workflows, however, sit at the intersection of volunteer time scarcity, print-first constraints, and relational trust that must survive a failed output.
When the latticework includes Munger’s own inversion model—ask what would guarantee failure—the missing pieces surface quickly. An AI tool fails in a church not because its perplexity score is too high but because the distribution path never intersects the actual decision point, the positioning triggers the wrong mental category (“another dashboard”), and the handoff leaves the non-technical user holding raw text with no next action defined.
Applying the latticework means treating distribution, positioning, and handoff as first-order variables measured with the same rigor once reserved for model accuracy. It also surfaces second-order effects: a poorly positioned tool burns relational capital with gatekeepers who then block later experiments, even better ones.
What changes when you stop optimizing the model first
The shift begins with mapping the exact physical or digital surface where the output must appear. In the children’s ministry case that surface was the single sheet of paper handed to a volunteer at 8:15 a.m. Once that surface was named, the pilot moved from prompt engineering to a print-queue integration that cut three clicks and removed the login requirement.
Positioning changes next. Instead of announcing an “AI co-preacher,” the revised framing described a “first-draft generator that still needs your voice.” The language matched the existing mental model pastors already held about sermon prep and lowered the activation energy required to try it.
Handoff receives the same concrete treatment. The pastoral care pilot added an explicit “copy to follow-up list” button that wrote directly into the spreadsheet already in use. Adoption among the same staff group moved from near zero to consistent weekly use within four weeks.
The model itself was not degraded; its scope simply stopped expanding until the surrounding lattice was in place. Accuracy became a supporting metric rather than the lead indicator.
Your Turn: Apply This Today
- Open your current AI experiment’s project doc and list the three non-quality factors—distribution path, positioning language, and handoff step—on a single line each.
- For the distribution path, write the exact URL, email subject line, or physical handoff the end user will encounter first and schedule a test with one actual user this week.
- For positioning, draft the one-sentence description you will use in the first message to that user and run it past a non-technical colleague for clarity before sending.
- For the handoff, identify the exact next artifact the user must produce after seeing the AI output and add one automated or one-click bridge to that artifact.
- Log the three factors plus the corresponding test steps in a shared note visible to the full pilot team by end of day tomorrow.
- Review the log in your next stand-up and mark which factor moved closest to the end user’s actual workflow surface.
The same lattice that exposed the bias in my earlier pilots now shapes every new experiment. Two recent posts on the blog explore adjacent angles: “The Bias That Quietly Killed Three Ministry Pilots” and “The Routing Layer No Ministry Pilot Logged”.
I consult with product leaders and ministry technology teams on AI pilot design, distribution mapping, and non-technical user handoffs. Let’s talk.