The children’s director shoved her laptop across the kitchen table. The screen showed three Figma frames she had built after midnight. One had a big green button for “daily verse push.” Another split the screen between parent notes and a child progress bar. The third tried to guess what a family needed based on last week’s attendance. This is the moment every AI ministry tool faces: the dataset points one way and the person who knows the work points another.
She pointed at the usage logs open in another tab. “None of this matches what those numbers say parents want,” she said. The logs came from the first two weeks of an AI pilot. They showed quick taps on random verses and almost no return visits after day three. She had watched real parents in her church that same week. They opened the app once during carpool, closed it when the toddler started crying, and never came back.
The data said simplify the verse feed. Her hands said something else. She had already seen what happened when the team followed the numbers last quarter. The feature shipped, the graphs looked clean for fourteen days, and then the actual ministry leaders stopped logging in.
Solomon faced two women who both claimed the same child. He did not run a survey. He did not average their stories. He called for a sword. The move looked reckless until the real mother spoke. The test was never about the data in front of him. It was about whose voice actually belonged in the room.
That same test sits in front of every team shipping the first version of an AI ministry tool. The early logs come from people who found the prototype link on social media or in a beta signup form. They are not the children’s director at 9:40 p.m. They are not the parent who only opens the app when the van is moving. Their clicks tell you what curious users do, not what exhausted ministry volunteers will keep doing after month three.
The Inbox That Became the Real Spec
The children’s director kept every parent email in one folder. She had printed the last month of messages and spread them on the table next to the laptop. One mother wrote that she needed the app to remind her of the exact craft supplies already in her kitchen drawer. Another asked for a single sentence she could read to her son while she stirred dinner. None of those requests showed up in the first two weeks of usage data because the parents who wrote them had not yet downloaded the pilot.
I watched her map each printed email to a screen. The green button disappeared. The progress bar stayed only if it could be filled by a single tap during pickup line. The verse feed became a list of three options, each tied to an object already in a typical home. She built the change in forty minutes. The logs never would have surfaced those constraints because the people writing the emails were not yet in the dataset.
Why the First Ten Users of an AI Ministry Tool Lied
The first ten users of that pilot were all early adopters who liked trying new church apps. They tapped through every screen the day they signed up. They answered every prompt. Their behavior looked like engagement until the team checked the actual church roster. None of those ten people served in children’s ministry. None of them had kids in the current Sunday school year. Their taps reflected curiosity, not repetition under real load. An AI ministry tool that trusts those first taps optimizes for the wrong people.
Solomon’s test worked because he forced the claimants to reveal what they would actually protect. The early dataset never creates that pressure. It records what people will do once, not what they will defend when the cost is their own time on a Tuesday night.
How Solomon’s Split Decision Shows Up in Roadmaps
Teams keep shipping the averaged version. They see the logs favor short verses and remove the longer reflection. They see the quick taps and kill the extra confirmation step. Six weeks later the real users have left because the thing they needed was never measured.
The fix is not more data. It is deciding whose voice gets the sword. On the current roadmap that means looking at every AI pilot feature and asking which one would survive if the only inputs were the printed emails from the children’s director’s folder. Features that only the beta users love get cut. Features that match the inbox stay even when the graphs look thin in week one.
Your Turn: Apply This Today
- Pick the single AI pilot feature your team plans to ship in the next four weeks and list the three data sources that currently justify keeping it.
- Find the printed or saved messages from the actual ministry volunteers who would use the feature after launch and map each one to a screen or flow.
- Remove any screen element that cannot be completed in one tap during a carpool or while stirring dinner.
- Run the remaining flow against the first ten users from your beta list and note how many of them actually serve in children’s ministry or lead volunteers.
- Delete the feature if fewer than half of those ten users match the real ministry role the inbox describes.
- Write the new version of the feature using only constraints pulled from the inbox and ship that version to the next five volunteers who email you this week.
The same pattern showed up in the post about the children’s director whose inbox became the real product spec and in the post about the 1997 lesson AI product teams keep missing.
I consult with ministry product leaders on early-stage AI features and deciding which data sources actually belong on the roadmap. Let’s talk.