The Review Layer We Built After an Agent Merge Broke Our Volunteer Text Thread

The children’s ministry lead held the phone to her ear with one shoulder while her free hand scrolled through the old room list on the shared spreadsheet. A volunteer on the other end kept repeating that the 3rd grade class still thought they met in the east wing. At 2:40 the repo notification popped on my laptop. The agent had already merged its rearchitected schedule module, complete with new status fields and a fresh calendar sync.

The lead hung up, opened the calendar, and immediately said the volunteer names had disappeared from the notification column. She refreshed twice. Nothing. The agent output had overwritten the exact handoff the team used every Tuesday to text the right people before anyone showed up at the wrong door. That gap is why we ended up building an agent review layer between the merge and the live schedule.

That single merge exposed the gap between clean code and live ministry movement. The new module looked complete in the pull request. It passed every automated test the team had written. Yet it severed the one thread the volunteers actually followed when plans shifted at the last minute.

Teresa Torres calls this continuous discovery: the work of staying in direct contact with users while the product changes, not after. She insists the discovery work never ends because the context keeps moving. In ministry products the context moves through text threads, not dashboards. The PM who treats the agent merge as finished work has already stopped watching.

The first problem showed up in the notification step. The old system sent a simple SMS whenever a room changed. The new module routed everything through an event status that required the volunteer to open an app. No one on the children’s team had ever opened that app during a Tuesday scramble. The lead had to rebuild the notification by hand, copying names back into the old spreadsheet so the texts would still go out.

This is not an edge case. It is the normal state of ministry handoffs. Volunteers operate on whatever channel already works when the kid starts climbing the counter and the phone rings again. An agent that optimizes for data cleanliness without preserving that channel creates new work instead of removing it.

The second problem revealed itself when I pulled up the actual text thread the lead kept open on her phone. The PR diff showed field renames and a new foreign key. The thread showed a volunteer asking whether the craft supplies would still be in the room by 6:15. The agent had no way to answer that question because the rearchitecture had dropped the supply note field entirely. Continuous discovery required reading the thread first, then checking whether the module still supported the conversation that actually happened.

I started keeping a running log of every text that mentioned the schedule. Each entry noted what information moved between people and which tool carried it. After a couple of weeks the pattern was clear. The shared calendar was only the official record. The real coordination lived in the thread that updated the calendar. Any agent change had to leave that thread intact or the entire handoff broke.

The third problem followed directly. The review layer could not live inside the agent or inside the calendar alone. It had to sit between them. One person on the team now reads the agent output, checks the current text thread, and decides what actually gets pushed to the shared calendar. The step takes a few minutes on a normal Tuesday. It prevents the scramble that used to happen when the wrong room list reached volunteers.

That review layer is not extra process. It is the fit test Torres describes. The PM keeps an explicit check between every agent change and the live handoff the ministry team still runs. Without it, faster iteration simply multiplies the places where the handoff can fail.

The merge that looked clean broke the volunteer notification step

The agent’s schedule module replaced a manual lookup with an automated status field. On paper the change reduced clicks. In practice it removed the one field the lead used to trigger the SMS blast. She had to recreate the list by copying names from the old spreadsheet into a separate messaging tool. The time saved in the repo was spent again on her phone while the volunteer waited.

Ministry teams do not measure success by merged pull requests. They measure it by whether the right people arrive at the right room with the right supplies. When the notification step vanished, arrival times slipped noticeably. That delay compounded across every grade level that met on Tuesday nights.

The pattern repeated with other agent merges. Each one optimized for internal data consistency while ignoring the external thread that carried the actual coordination. The lead began keeping a printed copy of the old list next to her laptop as backup. She did not trust the new module to preserve what the text thread needed.

Continuous discovery meant watching the actual text thread, not the PR diff

Torres’s method requires ongoing contact with the people who use the product in its real context. For this team the real context was the group text that updated every time a volunteer called in sick or a supply ran out. The PR showed clean schema changes. The thread showed the supply note that still needed to travel with the room assignment.

I sat with the lead for one full Tuesday cycle and copied every message that touched the schedule. The agent output never appeared in those messages. The lead translated the agent data into the thread format before sending anything to volunteers. That translation step was invisible in the repo but essential to the handoff.

Once the team saw the gap, they added a simple rule. No agent change could remove a field that appeared in the recent schedule-related texts. The rule forced the PM to read the thread before approving the merge. The review became part of the discovery loop instead of an afterthought.

The agent review layer had to sit between the agent output and the shared calendar

The layer is a single human checkpoint that reads both the agent proposal and the current text thread. It asks one question: does this change still let the lead send the right information in the format the volunteers already expect? If the answer is no, the change waits until the thread format is protected.

This checkpoint adds minutes, not hours. It prevents the hours the lead used to spend rebuilding lists after a merge. The team now runs the agent on a staging copy first, then applies only the pieces that survive the thread check. The shared calendar stays reliable because the review layer filters what reaches it.

The same pattern applies to other ministry agents. Sermon prep tools, curriculum planners, and attendance trackers all create clean outputs that still need translation into the channels volunteers actually use. The PM who skips the live handoff observation will keep shipping merges that look finished and still require manual repair.

Your Turn: Apply This Today

• Pick one active ministry agent this week and export the last ten messages from the volunteer text thread that mentions its output.
• Compare each message against the agent’s current fields and note which pieces the thread still carries manually.
• Add a single checkpoint task to the evaluation loop that requires reading the thread before approving any new merge.
• Run the next agent update in a staging copy first, then apply only the fields that match the thread format.
• Measure the time the ministry lead spends rebuilding lists after the change and compare it to the previous cycle.
• Document the checkpoint rule so the next PM who inherits the workflow knows the thread must stay intact.

The approach aligns with the evaluation layer described in “The PM Who Just Shipped an Agent and Now Needs an Evaluation Layer” and the handoff details in “Letter to the PM Who Just Inherited the Ministry Agent Schedule.”

I consult with ministry product leaders on agent evaluation layers, live handoff observations, and fitting rearchitectures to existing volunteer workflows. Let’s talk.