The Question Every PM Gets After the First Agent Ships

Last night the ping came through at 11:17. The agent had booked a volunteer who’d already said no twice, then fired off the curriculum link to the entire wrong list. Three replies landed before I even opened my laptop.

I kept staring at the thread and wondering what I actually owned now. Not the clicks or the approvals—I’d already handed those off. Just the quiet gap between what the agent did and what should have happened.

That gap is where every PM ends up living once the first agent ships.

Where the regret actually showed up

In one children’s ministry tool rollout, the agent was given the narrow job of matching volunteer availability to upcoming events. It worked for the first two weeks. Then it started routing new volunteers to the same three sites because those sites had the cleanest data in the system. The other sites, with messier records, got zero new people. No one had written down what “fair distribution” actually meant when the agent had to choose.

The regret did not appear in the logs. It appeared three weeks later when two site leaders called the same week to say their teams were collapsing from lack of help. The PM had to stand in front of the directors and explain why the distribution looked the way it did. That conversation was the place the regret landed.

Schön would have called the moment the PM realized the data skew the “surprise.” The reflective move was not to add another approval step. It was to name, out loud and in the product spec, the exact outcome the team refused to accept even if the agent executed perfectly.

Shifting from gatekeeper to the person who names the failure mode

Most PMs still see their job as keeping bad things from shipping. After agents arrive, that posture becomes impossible to sustain. The agent will ship something unexpected because that is what agents do when they operate on live data. The PM’s new job is to name the failure mode that matters before the agent ever runs.

One team I watched tried to solve this by requiring every agent action to route through a human reviewer. Usage dropped to almost nothing within a month. Volunteers did not want to click through another screen. The PM eventually removed the gate and instead wrote a single sentence into the requirements: if the agent assigns a volunteer to a role they have never done before, the system must surface that fact to the site leader within twenty-four hours. That sentence became the owned regret point.

The shift is from “I approve this” to “If this goes wrong, here is the exact signal I will be measured against.” It is narrower and harder to hide behind.

Hiring senior ICs who already carry that regret

The strongest senior individual contributors I have seen in these environments are the ones who have already lived through an agent or automation decision that produced real damage. They do not need training on reflective practice. They already feel the weight when they look at a proposed handoff.

In interviews I now ask a single concrete question: tell me about a time an automated system you shipped created a problem you had not named in advance. The candidates who pause and then describe the exact person who had to clean it up are the ones who already carry the regret. The ones who pivot to process improvements or “lessons learned” slides usually do not.

Those ICs become the people who can sit with a product spec and point to the sentence that is still missing. They do not need permission to name the failure mode because they have already paid the cost of leaving it unnamed.

Your Turn: Apply This Today

• Pick the agent that has been running longest without daily human review and list the three outcomes the team has explicitly said it will not accept even if the agent executes its instructions perfectly.
• Write one sentence that names the single human who will be asked to explain the outcome if any of those three things happens, then add that sentence to the requirements doc this week.
• Schedule a thirty-minute meeting with the senior IC who has shipped the most agent work and ask them to mark the current handoff map with the exact place where regret would land if the agent ran for a full month unsupervised.
• Review the last three support tickets or ministry leader complaints that mentioned the agent and trace each one back to the missing failure-mode sentence rather than the missing approval step.
• Change the next hiring debrief to include the regret question above and score candidates on whether they describe the human cost or only the process fix.
• Identify the current handoff point in your largest agent flow that still carries no single name and attach one owner by the end of this sprint, even if the title on the org chart stays the same.

Two earlier posts on this blog walked through similar handoff problems: “The Agent That Kept Running the Schedule While No One Was Watching” and “The Three-Person Pod That Still Needed a Fourth Set of Eyes.”

I consult with product leaders and ministry technology teams on agent oversight, regret mapping, and handoff protocols. Let’s talk.