Blog

Posted on

The PM Role That Stopped Shipping and Started Choosing

Workflow diagram and product brief representing product manager decisions

The question I keep getting is this: with agents collapsing the build step, what does a product manager actually own now? The answer is narrower and more decisive than most teams expect. The PM stops shipping features and starts choosing the exact places where human taste and judgment still touch real people on Sunday morning.

This shift exposes a common misread. Teams treat the model output as the finished product and assume the remaining work is review or polish. In practice the model removes volume but not consequence. The places where output meets a volunteer printing curriculum or a pastor checking a sermon illustration still require someone to decide what stays and what gets cut.

John Wesley’s three simple rules give the right lens. Do no harm. Do good. Attend to the ordinances of God. Applied to product work, these rules move from personal piety to operational discipline. They force the PM to treat every handoff as a point of potential harm or grace rather than a simple quality gate.

The Orchestration Layer That Replaced the Spec Doc

Spec documents used to carry the weight of decisions. Now the model generates options so quickly that the real artifact is the set of rules that decide which options reach a human. I watched one children’s ministry tool replace a 40-page spec with three checkpoints: does this output require a volunteer to explain it out loud, does it assume a printer is available, and does it leave the user with a next action that takes under seven minutes.

Those checkpoints are not features. They are filters the PM owns. Without them the model produces polished but unusable lessons that volunteers abandon after the first page. The orchestration layer is the living document that updates after each real Sunday failure rather than before each sprint.

Wesley would recognize this pattern. His societies did not rely on new rules written in advance. They relied on class leaders who knew exactly where harm could enter the weekly meeting. The PM now fills that role for the product.

Taste Decisions That Survive Agent Output

Most taste decisions used to happen during build. Now they happen at the moment of selection. In one case a model produced 12 possible illustrations for a kids’ lesson on generosity. Eleven were theologically fine and visually strong. One used an image that worked in English but translated as mockery in two of the target languages. The model had no way to flag it.

The PM’s job is not to improve the prompt until the bad option disappears. The job is to keep the final choice inside the three Wesley rules. Does this option risk harm to a child reading it in another culture? Does it do obvious good for the volunteer who has to teach it? Does it leave room for the local church to adapt it as a means of grace rather than a finished script?

These questions cannot be fully automated because they depend on knowledge of actual Sunday contexts. The model can surface options. Only the PM can apply the filter that protects the user who never asked for more options.

The Handoff Metric No Dashboard Tracks Yet

Completion rate still matters, but it arrives too late. The metric worth tracking is the percentage of agent output that reaches a human without requiring rework on Monday morning. In the SermonCentral workflow we began logging every instance where a generated resource had to be edited after a volunteer tried to use it in real time. The number dropped only when the PM owned the three taste checkpoints instead of delegating them.

This metric is invisible to standard analytics because the rework happens outside the product. It shows up in email threads, printed pages with handwritten corrections, and volunteer churn. Wesley’s first rule, do no harm, translates directly into keeping that rework number near zero at the point of human contact.

The teams that still measure only velocity or output volume miss the actual constraint. The constraint is no longer production. It is the quality of the last human decision before the resource leaves the building.

Your Turn: Apply This Today

  • Map the three specific handoff points in your current feature where model output first touches a real user and write them down before the next sprint planning.
  • Define the exact taste checkpoint at each handoff using one of Wesley’s three rules and log it in the same place your team keeps acceptance criteria.
  • Review the last agent-generated artifact that reached a Sunday morning user and mark every element that would have failed one of the three checkpoints.
  • Assign yourself ownership of the final selection step for one upcoming release instead of routing it through an automated review.
  • Track the Monday morning rework count for the next two weeks and note which checkpoint would have prevented each instance.
  • Write a one-paragraph description of what “attend to the ordinances” means for your product’s actual users and share it with the two engineers closest to the agent pipeline.

The same pattern showed up in The Inner Skills I Let the Model Train Out of Me and The PM Who Inherited the 8x Target. Both pieces trace what happens when the human filter disappears.

I consult with product leaders building ministry tools on AI orchestration layers, taste decisions at human handoffs, and defining non-dashboard metrics. Let’s talk.

Posted on

The Year I Optimized for Tokens Instead of Presence

Printed papers representing the print step where presence matters

I spent a year routing every product decision through token budgets and latency targets. The rule was simple: if an AI could generate the draft, the spec, or the volunteer script in under four seconds, we shipped it. What I missed was the moment the curriculum stopped matching what a real 7-minute volunteer could actually finish on a Tuesday night with a stack of printed pages in front of them.

The cost showed up in completion rates. We had higher output and lower retention. The volunteers kept opening the materials once, getting stuck on the third activity, and printing something else instead. I kept measuring the wrong thing because the model made it easy to keep measuring the same thing.

Charlie Munger’s latticework of mental models insists that a single lens will always blind you to what the model cannot see. I had treated speed and token cost as the only two models that mattered. Presence was not in the lattice at all.

The routing habit that trained out my own judgment

Every ticket went through the same path. First the model wrote the acceptance criteria, then it suggested the flow, then it generated the test cases. I reviewed the output for factual errors and moved on. The loop felt disciplined until I noticed I was no longer asking whether the volunteer would still be standing at the table when the third page asked them to cut out six small crosses.

The habit removed friction, but it also removed the friction that forces judgment. When the model cannot see the table, the scissors, or the child who needs the activity finished before the parents arrive, the only remaining signal is whether the tokens stayed under budget. That signal is too narrow for ministry work.

After six months the pattern was clear. Features that scored highest on our internal efficiency dashboard were the same ones that generated the most Monday-morning support tickets from volunteers who could not finish the lesson.

Presence as the override at the print step

The print step is where the abstract workflow becomes physical. I started showing up at two different churches during the hour when volunteers actually assembled the materials. One Tuesday I watched a leader try to follow a three-column layout on an 8.5-by-11 sheet that had already been folded twice. The model had optimized the layout for screen reading. The volunteer had no screen.

That single observation overturned three accepted stories we had shipped in the previous quarter. The fix was not another prompt. It was adding a hard stop: no layout change could ship until someone had printed it on the exact paper stock the volunteer would use and completed the activity with a stopwatch running.

The latticework requires multiple models to stay in tension. Token cost and completion time under real conditions cannot be collapsed into one number. When they conflict, the print step wins because that is where the actual work happens.

How emotional intelligence shows up in a sprint review

Sprint reviews used to be a slide deck of velocity and error rates. After the print-step visits I changed the format. The last ten minutes became a short video of a volunteer attempting the new flow while thinking out loud. No narration, no cuts. The team watched the pauses, the page flips, the quiet “this doesn’t make sense” that never appeared in the analytics.

The first time we ran the video, two planned features were removed before the next sprint began. Not because the metrics said they were slow, but because the volunteer’s frustration was audible. That data had always existed; we had simply routed around the channel that carried it.

Emotional intelligence in this setting is not a soft skill layered on top of product work. It is the additional mental model that tells you when the existing metrics have stopped tracking the outcome that actually matters to the volunteer. Without it the lattice stays incomplete.

Your Turn: Apply This Today

  • Pick one live ministry workflow that touches volunteers this week and block fifteen minutes on your calendar to watch it in person without a laptop open.
  • Print the exact materials the volunteer will use on the same paper they receive and complete the full activity yourself with a timer running.
  • Write down the single moment where the workflow required a judgment the current model cannot see, then log that exact moment in the ticket system as a required check.
  • In the next sprint review, replace one metrics slide with a two-minute unedited clip of a real user attempting the task and end the meeting without discussion.
  • Remove one previously accepted efficiency target that no longer matches what you observed during the print step and document the removal in the same place you track token savings.
  • Schedule the same fifteen-minute presence audit on the same workflow for next week and record only the one adjustment you made because you were physically there.

The Morning the Print Step Broke and The Inner Skills I Let the Model Train Out of Me both trace the same gap between what the model optimizes and what the volunteer actually experiences.

I consult with product leaders and ministry teams on presence audits, sprint-review redesign, and workflow overrides that protect volunteer completion. Let’s talk.

Posted on

The Quality Erosion No 8x Claim Ever Mentions

Hands using a caliper for quality control, representing AI agent review layers

The fastest agent stacks in ministry tools do not eliminate bottlenecks. They delete the review layers that surface failures before they reach the volunteer who opens the app on a Saturday night with no backup plan.

This pattern shows up when teams optimize for output velocity above all else. One agent generates the schedule, another approves the assignment, a third pushes the notification. Each step removes a human checkpoint that once caught the mismatch between a new curriculum and a classroom without the right materials. The result is not faster ministry. It is ministry that breaks in the exact moment it is needed most.

Jensen Huang has argued that sovereign control over AI systems requires owning the full stack of constraints, not just the generation layer. Applied to product work, that means the team that builds the agent must also own the verification surface. Without that ownership, speed becomes a tax paid later in rework, trust erosion, and quiet volunteer attrition.

The security review that disappeared after the first agent

The first agent that auto-assigns volunteers usually passes internal tests. It respects role permissions and avoids obvious conflicts. The second agent, the one that routes last-minute swaps, rarely receives the same scrutiny. It operates on fresher data and therefore inherits whatever drift has entered the underlying volunteer profile fields since the last manual audit.

That drift shows up as a children’s ministry coordinator receiving a high-school curriculum assignment because the system still lists her as available for any age group. The agent chain treats the profile as ground truth. The missing step is the one that once required a human to confirm the profile had not been updated in twelve months. Once the second agent ships, that confirmation step is treated as legacy process rather than active control surface.

Teams that measure only agent completion rate never see the downstream ticket volume. The tickets arrive as parent complaints or volunteer no-shows instead of system errors. By then the original optimization story has already been told to leadership.

The quality signal that only appears on Sunday

Volunteer scheduling systems produce their clearest failure data on Sunday morning. A printed roster that does not match the actual check-in list, a notification that arrived after the volunteer had already left for church, a curriculum link that points to last quarter’s material. These signals are cheap to observe in person and expensive to reconstruct from logs.

Agent workflows push that observation moment later. The schedule looks complete in the dashboard on Thursday. The quality gap only becomes visible when the volunteer arrives and the room is missing supplies. By treating Sunday data as anecdotal rather than the primary signal, teams lose the ability to tune the reversal rules that matter.

Huang’s point about sovereign systems is that the owner must decide what counts as acceptable output before deployment, not after. In this domain the acceptable output includes zero mismatched age-group assignments on Sunday. Anything less transfers the verification cost onto the volunteer.

Reinstating the manual gate without slowing the whole system

The manual gate does not need to sit in front of every agent action. It needs to sit in front of the actions that change a volunteer’s actual Sunday morning. A lightweight rule can flag any swap that crosses age bands or curriculum types and route only those cases to a human for confirmation. Everything else proceeds.

The cost is measured in minutes per week, not hours. The return is measured in avoided no-shows and retained volunteers. Most teams reject this approach because they have already framed every added step as a violation of the 8x target. The frame itself is the problem. The target was never defined against the cost of a volunteer who stops opening the app.

Reinstating the gate also requires updating the success metric. Completion rate of the agent chain is replaced by completion rate of the actual volunteer assignment on Sunday. The two numbers diverge quickly once the review layer is restored.

Your Turn: Apply This Today

  • Identify the single agent action in your current volunteer scheduling flow that changes an age-group assignment and add an explicit reversal protocol that requires human confirmation before that change commits.
  • Run the next scheduling cycle with that reversal turned on and log every flagged case along with the time required to resolve it.
  • Compare the flagged cases against the previous three cycles that ran without the reversal and count how many would have produced a Sunday mismatch.
  • Update the dashboard metric from agent completion percentage to actual volunteer check-in match rate measured on Sunday afternoon.
  • Document the total human minutes spent on reversals for one full month and present that number alongside any 8x output claims made about the agent stack.
  • Remove the reversal only after the mismatch rate stays below two percent for four consecutive weeks.

The same erosion pattern appears in curriculum publishing and notification systems. Both “The 8x Claim That Hid the Real Cost” and “The 8x Output That Created the Monday Morning Rework” trace the downstream cost when review steps are treated as optional.

I consult with product leaders building ministry tools on agent workflows and review gates. Let’s talk.

View all posts