A product team reported their AI-assisted planning workflow delivered eight times the output volume in the first month. It is a textbook AI productivity paradox: the volume goes up while the real value quietly goes down. The dashboard showed more lesson outlines, more email drafts, more volunteer schedules generated per hour than any prior quarter. Yet six weeks later the same team watched completion rates among their 7-minute volunteers drop below the baseline they had before the new system.
The obvious reading treats the multiplier as pure gain. The data actually pointed the other direction once second-order effects entered the picture. Volunteers who received the polished outputs no longer understood the reasoning chain that produced them. When a schedule conflict appeared on Sunday morning, they lacked the context to adjust it without breaking the original intent.
This is the foundational misread that causes product teams to celebrate velocity while quietly eroding judgment at the edge. Charlie Munger’s latticework demands that any single model, in this case raw output multiplication, must be cross-checked against at least two other models before it earns trust. One model tracks first-order production. Another tracks whether the recipient can still reconstruct the decision logic six weeks later. When those two models conflict, the latticework flags the multiplier as incomplete rather than impressive.
Where the AI Productivity Paradox Starts: the 8x Number
The eightfold increase came from an agent loop that ingested past curriculum, current calendar constraints, and volunteer availability, then emitted complete weekly plans. The prompt engineering was tight. Token usage stayed low. Human review time fell from forty minutes to five. On paper the system looked like a decisive win for any ministry running on volunteer labor.
The number ignored what happened after the plan left the system. Volunteers received a finished document rather than the intermediate constraints the agent had weighed. They could execute the steps, but they could not explain why a particular story was chosen for a given age group or why one room was double-booked on purpose. When the inevitable exception arrived, execution stalled because the mental model had never been transferred.
Munger would have asked which additional model revealed the omission. In this case the missing model was the volunteer’s ability to re-derive the plan under partial information. The 8x figure measured only the production side of the lattice.
The Hidden Regression Behind the AI Productivity Paradox
Regression appeared first in small decisions that used to be automatic. A volunteer who once swapped two craft supplies without asking now waited for the next planning cycle. Another who used to shorten a lesson when time ran long now read every line even when the room was restless. The surface metrics still looked acceptable until the cumulative effect reached the children and parents.
The second-order failure mode is simple: when output volume rises faster than decision context, the volunteer shifts from owner to operator. Ownership requires the ability to defend a change. Operation only requires following the last instruction. Sunday is when the difference becomes visible because exceptions arrive faster than the next agent run.
Teams that track only generation counts miss this shift entirely. The latticework requires a second measurement: the percentage of volunteers who can state the original constraint set without opening the document. When that percentage falls, the multiplier has begun to subtract capability rather than add it.
The Logging Practice That Surfaces It Early
Economists have debated a similar gap for decades under the name the productivity paradox: new technology shows up everywhere except in the measured results.
The practical countermeasure is a minimal log kept by the volunteer, not the system. After each handoff the volunteer writes two sentences: the constraint they remember most clearly and the one change they would make if the constraint were removed. The log takes less than sixty seconds and travels with the printed plan.
Review happens once a week. A product owner or lead volunteer scans the accumulated sentences for patterns. Repeated inability to name a constraint signals that the agent output has become opaque. A repeated suggested change that the agent would have rejected reveals a missing rule in the prompt. Both signals appear before completion rates move.
This practice adds one override protocol and one weekly log review to any agent loop already running. The override lets a volunteer mark a plan “context needed” and receive the constraint list before the next cycle. The review meeting lasts fifteen minutes and uses only the volunteer sentences, never the original agent trace.
Your Turn: Apply This Today
- Pick one recurring agent output this week and require the volunteer to add the two-sentence log before they leave the room.
- Schedule a fifteen-minute Friday review that reads only the volunteer sentences and flags any plan where the constraint cannot be restated.
- Add an override button in the handoff interface labeled “send constraints” that bypasses polished output and returns the raw decision factors.
- Measure the percentage of volunteers who correctly name the top constraint on Monday morning; treat any drop below 80 percent as a system defect.
- Remove one generation step from the agent prompt and replace it with an explicit request that the volunteer must confirm the constraint in their own words.
- Run the same lattice check on any new agent you ship: first measure output volume, then measure retained decision context at the two-week mark.
Two earlier posts examined the same tension from different angles. “When Speed Outruns Context” traced how notification volume alone can erode volunteer judgment, and “Constraint-First Prompts” showed the minimal prompt change that keeps reasoning visible. Both posts are linked from the archive.
I consult with product leaders building AI workflows for volunteer-driven ministries on output measurement, constraint logging, and second-order failure detection. Let’s talk.