The Context Layer I Pretended Was Optional Until Token Bills Arrived

I still have the screenshot from that Tuesday. A children’s ministry coordinator in Ohio had forwarded the same signup list three times because the agent kept dropping the prior week’s commitments. No error message, just clean new suggestions that ignored everything we’d already confirmed.

I’d told the team the model would “figure it out” from recent messages. The prompt was tight, the activity feed was fresh, and we shipped. The first token bill that month was larger than the entire previous quarter combined, but the real hit was watching volunteers quietly stop trusting the tool.

That gap between what the model could see and what it needed to remember — the context layer no one had actually built — is where most of these agents quietly die.

Charlie Munger’s latticework idea helps here. He described the habit of building multiple mental models that cross-check one another instead of relying on a single narrow lens. In product terms, that means treating the context layer not as a single store but as overlapping layers—roster state, routing rules, and access history—that must remain legible to each other.

How Lost Roster State Shows Up in Real Scheduling

Take a typical Wednesday-night program that runs on thirty volunteers. The coordinator used to keep a shared spreadsheet with columns for last confirmed date, preferred role, and any notes from the previous handoff. When the agent replaced the sheet, those columns disappeared from the model’s working memory after each session.

Within two weeks the agent began assigning the same person to conflicting rooms because it had no record of the prior commitment. The coordinator spent extra hours every Monday correcting the output instead of preparing curriculum. The pattern is common: the agent optimizes for the current prompt while the team pays for the missing longitudinal view.

Teams that kept even a minimal state file alongside the agent saw fewer collisions. The file was not elegant, but it preserved the handoff chain the volunteers actually used to trust one another.

Dynamic Routing as the Practical Fix for Small Teams

Small ministry teams rarely have dedicated ops staff. Their routing decisions depend on who answered the last text thread and who owes a favor from the previous quarter. An agent that cannot surface that informal history forces the coordinator to reconstruct the social graph from scratch every week.

One team I observed added a lightweight routing table that logged every assignment with a timestamp and the name of the person who last updated it. The agent consulted the table before generating new slots. Conflicts dropped because the model now carried forward the same continuity the coordinator used to track mentally.

The table stayed small—under two hundred rows—and lived in the same database as the agent’s output. No new infrastructure was required, only the discipline to treat prior handoffs as first-class context rather than optional color.

The Context Layer Hiding Inside Your Security Review

Security reviews often push teams to strip identifiers and session history for compliance. In ministry settings that same stripping removes the exact chain of custody volunteers need when questions arise about who signed up whom.

A workable middle path logs the minimal set of prior actions required for continuity—date of last confirmation, role preference, and the name of the person who recorded it—while redacting anything outside the team’s agreed boundary. The security layer and the context layer become the same artifact instead of opposing constraints.

Teams that made this choice could still pass audits because the retained data stayed inside the organization and carried explicit expiration rules. The agent retained enough history to avoid re-asking the same availability questions every cycle.

Your Turn: Apply This Today

• Pull the last thirty days of scheduling output from one agent workflow and list every field that disappeared between sessions.
• Identify the single handoff data point volunteers mention most often in chat threads and add it to the agent’s context refresh this week.
• Write a five-row test file that replays the last three handoffs and run the agent against it before the next schedule generation.
• Tag each retained field with an expiration date so the security review can see exactly what history is kept and why.
• Share the updated context schema with the two coordinators who handle the most manual corrections and ask which row still feels missing.
• Measure the change in coordinator correction time over the next two scheduling cycles and keep only the fields that move the number.

The Token Cost Drop That Only Appeared After We Added the Knowledge Graph and The PM Who Just Shipped an Agent and Now Needs an Evaluation Layer both trace the same pattern of missing longitudinal state.

I consult with product leaders building ministry agents on context layers, handoff preservation, and evaluation that respects volunteer history. Let’s talk.