The Token Cost Drop That Only Appeared After We Added the Knowledge Graph

Tuesday night the group texts were flying. One volunteer was stuck in traffic, another needed the key for the side door, and the third had just remembered that the new family’s kid had an allergy note that never made it into the spreadsheet. I watched the scheduling agent spin through query after query, each one longer than the last, and I remember thinking the whole thing felt less like help and more like expensive noise.

We had already tried tightening the prompts. That bought us nothing. What finally changed the meter was feeding the agent the exact three-step handoff the team had been doing on paper for two years: who confirms the room, who checks the list, who sends the text when someone drops. Once the graph held that sequence, the extra tokens just stopped showing up.

The saving wasn’t from adding structure in general. It was from adding the structure they already trusted when the Wi-Fi was spotty and the coffee was gone.

The number that looked too good to be true

The first run showed 1.8 million fewer tokens across a four-week window. Inference costs fell from $1,240 to $645 for the same volume of weekly planning sessions. The agent still produced complete volunteer assignments and print-ready handouts.

We had expected some improvement from retrieval alone. What we did not expect was that the saving came almost entirely from fewer repeated context tokens. The model stopped re-explaining the same role definitions and location constraints on every turn because the graph already held the live state of each handoff.

The complication surfaced when we tried to reproduce the result on a second team using an off-the-shelf organizational ontology. Token counts stayed flat. The structure was correct in theory but invisible to the actual sequence of decisions the volunteers made each Tuesday night.

How the real coordination data differed from the vendor template

The vendor template assumed a clean hierarchy: event, role, person, status. Our logs showed something messier. A typical handoff ran from printed roster to text message confirmation to last-minute swap on arrival. The graph nodes had to capture the message thread ID, the printed page version, and the verbal override that never made it back into any system.

Continuous discovery meant sitting with the children’s ministry coordinator while she updated the sheet. We watched her cross out a name, write a replacement in the margin, then photograph the margin and send it to the team. That photograph became the live edge in the graph. The vendor template had no slot for it.

Once we added the three nodes that matched the observed sequence—print version, confirmation thread, on-site override—token usage dropped on the next billing cycle. The model could now reference the current state instead of re-deriving it from raw text each time.

The test that proved the graph was worth the extra modeling work

We ran an A/B test across two identical volunteer pools for six weeks. One side used the enriched graph. The other used the same agent with a flat document store. The flat version required 2.3 times more tokens to reach the same assignment quality score.

Quality was measured by volunteer completion rate, the metric we track from the SermonCentral work. Teams on the graph side saw completion rise from 71 percent to 84 percent. The flat store produced more correct assignments on paper but fewer volunteers who actually showed up, because the output still required extra clarification calls.

The extra modeling work was two afternoons of observation plus one structured interview per team. Torres would call that the cost of staying in contact with the work. Skipping it left the agent producing technically valid but practically unusable plans.

Your Turn: Apply This Today

• Pick one volunteer workflow that runs every week and list the exact three handoff points that happen outside any current tool.
• Shadow two volunteers through those three points this week and note the artifact each person actually touches or sends.
• Draw the minimal graph nodes that would hold those artifacts in a single state the agent can read on the next turn.
• Replace one generic entity in your current schema with the handoff-specific node you just defined and measure token count for the same query set.
• Run the query set again after one week of live use and check whether the cost saving appears only after the node reflects real overrides.
• Schedule the next observation session before you declare the graph finished.

The pattern here shows up again in The Data Moat That Actually Matters for Small Ministry Teams and The Knowledge Layer Ministry Teams Forgot to Budget.

I consult with ministry product leaders on volunteer workflow graphs and continuous discovery for AI cost control. Let’s talk.