Blog

The bedtime rule, often called the Ten-Minute Bedtime Rule, shows up in product circles through frameworks like those promoted by consultants drawing from books such as “The Effective Executive” and later PM training programs. It instructs teams to close the day by writing a short list of observations, model outputs, or user signals before sleep. The claim is that this tidy capture step turns scattered input into retained knowledge without extra overhead.

The rule breaks once AI features ship because the input no longer arrives as discrete daily packets. Live agents generate continuous streams of partial answers, edge-case refusals, and volunteer rephrasings that arrive at 11 p.m. and again at 6 a.m. The ten-minute window cannot sort which of those outputs already changed the product behavior that same afternoon.

This separation between capture and action worked when product cycles moved in weeks. AI rollouts compress the cycle to hours, so the habit of parking insights for later review creates lag that users feel immediately.

This is the foundational misread that causes product teams to treat learning as an after-hours activity rather than the material that must alter the next deployment.

John Wesley’s three rules offer the needed lens. He told early Methodists to do no harm, do good, and stay in love with God through ordinary disciplines. Applied to product work, the rules become filters that decide which incoming signal requires an immediate change to the shipped system rather than another note in a document.

Why the bedtime rule breaks once live agents ship

At SermonCentral we released a simple agent that suggested age-appropriate rephrasings for children’s ministry scripts. Within two days the agent began softening references to judgment in ways that matched the tone of one large church network but clashed with another. The bedtime list captured the pattern on night one. By night three the mismatch had already reached volunteers who printed the scripts and used them.

The ten-minute capture recorded the observation correctly. It did not trigger any rollback or prompt adjustment because the rule treats the note as sufficient. In practice the agent continued producing the same softened language until a manual review three days later.

Real usage data from the smallest churches showed the problem first. Those volunteers do not file tickets. They simply stop returning to the tool. The bedtime list had no mechanism to surface that drop-off against the agent’s output log.

Wesley’s rules turn raw model output into a decision filter

The first rule, do no harm, forces a check against the most vulnerable user before any output stays live. For the children’s script agent this meant asking whether a suggested rephrasing would confuse a new volunteer who had never taught the story before. If the answer was unclear, the output route changed to a flagged review queue rather than direct suggestion.

The second rule, do good, asks what concrete action the output should enable that day. In the Bible Gateway experiments this looked like requiring every model suggestion to point toward a next step the reader could take on the same visit, such as a short audio clip or a parallel passage. Outputs that only added information without enabling that step were suppressed.

The third rule, stay in love with God through ordinary disciplines, maps to keeping the product tied to the actual rhythm of ministry work. For agents this meant testing whether the suggestion still made sense when the user had only seven minutes between services. Many clever completions failed that test and were removed.

These filters run at decision time, not at the end of the day. They force the product manager to judge the output against live constraints rather than collect it for later sorting.

What replaces the bedtime rule: ownership transfer

The shift is from recording an insight to assigning the change it requires. When an agent surfaces a new refusal pattern, the next step is not a note but a one-line ownership statement that names the person who will adjust the prompt or the guardrail before the next deployment window.

At one rollout the team moved from end-of-day summaries to a shared log that required each entry to include the exact parameter or content rule that would change as a result. Entries without that line were deleted after twenty-four hours. The volume of notes dropped sharply, but the number of actual prompt changes rose.

This transfer also surfaces when the input should not change the product at all. Some model behaviors look novel in the moment yet match patterns already handled by existing volunteer workflows. Wesley’s first rule catches those cases quickly because the test is whether harm would occur if nothing changed.

Even classic guidance on healthy routines assumes a stable input, which is exactly why the bedtime rule fails once shipped AI behavior keeps shifting.

Your Turn: Apply This Today

• Pick the single AI feature now in production that produces the most daily output and add a one-question harm check to its review log before any new suggestion reaches users.
• Require every model experiment this week to name the exact deployment parameter or content rule that will change if the experiment succeeds, and delete any experiment notes that omit that line.
• Run the do-good test on three recent agent outputs by asking whether each one points a volunteer to an action they can finish in the same session; remove any that do not.
• Review the last forty-eight hours of agent logs against the seven-minute volunteer window and flag any suggestion that requires more context than a printed page can hold.
• Assign ownership for the top three recurring edge cases you logged this week by writing the owner’s name and the next deployment date next to each case.
• Delete every capture note older than seven days that has not produced a shipped change, then move the remaining items into the live decision queue.

The CPO Who Captured What the Spreadsheet Missed shows what happens when raw signals stay disconnected from decisions. The Trust Layer No One Budgets For traces how unfiltered agent behavior erodes volunteer confidence over time.

I consult with product leaders on AI feature rollouts, input filtering for live decisions, and ownership transfer in ministry-facing tools. Let’s talk.

The career question product managers keep asking me after the promotion lands is whether the new title actually expands what they can decide without approval, and it is a career question no raise answers on its own. The answer is no, not unless the scope of their circle of competence has also moved outward in concrete ways.

Most keep measuring progress by the old markers—headcount, budget line, and the size of the roadmap they inherit. Those markers no longer track real ownership once AI tools compress the time between idea and shipped feature.

The Career Question When Compensation and Scope Drift Apart

The promotion conversation usually ends with a number and a new business card. What it rarely surfaces is whether the person now owns decisions that used to sit two layers above them. In practice the gap shows up six months later when an AI-generated prototype lands on their desk and they realize they still need three sign-offs before they can run it with real users.

This mismatch is not unique to faith-tech, but it appears faster here because the user base is smaller and the success metrics are softer. A children’s ministry volunteer who prints a lesson on Tuesday does not file a support ticket when the experience feels off—she simply stops coming back. The PM who cannot change the print layout without another round of reviews never learns what that volunteer actually needed.

Gallup’s research on what actually drives engagement at work reframes this career question well. Charlie Munger’s circle of competence is useful here because it forces the question away from title and toward boundary. Munger described the circle as the area where you can make decisions with high accuracy because you have seen the patterns before. Outside that circle, even smart people produce expensive errors. The promotion does not enlarge the circle; deliberate work does.

Circle of Competence Applied to AI Product Roles

In AI-era product work the boundary matters more because models now generate options faster than teams can evaluate them. A PM whose competence stops at writing requirements will green-light features that look good in a demo and fail with the smallest churches. The PM who has spent time watching volunteers navigate the same workflow on paper knows which generated option will actually survive contact with a seven-minute prep window.

Expanding the circle requires choosing a narrow slice and owning the outcome end to end. At one ministry tool I watched a PM take responsibility for the entire flow from volunteer signup to lesson completion. She removed three internal review steps and measured only whether the printed page reached the classroom. Usage rose because she had narrowed her circle to something she could actually judge without waiting for data that never arrived in time.

The same pattern holds for larger platforms. When the team at a high-traffic Bible site decided to test an AI-assisted reading plan, the PM who succeeded was the one who had previously owned the print PDF version used by small groups. She already knew which formatting constraints mattered. The model suggestions were filtered against that lived constraint rather than against abstract engagement metrics.

The Career Question the Old Faith-Tech Playbook Cannot Answer

The inherited playbook says move up by managing larger teams and bigger roadmaps. In ministry tools that path often leads to more meetings and less contact with the actual user. The volunteer who prepares a lesson in a church basement does not care about the PM’s headcount; she cares whether the material fits on one page and survives a spilled coffee cup.

AI accelerates the break because it removes the old friction of building. A feature that once took a quarter can now ship in a week. The limiting factor becomes judgment about whether the feature should ship at all. That judgment only improves inside a competence circle that includes direct observation of the user under real constraints.

Teams that keep promoting on the old signals end up with leaders who can describe strategy but cannot tell whether a generated outline will work for a volunteer with no formal training. The organizations that notice the drift are the ones that start asking different questions in promotion reviews: what specific decision boundary has this person moved in the last twelve months, and what evidence shows the boundary is now reliable.

Your Turn: Apply This Today

• List every current project and mark the single decision on each that you cannot make without another person’s approval; choose one to own fully this quarter.
• Shadow one user completing their actual task with the current product for at least ninety minutes and record the exact moment they adapt or abandon the flow.
• Write a one-paragraph definition of the competence boundary you claim today and share it with your manager before the next role conversation.
• Run one AI-generated option through your existing user constraint set and reject it on paper before any build work begins.
• Identify the last three features that shipped under your name and note whether you personally observed the user outcome or relied on secondhand reports.
• Schedule a thirty-minute review with a peer PM who works on a different product line and compare where each of you can act without escalation.

The CPO Who Captured What the Spreadsheet Missed and The CPO Who Started Asking What Kept Her Volunteers Up both trace the same pattern: real ownership grows only after someone redraws their own competence boundary.

I consult with mid-career product leaders on mapping competence boundaries, AI-era role decisions, and ownership growth in mission-driven tools. Let’s talk.

The screen glowed under the dim office lights as the triage agent pushed its first batch of cards, and the judgment call it could not make was already waiting. The children’s director’s eyes narrowed at the top result, a flagged prayer request from a single mom whose kid had missed three Sundays. She tapped override without checking the 87 percent score, then two more before the next refresh cycle even started.

Her coffee had gone cold beside the keyboard. The room stayed quiet except for the low hum of the laptop fan. She saved the batch and stood up, already thinking through which volunteer would see the corrected version first thing tomorrow.

That single sequence of clicks showed the gap no model can close on its own. The scores measured pattern fit, not the weight of the actual people on the other side. When the override happens this early, it reveals the constraint that matters most for any v1: the builder still has to carry live context the dataset never captured.

This is the exact spot where over-reliance on AI starts to erode the judgment that later versions depend on. Teams begin treating high confidence numbers as permission to stop asking the harder questions. The product ships with cleaner logs but thinner instincts.

The judgment call Solomon would recognize

Solomon’s judgment offers the clearest lens here. Two women claimed the same living child. No dataset or probability score settled it. Solomon forced the decision into the open by proposing to split the child, then watched which response revealed the real mother. The test did not measure accuracy against past cases. It created a live situation that exposed the difference between claim and care.

Applied to agent handoffs, the same principle holds. When an AI triage system surfaces a recommendation, the override moment functions like Solomon’s sword. It forces the builder to decide whether the model’s pattern match aligns with the actual stakes. Skip that step repeatedly and the team loses the ability to notice when the pattern itself is incomplete.

The children’s director’s overrides worked because she still held the names and the absences in her head. The agent had access only to attendance flags and text strings. Her decision loop included the detail the model could not weight. Without that loop, the next release would optimize for cleaner data rather than better ministry outcomes.

Solomon did not ask for more evidence first. He set a condition that made the real priority visible in real time. Modern agent systems invert this. They present polished outputs and invite teams to accept them unless something obvious breaks. The result is a gradual shift where builders review less and ratify more.

Why your v1 needs a protected judgment call

Product teams building v1 agents for ministry tools see this pattern most clearly when usage data starts rolling in. The metrics look stable because the model handles volume. The edge cases that matter, the ones involving real volunteers and irregular schedules, only surface when someone still practices the override habit.

NN/g research on keeping a human in the loop explains why a deliberate judgment call beats a confident model score. The constraint tightens as models accelerate. Faster inference means more decision cards appear before a builder finishes evaluating the first one. Judgment slots, the mental space required to hold context and make the call, stay fixed. They cannot scale with token speed.

Teams that protect those slots treat overrides as scheduled work rather than exceptions. They block time after each agent run to review a small set of cases without the scores visible. The practice keeps the builder’s sense of what counts sharp instead of letting it atrophy.

The same protection shows up in how handoff points get designed. Instead of routing every low-stakes item through the model, the system leaves certain categories for direct human entry. This preserves the live context that later training data will need. Without it, subsequent versions inherit the blind spots of the first release.

Safeguards that keep the human in the loop

Another safeguard comes from requiring the builder to write a short note on each override before the card closes. The note forces articulation of the missing variable. Over weeks those notes become the clearest signal of where the model still needs human weight.

Protecting judgment also means limiting the number of agent-driven decisions any single builder reviews in one sitting. Fatigue turns overrides into rubber stamps. Capping the load keeps each one deliberate.

Finally, the practice extends to how teams document wins. When an override later proves correct in follow-up conversations with users, that outcome gets recorded with the same detail as model successes. The record reinforces that judgment remains the durable asset.

Your Turn: Apply This Today
– Block thirty minutes on your calendar this week to review five agent outputs with the scores hidden before you check them.
– Pick one category of decisions your current v1 agent handles and route it back to direct entry for the next sprint.
– After every override this month, write one sentence explaining the variable the model missed and store it in a shared note.
– Limit yourself to reviewing no more than eight agent cards in any single session for the next two weeks.
– At the end of this week, pull the three overrides that felt hardest and discuss them with one teammate without referencing the scores.
– Schedule a recurring thirty-minute slot every Friday to read the override notes from the prior week out loud to yourself.

The CPO Who Captured What the Spreadsheet Missed shows how one leader built a parallel record of the calls models could not make. The Trust Layer No One Budgets For traces what happens when those records never get created.

I consult with product leaders building AI agents for ministry tools on v1 judgment practices and override design. Let’s talk.