Teaching a CRM to qualify its own deals

The problem

The company qualifies deals with MEDIC, a checklist framework: Metrics, Economic buyer, Decision criteria and Decision process, Identify pain, Champion. The answers to those questions were being said on calls and recorded in meeting notes. The structured MEDIC fields next to each deal, the ones pipeline reviews actually read, stayed empty or stale.

That gap is structural, not a discipline problem. Notes are where writing is easy; fields are what reports read; keeping the two in sync is a tax that busy people defer indefinitely. The evidence does not disappear, it becomes unreachable: answering "who controls the budget here?" means rereading every note on the deal, so in practice nobody answers it. The decay is also worst on the deals deepest in the pipeline, because they have the most notes and the oldest field values.

What we built

A scheduled agent that runs every day. For each deal with fresh notes it makes one model call, asks for findings per MEDIC concept, and writes the survivors into suggestion fields: a parallel set of fields the agent owns, each prefixed so its origin is unmistakable. Each concept is really three fields, the suggested value, a _source field holding a link to the note it came from, and a _status field the salesperson flips as they accept or reject the suggestion.

The salesperson's own fields are never touched. When the notes disagree with what a human wrote, the disagreement surfaces as a suggestion sitting next to the human value, not as an edit to someone's work.

Safety design

Three properties carried most of the weight.

No write path to human fields. The agent signs in to the CRM under its own identity, and its code contains no path that writes to a human-owned field. Zero writes to human fields is a schema property, not a prompt instruction, so it holds even when the model misbehaves.

No citation, no write. Every finding must return the source note's ID and the verbatim sentence that supports it. A plain code check after the model call drops anything that does not comply, including findings that cite a note the model was never shown, which is exactly what a fabricated citation looks like. Because uncited findings are discarded in code, the citation rate is 100% by construction.

Default to nothing. Going live requires two separate switches at once: an --apply flag on the run and a live-writes setting in the environment. With either one missing, the agent decides everything and writes nothing. Every decision is logged, including the non-events: skipped because a human owned the field, or would-have-written-but-dry-run.

What happened

Three production failures, none of them crashes. All three were the agent quietly doing nothing useful while looking healthy, and all three were caught through the log.

1. The sources were not what the spec said. The plan assumed calls, emails, and notes. The live workspace exposed no call transcripts through the API and had no email object at all. Everything usable lived in notes, so the agent became notes-first, with the call and email handling built but switched off.
2. Dry-run rows poisoned the already-handled log. The agent skips findings it has already handled, so a daily job does not re-suggest the same thing forever. The original check did not distinguish rehearsal rows from real writes. Months of dry-run testing filled the log, and on the first live day the agent recognized its own dry-run ghosts and wrote almost nothing. The fix was one clause: count a finding as handled only if it actually wrote.
3. A 50-deal cap pointed at the wrong end of the pipeline. A per-run limit left over from testing, combined with a newest-first sort, meant the agent processed the fifty most recently created deals, the ones with no notes yet, and never reached the older, late-stage deals where the evidence actually lives.

What this looked like day to day

The agent runs every morning. Suggestions appear next to the fields they concern, each with its quote, so a salesperson can judge one in seconds without opening the note. When someone asks why a deal's champion is listed as a particular person, there is a sentence to point at instead of a shrug.

This page is the engagement-shaped summary. The full build story, with the decision logic, the citation-gate code, and what each failure taught, is the essay Teaching a CRM to qualify its own deals.