Finding the Clients Who Had Already Left
How we helped a digital marketing agency see which clients had already left, by reading every client conversation for silence and relationship-ending language, with one AI judge that must quote its evidence and a person who confirms every finding.
15 of 17
Churned clients caught by one silence signal
1 of 17
Marked churned in the agency's own records
$0.003
Per AI judgment
CLIENT OVERVIEW
A digital marketing agency with more than 100 client accounts. Its conversations with each client run through support tickets, a project tool, meetings, and email. The agency's records of who had churned lagged reality: an account could go quiet, or a client could write that the relationship was ending, and the account still looked active. Leadership wanted to know which relationships had actually ended, with evidence a person could check.
THE PROBLEM
What was broken
The client list lagged reality
The agency's records said who was active, but they didn't match what clients were saying. An account that had gone quiet, or a client who had written that the work was ending, could still show as active. Nobody could say with confidence how many relationships had really ended.
The evidence lived in four places
What a client said was spread across support tickets, the project tool, meetings, and email. No one person read all of it, so a goodbye in one channel or a long silence across all of them never reached anyone as a single signal.
Churn doesn't always arrive as a cancellation
Some clients write that they're leaving. Others go quiet, or their project picks up an offboarding checklist. Catching that means tracking what stopped happening, not only what was said.
THE GOALS
What success looks like
Read each client's messages across tickets, the project tool, meetings, and email
Flag relationship-ending language and long silences, with the evidence attached
Have a person confirm every proposal, with nothing applied automatically
Check every signal against a year of real churn
OUR APPROACH & SOLUTION
Finding churn that had already happened before predicting any
We didn't start by predicting churn. We started by finding it. The detectors look for churn that has already happened, in relationship-ending language and in long silences, and we checked them against a year of client history. Every finding is a proposal with its evidence attached, and a person confirms it. This is a look back, not a forecast, and the numbers here are what it found. Next on the plan is a post-mortem that explains why each client left.
Phase 1
Client Health Review
Built a client health review that reads each client's messages across support tickets, the project tool, meetings, and email, drawing on the platform's 12 integrations.
Phase 2
Churn-Language Detector
Added a detector for explicit relationship-ending language in client email and tickets, so a client who writes that the work is ending gets flagged for review.
Phase 3
Departures Detector
Merged on October 30, 2026: three activity clocks, evidence excerpts, and one AI judge that reads the excerpts and must quote its evidence. Email evidence was added on November 3, and the detector turned on by default on November 5.
Phase 4
Alert Calibration
The first version over-alerted, and 114 accounts showed a critical count. A calibration pass cut high-or-critical risks per account review from 3.9 to between 0.6 and 1.4 in a 10-review replay.
Phase 5
Look Back and Production Read
A look back over one year found 17 churned engagements with written evidence and tested each signal against them. A later production read found 10 churned, 2 likely churned, and 2 scope-change assessments, each with a verified quote.
WHAT WE BUILT
Features and capabilities
01
Client Health Review
Reads each client's messages across support tickets, the project tool, meetings, and email. The platform has 12 integrations, so a review sees the relationship across channels instead of one inbox at a time.
02
Churn-Language Detector
Looks for explicit relationship-ending language in client email and tickets, so a written goodbye gets flagged for review wherever the client sent it.
03
Three Activity Clocks
The departures detector runs three activity clocks for each account. In the look back, one silence signal, 60 or more days without human contact, caught 15 of 17 churned engagements.
04
Evidence Excerpts
Every finding carries the excerpts behind it, so the person reviewing it reads what the client said instead of a bare score.
05
One AI Judge That Quotes Its Evidence
One AI judge reads the evidence for an account and must quote it to support its call. Each judge call costs about $0.003. In a later production read, every churned, likely churned, and scope-change assessment came with a verified quote.
06
A Person Confirms Every Proposal
Each finding is a proposal. A person confirms it, and nothing is applied automatically, so no client is marked churned on the AI's word alone.
07
Calibrated Alerts
Risk levels were calibrated in a 10-review replay, so an account review surfaces between 0.6 and 1.4 high-or-critical risks instead of 3.9, and a flag stands out.
KEY TAKEAWAYS & IMPACT
What changed for this business
15 of 17
Churned engagements caught by one silence signal
In the one-year look back, 60 or more days without human contact caught 15 of the 17 churned engagements that had written evidence. It's a look back, not a forecast: these clients had already left, and the signal shows the record already said so.
1 of 17
Marked churned in the agency's own records
Of the same 17 engagements, the agency's own records had marked 1 as churned. All 7 projects with an offboarding checklist had churned too. The evidence was already in the agency's tools. It wasn't in the client list.
0.6 to 1.4
High-or-critical risks per review, down from 3.9
The first version showed a critical count on 114 accounts. An alert on every account is the same as no alerts, so we calibrated. In a 10-review replay, high-or-critical risks per account review fell from 3.9 to between 0.6 and 1.4.
THE OUTCOME
The wins
-
Leadership can see which client relationships had already ended, with the quote that shows it, instead of relying on records that had marked 1 of 17.
-
A later production read found 10 churned, 2 likely churned, and 2 scope-change assessments, each with a verified quote.
-
All 7 projects with an offboarding checklist had churned, which gives the team a clear signal in its own project tool.
-
A person confirms every proposal and nothing is applied automatically, so no client is marked churned on the AI's word alone.
-
After calibration, an account review surfaces between 0.6 and 1.4 high-or-critical risks instead of 3.9, so a flag means something.
-
Each AI judgment costs about $0.003 and has to quote its evidence, so any flag can be checked against what the client said.