Patterns, Not Tickets

When you look at tickets one by one, you fix issues. When you look at them together, you fix systems.

That line is Colin’s, and it is the whole argument. The steps below are an attempt to write down the method underneath the two builds he has published in detail: the churn analysis that cost about one hundred dollars, and the Zendesk run that triaged 650 tickets in thirty minutes. Both are documented as workflow studies.

The five steps

1. Pick a pile you already own

Not a new data collection project. Tickets, call transcripts, churn records, onboarding notes, QA scores. The qualifying test is that the data already exists, somebody already reads it one row at a time, and nobody has ever read all of it at once.

2. Flatten it before you ask anything of it

Pull it into one place and summarize per unit, per customer, per ticket, per call. This is the dull step and it is where the cost lives. In the churn build the pull came from Snowflake and each customer record was summarized and cleaned before any categorization happened.

3. Let the model derive the categories

Do not hand it your existing taxonomy. Your taxonomy is what produced the blind spot. Ask for categories derived from the data, with a confidence score attached to each assignment, so that the low-confidence pile is visible rather than averaged away.

4. Cross-reference against the systems that hold the other half

A ticket on its own is a complaint. A ticket read against Jira, Slack, the knowledge base and a vector database is a signal about a documentation gap, a release, or a training problem. The triage run found a documentation and training gap driving escalations that no single ticket revealed.

5. Stop at the finding that changes a system

The deliverable is not a labeled dataset. It is a small number of findings that each imply a change to how something runs. In the churn case, the finding was that a large share of recorded churn was business closures and acquisitions, which meant the retention metric itself was wrong. That is a system fix. A list of three hundred miscategorized accounts is not.

What this method does not do

It does not replace reading. Somebody still has to look at the low-confidence pile and at the sample. It does not produce a number you can take to a board without checking it. And it does not work on a pile nobody currently reads at all, because then there is no baseline to be surprised against.

Cost discipline

State the real figure. The churn analysis is useful as a reference point precisely because the published number is about one hundred dollars and about one hour, not “dramatically faster and cheaper.” A method reported without its cost is a vendor claim.