Most engineering organizations know whether their production systems are healthy.
They track latency, errors, deployments, incidents, and test failures.
Ask whether the codebase itself is getting healthier or harder to change, and the answer gets fuzzy.
Where is complexity accumulating? Who owns it? Is it getting better or worse? How big would a cleanup be?
AI coding agents give us a new way to answer these questions. Aaron Francis has shown a great example with agents auditing code for drift, unnecessary abstractions, dead code, weak error handling, and naming problems.
But how much of a code health audit actually needs AI?
Our rule at Codemod is simple: if a question can be answered reliably with deterministic analysis, do that first.
We tested this on Backstage, a popular production-scale open-source codebase used by many enterprises. A TypeScript mining codemod, typescript-impossible-states-mining, scanned 839,957 LOC across 7,636 files in 5.33 seconds, finding 70 candidates across 51 files.
Here's what that makes possible.
If you can define a code health problem structurally, you probably don't need AI to find it
Consider:
type RequestState = {
isLoading: boolean;
isSuccess: boolean;
isError: boolean;
data?: Data;
error?: Error;
};The type system allows contradictory states.
Compare that with:
type RequestState =
| { status: "loading" }
| { status: "success"; data: Data }
| { status: "error"; error: Error };You don't need a frontier model to spot this structural difference. An AST-based miner can find patterns like this across an entire codebase, quickly and consistently.
We turned three common code health patterns into deterministic miners
We built mining codemods for TypeScript and Python.
The TypeScript miner looks for three patterns:
- Boolean flag clusters that may represent one underlying state.
- Duplicated state
, such as
status: "loading"alongsideisLoading: true. - Nested linear lookups
, such as
.find()inside.map().
These aren't declarations that the code is bad. They're candidates for review.
A miner can establish that a pattern exists. Engineering judgment decides whether it's worth changing.
We scanned 840K lines of Backstage in 5 seconds and found 70 candidates across 51 files
The Backstage results:
- 839,957 LOC
- 7,636 files
- 5.33s scan time
- 70 findings across 51 files
- 62 high-confidence **
collections** findings - 8 medium-confidence **
state\_representation** findings
That's more useful than a "code health score."
A score of 73/100 doesn't tell me what to do. Seventy concrete findings across 51 files gives me something I can inspect, scope, and potentially fix.
Confidence matters too. A .find() inside a .map() is straightforward to detect. A boolean cluster requires more interpretation.
The dashboard keeps those differences visible instead of hiding them behind one number.
At org scale, where a problem lives matters more than how many times it appears
Three hundred findings spread across 80 repositories is a different problem from 300 findings concentrated in one subsystem.
That's why we attach metadata to findings.
In Backstage, findings were concentrated in plugins including catalog-backend-module-gitlab and scaffolder.
Excluding the catchall @backstage/maintainers, ownership was concentrated under @backstage/tooling-maintainers and @backstage/framework-maintainers.
Now you can see the blast radius before writing the first migration PR.
A deterministic audit turns code health from a snapshot into a metric you can track
Run the same miner every week and you can ask:
Are we introducing this pattern faster than we're removing it?
After a cleanup, the count should fall. If it climbs again, the problem is returning.
And because the detector is deterministic, the measurement stays stable between runs.
Use deterministic analysis for facts and AI for questions that require judgment
Some questions need context:
- Is this abstraction earning its keep?
- Does this module boundary make sense?
- Is this naming misleading?
- Has the architecture drifted?
AI is useful there.
But "is .find() being called inside .map()?" doesn't need an agent.
Neither does "does this type encode the same state with both strings and booleans?"
So split the audit.
Use parsers and static analysis to establish facts. Spend AI reasoning on intent, architecture, context, and tradeoffs.
Explicit state models may make unfamiliar code easier for agents to reason about
There's another hypothesis worth testing.
When code represents state through several booleans, both humans and agents have to infer which combinations are valid.
A discriminated union puts those rules directly in the type system.
My hypothesis is that as agents make larger changes across unfamiliar codebases, explicit models will require less inference and make changes safer.
That's something worth measuring rather than assuming.
Mine what you can measure, then spend AI reasoning on the hard parts
The useful model isn't deterministic analysis or AI.
It's both.
Deterministic miners tell you what can be measured precisely. AI handles questions that require understanding the surrounding system.
Insights then answers the questions tech leads actually care about:
Where is the problem? Who owns it? How big is the cleanup? Is it getting better or worse?
That's more actionable than an audit report saying the codebase has "architectural inconsistencies."
Any structural rule your organization cares about can become a continuous code health signal
Our rules are just examples.
You could mine deprecated internal APIs, error-handling patterns, old framework conventions, state modeling, or company-specific architectural rules.
If you can define the pattern deterministically, you can find it, measure it, track it, and scope the work required to change it.
You can build these mining codemods yourself. If you want help defining one for your codebase, reach out to the Codemod team.
The goal isn't to remove AI from code auditing.
It's to give AI fewer obvious things to figure out.