AI Codebase Documentation: What Poltergit Reveals
See how Poltergit uses local AI, Git history and verifiable citations to explain legacy code—and where this approach helps, fails and matters for teams.
Poltergit Shows How AI Can Explain the Code Nobody Remembers
Every mature codebase has lines that are hard to touch because nobody can say why they're there. A retry count or a strange constant might have started as a fix for a real production incident. Years later, the original author has moved on, the commit message says "update," and removing the code feels riskier than leaving it alone.
Poltergit is an open-source experiment in using AI to recover some of that missing context from Git history. Point it at a line and it collects relevant commits - and, if you want, related GitHub discussions - then asks a local language model to explain what the evidence says. The key constraint is simple: any claim has to include exact, checked quotations. If the record doesn't support an explanation, it's supposed to say so.
That makes Poltergit more than a coding-assistant demo. It's a real case study in where AI can help with maintenance work, how to make answers auditable, and why "runs locally" doesn't automatically solve every privacy or reliability concern. Here's how the system works, what it gets right, and what engineering teams should take from it.
Why does git blame fail to explain legacy code?
Git can tell you when a line changed and who committed it. It can't reliably tell you the operational reason behind it. And that operational reason matters, because code is often shaped by events that don't show up in the code itself: a customer incident, an upstream outage, a timing constraint, or a workaround for a dependency that has since been replaced.
Picture finding time.sleep(3) in a reconnect path. It might look arbitrary. But deleting it could trigger a reconnect storm if the gateway temporarily holds closed sessions. The code is visible; the context that makes it safe is either buried in an old commit message or pull request - or simply missing.
This is a documentation problem, but it's not only a documentation problem. Teams rarely have time to write a perfect explanation for every non-obvious decision, and even good documentation drifts as the code changes. The practical challenge is how to retrieve evidence more easily without turning an AI-generated guess into a new, confident-sounding source of misinformation.
Why a line-level question is a useful starting point
A repository-wide question like "Explain this service" is broad enough to invite vague answers. A question about a single line forces a bounded task: find the line's history, gather relevant evidence, and explain what that evidence does (and doesn't) support.
Poltergit uses Git's line-history capability, git log -L, to follow a line across commits even when it moves. That gives it a narrower, more targeted evidence set than simply searching recent commit messages. If you enable it, the tool can also pull in related pull requests, reviews, and issues from GitHub - where the rationale is sometimes spelled out more clearly.
There's also a hard boundary here: Poltergit can only recover what the record contains. Sometimes a concise commit message really does preserve the reason. Sometimes you get a weak message like "update." Poltergit's job is to treat that absence as a real result; otherwise, a polished explanation can hide the fact that nobody ever recorded the answer.
The maintenance cost of forgotten decisions
When engineers can't establish why a line exists, they usually end up with two bad options: leave it alone, or change it while guessing. Leaving it alone adds complexity. Changing it adds regression risk. A tool that reduces the time spent digging through history can improve those decisions - even if it can't always produce a definitive explanation.
The goal isn't to make every old line instantly intelligible. It's to help a developer tell the difference between "there is evidence for this behavior," "there is relevant but incomplete evidence," and "the record is silent." Those are different situations, and a good tool shouldn't blend them together.
What does Poltergit do?
Poltergit is an open-source Python tool with a command-line interface and a small local web application called the séance room. Its central command asks why a particular line exists. It collects the line's history, sends the relevant material to a locally running model, and returns an answer with citations to the source material.
In the demo, line 13 contains time.sleep(3). The model explains that the delay helps prevent a reconnect storm while a reservation expires. The citation points to a commit whose message describes a gateway session reservation of two seconds. The explanation isn't meant to stand on its own as "AI knowledge" - you can inspect the supporting words.
A second example shows why silence matters. A constant is changed in a commit titled "update." There's no clear rationale to retrieve, so the appropriate response is that the record doesn't explain the change. During testing, the model once tried to provide a reason with a quotation that didn't actually match its source. Poltergit rejected the answer and showed labelled source excerpts instead.
From a line query to a local answer
The workflow is intentionally small:
- Choose a file and line, such as
src/client.py:13. - Use Git to trace that line through relevant commits.
- Optionally include related GitHub pull requests, reviews, and issues.
- Ask the local model to return claims in a structured format, with a source and exact quotation for each claim.
- Validate those quotations before showing the answer.
The tool reads committed HEAD and warns about uncommitted changes. That's a sensible scope choice: historical intent shouldn't quietly mix in edits you haven't committed. It also trims the model prompt without throwing away the oldest line history, which can matter for when a behavior first appeared.
More than an explanation interface
Poltergit also has an ofrenda command that creates a memorial page for deleted files and Python functions. It records when an item first appeared, how long it stayed in the project, and which commit removed it. Function birth dates are estimated by comparing Python syntax trees across as many as 200 revisions of a file.
There's a product insight here: maintenance isn't only about understanding what exists now. Teams sometimes need to understand what was removed - and why. A historical view can help during migrations, incident reviews, or attempts to restore behavior that was intentionally retired.
The project also includes local text-to-speech, so the answer can be read aloud, and it ships with a reproducible demo repository with six commits. Those features make the project memorable, but the core engineering idea is narrower: turn repository history into evidence-backed explanations, while making uncertainty visible.
How does Poltergit make AI answers auditable?
The most important design choice isn't which model speaks. It's what the system demands before it shows you anything. Poltergit asks Gemma 4 E4B, running through Ollama, to return structured JSON. Each claim must point to a source and include an exact quotation. Then the application checks that quotation against the cited source before presenting the response.
The implementation enforces two rules: the quote must be at least 12 characters long and must appear word for word in the cited material; and the answer has to speak about the code, not imitate what the original developer "must have meant." If validation fails, the generated answer is thrown out. Instead, the user sees labelled source excerpts.
This is a useful safeguard, but it's not a guarantee the conclusion is correct. A model can select a real quote and still draw the wrong conclusion. Quote matching verifies that the cited words exist; it doesn't verify that the reasoning faithfully represents the full history. In other words, the design makes an answer easier to inspect - but not infallible.
The difference between evidence and correctness
A useful way to evaluate the feature is to separate three questions:
- Is the evidence real? Does the cited text actually exist in the source? Poltergit checks this mechanically.
- Does the evidence support the claim? That part still takes judgment. A quotation can be accurate but incomplete or ambiguous.
- Is the answer useful for the decision you're trying to make? A historical explanation doesn't, by itself, prove the code is still needed today.
That separation matters because it prevents a common mistake in AI product design: treating citations as a complete trust solution. They improve traceability. They don't eliminate interpretation, stale assumptions, or missing context.
Retrieval has to reach beyond the obvious commit
Line history can be too narrow. In Poltergit's example, a retry loop is touched only by the commit that introduced the client. The reason for the retry count shows up elsewhere - another change that increased MAX_RETRIES from three to seven.
To find that kind of related history, Poltergit can index repository commits using EmbeddingGemma and store vectors in PostgreSQL with pgvector. It combines vector similarity with PostgreSQL full-text search using Reciprocal Rank Fusion. The search is scoped by repository, revision, and embedding model, which helps keep results from unrelated histories from getting mixed together.
This is a practical hybrid retrieval pattern: semantic search can surface conceptually related changes, while keyword search can pull up exact terms. It also adds operational dependencies. Teams need to configure the database, indexing, and revision boundaries - and test retrieval quality on their own repositories rather than assuming the "plausible" results are actually relevant.
Observability is part of the product
Each séance can be traced as an agent transaction, with spans for history retrieval, model inference, and evidence validation. In the documented CPU-only run, history retrieval took 677 milliseconds, quote checking took 0.12 milliseconds, and Gemma inference took 2.89 minutes. The inference used 728 input tokens and produced 834 output tokens, with no per-request model API charge.
Those timings matter because they explain why observability is still useful even for a local tool. A first run that looks like a failure might just be a slow model call that trips a 180-second timeout. Traces also helped identify missing AI spans and telemetry lost when the command-line process exited before flushing events. In each case, the fix came from seeing the actual execution path - not guessing.
Where does this approach create business value - and where are the limits?
The simplest business case is reduced investigation time. If a developer can quickly find the commit or discussion behind a fragile behavior, they can make a better-informed maintenance decision. That still doesn't guarantee fewer incidents or faster delivery for every team. The payoff depends on how much useful rationale exists in the repository and how often engineers currently spend time reconstructing it.
Local inference also changes the cost profile. Poltergit uses Gemma 4 for answers, EmbeddingGemma for vectors, and Kokoro for speech, all run locally. In the described setup, that means there's no per-query API charge for inference. That can be appealing for a tool used repeatedly across a large legacy estate - where metered calls might make people use it less than they should.
But "local AI" isn't the same thing as "nothing leaves the laptop." The project's semantic index can be stored in a free Tiger Cloud PostgreSQL service. That means embeddings and repository history might be sent to a database you choose to operate, even if the model inference stays local. The details of configuration and your threat model matter more than the label.
Privacy depends on the whole data path
Poltergit also instruments sessions with Sentry. The documented configuration turns off default personally identifiable information collection and avoids sending prompts or source code; traces include operational details like timing, model, and token counts. That separation is helpful, but anyone considering a similar setup should verify exactly what their configuration and SDK versions will send.
A practical review should cover:
- Where are raw commits, embeddings, and pull-request discussions stored?
- Which services receive telemetry, and what fields are included?
- Can repositories contain secrets or personal information in history?
- What retention and access controls apply to the index and traces?
- Are local models and their dependencies approved for the organization?
These questions matter for internal security policies and, where applicable, privacy obligations like the GDPR. Local inference can reduce exposure to hosted model providers, but it doesn't remove the need to map every data flow.
The economic case includes setup and latency
"No per-query inference cost" doesn't mean "no cost." Local models consume CPU, memory, and developer time. In the CPU-only example, cited inference took almost three minutes. That might be fine for an occasional archaeology task, but it's a problem for fast edit-run loops. Hardware, model size, response length, and how often the tool gets used all affect the trade-off.
There's also an infrastructure cost beyond the obvious: vector database setup, indexing, model installation, and monitoring. If a team can answer most questions with git log and a few minutes of inspection, those components may not pay for themselves. But if you have a large codebase, frequent handovers, and recurring investigations, it becomes more reasonable to evaluate the retrieval layer.
The right business question isn't "Is AI cheaper than an API?" It's "Does this system reduce the total cost of understanding code enough to justify latency, setup, and governance?"
What should teams evaluate before adopting it?
Poltergit is a useful prototype and a clear demonstration of a design pattern. Before rolling something like this out across a production engineering org, test the workflow against real maintenance problems. A demo repository is reproducible, but it can't represent the quality, gaps, or conventions in your own history.
Start with a small sample of genuinely confusing lines: retry loops, delays, feature flags, compatibility branches, and constants with unclear names. For each one, write down what a senior engineer believes the rationale is - then compare the tool's answers and sources. Include cases where the history is incomplete. The point isn't to reward confident prose. It's to see whether the system actually finds useful evidence, makes uncertainty visible, and avoids inventing a rationale.
A practical evaluation checklist
- Retrieval quality: Does line history find the relevant changes? Does hybrid search pull in related commits without flooding the output with noise?
- Evidence quality: Are citations exact, readable, and sufficient to verify the claim? Can users inspect the original context without relying on a tiny snippet?
- Failure behavior: When evidence is missing or validation fails, does the tool clearly say so instead of presenting a plausible guess?
- Latency: Is inference time acceptable on the machines developers actually use? Test cold starts as well as warm runs.
- Data boundaries: Where do repository contents, embeddings, pull-request text, and telemetry go? Can you run the system without relying on services your security team hasn't approved?
- Maintenance burden: Who updates the models, database schema, integrations, and indexing logic? What happens when a repository is rebased or history changes?
- Decision fit: Does the answer help a developer decide what to inspect next, or does it encourage people to treat historical intent as proof that the code is still necessary?
One rule that's worth taking seriously: make it easy to verify an answer. If someone can't jump from a claim to its source quickly, the citation becomes decoration. Also, silence should be allowed. If product metrics reward an answer for every query, the system will get pushed toward confident completion even when the evidence is weak.
Keep the human decision where it belongs
Historical context can explain why code was introduced. It can't decide whether the original constraint still applies. The gateway may have changed. A dependency may have been replaced. A timeout might have been a workaround for a bug that no longer exists.
Use a tool like this as an investigation assistant, not an authorization system. Let it shorten the path to relevant evidence, then validate that evidence against current behavior, tests, and system architecture. For higher-risk changes, the answer should be a starting point for review - not a replacement for engineering judgment.
Conclusion
Poltergit tackles a real engineering problem: code often outlives the people and conversations that explain it. Its strongest idea isn't that AI can talk confidently about a line of code. It's that AI should point to evidence, have that evidence checked, and be willing to say when the historical record is silent.
For teams looking at AI in maintenance workflows, that's a solid standard to aim for. Keep the question bounded, make the evidence inspectable, measure retrieval and latency on real repositories, and map every data flow. Then treat the result as a faster route into the history - not as a substitute for engineering judgment.