Local LLM for Git History: A Practical AI Experiment

The past is useful, but incomplete

Commit messages aren't a full project diary. They can be short, inconsistent, or written mostly for someone debugging later. "fix typo" tells you almost nothing about the stress around a release. And the repo itself can miss plenty of real decision-making - things that happened in meetings, in client conversations, or in issue trackers.

So the model shouldn't be treated like an archivist with a complete view. It's working with whatever evidence is in the commit log. git-ghost's system prompt asks Gemma to stick to facts that are present in that log. That's a good guardrail: it can interpret what it sees, but it shouldn't claim a file or turning point existed unless the history backs it up.

Even with that constraint, accuracy isn't guaranteed. A language model can misread a message or over-interpret the sequence of commits. In practice, you should treat the result as a prompted reflection - not an authoritative account of your project's history.

Why make the interaction playful?

The tool frames the response as a "ghost" in three parts: "I remember..." for the project's beginnings, "I see you now..." for what changed, and "Before I fade..." for one concrete suggestion. The Día de los Muertos framing gives the interaction a shape, without turning it into a generic productivity dashboard.

That choice matters. You don't always need to optimize a work metric to build something useful. Sometimes a small, well-framed interaction helps people notice patterns they'd normally gloss over. The key is being honest about the value: git-ghost offers a perspective on your commit log, not a substitute for talking with the people who built the software.

A small input makes a more focused feature

This isn't meant to be a general code assistant. It doesn't need to scan every file or answer arbitrary questions about the whole codebase. Its job is to turn selected history into a structured response.

That limited scope is a real design advantage. It's easier to explain, and it gives the prompt a clear boundary. There's also a practical security angle: a full repo can include secrets, personal data, generated files, and lots of irrelevant implementation detail. Even though a commit log can still contain sensitive information, it's usually a smaller, narrower input. Picking the smallest useful context reduces what the model has to interpret and lowers the chance you accidentally feed it things you shouldn't.

Of course, there's a trade-off. Commit messages often don't include enough detail to support a truly meaningful observation. If the tool ever needs richer context, it should ask for it explicitly - ideally through an option the user controls - rather than quietly uploading (or reading) the entire repo.

Test the integration without downloading a model

The project includes a unit test that starts a fake Ollama server and creates a temporary Git repository. That means the test can exercise the request logic and the repo-handling code without installing a model or relying on a real inference response.

This is a solid way to test small AI integrations. You can't make a deterministic unit test prove what a language model will always say. But you can test the parts around it: that the right commits are collected, the request is formed correctly, the stream is handled properly, and failures show up in predictable ways.

And in real systems, keep those tests separate from model-quality evaluation. A fake server confirms the plumbing; representative repos plus human review are what you use to check whether outputs stay grounded, useful, and appropriately cautious. Otherwise you end up confusing "the integration didn't crash" with "the AI behavior is reliable."

Local is a deployment choice, not a blanket privacy guarantee

Running the model locally removes one specific risk: you don't have to send the prompt to a hosted inference API. But it doesn't automatically make everything "private" in every sense. You still need basic access controls on the machine, avoid logging sensitive prompts unnecessarily, and make sure you understand which model and runtime are actually in play.

That distinction matters for teams. "Local AI" isn't a free pass on threat modeling. Before using a local model with company repos, check runtime configuration, whether the endpoint is exposed, machine policies, and what kinds of information show up in commit messages. And if someone modifies the tool to hit a remote endpoint, the privacy story changes immediately.

Local execution also shifts costs rather than deleting them. There may be no per-API-call cost, but inference uses your machine's resources and can be slower than a hosted setup. A small default model makes the experiment more approachable, but quality and speed depend on your hardware and the task. In the end, the right choice is the one that performs well enough for the use case - not the biggest model you can run.

Open weights make the behavior more inspectable and adjustable

Using an open-weight model with a small, readable client gives users more control. They can inspect how the prompt is assembled, swap models, or change the output format to match their repo. That matters because the output is subjective: one developer wants something warmer, another wants a more technical, straight-to-the-point summary.

It's also worth saying plainly: this isn't automatically a business case for every team. Hosted systems can be easier to operate and may support capabilities a local setup can't match. But when the data is personal, the task is bounded, and the user cares about control, local inference can be a better fit than a generic cloud integration.

Design for evidence, not just fluent output

Language models can tell a convincing story even when the evidence is thin. That's exactly why git-ghost's prompt asks the model to use only facts from the commit log.

For a business tool, it helps to go further: make the source material easy to inspect, separate direct evidence from interpretation, and give users a way to correct or reject the result.

A practical review checklist looks like this:

• Input: Is the model getting only what it needs for this task?
• Grounding: Can a user trace key claims back to a source record?
• Uncertainty: Does the interface show where the source material is thin?
• Fallback: Does the tool fail safely if Ollama is unavailable or the model output is unusable?
• Evaluation: Are representative examples reviewed by people who understand the underlying work?

These aren't pointless bureaucracy. They help you see where an AI feature saves time and where it might create false confidence. If a wrong answer could actually cost someone something, the review bar needs to go up accordingly.

Keep the feature small until its value is clear

The strongest reason to start with a small AI tool isn't that small tools are always better. It's that a narrow experiment lets you learn with less operational and privacy risk. git-ghost can test whether developers enjoy revisiting their work through the project's history without needing accounts, a cloud service, or a big product surface area.

If people genuinely want more, the next step shouldn't be "add features automatically." Ask what users found valuable: the historical summary, the advice, the local execution, or just the ritual of looking back. Different answers point to different product decisions. If users only want a brief reflection, adding repository-wide analysis can increase complexity without adding much value.

For teams, a useful metric isn't whether the output sounds impressive. It's whether the feature helps someone make a better decision or finish a real task - while keeping the limitations obvious. If it does neither, then the "novelty" might be the whole product. That's not necessarily bad, as long as it's honest about what it is.