Formal Verification of Signal: What AI Proved in Code11 min read2,181 words

Formal Verification of Signal: What AI Proved in Code

Learn how AI agents and Lean helped verify Signal’s core cryptographic code, what the proof covers, its limits, and what security teams can apply in practice.

formal verification of SignalAI formal verificationLean theorem provingcryptographic protocol securitysoftware verification

Formal Verification of Signal: What AI Proved in Code

What AI-Proved Formal Verification Means for Signal

Formal verification has a practical catch: it's hard to prove a security design is sound, and it's often even harder to prove that real, production code actually implements that design. The work has to translate messy code into a mathematical model, spell out the properties precisely, and then scrutinize every mismatch between the model and what ships. For many teams, that cost has meant verification stays in the lab rather than the release pipeline.

Google DeepMind says its AlphaProof Code framework helped formally verify key security properties in Signal's core cryptographic implementation. AI agents generated more than six million lines of Lean proof over four weeks. Human experts defined and reviewed the security specifications, and Lean mechanically checked the proofs. That's a notable result - but it doesn't mean an AI independently certified the entire Signal app, nor does it prove the app can never have a vulnerability.

This article looks at what the team verified, how the human-and-agent workflow worked, why implementation-level checking matters, and what engineering leaders can take away without overreading the scope. The core lesson isn't just that AI can write proofs. It's how to combine automation with precise specifications, independent checking, and targeted human review.

What AlphaProof Code verified in Signal

The project focused on key security properties in Signal's core cryptographic code. Signal's protocol is designed to protect private conversations, including when a device may be temporarily compromised. The researchers took selected security goals and turned them into formal, mathematical specifications, then set out to prove that the relevant implementation satisfies those properties.

That distinction is important. A protocol can look correct on paper and still be implemented wrong. A small issue in state handling, a gap between the specification and the code, or an overlooked edge case can erode the protection users count on. Formal verification is meant to reduce that gap by making the claim precise and checking it systematically, instead of relying only on testing or expert review.

The reported scope is the core cryptographic implementation and the properties specified for it. It's not a proof that every part of the Signal app is flawless. A messaging product also depends on other areas - account security, device configuration, operating systems, network behavior, user interfaces, and operational processes. In other words, this should be read as a deep verification of one security-critical component, not a blanket guarantee of the entire product.

The team also tested whether the system could catch known defects. Researchers reintroduced two real bugs that had previously been fixed into an isolated version of Signal's code. In both cases, the system produced a machine-checked proof showing that the buggy version did not satisfy the relevant security property. That's useful: it suggests the workflow can distinguish at least some insecure implementations from correct ones. It's still not proof that every unknown defect would be found.

The practical takeaway, then, is specific. A widely deployed protocol's cryptographic implementation received formal assurance at a depth that the researchers say hadn't been achieved before for a protocol of this kind. The value is in strengthening confidence for a defined set of protections - while keeping the boundary of what was actually proved clearly in view.

How AI agents and Lean worked together

AlphaProof Code is described as a multi-model agent framework that splits formal verification into roles. Researchers and AI models helped translate Signal's security goals into formal specifications. Human experts reviewed those specifications. After that, AI agents powered by Gemini Carbon generated Lean code intended to prove that the implementation met the specifications.

Lean is a proof assistant: it verifies whether a proof follows from its definitions and rules. That matters because generated output shouldn't be treated as automatically correct just because it looks reasonable or came from a capable model. The system has to produce a proof that Lean will accept. The language model can help draft proof steps, but the checker provides the separate, mechanical validation.

The reported scale was more than six million lines of generated Lean. That number isn't, by itself, a measure of proof quality. Bigger proof efforts can include extra supporting definitions and intermediate steps, and volume alone doesn't tell you whether the underlying security claim is the right one. What counts more is the division of responsibility: people determine and review what should be proved, agents help with the heavy proof construction work, and the proof checker validates the result against the formalized claim.

The workflow also used AI agents to audit the generated proof and flag areas that needed human attention. DeepMind reports that experts had to review about one percent of the codebase for this targeted process. That figure shouldn't be read as "almost all security review is solved." It's about how much review was needed within this specific verification workflow. When the assumptions, specifications, and scope boundaries are on the line, human judgment still matters.

The project finished in four weeks. That's faster than the years such work has historically taken for expert teams. Still, it's not a universal estimate. Timing will depend on how the code is structured, how much is in scope, how clear the security requirements are, and whether you have people who understand both the protocol and the proof environment.

Why implementation-level verification matters

Security analysis often starts with the design of a cryptographic protocol. That step is necessary - an incorrect design can't be fixed by careful coding. But even a correct design doesn't automatically mean the deployed code is correct. The implementation has to preserve the design's behavior across real code paths, and in this case down to what runs on devices after compilation.

That's where the hard engineering work lives. A specification is an abstraction. Production software includes concrete data structures, state transitions, and interactions that may not map neatly onto a high-level protocol description. To make a formal claim about code, researchers have to connect those details to mathematical properties. A proof that covers only the design leaves the implementation gap open; a proof that reaches the implementation offers stronger evidence - as long as the scope and assumptions are spelled out.

For Signal, the properties included protections like keeping messages secure before and after a device is temporarily compromised. These aren't casual promises. They hinge on how cryptographic state evolves and how the system handles compromise and recovery. Formalizing that behavior requires experts to define exactly what "secure" means inside the model. Then the proof shows that the modeled implementation satisfies that definition - not that it answers every possible security question about the product.

This is why "formally verified" shouldn't be treated as "bug-free." A proof only establishes what its specification says, under the assumptions encoded in the model. If the requirement is incomplete, a relevant component is outside scope, or an assumption doesn't hold in deployment, the proof can't paper over the gap. And even with theorem proving, you still need threat modeling, secure development practices, operational controls, and ongoing review of changes after verification.

Even so, implementation-level proof can raise the level of assurance available to teams. Testing exercises behavior across selected inputs and conditions. Expert review can catch subtle mistakes, but it's limited by time and attention. A machine-checked proof can establish a property across the formalized cases, and it doesn't depend on a reviewer manually re-deriving every inference. It's not perfect security; it's a stronger, more explicit argument about a defined part of the system.

What engineering teams should learn from the project

If you're a security leader, the first lesson is to pick the claim before you pick the tool. "Verify our software" isn't a useful brief. Teams need a concrete property with real consequences - like whether a specific cryptographic state transition preserves a confidentiality guarantee under stated conditions. Narrow, high-value claims are easier to reason about, and they end up being more actionable than broad assurances that don't translate cleanly into checks.

A practical adoption path might look like this:

  1. Select a high-impact boundary. Start with a component where a defect could cause material user, safety, or business risk. Cryptographic code, payment authorization logic, and safety-critical device controls may justify the cost; a low-risk feature probably won't.
  2. Write down the security property. Define what must stay true, under which conditions, and what you're explicitly excluding. Have domain experts review the specification before asking agents to prove it.
  3. Map the implementation to the claim. Identify the source code, build steps, and compiled artifacts within scope. If the proof doesn't reach deployed behavior, record that limitation instead of implying broader coverage.
  4. Use automation for proof construction, not authority. Agents can cut down manual effort building and exploring proofs. Keep a trusted checker in the loop, and preserve the ability to inspect assumptions, failed cases, and human review flags.
  5. Budget for ongoing maintenance. Software changes. A proof tied to a previous version isn't automatic evidence for a later release. Fold proof updates into change review and decide which changes require re-verification.

The business case tends to be strongest when the cost of failure is high and the component changes at a manageable pace. Formal verification can take specialist time and can surface unclear or contradictory requirements before a proof ever exists. That's often worth it when the alternative is a low-probability failure with high consequences - but it's rarely a sensible "do it everywhere" mandate.

There are also organizational dependencies. Teams need people who can translate security intent into precise specifications, maintain the formal model, and judge what falls outside it. AI can reduce some proof-writing labor, but it doesn't remove the need for those skills. The scarce resource may shift from building every proof step by hand to deciding what's worth proving and making sure the specification reflects the real threat model.

For technology executives, the question isn't whether every team should adopt theorem proving immediately. It's whether a critical product claim is important enough that stronger evidence is worth the cost. The Signal project suggests AI-assisted workflows may make deeper verification practical for more systems than before. But the proof still needs scoping, review, and maintenance like any other security control.

Key takeaways

  • AlphaProof Code helped verify specified security properties of Signal's core cryptographic implementation, not every part of the app.
  • Human experts reviewed the formal specifications; AI agents generated Lean proofs; Lean checked them mechanically.
  • DeepMind reports more than six million lines of generated Lean and a four-week verification effort.
  • The workflow flagged areas for human review, but it doesn't make human security judgment optional.
  • Formal proof increases confidence in a defined claim. It doesn't guarantee the absence of vulnerabilities or that all deployment assumptions hold.
  • Best candidates are high-impact components with clear properties and a realistic plan to keep proofs updated as code changes.

Conclusion

The Signal verification project stands out because it brings AI into a demanding engineering task without treating AI as the final authority. People defined and reviewed the claims, agents helped construct proofs at scale, and Lean checked the results. The combination made a deep implementation-level effort possible in weeks, while keeping formal validation separate from model-generated text.

For teams building security-critical software, the next useful step isn't to declare the system verified. Pick one high-consequence property, define its scope and assumptions, and assess whether the proof can be maintained alongside the code. AI may reduce the cost of getting there, but the quality of the claim - and the discipline to keep it current - remains the team's responsibility.

Frequently Asked Questions

The work formally verified selected security properties of Signal’s core cryptographic implementation. Researchers and AI models helped define the specifications, human experts reviewed them, and agents generated Lean proofs. Lean checked those proofs. The result is bounded to the specified properties and code in scope, not the entire Signal app.
No. A formal proof establishes that the modeled code satisfies a specific formal specification, under stated assumptions. It does not cover every product component, threat, operating-system behavior or future code change. The work strengthens assurance in defined cryptographic protections; it does not eliminate the need for security reviews, testing and ongoing maintenance.
Human experts helped define the security properties and reviewed every specification, then examined areas flagged by the agents. AI agents generated proof code, while Lean mechanically checked whether the proofs followed from the formal definitions and rules. The separation matters: generated output was not accepted merely because an AI produced it.

Related articles