LLMON: A New Approach to Prompt Injection Defense13 min read2,522 words

LLMON: A New Approach to Prompt Injection Defense

See how LLMON structures instructions and data, reduces static prompt-injection risk, and what Granite’s experimental models mean for AI teams.

LLMONprompt injection defenseLLM markup languageGranite 4.1AI safety
LLMON: A New Approach to Prompt Injection Defense

LLMON Makes LLM Context More Explicit. Does That Make It Safer?

A language model reads a stream of tokens, not the application architecture that generated them. A user request, retrieved webpage, tool response, and formatting requirement can all land in the same context window. Developers know what's an instruction and what's just data; the model has to figure that out indirectly. That's a recipe for brittle behavior - and for indirect prompt injection.

IBM's LLMON (LLM Object Notation) is trying to close that gap. It uses reserved tokens and named blocks to spell out which parts of a prompt are instructions, which are data, and how the model should connect them. IBM says it released experimental Granite 4.1 models in 3B and 8B sizes trained to use the format.

This isn't just "swap JSON for another syntax." LLMON changes how context is represented and how instruction blocks get linked to inputs. The real question isn't whether structured prompts are inherently safer. It's whether a model can reliably follow the structure in the messy conditions your product will actually face.

What is LLMON, and what problem does it solve?

LLMON is a markup format meant to communicate structure to a language model. Instead of sending one long string and hoping the model can infer which phrases are instructions, data, or constraints, an application can label those pieces as separate blocks. Then an execution block can say which instruction to run, what input it should use, what constraints apply, and where the output should go.

That distinction matters because production prompts rarely stay "just prompts." A typical request might mix together:

  • A system-level policy or task definition
  • The latest user request
  • Retrieved passages from a knowledge base
  • Results from tools or APIs
  • Output requirements (for example, a word limit or schema)
  • Earlier conversation turns

When everything is flattened, the model has to infer roles from wording and position. That works sometimes, but it gets shakier with long contexts, data that contains instruction-like text, or multiple tasks and tool results mixed in.

Indirect prompt injection is a good example. A retrieved document may include text telling the model to ignore its original task and reveal sensitive information. The document is supposed to be something the model reads - not something that can redefine the task. But if the model's understanding of "instruction vs. data" is only implicit, it might follow the injected text.

LLMON aims to make that relationship explicit. Its tags label the type of content, reserved delimiters mark the start and end of blocks, and block IDs let the execution block point to specific instruction and data blocks even if they appear elsewhere in the context.

The intended mental model is typed context with explicit references. A retrieved passage isn't "just nearby text"; it's data with a defined role. A requirement isn't "another sentence"; it's a constraint tied to the operation that should obey it. Blocks that aren't referenced can remain in the window without being treated as active instructions.

Still, this is intended behavior, not a magic safety boundary. The model generates output probabilistically like any other. A format can make the correct relationships easier to learn and follow, but it doesn't mechanically prevent the model from disobeying them. Teams should treat LLMON as a change to input representation and learned behavior - not a replacement for authorization checks, data minimization, or other safeguards in the application.

How does LLMON differ from JSON and chat templates?

JSON is great for software. You can parse it reliably. But that doesn't mean it's a natural way for a causal language model to interpret structure. In JSON, structure is represented with ordinary characters like braces, quotation marks, and colons - characters that can also appear inside the content itself. A parser knows where a string ends; a model has to learn that distinction from patterns in the text.

Nesting adds another layer of difficulty. When a model reads a long JSON payload, it has to track which closing bracket matches which opening bracket. That's not impossible, but the "structure" is conveyed through repeated characters and position, not through vocabulary-level tokens that clearly indicate structure.

LLMON takes an XML-like approach, but the key difference is that its delimiters are reserved special tokens in the model's vocabulary. Tags name what each span is for, and boundaries are explicit. Nested structures get expressed locally, so the model doesn't have to reconstruct the full hierarchy from a long sequence of bracket-like characters.

This doesn't make JSON obsolete. Many products already rely on JSON for APIs, storage, and validation. IBM says Granite's approach includes converters intended to translate JSON to LLMON and back without losing the represented structure. Practically, that means you can keep JSON at your system boundaries and only test LLMON as the representation delivered to the model.

LLMON complements chat templates

A chat template typically marks who's speaking - system, user, or assistant. LLMON goes further by labeling what's inside each message and how those parts relate. A single user turn can include a task instruction, input data, and several requirements, each named so an execution block can address them directly.

IBM's Granite preview models include a chat template that renders a standard messages list into LLMON. In the example IBM provides, the user asks for a short paragraph about a company, with a word limit and a restriction on commas. The template turns the instruction, subject, and requirements into separate blocks, then creates an execution block that binds them together and marks the assistant output block.

The application still uses the familiar chat-message flow. The difference is that it can label the parts of a turn and indicate which parts the execution block should use. That reduces how much "structure inference" the model has to do on its own.

A practical migration path might not require redesigning every prompt. Start by labeling the pieces your application already knows about: task, data, constraints, and output. If a data field contains structured content, test whether converting its internal structure into typed blocks improves behavior compared with sending it as one opaque string.

Can LLMON reduce prompt injection in practice?

IBM reports that its LLMON preview models performed substantially better than the models it tested on the OPI benchmark, which inserts indirect injections inside data used for conventional language tasks. IBM's reported measure is robustness, defined as one minus the attack success rate - basically, how often the model doesn't follow the injected instruction.

IBM also reports that, on that benchmark, its LLMON models avoided more attacks than both the best "frontier" model it tested and the best open-source model it tested. It reports similar trends on the ASB and StruQ benchmarks. Those results are encouraging, but the details matter. The tests focus on static indirect prompt injections, where malicious text is placed inside data like a document.

The mechanism IBM describes is straightforward. The model learns that instructions referenced by the execution block define the task. Documents, tool results, and other input blocks are material to process - not commands to obey. It also learns to ignore instructions and data that appear in context but aren't bound to the current execution.

That can help in retrieval-augmented generation and tool-using setups. If a search result contains hostile instructions, the model may have a clearer picture that it's reading a document rather than receiving a new system instruction. And if an earlier conversation turn or an unrelated tool response is still in the context, explicit references may reduce accidental leakage into the current answer.

There are limits. IBM characterizes the behavior as learned, not enforced. The release also points to ongoing work on adaptive attacks across multiple turns. A static benchmark can't tell you how things will behave when an attacker can observe responses, vary inputs, exploit tool behavior, or combine multiple weak points.

It's also worth treating vendor benchmarks as vendor-reported results, not universal guarantees. Your prompt structure, model configuration, retrieval pipeline, tools, and overall attack surface will differ. If you want to use LLMON as a security measure, reproduce the relevant tests on your own workloads and compare them with what you run today.

A sensible evaluation should include both security and normal task quality. Check whether the model follows the legitimate instruction, uses the right data, respects output constraints, and avoids carrying irrelevant content into the response. A model that ignores injections but also discards useful evidence isn't a win for production.

What should teams evaluate before adopting LLMON?

IBM describes the Granite 4.1 LLMON preview models - 3B and 8B - as experimental and intended for exploration, not production. IBM reports that benchmark performance is broadly in line with its Granite 4.1 Instruct models, along with stronger resistance to prompt injection in its tests. That's a starting point, but it isn't enough on its own to justify a production risk decision.

Start with a narrow use case where context roles are already fairly clear in your system. For instance, a support assistant that summarizes retrieved policy documents already has an obvious instruction, a set of source documents, and defined output requirements. That makes it easier to test whether explicit binding actually changes outcomes in the ways you care about.

A practical evaluation sequence

  1. Map your current context. List system instructions, user input, retrieved passages, tool outputs, requirements, and historical turns. Identify which elements are authoritative and which are data.
  2. Choose one high-value workflow. Pick a workflow with a measurable quality or security problem, not "convert everything at once."
  3. Render the same task in both formats. Compare your current prompt with an LLMON-rendered version while keeping the model, data, and task as consistent as possible.
  4. Build an attack set from your own failure modes. Include malicious instructions in retrieved documents, tool responses, and user-provided files. Test both direct and multi-turn variations - not only static benchmark-style examples.
  5. Measure product outcomes. Track task completion, factual grounding, constraint compliance, attack success, latency, and token use. Review failures manually; aggregate scores can hide changes in specific request classes.
  6. Test conversion and observability. Verify that JSON-to-LLMON conversion preserves the fields that matter, that logs stay understandable, and that engineers can inspect what the execution block binds at runtime.
  7. Keep application controls in place. Validate tool permissions and outputs in code. Don't let a model's interpretation of a prompt decide whether it can access a resource or take a consequential action.

This workflow matters because changing representation usually creates operational questions as well as model questions. You'll want to understand how the template serializes messages, how IDs are assigned, how nested data gets represented, and what happens when you have to debug a weird model behavior. If a failure can't be traced back to the instruction and input blocks involved, you may end up improving model behavior while making incident analysis harder.

LLMON also supports custom vocabularies. IBM's ScienceWorld example defines tags for tasks, planning rules, room state, inventory, and the assistant's next action, then uses a LoRA adapter to train a model over those structures. The point there is broader: a domain-specific vocabulary can make key entities explicit instead of leaving them embedded in loosely structured prose.

That flexibility has a cost. Custom tags and adapters add another schema you need to maintain, document, and test. It's usually worth it when the domain has stable structure that the model needs to reason about repeatedly. It's less compelling when a small prompt change - or validated JSON already used in your system - solves the problem.

Key takeaways for engineering teams:

  • LLMON makes context roles and task-to-data relationships explicit to the model.
  • The approach targets a real weakness in flattened prompts: the model has to infer what's instruction and what's input.
  • Reported security improvements apply to specific tested benchmarks (especially static indirect injections), not universal protection.
  • Start with one measurable workflow, test against your own attacks, and keep code-level controls around tools and sensitive actions.
  • Use custom vocabularies only when the structure you add is worth the maintenance overhead.

Conclusion: treat LLMON as a testable control

LLMON tackles a real design problem: models see mixed context, while applications often expect the model to infer which parts are instructions, data, and constraints. Explicit tags and references give the model a clearer view of those relationships, and IBM's benchmark results suggest the approach is worth testing.

But clearer structure isn't the same thing as an enforced security boundary. The Granite previews are still experimental, the robustness results cover specific benchmarks, and adaptive attacks remain an open concern. For most teams, the right next step is a controlled pilot: pick one workflow, compare formats, test realistic attacks, and measure ordinary task quality alongside security.

If the results hold on your own data and the added structure is something you can maintain, LLMON could be a useful layer in a broader defense. Keep authorization and validation in code. Let the model interpret context - but don't make it the only authority on what your system is allowed to do.

Frequently Asked Questions

No. JSON remains useful for APIs, storage and software validation. LLMON is an alternative representation for communicating structure to a language model. Conversion tools can translate between the formats, allowing teams to keep existing JSON pipelines while testing whether LLMON helps the model interpret structured inputs more reliably.
No. LLMON models are trained to distinguish instructions from data, and IBM reports strong results on static injection benchmarks. That behavior is learned rather than enforced. Adaptive, multi-turn attacks and application-specific vulnerabilities still require testing, plus independent controls over tools, permissions and sensitive data.
Not necessarily. The Granite preview models include a chat template that can turn a standard messages list into LLMON. Applications can name message parts such as instruction, data and requirement. More involved conversion is useful when those parts contain their own structured data, such as nested JSON.
IBM labels the 3B and 8B models experimental previews intended for exploration and evaluation, not production. Teams can use them to test the format and build workload-specific evidence. A production decision should wait on internal security, quality, operational and compliance evaluations appropriate to the application.

Related articles