AI Agent Memory 🚀: Portable Context for Enterprise AI
Discover why AI agent memory and an AI context layer matter for enterprise AI agents—learn what buyers must evaluate for trusted, portable AI memory.
Why Enterprise AI Needs Memory It Can Take With It
Enterprise AI has a context problem. Models can generate solid answers, but they often don't have the company-specific knowledge that makes those answers reliable - like who owns a process, which system holds the latest truth, and how decisions actually get made. Just connecting an AI assistant to more software doesn't automatically fix that. It can also slow things down, raise costs, and make governance harder.
Glean CEO Arvind Jain argues that a separate context layer can close some of those gaps. In a TNW interview at HumanX in Amsterdam, he walked through Glean's approach to enterprise search, agent memory, and model choice. One point, though, sticks: Glean wants the memory agents build about people and companies to stay portable, instead of being locked inside a single AI vendor.
That raises practical questions for buyers. What should an enterprise AI agent remember? Who should control that information? And when does a context platform earn its keep? This piece looks at Jain's claims, the business logic behind them, and a way to evaluate the approach without treating any one vendor's architecture as the answer.
Why enterprise AI needs a context layer
Glean started as a search product for company information. It connects to tools like Salesforce, HubSpot, Jira, GitHub, and HR or IT systems, then helps people find what they need across them. Jain describes the current product as an AI coworker and agent platform that can use models from multiple providers.
The distinction between access and context matters. A model might be allowed to query company systems, but that doesn't mean it understands how the organization works. It can pull a project document without knowing whether it's current, identify a person without knowing their role in a decision, or surface conflicting answers without knowing which one the team trusts.
A context layer tries to make those relationships available to the model. Jain said Glean builds knowledge graphs that represent people, concepts, and processes. The catch is that the value of any graph depends on whether it reflects reality, stays up to date, and keeps the access rules from the underlying systems. A "diagram of the company" isn't automatically useful - what matters is whether it helps an agent pull the right information for the right task.
The information that never made it into a database
Jain estimates that 80% to 90% of how work gets done lives in people's heads rather than in formal documents or databases. Treat that as his assessment, not a universal measurement. The real issue is familiar in big organizations: important knowledge often gets distributed across conversations, habits, and informal working relationships.
A new hire may not know which of three documents is the final source. A support rep might know when exceptions get approved, even if the policy page never says so. A product manager may understand why a decision was made, but the rationale never landed in a system of record.
AI can help surface patterns in the information employees already create. But more data isn't the same thing as useful institutional knowledge. Systems also need ways to separate durable context from one-off comments, private information, and outdated instructions. When that line is blurry, an agent can confidently repeat stale or irrelevant information at scale.
Context as a layer, not another destination
Jain says customers can use Glean without making its app the main interface. In his framing, another assistant - like Claude - can ask Glean for relevant company context, instead of searching across dozens of systems on its own.
That's different from asking an organization to replace all its existing tools. It puts Glean in the middle, translating company knowledge into whatever AI interface employees already use. The upside is less duplicated integration work and more consistent retrieval. The trade-off is simple: it adds another system between the user and the data.
For buyers, the useful question isn't whether a context layer sounds modern. It's whether it improves task completion in ways you can measure. Does it retrieve better evidence? Does it cut down time spent searching? Can teams trace where an answer came from? And does it work across the applications your people already rely on, instead of creating yet another place they have to remember to check?
Portable memory is a governance decision
AI agents can accumulate useful information about how people work: preferred formats, recurring tasks, feedback on drafts, and corrections to earlier outputs. That kind of memory can make an assistant more helpful over time. It can also create a new dependency - if the memory stays trapped inside one provider's product and can't be exported or reused elsewhere.
Jain's position is that Glean stores this in a standard format controlled by the company. He said the same idea applies to information agents learn from human feedback. His stated goal was to make Glean "very easily replaceable." That's a notable claim in a market where vendors often benefit when customers rely on proprietary data, workflows, and integrations.
The core argument is reasonable: if a company can move its agent memory, switching models or providers becomes less disruptive. But portability isn't a slogan. It has to be proven through actual data formats, export tools, documentation, and migration testing. A memory store can be "exportable" in theory and still be painful to use elsewhere if its structure or meaning is proprietary.
Decide what an agent is allowed to remember
Before choosing where agent memory lives, define what counts as memory. A practical starting point is to split it into categories:
- User preferences: preferred tone, format, language, or level of detail.
- Task context: information needed to complete a specific request or workflow.
- Company knowledge: policies, projects, people, processes, and approved documentation.
- Feedback: corrections and evaluations that improve future outputs.
- Sensitive or personal information: data that should be excluded, restricted, or retained only under explicit rules.
Different categories have different owners and retention needs. One person's preference for concise summaries may travel with that employee. A company process should stay under company control. A private message shouldn't automatically become organization-wide knowledge just because an AI system can access it.
Jain said Glean follows existing company permissions, and that personal information (like LinkedIn messages) stays with the employee rather than becoming company data. Those boundaries matter for the product's trust model. If you're evaluating something similar, verify how those boundaries work in your environment - especially permission sync, revocation behavior, and what happens to stored context if a user changes roles or leaves.
Keep access control out of the model's judgment
Jain emphasizes that permission checks are programmatic, not left to the AI. That distinction is important. Language models are probabilistic systems; authorization has to be enforced consistently. A model shouldn't decide whether an employee can view someone else's inbox based on how it interprets a prompt.
The model can help interpret a request or summarize information the user is already authorized to access. The surrounding system should decide what that user can access in the first place. In other words: use AI where flexible language understanding helps, and use deterministic controls for permissions, identity, and policy enforcement.
That design still needs real testing. Permissions inherited from source applications can be messy, and access changes can outpace index updates. Buyers should ask how quickly permission changes propagate, whether retrieved content is filtered before it reaches the model, and whether logs make it possible to investigate an improper disclosure. "It follows existing permissions" is something you validate - not something you accept at face value.
Model choice is an operating-cost decision
Jain says Glean would rather use a model other than Anthropic's Claude for 85% to 90% of enterprise tasks, describing Claude as very expensive for many of those workloads. He also says some GPT models cost an order of magnitude less and can handle much of the same work. Those are Jain's characterizations - actual cost comparisons depend on the specific model versions, negotiated pricing, task design, input length, and output quality.
The broader point is that an enterprise agent platform shouldn't assume every task needs the most capable - and usually the most expensive - model available. A straightforward classification, extraction, or routing task may work fine with a smaller model. A more complex analysis might justify a stronger one. Using a premium model for every request is easy to set up, but it can turn usage growth into an ongoing cost problem.
Route tasks by quality, latency and cost
A workable model strategy starts by grouping tasks by what they require. For each group, set a minimum acceptable quality level, a latency target, and a cost ceiling. Then compare candidate models using representative company data - not just generic benchmark numbers.
A practical routing policy might look like this:
- Use a lower-cost model for predictable, bounded tasks like categorization or standard extraction.
- Escalate to a more capable model when the task involves ambiguity, multiple sources, or reasoning with real consequences.
- Require human review when an error could create legal, financial, safety, or serious customer impact.
- Track failure rates and rework, not only cost per model call.
This shouldn't be treated as "set it once and forget it." Models change, pricing changes, and people find new ways to use the system. A model that looks good on short prompts may behave differently with long policy documents or conflicting sources. Evaluation should reflect what employees actually do.
Measure whether context improves the result
Jain says Glean's studies showed that adding its context layer made Claude twice as fast, cut token use in half, and led to work being preferred five times more often. Those are company-reported results from the interview; the source doesn't include the study design, sample size, or evaluation method. Treat them as claims worth checking, not guaranteed outcomes for every customer.
A buyer can test the same idea with a controlled pilot. Run the same tasks with and without the context layer, keeping model settings equivalent where possible. Measure:
- Answer quality, including whether sources support the response.
- Time to completion, plus employee time saved after reviewing the result.
- Model and platform cost per successfully completed task.
- Error rates, corrections, and escalations to a person.
- User preference across a representative group, not just early adopters.
The key metric isn't token savings in isolation. It's the cost and reliability of completing a real job. If a context layer reduces model usage but adds integration, licensing, and maintenance costs, the total economics may still be positive - or they may not. A good pilot makes that trade-off visible.
What enterprise buyers should evaluate
Glean says it passed $300 million in annual recurring revenue in May 2026, 15 months after reporting $100 million. Jain also says the company has more than 1,000 large enterprise customers, millions of users, and around 1,500 employees. He told TNW that Glean has no plans to raise more money and no concrete plan to go public, though it has been preparing internal processes that would make a public listing possible. Those numbers give market context, but they don't, on their own, answer whether the product fits your organization.
Start with a defined workflow, not a broad mandate to "add AI." Search across company systems, support resolution, sales preparation, and employee onboarding each come with different data quality issues, permission constraints, and expectations. A pilot that tries to prove everything at once often turns into a polished demo - and not much evidence you can use for procurement.
Use a practical evaluation checklist
Before committing to an enterprise AI context platform, get crisp answers to these questions:
- What specific work should improve? Name the task, its current baseline, and who owns the outcome.
- Which systems are in scope? Confirm integrations, data types, update frequency, and known gaps.
- How are permissions enforced? Test access with users who have different roles, and verify that revocations take effect quickly.
- What does the system store? Separate source data, derived context, user preferences, agent feedback, and logs.
- Can memory be exported and used elsewhere? Request a real export and test whether another system can interpret it.
- How are models selected? Understand routing rules, fallbacks, provider options, and cost controls.
- How will success be measured? Track quality, time saved, cost per completed task, and how much human correction is needed.
- Who owns ongoing operations? Assign responsibility for connectors, permissions, evaluations, incident response, and employee feedback.
Jain also called hiring and aligning people the hardest part of scaling Glean, saying a CEO should spend much of his time building the team. That lesson applies well beyond Glean. Enterprise AI isn't just an integration project. Someone has to own the workflows, set policy, handle failures when systems break, and stop local teams from creating conflicting versions of the same process.
Glean's origin story points to the same operational problem. Jain says the idea came from his experience at Rubrik, where the company grew quickly and employees struggled to find information. His earlier work on Google Search informed how the product finds knowledge across systems. That pattern shows up a lot: growth increases the volume of information, and it also increases the coordination needed to locate and trust it.
For buyers, the implication is pretty direct. An AI assistant can't fix every organizational problem. If source data is outdated, ownership is unclear, or teams disagree about what the process actually is, adding a retrieval layer can expose those weaknesses instead of resolving them. Sometimes the first step is cleaning up the underlying systems and responsibilities.
A decision rule for the first deployment
Start with a workflow where the information matters, permissions are understandable, and a person can verify the result. Set a baseline, run a limited pilot, and include the people who do the work every day. Expand only when the evidence shows the system improves the workflow without creating unacceptable security risk, cost, or maintenance burden.
That approach may sound less flashy than launching a company-wide agent program. It also tends to produce something people can actually trust and operate. Jain's portability argument is most useful when treated as a procurement and architecture requirement: preserve the ability to change providers, but verify that the surrounding controls and data portability are real in practice.
Conclusion: Buy for outcomes, preserve your options
Jain's argument brings together two ideas that are easy to separate but stronger when you treat them as a package: enterprise AI needs better context, and the memory that makes AI useful shouldn't turn into accidental lock-in. A context platform can help agents find company knowledge and may lower the cost of using powerful models. It also becomes part of your organization's data and security architecture, so its claims need to be tested against real workloads.
Start with a narrow workflow, set a baseline, validate permissions, and test memory portability before scaling. If the system improves completed work and stays operable as models and vendors change, expand it. If it only produces a compelling demo, don't confuse that with a production case.
For the full interview and Jain's comments on Glean's strategy, see TNW's conversation with Arvind Jain.