Agentic Analytics: Improve AI Insights With Context

What does insight quality mean in agentic analytics?

An analytics agent produces more than a chart or a query result. Depending on how it's configured, it might describe a change, explain plausible causes, recommend a response, or take action through another system. Those outputs carry different risk levels - so wrapping them all under one broad label like "accuracy" hides what actually matters.

The report's headline figure is that, on average, 48% of agent-generated insights at surveyed organizations lead to actions that improve the business. Respondents said the figure was 37% a year earlier. That's progress, but it also means more than half of the insights weren't tied to improvement. And "not used" doesn't always mean "wrong": some insights may have been irrelevant, redundant, mistimed, or simply not acted on. That distinction matters when you decide what to fix.

A useful evaluation separates at least three stages:

• Descriptive quality: Did the agent correctly report what happened? For example, did it calculate the change in weekly orders using the approved definition and the right comparison period?
• Diagnostic quality: Did it identify plausible contributing factors and separate evidence from speculation? A correlation with a campaign isn't proof the campaign caused the change.
• Prescriptive quality: Is the recommended action suitable for the business objective, constraints, and likely consequences? This is where a correct observation can still lead to a bad decision.

The report says respondents rated descriptive insights more positively than prescriptive ones. That tracks with how work actually goes. Describing a movement is one thing; choosing what to do about it means dealing with trade-offs - margin versus volume, short-term conversion versus customer retention, or speed versus compliance risk.

For a product team, an agent might correctly identify that trial-to-paid conversion fell. A weaker system might immediately recommend a discount. A better one checks whether the decline is concentrated in a segment, whether a product release changed onboarding, and whether discounting would harm unit economics. The test isn't whether the recommendation sounds confident. The test is whether the evidence and assumptions hold up under review - and whether the action improves the outcome you care about.

Why do analytics agents produce weak insights?

The leading reported cause is missing business context. In the survey, 55% of executives cited insufficient context when agents produced substandard insights. Another 45% pointed to fragmented or inaccessible data. These issues overlap: an agent can't use information it can't reach, but "having access" also doesn't automatically tell it what the information means.

Business context includes definitions, ownership, time periods, relationships, and operating rules. A database might have fields named net_revenue, active_customer, and churned_account, but those names don't tell the agent which definition Finance uses, whether a trial user counts as active, or how the company handles refunds. People settle those questions through institutional knowledge. The agent needs that knowledge encoded in a usable form.

The same problem shows up in questions that sound simple. If someone asks, "Which region is underperforming?", the agent needs to know which metric matters, what period to compare, whether regions share a common target, and how to handle missing data. Without those rules, it may return a neat ranking built on inconsistent definitions. More model capacity won't fix a metric that doesn't have an agreed meaning.

Fragmentation makes things worse. Customer, product, marketing, and finance data often live in different systems with different identifiers and update schedules. An agent can still produce a coherent narrative from partial information - sometimes more dangerous than an obvious error, because users may not realize what the system couldn't see. Access controls also matter: broad access can improve coverage while sending information to the wrong users or workflows.

There's also a workflow problem. The report found that 52% of surveyed organizations generate more agent insights than they can evaluate and act on. Only 24% said they can evaluate all of them. When teams are overloaded, review becomes case-by-case and human-intensive. A queue full of low-value alerts can bury the few recommendations that are actually worth attention.

So weak outcomes aren't only a model problem. They can come from undefined metrics, inaccessible data, poor prioritization, or a process that has no owner for validating recommendations. Treating everything as a "prompting" issue misses the point. Before you change models, identify where the failure happens: is the answer factually wrong, contextually incomplete, operationally irrelevant, or just not reaching someone who can act?

How do semantic layers and governance improve agent outputs?

A semantic layer gives analytics tools a shared way to interpret business data. Instead of asking each agent (or analyst) to reconstruct what "revenue" means, the layer can expose approved metrics, dimensions, relationships, and calculation rules. Think of it as a bridge between technical data structures and the language your teams use to make decisions.

For example, an organization might define net revenue as recognized revenue after discounts and refunds, specify which date field controls the reporting period, and document how currencies get converted. With that in place, the agent can apply the same definition across questions. A semantic layer doesn't magically guarantee good recommendations - but it reduces a common source of inconsistency: different systems answering the same business question in different ways.

The report says 45% of surveyed organizations are adding a semantic layer to their data architecture. That's encouraging, but it's not a one-and-done metadata project. Definitions change. Teams disagree. New products introduce new measures. If nobody owns the definitions or reviews updates, the semantic layer can drift into being an authoritative source of outdated assumptions.

Governance is the other half. In the report, 60% of respondents prioritized strengthening AI and data governance to improve agent insights. In practice, governance should answer questions like:

• Who owns each important metric, and who approves definition changes?
• Which datasets can an agent use, and under what access controls?
• What evidence must accompany an explanation or recommendation?
• Which actions need human approval, and which can be automated?
• How are errors, user corrections, and business outcomes recorded?

The right level of control depends on the cost of getting it wrong. An agent that drafts an internal weekly summary can get lighter review than one that changes prices, approves payments, or sends customer communications. A sensible design makes permissions and approval requirements explicit, rather than assuming users will notice when a task has become high-risk.

There's a trade-off here. More governance can slow experimentation, especially if every new metric triggers a long approval cycle. Too little governance creates conflicting definitions and unclear accountability. A workable approach is to govern high-impact measures and actions tightly, while still allowing experimentation in clearly labeled, lower-risk areas. The goal isn't a perfect catalog of every field. It's a dependable foundation for the questions and actions that matter most.

How can organizations evaluate and control agent insights?

Evaluation should test the full path from question to business outcome - not just whether the model can produce an answer that sounds reasonable. Start with a defined use case and a small set of representative questions. Include routine cases, edge cases, ambiguous requests, and scenarios where the correct response is to ask for clarification or decline to recommend an action.

For each test, record what the agent used: metric definition, data sources, filters, time period, and any assumptions. A reviewer should be able to reproduce the result. If the system can't show evidence or explain which definition it applied, the issue may be traceability, not the wording of the response.

A practical scorecard can cover four dimensions:

1. Data correctness: Are the source records, joins, filters, and calculations right?
2. Context fit: Did the agent apply the organization's approved definitions and relevant constraints?
3. Decision usefulness: Is the output timely, relevant, and specific enough for someone to act on?
4. Outcome and risk: Did the action improve the intended measure without creating unacceptable costs or side effects?

Keep descriptive and prescriptive performance separate. A system can be good at spotting unusual movements while still being unreliable at recommending interventions. If that's the case, deploy it as a monitoring and investigation aid - not as an autonomous decision-maker. That isn't "failure." It's a boundary.

You also need controls for insight volume. If teams receive more recommendations than they can review, adding generation capacity just worsens the operational problem. Rank insights by materiality, confidence, novelty, and the cost of being wrong. Group repeated alerts, suppress known low-value patterns, and set thresholds based on the team's actual capacity. The best threshold isn't the one that produces the most alerts - it's the one that protects attention for signals worth investigating.

Human review shouldn't be an afterthought. Specify who reviews recommendations, what evidence they need, how quickly they must respond, and what happens when they disagree. Record overrides and later outcomes. Over time, those records help you see whether the agent keeps misunderstanding a segment, whether an approved metric needs revision, or whether the business process is blocking useful action.

Finally, measure outcomes with a baseline and - when possible - a comparison. If a recommendation changes a customer offer, compare the result with a suitable control or established baseline rather than assuming the agent "caused" the change. The report's link between insights and business improvement is a useful indicator, but each organization still needs a way to determine whether the agent's contribution is actually causal - and worth the operating cost.

What should a practical implementation plan look like?

Start with a narrow decision, not a company-wide ambition. Pick a recurring workflow where the data is available, the decision owner is known, and the cost of a mistaken recommendation is manageable. Examples include identifying inventory items at risk of stockout, flagging unusual growth in support backlog, or summarizing changes in a sales pipeline for a manager to review.

Define the outcome before selecting an agent. "Provide better insights" isn't measurable. A more useful target could be reducing the time it takes an operations lead to identify stockout risk, while keeping false alerts below a level the team can handle. Include both the benefit and the operational burden. If an agent saves analyst time but creates a new review queue, it's moved work rather than removed it.

Next, make business rules explicit. Document the metric, its owner, the relevant data sources, the refresh schedule, and known limitations. Clarify what the agent is allowed to do. For an initial deployment, it might read approved data and draft recommendations, but not execute any changes. That boundary helps you learn how the system behaves before granting higher-impact permissions.

A staged rollout can look like this:

• Baseline: Record current decision time, error rates, workload, and business outcome.
• Offline evaluation: Test the agent against historical and edge-case questions, including cases where it should not answer confidently.
• Shadow mode: Generate insights without showing them to decision-makers or triggering actions. Compare outputs with actual decisions and outcomes.
• Assisted use: Show evidence and recommendations to a named owner, who approves or rejects them and records why.
• Limited automation: Automate only the specific, reversible actions that meet pre-agreed quality and monitoring thresholds.

At each stage, review failure examples - not just average scores. A 90% success rate can still be unacceptable if the remaining 10% includes expensive or discriminatory outcomes. Conversely, a lower score may be fine for low-impact summaries if users can verify them quickly. Risk depends on the task and the consequences, not just the model's benchmark.

Define a stop condition up front. Pause or roll back if data freshness drops, the agent starts using unapproved definitions, alert volume exceeds review capacity, or outcomes move in the wrong direction. Assign an owner who can make that call. Without clear ownership, teams tend to keep a system running simply because it's already deployed, even after the evidence no longer supports it.

This approach is less dramatic than launching an autonomous analyst across every department. It also gives you better evidence about where automation works, what context is missing, and whether the savings justify the governance and maintenance effort.