Jagged AGI: What Advanced AI Can—and Still Can't—Do
See what Jagged AGI means in practice: where advanced AI performs at expert level, where it breaks down, and how businesses can deploy it more safely.
Jagged AGI: Powerful Across Domains, Uneven in Practice
Advanced AI can now produce work that looks like it came from a specialist: interactive software, research write-ups, creative digital experiences, and fairly believable business plans. That range makes it tempting to ask whether we've reached artificial general intelligence. But impressive output in a few settings doesn't automatically mean a system can judge its own work, handle uncertainty, or take responsibility for a business outcome.
A useful way to describe the gap is Jagged AGI: AI that can reach expert-level performance across many intellectual tasks, while still being unexpectedly weak in other parts of the same job. In reported experiments with a recent Claude model, the strongest results tended to come from digital projects with clear goals and room for iteration. The failures weren't mainly "it made stuff up" or "it can't code." They were more about judgment: too much process, repetitive ideas, work that was technically fine but dull, and problems the model didn't catch on its own.
This article looks at what those experiments suggest - where the model did well, why raw capability is a weak measure of readiness, and how to use more capable AI without confusing a polished demo with dependable autonomy.
What does Jagged AGI mean?
There's no single agreed definition of artificial general intelligence. Some definitions focus on breadth: can a system handle many kinds of intellectual work? Others emphasize human-level performance, adaptability, or the ability to learn new tasks without extensive retraining. The definition matters because a system can look "general" under one test and narrow under another.
One practical definition is an AI that performs about as well as most human experts on most computer-based intellectual tasks. That's narrower than "can do most jobs," because many jobs involve physical action, trust and negotiation, real accountability, and constant interaction with people. Computer work is often only part of the job.
Under that narrower definition, Jagged AGI describes a system that's broadly capable but uneven. It might solve a hard coding problem on the first attempt, then miss that the result is tedious or overcomplicated. It might produce a plausible research design, then fail to keep the paper concise or verify that its analysis was preregistered before looking at results. The key issue isn't just whether the system can complete a task. It's whether the quality holds across the whole job, including planning, evaluation, and correction.
The difference between range and reliability
A model that can write code, analyze data, and generate research designs has range. That part is real - and useful. But range isn't the same thing as reliability. In practice, the question isn't whether it can produce one impressive result. It's whether it can produce acceptable results repeatedly, recognize when things are different, and recover when the task changes.
A useful analogy is a skilled employee who can turn in excellent work across several areas but still needs a manager to step in when the deliverable runs long, the process gets stuck, or a customer-facing decision carries real risk. They're valuable. That doesn't automatically mean they're ready to run the operation on their own.
Why old failure tests are not enough
Early AI failures were often obvious: fabricated facts, broken code, or basic calculation errors. Those failures still matter. But more capable systems can produce work that's "mostly correct" and still fail the assignment in ways that matter.
So the evaluation problem gets harder. You need to assess not only factual accuracy, but also scope, usefulness, taste, efficiency, and whether the system can recognize when its output isn't good enough. A checklist can catch missing citations. It usually won't tell you that a course feels lifeless - or that a project has accumulated so many approval steps that it can no longer move.
What can advanced AI do across different fields?
The strongest evidence for generality comes from a model handling different kinds of work, not from one polished demonstration. In the reported tests, the model was asked to create a short video, turn a small programmable keyboard into a game device, develop a business concept, reconstruct a literature course, conduct empirical research, and build a set of interactive 3D worlds.
Those aren't the same kind of task. They demand different mixes of coding, research, writing, design, and planning. What stood out was how often the model could produce a credible first version with limited direction.
From a prompt to a working artifact
In the hardware experiment, the model was given a small USB keyboard with a few illuminated keys, a joystick, and a knob. With no public instructions available to the tester, it examined the software used to control the device, figured out how to work with the inputs, and created several simple games. The capability wasn't just "writing code." It was taking an unfamiliar interface, making it usable in a real application, and working within the device's constraints.
That same pattern showed up in creative work. Given a broad brief for an impressive, embeddable demonstration, the model produced a video concept with a clear structure and a twist. In another project, it built a set of code-generated 3D environments from a short description. The outputs ranged from visual jokes to scenes inspired by art, literature, and natural landscapes. Importantly, the environments were rendered in code rather than assembled from pre-made assets.
The practical takeaway is that advanced models can compress multiple steps: interpret a brief, research an unfamiliar area, create an artifact, and revise it. That can reduce the cost of trying ideas that used to take a specialist team - or weeks of effort.
Research and business planning
The model also developed a business concept about identifying public-sector equipment opportunities before they went to bid. It screened possible ideas, reviewed local government records, and proposed a service for equipment sales representatives. The plan was plausible enough to deserve consideration, but the evaluator stopped before letting it contact customers or operate independently. Infrastructure needs, sales realities, and whether the eventual service would be good were left untested.
In a separate research project, the model selected a question about filing fees and company formation, gathered public data, and drafted an empirical analysis. The reported results suggested that lower fees were associated with more registrations, without a similar jump in later indicators of business activity. The design looked promising, but the work wasn't independently verified, and the plan hadn't been preregistered before analysis began.
Those caveats aren't small footnotes. They mark the line between putting out something plausible and showing that it's trustworthy. An AI can speed up the first step without automatically completing the second.
Why does capable AI still produce weak work?
The less obvious failures in these experiments often came down to judgment and self-evaluation. A literature course built around a difficult historical syllabus was accurate but initially joyless. When asked to make it more engaging, the model veered into an overly childish design. In the research project, each critique triggered another analysis cycle until the draft ballooned to 182 pages. In the 3D project, the agents repeated motifs and set up an approval system that slowed production dramatically.
These aren't the old "invented a source" failures. They're failures of proportion: doing too much, optimizing the wrong thing, or missing what a human would treat as the actual point of the work.
The model may optimize what it can measure
In the 3D project, the request to polish existing worlds pushed the agents to compare new versions against a set of screenshots. They then optimized for those specific images. That can improve a narrow score while making the overall creative experience worse. It's the familiar systems problem: once a proxy becomes the target, the output can drift away from the real objective.
The agents eventually built a large rulebook and added multiple review stages. The idea was to protect quality, but the process grew without a corresponding way to remove rules or check whether the process itself still made sense. The result was an AI-driven bureaucracy: more formal oversight, slower output, and a lot of effort spent managing the machinery.
For businesses, that's the warning: adding evaluators doesn't guarantee better judgment. If every evaluation measures a narrow property, the system may get better at passing tests rather than delivering value.
Critique is not the same as taste
The projects used critics, fact-checkers, and review agents. Those tools caught factual issues and suggested fixes. They didn't reliably notice when a website felt dull, a paper had become bloated, or a creative process had tied itself into knots. A quality check can be thorough and still miss the most important question: Is this useful and good for the intended audience?
Some of that gap is subjective. A manager asking for "more polished" can be too vague to steer a team reliably. Aesthetic preferences also differ. Still, uncertainty about whether the brief is wrong, the system is failing, or the reviewer is missing something is itself an operational problem. If you can't explain why the result missed, you can't confidently decide what to change next time.
The strongest limitation may be the model's uneven ability to notice when its own work is only "adequate." Experts don't just generate answers; they recognize when something is incomplete, disproportionate, or the wrong fit for the situation. That monitoring function is still a weak point.
What does Jagged AGI mean for business and research?
For businesses, the immediate opportunity isn't replacing a whole role with a model. It's cutting the cost of work that's digital, bounded, and reviewable. A model can investigate a market, draft a prototype, organize public information, or produce a first-pass analysis. Then a human can decide whether the work is valid, whether it fits the product, and whether it's safe to act on.
This division of labor is more realistic than asking an AI to "run the business" with minimal supervision. Running a business involves far more than producing a plan. It requires customer discovery, delivery, pricing, cash management, legal compliance, handling exceptions, and earning trust over time. A model can contribute across those areas without being dependable enough to own the outcome.
Where the leverage is strongest
AI is most useful when the task has three traits:
- A concrete output: a prototype, report, research draft, or data set you can inspect.
- A bounded environment: the system works with known tools and sources rather than operating freely across external systems.
- A practical review path: a human can verify key claims or test the result before it reaches customers.
That's why a model can be a strong partner for exploring a product concept or drafting something first, but a poor choice for unmonitored customer outreach. The second case introduces reputational and financial risk that's hard to capture in a prompt.
Research faces a verification bottleneck
AI can generate analysis and writing faster than most teams can review it. That creates a specific bottleneck for research: the scarce resource shifts from producing a draft to validating the claims, data, and methods behind it. A paper can sound rigorous while still having timing problems in its design, unverified results, or language that makes it unclear what was actually done.
That doesn't make AI-assisted research pointless. It changes the cost structure. When systems can generate plausible work cheaply, it becomes more important to preserve provenance, reproducibility, and human ownership of the conclusions. A team should be able to trace where the data came from, which analyses were planned before results were seen, and what changed after analysis began.
For business and research alike, the real question isn't "Can AI do this?" It's: "What evidence would justify trusting this output, and who is responsible if that evidence turns out to be wrong?"
How should teams work with Jagged AI?
Treat an advanced model like a capable project partner whose strengths and blind spots vary by task. That means structuring the work so the system can move quickly while humans stay in control of consequential decisions. The goal isn't adding people to every keystroke. It's putting review where errors hurt most - and where the model's judgment is least reliable.
A practical deployment sequence
- Choose a bounded task. Start with a deliverable you can inspect - such as a prototype, a draft analysis, or a structured summary. Don't begin with an open-ended mandate to run a business or contact customers.
- Define the outcome, not just the activity. "Review these records" tells you what to do. "Produce a dated list of potential projects, with source evidence and confidence levels" describes something you can check.
- Set limits before you grant autonomy. Specify what the model can read, change, publish, spend, or send. Keep external actions behind human approval until the system proves reliable under realistic conditions.
- Evaluate the whole deliverable. Check accuracy, but also usefulness, length, repetition, tone, cost, and time to completion. A technically correct result can still frustrate the user or create an operational burden.
- Test whether the process can recover. Add edge cases, conflicting information, and ambiguous requests. Watch whether the system asks clarifying questions, flags uncertainty, and revises appropriately - rather than confidently running with a bad assumption.
- Keep human accountability explicit. Assign a named person to approve consequential outputs. AI agents can perform checks, but they don't carry responsibility for business, legal, or research decisions.
Watch for process growth
Long-running projects can accumulate agents, rules, and approval steps. Some structure helps. Unchecked process growth doesn't. Track whether each new review step catches a meaningful failure mode, and remove steps that add delay without improving outcomes. A system shouldn't need more oversight than the value of the work it produces.
Also set a stopping rule. For example: after two rounds of revision, a human decides whether to continue, narrow the task, or abandon the draft. Without that, a model can keep responding to criticism with more analysis and more text - without ever clearly saying whether the result is ready.
Keep feedback specific and consequential
In the reported projects, human feedback made a real difference. The best feedback isn't just "try again." It explains what went wrong in a way the next iteration can use: the audience is expert, the page feels repetitive, the analysis doesn't establish causality, or the review process is taking longer than the work itself.
This feedback loop is where the leverage is. A model can make changes in minutes, but someone still needs to notice which change actually matters. The more open-ended the work, the more valuable that human steering becomes.
Conclusion
Jagged AGI is a useful lens because it avoids two unhelpful extremes: dismissing advanced AI as a novelty, and treating broad capability as proof of dependable autonomy. These systems can already compress a lot of research, coding, and creative work into a short time. They can also miss the issues that make a result truly useful.
For now, the safest approach is to give AI bounded, inspectable work and let human judgment govern quality, risk, and consequences. Measure the whole system - including review overhead and process drift - not just how fast the first draft appears. The model may be able to do more than your team expected. But deployment still has to account for what it doesn't notice.