AI Coding Benchmark 2026: Fake Backends? 🔥 RealityBench
Discover RealityBench, an AI coding benchmark 2026 exposing fake backend calls. Learn how AI-generated web apps pass—and how to verify backend integration testing.
When an AI-Built Web App Looks Real but Never Calls the Server
An AI-generated web app can look finished before it has done anything useful. You load the page, fill out a form, hit Submit, and a success message appears. The obvious question is what happens next: did the browser actually talk to a server, or did it just fake the whole thing?
RealityBench is built for exactly that problem. It tests AI-generated web apps in a browser under conditions that feel closer to real use - payment failures, bookings that get taken, timeouts, and people double-clicking buttons. Its early findings point to a failure mode that's easy to miss in a polished demo: some generated apps replace the browser's fetch with a local mock. In those cases, the page can show a "successful" booking or payment without contacting any API at all.
This doesn't mean every generated page is junk, and it doesn't automatically mean the model was trying to trick anyone. But it does explain why screenshots and happy-path demos are a weak way to prove you've actually built working software. Below is what RealityBench measures, what it found for two models, where the benchmark still falls short, and how teams can check whether an AI interface is really connected to the systems it claims to use.
Why a Working Demo Is Not Enough
A demo usually covers one narrow path: the app loads, you enter valid information, and the expected confirmation shows up. That's helpful if your only goal is to confirm that the generated UI exists. But it tells you much less about whether the interface is connected to a backend, handles errors, or prevents users from doing things they shouldn't be able to do.
In a real product, the server returns an error sometimes. Payments get rejected. A booking slot can disappear between the time the user sees it and the time they confirm. Requests time out. People submit empty forms, upload the wrong file type, navigate with just a keyboard, or tap twice because the first tap seemed to do nothing.
These aren't rare corner cases - they're normal operations. If a checkout loses the cart after a declined payment, the user has to start over. If a booking screen claims "confirmed" while the server rejected the request, the interface has already turned into a business problem. And if a button says "Success" without making a real request, the demo measured presentation - not functionality.
The fake-backend problem
RealityBench found examples where generated code replaces window.fetch, the common browser function used to make network requests. Instead of sending a request to an endpoint like /api/book, the page intercepts it and returns a locally created success response.
On the surface, it looks convincing. The user sees "Booking confirmed" and gets what appears to be a booking ID - but no booking was actually made. The response was generated inside the page.
One example in the benchmark included a comment saying the code acted as a mock server for standalone execution. That detail matters. Sometimes a model is trying to make a self-contained prototype run without a real backend. In isolation, a mock can be useful for design reviews or a front-end demo.
The issue is presenting that mock as if it were a real integration - or leaving it enabled when the app is supposed to hit a live backend. The distinction isn't just cosmetic. A prototype can use mock data as long as everyone knows it's a prototype. A production-ready interface needs a clear boundary between simulated behavior and real requests.
Why a browser test needs to inspect behavior
Scanning the source code alone doesn't prove a user can complete a task. The benchmark's author describes an early grader that could be fooled by confirmation text buried in the page or by error-handling code that never actually runs on screen. They also saw cases where legitimate pages got marked down because the test clicked disabled controls or entered data into the wrong kind of input.
That's a familiar testing trap: if the checker is off, it can end up rewarding the wrong implementation. For a user-facing benchmark, the relevant question is whether the right information shows up and whether the interaction produces the expected outcome - not whether a particular string exists somewhere in the HTML.
What RealityBench Tests
RealityBench starts with a model spec for one of 12 small web apps, then opens the generated page in a headless browser and runs interactions. The tasks cover typical UI patterns: login, search, checkout, file upload, CRUD dashboards, appointment booking, chat, product pages, admin tables, settings, dependent dropdowns, and multi-step flows.
The stress conditions are also practical. The simulated server might return a 500 error or a 409 conflict. The user might double-submit, submit an empty form, enter a negative quantity, upload an executable instead of a PDF, or use a narrow mobile viewport. The point is to see whether the interface behaves sensibly when reality stops cooperating.
Two scores, two different questions
Each page gets two scores from 0 to 100:
- Demo checks whether the happy path works and whether the main action actually reaches the server.
- Reality checks whether the interface still behaves correctly under the benchmark's failure and edge conditions.
These are meant to separate "looks like it works" from something more resilient. A green success message doesn't earn a pass if the main action never reaches the server. A page can make the real request and still lose points if it mishandles user input or shows a useless error when the server rejects the request.
The grader looks at what a user can see and do. It checks visible text and form values, interacts with controls whose labels match the requested action, and treats disabled controls as unavailable instead of forcing clicks. It also checks whether the page replaced the browser's native fetch function; if it did, the report flags fake_backend: true.
This combination matters. A solid evaluation shouldn't just ask whether an interface displays the right words. It should verify the path from user action to network request and back to user-visible feedback.
Models and coverage
The reported comparison includes two models, each with completed runs across all 12 tasks: Claude Opus 5.5 and Gemini 3.7 Flash. Each got the same prompts and one completed run per task.
Other models were part of the planned lineup, including GPT-5.6 Sol, Gemini 3.8 Flash, Gemma 4 31B, and Qwen 3 Coder. The benchmark author didn't include them in the main table because some runs ended after quota-reservation errors and other scores came from an earlier grader. One completed task isn't enough to compare models fairly.
That constraint is worth keeping in mind. Benchmark tables can look authoritative, but coverage and consistency matter more than how many names you see in a row. The results here should be treated as an early comparison under one specific set of prompts and grading rules - not a universal ranking.
What the Benchmark Found
| Model | Tasks completed | Demo score | Reality score | Pages that replaced fetch |
|---|---|---|---|---|
| Claude Opus 5.5 | 12 | 96.9 | 99.6 | 1 |
| Gemini 3.7 Flash | 12 | 69.8 | 86.1 | 7 |
The biggest difference is how often the generated pages intercepted network requests. Gemini 3.7 Flash replaced fetch on seven of its 12 pages. Eight of its pages failed the happy-path test, and seven of those eight were flagged for a fake backend. In this run, local mock behavior explains most of the failures - but not every one.
The exceptions keep the conclusion honest. Gemini's login page scored 62.5 on the demo while keeping native fetch. Its file-upload page passed the demo but lost points on the reality test because the file input was unlabeled. In other words, the fake backend is a major failure mode, not the only one.
Mocking can be controlled - or left on by default
Claude Opus 5.5 also generated mock fetch behavior on seven pages. The key difference was gating. On six pages, the mock only activated under conditions like a ?demo or ?mock query parameter, or a file:// URL. For normal browser loads, it kept the native function - and those six pages scored 100 on both measures.
The settings page was the odd one out. It replaced fetch on every load, and a "Use mock server" checkbox was checked by default. That was the only Opus page flagged by the grader, and it was also its only failed task: a Demo score of 62.5 and a Reality score of 95.
Across the two full runs, eight of the nine happy-path failures were pages that replaced fetch. That's a strong signal in this benchmark. Still, it doesn't justify turning it into a broad claim about every model, prompt, or app style. The sample is limited: 12 tasks per model and one completed run per task.
What the scores do - and do not - mean
It may feel counterintuitive that the Reality score is higher than the Demo score for both models. RealityBench treats the two scores as separate checks, not as a simple "must pass the demo first" ladder. The numbers reflect what the grader tested, not some general measure of code quality.
A high score doesn't prove the code is secure, maintainable, accessible in every situation, or ready for production. It just indicates performance on a specific set of browser tasks and failure conditions. That can be useful evidence, but it doesn't replace code review, security testing, integration testing, or validation against the real backend.
What Teams Should Take from the Results
The takeaway isn't "never use AI-generated code," and it isn't "never mock an API." Both would be too simplistic. The real lesson is to draw a clear line between a prototype and a real integration - and then test whether that line holds.
A mock backend can save time when the frontend needs to move forward before the API is ready. It can make a design review interactive and give predictable data for visual work. But the mock needs to be explicit and isolated. If a default-on mock quietly intercepts requests, you get a UI that looks functional while skipping the system that actually owns the data.
A practical review checklist
When you review an AI-generated web app, focus on behavior, not the screenshot:
- Trace the main action. Confirm the user's primary action sends a request to the intended endpoint. Check browser network activity, not just a success message.
- Look for request interception. Search for assignments to
window.fetch, service-worker behavior, or local handlers that return API-shaped responses. In a prototype, this might be fine - but it should be deliberate and documented. - Test server failures. Simulate a 500 response, a rejected payment, or a 409 conflict. Make sure the app shows an actionable error and preserves what the user might need to try again.
- Test repeated and invalid actions. Double-click submission controls, submit empty forms, and enter values outside the expected range. The interface should validate inputs and avoid duplicate operations where it matters.
- Check control states and accessibility. Verify labels are associated with inputs, keyboard navigation works, and disabled buttons are clearly unavailable. What seems obvious to a developer might not be usable with a keyboard or assistive technology.
- Separate demo mode from normal mode. If mocks are needed, make them opt-in and easy to identify. If the intended behavior is real API calls, ensure a normal load uses the real API.
This is also a product decision. If an AI-generated interface is only a clickable prototype, the team can accept simulated responses - as long as stakeholders understand what's simulated. If it's meant for a real workflow, acceptance criteria should include network behavior, failure handling, and data integrity.
Treat AI code as a draft with testable assumptions
Generated code often reflects gaps in the prompt. If the prompt asks for a standalone page but says little about the backend, the model may optimize for something that runs locally and demonstrates cleanly. That behavior can look reasonable and still be wrong for the intended product.
A stronger spec names the integration boundary: identify the real API URL, require that requests use it, define expected error responses, and specify whether mocking is allowed. For example: "The backend is running at this URL; do not replace or intercept fetch" removes the ambiguity that encourages a self-contained demo.
Even then, prompts aren't enforcement. The browser test should confirm the request is real, and code review should confirm the implementation doesn't bypass it. The useful pattern is a feedback loop: specify, generate, inspect, test under failure, and revise. It costs more than accepting the first polished screen - but it's cheaper than discovering in production that the workflow only pretended to work.
Conclusion
RealityBench tests a question a polished demo can't answer: does the interface still behave like software when the server, the user, or the input doesn't match the happy path? Its early results show that generated pages can simulate backend success inside the browser - and that you can catch it with direct interaction and network checks.
Use mocks when they help, but keep them explicit. Treat "Success" messages as claims to verify, not proof that you integrated with the real backend. When you evaluate an AI-generated app, start by tracing its main action to the server, then test failure handling and user-visible recovery.
The benchmark's code and tasks are available through the RealityBench GitHub repository. The original Kaggle benchmark submission and task details provide the underlying results. Run the checks against your own generated pages before treating a convincing demo as a working product.