AI Due Diligence: Lessons From a Theranos Whistleblower
Learn why Tyler Shultz says AI claims can be hard to verify, what investors should test, and how overreaching NDAs may deter employees from speaking up.
AI Due Diligence: What Investors and Labs Should Learn From Theranos
A slick AI demo can make a product look far more capable than the system behind it. That mismatch matters when investors are deciding where to put money, when customers are weighing a vendor, and when employees are trying to raise alarms if the marketing doesn't match reality.
Tyler Shultz saw what happens when scrutiny doesn't keep up. As a scientist at Theranos, he raised concerns about the blood-testing company before helping a Wall Street Journal reporter expose its fraud. In a 2026 interview on The San Francisco Standard's PST podcast, Shultz said AI is hard to vet, investors shouldn't rely on a founder's word, and restrictive agreements can make it harder for employees to report problems.
His point isn't that every AI company is headed for Theranos-style fraud. It's that fast-moving markets, impressive demos, and limited visibility into the technical details create the same conditions for over-trust. This article lays out how to assess AI claims, where NDAs can raise risks, and what investors and teams can do before things turn into a public incident.
Why is AI difficult for investors to verify?
AI products are unusually easy to present in ways that blur where the result actually comes from. A demo might use a third-party model, an operator running things behind the scenes, cherry-picked inputs, or a narrow workflow that doesn't reflect normal use. A strong output is proof that something worked in that moment - but it isn't proof that the product is repeatable or that the company has a durable technical advantage.
Shultz described the challenge as separating something genuinely novel from what he called a "ChatGPT wrapper." That line isn't always clean. Using an existing model can still be valuable if the company solves a real customer problem through reliable workflows, useful data, integrations, security, or support. The real question is simpler: does the company describe what it built accurately - and do customers get enough value to pay for it?
A demo is not proof of a business
A live demonstration can be built around a narrow set of conditions. It may not show error rates, performance on unfamiliar inputs, latency, operating costs, or how often a human needs to step in. Those details are what determine whether a product can be used safely and profitably once it leaves the demo environment.
For instance, an AI tool that summarizes support calls might look great on clean recordings. Due diligence should also test background noise, multiple speakers, accents, missing context, and what happens when the summary is wrong. A polished sample output doesn't tell you much about how the system behaves across the full mess of real customer data.
The claims investors need to separate
When a company makes an AI claim, it helps to split it into three pieces:
- Capability: What does the system do, and under what conditions?
- Repeatability: Can it deliver acceptable results across realistic inputs and over time?
- Economics: Can the company deliver that performance at a price customers will accept?
This prevents a common failure mode: treating technical possibility as commercial readiness. A prototype can impress without being reliable. A reliable product can still have weak unit economics. And a business with strong economics might have little technical differentiation, which changes what investors should demand to judge long-term defensibility.
How should investors conduct AI due diligence?
Shultz's basic advice is straightforward: don't treat a short meeting or a confident founder as enough. He argued that the days of a 30-minute coffee followed by $100,000 checks should be over. That lesson applies well beyond early-stage angel investing - the level of diligence should track the size of the claim, the size of the investment, and what happens if the claim turns out to be wrong.
A good process stress-tests the company's story from multiple angles. You don't need to replicate every research effort, but you do need to understand what the system does, what it depends on, and what evidence supports the business case. Ask for repeatable demonstrations, access to technical reviewers who can dig in, and references from people using the product in real operating conditions - not just pilots and promises.
A practical diligence sequence
- Write down the claims. Get specific about accuracy, speed, autonomy, proprietary data, customer adoption, and costs. Vague language is hard to verify, so ask the company to define its terms.
- Ask for the test conditions. Find out what data was used, how the evaluation set was chosen, what was excluded, and whether results were independently checked. A metric without a measurement method isn't a meaningful comparison point.
- Test realistic edge cases. Ask what happens with incomplete, ambiguous, adversarial inputs - or anything outside the intended use. In high-impact settings, look closely at human review and escalation.
- Trace dependencies. Identify what relies on third-party models, cloud providers, external datasets, or manual work. These aren't automatically "bad," but they affect margins, resilience, and control.
- Check customer evidence. Talk to customers about deployment time, ongoing human effort, reliability, renewal decisions, and measurable value. A pilot or letter of intent isn't the same as repeated paid usage.
- Pressure-test the economics. Estimate inference, infrastructure, support, and evaluation costs at realistic usage levels. A system that's cheap in a small trial can get expensive fast as adoption grows.
Look for consistency, not a perfect model
No AI system is error-free. What matters is the threshold for the intended use. A drafting assistant can handle a different error profile than a tool used to prioritize medical cases or approve financial decisions. What you want to see is whether the company understands its limitations, measures them, and has controls proportionate to the harm a mistake could cause.
Good diligence also checks whether leadership, engineering, and customers tell the same story. If they don't, dig into the gap before calling it "just communication." It may point to an immature product, weak internal controls, or a business plan built on capabilities that haven't actually been demonstrated.
Can AI company NDAs stop employees from speaking up?
Shultz warned that some technology workers face NDAs and non-disparagement agreements he described as excessively broad and "likely illegal." That's his assessment, not a blanket ruling about every contract used by every AI company. Enforceability depends on the wording, the facts, and the applicable law.
Even when a contract can't lawfully block protected reporting, the real-world concern is still there. Employees may not know what rights they have. They may worry about expensive legal fights. Or they may fear that contacting a regulator will hurt their career. Shultz said that at Theranos he signed NDAs that left him believing he couldn't talk to a lawyer or a regulator. He ultimately contacted regulators using a fake name and email address.
That experience points to a gap between formal rights and what workers think they're allowed to do. If someone believes every conversation is off-limits, a theoretical legal safeguard may not be enough to get a safety issue into the hands of someone who can act on it.
What employees should do when they see a serious issue
If you work at an AI company and think a product, safety process, or public claim is misleading, keep your response deliberate:
- Preserve relevant facts lawfully. Record dates, decisions, and the basis for your concern. Don't take confidential data or trade secrets out of the company without qualified legal advice.
- Get independent legal guidance. Ask a lawyer who understands employment and whistleblower issues what your agreement permits and what protections may apply. Some lawyers take whistleblower matters on contingency, but eligibility and terms vary.
- Use appropriate reporting channels. Depending on the situation, that could mean internal compliance, a regulator, or another legally protected route. Don't assume a manager is the only option.
- Avoid relying on a manager's interpretation of your contract. The employer may have a legitimate interest in protecting confidential information, but that doesn't answer what you can legally disclose to a regulator or a lawyer.
This is not legal advice. Rules vary by jurisdiction, including between US states and between the US and the UK. Employees should get advice tailored to their situation instead of relying on a general article or a company summary of its own policy.
What employers should make explicit
Companies should explain, in plain language, how confidentiality obligations interact with legally protected reports, access to counsel, and regulator communications. The legal documents should protect real confidential information without implying that employees are barred from reporting unlawful conduct or safety concerns when the law allows it.
Clarity is part of risk management. If employees can't tell whether they can raise a concern, the company may learn about problems later than it otherwise would. A documented reporting path, clear expectations about anti-retaliation, and independent escalation options make it easier to surface issues while they're still fixable.
What can AI companies learn from Theranos?
The Theranos comparison has to be handled carefully. The cases aren't interchangeable, and ambitious AI claims don't automatically mean fraud. Still, the story offers a clear lesson about how authority can pile up when a company's claims get less scrutiny than its reputation, fundraising momentum, or public profile.
Shultz said Theranos investors were victims on paper, but he also argued that inadequate diligence gave Elizabeth Holmes power she shouldn't have had. That distinction matters. Investors can be misled, but they still have a responsibility to test extraordinary claims - especially when their decisions affect employees, customers, and later investors.
The same logic applies to AI. Technical complexity can become a convenient excuse for accepting claims that are hard to evaluate. Complexity should lead to better questions and qualified review, not automatic deference. If a company can't explain what it measured, what's still uncertain, and where human work is involved, calling the system "proprietary" doesn't make it more credible.
Build an organization where bad news can travel
For AI leaders, internal reporting is part of product quality. Teams need a route to flag evaluation failures, unexpected model behavior, misleading sales claims, or safety issues without being treated as disloyal. That doesn't mean every disagreement is a whistleblower case. It does mean concerns should be assessed based on evidence, with clear ownership and a record of how they were resolved.
A workable internal system includes:
- Named owners for product safety and evaluation issues.
- A documented path for escalating concerns beyond the immediate manager.
- A process for recording incidents, decisions, and corrective actions.
- Clear boundaries between protecting confidential information and restricting legally protected reporting.
- Follow-up that explains what happened to a concern, when appropriate.
These mechanisms work best before a crisis. Once external scrutiny arrives, employees will naturally question whether internal channels are independent - and whether they're safe to use.
Keep claims, controls, and incentives aligned
A company's risk goes up when public promises get ahead of the evaluation process - or when sales incentives reward claims engineering can't actually support. Leaders should review customer-facing statements alongside evidence and make sure the people doing testing can raise objections before launch.
Investors can reinforce that discipline too. Ask how the company handles failures, not just how it reports successes. Probe what changes after an evaluation fails, whether customers are told about material limitations, and who has the power to pause a launch. The answers usually reveal more about governance than a polished "safety" statement ever will.
Conclusion
Shultz's warning isn't really about predicting another Theranos. It's about refusing to let speed and technical complexity replace scrutiny. AI claims should be tested against real-world conditions, business economics, and the people who will actually use the product. Investors should calibrate diligence to the stakes. Companies should make it safe and clear for employees to raise concerns.
The standard is simple: ask what has been demonstrated, how it was measured, what could fail, and who can speak up if it does. If a company can answer those questions with evidence and real processes, that's more meaningful than confidence in a pitch or a contract.