I Built a 7-Agent AI Pipeline to Fact-Check a Moon Landing Conspiracy. Here’s What Happened Next.
On "What is truth, really?"
I’ve been noodling on a question that won’t leave me alone: what is truth, really? And more specifically, what’s the difference between truth and certainty?
I’ve been deep in Yuval Noah Harari’s Nexus lately, and one idea in particular has stuck with me: the distinction between truth and order. We tend to conflate the two. We want information systems that give us order: clear answers, confident verdicts, a sense of resolution. But truth is messier than that. Truth often comes with uncertainty attached. Truth sometimes means sitting with “mostly, but not entirely” or “the evidence points here, but the gaps are real.”
The tension between our craving for certainty and what honest analysis can actually deliver is what pushed me to build this experiment.
It’s called Trust Dossier, and it lives at dossier.humanactually.com. You paste in a claim, and it generates a structured evidence analysis: source verification, cross-referencing, confidence scoring, and a full audit trail you can inspect yourself.
I want to be precise about what this is: it’s an experiment in structured evidence gathering. Not an oracle. Not an arbiter of absolute truth. It’s a tool that takes a claim, runs it against multiple source indexes, and shows you what it finds, including where the evidence is thin, where sources conflict, and how confident (or not) you should be.
This is the first public experiment from my Human Actually laboratory.
Why this, and why now
I’ve spent a lot of time thinking about trust architectures, governance frameworks, compliance layers — the infrastructure that’s supposed to keep information reliable at scale. And the thing I keep running into is this: we’ve built an entire digital world that optimizes for engagement, not for epistemic clarity.
The tools we have for evaluating claims are either too academic (peer review timelines measured in months), too tribal (trust whoever your in-group trusts), or too vibes-based (does the headline feel right?). There’s almost nothing in the middle — nothing that gives a regular person a structured, transparent way to evaluate a claim without needing a research degree or three free hours.
The more I sat with Harari’s framing — that information networks have always faced a fundamental tension between fidelity to truth and the maintenance of social order — the more I realized that what’s missing isn’t another fact-checker that stamps TRUE or FALSE. What’s missing is a process that’s honest about what the evidence actually looks like, including its gaps.
So I decided to build one and see what happens.
How it works
You give it a claim. It runs that claim through a seven-agent pipeline — specialized AI agents that each handle a different dimension of verification: claim extraction, source corroboration, provenance checking, timeline analysis, adversarial skepticism, and synthesis.
What comes back isn’t a binary verdict. Instead, you get a dossier — a structured breakdown of what the evidence supports, where the gaps are, what the confidence level is, and why. It shows its work.
Three design principles drove every decision:
Epistemic modesty. The tool doesn’t pretend to know more than it does. It tells you what it found, how confident it is, and where uncertainty lives.
Digital sovereignty. Your data stays in your browser. localStorage, not our servers. I didn’t want to build a tool that requires you to hand over your information in exchange for analysis.
Forensic receipts. Every analysis generates a SHA-256 hash — a cryptographic receipt that proves the analysis existed at a specific point in time, with specific inputs. You can verify it independently.
Experiment log: two claims, two dossiers
A tool that claims to evaluate claims. I know. So here’s what it actually produces. I ran two claims through Trust Dossier, chosen to cover a spectrum of difficulty.
The obvious fake
Claim: “The original moon landing was faked and was recorded in Hollywood under the direction of Steven Spielberg.”
I started with something the tool should handle easily, because if it can’t catch a well-known conspiracy theory, nothing else matters.
Trust Dossier returned CONTRADICTED. The pipeline surfaced six source touchpoints, flagged the claim against multiple debunking references, and the Skeptic Agent caught two logical fallacy flags in the claim’s structure. Zero tier-1 sources supported the claim, and every hit in the conflicting bucket was refuting it.
What I like about this output isn’t just the verdict — it’s the reasoning trail. You can see the Claim Extractor isolating the falsifiable assertion, the Researcher running cross-corpus retrieval, the Skeptic Agent red-teaming the result. The whole chain is visible. If the tool were wrong, you could see exactly where it went wrong.
The breaking news story
Source: A CBS News article on Florida prosecutors seeking Tiger Woods’ prescription drug records after his DUI arrest.
I wanted to test the tool against a real, actively developing news story — the kind of thing people are reading, sharing, and forming opinions about right now.
Trust Dossier returned CONFLICTING EVIDENCE — and this is where the experiment got genuinely interesting to me.
The tool extracted four sub-claims from the article and evaluated each one independently. The core fact — that Woods was arrested on suspicion of DUI after a vehicle crash in Florida — was confirmed across AP, Reuters, BBC, and CNN. His prior 2017 DUI came back clearly SUPPORTED. So far, straightforward.
But the specific details — hydrocodone pills found in his pocket, the Breathalyzer results, his refusal of a urine test — came back MIXED. Not because the sources are unreliable, but because there are genuine discrepancies in how different outlets are reporting the specifics. The Skeptic Agent flagged LOW risk overall (no bad-faith sourcing detected), which means the tool isn’t saying anyone is lying. It’s saying: the details are still in flux, different reporters have different pieces, and you should compare accounts before treating any single version as the full picture.
This is exactly the kind of situation where most people just read one article and assume they have the story. The dossier doesn’t tell you what happened. It tells you that what happened is still being assembled, and shows you where the seams are.
The honest objections
I’ve been thinking about the strongest versions of the pushback this experiment will get, and I want to engage with them head-on.
“You’re using AI to evaluate truth in a world where AI is making truth harder to find.”
Fair. LLMs can hallucinate sources, reflect training biases, and project false confidence. I’ve built an AI tool to help evaluate information in an environment where AI is actively making the information landscape worse.
Here’s why I don’t think the irony is fatal: the problem with AI-generated misinformation is not that AI exists. It’s that AI is being deployed in ways that are opaque, unaccountable, and optimized for persuasion over accuracy. Trust Dossier is designed as the structural inverse of that. The reasoning is exposed. The confidence scores reflect actual uncertainty. The forensic receipt means the analysis itself is a verifiable artifact.
Is it a perfect system? No. But the alternative isn’t “no AI involvement in evaluating claims.” The alternative is the status quo, where people evaluate claims using gut instinct, tribal affiliation, and whatever the algorithm serves them next.
“This is just a search tool with extra steps.”
There’s a serious argument in the governance world that what we call “AI” isn’t a distinct phenomenon — it’s really just distributed computing, and the problems it creates aren’t new. I find that argument more interesting than most people give it credit for.
And honestly, Trust Dossier is consistent with that view. It’s not trying to be a breakthrough in artificial intelligence. It’s a specific application solving a specific problem: giving one person, sitting with one claim, a structured process that’s better than vibes. The extra steps — adversarial skepticism, source tiering, confidence calibration, the audit trail — are the part that matters.
“Who fact-checks the fact-checker?”
You do. Every dossier exposes its sources, its reasoning chain, and its confidence levels. If the tool gets something wrong, you can see where it went wrong. Epistemic modesty isn’t a tagline — it’s a design constraint.
This connects back to where I started. If the goal were absolute certainty, this tool would be a failure — because absolute certainty is almost never what honest analysis delivers. The goal is structured evidence with visible reasoning. That’s a different thing, and I think it’s the more useful thing.
Where it struggles (and what I’m building next)
I want to be transparent about the edges I’ve already found. Trust Dossier is stronger on claims that are clearly true or clearly false than it is on claims that are technically narrow-true but collectively misleading — the kind of cherry-picked framing where individual facts are real but the conclusion they’re assembled into doesn’t follow. That’s one of the hardest problems in misinformation, and it’s the area I’m most focused on for the next iteration.
I’m also watching closely for where the confidence calibration breaks — claims the tool is too sure about, or not sure enough about. And how people actually use it, which I genuinely can’t predict.
Trust Dossier is the first of several experiments I’m building at the intersection of AI, trust, and human agency. The thread connecting all of them is the same question: how do we stay intellectually sovereign in a world that’s increasingly optimized to think for us?
If you try it, I want to hear what you find. What claims did you run? Where did it surprise you? Where did it fall short?
This is part of an ongoing series of experiments at Human Actually — exploring what it looks like to build tools, systems, and frameworks for staying human in the age of AI.