From manuscript to evidence-backed report.
RefCheck verifies each scientific claim against the publication it cites. Every finding is supported by text from the source.
- 01
Claim identification
Each claim is identified and linked to the reference it cites.
- 02
Source retrieval
The cited publication is retrieved from authorized and open sources.
- 03
Evidence extraction
A language model specialized in extraction selects the relevant source passages and confirms each one verbatim.
- 04
Multi-layer LLM assessment
Multiple layers of language models assess each claim against the evidence, several times and independently.
- 05
Error classification
Unsupported claims receive a named error type.
- 06
Report
Claim and source side by side, highlighted discrepancy, explanation and editorial guidance. Unsupported claims and reference problems are shown first. Claims that could not be confirmed appear in a separate block.
Two new layers, developed with publishers
Both layers extend the current architecture and are implemented in collaboration with publishers.
Internal-signal filter
Sparse autoencoders and linear probes read internal signals of language models to identify clearly supported claims before the LLM assessment. In development experiments, this reduced LLM judge evaluations by 17%. Fewer judge evaluations lower the cost of each manuscript and make verification at scale more efficient.
Agentic verification
A multi-agent system for complex and unresolved cases. An orchestrator agent decides what each case needs and assigns the work to specialized agents that operate independently: one retrieves additional evidence, another extracts new passages, and others assess the claim again. The orchestrator reviews their results and stops when the case is resolved. Deeper analysis is reserved for the claims that need it.
Results
false positives on supported claims
of errors detected when the cited source was available
Development evaluation results; not yet measured on the held-out Gold TEST.
What the report shows
The source evidence contradicts the claim or does not report what the claim states.
Only the abstract was available, and it does not support the claim. The full-text paper must be checked.
The full text was read, but the repeated assessments did not reach a consistent verdict.
The cited source could not be retrieved, or the reference could not be identified.
The reference has a broken DOI or incomplete bibliographic data.
The report also shows how many claims were supported, and highlights them in the manuscript view.
Eight types of error
These are the subtle errors that AI-generated text produces when it cites real publications. Each unsupported claim in the report receives one of these types.
The source says the opposite
ManuscriptMetformin increased tumour growth in mice.
SourceMetformin reduced tumour growth in mice.
The study did not do that
ManuscriptA randomized trial showed that statins reduce dementia risk.
SourceA retrospective cohort study of statin users.
Data does not match the study
ManuscriptThe trial included 1,200 patients.
SourceThe trial included 120 patients.
Inflated figure
ManuscriptThe drug reduced relapse threefold.
SourceThe drug reduced relapse by 30%.
Figure taken out of context
ManuscriptThe response rate was 80%.
Source80% in patients with the EGFR mutation, 22% overall.
One measure confused with another
ManuscriptThe intervention reduced mortality by 25%.
SourceThe intervention reduced hospital admissions by 25%.
The source does not say that
ManuscriptVitamin D supplementation prevents depression.
SourceReports effects of vitamin D on bone density only.
Overstated certainty
ManuscriptAir pollution causes type 2 diabetes.
SourceAir pollution was associated with type 2 diabetes.
Examples are illustrative.
Material errors
RefCheck focuses on errors that change the scientific meaning of a claim. Differences in wording that preserve the meaning of the source are not reported as errors.
Scope and access to sources
RefCheck is designed for life sciences and biomedicine.
When a cited paper is behind a paywall, RefCheck works with the abstract. If the abstract does not support the claim, the claim is marked Unverified (full text needed) and the editor is asked to check the full-text paper. This depends on access to the full text, not on RefCheck. For this reason, we are seeking agreements with publishers to verify claims against the complete source.