← Back to RefCheck
How RefCheck Verifies Claims

From manuscript to evidence-backed report.

RefCheck verifies each scientific claim against the publication it cites. Every finding is supported by text from the source.

  1. 01

    Claim identification

    Each claim is identified and linked to the reference it cites.

  2. 02

    Source retrieval

    The cited publication is retrieved from authorized and open sources.

  3. 03

    Evidence extraction

    A language model specialized in extraction selects the relevant source passages and confirms each one verbatim.

  4. 04

    Multi-layer LLM assessment

    Multiple layers of language models assess each claim against the evidence, several times and independently.

  5. 05

    Error classification

    Unsupported claims receive a named error type.

  6. 06

    Report

    Claim and source side by side, highlighted discrepancy, explanation and editorial guidance. Unsupported claims and reference problems are shown first. Claims that could not be confirmed appear in a separate block.

In development

Two new layers, developed with publishers

Both layers extend the current architecture and are implemented in collaboration with publishers.

Internal-signal filter

Sparse autoencoders and linear probes read internal signals of language models to identify clearly supported claims before the LLM assessment. In development experiments, this reduced LLM judge evaluations by 17%. Fewer judge evaluations lower the cost of each manuscript and make verification at scale more efficient.

Agentic verification

A multi-agent system for complex and unresolved cases. An orchestrator agent decides what each case needs and assigns the work to specialized agents that operate independently: one retrieves additional evidence, another extracts new passages, and others assess the claim again. The orchestrator reviews their results and stops when the case is resolved. Deeper analysis is reserved for the claims that need it.

Results

0%

false positives on supported claims

100%

of errors detected when the cited source was available

Development evaluation results; not yet measured on the held-out Gold TEST.

What the report shows

Unsupported

The source evidence contradicts the claim or does not report what the claim states.

Unverified (full text needed)

Only the abstract was available, and it does not support the claim. The full-text paper must be checked.

Uncertain

The full text was read, but the repeated assessments did not reach a consistent verdict.

Not reviewed

The cited source could not be retrieved, or the reference could not be identified.

Metadata problems

The reference has a broken DOI or incomplete bibliographic data.

The report also shows how many claims were supported, and highlights them in the manuscript view.

Eight types of error

These are the subtle errors that AI-generated text produces when it cites real publications. Each unsupported claim in the report receives one of these types.

The source says the opposite

ManuscriptMetformin increased tumour growth in mice.

SourceMetformin reduced tumour growth in mice.

The study did not do that

ManuscriptA randomized trial showed that statins reduce dementia risk.

SourceA retrospective cohort study of statin users.

Data does not match the study

ManuscriptThe trial included 1,200 patients.

SourceThe trial included 120 patients.

Inflated figure

ManuscriptThe drug reduced relapse threefold.

SourceThe drug reduced relapse by 30%.

Figure taken out of context

ManuscriptThe response rate was 80%.

Source80% in patients with the EGFR mutation, 22% overall.

One measure confused with another

ManuscriptThe intervention reduced mortality by 25%.

SourceThe intervention reduced hospital admissions by 25%.

The source does not say that

ManuscriptVitamin D supplementation prevents depression.

SourceReports effects of vitamin D on bone density only.

Overstated certainty

ManuscriptAir pollution causes type 2 diabetes.

SourceAir pollution was associated with type 2 diabetes.

Examples are illustrative.

Material errors

RefCheck focuses on errors that change the scientific meaning of a claim. Differences in wording that preserve the meaning of the source are not reported as errors.

Scope and access to sources

RefCheck is designed for life sciences and biomedicine.

When a cited paper is behind a paywall, RefCheck works with the abstract. If the abstract does not support the claim, the claim is marked Unverified (full text needed) and the editor is asked to check the full-text paper. This depends on access to the full text, not on RefCheck. For this reason, we are seeking agreements with publishers to verify claims against the complete source.

Discuss a publisher pilot →