Why Evidence Synthesis Is Broken — And What Comes Next

The numbers tell the story

Evidence synthesis is the foundation of clinical guidelines — the recommendations that determine how millions of patients are treated. Systematic reviews and meta-analyses are the gold standard for producing this evidence. And by every meaningful metric, the process is failing the researchers who conduct it.

Here are the numbers.

1,139 hours per review

A comprehensive analysis of systematic review time requirements found that a single health sciences review consumes an average of 1,139 hours across all team members — roughly 30 weeks of full-time work. That includes 588 hours for protocol development, searching, and retrieval; 206 hours for report writing; 201 hours for administration; and 144 hours for statistical analysis.

This is not a niche problem. It is the baseline cost of producing a single piece of high-quality medical evidence.

10.76% screening error rate

A study analyzing 139,467 citations across 329,332 inclusion and exclusion decisions from 86 unique reviewers found that human reviewers have a combined error rate of 10.76% (95% CI: 7.43%–14.09%). More than one in ten screening decisions is wrong. The researchers concluded that "the gold standard is not perfect."

This error rate has practical consequences. Missed studies can change the conclusions of a review. Incorrectly included studies introduce noise. And in most workflows, there is no reliable mechanism to identify where errors occurred or propagated.

28% never completed

Of 8,137 Cochrane systematic review protocols, approximately 28% never progressed to a completed review. These represent years of registered research intent that produced no usable evidence.

64.3% never updated

Of the reviews that were completed, 64.3% were never updated by the time of analysis. Evidence goes stale. Guidelines that depend on that evidence fall behind the science. Patient care is informed by conclusions that may no longer hold.

Cochrane's living review attempt

The concept of living systematic reviews — continuously updated reviews that incorporate new evidence as it becomes available — is sound. The execution has proven difficult.

A 2025 mixed-methods study of Cochrane's COVID-19 living systematic reviews found that half of the living reviews went unmodified by mid-2023. Author teams struggled with publication delays, funding loss, and screening burden. Only one out of twenty-five living reviews successfully communicated its living status to readers after publication.

The methodology exists. The tooling to sustain it at scale does not.

67 documented problems

A living systematic review of systematic review problems cataloged 67 discrete flaws in the conduct, methods, and reporting of published reviews. The authors described these flaws as "a threat to credible science," given the central role systematic reviews play in informing clinical decisions and health policy.

These are not edge cases. They are structural problems with the way evidence synthesis is currently conducted.

The tools landscape: fragmented and incomplete

A 2025 scoping review identified 388 AI tools and platforms designed for evidence synthesis steps. That number sounds promising until you examine the distribution.

A separate systematic review of automated meta-analysis found that 57% of these tools address only data processing tasks like extraction and statistical modeling. Only 17% tackle advanced synthesis stages. And just 2% — a single study — explored full end-to-end process automation.

The most widely used tools — Covidence, Rayyan, DistillerSR — are screening assistants. They are useful for one step of an eight-step process. None of them cover the complete systematic review workflow. None are designed for living evidence synthesis.

Methodology is evolving faster than tools

The research methodology community continues to advance standards:

• PRISMA-NMA guidelines for network meta-analyses are being overhauled for greater transparency in statistical methods and intervention definitions.
• CONSORT-C 2026, published in The BMJ in March 2026, introduces 13 new reporting items for pediatric randomized trials — including youth-generated items reflecting direct patient input.
• PRISMA-LSR, published in The BMJ in November 2024, provides the first standardized reporting framework specifically for living systematic reviews.

These guidelines set a higher bar. The tools available to meet that bar have not kept pace.

What comes next

The gap between what evidence synthesis requires and what current tools provide is not shrinking. It is widening.

Regulators are moving toward continuous evidence. The FDA's single-trial policy intensifies the need for post-market evidence generation. The EMA's DARWIN EU network is producing 100 studies per year at a four-month median turnaround. Pharmacovigilance teams are adopting living evidence reviews that update continuously.

The tools that researchers use to produce this evidence should match the ambition of the standards they're expected to meet.

At Mapped Technologies, that's what we're building. Our first product, mapped, covers the complete eight-step systematic review workflow with AI assistance at every stage. Our next product targets living evidence synthesis — continuously updated, regulatory-grade, and built for the infrastructure that the FDA and EMA are constructing.

388 tools, none covering the full lifecycle. We're building what should exist.