Can AI Do Systematic Review Work Independently?

AI can assist with systematic reviews, but can ai do systematic review independently from search through interpretation? Current evidence does not justify treating AI as an autonomous reviewer. It can help with defined tasks, while researchers remain accountable for the protocol, completeness of the evidence, eligibility decisions, data, appraisal, and conclusions. [3][4]

A systematic review uses explicit methods to identify, select, appraise, and synthesize relevant research; a meta-analysis is one possible statistical component, not another name for the review itself. [2] The practical question is therefore not whether AI can “do a review,” but which tasks it can support, how its output is checked, and who makes consequential decisions.

Can AI do a systematic review on its own?

No: current evidence supports AI as assistance for discrete tasks, not as a substitute for an accountable review team. A review requires linked decisions about scope, searching, eligibility, appraisal, synthesis, and interpretation. Automating one step does not establish that the whole process is reliable.

“AI” also covers different methods. Performance observed with trained screening systems or active learning cannot automatically be attributed to a general-purpose large language model (LLM). Evidence about one model, dataset, or review task is not a guarantee for another. [1][6] Treating a tool as an independent reviewer would also make it harder to ensure that methods were applied consistently and errors were detected.

A governed workflow uses AI to reduce repetitive work while preserving human control over protocol choices, search completeness, final eligibility decisions, data verification, appraisal, and interpretation. [3][4] The tool may suggest or prioritize; the team remains responsible for deciding what the evidence means.

Which review tasks can AI help with?

AI can assist with selected evidence-synthesis tasks, particularly screening support, but its usefulness depends on the task and the review-specific validation. [1][5] For example, a system may prioritize records that appear more likely to meet eligibility criteria, helping reviewers decide what to inspect first. That is different from making the final inclusion decision.

Some screening approaches are semi-automated: they help order or flag records for human review. Fully automated approaches make relevance decisions without the same record-by-record human involvement. Evidence maps describe results for both kinds of systems, but performance varies by task and evaluation. [1][5] Results from abstract screening should not be assumed to establish accuracy for full-text decisions, data extraction, or risk-of-bias assessment.

AI can therefore be most defensible when its role is bounded and visible: the team specifies what the tool may do, keeps access to source records, and checks outputs against the underlying evidence. A tool’s ability to generate fluent text is not, by itself, evidence that its searches, extracted values, or methodological judgments are correct. [6]

What do recall and workload-reduction numbers actually mean?

Recall is the share of all relevant records that a process identifies; precision is the share of records labeled relevant that truly are relevant. [1] In screening, high recall matters because missed eligible studies can change the evidence base. But a high reported recall in a benchmark does not guarantee that a tool will perform equally well in a new review.

Evidence maps report different median screening results across approaches. One map reported median recall around 95% for active-learning approaches, with workload reduced by about half, and median recall of 88% for fully automated LLM abstract screening. [1] A 2025 AHRQ evidence map covering 95 studies reported median recall of 85% for fully automated zero-shot models and 97% for semi-automated models; semi-automated approaches had a median screening-burden reduction of 51%. [5] These are summaries across evaluations, not promises about a particular topic, dataset, or implementation.

Workload reduction also does not mean the remaining records are safe to ignore. A commonly used 95% recall target is not a guarantee; reaching a defensible stopping point depends on an adequately validated stopping rule. [1][7][8] Teams should pilot the tool on their own review materials, compare its decisions with human judgments, inspect disagreements, and document how validation informs continued use. A benchmark number alone cannot establish that the tool is safe for a new review.

Where does AI still make consequential mistakes?

AI can miss relevant studies and make errors in search, extraction, and appraisal, so its output requires source-level verification. In a review of 19 comparative studies of generative AI, the median share of relevant studies missed in search experiments was 91%; median incorrect data extraction was 14%, and median risk-of-bias assessment errors were 27%. [6] These figures describe the studies reviewed, not guaranteed error rates for every tool or review.

The distinction between prioritization and exclusion is especially important. Prioritization changes the order in which people inspect records; automatic exclusion removes records from consideration and can discard eligible evidence. The latter therefore needs stronger, review-specific validation and safeguards. [8][12] A team should not infer that an item is ineligible merely because a model ranks it low or labels it irrelevant.

Errors can also arise when the tool is applied to material unlike the data on which its performance was evaluated. Differences in the review question, eligibility criteria, databases, abstracts, full texts, and model or system may affect results. Because evidence is heterogeneous and task-specific, teams need to test the actual configuration they intend to use rather than transfer results uncritically from another evaluation. [1][5][6]

How should a review team use AI safely?

A review team should define the tool’s role in advance, validate it on the review’s own material, and retain human responsibility for consequential decisions. Reviewers should establish the protocol and eligibility criteria before using AI to support screening, then assess whether the tool behaves appropriately on records representative of the review.

A practical pilot can compare AI output with human judgments, examine false exclusions and disagreements, and record what the team changes as a result. Keep the source record available for checking extracted data and decisions. If the tool is used to exclude records rather than prioritize them, the team needs a stronger, review-specific basis for trusting that exclusion process. [8][12]

Human oversight is not a single final check. Reviewers remain responsible for search completeness and protocol decisions as well as eligibility, verification, appraisal, and interpretation. [3][4] For intervention reviews, Cochrane guidance requires two independent people for final full-text eligibility decisions; this is Cochrane guidance for that context, not a universal PRISMA requirement. [3][9]

What should researchers report about AI use?

Researchers should report enough information for readers to understand what the AI did and how its output was checked. Relevant details include the tool and version, its purpose, the dates of use, human oversight, the validation approach, and relevant prompts or outputs. [9][10][11] Reporting should make clear which decisions were made by people and which were supported or generated by the tool.

PRISMA is a reporting guideline, not an endorsement of a particular AI method. [9] Transparent reporting helps readers assess how the workflow was conducted; it does not, on its own, validate the method. A review should therefore describe validation and safeguards alongside the tool’s role, rather than treating disclosure as proof of accuracy.

Key takeaways

• AI can support defined systematic-review tasks, but current evidence does not justify autonomous end-to-end reviews. [1][4]
• Recall measures the share of relevant records identified; a high benchmark recall does not guarantee performance in a new review. [1][7][8]
• Prioritizing records for human review is not the same as automatically excluding them. [8][12]
• Review teams should verify AI-supported decisions against source material and retain responsibility for consequential judgments. [3][4]
• Reports should identify the tool and version, purpose and dates of use, human oversight, validation, and relevant prompts or outputs. [9][10][11]

FAQ

What is dual screening in systematic reviews?

Dual screening generally refers to having two people assess records independently for eligibility. In the specific context of final full-text eligibility decisions in intervention reviews, Cochrane guidance requires two independent people. [3] That guidance should not be mistaken for a universal PRISMA requirement. [9]

Does dual screening make AI screening unnecessary?

Not necessarily: human review and AI support serve different functions, and using a tool does not itself establish that screening decisions are dependable. A team should follow the applicable review guidance and validate any AI-supported workflow rather than assume the tool replaces independent human judgment. [3][4][8]

Is a 95% recall target enough to stop screening?

No. A commonly used 95% recall target is not a guarantee, and a defensible stopping decision requires an adequately validated stopping rule. [1][7][8] Teams need to assess the rule in the context of their review rather than rely on the threshold alone.

What should be reported when AI is used in a review?

Report the tool and version, its purpose and dates of use, the human oversight provided, and the validation approach. Include relevant prompts or outputs so readers can understand how the AI-supported workflow operated. [9][10][11]