|
Getting your Trinity Audio player ready...
|
Contents
- Introduction
- Summarizing a research paper
- Extracting data from a research paper
- Answering specific questions from your own PDF
- Evaluating a research paper against a set checklist
- Conclusion: Can AI read papers for me?
Introduction
AI is effective at solving one problem that plagues researchers: too many papers to read and too little time. AI is also good at making complex topics understandable to students and making dense, jargon-heavy text easier to read. But use AI wrongly and you end up with hallucinations: fake output that is plausibly real. This article will walk you through the right way to use AI when you’re trying to get insights from a research paper, with sample prompts for different use cases.
Summarizing a research paper
If you just type “summarize this paper” into ChatGPT, you’ll get a variation of the paper’s abstract. That’s not what you want. Tell AI what you want the summary for: deciding whether you need to read the paper, whether the paper should be included in a literature review, whether the paper supports a specific claim you’re making, etc.
Also, AI can skip the limitations of a paper or turn hedging language like “suggests” into confident claims. This is another thing to watch out for in an AI-generated summary.
Sample prompt
Summarize the attached paper for a researcher conducting a literature review on [your topic]. Use these sections: (1) Research question, (2) Study design and sample, (3) Key instruments or equipment, (4) Main findings with the key numbers, (5) Limitations the authors acknowledge. Keep it under 400 words. If something isn’t stated in the paper, write “not reported” rather than inferring it.
Extracting data from a research paper
For a systematic review or meta-analysis, you need to extract a handful of key details from 100+ papers: sample size, sample characteristics, effect size, etc. This used to be a manual task, and frankly, pretty boring. AI can speed this up, making data extraction a matter of minutes, not hours.
Sample prompt
Extract the following from the attached paper into a markdown table with one row per reported outcome: outcome name, measurement instrument, sample size analyzed, effect estimate, 95% CI, p-value, and the page or table number where you found it. Use “NR” for anything not reported. Do not calculate or estimate any value that isn’t printed in the paper.
Caveats for AI-assisted data extraction
- Data extraction from embedded figures and tables may not be complete or accurate.
- You may need to download and supply supplemental files from the article separately and re-feed them to your AI tool.
- Always ask AI for the specific paragraph, line, page or table that every value came from.
- Instruct AI to input “not found” or “not reported” if it can’t find a value.
Answering specific questions from your own PDF
While writing, you may also have questions about a specific paper that don’t surface in an AI-generated summary or the paper’s abstract. These could be “did the authors control for variable X?”, “which scale did the authors use to measure Y?”, “how did the authors define term J?” “how many samples came from B type of site”? Using AI, you can type in a query and get answers sourced directly from a single PDF that you upload.
Sample prompt
Using only the attached PDF, answer: how did the authors handle participants who dropped out before the 12-month follow-up? Quote the exact sentence(s) describing this and give the section name. If the paper does not specify, say so explicitly instead of describing what studies like this typically do.
Precautions for using AI ChatPDF tools
- Ask AI to quote the exact text each time from the paper.
- Instruct AI to say “Not reported” for details it can’t find
- AI may not be able to “scan” a figure to tell you what’s in it, e.g., determining a trend in a subgroup from a line chart or which group of genes was turned off in a heat map.
Evaluating a research paper against a set checklist
Researchers don’t evaluate papers based on vibes. Critical appraisal checklists exist precisely for papers to be evaluated on relevant, specific and objective criteria. These include CONSORT, STROBE, PRISMA and various families of tools from the Joanna Briggs Institute (JBI) or CASP (Critical Skills Appraisal Programme, UK). AI can be used to catch specific omissions from a paper, such as missing trial registration numbers or missing loss-to-follow-up data.
Sample prompt
Appraise the attached randomized trial against the CONSORT 2010 checklist. Make a table with three columns: 1) checklist item, 2) status (Met / Partially met / Not met / N/A), and 3) either a short quote and location from the paper, or a note that it’s absent.
Precautions while using AI for manuscript evaluation
- AI can’t tell you if the research question was interesting, the methodology and design matched the hypothesis, whether the data are fabricated, etc. Those require human judgement and expertise.
- Always instruct AI to flag absences, else it will hallucinate details to supply any gaps.
- Permission is murky when it comes to manuscripts that you’re peer reviewing. In general, do not upload to any public chatbot like ChatGPT a manuscript sent to you for peer review. Consult the handling editor about the journal’s AI use policy for peer reviewers and whether the paper can be run through journal-preferred institutional AI tools.
Conclusion: Can AI read papers for me?
AI is not a substitute for reading a paper yourself. It only makes reading, summarizing, extracting, and appraising a paper faster. Hallucination risk is lower when you use specific, narrow prompts as suggested in this article, but isn’t eliminated altogether. If you skip reading the paper, you’ll end up with superficial or incorrect output. Remember that AI tools are built to sound confident and plausible, so output from AI-assisted literature search still requires human verification.
