How to Read a Scientific Paper

– CG Science Unlocked –

The media tends to oversimplify research findings in the interest of generating sensational headlines. Too often, complex scientific papers are reduced to soundbites which – while entertaining – fail to accurately characterize results. Thankfully, you do not have to rely on media to interpret scientific findings for you. The internet has made it far easier to review the research yourself and evaluate the accuracy of dramatic claims.

Unsure where to start? Here are some tips to get you started.

Locate the source

News stories do not typically include full citations or links to referenced research, which means you will need to source the paper yourself. If you don’t know the title, Google Scholar is an excellent resource for locating academic journal articles as is PubMed, a research index run by the US National Institutes of Health.

To search, use keywords drawn from details in the news article. Depending on the topic, possibilities include:

• author name(s)
• the affiliated institution, e.g. hospital or university
• the name of the scientific journal
• subject, e.g. the disease or treatment studied

You can also filter your search for publication date and type of research (e.g. metanalysis, clinical trial). Given the sheer volume of research published on COVID-19 already, you may have to tweak your keywords and try several different combinations until you find useful results.

Once you find your article, the next step is to access it. Some pieces are ‘open access’, which means they are available online to anyone who wants to read them. Some, however, exist behind a paywall, which means you must pay to read them – and they are often prohibitively expensive.  

In these cases, we suggest emailing the author and asking them to send it to you directly. A quick Google search of the author’s name and the affiliated university will usually yield an email address. Sending a brief note and a sincere ‘thank you’ will often result in a response. Researchers are people, too, and many are happy to get requests for their work.

So, you have the paper. Now what?

* * *

Is it peer-reviewed?

When researchers submit a study to a traditional academic journal for publication, the editors of the journal request that other scientists and researchers review it to ensure that it meets high scientific standards. This process is called ‘peer-review’. The reviewing scientists either approve the article for publication or send feedback and critique back to the editors. If the paper is then rejected by the journal editors on the basis of the critique, the authors are often allowed to read the feedback, make revisions, and resubmit. This process can be lengthy, sometimes spanning more than a year.

Not all published studies are peer-reviewed. Several online publications, such as medRxiv, publish studies known as ‘pre-prints’, which are completed studies that have not yet been through the peer-review process. Pre-prints are released so that new findings can be made public more quickly. However, they may contain errors, and their conclusions may change significantly prior to publication in a traditional journal – if they are accepted at all. Pre-prints typically come with disclaimers that indicate they have not been peer-reviewed. Conference or Congress Abstracts are one type of non-peer-reviewed research that often elicit media attention. These are written for presentation at scientific conferences and summarise the key findings of completed studies before they are published.

When evaluating an article, it is important to consider whether it has been peer-reviewed. Lack of peer review does not necessarily mean that research is of poor quality—but it does mean that it has not been thoroughly vetted by impartial experts. In other words, good research can be found in pre-prints, but not all pre-prints are good research.

* * *

How is it structured?

You now know whether your article was peer-reviewed and it is time to read the paper.  Academic papers are generally organized into these sections:

• Abstract – briefly summarises the study and its results
• Introduction –  provides background and context and poses the question that the research is trying to answer
• Methods – describes how the research was conducted, including selection criteria, participants, design, materials, and procedure (*Determining the robustness of methodology can be difficult for non-subject matter experts. One good reason to lend more credence to peer-reviewed studies is that other experts have evaluated the methodology.)
• Results – communicates the study’s outcome and gives the results, often using figures, tables, and detailed statistical analysis
• Discussion/Conclusions – summarises results and explains their importance, reflecting on effectiveness and limitations of methods
• Future Scope/Recommendations – makes recommendations on the application of the findings and potential future research pathways

* * *

Keep asking questions

Now that you have a clearer idea of what the paper says and can better discern if the media’s interpretation was accurate. 

As you become more proficient, you will become more discerning in your critique and learn to identify the answers to essential questions:

• Who were the subjects, and how does this affect the interpretation?
• Where was the research conducted, and can the results apply elsewhere?
• Can the outcomes be extrapolated, or are they limited to a specific population?
• What was the sample size?  Was it big enough to justify the conclusion?
• When was the study conducted?  Are the results still applicable today?

While developing the ability to read and evaluate scientific papers can be challenging, it is worth the effort. Learning to critically evaluate an academic paper reduces reliance on media interpretations and limits the influence of third parties, who may have motivations for presenting information in a particular way. More broadly, it enhances your ability to think critically and develop well-informed opinions in other areas of your life. 

And, like any other skill, you will improve with practice.

* * *

For a deeper dive into science reading, see Jennifer Raff’s excellent blog “How to read and understand a scientific paper: a guide for non-scientists” (LSE blogs, 9 May 2016)