Published 21 May 2021
— CG Science Unlocked —
The media tends to oversimplify research findings in the interest of generating sensational headlines. Too often, complex scientific papers are reduced to soundbites which – while entertaining – fail to accurately characterize results. Thankfully, you do not have to rely on media to interpret scientific findings for you. The internet has made it far easier to review the research yourself and evaluate the accuracy of dramatic claims.
Unsure where to start? Here are some tips to get you started.
Locate the source
News stories do not typically include full citations or links to referenced research, which means you will need to source the paper yourself. If you don’t know the title, Google Scholar is an excellent resource for locating academic journal articles as is PubMed, a research index run by the US National Institutes of Health.
To search, use keywords drawn from details in the news article. Depending on the topic, possibilities include:
You can also filter your search for publication date and type of research (e.g. metanalysis, clinical trial). Given the sheer volume of research published on COVID-19 already, you may have to tweak your keywords and try several different combinations until you find useful results.
Once you find your article, the next step is to access it. Some pieces are ‘open access’, which means they are available online to anyone who wants to read them. Some, however, exist behind a paywall, which means you must pay to read them – and they are often prohibitively expensive.
In these cases, we suggest emailing the author and asking them to send it to you directly. A quick Google search of the author’s name and the affiliated university will usually yield an email address. Sending a brief note and a sincere ‘thank you’ will often result in a response. Researchers are people, too, and many are happy to get requests for their work.
So, you have the paper. Now what?
* * *
Is it peer-reviewed?
When researchers submit a study to a traditional academic journal for publication, the editors of the journal request that other scientists and researchers review it to ensure that it meets high scientific standards. This process is called ‘peer-review’. The reviewing scientists either approve the article for publication or send feedback and critique back to the editors. If the paper is then rejected by the journal editors on the basis of the critique, the authors are often allowed to read the feedback, make revisions, and resubmit. This process can be lengthy, sometimes spanning more than a year.
Not all published studies are peer-reviewed. Several online publications, such as medRxiv, publish studies known as ‘pre-prints’, which are completed studies that have not yet been through the peer-review process. Pre-prints are released so that new findings can be made public more quickly. However, they may contain errors, and their conclusions may change significantly prior to publication in a traditional journal – if they are accepted at all. Pre-prints typically come with disclaimers that indicate they have not been peer-reviewed. Conference or Congress Abstracts are one type of non-peer-reviewed research that often elicit media attention. These are written for presentation at scientific conferences and summarise the key findings of completed studies before they are published.
When evaluating an article, it is important to consider whether it has been peer-reviewed. Lack of peer review does not necessarily mean that research is of poor quality—but it does mean that it has not been thoroughly vetted by impartial experts. In other words, good research can be found in pre-prints, but not all pre-prints are good research.
* * *
How is it structured?
You now know whether your article was peer-reviewed and it is time to read the paper. Academic papers are generally organized into these sections:
* * *
Keep asking questions
Now that you have a clearer idea of what the paper says and can better discern if the media’s interpretation was accurate.
As you become more proficient, you will become more discerning in your critique and learn to identify the answers to essential questions:
While developing the ability to read and evaluate scientific papers can be challenging, it is worth the effort. Learning to critically evaluate an academic paper reduces reliance on media interpretations and limits the influence of third parties, who may have motivations for presenting information in a particular way. More broadly, it enhances your ability to think critically and develop well-informed opionions in other areas of your life.
And, like any other skill, you will improve with practice.
* * *
For a deeper dive into science reading, see Jennifer Raff’s excellent blog “How to read and understand a scientific paper: a guide for non-scientists” (LSE blogs, 9 May 2016)