Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.
Library Hours

Systematic Reviews

Select Studies

Appropriate studies will need to be selected from your search results based on the inclusion and exclusion criteria defined in your protocol. ​​Selection should be a methodical and well-documented process undertaken by at least two members of the review team to minimize bias.

Creating a Procedure

A minimum of two reviewers will need to run through all steps of the selection process independently, an important step to reducing selection bias. It's important that the methods for documenting the process, the inclusion and exclusion criteria, the method for eliminating duplicates, and the standards for critical evaluation and bias assessment are all clearly laid out and understood by everyone. 

Running a pilot of the study selection process on a sample set of studies can give you an idea of how well your procedure works, and whether anything needs to be clarified or refined.

Even following a clearly set procedure, it is likely that some discrepancies will exist between the results of each reviewer. Your team should also have a plan in place for resolving these disagreements. For example, disagreements may be resolved by both reviewers reaching a consensus through discussion, or by having a third independent reviewer to make the decision.

Documenting the Selection Process

You will need to keep a record of your study selection process, including information about articles that are excluded from the review, and your reasons for excluding them. ​You may wish to create screening forms to provide detailed information on your decisions to include or exclude individual studies. This flow chart from PRISMA can help you provide a summary of your process:

PRISMA Selection Flow Chart

Removing Duplicates

The first way to narrow your search results is by identifying and eliminating any duplicates. Citation management tools make this task easier, but be careful not to rely too much on their automatic processes. They may accidentally delete both copies of a study, or eliminate something that was not actually a duplicate.

Exact Duplicates:

There will likely be exact duplicates even within the same database, and certainly across multiple databases. These need to be eliminated to avoid skewing your results.

Close Duplicates:

These are often, but not always, the same publication that has just been entered as new database entries in slightly different ways (ex. Author as "Smith, JN" vs. "James N Smith").

Close duplicates can also be:

  • Translations of the same publication
  • A follow-up of the same study
  • Two publications reporting different sets of outcomes from the same study

Due to the bias that multiple reporting introduces, the Centre for Reviews and Dissemination recommends treating multiple reports of a study as a single study, but with references to all of the publications.

Using Your Inclusion and Exclusion Criteria

The next step will be to eliminate studies based on your pre-determined inclusion and exclusion criteria:

  • Scan the titles and abstracts to quickly eliminate studies that do not focus on your topic or meet your criteria. When in doubt, err on the side of over-inclusion.
  • Obtain the full-text documents of the remaining studies and do a more thorough evaluation of what meets your criteria.

Obtaining Full-Text Documents:

Sometimes it can be difficult to find full-text documents. If it is not available through the OSU Library system, you can often obtain it through: 

Contact a librarian if you're struggling to find and/or access any publications. We're here to help!

Quality Assessment

The quality of a systematic review's conclusions depends on the quality of the included research, from which the team extracts and synthesizes data. Unfortunately, not all research is reliable. Flawed study design, execution, or reporting can result in bias and impact results and conclusions. That's why study quality should be a final parameter in the decision to to include research within your subsequent analysis.

What is "study quality"?

  • Methodological quality: How the study has been designed and conducted.
  • Reporting quality: How the study has been described. You may need to contact study authors to fill in information gaps.

Some guidelines prefer to focus on the more specific term "risk of bias", which refers to the potential relationship between methodological characteristics and validity of the study's results.

Other issues that can impact study quality:

  • Appropriateness of study design to the research objective
  • Choice of outcome measure
  • Issues in the statistical analysis 
  • Generalizability

How to evaluate study quality:

Clearly, assessing the quality of a study has the potential to be very subjective, but, like all aspects of the systematic review, your methods must be consistent and reproducible. A wide variety of critical evaluation tools have been developed to assess the quality of individual studies. However, not all of these tools may be appropriate for your systematic review. For example, many guidelines now advise against using scales like the Jadad scale, which give studies a summary quality score, because they obscure the importance of individual elements.

Here are some useful tools for assessing study quality: