Choosing Outcomes

Once you completed the processes above, the next step is to work on the primary outcomes.  The process defines the expectation of the main outcome of the research.  Continuing with the example above, a primary outcome may simply be rates of respiratory depression.  Secondary outcomes are also explored in this step. A secondary outcomes from the example above might be mortality from respiratory depression or morbidities that may result from respiratory depression.  A primary outcome is always needed and secondary outcomes may of interest but are not necessary. 

Writing a Protocol

It is highly encouraged that you both register your systematic review and search for similar reviews in PROSPERO. PROSPERO is the international database of prospectively registered systematic reviews in health care.  Searching PROSPERO for similar reviews allows you to see if another group is currently working on the same review, necessitating a change in topic.  Registering your topic also helps in letting others know your topic is being addressed. Your protocol for the PROSPERO record should include the following: 

1. Review Title
2. Anticipated or actual start date, anticipated completion date, and current stage review
3. Named contact, email, address, phone number, and organizational affiliation
4. Review team members and their affiliations
5. Funding sources and conflicts of interest - if any
6. Collaborators - those working on the review, but not listed as a team member
7. Review question
8. Search details with URL if available - contact your librarian for help here!
9. Condition being studied, population, intervention, comparison
10. Study inclusion and exclusion - context
11. Primary Outcomes and Secondary outcomes
12. Data extraction procedures 
13. Risk assessment procedures
14. Data synthesis plan - analysis of subgroups planned
15. Type of review, language, country
16. Etc. - dissemination plans, keywords, any existing reviews on same topic? 

Literature Searching is the next big step in your systematic review.  A big part of a systematic review is a through and systematic review of the literature.  This is where your local medical librarians can be of assistance.  Librarians can work as part of the systematic review team  as a kind of project manager to conduct the literature search (including grey or unpublished sources), organize citations in bibliographic citation managers and article review software, help to narrow down the initial number of articles to review, create a PRISMA flow diagram for publication, and write up the methodology of the search for publication.  Librarians can also conduct basic editing on the document.

When searching the literature it is suggested that at least three databases are searched.  This creates overlap in your articles, but also ensures a higher rate of coverage on pertinent articles.  Database access varies by library, so you may need to check to see what databases your library can access.  Suggested databases include:

• PubMed/MEDLINE - basic biomedical database for United States/North American coverage
• EMBASE - European coverage
• Cochrane Library - both systematic reviews and DARE
• CINAHL - nursing and allied health
• PsychINFO - psychology and psychiatry
• Web of Science - expanded basic and social science coverage
• SCOPUS - newer and expansive coverage database
• LILACS - Central and South American and the Caribbean

Once searches are completed, the entire search strategy and resulting citations should be saved in a file along with the total number of records retrieved, the date of the search, and the database searched.  Any limits or focus elements used should be noted.

Once the literature search has been conducted the articles titles and abstracts must first be searched for articles relevant to your question. A librarian can assist in first removing any duplicates and many articles which fall under the exclusion criteria.  A librarian can also assist with putting the citations into an online database for review, such as Rayyan or Coevidence. These online software tools allow reviewers to vet articles separately and then come together for a consensus.  Articles can also be labeled with inclusion or exclusion data. 

At least two content experts should look through all articles with a third content expert as a tie breaker.  After the initial articles are screened and agreed on, the reviewers should obtain full-text of the remaining articles for a more in depth review.  A secondary review can then be conducted to narrow the list down even further.  This should leave the reviewers with a final list of articles which fit the study criteria. 

Your process should follow the PRISMA flow diagram below.  You should also use a copy of this flow diagram to document your search for publication or presentation.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097.

Extracting Data

Data extraction can be one of the tougher and more time consuming parts of the systematic review.  All content experts on the systematic review team should decide how they would like to collect data, with written checklists or more likely on an electronic spreadsheet. The use of Microsoft Excel or Google Spreadsheets can make the process much easier.  The content experts also need to decide what data should be collected.  This will depend heavily on your question and both primary and secondary outcomes.  You will need at least enough data to cover these areas as well as information for a basic demographic profile of participants. 

Evaluation of Study Quality

The evaluation of study quality can be done at the same time as data extraction.  This is simply taking each study and deciding on the quality of the methodology used by the study.  There are a few ranking systems which can be used:

• GRADE - Grading of Recommendations Assessment, Development, and Evaluation
• SORT - Strength of Recommendation Taxonomy
• CONSORT - Consolidated Standards of Reporting Trials 

Assessing risk of bias may also be included in your systematic review.  This is a portion required by Cochrane Reviews, but not included in all reviews.  However, bias in individual studies will effect the overall outcome of the systematic review.  General types of bias to look for: 

• ​Selection Bias - difference in baseline characteristics between groups.
• Performance Bias - difference in the care of groups.
• Detection Bias - differences in how outcomes are measured.
• Attrition Bias - difference in withdrawal rates between groups.
• Reporting Bias - differences in reported and unreported findings.

Statistical Assimilation of Data

Once all studies have been mined for data it is time to decide which studies had enough data to be included in a meta-analysis(if one is being done).  Generally, a statistician should be used for this portion of the review to assist the content experts in choosing and carrying out the best statistical tests for the data given. On the UAMS campus, you can contact 

the Translational Research Institute for biostats assistance.  If a content expert can also perform the statistical portion of the review the following parts should be considered: 

• Demographic Table
• Forest plot of all studies
• Forest plot of any sub-analyses

Writing a Systematic Review

You now have a set of articles with the data abstracted, your data analysis and you are ready to write the narrative portion of your systematic review. A systematic review is a type of research study, so will be written up similar to any research study with a few additional headings:

• Abstract
• Background
• Objectives
• Methods - criteria for selecting studies, search methods, data collection and analysis
• Results - description of studies, risk of bias in studies, effects of interventions
• Discussion
• Conclusions
• References - can include: included and excluded studies with reasons(from secondary review) 

Writing an Abstract

Writing a great abstract is key to having a good final research product for presentation or publication.  As a summary of your research and findings, your abstract should be limited to 100-250 words.  Since it can be difficult to condense your research these few words, the tips below will help with this process. 

• Start with a formal outline, even if not required, this structure will help you organize your thoughts.
  • Introduction: objective or hypothesis
  • Methods:  what you did
  • Results: what you found
  • Conclusion:  why it's important
• Be careful of word usage here.  You need to be concise and precise.
• Have someone edit your abstract.
• Assure the abstract aligns with submission guidelines.
• See AMEE Guide no. 108: Writing Competitive Research Conference Abstracts.

Writing a Title

The title of your project is the first thing that people will see.  Creating a good title for your presentation/publication/poster will capture the attention of visitors.  The following tips will help you as you develop your title:

• Brainstorm lots of keywords to help find the best mix.
• Be informative.
• Place your methodology or trial type at the end of your title.
• Use action words that concisely portrays the message of your project.
• Be aware of the length of your title. Don't let it get too long.
• See more ideas here: Academic Medicine - AM Rounds - Strategies for Writing and Effective Title.

Review articles should have anywhere from 2-5 authors.  Students, residents and fellows should keep in mind that they will always need to have a practicing clinician as an author on their article.

Authorship can be one of the trickiest parts of writing an article and should be dealt with up front.  The International Committee of Medical Journal Editors (ICMJE) has set guidelines for what authors are expected to contribute and what qualifies someone as an author.  These are the suggested guidelines to follow for the majority of clinical journals. The criteria for authorship are as follows:

1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
2. Drafting the work or revising it critically for important intellectual content; AND
3. Final approval of the version to be published; AND
4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Anyone not meeting all four criteria should be listed in the acknowledgements section of your paper.

Along with meeting author criteria, authors must decide amongst themselves who will be credited with the first author slot and the order of authors.  Traditionally the lead or corresponding author, who will handle organization, final edits, and submission, or the one who does the most writing will be the first author.  Mentors, senior faculty, and project PI's are generally placed as the last author.