critical review of methodology

• Linguistics
• Composition Studies

Writing Critical Reviews: A Step-by-Step Guide

• February 2022
• In book: Study Skills for International Postgraduates (Second Edition) (pp.194-207)
• Edition: 2nd
• Chapter: 11
• Publisher: Bloomsbury

• University of Melbourne

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• BUS HIST REV
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Methodological Approaches to Literature Review

• Living reference work entry
• First Online: 09 May 2023
• Cite this living reference work entry

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  

575 Accesses

1 Citations

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Similar content being viewed by others

Reviewing Literature for and as Research

Discussion and Conclusion

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Write a Critical Review

Introduction, how can i improve my critical review, ask us: chat, email, visit or call.

Video: How to Integrate Critical Voice into Your Literature Review

Video: Note-taking and Writing Tips to Avoid Plagiarism

More help: Writing

• Book Writing Appointments Get help on your writing assignments.
• To introduce the source, its main ideas, key details, and its place within the field
• To present your assessment of the quality of the source

In general, the introduction of your critical review should include

• Author(s) name
• Title of the source 
• What is the author's central purpose?
• What methods or theoretical frameworks were used to accomplish this purpose?
• What topic areas, chapters, sections, or key points did the author use to structure the source?
• What were the results or findings of the study?
• How were the results or findings interpreted? How were they related to the original problem (author's view of evidence rather than objective findings)?
• Who conducted the research? What were/are their interests?
• Why did they do this research?
• Was this research pertinent only within the author’s field, or did it have broader (even global) relevance?
• On what prior research was this source-based? What gap is the author attempting to address?
• How important was the research question posed by the researcher?
• Your overall opinion of the quality of the source. Think of this like a thesis or main argument.
• Present your evaluation of the source, providing evidence from the text (or other sources) to support your assessment.

In general, the body of your critical review should include

• Is the material organized logically and with appropriate headings?
• Are there stylistic problems in logical, clarity or language?
• Were the author(s) able to answer the question (test the hypothesis) raised
• What was the objective of the study?
• Does all the information lead coherently to the purpose of the study?
• Are the methods valid for studying the problem or gap?
• Could the study be duplicated from the information provided?
• Is the experimental design logical and reliable?
• How are the data organized? Is it logical and interpretable?
• Do the results reveal what the researcher intended?
• Do the authors present a logical interpretation of the results?
• Have the limitations of the research been addressed?
• Does the study consider other key studies in the field or other research possibilities or directions?
• How was the significance of the work described?
• Follow the structure of the journal article (e.g. Introduction, Methods, Results, Discussion) - highlighting the strengths and weaknesses in each section
• Present the weaknesses of the article, and then the strengths of the article (or vice versa).
• Group your ideas according to different research themes presented in the source
• Group the strengths and weaknesses of the article into the following areas: originality, reliability, validity, relevance, and presentation

Purpose: 

• To summarize the strengths and weaknesses of the article as a whole
• To assert the article’s practical and theoretical significance

In general, the conclusion of your critical review should include

• A restatement of your overall opinion
• A summary of the key strengths and weaknesses of the research that support your overall opinion of the source
• Did the research reported in this source result in the formation of new questions, theories or hypotheses by the authors or other researchers?
• Have other researchers subsequently supported or refuted the observations or interpretations of these authors?
• Did the research provide new factual information, a new understanding of a phenomenon in the field, a new research technique?
• Did the research produce any practical applications? 
• What are the social, political, technological, or medical implications of this research?
• How do you evaluate the significance of the research? 
• Find out what style guide you are required to follow (e.g., APA, MLA, Chicago) and follow the guidelines to create a reference list (may be called a bibliography or works cited).
• Be sure to include citations in the text when you refer to the source itself or external sources. 
• Check out our Cite Your Sources Guide for more information. 
• Read assignment instructions carefully and refer to them throughout the writing process.
• Make an outline of your main sections before you write.
• If your professor does not assign a topic or source, you must choose one yourself. Select a source that interests you and is written clearly so you can understand it.
• << Previous: Start Here
• Last Updated: Apr 25, 2024 1:53 PM
• URL: https://guides.lib.uoguelph.ca/CriticalReview

Suggest an edit to this guide

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

• Open access
• Published: 11 August 2009

Methods for the synthesis of qualitative research: a critical review

• Elaine Barnett-Page 1 &
• James Thomas 1  

BMC Medical Research Methodology volume  9 , Article number:  59 ( 2009 ) Cite this article

198k Accesses

1142 Citations

40 Altmetric

Metrics details

In recent years, a growing number of methods for synthesising qualitative research have emerged, particularly in relation to health-related research. There is a need for both researchers and commissioners to be able to distinguish between these methods and to select which method is the most appropriate to their situation.

A number of methodological and conceptual links between these methods were identified and explored, while contrasting epistemological positions explained differences in approaches to issues such as quality assessment and extent of iteration. Methods broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences.

Methods for qualitative synthesis vary across a range of dimensions. Commissioners of qualitative syntheses might wish to consider the kind of product they want and select their method – or type of method – accordingly.

Peer Review reports

The range of different methods for synthesising qualitative research has been growing over recent years [ 1 , 2 ], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [ 3 ]. While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe how quantitative research is synthesised, far more terms are used to describe the synthesis of qualitative research. This profusion of terms can mask some of the basic similarities in approach that the different methods share, and also lead to some confusion regarding which method is most appropriate in a given situation. This paper does not argue that the various nomenclatures are unnecessary, but rather seeks to draw together and review the full range of methods of synthesis available to assist future reviewers in selecting a method that is fit for their purpose. It also represents an attempt to guide the reader through some of the varied terminology to spring up around qualitative synthesis. Other helpful reviews of synthesis methods have been undertaken in recent years with slightly different foci to this paper. Two recent studies have focused on describing and critiquing methods for the integration of qualitative research with quantitative [ 4 , 5 ] rather than exclusively examining the detail and rationale of methods for the synthesis of qualitative research. Two other significant pieces of work give practical advice for conducting the synthesis of qualitative research, but do not discuss the full range of methods available [ 6 , 7 ]. We begin our Discussion by outlining each method of synthesis in turn, before comparing and contrasting characteristics of these different methods across a range of dimensions. Readers who are more familiar with the synthesis methods described here may prefer to turn straight to the 'dimensions of difference' analysis in the second part of the Discussion.

Overview of synthesis methods

Meta-ethnography.

In their seminal work of 1988, Noblit and Hare proposed meta-ethnography as an alternative to meta-analysis [ 8 ]. They cited Strike and Posner's [ 9 ] definition of synthesis as an activity in which separate parts are brought together to form a 'whole'; this construction of the whole is essentially characterised by some degree of innovation, so that the result is greater than the sum of its parts. They also borrowed from Turner's theory of social explanation [ 10 ], a key tenet of which was building 'comparative understanding' [[ 8 ], p22] rather than aggregating data.

To Noblit and Hare, synthesis provided an answer to the question of 'how to "put together" written interpretive accounts' [[ 8 ], p7], where mere integration would not be appropriate. Noblit and Hare's early work synthesised research from the field of education.

Three different methods of synthesis are used in meta-ethnography. One involves the 'translation' of concepts from individual studies into one another, thereby evolving overarching concepts or metaphors. Noblit and Hare called this process reciprocal translational analysis (RTA). Refutational synthesis involves exploring and explaining contradictions between individual studies. Lines-of-argument (LOA) synthesis involves building up a picture of the whole (i.e. culture, organisation etc) from studies of its parts. The authors conceptualised this latter approach as a type of grounded theorising.

Britten et al [ 11 ] and Campbell et al [ 12 ] have both conducted evaluations of meta-ethnography and claim to have succeeded, by using this method, in producing theories with greater explanatory power than could be achieved in a narrative literature review. While both these evaluations used small numbers of studies, more recently Pound et al [ 13 ] conducted both an RTA and an LOA synthesis using a much larger number of studies (37) on resisting medicines. These studies demonstrate that meta-ethnography has evolved since Noblit and Hare first introduced it. Campbell et al claim to have applied the method successfully to non-ethnographical studies. Based on their reading of Schutz [ 14 ], Britten et al have developed both second and third order constructs in their synthesis (Noblit and Hare briefly allude to the possibility of a 'second level of synthesis' [[ 8 ], p28] but do not demonstrate or further develop the idea).

In a more recent development, Sandelowski & Barroso [ 15 ] write of adapting RTA by using it to ' integrate findings interpretively, as opposed to comparing them interpretively' (p204). The former would involve looking to see whether the same concept, theory etc exists in different studies; the latter would involve the construction of a bigger picture or theory (i.e. LOA synthesis). They also talk about comparing or integrating imported concepts (e.g. from other disciplines) as well as those evolved 'in vivo'.

Grounded theory

Kearney [ 16 ], Eaves [ 17 ] and Finfgeld [ 18 ] have all adapted grounded theory to formulate a method of synthesis. Key methods and assumptions of grounded theory, as originally formulated and subsequently refined by Glaser and Strauss [ 19 ] and Strauss and Corbin [ 20 , 21 ], include: simultaneous phases of data collection and analysis; an inductive approach to analysis, allowing the theory to emerge from the data; the use of the constant comparison method; the use of theoretical sampling to reach theoretical saturation; and the generation of new theory. Eaves cited grounded theorists Charmaz [ 22 ] and Chesler [ 23 ], as well as Strauss and Corbin [ 20 ], as informing her approach to synthesis.

Glaser and Strauss [ 19 ] foresaw a time when a substantive body of grounded research should be pushed towards a higher, more abstract level. As a piece of methodological work, Eaves undertook her own synthesis of the synthesis methods used by these authors to produce her own clear and explicit guide to synthesis in grounded formal theory. Kearney stated that 'grounded formal theory', as she termed this method of synthesis, 'is suited to study of phenomena involving processes of contextualized understanding and action' [[ 24 ], p180] and, as such, is particularly applicable to nurses' research interests.

As Kearney suggested, the examples examined here were largely dominated by research in nursing. Eaves synthesised studies on care-giving in rural African-American families for elderly stroke survivors; Finfgeld on courage among individuals with long-term health problems; Kearney on women's experiences of domestic violence.

Kearney explicitly chose 'grounded formal theory' because it matches 'like' with 'like': that is, it applies the same methods that have been used to generate the original grounded theories included in the synthesis – produced by constant comparison and theoretical sampling – to generate a higher-level grounded theory. The wish to match 'like' with 'like' is also implicit in Eaves' paper. This distinguishes grounded formal theory from more recent applications of meta-ethnography, which have sought to include qualitative research using diverse methodological approaches [ 12 ].

• Thematic Synthesis

Thomas and Harden [ 25 ] have developed an approach to synthesis which they term 'thematic synthesis'. This combines and adapts approaches from both meta-ethnography and grounded theory. The method was developed out of a need to conduct reviews that addressed questions relating to intervention need, appropriateness and acceptability – as well as those relating to effectiveness – without compromising on key principles developed in systematic reviews. They applied thematic synthesis in a review of the barriers to, and facilitators of, healthy eating amongst children.

Free codes of findings are organised into 'descriptive' themes, which are then further interpreted to yield 'analytical' themes. This approach shares characteristics with later adaptations of meta-ethnography, in that the analytical themes are comparable to 'third order interpretations' and that the development of descriptive and analytical themes using coding invoke reciprocal 'translation'. It also shares much with grounded theory, in that the approach is inductive and themes are developed using a 'constant comparison' method. A novel aspect of their approach is the use of computer software to code the results of included studies line-by-line, thus borrowing another technique from methods usually used to analyse primary research.

Textual Narrative Synthesis

Textual narrative synthesis is an approach which arranges studies into more homogenous groups. Lucas et al [ 26 ] comment that it has proved useful in synthesising evidence of different types (qualitative, quantitative, economic etc). Typically, study characteristics, context, quality and findings are reported on according to a standard format and similarities and differences are compared across studies. Structured summaries may also be developed, elaborating on and putting into context the extracted data [ 27 ].

Lucas et al [ 26 ] compared thematic synthesis with textual narrative synthesis. They found that 'thematic synthesis holds most potential for hypothesis generation' whereas textual narrative synthesis is more likely to make transparent heterogeneity between studies (as does meta-ethnography, with refutational synthesis) and issues of quality appraisal. This is possibly because textual narrative synthesis makes clearer the context and characteristics of each study, while the thematic approach organises data according to themes. However, Lucas et al found that textual narrative synthesis is 'less good at identifying commonality' (p2); the authors do not make explicit why this should be, although it may be that organising according to themes, as the thematic approach does, is comparatively more successful in revealing commonality.

Paterson et al [ 28 ] have evolved a multi-faceted approach to synthesis, which they call 'meta-study'. The sociologist Zhao [ 29 ], drawing on Ritzer's work [ 30 ], outlined three components of analysis, which they proposed should be undertaken prior to synthesis. These are meta-data-analysis (the analysis of findings), meta-method (the analysis of methods) and meta-theory (the analysis of theory). Collectively, these three elements of analysis, culminating in synthesis, make up the practice of 'meta-study'. Paterson et al pointed out that the different components of analysis may be conducted concurrently.

Paterson et al argued that primary research is a construction; secondary research is therefore a construction of a construction. There is need for an approach that recognises this, and that also recognises research to be a product of its social, historical and ideological context. Such an approach would be useful in accounting for differences in research findings. For Paterson et al, there is no such thing as 'absolute truth'.

Meta-study was developed to study the experiences of adults living with a chronic illness. Meta-data-analysis was conceived of by Paterson et al in similar terms to Noblit and Hare's meta-ethnography (see above), in that it is essentially interpretive and seeks to reveal similarities and discrepancies among accounts of a particular phenomenon. Meta-method involves the examination of the methodologies of the individual studies under review. Part of the process of meta-method is to consider different aspects of methodology such as sampling, data collection, research design etc, similar to procedures others have called 'critical appraisal' (CASP [ 31 ]). However, Paterson et al take their critique to a deeper level by establishing the underlying assumptions of the methodologies used and the relationship between research outcomes and methods used. Meta-theory involves scrutiny of the philosophical and theoretical assumptions of the included research papers; this includes looking at the wider context in which new theory is generated. Paterson et al described meta-synthesis as a process which creates a new interpretation which accounts for the results of all three elements of analysis. The process of synthesis is iterative and reflexive and the authors were unwilling to oversimplify the process by 'codifying' procedures for bringing all three components of analysis together.

Meta-narrative

Greenhalgh et al [ 32 ]'s meta-narrative approach to synthesis arose out of the need to synthesise evidence to inform complex policy-making questions and was assisted by the formation of a multi-disciplinary team. Their approach to review was informed by Thomas Kuhn's The Structure of Scientific Revolutions [ 33 ], in which he proposed that knowledge is produced within particular paradigms which have their own assumptions about theory, about what is a legitimate object of study, about what are legitimate research questions and about what constitutes a finding. Paradigms also tend to develop through time according to a particular set of stages, central to which is the stage of 'normal science', in which the particular standards of the paradigm are largely unchallenged and seen to be self-evident. As Greenhalgh et al pointed out, Kuhn saw paradigms as largely incommensurable: 'that is, an empirical discovery made using one set of concepts, theories, methods and instruments cannot be satisfactorily explained through a different paradigmatic lens' [[ 32 ], p419].

Greenhalgh et al synthesised research from a wide range of disciplines; their research question related to the diffusion of innovations in health service delivery and organisation. They thus identified a need to synthesise findings from research which contains many different theories arising from many different disciplines and study designs.

Based on Kuhn's work, Greenhalgh et al proposed that, across different paradigms, there were multiple – and potentially mutually contradictory – ways of understanding the concept at the heart of their review, namely the diffusion of innovation. Bearing this in mind, the reviewers deliberately chose to select key papers from a number of different research 'paradigms' or 'traditions', both within and beyond healthcare, guided by their multidisciplinary research team. They took as their unit of analysis the 'unfolding "storyline" of a research tradition over time' [[ 32 ], p417) and sought to understand diffusion of innovation as it was conceptualised in each of these traditions. Key features of each tradition were mapped: historical roots, scope, theoretical basis; research questions asked and methods/instruments used; main empirical findings; historical development of the body of knowledge (how have earlier findings led to later findings); and strengths and limitations of the tradition. The results of this exercise led to maps of 13 'meta-narratives' in total, from which seven key dimensions, or themes, were identified and distilled for the synthesis phase of the review.

Critical Interpretive Synthesis

Dixon-Woods et al [ 34 ] developed their own approach to synthesising multi-disciplinary and multi-method evidence, termed 'critical interpretive synthesis', while researching access to healthcare by vulnerable groups. Critical interpretive synthesis is an adaptation of meta-ethnography, as well as borrowing techniques from grounded theory. The authors stated that they needed to adapt traditional meta-ethnographic methods for synthesis, since these had never been applied to quantitative as well as qualitative data, nor had they been applied to a substantial body of data (in this case, 119 papers).

Dixon-Woods et al presented critical interpretive synthesis as an approach to the whole process of review, rather than to just the synthesis component. It involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers [ 35 ]. The authors also stress, as a defining characteristic, critical interpretive synthesis's critical approach to the literature in terms of deconstructing research traditions or theoretical assumptions as a means of contextualising findings.

Dixon-Woods et al rejected reciprocal translational analysis (RTA) as this produced 'only a summary in terms that have already been used in the literature' [[ 34 ], p5], which was seen as less helpful when dealing with a large and diverse body of literature. Instead, Dixon-Woods et al adopted a lines-of-argument (LOA) synthesis, in which – rejecting the difference between first, second and third order constructs – they instead developed 'synthetic constructs' which were then linked with constructs arising directly from the literature.

The influence of grounded theory can be seen in particular in critical interpretive synthesis's inductive approach to formulating the review question and to developing categories and concepts, rejecting a 'stage' approach to systematic reviewing, and in selecting papers using theoretical sampling. Dixon-Woods et al also claim that critical interpretive synthesis is distinct in its 'explicit orientation towards theory generation' [[ 34 ], p9].

Ecological Triangulation

Jim Banning is the author of 'ecological triangulation' or 'ecological sentence synthesis', applying this method to the evidence for what works for youth with disabilities. He borrows from Webb et al [ 36 ] and Denzin [ 37 ] the concept of triangulation, in which phenomena are studied from a variety of vantage points. His rationale is that building an 'evidence base' of effectiveness requires the synthesis of cumulative, multi-faceted evidence in order to find out 'what intervention works for what kind of outcomes for what kind of persons under what kind of conditions' [[ 38 ], p1].

Ecological triangulation unpicks the mutually interdependent relationships between behaviour, persons and environments. The method requires that, for data extraction and synthesis, 'ecological sentences' are formulated following the pattern: 'With this intervention, these outcomes occur with these population foci and within these grades (ages), with these genders ... and these ethnicities in these settings' [[ 39 ], p1].

Framework Synthesis

Brunton et al [ 40 ] and Oliver et al [ 41 ] have applied a 'framework synthesis' approach in their reviews. Framework synthesis is based on framework analysis, which was outlined by Pope, Ziebland and Mays [ 42 ], and draws upon the work of Ritchie and Spencer [ 43 ] and Miles and Huberman [ 44 ]. Its rationale is that qualitative research produces large amounts of textual data in the form of transcripts, observational fieldnotes etc. The sheer wealth of information poses a challenge for rigorous analysis. Framework synthesis offers a highly structured approach to organising and analysing data (e.g. indexing using numerical codes, rearranging data into charts etc).

Brunton et al applied the approach to a review of children's, young people's and parents' views of walking and cycling; Oliver et al to an analysis of public involvement in health services research. Framework synthesis is distinct from the other methods outlined here in that it utilises an a priori 'framework' – informed by background material and team discussions – to extract and synthesise findings. As such, it is largely a deductive approach although, in addition to topics identified by the framework, new topics may be developed and incorporated as they emerge from the data. The synthetic product can be expressed in the form of a chart for each key dimension identified, which may be used to map the nature and range of the concept under study and find associations between themes and exceptions to these [ 40 ].

'Fledgling' approaches

There are three other approaches to synthesis which have not yet been widely used. One is an approach using content analysis [ 45 , 46 ] in which text is condensed into fewer content-related categories. Another is 'meta-interpretation' [ 47 ], featuring the following: an ideographic rather than pre-determined approach to the development of exclusion criteria; a focus on meaning in context; interpretations as raw data for synthesis (although this feature doesn't distinguish it from other synthesis methods); an iterative approach to the theoretical sampling of studies for synthesis; and a transparent audit trail demonstrating the trustworthiness of the synthesis.

In addition to the synthesis methods discussed above, Sandelowski and Barroso propose a method they call 'qualitative metasummary' [ 15 ]. It is mentioned here as a new and original approach to handling a collection of qualitative studies but is qualitatively different to the other methods described here since it is aggregative; that is, findings are accumulated and summarised rather than 'transformed'. Metasummary is a way of producing a 'map' of the contents of qualitative studies and – according to Sandelowski and Barroso – 'reflect [s] a quantitative logic' [[ 15 ], p151]. The frequency of each finding is determined and the higher the frequency of a particular finding, the greater its validity. The authors even discuss the calculation of 'effect sizes' for qualitative findings. Qualitative metasummaries can be undertaken as an end in themselves or may serve as a basis for a further synthesis.

Dimensions of difference

Having outlined the range of methods identified, we now turn to an examination of how they compare with one another. It is clear that they have come from many different contexts and have different approaches to understanding knowledge, but what do these differences mean in practice? Our framework for this analysis is shown in Additional file 1 : dimensions of difference [ 48 ]. We have examined the epistemology of each of the methods and found that, to some extent, this explains the need for different methods and their various approaches to synthesis.

Epistemology

The first dimension that we will consider is that of the researchers' epistemological assumptions. Spencer et al [ 49 ] outline a range of epistemological positions, which might be organised into a spectrum as follows:

Subjective idealism : there is no shared reality independent of multiple alternative human constructions

Objective idealism : there is a world of collectively shared understandings

Critical realism : knowledge of reality is mediated by our perceptions and beliefs

Scientific realism : it is possible for knowledge to approximate closely an external reality

Naïve realism : reality exists independently of human constructions and can be known directly [ 49 , 45 , 46 ].

Thus, at one end of the spectrum we have a highly constructivist view of knowledge and, at the other, an unproblematized 'direct window onto the world' view.

Nearly all of positions along this spectrum are represented in the range of methodological approaches to synthesis covered in this paper. The originators of meta-narrative synthesis, critical interpretive synthesis and meta-study all articulate what might be termed a 'subjective idealist' approach to knowledge. Paterson et al [ 28 ] state that meta-study shies away from creating 'grand theories' within the health or social sciences and assume that no single objective reality will be found. Primary studies, they argue, are themselves constructions; meta-synthesis, then, 'deals with constructions of constructions' (p7). Greenhalgh et al [ 32 ] also view knowledge as a product of its disciplinary paradigm and use this to explain conflicting findings: again, the authors neither seek, nor expect to find, one final, non-contestable answer to their research question. Critical interpretive synthesis is similar in seeking to place literature within its context, to question its assumptions and to produce a theoretical model of a phenomenon which – because highly interpretive – may not be reproducible by different research teams at alternative points in time [[ 34 ], p11].

Methods used to synthesise grounded theory studies in order to produce a higher level of grounded theory [ 24 ] appear to be informed by 'objective idealism', as does meta-ethnography. Kearney argues for the near-universal applicability of a 'ready-to-wear' theory across contexts and populations. This approach is clearly distinct from one which recognises multiple realities. The emphasis is on examining commonalities amongst, rather than discrepancies between, accounts. This emphasis is similarly apparent in most meta-ethnographies, which are conducted either according to Noblit and Hare's 'reciprocal translational analysis' technique or to their 'lines-of-argument' technique and which seek to provide a 'whole' which has a greater explanatory power. Although Noblit and Hare also propose 'refutational synthesis', in which contradictory findings might be explored, there are few examples of this having been undertaken in practice, and the aim of the method appears to be to explain and explore differences due to context, rather than multiple realities.

Despite an assumption of a reality which is perhaps less contestable than those of meta-narrative synthesis, critical interpretive synthesis and meta-study, both grounded formal theory and meta-ethnography place a great deal of emphasis on the interpretive nature of their methods. This still supposes a degree of constructivism. Although less explicit about how their methods are informed, it seems that both thematic synthesis and framework synthesis – while also involving some interpretation of data – share an even less problematized view of reality and a greater assumption that their synthetic products are reproducible and correspond to a shared reality. This is also implicit in the fact that such products are designed directly to inform policy and practice, a characteristic shared by ecological triangulation. Notably, ecological triangulation, according to Banning, can be either realist or idealist. Banning argues that the interpretation of triangulation can either be one in which multiple viewpoints converge on a point to produce confirming evidence (i.e. one definitive answer to the research question) or an idealist one, in which the complexity of multiple viewpoints is represented. Thus, although ecological triangulation views reality as complex, the approach assumes that it can be approximately knowable (at least when the realist view of ecological triangulation is adopted) and that interventions can and should be modelled according to the products of its syntheses.

While pigeonholing different methods into specific epistemological positions is a problematic process, we do suggest that the contrasting epistemologies of different researchers is one way of explaining why we have – and need – different methods for synthesis.

Variation in terms of the extent of iteration during the review process is another key dimension. All synthesis methods include some iteration but the degree varies. Meta-ethnography, grounded theory and thematic synthesis all include iteration at the synthesis stage; both framework synthesis and critical interpretive synthesis involve iterative literature searching – in the case of critical interpretive synthesis, it is not clear whether iteration occurs during the rest of the review process. Meta-narrative also involves iteration at every stage. Banning does not mention iteration in outlining ecological triangulation and neither do Lucas or Thomas and Harden for thematic narrative synthesis.

It seems that the more idealist the approach, the greater the extent of iteration. This might be because a large degree of iteration does not sit well with a more 'positivist' ideal of procedural objectivity; in particular, the notion that the robustness of the synthetic product depends in part on the reviewers stating up front in a protocol their searching strategies, inclusion/exclusion criteria etc, and being seen not to alter these at a later stage.

Quality assessment

Another dimension along which we can look at different synthesis methods is that of quality assessment. When the approaches to the assessment of the quality of studies retrieved for review are examined, there is again a wide methodological variation. It might be expected that the further towards the 'realism' end of the epistemological spectrum a method of synthesis falls, the greater the emphasis on quality assessment. In fact, this is only partially the case.

Framework synthesis, thematic narrative synthesis and thematic synthesis – methods which might be classified as sharing a 'critical realist' approach – all have highly specified approaches to quality assessment. The review in which framework synthesis was developed applied ten quality criteria: two on quality and reporting of sampling methods, four to the quality of the description of the sample in the study, two to the reliability and validity of the tools used to collect data and one on whether studies used appropriate methods for helping people to express their views. Studies which did not meet a certain number of quality criteria were excluded from contributing to findings. Similarly, in the example review for thematic synthesis, 12 criteria were applied: five related to reporting aims, context, rationale, methods and findings; four relating to reliability and validity; and three relating to the appropriateness of methods for ensuring that findings were rooted in participants' own perspectives. Studies which were deemed to have significant flaws were excluded and sensitivity analyses were used to assess the possible impact of study quality on the review's findings. Thomas and Harden's use of thematic narrative synthesis similarly applied quality criteria and developed criteria additional to those they found in the literature on quality assessment, relating to the extent to which people's views and perspectives had been privileged by researchers. It is worth noting not only that these methods apply quality criteria but that they are explicit about what they are: assessing quality is a key component in the review process for both of these methods. Likewise, Banning – the originator of ecological triangulation – sees quality assessment as important and adapts the Design and Implementation Assessment Device (DIAD) Version 0.3 (a quality assessment tool for quantitative research) for use when appraising qualitative studies [ 50 ]. Again, Banning writes of excluding studies deemed to be of poor quality.

Greenhalgh et al's meta-narrative review [ 32 ] modified a range of existing quality assessment tools to evaluate studies according to validity and robustness of methods; sample size and power; and validity of conclusions. The authors imply, but are not explicit, that this process formed the basis for the exclusion of some studies. Although not quite so clear about quality assessment methods as framework and thematic synthesis, it might be argued that meta-narrative synthesis shows a greater commitment to the concept that research can and should be assessed for quality than either meta-ethnography or grounded formal theory. The originators of meta-ethnography, Noblit and Hare [ 8 ], originally discussed quality in terms of quality of metaphor, while more recent use of this method has used amended versions of CASP (the Critical Appraisal Skills Programme tool, [ 31 ]), yet has only referred to studies being excluded on the basis of lack of relevance or because they weren't 'qualitative' studies [ 8 ]. In grounded theory, quality assessment is only discussed in terms of a 'personal note' being made on the context, quality and usefulness of each study. However, contrary to expectation, meta-narrative synthesis lies at the extreme end of the idealism/realism spectrum – as a subjective idealist approach – while meta-ethnography and grounded theory are classified as objective idealist approaches.

Finally, meta-study and critical interpretive synthesis – two more subjective idealist approaches – look to the content and utility of findings rather than methodology in order to establish quality. While earlier forms of meta-study included only studies which demonstrated 'epistemological soundness', in its most recent form [ 51 ] this method has sought to include all relevant studies, excluding only those deemed not to be 'qualitative' research. Critical interpretive synthesis also conforms to what we might expect of its approach to quality assessment: quality of research is judged as the extent to which it informs theory. The threshold of inclusion is informed by expertise and instinct rather than being articulated a priori.

In terms of quality assessment, it might be important to consider the academic context in which these various methods of synthesis developed. The reason why thematic synthesis, framework synthesis and ecological triangulation have such highly specified approaches to quality assessment may be that each of these was developed for a particular task, i.e. to conduct a multi-method review in which randomised controlled trials (RCTs) were included. The concept of quality assessment in relation to RCTs is much less contested and there is general agreement on criteria against which quality should be judged.

Problematizing the literature

Critical interpretive synthesis, the meta-narrative approach and the meta-theory element of meta-study all share some common ground in that their review and synthesis processes include examining all aspects of the context in which knowledge is produced. In conducting a review on access to healthcare by vulnerable groups, critical interpretive synthesis sought to question 'the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions' [[ 34 ], p6]. Although not claiming to have been directly influenced by Greenhalgh et al's meta-narrative approach, Dixon-Woods et al do cite it as sharing similar characteristics in the sense that it critiques the literature it reviews.

Meta-study uses meta-theory to describe and deconstruct the theories that shape a body of research and to assess its quality. One aspect of this process is to examine the historical evolution of each theory and to put it in its socio-political context, which invites direct comparison with meta-narrative synthesis. Greenhalgh et al put a similar emphasis on placing research findings within their social and historical context, often as a means of seeking to explain heterogeneity of findings. In addition, meta-narrative shares with critical interpretive synthesis an iterative approach to searching and selecting from the literature.

Framework synthesis, thematic synthesis, textual narrative synthesis, meta-ethnography and grounded theory do not share the same approach to problematizing the literature as critical interpretive synthesis, meta-study and meta-narrative. In part, this may be explained by the extent to which studies included in the synthesis represented a broad range of approaches or methodologies. This, in turn, may reflect the broadness of the review question and the extent to which the concepts contained within the question are pre-defined within the literature. In the case of both the critical interpretive synthesis and meta-narrative reviews, terminology was elastic and/or the question formed iteratively. Similarly, both reviews placed great emphasis on employing multi-disciplinary research teams. Approaches which do not critique the literature in the same way tend to have more narrowly-focused questions. They also tend to include a more limited range of studies: grounded theory synthesis includes grounded theory studies, meta-ethnography (in its original form, as applied by Noblit and Hare) ethnographies. The thematic synthesis incorporated studies based on only a narrow range of qualitative methodologies (interviews and focus groups) which were informed by a similarly narrow range of epistemological assumptions. It may be that the authors of such syntheses saw no need for including such a critique in their review process.

Similarities and differences between primary studies

Most methods of synthesis are applicable to heterogeneous data (i.e. studies which use contrasting methodologies) apart from early meta-ethnography and synthesis informed by grounded theory. All methods of synthesis state that, at some level, studies are compared; many are not so explicit about how this is done, though some are. Meta-ethnography is one of the most explicit: it describes the act of 'translation' where terms and concepts which have resonance with one another are subsumed into 'higher order constructs'. Grounded theory, as represented by Eaves [ 17 ], is undertaken according to a long list of steps and sub-steps, includes the production of generalizations about concepts/categories, which comes from classifying these categories. In meta-narrative synthesis, comparable studies are grouped together at the appraisal phase of review.

Perhaps more interesting are the ways in which differences between studies are explored. Those methods with a greater emphasis on critical appraisal may tend (although this is not always made explicit) to use differences in method to explain differences in finding. Meta-ethnography proposes 'refutational synthesis' to explain differences, although there are few examples of this in the literature. Some synthesis methods – for example, thematic synthesis – look at other characteristics of the studies under review, whether types of participants and their context vary, and whether this can explain differences in perspective.

All of these methods, then, look within the studies to explain differences. Other methods look beyond the study itself to the context in which it was produced. Critical interpretive synthesis and meta-study look at differences in theory or in socio-economic context. Critical interpretive synthesis, like meta-narrative, also explores epistemological orientation. Meta-narrative is unique in concerning itself with disciplinary paradigm (i.e. the story of the discipline as it progresses). It is also distinctive in that it treats conflicting findings as 'higher order data' [[ 32 ], p420], so that the main emphasis of the synthesis appears to be on examining and explaining contradictions in the literature.

Going 'beyond' the primary studies

Synthesis is sometimes defined as a process resulting in a product, a 'whole', which is more than the sum of its parts. However, the methods reviewed here vary in the extent to which they attempt to 'go beyond' the primary studies and transform the data. Some methods – textual narrative synthesis, ecological triangulation and framework synthesis – focus on describing and summarising their primary data (often in a highly structured and detailed way) and translating the studies into one another. Others – meta-ethnography, grounded theory, thematic synthesis, meta-study, meta-narrative and critical interpretive synthesis – seek to push beyond the original data to a fresh interpretation of the phenomena under review. A key feature of thematic synthesis is its clear differentiation between these two stages.

Different methods have different mechanisms for going beyond the primary studies, although some are more explicit than others about what these entail. Meta-ethnography proposes a 'Line of Argument' (LOA) synthesis in which an interpretation is constructed to both link and explain a set of parts. Critical interpretive synthesis based its synthesis methods on those of meta-ethnography, developing an LOA using what the authors term 'synthetic constructs' (akin to 'third order constructs' in meta-ethnography) to create a 'synthesising argument'. Dixon-Woods et al claim that this is an advance on Britten et al's methods, in that they reject the difference between first, second and third order constructs.

Meta-narrative, as outlined above, focuses on conflicting findings and constructs theories to explain these in terms of differing paradigms. Meta study derives questions from each of its three components to which it subjects the dataset and inductively generates a number of theoretical claims in relation to it. According to Eaves' model of grounded theory [ 17 ], mini-theories are integrated to produce an explanatory framework. In ecological triangulation, the 'axial' codes – or second level codes evolved from the initial deductive open codes – are used to produce Banning's 'ecological sentence' [ 39 ].

The synthetic product

In overviewing and comparing different qualitative synthesis methods, the ultimate question relates to the utility of the synthetic product: what is it for? It is clear that some methods of synthesis – namely, thematic synthesis, textual narrative synthesis, framework synthesis and ecological triangulation – view themselves as producing an output that is directly applicable to policy makers and designers of interventions. The example of framework synthesis examined here (on children's, young people's and parents' views of walking and cycling) involved policy makers and practitioners in directing the focus of the synthesis and used the themes derived from the synthesis to infer what kind of interventions might be most effective in encouraging walking and cycling. Likewise, the products of the thematic synthesis took the form of practical recommendations for interventions (e.g. 'do not promote fruit and vegetables in the same way in the same intervention'). The extent to which policy makers and practitioners are involved in informing either synthesis or recommendation is less clear from the documents published on ecological triangulation, but the aim certainly is to directly inform practice.

The outputs of synthesis methods which have a more constructivist orientation – meta-study, meta-narrative, meta-ethnography, grounded theory, critical interpretive synthesis – tend to look rather different. They are generally more complex and conceptual, sometimes operating on the symbolic or metaphorical level, and requiring a further process of interpretation by policy makers and practitioners in order for them to inform practice. This is not to say, however, that they are not useful for practice, more that they are doing different work. However, it may be that, in the absence of further interpretation, they are more useful for informing other researchers and theoreticians.

Looking across dimensions

After examining the dimensions of difference of our included methods, what picture ultimately emerges? It seems clear that, while similar in some respects, there are genuine differences in approach to the synthesis of what is essentially textual data. To some extent, these differences can be explained by the epistemological assumptions that underpin each method. Our methods split into two broad camps: the idealist and the realist (see Table 1 for a summary). Idealist approaches generally tend to have a more iterative approach to searching (and the review process), have less a priori quality assessment procedures and are more inclined to problematize the literature. Realist approaches are characterised by a more linear approach to searching and review, have clearer and more well-developed approaches to quality assessment, and do not problematize the literature.

Mapping the relationships between methods

What is interesting is the relationship between these methods of synthesis, the conceptual links between them, and the extent to which the originators cite – or, in some cases, don't cite – one another. Some methods directly build on others – framework synthesis builds on framework analysis, for example, while grounded theory and constant comparative analysis build on grounded theory. Others further develop existing methods – meta-study, critical interpretive synthesis and meta-narrative all adapt aspects of meta-ethnography, while also importing concepts from other theorists (critical interpretive synthesis also adapts grounded theory techniques).

Some methods share a clear conceptual link, without directly citing one another: for example, the analytical themes developed during thematic synthesis are comparable to the third order interpretations of meta-ethnography. The meta-theory aspect of meta-study is echoed in both meta-narrative synthesis and critical interpretive synthesis (see 'Problematizing the literature, above); however, the originators of critical interpretive synthesis only refer to the originators of meta-study in relation to their use of sampling techniques.

While methods for qualitative synthesis have many similarities, there are clear differences in approach between them, many of which can be explained by taking account of a given method's epistemology.

However, within the two broad idealist/realist categories, any differences between methods in terms of outputs appear to be small.

Since many systematic reviews are designed to inform policy and practice, it is important to select a method – or type of method – that will produce the kind of conclusions needed. However, it is acknowledged that this is not always simple or even possible to achieve in practice.

The approaches that result in more easily translatable messages for policy-makers and practitioners may appear to be more attractive than the others; but we do need to take account lessons from the more idealist end of the spectrum, that some perspectives are not universal.

Dixon-Woods M, Agarwhal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Pol. 2005, 10 (1): 45-53b. 10.1258/1355819052801804.

Article   Google Scholar  

Barbour RS, Barbour M: Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract. 2003, 9 (2): 179-186. 10.1046/j.1365-2753.2003.00371.x.

Article   PubMed   Google Scholar  

Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Pol. 2005, 10 (Suppl 1): 6-20. 10.1258/1355819054308576.

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Shaw RL, Smith J, Sutton A, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006, 6: 27-44. 10.1177/1468794106058867.

Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. 2007, Maidenhead: Open University Press

Google Scholar  

Thorne S, Jenson L, Kearney MH, Noblit G, Sandelowski M: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004, 14: 1342-1365. 10.1177/1049732304269888.

Centre for Reviews and Dissemination: Systematic Reviews. CRD's Guidance for Undertaking Reviews in Health Care. 2008, York: CRD

Noblit GW, Hare RD: Meta-Ethnography: Synthesizing Qualitative Studies. 1988, London: Sage

Book   Google Scholar  

Strike K, Posner G: Types of synthesis and their criteria. Knowledge Structure and Use. Edited by: Ward S, Reed L. 1983, Philadelphia: Temple University Press

Turner S: Sociological Explanation as Translation. 1980, New York: Cambridge University Press

Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesis qualitative research: a worked example. J Health Serv Res. 2002, 7: 209-15. 10.1258/135581902320432732.

Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003, 65: 671-84. 10.1016/S0277-9536(02)00064-3.

Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R: Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005, 61: 133-155. 10.1016/j.socscimed.2004.11.063.

Schutz A: Collected Paper. 1962, The Hague: Martinus Nijhoff, 1:

Sandelowski M, Barroso J: Handbook for Synthesizing Qualitative Research. 2007, New York: Springer Publishing Company

Kearney MH: Enduring love: a grounded formal theory of women's experience of domestic violence. Research Nurs Health. 2001, 24: 270-82. 10.1002/nur.1029.

Article   CAS   Google Scholar  

Eaves YD: A synthesis technique for grounded theory data analysis. J Adv Nurs. 2001, 35: 654-63. 10.1046/j.1365-2648.2001.01897.x.

Article   CAS   PubMed   Google Scholar  

Finfgeld D: Courage as a process of pushing beyond the struggle. Qual Health Res. 1999, 9: 803-814. 10.1177/104973299129122298.

Glaser BG, Strauss AL: The Discovery of Grounded Theory: Strategies for Qualitative Research. 1967, New York: Aldine De Gruyter

Strauss AL, Corbin J: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 1990, Newbury Park, CA: Sage

Strauss AL, Corbin J: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 1998, Thousand Oaks, CA: Sage

Charmaz K: The grounded theory method: an explication and interpretation. Contemporary Field Research: A Collection of Readings. Edited by: Emerson RM. 1983, Waveland Press: Prospect Heights, IL, 109-126.

Chesler MA: Professionals' Views of the Dangers of Self-Help Groups: Explicating a Grounded Theoretical Approach. 1987, [Michigan]: Department of Sociology, University of Michigan, Ann Arbour Centre for Research on Social Organisation, Working Paper Series

Kearney MH: Ready-to-wear: discovering grounded formal theory. Res Nurs Health. 1988, 21: 179-186. 10.1002/(SICI)1098-240X(199804)21:2<179::AID-NUR8>3.0.CO;2-G.

Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth. 2008, 8: 45-10.1186/1471-2288-8-45.

Lucas PJ, Arai L, Baird , Law C, Roberts HM: Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Meth. 2007, 7 (4):

Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community H. 2004, 58: 794-800. 10.1136/jech.2003.014829.

Paterson BL, Thorne SE, Canam C, Jillings C: Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. 2001, Thousand Oaks, CA: Sage Publications

Zhao S: Metatheory, metamethod, meta-data-analysis: what, why and how?. Sociol Perspect. 1991, 34: 377-390.

Ritzer G: Metatheorizing in Sociology. 1991, Lexington, MA: Lexington Books

CASP (Critical Appraisal Skills Programme). date unknown, [ http://www.phru.nhs.uk/Pages/PHD/CASP.htm ]

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005, 61: 417-30. 10.1016/j.socscimed.2004.12.001.

Kuhn TS: The Structure of Scientific Revolutions. 1962, Chicago: University of Chicago Press

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Meth. 2006, 6 (35):

Gough D: Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Applied and Practice-based Research. Edited by: Furlong J, Oancea A. 2007, Special Edition of Research Papers in Education, 22 (2): 213-228.

Webb EJ, Campbell DT, Schwartz RD, Sechrest L: Unobtrusive Measures. 1966, Chicago: Rand McNally

Denzin NK: The Research Act: a Theoretical Introduction to Sociological Methods. 1978, New York: McGraw-Hill

Banning J: Ecological Triangulation. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Triangualtion.pdf ]

Banning J: Ecological Sentence Synthesis. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Sentence%20Synthesis.pdf ]

Brunton G, Oliver S, Oliver K, Lorenc T: A Synthesis of Research Addressing Children's, Young People's and Parents' Views of Walking and Cycling for Transport. 2006, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London

Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley A, Gabbay J, Stein K, Buchanan P, Gyte G: A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008, 11: 72-84. 10.1111/j.1369-7625.2007.00476.x.

Pope C, Ziebland S, Mays N: Qualitative research in health care: analysing qualitative data. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ritchie J, Spencer L: Qualitative data analysis for applied policy research. Analysing Qualitative Data. Edited by: Bryman A, Burgess R. 1993, London: Routledge, 173-194.

Miles M, Huberman A: Qualitative Data Analysis. 1984, London: Sage

Evans D, Fitzgerald M: Reasons for physically restraining patients and residents: a systematic review and content analysis. Int J Nurs Stud. 2002, 39: 739-743. 10.1016/S0020-7489(02)00015-9.

Suikkala A, Leino-Kilpi H: Nursing student-patient relationships: a review of the literature from 1984–1998. J Adv Nurs. 2000, 33: 42-50. 10.1046/j.1365-2648.2001.01636.x.

Weed M: 'Meta-interpretation': a method for the interpretive synthesis of qualitative research. Forum: Qual Soc Res. 2005, 6: Art 37-

Gough D, Thomas J: Dimensions of difference in systematic reviews. [ http://www.ncrm.ac.uk/RMF2008/festival/programme/sys1 ]

Spencer L, Ritchie J, Lewis J, Dillon L: Quality in Qualitative Evaluation: a Framework for Assessing Research Evidence. 2003, London: Government Chief Social Researcher's Office

Banning J: Design and Implementation Assessment Device (DIAD) Version 0.3: A response from a qualitative perspective. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Design%20and%20Implementation%20Assessment%20Device.pdf ]

Paterson BL: Coming out as ill: understanding self-disclosure in chronic illness from a meta-synthesis of qualitative research. Reviewing Research Evidence for Nursing Practice. Edited by: Webb C, Roe B. 2007, [Oxford]: Blackwell Publishing Ltd, 73-83.

Chapter   Google Scholar  

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/9/59/prepub

Download references

Acknowledgements

The authors would like to acknowledge the helpful contributions of the following in commenting on earlier drafts of this paper: David Gough, Sandy Oliver, Angela Harden, Mary Dixon-Woods, Trisha Greenhalgh and Barbara L. Paterson. We would also like to thank the peer reviewers: Helen J Smith, Rosaline Barbour and Mark Rodgers for their helpful reviews. The methodological development was supported by the Department of Health (England) and the ESRC through the Methods for Research Synthesis Node of the National Centre for Research Methods (NCRM). An earlier draft of this paper currently appears as a working paper on the National Centre for Research Methods' website http://www.ncrm.ac.uk/ .

Author information

Authors and affiliations.

Social Science Research Unit, Evidence for Policy and Practice Information and Co-ordinating (EPPI-) Centre, 18 Woburn Square, London, WC1H 0NS, UK

Elaine Barnett-Page & James Thomas

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Elaine Barnett-Page .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

Both authors made substantial contributions, with EBP taking a lead on writing and JT on the analytical framework. Both authors read and approved the final manuscript.

Electronic supplementary material

12874_2009_375_moesm1_esm.doc.

Additional file 1: Dimensions of difference. Ranging from subjective idealism through objective idealism and critical realism to scientific realism to naïve realism (DOC 46 KB)

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Barnett-Page, E., Thomas, J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9 , 59 (2009). https://doi.org/10.1186/1471-2288-9-59

Download citation

Received : 09 March 2009

Accepted : 11 August 2009

Published : 11 August 2009

DOI : https://doi.org/10.1186/1471-2288-9-59

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Narrative Synthesis
• Theoretical Sampling
• Qualitative Synthesis
• Order Construct

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

How to Write Critical Reviews

When you are asked to write a critical review of a book or article, you will need to identify, summarize, and evaluate the ideas and information the author has presented. In other words, you will be examining another person’s thoughts on a topic from your point of view.

Your stand must go beyond your “gut reaction” to the work and be based on your knowledge (readings, lecture, experience) of the topic as well as on factors such as criteria stated in your assignment or discussed by you and your instructor.

Make your stand clear at the beginning of your review, in your evaluations of specific parts, and in your concluding commentary.

Remember that your goal should be to make a few key points about the book or article, not to discuss everything the author writes.

Understanding the Assignment

To write a good critical review, you will have to engage in the mental processes of analyzing (taking apart) the work–deciding what its major components are and determining how these parts (i.e., paragraphs, sections, or chapters) contribute to the work as a whole.

Analyzing the work will help you focus on how and why the author makes certain points and prevent you from merely summarizing what the author says. Assuming the role of an analytical reader will also help you to determine whether or not the author fulfills the stated purpose of the book or article and enhances your understanding or knowledge of a particular topic.

Be sure to read your assignment thoroughly before you read the article or book. Your instructor may have included specific guidelines for you to follow. Keeping these guidelines in mind as you read the article or book can really help you write your paper!

Also, note where the work connects with what you’ve studied in the course. You can make the most efficient use of your reading and notetaking time if you are an active reader; that is, keep relevant questions in mind and jot down page numbers as well as your responses to ideas that appear to be significant as you read.

Please note: The length of your introduction and overview, the number of points you choose to review, and the length of your conclusion should be proportionate to the page limit stated in your assignment and should reflect the complexity of the material being reviewed as well as the expectations of your reader.

Write the introduction

Below are a few guidelines to help you write the introduction to your critical review.

Introduce your review appropriately

Begin your review with an introduction appropriate to your assignment.

If your assignment asks you to review only one book and not to use outside sources, your introduction will focus on identifying the author, the title, the main topic or issue presented in the book, and the author’s purpose in writing the book.

If your assignment asks you to review the book as it relates to issues or themes discussed in the course, or to review two or more books on the same topic, your introduction must also encompass those expectations.

Explain relationships

For example, before you can review two books on a topic, you must explain to your reader in your introduction how they are related to one another.

Within this shared context (or under this “umbrella”) you can then review comparable aspects of both books, pointing out where the authors agree and differ.

In other words, the more complicated your assignment is, the more your introduction must accomplish.

Finally, the introduction to a book review is always the place for you to establish your position as the reviewer (your thesis about the author’s thesis).

As you write, consider the following questions:

• Is the book a memoir, a treatise, a collection of facts, an extended argument, etc.? Is the article a documentary, a write-up of primary research, a position paper, etc.?
• Who is the author? What does the preface or foreword tell you about the author’s purpose, background, and credentials? What is the author’s approach to the topic (as a journalist? a historian? a researcher?)?
• What is the main topic or problem addressed? How does the work relate to a discipline, to a profession, to a particular audience, or to other works on the topic?
• What is your critical evaluation of the work (your thesis)? Why have you taken that position? What criteria are you basing your position on?

Provide an overview

In your introduction, you will also want to provide an overview. An overview supplies your reader with certain general information not appropriate for including in the introduction but necessary to understanding the body of the review.

Generally, an overview describes your book’s division into chapters, sections, or points of discussion. An overview may also include background information about the topic, about your stand, or about the criteria you will use for evaluation.

The overview and the introduction work together to provide a comprehensive beginning for (a “springboard” into) your review.

• What are the author’s basic premises? What issues are raised, or what themes emerge? What situation (i.e., racism on college campuses) provides a basis for the author’s assertions?
• How informed is my reader? What background information is relevant to the entire book and should be placed here rather than in a body paragraph?

Write the body

The body is the center of your paper, where you draw out your main arguments. Below are some guidelines to help you write it.

Organize using a logical plan

Organize the body of your review according to a logical plan. Here are two options:

• First, summarize, in a series of paragraphs, those major points from the book that you plan to discuss; incorporating each major point into a topic sentence for a paragraph is an effective organizational strategy. Second, discuss and evaluate these points in a following group of paragraphs. (There are two dangers lurking in this pattern–you may allot too many paragraphs to summary and too few to evaluation, or you may re-summarize too many points from the book in your evaluation section.)
• Alternatively, you can summarize and evaluate the major points you have chosen from the book in a point-by-point schema. That means you will discuss and evaluate point one within the same paragraph (or in several if the point is significant and warrants extended discussion) before you summarize and evaluate point two, point three, etc., moving in a logical sequence from point to point to point. Here again, it is effective to use the topic sentence of each paragraph to identify the point from the book that you plan to summarize or evaluate.

Questions to keep in mind as you write

With either organizational pattern, consider the following questions:

• What are the author’s most important points? How do these relate to one another? (Make relationships clear by using transitions: “In contrast,” an equally strong argument,” “moreover,” “a final conclusion,” etc.).
• What types of evidence or information does the author present to support his or her points? Is this evidence convincing, controversial, factual, one-sided, etc.? (Consider the use of primary historical material, case studies, narratives, recent scientific findings, statistics.)
• Where does the author do a good job of conveying factual material as well as personal perspective? Where does the author fail to do so? If solutions to a problem are offered, are they believable, misguided, or promising?
• Which parts of the work (particular arguments, descriptions, chapters, etc.) are most effective and which parts are least effective? Why?
• Where (if at all) does the author convey personal prejudice, support illogical relationships, or present evidence out of its appropriate context?

Keep your opinions distinct and cite your sources

Remember, as you discuss the author’s major points, be sure to distinguish consistently between the author’s opinions and your own.

Keep the summary portions of your discussion concise, remembering that your task as a reviewer is to re-see the author’s work, not to re-tell it.

And, importantly, if you refer to ideas from other books and articles or from lecture and course materials, always document your sources, or else you might wander into the realm of plagiarism.

Include only that material which has relevance for your review and use direct quotations sparingly. The Writing Center has other handouts to help you paraphrase text and introduce quotations.

Write the conclusion

You will want to use the conclusion to state your overall critical evaluation.

You have already discussed the major points the author makes, examined how the author supports arguments, and evaluated the quality or effectiveness of specific aspects of the book or article.

Now you must make an evaluation of the work as a whole, determining such things as whether or not the author achieves the stated or implied purpose and if the work makes a significant contribution to an existing body of knowledge.

Consider the following questions:

• Is the work appropriately subjective or objective according to the author’s purpose?
• How well does the work maintain its stated or implied focus? Does the author present extraneous material? Does the author exclude or ignore relevant information?
• How well has the author achieved the overall purpose of the book or article? What contribution does the work make to an existing body of knowledge or to a specific group of readers? Can you justify the use of this work in a particular course?
• What is the most important final comment you wish to make about the book or article? Do you have any suggestions for the direction of future research in the area? What has reading this work done for you or demonstrated to you?

Academic and Professional Writing

This is an accordion element with a series of buttons that open and close related content panels.

Analysis Papers

Reading Poetry

A Short Guide to Close Reading for Literary Analysis

Using Literary Quotations

Play Reviews

Writing a Rhetorical Précis to Analyze Nonfiction Texts

Incorporating Interview Data

Grant Proposals

Planning and Writing a Grant Proposal: The Basics

Additional Resources for Grants and Proposal Writing

Job Materials and Application Essays

Writing Personal Statements for Ph.D. Programs

• Before you begin: useful tips for writing your essay
• Guided brainstorming exercises
• Get more help with your essay
• Frequently Asked Questions

Resume Writing Tips

CV Writing Tips

Cover Letters

Business Letters

Proposals and Dissertations

Resources for Proposal Writers

Resources for Dissertators

Research Papers

Planning and Writing Research Papers

Quoting and Paraphrasing

Writing Annotated Bibliographies

Creating Poster Presentations

Writing an Abstract for Your Research Paper

Thank-You Notes

Advice for Students Writing Thank-You Notes to Donors

Reading for a Review

Critical Reviews

Writing a Review of Literature

Scientific Reports

Scientific Report Format

Sample Lab Assignment

Writing for the Web

Writing an Effective Blog Post

Writing for Social Media: A Guide for Academics

PSY290 - Research Methods

• Identifying & Locating Empirical Research Articles
• Survey & Test Instruments

Writing a Critical Review

Sample summaries, verbs to help you write the summary, how to read a scholarly article.

• APA Citation Style Help

A critical review is an academic appraisal of an article that offers both a summary and critical comment. They are useful in evaluating the relevance of a source to your academic needs. They demonstrate that you have understood the text and that you can analyze the main arguments or findings. It is not just a summary; it is an evaluation of what the author has said on a topic. It’s critical in that you thoughtfully consider the validity and accuracy of the author’s claims and that you identify other valid points of view.

An effective critical review has three parts:

• APA citation of article
• Clearly summarizes the purpose for the article and identifies the strengths and weaknesses of the research. (In your own words – no quotations.)
• Evaluates the contribution of the article to the discipline or broad subject area and how it relates to your own research.

Steps to Write a Critical Review:

• Create and APA style citation for the article you are reviewing.
• Skim the text: Read the title, abstract, introduction, and conclusion.
• Read the entire article in order to identify its main ideas and purpose.

Q. What were the authors investigating? What is their thesis? Q. What did the authors hope to discover?

        D. Pay close attention to the methods used by the authors to collection information.

Q. What are the characteristics of the participants? (e.g.) Age/gender/ethnicity

Q. What was the procedure or experimental method/surveys used?

Q. Are their any flaws in the design of their study?

  E. Review the main findings in the “Discussion” or “Conclusion” section. This will help you to evaluate the validity of their evidence, and the credibility of the authors.             Q.   Are their conclusions convincing?            Q.   Were their results significant? If so, describe how they were significant.  F. Evaluate the usefulness of the text to YOU in the context of your own research.

Q. How does this article assist you in your research?

Q. How does it enhance your understanding of this issue?

Q. What gaps in your research does it fill?

Good Summary:

Hock, S., & Rochford, R. A. (2010). A letter-writing campaign: linking academic success and civic engagement. Journal  of Community Engagement and Scholarship, 3 (2), 76-82.

Hock & Rochford (2010) describe how two classes of developmental writing students were engaged in a service-learning project to support the preservation of an on-campus historical site. The goal of the assignment was to help students to see how they have influence in their community by acting as engaged citizens, and to improve their scores on the ACT Writing Sample Assessment (WSA) exam. The authors report that students in developmental classes often feel disempowered, especially when English is not their first language. This assignment not only assisted them in elevating their written communication skills, but it also gave real-life significance to the assignment, and by extension made them feel like empowered members of the community. The advancement in student scores serves as evidence to support my research that when students are given assignments which permit local advocacy and active participation, their academic performance also improves.

Bad Summary:

Two ELL classes complete a service-learning project and improve their writing scores. This article was good because it provided me with lots of information I can use. The students learned a lot in their service-learning project and they passed the ACT exam.  

Remember you're describing what someone else has said. Use verbal cues to make this clear to your reader.  Here are some suggested verbs to use: 

* Adapted from: http://www.laspositascollege.edu/raw/summaries.php

• << Previous: Survey & Test Instruments
• Next: APA Citation Style Help >>
• Last Updated: Jun 6, 2024 10:25 AM
• URL: https://paradisevalley.libguides.com/PSY290

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley Open Access Collection

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure  1 . A list of excluded studies with reasons can be found in Appendix B .

Study selection flowchart

3.1. Characteristics of included reviews

Table  1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n  = 3), literature search ( n  = 3), study selection ( n  = 2), data extraction ( n  = 1), and RoB assessment ( n  = 2) (Table  2 ).

Summary of findings from review evaluating systematic review methods

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n  = 4), Hopewell et al. 20 and Crumley et al. 19 ( n  = 30), and Robson et al. 14 and Morissette et al. 23 ( n  = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table  2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration  or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: May 22, 2024 8:22 PM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved June 11, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Publish with us
• About the journal
• Meet the editors
• Specialist reviews
• BMJ Journals

You are here

• Volume 3, Issue 1
• Assessing the methodological quality and risk of bias of systematic reviews: primer for authors of overviews of systematic reviews
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-7825-6765 Carole Lunny 1 , 2 ,
• Salmaan Kanji 3 , 4 ,
• Pierre Thabet 5 ,
• Anna-Bettina Haidich 6 ,
• Konstantinos I Bougioukas 6 and
• Dawid Pieper 7
• 1 Knowledge Translation Program, Li Ka Shing Knowledge Institute , St Michael's Hospital , Toronto , ON , Canada
• 2 Cochrane Hyptertension Review Group , Cochrane Canada , Vancouver , BC , Canada
• 3 Ottawa Hospital , Ottawa , ON , Canada
• 4 Ottawa Hospital Research Institute , Ottawa , ON , Canada
• 5 School of Pharmaceutical Sciences University of Ottawa , Hôpital Montfort , Ottawa , ON , Canada
• 6 Department of Hygiene, Social-Preventive Medicine and Medical Statistics , Aristotle University of Thessaloniki Faculty of Health Sciences , Thessaloniki , Central Macedonia , Greece
• 7 Faculty of Health Sciences Brandenburg , Brandenburg Medical School Theodor Fontane Ruppin Clinics , Neuruppin , Brandenburg , Germany
• Correspondence to Dr Carole Lunny, Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, Canada; carole.lunny{at}ubc.ca

https://doi.org/10.1136/bmjmed-2023-000604

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• Epidemiology
• Research design
• Public health

Key messages

Systematic reviews underpin evidence based healthcare decision making, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid recommendations

Overviews of reviews (also known as umbrella reviews, meta-reviews, or reviews of reviews) evaluate biases at the systematic review level, among others, but proper use of tools for this purpose require training, time, and an appreciation of their strengths and limitations

AMSTAR-2 and ROBIS are the two most popular and rigorous critical appraisal tools used for appraising systematic reviews

The AMSTAR-2 16-item checklist focuses on methodological quality of systematic reviews of healthcare interventions, and incorporates aspects of review conduct, reporting comprehensiveness, and risk of bias as specific items

ROBIS is a domain based tool with 19 items focusing on risk of biases in a systematic review (eg, selective reporting of outcomes or analyses) of healthcare interventions and contains items related to risk of bias in results and conclusions, relevance, and an item about risk of interpretation bias or ’spin’

Carole Lunny and colleagues consider methods such as AMSTAR-2 and ROBIS tools to evaluate the methodological quality and risk of bias of systematic reviews of intervention effects that are included in overviews of reviews

Introduction

Overviews of reviews, which synthesise the findings of systematic reviews, 1 have significantly increased in publication over the past decade. 2 However, the terminology used to describe them is not agreed in consensus, with terms such as umbrella reviews, meta-reviews, and reviews of reviews being used interchangeably to mean overviews of reviews. Methods research has been ongoing since the 2010s to develop effective approaches for conducting overviews of reviews and addressing their unique characteristics. 3–7 Overview authors use various approaches to assess the methodological quality and risk of bias in their included systematic reviews, and they apply these assessments to inform the overviews' results and conclusions. However, proper use of tools for this purpose require training, time, and an appreciation of their strengths and limitations. This methods primer aims to address the inconsistency in assessing and reporting bias in systematic reviews of intervention effects included within overviews, and focuses on presenting the different validated tools, comparing them, and providing guidance on the interpretation and reporting of these assessments.

Assessing the methodological quality and risk of bias

Assessment tools.

In 2016, ROBIS was developed to assess risk of bias in systematic reviews, 11 ROBIS consists of three phases: assessment of relevance (optional), identification of bias concerns with the review process, and judgement of the overall risk of bias in the review. The tool focuses on four domains: study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis and findings. ROBIS helps reviewers identify potential biases in these domains by asking specific questions related to the review's methods and reporting. The tool underwent content validity and reliability testing to ensure its accuracy and consistency in assessing the risk of bias in systematic reviews.

In 2017, an update to AMSTAR, called AMSTAR-2, 10 aimed to assess methodological quality of systematic reviews, and involved inter-rater reliability and usability testing. AMSTAR-2 consists of 16 items that evaluate various aspects of the systematic review process, including the research question formulation, study selection and data extraction, assessment of risk of bias in individual studies, consideration of publication bias, and appropriate statistical analysis. This tool also assesses the overall methodological quality and risk of bias in the review, providing a comprehensive evaluation.

The decision about how to evaluate overall risk of bias for ROBIS is made at the assessors' discretion, as opposed to the AMSTAR-2 overall judgement, which is prescribed by AMSTAR-2 guidance. Examples of how to interpret methodological quality and risk of bias assessments, and how to make an overall judgement are found in box 1 .

Decision rules: how to decide that the results of a review are of high quality or at low risk of bias overall

Decision rules are a priori strategies used to specify rules to define explicitly how each item is rated, as well as how an overall judgement is made about a specific systematic review with the AMSTAR-2 and ROBIS tools. In the case of AMSTAR-2, the authors who are using the tool stipulate how to come to an overall high quality rating in the results of the review, but not how to rate each item. For example, item 15 of AMSTAR-2 asks assessors whether an adequate investigation of publication bias (small study bias) was conducted and whether its likely effect on the results was discussed. However, the AMSTAR-2 team did not specify what happens when 10 studies or fewer were included (ie, the analysis will be underpowered to detect publication bias), what methods to detect publication bias are recommended, and if publication bias is detected, how it should be discussed (ie, as a systematic review limitation).

The ROBIS tool equally does not specify what decision rules should be used for assessment of risk of bias, nor how to come to an overall judgement. For example, item 4.6 of ROBIS ("Were biases in primary studies minimal or addressed in the synthesis?") is similar to item 12 of AMSTAR-2 ("If meta-analysis was performed, did the review authors assess the potential impact of risk of bias in individual studies on the results of the meta-analysis?"). Of note, risk of bias should be assessed in any systematic review regardless of whether a meta-analysis was performed. A possible decision rule for answering these two questions when considering whether bias was adressed and considered in the results and their interpretation could be to respond "Yes" or "Probably/Partial Yes" if:

All studies received a low risk of bias rating; and

Studies were judged at high risk of bias and sensitivity analyses (grouping high v low risk studies in a meta-analysis) or adjustment approaches were used

For a "No" response:

Important biases were suspected to have been in the included studies that have been ignored by the review authors; or

Risk of bias was not assessed at all in the included studies; or

Bias was assessed but authors did not incorporate it into findings, discussion, and conclusions

Based on the above decision rules, how would the following statement be rated? "We planned on conducting sensitivity analysis on the studies based on their level of risk of bias. Most of the included studies had a similar risk of bias across all the domains except for industry sponsorship bias and incomplete data for total testosterone. Due to the inadequate number of studies, we were not able to conduct a sensitivity analysis on the included studies based on industry sponsorship."

For overall judgements, a decision rule could be that if one or more ROBIS domains are at high risk of bias, then the overall study is deemed at high risk of bias. For AMSTAR-2, the authors of the tool have stipulated that the review is considered of low or critical low quality when any of the subset of seven ‘critical’ items have one or more critical flaws. While the decisions about how to rate the items and make overall judgements can be debated, the grounds on which overview authors make these decisions should be noted explicitly in the manuscript or in an appendix, as then the assessment results will be transparent and reproducible.

Cautionary note: empirical evidence does not currently support the assignment of scores to items that are met in a risk of bias tool followed by the summation or averaging of these scores to produce a numerical measure of risk of bias. A thoughtful, nuanced, and customised overall judgement is required that considers all items with suspected bias on the basis of specific context.

The AMSTAR-2 and ROBIS tools were designed to assess systematic reviews with pairwise meta-analysis only. A more recent tool under development aims to assess the potential biases and limitations in network meta-analyses. 12 13 Guidance documents (eg, Cochrane 14 and JBI 15 ) recommend overview authors use ROBIS or AMSTAR-2 when comparing and critically appraising systematic reviews over other available tools. Figure 1 presents two example assessments conducted by our team, the ROBIS assessment of Normansell and colleagues 16 is presented at the domain level, and the AMSTAR-2 assessment of Puig and colleagues 17 is presented by item. Items are backed by quotes and rationales to support the answers chosen, for full transparency, and to help when comparing assessments between two independent assessors ( figure 2 ).

• Download figure
• Open in new tab
• Download powerpoint

Example assessments using ROBIS of Normansell 16 and AMSTAR-2 of Puig 17 . The ROBIS assessment is presented by domain and the AMSTAR-2 assessment by individual items. ROBIS's phase one, where the assessor considers the relevance of the systematic review questions to the overview's question, is not shown. The decision about how to evaluate overall risk of bias for ROBIS is made at the assessors' discretion, as opposed to the AMSTAR-2 overall judgement, which is prescribed by AMSTAR-2 guidance

PICO framework stands for patient or problem, intervention or exposure, comparison or control, and outcomes. DLQI=dermatology life quality index; DMARDs=disease modifying anti-rheumatic drugs; PASI=psoriasis area and severity index; RCT=randomised controlled trial

Comparison of AMSTAR-2 and ROBIS

Both the AMSTAR-2 and ROBIS tools provide structured guidelines for reviewers to evaluate and report on methodological strengths and weaknesses as well as potential biases in systematic reviews, contributing to the overall reliability and credibility of the evidence presented.Considerable overlap exists between the items of the two tools ( figure 1 ). In the documentation for each tool, AMSTAR-2 states that it was developed for systematic reviews of healthcare interventions whereas ROBIS states that it is aimed at reviews of healthcare interventions, diagnosis, prognosis, and biological cause. In practice, the ROBIS tool is generic and its signalling questions relate to interventions in the clinical or public health fields. Questions specific to systematic reviews of diagnosis, prognosis, and biological cause are not found in the tool. AMSTAR-2 was developed to assess methodological quality (which includes indicators of risk of bias) while ROBIS was developed primarily to assess risk of bias but also includes items that address methodological quality.

AMSTAR-2 focuses more on reporting comprehensiveness (eg, reporting of study designs for inclusion and reporting on excluded studies with justification) and methodological quality or transparency constructs (eg, pre-established protocol, sources of funding of primary studies, and reviewers' competing interests). Whereas ROBIS focuses on items related to identification of the different biases (eg, selective reporting of outcomes or analyses and publication bias). Bias occurs when factors systematically affect the results and conclusions of a review and cause them to be systematically different from the truth. 1 Systematic reviews affected by bias can be inaccurate; for example, finding false positive or false negative intervention effects by systematically over or under estimating the true effect in the target population. Methodological quality focuses on methodological features associated with internal validity. In theory, assessing risk of bias is the preferred approach because a review might have good methodological quality while still being at high risk of bias. For example, a systematic review might have been conducted according to stated guidance, but some relevant databases were not searched for evidence (database selection bias) leaving out crucial primary studies that may affect the results of the review.

In general, assessors found that AMSTAR-2 was more straightforward and user friendly than ROBIS. 18 19 The two tools had similar inter-rater reliability. 18 20 21 The range in time taken to use AMSTAR-2 was similar to ROBIS (14-60 v 16-60 min) across three comparison studies 18 20 21 ( table 1 ). ROBIS users required training and practice in using the tool 22 23 and it was often understood and applied differently. 20 AMSTAR-2 has been criticised for unclear guidance on some items, 24–26 which can lead to varying interpretations and applications. ROBIS is accompanied by voluminous guidance, which can be difficult to manage by the user. 21–23

• View inline

Comparison of the AMSTAR-2 and ROBIS tools

While AMSTAR-2 and ROBIS are both widely used tools for assessing systematic reviews, in some situations, one may be preferred over the other. AMSTAR-2 may be preferred when:

the primary focus is evaluating the methodological quality of a systematic review of interventions;

the aim is to broadly assess aspects of review conduct, reporting comprehensiveness, and risk of bias; or

a relatively quick and easy to use tool is sought, because AMSTAR-2 has fewer items compared with ROBIS.

ROBIS may be preferred when:

the aim is to identify concerns with the review conduct that may point to risk of biases in the results and conclusions, as well as assessing relevance and minimising interpretation bias or ‘spin’;

a more nuanced tool is sought, which may involve more thoughtful assessment and time, because ROBIS contains more items compared with AMSTAR-2;

the aim is to assess multiple types of systematic reviews to compare risk of bias across them (eg, when preparing a clinical practice guideline).

Reporting and interpretation

When reporting and interpreting the overview results, assessors should note some key considerations with AMSTAR-2 and ROBIS assessments. Authors should first report methodological quality or bias assessment results by item, domain, and overall judgement. In addition, assessment should be reported at the outcome level as opposed to the systematic review level. 18 Several responses to AMSTAR-2 item 13 (whether risk of bias was discussed or interpreted) are possible when multiple outcomes (eg, mortality and adverse events) are reported in one systematic review. Ideally, results of intervention overviews should be reported by qualifying the inherent methodological quality or risk of bias in the included systematic reviews as potential limitations.

Subgrouping systematic reviews by low and high risk of bias using ROBIS can be a great way to determine whether authors of reviews of interventions that have a high risk of bias over emphasised their findings and conclusions. Subgrouping also allows overview authors to exclude systematic reviews that are at a high risk of bias from the synthesis. However, using only one single criteria (ie, the systematic reviews at low risk of bias) for inclusion in analyses can result in unintended loss of information through exclusion of important systematic review data (eg, by excluding the systematic review with the greatest number of unique trials).

Conclusions

Overviews are used by guideline developers and policy makers to summarise large bodies of evidence in consideration of interventions of interest on a given topic. Using the appropriate tools to critically appraise included systematic reviews of intervention effects means that a complete assessment of methodological quality and all the potential biases are considered. Systematic reviews vary considerably by method, how data are synthesised, and how results and conclusions are reported, therefore. an assessment of potential biases is necessary to consider their reproducibility, trustworthiness, and usefulness for end users. At this time, the recommended tools to assess methodological quality and bias among systematic reviews included in overviews are AMSTAR-2 and ROBIS. Proper use of these tools for this purpose requires training, time, and methodological insight.

• Higgins JP ,
• Chandler J , et al
• Neelakant T ,
• Chen A , et al
• Hartling L ,
• Chisholm A ,
• Thomson D , et al
• Pollock A ,
• Campbell P , et al
• Buechter R ,
• Jerinic P , et al
• Begley CM , et al
• Thomson D ,
• Russell K ,
• Becker L , et al
• Brennan SE ,
• McDonald S , et al
• Whiting P ,
• Savović J , et al
• Reeves BC ,
• Wells G , et al
• Savović J ,
• Higgins JPT , et al
• Veroniki A-A ,
• Veroniki AA ,
• Hutton B , et al
• Pollock M ,
• Fernandes RM ,
• Becker LA , et al
• Aromataris E ,
• Fernandez RS ,
• Godfrey C , et al
• Normansell R ,
• Waterson S , et al
• Mollon P , et al
• Whitmarsh A ,
• Leach V , et al
• Eikermann M
• Duarte G , et al
• González-Lorenzo M , et al
• Cinquini M ,
• Gonzalez-Lorenzo M , et al
• Prengel P , et al
• Holmer HK ,
• Wegewitz U ,
• Weikert B ,
• Fishta A , et al

X @carole_lunny, @PierreThabet, @Bugiukas

Contributors CL conceived the idea of the study and drafted the manuscript; CL, PS, DW, KB, SK, and ABH edited the manuscript and read and approved the final manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests We have read and understood the BMJ policy on declaration of interests and declare the following interests: none.

Provenance and peer review Commissioned; externally peer reviewed.

Read the full text or download the PDF:

• Open access
• Published: 30 May 2024

Barriers and facilitators to mental health treatment access and engagement for LGBTQA+ people with psychosis: a scoping review protocol

• Cláudia C. Gonçalves   ORCID: orcid.org/0000-0001-6767-0920 1 ,
• Zoe Waters 2 ,
• Shae E. Quirk 1 ,
• Peter M. Haddad 1 , 3 ,
• Ashleigh Lin 4 ,
• Lana J. Williams 1 &
• Alison R. Yung 1 , 5  

Systematic Reviews volume  13 , Article number:  143 ( 2024 ) Cite this article

262 Accesses

Metrics details

The prevalence of psychosis has been shown to be disproportionately high amongst sexual and gender minority individuals. However, there is currently little consideration of the unique needs of this population in mental health treatment, with LGBTQA+ individuals facing barriers in accessing timely and non-stigmatising support for psychotic experiences. This issue deserves attention as delays to help-seeking and poor engagement with treatment predict worsened clinical and functional outcomes for people with psychosis. The present protocol describes the methodology for a scoping review which will aim to identify barriers and facilitators faced by LGBTQA+ individuals across the psychosis spectrum in help-seeking and accessing mental health support.

A comprehensive search strategy will be used to search Medline, PsycINFO, Embase, Scopus, LGBTQ+ Source, and grey literature. Original studies of any design, setting, and publication date will be included if they discuss barriers and facilitators to mental health treatment access and engagement for LGBTQA+ people with experiences of psychosis. Two reviewers will independently screen titles/abstracts and full-text articles for inclusion in the review. Both reviewers will then extract the relevant data according to pre-determined criteria, and study quality will be assessed using the Joanna Briggs Institute (JBI) critical appraisal checklists. Key data from included studies will be synthesised in narrative form according to the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews.

The results of this review will provide a comprehensive account of the current and historical barriers and facilitators to mental healthcare faced by LGBTQA+ people with psychotic symptoms and experiences. It is anticipated that the findings from this review will be relevant to clinical and community services and inform future research. Findings will be disseminated through publication in a peer-reviewed journal and presented at conferences.

Scoping review registration

This protocol is registered in Open Science Framework Registries ( https://doi.org/10.17605/OSF.IO/AT6FC ).

Peer Review reports

The prevalence of psychotic disorders in the general population has been estimated to be around 0.27–0.75% [ 1 , 2 ], with the lifetime prevalence of ever having a psychotic experience being estimated at 5.8% [ 3 ]. However, rates of psychotic symptoms and experiences are disproportionately high amongst LGBTQA+ populations, with non-heterosexual individuals estimated to be 1.99–3.75 times more likely to experience psychosis than their heterosexual peers [ 4 , 5 , 6 , 7 ]. Additionally, it has been estimated that transgender or gender non-conforming (henceforth trans) individuals are 2.46–49.7 times more likely than their cisgender peers (i.e. individuals whose gender identity is the same as their birth registered sex) to receive a psychotic disorder diagnosis [ 8 , 9 ]. The increased rates of psychotic experiences noted amongst gender and sexual minorities may be explained by evidence indicating that LGBTQA+ people are also exposed to risk factors for psychosis at a far greater rate than members of the general population, such as childhood adversity [ 10 , 11 , 12 ], minority stress [ 13 ], discrimination [ 14 ], and stigma [ 15 , 16 ]. Furthermore, there is added potential for diagnostic biases leading to over-diagnosing psychosis in gender diverse individuals, whose gender expression and dysphoria may be pathologized by mental health service providers [ 8 ].

Despite these concerning statistics, there is very little research examining the experiences of LGBTQA+ people with psychosis, and limited consideration of the unique needs these individuals may have in accessing and engaging with mental health services. While timely access to treatment has consistently been associated with better symptomatic and functional outcomes for people with psychosis [ 17 , 18 ], there are often delays to treatment initiation which are worsened for LGBTQA+ individuals [ 19 , 20 ]. These individuals face additional barriers to accessing adequate mental health support compared to cisgender/heterosexual people [ 19 ] and may need to experiment with several mental health services before finding culturally competent care [ 20 ]. This in turn may lead to longer duration of untreated psychosis. Additionally, there seems to be a lack of targeted support for this population from healthcare providers, with LGBTQA+ individuals with serious mental health concerns reporting higher rates of dissatisfaction with psychiatric services than their cisgender and heterosexual counterparts [ 7 , 14 , 21 ]. However, the extent of these differences varies across contexts [ 22 ], potentially due to improved education around stigma and LGBTQA+ issues within a subset of mental health services.

Nonetheless, stigma remains one of the highest cited barriers to help-seeking for mental health problems, particularly with regard to concerns around disclosure [ 23 ], which can be particularly challenging for people experiencing psychosis [ 24 , 25 ]. Stigma stress in young people at risk for psychosis is associated with less positive attitudes towards help-seeking regarding both psychiatric medication and psychotherapy [ 26 ], potentially partly due to fears of judgement and being treated differently by service providers [ 27 ]. This issue may be compounded for people who also belong to minoritized groups [ 23 , 28 ], particularly as LGBTQA+ individuals have reported experiencing frequent stigma and encountering uninformed staff when accessing mental healthcare [ 7 , 29 ]. Furthermore, stigma-fuelled hesitance to access services may be heightened for trans people [ 30 ] whose identities have historically been pathologized and conflated with experiences of psychosis [ 31 ].

Even when individuals manage to overcome barriers to access support, there are added challenges to maintaining adequate treatment engagement. In a large online study, half of trans and nearly one third of LGB participants reported having stopped using mental health services in the past because of negative experiences related to their gender identity or sexuality [ 20 ]. This can be particularly problematic as experiences of stigma predict poorer medication adherence in psychosis [ 32 ] which subsequently multiplies the risk for relapse and suicide [ 33 ]. While no research to date has explored non-adherence rates in people with psychosis who are LGBTQA+, concerns around suicidality are heightened for individuals who are gender and sexuality diverse [ 34 , 35 , 36 ].

Generally, there is rising demand for mental healthcare that specifically addresses the needs of gender and sexual minority individuals and promotes respect for diversity, equity, and inclusion [ 29 , 37 ]. This is particularly salient as positive relationships with staff are associated with better medication adherence for people with psychosis [ 38 ] and healthcare providers with LGBTQA+-specific mandates have demonstrated higher satisfaction rates for LGBTQA+ individuals [ 20 ]. Mental health services need to adapt treatment options to acknowledge minority stress factors for those with stigmatised identities and, perhaps more importantly, how these intersect and interact to increase inequalities in people from minoritized groups accessing and benefiting from treatment [ 37 , 39 ].

Additionally, gender affirming care needs to be recognised as an important facet of mental health treatment for many trans individuals, as it is associated with positive outcomes such as improvements in quality of life and psychological functioning [ 40 , 41 , 42 ] and reductions in psychiatric symptom severity and need for subsequent mental health treatment [ 8 , 43 ]. While there are additional barriers in access to gender affirming care for individuals with psychosis, this treatment has shown success in parallel with treatment to address psychosis symptom stabilisation [ 19 , 44 ]. The importance of affirmation is echoed by the finding that many negative experiences of LGBTQA+ participants with mental health services could be avoided simply by respecting people’s pronouns and using gender-neutral language [ 20 ].

To ensure timely access to appropriate treatment for LGBTQA+ people with psychosis, there is a need for improved understanding of the factors which challenge and facilitate help-seeking and engagement with mental health support. A preliminary search of Google Scholar, Medline, the Cochrane Database of Systematic Reviews, and PROSPERO was conducted and revealed no existing or planned reviews exploring benefits and/or obstacles to mental health treatment specific to this population. Therefore, the proposed review seeks to comprehensively search and appraise the existing literature to identify and summarise a range of barriers and facilitators to adequate mental health support faced by LGBTQA+ people with experiences of psychosis. This will allow for the mapping of the types of evidence available and identification of any knowledge gaps. Moreover, we hope to guide future decision-making in mental healthcare to improve service accessibility for LGBTQA+ individuals with psychosis and to set the foundations for future research that centres this marginalised population. Based on published guidance [ 45 , 46 , 47 ], a scoping review methodology was identified as the most appropriate approach to address these aims.

Selection criteria

This scoping review protocol has been developed in compliance with the JBI Manual for Evidence Synthesis [ 48 ] and, where relevant, the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) checklist [ 49 ] (see Additional file 1). In the event of protocol amendments, the date, justification, and description for each amendment will be provided.

Due to the limited literature around the topic of this review, any primary original study design, setting, and publication date will be considered for inclusion. Publications written in English will be included, and articles in other languages may be considered pending time and cost constraints around translation. Publications will be excluded if the full text is not available upon request from authors.

The PCC (Population, Concept, Context) framework was used to develop the inclusion criteria for this scoping review:

This review will include individuals of any age who are LGBTQA+ and have had experiences of psychosis. For the purposes of this review, ‘LGBTQA+ individuals’ will be broadly defined as any individual that is not heterosexual and/or cisgender or anyone who engages in same-gender sexual behaviour. Studies may include participants who are cisgender and heterosexual if they separately report outcomes for LGBTQA+ individuals. Within this review, the term ‘psychosis’ includes (i) any diagnosis of a psychotic disorder, such as schizophrenia spectrum disorders, mood disorders with psychotic features, delusional disorders, and drug-induced psychotic disorders, (ii) sub-threshold psychotic symptoms, such as those present in ultra-high risk (UHR), clinical high risk (CHR), or at risk mental state (ARMS) individuals, and (iii) any psychotic-like symptoms or experiences. Studies may include participants with multiple diagnoses if they separately report outcomes for individuals on the psychosis spectrum.

This review will include publications which discuss potential barriers and/or facilitators to mental health help-seeking and/or engagement with mental health treatment. ‘Barriers’ will be operationalised as any factors which may delay or prevent individuals from accessing and engaging with appropriate mental health support. These may include lack of mental health education, experienced or internalised stigma, experiences of discrimination from health services, and lack of inclusivity in health services. ‘Facilitators’ will be operationalised as any factors which may promote timely help-seeking and engagement with sources of support. These may include improved access to mental health education, positive sources of social support, and welcoming and inclusive services. Mental health help-seeking will be broadly defined as any attempt to seek and access formal or informal support to address a mental health concern related to experiences of psychosis (e.g. making an initial appointment with a service provider, seeking help from a friend). Mental health treatment engagement will be broadly defined as adherence and active participation in the treatment that is offered by a source of support (e.g. attending scheduled appointments, taking medication as prescribed, openly communicating with service providers).

This review may include research encompassing any setting in which mental healthcare is provided. This is likely to include formal healthcare settings such as community mental health teams or inpatient clinics as well as informal settings such as LGBTQA+ spaces or informal peer support. Studies will be excluded if they focus exclusively on physical health treatment.

Search strategy

Database searches will be conducted in Medline, PsycINFO, Embase, Scopus, and LGBTQ+ Source. The full search strategy for this protocol is available (see Additional file 2). This strategy has been collaboratively developed and evaluated by a scholarly services health librarian. Searches will include subject headings relevant to each database and title/abstract keywords relating to three main concepts: (i) LGBTQA+ identity, (ii) experiences of psychosis, and (iii) mental health treatment. Keywords for each concept will be combined using the Boolean operator ‘OR’, and the three concepts will be combined using ‘AND’. This search strategy was appropriately translated for each of the selected databases. There will be no limitations on language or publication date at this stage to maximise the breadth of the literature captured. Publications returned from these searches will be exported to EndNote. Searches will be re-run prior to the final analysis to capture any newly published studies.

The database searches will be supplemented by searching the grey literature as per the eligibility criteria detailed above. These may include theses and dissertations, conference proceedings, reports from mental health services, and policy documents from LGBTQA+ groups. Google and Google Scholar will be searched using a combination of clauses for psychosis (Psychosis OR psychotic OR schizophrenia OR schizoaffective), treatment (treatment or “help-seeking”), and queer identity. The latter concept will have three clauses for three separate searches, with one including broad queer identity (LGBT), one specific to non-heterosexual individuals (gay OR lesbian OR homosexual OR bisexual OR queer OR asexual), and one specific to trans individuals (transgender OR transsexual OR transexual OR “non-binary” OR “gender minority”). Additionally, reference lists and citing literature will be manually searched for each paper included in the review to capture any articles and policy documents not previously identified.

Data selection

Search results will be imported into Covidence using EndNote, and duplicates will be eliminated. Titles and abstracts will be screened by the first and second authors according to pre-defined screening criteria, which will be discussed by the authors and piloted prior to screening. These criteria will consider whether the articles included LGBTQA+ participants with experiences of psychosis (as operationalised above) in relation to mental health help-seeking and/or treatment. Full texts of relevant articles will then be obtained and screened by the first and second reviewer in accordance with the full inclusion and exclusion criteria after initial piloting to maximise inter-rater reliability. Decisions on inclusion and exclusion will be blinded and recorded on Covidence. Potential discrepancies will be resolved through discussion, and when consensus cannot be reached, these will be resolved by the supervising author. The process of study selection will be documented using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram [ 50 ].

Data extraction

Data extraction will be performed independently by two reviewers using Covidence. Prior to beginning final extraction, both reviewers will independently pilot the extraction tool using a sample of five included studies and discuss any necessary changes. Information extracted is planned to include the following: title, author name(s), year of publication, country in which the study was conducted, study design, sample size, population of focus (i.e. sexual minorities, gender minorities, or both), sample demographics (i.e. age, gender identity, and sexual orientation), setting (e.g. early intervention service, community mental health team, etc.), psychosis characteristics (e.g. diagnoses included, severity of symptoms, etc.), type of treatment (e.g. cognitive behavioural therapy, antipsychotic medication, etc.), and any barriers and/or facilitators identified according to the aforementioned operationalised definitions. Disagreements will be resolved through discussion between the two reviewers and, when necessary, final decisions will be made by a senior supervisor. Once extracted, information will be recorded in Excel. Lead authors of papers will be contacted by the primary review author in cases where there is missing or insufficient data.

Quality assessment

Due to the expected heterogeneity in the types of studies that may be included in this review (e.g. qualitative studies, randomised controlled trials, case control studies, case reports), the relevant revised Joanna Briggs Institute (JBI) critical appraisal checklists [ 51 ] will be used to assess risk of bias and study quality for each study design. Two reviewers will independently use these checklists to assess each paper that is included following the full-text screening. If there are discrepancies in article ratings, these will be resolved through discussion between the two authors. If no consensus is reached, discrepancies will be resolved by a senior supervisor. In line with the scoping nature of this review, low-quality studies will not be excluded from the synthesis.

Evidence synthesis

Data from included studies will be synthesised using a narrative synthesis approach in accordance with the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [ 52 ]. A preliminary descriptive synthesis will be conducted by tabulating the extracted data elements from each study alongside quality assessment results and developing an initial description of the barriers and facilitators to (1) accessing and (2) engaging with mental health support that are identified in the literature. This initial synthesis will then be interrogated and refined to contextualise these barriers and facilitators in the setting, population, and methodology of each study to form the basis for an interpretative synthesis.

This review will not use a pre-existing thematic framework to categorise barriers and facilitators as it is expected that the factors identified will not neatly fit into existing criteria. Instead, these will be conceptualised according to overarching themes as interrelated factors, so that potentially complex interactions between barriers and facilitators within and across relevant studies may be explored through concept mapping. If most of the studies included are qualitative, there may also be scope for a partial meta-synthesis. To avoid oversimplifying the concept of ‘barriers and facilitators’ (see criticism by Bach-Mortensen & Verboom [ 53 ]), this data synthesis will be followed by a critical reflection of the findings through the lens of the socio-political contexts which may give rise to the barriers and facilitators identified, exploring the complexities necessary for any changes to be implemented in mental health services.

If the extracted data indicate that gender minority and sexual minority individuals experience unique or different barriers and/or facilitators to each other, these population groups will be analysed separately as opposed to findings being generalised across the LGBTQA+ spectrum. Furthermore, if there is scope to do so, analyses may be conducted to investigate how perceived barriers and facilitators for this population may have changed over time (i.e. according to publication date) as definitions of psychosis evolve and LGBTQA+ individuals gain visibility in clinical services.

The proposed review will add to the literature around mental health treatment for LGBTQA+ people with psychosis. It will provide a thorough account of the barriers and facilitators to accessing and engaging with support faced by this population and may inform future research and clinical practice.

In terms of limitations, this review will be constrained by the existing literature and may therefore not be sufficiently comprehensive in reflecting the barriers and facilitators experienced by subgroups within the broader LGBTQA+ community. Additionally, although broad inclusion criteria are necessary to capture the full breadth of research conducted in this topic, included studies are likely to be heterogeneous and varied in terms of their methodology and population which may complicate data synthesis.

Nonetheless, it is anticipated that the findings from this review will provide the most comprehensive synthesis to date of the issues driving low help-seeking and treatment engagement in people across the psychosis spectrum who are LGBTQA+. This review will likely also identify gaps in the literature which may inform avenues for future research, and the factors identified in this review will be considered in subsequent research by the authors.

Additionally, findings will be relevant to healthcare providers that offer support to people with psychosis who may have intersecting LGBTQA+ identities as well as LGBTQA+ organisations which offer support to LGBTQA+ people who may be experiencing distressing psychotic experiences. These services are likely to benefit from an increased awareness of the factors which may improve or hinder accessibility for these subsets of their target populations. Therefore, results from this review may inform decision-making around the implementation of service-wide policy changes.

The findings of this review will be disseminated through the publication of an article in a peer-reviewed journal and presented at relevant conferences in Australia and/or internationally. Additionally, the completed review will form part of the lead author’s doctoral thesis.

Availability of data and materials

Not applicable for this protocol.

Abbreviations

• At risk mental state

Clinical high risk for psychosis

Joanna Briggs Institute

Lesbian, gay, and bisexual

Lesbian, gay, bisexual, transgender, queer or questioning, asexual or aromantic, and more

Population, Concept, Context

Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols

Ultra-high risk for psychosis

Institute for Health Metrics and Evaluation (IHME). Global Burden of Disease Study (GBD) Results. Seattle, WA: IHME, University of Washington. 2020. https://vizhub.healthdata.org/gbd-results/ . Accessed 26 May 2023

Moreno-Kustner B, Martin C, Pastor L. Prevalence of psychotic disorders and its association with methodological issues. A systematic review and meta-analyses. PLoS One. 2018;13(4):e0195687. https://doi.org/10.1371/journal.pone.0195687 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

McGrath JJ, Saha S, Al-Hamzawi A, Alonso J, Bromet EJ, Bruffaerts R, et al. Psychotic experiences in the general population: a cross-national analysis based on 31 261 respondents from 18 countries. JAMA Psychiatry. 2015;72(7):697–705. https://doi.org/10.1001/jamapsychiatry.2015.0575 .

Article   PubMed   PubMed Central   Google Scholar  

Chakraborty A, McManus S, Brugha TS, Bebbington P, King M. Mental health of the non-heterosexual population of England. Br J Psychiatry. 2011;198(2):143–8. https://doi.org/10.1192/bjp.bp.110.082271 .

Article   PubMed   Google Scholar  

Gevonden M, Selten J, Myin-Germeys I, De Graaf R, Ten Have M, Van Dorsselaer S, et al. Sexual minority status and psychotic symptoms: findings from the Netherlands Mental Health Survey and Incidence Studies (NEMESIS). Psychol Med. 2014;44(2):421–33. https://doi.org/10.1017/S0033291713000718 .

Article   CAS   PubMed   Google Scholar  

Jacob L, Smith L, McDermott D, Haro JM, Stickley A, Koyanagi A. Relationship between sexual orientation and psychotic experiences in the general population in England. Psychol Med. 2021;51(1):138–46. https://doi.org/10.1017/S003329171900309X .

Welch S, Collings SCD, Howden-Chapman P. Lesbians in New Zealand: their mental health and satisfaction with mental health services. Aust N Z J Psychiatry. 2000;34(2):256–63. https://doi.org/10.1080/j.1440-1614.2000.00710.x .

Barr SM, Roberts D, Thakkar KN. Psychosis in transgender and gender non-conforming individuals: a review of the literature and a call for more research. Psychiatry Res. 2021;306:114272. https://doi.org/10.1016/j.psychres.2021.114272 .

Hanna B, Desai R, Parekh T, Guirguis E, Kumar G, Sachdeva R. Psychiatric disorders in the U.S. transgender population. Ann Epidemiol. 2019;39:1–7. https://doi.org/10.1016/j.annepidem.2019.09.009 .

Schneeberger AR, Dietl MF, Muenzenmaier KH, Huber CG, Lang UE. Stressful childhood experiences and health outcomes in sexual minority populations: a systematic review. Soc Psychiatry Psychiatr Epidemiol. 2014;49:1427–45. https://doi.org/10.1007/s00127-014-0854-8 .

Stanton KJ, Denietolis B, Goodwin BJ, Dvir Y. Childhood trauma and psychosis: an updated review. Child Adolesc Psychiatr Clin N Am. 2020;29(1):115–29. https://doi.org/10.1016/j.chc.2019.08.004 .

Varese F, Smeets F, Drukker M, Lieverse R, Lataster T, Viechtbauer W, et al. Childhood adversities increase the risk of psychosis: a meta-analysis of patient-control, prospective- and cross-sectional cohort studies. Schizophr Bull. 2012;38(4):661–71. https://doi.org/10.1093/schbul/sbs050 .

Mongelli F, Perrone D, Balducci J, Saccheti A, Ferrari S, Mattei G, et al. Minority stress and mental health among LGBT populations: an update on the evidence. Minerva Psichiatr. 2019;60(1):27–50. https://doi.org/10.23736/S0391-1772.18.01995-7 .

Article   Google Scholar  

Kidd SA, Howison M, Pilling M, Ross LE, McKenzie K. Severe mental illness in LGBT populations: a scoping review. Psychiatr Serv. 2016;67(7):779–83. https://doi.org/10.1176/appi.ps.201500209 .

Hatzenbuehler ML, Pachankis JE. Stigma and minority stress as social determinants of health among lesbian, gay, bisexual, and transgender youth. Pediatr Clin North Am. 2016;63(6):985–97. https://doi.org/10.1016/j.pcl.2016.07.003 .

Rüsch N, Heekeren K, Theodoridou A, Müller M, Corrigan PW, Mayer B, et al. Stigma as a stressor and transition to schizophrenia after one year among young people at risk of psychosis. Schizophr Res. 2015;166(1–3):43–8. https://doi.org/10.1016/j.schres.2015.05.027 .

Howes OD, Whitehurst T, Shatalina E, Townsend L, Onwordi EC, Mak TLA, et al. The clinical significance of duration of untreated psychosis: an umbrella review and random-effects meta-analysis. World Psychiatry. 2021;20(1):75–95. https://doi.org/10.1002/wps.20822 .

McGorry PD. Early intervention in psychosis: obvious, effective, overdue. J Nerv Ment Dis. 2015;203(5):310–308. https://doi.org/10.1097/NMD.0000000000000284 .

Peta JL. The Oxford Handbook of Sexual and Gender Minority Mental Health. Oxford (GB): Oxford University Press; 2020. Chapter 11, Schizophrenia spectrum and other psychotic disorders among sexual and gender minority populations; 125-134. Available from: The Oxford Handbook of Sexual and Gender Minority Mental Health - Google Books. Accessed 01 Jun 2023

Simeonov D, Steele LS, Anderson S, Ross LE. Perceived satisfaction with mental health services in the lesbian, gay, bisexual, transgender, and transsexual communities in Ontario, Canada: an internet-based survey. Can J Commun Ment Health. 2015;34(1):31–44. https://doi.org/10.7870/cjcmh-2014-037 .

Avery AM, Hellman RE, Sudderth LK. Satisfaction with mental health services among sexual minorities with major mental illness. Am J Public Health. 2001;91(6):990–1. https://doi.org/10.2105/AJPH.91.6.990 .

Plöderl M, Mestel R, Fartacek C. Differences by sexual orientation in treatment outcome and satisfaction with treatment among inpatients of a German psychiatric clinic. PLoS ONE. 2022;17(1):e0262928. https://doi.org/10.1371/journal.pone.0262928 .

Clement S, Schauman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, et al. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med. 2015;45(1):11–27. https://doi.org/10.1017/S0033291714000129 .

Pyle M, Morrison AP. “It’s just a very taboo and secretive kind of thing”: making sense of living with stigma and discrimination from accounts of people with psychosis. Psychosis. 2014;6(3):195–205. https://doi.org/10.1080/17522439.2013.834458 .

Wood L, Burke E, Byrne R, Pyle M, Chapman N, Morrison AP. Stigma in psychosis: a thematic synthesis of current qualitative evidence. Psychosis. 2015;7(2):152–65. https://doi.org/10.1080/17522439.2014.926561 .

Rüsch N, Heekeren K, Theodoridou A, Dvorsky D, Müller M, Paust T, et al. Attitudes towards help-seeking and stigma among young people at risk for psychosis. Psychiatry Res. 2013;210(3):1313–5. https://doi.org/10.1016/j.psychres.2013.08.028 .

Gronholm PC, Thornicroft G, Laurens KR, Evans-Lacko S. Mental health-related stigma and pathways to care for people at risk of psychotic disorders or experiencing first-episode psychosis: a systematic review. Psychol Med. 2017;47(11):1867–79. https://doi.org/10.1017/S0033291717000344 .

Rossman K, Salamanca P, Macapagal K. A qualitative study examining young adults’ experiences of disclosure and nondisclosure of LGBTQ identity to health care providers. J Homosex. 2017;64(10):1390–410. https://doi.org/10.1080/00918369.2017.1321379 .

Rees SM, Crowe M, Harris S. The lesbian, gay, bisexual and transgender communities’ mental health care needs and experiences of mental health services: an integrative review of qualitative studies. J Psychiat Mental Health Nurs. 2020;28(4):578–89. https://doi.org/10.1111/jpm.12720 .

Shipherd JC, Green KE, Abramovitz A. Transgender clients: identifying and minimizing barriers to mental health treatment. J Gay Lesbian Ment Health. 2010;14(2):94–108. https://doi.org/10.1080/19359701003622875 .

Hoening J, Kenna JC. The nosological position of transsexualism. Arch Sex Behav. 1974;3:273–87. https://doi.org/10.1007/BF01541490 .

Eliasson ET, McNamee L, Swanson L, Lawrie SM, Schwannauer M. Unpacking stigma: meta-analyses of correlates and moderators of personal stigma in psychosis. Clin Psychol Rev. 2021;89:102077. https://doi.org/10.1016/j.cpr.2021.102077 .

Lally J, MacCabe JH. Antipsychotic medication in schizophrenia: a review. Br Med Bull. 2015;114(1):169–79. https://doi.org/10.1093/bmb/ldv017 .

De Lange J, Baams L, van Bergen D, Bos HMW, Bosker RJ. Minority stress and suicidal ideation and suicide attempts among LGBT adolescents and young adults: a meta-analysis. LGBT Health. 2022;9(4):222–37. https://doi.org/10.1089/lgbt.2021.0106 .

Skerrett DM, Kolves K, De Leo D. Are LGBT populations at higher risk for suicidal behaviours in Australia? Research findings and implications. J Homosex. 2015;62(7):883–901. https://doi.org/10.1080/00918369.2014.1003009 .

Strauss P, Cook A, Winter S, Watson V, Toussaint DW, Lin A. Trans Pathways: the mental health experiences and care pathways of trans young people: summary of results. Perth, Australia: Telethon Kids Institute. 2017. trans-pathways-report.pdf (telethonkids.org.au). Accessed 16 June 2023

DeLuca JS, Novacek DM, Adery LH, Herrera SN, Landa Y, Corcoran CM, et al. Equity in mental health services for youth at clinical high risk for psychosis: considering marginalized identities and stressors. Evid Based Pract Child Adolesc Ment Health. 2022;7(2):176–97. https://doi.org/10.1080/23794925.2022.2042874 .

Day JC, Bentall RP, Roberts C, Randall F, Rogers A, Cattell D, et al. Attitudes toward antipsychotic medication: the impact of clinical variables and relationships with health professionals. Arch Gen Psychiatry. 2005;62(7):717–24. https://doi.org/10.1001/archpsyc.62.7.717 .

Coyne CA, Poquiz JL, Janssen A, Chen D. Evidence-based psychological practice for transgender and non-binary youth: defining the need, framework for treatment adaptation, and future directions. Evid Based Pract Child Adolesc Ment Health. 2020;5(3):340–53. https://doi.org/10.1080/23794925.2020.1765433 .

Baker KE, Wilson LM, Sharma R, Dukhanin V, McArthur K, Robinson KA. Hormone therapy, mental health, and quality of life among transgender people: a systematic review. J Endocr Soc. 2021;5(4):bvab011. https://doi.org/10.1210/jendso/bvab011 .

Hughto JMW, Reisner SL. A systematic review of the effects of hormone therapy on psychological functioning and quality of life in transgender individuals. Transgend Health. 2016;1(1):21–31. https://doi.org/10.1089/trgh.2015.0008 .

Wernick JA, Busa S, Matouk K, Nicholson J, Janssen A. A systematic review of the psychological benefits of gender-affirming surgery. Urol Clin North Am. 2019;46(4):475–86. https://doi.org/10.1016/j.ucl.2019.07.002 .

Bränström R, Packanhkis JE. Reductions in mental health treatment utilization among transgender individuals after gender-affirming surgeries: a total population study. Am J Psychiatry. 2019;177(8):727–34. https://doi.org/10.1176/appi.ajp.2019.19010080 .

Meijer JH, Eeckhout GM, van Vlerken RHT, de Vries ALC. Gender dysphoria and co-existing psychosis: review and four case examples of successful gender affirmative treatment. LGBT Health. 2017;4(2):106–14. https://doi.org/10.1089/lgbt.2016.0133 .

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143. https://doi.org/10.1186/s12874-018-0611-x .

Munn Z, Pollock D, Khalil H, Alexander L, McInerney P, Godfrey CM, et al. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evid Synth. 2022;20(4):950–2. https://doi.org/10.11124/JBIES-21-00483 .

Peters MDJ, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Rev. 2021;10(1):263. https://doi.org/10.1186/s13643-021-01821-3 .

Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI Manual for Evidence Synthesis. JBI; 2024. https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-24-01

Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350: g7647. https://doi.org/10.1136/bmj.g7647 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;372:n71. https://doi.org/10.1136/bmj.n71 .

Barker TH, Stone JC, Sears K, Klugar M, Leonardi-Bee J, Tufanaru C, et al. Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process. JBI Evid Synth. 2023;21(3):478–93. https://doi.org/10.11124/JBIES-22-00125 .

Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC methods programme. Lancaster University; 2006. https://doi.org/10.13140/2.1.1018.4643

Bach-Mortensen AM, Verboom B. Barriers and facilitators systematic reviews in health: a methodological review and recommendations for reviewers. Res Synth Methods. 2020;11(6):743–59. https://doi.org/10.1002/jrsm.1447 .

Download references

Acknowledgements

The authors would like to acknowledge the support of Ms Olivia Larobina, Scholarly Services Librarian (STEMM) at Deakin University, in the development of the search strategy.

CCG is funded by a Deakin University Postgraduate Research (DUPR) Scholarship. ZW is funded by a University of Western Australia Research Training Program (RTP) Scholarship. AL is supported by a National Health and Medical Research Council (NHMRC) Emerging Leaders Fellowship (2010063). LJW is supported by a NHMRC Emerging Leaders Fellowship (1174060). ARY is supported by a NHMRC Principal Research Fellowship (1136829). The funding providers had no role in the design and conduct of the study, or in the preparation, review, or approval of this manuscript.

Author information

Authors and affiliations.

Institute for Mental and Physical Health and Clinical Translation, Deakin University, Geelong, VIC, 3220, Australia

Cláudia C. Gonçalves, Shae E. Quirk, Peter M. Haddad, Lana J. Williams & Alison R. Yung

Telethon Kids Institute, University of Western Australia, Perth, WA, 6009, Australia

University Hospital Geelong, Barwon Health, Geelong, VIC, 3220, Australia

Peter M. Haddad

School of Population and Global Health, University of Western Australia, Perth, WA, 6009, Australia

Ashleigh Lin

School of Health Sciences, University of Manchester, Manchester, M13 9PL, UK

Alison R. Yung

You can also search for this author in PubMed   Google Scholar

Contributions

CCG is the guarantor. CCG conceptualised the review, developed the study design, and drafted the manuscript. CCG, ZW, and SQ collaborated with OL (Scholarly Services Librarian) to develop the search strategy. All authors critically reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Cláudia C. Gonçalves .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

In the last 3 years, PMH has received honoraria for lecturing from Janssen, NewBridge Pharmaceuticals, and Otsuka and royalties from edited textbooks (Cambridge University Press, Oxford University Press).

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. prisma-p 2015 checklist. completed prisma-p checklist for this systematic review protocol., 13643_2024_2566_moesm2_esm.docx.

Additional file 2. Search Strategy. Detailed search strategy for this systematic review, including search terms and relevant controlled vocabulary terms for each included database.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Gonçalves, C.C., Waters, Z., Quirk, S.E. et al. Barriers and facilitators to mental health treatment access and engagement for LGBTQA+ people with psychosis: a scoping review protocol. Syst Rev 13 , 143 (2024). https://doi.org/10.1186/s13643-024-02566-5

Download citation

Received : 04 July 2023

Accepted : 17 May 2024

Published : 30 May 2024

DOI : https://doi.org/10.1186/s13643-024-02566-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Treatment access
• Treatment engagement
• Facilitators
• Clinical high risk
• Ultra-high risk
• Psychotic experiences

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• More from M-W
• To save this word, you'll need to log in. Log In

methodology

Definition of methodology

Did you know.

Methodology and Science

The methodology employed in an experiment is essential to its success, and bad methodology has spoiled thousands of research projects. So whenever a piece of research is published in a scientific or medical journal, the researchers always carefully describe their methodology; otherwise, other scientists couldn't possibly judge the quality of what they've done.

Examples of methodology in a Sentence

These examples are programmatically compiled from various online sources to illustrate current usage of the word 'methodology.' Any opinions expressed in the examples do not represent those of Merriam-Webster or its editors. Send us feedback about these examples.

Word History

New Latin methodologia , from Latin methodus + -logia -logy

1800, in the meaning defined at sense 1

Articles Related to methodology

Are You Misusing These 10 Common Words?

You keep using that word. I do not think it means what you think it means.

Dictionary Entries Near methodology

methodologist

methods engineer

Cite this Entry

“Methodology.” Merriam-Webster.com Dictionary , Merriam-Webster, https://www.merriam-webster.com/dictionary/methodology. Accessed 11 Jun. 2024.

More from Merriam-Webster on methodology

Nglish: Translation of methodology for Spanish Speakers

Britannica English: Translation of methodology for Arabic Speakers

Britannica.com: Encyclopedia article about methodology

Subscribe to America's largest dictionary and get thousands more definitions and advanced search—ad free!

Can you solve 4 words at once?

Word of the day.

See Definitions and Examples »

Get Word of the Day daily email!

Popular in Grammar & Usage

What's the difference between 'fascism' and 'socialism', more commonly misspelled words, commonly misspelled words, how to use em dashes (—), en dashes (–) , and hyphens (-), absent letters that are heard anyway, popular in wordplay, the words of the week - june 7, 8 words for lesser-known musical instruments, 9 superb owl words, 10 words for lesser-known games and sports, your favorite band is in the dictionary, games & quizzes.

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think! opens in new tab/window

CRediT author statement

CRediT (Contributor Roles Taxonomy) was introduced with the intention of recognizing individual author contributions, reducing authorship disputes and facilitating collaboration. The idea came about following a 2012 collaborative workshop led by Harvard University and the Wellcome Trust, with input from researchers, the International Committee of Medical Journal Editors (ICMJE) and publishers, including Elsevier, represented by Cell Press.

CRediT offers authors the opportunity to share an accurate and detailed description of their diverse contributions to the published work.

The corresponding author is responsible for ensuring that the descriptions are accurate and agreed by all authors

The role(s) of all authors should be listed, using the relevant above categories

Authors may have contributed in multiple roles

CRediT in no way changes the journal’s criteria to qualify for authorship

CRediT statements should be provided during the submission process and will appear above the acknowledgment section of the published paper as shown further below.

*Reproduced from Brand et al. (2015), Learned Publishing 28(2), with permission of the authors.

Sample CRediT author statement

Zhang San:  Conceptualization, Methodology, Software  Priya Singh. : Data curation, Writing- Original draft preparation.  Wang Wu : Visualization, Investigation.  Jan Jansen :  Supervision. : Ajay Kumar : Software, Validation.:  Sun Qi:  Writing- Reviewing and Editing,

Read more about CRediT  here opens in new tab/window  or check out this  article from  Authors' Updat e:  CRediT where credit's due .

• Search Close search
• Find a journal
• Search calls for papers
• Journal Suggester
• Open access publishing

We’re here to help

Find guidance on Author Services

Open access

Methodology or method? A critical review of qualitative case study reports

• Cite this article
• https://doi.org/10.3402/qhw.v9.23606

Definitions of qualitative case study research

Current methodological issues in qualitative case study research, conflict of interest and funding.

• Full Article
• Figures & data
• Reprints & Permissions
• View PDF PDF

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

• Case studies
• health research
• research design
• interdisciplinary research
• qualitative research
• literature review

Case study research is an increasingly popular approach among qualitative researchers (Thomas, Citation 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines ( Citation Creswell, 2013b ; Citation Denzin & Lincoln, 2011b ; Merriam, Citation 2009 ; Ragin & Becker, Citation 1992 ; Stake, Citation 1995 ; Yin, Citation 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach ( Citation Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( Citation 1995 ) and Merriam ( Citation 2009 ), is situated in a social constructivist paradigm, whereas the second, by Citation Yin (2012) , Citation Flyvbjerg (2011) , and Eisenhardt ( Citation 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, Citation 2006 ; Meyer, Citation 2001 ; Thomas, Citation 2010 ; Tight, Citation 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, Citation 1995 ). Qualitative case study research, as described by Stake ( Citation 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, Citation 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, Citation 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, Citation 1998 ). Citation Thomas (2011) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, Citation 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” ( Citation Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, Citation 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( Citation 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, Citation 1995 , Citation 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, Citation 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, Citation 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, Citation 2009 , Citation 2012 ).

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, Citation 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, Citation 2013 ; Morse, Citation 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity ( Citation Denzin & Lincoln, 2011b ; Morse, Citation 2009 ). Qualitative research is “inherently multimethod” ( Citation Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, Citation 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, Citation 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( Citation 2000 , Citation 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( Citation 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, Citation 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, Citation 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, Citation 1989 ; Flyvbjerg, Citation 2006 , Citation 2011 ; Jensen & Rodgers, Citation 2001 ; Piekkari, Welch, & Paavilainen, Citation 2009 ; Tight, Citation 2010 ; Yin, Citation 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, Citation 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, Citation 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, Citation 2009 ; Stake, Citation 1995 ). Building on discussions relating to “fuzzy” (Bassey, Citation 2001 ), or naturalistic generalizations (Stake, Citation 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, Citation 2003 ; Morse et al., Citation 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, Citation 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, Citation 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, Citation 2009 ; Meyer, Citation 2001 ; Stake, Citation 1995 ), and has been likened to a “curious methodological limbo” (Gerring, Citation 2004 , p. 341) or “paradigmatic bridge” (Luck et al., Citation 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, Citation 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, Citation 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, Citation 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour ( Citation Creswell, 2013b ; Merriam, Citation 2009 ; Stake, Citation 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous ( Citation Denzin & Lincoln, 2011b ; Lincoln, Citation 1995 ; Sandelowski & Barroso, Citation 2002 ). The criteria proposed by Stake ( Citation 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( Citation 2009 ) and Citation Creswell (2013b) . Stake's ( Citation 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Table I Framework for assessing quality in qualitative case study research.

Study design.

The critical review method described by Grant and Booth ( Citation 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, Citation 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, Citation 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, Citation 2006 ); however, this was the most appropriate and accessible method available.

Table II International Journal of Qualitative Studies on Health and Well-being.

Search strategy.

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, Citation 2008 ). Consensus was to allocate to the methods category.

Table III Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( Citation 1995 ), two by Yin ( Citation 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors ( Citation Creswell, 2013b ; Merriam, Citation 2009 ; Stake, Citation 1995 ; Yin, Citation 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, Citation 2012 ; Coltart & Henwood, Citation 2012 ; Hooghe, Neimeyer, & Rober, Citation 2012 ; Roscigno et al., Citation 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, Citation 2012 ; Buzzanell & D'Enbeau, Citation 2009 ; Nagar-Ron & Motzafi-Haller, Citation 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, Citation 2001 ; Stake, Citation 1995 ), and maintain focus on the case ( Citation Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Citation Yeh (2013) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., Citation 2013 , p. 130). Citation Bronken et al. (2012) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, Citation 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, Citation 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, Citation 2012 ; Bronken et al., Citation 2012 ; Colón-Emeric et al., Citation 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, Citation 2012 ; Mawn et al., Citation 2010 ; Snyder-Young, Citation 2011 ). There were exceptions in the methods category (Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, Citation 2009 ; Buckley & Waring, Citation 2013 ; Cunsolo Willox et al., Citation 2013 ; De Haene, Grietens, & Verschueren, Citation 2010 ; Gratton & O'Donnell, Citation 2011 ; Sumsion, Citation 2013 ; Wimpenny & Savin-Baden, Citation 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, Citation 2012 ; Bronken et al., Citation 2012 ; Colón-Emeric et al., Citation 2010 ; Jackson et al., Citation 2012 ; Mawn et al., Citation 2010 ). In two studies, local cases were selected (Barone, Citation 2010 ; Fourie & Theron, Citation 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., Citation 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., Citation 2010 ; Jackson et al., Citation 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, Citation 2013 ). Citation Roscigno et al. (2012) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Citation Hooghe et al. (2012) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Citation Coltart and Henwood (2012) , provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Box 1. Article synopsis of case study research using Stake's tradition.

Box 2. article synopsis of case study research using yin's tradition., contextually bound case study.

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, Citation 2009 ; Ragin & Becker, Citation 1992 ; Stake, Citation 1995 ; Yin, Citation 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., Citation 2010 ; D'Enbeau, Buzzanell, & Duckworth, Citation 2010 ; Gallagher et al., Citation 2013 ; Hooghe et al., Citation 2012 ; Jackson et al., Citation 2012 ; Roscigno et al., Citation 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Citation Gallagher et al. (2013) , or the nursing home in Citation Colón-Emeric et al. (2010) . Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, Citation 2012 ; Buzzanell & D'Enbeau, Citation 2009 ; Nairn & Panelli, Citation 2009 ; Wimpenny & Savin-Baden, Citation 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, Citation 2009 ) and images (Jorrín-Abellán et al., Citation 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, Citation 1998 ). In Boxes 1 and 2, we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1, Citation Ledderer (2011) used a qualitative case study design informed by Stake's tradition. In Box 2, Citation Gillard, Witt, and Watts (2011) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2, we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, Citation 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, Citation 2009 ; D'Enbeau et al., Citation 2010 ; Gallagher et al., Citation 2013 ; Gillard et al., Citation 2011 ; Ledderer, Citation 2011 ; Nagar-Ron & Motzafi-Haller, Citation 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, Citation 2009 ; Stake, Citation 1995 ).

Citation Gillard et al. (2011) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Citation Nagar-Ron and Motzafi-Haller (2011) , while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, Citation 2009 ; Bronken et al., Citation 2012 ; Fincham, Scourfield, & Langer, Citation 2008 ; Fourie & Theron, Citation 2012 ; Hooghe et al., Citation 2012 ; Snyder-Young, Citation 2011 ; Yeh, Citation 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, Citation 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods ( Citation Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, Citation 2010 ; Bronken et al., Citation 2012 ; Hooghe et al., Citation 2012 ; Mawn et al., Citation 2010 ; Roscigno et al., Citation 2012 ; Yeh, Citation 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., Citation 2010 ; Coltart & Henwood, Citation 2012 ; Roscigno et al., Citation 2012 ; Snyder-Young, Citation 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, Citation 2012 ; Hooghe et al., Citation 2012 ; Jorrín-Abellán et al., Citation 2008 ; Mawn et al., Citation 2010 ; Rytterström et al., Citation 2013 ; Wimpenny & Savin-Baden, Citation 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., Citation 2010 ; Rytterström et al., Citation 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements ( Citation Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines ( Citation Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Citation Morse (2011) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, Citation 2007 ; Sandelowski, Citation 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors ( Citation Creswell, 2013a ; Merriam, Citation 2009 ; Stake, Citation 1995 ; Yin, Citation 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, Citation 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, Citation 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, Citation 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, Citation 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature ( Citation Hames, 2013 ; Citation Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed ( Citation Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., Citation 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., Citation 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( Citation 1995 ) criteria have been referenced elsewhere, and was deemed the best available ( Citation Creswell, 2013b ; Crowe et al., Citation 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, Citation 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research . 2012; 12 (6): 735–752. 10.1177/1468794112439008.   Web of Science ® Google Scholar
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry . 2009; 15 (5): 931–951. 10.1177/1077800408322789.   Google Scholar
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry . 2012; 18 (2): 121–133. 10.1177/1077800411429087.   Google Scholar
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research . 2008; 18 (4): 557–564. 10.1177/1049732307308514.   PubMed Web of Science ® Google Scholar
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research . 2003; 13 (6): 871–883. 10.1177/1049732303013006008.   PubMed Web of Science ® Google Scholar
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research . 2010; 20 (4): 453–464. 10.1177/1049732310361893.   Google Scholar
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education . 2001; 27 (1): 5–22. 10.1080/03054980123773.   Web of Science ® Google Scholar
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research . 2012; 22 (10): 1303–1316. 10.1177/1049732312450366.   PubMed Web of Science ® Google Scholar
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research . 2011; 21 (12): 1705–1718. 10.1177/1049732311417727.   Google Scholar
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research . 2013; 13 (2): 148–172. 10.1177/1468794112472280.   Google Scholar
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry . 2009; 15 (7): 1199–1224. 10.1177/1077800409338025.   Web of Science ® Google Scholar
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research . 2007; 17 (10): 1316–1328. 10.1177/1049732307306927.   PubMed Web of Science ® Google Scholar
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers). Qualitative Health Research . 2006; 16 (3): 423–435. 10.1177/1049732305285701.   PubMed Web of Science ® Google Scholar
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, etal. Regulation and mindful resident care in nursing homes. Qualitative Health Research . 2010; 20 (9): 1283–1294. 10.1177/1049732310369337.   PubMed Web of Science ® Google Scholar
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research . 2012; 12 (1): 35–52. 10.1177/1468794111426224.   Google Scholar
• Creswell J. W, Creswell J. W. Five qualitative approaches to inquiry. Qualitative inquiry and research design: Choosing among five approaches . 2013a; 3rd ed, Thousand Oaks, CA: Sage. 53–84.   Google Scholar
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches . 2013b; 3rd ed, Thousand Oaks, CA: Sage.   Google Scholar
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology . 2011; 11 (1): 1–9. 10.1186/1471-2288-11-100.   PubMed Web of Science ® Google Scholar
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government. Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research . 2013; 13 (2): 127–147. 10.1177/1468794112446105.   Google Scholar
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research . 2010; 20(12): 1664–1676.   PubMed Web of Science ® Google Scholar
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry . 2010; 16 (9): 709–720. 10.1177/1077800410374183.   Google Scholar
• Denzin N. K, Lincoln Y. S, Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. The SAGE handbook of qualitative research . 2011a; 4th ed, Thousand Oaks, CA: Sage. 1–6.   Google Scholar
• Denzin N. K, Lincoln Y. S. The SAGE handbook of qualitative research . 2011b; 4th ed, Thousand Oaks, CA: Sage.   Google Scholar
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research . 2012; 12 (2): 202–217. 10.1177/1468794111422040.   Web of Science ® Google Scholar
• Building theories from case study research. The Academy of Management Review . 14 (4): 532–550. 10.2307/258557.   Google Scholar
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research . 2008; 18 (6): 853–862. 10.1177/1049732307308945.   PubMed Web of Science ® Google Scholar
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care . 1999; 8 (2): 107–112. 10.1046/j.1365-2354.1999.00141.x.   Google Scholar
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry . 2006; 12 (2): 219–245. 10.1177/1077800405284.363.   Web of Science ® Google Scholar
• Flyvbjerg B, Denzin N. K, Lincoln Y. S. Case study. The SAGE handbook of qualitative research . 2011; 4th ed, Thousand Oaks, CA: Sage. 301–316.   Google Scholar
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research . 2012; 22 (10): 1355–1368. 10.1177/1049732312451871.   Web of Science ® Google Scholar
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research . 2013; 23 (3): 407–421. 10.1177/1049732312470521.   Google Scholar
• Gerring J. What is a case study and what is it good for?. American Political Science Review . 2004; 98 (2): 341–354. 10.1017/S0003055404001182.   Web of Science ® Google Scholar
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research . 2011; 21 (11): 1508–1526. 10.1177/1049732311413907.   PubMed Web of Science ® Google Scholar
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal . 2009; 26 : 91–108. 10.1111/j.1471-1842.2009.00848.x.   PubMed Web of Science ® Google Scholar
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research . 2011; 11 (2): 159–175. 10.1177/1468794110394068.   Web of Science ® Google Scholar
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing . 2013; 8 : 1. 10.3402/qhw.v8i0.20647.   Web of Science ® Google Scholar
• Hames I. COPE Ethical guidelines for peer reviewers. Committee on Publication Ethics, 1. 2013, March. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines .   Google Scholar
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research . 2012; 22 (9): 1220–1231. 10.1177/1049732312449209.   PubMed Web of Science ® Google Scholar
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research . 2012; 22 (11): 1468–1475. 10.1177/1049732312450323.   Google Scholar
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink?. Nursing Inquiry . 2013 10.1111/nin.12030.   Web of Science ® Google Scholar
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review . 2001; 61 (2): 235–246. 10.1111/0033-3352.00025.   Web of Science ® Google Scholar
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research?. Qualitative Inquiry . 2008; 14 (7): 1187–1204. 10.1177/1077800408318435.   Google Scholar
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research . 2011; 21 (1): 27–40. 10.1177/1049732310377451.   PubMed Google Scholar
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry . 1995; 1 (3): 275–289. 10.1177/107780049500100301.   Google Scholar
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry . 2006; 13 (2): 103–109. 10.1111/j.1440-1800.2006.00309.x.   PubMed Web of Science ® Google Scholar
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, etal. Health disparities among health care workers. Qualitative Health Research . 2010; 20 (1): 68–80. 10.1177/1049732309355590.   Google Scholar
• Merriam S. B. Qualitative research: A guide to design and implementation . 2009; 3rd ed, San Francisco, CA: Jossey-Bass.   Google Scholar
• Meyer C. B. A case in case study methodology. Field Methods . 2001; 13 (4): 329–352. 10.1177/1525822x0101300402.   Google Scholar
• Morse J. M. Mixing qualitative methods. Qualitative Health Research . 2009; 19 (11): 1523–1524. 10.1177/1049732309349360.   PubMed Web of Science ® Google Scholar
• Morse J. M. Molding qualitative health research. Qualitative Health Research . 2011; 21 (8): 1019–1021. 10.1177/1049732311404706.   PubMed Web of Science ® Google Scholar
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, etal. Considering the qualitative–quantitative language divide. Qualitative Health Research . 2011; 21 (9): 1302–1303. 10.1177/1049732310392386.   Google Scholar
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry . 2011; 17 (7): 653–663. 10.1177/1077800411414007.   Web of Science ® Google Scholar
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry . 2009; 15 (1): 96–112. 10.1177/1077800408318314.   Google Scholar
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry . 2012; 18 (5): 449–460. 10.1177/1077800412439530.   Google Scholar
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods . 2009; 12 (3): 567–589. 10.1177/1094428108319905.   Web of Science ® Google Scholar
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry . 1992; Cambridge: Cambridge University Press.   Google Scholar
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, etal. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research . 2012; 22 (9): 1232–1246. 10.1177/1049732312449210.   PubMed Web of Science ® Google Scholar
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing . 2007; 60 (4): 447–452. 10.1111/j.1365-2648.2007.04385.x.   PubMed Web of Science ® Google Scholar
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research . 2013; 23 : 1179–1187. 10.1177/1049732312470760.   Web of Science ® Google Scholar
• Sandelowski M. Whatever happened to qualitative description?. Research in Nursing & Health . 2000; 23 (4): 334–340. 10.1002/1098-240X.   PubMed Web of Science ® Google Scholar
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health . 2010; 33 (1): 77–84. 10.1002/nur.20362.   PubMed Web of Science ® Google Scholar
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods . 2002; 1(1): 74–108.   Google Scholar
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry . 2011; 17 (10): 943–951. 10.1177/1077800411425149.   Google Scholar
• Stake R. E. The case study method in social inquiry. Educational Researcher . 1978; 7 ( 2 ): 5–8. 10.3102/0013189X007002005.   Google Scholar
• Stake R. E. The art of case study research . 1995; Thousand Oaks, CA: Sage.   Google Scholar
• Stake R. E, Denzin N. K, Lincoln Y. S. Case studies. Strategies of qualitative inquiry . 1998; Thousand Oaks, CA: Sage. 86–109.   Google Scholar
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research . 2013; 14 (2): 149–165. 10.1177/1468794112468471..   Google Scholar
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry . 2010; 16 (7): 575–582. 10.1177/1077800410372601.   Web of Science ® Google Scholar
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry . 2011; 17 (6): 511–521. 10.1177/1077800411409884.   Web of Science ® Google Scholar
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology . 2010; 13 (4): 329–339. 10.1080/13645570903187181.   Web of Science ® Google Scholar
• Wager E, Kleinert S, Mayer T, Steneck N. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. Promoting research integrity in a global environment . 2010a; Singapore: Imperial College Press/World Scientific. 309–316.   Google Scholar
• Wager E, Kleinert S, Mayer T, Steneck N. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. Promoting research integrity in a global environment . 2010b; Singapore: Imperial College Press/World Scientific. 317–328.   Google Scholar
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing . 2000; 33 (6): 798–805. 10.1046/j.1365-2648.2001.01720.x.   Google Scholar
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry . 2012; 18 (8): 689–698. 10.1177/1077800412452854.   Web of Science ® Google Scholar
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry . 2013; 19 (4): 298–309. 10.1177/1077800412471516.   Google Scholar
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research . 1999; 34(5 Pt 2): 1209–1224.   PubMed Web of Science ® Google Scholar
• Yin R. K. Case study research: Design and methods . 2009; 4th ed, Thousand Oaks, CA: Sage.   Google Scholar
• Yin R. K. Applications of case study research . 2012; 3rd ed, Thousand Oaks, CA: Sage.   Google Scholar
• Back to Top

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations. Articles with the Crossref icon will open in a new tab.

• People also read
• Recommended articles

To cite this article:

Download citation, your download is now in progress and you may close this window.

• Choose new content alerts to be informed about new research of interest to you
• Easy remote access to your institution's subscriptions on any device, from any location
• Save your searches and schedule alerts to send you new results
• Export your search results into a .csv file to support your research

Login or register to access this feature

Register now or learn more

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Test for Fentanyl
• if You Think Someone is Overdosing
• Stop Overdose
• Naloxone FAQs
• Stigma Reduction

About Stop Overdose

• Through preliminary research and strategic workshops, CDC identified four areas of focus to address the evolving drug overdose crisis.
• Stop Overdose resources speak to the reality of drug use, provide practical ways to prevent overdoses, educate about the risks of illegal drug use, and show ways to get help.

Drugs take nearly 300 lives every day. 1 To address the increasing number of overdose deaths related to both prescription opioids and illegal drugs, we created a website to educate people who use drugs about the dangers of illegally manufactured fentanyl, the risks and consequences of mixing drugs, the lifesaving power of naloxone, and the importance of reducing stigma around recovery and treatment options. Together, we can stop drug overdoses and save lives.

What you can do

• Get the facts on fentanyl
• Learn about lifesaving naloxone
• Understand the risks of polysubstance use
• Reduce stigma around recovery and treatment

Explore and download Stop Overdose and other educational materials on CDC's Overdose Resource Exchange .

• Centers for Disease Control and Prevention, National Center for Health Statistics. National Vital Statistics System, Mortality 2018-2021 on CDC WONDER Online Database, released in 2023. Data are from the Multiple Cause of Death Files, 2018-2021, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Accessed at http://wonder.cdc.gov/mcd-icd10-expanded.html on Mar 5, 2024

Every day, drugs claim hundreds of lives. The Stop Overdose website educates drug users on fentanyl, naloxone, polysubstance use, and dealing with stigma.