thematic analysis in systematic literature review

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Practical thematic...

Practical thematic analysis: a guide for multidisciplinary health services research teams engaging in qualitative analysis

• Related content
• Peer review
• Catherine H Saunders , scientist and assistant professor 1 2 ,
• Ailyn Sierpe , research project coordinator 2 ,
• Christian von Plessen , senior physician 3 ,
• Alice M Kennedy , research project manager 2 4 ,
• Laura C Leviton , senior adviser 5 ,
• Steven L Bernstein , chief research officer 1 ,
• Jenaya Goldwag , resident physician 1 ,
• Joel R King , research assistant 2 ,
• Christine M Marx , patient associate 6 ,
• Jacqueline A Pogue , research project manager 2 ,
• Richard K Saunders , staff physician 1 ,
• Aricca Van Citters , senior research scientist 2 ,
• Renata W Yen , doctoral student 2 ,
• Glyn Elwyn , professor 2 ,
• JoAnna K Leyenaar , associate professor 1 2
• on behalf of the Coproduction Laboratory
• 1 Dartmouth Health, Lebanon, NH, USA
• 2 Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth College, Lebanon, NH, USA
• 3 Center for Primary Care and Public Health (Unisanté), Lausanne, Switzerland
• 4 Jönköping Academy for Improvement of Health and Welfare, School of Health and Welfare, Jönköping University, Jönköping, Sweden
• 5 Highland Park, NJ, USA
• 6 Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, St Louis, MO, USA
• Correspondence to: C H Saunders catherine.hylas.saunders{at}dartmouth.edu
• Accepted 26 April 2023

Qualitative research methods explore and provide deep contextual understanding of real world issues, including people’s beliefs, perspectives, and experiences. Whether through analysis of interviews, focus groups, structured observation, or multimedia data, qualitative methods offer unique insights in applied health services research that other approaches cannot deliver. However, many clinicians and researchers hesitate to use these methods, or might not use them effectively, which can leave relevant areas of inquiry inadequately explored. Thematic analysis is one of the most common and flexible methods to examine qualitative data collected in health services research. This article offers practical thematic analysis as a step-by-step approach to qualitative analysis for health services researchers, with a focus on accessibility for patients, care partners, clinicians, and others new to thematic analysis. Along with detailed instructions covering three steps of reading, coding, and theming, the article includes additional novel and practical guidance on how to draft effective codes, conduct a thematic analysis session, and develop meaningful themes. This approach aims to improve consistency and rigor in thematic analysis, while also making this method more accessible for multidisciplinary research teams.

Through qualitative methods, researchers can provide deep contextual understanding of real world issues, and generate new knowledge to inform hypotheses, theories, research, and clinical care. Approaches to data collection are varied, including interviews, focus groups, structured observation, and analysis of multimedia data, with qualitative research questions aimed at understanding the how and why of human experience. 1 2 Qualitative methods produce unique insights in applied health services research that other approaches cannot deliver. In particular, researchers acknowledge that thematic analysis is a flexible and powerful method of systematically generating robust qualitative research findings by identifying, analysing, and reporting patterns (themes) within data. 3 4 5 6 Although qualitative methods are increasingly valued for answering clinical research questions, many researchers are unsure how to apply them or consider them too time consuming to be useful in responding to practical challenges 7 or pressing situations such as public health emergencies. 8 Consequently, researchers might hesitate to use them, or use them improperly. 9 10 11

Although much has been written about how to perform thematic analysis, practical guidance for non-specialists is sparse. 3 5 6 12 13 In the multidisciplinary field of health services research, qualitative data analysis can confound experienced researchers and novices alike, which can stoke concerns about rigor, particularly for those more familiar with quantitative approaches. 14 Since qualitative methods are an area of specialisation, support from experts is beneficial. However, because non-specialist perspectives can enhance data interpretation and enrich findings, there is a case for making thematic analysis easier, more rapid, and more efficient, 8 particularly for patients, care partners, clinicians, and other stakeholders. A practical guide to thematic analysis might encourage those on the ground to use these methods in their work, unearthing insights that would otherwise remain undiscovered.

Given the need for more accessible qualitative analysis approaches, we present a simple, rigorous, and efficient three step guide for practical thematic analysis. We include new guidance on the mechanics of thematic analysis, including developing codes, constructing meaningful themes, and hosting a thematic analysis session. We also discuss common pitfalls in thematic analysis and how to avoid them.

Summary points

Qualitative methods are increasingly valued in applied health services research, but multidisciplinary research teams often lack accessible step-by-step guidance and might struggle to use these approaches

A newly developed approach, practical thematic analysis, uses three simple steps: reading, coding, and theming

Based on Braun and Clarke’s reflexive thematic analysis, our streamlined yet rigorous approach is designed for multidisciplinary health services research teams, including patients, care partners, and clinicians

This article also provides companion materials including a slide presentation for teaching practical thematic analysis to research teams, a sample thematic analysis session agenda, a theme coproduction template for use during the session, and guidance on using standardised reporting criteria for qualitative research

In their seminal work, Braun and Clarke developed a six phase approach to reflexive thematic analysis. 4 12 We built on their method to develop practical thematic analysis ( box 1 , fig 1 ), which is a simplified and instructive approach that retains the substantive elements of their six phases. Braun and Clarke’s phase 1 (familiarising yourself with the dataset) is represented in our first step of reading. Phase 2 (coding) remains as our second step of coding. Phases 3 (generating initial themes), 4 (developing and reviewing themes), and 5 (refining, defining, and naming themes) are represented in our third step of theming. Phase 6 (writing up) also occurs during this third step of theming, but after a thematic analysis session. 4 12

Key features and applications of practical thematic analysis

Step 1: reading.

All manuscript authors read the data

All manuscript authors write summary memos

Step 2: Coding

Coders perform both data management and early data analysis

Codes are complete thoughts or sentences, not categories

Step 3: Theming

Researchers host a thematic analysis session and share different perspectives

Themes are complete thoughts or sentences, not categories

Applications

For use by practicing clinicians, patients and care partners, students, interdisciplinary teams, and those new to qualitative research

When important insights from healthcare professionals are inaccessible because they do not have qualitative methods training

When time and resources are limited

Steps in practical thematic analysis

• Download figure
• Open in new tab
• Download powerpoint

We present linear steps, but as qualitative research is usually iterative, so too is thematic analysis. 15 Qualitative researchers circle back to earlier work to check whether their interpretations still make sense in the light of additional insights, adapting as necessary. While we focus here on the practical application of thematic analysis in health services research, we recognise our approach exists in the context of the broader literature on thematic analysis and the theoretical underpinnings of qualitative methods as a whole. For a more detailed discussion of these theoretical points, as well as other methods widely used in health services research, we recommend reviewing the sources outlined in supplemental material 1. A strong and nuanced understanding of the context and underlying principles of thematic analysis will allow for higher quality research. 16

Practical thematic analysis is a highly flexible approach that can draw out valuable findings and generate new hypotheses, including in cases with a lack of previous research to build on. The approach can also be used with a variety of data, such as transcripts from interviews or focus groups, patient encounter transcripts, professional publications, observational field notes, and online activity logs. Importantly, successful practical thematic analysis is predicated on having high quality data collected with rigorous methods. We do not describe qualitative research design or data collection here. 11 17

In supplemental material 1, we summarise the foundational methods, concepts, and terminology in qualitative research. Along with our guide below, we include a companion slide presentation for teaching practical thematic analysis to research teams in supplemental material 2. We provide a theme coproduction template for teams to use during thematic analysis sessions in supplemental material 3. Our method aligns with the major qualitative reporting frameworks, including the Consolidated Criteria for Reporting Qualitative Research (COREQ). 18 We indicate the corresponding step in practical thematic analysis for each COREQ item in supplemental material 4.

Familiarisation and memoing

We encourage all manuscript authors to review the full dataset (eg, interview transcripts) to familiarise themselves with it. This task is most critical for those who will later be engaged in the coding and theming steps. Although time consuming, it is the best way to involve team members in the intellectual work of data interpretation, so that they can contribute to the analysis and contextualise the results. If this task is not feasible given time limitations or large quantities of data, the data can be divided across team members. In this case, each piece of data should be read by at least two individuals who ideally represent different professional roles or perspectives.

We recommend that researchers reflect on the data and independently write memos, defined as brief notes on thoughts and questions that arise during reading, and a summary of their impressions of the dataset. 2 19 Memoing is an opportunity to gain insights from varying perspectives, particularly from patients, care partners, clinicians, and others. It also gives researchers the opportunity to begin to scope which elements of and concepts in the dataset are relevant to the research question.

Data saturation

The concept of data saturation ( box 2 ) is a foundation of qualitative research. It is defined as the point in analysis at which new data tend to be redundant of data already collected. 21 Qualitative researchers are expected to report their approach to data saturation. 18 Because thematic analysis is iterative, the team should discuss saturation throughout the entire process, beginning with data collection and continuing through all steps of the analysis. 22 During step 1 (reading), team members might discuss data saturation in the context of summary memos. Conversations about saturation continue during step 2 (coding), with confirmation that saturation has been achieved during step 3 (theming). As a rule of thumb, researchers can often achieve saturation in 9-17 interviews or 4-8 focus groups, but this will vary depending on the specific characteristics of the study. 23

Data saturation in context

Braun and Clarke discourage the use of data saturation to determine sample size (eg, number of interviews), because it assumes that there is an objective truth to be captured in the data (sometimes known as a positivist perspective). 20 Qualitative researchers often try to avoid positivist approaches, arguing that there is no one true way of seeing the world, and will instead aim to gather multiple perspectives. 5 Although this theoretical debate with qualitative methods is important, we recognise that a priori estimates of saturation are often needed, particularly for investigators newer to qualitative research who might want a more pragmatic and applied approach. In addition, saturation based, sample size estimation can be particularly helpful in grant proposals. However, researchers should still follow a priori sample size estimation with a discussion to confirm saturation has been achieved.

Definition of coding

We describe codes as labels for concepts in the data that are directly relevant to the study objective. Historically, the purpose of coding was to distil the large amount of data collected into conceptually similar buckets so that researchers could review it in aggregate and identify key themes. 5 24 We advocate for a more analytical approach than is typical with thematic analysis. With our method, coding is both the foundation for and the beginning of thematic analysis—that is, early data analysis, management, and reduction occur simultaneously rather than as different steps. This approach moves the team more efficiently towards being able to describe themes.

Building the coding team

Coders are the research team members who directly assign codes to the data, reading all material and systematically labelling relevant data with appropriate codes. Ideally, at least two researchers would code every discrete data document, such as one interview transcript. 25 If this task is not possible, individual coders can each code a subset of the data that is carefully selected for key characteristics (sometimes known as purposive selection). 26 When using this approach, we recommend that at least 10% of data be coded by two or more coders to ensure consistency in codebook application. We also recommend coding teams of no more than four to five people, for practical reasons concerning maintaining consistency.

Clinicians, patients, and care partners bring unique perspectives to coding and enrich the analytical process. 27 Therefore, we recommend choosing coders with a mix of relevant experiences so that they can challenge and contextualise each other’s interpretations based on their own perspectives and opinions ( box 3 ). We recommend including both coders who collected the data and those who are naive to it, if possible, given their different perspectives. We also recommend all coders review the summary memos from the reading step so that key concepts identified by those not involved in coding can be integrated into the analytical process. In practice, this review means coding the memos themselves and discussing them during the code development process. This approach ensures that the team considers a diversity of perspectives.

Coding teams in context

The recommendation to use multiple coders is a departure from Braun and Clarke. 28 29 When the views, experiences, and training of each coder (sometimes known as positionality) 30 are carefully considered, having multiple coders can enhance interpretation and enrich findings. When these perspectives are combined in a team setting, researchers can create shared meaning from the data. Along with the practical consideration of distributing the workload, 31 inclusion of these multiple perspectives increases the overall quality of the analysis by mitigating the impact of any one coder’s perspective. 30

Coding tools

Qualitative analysis software facilitates coding and managing large datasets but does not perform the analytical work. The researchers must perform the analysis themselves. Most programs support queries and collaborative coding by multiple users. 32 Important factors to consider when choosing software can include accessibility, cost, interoperability, the look and feel of code reports, and the ease of colour coding and merging codes. Coders can also use low tech solutions, including highlighters, word processors, or spreadsheets.

Drafting effective codes

To draft effective codes, we recommend that the coders review each document line by line. 33 As they progress, they can assign codes to segments of data representing passages of interest. 34 Coders can also assign multiple codes to the same passage. Consensus among coders on what constitutes a minimum or maximum amount of text for assigning a code is helpful. As a general rule, meaningful segments of text for coding are shorter than one paragraph, but longer than a few words. Coders should keep the study objective in mind when determining which data are relevant ( box 4 ).

Code types in context

Similar to Braun and Clarke’s approach, practical thematic analysis does not specify whether codes are based on what is evident from the data (sometimes known as semantic) or whether they are based on what can be inferred at a deeper level from the data (sometimes known as latent). 4 12 35 It also does not specify whether they are derived from the data (sometimes known as inductive) or determined ahead of time (sometimes known as deductive). 11 35 Instead, it should be noted that health services researchers conducting qualitative studies often adopt all these approaches to coding (sometimes known as hybrid analysis). 3

In practical thematic analysis, codes should be more descriptive than general categorical labels that simply group data with shared characteristics. At a minimum, codes should form a complete (or full) thought. An easy way to conceptualise full thought codes is as complete sentences with subjects and verbs ( table 1 ), although full sentence coding is not always necessary. With full thought codes, researchers think about the data more deeply and capture this insight in the codes. This coding facilitates the entire analytical process and is especially valuable when moving from codes to broader themes. Experienced qualitative researchers often intuitively use full thought or sentence codes, but this practice has not been explicitly articulated as a path to higher quality coding elsewhere in the literature. 6

Example transcript with codes used in practical thematic analysis 36

• View inline

Depending on the nature of the data, codes might either fall into flat categories or be arranged hierarchically. Flat categories are most common when the data deal with topics on the same conceptual level. In other words, one topic is not a subset of another topic. By contrast, hierarchical codes are more appropriate for concepts that naturally fall above or below each other. Hierarchical coding can also be a useful form of data management and might be necessary when working with a large or complex dataset. 5 Codes grouped into these categories can also make it easier to naturally transition into generating themes from the initial codes. 5 These decisions between flat versus hierarchical coding are part of the work of the coding team. In both cases, coders should ensure that their code structures are guided by their research questions.

Developing the codebook

A codebook is a shared document that lists code labels and comprehensive descriptions for each code, as well as examples observed within the data. Good code descriptions are precise and specific so that coders can consistently assign the same codes to relevant data or articulate why another coder would do so. Codebook development is iterative and involves input from the entire coding team. However, as those closest to the data, coders must resist undue influence, real or perceived, from other team members with conflicting opinions—it is important to mitigate the risk that more senior researchers, like principal investigators, exert undue influence on the coders’ perspectives.

In practical thematic analysis, coders begin codebook development by independently coding a small portion of the data, such as two to three transcripts or other units of analysis. Coders then individually produce their initial codebooks. This task will require them to reflect on, organise, and clarify codes. The coders then meet to reconcile the draft codebooks, which can often be difficult, as some coders tend to lump several concepts together while others will split them into more specific codes. Discussing disagreements and negotiating consensus are necessary parts of early data analysis. Once the codebook is relatively stable, we recommend soliciting input on the codes from all manuscript authors. Yet, coders must ultimately be empowered to finalise the details so that they are comfortable working with the codebook across a large quantity of data.

Assigning codes to the data

After developing the codebook, coders will use it to assign codes to the remaining data. While the codebook’s overall structure should remain constant, coders might continue to add codes corresponding to any new concepts observed in the data. If new codes are added, coders should review the data they have already coded and determine whether the new codes apply. Qualitative data analysis software can be useful for editing or merging codes.

We recommend that coders periodically compare their code occurrences ( box 5 ), with more frequent check-ins if substantial disagreements occur. In the event of large discrepancies in the codes assigned, coders should revise the codebook to ensure that code descriptions are sufficiently clear and comprehensive to support coding alignment going forward. Because coding is an iterative process, the team can adjust the codebook as needed. 5 28 29

Quantitative coding in context

Researchers should generally avoid reporting code counts in thematic analysis. However, counts can be a useful proxy in maintaining alignment between coders on key concepts. 26 In practice, therefore, researchers should make sure that all coders working on the same piece of data assign the same codes with a similar pattern and that their memoing and overall assessment of the data are aligned. 37 However, the frequency of a code alone is not an indicator of its importance. It is more important that coders agree on the most salient points in the data; reviewing and discussing summary memos can be helpful here. 5

Researchers might disagree on whether or not to calculate and report inter-rater reliability. We note that quantitative tests for agreement, such as kappa statistics or intraclass correlation coefficients, can be distracting and might not provide meaningful results in qualitative analyses. Similarly, Braun and Clarke argue that expecting perfect alignment on coding is inconsistent with the goal of co-constructing meaning. 28 29 Overall consensus on codes’ salience and contributions to themes is the most important factor.

Definition of themes

Themes are meta-constructs that rise above codes and unite the dataset ( box 6 , fig 2 ). They should be clearly evident, repeated throughout the dataset, and relevant to the research questions. 38 While codes are often explicit descriptions of the content in the dataset, themes are usually more conceptual and knit the codes together. 39 Some researchers hypothesise that theme development is loosely described in the literature because qualitative researchers simply intuit themes during the analytical process. 39 In practical thematic analysis, we offer a concrete process that should make developing meaningful themes straightforward.

Themes in context

According to Braun and Clarke, a theme “captures something important about the data in relation to the research question and represents some level of patterned response or meaning within the data set.” 4 Similarly, Braun and Clarke advise against themes as domain summaries. While different approaches can draw out themes from codes, the process begins by identifying patterns. 28 35 Like Braun and Clarke and others, we recommend that researchers consider the salience of certain themes, their prevalence in the dataset, and their keyness (ie, how relevant the themes are to the overarching research questions). 4 12 34

Use of themes in practical thematic analysis

Constructing meaningful themes

After coding all the data, each coder should independently reflect on the team’s summary memos (step 1), the codebook (step 2), and the coded data itself to develop draft themes (step 3). It can be illuminating for coders to review all excerpts associated with each code, so that they derive themes directly from the data. Researchers should remain focused on the research question during this step, so that themes have a clear relation with the overall project aim. Use of qualitative analysis software will make it easy to view each segment of data tagged with each code. Themes might neatly correspond to groups of codes. Or—more likely—they will unite codes and data in unexpected ways. A whiteboard or presentation slides might be helpful to organise, craft, and revise themes. We also provide a template for coproducing themes (supplemental material 3). As with codebook justification, team members will ideally produce individual drafts of the themes that they have identified in the data. They can then discuss these with the group and reach alignment or consensus on the final themes.

The team should ensure that all themes are salient, meaning that they are: supported by the data, relevant to the study objectives, and important. Similar to codes, themes are framed as complete thoughts or sentences, not categories. While codes and themes might appear to be similar to each other, the key distinction is that the themes represent a broader concept. Table 2 shows examples of codes and their corresponding themes from a previously published project that used practical thematic analysis. 36 Identifying three to four key themes that comprise a broader overarching theme is a useful approach. Themes can also have subthemes, if appropriate. 40 41 42 43 44

Example codes with themes in practical thematic analysis 36

Thematic analysis session

After each coder has independently produced draft themes, a carefully selected subset of the manuscript team meets for a thematic analysis session ( table 3 ). The purpose of this session is to discuss and reach alignment or consensus on the final themes. We recommend a session of three to five hours, either in-person or virtually.

Example agenda of thematic analysis session

The composition of the thematic analysis session team is important, as each person’s perspectives will shape the results. This group is usually a small subset of the broader research team, with three to seven individuals. We recommend that primary and senior authors work together to include people with diverse experiences related to the research topic. They should aim for a range of personalities and professional identities, particularly those of clinicians, trainees, patients, and care partners. At a minimum, all coders and primary and senior authors should participate in the thematic analysis session.

The session begins with each coder presenting their draft themes with supporting quotes from the data. 5 Through respectful and collaborative deliberation, the group will develop a shared set of final themes.

One team member facilitates the session. A firm, confident, and consistent facilitation style with good listening skills is critical. For practical reasons, this person is not usually one of the primary coders. Hierarchies in teams cannot be entirely flattened, but acknowledging them and appointing an external facilitator can reduce their impact. The facilitator can ensure that all voices are heard. For example, they might ask for perspectives from patient partners or more junior researchers, and follow up on comments from senior researchers to say, “We have heard your perspective and it is important; we want to make sure all perspectives in the room are equally considered.” Or, “I hear [senior person] is offering [x] idea, I’d like to hear other perspectives in the room.” The role of the facilitator is critical in the thematic analysis session. The facilitator might also privately discuss with more senior researchers, such as principal investigators and senior authors, the importance of being aware of their influence over others and respecting and eliciting the perspectives of more junior researchers, such as patients, care partners, and students.

To our knowledge, this discrete thematic analysis session is a novel contribution of practical thematic analysis. It helps efficiently incorporate diverse perspectives using the session agenda and theme coproduction template (supplemental material 3) and makes the process of constructing themes transparent to the entire research team.

Writing the report

We recommend beginning the results narrative with a summary of all relevant themes emerging from the analysis, followed by a subheading for each theme. Each subsection begins with a brief description of the theme and is illustrated with relevant quotes, which are contextualised and explained. The write-up should not simply be a list, but should contain meaningful analysis and insight from the researchers, including descriptions of how different stakeholders might have experienced a particular situation differently or unexpectedly.

In addition to weaving quotes into the results narrative, quotes can be presented in a table. This strategy is a particularly helpful when submitting to clinical journals with tight word count limitations. Quote tables might also be effective in illustrating areas of agreement and disagreement across stakeholder groups, with columns representing different groups and rows representing each theme or subtheme. Quotes should include an anonymous label for each participant and any relevant characteristics, such as role or gender. The aim is to produce rich descriptions. 5 We recommend against repeating quotations across multiple themes in the report, so as to avoid confusion. The template for coproducing themes (supplemental material 3) allows documentation of quotes supporting each theme, which might also be useful during report writing.

Visual illustrations such as a thematic map or figure of the findings can help communicate themes efficiently. 4 36 42 44 If a figure is not possible, a simple list can suffice. 36 Both must clearly present the main themes with subthemes. Thematic figures can facilitate confirmation that the researchers’ interpretations reflect the study populations’ perspectives (sometimes known as member checking), because authors can invite discussions about the figure and descriptions of findings and supporting quotes. 46 This process can enhance the validity of the results. 46

In supplemental material 4, we provide additional guidance on reporting thematic analysis consistent with COREQ. 18 Commonly used in health services research, COREQ outlines a standardised list of items to be included in qualitative research reports ( box 7 ).

Reporting in context

We note that use of COREQ or any other reporting guidelines does not in itself produce high quality work and should not be used as a substitute for general methodological rigor. Rather, researchers must consider rigor throughout the entire research process. As the issue of how to conceptualise and achieve rigorous qualitative research continues to be debated, 47 48 we encourage researchers to explicitly discuss how they have looked at methodological rigor in their reports. Specifically, we point researchers to Braun and Clarke’s 2021 tool for evaluating thematic analysis manuscripts for publication (“Twenty questions to guide assessment of TA [thematic analysis] research quality”). 16

Avoiding common pitfalls

Awareness of common mistakes can help researchers avoid improper use of qualitative methods. Improper use can, for example, prevent researchers from developing meaningful themes and can risk drawing inappropriate conclusions from the data. Braun and Clarke also warn of poor quality in qualitative research, noting that “coherence and integrity of published research does not always hold.” 16

Weak themes

An important distinction between high and low quality themes is that high quality themes are descriptive and complete thoughts. As such, they often contain subjects and verbs, and can be expressed as full sentences ( table 2 ). Themes that are simply descriptive categories or topics could fail to impart meaningful knowledge beyond categorisation. 16 49 50

Researchers will often move from coding directly to writing up themes, without performing the work of theming or hosting a thematic analysis session. Skipping concerted theming often results in themes that look more like categories than unifying threads across the data.

Unfocused analysis

Because data collection for qualitative research is often semi-structured (eg, interviews, focus groups), not all data will be directly relevant to the research question at hand. To avoid unfocused analysis and a correspondingly unfocused manuscript, we recommend that all team members keep the research objective in front of them at every stage, from reading to coding to theming. During the thematic analysis session, we recommend that the research question be written on a whiteboard so that all team members can refer back to it, and so that the facilitator can ensure that conversations about themes occur in the context of this question. Consistently focusing on the research question can help to ensure that the final report directly answers it, as opposed to the many other interesting insights that might emerge during the qualitative research process. Such insights can be picked up in a secondary analysis if desired.

Inappropriate quantification

Presenting findings quantitatively (eg, “We found 18 instances of participants mentioning safety concerns about the vaccines”) is generally undesirable in practical thematic analysis reporting. 51 Descriptive terms are more appropriate (eg, “participants had substantial concerns about the vaccines,” or “several participants were concerned about this”). This descriptive presentation is critical because qualitative data might not be consistently elicited across participants, meaning that some individuals might share certain information while others do not, simply based on how conversations evolve. Additionally, qualitative research does not aim to draw inferences outside its specific sample. Emphasising numbers in thematic analysis can lead to readers incorrectly generalising the findings. Although peer reviewers unfamiliar with thematic analysis often request this type of quantification, practitioners of practical thematic analysis can confidently defend their decision to avoid it. If quantification is methodologically important, we recommend simultaneously conducting a survey or incorporating standardised interview techniques into the interview guide. 11

Neglecting group dynamics

Researchers should concertedly consider group dynamics in the research team. Particular attention should be paid to power relations and the personality of team members, which can include aspects such as who most often speaks, who defines concepts, and who resolves disagreements that might arise within the group. 52

The perspectives of patient and care partners are particularly important to cultivate. Ideally, patient partners are meaningfully embedded in studies from start to finish, not just for practical thematic analysis. 53 Meaningful engagement can build trust, which makes it easier for patient partners to ask questions, request clarification, and share their perspectives. Professional team members should actively encourage patient partners by emphasising that their expertise is critically important and valued. Noting when a patient partner might be best positioned to offer their perspective can be particularly powerful.

Insufficient time allocation

Researchers must allocate enough time to complete thematic analysis. Working with qualitative data takes time, especially because it is often not a linear process. As the strength of thematic analysis lies in its ability to make use of the rich details and complexities of the data, we recommend careful planning for the time required to read and code each document.

Estimating the necessary time can be challenging. For step 1 (reading), researchers can roughly calculate the time required based on the time needed to read and reflect on one piece of data. For step 2 (coding), the total amount of time needed can be extrapolated from the time needed to code one document during codebook development. We also recommend three to five hours for the thematic analysis session itself, although coders will need to independently develop their draft themes beforehand. Although the time required for practical thematic analysis is variable, teams should be able to estimate their own required effort with these guidelines.

Practical thematic analysis builds on the foundational work of Braun and Clarke. 4 16 We have reframed their six phase process into three condensed steps of reading, coding, and theming. While we have maintained important elements of Braun and Clarke’s reflexive thematic analysis, we believe that practical thematic analysis is conceptually simpler and easier to teach to less experienced researchers and non-researcher stakeholders. For teams with different levels of familiarity with qualitative methods, this approach presents a clear roadmap to the reading, coding, and theming of qualitative data. Our practical thematic analysis approach promotes efficient learning by doing—experiential learning. 12 29 Practical thematic analysis avoids the risk of relying on complex descriptions of methods and theory and places more emphasis on obtaining meaningful insights from those close to real world clinical environments. Although practical thematic analysis can be used to perform intensive theory based analyses, it lends itself more readily to accelerated, pragmatic approaches.

Strengths and limitations

Our approach is designed to smooth the qualitative analysis process and yield high quality themes. Yet, researchers should note that poorly performed analyses will still produce low quality results. Practical thematic analysis is a qualitative analytical approach; it does not look at study design, data collection, or other important elements of qualitative research. It also might not be the right choice for every qualitative research project. We recommend it for applied health services research questions, where diverse perspectives and simplicity might be valuable.

We also urge researchers to improve internal validity through triangulation methods, such as member checking (supplemental material 1). 46 Member checking could include soliciting input on high level themes, theme definitions, and quotations from participants. This approach might increase rigor.

Implications

We hope that by providing clear and simple instructions for practical thematic analysis, a broader range of researchers will be more inclined to use these methods. Increased transparency and familiarity with qualitative approaches can enhance researchers’ ability to both interpret qualitative studies and offer up new findings themselves. In addition, it can have usefulness in training and reporting. A major strength of this approach is to facilitate meaningful inclusion of patient and care partner perspectives, because their lived experiences can be particularly valuable in data interpretation and the resulting findings. 11 30 As clinicians are especially pressed for time, they might also appreciate a practical set of instructions that can be immediately used to leverage their insights and access to patients and clinical settings, and increase the impact of qualitative research through timely results. 8

Practical thematic analysis is a simplified approach to performing thematic analysis in health services research, a field where the experiences of patients, care partners, and clinicians are of inherent interest. We hope that it will be accessible to those individuals new to qualitative methods, including patients, care partners, clinicians, and other health services researchers. We intend to empower multidisciplinary research teams to explore unanswered questions and make new, important, and rigorous contributions to our understanding of important clinical and health systems research.

Acknowledgments

All members of the Coproduction Laboratory provided input that shaped this manuscript during laboratory meetings. We acknowledge advice from Elizabeth Carpenter-Song, an expert in qualitative methods.

Coproduction Laboratory group contributors: Stephanie C Acquilano ( http://orcid.org/0000-0002-1215-5531 ), Julie Doherty ( http://orcid.org/0000-0002-5279-6536 ), Rachel C Forcino ( http://orcid.org/0000-0001-9938-4830 ), Tina Foster ( http://orcid.org/0000-0001-6239-4031 ), Megan Holthoff, Christopher R Jacobs ( http://orcid.org/0000-0001-5324-8657 ), Lisa C Johnson ( http://orcid.org/0000-0001-7448-4931 ), Elaine T Kiriakopoulos, Kathryn Kirkland ( http://orcid.org/0000-0002-9851-926X ), Meredith A MacMartin ( http://orcid.org/0000-0002-6614-6091 ), Emily A Morgan, Eugene Nelson, Elizabeth O’Donnell, Brant Oliver ( http://orcid.org/0000-0002-7399-622X ), Danielle Schubbe ( http://orcid.org/0000-0002-9858-1805 ), Gabrielle Stevens ( http://orcid.org/0000-0001-9001-178X ), Rachael P Thomeer ( http://orcid.org/0000-0002-5974-3840 ).

Contributors: Practical thematic analysis, an approach designed for multidisciplinary health services teams new to qualitative research, was based on CHS’s experiences teaching thematic analysis to clinical teams and students. We have drawn heavily from qualitative methods literature. CHS is the guarantor of the article. CHS, AS, CvP, AMK, JRK, and JAP contributed to drafting the manuscript. AS, JG, CMM, JAP, and RWY provided feedback on their experiences using practical thematic analysis. CvP, LCL, SLB, AVC, GE, and JKL advised on qualitative methods in health services research, given extensive experience. All authors meaningfully edited the manuscript content, including AVC and RKS. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This manuscript did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Provenance and peer review: Not commissioned; externally peer reviewed.

• Ziebland S ,
• ↵ A Hybrid Approach to Thematic Analysis in Qualitative Research: Using a Practical Example. 2018. https://methods.sagepub.com/case/hybrid-approach-thematic-analysis-qualitative-research-a-practical-example .
• Maguire M ,
• Vindrola-Padros C ,
• Vindrola-Padros B
• ↵ Vindrola-Padros C. Rapid Ethnographies: A Practical Guide . Cambridge University Press 2021. https://play.google.com/store/books/details?id=n80HEAAAQBAJ
• Schroter S ,
• Merino JG ,
• Barbeau A ,
• ↵ Padgett DK. Qualitative and Mixed Methods in Public Health . SAGE Publications 2011. https://play.google.com/store/books/details?id=LcYgAQAAQBAJ
• Scharp KM ,
• Korstjens I
• Barnett-Page E ,
• ↵ Guest G, Namey EE, Mitchell ML. Collecting Qualitative Data: A Field Manual for Applied Research . SAGE 2013. https://play.google.com/store/books/details?id=-3rmWYKtloC
• Sainsbury P ,
• Emerson RM ,
• Saunders B ,
• Kingstone T ,
• Hennink MM ,
• Kaiser BN ,
• Hennink M ,
• O’Connor C ,
• ↵ Yen RW, Schubbe D, Walling L, et al. Patient engagement in the What Matters Most trial: experiences and future implications for research. Poster presented at International Shared Decision Making conference, Quebec City, Canada. July 2019.
• ↵ Got questions about Thematic Analysis? We have prepared some answers to common ones. https://www.thematicanalysis.net/faqs/ (accessed 9 Nov 2022).
• ↵ Braun V, Clarke V. Thematic Analysis. SAGE Publications. 2022. https://uk.sagepub.com/en-gb/eur/thematic-analysis/book248481 .
• Kalpokas N ,
• Radivojevic I
• Campbell KA ,
• Durepos P ,
• ↵ Understanding Thematic Analysis. https://www.thematicanalysis.net/understanding-ta/ .
• Saunders CH ,
• Stevens G ,
• CONFIDENT Study Long-Term Care Partners
• MacQueen K ,
• Vaismoradi M ,
• Turunen H ,
• Schott SL ,
• Berkowitz J ,
• Carpenter-Song EA ,
• Goldwag JL ,
• Durand MA ,
• Goldwag J ,
• Saunders C ,
• Mishra MK ,
• Rodriguez HP ,
• Shortell SM ,
• Verdinelli S ,
• Scagnoli NI
• Campbell C ,
• Sparkes AC ,
• McGannon KR
• Sandelowski M ,
• Connelly LM ,
• O’Malley AJ ,

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Do Thematic Analysis | Step-by-Step Guide & Examples

How to Do Thematic Analysis | Step-by-Step Guide & Examples

Published on September 6, 2019 by Jack Caulfield . Revised on June 22, 2023.

Thematic analysis is a method of analyzing qualitative data . It is usually applied to a set of texts, such as an interview or transcripts . The researcher closely examines the data to identify common themes – topics, ideas and patterns of meaning that come up repeatedly.

There are various approaches to conducting thematic analysis, but the most common form follows a six-step process: familiarization, coding, generating themes, reviewing themes, defining and naming themes, and writing up. Following this process can also help you avoid confirmation bias when formulating your analysis.

This process was originally developed for psychology research by Virginia Braun and Victoria Clarke . However, thematic analysis is a flexible method that can be adapted to many different kinds of research.

Table of contents

When to use thematic analysis, different approaches to thematic analysis, step 1: familiarization, step 2: coding, step 3: generating themes, step 4: reviewing themes, step 5: defining and naming themes, step 6: writing up, other interesting articles.

Thematic analysis is a good approach to research where you’re trying to find out something about people’s views, opinions, knowledge, experiences or values from a set of qualitative data – for example, interview transcripts , social media profiles, or survey responses .

Some types of research questions you might use thematic analysis to answer:

• How do patients perceive doctors in a hospital setting?
• What are young women’s experiences on dating sites?
• What are non-experts’ ideas and opinions about climate change?
• How is gender constructed in high school history teaching?

To answer any of these questions, you would collect data from a group of relevant participants and then analyze it. Thematic analysis allows you a lot of flexibility in interpreting the data, and allows you to approach large data sets more easily by sorting them into broad themes.

However, it also involves the risk of missing nuances in the data. Thematic analysis is often quite subjective and relies on the researcher’s judgement, so you have to reflect carefully on your own choices and interpretations.

Pay close attention to the data to ensure that you’re not picking up on things that are not there – or obscuring things that are.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once you’ve decided to use thematic analysis, there are different approaches to consider.

There’s the distinction between inductive and deductive approaches:

• An inductive approach involves allowing the data to determine your themes.
• A deductive approach involves coming to the data with some preconceived themes you expect to find reflected there, based on theory or existing knowledge.

Ask yourself: Does my theoretical framework give me a strong idea of what kind of themes I expect to find in the data (deductive), or am I planning to develop my own framework based on what I find (inductive)?

There’s also the distinction between a semantic and a latent approach:

• A semantic approach involves analyzing the explicit content of the data.
• A latent approach involves reading into the subtext and assumptions underlying the data.

Ask yourself: Am I interested in people’s stated opinions (semantic) or in what their statements reveal about their assumptions and social context (latent)?

After you’ve decided thematic analysis is the right method for analyzing your data, and you’ve thought about the approach you’re going to take, you can follow the six steps developed by Braun and Clarke .

The first step is to get to know our data. It’s important to get a thorough overview of all the data we collected before we start analyzing individual items.

This might involve transcribing audio , reading through the text and taking initial notes, and generally looking through the data to get familiar with it.

Next up, we need to code the data. Coding means highlighting sections of our text – usually phrases or sentences – and coming up with shorthand labels or “codes” to describe their content.

Let’s take a short example text. Say we’re researching perceptions of climate change among conservative voters aged 50 and up, and we have collected data through a series of interviews. An extract from one interview looks like this:

In this extract, we’ve highlighted various phrases in different colors corresponding to different codes. Each code describes the idea or feeling expressed in that part of the text.

At this stage, we want to be thorough: we go through the transcript of every interview and highlight everything that jumps out as relevant or potentially interesting. As well as highlighting all the phrases and sentences that match these codes, we can keep adding new codes as we go through the text.

After we’ve been through the text, we collate together all the data into groups identified by code. These codes allow us to gain a a condensed overview of the main points and common meanings that recur throughout the data.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Next, we look over the codes we’ve created, identify patterns among them, and start coming up with themes.

Themes are generally broader than codes. Most of the time, you’ll combine several codes into a single theme. In our example, we might start combining codes into themes like this:

At this stage, we might decide that some of our codes are too vague or not relevant enough (for example, because they don’t appear very often in the data), so they can be discarded.

Other codes might become themes in their own right. In our example, we decided that the code “uncertainty” made sense as a theme, with some other codes incorporated into it.

Again, what we decide will vary according to what we’re trying to find out. We want to create potential themes that tell us something helpful about the data for our purposes.

Now we have to make sure that our themes are useful and accurate representations of the data. Here, we return to the data set and compare our themes against it. Are we missing anything? Are these themes really present in the data? What can we change to make our themes work better?

If we encounter problems with our themes, we might split them up, combine them, discard them or create new ones: whatever makes them more useful and accurate.

For example, we might decide upon looking through the data that “changing terminology” fits better under the “uncertainty” theme than under “distrust of experts,” since the data labelled with this code involves confusion, not necessarily distrust.

Now that you have a final list of themes, it’s time to name and define each of them.

Defining themes involves formulating exactly what we mean by each theme and figuring out how it helps us understand the data.

Naming themes involves coming up with a succinct and easily understandable name for each theme.

For example, we might look at “distrust of experts” and determine exactly who we mean by “experts” in this theme. We might decide that a better name for the theme is “distrust of authority” or “conspiracy thinking”.

Finally, we’ll write up our analysis of the data. Like all academic texts, writing up a thematic analysis requires an introduction to establish our research question, aims and approach.

We should also include a methodology section, describing how we collected the data (e.g. through semi-structured interviews or open-ended survey questions ) and explaining how we conducted the thematic analysis itself.

The results or findings section usually addresses each theme in turn. We describe how often the themes come up and what they mean, including examples from the data as evidence. Finally, our conclusion explains the main takeaways and shows how the analysis has answered our research question.

In our example, we might argue that conspiracy thinking about climate change is widespread among older conservative voters, point out the uncertainty with which many voters view the issue, and discuss the role of misinformation in respondents’ perceptions.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Discourse analysis
• Cohort study
• Peer review
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. (2023, June 22). How to Do Thematic Analysis | Step-by-Step Guide & Examples. Scribbr. Retrieved April 1, 2024, from https://www.scribbr.com/methodology/thematic-analysis/

Is this article helpful?

Jack Caulfield

Other students also liked, what is qualitative research | methods & examples, inductive vs. deductive research approach | steps & examples, critical discourse analysis | definition, guide & examples, what is your plagiarism score.

Educational resources and simple solutions for your research journey

How to Write a Thematic Literature Review: A Beginner’s Guide

Literature reviews provide a comprehensive understanding of existing knowledge in a particular field, offer insights into gaps and trends, and ultimately lay the foundation for innovative research. However, when tackling complex topics spanning multiple issues, the conventional approach of a standard literature review might not suffice. Many researchers present a literature review without giving any thought to its organization or structure, but this is where a thematic literature review comes into play. In this article, we will explore the significance of thematic reviews, delve into how and when to undertake them, and offer invaluable guidance on structuring and crafting a compelling thematic literature review.

Table of Contents

What is a thematic literature review?

A thematic literature review, also known as a thematic review, involves organizing and synthesizing the existing literature based on recurring themes or topics rather than a chronological or methodological sequence. Typically, when a student or researcher works intensively on their research there are many sub-domains or associated spheres of knowledge that one encounters. While these may not have a direct bearing on the main idea being explored, they provide a much-needed background or context to the discussion. This is where a thematic literature review is useful when dealing with complex research questions that involve multiple facets, as it allows for a more in-depth exploration of specific themes within the broader context.

When to opt for thematic literature review?

It is common practice for early career researchers and students to collate all the literature reviews they have undertaken under one single broad umbrella. However, when working on a literature review that involves multiple themes, lack of organization and structure can slow you down and create confusion. Deciding to embark on a thematic literature review is a strategic choice that should align with your research objectives. Here are some scenarios where opting for a thematic review is advantageous:

• Broad Research Questions: When your research question spans across various dimensions and cannot be adequately addressed through a traditional literature review.
• Interdisciplinary Research: In cases where your research draws from multiple disciplines, a thematic review helps in synthesizing diverse literature cohesively.
• Emerging Research Areas: When exploring emerging fields or topics with limited existing literature, a thematic review can provide valuable insights by focusing on available themes.
• Complex Issues: Thematic reviews are ideal for dissecting complex issues with multiple contributing factors or dimensions.

Advantages of a Thematic Literature Review

With better comprehension and broad insights, thematic literature reviews can help in identifying possible research gaps across themes. A thematic literature review has several advantages over a general or broad-based approach, especially for those working on multiple related themes.

• It provides a comprehensive understanding of specific themes within a broader context, allowing for a deep exploration of relevant literature.
• Thematic reviews offer a structured approach to organizing and synthesizing diverse sources, making it easier to identify trends, patterns, and gaps.
• Researchers can focus on key themes, enabling a more detailed analysis of specific aspects of the research question.
• Thematic reviews facilitate the integration of literature from various disciplines, offering a holistic view of the topic.
• Researchers can provide targeted recommendations or insights related to specific themes, aiding in the formulation of research hypotheses.

Now that we know the benefits of a thematic literature review, what is the best way to arrange reviewed literature in a thematic format?

How to write a thematic literature review

To effectively structure and write a thematic literature review, follow these key steps:

• Define Your Research Question: Clearly define the overarching research question or topic you aim to explore thematically. When writing a thematic literature review, go through different literature review sections of published research work and understand the subtle nuances associated with this approach.
• Identify Themes: Analyze the literature to identify recurring themes or topics relevant to your research question. Categorize the bibliography by dividing them into relevant clusters or units, each dealing with a specific issue. For example, you can divide a topic based on a theoretical approach, methodology, discipline or by epistemology. A theoretical review of related literature for example, may also look to break down geography or issues pertaining to a single country into its different parts or along rural and urban divides.
• Organize the Literature: Group the literature into thematic clusters based on the identified themes. Each cluster represents a different aspect of your research question. It is up to you to define the different narratives of thematic literature reviews depending on the project being undertaken; there is no one formal way of doing this. You can weigh how specific areas stack up against others in terms of existing literature or studies and how many more aspects may need to be added or further looked into.
• Review and Synthesize: Within each thematic cluster, review and synthesize the relevant literature, highlighting key findings and insights. It is recommended to identify any theme-related strengths or weaknesses using an analytical lens.
• Integrate Themes: Analyze how the themes interact with each other, draw linkages between earlier studies and see how they contribute to your own research. A thematic literature review presents readers with a comprehensive overview of the literature available on and around the research topic.
• Provide a Framework: Develop a framework or conceptual model that illustrates the relationships between the themes. Present the most relevant part of the thematic review toward the end and study it in greater detail as it reflects the literature most relevant and directly related to the main research topic.
• Conclusion: Conclude your thematic literature review by summarizing the key findings and their implications for your research question. Be sure to highlight any gaps or areas requiring further investigation in this section.
• Cite and Reference: It is important to remember that a thematic review of literature for a PhD thesis or research paper lends greater credibility to the student or researcher. So ensure that you properly cite and reference all sources according to your chosen citation style.
• Edit and Proofread: Take some time to review your work, ensure proper structure and flow and eliminate any language, grammar, or spelling errors that could deviate reader attention. This will help you deliver a well-structured and elegantly written thematic literature review.

Thematic literature review example

In essence, a thematic literature review allows researchers to dissect complex topics into smaller manageable themes, providing a more focused and structured approach to literature synthesis. This method empowers researchers to gain deeper insights, identify gaps, and generate new knowledge within the context of their research.

To illustrate the process mentioned above, let’s consider an example of a thematic literature review in the context of sustainable development. Imagine the overarching research question is: “What are the key factors influencing sustainable urban planning?” Potential themes could include environmental sustainability, social equity, economic viability, and governance. Each theme would have a dedicated section in the review, summarizing relevant literature and discussing how these factors intersect and impact sustainable urban planning. Close with a strong conclusion that highlights research gaps or areas of investigation. Finally, review and refine the thematic literature review, adding citations and references as required.

In conclusion, when tackling multifaceted research questions, a thematic literature review proves to be an indispensable tool for researchers and students alike. By adopting this approach, scholars can navigate the intricate web of existing literature, unearth meaningful patterns, and contribute to the advancement of knowledge in their respective fields. We hope the information in this article helps you create thematic reviews that illuminate your path to new discoveries and innovative insights.

R Discovery is a literature search and research reading app that uses your interests to instantly create personalized reading feeds. Researchers can stay updated on the latest, most relevant content from its continually expanding library of 115M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex as well as prestigious publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more. The top-rated app in its space, R Discovery’s carefully curated features give you the power to choose what, where, and how you read research.     

Try the app for free or upgrade to R Discovery Prime , which unlocks unlimited access to app-only features that let you listen to research on the go, read in your language, invite collaborators, auto sync with reference managers and more. It’s like having the world of research at your fingertips! Choose a simpler, smarter way to find and read research – Get R Discovery Prime now at just US$39 a year!    

Related Posts

Essay Writing Basics: Strategies for PhD Success 

What is Inductive Reasoning? Definition, Types and Examples

• Open access
• Published: 18 April 2018

Protocol for a systematic review and thematic synthesis of patient experiences of central venous access devices in anti-cancer treatment

• Caoimhe Ryan   ORCID: orcid.org/0000-0001-7279-563X 1 ,
• Hannah Hesselgreaves 2 ,
• Olivia Wu 1 ,
• Jim Paul 3 ,
• Judith Dixon-Hughes 3 &
• Jonathan G. Moss 4  

Systematic Reviews volume  7 , Article number:  61 ( 2018 ) Cite this article

27k Accesses

26 Citations

6 Altmetric

Metrics details

Three types of central venous access devices (CVADs)—peripherally inserted central catheters (PICCs), skin-tunnelled central catheters (Hickman-type devices), and implantable chest wall Ports (Ports)—are routinely used in the intravenous administration of anti-cancer treatment. These devices avoid the need for peripheral cannulation and allow for home delivery of treatment. Assessments of these devices have tended to focus on medical and economic factors, but there is increased interest in the importance of patient experiences and perspectives in this area. The aim of this systematic review is to synthesise existing research regarding patient experiences of these CVADs to help clinicians guide, prepare, and support patients receiving CVADs for the administration of anti-cancer treatment.

A systematic search of MEDLINE, Embase, and CINAHL research databases will be carried out along with a supplementary reference list search. This review will include quantitative, qualitative, and mixed methods studies published in peer-review journals, reporting some aspect(s) of patient experiences or perspectives regarding the use of PICC, Hickman, or Port CVADs for the administration of anti-cancer drugs. The methodological quality and risk of bias of included papers will be assessed using the Mixed Methods Appraisal Tool (MMAT). Relevant outcome data will be extracted from included studies and analysed using a thematic synthesis approach.

The results section of the review will comprise thematic synthesis of quantitative studies, thematic synthesis of qualitative studies, and the aggregation of the two. Results will aim to offer an account of current understandings of patient experiences and perspective regarding PICC, Hickman-type, and Port devices in the context of anti-cancer treatment. Confidence in cumulative evidence will be assessed using the Confidence in the Evidence from Reviews of Qualitative research (CERQual) approach.

Systematic review registration

This systematic review protocol is registered with the International Prospective Register of Systematic Reviews (PROSPERO). Registration number: CRD42017065851 . This protocol was prepared using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols checklist (PRISMA-P) (Shamseer et al., BMJ 349: 2015).

Peer Review reports

Three types of central venous access devices (CVADs) are routinely used in the intravenous administration of anti-cancer treatment: peripherally inserted central catheters (PICCs), skin-tunnelled central catheters (Hickman-type devices), and implantable chest wall Ports (Ports).

All three deliver the drug into the superior vena cava, which drains directly into the right atrium of the heart. PICC lines are inserted into a peripheral vein in the arm. The end of the line remains outside the body, emerging from a point usually above the elbow. Hickman-type devices are inserted into a central vein in the neck or upper chest. The end of the line is tunnelled under the skin to emerge from the chest, where it sits outside the body. Chest wall Ports constitute a closed-system that is fully implanted in the body with no external lines. Vascular access is via needle puncture.

The main benefit of the devices is that they avoid the need for peripheral cannulation which relies on peripheral arm veins that easily become occluded. Moreover, peripheral cannulation usually involves repeated needle insertions which patients tend to find unpleasant, even painful and distressing [ 1 ]. They also allow for home delivery of treatment via portable chemotherapy infusion pump.

Patient-centred approaches in the area of vascular access exist but are limited. Published reports on the use of vascular access devices in the context of anti-cancer treatment have tended to focus on medical and economic factors such as cost, maintenance, infection, and other complications [ 2 , 3 ], with much less attention given to patient experiences and perspectives. It is therefore difficult to gain a clear picture of how patients experience PICC, Hickman, and Port devices [ 4 ]. Developing a better understanding of these experiences would offer important insight into the roles these devices play in patients’ quality of life and in the burden of treatment. Moreover, a comparison of the effects and experiences of these different devices would be important in guiding device selection where more than one device is clinically suitable.

To our knowledge no systematic review has previously been carried out on this topic. The present review will draw together existing approaches to recording and assessing patient experiences of these CVADs in order to produce a coherent understanding of such experiences. This will help clinicians guide, prepare, and support patients receiving CVADs for the administration of anti-cancer treatment.

The aim of this review is to synthesise existing literature that investigates patient experiences with PICC, Hickman-type, and Port devices in the context of anti-cancer treatment.

This review will address the following questions: What are patients’ experiences and perspectives regarding the three different central venous access devices implanted for delivering long-term anti-cancer treatment: peripherally inserted central catheters (PICC), Hickman-type tunnelled catheters, and totally implanted chest wall Ports? What effect (if any) do these CVADs have on patients’ quality of life in the context of anti-cancer treatment, and what aspects of each device are important in this respect? To what extent is it possible to compare and contrast patient experiences of different CVADs? What are the limitations of the existing literature, and what additional research is needed?

Eligibility criteria

This review will include empirical studies published in peer-review journals meeting review-specific eligibility criteria. These criteria were developed in line with the existing search tools PICO (Population, Intervention, Comparison, Outcome) [ 5 ] and SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 6 ]. PICO is typically used in systematic reviews of quantitative research, answering clinical questions. SPIDER was adapted from the former for use in systematic reviews of qualitative and mixed method research. As our review will comprise quantitative, qualitative, and mixed methods studies, our eligibility criteria include elements from both.

As the use of PICCs and chest wall Ports in cancer treatment was not well established until the 1990s and involved a learning curve of several years, this review will consider only studies published from the year 1997 onward.

Participant population

Studies will be eligible for inclusion where participants are adult or paediatric patients (or parents/guardians of paediatric patients) diagnosed with any type of cancer including solid malignancy and haematologic malignancy, who have received PICC, Hickman-type or chest wall Port CVADs for the primary purpose of the administration of anti-cancer drugs.

Intervention

Included studies will involve the implantation (via any technique) of any of the following venous access devices for the long-term administration of any anti-cancer treatment: (i) peripherally inserted central catheters (PICC), (ii) subcutaneously tunnelled central catheters (Hickman-type device), and (iii) implantable chest wall Ports (Port).

Comparator/control

Studies will not be included or excluded on the basis of comparator. Where relevant and appropriate, comparisons between devices or other forms of venous access in the administration of anti-cancer treatment will be considered in analysis.

The review will include empirical studies using quantitative and/or qualitative analytic methods. It will exclude non-empirical studies but will not limit inclusion based on study design. Studies that report outcomes regarding CVADs but do not report or discuss any patient experience will also be excluded, as will studies that do not provide sufficient detail for a thematic synthesis—i.e. studies that do not contribute to themes or where data cannot be themed.

This review will include studies conducted in oncology settings such as hospitals, and clinics, or related contexts including home-treatment, patient care facilities, or patient groups/organisations.

Evaluation/outcomes

Finally, to be eligible for inclusion study, participants must have completed a measure (e.g. questionnaire, survey) or provided an account (e.g. interview, focus group) of some aspect of their own experience with or perspectives on CVADs or in the case of paediatric patients, where parents/guardians similarly report on some aspect of their own and their children’s experiences and perspectives (please see ‘ Outcomes and Prioritisation ’ section, below, for explanations of these outcomes).

Information sources

A systematic search strategy will be used to search MEDLINE, Embase, and CINAHL research databases. A further important component of our search strategy, especially for the purposes of identifying relevant qualitative research, involves forward and backward citation tracking of publications meeting the inclusion criteria, via Web of Science and Google Scholar. In addition, experts including authors of key papers will be contacted to minimise the likelihood of overlooking key sources.

Search strategy

Search terms will target three key domains: (i) central venous access devices, (ii) patient population, and (iii) patient experiences. Search terms for Hickman and Port devices and for cancer and chemotherapy will be adapted from an existing systematic review comparing the clinical effectiveness of these devices [ 7 ]. The search strategy has been constructed with advice and guidance from an information scientist. An example of search terms and strategy is provided in Appendix .

Data management

Details of all searches will be recorded. Search results will be downloaded to EndNote desktop software. Studies sourced through supplemental hand searching will be recorded and imported into EndNote.

Selection process

Duplicate publications will be removed. The titles and abstracts of all remaining publications will be assessed against inclusion criteria. Those not meeting these criteria will be excluded. All titles and abstracts identified at this stage will also be screened independently by a second reviewer. Disparities will be resolved through discussion or in consultation with a third party. Full texts of the remaining publications will be obtained for further review. Reasons for further exclusions at this stage will be documented. The selection process, including search results and reasons for exclusion at each stage of screening, will be recorded and represented in a PRISMA flow diagram (Additional file  1 )[ 8 ].

Data collection process

Data extraction will be performed by one reviewer using a data extraction form developed by the researchers for the purposes of this review. This form will be refined by the reviewer until the data extraction is complete, to ensure the appropriateness and usefulness of all fields.

For all studies, descriptive data will be extracted regarding type(s) of vascular access device included in the study, patient population, sample size, stated research aims, study context, study design, methodology, reason for measurement of patient experience, and timing of measurement (in relation to device placement).

For quantitative studies, outcome data including descriptive and inferential statistics will be extracted for measures of patient experiences or perspectives regarding CVADs. Details of measurement instruments will also be recorded. The purpose of qualitative synthesis is to ‘go beyond’ primary studies. This means the data to be extracted from the qualitative studies comprises the analyses and interpretations of the study authors including all themes, categories, theories, models, and similar. Therefore, all material labelled as results or findings by the authors as well as subsequent discussions and conclusions relating to CVADs will be extracted in full [ 9 ].

Outcomes and prioritisation

The primary outcomes of interest will be any aspect of patients’ (or parent/guardians’) self-reported experiences or perspectives relating to PICC, Hickman, or Port devices. The data to be collected will include quantitative and qualitative materials, i.e. survey or questionnaire data, qualitative themes, and participant quotations.

For our present purposes, ‘patient perspectives’ will include opinions, attitudes, and evaluations (including acceptance and satisfaction). Patient experiences will include emotions, physical sensations (e.g. pain, discomfort), psychological factors (e.g. stress, mood), and pragmatic factors (e.g. routine activities).

Our analyses will prioritise any of the above as they relate to device acceptance or preference or to patient wellbeing or quality of life. Secondary prioritisation will be given to any of the above as they relate to device placement (insertion).

Risk of bias in individual studies

Critical appraisal of methodological quality and risk of bias of included papers will be undertaken independently by two reviewers according to the Mixed Methods Appraisal Tool (MMAT) developed by Pluye and colleagues [ 10 ] for the appraisal of qualitative, quantitative, and mixed methods research. Following these guidelines, the reviewers will assess for all included studies whether (i) there are clear research questions and (ii) the data collected address the research question. The following assessments will then be made, depending on study type:

Qualitative studies

The reviewers will assess the appropriateness of data sources and analytical processes, the study’s consideration of context/setting, and the study’s consideration of researchers’ influence.

Randomised controlled trials

The reviewers will assess the study’s descriptions of randomisation and allocation concealment, the completeness of outcome data, and the level of drop-out rates.

Nonrandomised quantitative studies

The reviewers will assess the study’s minimisation of selection bias, use of appropriate measurement instruments, use of comparable groups across study conditions, and the completeness of outcome data.

Descriptive cross-sectional quantitative studies

The reviewers will assess the appropriateness of the study’s sampling strategy and the representativeness of the sample, use of appropriate measurement instruments, and acceptability of response rate.

Data analysis and synthesis

Data will be analysed using a narrative approach, specifically thematic synthesis. Our analysis will comprise three phases. The first phase will be a thematic synthesis of the patient experience and perspective data extracted from the quantitative studies selected for inclusion in our review. We use inclusive definitions of patient experience and perspectives in this review. Our search strategy therefore includes a range of broad search terms relating to these phenomena. In addition, to our knowledge, there are no validated or standardised measures relating to patient experiences of CVADs in common usage. For instance, one such measure recently developed in France (specific to Port devices) [ 11 ] has to date been used in a single pilot study [ 12 ]. Consequently, we anticipate significant heterogeneity with regard to quantitative outcomes, and these data will not be quantitatively synthesised. The second phase of analysis will be a thematic synthesis of the qualitative studies selected for inclusion. The final third stage will be an aggregation of both sets of analysis. In the interests of transparency, detailed accounts of each stage of analysis will be recorded. The second phase of analysis will be a thematic synthesis of the patient experience and perspective data extracted from qualitative studies selected for inclusion. The final third stage will be an aggregation of both sets of analysis. In the interests of transparency, detailed accounts of each stage of analysis will be recorded.

Analytic method

Thematic synthesis is an adaptation for the purpose of secondary data synthesis of ‘thematic analysis’ and provides a set of established methods and techniques for the identification and development of analytic themes in primary research data [ 9 ]. Thematic synthesis was selected for the purposes of this review for a several reasons [ 13 ]. First, it is well suited to our present objective of aggregating existing evidence and identifying patterns within data. Second, whilst it is most commonly associated with the synthesis of qualitative research outcomes, thematic synthesis is also used for the synthesis of quantitative research outcomes, particularly where there is heterogeneity in outcome variables and measurements. Finally, the process of thematic synthesis offers good transparency and outcomes are accessible.

Thematic synthesis involves three stages of analysis (all three stages will be applied to all included quantitative studies and repeated for all included qualitative studies). First, the data—here, those pertaining to patient experiences and perspectives—are coded. A coding frame will be developed comprising codes derived from the data. Coding will be carried out by the first reviewer and checked by a second reviewer. Disparities or discrepancies in coding will be resolved through discussion or in consultation with a third party if necessary; the coding frame will be adjusted accordingly.

In the second stage of analysis, similarities between codes will be identified. Codes will be grouped into ‘descriptive themes’ that capture and describe patterns in the data across studies. Each theme will be entered as columns into a table, and coded data from each study will illustrate the themes in rows, to facilitate comparison within and between studies, as part of the constant comparison analytic process [ 14 , 15 ]. Memos will be used as part of the analysis process and may be included in the summary table where clarity may be required about the interpretation of a piece of evidence [ 16 ]. The aim of this table is to demonstrate themes with illustrative data and capture similarities and differences within the data where possible—that is, to show how themes are generated but also show divergence of findings in each theme, where it applies.

The third stage of thematic synthesis will involve the development of analytic themes. The purpose of this phase of analysis is to ‘go beyond’ the primary reported data by synthesising findings across studies and interpreting their meaning in relation to our review research questions. This will comprise the narrative component of our analysis, which will provide narrative descriptions of each theme.

Once thematic synthesis has been completed for quantitative studies and for qualitative studies, a final phase of analysis will compare both sets of results, and the three central venous access devices included in the review. This process will aggregate results, providing an overall account of current understandings of patient experiences and perspectives of CVADs in the context of anti-cancer treatment.

Confidence in cumulative evidence

Confidence in discrete review findings will be assessed using the recently developed Confidence in the Evidence from Reviews of Qualitative research (CERQual) approach [ 17 ]. Assessment of confidence in a given review finding involves evaluating how likely it is that the finding represents a real phenomenon, i.e. genuine patient experiences of CVADs. This assessment will be based upon an evaluation of the following: (i) methodological limitations of the primary studies contributing to the finding, (ii) the relevance of the primary contributing studies with regard to the objectives of the systematic review, (iii) the coherence of the finding, and (iv) the adequacy of data supporting the finding.

For the purposes of this study, these evaluations will be conducted as follows:

Methodological limitations of contributing studies: With reference to our prior critical appraisal of methodological quality of these studies (see above), we will evaluate the extent to which the primary studies contributing to the finding are generally of good methodological quality.

Relevance of contributing studies: We will evaluate the extent to which the primary studies contributing to the finding are similar in context and aims/objectives to the present review. For instance, studies carried out with the primary aim of investigating some aspect of patients’ experiences of CVADs in the context of cancer treatment will be considered highly relevant. Those in which findings relating to patient experiences of CVADs are secondary or incidental to an alternative aim will be considered to be of low relevance.

Coherence of finding: We will assess the extent to which the finding offers a credible explanation for the patterns it describes (e.g. tendencies, relationships between relevant factors). This will involve assessing the consistency or inconsistency of the pattern across various research contexts represented by studies included in the review and the ability of the finding to account for notable variations across contexts. Findings will be judged to be of low coherence if they describe inconsistent or contradictory patterns and where inconsistencies are unexplained.

Adequacy of supporting data: We will assess the extent to which a given finding is supported by substantial evidence defined both in terms of number of studies contributing to a given study and the quantitative robustness or qualitative richness (i.e. fullness and depth) of the data provided by these studies.

The above evaluations will necessarily entail subjective judgements and will therefore be carried out by two reviewers working in collaboration. A summary table will list for each review finding—primary contributing studies, evaluations of the above four domains, an overall confidence rating (high, moderate, low, or very low), and an explanation of the rating judgement.

The results section of the review will summarise the findings of (i) the thematic synthesis of quantitative studies, (ii) the thematic synthesis of qualitative studies, and (iii) the aggregation (comparison and juxtaposition) of both sets of findings.

The results will offer an account of current understandings of patient experiences and perspective regarding PICC, Hickman-type, and Port devices in the context of anti-cancer treatment. They will explore the extent to which and/or the ways in which these devices affect patients’ quality of life. Comparisons between experiences of different devices, or between especially positive and negative experiences, will be explored as appropriate and similarities and differences discussed and tabulated, if the data lend themselves to this. Not all relevant comparisons can be identified in advance, so other comparisons may become important. Finally, the results of this review will consider the gaps in the existing literature and of ways in which existing data can be used to support patients.

Abbreviations

Confidence in the Evidence from Reviews of Qualitative research

Central venous access device

Mixed Methods Appraisal Tool for the appraisal of qualitative, quantitative, and mixed methods research

Peripherally inserted central catheter

Framework for the development of systematic literature searches. Acronym derived from key aspects of empirical studies: Population, Intervention, Comparison, Outcome

Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols

Adaptation of PICO for the development of systematic searches of qualitative literature. Acronym derived from key aspects of empirical studies: Sample, Phenomenon of Interest, Design, Evaluation

Robinson-Reilly M, Paliadelis P, Cruickshank M. Venous access: the patient experience. Support Care Cancer. 2016;24(3):1181–7.

Campbell WB, Elworthy S, Peerlinck I, Vanslembroek K, Bangur R. Sites of implantation for central venous access devices (Ports): a study of the experiences and preferences of patients. Eur J Vasc Endovasc Surg. 2004;644:642–4.

Sansivero GE. Features and selection of vascular access devices. Semin Oncol Nurs. 2010;26:88–101.

Article   PubMed   Google Scholar  

Corriere MA, Avise JA, Peterson LA, Stafford JM, Easterling D, Sr SB, Ip E, Hyde AS, Burke GL. Exploring patient involvement in decision making for vascular procedures. J Vasc Surg. 2015;62:1032–1039.e2.

Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123:A12–3.

CAS   PubMed   Google Scholar  

Cooke A, Smith D, Booth A. Beyond PICO. Qual Health Res. 2012;22:1435–43.

Kulkarni S, Wu O, Kasthuri R, Moss JG. Centrally inserted external catheters and totally implantable ports for the delivery of chemotherapy: a systematic review and meta-analysis of device-related complications. Cardiovasc Intervent Radiol. 2014;37:990–1008.

Article   CAS   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Article   PubMed   PubMed Central   Google Scholar  

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45.

Pluye P, Robert E, Cargo M, Bartlett G, O’Cathain A, Griffiths F, Boardman F, Gagnon MP, Rousseau MC: Proposal: a mixed methods appraisal tool for systematic mixed studies reviews. 2011. [ http://mixedmethodsappraisaltoolpublic.pbworks.com ].

Google Scholar  

Marcy PY, Mari V, Figl A, Ben-Taarit I, Fouché Y, Peyrade F, Follana P, Michel C, Chamorey E, Marcy PY. Patient satisfaction with and acceptance of their totally-implanted central venous catheter: construction and first validation of a questionnaire. J Cancer Ther. 2014;5:706–16.

Article   Google Scholar  

Burbridge B, Goyal K. Quality-of-life assessment: arm TIVAD versus chest TIVAD. J Vasc Access. 2016;17:527–34.

Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, Van Der Wilt GJ, Mozygemba K, Refolo P, Sacchini D, Tummers M, Rehfuess E: Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. 2016. [ http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-on-the-integrated-integratedassessment-of-complex-health-technologies-the-INTEGRATE-HTA-model.pdf ].

Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. New Brunswick and London: Aldine Pub. Co; 1967.

Strauss A, Corbin C. Basics of qualitative research. Thousand Oaks: Sage Publicatins Ltd; 1998.

Finfgeld-Connett D. Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qual Res. 2014;14:341–52.

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.

Download references

Acknowledgements

Not applicable

Availability of data and materials

Author information, authors and affiliations.

Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK

Caoimhe Ryan & Olivia Wu

School of Medical Education, Newcastle University, Newcastle upon Tyne, UK

Hannah Hesselgreaves

Institute of Cancer Sciences, University of Glasgow, Glasgow, UK

Jim Paul & Judith Dixon-Hughes

Gartnavel General Hospital, Glasgow, UK

Jonathan G. Moss

You can also search for this author in PubMed   Google Scholar

Contributions

CR drafted this manuscript. HH contributed to the design of the protocol and provided input across all sections. OW provided guidance during the initial development of this protocol and offered comments on full drafts of the protocol. JDH was consulted during the development of the protocol and provided comments on full drafts. JP was consulted during the development of the protocol and provided comments on full drafts. JM offered expertise regarding limitations by year for the systematic review described as well as comments on full drafts of the protocol. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Caoimhe Ryan .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

PRISMA-P checklist. (DOCX 35 kb)

Ovid MEDLINE(R) via Ovid 1946 to Week 2 2017

exp Vascular Access Devices/

Catheterization, Central Venous/

(central venous adj5 (catheter* or access)).tw.

percutaneous catheter*.tw.

(venous adj5 (catheter* or access)).tw.

vascular access.tw.

hickman*.tw.

((perhiperally adj5 catheter*) or PICC*).tw.

(implant* adj5 catheter*).tw.

(venous adj5 Port*).tw.

(Portacath* or Port-a-cath*).tw.

(implant* adj5 Port*).tw.

(implant* adj5 reservoir*).tw.

(subcutaneous* adj5 Port*).tw.

(tunnel* adj5 catheter*).tw.

exp Neoplasms/

chemotherapy.tw.

(tumor* or tumour*).tw.

cancer*.tw.

malignan*.tw.

oncolog*.tw.

exp Patient Satisfaction/

(patient* adj5 experience*).tw.

(patient* adj5 perspective*).tw.

(patient* adj5 view*).tw.

(patient* adj5 attitude*).tw.

(patient* adj5 opinion*).tw.

(patient* adj5 satisf*).tw.

(patient* adj5 accept*).tw.

(patient* adj5 evaluat*).tw.

(patient* adj5 assess*).tw.

(patient* adj5 choice*).tw.

(patient* adj5 decision*).tw.

(patient* adj5 prefer*).tw.

(patient* adj5 (questionnaire* or survey*)).tw.

16 and 23 and 38

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Ryan, C., Hesselgreaves, H., Wu, O. et al. Protocol for a systematic review and thematic synthesis of patient experiences of central venous access devices in anti-cancer treatment. Syst Rev 7 , 61 (2018). https://doi.org/10.1186/s13643-018-0721-x

Download citation

Received : 19 September 2017

Accepted : 23 March 2018

Published : 18 April 2018

DOI : https://doi.org/10.1186/s13643-018-0721-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Central venous access devices
• Vascular access
• Chemotherapy
• Patient experience
• Patient perspectives

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Research article
• Open access
• Published: 10 July 2008

Methods for the thematic synthesis of qualitative research in systematic reviews

• James Thomas 1 &
• Angela Harden 1  

BMC Medical Research Methodology volume  8 , Article number:  45 ( 2008 ) Cite this article

370k Accesses

3840 Citations

102 Altmetric

Metrics details

There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies.

We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved.

We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses.

We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing.

Peer Review reports

The systematic review is an important technology for the evidence-informed policy and practice movement, which aims to bring research closer to decision-making [ 1 , 2 ]. This type of review uses rigorous and explicit methods to bring together the results of primary research in order to provide reliable answers to particular questions [ 3 – 6 ]. The picture that is presented aims to be distorted neither by biases in the review process nor by biases in the primary research which the review contains [ 7 – 10 ]. Systematic review methods are well-developed for certain types of research, such as randomised controlled trials (RCTs). Methods for reviewing qualitative research in a systematic way are still emerging, and there is much ongoing development and debate [ 11 – 14 ].

In this paper we present one approach to the synthesis of findings of qualitative research, which we have called 'thematic synthesis'. We have developed and applied these methods within several systematic reviews that address questions about people's perspectives and experiences [ 15 – 18 ]. The context for this methodological development is a programme of work in health promotion and public health (HP & PH), mostly funded by the English Department of Health, at the EPPI-Centre, in the Social Science Research Unit at the Institute of Education, University of London in the UK. Early systematic reviews at the EPPI-Centre addressed the question 'what works?' and contained research testing the effects of interventions. However, policy makers and other review users also posed questions about intervention need, appropriateness and acceptability, and factors influencing intervention implementation. To address these questions, our reviews began to include a wider range of research, including research often described as 'qualitative'. We began to focus, in particular, on research that aimed to understand the health issue in question from the experiences and point of view of the groups of people targeted by HP&PH interventions (We use the term 'qualitative' research cautiously because it encompasses a multitude of research methods at the same time as an assumed range of epistemological positions. In practice it is often difficult to classify research as being either 'qualitative' or 'quantitative' as much research contains aspects of both [ 19 – 22 ]. Because the term is in common use, however, we will employ it in this paper).

When we started the work for our first series of reviews which included qualitative research in 1999 [ 23 – 26 ], there was very little published material that described methods for synthesising this type of research. We therefore experimented with a variety of techniques borrowed from standard systematic review methods and methods for analysing primary qualitative research [ 15 ]. In later reviews, we were able to refine these methods and began to apply thematic analysis in a more explicit way. The methods for thematic synthesis described in this paper have so far been used explicitly in three systematic reviews [ 16 – 18 ].

The review used as an example in this paper

To illustrate the steps involved in a thematic synthesis we draw on a review of the barriers to, and facilitators of, healthy eating amongst children aged four to 10 years old [ 17 ]. The review was commissioned by the Department of Health, England to inform policy about how to encourage children to eat healthily in the light of recent surveys highlighting that British children are eating less than half the recommended five portions of fruit and vegetables per day. While we focus on the aspects of the review that relate to qualitative studies, the review was broader than this and combined answering traditional questions of effectiveness, through reviewing controlled trials, with questions relating to children's views of healthy eating, which were answered using qualitative studies. The qualitative studies were synthesised using 'thematic synthesis' – the subject of this paper. We compared the effectiveness of interventions which appeared to be in line with recommendations from the thematic synthesis with those that did not. This enabled us to see whether the understandings we had gained from the children's views helped us to explain differences in the effectiveness of different interventions: the thematic synthesis had enabled us to generate hypotheses which could be tested against the findings of the quantitative studies – hypotheses that we could not have generated without the thematic synthesis. The methods of this part of the review are published in Thomas et al . [ 27 ] and are discussed further in Harden and Thomas [ 21 ].

Qualitative research and systematic reviews

The act of seeking to synthesise qualitative research means stepping into more complex and contested territory than is the case when only RCTs are included in a review. First, methods are much less developed in this area, with fewer completed reviews available from which to learn, and second, the whole enterprise of synthesising qualitative research is itself hotly debated. Qualitative research, it is often proposed, is not generalisable and is specific to a particular context, time and group of participants. Thus, in bringing such research together, reviewers are open to the charge that they de-contextualise findings and wrongly assume that these are commensurable [ 11 , 13 ]. These are serious concerns which it is not the purpose of this paper to contest. We note, however, that a strong case has been made for qualitative research to be valued for the potential it has to inform policy and practice [ 11 , 28 – 30 ]. In our experience, users of reviews are interested in the answers that only qualitative research can provide, but are not able to handle the deluge of data that would result if they tried to locate, read and interpret all the relevant research themselves. Thus, if we acknowledge the unique importance of qualitative research, we need also to recognise that methods are required to bring its findings together for a wide audience – at the same time as preserving and respecting its essential context and complexity.

The earliest published work that we know of that deals with methods for synthesising qualitative research was written in 1988 by Noblit and Hare [ 31 ]. This book describes the way that ethnographic research might be synthesised, but the method has been shown to be applicable to qualitative research beyond ethnography [ 32 , 11 ]. As well as meta-ethnography, other methods have been developed more recently, including 'meta-study' [ 33 ], 'critical interpretive synthesis' [ 34 ] and 'metasynthesis' [ 13 ].

Many of the newer methods being developed have much in common with meta-ethnography, as originally described by Noblit and Hare, and often state explicitly that they are drawing on this work. In essence, this method involves identifying key concepts from studies and translating them into one another. The term 'translating' in this context refers to the process of taking concepts from one study and recognising the same concepts in another study, though they may not be expressed using identical words. Explanations or theories associated with these concepts are also extracted and a 'line of argument' may be developed, pulling corroborating concepts together and, crucially, going beyond the content of the original studies (though 'refutational' concepts might not be amenable to this process). Some have claimed that this notion of 'going beyond' the primary studies is a critical component of synthesis, and is what distinguishes it from the types of summaries of findings that typify traditional literature reviews [e.g. [ 32 ], p209]. In the words of Margarete Sandelowski, "metasyntheses are integrations that are more than the sum of parts, in that they offer novel interpretations of findings. These interpretations will not be found in any one research report but, rather, are inferences derived from taking all of the reports in a sample as a whole" [[ 14 ], p1358].

Thematic analysis has been identified as one of a range of potential methods for research synthesis alongside meta-ethnography and 'metasynthesis', though precisely what the method involves is unclear, and there are few examples of it being used for synthesising research [ 35 ]. We have adopted the term 'thematic synthesis', as we translated methods for the analysis of primary research – often termed 'thematic' – for use in systematic reviews [ 36 – 38 ]. As Boyatzis [[ 36 ], p4] has observed, thematic analysis is "not another qualitative method but a process that can be used with most, if not all, qualitative methods..." . Our approach concurs with this conceptualisation of thematic analysis, since the method we employed draws on other established methods but uses techniques commonly described as 'thematic analysis' in order to formalise the identification and development of themes.

We now move to a description of the methods we used in our example systematic review. While this paper has the traditional structure for reporting the results of a research project, the detailed methods (e.g. precise terms we used for searching) and results are available online. This paper identifies the particular issues that relate especially to reviewing qualitative research systematically and then to describing the activity of thematic synthesis in detail.

When searching for studies for inclusion in a 'traditional' statistical meta-analysis, the aim of searching is to locate all relevant studies. Failing to do this can undermine the statistical models that underpin the analysis and bias the results. However, Doyle [[ 39 ], p326] states that, "like meta-analysis, meta-ethnography utilizes multiple empirical studies but, unlike meta-analysis, the sample is purposive rather than exhaustive because the purpose is interpretive explanation and not prediction" . This suggests that it may not be necessary to locate every available study because, for example, the results of a conceptual synthesis will not change if ten rather than five studies contain the same concept, but will depend on the range of concepts found in the studies, their context, and whether they are in agreement or not. Thus, principles such as aiming for 'conceptual saturation' might be more appropriate when planning a search strategy for qualitative research, although it is not yet clear how these principles can be applied in practice. Similarly, other principles from primary qualitative research methods may also be 'borrowed' such as deliberately seeking studies which might act as negative cases, aiming for maximum variability and, in essence, designing the resulting set of studies to be heterogeneous, in some ways, instead of achieving the homogeneity that is often the aim in statistical meta-analyses.

However you look, qualitative research is difficult to find [ 40 – 42 ]. In our review, it was not possible to rely on simple electronic searches of databases. We needed to search extensively in 'grey' literature, ask authors of relevant papers if they knew of more studies, and look especially for book chapters, and we spent a lot of effort screening titles and abstracts by hand and looking through journals manually. In this sense, while we were not driven by the statistical imperative of locating every relevant study, when it actually came down to searching, we found that there was very little difference in the methods we had to use to find qualitative studies compared to the methods we use when searching for studies for inclusion in a meta-analysis.

Quality assessment

Assessing the quality of qualitative research has attracted much debate and there is little consensus regarding how quality should be assessed, who should assess quality, and, indeed, whether quality can or should be assessed in relation to 'qualitative' research at all [ 43 , 22 , 44 , 45 ]. We take the view that the quality of qualitative research should be assessed to avoid drawing unreliable conclusions. However, since there is little empirical evidence on which to base decisions for excluding studies based on quality assessment, we took the approach in this review to use 'sensitivity analyses' (described below) to assess the possible impact of study quality on the review's findings.

In our example review we assessed our studies according to 12 criteria, which were derived from existing sets of criteria proposed for assessing the quality of qualitative research [ 46 – 49 ], principles of good practice for conducting social research with children [ 50 ], and whether studies employed appropriate methods for addressing our review questions. The 12 criteria covered three main quality issues. Five related to the quality of the reporting of a study's aims, context, rationale, methods and findings (e.g. was there an adequate description of the sample used and the methods for how the sample was selected and recruited?). A further four criteria related to the sufficiency of the strategies employed to establish the reliability and validity of data collection tools and methods of analysis, and hence the validity of the findings. The final three criteria related to the assessment of the appropriateness of the study methods for ensuring that findings about the barriers to, and facilitators of, healthy eating were rooted in children's own perspectives (e.g. were data collection methods appropriate for helping children to express their views?).

Extracting data from studies

One issue which is difficult to deal with when synthesising 'qualitative' studies is 'what counts as data' or 'findings'? This problem is easily addressed when a statistical meta-analysis is being conducted: the numeric results of RCTs – for example, the mean difference in outcome between the intervention and control – are taken from published reports and are entered into the software package being used to calculate the pooled effect size [ 3 , 51 ].

Deciding what to abstract from the published report of a 'qualitative' study is much more difficult. Campbell et al . [ 11 ] extracted what they called the 'key concepts' from the qualitative studies they found about patients' experiences of diabetes and diabetes care. However, finding the key concepts in 'qualitative' research is not always straightforward either. As Sandelowski and Barroso [ 52 ] discovered, identifying the findings in qualitative research can be complicated by varied reporting styles or the misrepresentation of data as findings (as for example when data are used to 'let participants speak for themselves'). Sandelowski and Barroso [ 53 ] have argued that the findings of qualitative (and, indeed, all empirical) research are distinct from the data upon which they are based, the methods used to derive them, externally sourced data, and researchers' conclusions and implications.

In our example review, while it was relatively easy to identify 'data' in the studies – usually in the form of quotations from the children themselves – it was often difficult to identify key concepts or succinct summaries of findings, especially for studies that had undertaken relatively simple analyses and had not gone much further than describing and summarising what the children had said. To resolve this problem we took study findings to be all of the text labelled as 'results' or 'findings' in study reports – though we also found 'findings' in the abstracts which were not always reported in the same way in the text. Study reports ranged in size from a few pages to full final project reports. We entered all the results of the studies verbatim into QSR's NVivo software for qualitative data analysis. Where we had the documents in electronic form this process was straightforward even for large amounts of text. When electronic versions were not available, the results sections were either re-typed or scanned in using a flat-bed or pen scanner. (We have since adapted our own reviewing system, 'EPPI-Reviewer' [ 54 ], to handle this type of synthesis and the screenshots below show this software.)

Detailed methods for thematic synthesis

The synthesis took the form of three stages which overlapped to some degree: the free line-by-line coding of the findings of primary studies; the organisation of these 'free codes' into related areas to construct 'descriptive' themes; and the development of 'analytical' themes.

Stages one and two: coding text and developing descriptive themes

In our children and healthy eating review, we originally planned to extract and synthesise study findings according to our review questions regarding the barriers to, and facilitators of, healthy eating amongst children. It soon became apparent, however, that few study findings addressed these questions directly and it appeared that we were in danger of ending up with an empty synthesis. We were also concerned about imposing the a priori framework implied by our review questions onto study findings without allowing for the possibility that a different or modified framework may be a better fit. We therefore temporarily put our review questions to one side and started from the study findings themselves to conduct an thematic analysis.

There were eight relevant qualitative studies examining children's views of healthy eating. We entered the verbatim findings of these studies into our database. Three reviewers then independently coded each line of text according to its meaning and content. Figure 1 illustrates this line-by-line coding using our specialist reviewing software, EPPI-Reviewer, which includes a component designed to support thematic synthesis. The text which was taken from the report of the primary study is on the left and codes were created inductively to capture the meaning and content of each sentence. Codes could be structured, either in a tree form (as shown in the figure) or as 'free' codes – without a hierarchical structure.

line-by-line coding in EPPI-Reviewer.

The use of line-by-line coding enabled us to undertake what has been described as one of the key tasks in the synthesis of qualitative research: the translation of concepts from one study to another [ 32 , 55 ]. However, this process may not be regarded as a simple one of translation. As we coded each new study we added to our 'bank' of codes and developed new ones when necessary. As well as translating concepts between studies, we had already begun the process of synthesis (For another account of this process, see Doyle [[ 39 ], p331]). Every sentence had at least one code applied, and most were categorised using several codes (e.g. 'children prefer fruit to vegetables' or 'why eat healthily?'). Before completing this stage of the synthesis, we also examined all the text which had a given code applied to check consistency of interpretation and to see whether additional levels of coding were needed. (In grounded theory this is termed 'axial' coding; see Fisher [ 55 ] for further discussion of the application of axial coding in research synthesis.) This process created a total of 36 initial codes. For example, some of the text we coded as "bad food = nice, good food = awful" from one study [ 56 ] were:

'All the things that are bad for you are nice and all the things that are good for you are awful.' (Boys, year 6) [[ 56 ], p74]

'All adverts for healthy stuff go on about healthy things. The adverts for unhealthy things tell you how nice they taste.' [[ 56 ], p75]

Some children reported throwing away foods they knew had been put in because they were 'good for you' and only ate the crisps and chocolate . [[ 56 ], p75]

Reviewers looked for similarities and differences between the codes in order to start grouping them into a hierarchical tree structure. New codes were created to capture the meaning of groups of initial codes. This process resulted in a tree structure with several layers to organize a total of 12 descriptive themes (Figure 2 ). For example, the first layer divided the 12 themes into whether they were concerned with children's understandings of healthy eating or influences on children's food choice. The above example, about children's preferences for food, was placed in both areas, since the findings related both to children's reactions to the foods they were given, and to how they behaved when given the choice over what foods they might eat. A draft summary of the findings across the studies organized by the 12 descriptive themes was then written by one of the review authors. Two other review authors commented on this draft and a final version was agreed.

relationships between descriptive themes.

Stage three: generating analytical themes

Up until this point, we had produced a synthesis which kept very close to the original findings of the included studies. The findings of each study had been combined into a whole via a listing of themes which described children's perspectives on healthy eating. However, we did not yet have a synthesis product that addressed directly the concerns of our review – regarding how to promote healthy eating, in particular fruit and vegetable intake, amongst children. Neither had we 'gone beyond' the findings of the primary studies and generated additional concepts, understandings or hypotheses. As noted earlier, the idea or step of 'going beyond' the content of the original studies has been identified by some as the defining characteristic of synthesis [ 32 , 14 ].

This stage of a qualitative synthesis is the most difficult to describe and is, potentially, the most controversial, since it is dependent on the judgement and insights of the reviewers. The equivalent stage in meta-ethnography is the development of 'third order interpretations' which go beyond the content of original studies [ 32 , 11 ]. In our example, the step of 'going beyond' the content of the original studies was achieved by using the descriptive themes that emerged from our inductive analysis of study findings to answer the review questions we had temporarily put to one side. Reviewers inferred barriers and facilitators from the views children were expressing about healthy eating or food in general, captured by the descriptive themes, and then considered the implications of children's views for intervention development. Each reviewer first did this independently and then as a group. Through this discussion more abstract or analytical themes began to emerge. The barriers and facilitators and implications for intervention development were examined again in light of these themes and changes made as necessary. This cyclical process was repeated until the new themes were sufficiently abstract to describe and/or explain all of our initial descriptive themes, our inferred barriers and facilitators and implications for intervention development.

For example, five of the 12 descriptive themes concerned the influences on children's choice of foods (food preferences, perceptions of health benefits, knowledge behaviour gap, roles and responsibilities, non-influencing factors). From these, reviewers inferred several barriers and implications for intervention development. Children identified readily that taste was the major concern for them when selecting food and that health was either a secondary factor or, in some cases, a reason for rejecting food. Children also felt that buying healthy food was not a legitimate use of their pocket money, which they would use to buy sweets that could be enjoyed with friends. These perspectives indicated to us that branding fruit and vegetables as a 'tasty' rather than 'healthy' might be more effective in increasing consumption. As one child noted astutely, 'All adverts for healthy stuff go on about healthy things. The adverts for unhealthy things tell you how nice they taste.' [[ 56 ], p75]. We captured this line of argument in the analytical theme entitled 'Children do not see it as their role to be interested in health'. Altogether, this process resulted in the generation of six analytical themes which were associated with ten recommendations for interventions.

Six main issues emerged from the studies of children's views: (1) children do not see it as their role to be interested in health; (2) children do not see messages about future health as personally relevant or credible; (3) fruit, vegetables and confectionery have very different meanings for children; (4) children actively seek ways to exercise their own choices with regard to food; (5) children value eating as a social occasion; and (6) children see the contradiction between what is promoted in theory and what adults provide in practice. The review found that most interventions were based in school (though frequently with parental involvement) and often combined learning about the health benefits of fruit and vegetables with 'hands-on' experience in the form of food preparation and taste-testing. Interventions targeted at people with particular risk factors worked better than others, and multi-component interventions that combined the promotion of physical activity with healthy eating did not work as well as those that only concentrated on healthy eating. The studies of children's views suggested that fruit and vegetables should be treated in different ways in interventions, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective than those which were not.

Context and rigour in thematic synthesis

The process of translation, through the development of descriptive and analytical themes, can be carried out in a rigorous way that facilitates transparency of reporting. Since we aim to produce a synthesis that both generates 'abstract and formal theories' that are nevertheless 'empirically faithful to the cases from which they were developed' [[ 53 ], p1371], we see the explicit recording of the development of themes as being central to the method. The use of software as described can facilitate this by allowing reviewers to examine the contribution made to their findings by individual studies, groups of studies, or sub-populations within studies.

Some may argue against the synthesis of qualitative research on the grounds that the findings of individual studies are de-contextualised and that concepts identified in one setting are not applicable to others [ 32 ]. However, the act of synthesis could be viewed as similar to the role of a research user when reading a piece of qualitative research and deciding how useful it is to their own situation. In the case of synthesis, reviewers translate themes and concepts from one situation to another and can always be checking that each transfer is valid and whether there are any reasons that understandings gained in one context might not be transferred to another. We attempted to preserve context by providing structured summaries of each study detailing aims, methods and methodological quality, and setting and sample. This meant that readers of our review were able to judge for themselves whether or not the contexts of the studies the review contained were similar to their own. In the synthesis we also checked whether the emerging findings really were transferable across different study contexts. For example, we tried throughout the synthesis to distinguish between participants (e.g. boys and girls) where the primary research had made an appropriate distinction. We then looked to see whether some of our synthesis findings could be attributed to a particular group of children or setting. In the event, we did not find any themes that belonged to a specific group, but another outcome of this process was a realisation that the contextual information given in the reports of studies was very restricted indeed. It was therefore difficult to make the best use of context in our synthesis.

In checking that we were not translating concepts into situations where they did not belong, we were following a principle that others have followed when using synthesis methods to build grounded formal theory: that of grounding a text in the context in which it was constructed. As Margaret Kearney has noted "the conditions under which data were collected, analysis was done, findings were found, and products were written for each contributing report should be taken into consideration in developing a more generalized and abstract model" [[ 14 ], p1353]. Britten et al . [ 32 ] suggest that it may be important to make a deliberate attempt to include studies conducted across diverse settings to achieve the higher level of abstraction that is aimed for in a meta-ethnography.

Study quality and sensitivity analyses

We assessed the 'quality' of our studies with regard to the degree to which they represented the views of their participants. In doing this, we were locating the concept of 'quality' within the context of the purpose of our review – children's views – and not necessarily the context of the primary studies themselves. Our 'hierarchy of evidence', therefore, did not prioritise the research design of studies but emphasised the ability of the studies to answer our review question. A traditional systematic review of controlled trials would contain a quality assessment stage, the purpose of which is to exclude studies that do not provide a reliable answer to the review question. However, given that there were no accepted – or empirically tested – methods for excluding qualitative studies from syntheses on the basis of their quality [ 57 , 12 , 58 ], we included all studies regardless of their quality.

Nevertheless, our studies did differ according to the quality criteria they were assessed against and it was important that we considered this in some way. In systematic reviews of trials, 'sensitivity analyses' – analyses which test the effect on the synthesis of including and excluding findings from studies of differing quality – are often carried out. Dixon-Woods et al . [ 12 ] suggest that assessing the feasibility and worth of conducting sensitivity analyses within syntheses of qualitative research should be an important focus of synthesis methods work. After our thematic synthesis was complete, we examined the relative contributions of studies to our final analytic themes and recommendations for interventions. We found that the poorer quality studies contributed comparatively little to the synthesis and did not contain many unique themes; the better studies, on the other hand, appeared to have more developed analyses and contributed most to the synthesis.

This paper has discussed the rationale for reviewing and synthesising qualitative research in a systematic way and has outlined one specific approach for doing this: thematic synthesis. While it is not the only method which might be used – and we have discussed some of the other options available – we present it here as a tested technique that has worked in the systematic reviews in which it has been employed.

We have observed that one of the key tasks in the synthesis of qualitative research is the translation of concepts between studies. While the activity of translating concepts is usually undertaken in the few syntheses of qualitative research that exist, there are few examples that specify the detail of how this translation is actually carried out. The example above shows how we achieved the translation of concepts across studies through the use of line-by-line coding, the organisation of these codes into descriptive themes, and the generation of analytical themes through the application of a higher level theoretical framework. This paper therefore also demonstrates how the methods and process of a thematic synthesis can be written up in a transparent way.

This paper goes some way to addressing concerns regarding the use of thematic analysis in research synthesis raised by Dixon-Woods and colleagues who argue that the approach can lack transparency due to a failure to distinguish between 'data-driven' or 'theory-driven' approaches. Moreover they suggest that, "if thematic analysis is limited to summarising themes reported in primary studies, it offers little by way of theoretical structure within which to develop higher order thematic categories..." [[ 35 ], p47]. Part of the problem, they observe, is that the precise methods of thematic synthesis are unclear. Our approach contains a clear separation between the 'data-driven' descriptive themes and the 'theory-driven' analytical themes and demonstrates how the review questions provided a theoretical structure within which it became possible to develop higher order thematic categories.

The theme of 'going beyond' the content of the primary studies was discussed earlier. Citing Strike and Posner [ 59 ], Campbell et al . [[ 11 ], p672] also suggest that synthesis "involves some degree of conceptual innovation, or employment of concepts not found in the characterisation of the parts and a means of creating the whole" . This was certainly true of the example given in this paper. We used a series of questions, derived from the main topic of our review, to focus an examination of our descriptive themes and we do not find our recommendations for interventions contained in the findings of the primary studies: these were new propositions generated by the reviewers in the light of the synthesis. The method also demonstrates that it is possible to synthesise without conceptual innovation. The initial synthesis, involving the translation of concepts between studies, was necessary in order for conceptual innovation to begin. One could argue that the conceptual innovation, in this case, was only necessary because the primary studies did not address our review question directly. In situations in which the primary studies are concerned directly with the review question, it may not be necessary to go beyond the contents of the original studies in order to produce a satisfactory synthesis (see, for example, Marston and King, [ 60 ]). Conceptually, our analytical themes are similar to the ultimate product of meta-ethnographies: third order interpretations [ 11 ], since both are explicit mechanisms for going beyond the content of the primary studies and presenting this in a transparent way. The main difference between them lies in their purposes. Third order interpretations bring together the implications of translating studies into one another in their own terms, whereas analytical themes are the result of interrogating a descriptive synthesis by placing it within an external theoretical framework (our review question and sub-questions). It may be, therefore, that analytical themes are more appropriate when a specific review question is being addressed (as often occurs when informing policy and practice), and third order interpretations should be used when a body of literature is being explored in and of itself, with broader, or emergent, review questions.

This paper is a contribution to the current developmental work taking place in understanding how best to bring together the findings of qualitative research to inform policy and practice. It is by no means the only method on offer but, by drawing on methods and principles from qualitative primary research, it benefits from the years of methodological development that underpins the research it seeks to synthesise.

Chalmers I: Trying to do more good than harm in policy and practice: the role of rigorous, transparent and up-to-date evaluations. Ann Am Acad Pol Soc Sci. 2003, 589: 22-40. 10.1177/0002716203254762.

Article   Google Scholar  

Oakley A: Social science and evidence-based everything: the case of education. Educ Rev. 2002, 54: 277-286. 10.1080/0013191022000016329.

Cooper H, Hedges L: The Handbook of Research Synthesis. 1994, New York: Russell Sage Foundation

Google Scholar  

EPPI-Centre: EPPI-Centre Methods for Conducting Systematic Reviews. 2006, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=89 ]

Higgins J, Green S, (Eds): Cochrane Handbook for Systematic Reviews of Interventions 4.2.6. 2006, Updated September 2006. Accessed 24th January 2007, [ http://www.cochrane.org/resources/handbook/ ]

Petticrew M, Roberts H: Systematic Reviews in the Social Sciences: A practical guide. 2006, Oxford: Blackwell Publishing

Book   Google Scholar  

Chalmers I, Hedges L, Cooper H: A brief history of research synthesis. Eval Health Prof. 2002, 25: 12-37. 10.1177/0163278702025001003.

Article   PubMed   Google Scholar  

Juni P, Altman D, Egger M: Assessing the quality of controlled clinical trials. BMJ. 2001, 323: 42-46. 10.1136/bmj.323.7303.42.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mulrow C: Systematic reviews: rationale for systematic reviews. BMJ. 1994, 309: 597-599.

White H: Scientific communication and literature retrieval. The Handbook of Research Synthesis. Edited by: Cooper H, Hedges L. 1994, New York: Russell Sage Foundation

Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003, 56: 671-684. 10.1016/S0277-9536(02)00064-3.

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006, 6: 27-44. 10.1177/1468794106058867.

Sandelowski M, Barroso J: Handbook for Synthesising Qualitative Research. 2007, New York: Springer

Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M: Qualitative meta-synthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004, 14: 1342-1365. 10.1177/1049732304269888.

Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people's views: an example from public health. J Epidemiol Community Health. 2004, 58: 794-800. 10.1136/jech.2003.014829.

Article   PubMed   PubMed Central   Google Scholar  

Harden A, Brunton G, Fletcher A, Oakley A: Young People, Pregnancy and Social Exclusion: A systematic synthesis of research evidence to identify effective, appropriate and promising approaches for prevention and support. 2006, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=674 ]

Thomas J, Sutcliffe K, Harden A, Oakley A, Oliver S, Rees R, Brunton G, Kavanagh J: Children and Healthy Eating: A systematic review of barriers and facilitators. 2003, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, accessed 4 th July 2008, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=246 ]

Thomas J, Kavanagh J, Tucker H, Burchett H, Tripney J, Oakley A: Accidental Injury, Risk-Taking Behaviour and the Social Circumstances in which Young People Live: A systematic review. 2007, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=1910 ]

Bryman A: Quantity and Quality in Social Research. 1998, London: Unwin

Hammersley M: What's Wrong with Ethnography?. 1992, London: Routledge

Harden A, Thomas J: Methodological issues in combining diverse study types in systematic reviews. Int J Soc Res Meth. 2005, 8: 257-271. 10.1080/13645570500155078.

Oakley A: Experiments in Knowing: Gender and methods in the social sciences. 2000, Cambridge: Polity Press

Harden A, Oakley A, Oliver S: Peer-delivered health promotion for young people: a systematic review of different study designs. Health Educ J. 2001, 60: 339-353. 10.1177/001789690106000406.

Harden A, Rees R, Shepherd J, Brunton G, Oliver S, Oakley A: Young People and Mental Health: A systematic review of barriers and facilitators. 2001, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=256 ]

Rees R, Harden A, Shepherd J, Brunton G, Oliver S, Oakley A: Young People and Physical Activity: A systematic review of barriers and facilitators. 2001, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=260 ]

Shepherd J, Harden A, Rees R, Brunton G, Oliver S, Oakley A: Young People and Healthy Eating: A systematic review of barriers and facilitators. 2001, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, [ http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=258 ]

Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J: Integrating qualitative research with trials in systematic reviews: an example from public health. BMJ. 2004, 328: 1010-1012. 10.1136/bmj.328.7446.1010.

Davies P: What is evidence-based education?. Br J Educ Stud. 1999, 47: 108-121. 10.1111/1467-8527.00106.

Newman M, Thompson C, Roberts AP: Helping practitioners understand the contribution of qualitative research to evidence-based practice. Evid Based Nurs. 2006, 9: 4-7. 10.1136/ebn.9.1.4.

Popay J: Moving Beyond Effectiveness in Evidence Synthesis. 2006, London: National Institute for Health and Clinical Excellence

Noblit GW, Hare RD: Meta-Ethnography: Synthesizing qualitative studies. 1988, Newbury Park: Sage

Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesise qualitative research: a worked example. J Health Serv Res Policy. 2002, 7: 209-215. 10.1258/135581902320432732.

Paterson B, Thorne S, Canam C, Jillings C: Meta-Study of Qualitative Health Research. 2001, Thousand Oaks, California: Sage

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretative synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006, 6: 35-10.1186/1471-2288-6-35.

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.

Boyatzis RE: Transforming Qualitative Information. 1998, Sage: Cleveland

Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol. 2006, 3: 77-101. 10.1191/1478088706qp063oa. [ http://science.uwe.ac.uk/psychology/drvictoriaclarke_files/thematicanalysis%20.pdf ]

Silverman D, Ed: Qualitative Research: Theory, method and practice. 1997, London: Sage

Doyle LH: Synthesis through meta-ethnography: paradoxes, enhancements, and possibilities. Qual Res. 2003, 3: 321-344. 10.1177/1468794103033003.

Barroso J, Gollop C, Sandelowski M, Meynell J, Pearce PF, Collins LJ: The challenges of searching for and retrieving qualitative studies. Western J Nurs Res. 2003, 25: 153-178. 10.1177/0193945902250034.

Walters LA, Wilczynski NL, Haynes RB, Hedges Team: Developing optimal search strategies for retrieving clinically relevant qualitative studies in EMBASE. Qual Health Res. 2006, 16: 162-8. 10.1177/1049732305284027.

Wong SSL, Wilczynski NL, Haynes RB: Developing optimal search strategies for detecting clinically relevant qualitative studies in Medline. Medinfo. 2004, 11: 311-314.

Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P: Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess. 1998, 2 (16):

Seale C: Quality in qualitative research. Qual Inq. 1999, 5: 465-478.

Spencer L, Ritchie J, Lewis J, Dillon L: Quality in Qualitative Evaluation: A framework for assessing research evidence. 2003, London: Cabinet Office

Boulton M, Fitzpatrick R, Swinburn C: Qualitative research in healthcare II: a structured review and evaluation of studies. J Eval Clin Pract. 1996, 2: 171-179. 10.1111/j.1365-2753.1996.tb00041.x.

Article   CAS   PubMed   Google Scholar  

Cobb A, Hagemaster J: Ten criteria for evaluating qualitative research proposals. J Nurs Educ. 1987, 26: 138-143.

CAS   PubMed   Google Scholar  

Mays N, Pope C: Rigour and qualitative research. BMJ. 1995, 311: 109-12.

Medical Sociology Group: Criteria for the evaluation of qualitative research papers. Med Sociol News. 1996, 22: 68-71.

Alderson P: Listening to Children. 1995, London: Barnardo's

Egger M, Davey-Smith G, Altman D: Systematic Reviews in Health Care: Meta-analysis in context. 2001, London: BMJ Publishing

Sandelowski M, Barroso J: Finding the findings in qualitative studies. J Nurs Scholarsh. 2002, 34: 213-219. 10.1111/j.1547-5069.2002.00213.x.

Sandelowski M: Using qualitative research. Qual Health Res. 2004, 14: 1366-1386. 10.1177/1049732304269672.

Thomas J, Brunton J: EPPI-Reviewer 3.0: Analysis and management of data for research synthesis. EPPI-Centre software. 2006, London: EPPI-Centre, Social Science Research Unit, Institute of Education

Fisher M, Qureshi H, Hardyman W, Homewood J: Using Qualitative Research in Systematic Reviews: Older people's views of hospital discharge. 2006, London: Social Care Institute for Excellence

Dixey R, Sahota P, Atwal S, Turner A: Children talking about healthy eating: data from focus groups with 300 9–11-year-olds. Nutr Bull. 2001, 26: 71-79. 10.1046/j.1467-3010.2001.00078.x.

Daly A, Willis K, Small R, Green J, Welch N, Kealy M, Hughes E: Hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol. 2007, 60: 43-49. 10.1016/j.jclinepi.2006.03.014.

Popay J: Moving beyond floccinaucinihilipilification: enhancing the utility of systematic reviews. J Clin Epidemiol. 2005, 58: 1079-80. 10.1016/j.jclinepi.2005.08.004.

Strike K, Posner G: Types of synthesis and their criteria. Knowledge Structure and Use: Implications for synthesis and interpretation. Edited by: Ward S, Reed L. 1983, Philadelphia: Temple University Press

Marston C, King E: Factors that shape young people's sexual behaviour: a systematic review. The Lancet. 2006, 368: 1581-86. 10.1016/S0140-6736(06)69662-1.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/8/45/prepub

Download references

Acknowledgements

The authors would like to thank Elaine Barnett-Page for her assistance in producing the draft paper, and David Gough, Ann Oakley and Sandy Oliver for their helpful comments. The review used an example in this paper was funded by the Department of Health (England). The methodological development was supported by Department of Health (England) and the ESRC through the Methods for Research Synthesis Node of the National Centre for Research Methods. In addition, Angela Harden held a senior research fellowship funded by the Department of Health (England) December 2003 – November 2007. The views expressed in this paper are those of the authors and are not necessarily those of the funding bodies.

Author information

Authors and affiliations.

EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, UK

James Thomas & Angela Harden

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to James Thomas .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

Both authors contributed equally to the paper and read and approved the final manuscript.

James Thomas and Angela Harden contributed equally to this work.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Thomas, J., Harden, A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 8 , 45 (2008). https://doi.org/10.1186/1471-2288-8-45

Download citation

Received : 17 April 2008

Accepted : 10 July 2008

Published : 10 July 2008

DOI : https://doi.org/10.1186/1471-2288-8-45

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative Research
• Primary Study
• Analytical Theme
• Healthy Eating
• Review Question

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Supplements
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 6, Issue 6
• Defining global health: findings from a systematic review and thematic analysis of the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-3263-9154 Melissa Salm 1 ,
• Mahima Ali 2 ,
• Mairead Minihane 2 ,
• Patricia Conrad 3
• 1 Anthropology , University of California Davis , Davis , California , USA
• 2 University of California Davis , Davis , California , USA
• 3 VM:PMI , University of California Davis , Davis , California , USA
• Correspondence to Melissa Salm; melsalm{at}gmail.com

Introduction Debate around a common definition of global health has seen extensive scholarly interest within the last two decades; however, consensus around a precise definition remains elusive. The objective of this study was to systematically review definitions of global health in the literature and offer grounded theoretical insights into what might be seen as relevant for establishing a common definition of global health.

Method A systematic review was conducted with qualitative synthesis of findings using peer-reviewed literature from key databases. Publications were identified by the keywords of ‘global health’ and ‘define’ or ‘definition’ or ‘defining’. Coding methods were used for qualitative analysis to identify recurring themes in definitions of global health published between 2009 and 2019.

Results The search resulted in 1363 publications, of which 78 were included. Qualitative analysis of the data generated four theoretical categories and associated subthemes delineating key aspects of global health. These included: (1) global health is a multiplex approach to worldwide health improvement taught and pursued at research institutions; (2) global health is an ethically oriented initiative that is guided by justice principles; (3) global health is a mode of governance that yields influence through problem identification, political decision-making, as well as the allocation and exchange of resources across borders and (4) global health is a vague yet versatile concept with multiple meanings, historical antecedents and an emergent future.

Conclusion Extant definitions of global health can be categorised thematically to designate areas of importance for stakeholders and to organise future debates on its definition. Future contributions to this debate may consider shifting from questioning the abstract ‘what’ of global health towards more pragmatic and reflexive questions about ‘who’ defines global health and towards what ends.

• health education and promotion
• health policy
• public health
• qualitative study
• systematic review

Data availability statement

No data are available. All data relevant to the study are included in the article or uploaded as supplementary information. n/a.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjgh-2021-005292

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key questions

What is already known.

Debate around a common definition of global health has seen extensive scholarly interest within the last two decades; despite the abundance of literature, ambiguity still persists around its precise definition.

No systematic reviews with thematic analysis have been conducted to explore extant definitions of global health nor to contribute to a comprehensive definition of global health.

What are the new findings?

We compile and thematically analyse extant definitions of global health and propose grounded theoretical insights into what might be seen as relevant for establishing a common definition of global health moving forward.

The need for a clear and concise definition of global health has the highest stakes in the domain of global health policy governance.

What do the new findings imply?

Stakeholders tend to define the ‘what’ of global health: its spaces, objects and practices. Our findings suggest that the debate around definition should shift to more pragmatic and reflexive questions regarding ‘who’ defines global health and towards what ends.

Introduction

Debate around a common definition of global health (GH) has seen extensive scholarly interest within the last two decades. In 2009, a widely circulated paper by Koplan and colleagues aimed to establish ‘a common definition of global health’ as distinct from its derivations in public health (PH) and international health (IH). 1 They rooted the definition of PH in the mid-19th century social reform movements of Europe and the USA, the growth of biological and medical knowledge, and the discipline’s emphasis on population-level health management. Similarly, they traced the evolution of IH back to its colonial roots in hygiene and tropical medicine (TM) through to the mid-20th century with its geographic focus on developing countries. GH, they argued, would require a distinctive definition of its own to be ‘more than a rephrasing of a common definition of PH or a politically correct updating of international health’. Their intervention built on prior research noting confusion and overlap among the three terms and thus a need to carefully articulate the important differences between them. 2–5 Additional stakeholders have since elaborated varied definitions of GH, yet consensus around its precise definition remains elusive.

To determine how GH is presently defined and to identify whether a common conceptualisation has been established, we conducted a qualitative systematic literature review (SLR) of the GH literature between 2009 and 2019. SLRs are a methodology used ‘to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question’. 6 Unlike unsystematic narrative reviews, SLRs use formal, repeatable and transparent, procedures for identifying, evaluating and interpreting available research, thus ensuring robust coverage of the current literature while reducing the biased presentation of available evidence. 7–9 Medical researchers and policy-makers have long relied on SLRs because they integrate and critically evaluate current knowledge to support decisions about important issues. 10 However, very few SLRs exploring aspects of GH have yet been published, 11–13 and no SLRs focusing on extant definitions of GH have been conducted. This paper fills that gap by exploring the thematic components of extant definitions and thereby contributes towards a comprehensive definition of GH.

Aims and objectives

The aim of this review is: (a) to examine how GH has been defined in the literature between 2009 and 2019, (b) to systematically analyse the core thematic categories undergirding extant definitions of GH and (c) to offer grounded theoretical insights into what might be seen as relevant for establishing a common definition of GH.

Aiming to capture definitions of GH in literature between 2009 and 2019, our team conducted a systematic review of the peer-reviewed literature following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines ( figure 1 ). 14 The sequential steps of our review process included the following.

• Download figure
• Open in new tab
• Download powerpoint

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of citation analysis and systematic literature review. 14

Search strategy: identify papers and relevant databases

Search technique.

The terms ‘global health’ AND ‘define’ OR ‘definition’ OR ‘defining’ were queried when they appeared in the title, abstract or keyword of studies. Published studies were identified through comprehensive searches of electronic databases accessible through the authors’ university library system (Web of Science, Scopus, Embase, PubMed, EBSCO). Citation tracking through Google Scholar was also completed.

Study selection criteria

Articles published in international peer-reviewed journals, including conference papers, book chapters and editorial material, were reviewed. The studies included were written in English and published between 2009 and 2019. The year 2009 was chosen as a starting point because this is the year in which Koplan et al published ‘Towards a Common Definition of Global Health’. For this review, the team excluded news articles, theses, book reviews and published papers that were not written in English.

Assessment strategy: appraise which papers to include in review

The protocol-driven search strategy required that articles included in the review must: (a) contain the keywords ‘global health’ and ‘definition’ and/or ‘define’; (b) be in the English language and (c) be published between 2009 and 2019. The number of articles containing these keywords was recorded, and all the titles uncovered in the search were imported into Mendeley, a software for managing citations. Duplicates were identified and removed, after which abstracts were screened to assess eligibility against the inclusion criteria. Full-text articles were retrieved for those that met the inclusion criteria and three team members read a designated number of the articles selected for full review. To be included in the data extraction sheet, each article needed to: (a) focus on and explicitly name GH, (b) offer an original definition or description of GH and/or (c) cite an already-existing definition of GH. Articles that mentioned the query terms without any relation to these requirements (eg, did not provide a definition of GH or descriptive data to support interpretations of a GH definition) were excluded. Assessment for relevance and content was conducted by two investigators who reviewed all identified articles independently. Disagreements were resolved by consensus with a third investigator.

Synthesis strategy: extract the data

Based on the research goals, the team designed an initial coding template in Google Sheets as a method of documentation, with the following coding variables: author, title, typology, definition(s), conclusions and conceptual dimensions. To achieve a high level of reliability, the review team open-coded the same five articles, compared their coding experiences, and reconciled differences before adopting a final coding template and evenly dividing the remaining articles to be analysed. Extracted data included the type of study or research paradigm of each publication, the location and disciplinary affiliation of each study based on the contact information of the corresponding author, definitions and descriptions of GH and specialised dimensions of GH. Whenever articles contained more than one definition or description of GH, those items were organised line-by-line under the author on the data extraction sheet.

Analysis strategy: analyse the data

The team conducted thematic analysis of the data to understand how GH has been defined since 2009. Our approach to thematic analysis was based on the guidelines described by Thomas and Harden 15 and further informed by principles in grounded theory. 16 Our strategy consisted of three main stages: Initial Coding—remaining open to all possible emergent themes indicated by readings of the data; 16 17 Focused Coding—categorising the data inductively based on thematic similarity at the level of description 17 and finally, Theoretical Coding—integrating thematic categories into core theoretical constructs at a higher level of analysis. 18

In the first cycle, open descriptive codes were generated (eg, differences between PH and IH, GH education requirements, social justice values) directly from the definitions and descriptions of GH found in the articles. Individual sentences defining or describing GH were treated as unique line items on the data extraction sheet and coded accordingly in order to generate a range of ideas and information on which to build.

In the second cycle, a focused thematic analysis was carried out to identify general relationships and patterns among definitions in the literature and to confirm significant links between the openly coded data. Thematic phrases (eg, GH is multidisciplinary, GH promotes equity) were developed and reapplied to coded definitions on the data extraction sheet. Team members wrote and attached analytic memos to each coded datum—reflecting on emergent patterns and further ‘codeweaving’, 18 which is a term for charting possible relationships among the coded data. At this stage, additional coding techniques were utilised. Attribute coding was applied as a management technique for logging information about the characteristics of each publication. 19 Data segments coded in this manner were extracted from the main data extraction form and reassembled together in a separate Google Sheet for further analysis. The team also coded extracted definitions of GH by type: (a) original definition, (b) cited definition, (c) original description to track possible relationships between citational practices and developments in the conceptualisation and definition of GH.

In the third cycle, thematic phrases were ordered according to frequency then commonality and abstracted for overriding significance into theoretical categories. At this stage, the conceptual level of analysis was raised from description to a more abstract, theoretical level leading to a grounded theory. This resulted in the construction of four thematic categories, which are presented below with their supporting subthemes.

Patient and public involvement

Patients and public were not directly involved in this review; we used publicly available data for the analysis.

The search strategy retrieved bibliographic records for 1363 papers. The assessment strategy resulted in the elimination of 1237 papers after the removal of duplicates. Consequently, 78 papers were subjected to our strategies of synthesis (data extraction) and analysis.

Characteristics of study

A variety of studies were included in this review. The majority (27) were commentaries, viewpoints or debates. 1 20–48 Twenty-four were grouped as review/overview articles. 45–68 There were 25 original research articles, of which 13 used qualitative methods, 69–81 11 used mixed-methods 82–92 and one 93 used quantitative data from a survey to proffer definitions of GH. Two studies included in the review were book chapters. 94 95

The typologic, geographic and disciplinary distribution of the studies in this review are shown in table 1 . Most studies were authored in North America (40), 1 20–31 39–41 43 46 47 50 54–58 61 63 66 68 70 73 74 76–80 83 84 86 87 89–91 94 followed by European countries (29), 22 26 28 32 34–38 42 44 45 48 51 52 59 62 64 65 67 71 75 82 85 88 92 93 95 96 countries in Asia (2), 33 72 Latin America and the Caribbean (2), 60 81 and New Zealand (1). 20 Disciplinary fields represented in our sample included health (56), 20 22–27 30–32 34–40 42 43 45–51 54–56 58–61 63–69 72 74 75 77–79 82–84 86 88–91 93 95 law, social and cultural professions (19), 1 20 28 29 33 41 44 52 53 57 62 70 71 73 76 80 81 87 92 94 and education (2). 20 31

• View inline

Summary of characteristics of retrieved publications

Attributes of definitions

All 78 studies under review defined, described and/or cited extant definitions of GH. The 34 papers shown in table 2 included descriptive definitions of GH that were formulated distinctly by its authors, that is, they were presented as original and without direct reference to other definitions.

How global health has been defined by academics since 2009

Several scholars engaged directly with the Koplan et al definition of GH 1 to stipulate definitions of their own. For example, some authors proposed amendments to Koplan et al that would place greater emphasis on inequity reduction and the need for collaboration, 20 particularly with institutional partners from developing countries. 73 Others were more critical of the broad yet weak conceptual idealism 86 of Koplan et al and recommended detaching normative objectives from its definition, 26 such as the value-laden concept of equity, which could compromise the definition’s technical neutrality by rendering it ideological. 91 Other authors sought to analytically clarify the meaning of ‘the global’ 26 in the definition provided by Koplan et al , distinguish it more clearly from IH 78 or dispute their distinction between GH and PH. 27 Indeed, the impact of the definition of GH proposed by Koplan et al has been substantial. It was variously adopted by the Consortium of Universities for Global Health, 47 the Canadian government, 23 Global Health for Family Medicine, 89 the German Academy of Sciences 75 and the Chinese Consortium of Universities for Global Health. 77

In general, GH was defined as a term, 37 51 95 and in particular, an umbrella term 49 75 or a concept; 69 and more broadly as a zone 76 or field 32 48 91 94 or area of research and practice, 1 56 as an achievable goal, 50 an approach, 48 82 a set of principles, 45 83 an organising framework for thinking and action 96 or a collection of problems. 35 94 GH was frequently contrasted to IH 32 35 68 69 94 95 and PH, 20 21 31 32 35 or else seen as indistinguishably from PH and IH. 27 Additionally, several papers explicitly specified and subsequently defined certain dimensions of GH, such as ‘global health governance’ (GHG), 32 33 35 38 42 51 52 58 69 80 81 87 ‘global health diplomacy’ (GHD), 24 28 95 ‘global health education’, 36 39 46–49 59 70 74 75 77 78 82 89 93 ‘global health security’, 26 41 76 88 92 97 98 ‘global health network’, 41 81 ‘global health actor’, 52 ‘global health ethics’, 69 ‘global health academics’ 64 67 and ‘global health social justice’ 61 (see table 3 ).

Frequently defined facets of ‘Global Health’ with exemplary definitions

Grounded theory approach based on thematic analysis

Definitions and descriptions of GH were aggregated into nine thematic codes reflecting the contents and scope of GH definitions, the functionality of those definitions and/or perceptions about defining GH. Codes were: (1) GH is a domain of research, healthcare and education, (2) GH is multifaceted (disciplinary, sectoral, cultural, national), (3) GH is rooted in a commitment to equity, (4) GH is a political field comprising power relations, (5) GH is problem-oriented, (6) GH transcends national borders, (7) GH is determined by globalisation and international interdependence, (8) conceptually, GH is either similar or dissimilar to PH, IH and TM and (9) GH is perceived as definitionally vague.

These codes were grouped selectively into higher analytical categories or theoretical statements as grounded in the literature: (1) GH is a multiplex approach to worldwide health improvement and form of expertise taught and researched through academic institutions, (2) GH is an ethos (ethical orientation and appeal) that is guided by justice principles, (3) GH is a mode of governance that yields degrees of national, international, transnational and supranational influence through political decision-making, problem identification, the allocation and exchange of resources across borders, (4) GH is a polysemous concept with many meanings and historical antecedents, and which has an emergent future ( table 4 ).

Defining global health with grounded theory analysis—table of themes, code categories and quotes from text

Theme: global health is a multiplex approach to worldwide health improvement taught and pursued through research institutions

Subtheme: gh is a domain of research, healthcare, education.

GH was repeatedly defined as an active field of knowledge production that is composed of the following key elements: research, education, training and practice related to health improvement. 1 20 21 23 32 33 35 38 40 44–49 52 55–58 61 63–69 72 74 75 77 78 80 82 90–92 94 Few authors defined GH as a new, independent discipline within the broader domain of medical knowledge, 17 33 38 46 63 74 80 82 90 and some outlined discipline-specific competencies that were considered integral to the definition of GH, at least in curriculum development; for example: clinical literacy, 80 medical humanities, 82 cross-cultural sensitivity, 33 38 46 59 63 80 90 experiential learning 47 and critical thinking skills. 72 82 Several authors defined GH as a diffuse arena of scholarship that spans an array of academic disciplines, including anthropology, engineering, law, agriculture and healthcare administration. 44 56 59 63–65 78 91 94 Others defined GH explicitly as a ‘transdiscipline’ that seeks to transcend the restricted gaze of any single discipline and consequently integrate knowledge from a variety of sources. 67 94 Several authors explicitly defined GH as a necessarily collaborative field. 1 20 22 24 36 43 45 47 57 61 63 68 77 78 80 91

Subtheme: GH is multifaceted (disciplinary, sectoral, cultural, national)

The prefix ‘multi-’ was consistently applied in definitions of GH to describe a perspective that focuses on the multitude of interrelated factors, dimensions, values and features that underpin health as well as efforts to improve and study it. There was broad agreement that multidisciplinarity is a defining characteristic of GH. 1 23 25 32–34 36 38 40 45–47 49 52 55–57 59 60 64–69 72 75 77 78 80 82 91 However, there was some debate whether multiple disciplines are always needed and beneficial—and therefore essential—to the definition of GH. 23 One author argued that the multidisciplinary nature of GH is precisely what differentiates it from PH and IH. 68 Although some claimed that GH, with its focus on social and economic determinants, is inherently ‘predisposed to include aspects of the liberal arts and social sciences’, 75 others critically observed that most GH educational opportunities still cater predominantly to medical students, 32 35 48 72 which suggests that greater efforts will be required to achieve multidisciplinarity in the field moving forward.

There was a correspondence between GH definitions citing multidisciplinarity and cultural competency. 32 33 38 48 49 56 78 82 90 Curiously, multisectorality was less frequently mentioned than multidisciplinarity in definitions of GH, though it was referenced in some papers. 20 22 43 52 66 83 86 95

Theme: global health is an ethical initiative that is guided by justice principles

Subtheme: gh is rooted in values of equity and social justice.

Equity and social justice were the two most commonly and explicitly referenced values undergirding GH definitions and goals. Equity was repeatedly framed as a ‘main objective’ 60 and core component of GH research and practice. 23 25 43 46 48 53 66 67 77 78 84 However, it remains unclear whether the authors in our sample share the same meaning of equity. Velji and Bryant defined equity broadly as ‘ensuring equal opportunities and resources to enable all people to achieve their fullest health potential’. 66 Meanwhile, others rooted their conceptualisation of equity more specifically in the principles of social justice 30 61 69 88 89 or the human rights concept of equality, 54 62 67 83 86 which asserts that ‘all people are equal in regard to dignity and rights, regardless of their origin and all biological, social or other specific differences’. 59 This postwar sensibility echoes the 1978 Alma Ata Declaration of ‘health for all’, 20 24 as well as a traditional humanitarian ideal, even if now associated with principles grounded in national and global security. 24 54 88

Occasionally, the terms ‘equity’ and ‘equality’ were used interchangeably, suggesting they possess a commonly shared valence and reciprocal relationship despite slight differences in signification. Whereas equity refers to the provision of resources and opportunity based on specific needs, equality connotes providing the same level of resources and opportunities for all. 86 Nevertheless, other scholars questioned whether equity or equality should be included in official definitions of GH, at all, 27 48 75 insofar as what counts as ‘equitable’ for one country may be different for another. 26 32 48

Theme: global health is a form of governance that yields national, international, transnational and supranational influence through political decision-making, problem identification, the allocation and exchange of resources across borders

Subtheme: gh is a political field comprising power relations at multiple scales.

Numerous papers defined GH as embedded within a political field comprising power relations at multiple scales. 20 22–24 26 28 29 31–33 35 41 42 45 48 51–54 56 58 60 63 66 70 72 76 79 87 95 ‘Political field’ refers here to a sphere of influence and jurisdiction wherein institutions determine governing modalities (eg, laws, policies, instruments) to assure a range of activities, such as determining priorities, coordinating stakeholders, regulating funding mechanisms, establishing accountability, allocating resources and providing access to health services for the general public. ‘Power relations’ refers to the capacity of institutions, individuals, instruments and ideas to affect the actions of others; and ‘at multiple scales’ refers to levels of analysis (ie, worldwide, regional, national, local, etc.).

Within the literature on GHG and GH security, authors argued the need for a universal definition of GH to shape policy frameworks that ensure compliance with IH law. 32 45 51 88 95 Here, it is important to note that the ability to shape GH policy is, itself, an exercise in power: some GH actors, defined as ‘individuals or organizations that operate transnationally with a primary intent to improve health’, 56 are more capacitated than others to impact the formulation of policies and amount of attention and resources that certain GH issues receive. 32 41 45 52 95 For example, several papers discussed how ‘GH actors’ like the World Bank and the WHO shaped discussions around the response to Ebola, leading to refined definitions of GHG 35 87 88 and GH security. 41 Similarly, definitions of GH in line with the 2015 United Nations Millennium Development Goals, were also commonly referenced, 25 35 45 51 reflecting the influence of certain GH actors on the conceptualisation of GH.

Subtheme: GH is determined by globalisation and international interdependence

Numerous authors linked interdependence and accelerating globalisation (the process of integrating governments and markets, and of connecting people worldwide) with the need for a cohesive definition of GH, particularly to address issues of governance. 24 32 35 45 68 88 GHG and GHD were outlined as two influential subdomains in which the interconnections between globalisation, foreign policy and international relations were viewed as indispensable to definitions of GH. Two articles quoted David P Fidler’s definition of GHG as ‘the use of formal and informal institutions, rules, and processes by states, intergovernmental organizations, and nonstate actors to deal with challenges to health that require cross-border collective action to address effectively’. 35 58 Elsewhere, GHD was described as ‘bringing together the disciplines of public health, international affairs, management, law and economics and focuses on negotiations that shape and manage the global policy environment for health’. 95

Subtheme: GH issues transcend national borders

Across several papers, we observed a common refrain that GH ‘crosses borders’ and ‘transcends national boundaries’. 1 20 23 42 45 52 60 67 68 74 Authors frequently described GH concerns as those exceeding the jurisdictional reaches of any individual nation-state alone. 34 42 45 51 52 54 77 95 One paper claimed that GH is ‘transnational by definition’, 74 and others characterised GH problems as those experienced transnationally. 20 32 48 50 68

Studies focusing on GH research and training frequently referenced specific diseases and health risks that ‘transcend national borders’ alongside parallel recommendations to include an international component in the development of GH curricula. 16 48 49 63 74 93 While crossing national borders to research and promote health for all is widely perceived as an historical condition for GH 24 that has led to GH’s emergence as an academic discipline, 63 several scholars argued that GH should also focus on domestic health disparities 1 27 38 46 and for local issues to be simultaneously understood as universal or worldwide 48 74 75 to the extent they may occur anywhere 22 and are almost always impacted by global phenomena. 56

Subtheme: GH is problem-oriented

Medical anthropologists, Arthur Kleinman and Paul Farmer, described GH as a collection of problems rather than a distinct discipline. 35 94 Several authors in our review delineated GH problems through identification of specific diseases, such as HIV/AIDS, malaria, TB, Zika and Ebola. 24 29 30 35 45 83 Lee and Brumme noted that it has become common for experts to define GH problems by identifying their objects, namely diseases, population groups and locations. 58 Indeed, some authors outlined GH problems as the set of challenges ‘among those most neglected in developing countries’, 86 among them: emerging infectious diseases and maternal and child health; 43 65 diabetes, cardiovascular disease and other noncommunicable diseases in ‘local’ communities 25 63 and even neurological disorders among refugees arriving in Europe. 93 How these types of object-based definitions of GH problems come to shape GH agendum is important to note.

Clark made a compelling argument against the definition of GH problems in terms of specific diseases, writing that such ‘medicalisation’ may ‘prove detrimental for how the world responds and resources actions designed to alleviate poor health and poverty, redress inequities, and save lives’. 72 Brada also argued against defining GH problems geographically and instead urged experts to consider how the processes by which GH and its quintessential spaces, namely ‘resource-limited’ and ‘resource-poor settings’, are actively constituted, reinforced and contested. 70 Several authors similarly suggested that focusing on the social, political, economic and cultural forces contributing to health inequity and diseases of poverty better captured the scope of GH problems than naming any particular set of diseases or places in the world. 33 43 56 58 69 72 73 86 92

Lack of consensus regarding what counts as a ‘true’ GH problem was linked to the lack of a clear and concise definition of GH. Indeed, several scholars argued that the current inability to define GH made it difficult for stakeholders to define precisely what the ‘problem’ is. 44 45 48 86 Furthermore, the diagnosis of GH problems determined what types of GH ‘solutions’ were proposed in response. For example, when GH problems were defined as universally shared and transnational, then cross-border solutions were developed; when GH issues were framed epidemiologically in terms of distributed risk, then actions targeting specific determinants and burdens were proposed. 1 20 23 67 68 92 When GH problems were framed as threats to inter/national security, strategies were formulated to protect borders, economies, health systems and to improve surveillance mechanisms. 41 45 54 76 80 88 When the problem of inequality drove definitions of GH, recommendations to alleviate poverty, food insecurity, poor sanitation, etc. were proposed. 32 53 60 72

Although Kuhlmann suggested that GH tends to over-prioritise problem-identification to the detriment of critical solution-oriented work, 31 our analysis suggests that the type, scope and quality of solutions proposed are contingent on the elaboration of problems. Similarly, Campbell wrote, ‘Unlike a science or an art, the field of global health is very much about providing solutions to current problems. As such, it would be short-sighted not to consider the causes of global health problems in order to better formulate the solutions. The causes ought to be included in a comprehensive and complete definition of the field’. 23

Theme: global health is a polysemous concept with historical antecedents and an emergent future

Subtheme: gh is conceptually dis/similar to ph, ih and tm.

GH was consistently traced back to and compared with PH, IH and TM. 1 20 27 32–34 43 57 69 71 75 84 86 88 Disagreement or confusion regarding the degrees of similarity and difference between these domains seemed to stem from a shared understanding that GH, in fact, evolved to a varying degree from each of these fields and does not, therefore, denote a clear-cut break with nor full-blown departure from any of them. 84 94

Several authors argued that the scope and scale of GH is distinct from PH. 1 20 32 69 71 Some argued that ‘public health is equated primarily with population-wide interventions; global health is concerned with all strategies for health improvement,’ including clinical care; 20 and that ‘public health acknowledges the state as a dominant actor, (while) global health recognizes the rise of other actors like international institutions’. 35 GH was also seen as placing a greater emphasis on multidisciplinarity and promoting a more expansive conceptualisation of ‘health’, itself, compared with PH. 69 Beyond the prevention of and response to biomedicalised health risks at the population level, Rowson defined GH as oriented towards the ‘underlying determinants of those problems, which are social, political and economic in nature.’ 32 It is questionable, however, to assume similar notions of health have not also been pursued in PH. Meanwhile, opposing views found GH and PH conceptually indistinguishable, 27 43 86 either as terms that could be used interchangeably, 95 or else as coconstitutive of one another, such that PH could be understood as a descriptive component of GH. 33 86

Differences between GH and IH echoed those drawn between GH and PH. For example, GH was characterised as more attentive to multidisciplinarity, while IH was said to implement a more limited biomedical approach to healthcare and health research. 1 69 95 Undergirding a major point of distinction between GH and IH was the belief that IH focuses on health problems in developing countries 1 22 32 43 45 48 54 83 86 93 and relies on ‘the flow of resources and knowledge from the developed to the developing world’, 32 whereas GH either is, or should be, more bidirectional. 1 45 84 In other cases, GH was described as comparable to IH, for example, when countries link GH efforts with development aid. 86 This is because the emphasis on delivering aid to poor countries reinforces an image of the world’s poor as needy subjects and, therefore, marks a continuation of IH and its sentiments under the guise of GH. 35

Finally, the field of TM was referenced to describe the evolutionary track of GH, particularly that GH is a modern-day product of the former. 20 25 57 69 75 84 A few authors critically pointed out that although GH has generally replaced TM and IH as terms embedded in histories of colonial power relations, many of the contemporary structures for governing and/or facilitating GH between countries today have remained largely the same, 25 48 54 62 suggesting that distinguishability between these terms too often occurs at the level of semantics.

Subtheme: GH is still vaguely defined

While GH was often described as a popular and well-established term, another key attribute repeated across the literature was its enduring vagueness. 23 25 26 31 33 43 45 48 52 62 74–77 81 86 Indeed, most papers commented on the term’s defiance of easy definition, its ambiguity and the lack of clarity regarding how people and organisations engaged in GH are using (or not using) the term to describe their interests. For example, Beaglehole and Bonita pointed out that research centres in low-income and middle-income countries are often engaged in GH issues but under other labels. 20 Some authors viewed the present lack of a clear and common definition as an obstacle endangering the coherence and maturation of the field. 33 35 45 For others, this indistinctness was thought to be precisely what gives GH such wide applicability, a certain degree of currency and political expediency. 45 76 81 86

A major concern cited was the lack of guidance for defining the term ‘global’ in GH. 26 34 43 48 75 As Bozorgmehr has outlined, the term is often used interchangeably within the GH community to mean ‘worldwide’, ‘everywhere’, ‘holistic’ and/or ‘issues that transcend national boundaries’. 48 This trend was noticeable within our review, as well. Engebretsen emphasised that GH ‘does not only allude to supranational dependency within the health field, but refers to a norm or vision for health with global ambitions’. 26 This view suggests that because the planet is populated by a multiplicity of positionings, perspectives and diverse world views, there can never be a truly a universal definition of ‘the global’ nor a global consensus around the definition of GH.

Finally, among studies that conducted original research into the definition of GH, several reported that study participants could not reach consensus on a definition. 52 74 75 77 Many thought it would be difficult if not impossible to arrive at a single, unified theoretical definition of GH, yet considered it important to formulate an operational definition of GH for guiding emerging activities related to GH. 23 45 77

This is the first study to systematically synthesise the literature defining GH and analyse the definitions found therein. All of the articles included in this study were published in peer-reviewed journals since 2009 indicating recent and steadfast interest in the topic of GH’s definition. This review examined GH definitions in the literature, and our thematic analysis focused on identifying recurrent themes across different definitions of GH.

Of the 78 articles included in this study, approximately one-third utilised empirical research methodologies to posit definitions of GH or else directly contribute towards the establishment of a common definition. Another one-third of papers summarised and discussed previously published definitions of GH (eg, reviews/overviews), while the remaining one-third suggested definitions of GH that were less grounded in analysis of empirical data than in the perspectives of its authors (eg, editorials, viewpoints). This systematic analysis indicated that the question of GH’s precise definition marks a point of controversy across fields of expertise. The variety of GH definitions posited by diverse experts in search of a common definition indicate that GH is multifaceted and polysemous.

In its broadest sense, GH can be defined as an area of research and practice committed to the application of overtly multidisciplinary, multisectoral and culturally sensitive approaches for reducing health disparities that transcend national borders. Indeed, it was most commonly defined across the literature in such general terms.

More specific definitions of GH were, of course, proposed by and considered valuable for many stakeholders in our review. Our analysis indicates that the precise definitions proposed by different experts were devised to serve particular functions. For example, narrow and concise definitions of GH were most frequently sought in the domains of governance and education, primarily for steering the development of policy frameworks and curricula, respectively. The imperative for an exact definition of GH in these subfields may be linked to bureaucratic demands for demarcating a technical term under which to classify specific activities, standardise certain functions, administer funds and direct workflow accordingly. It is also in this domain that authors most vociferously decried the absence of a unified and concise definition of GH, arguing this lack has led to ineffective initiatives, elusive methods for establishing accountability and instances of resource allocation based on ad hoc criteria—attractiveness to donors, public opinion, development agendum, foreign, economic or security policy priorities and so on—rather than via transparent mechanisms for adjudicating health need. 28 54 58 65 83 In contexts where health needs and upstream challenges were articulated, the lack of an agreed-upon definition oft impeded the policy process because stakeholders could not discern which GH issues among the multitude of different problems labelled as important were, in fact, the most pressing. 24 45 52 Because political indecision ramifies disproportionately for publics in countries where reliance on GH aid is a matter of life and death, establishing a clear definition of GH seems most crucial for the domain of governance.

We also found that detailed descriptions of GH’s specific conceptual and functional dimensions tended to reflect the specialisations or discipline-specific priorities of their authors. For example, definitions of GH stipulating the primacy of ‘cultural competency’ and ‘multidisciplinarity’ were more commonly proposed by interdisciplinary professionals in the literature on GH education than in journals of health policy, where definitions of GH were oriented more toward ‘security’ and ‘governance’ concerns. This suggests a correspondence between the subjective, experiential positions of the definers and the vocabulary they used to define or frame the need to define GH.

Unsurprisingly, we found that health professionals proposed the majority of definitions of GH in the literature. Additionally, the majority of publications and their authors were from higher income countries. Several authors in our review critically observed that GH has become institutionalised at a faster rate in higher income countries compared with lower and middle-income countries. 20 48 63 72 77 82 Their observations combined with our findings suggest that extant definitions of GH published in the literature or otherwise circulating in academic and professionalised spaces may unevenly reflect the interests and priorities of stakeholders from higher income countries. This suggests a need for greater diversity and inclusion in the debate on GH’s definition, as well as further reflexivity regarding who is defining GH, their means and motivations for doing so, and what these definitions put into action.

Interestingly, several articles published since 2019 have extended the debate on this topic of GH’s definition by directly engaging questions of geography and positionality: a recent commentary by King and Kolski defining GH ‘as public health somewhere else’ was met with pushback by those who argue that spatial definitions of GH are limited and limiting. 99–102

Limitations

To determine how GH is defined by experts in the literature, we ensured that the selection criteria developed for this study were broad enough to include a wide range of perspectives. Therefore, we included articles with varying degrees of evidentiary support, such as viewpoints, commentaries and editorials. Consequently, the results may be influenced by some of the primary researchers’ assumptions, projections, and biases. Backward citation tracking was used to add relevant articles to the review that had not been initially identified through database searching. This ensured that the review was exhaustive, however it also means that some conclusions drawn in the thematic analysis may have been influenced by this manual search strategy. By applying qualitative methods, this review provided a robust analysis of the thematic categories undergirding extant definitions of GH. A major limitation of this form of analysis is the extensive time required to develop and establish a code book and standardise the three coders’ use of the code book. However, this was deemed necessary to ensure consistency of judgement and intercoder reliability at each stage in the analysis. Another limitation of this study is that only articles written in English were included. To enhance the generalisability of results, future reviews should include data from non-English articles, especially if an inclusive, common definition of GH is to be achieved. Finally, this review was finalised prior to the emergence of the novel coronavirus. As such, future research should take into account new definitions of GH that emerge in light of the pandemic and lessons learnt.

Between 2009 and 2019, GH was most commonly defined in the literature in broad and general terms: as an area of research and practice committed to the application of multidisciplinary, multisectoral and culturally sensitive approaches for reducing health disparities that transcend national borders. More precise definitions exist to serve particular functions and tend to reflect the priorities of its definers. The four key themes that emerged from the present analysis are that GH is: (1) a multiplex approach to worldwide health improvement taught and researched through academic institutions; (2) an ethos that is guided by justice principles; (3) a mode of governance that yields influence through political decision-making, problem identification, the allocation and exchange of resources across borders and (4) a polysemous concept with historical antecedents and an emergent future. Findings from this thematic analysis have the potential to organise future conversations about which definition of GH is most common and/or most useful. Future discussions on the topic might shift from questioning the abstract ‘what’ of GH to more pragmatic and reflexive questions about ‘who’ defines GH and towards what ends.

Acknowledgments

Helpful comments by anonymous reviewers are acknowledged with thanks.

• Koplan JP ,
• Merson MH , et al
• Macfarlane SB ,
• Kickbusch I
• Liberati A ,
• Altman DG ,
• Tetzlaff J , et al
• Mengist W ,
• Soromessa T ,
• Carver JC ,
• Hassler E ,
• Hernandes E
• Missoni E , et al
• DiPace JI , et al
• Shamseer L ,
• Clarke M , et al
• Beaglehole R ,
• Bozorgmehr K
• Campbell RM ,
• De Cock KM ,
• Simone PM ,
• Davison V , et al
• Engebretsen E ,
• Bentley ME ,
• Buekens P , et al
• Johnston BD
• Gureje O , et al
• Kuhlmann AS ,
• Willott C ,
• Hughes R , et al
• Margolis CZ
• Kadandale S ,
• Nordström A , et al
• John CC , et al
• Speakman EM ,
• Joyner PU ,
• Aluttis C ,
• Arthur MAM ,
• Seidman G ,
• Chadi N , et al
• Bozorgmehr K ,
• Tinnemann P
• Sugarman J , Working Group on Ethics Guidelines for Global Health Training (WEIGHT)
• Bhutta ZA , et al
• Haffeld JB ,
• Røttingen J-A
• Hoffman SJ ,
• Stanhope M , et al
• Jogerst K ,
• Callender B ,
• Adams V , et al
• Schuster S ,
• Vajda C , et al
• Nascimento BR ,
• Brant LCC ,
• Moraes DN , et al
• Nemetchek B
• Steenhoff AP ,
• Crouse HL ,
• Lukolyo H , et al
• ↵ The International health regulations (IHR) – 10 years of global public health security . Wkly Epidemiol Rec 2017 ; 92 : 534 – 6 . pmid: http://www.ncbi.nlm.nih.gov/pubmed/28884994 OpenUrl PubMed
• Van Belle S ,
• Kickbusch I , et al
• Mendes IAC ,
• Klopper H , et al
• Benatar S ,
• Batra M , et al
• Havemann M ,
• Xu D , et al
• Peluso MJ ,
• Encandela J ,
• Hafler JP , et al
• Swanson RC ,
• Schensul JJ , et al
• Sienko KH ,
• Kaufmann EE
• Moore J , et al
• Sridharan S
• Erondu NA ,
• Marten R , et al
• Gostin LO ,
• Friedman EA
• Heymann DL ,
• Takemi K , et al
• Redwood-Campbell L ,
• Rouleau K , et al
• Keating EM ,
• Rowthorn V ,
• Schäferhoff M ,
• Kraus J , et al
• Sauerbier A ,
• Macerollo A ,
• Györfi O , et al
• Kleinman A , et al
• Gan Y , et al
• Turcotte-Tremblay A-M ,
• Fregonese F ,
• Kadio K , et al
• Herzig van Wees S ,
• Fischer SE ,
• Zielinski C , et al

Handling editor Seye Abimbola

Contributors MS initiated and designed the project. MS, MA and MM contributed to the implementation of the research, to the collection of data, analysis of the results and to the writing of the manuscript. PC supervised the project and provided feedback on the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Linked Articles

• Editorial The challenges of defining global health research Alberto L Garcia-Basteiro Seye Abimbola BMJ Global Health 2021; 6 - Published Online First: 30 Dec 2021. doi: 10.1136/bmjgh-2021-008169

Read the full text or download the PDF:

Electronic problem lists: a thematic analysis of a systematic literature review to identify aspects critical to success

Affiliations.

• 1 Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
• 2 Intermountain Healthcare, Salt Lake City, UT, USA.
• PMID: 29547974
• PMCID: PMC7647009
• DOI: 10.1093/jamia/ocy011

Objective: Problem list data is a driving force for many beneficial clinical tools, yet these data remain underutilized. We performed a systematic literature review, pulling insights from previous research, aggregating insights into themes, and distilling themes into actionable advice. We sought to learn what changes we could make to existing applications, to the clinical workflow, and to clinicians' perceptions that would improve problem list utilization and increase the prevalence of problems data in the electronic medical record.

Materials and methods: We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to systematically curate a corpus of pertinent articles. We performed a thematic analysis, looking for interesting excerpts and ideas. By aggregating excerpts from many authors, we gained broader, more inclusive insights into what makes a good problem list and what factors are conducive to its success.

Results: Analysis led to a list of 7 benefits of using the problem list, 15 aspects critical to problem list success, and knowledge to help inform policy development, such as consensus on what belongs on the problem list, who should maintain the problem list, and when.

Conclusions: A list of suggestions is made on ways in which the problem list can be improved to increase utilization by clinicians. There is also a need for standard measurements of the problem list, so that lists can be measured, compared, and discussed with rigor and a common vocabulary.

Publication types

• Meta-Analysis
• Systematic Review
• Medical Records Systems, Computerized*
• Medical Records, Problem-Oriented*

• Open access
• Published: 09 January 2024

Recurrence of post-traumatic stress disorder: systematic review of definitions, prevalence and predictors

• Samantha K Brooks 1 &
• Neil Greenberg 1  

BMC Psychiatry volume  24 , Article number:  37 ( 2024 ) Cite this article

1591 Accesses

3 Altmetric

Metrics details

Many people will experience a potentially traumatic event in their lifetime and a minority will go on to develop post-traumatic stress disorder (PTSD). A wealth of literature explores different trajectories of PTSD, focusing mostly on resilient, chronic, recovered and delayed-onset trajectories. Less is known about other potential trajectories such as recurring episodes of PTSD after initial recovery, and to date there has been no estimate of what percentage of those who initially recover from PTSD later go on to experience a recurrence. This systematic review aimed to synthesise existing literature to identify (i) how ‘recurrence’ of PTSD is defined in the literature; (ii) the prevalence of recurrent episodes of PTSD; and (iii) factors associated with recurrence.

A literature search of five electronic databases identified primary, quantitative studies relevant to the research aims. Reference lists of studies meeting pre-defined inclusion criteria were also hand-searched. Relevant data were extracted systematically from the included studies and results are reported narratively.

Searches identified 5,398 studies, and 35 were deemed relevant to the aims of the review. Results showed there is little consensus in the terminology or definitions used to refer to recurrence of PTSD. Because recurrence was defined and measured in different ways across the literature, and prevalence rates were reported in numerous different ways, it was not possible to perform meta-analysis to estimate the prevalence of recurrence. We also found no consistent evidence regarding predictors of PTSD recurrence.

A clear and consistent evidence-based definition of recurrence is urgently needed before the prevalence and predictors of recurrence can be truly understood.

Peer Review reports

Potentially traumatic events are common. Research suggests that over 70% of people will experience a potentially traumatic event (such as witnessing death or serious injury, automobile accident, life-threatening illness or injury, or violent encounter) in their lifetime [ 1 ]. Understandably, these events can be very distressing in the short-term and many people will experience acute post-traumatic symptoms in the immediate aftermath of a traumatic event, including intrusive symptoms (e.g. recurrent unwanted thoughts, nightmares); avoidance symptoms (e.g. emotional numbing, social withdrawing); hyperarousal (e.g. easily startled, feeling ‘on edge’); and physical symptoms (e.g. chest pain, dizziness) [ 2 ]. For the majority, these symptoms will decline naturally without intervention [ 3 ], typically within the first four weeks [ 2 ]. An important minority will find their symptoms persist for longer than a month. Those who continue to experience persistent re-experiencing of the traumatic event; avoidance of stimuli associated with the event; negative alterations in cognitions and mood and alterations in arousal and reactivity, causing clinical distress or functional impairment and not attributable to any other medical condition, are likely to be diagnosed with post-traumatic stress disorder (PTSD) [ 4 ]. Although only a minority of people who experience potentially traumatic events will go on to develop PTSD, it remains one of the most common mental disorders with lifetime prevalence estimated to be between 8% [ 5 ] and 12% [ 6 ]. PTSD is associated with reduced health-related quality of life and physical comorbidities, as well as major socio-economic costs [ 7 ].

The early 2000s saw a shift from studying PTSD itself as an outcome to studying change in symptoms as an outcome [ 8 ], with a wealth of studies using modelling approaches such as latent class growth analysis and latent growth mixture modelling to identify different trajectories of PTSD. Most of this literature identifies four trajectories, two of which are relatively stable trajectories ( chronic , a stable trajectory of post-traumatic stress symptoms, and resilient , a stable trajectory of healthy functioning after an adverse event), and two which display dynamic symptom patterns ( recovered , i.e. decreasing symptoms after an initial diagnosis of PTSD, and delayed-onset , i.e. increasing symptoms not meeting the diagnostic criteria for PTSD until potentially months or even years after traumatic exposure) [ 9 ]. Van de Schoot et al. [ 10 ] suggest that the two trajectories which typically occur less often (chronic and delayed-onset) are at risk of being overlooked by researchers or overwhelmed within the data by the larger trajectories. There may also be other less-researched or less-understood trajectories overlooked to an even greater extent. For example, one previous review [ 11 ] identified limited evidence of another, smaller trajectory referred to as a ‘relapsing’ or ‘recurring’ PTSD trajectory, in which individuals develop PTSD, are free from symptoms for long enough to be considered ‘recovered’, and then experience a recurrence of symptoms.

Recurrence is given relatively little attention in the PTSD literature, perhaps due to limitations of study methodologies and the complexities of studying recurrence. For example, Santiago et al. [ 11 ] note that few studies of PTSD follow participants for more than a year or with more than two assessments. Clearly, it would not be possible for researchers to identify recurrence of PTSD if data is only collected for two time-points: the only possible outcomes would be low symptom levels at each time-point (‘resilience’), high symptoms at each time-point (‘chronic’), or low level of symptoms at one time-point and a high level at the other (either ‘recovery’ or ‘delayed-onset’ depending on time-point at which symptoms were experienced). Additionally, studies which only follow up participants for a year or less are unlikely to clearly identify a recurrent trajectory of PTSD given the time needed to both recover and to experience a recurrent episode. The timing of PTSD assessment is also important: identification of PTSD recurrence relies on studies capturing the presence of symptoms during the recurrence, rather than before it occurs or after recurring symptoms have subsided. Therefore, it is perhaps unsurprising that the majority of the literature does not identify a ‘recurring’ trajectory of PTSD. Even studies which do identify recurrences often group these in with other trajectories: for example, Mota et al. [ 12 ] identified ‘recurrent’ cases of PTSD (individuals who had a lifetime diagnosis in 2002 and another post-2002 diagnosis reported in 2018), but grouped ‘persistent’ and ‘recurrent’ cases of PTSD together. Magruder et al. [ 13 ] identified a group of recurrent cases of PTSD – individuals who had lifetime PTSD pre-1992 but not a current diagnosis in 2002, who then had a diagnosis again in 2021, but these were grouped with ‘chronic’ cases. Karamustafalioglu et al. [ 14 ] simply include an ‘other’ group constituting both recurrent cases (individuals who met the criteria for PTSD diagnosis 1–3 months post-trauma and at the third follow-up 18–20 months post-trauma, but not at the second follow-up 6–10 months post-trauma) and others with delayed-onset PTSD which resolved. Boe et al. [ 15 ] identified a group of individuals with ‘reactivated’ PTSD who reported remission from PTSD in the first five years after the North Sea oil rig disaster of 1980 and a new episode at any point between 1985 and 2007. However, the authors suggest that there are blurred boundaries between delayed-onset and ‘reactivated’ PTSD, going on to include ‘possible delayed cases’ in their analysis of reactivated PTSD.

It is important to note that even the definitions of the more well-established trajectories of PTSD are not without their controversies. For example, Andrews et al. [ 16 ] point out the ambiguity in the criterion for delayed-onset PTSD, questioning whether ‘the onset of symptoms’ refers to any symptoms which might eventually lead to PTSD or only to full-blown PTSD itself. North et al. [ 17 ] comment on the ambiguities involved in the term remission (i.e. whether remission should be symptom-based or threshold-based) as well as the term onset (i.e. whether onset refers to first symptoms or first meeting diagnostic criteria). Definition of recovery also appears to differ from study to study, with some authors considering recovery to be symptom-based (i.e. no symptoms of the disorder remain) and others considering it to be threshold-based (i.e. some symptoms may remain, but they are beneath the diagnostic threshold) [ 18 ].

To date, several systematic reviews have been published which focus solely on only one PTSD trajectory. For example, previous reviews have focused on the delayed-onset trajectory [ 16 , 19 ]; the recovery trajectory [ 20 ]; and the resilient trajectory [ 21 ]. To date there has not been a literature review examining evidence of a recurrent trajectory of PTSD. Berge et al. [ 22 ] aimed to systematically review research on relapse in veterans but found no studies reporting actual rates of relapse or recurrence. Reviews have also explored the risk of relapse of various anxiety disorders, including PTSD, after discontinuation of antidepressants [ 23 ] and after cognitive behavioural therapy [ 24 ]. However, there have been no reviews attempting to quantify the risk of PTSD recurring, establish the predictors of recurrence, or quantify how much each predictive factor contributes to the risk of recurrence. The current review aimed to fill this gap in the literature by synthesising existing published data on how researchers define ‘recurrence’ of PTSD, recurrence rates of PTSD, and predictive factors of recurrence.

Having an appropriate understanding of recurrence is important as the concept needs to be properly understood in order to take steps to mitigate the risks of recurrent PTSD episodes. Mitigating the risk of PTSD recurring could benefit the health and wellbeing of trauma-exposed individuals and could reduce the socio-economic costs to the wider society [ 7 ]. The prevalence of recurrence is of particular importance to occupational medicine: regularly trauma-exposed organisations, for example, are often faced with decisions about when (and if) staff who have had and recovered from PTSD should return to the frontline duties. Understanding the risk of recurrent episodes may therefore have implications for those in charge of making such decisions. The present time is also a particularly relevant time to develop our understanding of recurrence of PTSD, as it is possible that the COVID-19 pandemic could contribute to recurrence. The pandemic has been declared a potential traumatic stressor, with research suggesting that COVID-19 survivors are at elevated risk of experiencing PTSD [ 25 ] and that PTSD symptoms may also develop due to quarantine [ 26 ], concerns about the health of loved ones, or economic loss as a result of the pandemic [ 27 ]. Hori et al. [ 28 ] suggest that the daily television updates regarding COVID-19 could trigger memories of surviving a previous traumatic situation, and exacerbate subthreshold PTSD symptoms. Therefore, experiencing the pandemic could potentially cause a recurrence of symptoms in people who have previously been diagnosed with PTSD.

The aim of this review was to collate literature which provides evidence of the lesser-studied ‘recurrent’ trajectory of PTSD and to identify: (i) the definitions of ‘recurrence’ used throughout the literature; (ii) prevalence of recurrence; and (iii) risk and protective factors for the recurrent trajectory of PTSD.

This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 29 ]. Our population of interest were people who had been diagnosed with, recovered from, and experienced a recurrence of PTSD (as diagnosed by a clinician or validated PTSD assessment tool). For the aim relating to prevalence of recurrent episodes, studies needed to involve a suitable design allowing prevalence to be assessed: for example, studies involving a population of people who had recovered from PTSD, followed over time to show how many had a recurrent episode and how many did not. For the other aims (i.e., definitions of recurrence and factors associated with recurrence), a comparison group was not necessary.

Registering the review

A protocol for the current review was developed and registered with PROSPERO on March 9th 2023 (registration number CRD42023405752). The only deviation from the protocol was the addition of another quality appraisal tool, due to finding a study design (retrospective analysis of existing health data) which we had not anticipated.

Eligibility criteria

To be included in the review, studies needed to (1) be published in peer-reviewed journals, (2) be published in the English language, (3) use quantitative methodology, (4) use a standardised tool to assess PTSD and (5) present data on recurrence rates of PTSD and/or factors associated with PTSD recurrence. There were no limitations relating to publication date or location of the studies. Case studies were excluded but there were no other exclusion criteria relating to population size.

Data searching and screening

A systematic literature search was carried out to examine definitions, prevalence rates and predictors of PTSD recurrence. Four electronic databases (Embase, PsycInfo, Medline and Web of Science) were searched on 24th November 2022, using a combination of search terms relating to PTSD, recurrence, and prevalence/predictors which were combined using Boolean operators. The full list of search terms is presented in Appendix 1 . The US Department of Veterans Affairs National Center for Post-Traumatic Stress Disorder’s PTSDPubs database (formerly PILOTS) was searched separately on the same date using the individual terms ‘recurrence’ and ‘recurrent’ and limited to peer-reviewed articles. Reference lists of articles deemed to meet the inclusion criteria were also hand-searched.

All citations resulting from the literature searches were downloaded to an EndNote library where duplicates were removed. The titles of all citations were then screened for relevance to the review, with any clearly not relevant being excluded. Abstracts were then screened for eligibility and the full texts of all remaining citations after abstract screening were located and read in their entirety to identify studies meeting all inclusion criteria. The literature searches and screening were carried out by the first author. The two authors met regularly throughout the screening process to discuss any uncertainties about inclusion or exclusion until a decision was reached.

Data extraction

The first author carried out data extraction of all citations deemed to meet the inclusion criteria. Data were extracted to a Microsoft Excel spreadsheet with the following headings: authors, year of publication, country, study design, sampling method, inclusion/exclusion criteria, study population size, socio-demographic characteristics of participants, type of trauma exposure, time-points at which PTSD was assessed, tools for assessing PTSD, definitions of recovery and recurrence, whether any PTSD treatment was received, prevalence rates of recurrence, and factors examined as potential predictors of recurrence.

Data synthesis

For the first aim of the review (relating to definitions of recurrence), we designed a table to present data relating to how ‘recurrence’ was understood and defined in each study. The tools used to diagnose and measure PTSD symptoms in the first place are important in understanding how PTSD is defined, so first the assessment tools used in each study were extracted into the table. Given that we wanted to understand the length of time an individual needs to be free of PTSD in order to be considered ‘recovered’, for each study we also included the time-points of PTSD assessment in the table. Next, we included the definitions of recovery and recurrence from each study, explained narratively in the table. We also added information to this table to report whether participants had received PTSD treatment during each study, as some studies focusing on interventions used ‘response to treatment’ in their definitions of recovery. We compared the different definitions used within the studies to establish whether there was consensus within the literature around (i) whether recovery and recurrence are symptom-based or threshold-based and (ii) how long the recovery period between initial diagnosis and recurrent episodes needs to be in order to be considered recurrent rather than chronic PTSD.

The second aim related to prevalence of PTSD recurrence. Due to the various research designs and definitions of ‘recurrence’ in the literature, as well as the different ways in which prevalence was reported, meta-analytic techniques could not be used. Rather, we presented the prevalence data as it was reported in each study. This sometimes meant presenting the prevalence of PTSD recurrence within an entire trauma-exposed population, including those who never experienced PTSD at any time. Other times, this meant presenting the prevalence of PTSD within a population who all had PTSD at one time-point, and other times this meant presenting the prevalence of PTSD within a group who had recovered from PTSD.

Finally, in order to explore factors associated with PTSD recurrence, all variables considered as potential covariates were recorded individually for each study. Each potential predictive factor was descriptively reported in a table, and any found to be significantly associated with experiencing PTSD recurrence were bolded to differentiate between non-significant and significant findings. Factors are also described narratively within the results section. Insights from thematic analysis [ 30 ] were used to group similar data together. For example, data relating to gender or age as predictors of recurrence were coded ‘socio-demographic’ and discussed together within the results.

Quality appraisal

We appraised the quality of studies using National Institutes of Health (NIH) tools: either the Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies or the Quality Assessment of Controlled Intervention Studies tool, depending on study design. Concurrent with other reviews [e.g. 31 ] we rated quality as ‘poor’ if studies scored 0–4/14, ‘fair’ if they scored 5–10/14 and ‘good’ if they scored 11–14/14. One study used retrospective analysis of existing health data, and for this study we used the MetaQAT Critical Appraisal Tool [ 32 ]. To keep the ratings consistent with our rating system for the studies appraised by NIH tools, we defined ‘poor’ quality as a score of 0–34%, ‘fair’ quality as a score of 35–72% and ‘good’ quality as a score of 78% or higher.

Literature searches yielded 5,398 citations of which 1,083 were duplicates. After title and abstract screening, 4,210 citations were excluded leaving 105 citations for full-text screening. After reading full texts of the remaining citations, 75 were excluded and an additional five studies were added after hand-searching reference lists. A total of 35 citations were included in the review [ 15 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ]. Figure  1 illustrates the screening process in a PRISMA flow diagram.

PRISMA flow diagram of screening process

Table  1 provides an overview of key characteristics of all included studies. Studies originated from the United States of America (n = 13), Denmark (n = 5), Israel (n = 4), China (n = 4), Norway (n = 2), the United Kingdom (n = 2), Japan (n = 1), the Netherlands (n = 1), Switzerland (n = 1), and Turkey (n = 1). The remaining study included participants in multiple different countries across Europe and Asia. Study populations ranged from 35 to 7,918 and included military personnel (n = 15), civilian adults (n = 14), children or adolescents (n = 4) or a combination of military and civilian adults (n = 2). Only three studies were rated as ‘good’ quality; the majority were rated ‘fair’.

Definitions of recurrence

Table  2 reports, for each study, the tools used to assess PTSD; time-points at which PTSD was assessed; definitions of recovery and recurrence; and whether the participants received PTSD treatment or not.

Terminology

The first aim of the review was to explore how ‘recurrence’ is defined in the literature. We found no consensus in terms of how this is defined. In fact, the studies used a variety of different terms to describe the emergence of new PTSD episodes after initial ‘recovery’, including ‘recurrence’ [ 33 , 37 , 44 , 47 , 64 , 65 ]; ‘relapse’ [ 35 , 36 , 40 , 49 , 50 , 52 , 53 , 57 ]; ‘reactivation’ [ 15 , 60 , 62 ]; ‘exacerbation/reactivation’ [ 61 ]; ‘relieved-worsening PTSD’ [ 34 , 48 , 51 , 63 ]; ‘response-remit’ trajectory [ 54 ]; ‘fluctuating course’ [ 58 ]; ‘intermittent cases’ [ 43 ]; ‘delayed increase in symptoms’ [ 46 ]; and the ‘relapsing/remitting’ trajectory [ 42 , 55 ]. Many others simply described recurrence as ‘symptom increase’ [ 38 ], ‘initial declines followed by symptom increases’ [ 56 ] or ‘exacerbation of symptoms’ [ 41 , 60 ]. Some studies did not name the trajectory at all; rather, they presented tables or flow charts showing the number of participants with PTSD at each time-point, from which it was possible for us to identify a sub-group of participants who were described as having PTSD at one time-point, not having it at least one follow-up, and then having it again at subsequent time-points [ 39 , 59 ]. Similarly, Hansen et al. [ 45 ] identified and commented on a sub-group of participants who met the criteria for PTSD, did not meet the criteria at a subsequent time-point, and then met the criteria again later, but they did not give this a name.

Criteria for recurrence

Several studies defined recurrence (or equivalent terminology such as relapse) as meeting diagnostic criteria for PTSD at a follow-up time-point after an initial ‘recovery’ period where they did not meet the cut-off for PTSD [ 33 , 35 , 37 , 39 , 43 , 45 , 46 , 58 , 65 ]. Holliday et al. [ 47 ] referred to ‘clinically meaningful change in PTSD symptoms’, which was also assumed to refer to clinical cut-off scores. Markowitz et al. [ 52 ] based the definition of relapse on similarity to baseline scores. Sungur and Kaya [ 64 ] defined recovery and recurrence as being asymptomatic and then symptomatic again, but it is not clear whether this referred to clinical cut-offs. One study defined ‘reactivation’ of PTSD as meeting full diagnostic criteria or being a sub-syndromal case [ 15 ]. Others were more vague and did not mention cut-offs, instead referring to dramatic or steep symptom increases [ 34 , 38 , 56 , 63 ], fluctuating symptoms [ 42 , 55 ], returning to pre-treatment levels of PTSD [ 54 ], symptoms which ‘decreased somewhat and increased drastically’ [ 48 ], symptoms which ‘decreased to a low level and increased again’ [ 49 , 50 ] or ‘steadily worsening’ symptoms [ 36 ]. DenVelde et al. [ 41 ] simply asked participants to self-report whether they had ‘experienced remissions and exacerbations’. Martenyi et al. [ 53 ] had multiple definitions of relapse, including increases in scores on their PTSD measures or ‘the clinical judgement of the investigator’. Others labelled the trajectory but did not specify the parameters of their definitions [ 51 , 60 , 61 , 62 , 66 ]. One study [ 57 ] used ‘being hospitalised’ as a proxy measure of PTSD recurrence, although this way of defining recurrence would obviously not capture individuals who developed recurring symptoms which were not severe enough to warrant hospitalisation; additionally, no criteria for hospitalisation were described. Similarly, Davidson et al. [ 40 ] described ‘relapse’ as PTSD scores reverting back to baseline or worse, or experiencing an ‘untoward clinical event’ including suicidality, hospitalisation, or dropping out of the study due to feeling progress was not being made.

We found little consensus as to how long participants needed to be symptom-free (or have reduced symptoms) in order to be considered ‘recovered’ prior to recurrence. The majority of studies simply based their definitions on the time-points of the study, suggesting that recurrence was identified if participants had PTSD at baseline, did not have PTSD during at least one follow-up, and then had PTSD again at a later follow-up. The time-points of follow-ups ranged from weeks to months to years. Only four studies suggested specific timeframes: three studies claimed that participants needed to be ‘recovered’ for eight weeks in order for later reports of PTSD to count as ‘recurrence’ rather than symptom fluctuation [ 35 , 37 , 66 ] whereas Zanarini et al. [ 65 ] reported that participants needed to be not meeting the PTSD criteria for at least two years in order to be considered ‘recovered’. Similarly, most studies did not clarify a time-scale for how long symptoms needed to be experienced in order to be considered a ‘recurrence’. Most studies again simply based their diagnosis on the scores participants happened to report on the days they were assessed. Few studies specified a time-frame: three [ 35 , 43 , 65 ] suggested a duration of four consecutive weeks of meeting their criteria for PTSD, while Benítez et al. [ 37 ] suggested two weeks of symptoms was sufficient to identify a recurrent episode.

Prevalence of recurrence

The review’s second aim was to explore PTSD recurrence rates. Table  3 presents data on the prevalence of recurrence of PTSD for each study. The second column of Table  3 presents the data that is reported in the original studies. The findings reported in this column are not easily comparable because studies reported recurrence rates in different ways. Some reported the percentage of the entire trauma-exposed sample who experienced PTSD recurrence (column 3 of Table  3 ). Others reported the percentage of those with PTSD who experienced recurrence (column 4 of Table  3 ) and the remaining studies reported the percentage of those who recovered from PTSD who experienced recurrence (column 5 of Table  3 ). Three studies [ 44 , 47 , 57 ] did not report the prevalence of recurrence, but were still included in the review as they included definitions and/or predictors of recurrence. One study [ 60 ] deliberately chose a sample who had all experienced recurrence; therefore, recurrence prevalence data for this study was not recorded in Table  3 as it would, by design, be 100%.

Most studies (19/35) reported the prevalence of recurrence within the entire trauma-exposed population. We would therefore expect prevalence rates to be extremely small, given that the majority of trauma-exposed people will not develop PTSD in the first place [ 3 ], let alone have recurrent episodes. However, in several studies this was not the case. Prevalence of recurrence ranged from 0.2% (for a sub-set of participants who did not directly witness the disaster in question) [ 45 ] to 57% of 63 women newly-diagnosed with ovarian cancer [ 43 ]. The latter study was carried out over 27 weeks and identified ‘intermittent cases’ who had PTSD at one time-point, no PTSD at a later time-point, and then PTSD again later on. We note that 27 weeks is a fairly short period of time for both recovery and recurrence to occur, and it is therefore possible that the data reflects symptom fluctuations rather than true recovery or recurrence. Overall, the mean prevalence of recurrent PTSD in trauma-exposed populations was 13.1%, and the median was 3.8%.

Five studies presented the prevalence of recurrence within populations diagnosed with PTSD. We would expect these prevalence rates to be higher than the prevalence rates of recurrence within full trauma-exposed samples, as they are based on populations who developed PTSD only. The rates were 4.9% [ 39 ], 15.4% [ 66 ], 24.5% [ 36 ], 28% [ 46 ] and 49.6% [ 41 ]. Mean and median prevalence of recurrent PTSD were both 24.5%.

Seven studies presented data on the prevalence of recurrence within sub-sets of study populations who had recovered from PTSD; therefore, the only possible trajectories for these participants would be recurrence or maintenance of recovery. Recurrence rates ranged from 5.8% (for a sub-set of participants treated with fluoxetine) [ 53 ] to 50% (for a sub-group treated with a placebo) [ 40 ]. Mean prevalence of recurrent PTSD was 25.4% and the median was 22.2%.

The three studies rated highest in quality [ 34 , 47 , 55 ] did not report similar findings relating to prevalence. Holliday et al. [ 47 ] did not present prevalence data at all. Andersen et al. [ 34 ] reported that 2% of participants followed the ‘relieved-worsening’ trajectory, whereas Osenbach et al. [ 55 ] reported that 35% of participants followed the ‘relapsing-remitting’ trajectory. Notably, Andersen et al.’s [ 34 ] participants were military personnel, whilst Osenbach et al.’s [ 55 ] participants were civilian trauma survivors. For this reason, we decided to look separately at recurrence rates in military and civilian participants. We also decided to look separately at data on children as children’s experiences during and after potentially traumatic events are likely to be distinct from those of adults [ 67 ]. Table  4 presents the mean and median recurrence rates for different populations.

Prevalence of PTSD recurrence in military populations

Fifteen studies focused on military personnel and veterans, three of which did not provide prevalence data and one of which included only participants with PTSD recurrence. Military studies which presented rates of recurrence in trauma-exposed populations (rather than focusing on people diagnosed with PTSD only) typically found low prevalence of recurrence: seven studies found prevalence rates under 4% [ 34 , 48 , 51 , 54 , 61 , 62 ]. Another study found a prevalence rate of 6% [ 38 ]. The only higher prevalence rates were reported by Solomon & Mikulincer [ 59 ], who reported recurrence rates of 24.4% for those with combat stress reactions (people referred for psychiatric intervention during the war) and 13.2% for participants who participated in combat in the same units but without need for psychiatric intervention during the war. This study assessed participants over twenty years, which may explain its higher prevalence rate than the majority of studies which were completed within two-and-a-half years or less. However, the study period was shorter than the forty-seven years of Solomon et al.’s [ 62 ] study, which reported only a 1.6% rate of recurrence. It is unclear why Solomon and Mikulincer [ 59 ] found much higher rates of recurrence.

Two military studies reported recurrence rates for PTSD-populations. These were 24.5% [ 36 ] and 49.6% [ 41 ]. We note that all of Armenta et al.’s [ 36 ] participants had comorbid depression at baseline. We also note some concerns about the reliability of DenVelde et al.’s study [ 41 ], which was a retrospective study asking participants to give complete life-history data at one time-point only.

One military study reported on the prevalence of recurrence in a sub-group of participants who had recovered. Solomon et al. [ 62 ], who reported a prevalence rate of 1.6% (out of the entire trauma-exposed sample) over the first forty-two years of the study, found in a follow-up at forty-seven years that 16.7% of those who had initially recovered experienced recurrence of PTSD during the COVID-19 pandemic.

Prevalence of PTSD recurrence in civilian adult populations

Fourteen studies focused on civilian adults. Findings relating to recurrence prevalence in entire trauma-exposed samples were mixed. Two studies reported rates of under 5% [ 45 , 58 ] in survivors of a terrorist attack and an earthquake respectively. Sungur and Kaya [ 64 ] reported a recurrence rate of 8.9% in survivors of the Sivas disaster, a religious fundamentalist protest which resulted in civilian deaths. Higher rates of recurrence were reported for survivors of an oil rig disaster (18.8%) [ 15 ], survivors of an oil spill (32%) [ 56 ], acutely injured trauma survivors (35%) [ 55 ] and women recently diagnosed with ovarian cancer (57%) [ 43 ].

For populations of civilians with PTSD only, recurrence rates were 4.9% [ 39 ] (type of trauma not reported), 15.4% [ 66 ] (trauma type varied), and 28% [ 46 ] (participants severely injured in accidents). Four studies reported data on the prevalence of recurrence in populations who had previously recovered from PTSD. Reported rates were 14% [ 52 ] (trauma type varied), 29.5% [ 37 ] (trauma type varied), 34% [ 35 ] (trauma type not reported) and 40% [ 65 ] (trauma type varied).

Prevalence of PTSD recurrence in children

Four studies focused on recurrence in adolescents / children, with mixed findings. Fan et al. [ 42 ] found that 3.3% of 1,573 earthquake survivors experienced ‘relapsing/remitting’ PTSD. Liang et al. [ 49 , 50 ] found that 17.7% of 301 earthquake survivors experienced the ‘relapsing’ trajectory of PTSD. An et al. [ 33 ] found that 37% of 246 adolescents experienced ‘recurrent dysfunction’ after experiencing an earthquake.

Prevalence of PTSD recurrence in combined military and civilian populations

Finally, two studies included both military and civilian participants; both of these studies were trials comparing fluoxetine to placebo treatment in people with PTSD. Davidson et al. [ 40 ] found that half of the placebo group relapsed after recovery, compared to 22.2% of the fluoxetine group. Martenyi et al. [ 53 ] reported lower rates of ‘relapse’: 16.1% of the placebo group and 5.8% of the fluoxetine group. The latter study followed up participants after 36 months, while Davidson et al. [ 40 ] followed up participants for a year after treatment.

Predictors of PTSD recurrence

The third and final aim of the present review was to identify factors associated with PTSD recurrence. Firstly, we note that (as shown in Table  2 ), participants in a number of studies had received some type of intervention during the study period, which was typically not accounted for in analyses of predictors. Many other studies did not report whether participants received treatment or not. Having treatment, whether it be medication, therapy, or a combination, is likely to be an important factor influencing PTSD trajectory, given that there are evidence-based treatments for the condition [ 68 ], but this was typically not explored.

Table  5 shows the factors considered as predictors in each study, with significant associations presented in bold. The majority of included studies (22/35) explored at least one covariate; the remaining studies either did not explore covariates or combined recurrent trajectories with other trajectories in their analyses of predictors. Of those studies which did explore covariates of recurrence, we found little consensus.

Sociodemographic factors

Gender was considered as a potential covariate by six studies; one [ 33 ] found that recurrent PTSD was associated with female gender while five studies (including two based on the same data-set) [ 49 , 50 ] found no significant gender association [ 35 , 36 , 42 , 49 , 50 ]. None of the three studies testing age as a covariate found a significant association [ 35 , 36 , 57 ]. One study of school-aged children found that children in a higher grade (i.e. older in age) were more likely to experience PTSD recurrence [ 33 ], while three studies of two cohorts [ 42 , 49 , 50 ] found no significant association between recurrence and school grade. Three studies considered race as a covariate, finding no significant association between PTSD recurrence and race [ 36 , 44 , 55 ]. Other socio-demographic characteristics considered included number of children in the family [ 42 ], marital status and level of education [ 36 ], none of which were found to be associated with PTSD recurrence. For military participants, there were no significant differences in service branch, service component or pay grade between the recurrent and rapid recovery groups [ 36 ].

Psychiatric history

Seven studies considered psychiatric history and concurrent diagnoses as potential covariates of PTSD recurrence, again with mixed findings. Recurrence was not found to be associated with other anxiety syndromes [ 36 ], baseline levels of anxiety [ 54 ], depressive symptoms [ 55 ], baseline levels of depression [ 54 ] or psychiatric history [ 55 ]. Ansell et al. [ 35 ] found that diagnoses of a number of co-morbid mental health disorders such as major depressive disorder and personality disorders such as schizotypal personality disorder, avoidant personality disorder and borderline personality disorder were not associated with recurrence, but participants with a baseline diagnosis of obsessive-compulsive personality disorder were significantly less likely to experience PTSD recurrence. Conversely, Perconte et al. [ 57 ] found that those who experienced recurrence were significantly more likely to report obsessive-compulsive symptoms than those whose symptoms improved without recurrence. Sakuma et al. [ 58 ] found that pre-disaster treatment for mental illness was significantly associated with PTSD recurrence, but note that the results should be interpreted carefully due to the very small number of participants in the ‘fluctuating symptoms’ group who appeared to have experienced recurrent episodes. Perconte et al. [ 57 ] found that, versus the improved symptoms group, those with PTSD recurrence were more likely to report depression, anxiety, hostility, phobic anxiety, somaticism and psychoticism; however, previous psychiatric hospitalisations and pre-treatment ratings of global pathology on a psychiatric scale did not predict recurrence. Finally, Madsen et al. [ 51 ] found that suicidal ideation was significantly higher in the ‘relieved-worsening PTSD’ group than the ‘low-stable’ group and that suicidal ideation was in fact highest in the recurrent (termed ‘relieved-worsening’) group than any other. However, it should be noted that suicidality was not assessed at baseline in this study, therefore it is not clear whether suicidal ideation is a cause or a consequence of PTSD recurrence.

Physical health

Fewer studies considered physical health as a potential predictor of PTSD recurrence. One study found no association between recurrence and disabling injury/illness, somatic symptoms or bodily pain [ 36 ] and another found no association between recurrence and prior treatment for physical illness [ 57 ]. However, obesity was a significant predictor of PTSD recurrence [ 36 ]. In terms of health-related behaviours, Armenta et al. [ 36 ] found no association between PTSD recurrence and smoking status, alcohol problems or sleep duration. However, Perconte et al. [ 57 ] found that higher weekly alcohol intake both before and at termination of PTSD treatment predicted recurrence.

Cognitive ability

Only one study [ 63 ] explored cognitive ability as a potential covariate, finding that the participants who were in the recurrent (termed ‘relieved-worsening PTSD’) group had significantly lower cognitive ability scores than those in the ‘low-stable’ group.

Trauma history and pre-trauma experiences

The review also found mixed evidence for trauma history as a predictor of PTSD recurrence. Liang et al. [ 49 , 50 ] found no association between pre-disaster traumatic experience and PTSD recurrence. Armenta et al. [ 36 ] found no association between recurrence and childhood sexual abuse, childhood verbal abuse, childhood neglect, sexual assault, physical assault, or ‘other life events’, but did find that participants reporting a history of childhood physical abuse were significantly more likely to experience PTSD recurrence. Holliday et al. [ 47 ] found that veterans who had experienced military sexual trauma (MST) had greater initial reductions in PTSD symptoms than those who had not experienced MST, but also experienced a ‘modestly greater’ recurrence of symptoms than those without MST, although this difference did not appear to reach statistical significance. Zanarini et al. [ 65 ] found that the presence of childhood sexual abuse history did not significantly predict time-to-recurrence, but severity of childhood sexual abuse, adult rape history, combination of childhood sexual abuse history and adult rape history, and experiencing sexual assault during study follow-up were associated with less time-to-recurrence. Osofsky et al. [ 56 ] found that abuse, emotional abuse, domestic violence, and greater number of traumas experienced were associated with recurrence of PTSD, and Osenbach et al. [ 55 ] found that recurrent life stressors significantly increased the odds of membership in chronic, relapsing or recovery groups rather than the resilient group. For military participants, one study found combat deployment was significantly associated with recurrent PTSD [ 36 ] while others found combat exposure was not associated with recurrence [ 54 , 57 ]. Finally, Fan et al. [ 42 ] found that compared to the recovery group, relapsing participants experienced significantly fewer negative life events 6-months post-disaster, but significantly more such events at the 24-month follow-up.

Few other pre-trauma experiences were considered. An et al. [ 33 ] found that those with recurrent PTSD were significantly more likely to have experienced academic burnout than those in the recovery trajectory, although there was no difference between the recurrent and delayed trajectories.

Experiences during and immediately after the traumatic experience

The review also found mixed evidence for an association between peri-traumatic experiences and PTSD recurrence. The most consistent finding related to how stressful the traumatic experience was perceived to be at the time. For example, risk of recurrence was significantly higher in those with combat stress reactions [ 59 ] and in those with higher stress relating to the disaster they had experienced [ 56 ], as well as with greater trauma severity [ 49 , 50 ]. However, recurrence was not found to be associated with subjective fear during the event [ 33 ]; directly witnessing a disaster [ 42 ]; property loss during the event [ 33 , 42 ]; property damage [ 42 ]; displacement due to property damage [ 58 ]; near-death experience [ 58 ]; or having a family member injured, killed or missing [ 42 , 58 ].

There was some evidence that initial post-traumatic stress symptoms immediately after the traumatic event could predict PTSD trajectory. Liang et al. [ 49 , 50 ], in a study of PTSD in children from two schools affected by an earthquake, found that children from one of the two schools (‘School 2’) were significantly more likely to experience PTSD recurrence than children from the other school (‘School 1’). Further investigations revealed that after adjusting for immediate post-traumatic stress symptoms the school no longer predicted relapse; those from School 2 had significantly greater post-traumatic stress symptoms immediately after the disaster, which the authors suggest might be due to School 1 providing sufficient psychological services as well as having the same students and teachers before and after the earthquake (therefore perhaps greater social support available), whereas School 2 had insufficient psychological services and consisted of teachers and students from several different schools which could not be reconstructed after the earthquake.

One study [ 58 ] considered occupational-related covariates of PTSD recurrence for disaster recovery workers. They found that having mainly disaster-related occupational duties and lack of rest due to occupational duties were not associated with recurrence, but perceived poor workplace communication did predict recurrence.

Post-trauma experiences and symptoms

An et al. [ 33 ] found that, compared to the delayed PTSD trajectory, those who experienced recurrence were less likely to have experienced post-traumatic growth after the traumatic event; however, there were no differences in post-traumatic growth between the recurrent and recovery groups. Fan et al. [ 42 ] found that neither positive coping nor negative coping six months post-disaster were associated with PTSD recurrence. In a military study, Karstoft et al. [ 48 ] found that poor adjustment to civilian life (i.e. difficulties with community reintegration after deployment) was significantly higher for the recurrent (‘relieved-worsening PTSD’) group than all other groups. However, it is not clear whether poor adjustment was a cause or an effect of PTSD symptoms worsening after initial improvement.

Two studies explored specific cluster symptoms. Murphy and Smith [ 54 ] found PTSD recurrence was not predicted by the magnitude of re-experiencing, avoidance, or hyperarousal symptoms. Boe et al. [ 15 ] found that the number of intrusion and avoidance symptoms five-and-a-half months post-trauma did not predict recurrence, but the number of intrusion and avoidance symptoms both fourteen months and five years after the disaster did predict recurrence.

Social support

Only three studies directly considered social support as a potential covariate. Armenta et al. [ 36 ] found no association between social support and PTSD recurrence, and Perconte et al. [ 57 ] found that family support did not predict recurrence. Fan et al. [ 42 ] found that level of social support six months after experiencing an earthquake was not associated with PTSD recurrence, but those in the ‘relapsing’ group reported significantly less social support 24 months after the earthquake than those in the ‘recovery’ group.

PTSD treatment

Most of the studies investigating treatment for PTSD found that not receiving interventions, or discontinuing treatment, were associated with PTSD recurrence. For example, Osenbach et al. [ 55 ] found that those who received ‘usual care’ only were significantly more likely to experience recurrence than those who received interventions designed to reduce post-traumatic symptoms. Davidson et al. [ 40 ] found that those who received placebo treatment were significantly more likely to experience recurrence than those who received fluoxetine. Martenyi et al. [ 53 ] found that those who discontinued fluoxetine treatment were significantly more likely to experience recurrence, especially for those with combat-related PTSD. However, Perconte et al. [ 57 ] found that number of weeks enrolled in treatment and number of treatment sessions attended did not significantly affect risk of recurrence. In this study, though, being hospitalised at least once since the termination of treatment was used as a proxy measure of ‘recurrence’ and so the findings are arguably not truly representative of actual recurrent episodes of PTSD. Overall, our findings indicated some evidence that treatment helped to avoid recurrent episodes.

In this study, we systematically reviewed 35 studies to identify definitions and prevalence of recurrent PTSD and factors associated with recurrence. It is important to define and operationalise recurrence as the concept needs to be understood in order to make prevention efforts. The health-related, social and economic costs of PTSD can be substantial. PTSD negatively affects individuals’ emotional wellbeing and physical health [ 7 ], impedes social relationships [ 69 ], limits productivity at work and increases sickness absence [ 70 ]. The direct costs (e.g., medical care costs) and indirect costs (e.g., costs of unemployment or reduced productivity) of PTSD can create substantial economic burden [ 7 , 71 ]. Determining the predictors of recurrence of PTSD (which can only be properly understood if ‘recurrence’ itself has a clear definition) is important for prevention efforts: identifying those most at risk for recurrent episodes would allow for the subsequent investigation of ways of mitigating or preventing the risk. However, we found little consensus as to how recurrence is defined, mixed evidence on the prevalence of recurrence and inconsistent findings relating to predictors of recurrence. This lack of clarity about what relapse or recurrence is, and is not, is a major barrier to understanding this important topic.

In a previous review exploring PTSD recurrence in veterans, Berge et al. [ 22 ] acknowledge that there is no generally accepted or used definition of recovery relating to psychological trauma. The definition of recurrence used in their review was the return of symptoms following a period of complete recovery, representing the start of a new and separate episode . However, it is not clear what length of time is covered by ‘a period of complete recovery’ nor what ‘complete recovery’ means. How many days, weeks, or months does an individual need to be free of symptoms of PTSD in order to be considered truly recovered? Is ‘symptom-free’ the only definition of recovery, or is ‘not meeting the criteria for PTSD’ enough? Our own review revealed that there is little consensus as to what recurrence means and the parameters for its definition. Even the terminology used varied across studies, with ‘relapse’, ‘recurrence’, ‘reactivation’ and numerous other terms often used to describe what essentially appeared to be the same concept. There was no consensus as to how long an individual needed to be free of symptoms in order to be considered recovered, nor for how long symptoms needed to recur in order to be considered a recurrent episode. Most studies simply defined recurrence as a change in symptoms between assessments, meaning that whether or not an individual was defined as having a recurrent episode or not very much depended on the scores they reported at arbitrary time-points. Even minor symptom fluctuations could cause someone to change from being identified as a ‘case’ to ‘recovered’ and vice versa. Because PTSD tended to be examined using prospective studies where symptoms were assessed at predetermined assessment points, it is possible that individuals may have onsets of PTSD after one assessment and then remit before the next. With no retrospective assessment between time-points, it is difficult to assess the true prevalence of recurrence. Andrews et al. [ 16 ] make a similar point in relation to delayed onset PTSD, suggesting the absence of information about symptoms outside of the predetermined time-points of studies means that estimates of delayed onset PTSD may be unreliable.

The second aim of the review was to examine the prevalence of PTSD recurrence in existing literature. Given the numerous different ways of assessing PTSD, defining initial recovery and defining recurrence, as well as the differing time-points at which PTSD was assessed across studies, we suggest that the current data on recurrence prevalence is not especially meaningful. We found very different prevalence rates reported within the literature, with data suggesting that anywhere between 0.2% and 57% of trauma-exposed populations might experience recurrent episodes of PTSD. Some of the higher percentages we found seem greater than we would expect, given that only a minority of trauma-exposed people are likely to develop PTSD in the first place – let alone suffer from it, recover from it, and experience a recurrent episode. We would expect that studies carried out over a longer period of time would find higher recurrence rates, simply because in these studies there is more time for recurrent episodes to occur. However, the highest prevalence rate (57%) was found in a study which took place over only 27 weeks [ 43 ]; the authors labelled these participants as ‘intermittent cases’ and it appears likely that symptom fluctuation, rather than true recovery and recurrence, occurred in this study – and potentially many others. Additionally, studies did not typically control for exposure to subsequent trauma, meaning that ‘recurrences’ of PTSD identified may actually be new episodes, rather than a relapse. Further research studies, especially research involving assessments over a number of years, are needed to establish the true prevalence of recurrent PTSD which also needs to be clearly defined with an agreed time period between remission and relapse.

It has been proposed that recurrence rates might increase with old age. Murray [ 72 ] suggests that PTSD can be ‘reactivated’ in older age because physical illnesses become more common, which can reactivate traumatic memories; increased dependence on others due to ageing can reactivate feelings of helplessness; and loss of structure and identity caused by retirement can similarly reactivate traumatic symptoms. Other factors relating to ageing such as decline of cognitive function, difficulty controlling ruminations, reminiscing, and late-life stressors such as serious illness, surgical procedures and death of spouses, siblings or close friends can either directly remind the person of their previous traumatic experience(s) or can induce similar feelings of vulnerability [ 73 ]. Three studies of adults in this review did not find age predicted recurrence [ 35 , 36 , 57 ]; however, the populations trended young overall, with each of the three studies reporting the mean age of participants was under 40. We suggest, then, that more studies of older adults with lifetime PTSD are needed to establish whether this group are at increased risk of recurrence.

The third aim of this review was to understand factors associated with PTSD recurrence. Although a number of potential covariates were considered, most were not investigated by more than a few studies, and findings were varied and inconsistent. Of the covariates investigated by multiple studies, none were found to have significant associations with recurrence across all studies. It was therefore not possible to quantify the extent to which potential risk factors contribute to the risk of recurrence. One reason for the inconsistent findings might be the relatively small numbers of participants with recurrent PTSD in many of the studies. We note also that most studies did not consider either subsequent trauma or treatment impact in their analysis of predictors of recurrence.

We did not find strong evidence of an association between PTSD recurrence and comorbid psychiatric conditions. Recurrence of other mental health disorders, such as anxiety, is reportedly associated with comorbid psychiatric conditions including major depression, alcohol and substance use disorders [ 74 ]. Additionally, comorbid disorders have been found to be associated with an ‘unfavourable long-term course’ of PTSD [ 18 ]. However, in a review of predictors of developing PTSD, Brewin et al. [ 75 ] found that while psychiatric history was associated with development of PTSD, it was not a strong risk factor – factors operating during or after the traumatic exposure had greater effects than the pre-trauma factors. Many studies in this review found no evidence of a relationship between PTSD recurrence and other mental health conditions; in those that did find a relationship, it was not always clear whether the other conditions pre-dated the recurrent PTSD episode or not. Overall, the most consistent evidence we found indicated that recurrence of PTSD was associated with greater stress and traumatic response at the time of the traumatic experience.

We did not find evidence to suggest that trauma type may affect recurrence. Many studies examined PTSD trajectories after a single traumatic event. Those that did include participants who had experienced various different types of trauma did not consider trauma type as a potential predictor of recurrence. Given the wide variations in methodology, it was not appropriate for us to compare recurrence rates for different trauma types within the review. Future research should include participants who have experienced different types of trauma and should consider trauma type as a potential predictor of PTSD trajectory.

Only one study assessed PTSD during the COVID-19 pandemic, with Solomon et al. [ 62 ] reporting that 16.7% of initially-recovered participants experienced recurrence during the pandemic. However, it is not clear how many of this cohort may also have experienced recurrence before the pandemic, and without being able to make that comparison, we cannot ascertain the extent to which recurrence was exacerbated by the pandemic. Additionally, the percentage (16.7%) is similar to recurrence rates in several other, non-COVID studies. Ideally, future studies will present data on PTSD recurrence rates for one cohort at regular intervals, including data collected during or after the COVID-19 pandemic, to ascertain whether the pandemic did affect recurrence rates.

In their review, Steinert et al. [ 18 ] identified older age, higher education, greater trauma severity, higher baseline symptoms, more physical/functional impairments, and poorer social support as predictors of ‘unfavourable’ long-term course of PTSD. These were identified as predictors due to being reported in at least two studies within their review. The current review did not find consistent evidence that age, education, trauma severity, baseline symptoms, impairments or social support predicted recurrence – although age was only considered in studies of young people. We found some evidence from treatment studies that fluoxetine reduced the risk of recurrence, as did participation in an intervention involving a combination of motivational interviewing, behavioural activation and pharmacotherapy. It is therefore difficult to make recommendations relevant to occupational health, as we had hoped to do. Managers of trauma-exposed employees who have developed PTSD may have questions around whether recovered individuals can go back to frontline work, or whether they risk experiencing a recurrence of PTSD. Our findings tentatively suggest that recurrence might be relatively rare (rates of recurrence ranged from 0.2 − 57% in full trauma-exposed samples, mean 13.1%; 4.9 − 49.6% in PTSD-only subgroups, mean 24.5%; and 5.8 − 50% for recovered subgroups, mean 25.4%) but clearer definitions and assessments of recurrence are needed to substantiate that claim. As we found no consistent evidence of predictors of recurrence, it was therefore not possible to identify which sub-groups of people might be more likely to have their PTSD recur. We did find evidence from two studies that recurrence was more prevalent in groups of PTSD patients treated with placebos compared to PTSD patients treated with fluoxetine, suggesting that medication appears at least somewhat effective in reducing the risk of recurrence. However, we found no studies looking at the impact of first-line treatments on relapse (i.e. trauma-focused cognitive behavioural therapy [ 76 ] or eye movement desensitisation and reprocessing [ 77 , 78 ]) which is a major gap in the literature. Whilst more, high-quality studies are carried out, employers should ensure that workers get evidence-based treatments and have an occupational mental health assessment on completion of potentially traumatic work to provide an expert judgement, given that we cannot identify any clear risk factors from the literature.

The key limitation of the literature on PTSD recurrence is that it is not always easy to differentiate between recurrence and symptom fluctuation, and it is also difficult to know what ‘recovery’ truly means. It is not clear how many of the so-called ‘recovered’ participants within the reviewed studies may have been close to clinical thresholds for PTSD at the assessment points. Rather than moving from distinct ‘recovered’ to ‘recurrent episodes’, it may be that individuals only experienced small fluctuations in PTSD symptoms, moving them above and below the symptom thresholds. Indeed, the authors of several of the included studies remarked on the difficulties in identifying PTSD trajectories. In Boe et al.’s [ 15 ] study, clinical interviews were conducted by two clinical psychologists who were trained and supervised by an experienced clinician and trauma researcher and even these experienced individuals had difficulties identifying recurrence of PTSD, with one case being recategorised from ‘full-blown PTSD reactivation’ to ‘sub-syndromal reactivation’ after discussion between the researchers. Markowitz et al. [ 52 ] pointed out that, as they defined relapse as ‘loss of response (to treatment) status’, relapse might reflect barely crossing that threshold: indeed, more in-depth analysis of their six ‘relapsers’ showed that all but one still showed some, albeit more modest, treatment benefit relative to their baseline PTSD severity.

Sakuma et al. [ 58 ] discussed their finding of a ‘fluctuating’ trajectory (and lack of a delayed-onset trajectory), differing from the typical four trajectories widely accepted within the PTSD literature. They suggested the difference may be due to variations in the duration of study periods and characteristics of the study samples. The majority of studies which produce the typical four trajectories are conducted over short periods between a few months and two years [ 9 ], compared to the longer (54-month) period of Sakuma et al.’s [ 58 ] study: the trajectory commonly identified as ‘delayed onset’ could really be a fluctuating trajectory if examined over a longer period. Or, it could reflect a gradual accumulation of symptoms resulting in a delayed presentation of PTSD, rather than delayed onset.

The time-points of assessments could also affect reported prevalence rates. For example, Sungur & Kaya [ 64 ] pointed out that some of their ‘recurrent’ cases would have been considered ‘recovered’ if the study period had been shorter or if participants had not been reassessed at the particular time-points chosen. They also noted that symptoms across the entire participant population seemed to be higher at particular times during the study (namely, at the anniversary of the event and at the time of a disappointing result of a court hearing for compensation), suggesting that the nature and course of PTSD might be influenced by particular events which might trigger unwanted memories of the traumatic event. In the current review, most studies assessed participants for at least a year, but not all: five [ 38 , 39 , 43 , 52 , 53 ] followed participants for less than a year. Additionally, two studies [ 44 , 47 ] reported assessing participants pre-treatment and four months post-treatment but it was not clear how long treatment lasted.

We suggest that PTSD recurrence may not have been adequately assessed in many of the included studies. For example, Chopra et al. [ 39 ] described how, in order to minimise respondent burden, assessors were expected to stop inquiring about PTSD symptoms if participants were unlikely to meet the criteria and if they answered no to particular questions on the assessment tool. This could mean that some individuals who did have recurrent episodes of PTSD were not identified as they did not complete the full measures. Additionally, we found that a number of studies had very vague definitions of recurrence, such as ‘increasing symptoms’, where it was unclear what exactly this meant. Others used hospitalisation as a proxy measure for recurrence, or simply asked participants whether they perceived their symptoms had been exacerbated and in one case used the investigator’s own judgements as a way of determining recurrence. It is therefore likely that some recurrent cases may have been missed while others who never truly ‘recovered’ at all may have been reported to have experienced recurrence. Overall, the vague and inconsistent ways of assessing recurrence mean it is currently impossible to ascertain true recurrence rates within existing literature.

It is also possible that recurrent trajectories of PTSD appear in studies which do not identify them as such. For example, in Andrews et al.’s [ 16 ] review, the authors note that some cases of ‘delayed-onset PTSD’ in veterans of relatively old age with long intervals to first onset may in fact have had episodes of PTSD soon after their traumatic experiences which were undisclosed or forgotten. In other words, some cases of supposedly ‘delayed-onset’ PTSD might actually be recurrent cases. Andrews et al. [ 16 ] also point out that many of the studies included in their review of delayed-onset PTSD did not assess whether respondents could have had onsets of PTSD and then remitted before the next assessment point – which could lead to both over- and under-estimates of delayed-onset rates of PTSD. Indeed, the studies included in our own review tended to focus only on the scores at the various time-points and did not explore participants’ perceptions of symptom fluctuations outside of the time-points set by the study.

Limitations

There are a number of limitations of the literature reviewed. Many did not collect data on whether participants had undergone any intervention or not, and those that did tended not to include this as a potential confounding variable. The majority of studies did not assess whether participants experienced additional potentially traumatic experiences between PTSD assessments. Many did not define the parameters of ‘recovery’ and ‘recurrence’ and it is not clear whether recurrent episodes identified were truly recurrent episodes or merely symptom fluctuations. Many did not collect data on whether or not participants received any treatment for PTSD between data collection time-points, and many of those which did ask participants whether they had received any treatment did not distinguish between types of treatment. It is therefore unclear if, and how many, participants in many studies received any evidence-based PTSD treatment or not. Additionally, the majority of studies did not collect data on the time period of any treatment received. Some studies had extremely long gaps (e.g., decades) between assessments which could mean that recurrences were missed.

There are also limitations of the review process itself. Firstly, the screening, data extraction and quality appraisal were carried out by one author. Although decisions about exclusion or inclusion were discussed with the second author, it would have been preferable to have multiple screeners. We limited the review to English-language studies only, meaning that important studies published in other languages would have been missed. We included only studies which identified ‘recurrent episodes’ (or equivalent terminology e.g. relapse, reactivation); studies which identified no recurrent trajectory were not reviewed. It may be that these studies did not include a sufficient number of assessments to pick up on recurrent episodes, but it may also be that no participants in these studies experienced recurrence and therefore the true prevalence of recurrence may be lower than this review suggests.

Conclusions and implications

The main conclusion that can be drawn from the current review is that, moving forward, better clarity and consensus regarding the definition and identification of recurrent PTSD are urgently needed. Berge et al. [ 22 ] suggest that consistent definitions of relapse-related terms, supported by empirical research, are required in order to make studies of PTSD trajectories more robust. The findings of this review support this suggestion. Experts in the field should agree on an appropriate definition of recurrence (i.e. symptom-based or threshold-based) and should agree how long an individual needs to be ‘better’ for in order to be considered recovered as well as how long an individual needs to experience symptoms for in order to be considered as having a recurrent episode. Recurrence is arguably better-defined for recurrent depressive disorder, with the ICD-11 stating that recurrence is characterised by a history of depressive episodes separated by at least several months without significant mood disturbance [ 79 ]. However, further clarity is still needed. How many months is ‘several’? What are ‘significant’ symptoms? Still, we suggest this might be a useful starting point for a working definition of recurrent PTSD: a history of episodes of PTSD separated by at least several (i.e., three) months without significant (i.e., meeting diagnostic criteria) PTSD symptoms . However, further research is necessary to clarify whether these parameters (i.e. three months as a time period, symptom thresholds as a diagnostic tool) are the most appropriate to use. Using consistent terminology within the literature would make it easier to researchers in the future to understand true prevalence rates of PTSD recurrence and to compare them across studies. Further research allowing for the identification of recurrent PTSD episodes is needed. We believe the gold standard for assessing PTSD and properly identifying its trajectories, including recurrent trajectories, would be using the Clinician Administered PTSD Scale (CAPS) [ 80 ], or other validated questionnaires, at multiple specific time points over a long period of time. Figure  2 summarises the findings of the review and the proposed next steps based on our findings.

Summary of review and suggested next steps

It is important to understand recurrence in order to take steps towards reducing the risk of PTSD recurring. However, due to the inconsistent findings relating to predictors of recurrence, it is difficult to draw conclusions about the best ways of preventing or minimising recurrence. We suggest that ensuring that people who develop PTSD are provided with timely, evidence-based treatments is a logical first step [ 68 ]. Second, awareness of ‘early warning sign’ symptoms and ‘triggers’ might be useful, as well as awareness of effective coping strategies and how to access support. That is, if people with PTSD are able to recognise when they are struggling more and acknowledge that they need to be proactive in ensuring symptoms do not develop into full-blown PTSD again, they may be able to draw on their coping skills or reach out for formal or informal support when a recurrent episode seems imminent and may be able to stave off the recurrent episode. We also suggest that reframing the re-emergence of symptoms in a more positive way might be useful: instead of feeling defeated that symptoms have recurred, people could remind themselves that they have recovered once and therefore know that they are capable of doing so again. Within organisational settings, it is also important to foster an environment in which people who have any mental health condition, including PTSD, feel confident that asking for help will not lead to stigmatisation or increase the likelihood of inappropriate job loss. It may also be helpful to incorporate relapse prevention, understanding ‘warning signs’ of recurrent episodes and positive reframing into PTSD treatment programmes.

Data availability

All data generated or analysed during this study are included in this published article.

Abbreviations

Clinician-Administered PTSD Scale

Military sexual trauma

National Institutes for Health

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Post-Traumatic Stress Disorder

Benjet C, Bromet E, Karam EG, Kessler RC, McLaughlin KA, Ruscio AM, et al. The epidemiology of traumatic event exposure worldwide: results from the World Mental Health Survey Consortium. Psychol Med. 2016;46(2):327–43.

Article   CAS   PubMed   Google Scholar  

Brooks SK, Greenberg N. Preventing and treating trauma-related mental health problems. In: Lax P, editor. Textbook of acute trauma care. Switzerland: Springer Cham; 2022. pp. 829–46.

Chapter   Google Scholar  

Bonanno GA. Loss, trauma, and human resilience: have we underestimated the human capacity to thrive after extremely aversive events? Am Psychol. 2004;59(1):20–8.

Article   PubMed   Google Scholar  

American Psychiatric Association. Diagnostic and statistical manual of mental disorders (fifth edition). ; 2013. https://doi.org/10.1176/appi.books.9780890425596 .

Messman-Moore TL, Cook NK. Posttraumatic stress disorder. In: H. S. Friedman, editor. Encyclopedia of mental health (second edition). San Diego: Academic Press; 2016. pp. 308 – 12.

Spottswood M, Davydow DS, Huang H. The prevalence of posttraumatic stress disorder in primary care: a systematic review. Harv Rev Psychiatry. 2017;25(4):159–69. https://doi.org/10.1097/HRP.0000000000000136 .

Article   PubMed   PubMed Central   Google Scholar  

Kapfhammer HP. Acute and long-term mental and physical sequelae in the aftermath of traumatic exposure - some remarks on the body keeps the score. Psychiatr Danub. 2018;30(3):254–72. https://doi.org/10.24869/psyd.2018.254 .

Peleg T, Shalev AY. Longitudinal studies of PTSD: overview of findings and methods. CNS Spectr. 2006;11(8):589–602. https://doi.org/10.1017/s109285290001364x .

Galatzer-Levy IR, Huang SH, Bonanno GA. Trajectories of resilience and dysfunction following potential trauma: a review and statistical evaluation. Clin Psychol Rev. 2018;63:41–55. https://doi.org/10.1016/j.cpr.2018.05.008 .

van de Schoot R, Sijbrandij M, Depaoli S, Winter SD, Olff M, van Loey NE. Bayesian PTSD-trajectory analysis with informed priors based on a systematic literature search and expert elicitation. Multivar Behav Res. 2018;53(2):267–91. https://doi.org/10.1080/00273171.2017.1412293 .

Article   Google Scholar  

Santiago PN, Ursano RJ, Gray CL, Pynoos RS, Spiegel D, Lewis-Fernandez R, et al. A systematic review of PTSD prevalence and trajectories in DSM-5 defined trauma exposed populations: intentional and non-intentional traumatic events. PLoS ONE. 2013;8(4):5. https://doi.org/10.1371/journal.pone.0059236 .

Article   CAS   Google Scholar  

Mota N, Bolton SL, Enns MW, Afifi TO, El-Gabalawy R, Sommer JL, et al. Course and predictors of posttraumatic stress disorder in the Canadian Armed forces: a nationally representative, 16-year follow-up study. Can J Psychiatry. 2021;66(11):982–95. https://doi.org/10.1177/0706743721989167 .

Magruder KM, Goldberg J, Forsberg CW, Friedman MJ, Litz BT, Vaccarino V, et al. Long-term trajectories of PTSD in Vietnam-era veterans: the course and consequences of PTSD in twins. J Trauma Stress. 2016;29(1):5–16. https://doi.org/10.1002/jts.22075 .

Karamustafalioglu OK, Zohar J, Guveli M, Gal G, Bakirn B, Fostick L, et al. Natural course of posttraumatic stress disorder: a 20-month prospective study of Turkish Earthquake survivors. J Clin Psychiatry. 2006;67(6):882–9. https://doi.org/10.4088/JCP.v67n0604 .

Boe HJ, Holgersen KH, Holen A. Reactivation of posttraumatic stress in male Disaster survivors: the role of residual symptoms. J Anxiety Disord. 2010;24(4):397–402doi. https://doi.org/10.1016/j.janxdis.2010.02.003 .

Andrews B, Brewin CR, Philpott R, Stewart L. Delayed-onset posttraumatic stress disorder: a systematic review of the evidence. Am J Psychiatry. 2007;164(9):1319–26. https://doi.org/10.1176/appi.ajp.2007.06091491 .

North CS, Oliver J. Analysis of the longitudinal course of PTSD in 716 survivors of 10 Disasters. Soc Psychiatry Psychiatr Epidemiol. 2013;48(8):1189–97. https://doi.org/10.1007/s00127-012-0639-x .

Steinert C, Hofmann M, Leichsenring F, Kruse J. The course of PTSD in naturalistic long-term studies: high variability of outcomes. A systematic review. Nord J Psychiatry. 2015;69(7):483–96. https://doi.org/10.3109/08039488.2015.1005023 .

Bonde JPE, Jensen JH, Smid GE, Flachs EM, Elklit A, Mors O, Videbech P. Time course of symptoms in posttraumatic stress disorder with delayed expression: a systematic review. Acta Psychiatr Scand. 2022;145(2):116–31. https://doi.org/10.1111/acps.13372 .

Morina N, Wicherts JM, Lobbrecht J, Priebe S. Remission from post-traumatic stress disorder in adults: a systematic review of long term outcome studies. Clin Psychol Rev. 2014;34(3):249–55. https://doi.org/10.1016/j.cpr.2014.03.002 .

Pavlacic JM, Buchanan EM, McCaslin SE, Schulenberg SE, Young JN. A systematic review of posttraumatic stress and resilience trajectories: identifying predictors for future treatment of veterans and service members. Prof Psychol Res Pr. 2022;3266–75. https://doi.org/10.1037/pro0000451 .

Berge EE, Hagen R, Halvorsen JO. PTSD relapse in veterans of Iraq and Afghanistan: a systematic review. Mil Psychol. 2020;32(4):300–12. https://doi.org/10.1080/08995605.2020.1754123 .

Article   PubMed Central   Google Scholar  

Batelaan NM, Bosman RC, Muntingh A, Scholten WD, Huijbregts KM, van Balkom AJLM. Risk of relapse after antidepressant discontinuation in anxiety disorders, obsessive-compulsive disorder, and post-traumatic stress disorder: systematic review and meta-analysis of relapse prevention trials. BMJ. 2017;358:j3927. https://doi.org/10.1136/bmj.j3927 .

Levy HC, O’Bryan EM, Tolin DF. A meta-analysis of relapse rates in cognitive-behavioural therapy for anxiety disorders. J Anxiety Disord. 2021;81:102407. https://doi.org/10.1016/j.janxdis.2021.102407 .

Nagarajan R, Krishnamoorthy Y, Basavarachar V, Dakshinamoorthy R. Prevalence of post-traumatic stress disorder among survivors of severe COVID-19 Infections: a systematic review and meta-analysis. J Affect Disord. 2022;299:52–9.

Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, Rubin GJ. The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. Lancet. 2020;395(10227):912–20.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bonsaksen T, Heir T, Schou-Bredal I, Ekeberg Ø, Skogstad L, Grimholt TK. Post-traumatic stress disorder and associated factors during the early stage of the COVID-19 pandemic in Norway. Int J Environ Res Public Health. 2020;17(24):9210.

Hori A, Sawano T, Ozaki A, Tsubokura M. Exacerbation of subthreshold PTSD symptoms in a Great East Japan Earthquake survivor in the context of the COVID-19 pandemic. Case Rep Psychiatry. 2021;2021:6699775. https://doi.org/10.1155/2021/6699775 .

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Bagias C, Sukumar N, Weldeselassie Y, Oyebode O, Saravanan P. Cord blood adipocytokines and body composition in early childhood: a systematic review and meta-analysis. Int J Environ Res Public Health. 2021;18(4):1897. https://doi.org/10.3390/ijerph18041897 .

Rosella L, Bowman C, Pach B, Morgan S, Fitzpatrick T, Goel V. The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT). Public Health. 2016;136:57–65.

An YY, Huang JL, Yeung ETF, Hou WK. Academic burnout and posttraumatic growth predict trajectories of posttraumatic stress disorder symptoms of adolescents following Yancheng Tornado in China. Int J Stress Manage. 2022;29(2):143–53. https://doi.org/10.1037/str0000240 .

Andersen SB, Karstoft KI, Bertelsen M, Madsen T. Latent trajectories of trauma symptoms and resilience: the 3-year longitudinal prospective USPER study of Danish veterans deployed in Afghanistan. J Clin Psychiatry. 2014;75(9):1001–8. https://doi.org/10.4088/JCP.13m08914 .

Ansell EB, Pinto A, Edelen MO, Markowitz JC, Sanislow CA, Yen S, et al. The association of personality disorders with the prospective 7-year course of anxiety disorders. Psychol Med. 2011;41(5):1019–28.

Armenta RF, Walter KH, Geronimo-Hara TR, Porter B, Stander VA, LeardMann CA, et al. Longitudinal trajectories of comorbid PTSD and depression symptoms among US service members and veterans. BMC Psychiatry. 2019;19(1):12. https://doi.org/10.1186/s12888-019-2375-1 .

Benítez CIP, Zlotnick C, Stout RI, Lou FJ, Dyck I, Weisberg R, Keller M. (2012). A 5-year longitudinal study of posttraumatic stress disorder in primary care patients. Psychopathol. 2012;45(5):286 – 93. https://doi.org/10.1159/000331595 .

Berntsen D, Johannessen KB, Thomsen YD, Bertelsen M, Hoyle RH, Rubin DC. Peace and War: trajectories of posttraumatic stress disorder symptoms before, during, and after military deployment in Afghanistan. Psychol Sci. 2012;23(12):1557–65. https://doi.org/10.1177/0956797612457389 .

Chopra MP, Zhang H, Kaiser AP, Moye JA, Llorente MD, Oslin DW, Spiro IA. PTSD is a chronic, fluctuating disorder affecting the mental quality of life in older adults. Am J Geriatr Psychiatry. 2014;22(1):86–97. https://doi.org/10.1016/j.jagp.2013.01.064 .

Davidson JRT, Connor KM, Hertzberg MA, Weisler RH, Wilson WH, Payne VM. Maintenance therapy with fluoxetine in posttraumatic stress disorder: a placebo-controlled discontinuation study. J Clin Pharmacol. 2005;25(2):166–9. https://doi.org/10.1097/01.jcp.0000155817.21467.6c .

DenVelde WO, Hovens JE, Aarts PGH, FreyWouters E, Falger PRJ, VanDuijn H, et al. Prevalence and course of posttraumatic stress disorder in Dutch veterans of the civilian resistance during World War II: an overview. Psychol Rep. 1996;78(2):519–29. https://doi.org/10.2466/pr0.1996.78.2.519 .

Fan F, Long K, Zhou Y, Zheng Y, Liu X. Longitudinal trajectories of post-traumatic stress disorder symptoms among adolescents after the Wenchuan Earthquake in China. Psychol Med. 2015;45(13):2885–96. https://doi.org/10.1017/s0033291715000884 .

Gonçalves V, Jayson G, Tarrier N. A longitudinal investigation of posttraumatic stress disorder in patients with Ovarian cancer. J Psychosom Res. 2011;70(5):422–31. https://doi.org/10.1016/j.jpsychores.2010.09.017 .

Gross GM, Smith N, Holliday R, Rozek DC, Hoff R, Harpaz-Rotem I. Racial disparities in clinical outcomes of Veterans affairs residential PTSD treatment between Black and White veterans. Psychiatr Serv. 2022;73(2):126–32. https://doi.org/10.1176/appi.ps.202000783 .

Hansen MB, Birkeland MS, Nissen A, Blix I, Solberg O, Heir T. Prevalence and course of symptom-defined PTSD in individuals directly or indirectly exposed to terror: a longitudinal study. Psychiatry. 2017;80(2):171–83. https://doi.org/10.1080/00332747.2016.1230983 .

Hepp U, Moergeli H, Buchi S, Bruchhaus-Steinert H, Kraemer B, Sensky T, et al. Post-traumatic stress disorder in serious accidental injury: 3-year follow-up study. Br J Psychiatry. 2008;192(5):376–83. https://doi.org/10.1192/bjp.bp.106.030569 .

Holliday R, Smith NB, Holder N, Gross GM, Monteith LL, Maguen S, et al. Comparing the effectiveness of VA residential PTSD treatment for veterans who do and do not report a history of MST: a national investigation. J Psychiatr Res. 2020;122:42. https://doi.org/10.1016/j.jpsychires.2019.12.012

Karstoft KI, Armour C, Andersen SB, Bertelsen M, Madsen T. Community integration after deployment to Afghanistan: a longitudinal investigation of Danish soldiers. Soc Psychiatry Psychiatr Epidemiol. 2015;50(4):653–60. https://doi.org/10.1007/s00127-014-0973-2 .

Liang YM, Cheng J, Zhou YY, Liu ZK. Trajectories of posttraumatic stress disorders among children after the Wenchuan Earthquake: a four-year longitudinal study. Eur J Psychotraumatology. 2019;10(1):11. https://doi.org/10.1080/20008198.2019.1586266 .

Liang YM, Zhou YY, Liu ZK. Consistencies and differences in posttraumatic stress disorder and depression trajectories from the Wenchuan Earthquake among children over a 4 year period. J Affect Disord. 2021;279:9–16. https://doi.org/10.1016/j.jad.2020.09.107 .

Madsen T, Karstoft K-I, Bertelsen M, Andersen SB. Postdeployment suicidal ideations and trajectories of posttraumatic stress disorder in Danish soldiers: a 3-year follow-up of the USPER study. J Clin Psychiatry. 2014;75(9):994–1000. https://doi.org/10.4088/JCP.13m08910 .

Markowitz JC, Choo T-H, Neria Y. Do acute benefits of interpersonal psychotherapy for posttraumatic stress disorder endure? Can J Psychiatry. 2018;63(1):37–43. https://doi.org/10.1177/0706743717720690 .

Martenyi F, Brown EB, Zhang H, Koke SC, Prakash A. Fluoxetine v placebo in prevention of relapse in post-traumatic stress disorder. Br J Psychiatry. 2002;181(4):315 – 20. doi:10/1192/bjp.181.4.315.

Murphy D, Smith KV. Treatment efficacy for veterans with posttraumatic stress disorder: latent class trajectories of treatment response and their predictors. J Trauma Stress. 2018;31(5):753–63. https://doi.org/10.1002/jts.22333 .

Osenbach JE, Lewis C, Rosenfeld B, Russo J, Ingraham LM, Peterson R, et al. Exploring the longitudinal trajectories of posttraumatic stress disorder in injured trauma survivors. Psychiatry. 2014;77(4):386–97. https://doi.org/10.1521/psyc.2014.77.4.386 .

Osofsky HJ, Weems CF, Hansel TC, Speier AH, Osofsky JD, Graham R, et al. Identifying trajectories of change to improve understanding of integrated health care outcomes on PTSD symptoms post Disaster. Fam Syst Health. 2017;35(2):155–66. https://doi.org/10.1037/fsh0000274 .

Perconte ST, Griger ML. Comparison of successful, unsuccessful, and relapsed Vietnam veterans treated for posttraumatic stress disorder. J Nerv Ment Dis. 1991;179(9):558–62.

Sakuma A, Ueda I, Shoji W, Tomita H, Matsuoka H, Matsumoto K. Trajectories for post-traumatic stress disorder symptoms among local Disaster recovery workers following the Great East Japan Earthquake: Group-based trajectory modeling. J Affect Dis. 2020;274:742-51. https://doi.org/10.1016/j.jad.2020.05.152 .

Solomon Z, Mikulincer M. Trajectories of PTSD: a 20-year longitudinal study. Am J Psychiatry. 2006;163(4):659–66. https://doi.org/10.1176/ajp.2006.163.4.659 .

Solomon Z, Garb R, Bleich A, Grupper D. Reactivation of combat-related posttraumatic stress disorder. Am J Psychiatry. 1987;144(1):51–5.

Solomon Z, Bachem R, Levin Y, Crompton L, Ginzburg K. Long-term trajectories of posttraumatic stress disorder: categorical versus continuous assessment. Psychiatry. 2018;81(4):376–90. https://doi.org/10.1080/00332747.2018.1485369 .

Solomon Z, Mikulincer M, Ohry A, Ginzburg K. Prior trauma, PTSD long-term trajectories, and risk for PTSD during the COVID-19 pandemic: a 29-year longitudinal study. J Psychiatr Res. 2021;141:140–5. https://doi.org/10.1016/j.jpsychires.2021.06.031 .

Sørensen HJ, Andersen SB, Karstoft KI, Madsen T. The influence of pre-deployment cognitive ability on post-traumatic stress disorder symptoms and trajectories: the Danish USPER follow-up study of Afghanistan veterans. J Affect Disord. 2016;196:148–53. https://doi.org/10.1016/j.jad.2016.02.037 .

Sungur M, Kaya B. The onset and longitudinal course of a man-made post-traumatic morbidity: survivors of the Sivas Disaster. Int J Psychiatry Clin Pract. 2001;5(3):195–202. https://doi.org/10.1080/136515001317021662 .

Zanarini MC, Horz S, Frankenburg FR, Weingeroff J, Reich DB, Fitzmaurice G. The 10-year course of PTSD in borderline patients and axis II comparison subjects. Acta Psychiatr Scand. 2011;124(5):349–56. https://doi.org/10.1111/j.1600-0447.2011.01717.x .

Zlotnick C, Warshaw M, Shea MT, Allsworth J, Pearlstein T, Keller MB. Chronicity in posttraumatic stress disorder (PTSD) and predictors of course of comorbid PTSD in patients with anxiety disorders. J Trauma Stress. 1999;12(1):89–100. https://doi.org/10.1023/A:1024746316245 .

Lai BS, Lewis R, Livings MS, La Greca AM, Esnard AM. Posttraumatic stress symptom trajectories among children after Disaster exposure: a review. J Trauma Stress. 2017;30(6):571–82. https://doi.org/10.1002/jts.22242 .

National Institute for Health and Care Excellence. Post-traumatic stress disorder. 2018. https://www.nice.org.uk/guidance/ng116

Campbell SB, Renshaw KD. Posttraumatic stress disorder and relationship functioning: a comprehensive review and organizational framework. Clin Psychol Rev. 2018;65:152–62. https://doi.org/10.1016/j.cpr.2018.08.003 .

Stergiopoulos E, Cimo A, Cheng C, Bonato S, Dewa CS. Interventions to improve work outcomes in work-related PTSD: a systematic review. BMC Public Health. 2011;11:838. https://doi.org/10.1186/1471-2458-11-838

Davis LL, Schein J, Cloutier M, Gagnon-Sanschagrin P, Maitland J, Urganus A, Guerin A, Lefebvre P, Houle CR. The economic burden of posttraumatic stress disorder in the United States from a societal perspective. J Clin Psychiatry. 2022;83(3):21m14116. https://doi.org/10.4088/JCP.21m14116

Murray A. Recurrence of post traumatic stress disorder. Nurs Older People. 2005;17(6):24–30.

Floyd M, Rice J, Black SR. Recurrence of posttraumatic stress disorder in late life: a cognitive aging perspective. J Clin Geropsychol. 2002;8(4):303–11. https://doi.org/10.1023/A:1019679307628 .

Bruce SE, Yonkers KA, Otto MW, Eisen JL, Weisberg RB, Pagano M, et al. Influence of psychiatric comorbidity on recovery and recurrence in generalized anxiety disorder, social phobia, and panic disorder: a 12-year prospective study. Am J Psychiatry. 2005;162(6):1179–87.

Brewin CR, Andrews B, Valentine JD. Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults. J Consult Clin Psychol. 2000;68(5):748–66.

Cohen JA, Mannarino AP, Deblinger E. Treating trauma and traumatic grief in children and adolescents. New York: Guildford; 2010.

Google Scholar  

Shapiro F. Efficacy of the eye movement desensitization procedure in the treatment of traumatic memories. J Trauma Stress. 1989;2(2):199–223.

Shapiro F. Eye movement desensitization and reprocessing (EMDR): evaluation of controlled PTSD research. J Behav Ther Exp Psychiatry. 1996;27(3):209–18.

World Health Organization. 6A71 Recurrent depressive disorder. In International statistical classification of diseases and related health problems (11th ed.). ; 2019. https://icd.who.int/browse11/l-m/en#/http%3A%2F%2Fid.who.int%2Ficd%2Fentity%2F1194756772 .

Weathers FW, Blake DD, Schnurr PP, Kaloupek DG, Marx BP, Keane TM. The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). [Assessment]. 2013. Available from www.ptsd.va.gov .

Download references

This study was funded by the National Institute for Health and Care Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between the UK Health Security Agency, King’s College London and the University of East Anglia. The views expressed are those of the author(s) and not necessarily those of the NIHR, UKHSA or the Department of Health and Social Care. For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising. The funders had no role in carrying out the review or preparing the manuscript for publication.

Author information

Authors and affiliations.

Department of Psychological Medicine, King’s College London, Weston Education Centre, SE5 9RJ, London, United Kingdom

Samantha K Brooks & Neil Greenberg

You can also search for this author in PubMed   Google Scholar

Contributions

NG and SKB conceptualised the review. SKB carried out searches, screening, data extraction and analysis. NG contributed to analysis. SKB wrote the first draft of the manuscript and NG edited the manuscript. Both authors reviewed the final manuscript.

Corresponding author

Correspondence to Samantha K Brooks .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

NG runs March on Stress Ltd which is a psychological health consultancy and carries out expert witness work which includes PTSD cases.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Brooks, S.K., Greenberg, N. Recurrence of post-traumatic stress disorder: systematic review of definitions, prevalence and predictors. BMC Psychiatry 24 , 37 (2024). https://doi.org/10.1186/s12888-023-05460-x

Download citation

Received : 04 September 2023

Accepted : 13 December 2023

Published : 09 January 2024

DOI : https://doi.org/10.1186/s12888-023-05460-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Post-traumatic stress disorder

BMC Psychiatry

ISSN: 1471-244X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Bots in Software Development: A Systematic Literature Review and Thematic Analysis

• Published: 24 January 2024
• Volume 49 , pages 712–734, ( 2023 )

Cite this article

• R. Moguel-Sánchez   ORCID: orcid.org/0000-0001-7735-7110 1 ,
• C. S. Sergio Martínez-Palacios   ORCID: orcid.org/0000-0001-9795-1408 1 ,
• J. O. Ocharán-Hernández   ORCID: orcid.org/0000-0002-2598-1445 1 ,
• X. Limón   ORCID: orcid.org/0000-0003-4654-636X 1 &
• A. J. Sánchez-García   ORCID: orcid.org/0000-0002-2917-2960 1  

98 Accesses

Explore all metrics

Modern Software Engineering thrives with innovative tools that aid developers in creating better software grounded on quality standards. Software bots are an emerging and exciting trend in this regard, supporting numerous software development activities. As an emerging trend, few studies describe and analyze different bots in software development. This research presents a systematic literature review covering the state of the art of applied and proposed bots for software development. Our study spans literature from 2003 to 2022, with 82 different bots applied in software development activities, covering 83 primary studies. We found four bot archetypes: chatbots which focus on direct communication with developers to aid them, analysis bots that display helpful information in different tasks, repair bots for resolving software defects, and development bots that combine aspects of other bot technologies to provide a service to the developer. The primary benefits of using bots are increasing software quality, providing useful information to developers, and saving time through the partial or total automation of development activities. However, drawbacks are reported, including limited effectiveness in task completion, high coupling to third-party technologies, and some prejudice from developers toward bots and their contributions. We discovered that including Bots in software development is a promising field of research in software engineering that has yet to be fully explored.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Nagaria, B. and Hall, T., How software developers mitigate their errors when developing code, IEEE Trans. Software Eng., 2022, vol. 48, no. 6.

Suta, P., Lan, X., Wu, B., Mongkolnam, P., and Chan, J., An overview of machine learning in chatbots, Int. J. Mech. Eng. Rob. Res., 2020, vol. 9, no. 4, pp. 502–510.

Google Scholar  

Rainey, S.K., Brown, B., and Kirk, D.B., Bots, natural language processing, and machine learning, Tax Exec., 2017, vol. 69, p. 39.

Lebeuf, C.R., A taxonomy of software bots: Towards a deeper understanding of software bot characteristics, Ph.D. Thesis, 2018.

Lebeuf, C., Zagalsky, A., Foucault, M., and Storey, M.-A., Defining and classifying software bots: a faceted taxonomy, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 1–6.

Erlenhov, L., de Oliveira Neto, F.G., Scandariato, R., and Leitner, P., Current and future bots in software development, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 7–11.

Orgeolet, L., Foulquier, N., Misery, L., Redou, P., Pers, J.-O., Devauchelle-Pensec, V., and Saraux, A., Can artificial intelligence replace manual search for systematic literature? Review on cutaneous manifestations in primary Sjogren’s syndrome, Rheumatology, 2020, vol. 59, no. 4, pp. 811–819.

Article   Google Scholar  

Ciupe, A., Meza, S., and Orza, B., Systematic assessment of interactive instructional technologies in higher engineering education, in Proc. Int. Conf. on Interactive Collaborative Learning, Springer, 2020, pp. 797–804.

Jung, H., Lee, J., and Park, C., Deriving design principles for educational chatbots from empirical studies on human-chatbot interaction, J. Digital Contents Soc., 2020, vol. 21, pp. 487–493.

Bhirud, N., Tataale, S., Randive, S., and Nahar, S., A literature review on chatbots in healthcare domain, Int. J. Sci. Technol. Res., 2019, vol. 8, no. 7, pp. 225–231.

Perez, J.Q., Daradoumis, T., and Puig, J.M.M., Rediscovering the use of chatbots in education: A systematic literature review, Comput. Appl. Eng. Educ., 2020, vol. 28, no. 6, pp. 1549–1565.

Syvanen, S. and Valentini, C., Conversational agents in on-line organization–stakeholder interactions: A state-of-the-art analysis and implications for further research, J. Commun. Manag., 2020, vol. 24, no. 4.

Borsci, S., Malizia, A., Schmettow, M., Van Der Velde, F., Tariverdiyeva, G., Balaji, D., and Chamberlain, A., The chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents, Pers. Ubiquitous Comput., 2022, vol. 26, no. 1, pp. 95–119.

Feldt, R., de Oliveira Neto, F.G., and Torkar, R., Ways of applying artificial intelligence in software engineering, Proc. 6th IEEE/ACM Int. Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE), Gothenburg, 2018, pp. 35–41.

Erlenhov, L., Neto, F.G.D.O., and Leitner, P., An empirical study of bots in software development: characteristics and challenges from a practitioner’s perspective, in Proc. 28th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering, Association for Computing Machinery, 2020, pp. 445–455.

Santhanam, S., Hecking, T., Schreiber, A., and Wagner, S., Bots in software engineering: A systematic mapping study, Peer J. Comput. Sci., 2022, vol. 8, p. e866.

Lebeuf, C., Zagalsky, A., Foucault, M., and Storey, M.-A., Defining and classifying software bots: A faceted taxonomy, Proc. IEEE/ACM 1st Int. Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 1–6.

Kitchenham, B.A., Budgen, D., and Brereton, P., Evidence-Based Software Engineering and Systematic Reviews, CRC Press, 2015, vol. 4.

Book   Google Scholar  

Zhang, H., Babar, M.A., and Tell, P., Identifying relevant studies in software engineering, Inf. Software Technol., 2011, vol. 53, no. 6, pp. 625–637. https://doi.org/10.1016/j.infsof.2010.12.010

Zenodo: Bots and their Uses in Software Development: A Systematic Mapping Study [Data Set]. https://doi.org/10.5281/zenodo.7872403

Wohlin, C., Guidelines for snowballing in systematic literature studies and a replication in software engineering, Proc. 18th Int. Conf. on Evaluation and Assessment in Software Engineering, London, 2014.

Dybå, T. and Dingsoyr, T., Empirical studies of agile software development: A systematic review, Inf. Software Technol., 2008, vol. 50, no. 9–10, pp. 833–859.

Cruzes, D. and Dybå, T., Recommended Steps for Thematic Synthesis in Software Engineering, Trondheim: Dept. of Computer and Information Science, 2011.

Matthies, C., Dobrigkeit, F., and Hesse, G., An additional set of (automated) eyes: Chatbots for agile retrospectives, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE) , Montreal, 2019, pp. 34–37.

Balachandran, V., Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation, Proc. 35th Int. Conf. on Software Engineering (ICSE), San Francisco, 2013, pp. 931–940.

McConnell, S., Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers, Addison-Wesley, 2004.

Download references

This work was supported by ongoing institutional funding. No additional grants to carry out or direct this particular research were obtained

Author information

Authors and affiliations.

School of Statistics and Informatics, Universidad Veracruzana, Veracruz, Xalapa, México

R. Moguel-Sánchez, C. S. Sergio Martínez-Palacios, J. O. Ocharán-Hernández, X. Limón & A. J. Sánchez-García

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to R. Moguel-Sánchez , C. S. Sergio Martínez-Palacios , J. O. Ocharán-Hernández , X. Limón or A. J. Sánchez-García .

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note..

Pleiades Publishing remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Moguel-Sánchez, R., Martínez-Palacios, C.S., Ocharán-Hernández, J.O. et al. Bots in Software Development: A Systematic Literature Review and Thematic Analysis. Program Comput Soft 49 , 712–734 (2023). https://doi.org/10.1134/S0361768823080145

Download citation

Received : 14 June 2023

Revised : 12 July 2023

Accepted : 02 August 2023

Published : 24 January 2024

Issue Date : December 2023

DOI : https://doi.org/10.1134/S0361768823080145

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• development bots
• software development
• thematic synthesis
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Korean J Anesthesiol
• v.71(2); 2018 Apr

Introduction to systematic review and meta-analysis

1 Department of Anesthesiology and Pain Medicine, Inje University Seoul Paik Hospital, Seoul, Korea

2 Department of Anesthesiology and Pain Medicine, Chung-Ang University College of Medicine, Seoul, Korea

Systematic reviews and meta-analyses present results by combining and analyzing data from different studies conducted on similar research topics. In recent years, systematic reviews and meta-analyses have been actively performed in various fields including anesthesiology. These research methods are powerful tools that can overcome the difficulties in performing large-scale randomized controlled trials. However, the inclusion of studies with any biases or improperly assessed quality of evidence in systematic reviews and meta-analyses could yield misleading results. Therefore, various guidelines have been suggested for conducting systematic reviews and meta-analyses to help standardize them and improve their quality. Nonetheless, accepting the conclusions of many studies without understanding the meta-analysis can be dangerous. Therefore, this article provides an easy introduction to clinicians on performing and understanding meta-analyses.

Introduction

A systematic review collects all possible studies related to a given topic and design, and reviews and analyzes their results [ 1 ]. During the systematic review process, the quality of studies is evaluated, and a statistical meta-analysis of the study results is conducted on the basis of their quality. A meta-analysis is a valid, objective, and scientific method of analyzing and combining different results. Usually, in order to obtain more reliable results, a meta-analysis is mainly conducted on randomized controlled trials (RCTs), which have a high level of evidence [ 2 ] ( Fig. 1 ). Since 1999, various papers have presented guidelines for reporting meta-analyses of RCTs. Following the Quality of Reporting of Meta-analyses (QUORUM) statement [ 3 ], and the appearance of registers such as Cochrane Library’s Methodology Register, a large number of systematic literature reviews have been registered. In 2009, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [ 4 ] was published, and it greatly helped standardize and improve the quality of systematic reviews and meta-analyses [ 5 ].

Levels of evidence.

In anesthesiology, the importance of systematic reviews and meta-analyses has been highlighted, and they provide diagnostic and therapeutic value to various areas, including not only perioperative management but also intensive care and outpatient anesthesia [6–13]. Systematic reviews and meta-analyses include various topics, such as comparing various treatments of postoperative nausea and vomiting [ 14 , 15 ], comparing general anesthesia and regional anesthesia [ 16 – 18 ], comparing airway maintenance devices [ 8 , 19 ], comparing various methods of postoperative pain control (e.g., patient-controlled analgesia pumps, nerve block, or analgesics) [ 20 – 23 ], comparing the precision of various monitoring instruments [ 7 ], and meta-analysis of dose-response in various drugs [ 12 ].

Thus, literature reviews and meta-analyses are being conducted in diverse medical fields, and the aim of highlighting their importance is to help better extract accurate, good quality data from the flood of data being produced. However, a lack of understanding about systematic reviews and meta-analyses can lead to incorrect outcomes being derived from the review and analysis processes. If readers indiscriminately accept the results of the many meta-analyses that are published, incorrect data may be obtained. Therefore, in this review, we aim to describe the contents and methods used in systematic reviews and meta-analyses in a way that is easy to understand for future authors and readers of systematic review and meta-analysis.

Study Planning

It is easy to confuse systematic reviews and meta-analyses. A systematic review is an objective, reproducible method to find answers to a certain research question, by collecting all available studies related to that question and reviewing and analyzing their results. A meta-analysis differs from a systematic review in that it uses statistical methods on estimates from two or more different studies to form a pooled estimate [ 1 ]. Following a systematic review, if it is not possible to form a pooled estimate, it can be published as is without progressing to a meta-analysis; however, if it is possible to form a pooled estimate from the extracted data, a meta-analysis can be attempted. Systematic reviews and meta-analyses usually proceed according to the flowchart presented in Fig. 2 . We explain each of the stages below.

Flowchart illustrating a systematic review.

Formulating research questions

A systematic review attempts to gather all available empirical research by using clearly defined, systematic methods to obtain answers to a specific question. A meta-analysis is the statistical process of analyzing and combining results from several similar studies. Here, the definition of the word “similar” is not made clear, but when selecting a topic for the meta-analysis, it is essential to ensure that the different studies present data that can be combined. If the studies contain data on the same topic that can be combined, a meta-analysis can even be performed using data from only two studies. However, study selection via a systematic review is a precondition for performing a meta-analysis, and it is important to clearly define the Population, Intervention, Comparison, Outcomes (PICO) parameters that are central to evidence-based research. In addition, selection of the research topic is based on logical evidence, and it is important to select a topic that is familiar to readers without clearly confirmed the evidence [ 24 ].

Protocols and registration

In systematic reviews, prior registration of a detailed research plan is very important. In order to make the research process transparent, primary/secondary outcomes and methods are set in advance, and in the event of changes to the method, other researchers and readers are informed when, how, and why. Many studies are registered with an organization like PROSPERO ( http://www.crd.york.ac.uk/PROSPERO/ ), and the registration number is recorded when reporting the study, in order to share the protocol at the time of planning.

Defining inclusion and exclusion criteria

Information is included on the study design, patient characteristics, publication status (published or unpublished), language used, and research period. If there is a discrepancy between the number of patients included in the study and the number of patients included in the analysis, this needs to be clearly explained while describing the patient characteristics, to avoid confusing the reader.

Literature search and study selection

In order to secure proper basis for evidence-based research, it is essential to perform a broad search that includes as many studies as possible that meet the inclusion and exclusion criteria. Typically, the three bibliographic databases Medline, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL) are used. In domestic studies, the Korean databases KoreaMed, KMBASE, and RISS4U may be included. Effort is required to identify not only published studies but also abstracts, ongoing studies, and studies awaiting publication. Among the studies retrieved in the search, the researchers remove duplicate studies, select studies that meet the inclusion/exclusion criteria based on the abstracts, and then make the final selection of studies based on their full text. In order to maintain transparency and objectivity throughout this process, study selection is conducted independently by at least two investigators. When there is a inconsistency in opinions, intervention is required via debate or by a third reviewer. The methods for this process also need to be planned in advance. It is essential to ensure the reproducibility of the literature selection process [ 25 ].

Quality of evidence

However, well planned the systematic review or meta-analysis is, if the quality of evidence in the studies is low, the quality of the meta-analysis decreases and incorrect results can be obtained [ 26 ]. Even when using randomized studies with a high quality of evidence, evaluating the quality of evidence precisely helps determine the strength of recommendations in the meta-analysis. One method of evaluating the quality of evidence in non-randomized studies is the Newcastle-Ottawa Scale, provided by the Ottawa Hospital Research Institute 1) . However, we are mostly focusing on meta-analyses that use randomized studies.

If the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) system ( http://www.gradeworkinggroup.org/ ) is used, the quality of evidence is evaluated on the basis of the study limitations, inaccuracies, incompleteness of outcome data, indirectness of evidence, and risk of publication bias, and this is used to determine the strength of recommendations [ 27 ]. As shown in Table 1 , the study limitations are evaluated using the “risk of bias” method proposed by Cochrane 2) . This method classifies bias in randomized studies as “low,” “high,” or “unclear” on the basis of the presence or absence of six processes (random sequence generation, allocation concealment, blinding participants or investigators, incomplete outcome data, selective reporting, and other biases) [ 28 ].

The Cochrane Collaboration’s Tool for Assessing the Risk of Bias [ 28 ]

Data extraction

Two different investigators extract data based on the objectives and form of the study; thereafter, the extracted data are reviewed. Since the size and format of each variable are different, the size and format of the outcomes are also different, and slight changes may be required when combining the data [ 29 ]. If there are differences in the size and format of the outcome variables that cause difficulties combining the data, such as the use of different evaluation instruments or different evaluation timepoints, the analysis may be limited to a systematic review. The investigators resolve differences of opinion by debate, and if they fail to reach a consensus, a third-reviewer is consulted.

Data Analysis

The aim of a meta-analysis is to derive a conclusion with increased power and accuracy than what could not be able to achieve in individual studies. Therefore, before analysis, it is crucial to evaluate the direction of effect, size of effect, homogeneity of effects among studies, and strength of evidence [ 30 ]. Thereafter, the data are reviewed qualitatively and quantitatively. If it is determined that the different research outcomes cannot be combined, all the results and characteristics of the individual studies are displayed in a table or in a descriptive form; this is referred to as a qualitative review. A meta-analysis is a quantitative review, in which the clinical effectiveness is evaluated by calculating the weighted pooled estimate for the interventions in at least two separate studies.

The pooled estimate is the outcome of the meta-analysis, and is typically explained using a forest plot ( Figs. 3 and ​ and4). 4 ). The black squares in the forest plot are the odds ratios (ORs) and 95% confidence intervals in each study. The area of the squares represents the weight reflected in the meta-analysis. The black diamond represents the OR and 95% confidence interval calculated across all the included studies. The bold vertical line represents a lack of therapeutic effect (OR = 1); if the confidence interval includes OR = 1, it means no significant difference was found between the treatment and control groups.

Forest plot analyzed by two different models using the same data. (A) Fixed-effect model. (B) Random-effect model. The figure depicts individual trials as filled squares with the relative sample size and the solid line as the 95% confidence interval of the difference. The diamond shape indicates the pooled estimate and uncertainty for the combined effect. The vertical line indicates the treatment group shows no effect (OR = 1). Moreover, if the confidence interval includes 1, then the result shows no evidence of difference between the treatment and control groups.

Forest plot representing homogeneous data.

Dichotomous variables and continuous variables

In data analysis, outcome variables can be considered broadly in terms of dichotomous variables and continuous variables. When combining data from continuous variables, the mean difference (MD) and standardized mean difference (SMD) are used ( Table 2 ).

Summary of Meta-analysis Methods Available in RevMan [ 28 ]

The MD is the absolute difference in mean values between the groups, and the SMD is the mean difference between groups divided by the standard deviation. When results are presented in the same units, the MD can be used, but when results are presented in different units, the SMD should be used. When the MD is used, the combined units must be shown. A value of “0” for the MD or SMD indicates that the effects of the new treatment method and the existing treatment method are the same. A value lower than “0” means the new treatment method is less effective than the existing method, and a value greater than “0” means the new treatment is more effective than the existing method.

When combining data for dichotomous variables, the OR, risk ratio (RR), or risk difference (RD) can be used. The RR and RD can be used for RCTs, quasi-experimental studies, or cohort studies, and the OR can be used for other case-control studies or cross-sectional studies. However, because the OR is difficult to interpret, using the RR and RD, if possible, is recommended. If the outcome variable is a dichotomous variable, it can be presented as the number needed to treat (NNT), which is the minimum number of patients who need to be treated in the intervention group, compared to the control group, for a given event to occur in at least one patient. Based on Table 3 , in an RCT, if x is the probability of the event occurring in the control group and y is the probability of the event occurring in the intervention group, then x = c/(c + d), y = a/(a + b), and the absolute risk reduction (ARR) = x − y. NNT can be obtained as the reciprocal, 1/ARR.

Calculation of the Number Needed to Treat in the Dichotomous table

Fixed-effect models and random-effect models

In order to analyze effect size, two types of models can be used: a fixed-effect model or a random-effect model. A fixed-effect model assumes that the effect of treatment is the same, and that variation between results in different studies is due to random error. Thus, a fixed-effect model can be used when the studies are considered to have the same design and methodology, or when the variability in results within a study is small, and the variance is thought to be due to random error. Three common methods are used for weighted estimation in a fixed-effect model: 1) inverse variance-weighted estimation 3) , 2) Mantel-Haenszel estimation 4) , and 3) Peto estimation 5) .

A random-effect model assumes heterogeneity between the studies being combined, and these models are used when the studies are assumed different, even if a heterogeneity test does not show a significant result. Unlike a fixed-effect model, a random-effect model assumes that the size of the effect of treatment differs among studies. Thus, differences in variation among studies are thought to be due to not only random error but also between-study variability in results. Therefore, weight does not decrease greatly for studies with a small number of patients. Among methods for weighted estimation in a random-effect model, the DerSimonian and Laird method 6) is mostly used for dichotomous variables, as the simplest method, while inverse variance-weighted estimation is used for continuous variables, as with fixed-effect models. These four methods are all used in Review Manager software (The Cochrane Collaboration, UK), and are described in a study by Deeks et al. [ 31 ] ( Table 2 ). However, when the number of studies included in the analysis is less than 10, the Hartung-Knapp-Sidik-Jonkman method 7) can better reduce the risk of type 1 error than does the DerSimonian and Laird method [ 32 ].

Fig. 3 shows the results of analyzing outcome data using a fixed-effect model (A) and a random-effect model (B). As shown in Fig. 3 , while the results from large studies are weighted more heavily in the fixed-effect model, studies are given relatively similar weights irrespective of study size in the random-effect model. Although identical data were being analyzed, as shown in Fig. 3 , the significant result in the fixed-effect model was no longer significant in the random-effect model. One representative example of the small study effect in a random-effect model is the meta-analysis by Li et al. [ 33 ]. In a large-scale study, intravenous injection of magnesium was unrelated to acute myocardial infarction, but in the random-effect model, which included numerous small studies, the small study effect resulted in an association being found between intravenous injection of magnesium and myocardial infarction. This small study effect can be controlled for by using a sensitivity analysis, which is performed to examine the contribution of each of the included studies to the final meta-analysis result. In particular, when heterogeneity is suspected in the study methods or results, by changing certain data or analytical methods, this method makes it possible to verify whether the changes affect the robustness of the results, and to examine the causes of such effects [ 34 ].

Heterogeneity

Homogeneity test is a method whether the degree of heterogeneity is greater than would be expected to occur naturally when the effect size calculated from several studies is higher than the sampling error. This makes it possible to test whether the effect size calculated from several studies is the same. Three types of homogeneity tests can be used: 1) forest plot, 2) Cochrane’s Q test (chi-squared), and 3) Higgins I 2 statistics. In the forest plot, as shown in Fig. 4 , greater overlap between the confidence intervals indicates greater homogeneity. For the Q statistic, when the P value of the chi-squared test, calculated from the forest plot in Fig. 4 , is less than 0.1, it is considered to show statistical heterogeneity and a random-effect can be used. Finally, I 2 can be used [ 35 ].

I 2 , calculated as shown above, returns a value between 0 and 100%. A value less than 25% is considered to show strong homogeneity, a value of 50% is average, and a value greater than 75% indicates strong heterogeneity.

Even when the data cannot be shown to be homogeneous, a fixed-effect model can be used, ignoring the heterogeneity, and all the study results can be presented individually, without combining them. However, in many cases, a random-effect model is applied, as described above, and a subgroup analysis or meta-regression analysis is performed to explain the heterogeneity. In a subgroup analysis, the data are divided into subgroups that are expected to be homogeneous, and these subgroups are analyzed. This needs to be planned in the predetermined protocol before starting the meta-analysis. A meta-regression analysis is similar to a normal regression analysis, except that the heterogeneity between studies is modeled. This process involves performing a regression analysis of the pooled estimate for covariance at the study level, and so it is usually not considered when the number of studies is less than 10. Here, univariate and multivariate regression analyses can both be considered.

Publication bias

Publication bias is the most common type of reporting bias in meta-analyses. This refers to the distortion of meta-analysis outcomes due to the higher likelihood of publication of statistically significant studies rather than non-significant studies. In order to test the presence or absence of publication bias, first, a funnel plot can be used ( Fig. 5 ). Studies are plotted on a scatter plot with effect size on the x-axis and precision or total sample size on the y-axis. If the points form an upside-down funnel shape, with a broad base that narrows towards the top of the plot, this indicates the absence of a publication bias ( Fig. 5A ) [ 29 , 36 ]. On the other hand, if the plot shows an asymmetric shape, with no points on one side of the graph, then publication bias can be suspected ( Fig. 5B ). Second, to test publication bias statistically, Begg and Mazumdar’s rank correlation test 8) [ 37 ] or Egger’s test 9) [ 29 ] can be used. If publication bias is detected, the trim-and-fill method 10) can be used to correct the bias [ 38 ]. Fig. 6 displays results that show publication bias in Egger’s test, which has then been corrected using the trim-and-fill method using Comprehensive Meta-Analysis software (Biostat, USA).

Funnel plot showing the effect size on the x-axis and sample size on the y-axis as a scatter plot. (A) Funnel plot without publication bias. The individual plots are broader at the bottom and narrower at the top. (B) Funnel plot with publication bias. The individual plots are located asymmetrically.

Funnel plot adjusted using the trim-and-fill method. White circles: comparisons included. Black circles: inputted comparisons using the trim-and-fill method. White diamond: pooled observed log risk ratio. Black diamond: pooled inputted log risk ratio.

Result Presentation

When reporting the results of a systematic review or meta-analysis, the analytical content and methods should be described in detail. First, a flowchart is displayed with the literature search and selection process according to the inclusion/exclusion criteria. Second, a table is shown with the characteristics of the included studies. A table should also be included with information related to the quality of evidence, such as GRADE ( Table 4 ). Third, the results of data analysis are shown in a forest plot and funnel plot. Fourth, if the results use dichotomous data, the NNT values can be reported, as described above.

The GRADE Evidence Quality for Each Outcome

N: number of studies, ROB: risk of bias, PON: postoperative nausea, POV: postoperative vomiting, PONV: postoperative nausea and vomiting, CI: confidence interval, RR: risk ratio, AR: absolute risk.

When Review Manager software (The Cochrane Collaboration, UK) is used for the analysis, two types of P values are given. The first is the P value from the z-test, which tests the null hypothesis that the intervention has no effect. The second P value is from the chi-squared test, which tests the null hypothesis for a lack of heterogeneity. The statistical result for the intervention effect, which is generally considered the most important result in meta-analyses, is the z-test P value.

A common mistake when reporting results is, given a z-test P value greater than 0.05, to say there was “no statistical significance” or “no difference.” When evaluating statistical significance in a meta-analysis, a P value lower than 0.05 can be explained as “a significant difference in the effects of the two treatment methods.” However, the P value may appear non-significant whether or not there is a difference between the two treatment methods. In such a situation, it is better to announce “there was no strong evidence for an effect,” and to present the P value and confidence intervals. Another common mistake is to think that a smaller P value is indicative of a more significant effect. In meta-analyses of large-scale studies, the P value is more greatly affected by the number of studies and patients included, rather than by the significance of the results; therefore, care should be taken when interpreting the results of a meta-analysis.

When performing a systematic literature review or meta-analysis, if the quality of studies is not properly evaluated or if proper methodology is not strictly applied, the results can be biased and the outcomes can be incorrect. However, when systematic reviews and meta-analyses are properly implemented, they can yield powerful results that could usually only be achieved using large-scale RCTs, which are difficult to perform in individual studies. As our understanding of evidence-based medicine increases and its importance is better appreciated, the number of systematic reviews and meta-analyses will keep increasing. However, indiscriminate acceptance of the results of all these meta-analyses can be dangerous, and hence, we recommend that their results be received critically on the basis of a more accurate understanding.

1) http://www.ohri.ca .

2) http://methods.cochrane.org/bias/assessing-risk-bias-included-studies .

3) The inverse variance-weighted estimation method is useful if the number of studies is small with large sample sizes.

4) The Mantel-Haenszel estimation method is useful if the number of studies is large with small sample sizes.

5) The Peto estimation method is useful if the event rate is low or one of the two groups shows zero incidence.

6) The most popular and simplest statistical method used in Review Manager and Comprehensive Meta-analysis software.

7) Alternative random-effect model meta-analysis that has more adequate error rates than does the common DerSimonian and Laird method, especially when the number of studies is small. However, even with the Hartung-Knapp-Sidik-Jonkman method, when there are less than five studies with very unequal sizes, extra caution is needed.

8) The Begg and Mazumdar rank correlation test uses the correlation between the ranks of effect sizes and the ranks of their variances [ 37 ].

9) The degree of funnel plot asymmetry as measured by the intercept from the regression of standard normal deviates against precision [ 29 ].

10) If there are more small studies on one side, we expect the suppression of studies on the other side. Trimming yields the adjusted effect size and reduces the variance of the effects by adding the original studies back into the analysis as a mirror image of each study.