case study on relevance

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What the Case Study Method Really Teaches

• Nitin Nohria

Seven meta-skills that stick even if the cases fade from memory.

It’s been 100 years since Harvard Business School began using the case study method. Beyond teaching specific subject matter, the case study method excels in instilling meta-skills in students. This article explains the importance of seven such skills: preparation, discernment, bias recognition, judgement, collaboration, curiosity, and self-confidence.

During my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

• Nitin Nohria is the George F. Baker Jr. Professor at Harvard Business School and the former dean of HBS.

Partner Center

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

A case study research paper examines a person, place, event, condition, phenomenon, or other type of subject of analysis in order to extrapolate  key themes and results that help predict future trends, illuminate previously hidden issues that can be applied to practice, and/or provide a means for understanding an important research problem with greater clarity. A case study research paper usually examines a single subject of analysis, but case study papers can also be designed as a comparative investigation that shows relationships between two or more subjects. The methods used to study a case can rest within a quantitative, qualitative, or mixed-method investigative paradigm.

Case Studies. Writing@CSU. Colorado State University; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010 ; “What is a Case Study?” In Swanborn, Peter G. Case Study Research: What, Why and How? London: SAGE, 2010.

How to Approach Writing a Case Study Research Paper

General information about how to choose a topic to investigate can be found under the " Choosing a Research Problem " tab in the Organizing Your Social Sciences Research Paper writing guide. Review this page because it may help you identify a subject of analysis that can be investigated using a case study design.

However, identifying a case to investigate involves more than choosing the research problem . A case study encompasses a problem contextualized around the application of in-depth analysis, interpretation, and discussion, often resulting in specific recommendations for action or for improving existing conditions. As Seawright and Gerring note, practical considerations such as time and access to information can influence case selection, but these issues should not be the sole factors used in describing the methodological justification for identifying a particular case to study. Given this, selecting a case includes considering the following:

• The case represents an unusual or atypical example of a research problem that requires more in-depth analysis? Cases often represent a topic that rests on the fringes of prior investigations because the case may provide new ways of understanding the research problem. For example, if the research problem is to identify strategies to improve policies that support girl's access to secondary education in predominantly Muslim nations, you could consider using Azerbaijan as a case study rather than selecting a more obvious nation in the Middle East. Doing so may reveal important new insights into recommending how governments in other predominantly Muslim nations can formulate policies that support improved access to education for girls.
• The case provides important insight or illuminate a previously hidden problem? In-depth analysis of a case can be based on the hypothesis that the case study will reveal trends or issues that have not been exposed in prior research or will reveal new and important implications for practice. For example, anecdotal evidence may suggest drug use among homeless veterans is related to their patterns of travel throughout the day. Assuming prior studies have not looked at individual travel choices as a way to study access to illicit drug use, a case study that observes a homeless veteran could reveal how issues of personal mobility choices facilitate regular access to illicit drugs. Note that it is important to conduct a thorough literature review to ensure that your assumption about the need to reveal new insights or previously hidden problems is valid and evidence-based.
• The case challenges and offers a counter-point to prevailing assumptions? Over time, research on any given topic can fall into a trap of developing assumptions based on outdated studies that are still applied to new or changing conditions or the idea that something should simply be accepted as "common sense," even though the issue has not been thoroughly tested in current practice. A case study analysis may offer an opportunity to gather evidence that challenges prevailing assumptions about a research problem and provide a new set of recommendations applied to practice that have not been tested previously. For example, perhaps there has been a long practice among scholars to apply a particular theory in explaining the relationship between two subjects of analysis. Your case could challenge this assumption by applying an innovative theoretical framework [perhaps borrowed from another discipline] to explore whether this approach offers new ways of understanding the research problem. Taking a contrarian stance is one of the most important ways that new knowledge and understanding develops from existing literature.
• The case provides an opportunity to pursue action leading to the resolution of a problem? Another way to think about choosing a case to study is to consider how the results from investigating a particular case may result in findings that reveal ways in which to resolve an existing or emerging problem. For example, studying the case of an unforeseen incident, such as a fatal accident at a railroad crossing, can reveal hidden issues that could be applied to preventative measures that contribute to reducing the chance of accidents in the future. In this example, a case study investigating the accident could lead to a better understanding of where to strategically locate additional signals at other railroad crossings so as to better warn drivers of an approaching train, particularly when visibility is hindered by heavy rain, fog, or at night.
• The case offers a new direction in future research? A case study can be used as a tool for an exploratory investigation that highlights the need for further research about the problem. A case can be used when there are few studies that help predict an outcome or that establish a clear understanding about how best to proceed in addressing a problem. For example, after conducting a thorough literature review [very important!], you discover that little research exists showing the ways in which women contribute to promoting water conservation in rural communities of east central Africa. A case study of how women contribute to saving water in a rural village of Uganda can lay the foundation for understanding the need for more thorough research that documents how women in their roles as cooks and family caregivers think about water as a valuable resource within their community. This example of a case study could also point to the need for scholars to build new theoretical frameworks around the topic [e.g., applying feminist theories of work and family to the issue of water conservation].

Eisenhardt, Kathleen M. “Building Theories from Case Study Research.” Academy of Management Review 14 (October 1989): 532-550; Emmel, Nick. Sampling and Choosing Cases in Qualitative Research: A Realist Approach . Thousand Oaks, CA: SAGE Publications, 2013; Gerring, John. “What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Seawright, Jason and John Gerring. "Case Selection Techniques in Case Study Research." Political Research Quarterly 61 (June 2008): 294-308.

Structure and Writing Style

The purpose of a paper in the social sciences designed around a case study is to thoroughly investigate a subject of analysis in order to reveal a new understanding about the research problem and, in so doing, contributing new knowledge to what is already known from previous studies. In applied social sciences disciplines [e.g., education, social work, public administration, etc.], case studies may also be used to reveal best practices, highlight key programs, or investigate interesting aspects of professional work.

In general, the structure of a case study research paper is not all that different from a standard college-level research paper. However, there are subtle differences you should be aware of. Here are the key elements to organizing and writing a case study research paper.

I.  Introduction

As with any research paper, your introduction should serve as a roadmap for your readers to ascertain the scope and purpose of your study . The introduction to a case study research paper, however, should not only describe the research problem and its significance, but you should also succinctly describe why the case is being used and how it relates to addressing the problem. The two elements should be linked. With this in mind, a good introduction answers these four questions:

• What is being studied? Describe the research problem and describe the subject of analysis [the case] you have chosen to address the problem. Explain how they are linked and what elements of the case will help to expand knowledge and understanding about the problem.
• Why is this topic important to investigate? Describe the significance of the research problem and state why a case study design and the subject of analysis that the paper is designed around is appropriate in addressing the problem.
• What did we know about this topic before I did this study? Provide background that helps lead the reader into the more in-depth literature review to follow. If applicable, summarize prior case study research applied to the research problem and why it fails to adequately address the problem. Describe why your case will be useful. If no prior case studies have been used to address the research problem, explain why you have selected this subject of analysis.
• How will this study advance new knowledge or new ways of understanding? Explain why your case study will be suitable in helping to expand knowledge and understanding about the research problem.

Each of these questions should be addressed in no more than a few paragraphs. Exceptions to this can be when you are addressing a complex research problem or subject of analysis that requires more in-depth background information.

II.  Literature Review

The literature review for a case study research paper is generally structured the same as it is for any college-level research paper. The difference, however, is that the literature review is focused on providing background information and  enabling historical interpretation of the subject of analysis in relation to the research problem the case is intended to address . This includes synthesizing studies that help to:

• Place relevant works in the context of their contribution to understanding the case study being investigated . This would involve summarizing studies that have used a similar subject of analysis to investigate the research problem. If there is literature using the same or a very similar case to study, you need to explain why duplicating past research is important [e.g., conditions have changed; prior studies were conducted long ago, etc.].
• Describe the relationship each work has to the others under consideration that informs the reader why this case is applicable . Your literature review should include a description of any works that support using the case to investigate the research problem and the underlying research questions.
• Identify new ways to interpret prior research using the case study . If applicable, review any research that has examined the research problem using a different research design. Explain how your use of a case study design may reveal new knowledge or a new perspective or that can redirect research in an important new direction.
• Resolve conflicts amongst seemingly contradictory previous studies . This refers to synthesizing any literature that points to unresolved issues of concern about the research problem and describing how the subject of analysis that forms the case study can help resolve these existing contradictions.
• Point the way in fulfilling a need for additional research . Your review should examine any literature that lays a foundation for understanding why your case study design and the subject of analysis around which you have designed your study may reveal a new way of approaching the research problem or offer a perspective that points to the need for additional research.
• Expose any gaps that exist in the literature that the case study could help to fill . Summarize any literature that not only shows how your subject of analysis contributes to understanding the research problem, but how your case contributes to a new way of understanding the problem that prior research has failed to do.
• Locate your own research within the context of existing literature [very important!] . Collectively, your literature review should always place your case study within the larger domain of prior research about the problem. The overarching purpose of reviewing pertinent literature in a case study paper is to demonstrate that you have thoroughly identified and synthesized prior studies in relation to explaining the relevance of the case in addressing the research problem.

III.  Method

In this section, you explain why you selected a particular case [i.e., subject of analysis] and the strategy you used to identify and ultimately decide that your case was appropriate in addressing the research problem. The way you describe the methods used varies depending on the type of subject of analysis that constitutes your case study.

If your subject of analysis is an incident or event . In the social and behavioral sciences, the event or incident that represents the case to be studied is usually bounded by time and place, with a clear beginning and end and with an identifiable location or position relative to its surroundings. The subject of analysis can be a rare or critical event or it can focus on a typical or regular event. The purpose of studying a rare event is to illuminate new ways of thinking about the broader research problem or to test a hypothesis. Critical incident case studies must describe the method by which you identified the event and explain the process by which you determined the validity of this case to inform broader perspectives about the research problem or to reveal new findings. However, the event does not have to be a rare or uniquely significant to support new thinking about the research problem or to challenge an existing hypothesis. For example, Walo, Bull, and Breen conducted a case study to identify and evaluate the direct and indirect economic benefits and costs of a local sports event in the City of Lismore, New South Wales, Australia. The purpose of their study was to provide new insights from measuring the impact of a typical local sports event that prior studies could not measure well because they focused on large "mega-events." Whether the event is rare or not, the methods section should include an explanation of the following characteristics of the event: a) when did it take place; b) what were the underlying circumstances leading to the event; and, c) what were the consequences of the event in relation to the research problem.

If your subject of analysis is a person. Explain why you selected this particular individual to be studied and describe what experiences they have had that provide an opportunity to advance new understandings about the research problem. Mention any background about this person which might help the reader understand the significance of their experiences that make them worthy of study. This includes describing the relationships this person has had with other people, institutions, and/or events that support using them as the subject for a case study research paper. It is particularly important to differentiate the person as the subject of analysis from others and to succinctly explain how the person relates to examining the research problem [e.g., why is one politician in a particular local election used to show an increase in voter turnout from any other candidate running in the election]. Note that these issues apply to a specific group of people used as a case study unit of analysis [e.g., a classroom of students].

If your subject of analysis is a place. In general, a case study that investigates a place suggests a subject of analysis that is unique or special in some way and that this uniqueness can be used to build new understanding or knowledge about the research problem. A case study of a place must not only describe its various attributes relevant to the research problem [e.g., physical, social, historical, cultural, economic, political], but you must state the method by which you determined that this place will illuminate new understandings about the research problem. It is also important to articulate why a particular place as the case for study is being used if similar places also exist [i.e., if you are studying patterns of homeless encampments of veterans in open spaces, explain why you are studying Echo Park in Los Angeles rather than Griffith Park?]. If applicable, describe what type of human activity involving this place makes it a good choice to study [e.g., prior research suggests Echo Park has more homeless veterans].

If your subject of analysis is a phenomenon. A phenomenon refers to a fact, occurrence, or circumstance that can be studied or observed but with the cause or explanation to be in question. In this sense, a phenomenon that forms your subject of analysis can encompass anything that can be observed or presumed to exist but is not fully understood. In the social and behavioral sciences, the case usually focuses on human interaction within a complex physical, social, economic, cultural, or political system. For example, the phenomenon could be the observation that many vehicles used by ISIS fighters are small trucks with English language advertisements on them. The research problem could be that ISIS fighters are difficult to combat because they are highly mobile. The research questions could be how and by what means are these vehicles used by ISIS being supplied to the militants and how might supply lines to these vehicles be cut off? How might knowing the suppliers of these trucks reveal larger networks of collaborators and financial support? A case study of a phenomenon most often encompasses an in-depth analysis of a cause and effect that is grounded in an interactive relationship between people and their environment in some way.

NOTE:   The choice of the case or set of cases to study cannot appear random. Evidence that supports the method by which you identified and chose your subject of analysis should clearly support investigation of the research problem and linked to key findings from your literature review. Be sure to cite any studies that helped you determine that the case you chose was appropriate for examining the problem.

IV.  Discussion

The main elements of your discussion section are generally the same as any research paper, but centered around interpreting and drawing conclusions about the key findings from your analysis of the case study. Note that a general social sciences research paper may contain a separate section to report findings. However, in a paper designed around a case study, it is common to combine a description of the results with the discussion about their implications. The objectives of your discussion section should include the following:

Reiterate the Research Problem/State the Major Findings Briefly reiterate the research problem you are investigating and explain why the subject of analysis around which you designed the case study were used. You should then describe the findings revealed from your study of the case using direct, declarative, and succinct proclamation of the study results. Highlight any findings that were unexpected or especially profound.

Explain the Meaning of the Findings and Why They are Important Systematically explain the meaning of your case study findings and why you believe they are important. Begin this part of the section by repeating what you consider to be your most important or surprising finding first, then systematically review each finding. Be sure to thoroughly extrapolate what your analysis of the case can tell the reader about situations or conditions beyond the actual case that was studied while, at the same time, being careful not to misconstrue or conflate a finding that undermines the external validity of your conclusions.

Relate the Findings to Similar Studies No study in the social sciences is so novel or possesses such a restricted focus that it has absolutely no relation to previously published research. The discussion section should relate your case study results to those found in other studies, particularly if questions raised from prior studies served as the motivation for choosing your subject of analysis. This is important because comparing and contrasting the findings of other studies helps support the overall importance of your results and it highlights how and in what ways your case study design and the subject of analysis differs from prior research about the topic.

Consider Alternative Explanations of the Findings Remember that the purpose of social science research is to discover and not to prove. When writing the discussion section, you should carefully consider all possible explanations revealed by the case study results, rather than just those that fit your hypothesis or prior assumptions and biases. Be alert to what the in-depth analysis of the case may reveal about the research problem, including offering a contrarian perspective to what scholars have stated in prior research if that is how the findings can be interpreted from your case.

Acknowledge the Study's Limitations You can state the study's limitations in the conclusion section of your paper but describing the limitations of your subject of analysis in the discussion section provides an opportunity to identify the limitations and explain why they are not significant. This part of the discussion section should also note any unanswered questions or issues your case study could not address. More detailed information about how to document any limitations to your research can be found here .

Suggest Areas for Further Research Although your case study may offer important insights about the research problem, there are likely additional questions related to the problem that remain unanswered or findings that unexpectedly revealed themselves as a result of your in-depth analysis of the case. Be sure that the recommendations for further research are linked to the research problem and that you explain why your recommendations are valid in other contexts and based on the original assumptions of your study.

V.  Conclusion

As with any research paper, you should summarize your conclusion in clear, simple language; emphasize how the findings from your case study differs from or supports prior research and why. Do not simply reiterate the discussion section. Provide a synthesis of key findings presented in the paper to show how these converge to address the research problem. If you haven't already done so in the discussion section, be sure to document the limitations of your case study and any need for further research.

The function of your paper's conclusion is to: 1) reiterate the main argument supported by the findings from your case study; 2) state clearly the context, background, and necessity of pursuing the research problem using a case study design in relation to an issue, controversy, or a gap found from reviewing the literature; and, 3) provide a place to persuasively and succinctly restate the significance of your research problem, given that the reader has now been presented with in-depth information about the topic.

Consider the following points to help ensure your conclusion is appropriate:

• If the argument or purpose of your paper is complex, you may need to summarize these points for your reader.
• If prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the conclusion of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration of the case study's findings that returns the topic to the context provided by the introduction or within a new context that emerges from your case study findings.

Note that, depending on the discipline you are writing in or the preferences of your professor, the concluding paragraph may contain your final reflections on the evidence presented as it applies to practice or on the essay's central research problem. However, the nature of being introspective about the subject of analysis you have investigated will depend on whether you are explicitly asked to express your observations in this way.

Problems to Avoid

Overgeneralization One of the goals of a case study is to lay a foundation for understanding broader trends and issues applied to similar circumstances. However, be careful when drawing conclusions from your case study. They must be evidence-based and grounded in the results of the study; otherwise, it is merely speculation. Looking at a prior example, it would be incorrect to state that a factor in improving girls access to education in Azerbaijan and the policy implications this may have for improving access in other Muslim nations is due to girls access to social media if there is no documentary evidence from your case study to indicate this. There may be anecdotal evidence that retention rates were better for girls who were engaged with social media, but this observation would only point to the need for further research and would not be a definitive finding if this was not a part of your original research agenda.

Failure to Document Limitations No case is going to reveal all that needs to be understood about a research problem. Therefore, just as you have to clearly state the limitations of a general research study , you must describe the specific limitations inherent in the subject of analysis. For example, the case of studying how women conceptualize the need for water conservation in a village in Uganda could have limited application in other cultural contexts or in areas where fresh water from rivers or lakes is plentiful and, therefore, conservation is understood more in terms of managing access rather than preserving access to a scarce resource.

Failure to Extrapolate All Possible Implications Just as you don't want to over-generalize from your case study findings, you also have to be thorough in the consideration of all possible outcomes or recommendations derived from your findings. If you do not, your reader may question the validity of your analysis, particularly if you failed to document an obvious outcome from your case study research. For example, in the case of studying the accident at the railroad crossing to evaluate where and what types of warning signals should be located, you failed to take into consideration speed limit signage as well as warning signals. When designing your case study, be sure you have thoroughly addressed all aspects of the problem and do not leave gaps in your analysis that leave the reader questioning the results.

Case Studies. Writing@CSU. Colorado State University; Gerring, John. Case Study Research: Principles and Practices . New York: Cambridge University Press, 2007; Merriam, Sharan B. Qualitative Research and Case Study Applications in Education . Rev. ed. San Francisco, CA: Jossey-Bass, 1998; Miller, Lisa L. “The Use of Case Studies in Law and Social Science Research.” Annual Review of Law and Social Science 14 (2018): TBD; Mills, Albert J., Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Putney, LeAnn Grogan. "Case Study." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE Publications, 2010), pp. 116-120; Simons, Helen. Case Study Research in Practice . London: SAGE Publications, 2009;  Kratochwill,  Thomas R. and Joel R. Levin, editors. Single-Case Research Design and Analysis: New Development for Psychology and Education .  Hilldsale, NJ: Lawrence Erlbaum Associates, 1992; Swanborn, Peter G. Case Study Research: What, Why and How? London : SAGE, 2010; Yin, Robert K. Case Study Research: Design and Methods . 6th edition. Los Angeles, CA, SAGE Publications, 2014; Walo, Maree, Adrian Bull, and Helen Breen. “Achieving Economic Benefits at Local Events: A Case Study of a Local Sports Event.” Festival Management and Event Tourism 4 (1996): 95-106.

Writing Tip

At Least Five Misconceptions about Case Study Research

Social science case studies are often perceived as limited in their ability to create new knowledge because they are not randomly selected and findings cannot be generalized to larger populations. Flyvbjerg examines five misunderstandings about case study research and systematically "corrects" each one. To quote, these are:

Misunderstanding 1 :  General, theoretical [context-independent] knowledge is more valuable than concrete, practical [context-dependent] knowledge. Misunderstanding 2 :  One cannot generalize on the basis of an individual case; therefore, the case study cannot contribute to scientific development. Misunderstanding 3 :  The case study is most useful for generating hypotheses; that is, in the first stage of a total research process, whereas other methods are more suitable for hypotheses testing and theory building. Misunderstanding 4 :  The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions. Misunderstanding 5 :  It is often difficult to summarize and develop general propositions and theories on the basis of specific case studies [p. 221].

While writing your paper, think introspectively about how you addressed these misconceptions because to do so can help you strengthen the validity and reliability of your research by clarifying issues of case selection, the testing and challenging of existing assumptions, the interpretation of key findings, and the summation of case outcomes. Think of a case study research paper as a complete, in-depth narrative about the specific properties and key characteristics of your subject of analysis applied to the research problem.

Flyvbjerg, Bent. “Five Misunderstandings About Case-Study Research.” Qualitative Inquiry 12 (April 2006): 219-245.

• << Previous: Writing a Case Analysis Paper
• Next: Writing a Field Report >>
• Last Updated: Mar 6, 2024 1:00 PM
• URL: https://libguides.usc.edu/writingguide/assignments

• Open access
• Published: 27 June 2011

The case study approach

• Sarah Crowe 1 ,
• Kathrin Cresswell 2 ,
• Ann Robertson 2 ,
• Guro Huby 3 ,
• Anthony Avery 1 &
• Aziz Sheikh 2  

BMC Medical Research Methodology volume  11 , Article number:  100 ( 2011 ) Cite this article

770k Accesses

1035 Citations

37 Altmetric

Metrics details

The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.

Peer Review reports

Introduction

The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.

The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.

This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables 1 , 2 , 3 and 4 ) and those of others to illustrate our discussion[ 3 – 7 ].

What is a case study?

A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.

Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.

These are however not necessarily mutually exclusive categories. In the first of our examples (Table 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables 2 , 3 and 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 – 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].

What are case studies used for?

According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables 2 and 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.

Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].

How are case studies conducted?

Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.

Defining the case

Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].

For example, in our evaluation of the introduction of electronic health records in English hospitals (Table 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.

Selecting the case(s)

The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.

For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.

In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.

The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.

It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.

In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.

Collecting the data

In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 – 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table 2 )[ 4 ].

Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.

In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.

Analysing, interpreting and reporting case studies

Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.

The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table 4 )[ 6 ].

Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.

When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].

What are the potential pitfalls and how can these be avoided?

The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.

Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table 8 )[ 8 , 18 – 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table 9 )[ 8 ].

Conclusions

The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.

Yin RK: Case study research, design and method. 2009, London: Sage Publications Ltd., 4

Google Scholar  

Keen J, Packwood T: Qualitative research; case study evaluation. BMJ. 1995, 311: 444-446.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J, et al: Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009, 6 (10): 1-11.

Article   Google Scholar  

Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, et al: The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO). 2008, [ http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf ]

Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al: Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010, 41: c4564-

Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P, the Patient Safety Education Study Group: Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010, 15: 4-10. 10.1258/jhsrp.2009.009052.

Article   PubMed   Google Scholar  

van Harten WH, Casparie TF, Fisscher OA: The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002, 60 (1): 17-37. 10.1016/S0168-8510(01)00187-7.

Stake RE: The art of case study research. 1995, London: Sage Publications Ltd.

Sheikh A, Smeeth L, Ashcroft R: Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002, 52 (482): 746-51.

PubMed   PubMed Central   Google Scholar  

King G, Keohane R, Verba S: Designing Social Inquiry. 1996, Princeton: Princeton University Press

Doolin B: Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998, 13: 301-311. 10.1057/jit.1998.8.

George AL, Bennett A: Case studies and theory development in the social sciences. 2005, Cambridge, MA: MIT Press

Eccles M, the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implementation Science. 2006, 1: 1-8. 10.1186/1748-5908-1-1.

Article   PubMed Central   Google Scholar  

Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A: Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005, 365 (9456): 312-7.

Sheikh A, Panesar SS, Lasserson T, Netuveli G: Recruitment of ethnic minorities to asthma studies. Thorax. 2004, 59 (7): 634-

CAS   PubMed   PubMed Central   Google Scholar  

Hellström I, Nolan M, Lundh U: 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005, 4: 7-22. 10.1177/1471301205049188.

Som CV: Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005, 18: 463-477. 10.1108/09513550510608903.

Lincoln Y, Guba E: Naturalistic inquiry. 1985, Newbury Park: Sage Publications

Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2001, 322: 1115-1117. 10.1136/bmj.322.7294.1115.

Mays N, Pope C: Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.

Mason J: Qualitative researching. 2002, London: Sage

Brazier A, Cooke K, Moravan V: Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008, 7: 5-17. 10.1177/1534735407313395.

Miles MB, Huberman M: Qualitative data analysis: an expanded sourcebook. 1994, CA: Sage Publications Inc., 2

Pope C, Ziebland S, Mays N: Analysing qualitative data. Qualitative research in health care. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Cresswell KM, Worth A, Sheikh A: Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010, 10 (1): 67-10.1186/1472-6947-10-67.

Article   PubMed   PubMed Central   Google Scholar  

Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358: 483-488. 10.1016/S0140-6736(01)05627-6.

Article   CAS   PubMed   Google Scholar  

Yin R: Case study research: design and methods. 1994, Thousand Oaks, CA: Sage Publishing, 2

Yin R: Enhancing the quality of case studies in health services research. Health Serv Res. 1999, 34: 1209-1224.

Green J, Thorogood N: Qualitative methods for health research. 2009, Los Angeles: Sage, 2

Howcroft D, Trauth E: Handbook of Critical Information Systems Research, Theory and Application. 2005, Cheltenham, UK: Northampton, MA, USA: Edward Elgar

Book   Google Scholar  

Blakie N: Approaches to Social Enquiry. 1993, Cambridge: Polity Press

Doolin B: Power and resistance in the implementation of a medical management information system. Info Systems J. 2004, 14: 343-362. 10.1111/j.1365-2575.2004.00176.x.

Bloomfield BP, Best A: Management consultants: systems development, power and the translation of problems. Sociological Review. 1992, 40: 533-560.

Shanks G, Parr A: Positivist, single case study research in information systems: A critical analysis. Proceedings of the European Conference on Information Systems. 2003, Naples

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/11/100/prepub

Download references

Acknowledgements

We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.

Author information

Authors and affiliations.

Division of Primary Care, The University of Nottingham, Nottingham, UK

Sarah Crowe & Anthony Avery

Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK

Kathrin Cresswell, Ann Robertson & Aziz Sheikh

School of Health in Social Science, The University of Edinburgh, Edinburgh, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sarah Crowe .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Crowe, S., Cresswell, K., Robertson, A. et al. The case study approach. BMC Med Res Methodol 11 , 100 (2011). https://doi.org/10.1186/1471-2288-11-100

Download citation

Received : 29 November 2010

Accepted : 27 June 2011

Published : 27 June 2011

DOI : https://doi.org/10.1186/1471-2288-11-100

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Case Study Approach
• Electronic Health Record System
• Case Study Design
• Case Study Site
• Case Study Report

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Relevance and Refinements of Case Studies

Cite this chapter.

• Joachim Blatter 3 &
• Markus Haverland 4  

Part of the book series: Research Methods Series ((REMES))

2453 Accesses

1 Citations

Case studies are an interesting phenomenon in the social sciences. On the one hand, they have played a pivotal role in theory development and are still popular in almost all fields of the social sciences, with the notable exception of economics. On the other hand, they have been treated by most methodologists with skepticism and disdain. Many classic works in the social sciences illustrate the relevance — even prevalence — of case study research for most of the twentieth century. Developments in ontological reasoning, theory building, and epistemology, together with the sophistication of statistical techniques, seemed to reduce the appeal of the case study approach in the last decades of the twentieth century and led to the rise of large-N studies. Nevertheless, in recent years, we have witnessed a resurgent interest in case study research, accompanied by intensive methodological reflection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Author information

Authors and affiliations.

Department of Humanities and Social Sciences, University of Lucerne, Switzerland

Joachim Blatter ( Professor of Political Science )

Department of Public Administration, Erasmus University Rotterdam, The Netherlands

Markus Haverland ( Professor of Political Science )

You can also search for this author in PubMed   Google Scholar

Copyright information

© 2012 Joachim Blatter and Markus Haverland

About this chapter

Blatter, J., Haverland, M. (2012). Relevance and Refinements of Case Studies. In: Designing Case Studies. Research Methods Series. Palgrave Macmillan, London. https://doi.org/10.1057/9781137016669_1

Download citation

DOI : https://doi.org/10.1057/9781137016669_1

Publisher Name : Palgrave Macmillan, London

Print ISBN : 978-1-349-32085-1

Online ISBN : 978-1-137-01666-9

eBook Packages : Palgrave Political & Intern. Studies Collection Political Science and International Studies (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 21, Issue 1
• What is a case study?
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Roberta Heale 1 ,
• Alison Twycross 2
• 1 School of Nursing , Laurentian University , Sudbury , Ontario , Canada
• 2 School of Health and Social Care , London South Bank University , London , UK
• Correspondence to Dr Roberta Heale, School of Nursing, Laurentian University, Sudbury, ON P3E2C6, Canada; rheale{at}laurentian.ca

https://doi.org/10.1136/eb-2017-102845

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is it?

Case study is a research methodology, typically seen in social and life sciences. There is no one definition of case study research. 1 However, very simply… ‘a case study can be defined as an intensive study about a person, a group of people or a unit, which is aimed to generalize over several units’. 1 A case study has also been described as an intensive, systematic investigation of a single individual, group, community or some other unit in which the researcher examines in-depth data relating to several variables. 2

Often there are several similar cases to consider such as educational or social service programmes that are delivered from a number of locations. Although similar, they are complex and have unique features. In these circumstances, the evaluation of several, similar cases will provide a better answer to a research question than if only one case is examined, hence the multiple-case study. Stake asserts that the cases are grouped and viewed as one entity, called the quintain . 6  ‘We study what is similar and different about the cases to understand the quintain better’. 6

The steps when using case study methodology are the same as for other types of research. 6 The first step is defining the single case or identifying a group of similar cases that can then be incorporated into a multiple-case study. A search to determine what is known about the case(s) is typically conducted. This may include a review of the literature, grey literature, media, reports and more, which serves to establish a basic understanding of the cases and informs the development of research questions. Data in case studies are often, but not exclusively, qualitative in nature. In multiple-case studies, analysis within cases and across cases is conducted. Themes arise from the analyses and assertions about the cases as a whole, or the quintain, emerge. 6

Benefits and limitations of case studies

If a researcher wants to study a specific phenomenon arising from a particular entity, then a single-case study is warranted and will allow for a in-depth understanding of the single phenomenon and, as discussed above, would involve collecting several different types of data. This is illustrated in example 1 below.

Using a multiple-case research study allows for a more in-depth understanding of the cases as a unit, through comparison of similarities and differences of the individual cases embedded within the quintain. Evidence arising from multiple-case studies is often stronger and more reliable than from single-case research. Multiple-case studies allow for more comprehensive exploration of research questions and theory development. 6

Despite the advantages of case studies, there are limitations. The sheer volume of data is difficult to organise and data analysis and integration strategies need to be carefully thought through. There is also sometimes a temptation to veer away from the research focus. 2 Reporting of findings from multiple-case research studies is also challenging at times, 1 particularly in relation to the word limits for some journal papers.

Examples of case studies

Example 1: nurses’ paediatric pain management practices.

One of the authors of this paper (AT) has used a case study approach to explore nurses’ paediatric pain management practices. This involved collecting several datasets:

Observational data to gain a picture about actual pain management practices.

Questionnaire data about nurses’ knowledge about paediatric pain management practices and how well they felt they managed pain in children.

Questionnaire data about how critical nurses perceived pain management tasks to be.

These datasets were analysed separately and then compared 7–9 and demonstrated that nurses’ level of theoretical did not impact on the quality of their pain management practices. 7 Nor did individual nurse’s perceptions of how critical a task was effect the likelihood of them carrying out this task in practice. 8 There was also a difference in self-reported and observed practices 9 ; actual (observed) practices did not confirm to best practice guidelines, whereas self-reported practices tended to.

Example 2: quality of care for complex patients at Nurse Practitioner-Led Clinics (NPLCs)

The other author of this paper (RH) has conducted a multiple-case study to determine the quality of care for patients with complex clinical presentations in NPLCs in Ontario, Canada. 10 Five NPLCs served as individual cases that, together, represented the quatrain. Three types of data were collected including:

Review of documentation related to the NPLC model (media, annual reports, research articles, grey literature and regulatory legislation).

Interviews with nurse practitioners (NPs) practising at the five NPLCs to determine their perceptions of the impact of the NPLC model on the quality of care provided to patients with multimorbidity.

Chart audits conducted at the five NPLCs to determine the extent to which evidence-based guidelines were followed for patients with diabetes and at least one other chronic condition.

The three sources of data collected from the five NPLCs were analysed and themes arose related to the quality of care for complex patients at NPLCs. The multiple-case study confirmed that nurse practitioners are the primary care providers at the NPLCs, and this positively impacts the quality of care for patients with multimorbidity. Healthcare policy, such as lack of an increase in salary for NPs for 10 years, has resulted in issues in recruitment and retention of NPs at NPLCs. This, along with insufficient resources in the communities where NPLCs are located and high patient vulnerability at NPLCs, have a negative impact on the quality of care. 10

These examples illustrate how collecting data about a single case or multiple cases helps us to better understand the phenomenon in question. Case study methodology serves to provide a framework for evaluation and analysis of complex issues. It shines a light on the holistic nature of nursing practice and offers a perspective that informs improved patient care.

• Gustafsson J
• Calanzaro M
• Sandelowski M

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• AI Essentials for Business
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

5 Benefits of Learning Through the Case Study Method

• 28 Nov 2023

While several factors make HBS Online unique —including a global Community and real-world outcomes —active learning through the case study method rises to the top.

In a 2023 City Square Associates survey, 74 percent of HBS Online learners who also took a course from another provider said HBS Online’s case method and real-world examples were better by comparison.

Here’s a primer on the case method, five benefits you could gain, and how to experience it for yourself.

Access your free e-book today.

What Is the Harvard Business School Case Study Method?

The case study method , or case method , is a learning technique in which you’re presented with a real-world business challenge and asked how you’d solve it. After working through it yourself and with peers, you’re told how the scenario played out.

HBS pioneered the case method in 1922. Shortly before, in 1921, the first case was written.

“How do you go into an ambiguous situation and get to the bottom of it?” says HBS Professor Jan Rivkin, former senior associate dean and chair of HBS's master of business administration (MBA) program, in a video about the case method . “That skill—the skill of figuring out a course of inquiry to choose a course of action—that skill is as relevant today as it was in 1921.”

Originally developed for the in-person MBA classroom, HBS Online adapted the case method into an engaging, interactive online learning experience in 2014.

In HBS Online courses , you learn about each case from the business professional who experienced it. After reviewing their videos, you’re prompted to take their perspective and explain how you’d handle their situation.

You then get to read peers’ responses, “star” them, and comment to further the discussion. Afterward, you learn how the professional handled it and their key takeaways.

HBS Online’s adaptation of the case method incorporates the famed HBS “cold call,” in which you’re called on at random to make a decision without time to prepare.

“Learning came to life!” said Sheneka Balogun , chief administration officer and chief of staff at LeMoyne-Owen College, of her experience taking the Credential of Readiness (CORe) program . “The videos from the professors, the interactive cold calls where you were randomly selected to participate, and the case studies that enhanced and often captured the essence of objectives and learning goals were all embedded in each module. This made learning fun, engaging, and student-friendly.”

If you’re considering taking a course that leverages the case study method, here are five benefits you could experience.

5 Benefits of Learning Through Case Studies

1. take new perspectives.

The case method prompts you to consider a scenario from another person’s perspective. To work through the situation and come up with a solution, you must consider their circumstances, limitations, risk tolerance, stakeholders, resources, and potential consequences to assess how to respond.

Taking on new perspectives not only can help you navigate your own challenges but also others’. Putting yourself in someone else’s situation to understand their motivations and needs can go a long way when collaborating with stakeholders.

2. Hone Your Decision-Making Skills

Another skill you can build is the ability to make decisions effectively . The case study method forces you to use limited information to decide how to handle a problem—just like in the real world.

Throughout your career, you’ll need to make difficult decisions with incomplete or imperfect information—and sometimes, you won’t feel qualified to do so. Learning through the case method allows you to practice this skill in a low-stakes environment. When facing a real challenge, you’ll be better prepared to think quickly, collaborate with others, and present and defend your solution.

3. Become More Open-Minded

As you collaborate with peers on responses, it becomes clear that not everyone solves problems the same way. Exposing yourself to various approaches and perspectives can help you become a more open-minded professional.

When you’re part of a diverse group of learners from around the world, your experiences, cultures, and backgrounds contribute to a range of opinions on each case.

On the HBS Online course platform, you’re prompted to view and comment on others’ responses, and discussion is encouraged. This practice of considering others’ perspectives can make you more receptive in your career.

“You’d be surprised at how much you can learn from your peers,” said Ratnaditya Jonnalagadda , a software engineer who took CORe.

In addition to interacting with peers in the course platform, Jonnalagadda was part of the HBS Online Community , where he networked with other professionals and continued discussions sparked by course content.

“You get to understand your peers better, and students share examples of businesses implementing a concept from a module you just learned,” Jonnalagadda said. “It’s a very good way to cement the concepts in one's mind.”

4. Enhance Your Curiosity

One byproduct of taking on different perspectives is that it enables you to picture yourself in various roles, industries, and business functions.

“Each case offers an opportunity for students to see what resonates with them, what excites them, what bores them, which role they could imagine inhabiting in their careers,” says former HBS Dean Nitin Nohria in the Harvard Business Review . “Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders.”

Through the case method, you can “try on” roles you may not have considered and feel more prepared to change or advance your career .

5. Build Your Self-Confidence

Finally, learning through the case study method can build your confidence. Each time you assume a business leader’s perspective, aim to solve a new challenge, and express and defend your opinions and decisions to peers, you prepare to do the same in your career.

According to a 2022 City Square Associates survey , 84 percent of HBS Online learners report feeling more confident making business decisions after taking a course.

“Self-confidence is difficult to teach or coach, but the case study method seems to instill it in people,” Nohria says in the Harvard Business Review . “There may well be other ways of learning these meta-skills, such as the repeated experience gained through practice or guidance from a gifted coach. However, under the direction of a masterful teacher, the case method can engage students and help them develop powerful meta-skills like no other form of teaching.”

How to Experience the Case Study Method

If the case method seems like a good fit for your learning style, experience it for yourself by taking an HBS Online course. Offerings span seven subject areas, including:

• Business essentials
• Leadership and management
• Entrepreneurship and innovation
• Finance and accounting
• Business in society

No matter which course or credential program you choose, you’ll examine case studies from real business professionals, work through their challenges alongside peers, and gain valuable insights to apply to your career.

Are you interested in discovering how HBS Online can help advance your career? Explore our course catalog and download our free guide —complete with interactive workbook sections—to determine if online learning is right for you and which course to take.

About the Author

• Our Mission

Making Learning Relevant With Case Studies

The open-ended problems presented in case studies give students work that feels connected to their lives.

To prepare students for jobs that haven’t been created yet, we need to teach them how to be great problem solvers so that they’ll be ready for anything. One way to do this is by teaching content and skills using real-world case studies, a learning model that’s focused on reflection during the problem-solving process. It’s similar to project-based learning, but PBL is more focused on students creating a product.

Case studies have been used for years by businesses, law and medical schools, physicians on rounds, and artists critiquing work. Like other forms of problem-based learning, case studies can be accessible for every age group, both in one subject and in interdisciplinary work.

You can get started with case studies by tackling relatable questions like these with your students:

• How can we limit food waste in the cafeteria?
• How can we get our school to recycle and compost waste? (Or, if you want to be more complex, how can our school reduce its carbon footprint?)
• How can we improve school attendance?
• How can we reduce the number of people who get sick at school during cold and flu season?

Addressing questions like these leads students to identify topics they need to learn more about. In researching the first question, for example, students may see that they need to research food chains and nutrition. Students often ask, reasonably, why they need to learn something, or when they’ll use their knowledge in the future. Learning is most successful for students when the content and skills they’re studying are relevant, and case studies offer one way to create that sense of relevance.

Teaching With Case Studies

Ultimately, a case study is simply an interesting problem with many correct answers. What does case study work look like in classrooms? Teachers generally start by having students read the case or watch a video that summarizes the case. Students then work in small groups or individually to solve the case study. Teachers set milestones defining what students should accomplish to help them manage their time.

During the case study learning process, student assessment of learning should be focused on reflection. Arthur L. Costa and Bena Kallick’s Learning and Leading With Habits of Mind gives several examples of what this reflection can look like in a classroom: 

Journaling: At the end of each work period, have students write an entry summarizing what they worked on, what worked well, what didn’t, and why. Sentence starters and clear rubrics or guidelines will help students be successful. At the end of a case study project, as Costa and Kallick write, it’s helpful to have students “select significant learnings, envision how they could apply these learnings to future situations, and commit to an action plan to consciously modify their behaviors.”

Interviews: While working on a case study, students can interview each other about their progress and learning. Teachers can interview students individually or in small groups to assess their learning process and their progress.

Student discussion: Discussions can be unstructured—students can talk about what they worked on that day in a think-pair-share or as a full class—or structured, using Socratic seminars or fishbowl discussions. If your class is tackling a case study in small groups, create a second set of small groups with a representative from each of the case study groups so that the groups can share their learning.

4 Tips for Setting Up a Case Study

1. Identify a problem to investigate: This should be something accessible and relevant to students’ lives. The problem should also be challenging and complex enough to yield multiple solutions with many layers.

2. Give context: Think of this step as a movie preview or book summary. Hook the learners to help them understand just enough about the problem to want to learn more.

3. Have a clear rubric: Giving structure to your definition of quality group work and products will lead to stronger end products. You may be able to have your learners help build these definitions.

4. Provide structures for presenting solutions: The amount of scaffolding you build in depends on your students’ skill level and development. A case study product can be something like several pieces of evidence of students collaborating to solve the case study, and ultimately presenting their solution with a detailed slide deck or an essay—you can scaffold this by providing specified headings for the sections of the essay.

Problem-Based Teaching Resources

There are many high-quality, peer-reviewed resources that are open source and easily accessible online.

• The National Center for Case Study Teaching in Science at the University at Buffalo built an online collection of more than 800 cases that cover topics ranging from biochemistry to economics. There are resources for middle and high school students.
• Models of Excellence , a project maintained by EL Education and the Harvard Graduate School of Education, has examples of great problem- and project-based tasks—and corresponding exemplary student work—for grades pre-K to 12.
• The Interdisciplinary Journal of Problem-Based Learning at Purdue University is an open-source journal that publishes examples of problem-based learning in K–12 and post-secondary classrooms.
• The Tech Edvocate has a list of websites and tools related to problem-based learning.

In their book Problems as Possibilities , Linda Torp and Sara Sage write that at the elementary school level, students particularly appreciate how they feel that they are taken seriously when solving case studies. At the middle school level, “researchers stress the importance of relating middle school curriculum to issues of student concern and interest.” And high schoolers, they write, find the case study method “beneficial in preparing them for their future.”

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Case Study?

Weighing the pros and cons of this method of research

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Cara Lustik is a fact-checker and copywriter.

Verywell / Colleen Tighe

• Pros and Cons

What Types of Case Studies Are Out There?

Where do you find data for a case study, how do i write a psychology case study.

A case study is an in-depth study of one person, group, or event. In a case study, nearly every aspect of the subject's life and history is analyzed to seek patterns and causes of behavior. Case studies can be used in many different fields, including psychology, medicine, education, anthropology, political science, and social work.

The point of a case study is to learn as much as possible about an individual or group so that the information can be generalized to many others. Unfortunately, case studies tend to be highly subjective, and it is sometimes difficult to generalize results to a larger population.

While case studies focus on a single individual or group, they follow a format similar to other types of psychology writing. If you are writing a case study, we got you—here are some rules of APA format to reference.  

At a Glance

A case study, or an in-depth study of a person, group, or event, can be a useful research tool when used wisely. In many cases, case studies are best used in situations where it would be difficult or impossible for you to conduct an experiment. They are helpful for looking at unique situations and allow researchers to gather a lot of˜ information about a specific individual or group of people. However, it's important to be cautious of any bias we draw from them as they are highly subjective.

What Are the Benefits and Limitations of Case Studies?

A case study can have its strengths and weaknesses. Researchers must consider these pros and cons before deciding if this type of study is appropriate for their needs.

One of the greatest advantages of a case study is that it allows researchers to investigate things that are often difficult or impossible to replicate in a lab. Some other benefits of a case study:

• Allows researchers to capture information on the 'how,' 'what,' and 'why,' of something that's implemented
• Gives researchers the chance to collect information on why one strategy might be chosen over another
• Permits researchers to develop hypotheses that can be explored in experimental research

On the other hand, a case study can have some drawbacks:

• It cannot necessarily be generalized to the larger population
• Cannot demonstrate cause and effect
• It may not be scientifically rigorous
• It can lead to bias

Researchers may choose to perform a case study if they want to explore a unique or recently discovered phenomenon. Through their insights, researchers develop additional ideas and study questions that might be explored in future studies.

It's important to remember that the insights from case studies cannot be used to determine cause-and-effect relationships between variables. However, case studies may be used to develop hypotheses that can then be addressed in experimental research.

Case Study Examples

There have been a number of notable case studies in the history of psychology. Much of  Freud's work and theories were developed through individual case studies. Some great examples of case studies in psychology include:

• Anna O : Anna O. was a pseudonym of a woman named Bertha Pappenheim, a patient of a physician named Josef Breuer. While she was never a patient of Freud's, Freud and Breuer discussed her case extensively. The woman was experiencing symptoms of a condition that was then known as hysteria and found that talking about her problems helped relieve her symptoms. Her case played an important part in the development of talk therapy as an approach to mental health treatment.
• Phineas Gage : Phineas Gage was a railroad employee who experienced a terrible accident in which an explosion sent a metal rod through his skull, damaging important portions of his brain. Gage recovered from his accident but was left with serious changes in both personality and behavior.
• Genie : Genie was a young girl subjected to horrific abuse and isolation. The case study of Genie allowed researchers to study whether language learning was possible, even after missing critical periods for language development. Her case also served as an example of how scientific research may interfere with treatment and lead to further abuse of vulnerable individuals.

Such cases demonstrate how case research can be used to study things that researchers could not replicate in experimental settings. In Genie's case, her horrific abuse denied her the opportunity to learn a language at critical points in her development.

This is clearly not something researchers could ethically replicate, but conducting a case study on Genie allowed researchers to study phenomena that are otherwise impossible to reproduce.

There are a few different types of case studies that psychologists and other researchers might use:

• Collective case studies : These involve studying a group of individuals. Researchers might study a group of people in a certain setting or look at an entire community. For example, psychologists might explore how access to resources in a community has affected the collective mental well-being of those who live there.
• Descriptive case studies : These involve starting with a descriptive theory. The subjects are then observed, and the information gathered is compared to the pre-existing theory.
• Explanatory case studies : These   are often used to do causal investigations. In other words, researchers are interested in looking at factors that may have caused certain things to occur.
• Exploratory case studies : These are sometimes used as a prelude to further, more in-depth research. This allows researchers to gather more information before developing their research questions and hypotheses .
• Instrumental case studies : These occur when the individual or group allows researchers to understand more than what is initially obvious to observers.
• Intrinsic case studies : This type of case study is when the researcher has a personal interest in the case. Jean Piaget's observations of his own children are good examples of how an intrinsic case study can contribute to the development of a psychological theory.

The three main case study types often used are intrinsic, instrumental, and collective. Intrinsic case studies are useful for learning about unique cases. Instrumental case studies help look at an individual to learn more about a broader issue. A collective case study can be useful for looking at several cases simultaneously.

The type of case study that psychology researchers use depends on the unique characteristics of the situation and the case itself.

There are a number of different sources and methods that researchers can use to gather information about an individual or group. Six major sources that have been identified by researchers are:

• Archival records : Census records, survey records, and name lists are examples of archival records.
• Direct observation : This strategy involves observing the subject, often in a natural setting . While an individual observer is sometimes used, it is more common to utilize a group of observers.
• Documents : Letters, newspaper articles, administrative records, etc., are the types of documents often used as sources.
• Interviews : Interviews are one of the most important methods for gathering information in case studies. An interview can involve structured survey questions or more open-ended questions.
• Participant observation : When the researcher serves as a participant in events and observes the actions and outcomes, it is called participant observation.
• Physical artifacts : Tools, objects, instruments, and other artifacts are often observed during a direct observation of the subject.

If you have been directed to write a case study for a psychology course, be sure to check with your instructor for any specific guidelines you need to follow. If you are writing your case study for a professional publication, check with the publisher for their specific guidelines for submitting a case study.

Here is a general outline of what should be included in a case study.

Section 1: A Case History

This section will have the following structure and content:

Background information : The first section of your paper will present your client's background. Include factors such as age, gender, work, health status, family mental health history, family and social relationships, drug and alcohol history, life difficulties, goals, and coping skills and weaknesses.

Description of the presenting problem : In the next section of your case study, you will describe the problem or symptoms that the client presented with.

Describe any physical, emotional, or sensory symptoms reported by the client. Thoughts, feelings, and perceptions related to the symptoms should also be noted. Any screening or diagnostic assessments that are used should also be described in detail and all scores reported.

Your diagnosis : Provide your diagnosis and give the appropriate Diagnostic and Statistical Manual code. Explain how you reached your diagnosis, how the client's symptoms fit the diagnostic criteria for the disorder(s), or any possible difficulties in reaching a diagnosis.

Section 2: Treatment Plan

This portion of the paper will address the chosen treatment for the condition. This might also include the theoretical basis for the chosen treatment or any other evidence that might exist to support why this approach was chosen.

• Cognitive behavioral approach : Explain how a cognitive behavioral therapist would approach treatment. Offer background information on cognitive behavioral therapy and describe the treatment sessions, client response, and outcome of this type of treatment. Make note of any difficulties or successes encountered by your client during treatment.
• Humanistic approach : Describe a humanistic approach that could be used to treat your client, such as client-centered therapy . Provide information on the type of treatment you chose, the client's reaction to the treatment, and the end result of this approach. Explain why the treatment was successful or unsuccessful.
• Psychoanalytic approach : Describe how a psychoanalytic therapist would view the client's problem. Provide some background on the psychoanalytic approach and cite relevant references. Explain how psychoanalytic therapy would be used to treat the client, how the client would respond to therapy, and the effectiveness of this treatment approach.
• Pharmacological approach : If treatment primarily involves the use of medications, explain which medications were used and why. Provide background on the effectiveness of these medications and how monotherapy may compare with an approach that combines medications with therapy or other treatments.

This section of a case study should also include information about the treatment goals, process, and outcomes.

When you are writing a case study, you should also include a section where you discuss the case study itself, including the strengths and limitiations of the study. You should note how the findings of your case study might support previous research. 

In your discussion section, you should also describe some of the implications of your case study. What ideas or findings might require further exploration? How might researchers go about exploring some of these questions in additional studies?

Need More Tips?

Here are a few additional pointers to keep in mind when formatting your case study:

• Never refer to the subject of your case study as "the client." Instead, use their name or a pseudonym.
• Read examples of case studies to gain an idea about the style and format.
• Remember to use APA format when citing references .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach .  BMC Med Res Methodol . 2011;11:100.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach . BMC Med Res Methodol . 2011 Jun 27;11:100. doi:10.1186/1471-2288-11-100

Gagnon, Yves-Chantal.  The Case Study as Research Method: A Practical Handbook . Canada, Chicago Review Press Incorporated DBA Independent Pub Group, 2010.

Yin, Robert K. Case Study Research and Applications: Design and Methods . United States, SAGE Publications, 2017.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• Free Resources

Marketing Relevance Case Studies: Right offer, right time, right person

This article was published in the MarketingSherpa email newsletter . “We need to field the right offer at the right time for the right person,” Flint McGlaughlin taught in Sales Funnel Power: 3 steps to help you lay the foundation of your value proposition .

In this article, we bring you case studies to inspire your best thinking for getting these three elements right:

• Right time – Learn how Nestlé used AI personalization to vary the content visitors would receive based on time of day, along with many other factors, to get more clicks
• Right offer – See how a marketing agency changed what its short-form videos offer (compared to what it’s long-form videos offered) to grow YouTube subscribers  
• Right person – Read how a global provider of part-time CFOs tweaked its newly successful homepage headline to decrease the form conversion rate…but get more of their ideal prospect.

Quick Case Study #1: How Nestlé used AI for personalization to get 45% more clicks

El Major Nido is Nestlé’s online destination for the Hispanic community with the goal of building a community around food online. “The Hispanic community has a desire to stay connected with their culture while experiencing American culture through food. We created El Mejor Nido with an anchor in the consumer insights of food and culture and want to grow and evolve with our community through digital experiences,” said Margie Bravo, Multicultural Strategist, Nestlé .

Simple, findable recipes are the main drivers of traffic through the website, which currently has the second most traffic of any Nestlé site.

The team wanted to test recipe personalization on the site’s homepage and recipe detail page to see how big of an effect it would have on engagement.

Creative Sample #1: Curated and personalized recipe recommendations on product detail page of Nestlé’s food community website

For the control, every visitor saw the same four curated recipes. For the treatment, each visitor saw a different set of recipes based on factors unique to them, such as:

• Content factors – difficulty level, food trends, inventory, price, ingredients, diet, prep and cook times, complementary meals
• User factors – time of day, weather, location, device, first visit or recurring user, etc.

Different combinations of these different factors will yield different recipe recommendations on the homepage carousel and the carousel found under the recipe detail page.

Here’s an example scenario. Melissa and Taylor are coworkers and both looking at recipes in the office.

One way of “personalization” would be to personalize based on location – so if they are in the same location, they might both receive similar recommendations depending on how different the other factors are. For example, for Melissa who’s checking recipes right at the end of the workday, this might be easy recipes that are vegetarian with prep times less than 30 minutes. For Taylor who’s checking recipes earlier in the morning, this might be easy recipes, that are not vegetarian, but that serve four people.

Another factor was language. The personalization platform automatically redirected visitors to a page in their preferred language (Spanish or English) based on visitors’ browser language preference – 9,900 consumers had their language switched based on this preference.

The personalization is possible for anonymous visitors as well as returning visitors, even on the first visit. “There is an assumption that you have to know your user in order to personalize, but 90% of visitors to a website are first-time and anonymous. You still have to give them a personalized experience to keep them coming back and engaged,” said Diane Keng, CEO, Breinify (Nestlé’s AI-driven personalization platform).

The personalized treatment received 45% more clicks than the curated control.

“AI tools can have so many applications for your business, but for a marketer the most impactful is to help you enrich consumer data and understand your consumers better, to give them exactly what they are looking for,” Bravo said. “The most important things to remember when using technology is to remain consumer focused, to be data-driven and to work with partners that understand your business. You need to make the technology work for you and not the other way around.”

Quick Case #2: How marketing agency grew YouTube subscribers from 5,000 to 455,000 in 100 days

“With all the government scrutiny over TikTok right now, we recently launched a 100-day YouTube video campaign to repurpose TikTok videos as a back-up plan,” said Austin Armstong, CEO, Socialty Pro .

The team focused on only creating similar content to what was performing best for them on TikTok and posting those videos as YouTube Shorts. This strategy helped them rapidly grow their YouTube subscribers.

“YouTube Shorts are in a vertical video format similar to Instagram Reels or TikTok,” Armstrong said. “This success also came right after an announcement from YouTube at VidCon 2022 saying that people who view your YouTube Shorts will now be recommended to view your long-form content. Was this timing a coincidence that impacted the Socialty Pro YouTube Subscriber growth? All it took was one video to explode, and it seemed to unlock all of the similar videos on the channel.”

The team sought to reverse engineer why that video was so successful and use what they learned to inform their YouTube strategy.

From a title and topic perspective, the video was incredibly broad – “Top 5 Most Useful Websites.” It was designed to reach as many people as possible.

The team used a really polarizing opening hook in that video: “These websites feel illegal to know.” They then added a unique spin on a very popular style, using a conversation approach. In each video, one person is talking to another person, but it’s really just the same person.

Creative Sample #2: YouTube Shorts video that went viral for marketing agency

The team took the following lessons from this video, and used these lessons to create more videos:

• Instead of using SEO titles like it does with long-form content, their video titling strategy revolved around short and simple titles for these shorter videos. For example, “Top 3 website growth hacks” instead of “3 ways to get more website backlinks to increase domain authority.” The YouTube Shorts videos that did not work well were too specific.  
• Instead of posting videos that go deep on single topics, they focused on listicle-type videos using a list-based format. For example, “5 types” or “5 websites” were used in the front of the video titles. The YouTube Shorts that did not work well did not include lists.  
• Because watch time is everything on YouTube, the new videos focused on making every cut in the video fast paced, with no slow moment, to keep the viewer engaged in the content.
• They focused on ‘edu-taining’ viewers versus posting boring ‘talking head’ videos. Overall, the team began doing comedic sketches where Armstrong talked to a differently dressed version of himself with corny reactions.  
• At the end of each video, there is also a single call to action, which included subscribing to the Socialty Pro YouTube Channel to find even more useful website tips.

This similar video content posting strategy worked. Socialty Pro’s top 10 videos are all listicles with useful websites. And the company posted one to two of these YouTube Shorts every day to ride the momentum.

The team started using the Community tab on their channel to immediately engage with new subscribers.

Creative Sample #3: Community posts on marketing agency’s YouTube channel

On the Community tab, the team used polls to ask its audience what types of long-form videos they'd like to see more. Based on this feedback, the team made more of what the audience wanted to see. For example, Armstrong’s summary video about reaching 100,000 subscribers in three days came from a poll. And the polls were used to gain feedback for both long- form and short-form videos.

Overall, this growth hacking process and very small tweaks led the team to get up to three million views per day as well as gaining about 50,000 subscribers per day.

The result was the Socialty Pro YouTube Channel grew from 5,000 to 455,000 Subscribers. This rapid subscriber growth did not happen overnight. In fact, it took three years to reach 5,000 subscribers, but then it only took three days for everything to surpass the 100,000-subscriber mark using this video marketing strategy.

Quick Case Study #3: How fractional CFO company got 33% more high-quality form completions from homepage headline and subheadline test

The CFO Centre is a global provider of part-time CFOs. The company’s homepage had the following headline:

Superstar CFOs Helping Ambitious Entrepreneurs Achieve Extraordinary Things (unique part-time model)

That headline was accompanied by these subheadlines:

Increase cash | Increase profit | Increase valuation | Scale faster Affordable part-time model | No up-front recruitment fees | No tie-ins | World's #1 provider

Participating in the MECLABS SuperFunnel Research Cohort inspired a headline and subheadline test on The CFO Centre’s United Kingdom homepage (MECLABS is the parent organization of MarketingSherpa). Here is the new headline:

Break Through £2m Revenue with Powerful Reports and Expert Advice

And new sub headlines:

10,000+ entrepreneurs have built their businesses and the lives they want, with our part-time CFOs Affordable part-time model | No up-front recruitment fees | No tie-ins | World's #1 provider | Established 2001

And this is how the copy looked on the page…

Creative Sample #4: Treatment headline and sub headline copy for global provider of part-time CFOs

“The above was the highest impact statement that I could get through the business for the homepage (we’ll have more powerful headlines for specific services). It aims to predicate the subject and their gain in the first four words and quantify the track record,” said Kate Barlow, Head of Digital Group, The CFO Centre .

"In truth, there are many things the CFOs do – and often they deliver a full spectrum of services once we’re working with a client. We wanted to test a product-level VP (value proposition) with universal appeal. As we work through headlines on the site, I think we’ll begin to test headlines that reflect primary or process VP on the homepage but we wanted to make a start with this one," she said. "My aim is to demonstrate the power of the headline initially and then work on creating a stronger set of words once people see that there’s a huge opportunity for us to tap into. I believe that successful first steps and building my own credibility in terms of running effective tests will open the way to a more adventurous approach."

One driver for change to this headline/subheadline combo was to provide greater clarity to website visitors on whether or not they are the ideal prospect, explain what they will get (reports) and why that matters (to help them grow through their next revenue target). "We thought this headline would work better because it talks to the customer and their needs, rather than at the customer. It also feels less boastful than talking about 'superstar CFOs,' which, I’d say, falls into the realm of marketing hype!" Barlow said. The subheadline adds clarity about the service (fractional or part-time CFOs) and evidentials to support the assertion that the company is the world’s #1 provider.

During the first week, she used the number £1m and that increased completions of the form (“which is in completely the wrong place on the page!” Barlow said) by 250% and live chat interactions by 100%.

In week two, the team changed the number to £2m to fit more closely with their ideal prospects and the numbers decreased (but quality increased). Compared with the control week (the week preceding the first test), they are still seeing a 33% increase in form submissions and 9% increase in chats.

"The benefit to us as a business is that we’re getting more enquiries from our ideal prospects," Barlow said. "This test shows that we better serve the customer when we think deeply about what we’re offering and invest effort in distilling these thoughts into succinct messages. When we work hard to earn the right to gain our prospects’ attention and we can get them to engage deeper (Micro-Yes 1 and Micro-Yes 2), we achieve better results immediately. We’ll definitely be testing more headlines in future but we’ll also be redesigning our homepage to get our micro-yeses in sequence in the future."

But she also cautioned about the limitations of the results. “It's important to add that the numbers are tiny. They don’t come close to meeting a stochastic sample size because we don’t have tons of traffic to the site. However, I’ve now engaged a couple of other countries in the testing process,” she said.

“Alongside this, I’m working on the [MECLABS] SuperFunnel project and I’ll get this ready to launch when Flint [McGlaughlin] wants us to go live. It is incredibly inspiring and uplifting to see so many elements of marketing practice click into place through this process,” Barlow said.

Related Resources

Email Marketing: 3 award-winning lessons about relevance

Marketers Stand Together: 8 crucial conversion optimization lessons from MarketingExperiments videos

Timing and Email Marketing: Sunday generated 23% higher clickthrough than Tuesday in test

Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription? Click Here to Manage Subscriptions

Get Better Business Results With a Skillfully Applied Customer-first Marketing Strategy

The customer-first approach of MarketingSherpa’s agency services can help you build the most effective strategy to serve customers and improve results, and then implement it across every customer touchpoint.

Get headlines, value prop, competitive analysis, and more.

Marketer Vs Machine

Marketer Vs Machine: We need to train the marketer to train the machine.

Free Marketing Course

Become a Marketer-Philosopher: Create and optimize high-converting webpages (with this free online marketing course)

Project and Ideas Pitch Template

A free template to help you win approval for your proposed projects and campaigns

Six Quick CTA checklists

These CTA checklists are specifically designed for your team — something practical to hold up against your CTAs to help the time-pressed marketer quickly consider the customer psychology of your “asks” and how you can improve them.

Infographic: How to Create a Model of Your Customer’s Mind

You need a repeatable methodology focused on building your organization’s customer wisdom throughout your campaigns and websites. This infographic can get you started.

Infographic: 21 Psychological Elements that Power Effective Web Design

To build an effective page from scratch, you need to begin with the psychology of your customer. This infographic can get you started.

Receive the latest case studies and data on email, lead gen, and social media along with MarketingSherpa updates and promotions.

• Your Email Account
• Customer Service Q&A
• Search Library
• Content Directory:

Questions? Contact Customer Service at [email protected]

© 2000-2024 MarketingSherpa LLC, ISSN 1559-5137 Editorial HQ: MarketingSherpa LLC, PO Box 50032, Jacksonville Beach, FL 32240

The views and opinions expressed in the articles of this website are strictly those of the author and do not necessarily reflect in any way the views of MarketingSherpa, its affiliates, or its employees.

Hey there! Free trials are available for Standard and Essentials plans. Start for free today.

What Is a Case Study and Why You Should Use Them

Case studies can provide more insights into your business while helping you conduct further research with robust qualitative data analysis to learn more.

If you're in charge of running a company, then you're likely always looking for new ways to run your business more efficiently and increase your customer base while streamlining as many processes as possible.

Unfortunately, it can sometimes be difficult to determine how to go about implementing the proper program in order to be successful. This is why many business owners opt to conduct a case study, which can help significantly. Whether you've been struggling with brand consistency or some other problem, the right case study can identify why your problem exists as well as provide a way to rectify it.

A case study is a great tool that many businesses aren't even aware exists, and there are marketing experts like Mailchimp who can provide you with step-by-step assistance with implementing a plan with a case study. Many companies discover that not only do they need to start a blog in order to improve business, but they also need to create specific and relevant blog titles.

If your company already has a blog, then optimizing your blog posts may be helpful. Regardless of the obstacles that are preventing you from achieving all your professional goals, a case study can work wonders in helping you reverse this issue.

What is a case study?

A case study is a comprehensive report of the results of theory testing or examining emerging themes of a business in real life context. Case studies are also often used in the healthcare industry, conducting health services research with primary research interest around routinely collected healthcare data.

However, for businesses, the purpose of a case study is to help small business owners or company leaders identify the issues and conduct further research into what may be preventing success through information collection, client or customer interviews, and in-depth data analysis.

Knowing the case study definition is crucial for any business owner. By identifying the issues that are hindering a company from achieving all its goals, it's easier to make the necessary corrections to promote success through influenced data collection.

Why are case studies important?

Now that we've answered the questions, "what is a case study?" Why are case studies important? Some of the top reasons why case studies are important include:

• Understand complex issues: Even after you conduct a significant amount of market research , you might have a difficult time understanding exactly what it means. While you might have the basics down, conducting a case study can help you see how that information is applied. Then, when you see how the information can make a difference in business decisions, it could make it easier to understand complex issues.
• Collect data: A case study can also help with data tracking . A case study is a data collection method that can help you describe the information that you have available to you. Then, you can present that information in a way the reader can understand.
• Conduct evaluations: As you learn more about how to write a case study, remember that you can also use a case study to conduct evaluations of a specific situation. A case study is a great way to learn more about complex situations, and you can evaluate how various people responded in that situation. By conducting a case study evaluation, you can learn more about what has worked well, what has not, and what you might want to change in the future.
• Identify potential solutions: A case study can also help you identify solutions to potential problems. If you have an issue in your business that you are trying to solve, you may be able to take a look at a case study where someone has dealt with a similar situation in the past. For example, you may uncover data bias in a specific solution that you would like to address when you tackle the issue on your own. If you need help solving a difficult problem, a case study may be able to help you.

Remember that you can also use case studies to target your audience . If you want to show your audience that you have a significant level of expertise in a field, you may want to publish some case studies that you have handled in the past. Then, when your audience sees that you have had success in a specific area, they may be more likely to provide you with their business. In essence, case studies can be looked at as the original method of social proof, showcasing exactly how you can help someone solve their problems.

What are the benefits of writing a business case study?

Although writing a case study can seem like a tedious task, there are many benefits to conducting one through an in depth qualitative research process.

• Industry understanding: First of all, a case study can give you an in-depth understanding of your industry through a particular conceptual framework and help you identify hidden problems that are preventing you from transcending into the business world.
• Develop theories: If you decide to write a business case study, it provides you with an opportunity to develop new theories. You might have a theory about how to solve a specific problem, but you need to write a business case study to see exactly how that theory has unfolded in the past. Then, you can figure out if you want to apply your theory to a similar issue in the future.
• Evaluate interventions: When you write a business case study that focuses on a specific situation you have been through in the past, you can uncover whether that intervention was truly helpful. This can make it easier to figure out whether you want to use the same intervention in a similar situation in the future.
• Identify best practices: If you want to stay on top of the best practices in your field, conducting case studies can help by allowing you to identify patterns and trends and develop a new list of best practices that you can follow in the future.
• Versatility: Writing a case study also provides you with more versatility. If you want to expand your business applications, you need to figure out how you respond to various problems. When you run a business case study, you open the door to new opportunities, new applications, and new techniques that could help you make a difference in your business down the road.
• Solve problems: Writing a great case study can dramatically improve your chances of reversing your problem and improving your business.
• These are just a few of the biggest benefits you might experience if you decide to publish your case studies. They can be an effective tool for learning, showcasing your talents, and teaching some of your other employees. If you want to grow your audience , you may want to consider publishing some case studies.

What are the limitations of case studies?

Case studies can be a wonderful tool for any business of any size to use to gain an in-depth understanding of their clients, products, customers, or services, but there are limitations.

One limitation of case studies is the fact that, unless there are other recently published examples, there is nothing to compare them to since, most of the time, you are conducting a single, not multiple, case studies.

Another limitation is the fact that most case studies can lack scientific evidence.

Types of case studies

There are specific types of case studies to choose from, and each specific type will yield different results. Some case study types even overlap, which is sometimes more favorable, as they provide even more pertinent data.

Here are overviews of the different types of case studies, each with its own theoretical framework, so you can determine which type would be most effective for helping you meet your goals.

Explanatory case studies

Explanatory case studies are pretty straightforward, as they're not difficult to interpret. This type of case study is best if there aren't many variables involved because explanatory case studies can easily answer questions like "how" and "why" through theory development.

Exploratory case studies

An exploratory case study does exactly what its name implies: it goes into specific detail about the topic at hand in a natural, real-life context with qualitative research.

The benefits of exploratory case studies are limitless, with the main one being that it offers a great deal of flexibility. Having flexibility when writing a case study is important because you can't always predict what obstacles might arise during the qualitative research process.

Collective case studies

Collective case studies require you to study many different individuals in order to obtain usable data.

Case studies that involve an investigation of people will involve many different variables, all of which can't be predicted. Despite this fact, there are many benefits of collective case studies, including the fact that it allows an ongoing analysis of the data collected.

Intrinsic case studies

This type of study differs from the others as it focuses on the inquiry of one specific instance among many possibilities.

Many people prefer these types of case studies because it allows them to learn about the particular instance that they wish to investigate further.

Instrumental case studies

An instrumental case study is similar to an intrinsic one, as it focuses on a particular instance, whether it's a person, organization, or something different.

One thing that differentiates instrumental case studies from intrinsic ones is the fact that instrumental case studies aren't chosen merely because a person is interested in learning about a specific instance.

Tips for writing a case study

If you have decided to write case studies for your company, then you may be unsure of where to start or which type to conduct.

However, it doesn't have to be difficult or confusing to begin conducting a case study that will help you identify ways to improve your business.

Here are some helpful tips for writing your case studies:

1. Your case study must be written in the proper format

When writing a case study, the format that you should be similar to this:

Administrative summary

The executive summary is an overview of what your report will contain, written in a concise manner while providing real-life context.

Despite the fact that the executive summary should appear at the beginning of your case studies, it shouldn't be written until you've completed the entire report because if you write it before you finish the report, this summary may not be completely accurate.

Key problem statement

In this section of your case study, you will briefly describe the problem that you hope to solve by conducting the study. You will have the opportunity to elaborate on the problem that you're focusing on as you get into the breadth of the report.

Problem exploration

This part of the case study isn't as brief as the other two, and it goes into more detail about the problem at hand. Your problem exploration must include why the identified problem needs to be solved as well as the urgency of solving it.

Additionally, it must include justification for conducting the problem-solving, as the benefits must outweigh the efforts and costs.

Proposed resolution

This case study section will also be lengthier than the first two. It must include how you propose going about rectifying the problem. The "recommended solution" section must also include potential obstacles that you might experience, as well as how these will be managed.

Furthermore, you will need to list alternative solutions and explain the reason the chosen solution is best. Charts can enhance your report and make it easier to read, and provide as much proof to substantiate your claim as possible.

Overview of monetary consideration

An overview of monetary consideration is essential for all case studies, as it will be used to convince all involved parties why your project should be funded. You must successfully convince them that the cost is worth the investment it will require. It's important that you stress the necessity for this particular case study and explain the expected outcome.

Execution timeline

In the execution times of case studies, you explain how long you predict it will take to implement your study. The shorter the time it will take to implement your plan, the more apt it is to be approved. However, be sure to provide a reasonable timeline, taking into consideration any additional time that might be needed due to obstacles.

Always include a conclusion in your case study. This is where you will briefly wrap up your entire proposal, stressing the benefits of completing the data collection and data analysis in order to rectify your problem.

2. Make it clear and comprehensive

You want to write your case studies with as much clarity as possible so that every aspect of the report is understood. Be sure to double-check your grammar, spelling, punctuation, and more, as you don't want to submit a poorly-written document.

Not only would a poorly-written case study fail to prove that what you are trying to achieve is important, but it would also increase the chances that your report will be tossed aside and not taken seriously.

3. Don't rush through the process

Writing the perfect case study takes time and patience. Rushing could result in your forgetting to include information that is crucial to your entire study. Don't waste your time creating a study that simply isn't ready. Take the necessary time to perform all the research necessary to write the best case study possible.

Depending on the case study, conducting case study research could mean using qualitative methods, quantitative methods, or both. Qualitative research questions focus on non-numerical data, such as how people feel, their beliefs, their experiences, and so on.

Meanwhile, quantitative research questions focus on numerical or statistical data collection to explain causal links or get an in-depth picture.

It is also important to collect insightful and constructive feedback. This will help you better understand the outcome as well as any changes you need to make to future case studies. Consider using formal and informal ways to collect feedback to ensure that you get a range of opinions and perspectives.

4. Be confident in your theory development

While writing your case study or conducting your formal experimental investigation, you should have confidence in yourself and what you're proposing in your report. If you took the time to gather all the pertinent data collected to complete the report, don't second-guess yourself or doubt your abilities. If you believe your report will be amazing, then it likely will be.

5. Case studies and all qualitative research are long

It's expected that multiple case studies are going to be incredibly boring, and there is no way around this. However, it doesn't mean you can choose your language carefully in order to keep your audience as engaged as possible.

If your audience loses interest in your case study at the beginning, for whatever reason, then this increases the likelihood that your case study will not be funded.

Case study examples

If you want to learn more about how to write a case study, it might be beneficial to take a look at a few case study examples. Below are a few interesting case study examples you may want to take a closer look at.

• Phineas Gage by John Martin Marlow : One of the most famous case studies comes from the medical field, and it is about the story of Phineas Gage, a man who had a railroad spike driven through his head in 1848. As he was working on a railroad, an explosive charge went off prematurely, sending a railroad rod through his head. Even though he survived this incident, he lost his left eye. However, Phineas Gage was studied extensively over the years because his experiences had a significant, lasting impact on his personality. This served as a case study because his injury showed different parts of the brain have different functions.
• Kitty Genovese and the bystander effect : This is a tragic case study that discusses the murder of Kitty Genovese, a woman attacked and murdered in Queens, New York City. Shockingly, while numerous neighbors watched the scene, nobody called for help because they assumed someone else would. This case study helped to define the bystander effect, which is when a person fails to intervene during an emergency because other people are around.
• Henry Molaison and the study of memory : Henry Molaison lost his memory and suffered from debilitating amnesia. He suffered from childhood epilepsy, and medical professionals attempted to remove the part of his brain that was causing his seizures. He had a portion of his brain removed, but it completely took away his ability to hold memories. Even though he went on to live until the age of 82, he was always forced to live in the present moment, as he was completely unable to form new memories.

Case study FAQs

When should you do a case study.

There are several scenarios when conducting a case study can be beneficial. Case studies are often used when there's a "why" or "how" question that needs to be answered. Case studies are also beneficial when trying to understand a complex phenomenon, there's limited research on a topic, or when you're looking for practical solutions to a problem.

How can case study results be used to make business decisions?

You can use the results from a case study to make future business decisions if you find yourself in a similar situation. As you assess the results of a case study, you can identify best practices, evaluate the effectiveness of an intervention, generate new and creative ideas, or get a better understanding of customer needs.

How are case studies different from other research methodologies?

When compared to other research methodologies, such as experimental or qualitative research methodology, a case study does not require a representative sample. For example, if you are performing quantitative research, you have a lot of subjects that expand your sample size. If you are performing experimental research, you may have a random sample in front of you. A case study is usually designed to deliberately focus on unusual situations, which allows it to shed new light on a specific business research problem.

Writing multiple case studies for your business

If you're feeling overwhelmed by the idea of writing a case study and it seems completely foreign, then you aren't alone. Writing a case study for a business is a very big deal, but fortunately, there is help available because an example of a case study doesn't always help.

Mailchimp, a well-known marketing company that provides comprehensive marketing support for all sorts of businesses, can assist you with your case study, or you can review one of their own recently published examples.

Mailchimp can assist you with developing the most effective content strategy to increase your chances of being as successful as possible. Mailchimp's content studio is a great tool that can help your business immensely.

Importance of a Case Study

Aug 1, 2018 | coursework writing , case study , research paper , Writing

A case study is research method that involves an up-close, in-depth and detailed investigation of a subject of study and its related contextual position. They can be produced following a form of research. A case study helps in bringing the understanding of a complex issue or object. It can extend experience or add strength to the existing knowledge through previous research. Their contextual analysis revolves around a limited number of events or conditions and how they relate. The case study has been used by researchers for a long time and has been applied in different disciplines. It has been widely used in social sciences as a qualitative research method to investigate contemporary real-life situations and has provided a foundation of application of ideas and extension of methods.

It has been defined as an empirical inquiry that examines a contemporary phenomenon within the context of its real life. However, some people have disagreed with this research method arguing that the study of a small number of cases does not offer enough ground to establish reliability or generality of findings. Others have argued that a case study is only used when applied as an exploratory tool, yet most researchers continue using it successfully in carefully planned studies that concern real-life situations, problems, and issues.

Case studies will more often than not appear in journals or professional conferences instead of popular works. A case study may be an individual, organization, action, event existing in a given time and place. For instance, there are case studies of individuals and clinical practices. When the term “case” is used in a claim, an argument, or a proposition; it can be the subject of a litany of research methods. A case study will involve quantitative and qualitative methods of research.

Researchers, on the other hand, are always spoilt for choice when they are determining the tools to use in dealing with their research question. This is because there is an array of both qualitative and quantitative research tools. They can be based on in-depth case studies or desk-based literature reviews. When using case study, the researcher will get an in-depth investigation of a phenomenon, individual, or an event. They help in investigating and understanding the underlying principles in an occurrence within a real-life context.

• They are comprehensive Case studies enable a holistic review. A researcher can use a range of tools which he would otherwise not apply when using other stand-alone research techniques. This gives his time to develop an in-depth understanding of the topic and establish a credible platform to investigate the factors that affect a case study in extensive detail.
• Case studies reduce bias They give room to the diversity of perspectives as opposed to when one is using a single view of a person you get with a survey response or an interview. It eliminates chances of potential bias by giving an opportunity to gain a greater understanding of the subject under investigation. Lack of bias dilutes the agenda of a given individual.
• Broad relevance One of the criticisms that case study method gets is that the findings cannot be generalized. However, when a case study is part of broader research can explore common problems in detail.
• Permissions The identity of the research participants is crucial in painting a real picture of whatever that is taking place. Many researchers have found out that participants are more comfortable in situation s where they are sure that the identity will remain anonymous. However, this presents a challenge given the comprehensive nature of the study. In-depth case studies will require one to seek confirmation that the leading research participant agrees that the material is accurate and anonymous. This enables the confidence on the part of the researcher as well as the participant. Gaining permission can take quite some time and could culminate to additional restatements of the published research.
• Time Case studies consume time. You have to plan for multiple interviews, waiting for the data to come in; coordinating focus groups can take quite a substantial, amount of time. For instance, if you are depending on a voluntary case study participant who is going on with his daily business, that might present a challenge. You can overcome these issues by offering incentives to your participants and then outline what you expect from each of them from the onset and sending deadline notifications in advance. This helps in receiving the data early enough.
• Decide and define the research questions
• Select your case studies and determine the techniques for data collection and analysis
• Prepare to engage in data collection
• Collect data in the field
• Cary out data evaluation and analysis
• Write your report

You have to decide on the questions you want to use in your research . They are a referral for the researcher as he seeks to provide answers to them. The researcher has to establish the focus of the student by coming up with questions that concern the problem or the situation being studied and to determine the purpose of the study. The case study here might involve a program, an entity, a person, or a group of people. Each object has a relationship that is connected to social, political, historical, and personal issues. This provides wide-ranging possibilities for questions and adds complexities to the case study.

The case study must answer questions that begin with “why” or “how.” These questions are directed to a limited number of events or conditions and their inter-relationships. One way that enables researchers to formulate these questions is by conducting a literature review. This allows them to establish what previous researchers found out, and they enable in formulating insightful questions necessary for the examination of the problem. Well defined questions from the onset direct one on where to get more evidence and also helps in determining analysis method for the study, The definition of case study purpose, the literature review and the early decision on the potential audience for the final report will help in providing guidance on how the study will be conducted and published.

The design phase of the case study research gives the researcher an opportunity to decide what the approach will be when it comes to selecting the single or multiple real-life cases for examination. It will also help in deciding the data gathering approaches and the instruments used. When working with multiple cases, each case is treated individually. The conclusion of the individual case can be used as information that contributes to the whole study.

Excellent case studies often select and carefully examine the existing choices in the research tools at disposal with the objective of raising the study validity. You can create boundaries when you do careful discrimination when you are carrying out the selection. One of the strengths of case study method is by the use of multiple techniques and sources in the process of gathering data. The researcher makes an early determination of the evidence he has to gather and the analysis method he will apply for him to answer the research questions. The data gathered may be largely qualitative but can also be quantitative. One can use surveys, interviews, observation, documentation review, or the collection of physical artifacts as tools of data collection. The researcher ought to distribute the data gathering tools systematically when collecting the evidence. The researcher must ensure that the research is constructed to achieve, internal validity, external validity, construct validity and reliability. This should be achieved during the design phase.

Case study often generates big amounts of data from multiple sources. As such, it is important to organize your data systematically to prevent cases of confusion or getting overwhelmed by the incoming data. This helps the researcher to maintain sight of the original research purpose and questions.

One can prepare databases to help in sorting, categorizing, storing, and retrieving data for analysis. Some of the best case studies carries out training for the researchers to establish clear protocols and procedures early enough before the fieldwork kicks off. They also conduct a pilot study well in advance to remove barriers and problems in the field. Once the training is done, the last step is to select a pilot site where each data gathering method is put to the test to uncover problem areas and correct them early.

The researcher must ensure that the evidence and the issue under investigation are have maintained their relationship. It is possible for the researcher to enter data into a database and physically store it. However, he has to document, classify, and cross-reference all evidence for it to be efficiently recalled for examination and sorting as the study continues.

The researcher must collect and store data comprehensively and systematically. This should be done in formats that are easy to reference and sort to enable him in identifying possible lines of inquiry. Successful case studies utilize field notes and databases in categorizing and referencing data, so that is it readily available for interpretation. Field notes take records of feelings, intuitive hunches; pose questions, document work in progress. Stories, testimonies, and illustrations are useful in later reports. Some techniques require the researcher to place information into arrays, matrices of categories, flow charts or other displays as well as tabulations of event frequency. If there is conflicting evidence, the researcher must probe the differences deeper and identify the source of conflict. The researcher must provide answers to the “how” and “why” research questions.

The researcher has to examine raw data using different interpretations. This enables him to draw linkages between the outcomes and the research object bearing in mind the research questions. The researcher must have an open mind during the data evaluation and analysis process. The researcher can strengthen the research findings and conclusion thanks to the multiple data collection methods and analysis techniques he had applied.

The kind of tactics used by the researcher during the analysis compels him to go beyond the initial impressions to improve chances of reliable and accurate findings.

• Preparing the report

Excellent case studies interpret data in ways that they make it easy to understand a problem hitherto complex. It allows the reader to question and examine the study and arrive at an understanding that the researcher was independence. The written report aims to convey to the reader a simplified experience of the issue that was once complex. With case studies, the reader can access the information publicly in ways that may lead him to utilize the experience in his real-life situation.

The report can be written in a manner that handles each case on its separate chapter or giving it a chronological recounting. The researchers at times use the report writing process to do a critical examination of the document to identify ways through which the report might be incomplete. The researcher can use the representative audience to carry out a review and present comments on the same. The comments are the premise upon which the revisions of the documents are made. Sometimes it is recommended to have a journalist in the review audience whereas others argue that the participants should review the document. Those are the steps used in a case study research.

With case studies, the researcher will get a more concrete and unbiased understanding of a given complex situation. This is achieved using a range of search tools. With a real-life view, the research can give leeway for the recommendation of practical solutions to challenges. Case studies are important, and the challenges involved can be surmounted planning, background research and an informed selection of all the participants. If the case study approach works for you, utilize it.

Recent Posts

• Writing an essay on a book
• Case studies: everything you need to know
• Tips to help you make your essay longer
• Writing sociology essays: Sociology essay tips and topics to consider
• The best way to formulate an interesting research paper introduction

Recent Comments

• December 2019
• November 2019
• October 2019
• September 2019
• August 2019
• February 2019
• January 2019
• December 2018
• November 2018
• October 2018
• September 2018
• August 2018
• Abstract Writing
• Annotated Bibliography
• application essay
• Argumentative Essays
• Article Writing
• assignment writing
• book review
• business plan
• business proposal
• chicago style format
• Citing A Research Paper
• Classification Essay
• communication skills
• coursework writing
• critical thinking paper
• Descriptive Essay
• Descriptive Essay Topics
• Dissertation Proposal Writing
• Dissertation writing
• Essay Editing
• Essay on leadership
• Essay Writing
• essay writing skills
• exam preparation
• Homework writing
• Journal Writing
• literature review
• movie review
• Online Writing
• Online Writing Services
• paper writing
• persuasive writing
• philosophy paper
• poem writing
• Presentation writing
• psychology paper
• Report Writing
• research design
• research methods
• research paper
• review writing
• Scholarship essay
• sociology essay
• speech writing
• thematic essays
• thesis paper
• Thesis paper writing
• thesis writing
• thesis writing services
• Uncategorized
• Writing guide

• Growth Marketing
• Content Strategy
• SEO Strategy
• Health Care
• Guide to Growth Marketing
• Guide to Digital PR
• SEO Strategy: The Ultimate Guide
• Website Content Strategy Guide

Client Success Stories

Bringing meaning back to the screen.

Just What the Doctor Ordered

• Open access
• Published: 16 April 2024

How does the external context affect an implementation processes? A qualitative study investigating the impact of macro-level variables on the implementation of goal-oriented primary care

• Ine Huybrechts   ORCID: orcid.org/0000-0003-0288-1756 1 , 2 ,
• Anja Declercq 3 , 4 ,
• Emily Verté 1 , 2 ,
• Peter Raeymaeckers 5   na1 &
• Sibyl Anthierens 1   na1

on behalf of the Primary Care Academy

Implementation Science volume  19 , Article number:  32 ( 2024 ) Cite this article

667 Accesses

8 Altmetric

Metrics details

Although the importance of context in implementation science is not disputed, knowledge about the actual impact of external context variables on implementation processes remains rather fragmented. Current frameworks, models, and studies merely describe macro-level barriers and facilitators, without acknowledging their dynamic character and how they impact and steer implementation. Including organizational theories in implementation frameworks could be a way of tackling this problem. In this study, we therefore investigate how organizational theories can contribute to our understanding of the ways in which external context variables shape implementation processes. We use the implementation process of goal-oriented primary care in Belgium as a case.

A qualitative study using in-depth semi-structured interviews was conducted with actors from a variety of primary care organizations. Data was collected and analyzed with an iterative approach. We assessed the potential of four organizational theories to enrich our understanding of the impact of external context variables on implementation processes. The organizational theories assessed are as follows: institutional theory, resource dependency theory, network theory, and contingency theory. Data analysis was based on a combination of inductive and deductive thematic analysis techniques using NVivo 12.

Institutional theory helps to understand mechanisms that steer and facilitate the implementation of goal-oriented care through regulatory and policy measures. For example, the Flemish government issued policy for facilitating more integrated, person-centered care by means of newly created institutions, incentives, expectations, and other regulatory factors. The three other organizational theories describe both counteracting or reinforcing mechanisms. The financial system hampers interprofessional collaboration, which is key for GOC. Networks between primary care providers and health and/or social care organizations on the one hand facilitate GOC, while on the other hand, technology to support interprofessional collaboration is lacking. Contingent variables such as the aging population and increasing workload and complexity within primary care create circumstances in which GOC is presented as a possible answer.

Conclusions

Insights and propositions that derive from organizational theories can be utilized to expand our knowledge on how external context variables affect implementation processes. These insights can be combined with or integrated into existing implementation frameworks and models to increase their explanatory power.

Peer Review reports

Contributions to literature

Knowledge on how external context variables affect implementation processes tends to be rather fragmented. Insights on external context in implementation research often remain limited to merely describing macro-context barriers and facilitators.

Organizational theories contribute to our understanding on the impact of external context to an implementation process by explaining the complex interactions between organizations and their environments.

Findings can be utilized to help explain the mechanism of change in an implementation process and can be combined with or integrated into existing implementation frameworks and models to gain a broader picture on how external context affects implementation processes.

In this study, we integrate organizational theories to provide a profound analysis on how external context influences the implementation of complex interventions. There is a growing recognition that the context in which an intervention takes place highly influences implementation outcomes [ 1 , 2 ]. Despite its importance, researchers are challenged by the lack of a clear definition of context. Most implementation frameworks and models do not define context as such, but describe categories or elements of context, without capturing it as a whole [ 2 , 3 ]. Studies often distinguish between internal and external context: micro- and meso-level internal context variables are specific to a person, team, or organization. Macro-level external context variables consist of variables on a broader, socio-economic and policy level that are beyond one’s control [ 4 ].

Overall, literature provides a rather fragmented and limited perspective on how external context influences the implementation process of a complex intervention. Attempts are made to define, categorize, and conceptualize external context [ 5 , 6 ]. Certain implementation frameworks and models specifically mention external context, such as the conceptual model of evidence-based practice implementation in public service sectors [ 7 ], the Consolidated Framework for Implementation Research [ 8 ], or the i-PARiHS framework [ 9 ]. However, they remain limited to identifying and describing external context variables. Few studies are conducted that specifically point towards the actual impact of macro-level barriers and facilitators [ 10 , 11 , 12 ] but only provide limited insights in how these shape an implementation process. Nonetheless, external contextual variables can be highly disruptive for an organization’s implementation efforts, for example, when fluctuations in funding occur or when new legislation or technology is introduced [ 13 ]. In order to build a more comprehensive view on external context influences, we need an elaborative theoretical perspective.

Organizational theories as a frame of reference

To better understand how the external context affects the implementation process of a primary care intervention, we build upon research of Birken et al. [ 13 ] who demonstrate the explanatory power of organizational theories. Organizational theories can help explain the complex interactions between organizations and their environments [ 13 ], providing understanding on the impact of external context on the mechanism of change in an implementation process. We focus on three of the theories Birken et al. [ 8 ] put forward: institutional theory, resource dependency theory, and contingency theory. We also include network theory in recognition of the importance of interorganizational context and social ties between various actors, especially in primary care settings which are characterized by a multitude of diverse actors (meaning: participants of a process).

These four organizational theories demonstrate the ways in which organizations interact with their external environment in order to sustain and fulfill their core activities. All four of them do this with a different lens. Institutional theory states that an organization will aim to fulfil the expectations, values, or norms that are posed upon them in order to achieve a fit with their environment [ 14 ]. This theory helps to understand the relationships between organizations and actors and the institutional context in which they operate. Institutions can broadly be defined as a set of expectations for social or organizational behavior that can take the form formal structures such as regulatory entities, legislation, or procedures [ 15 ]. Resource dependency theory explains actions and decisions of organizations in terms of their dependence on critical and important resources. It postulates that organizations will respond to their external environment to secure the resources they need to operate [ 16 , 17 ]. This theory helps to gain insight in how fiscal variables can shape the adoption of an innovation. Contingency theory presupposes that an organizations’ effectiveness depends on the congruence between situational factors and organizational characteristics [ 18 ]. External context variables such as social and economic change and pressure can impact the way in which an innovation will be integrated. Lastly, network theory in its broader sense underlines the strength of networks: collaborating in networks can establish an effectiveness in which outcomes are achieved that could not be realized by individual organizations acting independently. Networks are about connecting or sharing information, resources, activities, and competences of three or more organizations aiming to achieve a shared goal or outcome [ 19 , 20 ]. Investigating networks helps to gain understanding of the importance of the interorganizational context and how social ties between organizations affect the implementation process of a complex intervention.

Goal-oriented care in Flanders as a case

In this study, we focus on the implementation of the approach goal-oriented care (GOC) in primary care in Flanders, the Dutch-speaking region in Belgium. Primary care is a highly institutionalized and regulated setting with a high level of professionalism. Healthcare organizations can be viewed as complex adaptive systems that are increasingly interdependent [ 21 ]. The primary care landscape in Flanders is characterized by many primary care providers (PCPs) being either self-employed or working in group practices or community health centers. They are organized and financed at different levels (federal, regional, local). In 2015–2019, a primary care reform was initiated in Flanders in which the region was geographically divided into 60 primary care zones that are governed by care councils. The Flemish Institute of Primary Care was created as a supporting institution aiming to strengthen the collaboration between primary care health and welfare actors. The complex and multisectoral nature of primary care in Flanders forms an interesting setting to gain understanding in how macro-level context variables affect implementation processes.

The concept of GOC implies a paradigm shift [ 22 ] that shifts away from a disease or problem-oriented focus towards a person-centered focus that departs from “what matters to the patient.” Boeykens et al. [ 23 ] state in their concept analysis that GOC could be described as a healthcare approach encompassing a multifaceted, dynamic, and iterative process underpinned by the patient’s context and values. The process is characterized by three stages: goal elicitations, goal setting, and goal evaluation in which patients’ needs and preferences form the common thread. It is an approach in which PCPs and patients collaborate to identify personal life goals and to align care with those goals [ 23 ]. An illustration of how this manifests at individual level can be found in Table 1 . The concept of GOC was incorporated in Flemish policies and included in the primary care reform in 2015–2019. It has gained interest in research and policy as a potential catalyst for integrated care [ 24 ]. As such, the implementation of GOC in Flanders provides an opportunity to investigate the external context of a complex primary care intervention. Our main research question is as follows: what can organizational theories tell us about the influence of external context variables on the implementation process of GOC?

We assess the potential of four organizational theories to enrich our understanding of the impact of external context variables on implementation processes. The organizational theories assessed are as follows: institutional theory, resource dependency theory, network theory, and contingency theory. Qualitative research methods are most suitable to investigate such complex matters, as they can help answer “how” and “why” questions on implementation [ 25 ]. We conducted online, semi-structured in-depth interviews with various primary care actors. These actors all had some level of experience at either meso- or micro-level with GOC implementation efforts.

Sample selection

For our purposive sample, we used the following inclusion criteria: 1) working in a Flemish health/social care context in which initiatives are taken to implement GOC and 2) having at least 6 months of experience. For recruitment, we made an overview of all possible stakeholders that are active in GOC by calling upon the network of the Primary Care Academy (PCA) Footnote 1 . Additionally, a snowballing approach was used in which respondents could refer to other relevant stakeholders at the end of each interview. This leads to respondents with different backgrounds (not only medical) and varying roles, such as being a staff member, project coordinator, or policy maker. We aimed at a maximum variation in the type of organizations which were represented by respondents, such as different governmental institutions and a variety of healthcare/social care organizations. In some cases, paired interviews were conducted [ 26 ] if the respondents were considered complementary in terms of expertise, background, and experience with the topic. An information letter and a request to participate was send to each stakeholder by e-mail. One reminder was sent in case of nonresponse.

Data collection

Interviews were conducted between January and June 2022 by a sociologist trained in qualitative research methods. Interviewing took place online using the software Microsoft Teams and were audio-recorded and transcribed verbatim. A semi-structured interview guide was used, which included (1) an exploration of the concept of GOC and how the respondent relates to this topic, (2) questions on how GOC became a topic of interest and initiatives within the respondent’s setting, and (3) the perceived barriers and facilitators for implementation. An iterative approach was used between data collection and data analysis, meaning that the interview guide underwent minor adjustments based on proceeding insights from earlier interviews in order to get richer data.

Data analysis

All data were thematically analyzed, both inductively and deductively, supported by the software NVivo 12©. For the inductive part, implicit and explicit ideas within the qualitative data were identified and described [ 27 ]. The broader research team, with backgrounds in sociology, medical sciences, and social work, discussed these initial analyses and results. The main researcher then further elaborated this into a broad understanding. This was followed by a deductive part, in which characteristics and perspectives from organizational theories were used as sensitizing concepts, inspired by research from Birken et al. [ 13 ]. This provided a frame of reference and direction, adding interpretive value to our analysis [ 28 ]. These analyses were subject of peer debriefing with our cooperating research team to validate whether these results aligned with their knowledge of GOC processes. This enhances the trustworthiness and credibility of our results [ 29 , 30 ]. Data analysis was done in Dutch, but illustrative quotes were translated into English.

In-depth interviews were performed with n = 23 respondents (see Table 2 ): five interviews were duo interviews, and one interview took place with n = 3 respondents representing one organization. We had n = 6 refusals: n = 3 because of time restraints, n = 1 did not feel sufficiently knowledgeable about the topic, n = 1 changed professional function, and there was n = 1 nonresponse. Respondents had various ways in which they related towards the macro-context: we included actors that formed part of external context (e.g., the Flemish Agency of Care and Health), actors that facilitate and strengthen organizations in the implementation of GOC (e.g., the umbrella organization for community health centers), and actors that actively convey GOC inside and outside their setting (e.g., an autonomous and integral home care service). Interviews lasted between 47 and 72 min. Table 3 gives an overview on the main findings of our deductive analysis with their respective links to the propositions of each of the organizational theories that we applied as a lens.

Institutional theory: laying foundations for a shift towards GOC

For the implementation of GOC in primary care, looking at the data with an institutional theory lens helps us understand the way in which primary care organizations will respond to social structures surrounding them. Institutional theory describes the influence of institutions, which give shape to organizational fields: “organizations that, in the aggregate, constitute a recognized area of institutional life [ 31 ], p. 148. Prevailing institutions within primary care in Flanders can affect how organizations within such organizational fields fulfil their activities. Throughout our interviews, we recognized several dynamics that are being described in institutional theory.

First of all, the changing landscape of primary care in Flanders (see 1.2) was often brought up as a dynamic in which GOC is intertwined with other changes. Respondents mention an overall tendency to reform primary care to becoming more integrated and the ideas of person-centered care becoming more upfront. These expectations in how primary care should be approached seem to affect the organizational field of primary care: “You could tell that in people’s minds they are ready to look into what it actually means to put the patient, the person central. — INT01” Various policy actors are committed to further steer towards these approaches: “the government has called it the direction that we all have to move towards. — INT23” It was part of the foundations for the most recent primary care reform, leading to the creation of demographic primary care zones governed by care councils and the Flemish Institute of Primary Care as supporting institution.

These newly established actors were viewed by our respondents as catalysts of GOC. They pushed towards the aims to depart from local settings and to establish connections between local actors. Overall, respondents emphasized their added value as they are close to the field and they truly connect primary care actors. “They [care councils] have picked up these concepts and have started working on it. At the moment they are truly the incubators and ecosystems, as they would call it in management slang. — INT04” For an innovation such as GOC to be diffused, they are viewed as the ideal actors who can function as a facilitator or conduit. They are uniquely positioned as they are closely in contact with the practice field and can be a top-down conduit for governmental actors but also are able to address the needs from bottom-up. “In this respect, people look at the primary care zones as the ideal partners. […] We can start bringing people together and have that helicopter view: what is it that truly connects you? — INT23” However, some respondents also mentioned their difficult governance structure due to representation of many disciplines and organizations.

Other regulatory factors were mentioned by respondents were other innovations or changes in primary care that were intentionally linked to GOC: e.g., the BelRAI Footnote 2 or Flemish Social Protection Footnote 3 . “The government also provides incentives. For example, family care services will gradually be obliged to work with the BelRAI screener. This way, you actually force them to start taking up GOC. — INT23” For GOC to be embedded in primary care, links with other regulatory requirements can steer PCPs towards GOC. Furthermore, it was sometimes mentioned that an important step would be for the policy level to acknowledge GOC as quality of care and to include the concept in quality standards. This would further formalize and enforce the institutional expectation to go towards person-centered care.

Currently, a challenge on institutional level as viewed by most respondents is that GOC is not or only to a limited extent incorporated in the basic education of most primary care disciplines. This leads to most of PCPs only having a limited understanding of GOC and different disciplines not having a shared language in this matter. “You have these primary health and welfare actors who each have their own approach, history and culture. To bring them together and to align them is challenging. — INT10” The absence of GOC as a topic in basic education is mentioned by various respondents as a current shortcoming in effectively implementing GOC in the wider primary care landscape.

Overall, GOC is viewed as our respondents as a topic that has recently gained a lot interest, both by individual PCPS, organizations, and governmental actors. The Flemish government has laid some foundations to facilitate this change with newly created institutions and incentives. However, other external context variables can interfere in how the concept of GOC is currently being picked up and what challenges arise.

Resource dependency theory: in search for a financial system that accommodates interprofessional collaboration

Another external context variable that affects how GOC can be introduced is the financial system that is at place. To analyze themes that were raised during the interviews with regard to finances, we utilized a resource dependency perspective. This theory presumes that organizations are dependent on financial resources and are seeking ways to ensure their continued functioning [ 16 , 17 ]. To a certain extent, this collides with the assumptions of institutional theory that foregrounds organization’s conformity to institutional pressures [ 32 ]. Resource dependency theory in contrast highlights differentiation of organizations that seek out competitive advantages [ 32 ].

In this context, respondents mention that their interest and willingness to move towards a GOC approach are held back by the current dominant system of pay for performance in the healthcare system. This financial system is experienced as restrictive, as it does not provide any incentive to PCPs for interprofessional collaboration, which is key for GOC. A switch to a flat fee system (in which a fixed fee is charged for each patient) or bundled payment was often mentioned as desirable. PCPs and health/social care organizations working in a context where they are financially rewarded for a trajectory or treatment of a patient in its entirety ensure that there is no tension with their necessity to obtain financial resources, as described in the resource dependency theory. Many of our respondents voice that community health centers are a good example. They cover different healthcare disciplines and operate with a fixed price per enrolled patient, regardless of the number of services for that patient. This promotes setting up preventive and health-promoting actions, which confirms our finding on the relevance of dedicated funding.

At the governmental level, the best way to finance and give incentives is said to be a point of discussion: “For years, we have been arguing about how to finance. Are we going to fund counsel coordination? Or counsel organization? Or care coordination? — INT04” Macro-level respondents do however mention financial incentives that are already in place to stimulate interprofessional collaboration: fees for multidisciplinary consultation being the most prominent. Other examples were given in which certain requirements were set for funding (e.g., Impulseo Footnote 4 , VIPA Footnote 5 ) that stimulate actors or settings in taking steps towards more interprofessional collaboration.

Nowadays, financial incentives to support organizations to engage in GOC tend to be project grants. However, a structural way to finance GOC approaches is currently lacking, according to our respondents. As a consequence, a long-term perspective for organizations is lacking; there is no stable financing and organizations are obliged to focus on projects instead of normalizing GOC in routine practice. According to a resource dependency perspective, the absence of financial incentives for practicing GOC hinders organizations in engaging with the approach, as they are focused on seeking out resources in order to fulfil their core activities.

A network-theory perspective: the importance of connectedness for the diffusion of an innovation

Throughout the interviews, interorganizational contextual elements were often addressed. A network theory lens states that collaborating in networks can lead to outcomes that could not be realized by individual organizations acting independently [ 19 , 20 ]. Networks consist of a set of actors such as PCPs or health/social care organizations along with a set of ties that link them [ 33 ]. These ties can be state-type ties (e.g., role based, cognitive) or event-type ties (e.g., through interactions, transactions). Both type of ties can enable a flow in which information or innovations can pass, as actors interact [ 33 ]. To analyze the implementation process of GOC and how this is diffused through various actors, a network theory perspective can help understand the importance of the connection between actors.

A first observation throughout the interviews in which we notice the importance of networks was in the mentioning of local initiatives that already existed before the creation of the primary care zones/care councils. In the area around Ghent, local multidisciplinary networks already organized community meetings, bringing together different PCPs on overarching topics relating to long-term care for patients with chronic conditions. These regions have a tradition of collaboration and connectedness of PCPs, which respondents mention to be highly valuable: “This ensures that we are more decisive, speaking from one voice with regards to what we want to stand for. — INT23” Respondents voice that the existence of such local networks has had a positive effect on the diffusion of ideas such as GOC, as trust between different actors was already established.

Further mentioning of the importance of networks could be found in respondents acknowledging one of the presumptions of network theory: working collaboratively towards a specific objective leads to outcomes that cannot be realized independently. This is especially true for GOC, an approach that in essence requires different disciplines to work together: “When only one GP, nurse or social worker starts working on it, it makes no sense. Everyone who is involved with that person needs to be on board. Actually, you need to finetune teams surrounding a person — INT11.” This is why several policy-level respondents mentioned that emphasis was placed on organizing GOC initiatives in a neighborhood-oriented way, in which accessible, inclusive care is aimed at by strengthening social cohesion. This way, different types of PCPs got to know each other through these sessions an GOC and would start to get aligned on what it means to provide GOC. However, in particular, self-employed PCPs are hard to reach. According to our respondents, occupational groups and care councils are suitable actors to engage these self-employed PCPs, but they are not always much involved in such a network .

To better connect PCPs and health/social care organizations, the absence of connectedness through the technological landscape is also mentioned. Current technological systems and platforms for documenting patient information do not allow for aligning and sharing between disciplines. In Flanders, there is a history of each discipline developing its own software, which lacks centralization or unification: “For years, they have decided to just leave it to the market, in such a way that you ended up with a proliferation of software, each discipline having its own package. — INT06” Most of the respondents mentioning this were aware that Flanders government is currently working on a unified digital care and support platform and were optimistic about its development.

Contingency theory: how environmental pressure can be a trigger for change

Our interviews were conducted during a rather dynamic and unique period of time in which the impact of social change and pressure was clearly visible: the Flemish primary care reform was ongoing which leads to the creation of care councils and VIVEL (see 3.1.1), and the COVID crisis impacted the functioning of these and other primary care actors. These observed effects of societal changes are reminiscent of the assumptions that are made in contingency theory. In essence, contingency theory presupposes that “organizational effectiveness results from fitting characteristics of the organization, such as its structure, to contingencies that reflect the situation of the organization [ 34 ], p. 1.” When it comes to the effects of the primary care reform and the COVID crisis, there were several mentions on how primary care actors reorganized their activities to adapt to these circumstances. Representatives of care councils/primary care zones whom we interviewed underlined that they were just at the point where they could again engage with their original action plans, not having to take up so many COVID-related tasks anymore. On the one hand, the COVID crisis had however forced them to immediately become functional and has also contributed that various primary care actors quickly got to know them. On the other hand, the COVID crisis has also kept them from their core activities for a while. On top of that, the crisis has also triggered a change the overall view towards data sharing. Some respondents mention a rather protectionist approach towards data sharing, while data sharing has become more normalized during the COVID crisis. This discussion was also relevant for the creation of a unified shared patient record in terms of documenting and sharing patient goals.

Other societal factors that were mentioned having an impact on the uptake of GOC are the demographic composition of a certain area. It was suggested that areas that are characterized by a patient population with more chronic care needs will be more likely to steer towards GOC as a way of coping with these complex cases. “You always have these GPs who blow it away immediately and question whether this is truly necessary. They will only become receptive to this when they experience needs for which GOC can be a solution — INT11.” On a macro-level, several respondents have mentioned how a driver for change is to have the necessity for change becoming very tangible. As PCPs are confronted with increasing numbers of patients with complex, chronic needs and their work becomes more demanding, the need for change becomes more acute. This finding is in line with what contingency theory underlines: changes in contingency (e.g., the population that is increasingly characterized by aging and multimorbidity) are an impetus for change for health/social care organizations to resolve this by adopting a structure that better fits the current environmental characteristics [ 34 ].

Our research demonstrates the applicability of organizational theories to help explain the impact that macro-level context variables have on an implementation process. These insights can be integrated into existing implementation frameworks and models to add the explanatory power of macro-level context variables, which is to date often neglected. The organizational theories demonstrate the ways in which organizations interact with their external environment in order to sustain and fulfill their core activities. As demonstrated in Fig. 1 , institutional theory largely explains how social expectations in the form of institutions lead towards the adoption or implementation of innovation, such as GOC. However, other organizational theories demonstrate how other macro-context elements on different areas can either strengthen or hamper the implementation process.

How organizational theories can help explain the way in which macro-level context variables affect implementation of an intervention

Departing from the mechanisms that are postulated by institutional theory, we observed that the shift towards GOC is part of a larger Flemish primary care reform in which and new institutions have been established and polices have been drawn up to go towards more integrated, person-centered care. To achieve this, governmental actors have placed emphasis on socialization of care, the local context, and establishing ties between organizations in order to become more complementary in providing primary health care [ 35 ]. With various initiatives surrounding this aim, the Flemish government is steering towards GOC. This is reminiscent of the mechanisms that are posed within institutional theory: organizations adapt to prevailing norms and expectations and mimic behaviors that are surrounding them [ 15 , 36 ].

Throughout our data, we came across concrete examples of how institutionalization takes place. DiMaggio and Powell [ 31 ] describe the subsequent process of isomorphism: organizations start to resemble each other as they are conforming to their institutional environment. A first mechanism through which this change occurs is coercive isomorphism and is clearly noticeable in our data. This type of isomorphism results from both formal and informal pressure coming from organizations from which a dependency relationship exists and from cultural expectations in the society [ 31 ]. Person-centered, GOC care is both formally propagated by governmental institutions and procedures and informally expected by current social tendencies. Care councils within primary care zones explicitly propagate and disseminate ideas and approaches that are desirable on policy level. Another form of isomorphism is professional isomorphism and relates to our finding that incorporation of GOC in basic education is currently lacking. The presumptions of professional isomorphism back up the importance of this: values, norms, and ideas that are developed during education are bound to find entrance within organizations as professionals start operating along these views.

Although many observations in our data back up the assumptions of institutional theory, it should be noticed that new initiatives such as the promotion of person-centered care and GOC can collide with earlier policy trends. Martens et al. [ 12 ] have examined the Belgian policy process relating three integrated care projects and concluded that although there is a strong support for a change towards a more patient-centered system, the current provider-driven system and institutional design complicate this objective. Furthermore, institutional theory tends to simplify actors as passive adopters of institutional norms and expectations and overlook the human agency and sensemaking that come with it [ 37 ]. For GOC, it is particularly true that PCPs will actively have to seek out their own style and fit the approach in their own way of working. Moreover, GOC was not just addressed as a governmental expectation but for many PCPs something they inherently stood behind.

Resources dependency theory poses that organizations are dependent on critical resources and adapt their way of working in response to those resources [ 17 ]. From our findings, it seems that the current financial system does not promote GOC, meaning that the mechanisms that are put forward in resources dependency theory are not set in motion. A macro-level analysis of barriers and facilitators in the implementation of integrated care in Belgium by Danhieux et al. [ 10 ] also points towards the financial system and data sharing as two of the main contextual determinants that affect implementation.

Throughout our data, the importance of a network approach was frequently mentioned. Interprofessional collaboration came forward as a prerequisite to make GOC happen, as well as active commitment on different levels. Burns, Nembhard, and Shortell [ 38 ] argue that research efforts on implementing person-centered, integrated care should have more focus on the use of social networks to study relational coordination. In terms of interprofessional collaboration, to date, Belgium has a limited tradition of working team-based with different disciplines [ 35 ]. However, when it comes to strengthening a cohesive primary care network, the recently established care councils have become an important facilitator. As a network governance structure, they resemble mostly a Network Administrative Organization (NAO): a separate, centralized administrative entity that is externally governed and not another member providing its own services [ 19 ]. According to Provan and Kenis [ 19 ], this type of governance form is most effective in a rather dense network with many participants, when the goal consensus is moderately high, characteristics that are indeed representative for the Flemish primary care landscape. This strengthens our observation that care councils have favorable characteristics and are well-positioned to facilitate the interorganizational context to implement GOC.

Lastly, the presumptions within contingency theory became apparent as respondents talked about how the need for change needs to become tangible for PCPs and organizations to take action, as they are increasingly faced with a shortage of time and means and more complex patient profiles. Furthermore, De Maeseneer [ 39 ] affirms our findings that the COVID-19 crisis could be employed as an opportunity to strengthen primary health care, as health becomes prioritized and its functioning becomes re-evaluated. Overall, contingency theory can help gain insight in how and why certain policy trends or decisions are made. A study of Bruns et al. [ 40 ] found that modifiable external context variables such as interagency collaboration were predictive for policy support for intervention adoption, while unmodifiable external context variable such as socio-economic composition of a region was more predictive for fiscal investments that are made.

Strengths and limitations

This study contributes to our overall understanding of implementation processes by looking into real-life implementation efforts for GOC in Flanders. It goes beyond a mere description of external context variables that affect implementation processes but aims to grasp which and how external context variables influence implementation processes. A variety of respondents from different organizations, with different backgrounds and perspectives, were interviewed, and results were analyzed by researchers with backgrounds in sociology, social work, and medical sciences. Results can not only be applied to further develop sustainable implementation plans for GOC but also enhance our understanding of how the external context influences and shapes implementation processes. As most research on contextual variables in implementation processes has until now mainly focused on internal context variables, knowledge on external context variables contributes to gaining a bigger picture of the mechanism of change.

However, this study is limited to the Flemish landscape, and external context variables and their dynamics might differ from other regions or countries. Furthermore, our study has examined and described how macro-level context variables affect the overall implementation processes of GOC. Further research is needed on the link between outer and inner contexts during implementation and sustainment, as explored by Lengninck-Hall et al. [ 41 ]. Another important consideration is that our sample only includes the “believers” in GOC and those who are already taking steps towards its implementation. It is possible that PCPs themselves or other relevant actors who are more skeptical about GOC have a different view on the policy and organizational processes that we explored. Furthermore, data triangulations in which this data is complemented with document analysis could have expanded our understanding and verified subjective perceptions of respondents.

Insights and propositions that derive from organizational theories can be utilized to expand our knowledge on how external context variables affect implementation processes. Our research demonstrates that the implementation of GOC in Flanders is steered and facilitated by regulatory and policy variables, which sets in motion mechanisms that are described in institutional theory. However, other external context variables interact with the implementation process and can further facilitate or hinder the overall implementation process. Assumptions and mechanisms explained within resource dependency theory, network theory, and contingency theory contribute to our understanding on how fiscal, technological, socio-economic, and interorganizational context variables affect an implementation process.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to confidentiality guaranteed to participants but are available from the corresponding author on reasonable request.

The Primary Care Academy (PCA) is a research and teaching network of four Flemish universities, six university colleges, the White and Yellow Cross (an organization for home nursing), and patient representatives that have included GOC as one of their main research domains.

BelRAI, the Belgian implementation of the interRAI assessment tools; these are scientific, internationally validated instruments enabling an assessment of social, psychological, and physical needs and possibilities of individuals in different care settings. The data follows the person and is shared between care professionals and care organizations.

The Flemish Social Protection is a mandatory insurance established by the Flemish government to provide a range of concessions to individuals with long-term care and support needs due to illness or disability.

Impulseo, financial support for general practitioners who start an individual practice or join a group practice

VIPA, grants for the realization of sustainable, accessible, and affordable healthcare infrastructure

Abbreviations

• Goal-oriented care

Primary care provider

Primary Care Academy

Squires JE, Graham ID, Hutchinson AM, Michie S, Francis JJ, Sales A, et al. Identifying the domains of context important to implementation science: a study protocol. Implement Sci. 2015;10(1):1–9.

Article   Google Scholar  

Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19(1):1–21.

Rogers L, De Brún A, McAuliffe E. Defining and assessing context in healthcare implementation studies: a systematic review. BMC Health Serv Res. 2020;20(1):1–24.

Huybrechts I, Declercq A, Verté E, Raeymaeckers P, Anthierens S. The building blocks of implementation frameworks and models in primary care: a narrative review. Front Public Health. 2021;9:675171.

Article   PubMed   PubMed Central   Google Scholar  

Hamilton AB, Mittman BS, Eccles AM, Hutchinson CS, Wyatt GE. Conceptualizing and measuring external context in implementation science: studying the impacts of regulatory, fiscal, technological and social change. Implement Sci. 2015;10 BioMed Central.

Watson DP, Adams EL, Shue S, Coates H, McGuire A, Chesher J, et al. Defining the external implementation context: an integrative systematic literature review. BMC Health Serv Res. 2018;18(1):1–14.

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38:4–23.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15.

Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2015;11(1):1–13.

Danhieux K, Martens M, Colman E, Wouters E, Remmen R, Van Olmen J, et al. What makes integration of chronic care so difficult? A macro-level analysis of barriers and facilitators in Belgium. International. J Integr Care. 2021;21(4).

Hamilton AB, Mittman BS, Campbell D, Hutchinson C, Liu H, Moss NJ, Wyatt GE. Understanding the impact of external context on community-based implementation of an evidence-based HIV risk reduction intervention. BMC Health Serv Res. 2018;18(1):1–10.

Martens M, Danhieux K, Van Belle S, Wouters E, Van Damme W, Remmen R, et al. Integration or fragmentation of health care? Examining policies and politics in a Belgian case study. Int J Health Policy Manag. 2022;11(9):1668.

PubMed   Google Scholar  

Birken SA, Bunger AC, Powell BJ, Turner K, Clary AS, Klaman SL, et al. Organizational theory for dissemination and implementation research. Implement Sci. 2017;12(1):1–15.

Powell WW, DiMaggio PJ. The new institutionalism in organizational analysis. University of Chicago Press; 2012.

Google Scholar  

Zucker LG. Institutional theories of organization. Annu Rev Sociol. 1987;13(1):443–64.

Hillman AJ, Withers MC, Collins BJ. Resource dependence theory: a review. J Manag. 2009;35(6):1404–27.

Nienhüser W. Resource dependence theory-how well does it explain behavior of organizations? Management Revue; 2008. p. 9–32.

Lammers CJ, Mijs AA, Noort WJ. Organisaties vergelijkenderwijs: ontwikkeling en relevantie van het sociologisch denken over organisaties. Het Spectrum. 2000;6.

Provan KG, Kenis P. Modes of network governance: structure, management, and effectiveness. J Public Adm Res Theory. 2008;18(2):229–52.

Kenis P, Provan K. Het network-governance-perspectief. Business performance management Sturen op prestatie en resultaat; 2008. p. 296–312.

Begun JW, Zimmerman B, Dooley K. Health care organizations as complex adaptive systems. Adv Health Care Org Theory. 2003;253:288.

Mold JW. Failure of the problem-oriented medical paradigm and a person-centered alternative. Ann Fam Med. 2022;20(2):145–8.

Boeykens D, Boeckxstaens P, De Sutter A, Lahousse L, Pype P, De Vriendt P, et al. Goal-oriented care for patients with chronic conditions or multimorbidity in primary care: a scoping review and concept analysis. PLoS One. 2022;17(2):e0262843.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gray CS, Grudniewicz A, Armas A, Mold J, Im J, Boeckxstaens P. Goal-oriented care: a catalyst for person-centred system integration. Int J Integr Care. 2020;20(4).

Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516.

Wilson AD, Onwuegbuzie AJ, Manning LP. Using paired depth interviews to collect qualitative data. Qual Rep. 2016;21(9):1549.

Guest G, MacQueen KM, Namey EE. Applied thematic analysis. Sage Publications; 2011.

Bowen GA. Grounded theory and sensitizing concepts. Int J Qual Methods. 2006;5(3):12–23.

Connelly LM. Trustworthiness in qualitative research. Medsurg Nurs. 2016;25(6):435.

Morse JM, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1(2):13–22.

DiMaggio PJ, Powell WW. The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. Am Sociol Rev. 1983;147-60.

de la Luz F-AM, Valle-Cabrera R. Reconciling institutional theory with organizational theories: how neoinstitutionalism resolves five paradoxes. J Organ Chang Manag. 2006;19(4):503–17.

Borgatti SP, Halgin DS. On network theory. Organ Sci. 2011;22(5):1168–81.

Donaldson L. The contingency theory of organizations. Sage; 2001.

Book   Google Scholar  

De Maeseneer J, Galle A. Belgium’s healthcare system: the way forward to address the challenges of the 21st century: comment on “Integration or Fragmentation of Health Care? Examining Policies and Politics in a Belgian Case Study”. Int J Health Policy Manag. 2023;12.

Dadich A, Doloswala N. What can organisational theory offer knowledge translation in healthcare? A thematic and lexical analysis. BMC Health Serv Res. 2018;18(1):1–20.

Jensen TB, Kjærgaard A, Svejvig P. Using institutional theory with sensemaking theory: a case study of information system implementation in healthcare. J Inf Technol. 2009;24(4):343–53.

Burns LR, Nembhard IM, Shortell SM. Integrating network theory into the study of integrated healthcare. Soc Sci Med. 2022;296:114664.

Article   PubMed   Google Scholar  

De Maeseneer J. COVID-19: using the crisis as an opportunity to strengthen primary health care. Prim Health Care Res Dev. 2021;22:e73.

Bruns EJ, Parker EM, Hensley S, Pullmann MD, Benjamin PH, Lyon AR, Hoagwood KE. The role of the outer setting in implementation: associations between state demographic, fiscal, and policy factors and use of evidence-based treatments in mental healthcare. Implement Sci. 2019;14:1–13.

Lengnick-Hall R, Stadnick NA, Dickson KS, Moullin JC, Aarons GA. Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment. Implement Sci. 2021;16:1–13.

Download references

Acknowledgements

We are grateful for the partnership with the Primary Care Academy (academie-eerstelijn.be) and want to thank the King Baudouin Foundation and Fund Daniël De Coninck for the opportunity they offer us for conducting research and have impact on the primary care of Flanders, Belgium. The consortium of the Primary Care Academy consists of the following: lead author: Roy Remmen—[email protected]—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Emily Verté—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium, and Department of Family Medicine and Chronic Care, Faculty of Medicine and Pharmacy, Vrije Universiteit Brussel, Brussel, Belgium; Muhammed Mustafa Sirimsi—Centre for Research and Innovation in Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Peter Van Bogaert—Workforce Management and Outcomes Research in Care, Faculty of Medicine and Health Sciences, University of Antwerp, Belgium; Hans De Loof—Laboratory of Physio-Pharmacology, Faculty of Pharmaceutical Biomedical and Veterinary Sciences, University of Antwerp, Belgium; Kris Van den Broeck—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Sibyl Anthierens—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Ine Huybrechts—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Peter Raeymaeckers—Department of Sociology, Faculty of Social Sciences, University of Antwerp, Belgium; Veerle Bufel—Department of Sociology, Centre for Population, Family and Health, Faculty of Social Sciences, University of Antwerp, Belgium; Dirk Devroey—Department of Family Medicine and Chronic Care, Faculty of Medicine and Pharmacy, Vrije Universiteit Brussel, Brussel; Bert Aertgeerts—Academic Centre for General Practice, Faculty of Medicine, KU Leuven, Leuven, and Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven; Birgitte Schoenmakers—Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven, Belgium; Lotte Timmermans—Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven, Belgium; Veerle Foulon—Department of Pharmaceutical and Pharmacological Sciences, Faculty Pharmaceutical Sciences, KU Leuven, Leuven, Belgium; Anja Declercq—LUCAS-Centre for Care Research and Consultancy, Faculty of Social Sciences, KU Leuven, Leuven, Belgium; Dominique Van de Velde, Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Pauline Boeckxstaens—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; An De Sutter—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Patricia De Vriendt—Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Frailty in Ageing (FRIA) Research Group, Department of Gerontology and Mental Health and Wellbeing (MENT) Research Group, Faculty of Medicine and Pharmacy, Vrije Universiteit, Brussels, Belgium, and Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Lies Lahousse—Department of Bioanalysis, Faculty of Pharmaceutical Sciences, Ghent University, Ghent, Belgium; Peter Pype—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, End-of-Life Care Research Group, Faculty of Medicine and Health Sciences, Vrije Universiteit Brussel and Ghent University, Ghent, Belgium; Dagje Boeykens—Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Ann Van Hecke—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, University Centre of Nursing and Midwifery, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Peter Decat—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Rudi Roose—Department of Social Work and Social Pedagogy, Faculty of Psychology and Educational Sciences, University Ghent, Belgium; Sandra Martin—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Erica Rutten—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Sam Pless—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Anouk Tuinstra—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Vanessa Gauwe—Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Didier ReynaertE-QUAL, University College of Applied Sciences Ghent, Ghent, Belgium; Leen Van Landschoot—Department of Nursing, University of Applied Sciences Ghent, Ghent, Belgium; Maja Lopez Hartmann—Department of Welfare and Health, Karel de Grote University of Applied Sciences and Arts, Antwerp, Belgium; Tony Claeys—LiveLab, VIVES University of Applied Sciences, Kortrijk, Belgium; Hilde Vandenhoudt—LiCalab, Thomas University of Applied Sciences, Turnhout, Belgium; Kristel De Vliegher—Department of Nursing–Homecare, White-Yellow Cross, Brussels, Belgium; and Susanne Op de Beeck—Flemish Patient Platform, Heverlee, Belgium.

This research was funded by fund Daniël De Coninck, King Baudouin Foundation, Belgium. The funder had no involvement in this study. Grant number: 2019-J5170820-211,588.

Author information

Peter Raeymaeckers and Sibyl Anthierens have contributed equally to this work and share senior last authorship.

Authors and Affiliations

Department of Family Medicine and Population Health, University of Antwerp, Doornstraat 331, 2610, Antwerp, Belgium

Ine Huybrechts, Emily Verté & Sibyl Anthierens

Department of Family Medicine and Chronic Care, Vrije Universiteit Brussel, Laarbeeklaan 103, 1090, Jette/Brussels, Belgium

Ine Huybrechts & Emily Verté

LUCAS — Centre for Care Research and Consultancy, KU Leuven, Minderbroedersstraat 8/5310, 3000, Leuven, Belgium

Anja Declercq

Center for Sociological Research, Faculty of Social Sciences, KU Leuven, Parkstraat 45/3601, 3000, Leuven, Belgium

Department of Social Work, University of Antwerp, St-Jacobstraat 2, 2000, Antwerp, Belgium

Peter Raeymaeckers

You can also search for this author in PubMed   Google Scholar

• , Emily Verté
• , Muhammed Mustafa Sirimsi
• , Peter Van Bogaert
• , Hans De Loof
• , Kris Van den Broeck
• , Sibyl Anthierens
• , Ine Huybrechts
• , Peter Raeymaeckers
• , Veerle Bufel
• , Dirk Devroey
• , Bert Aertgeerts
• , Birgitte Schoenmakers
• , Lotte Timmermans
• , Veerle Foulon
• , Anja Declerq
• , Dominique Van de Velde
• , Pauline Boeckxstaens
• , An De Sutter
• , Patricia De Vriendt
• , Lies Lahousse
• , Peter Pype
• , Dagje Boeykens
• , Ann Van Hecke
• , Peter Decat
• , Rudi Roose
• , Sandra Martin
• , Erica Rutten
• , Sam Pless
• , Anouk Tuinstra
• , Vanessa Gauwe
• , Leen Van Landschoot
• , Maja Lopez Hartmann
• , Tony Claeys
• , Hilde Vandenhoudt
• , Kristel De Vliegher
•  & Susanne Op de Beeck

Contributions

IH wrote the main manuscript text. AD, EV, PR, and SA contributed to the different steps of the making of this manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Ine Huybrechts .

Ethics declarations

Ethics approval and consent to participate.

The study protocol was approved by the Medical Ethics Committee of the University of Antwerp/Antwerp University Hospital (reference: 2021-1690). All participants received verbal and written information about the purpose and methods of the study and gave written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Huybrechts, I., Declercq, A., Verté, E. et al. How does the external context affect an implementation processes? A qualitative study investigating the impact of macro-level variables on the implementation of goal-oriented primary care. Implementation Sci 19 , 32 (2024). https://doi.org/10.1186/s13012-024-01360-0

Download citation

Received : 03 January 2024

Accepted : 28 March 2024

Published : 16 April 2024

DOI : https://doi.org/10.1186/s13012-024-01360-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Contingency theory
• External context
• Institutional theory
• Primary care
• Implementation process
• Macro-context
• Network theory
• Organizational theories
• Resource dependency theory

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

CASE STUDY: Serving Relevant Content to International Customers

Global tech brand engages Welocalize to enhance its content relevance program.   This case study explores how a global technology brand partnered with Welocalize to significantly enhance its content relevance program, effectively scaling across international markets to deliver reliable search results. Together, we achieved:  Onboarded over 1300 raters across 15 locales in the first 20…

Global tech brand engages Welocalize to enhance its content relevance program.

This case study explores how a global technology brand partnered with Welocalize to significantly enhance its content relevance program, effectively scaling across international markets to deliver reliable search results. Together, we achieved:  

• Onboarded over 1300 raters across 15 locales in the first 20 months  
• 150 million tasks completed annually  

Scroll 👇 to read the case study. If you have further questions, feel free to get in touch .  

The Challenge

Our technology client required a system capable of handling diverse workflows across numerous languages, with a requirement for data relevance, search relevance, content moderation, and ad quality. The diversity of the content, including image, text, audio, and video workflows, posed a substantial challenge.  

Inconsistent Quality and Relevance

There were concerns regarding the consistent quality and relevance of content across the vast and varied customer base. The global reach necessitated expertise in over 65 languages, involving distinct market cultural nuances and preferences.  

Scalability and Adaptability

Given the expansive global markets and the requirement to manage over 8,000 remote workers, the system needed to be adaptable to various workflows and rapidly scale to meet demand while maintaining high-quality outputs.  

The Solution

A collaboration with Welocalize was initiated to tackle these challenges. Welocalize devised a comprehensive remote worker model, incorporating the client’s task platform to create a fully managed service solution:  

Diverse and Expert Teams  

Welocalize assembled a team of Process Strategists, Quality Program Architects, an Internal Quality Assessor, and other experts to create a system for managing remote workers. This team was instrumental in developing more than 125 workflows, covering 65 languages, and effectively managing remote workers involved in tasks like content rating, grading, and annotating.  

Dynamic and Responsive Strategy

A responsive and flexible team was established, capable of quickly adapting to and implementing changes. Welocalize provided quality assurance and emphasized data-driven reporting and confidentiality. The well-being of the rater teams was a high priority, with programs put in place to protect remote workers.  

The Results

Within the first 20 months, we achieved remarkable scale, with over 1300 raters onboarded across 15 locales, completing a staggering 22 million tasks. The continued collaboration has fostered an efficient environment, with over 150 million tasks completed annually.

“As our relationship continues to flourish, our collaboration stands as a testament to what can be achieved through a dedicated and expert approach to content relevance and moderation at a global scale.”  

Through an emphasis on fair compensation audits and adherence to DEI and accessibility norms, the partnership ensured quality and compliance without compromise. Integrating a quality control and super-rater program further reinforced the commitment to quality.  

Furthermore, the collaboration incorporated various pricing options, including cost-per-task and performance-based pricing, alongside regular reviews to foster a cost-neutral continuous improvement environment, demonstrating a commitment to quality and financial efficiency.  

If you are interested in enhancing your content relevance program, we can help. Contact us to explore possibilities.  

Loading metrics

Open Access

Peer-reviewed

Research Article

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected] (AJT); [email protected] (DSJT)

Affiliations University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom, Oxford University Clinical Academic Graduate School, University of Oxford, Oxford, United Kingdom

Roles Data curation, Investigation, Writing – review & editing

Affiliation University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom

Affiliation Eye Institute, Cleveland Clinic Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates

Roles Data curation, Investigation, Writing – original draft, Writing – review & editing

Affiliations University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom, Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, United Kingdom

Roles Data curation, Investigation

Affiliation West Suffolk NHS Foundation Trust, Bury St Edmunds, United Kingdom

Affiliation Manchester Royal Eye Hospital, Manchester University NHS Foundation Trust, Manchester, United Kingdom

Affiliation Birmingham and Midland Eye Centre, Sandwell and West Birmingham NHS Foundation Trust, Birmingham, United Kingdom

Affiliation Department of Ophthalmology, Chang Gung Memorial Hospital, Linkou Medical Center, Taoyuan, Taiwan

Affiliation Yong Loo Lin School of Medicine, National University of Singapore, Singapore

Roles Data curation, Investigation, Project administration, Writing – review & editing

Affiliation Bedfordshire Hospitals NHS Foundation Trust, Luton and Dunstable, United Kingdom

Affiliation Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore

Roles Writing – review & editing

Affiliations Birmingham and Midland Eye Centre, Sandwell and West Birmingham NHS Foundation Trust, Birmingham, United Kingdom, Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom

Roles Funding acquisition, Project administration

Affiliations Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Byers Eye Institute, Stanford University, Palo Alto, California, United States of America

•  [ ... ],

Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

Affiliations Birmingham and Midland Eye Centre, Sandwell and West Birmingham NHS Foundation Trust, Birmingham, United Kingdom, Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom, Academic Ophthalmology, School of Medicine, University of Nottingham, Nottingham, United Kingdom

• [ view all ]
• [ view less ]
• Arun James Thirunavukarasu, 
• Shathar Mahmood, 
• Andrew Malem, 
• William Paul Foster, 
• Rohan Sanghera, 
• Refaat Hassan, 
• Sean Zhou, 
• Shiao Wei Wong, 
• Yee Ling Wong, 

• Published: April 17, 2024
• https://doi.org/10.1371/journal.pdig.0000341
• Reader Comments

Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types ( p >0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher ( p <0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.

Author summary

Large language models (LLMs) are the most sophisticated form of language-based artificial intelligence. LLMs have the potential to improve healthcare, and experiments and trials are ongoing to explore potential avenues for LLMs to improve patient care. Here, we test state-of-the-art LLMs on challenging questions used to assess the aptitude of eye doctors (ophthalmologists) in the United Kingdom before they can be deemed fully qualified. We compare the performance of these LLMs to fully trained ophthalmologists as well as doctors in training to gauge the aptitude of the LLMs for providing advice to patients about eye health. One of the LLMs, GPT-4, exhibits favourable performance when compared with fully qualified and training ophthalmologists; and comparisons with its predecessor model, GPT-3.5, indicate that this superior performance is due to improved accuracy and relevance of model responses. LLMs are approaching expert-level ophthalmological knowledge and reasoning, and may be useful for providing eye-related advice where access to healthcare professionals is limited. Further research is required to explore potential avenues of clinical deployment.

Citation: Thirunavukarasu AJ, Mahmood S, Malem A, Foster WP, Sanghera R, Hassan R, et al. (2024) Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study. PLOS Digit Health 3(4): e0000341. https://doi.org/10.1371/journal.pdig.0000341

Editor: Man Luo, Mayo Clinic Scottsdale, UNITED STATES

Received: July 31, 2023; Accepted: February 26, 2024; Published: April 17, 2024

Copyright: © 2024 Thirunavukarasu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are available as supplementary information , excluding copyrighted material from the textbook used for experiments.

Funding: DSWT is supported by the National Medical Research Council, Singapore (NMCR/HSRG/0087/2018; MOH-000655-00; MOH-001014-00), Duke-NUS Medical School (Duke-NUS/RSF/2021/0018; 05/FY2020/EX/15-A58), and Agency for Science, Technology and Research (A20H4g2141; H20C6a0032). DSJT is supported by a Medical Research Council / Fight for Sight Clinical Research Fellowship (MR/T001674/1). These funders were not involved in the conception, execution, or reporting of this review.

Competing interests: AM is a member of the Panel of Examiners of the Royal College of Ophthalmologists and performs unpaid work as an FRCOphth examiner. DSWT holds a patent on a deep learning system to detect retinal disease. DSJT authored the book used in the study and receives royalty from its sales. The other authors have no competing interests to declare.

Introduction

Generative Pre-trained Transformer 3.5 (GPT-3.5) and 4 (GPT-4) are large language models (LLMs) trained on datasets containing hundreds of billions of words from articles, books, and other internet sources [ 1 , 2 ]. ChatGPT is an online chatbot which uses GPT-3.5 or GPT-4 to provide bespoke responses to human users’ queries [ 3 ]. LLMs have revolutionised the field of natural language processing, and ChatGPT has attracted significant attention in medicine for attaining passing level performance in medical school examinations and providing more accurate and empathetic messages than human doctors in response to patient queries on a social media platform [ 3 , 4 , 5 , 6 ]. While GPT-3.5 performance in more specialised examinations has been inadequate, GPT-4 is thought to represent a significant advancement in terms of medical knowledge and reasoning [ 3 , 7 , 8 ]. Other LLMs in wide use include Pathways Language Model 2 (PaLM 2) and Large Language Model Meta AI 2 (LLaMA 2) [ 3 ], [ 9 , p. 2], [ 10 ].

Applications and trials of LLMs in ophthalmological settings has been limited despite ChatGPT’s performance in questions relating to ‘eyes and vision’ being superior to other subjects in an examination for general practitioners [ 7 , 11 ]. ChatGPT has been trialled on the North American Ophthalmology Knowledge Assessment Program (OKAP), and Fellowship of the Royal College of Ophthalmologists (FRCOphth) Part 1 and Part 2 examinations. In both cases, relatively poor results have been reported for GPT-3.5, with significant improvement exhibited by GPT-4 [ 12 , 13 , 14 , 15 , 16 ]. However, previous studies are afflicted by two important issues which may affect their validity and interpretability. First, so-called ‘contamination’, where test material features in the pretraining data used to develop LLMs, may result in inflated performance as models recall previously seen text rather than using clinical reasoning to provide an answer. Second, examination performance in and of itself provides little information regarding the potential of models to contribute to clinical practice as a medical-assistance tool [ 3 ]. Clinical benchmarks are required to understanding the meaning and implications of scores in ophthalmological examinations attained by LLMs and are a necessary precursor to clinical trials of LLM-based interventions.

Here, we used FRCOphth Part 2 examination questions to gauge the ophthalmological knowledge base and reasoning capability of LLMs using fully qualified and currently training ophthalmologists as clinical benchmarks. These questions were not freely available online, minimising the risk of contamination. The FRCOphth Part 2 Written Examination tests the clinical knowledge and skills of ophthalmologists in training using multiple choice questions with no negative marking and must be passed to fully qualify as a specialist eye doctor in the United Kingdom.

Question extraction

FRCOphth Part 2 questions were sourced from a textbook for doctors preparing to take the examination [ 17 ]. This textbook is not freely available on the internet, making the possibility of its content being included in LLMs’ training datasets unlikely [ 1 ]. All 360 multiple-choice questions from the textbook’s six chapters were extracted, and a 90-question mock examination from the textbook was segregated for LLM and doctor comparisons. Two researchers matched the subject categories of the practice papers’ questions to those defined in the Royal College of Ophthalmologists’ documentation concerning the FRCOphth Part 2 written examination. Similarly, two researchers categorised each question as first order recall or higher order reasoning, corresponding to ‘remembering’ and ‘applying’ or ‘analysing’ in Bloom’s taxonomy, respectively [ 18 ]. Disagreement between classification decisions was resolved by a third researcher casting a deciding vote. Questions containing non-plain text elements such as images were excluded as these could not be inputted to the LLM applications.

Trialling large language models

Every eligible question was inputted into ChatGPT (GPT-3.5 and GPT-4 versions; OpenAI, San Francisco, California, United States of America) between April 29 and May 10, 2023. The answers provided by GPT-3.5 and GPT-4 were recorded and their whole reply to each question was recorded for further analysis. If ChatGPT failed to provide a definitive answer, the question was re-trialled up to three times, after which ChatGPT’s answer was recorded as ‘null’ if no answer was provided. Correct answers (‘ground truth’) were defined as the answers provided by the textbook and were recorded for every eligible question to facilitate calculation of performance. Upon their release, Bard (Google LLC, Mountain View, California, USA) and HuggingChat (Hugging Face, Inc., New York City, USA) were used to trial PaLM 2 (Google LLC) and LLaMA (Meta, Menlo Park, California, USA) respectively on the portion of the textbook corresponding to a 90-question examination, adhering to the same procedures between June 20 and July 2, 2023.

Clinical benchmarks

To gauge the performance, accuracy, and relevance of LLM outputs, five expert ophthalmologists who had all passed the FRCOphth Part 2 (E1-E5), three trainees (residents) currently in ophthalmology training programmes (T1-T3), and two unspecialised ( i . e . not in ophthalmology training) junior doctors (J1-J2) first answered the 90-question mock examination independently, without reference to textbooks, the internet, or LLMs’ recorded answers. As with the LLMs, doctors’ performance was calculated with reference to the correct answers provided by the textbook. After completing the examination, ophthalmologists graded the whole output of GPT-3.5 and GPT-4 on a Likert scale from 1–5 (very bad, bad, neutral, good, very good) to qualitatively appraise accuracy of information provided and relevance of outputs to the question used as an input prompt. For these appraisals, ophthalmologists were blind to the LLM source (which was presented in a randomised order) and to their previous answers to the same questions, but they could refer to the question text and correct answer and explanation provided by the textbook. Procedures are comprehensively described in the protocol issued to the ophthalmologists ( S1 Protocol ).

Our null hypothesis was that LLMs and doctors would exhibit similar performance, supported by results in a wide range of medical examinations [ 3 , 6 ]. Prospective power analysis was conducted which indicated that 63 questions were required to identify a 10% superior performance of an LLM to human performance at a 5% significance level (type 1 error rate) with 80% power (20% type 2 error rate). This indicated that the 90-question examination in our experiments was more than sufficient to detect ~10% differences in overall performance. The whole 90-question mock examination was used to avoid over- or under-sampling certain question types with respect to actual FRCOphth papers. To verify that the mock examination was representative of the FRCOphth Part 2 examination, expert ophthalmologists were asked to rate the difficulty of questions used here in comparison to official examinations on a 5-point Likert scale (“much easier”, “somewhat easier”, “similar”, “somewhat more difficult”, “much more difficult”).

Statistical analysis

Performance of doctors and LLMs were compared using chi-squared (χ 2 ) tests. Agreement between answers provided by doctors and LLMs was quantified through calculation of Kappa statistics, interpreted in accordance with McHugh’s recommendations [ 19 ]. To further explore the strengths and weaknesses of the answer providers, performance was stratified by question type (first order fact recall or higher order reasoning) and subject using a chi-squared or Fisher’s exact test where appropriate. Likert scale data corresponding to the accuracy and relevance of GPT-3.5 and GPT-4 responses to the same questions were analysed with paired t -tests with the Bonferroni correction applied to mitigate the risk of false positive results due to multiple-testing—parametric testing was justified by a sufficient sample size [ 20 ]. A chi-squared test was used to quantify the significance of any difference in overall preference of ophthalmologists choosing between GPT-3.5 and GPT-4 responses. Statistical significance was concluded where p < 0.05. For additional contextualisation, examination statistics corresponding to FRCOphth Part 2 written examinations taken between July 2017 and December 2022 were collected from Royal College of Ophthalmologists examiners’ reports [ 21 ]. These statistics facilitated comparisons between human and LLM performance in the mock examination with the performance of actual candidates in recent examinations. Failure cases where all LLMs provided an incorrect answer were appraised qualitatively to explore any specific weaknesses of the technology.

Statistical analysis was conducted in R (version 4.1.2; R Foundation for Statistical Computing, Vienna, Austria), and figures were produced in Affinity Designer (version 1.10.6; Serif Ltd, West Bridgford, Nottinghamshire, United Kingdom).

Questions sources

Of 360 questions in the textbook, 347 questions (including 87 of the 90 questions from the mock examination chapter) were included [ 17 ]. Exclusions were all due to non-text elements such as images and tables which could not be inputted into LLM chatbot interfaces. The distribution of question types and subjects within the whole set and mock examination set of questions is summarised in Table 1 and S1 Table alongside performance.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

Question subject and type distributions presented alongside scores attained by LLMs (GPT-3.5, GPT-4, LLaMA, and PaLM 2), expert ophthalmologists (E1-E5), ophthalmology trainees (T1-T3), and unspecialised junior doctors (J1-J2). Median scores do not necessarily sum to the overall median score, as fractional scores are impossible.

https://doi.org/10.1371/journal.pdig.0000341.t001

GPT-4 represents a significant advance on GPT-3.5 in ophthalmological knowledge and reasoning.

Overall performance over 347 questions was significantly higher for GPT-4 (61.7%) than GPT-3.5 (48.41%; χ 2 = 12.32, p <0.01), with results detailed in S1 Fig and S1 Table . ChatGPT performance was consistent across question types and subjects ( S1 Table ). For GPT-4, no significant variation was observed with respect to first order and higher order questions (χ 2 = 0.22, p = 0.64), or subjects defined by the Royal College of Ophthalmologists (Fisher’s exact test over 2000 iterations, p = 0.23). Similar results were observed for GPT-3.5 with respect to first and second order questions (χ 2 = 0.08, p = 0.77), and subjects (Fisher’s exact test over 2000 iterations, p = 0.28). Performance and variation within the 87-question mock examination was very similar to the overall performance over 347 questions, and subsequent experiments were therefore restricted to that representative set of questions.

GPT-4 compares well with other LLMs, junior and trainee doctors and ophthalmology experts.

Performance in the mock examination is summarised in Fig 1 —GPT-4 (69%) was the top-scoring model, performing to a significantly higher standard than GPT-3.5 (48%; χ 2 = 7.33, p < 0.01) and LLaMA (32%; χ 2 = 22.77, p < 0.01), but statistically similarly to PaLM 2 (56%) despite a superior score (χ 2 = 2.81, p = 0.09). LLaMA exhibited the lowest examination score, significantly weaker than GPT-3.5 (χ 2 = 4.58, p = 0.03) and PaLM-2 (χ 2 = 10.01, p < 0.01) as well as GPT-4.

Examination performance in the 87-question mock examination used to trial LLMs (GPT-3.5, GPT-4, LLaMA, and PaLM 2), expert ophthalmologists (E1-E5), ophthalmology trainees (T1-T3), and unspecialised junior doctors (J1-J2). Dotted lines depict the mean performance of expert ophthalmologists (66/87; 76%), ophthalmology trainees (60/87; 69%), and unspecialised junior doctors (37/87; 43%). The performance of GPT-4 lay within the range of expert ophthalmologists and ophthalmology trainees.

https://doi.org/10.1371/journal.pdig.0000341.g001

The performance of GPT-4 was statistically similar to the mean score attained by expert ophthalmologists ( Fig 1 ; χ 2 = 1.18, p = 0.28). Moreover, GPT-4’s performance exceeded the mean mark attained across FRCOphth Part 2 written examination candidates between 2017–2022 (66.06%), mean pass mark according to standard setting (61.31%), and the mean official mark required to pass the examination after adjustment (63.75%), as detailed in S2 Table . In individual comparisons with expert ophthalmologists, GPT-4 was equivalent in 3 cases (χ 2 tests, p > 0.05, S3 Table ), and inferior in 2 cases (χ 2 tests, p < 0.05; Table 2 ). In comparisons with ophthalmology trainees, GPT-4 was equivalent to all three ophthalmology trainees (χ 2 tests, p > 0.05; Table 2 ). GPT-4 was significantly superior to both unspecialised trainee doctors (χ 2 tests, p < 0.05; Table 2 ). Doctors were anonymised in analysis, but their ophthalmological experience is summarised in S3 Table . Unsurprisingly, junior doctors (J1-J2) attained lower scores than expert ophthalmologists (E1-E5; t = 7.18, p < 0.01), and ophthalmology trainees (T1-T3; t = 11.18, p < 0.01), illustrated in Fig 1 . Ophthalmology trainees approached expert-level scores with no significant difference between the groups ( t = 1.55, p = 0.18). None of the other LLMs matched any of the expert ophthalmologists, mean mark of real examination candidates, or FRCOphth Part 2 pass mark.

Expert ophthalmologists agreed that the mock examination was a faithful representation of actual FRCOphth Part 2 Written Examination papers with a mean and median score of 3/5 (range 2-4/5).

Results of pair-wise comparisons of examination performance between GPT-4 and the other answer providers. Significantly greater performance for GPT-4 is highlighted green, significantly inferior performance for GPT-4 is highlighted orange. GPT-4 was superior to all other LLMs and unspecialised junior doctors, and equivalent to most expert ophthalmologists and all ophthalmology trainees.

https://doi.org/10.1371/journal.pdig.0000341.t002

LLM strengths and weaknesses are similar to doctors.

Agreement between answers given by LLMs, expert ophthalmologists, and trainee doctors was generally absent (0 ≤ κ < 0.2), minimal (0.2 ≤ κ < 0.4), or weak (0.4 ≤ κ < 0.6), with moderate agreement only recorded for one pairing between the two highest performing ophthalmologists ( Fig 2 ; κ = 0.64) [ 19 ]. Disagreement was primarily the result of general differences in knowledge and reasoning ability, illustrated by strong negative correlation between Kappa statistic (quantifying agreement) and difference in examination performance (Pearson’s r = -0.63, p < 0.01). Answer providers with more similar scores exhibited greater agreement overall irrespective of their category (LLM, expert ophthalmologist, ophthalmology trainee, or junior doctor).

Agreement correlates strongly with overall performance and stratification analysis found no particular question type or subject was associated with better performance of LLMs or doctors, indicating that LLM knowledge and reasoning ability is general across ophthalmology rather than restricted to particular subspecialties or question types.

https://doi.org/10.1371/journal.pdig.0000341.g002

Stratification analysis was undertaken to identify any specific strengths and weaknesses of LLMs with respect to expert ophthalmologists and trainee doctors ( Table 1 and S4 Table ). No significant difference between performance in first order fact recall and higher order reasoning questions was observed among any of the LLMs, expert ophthalmologists, ophthalmology trainees, or unspecialised junior doctors ( S4 Table ; χ 2 tests, p > 0.05). Similarly, only J1 (junior doctor yet to commence ophthalmology training) exhibited statistically significant variation in performance between subjects ( S4 Table ; Fisher’s exact tests over 2000 iterations, p = 0.02); all other doctors and LLMs exhibited no significant variation (Fisher’s exact tests over 2000 iterations, p > 0.05). To explore whether consistency was due to an insufficient sample size, similar analyses were run for GPT-3.5 and GPT-4 performance over the larger set of 347 questions ( S1 Table ; S4 Table ). As with the mock examination, no significant differences in performance across question types ( S4 Table ; χ 2 tests, p > 0.05) or subjects ( S4 Table ; Fisher’s exact tests over 2000 iterations, p > 0.05) were observed.

LLM examination performance translates to subjective preference indicated by expert ophthalmologists.

Ophthalmologists’ appraisal of GPT-4 and GPT-3.5 outputs indicated a marked preference for the former over the latter, mirroring objective performance in the mock examination and over the whole textbook. GPT-4 exhibited significantly ( t -test with Bonferroni correction, p < 0.05) higher accuracy and relevance than GPT-3.5 according to all five ophthalmologists’ grading ( Table 3 ). Differences were visually obvious, with GPT-4 exhibiting much higher rates of attaining the highest scores for accuracy and relevance than GPT-3.5 ( Fig 3 ). This superiority was reflected in ophthalmologists’ qualitative preference indications: GPT-4 responses were preferred to GPT-3.5 responses by every ophthalmologist with statistically significant skew in favour of GPT-4 (χ 2 test, p < 0.05; Table 3 ).

Accuracy (A) and relevance (B) ratings were provided by five expert ophthalmologists for ChatGPT (powered by GPT-3.5 and GPT-4) responses to 87 FRCOphth Part 2 mock examination questions. In every case, the accuracy and relevance of GPT-4 is significantly superior to GPT-3.5 (t-test with Bonferroni correct applied, p < 0.05). Pooled scores for accuracy (C) and relevance (D) from all five raters are presented in the bottom two plots, with GPT-3.5 (left bars) compared directly with GPT-4 (right bars).

https://doi.org/10.1371/journal.pdig.0000341.g003

t-test results with Bonferroni correction applied showing the superior accuracy and relevance of GPT-4 responses relative to GPT-3.5 responses in the opinion of five fully trained ophthalmologists (positive mean differences favour GPT-4), and χ 2 test showing that GPT-4 responses were preferred to GPT-3.5 responses by every ophthalmologist in their blinded qualitative appraisals.

https://doi.org/10.1371/journal.pdig.0000341.t003

Failure cases exhibit no association with subject, complexity, or human answers.

The LLM failure cases—where every LLM provided an incorrect answer—are summarised in Table 4 . While errors made by LLMs were occasionally similar to those made by trainee ophthalmologists and junior doctors, this association was not consistent ( Table 4 ). There was no preponderance of ophthalmological subject or first or higher order questions in the failure cases, and questions did not share a common theme, sentence structure, or grammatical construct ( Table 4 ). Examination questions are redacted here to avoid breaching copyright and prevent future LLMs accessing the test data during pretraining but can be provided on request.

Summary of LLM failure cases, where all models provided an incorrect answer to the FRCOphth Part 2 mock examination question. No associations were found with human answers, complexity, subject, theme, sentence structure, or grammatic constructs.

https://doi.org/10.1371/journal.pdig.0000341.t004

Here, we present a clinical benchmark to gauge the ophthalmological performance of LLMs, using a source of questions with very low risk of contamination as the utilised textbook is not freely available online [ 17 ]. Previous studies have suggested that ChatGPT can provide useful responses to ophthalmological queries, but often use online question sources which may have featured in LLMs’ pretraining datasets [ 7 , 12 , 15 , 22 ]. In addition, our employment of multiple LLMs as well as fully qualified and training doctors provides novel insight into the potential and limitations of state-of-the-art LLMs through head-to-head comparisons which provide clinical context and quantitative benchmarks of competence in ophthalmology. Subsequent research may leverage our questions and results to gauge the performance of new LLMs and applications as they emerge.

We make three primary observations. First, performance of GPT-4 compares well to expert ophthalmologists and ophthalmology trainees, and exhibits pass-worthy performance in an FRCOphth Part 2 mock examination. PaLM 2 did not attain pass-worthy performance or match expert ophthalmologists’ scores but was within the spread of trainee doctors’ performance. LLMs are approaching human expert-level knowledge and reasoning in ophthalmology, and significantly exceed the ability of non-specialist clinicians (represented here by unspecialised junior doctors) to answer ophthalmology questions. Second, clinician grading of model outputs suggests that GPT-4 exhibits improved accuracy and relevance when compared with GPT-3.5. Development is producing models which generate better outputs to ophthalmological queries in the opinion of expert human clinicians, which suggests that models are becoming more capable of providing useful assistance in clinical settings. Third, LLM performance was consistent across question subjects and types, distributed similarly to human performance, and exhibited comparable agreement between other LLMs and doctors when corrected for differences in overall performance. Together, this indicates that the ophthalmological knowledge and reasoning capability of LLMs is general rather than limited to certain subspecialties or tasks. LLM-driven natural language processing seems to facilitate similar—although idiosyncratic—clinical knowledge and reasoning to human clinicians, with no obvious blind spots precluding clinical use.

Similarly dramatic improvements in the performance of GPT-4 relative to GPT-3.5 have been reported in the context of the North American Ophthalmology Knowledge Assessment Program (OKAP) [ 13 , 15 ]. State-of-the-art models exhibit far more clinical promise than their predecessors, and expectations and development should be tailored accordingly. Results from the OKAP also suggest that improvement in performance is due to GPT-4 being more well-rounded than GPT-3.5 [ 13 ]. This increases the scope for potential applications of LLMs in ophthalmology, as development is eliminating weaknesses rather than optimising in narrow domains. This study shows that well-rounded LLM performance compares well with expert ophthalmologists, providing clinically relevant evidence that LLMs may be used to provide medical advice and assistance. Further improvement is expected as multimodal foundation models, perhaps based on LLMs such as GPT-4, emerge and facilitate compatibility with image-rich ophthalmological data [ 3 , 23 , 24 ].

Limitations

This study was limited by three factors. First, examination performance is an unvalidated indicator of clinical aptitude. We sought to ameliorate this limitation by employing expert ophthalmologists, ophthalmology trainees, and unspecialised junior doctors answering the same questions as clinical benchmarks; and compared LLM performance to real cohorts of candidates in recent FRCOphth examinations. However, it remains an issue that comparable performance to clinical experts in an examination does not necessarily demonstrate that an LLM can communicate with patients and practitioners or contribute to clinical decision making accurately and safely. Early trials of LLM chatbots have suggested that LLM responses may be equivalent or even superior to human doctors in terms of accuracy and empathy, and experiments using complicated case studies suggest that LLMs operate well even outside typical presentations and more common medical conditions [ 4 , 25 , 26 ]. In ophthalmology, GPT-3.5 and GPT-4 have been shown to be capable of providing precise and suitable triage decisions when queried with eye-related symptoms [ 22 , 27 ]. Further work is now warranted in conventional clinical settings.

Second, while the study was sufficiently powered to detect a less than 10% difference in overall performance, the relatively small number of questions in certain categories used for stratification analysis may mask significant differences in performance. Testing LLMs and clinicians with more questions may help establish where LLMs exhibit greater or lesser ability in ophthalmology. Furthermore, researchers using different ways to categorise questions may be able to identify specific strengths and weaknesses of LLMs and doctors which could help guide design of clinical LLM interventions.

Finally, experimental tasks were ‘zero-shot’ in that LLMs were not provided with any examples of correctly answered questions before it was queried with FRCOphth questions from the textbook. This mode of interrogation entails the maximal level of difficulty for LLMs, so it is conceivable that the ophthalmological knowledge and reasoning encoded within these models is actually even greater than indicated by results here [ 1 ]. Future research may seek to fine-tune LLMs by using more domain-specific text during pretraining and fine-tuning, or by providing examples of successfully completed tasks to further improve performance in that clinical task [ 3 ].

Future directions

Autonomous deployment of LLMs is currently precluded by inaccuracy and fact fabrication. Our study found that despite meeting expert standards, state-of-the-art LLMs such as GPT-4 do not match top-performing ophthalmologists [ 28 ]. Moreover, there remain controversial ethical questions about what roles should and should not be assigned to inanimate AI models, and to what extent human clinicians must remain responsible for their patients [ 3 ]. However, the remarkable performance of GPT-4 in ophthalmology examination questions suggests that LLMs may be able to provide useful input in clinical contexts, either to assist clinicians in their day-to-day work or with their education or preparation for examinations [ 3 , 13 , 14 , 27 ]. Further improvement in performance may be obtained by specific fine-tuning of models with high quality ophthalmological text data, requiring curation and deidentification [ 29 ]. GPT-4 may prove especially useful where access to ophthalmologists is limited: provision of advice, diagnosis, and management suggestions by a model with FRCOphth Part 2-level knowledge and reasoning ability is likely to be superior to non-specialist doctors and allied healthcare professionals working without support, as their exposure to and knowledge of eye care is limited [ 27 , 30 , 31 ].

However, close monitoring is essential to avoid mistakes caused by inaccuracy or fact fabrication [ 32 ]. Clinical applications would also benefit from an uncertainty indicator reducing the risk of erroneous decisions [ 7 ]. As LLM performance often correlates with the frequency of query terms’ representation in the model’s training dataset, a simple indicator of ‘familiarity’ could be engineered by calculating the relative frequency of query term representation in the training data [ 7 , 33 ]. Users could appraise familiarity to temper their confidence in answers provided by the LLM, perhaps reducing error. Moreover, ophthalmological applications require extensive validation, preferably with high quality randomised controlled trials to conclusively demonstrate benefit (or lack thereof) conferred to patients by LLM interventions [ 34 ]. Trials should be pragmatic so as not to inflate effect sizes beyond what may generalise to patients once interventions are implemented at scale [ 34 , 35 ]. In addition to patient outcomes, practitioner-related variables should also be considered: interventions aiming to improve efficiency should be specifically tested to ensure that they reduce rather than increase clinicians’ workload [ 3 ].

According to comparisons with expert and trainee doctors, state-of-the-art LLMs are approaching expert-level performance in advanced ophthalmology questions. GPT-4 attains pass-worthy performance in FRCOphth Part 2 questions and exceeds the scores of some expert ophthalmologists. As top-performing doctors exhibit superior scores, LLMs do not appear capable of replacing ophthalmologists, but state-of-the-art models could provide useful advice and assistance to non-specialists or patients where access to eye care professionals is limited [ 27 , 28 ]. Further research is required to design LLM-based interventions which may improve eye health outcomes, validate interventions in clinical trials, and engineer governance structures to regulate LLM applications as they begin to be deployed in clinical settings [ 36 ].

Supporting information

S1 fig. chatgpt performance in questions taken from the whole textbook..

Mosaic plot depicting the overall performance of ChatGPT versions powered by GPT-3.5 and GPT-4 in 360 FRCOphth Part 2 written examination questions. Performance was significantly higher for GPT-4 than GPT-3.5, and was close to mean human examination candidate performance and pass mark set by standard setting and after adjustment.

https://doi.org/10.1371/journal.pdig.0000341.s001

S1 Table. Question characteristics and performance of GPT-3.5 and GPT-4 over the whole textbook.

Similar observations were noted here to the smaller mock examination used for subsequent experiments. GPT-4 performs to a significantly higher standard than GPT-3.5

https://doi.org/10.1371/journal.pdig.0000341.s002

S2 Table. Examination statistics corresponding to FRCOphth Part 2 written examinations sat between July 2017-December 2022.

https://doi.org/10.1371/journal.pdig.0000341.s003

S3 Table. Experience of expert ophthalmologists (E1-E5), ophthalmology trainees (T1-T3), and unspecialised junior doctors (J1-J2) involved in experiments.

https://doi.org/10.1371/journal.pdig.0000341.s004

S4 Table. Results of statistical tests of variation in performance between question subjects and types, for each trialled LLM, expert ophthalmologist, and trainee doctor.

Statistically significant results are highlighted in green.

https://doi.org/10.1371/journal.pdig.0000341.s005

S1 Protocol. Procedures followed by ophthalmologists to grade the output of GPT-3.5 and GPT-4 in terms of accuracy, relevance, and rater-preference of model outputs.

https://doi.org/10.1371/journal.pdig.0000341.s006

Acknowledgments

The authors extend their thanks to Mr Arunachalam Thirunavukarasu (Betsi Cadwaladr University Health Board) for his advice and assistance with recruitment.

• 1. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. Language Models are Few-Shot Learners. In: Advances in Neural Information Processing Systems [Internet]. Curran Associates, Inc.; 2020 [cited 2023 Jan 30]. p. 1877–901. Available from: https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
• 2. OpenAI. GPT-4 Technical Report [Internet]. arXiv; 2023 [cited 2023 Apr 11]. Available from: http://arxiv.org/abs/2303.08774
• View Article
• PubMed/NCBI
• Google Scholar
• 9. Google. PaLM 2 Technical Report [Internet]. 2023 [cited 2023 May 11]. Available from: https://ai.google/static/documents/palm2techreport.pdf
• 17. Ting DSJ, Steel D. MCQs for FRCOphth Part 2. Oxford University Press; 2020. 253 p.
• 21. Part 2 Written FRCOphth Exam [Internet]. The Royal College of Ophthalmologists. [cited 2023 Jan 30]. Available from: https://www.rcophth.ac.uk/examinations/rcophth-exams/part-2-written-frcophth-exam/

• Case report
• Open access
• Published: 23 April 2024

Genetic exploration of Dravet syndrome: two case report

• Agung Triono 1 ,
• Elisabeth Siti Herini   ORCID: orcid.org/0000-0003-2571-8310 1 &

Journal of Medical Case Reports volume  18 , Article number:  215 ( 2024 ) Cite this article

Metrics details

Dravet syndrome is an infantile-onset developmental and epileptic encephalopathy (DEE) characterized by drug resistance, intractable seizures, and developmental comorbidities. This article focuses on manifestations in two Indonesian children with Javanese ethnicity who experienced Dravet syndrome with an SCN1A gene mutation, presenting genetic analysis findings using next-generation sequencing.

Case presentation

We present a case series involving two Indonesian children with Javanese ethnicity whom had their first febrile seizure at the age of 3 months, triggered after immunization. Both patients had global developmental delay and intractable seizures. We observed distinct genetic findings in both our cases. The first patient revealed heterozygous deletion mutation in three genes ( TTC21B , SCN1A , and SCN9A ). In our second patient, previously unreported mutation was discovered at canonical splice site upstream of exon 24 of the SCN1A gene. Our patient’s outcomes improved after therapeutic evaluation based on mutation findings When comparing clinical manifestations in our first and second patients, we found that the more severe the genetic mutation discovered, the more severe the patient’s clinical manifestations.

These findings emphasize the importance of comprehensive genetic testing beyond SCN1A , providing valuable insights for personalized management and tailored therapeutic interventions in patients with Dravet syndrome. Our study underscores the potential of next-generation sequencing in advancing genotype–phenotype correlations and enhancing diagnostic precision for effective disease management.

Peer Review reports

Dravet syndrome (DS), previously known as severe myoclonic epilepsy of infancy (SMEI), is an infantile-onset developmental and epileptic encephalopathy (DEE) characterized by drug resistance, intractable seizures, and comorbidities including intellectual disability, behavioral problems, sleep disturbances, gait disturbances, and an increased risk of sudden unexpected death in epilepsy [ 1 , 2 ]. The incidence of DS is approximately 1 in every 15,700 births [ 3 ]. The first symptom of DS is seizures in the first year of life, followed by developmental delay [ 1 ]. This first seizure is either generalized tonic–clonic or focal (occasionally hemiclonic) clonic, and in more than half of the cases, it is a febrile seizure, making it difficult to distinguish from a self-limiting febrile seizure. Infection, hot environment, exhaust, sunlight, or exercise can initiate an attack of DS [ 4 , 5 ]. Approximately 80% of patients with DS carry a pathogenic variant of the sodium channel alpha 1 subunit ( SCN1A ) gene resulting in haploinsufficiency Nav1.1, the alpha-1 subunit of the sodium channel. PCDH19, SCN2A, SCN8A, SCN1B, GABRA1, GABRB3, GABRG2, KCNA2, CHD2, CPLX1, HCN1A, and STXBP1 variants may also be involved in DS or DS-like phenotypes. Accordingly, genetic testing is required to identify other genes that play a role in the DS phenotype and to expand genotype-DS phenotype correlations to enhance the future management of this disease [ 6 ]. In the last decade, next-generation sequencing (NGS) technology has been able to analyze a set of genes (targeted panel sequencing), exome [(whole exome sequencing (WES)], or genome [whole genome sequencing (WGS)] in a single sequencing process, making it possible to diagnose rare diseases such as early childhood epilepsy [ 7 ]. Identification of the genetic basis of DS can provide additional information regarding pathophysiology, prognosis, and individual drug therapy options according to the patient’s condition.

We present a case series involving two children, one aged 11 years and 2 months, and the other aged 1 year and 4 months. Both children were diagnosed with DS, exhibiting symptoms of intractable seizures, global developmental delay, and seizures triggered by postimmunization fever. Despite displaying similar symptoms, the two individuals possess different genetic variants of the SCN1A gene and also possible novel mutation in DS. We also discuss the main clinical characteristics, treatment course, and management of DS at tertiary referral hospitals in Indonesia.

A boy with Javanese ethnicity aged 11 years and 2 months with uncontrollable seizures regularly visits our hospital. The patient had his first seizure at the age of 3 months with a duration of 15 min, and it was triggered after receiving diphtheria–pertussis–tetanus (DPT) immunization, which was accompanied by fever. The patient has about six to seven seizures per day for 1 min in the form of generalized tonic–clonic and absence seizures. He was the first child of nonconsanguineous healthy parents with normal prenatal and birth history. He has a younger sister with normal development. There is no history of family members with febrile seizure. The patient was born at 40 weeks of gestation, with a birth weight of 4000 g, length of 52 cm, and head circumference of 33 cm. The patient is currently experiencing global developmental delay and is still in kindergarten. He had learning difficulties and was unable to speak words at an age-appropriate level. He had delayed motor development and was unable to perform age-appropriate motor activities. Head circumference was 46.5 cm (microcephaly). There were no signs of meningeal irritation nor Babinski response. The motor examination revealed no increased tone in the upper and lower limb. Other systemic examinations revealed no abnormalities. Interictal electroencephalography (EEG) showed diffuse epileptiform irritative abnormality on a normal background (Fig.  1 ). Magnetic resonance imaging (MRI) of the brain showed cerebral atrophy, bilateral frontal subarachnoid enlargement, bilateral occipital lobe and polymicrogyria, and a neuroglial cyst in the right temporal lobe (Fig.  2 ). He was recommended to get genetic testing done since he was suspected of having DS.

Electroencephalography (EEG) shows diffuse epileptiform irritative abnormality on a normal background

Axial brain magnetic resonance imagery shows cerebral atrophy, bilateral frontal subarachnoid enlargement, bilateral occipital lobe polymicrogyria, and a neuroglial cyst in the right temporal lobe

Whole genome sequencing (WGS), whole exome sequencing (WES), and Sanger sequencing were performed at 3Billion (Seoul, Korea). The WGS and WES procedures were conducted according to the protocols of Richards et al . [ 8 ] and Seo et al . [ 9 ], respectively. Both WES and WGS are comprised of four main parts: (1) high-quality sequencing; (2) sequencing data analysis including alignment to the genome reference consortium human 37 (GRCh37)/hg19 for WES, also alignment to the genome reference consortium human 38 (GRCh38) and revised Cambridge reference sequence (rCRS) of the mitochondrial genome for WGS; (3) variant annotation and prioritization by EVIDENCE [a software that was developed in house to prioritize variants based on the American College of Medical Genetics and Genomics (ACMG) guidelines [ 10 ]]; and (4) variant interpretation in the context of the patient’s symptoms and reporting of disease-causing variants. Once EVIDENCE prioritizes the top candidate variants/genes, 3Billion’s highly-trained clinical/medical geneticists manually curate each variant to identify the disease-causing variant for reporting.

In our initial examination, we performed WES on patient 1 and subsequently identified a copy number variant (CNV), prompting us to proceed with WGS. The WGS analysis revealed a heterozygous pathogenic 552.9 Kb deletion variant in 2q24.3. The heterozygous deletion NC_000002.12:g.165811316_166364199delinsTGTACACTA at 2q24.3 spans across three genes ( TTC21B , SCN1A , and SCN9A ). The variant is not observed in the gnomAD SVs v2.1.1 dataset. SCN1A is subject to haploinsufficiency. Other pathogenic variants have been reported in this region. There are multiple similarly affected individuals reported with similar likely pathogenic copy–number–loss overlapping this region [ 11 , 12 ]. Therefore, this variant was classified as pathogenic. Due to region-spanning mutation in SCN1A, which suitable with clinical manifestation, the patient was diagnosed with DS (OMIM 607208: since we were unable to perform a Sanger sequencing study on both of the parents, the pattern of inheritance is still uncertain.

The arents were counseled about their child’s condition and agreed to undergo multipronged therapy. Before the patient was diagnosed with DS, he had received valproic acid (30 mg/kg per day), phenobarbital (2.5 mg/kg per day), and oxcarbazepine (5 mg/kg per day), also physio, occupation, and speech therapy but had not shown significant improvement. He was seizure-free for 3 months after oxcarbazepine was changed to levetiracetam (27 mg/kg per day). However, the patient then had another episodes of less than 5 minutes general tonic–clonic seizure (GTCS)-induced by fever. Interictal EEG was performed to evaluate his condition, and we found that the diffuse epileptiform irritative abnormality persisted.

A 1 year and 4 month-old-girl with Javanese ethnicity was referred to our hospital due experiencing myoclonic seizure followed by 20 minute GTCS at 3 months, after fever following DPT immunization. She then continued to experience generalized tonic–clonic seizures one to two times per day for 10–15 seconds. At 9 months of age, the patient received a second DPT immunization, and on the same day, she had another generalized tonic–clonic seizure that lasted > 30 minutes, resulting in her admission to the pediatric intensive care unit. Before the first seizure, the patient could lift her head, grasp a toy and make eye contact, but after that, she could neither lift her head nor grasp an object. The patient has no previous history of trauma.

She had a normal head circumference increased physiological reflexes in all extremities. Other systemic examinations revealed no abnormalities. Computed tomography (CT) scan examination of the head showed a subdural hygroma in the right and left frontoparietal region, without any other abnormalities (Fig.  3 ). Electroencephalography (EEG) at the beginning of the seizure did not show any abnormalities, but the EEG follow-up 7 months after the onset of the seizure showed an abnormal epileptiform (spike wave) with a normal background (Fig.  4 ). Thus, she was suspected of having DS and was recommended to undergo genetic examination.

Axial brain computed tomography scan shows a subdural hygroma in the right and left frontoparietal region, without any other abnormalities

Electroencephalography shows abnormal irritative epileptiform with a normal background

Whole exome sequencing (WES) showed a likely pathogenic variant identified as a heterozygous mutation of the SCN1A gene with genomic position 2-166859265-T-C (GRCh37), [NM_001165963.4:C.4003-2A > G [NP_001159435.1:p.?]. The variant is located in the canonical splice site upstream of exon 24 of SCN1A gene (NM_001165963.4 transcript). Since this variant is an essential splicing variant, the protein consequence is uncertain and therefore represented as (p.?). In this patient’s genetic mutation, the canonical junction site occurs which is expected to alter the junction and result in loss or disruption of normal protein function. However, using an in silico predictor, spliceAI ( https://spliceailookup.broadinstitute.org/ ), the variant is predicted to result in a loss of 22 base pairs at end of exon 24. This loss is expected to create a frameshift at the Gly1342 position. Sanger sequencing confirmed the patient’s genotype (Fig.  5 A), but the mother’s Sanger analysis was negative (Fig.  5 B). Due to familial issues, Sanger sequencing was not performed on the father, leaving the inheritance pattern unresolved.

A Sanger sequencing result of patient 2 showed a heterozygous mutation of the SCN1A gene with the genomic position 2-166859265-T-C (GRCh37), [NM_001165963.4:C.4003-2A >G [NP_001159435.1:p.?] (red arrow); and B Sanger sequencing result of patient 2’s mother showed normal sequence

The parents were counseled about their child’s condition and agreed to undergo multipronged therapy. Before patient was diagnosed with DS, she received clonazepam (0.01 mg/kg per day), valproic acid (29 mg/kg per day), and phenytoin (5 mg/kg per day), but seizure persisted. When phenytoin was stopped, with valproic acid (30 mg/kg per day) and clonazepam (0.04 mg/kg per day) adjusted, seizures were greatly decreased. Later, patient only experienced one seizure per year. The patient routinely received physio, speech, and occupational therapy.

When comparing the clinical features and outcomes of the two patients (Table  1 ), we found that our first patient, who had three medications, was still having a generalized seizure induced by fever with duration less than 5 minutes after they had been seizure-free for 3 months (at the age 11 years and 8 months. Our second patient, however, only experienced one seizure annually after receiving two medications (at the age 1 year and 10 months). This difference implies that the clinical state of the first patient was worse than that of the second.

Research on the identification of DS genetic mutations using NGS has never been done in Indonesia. In 2010, we conducted a study to identify pathogenic variants of the SCN1A gene using the Sanger sequencing method and successfully reported cases of novel SCN1A mutations in Indonesia in patients with severe myoclonic epilepsy in infancy (SMEI) and borderline SMEI (SMEB). The first boy identified with SMEI experienced a variety of seizures, including his first febrile seizure and general tonic–clonic seizure at 7 months of age, and later suffered from myoclonic seizures, left-sided hemiconvulsions, also focal convulsions without fever, along with delayed speech development. The second patient with SMEB had his first febrile seizures with GTCS after immunization at 3 months old, then later on experienced status epilepticus, GTCS, and atonic convulsions without fever [ 13 ]. We also conducted another research on the spectrum of generalized epilepsy with febrile seizure plus (GEFS+) focusing on clinical manifestations and SCN1A gene mutations. That study analyzed a total of 34 patients who suffered from SMEI (7 patients), SMEB (7 patients), febrile seizure plus (FS+) and absence/myoclonic/atonic/partial seizures (11 patients), and FS+ (9 patients) [ 14 ].

However, the research that we have done uses the Sanger sequencing genetic examination, which is expensive and takes considerable time. Additionally, it is unable to find any other gene besides SCN1A in patients with DS. A study by Djémié et al . in Belgium reported the discovery of 28 pathogenic variants of the SCN1A gene using the NGS method which were previously missed or undiagnosed using Sanger sequencing [ 7 ]. To link DS cases more effectively, we are attempting to conduct NGS genetic tests, specifically WES and WGS.

Dravet syndrome (DS) was infrequently reported in Indonesia due to its difficulty in diagnosis, misdiagnosis as febrile seizures or other epilepsy syndromes, or lack of follow-up and genetic testing in our country. According to the to the International League Against Epilepsy (ILAE) [ 15 ], the diagnostic criteria for this condition should consist of a number of the following symptoms: (1) a family history of epilepsy or febrile seizures; (2) normal development before seizures onset; (3) seizure before 1 year of age; (4) EEG with generalized spike and polyspike waves; (5) pleomorphic epilepsy (myoclonic, focal, clonic, absence, and generalized seizures); (6) focal abnormalities or early photosensitivity; (7) psychomotor retardation after 24 months; (8) exacerbation of seizures with increased body temperature; and (9) the appearance of subsequent ataxia, pyramidal signs or interictal myoclonus after the beginning of psychomotor slowing. Both of our patients had seizures beginning with increased body temperature and regression of development after seizure onset, which were resistant to the majority of anticonvulsant medications. The seizures began as generalized tonic–clonic seizures, followed by absence seizures. Both of our patients also experienced subsequent ataxia and pyramidal signs. Thus, they were suspected of having DS and were advised to undergo genetic testing.

Infants with DS have normal physical and psychomotor development at the time of their first seizure, which typically occurs between the ages of 5 and 8 months. In our case series, both of our patients experienced their first seizure at the age of 3 months [ 16 , 17 ]. In the first year of life, the most common form of seizure is febrile tonic–clonic. Some patients may experience myoclonic and dyscognitive seizures infrequently. Frequently, protracted seizures result in status epilepticus. In the first year of life, seizures are precipitated by fever/illness, immunization, and cleansing [ 16 ]. As the infant develops, he or she will experience a variety of seizure types, as well as fever and emotional stress, flashes of light, and overexertion being seizure triggers. The child with DS will develop hypotonia, ataxia, incoordination, and pyramidal signs, dysautonomia events, cognitive impairment, and behavioral disturbances such as attention deficit, hyperactivity, or autistic characteristics [ 15 ]. Some of the conditions above are very consistent with what happened to our patients.

The EEG performed during the early phases of the disease is normal. However, as the child grows, generalized spike waves with isolated or brief discharges of fast polyspike waves may be present [ 15 , 18 ]. In the first case, we found diffuse epileptiform irritative abnormality with a normal background, whereas in the second case, initially it was found normal, then a few months later it became abnormal irritative epileptiform with a normal background.

Genetic testing is developing rapidly and playing a significant role in the specific diagnosis and management of epilepsy [ 19 , 20 ]. Several genes with pathogenic mutations produce DS or DS-like phenotypes, which inevitably require different drug therapy approaches. Genes that cause DS can be grouped based on how they work: specifically, three sodium channel-related genes ( SCN2A, SCN8A , and SCN1B ), one potassium channel-related gene ( KCNA2 ), three gamma-aminobutyric acid receptors ( GABAR ) genes ( GABRA2, GABRB3 , and GABRG2 ), a cyclic nucleotide gated cation channel gene ( HCN1 ), and other functional genes including CHD2, CPLX1 , and STXBP1 . Approximately 80% of patients with DS have a pathogenic variant of the SCN1A gene, from which the majority of SCN1A variants are de novo, but 10% of people inherit the SCNA1 mutation from one or both parents [ 6 ]. Both of our patients had a mutation in the SCN1A gene, which is the most common mutation seen in DS.

Furthermore, TTC21B and SCN9A mutations were also found in our first patient. A study conducted by Suls et al . also reported a four generation Bulgarian family with epilepsy, revealing a heterozygous 400 kb deletion on chromosome 2q24 that included the SCN1A and TTC21B genes [ 21 ]. The patients exhibited variable phenotypes, but all experienced generalized tonic–clonic seizures around the first year of life, with some presenting myoclonic or absence seizures. Febrile seizures occurred in three of the four patients during infancy. Notably, one patient had mild mental retardation, another had psychomotor slowing, and a third had mental retardation from early infancy; all showed reduced seizures on medication. The findings in that study parallel the situation observed in our initial patient case. Meanwhile, a study by Singh et al . identified a heterozygous mutation in the SCN9A gene in two patients diagnosed with DS [ 22 ]. One of these patients also exhibited a mutation in the SCN1A gene. The study provided evidence suggesting that the SCN9A gene on chromosome 2q24 could potentially serve as a modifier for DS. Among 109 patients with DS, 8% were found to have an SCN9A mutation. This included six patients with double heterozygosity for SCN9A and SCN1A mutations and three patients with only heterozygous SCN9A mutations, supporting the notion of a multifactorial inheritance pattern [ 22 ]. The previous research confirmed the severity of clinical symptoms in our first patient, whom we identified mutations in the SCN1A, SCN9A , and TTC21B genes.

In the last decade, there has been a very rapid development of neurogenetic science and diagnostic technology. NGS is the latest method of genetic examination that allows for the discovery of causal mutations, including de novo, novel, and familial mutations related to epilepsy syndromes that have variable phenotypic features [ 23 ]. The first generation of DNA sequencing using the Sanger method could only examine one gene at a time and had limitations especially when examining large genomic regions, so the NGS method is more widely used today [ 7 , 23 ]. A study conducted by Kim et al . in Seoul reported an increase in diagnostic yield using WES after targeted panel sequencing with negative results in infantile onset epilepsy by 8%. This result suggests that WES assays increase the opportunity to search for new epilepsy genes and uncover less well-known epileptic phenotypes from known neurological diseases [ 24 ]. The WES examination also allows for the discovery of de novo or inherited mutations if the patient and both parents are examined [ 25 ].

According to the recommendations of the North American consensus panel, clobazam and valproic acid are the first-line therapies for antiepileptic drugs, followed by stiripentol, topiramate and levetiracetam. Patients with a suboptimal response to clobazam and valproic acid have been advised to consider the ketogenic diet as a second-line treatment [ 17 ]. SCN1A is a gene that codes for sodium channel channels, so drugs that work as sodium channel blockers, such as lamotrigine, phenytoin, carbamazepine, oxcarbazepine, lacosamide, and rufinamide, are contraindicated in patients with DS because they can increase the frequency of seizures [ 4 ]. After the failure of first- and second-line therapy, surgical therapies, such as vagus nerve stimulation (VNS), were moderately agreed upon and should be considered [ 17 ]. Besides medication, controlling infections and body temperature variations also showed to decrease the frequency of seizures and severity of the disease [ 18 ]. Initially, the first patient received oxcarbazepine and the second patient got phenytoin, which had been contraindicated to patients with DS. Futhermore, after eliminating medications that were contraindicated, both patients’ outcome improved.

In this study, we discovered unique mutations that have never been documented before, particularly in Indonesia, where NGS analysis of DS genetic variants has never been done. However, the limitation of this study, is that the information comes from two cases only. Further research is needed to explore more cases from Indonesia population.

In summary, our case series utilizing next-generation sequencing (NGS) unveils the intricate genetic landscape of Dravet syndrome (DS) in two Indonesian pediatric cases. By using WGS and WES, we identified distinct mutations in the SCN1A gene, as well as contributions from genes, such as TTC21B and SCN9A . The power of WGS lies in its ability to uncover rare pathogenic variants, including a 552.9 Kb deletion in the 2q24.3 region. These findings emphasize the importance of comprehensive genetic testing beyond SCN1A , providing valuable insights for personalized management and tailored therapeutic interventions in patients with DS. Our study underscores the potential of NGS in advancing genotype–phenotype correlations and enhancing diagnostic precision for effective disease management. Furthermore, we found that the clinical condition of the first patient was worse than that experienced by the second patient. This difference suggests that the more severe the genetic mutation detected, the more severe the clinical manifestations of the patient.

Availability of data and materials

The dataset used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

American College Of Medical Genetics

Copy number variant

Computed tomography

• Dravet syndrome

Developmental and epileptic encephalopathy

Electroencephalography

Febrile seizure plus

Generalized epilepsy with febrile seizure plus

Genome reference consortium human 37

Genome reference consortium human 38

General tonic clonic seizure

International league against epilepsy

Magnetic resonance imaging

• Next-generation sequencing

Revised Cambridge reference sequence

Sodium channel alpha 1 subunit

Severe myoclonic epilepsy of infancy-borderline

Severe myoclonic epilepsy in infancy

Vagus nerve stimulation

Whole-exome sequencing

Whole-genome sequencing

Zuberi SM, Wirrell E, Yozawitz E, Wilmshurst JM, Specchio N, Riney K, et al . ILAE classification and definition of epilepsy syndromes with onset in neonates and infants: position statement by the ILAE Task Force on Nosology and Definitions. Epilepsia. 2022;63(6):1349–97.

Article   PubMed   Google Scholar  

Wirrell EC, Hood V, Knupp KG, Meskis MA, Nabbout R, Scheffer IE, et al . International consensus on diagnosis and management of Dravet syndrome. Epilepsia. 2022;63(7):1761–77.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Isom LL, Knupp KG. Dravet syndrome: novel approaches for the most common genetic epilepsy. Neurotherapeutics. 2021;18(3):1524–34.

Cardenal-Muñoz E, Auvin S, Villanueva V, Cross JH, Zuberi SM, Lagae L, et al . Guidance on Dravet syndrome from infant to adult care: road map for treatment planning in Europe. Epilepsia Open. 2022;7(1):11–26.

Chen C, Fang F, Wang X, Lv J, Wang X, Jin H. Phenotypic and genotypic characteristics of SCN1A associated seizure diseases. Front Mol Neurosci. 2022;28(15): 821012.

Article   Google Scholar  

Ding J, Wang L, Jin Z, Qiang Y, Li W, Wang Y, et al . Do all roads lead to Rome? Genes causing Dravet syndrome and Dravet syndrome-like phenotypes. Front Neurol. 2022;11(13): 832380.

Djémié T, Weckhuysen S, Von Spiczak S, Carvill GL, Jaehn J, Anttonen A, et al . Pitfalls in genetic testing: the story of missed SCN1A mutations. Mol Genet Genomic Med. 2016;4(4):457–64.

Article   PubMed   PubMed Central   Google Scholar  

Richards S, Aziz N, Bale S, Bick D, Das S, Gastier-Foster J, et al . Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med. 2015;17(5):405–24.

Seo GH, Kim T, Choi IH, Park J, Lee J, Kim S, et al . Diagnostic yield and clinical utility of whole exome sequencing using an automated variant prioritization system, EVIDENCE . Clin Genet. 2020;98(6):562–70.

Riggs ER, Andersen EF, Cherry AM, Kantarci S, Kearney H, Patel A, et al . Technical standards for the interpretation and reporting of constitutional copy-number variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics (ACMG) and the Clinical Genome Resource (ClinGen). Genet Med. 2020;22(2):245–57.

Lim BC, Hwang H, Kim H, Chae JH, Choi J, Kim KJ, et al . Epilepsy phenotype associated with a chromosome 2q243 deletion involving SCN1A: migrating partial seizures of infancy or atypical Dravet syndrome? Epilepsy Res. 2015;109:34–9.

Article   CAS   PubMed   Google Scholar  

Fry AE, Rees E, Thompson R, Mantripragada K, Blake P, Jones G, et al . Pathogenic copy number variants and SCN1A mutations in patients with intellectual disability and childhood-onset epilepsy. BMC Med Genet. 2016;17(1):34.

Herini ES, Gunadi, Van Kempen MJA, Yusoff S, Sutaryo, Sunartini, et al. Novel SCN1A mutations in Indonesian patients with severe myoclonic epilepsy in infancy. Pediatr Int. 2010;52(2):234–9.

Herini ES, Gunadi, Harahap ISK, Yusoff S, Morikawa S, Patria SY, et al. Generalized epilepsy with febrile seizures plus (GEFS+) spectrum: clinical manifestations and SCN1A mutations in Indonesian patients. Epilepsy Res. 2010;90(1–2):132–9.

Anwar A, Saleem S, Patel UK, Arumaithurai K, Malik P. Dravet syndrome: an overview. Cureus. 2019. https://www.cureus.com/articles/20900-dravet-syndrome-an-overview . Accessed 20 Nov 2023.

Brunklaus A, Dorris L, Ellis R, Reavey E, Lee E, Forbes G, et al . The clinical utility of an SCN1A genetic diagnosis in infantile-onset epilepsy. Dev Med Child Neurol. 2013;55(2):154–61.

Wirrell EC, Laux L, Donner E, Jette N, Knupp K, Meskis MA, et al . Optimizing the diagnosis and management of Dravet syndrome: recommendations from a North American Consensus Panel. Pediatr Neurol. 2017;68:18-34.e3.

Yadav R, Shah S, Bhandari B, Marasini K, Mandal P, Murarka H, et al . Patient with Dravet syndrome: a case report. Clin Case Rep. 2022;10(5): e05840.

Møller RS, Dahl HA, Helbig I. The contribution of next generation sequencing to epilepsy genetics. Expert Rev Mol Diagn. 2015;15(12):1531–8.

Yozawitz E, Moshé SL. The influence of genetics on epilepsy syndromes in infancy and childhood. Acta Epileptol. 2022;4(1):41.

Suls A, Velizarova R, Yordanova I, Deprez L, Van Dyck T, Wauters J, et al . Four generations of epilepsy caused by an inherited microdeletion of the SCN1A gene. Am Acad Neurol. 2010;75(72):72–6.

CAS   Google Scholar  

Singh NA, Pappas C, Dahle EJ, Claes LRF, Pruess TH, De Jonghe P, et al . A role of SCN9A in human epilepsies, as a cause of febrile seizures and as a potential modifier of Dravet syndrome. PLoS Genet. 2009;5(9): e1000649.

Dunn P, Albury CL, Maksemous N, Benton MC, Sutherland HG, Smith RA, et al . Next generation sequencing methods for diagnosis of epilepsy syndromes. Front Genet. 2018;7(9):20.

Kim SY, Jang SS, Kim H, Hwang H, Choi JE, Chae J, et al . Genetic diagnosis of infantile-onset epilepsy in the clinic: application of whole-exome sequencing following epilepsy gene panel testing. Clin Genet. 2021;99(3):418–24.

Poduri A, Sheidley BR, Shostak S, Ottman R. Genetic testing in the epilepsies—developments and dilemmas. Nat Rev Neurol. 2014;10(5):293–9.

Download references

Acknowledgements

The authors express their gratitude to the patient and their families for their cooperation, as well as to all the staff and nurses who provided care for the patient. We are also thankful for the Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada for funding this research and providing English editing services for assistance in the editing and proofreading process. Additionally, we appreciate the assistance of Kristy Iskandar, Marissa Leviani Hadiyanto and Khansadhia Hasmaradana Mooiindie during the data collection and editing phases.

This study was supported by the Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, (Dana Masyarakat to ESH). The funding body did not influence the study design, data analysis, data interpretation, nor manuscript writing.

Author information

Authors and affiliations.

Department of Child Health, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Dr. Sardjito Hospital, Jl. Kesehatan No. 1, Yogyakarta, 55281, Indonesia

Agung Triono & Elisabeth Siti Herini

Pediatric Surgery Division, Department of Surgery/Genetics Working Group/Translational Research Unit, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Dr. Sardjito Hospital, Yogyakarta, 55281, Indonesia

You can also search for this author in PubMed   Google Scholar

Contributions

ESH, AG, and G made substantial contributions to the conception and design of the work. AG contributed to data acquisition. ESH, AG, and G performed the data analyses and the interpretation of the data. ESH and AG drafted the text and prepared the figures. ESH, AG, and G revised, read, and approved the final manuscript. All authors approve the present version for publication, and are accountable for all aspects related to the study.

Corresponding author

Correspondence to Elisabeth Siti Herini .

Ethics declarations

Ethics approval and consent to participate.

The Medical and Health Research Ethics Committee of the Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia approved all recruitment and research protocols. Eligible patients signed an informed consent form (ethical clearance number KE-FK-0455-EC-2023; dated March 2023). Patients older than 12 years old and/or their parents or guardian (for patients < 12 years old) signed a written informed consent form to be included in this study.

Consent for publication

Written informed consent was obtained from the patients’ legal guardians for publication of this case report and any accompanying images.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Triono, A., Herini, E.S. & Gunadi Genetic exploration of Dravet syndrome: two case report. J Med Case Reports 18 , 215 (2024). https://doi.org/10.1186/s13256-024-04514-2

Download citation

Received : 31 January 2024

Accepted : 18 March 2024

Published : 23 April 2024

DOI : https://doi.org/10.1186/s13256-024-04514-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Case series

Journal of Medical Case Reports

ISSN: 1752-1947

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 22 April 2024

Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán

• Andrew J. Charlton-Perez   ORCID: orcid.org/0000-0001-8179-6220 1 ,
• Helen F. Dacre   ORCID: orcid.org/0000-0003-4328-9126 1 ,
• Simon Driscoll 1 , 2 ,
• Suzanne L. Gray   ORCID: orcid.org/0000-0001-8658-362X 1 ,
• Ben Harvey   ORCID: orcid.org/0000-0002-6510-8181 1 , 3 ,
• Natalie J. Harvey 1 ,
• Kieran M. R. Hunt   ORCID: orcid.org/0000-0003-1480-3755 1 , 3 ,
• Robert W. Lee   ORCID: orcid.org/0000-0002-1946-5559 1 ,
• Ranjini Swaminathan 1 , 2 ,
• Remy Vandaele 1 , 2 &
• Ambrogio Volonté   ORCID: orcid.org/0000-0003-0278-952X 1 , 3  

npj Climate and Atmospheric Science volume  7 , Article number:  93 ( 2024 ) Cite this article

29 Altmetric

Metrics details

• Atmospheric dynamics
• Mathematics and computing

There has been huge recent interest in the potential of making operational weather forecasts using machine learning techniques. As they become a part of the weather forecasting toolbox, there is a pressing need to understand how well current machine learning models can simulate high-impact weather events. We compare short to medium-range forecasts of Storm Ciarán, a European windstorm that caused sixteen deaths and extensive damage in Northern Europe, made by machine learning and numerical weather prediction models. The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of the warm conveyor belt jet, and the large-scale dynamical drivers important for the rapid storm development such as the position of the storm relative to the upper-level jet exit. However, their ability to resolve the more detailed structures important for issuing weather warnings is more mixed. All of the machine learning models underestimate the peak amplitude of winds associated with the storm, only some machine learning models resolve the warm core seclusion and none of the machine learning models capture the sharp bent-back warm frontal gradient. Our study shows there is a great deal about the performance and properties of machine learning weather forecasts that can be derived from case studies of high-impact weather events such as Storm Ciarán.

Similar content being viewed by others

FuXi: a cascade machine learning forecasting system for 15-day global weather forecast

Lei Chen, Xiaohui Zhong, … Hao Li

Skilful nowcasting of extreme precipitation with NowcastNet

Yuchen Zhang, Mingsheng Long, … Jianmin Wang

Accurate medium-range global weather forecasting with 3D neural networks

Kaifeng Bi, Lingxi Xie, … Qi Tian

Introduction

During the 20th century and the first two decades of the 21st century, numerical weather prediction (NWP) transformed atmospheric science 1 . The combination of physical and mathematical understanding, the availability of high-performance computing and the expansion of the network of Earth system observation led to remarkable and continued progress in the skill and availability of weather forecasts. Numerical weather predictions are a ubiquitous part of modern life, with applications on many different timescales and in sectors as diverse as transport, agriculture, healthcare and recreation.

Over the last two years, machine learning (ML) techniques, a subset of the rapidly developing field of artificial intelligence (AI), have begun to be applied to the weather prediction problem in earnest. Whilst ML has had applications in climate science for many decades 2 , 3 , 4 , with these communities aware of its potential 5 , and is increasingly used for post-processing weather forecasts 6 , 7 , recent advances in ML and advancements in GPUs (Graphics Processing Units), have enabled the beginning of a ‘new dawn’ in the application of ML and AI techniques to weather and climate prediction 8 .

The publication of the WeatherBench dataset 9 and the 10-year roadmap for ML use by the European Centre for Medium-Range Weather Forecasts (ECMWF) 10 , amongst other developments, stimulated interest and investment in the development of ML models for weather forecasting. During 2022 and 2023, four ML models were developed by major technology companies to address the short to medium-range (0–10 day) forecasting problem. These models have all been shown to produce skillful 0–10 day forecasts of the 500 hPa geopotential height field, based on the widely used Anomaly Correlation Coefficient metric 11 . All four models use an encode-process-decode framework but with differing architectures:

FourCastNet 12 , developed by NVIDIA and based on Fourier Neural Operators (FNO) with a vision transformer architecture;

FourCastNet version 2 13 , which builds on FourCastNet by using spherical FNOs;

Pangu-Weather 14 , developed by Huawei and based on a three-dimensional Earth-specific transformer and hierarchical temporal aggregation; and

GraphCast 15 , developed by Google DeepMind and based on graph neural networks.

Similar techniques have been used to develop models for other forecast tasks (e.g., MetNet-3 for 12-h precipitation forecasts in the contiguous United States and 27 European countries 16 ). At the present time, ML models primarily produce deterministic forecasts, but rapid progress is being made in producing fully probabilistic forecasts 17 , 18 , 19 . All four ML models are extremely efficient when run on GPU or TPU (tensor processing unit) devices, typically producing 10-day forecasts in a few minutes.

Given the infancy of ML model weather prediction, to the author’s knowledge, there are no prior studies that compare how the four ML-models and NWP models capture individual, impactful weather events. Examination of individual weather events available from the papers that describe the ML models is limited to qualitative comparisons of the simulation of tropical cyclones and atmospheric rivers by FourCastNet 12 and quantitative assessment of the tracking error of tropical cyclones by Pangu-Weather 14 and GraphCast 15 . There are no published studies that examine ML model forecasts of extratropical windstorms 20 , despite their potential to cause multi-billion dollar damages 21 and increasing severity under climate and population change 22 .

In this study, we, therefore, seek to advance knowledge of the comparative performance of ML and NWP models by comparing their forecasts of Storm Ciarán, which affected several European countries during November 2023. This is a valuable out-of-sample test for the ML models because their training datasets all end before the beginning of 2023. We compare the ability of the models to capture the detailed physical structure of the storm and its impacts at two lead times over which operational weather forecasters were actively engaged in issuing weather warnings to the public. An accurate description of the physical structure of this, or any other, storm is a key component of forecasting its compound impact 23 and in constructing plausible storylines for end-users 24 .

Storm Ciarán and its associated impacts

Storm Ciarán was first seen as a low-pressure weather system south of Newfoundland at about 00 UTC on 31 October 2023. Based on surface analysis charts issued by the UK Met Office, it then tracked quickly across the North Atlantic, undergoing explosive deepening from 988 hPa at 00 UTC on 1 November to 954 hPa at 00 UTC on 2 November at which time it was located to the northeast of France. This deepening rate, 34 hPa in 24 h means that Ciarán was an extratropical cyclone “bomb” 25 . The lowest pressure recorded, 953 hPa at 06 UTC on 2 November, is a record low pressure for a November storm observed in England 26 . Figure 1 shows surface observations of the 10-m wind speed, cloud cover and mean sea level pressure (MSLP). The cyclonic circulation around the storm centre (with the lowest MSLP observed on the English south coast near the Isle of Wight) has a maximum wind speed of 65 knots on the Normandy coast in France.

The observations are shown as simplified station circles using conventional notation 54 . Circle shading indicates cloud cover in octas, wind barbs and feathers indicate wind speed in knots with the wind direction towards the circle, and numbers are the last three digits, including a decimal place, of the MSLP (in hPa) e.g., 543 equates to 954.3 hPa. Some thinning of observations has been performed for clarity and note that two ships both reported at 51.1 o N, 1.7 o E with different wind speeds and directions.

Although Storm Ciaran was not a classic Shapiro-Keyser cyclone 27 , clear banding in the vicinity of the tip of the cloud as it encircles the storm centre to the poleward side (called the cloud head) could be seen in satellite imagery before it made landfall in northern France. This banding suggests that a sting jet may have been present in Storm Ciarán 28 , however its identification requires methodologies beyond the scope of this study. Gusts of over 100 knots (51 m s -1 ) were reported in several locations in Brittany 29 , with a maximum of 111.7 knots (57.5 m s -1 ) recorded at Pointe du Raz at approximately 0200 UTC on 2 November 30 .

Across Northern Europe, at least 16 people were killed 31 . All flights were cancelled from Amsterdam Schiphol Airport and there were numerous cancellations from Spanish airports. An estimated 1.2 million households in northern France were left without electricity 32 and more than 1 million residents were cut off from the mobile telephone network. Brest and Quimper Airports were also shut and there was disruption to Eurostar operations 33 .

Approximately 10,000 homes in Cornwall were left without power, hundreds of schools were closed and many train services were disrupted by fallen trees. Gusts in Channel Islands ranged from 70-90 knots (36-46 m s -1 ) 32 with a maximum gust of 90 knots (46 ms -1 ) recorded in Alderney at approximately 08 UTC on 2 November 34 . Jersey also experienced a T6 tornado with estimated winds in the region of 161-186 mph (71–83 m s -1 ). Its 8-km track left a trail of destruction and tens of people needed to leave their homes. It is likely that this is the strongest tornado reported in the British Isles since the Gunnersbury tornado in December 1954 35 .

The 10-m wind speed and MSLP structure of Ciarán are shown in Fig. 2a, b at the times when it impacted the land: 00 and 06 UTC 2 November 2023. State-of-the-art model analyses, such as the IFS analysis used in this figure, represent the best three-dimensional estimates of the actual atmospheric state. The low-pressure centre of the storm tracked along the southern UK coast and the strongest winds (turning cyclonically) occurred in an arc in the southwest quadrant of the storm when the strong winds impacted Brittany and later more directly to the south of the low centre when they impacted the Channel Islands. The 10-m winds weakened significantly over land due to surface friction, no longer reaching the threshold for shading in the figure. They also weakened between the two times shown, with the peak winds falling by about 6 ms -1 , likely due to a combination of the storm making landfall and having already reached its mature stage. The observed wind speeds, shown by the overplotted colour-filled circles, are consistent with the analysed fields, away from the coastlines but exceed those analysed in some locations, notably some coastal locations and at the narrowest point of the English channel. The winds in these locations will be influenced by local mesoscale processes and so these exceedances are not unexpected given the resolution of the IFS model. The track of the storm, defined as the locations of its minimum MSLP according to IFS analyses, is shown by the black symbols joined by lines in panel (c). Ciarán had its genesis in the western North Atlantic around the time of the first track point shown (06 UTC 31 October) and travelled rapidly eastwards across the North Atlantic. The contours shown illustrate the MSLP and 250-hPa wind speed (i.e., the upper-level jet) at the start, middle and end times of the tracks and show how Ciarán evolved from a weak disturbance (with central MSLP exceeding 995 hPa) to a record-breaking deep storm as it crossed from the equatorward to the poleward side of the jet at about 06 UTC 1 November.

a , b Maps of 10-m wind speed (shading) and MSLP (contours) at a 00 UTC and b 06 UTC 2 November 2023 from the IFS analysis. Synoptic wind observations above 20 m s –1 are shown as coloured dots. c Six-hourly track points from the IFS analysis (blacks squares joined by lines) and the IFS HRES forecasts and AI models (coloured symbols as in Fig. 3a , b ) from 06 UTC 31 October to 06 UTC 06 UTC 2nd November 2023 (left to right) together with partial MSLP (grey contours in hPa) and 250-hPa wind speed (colour-filled contours) from the IFS analysis at 06 UTC on 31 October, 1 November and 2 November (left to right). The locations of the jet maxima at the longitude points of the MSLP minima at each time are indicated by the cyan squares connected by lines.

Track, intensification and wind impacts of Storm Ciarán

Ciarán’s track was well forecast by both the IFS HRES and ML-models (I. 2(c)) initialised at 00UTC on 31 October, although small differences in the location of the storm centre, and associated wind field, were critical for the accurate predictions of weather warnings along the southern English coast. Two days before Ciarán began to impact land and well before the start of its fast intensification, the spread in the position of the storm in ML models and NWP models is similar.

The evolution of the minimum mean sea pressure (MSLP) at the centre of the developing storm and its associated maximum 10-m wind speed are shown in Fig. 3 for the IFS analysis, IFS HRES forecast and ML model forecasts in panels (a, c), and for the ERA5 reanalysis, forecasts based on the ERA5 system, and the control (unperturbed) ensemble members of four NWP models in panels (b, d). Considering first the minimum (MSLP) evolution, all the forecasts closely follow both analysis products, capturing both the rapid deepening phase of the storm and its maximum intensity depth. The minimum MSLP at the end of the forecast (06 UTC 2 November) is 954 hPa in both the IFS analysis and ERA5. This value varies between 951 and 955 hPa for the ML models and between 950 and 953 hPa for the six NWP models (including the IFS HRES). In contrast, the spread in the maximum wind speed evolution is far greater. At the time of peak wind speed in both analyses, 00 UTC 2 November (48-h lead time), the value in the IFS analysis is 34 m s -1 . The IFS HRES forecast predicts this well (36 m s -1 ), while the other, generally slightly coarser resolution, NWP models mostly forecast slightly weaker winds (30-37 m s -1 ) with the NCEP model being a clear outlier, predicting winds of 40 m s -1 . The wind speeds forecast by the ML models are far too weak (25–26 m s -1 ), even in comparison with the analysis from ERA5 (30 m s -1 ). The ML models failed to capture the rapid intensification of the winds after about 06 UTC on 1 November (30-h lead time). Forecasts made using the ERA5 analysis system do not suffer from this low wind bias and so the underestimation is unlikely to be the result of training the ML models on the ERA5 data. The economic loss resulting from strong surface winds is often assumed to scale as the cube of normalised wind gust speed over a threshold (such as the 98 th percentile value) 36 , so even a small underestimation in predicted wind speed can be significant in terms of the subsequent losses.

a , c minimum MSLP and b , d maximum 10-m wind speed. a , b IFS analysis, IFS HRES forecast and forecasts from the four ML models. a , d ERA5, forecasts made from the ERA5 system and forecasts from control members of the ensemble forecasts from IFS (IFS ens_cntl), the Met Office (UKMO ens_cntl), the Japan Meteorological Agency (JMA ens_cntl) and National Centers for Environmental Prediction (NCEP ens_cntl). Note that the ECMWF HRES and IFS control members use the same model and resolution but are not bit-identical for technical computational reasons.

The differences in maximum 10-m wind speed are explored further in Fig. 4 which shows maps of the 10-m wind speed and MSLP for ERA5, the IFS HRES forecast and the four ML-models valid at 00 UTC 2 November, the time of peak wind speed in both analyses and when the strong winds made landfall in France. All the forecasts were initialised 48 h prior to this time (as for the data shown in Fig. 3 ). These maps can be compared directly with the IFS analysis fields shown in Fig. 2a . The region of strong winds is located in an arc in the region of the tight MSLP gradient in the southwest quadrant of the Ciarán in all seven maps. However, the ML models fail to predict the strongest winds in a band following the isobars (contours of constant MSLP) in the region of the tightest MSLP gradient, as is seen in the IFS HRES forecast, ERA5 and the IFS analysis. It is notable that, despite all the ML models being trained on ERA5, they fail to capture the structure and magnitude of the winds in ERA5 (including in forecasts made using the ERA5 system, as shown in Supp. Fig. 1a ) for this storm, implying that the far weaker winds found for the ML models compared to the NWP forecasts and IFS analysis are not simply a consequence of them being trained on a coarser resolution dataset. Note the NWP models used in Fig. 2 have a similar resolution to ERA5 (equivalent grid spacings of ERA5 ~ 31 km, Met Office ~20 km, JMA ~ 27 km, NCEP ~ 25 km) with the exception of the IFS ( ~ 9 km).

Maps of 10-m wind speed (shading) and MSLP (contours) from a ERA5 and b – f forecasts, initialised at 00 UTC 31 October 2023, from the b IFS HRES model and c – f ML models, as labelled.

Dynamical structure of Storm Ciarán

In this section, we evaluate the dynamics of Storm Ciarán during the final stage of its rapid development with a focus on the formation of strong winds at low altitudes. We compare the predictive capability of the ML models by comparing with the IFS HRES forecast and ERA5. The ML models are all trained on ERA5 and have the same resolution as the output provided for ERA5, allowing a fair comparison of model performance. The forecasts are all initialised at 00 UTC on 1 November, during the onset of Ciarán’s rapid intensification phase (Fig. 3a ). They were evaluated 18 h later (Fig. 5 ) and 24 h later (Fig. 6 ), when Storm Ciarán’s peak wind speeds were observed. By shifting the focus to these short lead times from the previous section, the aim is to highlight both the similarities in and differences between, the NWP and ML forecasts on timescales relevant for refining hazard warnings. To aid the reader, some key parts of the storm structure are labelled in Fig. 5a .

Maps show wind speed at 850 hPa (shading), wind speed at 250 hPa (65 ms –1 , cyan contour with high values in the bottom left of the panels), the wet-bulb potential temperature at 850 hPa (dark blue, light blue, light red and dark red contours indicating values increasing every 2.5 K from 280 K to 287.5 K), MSLP (thin grey contours), relative humidity with respect to water at 700 hPa (grey shading encircling regions above 80%, not shown for FourCastNet ( e ) as not available), the vertical component of relative vorticity at 850 hPa (light-to-dark green shading, from 3 × 10 −4 s –1 and then every 2 × 10 −4 s –1 ). a shows the structure in ERA5 while b – f show the structure from forecasts initialised at 00 UTC 1 November 2023. Note that the range of the wind speed colour bars in Figs. 5 and 6 is different to that in Figs. 2 and 4 . The main features of the cyclone described in the text are annotated in ( a ).

Contours of wind speed at 250 hPa and some of the contours of wet-bulb potential temperature at 850 hPa are not present as the associated values are not reached in the maps shown.

On 1 November 2023, Ciarán underwent significant intensification beneath the left exit region of an upper-level jet streak (Fig. 2c ). All the ML models captured the position and extent of the upper-level jet streak accurately with the minimum MSLP associated with Ciarán beneath the left exit region at 18 UTC (Fig. 5a–f ), a critical aspect of Storm Ciarán’s dynamics.

There is also consensus among the ML models concerning the general shape of the cyclone. Figure 5a–f shows the position of the selected moist isentropes, chosen to indicate the frontal locations and, by their separation, the frontal strengths. The position of the warm sector, identified as the region inside the 285 K moist isentrope, is characterised as a hooked feature in ERA5. The shape of the warm sector is well captured by all the ML models. The cloud head, represented by 700-hPa relative humidity above 80% (grey shading), is seen wrapping around the poleward side of the cyclone centre in ERA5. The IFS HRES, Graphcast and PanguWeather forecasts accurately depict the shape of the cloud head; however, the cloud head in FourCastNet v2 appears less curved than the other forecasts. FourCastNet forecast does not output a humidity variable at 700 hPa.

Despite capturing the general shape of Storm Ciarán, there are noticeable differences in the strength of frontal structures, indicated by the gradient in wet-bulb potential temperature (how close together moist isentropes are). This is true for the cold front, denoted by the 285-K and 287.5-K moist isentropes to the southeast of the cyclone centre and also for the “bent-back front”, i.e., the gradient between the 282.5-K and 285-K moist isentropes that wrap around the cyclone centre on its northwestern side. To the southwest of the low centre the moist isentropes indicate the bent-back front diverge, and this is known as the frontal-fracture region. While all the ML forecasts include a frontal-fracture region, they struggle to resolve the sharp across-front temperature gradient to the west and southwest of the low centre.

Cross-frontal wind shear is another indicator of frontal strength, and the values of the vertical component of 850-hPa relative vorticity (green shading in Fig. 5a–f ) near the bent-back front provide further evidence of the difficulties that the ML models have in simulating the frontal structures in the region. The hook-shaped narrow strip of high relative vorticity aligned with the bent-back warm front that is present in both IFS HRES and ERA5 (with maximum values up to 9 x 10 -4  s -1 and 7 x 10 -4  s -1 , respectively) becomes broader and weaker in the ML models. The discrepancy between the ERA5 (and also the forecasts based on the ERA5 system, see Supp. Fig. 1b ) and ML models in representing the sharpness of the bent-back front indicates that this shortcoming of the ML models is not solely due to model resolution.

This difference in frontal strength is particularly significant since it directly relates to the environment that can be conducive to the descent of a sting jet. Sting jets are coherent air flows that descend over a few hours from inside the tip of the cloud head at mid-tropospheric levels leading to a distinct mesoscale (perhaps 50–100 km across) region of near-surface stronger winds, and particularly gusts 37 . Among the models presented in this study, only the IFS forecasts and analysis have the necessary resolution to resolve a sting jet. It is crucial to recognise that the ML models, trained on coarse resolution data, are not equipped to discern features such as the presence of mesoscale jets. This limitation highlights the importance of using models with adequate resolution when predicting high-impact weather phenomena occurring at small spatial scales. However, our analysis suggests that the ML models struggle to represent frontal structures conducive to mesoscale high-impact features even when compared against NWP models with similar resolution, such as that used to generate ERA5.

The lack of sharpness of the bent-back warm front and cold front in the ML models impacts the strength of the wind speed maxima, as can be seen by turning the focus of evaluation to the region of strong winds. Figure 5a–f shows the 850-hPa wind speed (filled contours) to give an indication of the lower-tropospheric storm structure that is less influenced by the presence of land below than the 10-m winds shown previously; consequently, winds at this pressure level, roughly a km above the ground, are normally stronger than those nearer the surface. All the ML models consistently identify two regions of strong winds: one in the frontal-fracture region and another in the warm sector. The strong winds situated in the frontal-fracture region are associated with the tight pressure gradient near the tip of the bent-back warm front, the associated descent and acceleration where the moist isentropes spread out, and the alignment with the direction of propagation of the storm (with a possible local enhancement due to sting-jet descent in the IFS HRES, see the small-scale areas above 46 ms -1 ). The strong winds mostly in the core of the warm sector (enclosed by the 287.5-K moist isentropes) are associated with a broad jet, known as the warm conveyor belt jet, which ascends through the depth of the atmosphere from the top of the atmospheric boundary layer. The ability of the ML models to identify both the frontal-fracture and warm conveyor belt wind maxima (albeit with differences in the spatial structure and intensity of the latter) underscores their ability to accurately capture the general structure of extratropical cyclones. However, maximum wind speeds are weaker in the ML models than in ERA5, where they exceed 46 m s -1 (and even 48 m s -1 in the forecast based on the ERA5 system) in a broad region at the entrance of the frontal fracture. While Graphcast and FourCastNet display a small deficit of around 2 m s -1 , PanguWeather and FourCastNetv2 are roughly 4 m s -1 and 6 m s -1 lower, respectively.

We now turn our attention to the forecasts valid 6 h later, at 00UTC on 2 November 2023 (Fig. 6a–f ), the time of peak ERA5 wind speeds. At this stage of Ciarán’s evolution, analysis of the frontal-fracture region and the warm sector reveals several interesting features. ERA5 exhibits a region of warm air at the centre of the storm, which is separated from the main warm sector, a process known as warm core seclusion. Warm core seclusion occurs during the mature stage of extratropical cyclone development when cold air wraps around the low centre and cuts it off from the warm subtropical airmass. Relative vorticity starts decreasing along the bent-back front and generally increases near the cyclone centre as the front wraps around it. While the general evolution is captured by all models, the degree of clarity in the presence of a well-defined warm seclusion varies noticeably among the ML-models.

Focusing on the maximum winds in the frontal-fracture region, now compounded by the arrival of the cold conveyor belt (the main low-level jet in the cold sector, behind the cold front, of an extratropical cyclone), reveals that the ERA5, the forecast based on the ERA5 system (Supp. Fig. 1c ) and the IFS forecast all have peak 850-hPa wind speeds between 48-50 m s -1 . GraphCast and FourCastNet exhibit peak wind speeds 4–6 m s -1 lower. PanguWeather and FourCastNet-v2 have a larger weak bias, with wind speeds underestimated by 6-8 m s -1 . Wind maxima are consistently underestimated in the ML models when compared to the benchmarks provided by the ERA5 (and its forecast) and the IFS forecast. This discrepancy in predicting wind maxima at the time of peak winds and as they approach land is crucial for assessing the potential impact of Storm Ciaran’s surface winds and associated gusts.

By inspection, the structures of the MSLP fields in Figs. 5 and 6 are similar for the different models despite the differences in the wind speed structure and magnitude. This raises a further interesting question, is the discrepancy between the wind maxima in the conventional NWP and ML because the ML models do not reproduce the dynamical balances between the wind and pressure fields inherent in the conventional NWP models? This question is examined in detail in the Supplementary Material (and included Supp. Figs. 2 – 8 ) and a short summary is included here.

While the calculation of the geostrophic wind field (resulting from the balance of the pressure gradient and Coriolis forces) is relatively straightforward, calculating the more accurate gradient wind field (with the further inclusion of the centrifugal force associated with the curvature of a parcel trajectory) is more complex. Since both calculations require the evaluation of horizontal gradients in the geopotential field, an unphysical lack of smoothness on the smallest scales in all of the ML models becomes easily apparent and should be further investigated. Note that while the gradient wind should provide a better approximation to the frictionless large-scale flow (where that flow is curved) than the geostrophic wind, the flow can still differ from gradient wind balance due to unbalanced motions which may be physically realistic, particularly in high-resolution model output (and reality).

Smoothed geostrophic and gradient wind fields have physically plausible structures in both NWP and ML model outputs. While the strongest full winds are found in the NWP model outputs even after smoothing, the strongest gradient winds are not clearly different between the ML and NWP models. The differences between the smoothed full and gradient wind fields for the NWP and ML models have similar characteristic structures and magnitudes in strong wind regions of the storm. Within the limitations of the accuracy of our calculations, we cannot conclude that the weak winds in the ML model forecasts are the result of an inability to resolve the proper dynamical balances, but are likely to instead be related to inadequacies in the geopotential field, i.e., in the gradient and curvature of the geopotential contours.

In summary, the ML models represent the large-scale dynamical drivers key to the development of Storm Ciarán well, including the position of the storm relative to the upper-level jet exit. They also accurately capture the larger synoptic-scale structure of the cyclone such as the position of the cloud head, the shape of the warm sector and the location of the warm conveyor belt jet. The ability of the ML models to resolve the more detailed structure of the storm is more mixed. Only some ML models correctly resolve the warm core seclusion and none of them capture the sharp bent-back warm frontal gradient. ML models underestimate the magnitude of the strongest winds at the surface and in the free atmosphere (above the boundary layer), particularly in the frontal-fracture region near the end of the bent-back front. Note that this underestimation of the strongest wind speeds is not a consequence of the resolution of the output of the ML models or their training data, since it also applies when comparing against the ERA5 (and forecasts based on the ERA5 system) and NWP models with resolution similar to ERA5.

The contrasting ability of the four ML models considered to accurately forecast the large-scale dynamical properties of Storm Ciarán and its damaging winds serve to highlight the need for a more comprehensive assessment of this new and potentially transformative forecasting tool. More than 48 h before Storm Ciarán affected communities surrounding the English Channel, forecasts of the rapid MSLP deepening and track of the storms produced by the ML models were essentially indistinguishable from forecasts from an ensemble of conventional NWP models. Our analysis shows that the ML models were able to reproduce the upper-level flow that steered the developing storm into the left exit region of the jet and led to its rapid intensification. Many of the important dynamical features of the storm including the position of the cloud head, the shape of the warm sector and the location of cold and warm conveyor belt jets were also well captured by the ML models. The ML models do not seem to have been limited by the fact that a storm of comparable central pressure has never previously been observed over England during November. However, even in the relatively short ERA5 record, storms developing in a similar way with similar dynamical drivers (such as the upper-level jet) are common throughout winter and the ability of ML models to forecast more dynamically unusual storms, such as small-scale storms that develop rapidly from waves on pre-existing fronts, is an open question.

In contrast, when considering the damaging winds associated with Storm Ciarán in detail, forecasts from the ML models had significant errors and poorer performance than conventional NWP models. All four ML models failed to produce the narrow band of very strong winds at the surface that led to the most severe impacts, The ML models also failed to represent the strength of the cross-front thermal gradient in the bent-back front (a feature also dynamically linked to strong winds) and had variable success in producing the warm seclusion of air that formed in the centre of the storm in its mature stage.

Much further work, considering other storms, is needed to assess if the biases apparent in the simulation of Storm Ciarán are a systematic feature of this first generation of ML models. Increased scrutiny of the models is likely to lead to the identification of target areas for model improvement, as it has done for NWP models. Since the ML models are available to all through public repositories, this scrutiny is likely to enable rapid model improvement. Detailed documentation of the performance of ML models will also be critical to weather forecasters seeking to make greater use of the ML models as part of the forecasting process. Forecasting centres like ECMWF are already beginning to develop and test alpha versions of ML models that complement their existing capabilities 38 .

Based on a single case study, it would be premature to draw conclusions about the relative abilities of the four different approaches to ML weather forecasting exemplified by the different models. In particular, given we only had access to the ‘small’ version of the FourCastNet-v2 model it might be expected that this model would have a limited ability to produce the detailed properties of Storm Ciarán. Nonetheless, studies like ours are useful for identifying knowledge gaps in ML model development for forecasting, particularly in their ability to capture the structure of extreme weather patterns. This can be direct, such as the inclusion of necessary output variables (a 700-hPa humidity variable to identify the shape of the cloud head missing from the FourCastNet model), or through the formulation of more nuanced hypotheses for investigation. For instance, PanguWeather’s ability to capture the vertical component of the 850-hPa relative vorticity could be due to the model integrating height information across levels. Similarly, GraphCast’s ability to simulate the warm core seclusion better may be due to its multi-mesh representation rather than the spatial mixing used in the other models. All the ML models failed to capture the intense winds at the surface associated with Ciarán. As shown in the supplementary figures, wind errors in the lower troposphere are smaller. This may indicate that future models could benefit from including variables and the relationships across these variables that better characterise the planetary boundary layer in their training datasets. Most importantly, our analysis makes a strong case for a more robust evaluation of the forecasts from the ML models across all relevant spatio-temporal features of the physical phenomenon considered instead of isolated error metrics on individual variables.

The rapid acceleration of the forecasting capabilities of ML models as exemplified by our study of Storm Ciarán poses many new challenges and opportunities for atmospheric science 39 . Explainable AI (xAI) techniques 40 , 41 could be powerfully combined with the ML models we have considered to develop a deeper understanding of the reasons that they were able to produce skillful forecasts of Storm Ciarán in line with other attempts to unify ML and causal discovery methods 42 . The development of general-purpose, foundational models 43 could add further to the set of tools available to both forecast and understand high-impact weather events.

In this study, we compare forecasts produced by four different models based on machine learning methods. All are initialised from the same operational ECMWF analysis allowing a direct comparison with the current operational forecast of the ECMWF high-resolution model (CY48R1). The ML model forecasts are produced using the ai-models toolbox developed by ECMWF ( https://github.com/ecmwf-lab/ai-models ).

All four models considered are data-driven Deep Learning models and originally trained on a few ( ~ 4) decades worth of atmospheric and surface variables from the ERA5 dataset 44 at a resolution of 0.25°x 0.25° ( ~ 30 km resolution at the equator), which translates to 720 x 1440 grid cells. The ML models are all autoregressive, which means model output from a given time step can be used to predict output at the next time step. Model differences arise chiefly from the individual architectures, the selection of variables, parameterisations and training schemes briefly summarised in the technical details below:

FourCastNet 12 , uses the vision transformer (ViT) architecture with an Adaptive Fourier Neural Operator (AFNO) 45 . The AFNO enables dependencies across spatial and channel dimensions to be modelled efficiently at high resolutions where spatial token (feature) mixing occurs as a global convolution in the Fourier domain with FFTs. The model has a pre-training step where the AFNO is trained ahead on the ERA5 data with 20 different surface and atmospheric variables and then used for inference. The pre-training step learns mappings between X ( t ) and X ( t  + Δ t ) where t is a time step, Δ t is a time increment (set to 6 h) and X is a tensor of features called patches. In the second fine-tuning or inference step, the pre-trained model is used to produce inferences from a defined state X ( t ), first for X ( t + Δ t ) and this output from the models is itself then used to generate X ( t  + 2* Δ t ) or the output for the second time step. Thus, while the training of the model is resource intensive, it is a one-time cost and the inference step is very fast.

FourCastNet v2 13 is a development of the original FourCastNet model that uses Spherical Harmonics Neural Operators for modelling non-linear chaotic and dynamical systems on a sphere as opposed to flat Euclidean spaces. The model is trained with ERA5 data in a two-step process similar to FourCastNet—a single autoregressive step followed by fine-tuning. By learning global convolutions in computationally efficient manners, Fourier Neural Operators (such as those used in FourCastNet) are capable of accurately simulating long-range dependencies in spatio-temporal data. However, the Discrete Fourier Transform that FNOs rely on assumes a flat geometry, resulting in dissipation together with visual and spectral artefacts. The Spectral Fourier Neural Operators (SFNO), forming the basis for the FourCastNet v2 model architecture in its update from the FourCastNet model, in addition to having the desirable properties of FNOs also have translational or rotational equivariance. FourCastNet v2 is trained on a 73-channel subset of the ERA5 reanalysis dataset on single levels and pressure levels.

Pangu-Weather 14 consists of four deep neural networks with different lead times (time between input and output) of 1 h, 3 h, 6 h and 24 h. 5 upper atmosphere and 4 surface variables at 13 different pressure levels were used to train the model with a combined total of 256 million parameters. The overall deep network architecture is called 3DEST or 3D-Earth-specific Transformer that integrates height information into a new dimension thus capturing relationships between atmospheric variables across pressure levels, unlike similar transformer-based models such as FourCastNet. Data is fed into the neural network and a process called patch embedding is used to downsample the input data from individual grid cells into a 3D cube. This cube is then put through an encoder-decoder based on a ViT called the Swin transformer 25 with 16 blocks. The positional bias in the Swin transformer is replaced with an Earth-specific positional bias to reflect the fact that in a 2D projection of a sphere, distances between neighbouring points are not the same across all latitudes. The decoder is symmetric to the encoder. Although the transformer-based neural network has a large training time similar to FourCastNet, this is partially improved in Pangu-Weather by the use of a hierarchical temporal aggregation scheme that reduces cumulative forecast errors and also the forecast generation time. This is done by employing the neural network with the largest lead time iteratively for a forecast so that neural networks with shorter lead times are used closer to the forecast. The height integration and aggregated forecast schemes are also considered improvements over other transformer-based architectures.

GraphCast 15 is based on Graph Neural Networks (GNNs) 46 with around 36.7 million parameters. The model is trained with 5 surface and 6 atmospheric variables at 37 pressure levels resulting in 227 variables for every data point or grid cell. In the first step, the Encoder maps information from individual grid cells to nodes in a multi-mesh representation. The multi-mesh is derived as icosahedral meshes of increasing resolution from coarse (12 nodes) to fine (40,962 nodes). The second step has Processors using 16 GNN layers to propagate local and long-range information across the nodes on the multi-mesh through message passing. Finally, the decoder uses a single GNN layer to map the final processor layer’s multi-mesh representation back to the grid cells. GraphCast thus avoids the use of transformers and the associated scaling issues with higher resolutions that could result in large training times.

Numerical weather prediction model forecasts and analysis products

The ML model forecasts are compared to a set of forecasts from conventional numerical weather prediction (NWP) models to assess both systematic differences in the capabilities of the NWP and ML models and how the spread in the forecasts from the two architectures compare. Forecasts from the IFS HRES forecast and forecasts based on the ERA5 system (see below for a description of ERA5) were obtained from ECMWF and control (unperturbed) members of the ensemble forecasts for four models (the IFS 47 , the Met Office 48 , the Japan Meteorological Agency (JMA) 49 , and the National Centres for Environmental Prediction (NCEP) 50 ) were downloaded from the TIGGE archive 51 of operational global ensemble weather forecasts out to the medium range. The models chosen differ in their design and the resolution of the numerical model grid. Cycle 48r1 of the IFS was operational at the time of Storm Ciarán. Following the upgrade to this cycle in June 2023, the HRES and ensemble forecasts have the same resolution, equivalent to 9 km grid spacing. The Met Office, JMA and NCEP ensembles have grid spacings of approximately 20 km, 27 km and 25 km, respectively. The data for the four control ensemble members were all obtained regridded to a regular latitude-longitude grid of 0.5 degrees.

Analysis products

Both sets of models are compared to two analysis products (optimal blends of short-range forecasts and observations): the operational IFS analysis and ERA5 44 . The operational IFS analysis is produced using the IFS HRES forecast and has a resolution equivalent to 9 km grid spacing, ERA5 has a resolution equivalent to 31 km grid spacing. The IFS analysis and ERA5 were regridded to a regular latitude-longitude grid of 0.25 degrees.

ERA5 is used as an additional measure of forecasts ‘truth’ because the ML models all used ERA5 as their training data. Hence comparison with ERA5 indicates the skilfulness of these models relative to the best possible forecast given their training data. It is to be expected that the IFS analysis will include smaller-scale and higher amplitude weather features than ERA5 due to the use of a higher resolution model, despite being regridded to the same grid. It is also expected that the IFS analysis will be closer to the “truth” due to the use of higher resolution and an upgraded modelling system.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author upon reasonable request.

Code availability

Apart from the Python packages referenced in the Acknowledgements, the underlying code for this study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.

Bauer, P., Thorpe, A. & Brunet, G. The quiet revolution of numerical weather prediction. Nature 525 , 47–55 (2015).

Article   CAS   Google Scholar  

Krasnopolsky, V., Breaker, L. & Gemmill, W. A neural network as a nonlinear transfer function model for retrieving surface wind speeds from the special sensor microwave imager. J. Geophys. Res.: Oceans 100 , 11033–11045 (1995).

Article   Google Scholar  

Brajard, J., Jamet, C., Moulin, C. & Thiria, S. Use of a neuro-variational inversion for retrieving oceanic and atmospheric constituents from satellite ocean colour sensor: Application to absorbing aerosols. Neural Netw. 19 , 178–185 (2006).

Brenowitz, N. D. & Bretherton, C. S. Prognostic validation of a neural network unified physics parameterization. Geophys. Res. Lett. 45 , 6289–6298 (2018).

Jones, N. How machine learning could help to improve climate forecasts. Nature 548 , 379–380 (2017).

Rasp, S. & Lerch, S. Neural networks for postprocessing ensemble weather forecasts. Mon. Weather Rev. 146 , 3885–3900 (2018).

Grönquist, P. et al. Deep learning for post-processing ensemble weather forecasts. Philos. Trans. R. Soc. A 379 , 20200092 (2021).

Chantry, M., Christensen, H., Dueben, P. & Palmer, T. Opportunities and challenges for machine learning in weather and climate modelling: hard, medium and soft AI. Philos. Trans. R. Soc. A 379 , 20200083 (2021).

Rasp, S. et al. WeatherBench: a benchmark data set for data‐driven weather forecasting. J. Adv. Model. Earth Syst. 12 , e2020MS002203 (2020).

Dueben, P. et al. Machine Learning at ECMWF: a Roadmap for the Next 10 Years . (European Centre for Medium-Range Weather Forecasts, 2021).

Krishnamurti, T. et al. Improved skill for the anomaly correlation of geopotential heights at 500 hPa. Month. Weather Rev. 131 , 1082–1102 (2003).

Pathak, J. et al. Fourcastnet: a global data-driven high-resolution weather model using adaptive fourier neural operators. https://arxiv.org/abs/2202.11214 (2022).

Bonev, B. et al. Spherical fourier neural operators: learning stable dynamics on the sphere. https://arxiv.org/abs/2306.03838 (2023).

Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619 , 533–538 (2023).

Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382 , 1416–1421 (2023).

Andrychowicz, M. et al. Deep learning for day forecasts from sparse observations. https://arxiv.org/abs/2306.06079 (2023).

Weyn, J. A., Durran, D. R., Caruana, R. & Cresswell‐Clay, N. Sub‐seasonal forecasting with a large ensemble of deep‐learning weather prediction models. J. Adv. Modeling Earth Syst. 13 , e2021MS002502 (2021).

Li, L., Carver, R., Lopez-Gomez, I., Sha, F. & Anderson, J. Generative emulation of weather forecast ensembles with diffusion models. Science Advances. 10, eadk4489 (2024).

Price, I. et al. GenCast: Diffusion-based ensemble forecasting for medium-range weather. https://arxiv.org/abs/2312.15796 (2023).

Magnusson, L. in ECMWF Newsletter https://alderney-elec.com/weather-station/ (ECMWF, 2023).

Priestley, M. D., Dacre, H. F., Shaffrey, L. C., Hodges, K. I. & Pinto, J. G. The role of serial European windstorm clustering for extreme seasonal losses as determined from multi-centennial simulations of high-resolution global climate model data. Nat. Hazards Earth Syst. Sci. 18 , 2991–3006 (2018).

Little, A. S., Priestley, M. D. & Catto, J. L. Future increased risk from extratropical windstorms in northern Europe. Nat. Communications 14 , 4434 (2023).

Zscheischler, J. et al. A typology of compound weather and climate events. Nat. Rev. Earth Environ. 1 , 333–347 (2020).

Shepherd, T. G. et al. Storylines: an alternative approach to representing uncertainty in physical aspects of climate change. Clim. change 151 , 555–571 (2018).

Sanders, F. & Gyakum, J. R. Synoptic-dynamic climatology of the “bomb”. Mon. Weather Rev. 108 , 1589–1606 (1980).

Kendon, M. Storm Ciarán , 1 to 2 November 2023 . https://www.metoffice.gov.uk/binaries/content/assets/metofficegovuk/pdf/weather/learn-about/uk-past-events/interesting/2023/2023_09_storm_ciaran.pdf (2023).

Shapiro, M. A. & Keyser, D. Fronts, Jet Streams and the Tropopause . (Springer, 1990).

Gray, S. L. & Volonté, A. Why Storm Ciarán’s Winds Were So Strong . https://theconversation.com/why-storm-ciarans-winds-were-so-strong-216923 (2023).

Barroux, R. Storm Ciaran: A Night of Destruction and Howling Gales In Brittany . https://www.lemonde.fr/en/climate-change/article/2023/11/02/storm-ciaran-a-night-of-destruction-and-howling-gales-in-brittany_6222086_152.html (2023).

Meteociel.fr. Tableaux d’observations pour Pointe du Raz , https://www.meteociel.fr/temps-reel/obs_villes.php?jour2=2&mois2=10&annee2=2023&code2=7103 (2023).

France 24. At Least 16 Dead as Storm Ciaran pummels Western Europe . https://www.france24.com/en/live-news/20231103-ten-dead-as-storm-ciaran-batters-western-europe (2023).

Henley, J. Storm Ciarán: Seven People Killed and Dozens Injured in Europe , https://www.theguardian.com/world/2023/nov/02/storm-ciaran-people-killed-injured-storm-ciaran-batters-europe-wind-rain (2023).

Henshell, R. No Trains in Five French Regions and Airport Closes in Brest , https://www.connexionfrance.com/article/French-news/No-trains-in-five-French-regions-and-airport-closes-in-Brest (2023).

Alderney Electricity Limited. Alderney weather stations. https://alderney-elec.com/weather-station/ (2023).

Horton, S. Key Severe Damage Indicators along the Track of the November Jersey Tornado 2023 . (Tornado and Storm Research Organisation, 2023).

Klawa, M. & Ulbrich, U. A model for the estimation of storm losses and the identification of severe winter storms in Germany. Nat. Hazards Earth Syst. Sci. 3 , 725–732 (2003).

Clark, P. A. & Gray, S. L. Sting jets in extratropical cyclones: a review. Q. J. R. Meteorol. Soc. 144 , 943–969 (2018).

AIFS Team. ECMWF Unveils Alpha Version of New ML Model , https://www.ecmwf.int/en/about/media-centre/aifs-blog/2023/ECMWF-unveils-alpha-version-of-new-ML-model (2023).

Schultz, M. G. et al. Can deep learning beat numerical weather prediction? Philos. Trans. R. Soc. A 379 , 20200097 (2021).

Arrieta, A. B. et al. Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58 , 82–115 (2020).

Samek, W. & Müller, K.-R. Towards explainable artificial intelligence. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. 5–22 (Springer, 2019).

Camps-Valls, G. et al. Discovering causal relations and equations from data. Physics Reports. 1044, 1–68 (2023).

Bommasani, R. et al. On the opportunities and risks of foundation models. https://arxiv.org/abs/2312.15796 (2021).

Hersbach, H. et al. The ERA5 global reanalysis. Q. J. R. Meteorol. Soc. 146 , 1999–2049 (2020).

Guibas, J. et al. Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators. International Conference on Learning Representations (2022).

Battaglia, P. W. et al. Relational inductive biases, deep learning, and graph networks. https://arxiv.org/abs/1806.01261 (2018).

Wedi, N. et al. The Modelling Infrastructure of the Integrated Forecasting System: Recent advances and Future Challenges . (European Centre for Medium-Range Weather Forecasts, 2015).

Bowler, N. E., Arribas, A., Mylne, K. R., Robertson, K. B. & Beare, S. E. The MOGREPS short‐range ensemble prediction system. Q. J. R. Meteorol. Soc.: A J. Atmos. Sci. Appl. Meteorol. Phys. Oceanogr. 134 , 703–722 (2008).

Yamaguchi, H. et al. Introduction to JMA’s new Global Ensemble Prediction System. CAS/JSC WGNE Res. Activities Atmos. Ocean. Model. 42 , 6.13–16.14 (2018).

Google Scholar  

Zhou, X. et al. The development of the NCEP global ensemble forecast system version 12. Weather Forecast. 37 , 1069–1084 (2022).

Bougeault, P. et al. The THORPEX interactive grand global ensemble. Bull. Am. Meteorol. Soc. 91 , 1059–1072 (2010).

MetOffice. LAND SYNOP Reports from Land Stations Collected by the Met Office MetDB System . https://catalogue.ceda.ac.uk/uuid/9f80d42106ba708f92ada730ba321831 (2008).

MetOffice. SHIP SYNOP Reports from Ship, Buoy and Fixed Platform Stations Collected by the Met Office MetDB System . https://catalogue.ceda.ac.uk/uuid/65ca7898647cc3686492bcb8bb483a1c (2008).

National Weather Service. Sample Station Plot . https://www.wpc.ncep.noaa.gov/html/stationplot.shtml (2022).

Download references

Acknowledgements

We thank the ECMWF labs team for building the publicly available ai-models library which enabled us to produce and compare forecasts from the four ML models. This library can be accessed at https://github.com/ecmwf-lab/ai-models . We also thank the modelling groups who made the code for the ML models publicly available through the following repositories: • FourCastNet https://github.com/NVlabs/FourCastNet . • PanguWeather https://github.com/198808xc/Pangu-Weather . • GraphCast https://github.com/google-deepmind/graphcast . This work is partly based on TIGGE data. TIGGE (The International Grand Global Ensemble) is an initiative of the World Weather Research Programme (WWRP). We are also grateful to ECMWF for providing access to operational analysis products to members of the research team through national research accounts held through the Met Office. The work is partly funded by the UKRI Natural Environment Research Council (UKRI-NERC) through several grants held by contributors and by the Schmidt Futures Foundation. BH is funded by the UKRI-NERC CANARI programme (NE/W004984/1). NJH is funded by UKRI-NERC UMBRELLA (NE/X018555/1). KMRH is funded by a UKRI-NERC Independent Research Fellowship (MITRE; NE/W007924/1). RS is funded by UKRI-NERC TerraFIRMA (NE/W004895/1). AV is funded by UKRI-NERC Arctic Summer-time cyclones (NE/T006773/1). SD is funded and supported by the Schmidt Futures Foundation.

Author information

Authors and affiliations.

Department of Meteorology, Univ. of Reading, Reading, UK

Andrew J. Charlton-Perez, Helen F. Dacre, Simon Driscoll, Suzanne L. Gray, Ben Harvey, Natalie J. Harvey, Kieran M. R. Hunt, Robert W. Lee, Ranjini Swaminathan, Remy Vandaele & Ambrogio Volonté

National Centre for Earth Observation, Univ. of Reading, Reading, UK

Simon Driscoll, Ranjini Swaminathan & Remy Vandaele

National Centre for Atmospheric Science, Univ. of Reading, Reading, UK

Ben Harvey, Kieran M. R. Hunt & Ambrogio Volonté

You can also search for this author in PubMed   Google Scholar

Contributions

A.J.C.P. conceived the analysis and produced ML forecasts and initial characterisation of their performance. S.D., K.M.R.H., R.S. and R.V. contributed to the analysis and description of the ML models. N.J.H. led analysis of the properties of Storm Ciarán and its impacts. H.D., S.G., B.H. and A.V. produced the detailed dynamical analysis of the properties of Storm Ciarán in the forecasts including collecting NWP data and producing all figures. All authors contributed to discussions, writing, proofreading and editing the manuscript. All authors should be considered co-first authors as the work was completed collaboratively. The funders played no role in the study design, data collection, analysis and interpretation of data, or the writing of this manuscript.

Corresponding author

Correspondence to Andrew J. Charlton-Perez .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary material, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Charlton-Perez, A.J., Dacre, H.F., Driscoll, S. et al. Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán. npj Clim Atmos Sci 7 , 93 (2024). https://doi.org/10.1038/s41612-024-00638-w

Download citation

Received : 02 December 2023

Accepted : 31 March 2024

Published : 22 April 2024

DOI : https://doi.org/10.1038/s41612-024-00638-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Open access
• Published: 22 April 2024

The influence of maternal prepregnancy weight and gestational weight gain on the umbilical cord blood metabolome: a case–control study

• Xianxian Yuan   ORCID: orcid.org/0000-0001-8762-8471 1 ,
• Yuru Ma 1 ,
• Jia Wang 2 ,
• Yan Zhao 1 ,
• Wei Zheng 1 ,
• Ruihua Yang 1 ,
• Lirui Zhang 1 ,
• Xin Yan 1 &
• Guanghui Li   ORCID: orcid.org/0000-0003-2290-1515 1  

BMC Pregnancy and Childbirth volume  24 , Article number:  297 ( 2024 ) Cite this article

Metrics details

Maternal overweight/obesity and excessive gestational weight gain (GWG) are frequently reported to be risk factors for obesity and other metabolic disorders in offspring. Cord blood metabolites provide information on fetal nutritional and metabolic health and could provide an early window of detection of potential health issues among newborns. The aim of the study was to explore the impact of maternal prepregnancy overweight/obesity and excessive GWG on cord blood metabolic profiles.

A case control study including 33 pairs of mothers with prepregnancy overweight/obesity and their neonates, 30 pairs of mothers with excessive GWG and their neonates, and 32 control mother-neonate pairs. Untargeted metabolomic profiling of umbilical cord blood samples were performed using UHPLC‒MS/MS.

Forty-six metabolites exhibited a significant increase and 60 metabolites exhibited a significant reduction in umbilical cord blood from overweight and obese mothers compared with mothers with normal body weight. Steroid hormone biosynthesis and neuroactive ligand‒receptor interactions were the two top-ranking pathways enriched with these metabolites ( P  = 0.01 and 0.03, respectively). Compared with mothers with normal GWG, in mothers with excessive GWG, the levels of 63 metabolites were increased and those of 46 metabolites were decreased in umbilical cord blood. Biosynthesis of unsaturated fatty acids was the most altered pathway enriched with these metabolites ( P  < 0.01).

Conclusions

Prepregnancy overweight and obesity affected the fetal steroid hormone biosynthesis pathway, while excessive GWG affected fetal fatty acid metabolism. This emphasizes the importance of preconception weight loss and maintaining an appropriate GWG, which are beneficial for the long-term metabolic health of offspring.

Peer Review reports

The obesity epidemic is an important public health problem in developed and developing countries [ 1 ] and is associated with the emergence of chronic noncommunicable diseases, including type 2 diabetes mellitus (T2DM), hypertension, cardiovascular disease, nonalcoholic fatty liver disease (NAFLD), and cancer [ 2 , 3 , 4 ]. Maternal obesity is the most common metabolic disturbance in pregnancy, and the prevalence of obesity among women of childbearing age is 7.1% ~ 31.9% in some countries [ 5 ]. In China, the prevalence of overweight and obesity has also increased rapidly in the past four decades. Based on Chinese criteria, the latest national prevalence estimates for 2015–2019 were 34.3% for overweight and 16.4% for obesity in adults (≥ 18 years of age) [ 6 ].

Increasing evidence implicates overnutrition in utero as a major determinant of the health of offspring during childhood and adulthood, which is compatible with the developmental origins of health and disease (DOHaD) framework [ 7 ]. Maternal obesity and excessive gestational weight gain (GWG) are important risk factors for several adverse maternal outcomes, including gestational diabetes and hypertensive disorders, fetal death, and preterm birth [ 8 , 9 , 10 ]. More importantly, they have negative implications for offspring, both perinatally and later in life. Evidence from cohort studies focusing on offspring development confirms the relationship between maternal obesity/excessive GWG and offspring obesity programming [ 11 , 12 , 13 ]. Currently, there is no unified mechanism to explain the adverse outcomes associated with maternal obesity and excessive GWG, which may be the independent and interactive effects of the obese maternal phenotype itself and the diet associated with this phenotype. In addition to genetic and environmental factors, metabolic programming may also lead to the intergenerational transmission of obesity through epigenetic mechanisms.

Metabolomics, which reflects the metabolic phenotype of human subjects and animals, is the profiling of metabolites in biofluids, cells and tissues using high-throughput platforms, such as mass spectrometry. It has unique potential in identifying biomarkers for predicting occurrence, severity, and progression of diseases, as well as exploring underlying mechanistic abnormalities [ 14 , 15 ]. Umbilical cord metabolites can provide information about fetal nutritional and metabolic health, and may provide an early window for detection of potential health issues in newborns [ 16 ]. Previous studies have reported differences in umbilical cord metabolite profiles associated with maternal obesity [ 17 , 18 ]. However, the results were inconsistent due to differences in sample sizes, ethnicity and region, and mass spectrometry. In addition, most studies have not considered the difference in the effects of prepregnancy body mass index (BMI) and GWG on cord blood metabolites.

To investigate the relationship between early metabolic programming and the increased incidence of metabolic diseases in offspring, we studied the associations between elevated prepregnancy BMI/excessive GWG and umbilical cord metabolic profiles. Another purpose of this study was to explore whether there were differences in the effects of prepregnancy overweight/obesity and excessive GWG on cord blood metabolites.

Study population

This was a hospital-based, case control study that included singleton pregnant women who received prenatal care and delivered vaginally at Beijing Obstetrics and Gynecology Hospital, Capital Medical University, from January 2022 to March 2022. We selected 33 pregnant women with a prepregnancy BMI ≥ 24.0 kg/m 2 regardless of their gestational weight gain as the overweight/obese group, 30 pregnant women with a prepregnancy BMI of 18.5–23.9 kg/m 2 and a GWG > 14.0 kg as the excessive GWG group, and 32 pregnant women with a BMI of 18.5–23.9 kg/m 2 and a GWG of 8.0–14.0 kg as the control group. The ages of the three groups were matched (± 1.0 years), and the prepregnancy BMIs of the excessive GWG and control groups were matched (± 1.0 kg/m 2 ).

The inclusion criteria were women with singleton pregnancies, those aged between 20 and 45 years, those with full-term delivery (gestational age ≥ 37 weeks), those with a prepregnancy BMI ≥ 18.5 kg/m 2 , those without prepregnancy diabetes mellitus (DM) or hypertension, and those without gestational diabetes mellitus (GDM). The exclusion criteria were women with multiple pregnancies, those less than 20 years or more than 45 years old, those with a prepregnancy BMI < 18.5 kg/m 2 , those with prepregnancy DM, hypertension or GDM, and those without cord blood samples.

We classified pregnant women into BMI categories based on Chinese guidelines [ 19 ]: normal weight (prepregnancy BMI 18.5–23.9 kg/m 2 ), overweight (prepregnancy BMI 24.0–27.9 kg/m 2 ), and obese (prepregnancy BMI ≥ 28.0 kg/m 2 ). GWG guideline concordance was defined by the 2021 Chinese Nutrition Society recommendations according to prepregnancy BMI. The upper limits of GWG for normal weight, overweight, and obesity were 14.0 kg, 11.0 kg, and 9.0 kg, respectively.

Ethical approval and written informed consent were obtained from all participants. The study has been performed according to the Declaration of Helsinki, and the procedures have been approved by the ethics committees of Beijing Obstetrics and Gynecology Hospital, Capital Medical University (2021-KY-037).

Sample and data collection

Maternal and neonatal clinical data were collected from the electronic medical records system of Beijing Obstetrics and Gynecology Hospital. Maternal clinical characteristics included age, height, prepregnancy and predelivery weight, education level, smoking and drinking status during pregnancy, parity, conception method, comorbidities and complications of pregnancy, family history of DM and hypertension, gestational age, mode of delivery, and biochemical results during pregnancy. Prepregnancy BMI was calculated as prepregnancy weight in kilograms divided by the square of height in meters. GWG was determined by subtracting the prepregnancy weight in kilograms from the predelivery weight in kilograms. GDM was defined using the IAPDSG’s diagnostic criteria at 24 to 28 +6  weeks gestation and the fasting glucose and 1- and 2-h glucose concentrations at the time of the oral glucose tolerance test (OGTT). Neonatal clinical characteristics included sex, birth weight and length. Macrosomia was defined as a birth weight of 4,000 g or more [ 20 ]. Low birth weight (LBW) was defined as a birth weight less than 2,500 g [ 21 ].

Umbilical cord blood samples were obtained by trained midwives after clamping the cord at delivery. Whole blood samples were collected in EDTA tubes, refrigerated for < 24 h, and centrifuged at 2,000 r.p.m. at 4 ℃ for 10 min. Plasma aliquots were stored at -80 ℃ until shipment on dry ice to Novogene, Inc. (Beijing, China) for untargeted metabolomic analysis.

Untargeted metabolomic analyses

Ultrahigh-performance liquid chromatography tandem mass spectrometry (UHPLC‒MS/MS) analyses were performed using a Vanquish UHPLC system (Thermo Fisher, Germany) coupled with an Orbitrap Q Exactive™ HF mass spectrometer (Thermo Fisher, Germany) at Novogene Co., Ltd. (Beijing, China). Detailed descriptions of the sample preparation, mass spectrometry and automated metabolite identification procedures are described in the Supplementary materials .

Statistical analysis

Clinical data statistical analysis.

Quantitative data are shown as the mean ± standard deviation (SD) or median (interquartile range), and categorical data are presented as percentages. The Mann‒Whitney U test, chi-square test, and general linear repeated-measures model were used to assess the differences between the control and study groups when appropriate. A P value < 0.05 was considered statistically significant. All analyses were performed using Statistical Package of Social Sciences version 25.0 (SPSS 25.0) for Windows (SPSS Inc).

Umbilical cord metabolome statistical analysis

These metabolites were annotated using the Human Metabolome Database (HMDB) ( https://hmdb.ca/metabolites ), LIPIDMaps database ( http://www.lipidmaps.org/ ), and Kyoto Encylopaedia of Genes and Genomes (KEGG) database ( https://www.genome.jp/kegg/pathway.html ). Principal component analysis (PCA) and partial least-squares discriminant analysis (PLS-DA) were performed at metaX. We applied univariate analysis ( T test) to calculate the statistical significance ( P value). Metabolites with a variable importance for the projection (VIP) > 1, a P value < 0.05 and a fold change (FC) ≥ 2 or FC ≤ 0.5 were considered to be differential metabolites. A false discovery rate (FDR) control was implemented to correct for multiple comparisons. The q -value in the FDR control was defined as the FDR analog of the P -value. In this study, the q -value was set at 0.2. For clustering heatmaps, the data were normalized using z scores of the intensity areas of differential metabolites and were plotted by the Pheatmap package in R language.

The correlations among differential metabolites were analyzed by cor () in R language (method = Pearson). Statistically significant correlations among differential metabolites were calculated by cor.mtest () in R language. A P value < 0.05 was considered statistically significant, and correlation plots were plotted by the corrplot package in R language. The functions of these metabolites and metabolic pathways were studied using the KEGG database. The metabolic pathway enrichment analysis of differential metabolites was performed when the ratio was satisfied by x/n > y/N, and the metabolic pathway was considered significantly enriched when P  < 0.05.

Demographic characteristics of study participants

The demographic and clinical characteristics of the three population groups enrolled in the study are summarized in Table  1 . Mothers had no significant difference regarding their ages or gestational ages. Compared to the mothers in the excessive GWG and control groups, those in the prepregnancy overweight/obesity group had a significantly higher prepregnancy BMI (25.6 (24.5, 27.2) kg/m 2 ). However, there was no significant difference in prepregnancy BMI between mothers in the excessive GWG group (20.3 ± 1.2 kg/m 2 ) and mothers in the control group (20.6 ± 1.5 kg/m 2 ). Mothers in the excessive GWG group had the highest GWG (17.0 (15.5, 19.1) kg) among the three groups. The mean GWG of the mothers in the prepregnancy overweight/obesity group was 12.9 ± 3.8 kg, which was similar to that of the control group (11.8 ± 1.5 kg). It was noteworthy that among the 33 prepregnancy overweight/obese pregnant women, 20 of them had appropriate GWG, 1 had insufficient GWG, and 12 had excessive GWG. The proportion of mothers who underwent invitro fertilization and embryo transfer (IVF-ET) in the prepregnancy overweight/obesity group (15.2%) was significantly higher than that in the excessive GWG and control groups. There were no statistically significant differences in the proportions of pregnancy outcomes among the three groups, including preeclampsia, premature rupture of membranes, postpartum hemorrhage, macrosomia, and LBW. The babies in the three groups showed no significant difference regarding their birth weights or lengths.

The biochemical parameters of the mothers during pregnancy are shown in Table  2 . The levels of triglyceride (TG) and uric acid (UA) of mothers in the prepregnancy overweight/obesity group were significantly higher than those of the mothers in the excessive GWG and control groups in the first trimester. However, there was no significant difference in the blood glucose and lipid levels in the second and third trimesters of pregnancy among the three groups.

PCA and PLS-DA analysis of cord blood metabolites

Functional and taxonomic annotations of the identified metabolites included the HMDB classification annotations, LIPID MAPS classification annotations, and KEGG pathway annotations. Those cord blood metabolites included lipids and lipid-like molecules, organic acids and their derivatives, and organoheterocyclic compounds, which were mainly involved in metabolism. To better understand the structure of the cord blood metabolome in cases versus controls, we used unsupervised PCA to identify metabolites contributing the most to observed differences in the dataset. PCA did not clearly separate the three groups. We next used PLS-DA to identify metabolites that were predictive of case versus control status. PLS-DA clearly distinguished the cases from the controls (Fig.  1 ), the prepregnancy overweight/obesity group vs. the control group (R2Y = 0.82, Q2Y = 0.37; R2Y = 0.77, Q2Y = 0.13, respectively) (Fig.  1 A), and the excessive GWG group vs. the control group (R2Y = 0.76, Q2Y = 0.16; R2Y = 0.81, Q2Y = 0.41) (Fig.  1 B).

PLS-DA of identified cord blood metabolites. A the prepregnancy overweight/obesity group vs. the control group; B the excessive GWG group vs. the control group. (a) PLS-DA score. The horizontal coordinates are the score of the sample on the first principal component; the longitudinal coordinates are the score of the sample on the second principal component; R2Y represents the interpretation rate of the model, and Q2Y is used to evaluate the predictive ability of the PLS-DA model, and when R2Y is greater than Q2Y, it means that the model is well established. (b) PLS-DA valid. Horizontal coordinates represent the correlation between randomly grouped Y and the original group Y, and vertical coordinates represent the scores of R2 and Q2. (1) POS, positive metabolites; (2) NEG, negative metabolites

Maternal prepregnancy overweight/obesity

Screening differential metabolites according to a PLS-DA VIP > 1.0, a FC > 1.2 or < 0.833 and a P value < 0.05, a total of 106 cord blood metabolites (77 positive metabolites and 29 negative metabolites) differed between the prepregnancy overweight/obesity group and the control group. Compared with those in the control group, the levels of 46 metabolites (19 positive metabolites and 27 negative metabolites) were increased in the prepregnancy overweight/obesity group, among which octopamine was the metabolite with the largest increase, followed by (2S)-4-Oxo-2-phenyl-3,4-dihydro-2H-chromen-7-yl beta-D-glucopyranoside, N-tetradecanamide, stearamide, and methanandamide (Fig.  2 A). Compared with the control group, in the prepregnancy overweight/obesity group, there were 60 metabolites (58 positive metabolites and 2 negative metabolites) with reduced concentrations, among which senecionine was the metabolite with the largest decrease, followed by 3-(methylsulfonyl)-2H-chromen-2-one, methyl EudesMate, cuminaldehyde, and 2-(tert-butyl)-1,3-thiazolane-4-carboxylic acid (Fig.  2 A).

Stem plots of differential cord blood metabolites. A the prepregnancy overweight/obesity group vs. the control group; B the excessive GWG group vs. the control group. (1) positive metabolites; (2) negative metabolites. Notes: The color of the dot in the stem plots represents the upward and lower adjustment, the blue represents downward, and the red represents upward. The length of the rod represents the size of log2 (FC), and the size of the dot represents the size of the VIP value

A hierarchical analysis of the two groups of differential metabolites obtained was carried out, and the difference in metabolic expression patterns between the two groups and within the same comparison was obtained, which is shown in Fig.  3 . KEGG pathway analysis of differential cord blood metabolites associated with the prepregnancy overweight/obesity group versus the control group is shown in Table  3 and Fig.  4 A. The metabolite enrichment analysis revealed that steroid hormone biosynthesis ( P value = 0.01) and neuroactive ligand‒receptor interactions ( P value = 0.03) were the two pathways that were most altered between the prepregnancy overweight/obesity group and the control group. 19 metabolites were distributed in the pathway of steroid hormone biosynthesis, and 4 metabolites were distributed in the pathway of neuroactive ligand‒receptor interactions. In the steroid hormone biosynthesis pathway, the levels of corticosterone, 11-deoxycortisol, cortisol, testosterone, and 7α-hydroxytestosterone were decreased in the prepregnancy overweight/obesity group relative to those in the control group. In the neuroactive ligand‒receptor interaction pathway, the level of cortisol was decreased and the levels of trace amines were increased in the prepregnancy overweight/obesity group relative to the control group.

Clustering heat maps of differential cord blood metabolites of the three groups. A positive metabolites; B negative metabolites. Notes: Longitudinal clustering of samples and trans-verse clustering of metabolites. The shorter the clustering branches, the higher the similarity. Through horizontal comparison, we can see the relationship between groups of metabolite content clustering

KEGG enrichment scatterplots (a) and net (b) of differential cord blood metabolites. A the prepregnancy overweight/obesity group vs. the control group; B the excessive GWG group vs. the control group. (1) positive metabolites; (2) negative metabolites. Notes: (a) The horizontal co-ordinates in the figure are x/y (the number of differential metabolites in the corresponding metabolic pathway/the total number of total metabolites identified in this pathway). The value represents the enrichment degree of differential metabolites in the pathway. The color of the point rep-resents the P -value of the hypergeometric test, and the size of the point represents the number of differential metabolites in the corresponding pathway. (b) The red dot represents a metabolic pathway, the yellow dot represents a substance-related regulatory enzyme information, the green dot represents the background substance of a metabolic pathway, the purple dot represents the molecular module information of a class of substances, the blue dot represents a substance chemical reaction, and the green square represents the differential substance obtained by this comparison

Maternal excessive GWG

A total of 109 cord blood metabolites (52 positive metabolites and 57 negative metabolites) differed between the excessive GWG group and the control group. Compared with the control group, in the excessive GWG group, there were 63 metabolites (15 positive metabolites and 48 negative metabolites) with increased concentrations, among which 2-thio-acetyl MAGE was the metabolite with the largest increase, followed by PC (7:0/8:0), lysopc 16:2 (2 N isomer), MGMG (18:2), and thromboxane B2 (Fig.  2 B). Compared with the levels in the control group, the levels of 46 metabolites (37 positive metabolites and 9 negative metabolites) in the excessive GWG group were reduced, among which hippuric acid had the largest decrease, followed by 8-hydroxyquinoline, gamithromycin, 2-phenylglycine, and cefmetazole (Fig.  2 B).

A hierarchical analysis of differential metabolites obtained in the two groups was carried out, and the difference in metabolic expression patterns between the two groups and within the same comparison was obtained, which is shown in Fig.  3 . KEGG pathway analysis of the cord blood metabolites associated with the excessive GWG group versus the control group is shown in Table  4 and Fig.  4 B. The metabolite enrichment analysis revealed that biosynthesis of unsaturated fatty acids was the most altered pathway between the excessive GWG and control groups ( P value < 0.01). There were 13 metabolites distributed in the enriched pathway. The levels of docosapentaenoic acid (DPA), docosahexaenoic acid (DHA), arachidonic acid, adrenic acid, palmitic acid, stearic acid, behenic acid, lignoceric acid, and erucic acid were increased in the excessive GWG group relative to those in the control group.

Our present study found that both maternal prepregnancy overweight/obesity and excessive GWG could affect umbilical cord blood metabolites, and they had different effects on these metabolites. Regardless of their gestational weight gain, the umbilical cord blood of prepregnancy overweight and obese mothers had 46 metabolites increased and 60 metabolites decreased compared with the umbilical cord blood of mothers with normal body weight and appropriate GWG. Steroid hormone biosynthesis and neuroactive ligand‒receptor interactions were the two top-ranking pathways enriched with these metabolites. Compared with mothers with normal prepregnancy BMI and appropriate GWG, in mothers with normal prepregnancy BMI but excessive GWG, the levels of 63 metabolites were increased and those of 46 metabolites were decreased in umbilical cord blood. Biosynthesis of unsaturated fatty acids was the most altered pathway enriched with these metabolites.

There were many differential metabolites in the cord blood between the prepregnancy overweight/obesity group and the control group and between the excessive GWG group and the control group. However, the roles of most of these differential metabolites are unknown. The levels of stearamide and methanandamide were increased in the prepregnancy overweight/obesity group. Stearamide, also known as octadecanamide or kemamide S, belongs to the class of organic compounds known as carboximidic acids. Stearamide, which is increased in the serum of patients with hepatic cirrhosis and sepsis, may be associated with the systemic inflammatory state [ 22 , 23 ]. Methanandamide is a stable analog of anandamide that participates in energy balance mainly by activating cannabinoid receptors. Methanandamide dose-dependently inhibits and excites tension-sensitive gastric vagal afferents (GVAs), which play a role in appetite regulation [ 24 ]. In mice fed a high-fat diet, only an inhibitory effect of methanandamide was observed, and GVA responses to tension were dampened [ 24 , 25 ]. These changes may contribute to the development and/or maintenance of obesity. Moreover, methanandamide can produce dose-related hypothermia and attenuate cocaine-induced hyperthermia by a cannabinoid 1-dopamine D2 receptor mechanism [ 26 ].

Metabolomic pathway analysis of the cord blood metabolite features in the prepregnancy overweight and obesity group identified two filtered significant pathways: steroid hormone biosynthesis and neuroactive ligand‒receptor interaction pathways. In the steroid hormone biosynthesis pathway, the levels of several glucocorticoids (including corticosterone, 11-deoxycortisol, cortisol, testosterone, and 7α-hydroxytestosterone) were decreased in the prepregnancy overweight/obesity group. In addition to the physiological role of glucocorticoids in the healthy neuroendocrine development and maturation of fetuses and babies, glucocorticoids are essential to human health by regulating different physiological events in mature organs and tissues, such as glucose metabolism, lipid biosynthesis and distribution, food intake, thermogenesis, and mood and learning patterns [ 27 ]. Glucocorticoids have been considered as a link between adverse early-life conditions and the development of metabolic disorders in later life [ 28 , 29 , 30 ]. However, there is still much controversy regarding the role of maternal obesity in the fetal–steroid hormone biosynthesis pathway. Studies of maternal obesity animal models showed that corticosterone and cortisol levels were increased in the offspring of obese mothers [ 31 , 32 ]. A study reported by Satu M Kumpulainen et al. showed that young adults born to mothers with higher early pregnancy BMIs show lower average levels of diurnal cortisol, especially in the morning [ 33 ]. Laura I. Stirrat et al. found that increased maternal BMI was associated with lower maternal cortisol, corticosterone, and 11-dehydrocorticosterone levels. However, there were no associations between maternal BMI and glucocorticoid levels in the cord blood [ 34 ]. The differences in the study protocols of these previous studies may explain the mixed findings, such as cortisol measured from peripheral blood, cord blood or saliva; variation in measurement time points; the number of samples. Although the effect of maternal obesity on fetal steroid hormone levels is controversial, dysregulation of glucocorticoids may be a plausible mechanism by which maternal obesity can increase the risk of metabolic disorders and mental health disorders in offspring.

The effect of excessive GWG on umbilical cord blood metabolites is different from that of maternal overweight and obesity. Compared with the control group, in the excessive GWG group, the level of thromboxane B2 was increased and the level of hippuric acid was decreased. Thromboxane B2, which is important in the platelet release reaction, is a stable, physiologically active compound formed in vivo from prostaglandin endoperoxides. Hippuric acid is an acyl glycine formed from the conjugation of benzoic acid with glycine. Several studies have confirmed that both thromboxane B2 and hippuric acid levels are associated with diet. Dietary fatty acids affect platelet thromboxane production [ 35 , 36 , 37 ]. In our study, several fatty acids (e.g., palmitic acid, stearic acid, behenic acid, and lignoceric acid) in the excessive GWG group were also increased, which may have led to the increase in thromboxane B2 levels. Hippuric acid can be detected after the consumption of whole grains and anthocyanin-rich bilberries [ 38 , 39 ]. A healthy diet intervention increased the signals for hippuric acid to incorporate polyunsaturated fatty acids [ 38 ], and the low level of hippuric acid was associated with lower fruit-vegetable intakes [ 39 ]. Maternal overnutrition and unhealthy dietary patterns are the main reasons for excessive GWG [ 40 , 41 ]. Therefore, we speculated that the differences in thromboxane B2 and hippuric acid between the excessive GWG and control groups were associated with maternal diet during pregnancy. The effect of these differential metabolites on the long-term metabolic health of offspring after birth needs further study.

Metabolomic pathway analysis of the cord blood metabolite features in the excessive GWG group identified that biosynthesis of unsaturated fatty acids was the filtered significant pathway. The levels of several fatty acids in this pathway were increased in the excessive GWG group, including long-chain saturated fatty acids (e.g., palmitic acid (C 16:0), stearic acid (C 18:0), behenic acid (C 22:0), and lignoceric acid (C 23:0)), monounsaturated fatty acids (erucic acid), and polyunsaturated fatty acids (e.g., DPA, DHA, arachidonic acid, and adrenic acid). Because perinatal fatty acid status can be influenced by maternal dietary modifications or supplementation [ 42 ], we speculated that maternal diet during pregnancy caused the difference in umbilical cord blood fatty acids between the excessive GWG and control groups. A large body of evidence from mechanistic studies supports the potential of fatty acids to influence later obesity. However, the possible mechanisms and observed relationships are complex and related to the types and patterns of fatty acids [ 43 , 44 ]. Maternal dietary fatty acids have been found to induce hypothalamic inflammation, cause epigenetic changes, and alter the mechanisms of energy control in offspring [ 43 ]. Evidence from cell culture and rodent studies showed that polyunsaturated fatty acids might serve several complex roles in fetuses, including the stimulation and/or inhibition regulation of adipocyte differentiation [ 44 ]. The questions of whether lower n-6 long-chain polyunsaturated fatty acid levels or higher n-3 long-chain polyunsaturated fatty acid levels are of more relevance and whether the long-term effects differ with different offspring ages remain [ 44 ]. Although there is a biologically plausible case for the relevance of perinatal fatty acid status in later obesity risk, available data in humans suggest that the influence of achievable modification of perinatal n-3/n-6 status is not sufficient to influence offspring obesity risk in the general population [ 45 ]. Further studies seem justified to clarify the reasons.

The advantage of our present study is that we simultaneously analyzed the effects of prepregnancy overweight/obesity and excessive GWG on cord blood metabolites and explored their differences. In addition, to exclude the effect of hyperglycemia on cord blood metabolites, both women with prepregnancy diabetes mellitus and gestational diabetes mellitus were excluded from our study. The limitation of our study is that it was a single-center study with a small sample, especially in the prepregnancy overweight/obesity group. In the future, we can expand the sample size and conduct a subgroup analysis of the prepregnancy overweight/obesity group and analyze the differences in the effects of different degrees of obesity on cord blood metabolites. The prepregnancy overweight/obesity group can be further divided into an appropriate GWG group and an excessive GWG group, and the differences in the effects of these two groups on umbilical cord blood metabolites can be analyzed. Moreover, the dietary pattern of the pregnant woman could affect the production of cord blood metabolites. We did not investigate the dietary patterns of the mothers in this study, which is another limitation of this study. In future studies, we should investigate maternal dietary patterns as a very important confounding variable.

In conclusion, our present study confirmed that both prepregnancy overweight/obesity and excessive GWG could affect umbilical cord blood metabolites, and they had different effects on these metabolites. Prepregnancy overweight and obesity affected the fetal steroid hormone biosynthesis pathway, while normal prepregnancy body weight but excessive GWG affected fetal fatty acid metabolism. This emphasizes the importance of preconception weight loss and maintaining an appropriate GWG, which are beneficial for the long-term metabolic health of offspring.

Availability of data and materials

Data sets generated during the current study are not publicly available but will be available from the corresponding author at a reasonable request. Responses to the request for the raw data will be judged by a committee including XXY and GHL.

Abbreviations

Excessive gestational weight gain

Ultrahigh-performance liquid chromatography tandem mass spectrometry

Type 2 diabetes mellitus

Nonalcoholic fatty liver disease

The developmental origins of health and disease

Body mass index

Diabetes mellitus

Gestational diabetes mellitus

Oral glucose tolerance test

Low birth weight

Standard deviation

The Human Metabolome Database

Kyoto Encylopaedia of Genes and Genomes

Principal component analysis

Partial least-squares discriminant analysis

Importance for the projection

Fold change

Invitro fertilization and embryo transfer

Triglyceride

Docosapentaenoic acid

Docosahexaenoic acid

Gastric vagal afferents

Collaborators GBDO, Afshin A, Forouzanfar MH, Reitsma MB, Sur P, Estep K, Lee A, Marczak L, Mokdad AH, Moradi-Lakeh M, et al. Health effects of overweight and obesity in 195 countries over 25 years. N Engl J Med. 2017;377(1):13–27.

Article   Google Scholar  

Bjerregaard LG, Jensen BW, Angquist L, Osler M, Sorensen TIA, Baker JL. Change in overweight from childhood to early adulthood and risk of type 2 diabetes. N Engl J Med. 2018;378(14):1302–12.

Article   PubMed   Google Scholar  

Sharma V, Coleman S, Nixon J, Sharples L, Hamilton-Shield J, Rutter H, Bryant M. A systematic review and meta-analysis estimating the population prevalence of comorbidities in children and adolescents aged 5 to 18 years. Obes Rev. 2019;20(10):1341–9.

Article   PubMed   PubMed Central   Google Scholar  

Llewellyn A, Simmonds M, Owen CG, Woolacott N. Childhood obesity as a predictor of morbidity in adulthood: a systematic review and meta-analysis. Obes Rev. 2016;17(1):56–67.

Article   CAS   PubMed   Google Scholar  

Poston L, Caleyachetty R, Cnattingius S, Corvalan C, Uauy R, Herring S, Gillman MW. Preconceptional and maternal obesity: epidemiology and health consequences. Lancet Diabetes Endocrinol. 2016;4(12):1025–36.

Pan XF, Wang L, Pan A. Epidemiology and determinants of obesity in China. Lancet Diabetes Endocrinol. 2021;9(6):373–92.

Barker DJ. The developmental origins of adult disease. J Am Coll Nutr. 2004;23(6 Suppl):588S-595S.

LifeCycle Project-Maternal O, Childhood Outcomes Study G, Voerman E, Santos S, Inskip H, Amiano P, Barros H, Charles MA, Chatzi L, Chrousos GP, et al. Association of gestational weight gain with adverse maternal and infant outcomes. JAMA. 2019;321(17):1702–15.

Aune D, Saugstad OD, Henriksen T, Tonstad S. Maternal body mass index and the risk of fetal death, stillbirth, and infant death: a systematic review and meta-analysis. JAMA. 2014;311(15):1536–46.

Ukah UV, Bayrampour H, Sabr Y, Razaz N, Chan WS, Lim KI, Lisonkova S. Association between gestational weight gain and severe adverse birth outcomes in Washington State, US: a population-based retrospective cohort study, 2004–2013. PLoS Med. 2019;16(12):e1003009.

Starling AP, Brinton JT, Glueck DH, Shapiro AL, Harrod CS, Lynch AM, Siega-Riz AM, Dabelea D. Associations of maternal BMI and gestational weight gain with neonatal adiposity in the Healthy Start study. Am J Clin Nutr. 2015;101(2):302–9.

Voerman E, Santos S, Patro Golab B, Amiano P, Ballester F, Barros H, Bergstrom A, Charles MA, Chatzi L, Chevrier C, et al. Maternal body mass index, gestational weight gain, and the risk of overweight and obesity across childhood: an individual participant data meta-analysis. PLoS Med. 2019;16(2):e1002744.

Heslehurst N, Vieira R, Akhter Z, Bailey H, Slack E, Ngongalah L, Pemu A, Rankin J. The association between maternal body mass index and child obesity: a systematic review and meta-analysis. PLoS Med. 2019;16(6):e1002817.

Newgard CB. Metabolomics and metabolic diseases: where do we stand? Cell Metab. 2017;25(1):43–56.

Johnson CH, Ivanisevic J, Siuzdak G. Metabolomics: beyond biomarkers and towards mechanisms. Nat Rev Mol Cell Biol. 2016;17(7):451–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Hivert MF, Perng W, Watkins SM, Newgard CS, Kenny LC, Kristal BS, Patti ME, Isganaitis E, DeMeo DL, Oken E, et al. Metabolomics in the developmental origins of obesity and its cardiometabolic consequences. J Dev Orig Health Dis. 2015;6(2):65–78.

Schlueter RJ, Al-Akwaa FM, Benny PA, Gurary A, Xie G, Jia W, Chun SJ, Chern I, Garmire LX. Prepregnant obesity of mothers in a multiethnic cohort is associated with cord blood metabolomic changes in offspring. J Proteome Res. 2020;19(4):1361–74.

Shokry E, Marchioro L, Uhl O, Bermudez MG, Garcia-Santos JA, Segura MT, Campoy C, Koletzko B. Impact of maternal BMI and gestational diabetes mellitus on maternal and cord blood metabolome: results from the PREOBE cohort study. Acta Diabetol. 2019;56(4):421–30.

Chen C, Lu FC, Department of Disease Control Ministry of Health PRC. The guidelines for prevention and control of overweight and obesity in Chinese adults. Biomed Environ Sci. 2004;17(Suppl):1–36.

PubMed   Google Scholar  

The American College of Obstetricians and Gynecologists. Macrosomia: ACOG practice bulletin, number 216. Obstet Gynecol. 2020;135(1):e18–e35.

Goldenberg RL, Culhane JF. Low birth weight in the United States. Am J Clin Nutr. 2007;85(2):584S-590S.

Lian JS, Liu W, Hao SR, Guo YZ, Huang HJ, Chen DY, Xie Q, Pan XP, Xu W, Yuan WX, et al. A serum metabonomic study on the difference between alcohol- and HBV-induced liver cirrhosis by ultraperformance liquid chromatography coupled to mass spectrometry plus quadrupole time-of-flight mass spectrometry. Chin Med J (Engl). 2011;124(9):1367–73.

CAS   PubMed   Google Scholar  

Ding W, Xu S, Zhou B, Zhou R, Liu P, Hui X, Long Y, Su L. Dynamic plasma lipidomic analysis revealed cholesterol ester and amides associated with sepsis development in critically Ill patients after cardiovascular surgery with cardiopulmonary bypass. J Pers Med. 2022;12(11):1838.

Christie S, O’Rielly R, Li H, Nunez-Salces M, Wittert GA, Page AJ. Modulatory effect of methanandamide on gastric vagal afferent satiety signals depends on nutritional status. J Physiol. 2020;598(11):2169–82.

Christie S, O’Rielly R, Li H, Wittert GA, Page AJ. High fat diet induced obesity alters endocannabinoid and ghrelin mediated regulation of components of the endocannabinoid system in nodose ganglia. Peptides. 2020;131:170371.

Rasmussen BA, Kim E, Unterwald EM, Rawls SM. Methanandamide attenuates cocaine-induced hyperthermia in rats by a cannabinoid CB1-dopamine D2 receptor mechanism. Brain Res. 2009;1260:7–14.

Facchi JC, Lima TAL, Oliveira LR, Costermani HO, Miranda GDS, de Oliveira JC. Perinatal programming of metabolic diseases: the role of glucocorticoids. Metabolism. 2020;104:154047.

Reynolds RM, Walker BR, Syddall HE, Andrew R, Wood PJ, Whorwood CB, Phillips DI. Altered control of cortisol secretion in adult men with low birth weight and cardiovascular risk factors. J Clin Endocrinol Metab. 2001;86(1):245–50.

Valtat B, Dupuis C, Zenaty D, Singh-Estivalet A, Tronche F, Breant B, Blondeau B. Genetic evidence of the programming of beta cell mass and function by glucocorticoids in mice. Diabetologia. 2011;54(2):350–9.

Jia Y, Li R, Cong R, Yang X, Sun Q, Parvizi N, Zhao R. Maternal low-protein diet affects epigenetic regulation of hepatic mitochondrial DNA transcription in a sex-specific manner in newborn piglets associated with GR binding to its promoter. PLoS ONE. 2013;8(5):e63855.

Rodriguez JS, Rodriguez-Gonzalez GL, Reyes-Castro LA, Ibanez C, Ramirez A, Chavira R, Larrea F, Nathanielsz PW, Zambrano E. Maternal obesity in the rat programs male offspring exploratory, learning and motivation behavior: prevention by dietary intervention pre-gestation or in gestation. Int J Dev Neurosci. 2012;30(2):75–81.

Tuersunjiang N, Odhiambo JF, Long NM, Shasa DR, Nathanielsz PW, Ford SP. Diet reduction to requirements in obese/overfed ewes from early gestation prevents glucose/insulin dysregulation and returns fetal adiposity and organ development to control levels. Am J Physiol Endocrinol Metab. 2013;305(7):E868-878.

Kumpulainen SM, Heinonen K, Kaseva N, Andersson S, Lano A, Reynolds RM, Wolke D, Kajantie E, Eriksson JG, Raikkonen K. Maternal early pregnancy body mass index and diurnal salivary cortisol in young adult offspring. Psychoneuroendocrinology. 2019;104:89–99.

Stirrat LI, Just G, Homer NZM, Andrew R, Norman JE, Reynolds RM. Glucocorticoids are lower at delivery in maternal, but not cord blood of obese pregnancies. Sci Rep. 2017;7(1):10263.

Prisco D, Filippini M, Francalanci I, Paniccia R, Gensini GF, Serneri GG. Effect of n-3 fatty acid ethyl ester supplementation on fatty acid composition of the single platelet phospholipids and on platelet functions. Metabolism. 1995;44(5):562–9.

Kaapa P, Uhari M, Nikkari T, Viinikka L, Ylikorkala O. Dietary fatty acids and platelet thromboxane production in puerperal women and their offspring. Am J Obstet Gynecol. 1986;155(1):146–9.

Teng KT, Chang CY, Kanthimathi MS, Tan AT, Nesaretnam K. Effects of amount and type of dietary fats on postprandial lipemia and thrombogenic markers in individuals with metabolic syndrome. Atherosclerosis. 2015;242(1):281–7.

Hanhineva K, Lankinen MA, Pedret A, Schwab U, Kolehmainen M, Paananen J, de Mello V, Sola R, Lehtonen M, Poutanen K, et al. Nontargeted metabolite profiling discriminates diet-specific biomarkers for consumption of whole grains, fatty fish, and bilberries in a randomized controlled trial. J Nutr. 2015;145(1):7–17.

Brunelli L, Davin A, Sestito G, Mimmi MC, De Simone G, Balducci C, Pansarasa O, Forloni G, Cereda C, Pastorelli R, et al. Plasmatic hippuric acid as a hallmark of frailty in an Italian cohort: the mediation effect of fruit-vegetable intake. J Gerontol A Biol Sci Med Sci. 2021;76(12):2081–9.

Ferreira LB, Lobo CV, Miranda A, Carvalho BDC, Santos LCD. Dietary patterns during pregnancy and gestational weight gain: a systematic review. Rev Bras Ginecol Obstet. 2022;44(5):540–7.

Tielemans MJ, Garcia AH, Peralta Santos A, Bramer WM, Luksa N, Luvizotto MJ, Moreira E, Topi G, de Jonge EA, Visser TL, et al. Macronutrient composition and gestational weight gain: a systematic review. Am J Clin Nutr. 2016;103(1):83–99.

Lewis RM, Wadsack C, Desoye G. Placental fatty acid transfer. Curr Opin Clin Nutr Metab Care. 2018;21(2):78–82.

Cesar HC, Pisani LP. Fatty-acid-mediated hypothalamic inflammation and epigenetic programming. J Nutr Biochem. 2017;42:1–6.

Demmelmair H, Koletzko B. Perinatal polyunsaturated fatty acid status and obesity risk. Nutrients. 2021;13(11):3882.

Hauner H, Brunner S. Early fatty acid exposure and later obesity risk. Curr Opin Clin Nutr Metab Care. 2015;18(2):113–7.

Download references

Acknowledgements

The authors thank the study participants for their involvement and research assistants for their help conducting the study.

This research was funded by the Beijing Natural Science Foundation, grant number 7214231.

Author information

Authors and affiliations.

Division of Endocrinology and Metabolism, Department of Obstetrics, Beijing Obstetrics and Gynecology Hospital, Capital Medical University, Beijing Maternal and Child Health Care Hospital, No. 251, Yaojiayuan Road, Chaoyang District, Beijing, 100026, China

Xianxian Yuan, Yuru Ma, Yan Zhao, Wei Zheng, Ruihua Yang, Lirui Zhang, Xin Yan & Guanghui Li

Department of Obstetrics and Gynecology, The Second Hospital of Jilin University, Changchun, 130041, Jilin, China

You can also search for this author in PubMed   Google Scholar

Contributions

XXY designed the study. XXY, WZ, LRZ and XY analyzed the data. YRM, JW, YZ and RHY took part in data collection and management. XXY wrote the manuscript. XXY and GHL reviewed the manuscript and contributed to manuscript revision. All authors contributed to the article and approved the submitted version. All authors reviewed the manuscript.

Corresponding author

Correspondence to Guanghui Li .

Ethics declarations

Ethics approval and consent to participate.

This study has been performed in accordance with the Declaration of Helsinki and has been approved by the ethics committee of Beijing Obstetrics and Gynecology Hospital, Capital Medical University (2021-KY-037). Informed consent was obtained from all subjects involved in the study to publish this paper. All methods were carried out in accordance with relevant guidelines and regulations in the declaration.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Yuan, X., Ma, Y., Wang, J. et al. The influence of maternal prepregnancy weight and gestational weight gain on the umbilical cord blood metabolome: a case–control study. BMC Pregnancy Childbirth 24 , 297 (2024). https://doi.org/10.1186/s12884-024-06507-x

Download citation

Received : 30 September 2023

Accepted : 11 April 2024

Published : 22 April 2024

DOI : https://doi.org/10.1186/s12884-024-06507-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Maternal obesity
• Gestational weight gain
• Offspring health
• Metabolites
• Umbilical cord blood

BMC Pregnancy and Childbirth

ISSN: 1471-2393

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

The case study approach

Sarah crowe.

1 Division of Primary Care, The University of Nottingham, Nottingham, UK

Kathrin Cresswell

2 Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK

Ann Robertson

3 School of Health in Social Science, The University of Edinburgh, Edinburgh, UK

Anthony Avery

Aziz sheikh.

The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.

Introduction

The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.

The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.

This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables ​ Tables1, 1 , ​ ,2, 2 , ​ ,3 3 and ​ and4) 4 ) and those of others to illustrate our discussion[ 3 - 7 ].

Example of a case study investigating the reasons for differences in recruitment rates of minority ethnic people in asthma research[ 3 ]

Example of a case study investigating the process of planning and implementing a service in Primary Care Organisations[ 4 ]

Example of a case study investigating the introduction of the electronic health records[ 5 ]

Example of a case study investigating the formal and informal ways students learn about patient safety[ 6 ]

What is a case study?

A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table ​ (Table5), 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.

Definitions of a case study

Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.

These are however not necessarily mutually exclusive categories. In the first of our examples (Table ​ (Table1), 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables ​ Tables2, 2 , ​ ,3 3 and ​ and4) 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 - 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table ​ (Table2) 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].

What are case studies used for?

According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables ​ Tables2 2 and ​ and3, 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table ​ (Table4 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.

Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table ​ (Table6). 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].

Example of epistemological approaches that may be used in case study research

How are case studies conducted?

Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.

Defining the case

Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table ​ Table7 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].

Example of a checklist for rating a case study proposal[ 8 ]

For example, in our evaluation of the introduction of electronic health records in English hospitals (Table ​ (Table3), 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.

Selecting the case(s)

The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table ​ (Table1) 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.

For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.

In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.

The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table ​ Table3) 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.

It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.

In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.

Collecting the data

In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 - 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table ​ (Table2 2 )[ 4 ].

Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.

In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.

Analysing, interpreting and reporting case studies

Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.

The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table ​ (Table1 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table ​ (Table3 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table ​ (Table4 4 )[ 6 ].

Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.

When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table ​ Table3, 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].

What are the potential pitfalls and how can these be avoided?

The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table ​ (Table4), 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.

Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table ​ Table8 8 )[ 8 , 18 - 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table ​ (Table9 9 )[ 8 ].

Potential pitfalls and mitigating actions when undertaking case study research

Stake's checklist for assessing the quality of a case study report[ 8 ]

Conclusions

The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/11/100/prepub

Acknowledgements

We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.

• Yin RK. Case study research, design and method. 4. London: Sage Publications Ltd.; 2009. [ Google Scholar ]
• Keen J, Packwood T. Qualitative research; case study evaluation. BMJ. 1995; 311 :444–446. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J. et al. Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009; 6 (10):1–11. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO) 2008. http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf
• Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T. et al. Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010; 41 :c4564. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P. the Patient Safety Education Study Group. Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010; 15 :4–10. doi: 10.1258/jhsrp.2009.009052. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• van Harten WH, Casparie TF, Fisscher OA. The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002; 60 (1):17–37. doi: 10.1016/S0168-8510(01)00187-7. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stake RE. The art of case study research. London: Sage Publications Ltd.; 1995. [ Google Scholar ]
• Sheikh A, Smeeth L, Ashcroft R. Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002; 52 (482):746–51. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• King G, Keohane R, Verba S. Designing Social Inquiry. Princeton: Princeton University Press; 1996. [ Google Scholar ]
• Doolin B. Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998; 13 :301–311. doi: 10.1057/jit.1998.8. [ CrossRef ] [ Google Scholar ]
• George AL, Bennett A. Case studies and theory development in the social sciences. Cambridge, MA: MIT Press; 2005. [ Google Scholar ]
• Eccles M. the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) Designing theoretically-informed implementation interventions. Implementation Science. 2006; 1 :1–8. doi: 10.1186/1748-5908-1-1. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A. Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005; 365 (9456):312–7. [ PubMed ] [ Google Scholar ]
• Sheikh A, Panesar SS, Lasserson T, Netuveli G. Recruitment of ethnic minorities to asthma studies. Thorax. 2004; 59 (7):634. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Hellström I, Nolan M, Lundh U. 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005; 4 :7–22. doi: 10.1177/1471301205049188. [ CrossRef ] [ Google Scholar ]
• Som CV. Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005; 18 :463–477. doi: 10.1108/09513550510608903. [ CrossRef ] [ Google Scholar ]
• Lincoln Y, Guba E. Naturalistic inquiry. Newbury Park: Sage Publications; 1985. [ Google Scholar ]
• Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001; 322 :1115–1117. doi: 10.1136/bmj.322.7294.1115. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mays N, Pope C. Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000; 320 :50–52. doi: 10.1136/bmj.320.7226.50. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mason J. Qualitative researching. London: Sage; 2002. [ Google Scholar ]
• Brazier A, Cooke K, Moravan V. Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008; 7 :5–17. doi: 10.1177/1534735407313395. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Miles MB, Huberman M. Qualitative data analysis: an expanded sourcebook. 2. CA: Sage Publications Inc.; 1994. [ Google Scholar ]
• Pope C, Ziebland S, Mays N. Analysing qualitative data. Qualitative research in health care. BMJ. 2000; 320 :114–116. doi: 10.1136/bmj.320.7227.114. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cresswell KM, Worth A, Sheikh A. Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010; 10 (1):67. doi: 10.1186/1472-6947-10-67. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001; 358 :483–488. doi: 10.1016/S0140-6736(01)05627-6. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Yin R. Case study research: design and methods. 2. Thousand Oaks, CA: Sage Publishing; 1994. [ Google Scholar ]
• Yin R. Enhancing the quality of case studies in health services research. Health Serv Res. 1999; 34 :1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Green J, Thorogood N. Qualitative methods for health research. 2. Los Angeles: Sage; 2009. [ Google Scholar ]
• Howcroft D, Trauth E. Handbook of Critical Information Systems Research, Theory and Application. Cheltenham, UK: Northampton, MA, USA: Edward Elgar; 2005. [ Google Scholar ]
• Blakie N. Approaches to Social Enquiry. Cambridge: Polity Press; 1993. [ Google Scholar ]
• Doolin B. Power and resistance in the implementation of a medical management information system. Info Systems J. 2004; 14 :343–362. doi: 10.1111/j.1365-2575.2004.00176.x. [ CrossRef ] [ Google Scholar ]
• Bloomfield BP, Best A. Management consultants: systems development, power and the translation of problems. Sociological Review. 1992; 40 :533–560. [ Google Scholar ]
• Shanks G, Parr A. Proceedings of the European Conference on Information Systems. Naples; 2003. Positivist, single case study research in information systems: A critical analysis. [ Google Scholar ]