case study monitoring and evaluation

• Case Study Evaluation Approach
• Learning Center

A case study evaluation approach can be an incredibly powerful tool for monitoring and evaluating complex programs and policies. By identifying common themes and patterns, this approach allows us to better understand the successes and challenges faced by the program. In this article, we’ll explore the benefits of using a case study evaluation approach in the monitoring and evaluation of projects, programs, and public policies.

Table of Contents

Introduction to Case Study Evaluation Approach

The advantages of a case study evaluation approach, types of case studies, potential challenges with a case study evaluation approach, guiding principles for successful implementation of a case study evaluation approach.

• Benefits of Incorporating the Case Study Evaluation Approach in the Monitoring and Evaluation of Projects and Programs

A case study evaluation approach is a great way to gain an in-depth understanding of a particular issue or situation. This type of approach allows the researcher to observe, analyze, and assess the effects of a particular situation on individuals or groups.

An individual, a location, or a project may serve as the focal point of a case study’s attention. Quantitative and qualitative data are frequently used in conjunction with one another.

It also allows the researcher to gain insights into how people react to external influences. By using a case study evaluation approach, researchers can gain insights into how certain factors such as policy change or a new technology have impacted individuals and communities. The data gathered through this approach can be used to formulate effective strategies for responding to changes and challenges. Ultimately, this monitoring and evaluation approach helps organizations make better decision about the implementation of their plans.

This approach can be used to assess the effectiveness of a policy, program, or initiative by considering specific elements such as implementation processes, outcomes, and impact. A case study evaluation approach can provide an in-depth understanding of the effectiveness of a program by closely examining the processes involved in its implementation. This includes understanding the context, stakeholders, and resources to gain insight into how well a program is functioning or has been executed. By evaluating these elements, it can help to identify areas for improvement and suggest potential solutions. The findings from this approach can then be used to inform decisions about policies, programs, and initiatives for improved outcomes.

It is also useful for determining if other policies, programs, or initiatives could be applied to similar situations in order to achieve similar results or improved outcomes. All in all, the case study monitoring evaluation approach is an effective method for determining the effectiveness of specific policies, programs, or initiatives. By researching and analyzing the successes of previous cases, this approach can be used to identify similar approaches that could be applied to similar situations in order to achieve similar results or improved outcomes.

A case study evaluation approach offers the advantage of providing in-depth insight into a particular program or policy. This can be accomplished by analyzing data and observations collected from a range of stakeholders such as program participants, service providers, and community members. The monitoring and evaluation approach is used to assess the impact of programs and inform the decision-making process to ensure successful implementation. The case study monitoring and evaluation approach can help identify any underlying issues that need to be addressed in order to improve program effectiveness. It also provides a reality check on how successful programs are actually working, allowing organizations to make adjustments as needed. Overall, a case study monitoring and evaluation approach helps to ensure that policies and programs are achieving their objectives while providing valuable insight into how they are performing overall.

By taking a qualitative approach to data collection and analysis, case study evaluations are able to capture nuances in the context of a particular program or policy that can be overlooked when relying solely on quantitative methods. Using this approach, insights can be gleaned from looking at the individual experiences and perspectives of actors involved, providing a more detailed understanding of the impact of the program or policy than is possible with other evaluation methodologies. As such, case study monitoring evaluation is an invaluable tool in assessing the effectiveness of a particular initiative, enabling more informed decision-making as well as more effective implementation of programs and policies.

Furthermore, this approach is an effective way to uncover experiential information that can help to inform the ongoing improvement of policy and programming over time All in all, the case study monitoring evaluation approach offers an effective way to uncover experiential information necessary to inform the ongoing improvement of policy and programming. By analyzing the data gathered from this systematic approach, stakeholders can gain deeper insight into how best to make meaningful and long-term changes in their respective organizations.

Case studies come in a variety of forms, each of which can be put to a unique set of evaluation tasks. Evaluators have come to a consensus on describing six distinct sorts of case studies, which are as follows: illustrative, exploratory, critical instance, program implementation, program effects, and cumulative.

Illustrative Case Study

An illustrative case study is a type of case study that is used to provide a detailed and descriptive account of a particular event, situation, or phenomenon. It is often used in research to provide a clear understanding of a complex issue, and to illustrate the practical application of theories or concepts.

An illustrative case study typically uses qualitative data, such as interviews, surveys, or observations, to provide a detailed account of the unit being studied. The case study may also include quantitative data, such as statistics or numerical measurements, to provide additional context or to support the qualitative data.

The goal of an illustrative case study is to provide a rich and detailed description of the unit being studied, and to use this information to illustrate broader themes or concepts. For example, an illustrative case study of a successful community development project may be used to illustrate the importance of community engagement and collaboration in achieving development goals.

One of the strengths of an illustrative case study is its ability to provide a detailed and nuanced understanding of a particular issue or phenomenon. By focusing on a single case, the researcher is able to provide a detailed and in-depth analysis that may not be possible through other research methods.

However, one limitation of an illustrative case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a single unit, it may not be representative of other similar units or situations.

A well-executed case study can shed light on wider research topics or concepts through its thorough and descriptive analysis of a specific event or phenomenon.

Exploratory Case Study

An exploratory case study is a type of case study that is used to investigate a new or previously unexplored phenomenon or issue. It is often used in research when the topic is relatively unknown or when there is little existing literature on the topic.

Exploratory case studies are typically qualitative in nature and use a variety of methods to collect data, such as interviews, observations, and document analysis. The focus of the study is to gather as much information as possible about the phenomenon being studied and to identify new and emerging themes or patterns.

The goal of an exploratory case study is to provide a foundation for further research and to generate hypotheses about the phenomenon being studied. By exploring the topic in-depth, the researcher can identify new areas of research and generate new questions to guide future research.

One of the strengths of an exploratory case study is its ability to provide a rich and detailed understanding of a new or emerging phenomenon. By using a variety of data collection methods, the researcher can gather a broad range of data and perspectives to gain a more comprehensive understanding of the phenomenon being studied.

However, one limitation of an exploratory case study is that the findings may not be generalizable to other contexts or populations. Because the study is focused on a new or previously unexplored phenomenon, the findings may not be applicable to other situations or populations.

Exploratory case studies are an effective research strategy for learning about novel occurrences, developing research hypotheses, and gaining a deep familiarity with a topic of study.

Critical Instance Case Study

A critical instance case study is a type of case study that focuses on a specific event or situation that is critical to understanding a broader issue or phenomenon. The goal of a critical instance case study is to analyze the event in depth and to draw conclusions about the broader issue or phenomenon based on the analysis.

A critical instance case study typically uses qualitative data, such as interviews, observations, or document analysis, to provide a detailed and nuanced understanding of the event being studied. The data are analyzed using various methods, such as content analysis or thematic analysis, to identify patterns and themes that emerge from the data.

The critical instance case study is often used in research when a particular event or situation is critical to understanding a broader issue or phenomenon. For example, a critical instance case study of a successful disaster response effort may be used to identify key factors that contributed to the success of the response, and to draw conclusions about effective disaster response strategies more broadly.

One of the strengths of a critical instance case study is its ability to provide a detailed and in-depth analysis of a particular event or situation. By focusing on a critical instance, the researcher is able to provide a rich and nuanced understanding of the event, and to draw conclusions about broader issues or phenomena based on the analysis.

However, one limitation of a critical instance case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a specific event or situation, the findings may not be applicable to other similar events or situations.

A critical instance case study is a valuable research method that can provide a detailed and nuanced understanding of a particular event or situation and can be used to draw conclusions about broader issues or phenomena based on the analysis.

Program Implementation Program Implementation

A program implementation case study is a type of case study that focuses on the implementation of a particular program or intervention. The goal of the case study is to provide a detailed and comprehensive account of the program implementation process, and to identify factors that contributed to the success or failure of the program.

Program implementation case studies typically use qualitative data, such as interviews, observations, and document analysis, to provide a detailed and nuanced understanding of the program implementation process. The data are analyzed using various methods, such as content analysis or thematic analysis, to identify patterns and themes that emerge from the data.

The program implementation case study is often used in research to evaluate the effectiveness of a particular program or intervention, and to identify strategies for improving program implementation in the future. For example, a program implementation case study of a school-based health program may be used to identify key factors that contributed to the success or failure of the program, and to make recommendations for improving program implementation in similar settings.

One of the strengths of a program implementation case study is its ability to provide a detailed and comprehensive account of the program implementation process. By using qualitative data, the researcher is able to capture the complexity and nuance of the implementation process, and to identify factors that may not be captured by quantitative data alone.

However, one limitation of a program implementation case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a specific program or intervention, the findings may not be applicable to other programs or interventions in different settings.

An effective research tool, a case study of program implementation may illuminate the intricacies of the implementation process and point the way towards future enhancements.

Program Effects Case Study

A program effects case study is a research method that evaluates the effectiveness of a particular program or intervention by examining its outcomes or effects. The purpose of this type of case study is to provide a detailed and comprehensive account of the program’s impact on its intended participants or target population.

A program effects case study typically employs both quantitative and qualitative data collection methods, such as surveys, interviews, and observations, to evaluate the program’s impact on the target population. The data is then analyzed using statistical and thematic analysis to identify patterns and themes that emerge from the data.

The program effects case study is often used to evaluate the success of a program and identify areas for improvement. For example, a program effects case study of a community-based HIV prevention program may evaluate the program’s effectiveness in reducing HIV transmission rates among high-risk populations and identify factors that contributed to the program’s success.

One of the strengths of a program effects case study is its ability to provide a detailed and nuanced understanding of a program’s impact on its intended participants or target population. By using both quantitative and qualitative data, the researcher can capture both the objective and subjective outcomes of the program and identify factors that may have contributed to the outcomes.

However, a limitation of the program effects case study is that it may not be generalizable to other populations or contexts. Since the case study focuses on a particular program and population, the findings may not be applicable to other programs or populations in different settings.

A program effects case study is a good way to do research because it can give a detailed look at how a program affects the people it is meant for. This kind of case study can be used to figure out what needs to be changed and how to make programs that work better.

Cumulative Case Study

A cumulative case study is a type of case study that involves the collection and analysis of multiple cases to draw broader conclusions. Unlike a single-case study, which focuses on one specific case, a cumulative case study combines multiple cases to provide a more comprehensive understanding of a phenomenon.

The purpose of a cumulative case study is to build up a body of evidence through the examination of multiple cases. The cases are typically selected to represent a range of variations or perspectives on the phenomenon of interest. Data is collected from each case using a range of methods, such as interviews, surveys, and observations.

The data is then analyzed across cases to identify common themes, patterns, and trends. The analysis may involve both qualitative and quantitative methods, such as thematic analysis and statistical analysis.

The cumulative case study is often used in research to develop and test theories about a phenomenon. For example, a cumulative case study of successful community-based health programs may be used to identify common factors that contribute to program success, and to develop a theory about effective community-based health program design.

One of the strengths of the cumulative case study is its ability to draw on a range of cases to build a more comprehensive understanding of a phenomenon. By examining multiple cases, the researcher can identify patterns and trends that may not be evident in a single case study. This allows for a more nuanced understanding of the phenomenon and helps to develop more robust theories.

However, one limitation of the cumulative case study is that it can be time-consuming and resource-intensive to collect and analyze data from multiple cases. Additionally, the selection of cases may introduce bias if the cases are not representative of the population of interest.

In summary, a cumulative case study is a valuable research method that can provide a more comprehensive understanding of a phenomenon by examining multiple cases. This type of case study is particularly useful for developing and testing theories and identifying common themes and patterns across cases.

When conducting a case study evaluation approach, one of the main challenges is the need to establish a contextually relevant research design that accounts for the unique factors of the case being studied. This requires close monitoring of the case, its environment, and relevant stakeholders. In addition, the researcher must build a framework for the collection and analysis of data that is able to draw meaningful conclusions and provide valid insights into the dynamics of the case. Ultimately, an effective case study monitoring evaluation approach will allow researchers to form an accurate understanding of their research subject.

Additionally, depending on the size and scope of the case, there may be concerns regarding the availability of resources and personnel that could be allocated to data collection and analysis. To address these issues, a case study monitoring evaluation approach can be adopted, which would involve a mix of different methods such as interviews, surveys, focus groups and document reviews. Such an approach could provide valuable insights into the effectiveness and implementation of the case in question. Additionally, this type of evaluation can be tailored to the specific needs of the case study to ensure that all relevant data is collected and respected.

When dealing with a highly sensitive or confidential subject matter within a case study, researchers must take extra measures to prevent bias during data collection as well as protect participant anonymity while also collecting valid data in order to ensure reliable results

Moreover, when conducting a case study evaluation it is important to consider the potential implications of the data gathered. By taking extra measures to prevent bias and protect participant anonymity, researchers can ensure reliable results while also collecting valid data. Maintaining confidentiality and deploying ethical research practices are essential when conducting a case study to ensure an unbiased and accurate monitoring evaluation.

When planning and implementing a case study evaluation approach, it is important to ensure the guiding principles of research quality, data collection, and analysis are met. To ensure these principles are upheld, it is essential to develop a comprehensive monitoring and evaluation plan. This plan should clearly outline the steps to be taken during the data collection and analysis process. Furthermore, the plan should provide detailed descriptions of the project objectives, target population, key indicators, and timeline. It is also important to include metrics or benchmarks to monitor progress and identify any potential areas for improvement. By implementing such an approach, it will be possible to ensure that the case study evaluation approach yields valid and reliable results.

To ensure successful implementation, it is essential to establish a reliable data collection process that includes detailed information such as the scope of the study, the participants involved, and the methods used to collect data. Additionally, it is important to have a clear understanding of what will be examined through the evaluation process and how the results will be used. All in all, it is essential to establish a sound monitoring evaluation approach for a successful case study implementation. This includes creating a reliable data collection process that encompasses the scope of the study, the participants involved, and the methods used to collect data. It is also imperative to have an understanding of what will be examined and how the results will be utilized. Ultimately, effective planning is key to ensure that the evaluation process yields meaningful insights.

Benefits of Incorporating the Case Study Evaluation Approach in the Monitoring and Evaluation of Projects and Programmes

Using a case study approach in monitoring and evaluation allows for a more detailed and in-depth exploration of the project’s success, helping to identify key areas of improvement and successes that may have been overlooked through traditional evaluation. Through this case study method, specific data can be collected and analyzed to identify trends and different perspectives that can support the evaluation process. This data can allow stakeholders to gain a better understanding of the project’s successes and failures, helping them make informed decisions on how to strengthen current activities or shape future initiatives. From a monitoring and evaluation standpoint, this approach can provide an increased level of accuracy in terms of accurately assessing the effectiveness of the project.

This can provide valuable insights into what works—and what doesn’t—when it comes to implementing projects and programs, aiding decision-makers in making future plans that better meet their objectives However, monitoring and evaluation is just one approach to assessing the success of a case study. It does provide a useful insight into what initiatives may be successful, but it is important to note that there are other effective research methods, such as surveys and interviews, that can also help to further evaluate the success of a project or program.

In conclusion, a case study evaluation approach can be incredibly useful in monitoring and evaluating complex programs and policies. By exploring key themes, patterns and relationships, organizations can gain a detailed understanding of the successes, challenges and limitations of their program or policy. This understanding can then be used to inform decision-making and improve outcomes for those involved. With its ability to provide an in-depth understanding of a program or policy, the case study evaluation approach has become an invaluable tool for monitoring and evaluation professionals.

Leave a Comment Cancel Reply

Your email address will not be published.

How strong is my Resume?

Only 2% of resumes land interviews.

Land a better, higher-paying career

Jobs for You

Information coordinator – usaid guatemala planning and program support office.

• United States (Remote)

Director of Market Influence

• Atlanta, GA, USA
• Habitat for Humanity International

CLA Coordinator/Report Officer

• Bosnia and Herzegovina

Director of Collaborating, Learning, and Adapting (CLA) – Bosnia and Herzegovina

Senior transition and closeout consultant, global health technical and mission support (gh-tams), intern – pricing and budget.

• United States

Research Technical Advisor

• South Bend, IN, USA (Remote)
• University of Notre Dame

YMELP II Short-Term Technical Assistance (STTA)

Water, sanitation and hygiene advisor (wash) – usaid/drc.

• Democratic Republic of the Congo

Health Supply Chain Specialist – USAID/DRC

Chief of party – bosnia and herzegovina, project manager i, business development associate, director of finance and administration, request for information – collecting information on potential partners for local works evaluation.

• Washington, USA

Services you might be interested in

Useful guides ....

How to Create a Strong Resume

Monitoring And Evaluation Specialist Resume

Resume Length for the International Development Sector

Types of Evaluation

Monitoring, Evaluation, Accountability, and Learning (MEAL)

LAND A JOB REFERRAL IN 2 WEEKS (NO ONLINE APPS!)

Sign Up & To Get My Free Referral Toolkit Now:

The Monitoring and Evaluation Toolkit

This section asks:

What is a case study?

• What are the different types of case study ?
• What are the advantages and disadvantages of a case study ?
• How to Use Case Studies as part of your Monitoring & Evaluation?

There are many different text books and websites explaining the use of case studies and this section draws heavily on those of Lamar University and the NCBI (worked examples), as well as on the author’s own extensive research experience.

If you are monitoring/ evaluating a project, you may already have obtained general information about your target school, village, hospital or farming community. But the information you have is broad and imprecise. It may contain a lot of statistics but may not give you a feel for what is really going on in that village, school, hospital or farming community.

Case studies can provide this depth. They focus on a particular person, patient, village, group within a community or other sub-set of a wider group. They can be used to illustrate wider trends or to show that the case you are examining is broadly similar to other cases or really quite different.

In other words, a case study examines a person, place, event, phenomenon, or other type of subject of analysis in order to extrapolate key themes and results that help predict future trends, illuminate previously hidden issues that can be applied to practice, and/or provide a means for understanding an important research problem with greater clarity.

A case study paper usually examines a single subject of analysis, but case study papers can also be designed as a comparative investigation that shows relationships between two or among more than two subjects. The methods used to study a case can rest within a quantitative, qualitative, or a mixture of the two.

Different types of case study

There are many types of case study. Drawing on the work of Lamar University and the NCBI , some of the best-known types are set out below.

It is best not to worry too much about the nuances that differentiate types of case study. The key is to recognise that the case study is a detailed illustration of how your project or programme has worked or failed to work on an individual, hospital, school, target community or other group/ economic sector.

• Explanatory case studies aim to answer ‘how’ or ’why’ questions with little control on behalf of researcher over occurrence of events. This type of case studies focus on phenomena within the contexts of real-life situations. Example: “An investigation into the reasons of the global financial and economic crisis of 2008 – 2010.”
• Descriptive case studies aim to analyze the sequence of interpersonal events after a certain amount of time has passed. Studies in business research belonging to this category usually describe culture or sub-culture, and they attempt to discover the key phenomena. Example: Impact of increasing levels of funding for prosthetic limbs on the employment opportunities of amputees. A case study of the West Point community of Monrovia (Liberia).
• Exploratory case studies aim to find answers to the questions of ‘what’ or ‘who’. Exploratory case study data collection method is often accompanied by additional data collection method(s) such as interviews, questionnaires, experiments etc. Example: “A study into differences of local community governance practices between a town in francophone Cameroon and a similar-sized town in anglophone Cameroon.”
• Critical instance : This examines a single instance of unique interest, or serves as a critical test of an assertion about a programme, problem or strategy. The focus might be on the economic or human cost of a tsunami or volcanic eruption in a particular area.
• Representative : This relates to case which is typical in nature and representative of other cases that you might examine. An example might be a mother, with a part-time job and four children, living in a community where this is the norm
• Deviant : This refers to a case which is out of line with others. Deviant cases can be particularly interesting and often attract greater attention from analysts. A patient with immunity to a particular virus is worth studying as that study might provide clues to a possible cure to that virus
• Prototypical : This involves a case which is ahead of the curve in some way and has the capacity to set a trend. A particular African town or city may be a free bicyle loan scheme and the experiences of that town might suggest a future path to be followed by other towns and regions.
• Most similar cases : Here you are looking at more than one case and you have selected two cases which have a preponderance of features in common. You might for example be looking at two schools, each of which teaches boys aged from 11-15 and each of which charges similar fees. They are located in the same country but are in different regions where the local authorities devote different levels of resource to secondary school education. You may have a project in each of these areas and you may wish to explain why your project has been more successful in one than the other.
• Most dissimilar cases : these are cases which are, in most key respects, very different and where you might expect to find different outcomes. You might for example select a class of top-ranking pupils and compare it with a class of bottom-ranking puils. This could help to bring out the factors that contribute to or detract from academic success.

Advantages and Disadvantages of Case Study Method

• It helps explain how and why a phenomenon has occurred, thereby going beyond numerical data
• It allows the integration of qualitative and quantitative data collection and analysis methods
• It provides rich (or ‘thick) detail and is well suited to capturing complexities of real-life situations and the challenges facing real people
• Case studies (sometimes illustrated with quotations from beneficiairies/ stakeholder and with photographs) are often included as boxes in project reports and evaluations, thereby adding adding a human dimension to an otherwise dry description and data.
• Case studies may offer you an opportunity to gather evidence that challenges prevailing assumptions about a research problem and provide a new set of recommendations applied to practice that have not been tested previously.

Disadvantages

• Case studies may be marked by a lack of rigour (e.g. a study may not be sufficiently in-depth or a single case study may not be sufficient)
• Single case studies may offer very little basis for generalisations of findings and conclusions.
• Case studies often tend to be success stories (so they may involve a degree of bias).

Where to next?

Click here to return to the top of the page, here to return to step 3 (Data checking) and here to see a short worked example of a metrics-based evaluation.

15.7 Evaluation: Presentation and Analysis of Case Study

Learning outcomes.

By the end of this section, you will be able to:

• Revise writing to follow the genre conventions of case studies.
• Evaluate the effectiveness and quality of a case study report.

Case studies follow a structure of background and context , methods , findings , and analysis . Body paragraphs should have main points and concrete details. In addition, case studies are written in formal language with precise wording and with a specific purpose and audience (generally other professionals in the field) in mind. Case studies also adhere to the conventions of the discipline’s formatting guide ( APA Documentation and Format in this study). Compare your case study with the following rubric as a final check.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/writing-guide/pages/1-unit-introduction

• Authors: Michelle Bachelor Robinson, Maria Jerskey, featuring Toby Fulwiler
• Publisher/website: OpenStax
• Book title: Writing Guide with Handbook
• Publication date: Dec 21, 2021
• Location: Houston, Texas
• Book URL: https://openstax.org/books/writing-guide/pages/1-unit-introduction
• Section URL: https://openstax.org/books/writing-guide/pages/15-7-evaluation-presentation-and-analysis-of-case-study

© Dec 19, 2023 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

• Open access
• Published: 27 November 2020

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials

• Aileen Grant 1 ,
• Carol Bugge 2 &
• Mary Wells 3  

Trials volume  21 , Article number:  982 ( 2020 ) Cite this article

12k Accesses

10 Citations

5 Altmetric

Metrics details

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation of context. Case study is one appropriate methodology, but there is little guidance about what case study design can offer the study of context in trials. We address this gap in the literature by presenting a number of important considerations for process evaluation using a case study design.

In this paper, we define context, the relationship between complex interventions and context, and describe case study design methodology. A well-designed process evaluation using case study should consider the following core components: the purpose; definition of the intervention; the trial design, the case, the theories or logic models underpinning the intervention, the sampling approach and the conceptual or theoretical framework. We describe each of these in detail and highlight with examples from recently published process evaluations.

Conclusions

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation. We provide a comprehensive overview of the issues for process evaluation design to consider when using a case study design.

Trial registration

DQIP - ClinicalTrials.gov number, NCT01425502 - OPAL - ISRCTN57746448

Peer Review reports

Contribution to the literature

We illustrate how case study methodology can explore the complex, dynamic and uncertain relationship between context and interventions within trials.

We depict different case study designs and illustrate there is not one formula and that design needs to be tailored to the context and trial design.

Case study can support comparisons between intervention and control arms and between cases within arms to uncover and explain differences in detail.

We argue that case study can illustrate how components have evolved and been redefined through implementation.

Key issues for consideration in case study design within process evaluations are presented and illustrated with examples.

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail and whether they can be transferred to other settings and populations. However, historically, not all trials have had a process evaluation component, nor have they sufficiently reported aspects of context, resulting in poor uptake of trial findings [ 1 ]. Considerations of context are often absent from published process evaluations, with few studies acknowledging, taking account of or describing context during implementation, or assessing the impact of context on implementation [ 2 , 3 ]. At present, evidence from trials is not being used in a timely manner [ 4 , 5 ], and this can negatively impact on patient benefit and experience [ 6 ]. It takes on average 17 years for knowledge from research to be implemented into practice [ 7 ]. Suitable methodologies are therefore needed that allow for context to be exposed; one appropriate methodological approach is case study [ 8 , 9 ].

In 2015, the Medical Research Council (MRC) published guidance for process evaluations [ 10 ]. This was a key milestone in legitimising as well as providing tools, methods and a framework for conducting process evaluations. Nevertheless, as with all guidance, there is a need for reflection, challenge and refinement. There have been a number of critiques of the MRC guidance, including that interventions should be considered as events in systems [ 11 , 12 , 13 , 14 ]; a need for better use, critique and development of theories [ 15 , 16 , 17 ]; and a need for more guidance on integrating qualitative and quantitative data [ 18 , 19 ]. Although the MRC process evaluation guidance does consider appropriate qualitative and quantitative methods, it does not mention case study design and what it can offer the study of context in trials.

The case study methodology is ideally suited to real-world, sustainable intervention development and evaluation because it can explore and examine contemporary complex phenomena, in depth, in numerous contexts and using multiple sources of data [ 8 ]. Case study design can capture the complexity of the case, the relationship between the intervention and the context and how the intervention worked (or not) [ 8 ]. There are a number of textbooks on a case study within the social science fields [ 8 , 9 , 20 ], but there are no case study textbooks and a paucity of useful texts on how to design, conduct and report case study within the health arena. Few examples exist within the trial design and evaluation literature [ 3 , 21 ]. Therefore, guidance to enable well-designed process evaluations using case study methodology is required.

We aim to address the gap in the literature by presenting a number of important considerations for process evaluation using a case study design. First, we define the context and describe the relationship between complex health interventions and context.

What is context?

While there is growing recognition that context interacts with the intervention to impact on the intervention’s effectiveness [ 22 ], context is still poorly defined and conceptualised. There are a number of different definitions in the literature, but as Bate et al. explained ‘almost universally, we find context to be an overworked word in everyday dialogue but a massively understudied and misunderstood concept’ [ 23 ]. Ovretveit defines context as ‘everything the intervention is not’ [ 24 ]. This last definition is used by the MRC framework for process evaluations [ 25 ]; however; the problem with this definition is that it is highly dependent on how the intervention is defined. We have found Pfadenhauer et al.’s definition useful:

Context is conceptualised as a set of characteristics and circumstances that consist of active and unique factors that surround the implementation. As such it is not a backdrop for implementation but interacts, influences, modifies and facilitates or constrains the intervention and its implementation. Context is usually considered in relation to an intervention or object, with which it actively interacts. A boundary between the concepts of context and setting is discernible: setting refers to the physical, specific location in which the intervention is put into practice. Context is much more versatile, embracing not only the setting but also roles, interactions and relationships [ 22 ].

Traditionally, context has been conceptualised in terms of barriers and facilitators, but what is a barrier in one context may be a facilitator in another, so it is the relationship and dynamics between the intervention and context which are the most important [ 26 ]. There is a need for empirical research to really understand how different contextual factors relate to each other and to the intervention. At present, research studies often list common contextual factors, but without a depth of meaning and understanding, such as government or health board policies, organisational structures, professional and patient attitudes, behaviours and beliefs [ 27 ]. The case study methodology is well placed to understand the relationship between context and intervention where these boundaries may not be clearly evident. It offers a means of unpicking the contextual conditions which are pertinent to effective implementation.

The relationship between complex health interventions and context

Health interventions are generally made up of a number of different components and are considered complex due to the influence of context on their implementation and outcomes [ 3 , 28 ]. Complex interventions are often reliant on the engagement of practitioners and patients, so their attitudes, behaviours, beliefs and cultures influence whether and how an intervention is effective or not. Interventions are context-sensitive; they interact with the environment in which they are implemented. In fact, many argue that interventions are a product of their context, and indeed, outcomes are likely to be a product of the intervention and its context [ 3 , 29 ]. Within a trial, there is also the influence of the research context too—so the observed outcome could be due to the intervention alone, elements of the context within which the intervention is being delivered, elements of the research process or a combination of all three. Therefore, it can be difficult and unhelpful to separate the intervention from the context within which it was evaluated because the intervention and context are likely to have evolved together over time. As a result, the same intervention can look and behave differently in different contexts, so it is important this is known, understood and reported [ 3 ]. Finally, the intervention context is dynamic; the people, organisations and systems change over time, [ 3 ] which requires practitioners and patients to respond, and they may do this by adapting the intervention or contextual factors. So, to enable researchers to replicate successful interventions, or to explain why the intervention was not successful, it is not enough to describe the components of the intervention, they need to be described by their relationship to their context and resources [ 3 , 28 ].

What is a case study?

Case study methodology aims to provide an in-depth, holistic, balanced, detailed and complete picture of complex contemporary phenomena in its natural context [ 8 , 9 , 20 ]. In this case, the phenomena are the implementation of complex interventions in a trial. Case study methodology takes the view that the phenomena can be more than the sum of their parts and have to be understood as a whole [ 30 ]. It is differentiated from a clinical case study by its analytical focus [ 20 ].

The methodology is particularly useful when linked to trials because some of the features of the design naturally fill the gaps in knowledge generated by trials. Given the methodological focus on understanding phenomena in the round, case study methodology is typified by the use of multiple sources of data, which are more commonly qualitatively guided [ 31 ]. The case study methodology is not epistemologically specific, like realist evaluation, and can be used with different epistemologies [ 32 ], and with different theories, such as Normalisation Process Theory (which explores how staff work together to implement a new intervention) or the Consolidated Framework for Implementation Research (which provides a menu of constructs associated with effective implementation) [ 33 , 34 , 35 ]. Realist evaluation can be used to explore the relationship between context, mechanism and outcome, but case study differs from realist evaluation by its focus on a holistic and in-depth understanding of the relationship between an intervention and the contemporary context in which it was implemented [ 36 ]. Case study enables researchers to choose epistemologies and theories which suit the nature of the enquiry and their theoretical preferences.

Designing a process evaluation using case study

An important part of any study is the research design. Due to their varied philosophical positions, the seminal authors in the field of case study have different epistemic views as to how a case study should be conducted [ 8 , 9 ]. Stake takes an interpretative approach (interested in how people make sense of their world), and Yin has more positivistic leanings, arguing for objectivity, validity and generalisability [ 8 , 9 ].

Regardless of the philosophical background, a well-designed process evaluation using case study should consider the following core components: the purpose; the definition of the intervention, the trial design, the case, and the theories or logic models underpinning the intervention; the sampling approach; and the conceptual or theoretical framework [ 8 , 9 , 20 , 31 , 33 ]. We now discuss these critical components in turn, with reference to two process evaluations that used case study design, the DQIP and OPAL studies [ 21 , 37 , 38 , 39 , 40 , 41 ].

The purpose of a process evaluation is to evaluate and explain the relationship between the intervention and its components, to context and outcome. It can help inform judgements about validity (by exploring the intervention components and their relationship with one another (construct validity), the connections between intervention and outcomes (internal validity) and the relationship between intervention and context (external validity)). It can also distinguish between implementation failure (where the intervention is poorly delivered) and intervention failure (intervention design is flawed) [ 42 , 43 ]. By using a case study to explicitly understand the relationship between context and the intervention during implementation, the process evaluation can explain the intervention effects and the potential generalisability and optimisation into routine practice [ 44 ].

The DQIP process evaluation aimed to qualitatively explore how patients and GP practices responded to an intervention designed to reduce high-risk prescribing of nonsteroidal anti-inflammatory drugs (NSAIDs) and/or antiplatelet agents (see Table  1 ) and quantitatively examine how change in high-risk prescribing was associated with practice characteristics and implementation processes. The OPAL process evaluation (see Table  2 ) aimed to quantitatively understand the factors which influenced the effectiveness of a pelvic floor muscle training intervention for women with urinary incontinence and qualitatively explore the participants’ experiences of treatment and adherence.

Defining the intervention and exploring the theories or assumptions underpinning the intervention design

Process evaluations should also explore the utility of the theories or assumptions underpinning intervention design [ 49 ]. Not all theories underpinning interventions are based on a formal theory, but they based on assumptions as to how the intervention is expected to work. These can be depicted as a logic model or theory of change [ 25 ]. To capture how the intervention and context evolve requires the intervention and its expected mechanisms to be clearly defined at the outset [ 50 ]. Hawe and colleagues recommend defining interventions by function (what processes make the intervention work) rather than form (what is delivered) [ 51 ]. However, in some cases, it may be useful to know if some of the components are redundant in certain contexts or if there is a synergistic effect between all the intervention components.

The DQIP trial delivered two interventions, one intervention was delivered to professionals with high fidelity and then professionals delivered the other intervention to patients by form rather than function allowing adaptations to the local context as appropriate. The assumptions underpinning intervention delivery were prespecified in a logic model published in the process evaluation protocol [ 52 ].

Case study is well placed to challenge or reinforce the theoretical assumptions or redefine these based on the relationship between the intervention and context. Yin advocates the use of theoretical propositions; these direct attention to specific aspects of the study for investigation [ 8 ] can be based on the underlying assumptions and tested during the course of the process evaluation. In case studies, using an epistemic position more aligned with Yin can enable research questions to be designed, which seek to expose patterns of unanticipated as well as expected relationships [ 9 ]. The OPAL trial was more closely aligned with Yin, where the research team predefined some of their theoretical assumptions, based on how the intervention was expected to work. The relevant parts of the data analysis then drew on data to support or refute the theoretical propositions. This was particularly useful for the trial as the prespecified theoretical propositions linked to the mechanisms of action on which the intervention was anticipated to have an effect (or not).

Tailoring to the trial design

Process evaluations need to be tailored to the trial, the intervention and the outcomes being measured [ 45 ]. For example, in a stepped wedge design (where the intervention is delivered in a phased manner), researchers should try to ensure process data are captured at relevant time points or in a two-arm or multiple arm trial, ensure data is collected from the control group(s) as well as the intervention group(s). In the DQIP trial, a stepped wedge trial, at least one process evaluation case, was sampled per cohort. Trials often continue to measure outcomes after delivery of the intervention has ceased, so researchers should also consider capturing ‘follow-up’ data on contextual factors, which may continue to influence the outcome measure. The OPAL trial had two active treatment arms so collected process data from both arms. In addition, as the trial was interested in long-term adherence, the trial and the process evaluation collected data from participants for 2 years after the intervention was initially delivered, providing 24 months follow-up data, in line with the primary outcome for the trial.

Defining the case

Case studies can include single or multiple cases in their design. Single case studies usually sample typical or unique cases, their advantage being the depth and richness that can be achieved over a long period of time. The advantages of multiple case study design are that cases can be compared to generate a greater depth of analysis. Multiple case study sampling may be carried out in order to test for replication or contradiction [ 8 ]. Given that trials are often conducted over a number of sites, a multiple case study design is more sensible for process evaluations, as there is likely to be variation in implementation between sites. Case definition may occur at a variety of levels but is most appropriate if it reflects the trial design. For example, a case in an individual patient level trial is likely to be defined as a person/patient (e.g. a woman with urinary incontinence—OPAL trial) whereas in a cluster trial, a case is like to be a cluster, such as an organisation (e.g. a general practice—DQIP trial). Of course, the process evaluation could explore cases with less distinct boundaries, such as communities or relationships; however, the clarity with which these cases are defined is important, in order to scope the nature of the data that will be generated.

Carefully sampled cases are critical to a good case study as sampling helps inform the quality of the inferences that can be made from the data [ 53 ]. In both qualitative and quantitative research, how and how many participants to sample must be decided when planning the study. Quantitative sampling techniques generally aim to achieve a random sample. Qualitative research generally uses purposive samples to achieve data saturation, occurring when the incoming data produces little or no new information to address the research questions. The term data saturation has evolved from theoretical saturation in conventional grounded theory studies; however, its relevance to other types of studies is contentious as the term saturation seems to be widely used but poorly justified [ 54 ]. Empirical evidence suggests that for in-depth interview studies, saturation occurs at 12 interviews for thematic saturation, but typically more would be needed for a heterogenous sample higher degrees of saturation [ 55 , 56 ]. Both DQIP and OPAL case studies were huge with OPAL designed to interview each of the 40 individual cases four times and DQIP designed to interview the lead DQIP general practitioner (GP) twice (to capture change over time), another GP and the practice manager from each of the 10 organisational cases. Despite the plethora of mixed methods research textbooks, there is very little about sampling as discussions typically link to method (e.g. interviews) rather than paradigm (e.g. case study).

Purposive sampling can improve the generalisability of the process evaluation by sampling for greater contextual diversity. The typical or average case is often not the richest source of information. Outliers can often reveal more important insights, because they may reflect the implementation of the intervention using different processes. Cases can be selected from a number of criteria, which are not mutually exclusive, to enable a rich and detailed picture to be built across sites [ 53 ]. To avoid the Hawthorne effect, it is recommended that process evaluations sample from both intervention and control sites, which enables comparison and explanation. There is always a trade-off between breadth and depth in sampling, so it is important to note that often quantity does not mean quality and that carefully sampled cases can provide powerful illustrative examples of how the intervention worked in practice, the relationship between the intervention and context and how and why they evolved together. The qualitative components of both DQIP and OPAL process evaluations aimed for maximum variation sampling. Please see Table  1 for further information on how DQIP’s sampling frame was important for providing contextual information on processes influencing effective implementation of the intervention.

Conceptual and theoretical framework

A conceptual or theoretical framework helps to frame data collection and analysis [ 57 ]. Theories can also underpin propositions, which can be tested in the process evaluation. Process evaluations produce intervention-dependent knowledge, and theories help make the research findings more generalizable by providing a common language [ 16 ]. There are a number of mid-range theories which have been designed to be used with process evaluation [ 34 , 35 , 58 ]. The choice of the appropriate conceptual or theoretical framework is, however, dependent on the philosophical and professional background of the research. The two examples within this paper used our own framework for the design of process evaluations, which proposes a number of candidate processes which can be explored, for example, recruitment, delivery, response, maintenance and context [ 45 ]. This framework was published before the MRC guidance on process evaluations, and both the DQIP and OPAL process evaluations were designed before the MRC guidance was published. The DQIP process evaluation explored all candidates in the framework whereas the OPAL process evaluation selected four candidates, illustrating that process evaluations can be selective in what they explore based on the purpose, research questions and resources. Furthermore, as Kislov and colleagues argue, we also have a responsibility to critique the theoretical framework underpinning the evaluation and refine theories to advance knowledge [ 59 ].

Data collection

An important consideration is what data to collect or measure and when. Case study methodology supports a range of data collection methods, both qualitative and quantitative, to best answer the research questions. As the aim of the case study is to gain an in-depth understanding of phenomena in context, methods are more commonly qualitative or mixed method in nature. Qualitative methods such as interviews, focus groups and observation offer rich descriptions of the setting, delivery of the intervention in each site and arm, how the intervention was perceived by the professionals delivering the intervention and the patients receiving the intervention. Quantitative methods can measure recruitment, fidelity and dose and establish which characteristics are associated with adoption, delivery and effectiveness. To ensure an understanding of the complexity of the relationship between the intervention and context, the case study should rely on multiple sources of data and triangulate these to confirm and corroborate the findings [ 8 ]. Process evaluations might consider using routine data collected in the trial across all sites and additional qualitative data across carefully sampled sites for a more nuanced picture within reasonable resource constraints. Mixed methods allow researchers to ask more complex questions and collect richer data than can be collected by one method alone [ 60 ]. The use of multiple sources of data allows data triangulation, which increases a study’s internal validity but also provides a more in-depth and holistic depiction of the case [ 20 ]. For example, in the DQIP process evaluation, the quantitative component used routinely collected data from all sites participating in the trial and purposively sampled cases for a more in-depth qualitative exploration [ 21 , 38 , 39 ].

The timing of data collection is crucial to study design, especially within a process evaluation where data collection can potentially influence the trial outcome. Process evaluations are generally in parallel or retrospective to the trial. The advantage of a retrospective design is that the evaluation itself is less likely to influence the trial outcome. However, the disadvantages include recall bias, lack of sensitivity to nuances and an inability to iteratively explore the relationship between intervention and outcome as it develops. To capture the dynamic relationship between intervention and context, the process evaluation needs to be parallel and longitudinal to the trial. Longitudinal methodological design is rare, but it is needed to capture the dynamic nature of implementation [ 40 ]. How the intervention is delivered is likely to change over time as it interacts with context. For example, as professionals deliver the intervention, they become more familiar with it, and it becomes more embedded into systems. The OPAL process evaluation was a longitudinal, mixed methods process evaluation where the quantitative component had been predefined and built into trial data collection systems. Data collection in both the qualitative and quantitative components mirrored the trial data collection points, which were longitudinal to capture adherence and contextual changes over time.

There is a lot of attention in the recent literature towards a systems approach to understanding interventions in context, which suggests interventions are ‘events within systems’ [ 61 , 62 ]. This framing highlights the dynamic nature of context, suggesting that interventions are an attempt to change systems dynamics. This conceptualisation would suggest that the study design should collect contextual data before and after implementation to assess the effect of the intervention on the context and vice versa.

Data analysis

Designing a rigorous analysis plan is particularly important for multiple case studies, where researchers must decide whether their approach to analysis is case or variable based. Case-based analysis is the most common, and analytic strategies must be clearly articulated for within and across case analysis. A multiple case study design can consist of multiple cases, where each case is analysed at the case level, or of multiple embedded cases, where data from all the cases are pulled together for analysis at some level. For example, OPAL analysis was at the case level, but all the cases for the intervention and control arms were pulled together at the arm level for more in-depth analysis and comparison. For Yin, analytical strategies rely on theoretical propositions, but for Stake, analysis works from the data to develop theory. In OPAL and DQIP, case summaries were written to summarise the cases and detail within-case analysis. Each of the studies structured these differently based on the phenomena of interest and the analytic technique. DQIP applied an approach more akin to Stake [ 9 ], with the cases summarised around inductive themes whereas OPAL applied a Yin [ 8 ] type approach using theoretical propositions around which the case summaries were structured. As the data for each case had been collected through longitudinal interviews, the case summaries were able to capture changes over time. It is beyond the scope of this paper to discuss different analytic techniques; however, to ensure the holistic examination of the intervention(s) in context, it is important to clearly articulate and demonstrate how data is integrated and synthesised [ 31 ].

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation [ 38 ]. Case study can enable comparisons within and across intervention and control arms and enable the evolving relationship between intervention and context to be captured holistically rather than considering processes in isolation. Utilising a longitudinal design can enable the dynamic relationship between context and intervention to be captured in real time. This information is fundamental to holistically explaining what intervention was implemented, understanding how and why the intervention worked or not and informing the transferability of the intervention into routine clinical practice.

Case study designs are not prescriptive, but process evaluations using case study should consider the purpose, trial design, the theories or assumptions underpinning the intervention, and the conceptual and theoretical frameworks informing the evaluation. We have discussed each of these considerations in turn, providing a comprehensive overview of issues for process evaluations using a case study design. There is no single or best way to conduct a process evaluation or a case study, but researchers need to make informed choices about the process evaluation design. Although this paper focuses on process evaluations, we recognise that case study design could also be useful during intervention development and feasibility trials. Elements of this paper are also applicable to other study designs involving trials.

Availability of data and materials

No data and materials were used.

Abbreviations

Data-driven Quality Improvement in Primary Care

Medical Research Council

Nonsteroidal anti-inflammatory drugs

Optimizing Pelvic Floor Muscle Exercises to Achieve Long-term benefits

Blencowe NB. Systematic review of intervention design and delivery in pragmatic and explanatory surgical randomized clinical trials. Br J Surg. 2015;102:1037–47.

Article   CAS   PubMed   Google Scholar  

Dixon-Woods M. The problem of context in quality improvement. In: Foundation TH, editor. Perspectives on context: The Health Foundation; 2014.

Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):95.

Article   PubMed   PubMed Central   Google Scholar  

Grant A, Sullivan F, Dowell J. An ethnographic exploration of influences on prescribing in general practice: why is there variation in prescribing practices? Implement Sci. 2013;8(1):72.

Lang ES, Wyer PC, Haynes RB. Knowledge translation: closing the evidence-to-practice gap. Ann Emerg Med. 2007;49(3):355–63.

Article   PubMed   Google Scholar  

Ward V, House AF, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy. 2009;14(3):156–64.

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

Yin R. Case study research and applications: design and methods. Los Angeles: Sage Publications Inc; 2018.

Google Scholar  

Stake R. The art of case study research. Thousand Oaks, California: Sage Publications Ltd; 1995.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350.

Hawe P. Minimal, negligible and negligent interventions. Soc Sci Med. 2015;138:265–8.

Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, Murphy S. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation. 2018;25(1):23–45.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, Greaves F, Harper L, Hawe P, Moore L, et al. The need for a complex systems model of evidence for public health. Lancet. 2017;390(10112):2602–4.

Moore G, Cambon L, Michie S, Arwidson P, Ninot G, Ferron C, Potvin L, Kellou N, Charlesworth J, Alla F, et al. Population health intervention research: the place of theories. Trials. 2019;20(1):285.

Kislov R. Engaging with theory: from theoretically informed to theoretically informative improvement research. BMJ Qual Saf. 2019;28(3):177–9.

Boulton R, Sandall J, Sevdalis N. The cultural politics of ‘Implementation Science’. J Med Human. 2020;41(3):379-94. h https://doi.org/10.1007/s10912-020-09607-9 .

Cheng KKF, Metcalfe A. Qualitative methods and process evaluation in clinical trials context: where to head to? Int J Qual Methods. 2018;17(1):1609406918774212.

Article   Google Scholar  

Richards DA, Bazeley P, Borglin G, Craig P, Emsley R, Frost J, Hill J, Horwood J, Hutchings HA, Jinks C, et al. Integrating quantitative and qualitative data and findings when undertaking randomised controlled trials. BMJ Open. 2019;9(11):e032081.

Thomas G. How to do your case study, 2nd edition edn. London: Sage Publications Ltd; 2016.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing. BMJ Open. 2017;7(3).

Pfadenhauer L, Rohwer A, Burns J, Booth A, Lysdahl KB, Hofmann B, Gerhardus A, Mozygemba K, Tummers M, Wahlster P, et al. Guidance for the assessment of context and implementation in health technology assessments (HTA) and systematic reviews of complex interventions: the Context and Implementation of Complex Interventions (CICI) framework: Integrate-HTA; 2016.

Bate P, Robert G, Fulop N, Ovretveit J, Dixon-Woods M. Perspectives on context. London: The Health Foundation; 2014.

Ovretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20.

Medical Research Council: Process evaluation of complex interventions: UK Medical Research Council (MRC) guidance. 2015.

May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.

Bate P. Context is everything. In: Perpesctives on Context. The Health Foundation 2014.

Horton TJ, Illingworth JH, Warburton WHP. Overcoming challenges in codifying and replicating complex health care interventions. Health Aff. 2018;37(2):191–7.

O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns. 1998;33:267–79.

Creswell J, Poth C. Qualiative inquiry and research design, fourth edition edn. Thousan Oaks, California: Sage Publications; 2018.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Takahashi ARW, Araujo L. Case study research: opening up research opportunities. RAUSP Manage J. 2020;55(1):100–11.

Tight M. Understanding case study research, small-scale research with meaning. London: Sage Publications; 2017.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalisation process theory. Sociology. 2009;43:535.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice. A consolidated framework for advancing implementation science. Implement Sci. 2009;4.

Pawson R, Tilley N. Realist evaluation. London: Sage; 1997.

Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing - a trial of education, informatics & financial incentives. N Engl J Med. 2016;374:1053–64.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci. 2017;12(1):4.

Dreischulte T, Grant A, Hapca A, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: quantitative examination of variation between practices in recruitment, implementation and effectiveness. BMJ Open. 2018;8(1):e017133.

Grant A, Dean S, Hay-Smith J, Hagen S, McClurg D, Taylor A, Kovandzic M, Bugge C. Effectiveness and cost-effectiveness randomised controlled trial of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL (Optimising Pelvic Floor Exercises to Achieve Long-term benefits) trial mixed methods longitudinal qualitative case study and process evaluation. BMJ Open. 2019;9(2):e024152.

Hagen S, McClurg D, Bugge C, Hay-Smith J, Dean SG, Elders A, Glazener C, Abdel-fattah M, Agur WI, Booth J, et al. Effectiveness and cost-effectiveness of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL randomised trial. BMJ Open. 2019;9(2):e024153.

Steckler A, Linnan L. Process evaluation for public health interventions and research; 2002.

Durlak JA. Why programme implementation is so important. J Prev Intervent Commun. 1998;17(2):5–18.

Bonell C, Oakley A, Hargreaves J, VS, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333(7563):346–9.

Article   CAS   Google Scholar  

Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.

Yin R. Case study research: design and methods. London: Sage Publications; 2003.

Bugge C, Hay-Smith J, Grant A, Taylor A, Hagen S, McClurg D, Dean S: A 24 month longitudinal qualitative study of women’s experience of electromyography biofeedback pelvic floor muscle training (PFMT) and PFMT alone for urinary incontinence: adherence, outcome and context. ICS Gothenburg 2019 2019. https://www.ics.org/2019/abstract/473 . Access 10.9.2020.

Suzanne Hagen, Andrew Elders, Susan Stratton, Nicole Sergenson, Carol Bugge, Sarah Dean, Jean Hay-Smith, Mary Kilonzo, Maria Dimitrova, Mohamed Abdel-Fattah, Wael Agur, Jo Booth, Cathryn Glazener, Karen Guerrero, Alison McDonald, John Norrie, Louise R Williams, Doreen McClurg. Effectiveness of pelvic floor muscle training with and without electromyographic biofeedback for urinary incontinence in women: multicentre randomised controlled trial BMJ 2020;371. https://doi.org/10.1136/bmj.m3719 .

Cook TD. Emergent principles for the design, implementation, and analysis of cluster-based experiments in social science. Ann Am Acad Pol Soc Sci. 2005;599(1):176–98.

Hoffmann T, Glasziou P, Boutron I, Milne R, Perera R, Moher D. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348.

Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? Br Med J. 2004;328(7455):1561–3.

Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomised trial to improve the safety of NSAID and antiplatelet prescribing: Data-driven Quality Improvement in Primary Care. Trials. 2012;13:154.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12(2):219–45.

Thorne S. The great saturation debate: what the “S word” means and doesn’t mean in qualitative research reporting. Can J Nurs Res. 2020;52(1):3–5.

Guest G, Bunce A, Johnson L. How many interviews are enough?: an experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.

Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One. 2020;15(5):e0232076.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.

Rycroft-Malone J. The PARIHS framework: a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;4:297-304.

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103.

Cresswell JW, Plano Clark VL. Designing and conducting mixed methods research. Thousand Oaks: Sage Publications Ltd; 2007.

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

Craig P, Ruggiero E, Frohlich KL, Mykhalovskiy E, White M. Taking account of context in population health intervention research: guidance for producers, users and funders of research: National Institute for Health Research; 2018. https://www.ncbi.nlm.nih.gov/books/NBK498645/pdf/Bookshelf_NBK498645.pdf .

Download references

Acknowledgements

We would like to thank Professor Shaun Treweek for the discussions about context in trials.

No funding was received for this work.

Author information

Authors and affiliations.

School of Nursing, Midwifery and Paramedic Practice, Robert Gordon University, Garthdee Road, Aberdeen, AB10 7QB, UK

Aileen Grant

Faculty of Health Sciences and Sport, University of Stirling, Pathfoot Building, Stirling, FK9 4LA, UK

Carol Bugge

Department of Surgery and Cancer, Imperial College London, Charing Cross Campus, London, W6 8RP, UK

You can also search for this author in PubMed   Google Scholar

Contributions

AG, CB and MW conceptualised the study. AG wrote the paper. CB and MW commented on the drafts. All authors have approved the final manuscript.

Corresponding author

Correspondence to Aileen Grant .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval and consent to participate is not appropriate as no participants were included.

Consent for publication

Consent for publication is not required as no participants were included.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Grant, A., Bugge, C. & Wells, M. Designing process evaluations using case study to explore the context of complex interventions evaluated in trials. Trials 21 , 982 (2020). https://doi.org/10.1186/s13063-020-04880-4

Download citation

Received : 09 April 2020

Accepted : 06 November 2020

Published : 27 November 2020

DOI : https://doi.org/10.1186/s13063-020-04880-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Process evaluation
• Case study design

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Publication: World Bank Resilience M&E: Good Practice Case Studies

Files in English

Link to data set, report series, other publications in this report series, journal volume, journal issue, collections, associated urls, associated content.

This site uses cookies to optimize functionality and give you the best possible experience. If you continue to navigate this website beyond this page, cookies will be placed on your browser. To learn more about cookies, click here .

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Monitoring and Evaluation in the Public Sector: A Case Study of the Department of Rural Development and Land Reform in South Africa

Since the publication of the Government-Wide Monitoring and Evaluation Policy Framework (GWM&EPF) by the Presidency in South Africa (SA), several policy documents giving direction, clarifying context, purpose, vision, and strategies of M&E were developed. In many instances broad guidelines stipulate how M&E should be implemented at the institutional level, and linked with managerial systems such as planning, budgeting, project management and reporting. This research was undertaken to examine how the „institutionalisation‟ of M&E supports meaningful project implementation within the public sector in South Africa (SA), with specific reference to the Department of Rural Development and Land Reform (DRD&LR). This paper provides a theoretical and analytical framework on how M&E should be “institutionalised”, by emphasising that the IM&E is essential in the public sector, to both improve service delivery and ensure good governance. It is also argued that the M&E has the potential to support meaningful implementation, promote organisational development, enhance organisational learning and support service delivery.

Related Papers

James Malesela

The global context provides an important platform for governments to build and sustain their M&E systems by adopting the best practices and lessons. Monitoring and evaluation (M&E) in South African government has gradually been recognised as a mechanism to enhance good governance. The advent of framework for the government-wide M&E inculcates a culture of reflection and importance of keeping track of the policy, programme or project implementation. M&E form an indispensable part of public management and administrative tools accessible for managers to improve the business processes of the institution. M&E therefore provides a significant panacea for the growing pressure on the institutions to enhance good governance. The principles of good governance comprise accountability, transparency, rule of law, public participation, responsiveness and effectiveness. These principles correlate precisely with the values governing public administration enshrined in the Constitution of the Republic of South Africa, 1996. They serve as standards and indicators to monitor and measure performance. The relevance of monitoring, evaluation and good governance in Public Administration is inevitable. M&E cuts across the generic administrative and managerial functions of public administration while good governance demonstrates/exhibits the outcome of functional M&E.

Niringiye Ignatius

Philipp Krause

Roan Neethling , daniel meyer

The 1994 democratic rule and Constitution of 1996 shaped the way in which service delivery would be transformed in South Africa. This was done by developing new structures and policies that would ultimately attempt to create equity and fairness in the provision of services within the municipal sphere to all residents. This article analyses the perceptions of business owners regarding the creation of an enabling environment and service delivery within one of the best performing municipalities in Gauteng: the Midvaal Local Municipal area. A total of 50 business owners were interviewed by means of a quantitative questionnaire. Data were statistically analysed by using descriptive data as well as a chi-square cross tabulation. The results revealed that the general perception of service delivery is above acceptable levels. However, in some categories the business owners were less satisfied regarding land use and zoning process and regulations. Overall, the business owners felt that the local government was creating an enabling environment for business to prosper. No significant statistical difference was found regarding perceptions of service delivery and the enabling environment, between small and large businesses in the study area. This type of analysis provides the foundation for improved service delivery and policy development and allows for future comparative analysis research into local government. The research has also placed the relationship between good governance, service delivery and the creation of an enabling environment in the spotlight.

Zwelibanzi Mpehle

Gerrit Van der Waldt

Paschal ResearchTrainers

Lebogang L Nawa

The institutionalisation of cultural policy has, to date, become an effective tool for cultureled development in some parts of the world. South Africa is yet to fully embrace this phenomenon in its developmental matrix. While the government has introduced certain strategies, such as the Integrated Development Plan (IDP), to coordinate its post-apartheid development imperatives across all of its spheres, role players, such as politicians, town planners and developers, continue to carry on with their subjective approaches to development, without culture as the mediator. This perpetuates the fragmentation of spatial landscapes and infrastructure networks in these areas along racial and cultural lines. This article suggests that South Africa may benefit from formulating local, cultural policies for the revitalisation of decaying cities into new integrated, liveable and vibrant residential, business and sporting environs.

The principal question this study aims to answer is why and how a left-of-centre government not hobbled by heavy external leverage, with developmental state precedents, potentially positive macroeconomic fundamentals, and well-developed alternative policies for housing and urban reconstruction came to settle on a conservative housing policy founded on ‘precepts of the pre-democratic period’. Arguably, this policy is even more conservative than World Bank strictures and paradigms, whose advice the incoming democratic government ‘normally ignored’ and ‘tacitly rejected’. The study, which spans the period from the early 1990s to 2007, commences from the premise that housing is an expression and component of a society’s wider development agenda and is bound up with daily routines of the ordering and institutionalisation of social existence and social reproduction. It proposes an answer that resides in the mechanics and modalities of post-apartheid state construction and its associated techniques and technologies of societal penetration and regime legitimisation. The vagaries and vicissitudes of post-Cold War statecraft, the weight of history and legacy, strategic blundering, and the absence of a cognitive map and compass to guide post-apartheid statecraft, collectively contribute to past and present defects and deformities of our two decade-old developmentalism, writ large in our human settlements. Alternatives to the technocratic market developmentalism of our current housing praxis spotlight empowering shelter outcomes but were bastardised. This is not unrelated to the toxicity of mixing conservative governmentalities (neoliberal macroeconomic precepts, modernist planning orientations, supply-side citizenship and technocratic projections of state) with ‘ambiguated’ counter-governmentalities (self-empowerment, self-responsibilisation, the aestheticisation of poverty and heroic narratives about the poor). Underscored in the study is the contention that state developmentalism and civil society developmentalism rise and fall together, pivoting on (savvy) reconnection of economics and politics (the vertical axis of governance) and state and society (the horizontal axis). Without robust reconfiguration and recalibration of axes, the revamped or, more appropriately, reconditioned housing policy – Breaking New Ground – struggles to navigate the limitations of the First Decade settlement state shelter delivery regime and the Second Decade’s (weak) developmental state etho-politics. The prospects for success are contingent on structurally rewiring inherited and contemporary contacts and circuits of power, influence and money in order to tilt resource and institutional balances in favour of the poor. Present pasts and present futures, both here and abroad, offer resources for more transformative statecraft and sustainable human settlements, but only if we are prepared to challenge the underlying economic and political interests that to date have, and continue to, preclude such policies. History, experience and contemporary record show there are alternatives – another possible and necessary world – via small and large steps, millimetres and centimetres, trial and error.

RELATED PAPERS

UCT Master's Dissertation Series no. 5

Pieter U Pretorius

Myo Naing , Anne Mc Lennan

Jacob Fatile

Masters Degree Thesis

Stephen Baguma

Zukiswa Kota , Monica Hendricks , Eric Matambo

David Schaub-Jones

IOSR Journals

Kobus Muller , daniel meyer , Malcolm Wallis , Jacobus S Wessels , Liezel Lues , Xolile Thani , Thandi Matsiliza , melody brauns , Frederik M Uys

Rick de Satgé

Sonia Bakweleng Malapane

chris landsberg

Shikha Vyas-Doorgapersad

Inge Amundsen

Fanie Cloete

Carmen Alpin

Andrea Juan

Globalizations 12 (2)

The Future of Evaluation

Laila El Baradei , Nermine Wally , Doha Abdelhamid

Anil Kanjee , Yusuf Sayed

Tam O'Neil

African Evaluation Journal

Development Bank of Southern Africa–Operations …

Cherrel Africa

Francois B van Schalkwyk , Stefaan Verhulst , Glenn Maail , Piovesan Federico , Silvana Fumega

BRIDGE, IDS, Brighton

Bridget Byrne

Liz David-Barrett

Dr C L Pieterse

Ross R Worthington

Thabang Mpande

Victoria Awiti

Tristan Gorgens

University of Sussex. Doctoral thesis

Blanca Lopez

Robert Cameron

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

•   UoN Digital Repository Home
• Journal Articles
• Faculty of Engineering, Built Environment & Design (FEng / FBD)

Monitoring and evaluation: an urban project case study in Kenya

Collections.

• Faculty of Engineering, Built Environment & Design (FEng / FBD) [1463]

Let your search flow

Explore perspectives, what is a perspective.

Perspectives are different frameworks from which to explore the knowledge around sustainable sanitation and water management. Perspectives are like filters: they compile and structure the information that relate to a given focus theme, region or context. This allows you to quickly navigate to the content of your particular interest while promoting the holistic understanding of sustainable sanitation and water management.

Monitoring and evaluation - TARA (case study)

Executive Summary

This case study supports and illustrates the theoretic factsheet "Monitoring and evaluation (safe water business)" with practical insights.

TARA going from informal, to paper to a mobile app - M&E evolution in India

Informal infrequent M&E

TARAlife produces and sells liquid chlorine to purify drinking water, produced with Antenna Foundation ’s WATA™ technology converting salt and water with a simple electrolysis process into sodium hypochlorite (chlorine). When TARA started producing and selling chlorine branded Aqua+ (see picture) via its social enterprise TARAlife Pvt. Ltd. in 2012, TARA did not have a systematic M&E system in place to monitor sales and business activities. The head of TARAlife simply contacted each local partner by phone on an irregular basis to collect sales figures.

Paper-based

Recognising the importance of collecting customer and sales data in a more systematic way, TARA designed its first M&E system in 2013. This was a sales record booklet which included sections for customer data, sales data, and marketing materials. This system was not functioning properly, because each franchisee filled out the booklet slightly differently and the data were also not reported back to TARA headquarters consistently. This made the data from different regions and last mile agents difficult to compare. At the same time, TARA’s channel partners were having difficulties in managing their Aqua+ stocks, which was causing delayed orders and expired stocks.

To address both the issues of sales management as well as stock management, TARA started developing a mobile application, with support from a consultant. The app aims to make the sales reporting more user-friendly, more consistent and quicker. When developing the app, TARA realized that it could also be used to collect customer and impact-related data to assess the social, health and financial impacts of TARA's interventions.  Together with IRC, the framework for the app (see figure below) was developed based on the following four objectives:

• Retain & increase database of Aqua+ customers
• Track and record impact of intervention on health/overall quality of life
• Decrease or minimise sales lost and inventory costs
• Extend the application of the system to other products than Aqua+ over the long run

The key functions of the application are the following:

• Data collected through the app by micro franchisees: details about customers, micro franchisees, customer purchasing behaviour, baseline survey of potential new customers (i.e. current water disinfection practices, existing health status, medical expenses, etc.), and product feedback from customers.
• Analysis of the data captured with the application: real-time tracking of sales and micro franchisee performance in terms of meeting sales targets.
• Implementation of the data used: send reminders to customers about purchasing TARA products and send periodic messages about safe water awareness.
• Conduct an impact assessment survey (post intervention survey after 6 months of purchase).

The development of the mobile application is completed and it is about to enter the pilot test phase with micro franchisees at TARA Akshar locations in the state of Eastern Uttar Pradesh, India. The results from the pilot test are expected by 2018.

Lessons learnt from digitalising M&E

• Paper-based M&E has the advantage that surveys do not need to have access to a source of electricity, which can be advantageous in non-electrified rural areas.
• Paper-based M&E is more time-consuming as to get a clear overview and statistics data has to be fed into computers. During such process mistakes can occur and falsify data.
• The launch of the app-based M&E brings different advantages along: Data is now homogenously compiled and can soundly be tracked back to microfranchisees. It easily allows to make comparisons between regions, products and salespeople on a daily basis.
• App-based M&E allows to be adapted to a variety of products and can be duplicated when necessary.
• App-based M&E improve attractiveness of a safe water initiative or safe water enterprise for investments as impact is soundly collected and can be easily presented and accessed externally also.

Recommendations for implementing an app-based M&E system

• Developing and integrating app-based M&E is time-consuming and has its costs that have to be taken into account when reflecting on starting such project in your safe water initiative.
• In a long-term perspective is the use of app-based M&E inevitable as tendencies are in place of donors and impact investors to have sound track access to data and this if possible on a daily basis.

Safe Water and Jobs - Creating Access to Safe Water in India through Women-Led Service Delivery Models

Taralife sustainability solutions pvt. ltd, alternative versions to, perspective structure.

• Case Studies

You Might Be Interested In

• SDG Background
• Background on "bottom of the pyramid"
• Scaling Safe Water - The Need for an Industry Facilitator
• Operation and Maintenance

You want to stay up to date about water entrepreneurship?

Subscribe  here to the new Sanitation and Water Entrepreneurship Pact (SWEP) newsletter!

Contenidos de la ficha

Get regular updates on the latest innovations in SSWM, new perspectives and more!

Do you like our new look?

We'd love to know what you think of the new website – please send us your feedback.

Comparte con otros

Subscribe to our newsletter.

Loading metrics

Open Access

Peer-reviewed

Research Article

Public health policy impact evaluation: A potential use case for longitudinal monitoring of viruses in wastewater at small geographic scales

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation Department of Civil and Environmental Engineering, Stanford University, Stanford, California, United States of America

Roles Data curation, Validation, Writing – review & editing

Roles Investigation, Methodology, Writing – review & editing

Affiliations Department of Civil and Environmental Engineering, Stanford University, Stanford, California, United States of America, Codiga Resource Recovery Center, Stanford University, Stanford, California, United States of America

Roles Investigation, Methodology, Supervision, Writing – review & editing

Roles Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

• Elana M. G. Chan, 
• Amanda Bidwell, 
• Zongxi Li, 
• Sebastien Tilmans, 
• Alexandria B. Boehm

• Published: June 3, 2024
• https://doi.org/10.1371/journal.pwat.0000242
• Reader Comments

Public health policy impact evaluation is challenging to study because randomized controlled experiments are infeasible to conduct, and policy changes often coincide with non-policy events. Quasi-experiments do not use randomization and can provide useful knowledge for causal inference. Here we demonstrate how longitudinal wastewater monitoring of viruses at a small geographic scale may be used in a quasi-experimental design to evaluate the impact of COVID-19 public health policies on the spread of COVID-19 among a university population. We first evaluated the correlation between incident, reported COVID-19 cases and wastewater SARS-CoV-2 RNA concentrations and observed changes to the correlation over time, likely due to changes in testing requirements and testing options. Using a difference-in-differences approach, we then evaluated the association between university COVID-19 public health policy changes and levels of SARS-CoV-2 RNA concentrations in wastewater. We did not observe changes in SARS-CoV-2 RNA concentrations associated with most policy changes. Policy changes associated with a significant change in campus wastewater SARS-CoV-2 RNA concentrations included changes to face covering recommendations, indoor gathering bans, and routine surveillance testing requirements and availability.

Citation: Chan EMG, Bidwell A, Li Z, Tilmans S, Boehm AB (2024) Public health policy impact evaluation: A potential use case for longitudinal monitoring of viruses in wastewater at small geographic scales. PLOS Water 3(6): e0000242. https://doi.org/10.1371/journal.pwat.0000242

Editor: Ricardo Santos, Universidade Lisboa, Instituto superior Técnico, PORTUGAL

Received: February 2, 2024; Accepted: May 5, 2024; Published: June 3, 2024

Copyright: © 2024 Chan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Wastewater data are publicly available through the Stanford Digital Repository ( https://doi.org/10.25740/ch598gf0783 ).

Funding: This work was supported by the Provost’s Office of Stanford University to ABB with additional support from the Sergey Brin Family Foundation to ABB. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Nonpharmaceutical interventions (NPIs) aim to reduce the spread of an infectious disease in a community, especially when the community has little immunity to the pathogen or a vaccine is not yet available [ 1 ]. Examples of NPIs implemented in the United States at the start of the coronavirus disease 2019 (COVID-19) pandemic include face mask mandates, stay-at-home orders, non-essential business closures, and large gathering bans [ 2 ]. Although NPIs intended to benefit communities by flattening the epidemic curve—that is by reducing the peak number of cases and burden on the health care system—the implementation of NPIs also led to economic consequences and tolls on social well-being [ 3 – 5 ]. Governments and institutional leadership are tasked with balancing public health, social well-being, and economic prospects in the face of epidemics. Causal evidence can help policymakers and leaders make better-informed decisions in dire situations.

Following the initial wave of the pandemic, several studies empirically assessed the impact of NPIs on health-related outcomes. These studies suggested that NPIs reduced the spread of severe acute respiratory disease syndrome coronavirus 2 (SARS-CoV-2) virus, with school and workplace closures, business restrictions, large gathering bans, and mask mandates among the most impactful [ 6 – 10 ]. A review of the methodologies used by these studies found that around half analyzed raw outcome data and half analyzed computed outcome data (i.e., raw outcome data was used to compute another outcome) [ 11 ]. The most common raw outcomes analyzed were clinical surveillance reports (e.g., confirmed cases or deaths) and human mobility (e.g., tracking of mobile phones) [ 11 ]. The most common computed outcomes analyzed were COVID-19 growth rate and effective reproduction number [ 11 ].

Although clinical surveillance and mobile phone tracking are the most common sources of data used to evaluate NPIs, these data are not without biases and limitations. Counts of confirmed cases depend on clinical testing capacity and clinical testing rates, and deaths that occur outside of hospitals may be underreported [ 6 – 8 , 10 , 11 ]. Furthermore, clinical testing behaviors have drastically changed with the availability of self-administered antigen tests which are not reported to health departments [ 12 ]. Mobility data through tracking of mobile phones are unaffected by changes in clinical testing, but these data are not always publicly accessible, biased towards individuals who opt into location history sharing, and may not be a reliable proxy for SARS-CoV-2 transmission dynamics [ 13 , 14 ]. Wastewater monitoring, which gained heightened attention during the COVID-19 pandemic, is a promising data source because it does not suffer some of the limitations of clinical surveillance and mobility data for epidemiological inference.

Wastewater monitoring refers to the analysis of a sample of wastewater, which represents a pooled biological sample of the contributing population, for concentrations of infectious disease markers. Wastewater monitoring data capture contributions from both symptomatic and asymptomatic individuals and are not influenced by clinical testing availability or clinical test-seeking behaviors [ 15 ]. Studies reported that concentrations of SARS-CoV-2 RNA in wastewater solids are temporally correlated with laboratory-confirmed incident COVID-19 cases [ 16 – 19 ]. Several studies also demonstrated that wastewater monitoring can be used at geographic scales smaller than a sewershed (i.e., the population serviced by a wastewater treatment plant) to gain insight about COVID-19 incidence [ 20 – 32 ]. A potential use case for wastewater monitoring at subsewershed scales is to assess the impact of public health policies.

The World Health Organization (WHO) suggests sampling at finer spatial scales when using wastewater monitoring data to inform targeted control interventions [ 15 ]. Previous studies evaluating NPIs using clinical surveillance or mobility data were mostly conducted at national or subnational scales, and few of these studies investigated variation in the impact of NPIs on health-related outcomes among subpopulations [ 11 ]. NPIs may be more or less impactful in a subpopulation compared with the general population (e.g., due to different interaction patterns) or public health goals may differ among subpopulations (e.g., universities aim to maximize on-campus activity) [ 33 ]. Wastewater monitoring data may be well-suited to objectively assess NPIs, particularly among subpopulations and when clinical testing rates are low.

In this study, we evaluate the potential use case of wastewater monitoring data to empirically assess the impact of NPIs on the spread of COVID-19 among a university population. We begin by assessing the correlation between wastewater concentrations of SARS-CoV-2 RNA and reported COVID-19 incidence at Stanford University and evaluate changes to this correlation over time. Next, we evaluate the association between COVID-19 public health policies implemented at Stanford University and changes in wastewater concentrations of SARS-CoV-2 RNA using a difference-in-differences (DiD) approach. DiD is a quasi-experimental design commonly used in econometrics—although it was first used in 1854 by the English physician John Snow for epidemiologic purposes—that assesses the impact of an intervention on an outcome without the use of randomization [ 34 – 36 ]. DiD designs have been used by previous studies to empirically evaluate the causal effects of COVID-19 policies on clinical or mobility outcomes [ 37 – 44 ].

We used wastewater SARS-CoV-2 RNA monitoring data, COVID-19 case surveillance data, and dates associated with changes to campus COVID-19 public health policies between 29 July 2021 to 9 August 2023. During this timeframe, the residential communities for undergraduate and graduate students at Stanford University were open for all students to physically reside on campus. All calculations and statistical analyses were conducted in R (R Foundation for Statistical Computing version 4.1.3). This study was approved by the Stanford Institutional Review Board (IRB) for human subject research (IRB-59746). We did not obtain consent from individuals to preserve anonymity, and we did not have access to personally identifiable information during or after data collection.

2.1 Wastewater monitoring data

We used wastewater monitoring data from the Codiga Resource Recovery Center (CR2C) and the Palo Alto Regional Water Quality Control Plant (RWQCP) for this analysis. CR2C is a pilot scale wastewater treatment facility that services a portion of the Stanford University campus (California, USA). Buildings serviced include academic buildings and student and faculty housing; hospitals and clinics affiliated with the medical school are not serviced by CR2C (Fig A in S1 Text ) [ 45 , 46 ]. CR2C services approximately 10,000 people with an estimated daily flow of approximately 0.5 million gallons of wastewater each day [ 20 , 46 ]. CR2C is a subsewershed of the sewershed serviced by RWQCP which is operated by the City of Palo Alto (California, USA). RWQCP services approximately 236,000 people and treats approximately 20 million gallons of wastewater each day for Los Altos, Los Altos Hills, Mountain View, Palo Alto, Stanford University, and the East Palo Alto Sanitary District (Fig A in S1 Text ) [ 47 ].

Prospective, longitudinal wastewater sampling from CR2C and RWQCP began July 2021 and October 2020, respectively, and is currently ongoing. Briefly, wastewater settled solids are collected from both CR2C and RWQCP for laboratory processing. Settled solids samples at CR2C are generated from a 24-hour time proportional composite sample of the wastewater influent that is allowed to settle. Settled solids samples at RWQCP are “grab” samples from the primary clarifier; these samples are essentially composite samples because solids in the primary clarifier collect over 12–24 hours [ 48 ]. Six samples per week are collected from CR2C; seven samples per week are collected from RWQCP. Sampling from CR2C was temporarily reduced to two samples per week between 1 November 2022 and 31 December 2022. Details about sampling and processing methods used to measure the RNA targets, including quality assurance and quality control metrics, are registered in protocols.io [ 49 – 51 ] and have been described previously by Wolfe et al. [ 52 ] and Boehm et al. [ 53 ], so they are not repeated herein. Measurements and reporting in those other publications follow Environmental Microbiology Minimal Information (EMMI) guidelines. For this analysis, we used concentrations of the SARS-CoV-2 RNA N gene in wastewater settled solids in gene copies (gc) per gram (g) dry weight (gc/g), both unnormalized (N) and normalized by pepper mild mottle virus (PMMoV) RNA concentrations in wastewater settled solids in gc/g (N/PMMoV). The N gene target is located near the frequently used N2 assay target [ 54 ], and we have confirmed no mutation in the genomic target over the course of the pandemic [ 19 ]. PMMoV is a commonly used marker of wastewater fecal strength, and based on a mass balance model N/PMMoV should scale with disease incidence rate [ 18 , 55 , 56 ]. We used data between 29 July 2021 and 9 August 2023 (CR2C: 590 days; RWQCP: 736 days). The measured N gene concentration was below the limit of detection (approximately 1,000 gc/g) in 29 samples from CR2C. No samples from RWQCP were below the limit of detection. We imputed half the limit of detection (500 gc/g) for the N gene concentration for samples below the limit of detection. There were no non-detects for PMMoV in the dataset. Data from RWQCP between 16 November 2020 and 31 December 2022 have been published previously by Boehm et al. [ 53 ] and are publicly available through the Stanford Digital Repository ( https://doi.org/10.25740/cx529np1130 ) [ 57 ]. Data from CR2C are novel and not published elsewhere. All wastewater monitoring data used in this study are publicly available through the Stanford Digital Repository ( https://doi.org/10.25740/ch598gf0783 ) [ 58 ].

2.2 COVID-19 case surveillance data

Reported COVID-19 cases (hereafter “case data”) among students residing in the CR2C subsewershed are available from Stanford University. The date assigned to the positive test result is the date of specimen collection. We used case data between 29 July 2021 and 9 August 2023 for this analysis. The campus case data include positive test results from both student-reported self-administered antigen tests and laboratory-based PCR tests through the university’s surveillance testing program. The university’s surveillance testing program required vaccinated students to test once per week (twice per week for unvaccinated students) through 7 April 2022. Free, optional laboratory-based PCR testing continued to be available for students through 18 June 2023, so any cases thereafter were exclusively from student-reported self-administered tests. The CR2C subsewershed includes faculty and staff housing, but nonstudents residing in the CR2C subsewershed are not included in the university’s case data. Data provided by the state of California did not identify any COVID-19 cases in nonstudent housing areas during our entire analysis period.

2.3 Campus COVID-19 public health policies

Dates and details of changes to Stanford University’s COVID-19 public health policies were obtained from Stanford COVID-19 Health Alerts [ 59 ]. There were 15 unique dates on which campus COVID-19 public health policies changed during the study period ( Table 1 ). We categorized policies into three groups: masking (i.e., those involving the use of face coverings), mobility (i.e., those involving movement or gathering of individuals), and testing (i.e., those relating to laboratory-based surveillance testing). We included testing policies because we hypothesize that surveillance testing requirements and availability affect the number of asymptomatic cases interacting with the general university population and, in turn, SARS-CoV-2 transmission on campus. We further differentiated policies between those that enforced rules (i.e., restrictions) and those that relaxed existing rules (i.e., relaxations). More information about each policy is included in Table A in the S1 Text .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pwat.0000242.t001

2.4 Correlation analysis

Incident COVID-19 cases within the CR2C subsewershed were reported daily, whereas between 2–6 wastewater samples per week were collected and analyzed from CR2C during the analysis period. Clinical case surveillance data may further contain reporting biases on weekends. To compare the two time series, we calculated weekly average N concentrations, N/PMMoV concentrations, and incident COVID-19 cases for each epidemiological week (Sunday through Saturday). Neither raw nor log 10 -transformed weekly average N or N/PMMoV concentrations from CR2C were normally distributed (Shapiro-Wilk normality test, p < 0.01), so we used Kendall’s tau correlation to test the null hypothesis that weekly average wastewater SARS-CoV-2 RNA concentrations and weekly average incident COVID-19 cases in the CR2C subsewershed are not temporally correlated. We tested this null hypothesis using both unnormalized (N) and normalized (N/PMMoV) wastewater concentrations. We used the KendallTauB function from the DescTools R package to compute the 95% confidence interval for each tau estimate [ 60 ].

We further conducted three subgroup correlation analyses. First, we grouped the data by whether wastewater sample or clinical specimen collection occurred during the academic year (autumn, winter, or spring quarter) or nonacademic year (summer quarter). We used the date halfway between the last day of classes of the previous quarter and first day of classes of the following quarter to define the start and end of quarters [ 61 ]. Second, we grouped the data by whether wastewater sample or clinical specimen collection occurred before or after the requirement for laboratory-based surveillance testing was suspended for vaccinated and boosted students (7 April 2022) ( Table 1 ). The laboratory-based surveillance testing program required fully vaccinated students to test once a week (twice a week for unvaccinated students) and therefore intended to capture both symptomatic and asymptomatic cases through routine testing. Third, we grouped the data by whether wastewater sample or clinical specimen collection occurred before or after 1 May 2022 [ 19 ]. This date represents a point in time when self-administered COVID-19 antigen tests, the results of which are not reportable to health departments, were widely available [ 12 , 19 ]. For each subgroup, we grouped weekly average wastewater concentrations and incident case counts based on the end date of the epidemiological week. In total, we conducted 14 correlation analyses using subsets of the same datasets to test the same null hypothesis, so we used an alpha value of 0.05 / 14 = 0.004 to account for multiple hypothesis testing when interpreting the p-value associated with each tau estimate.

2.5 Policy impact evaluation

We used PMMoV-normalized wastewater concentrations for the remainder of the analysis as the correlation between incident COVID-19 cases and wastewater SARS-CoV-2 RNA concentrations were similar using N and N/PMMoV, and a mass balance model suggests the N/PMMoV ratio should scale with incidence rate [ 56 ]. PMMoV is also a conceptually valid normalization approach because (1) PMMoV is an indigenous wastewater virus and therefore may better correct for differences in virus recovery than an exogenous recovery control that is seeded into the sample such as bovine coronavirus (BCoV) and (2) PMMoV is of dietary origin and therefore can control for differences in the fecal strength of the wastestream [ 55 , 56 ]. To assess the association between campus COVID-19 public health policies and changes in N/PMMoV measurements at CR2C, we used a difference-in-differences (DiD) approach. For the DiD design, we assumed policies went into effect at midnight on the date of implementation (day = 0). We defined the pre-treatment period as the 14 days before a policy was implemented (days -14 to -1) and the post-treatment period as the 14 days after a policy was implemented (days 0 to 13). We chose 14 days because 14 days is the maximum incubation period for SARS-CoV-2 and people who shed SARS-CoV-2 RNA typically do so at the start of infection [ 62 – 66 ]. We assumed the RWQCP sewershed represents a reasonable comparison group for the CR2C subsewershed. With the exception of the East Palo Alto Sanitary District, RWQCP services cities in Santa Clara County, which is the same county that Stanford University is located in. Santa Clara County entered the least restrictive “Yellow Tier” of California’s Blueprint for a Safer Economy on 19 May 2021, which lifted most local orders [ 67 ]. Moreover, California met the criteria under the Blueprint for a Safer Economy to fully reopen the economy on 15 June 2021 [ 68 ]. Regular sampling began at CR2C on 29 July 2021; therefore, we assumed policies implemented by Stanford University thereafter ( Table 1 ) were only applicable to the CR2C subsewershed population and not the greater RWQCP sewershed population. The two exceptions were 3 August 2021 and 2 March 2022 because Santa Clara County also issued the same policies ( Table 1 ) [ 69 , 70 ]. Non-policy events, such as emergence of novel SARS-CoV-2 variants, may also affect SARS-CoV-2 transmission; however, CR2C and RWQCP are in the same geographic area, so we assumed most non-policy events occurred around the same time and are therefore accounted for in the DiD design. Further justification for using RWQCP as a comparison group is included in the S1 Text .

We used a multivariable linear regression model to implement our DiD approach ( Eq 1 ) [ 71 ]. A value of 0 for time represents the pre-treatment period (days -14 to -1), and a value of 1 represents the post-treatment period (days 0 to 13). A value of 0 for treated represents the untreated group (RWQCP), and a value of 1 represents the treated group (CR2C). The coefficient of the interaction between time and treated ( β 3 ) represents the DiD estimator, or the average treatment effect on the treated (ATT) [ 35 , 71 ]. In this study, a positive ATT value suggests a policy was associated with an increase in wastewater N/PMMoV concentrations; a negative ATT value suggests a policy was associated with a decrease in wastewater N/PMMoV concentrations. We recorded β 3 (the ATT) and the p-value associated with β 3 for each policy in Table 1 except for the two policies that Santa Clara County also issued (see above). R code for the DiD analysis is available through the Stanford Digital Repository ( https://doi.org/10.25740/ch598gf0783 ) [ 58 ].

3. Results and discussion

3.1 correlation between wastewater concentrations of sars-cov-2 rna and incident covid-19 cases.

Between 29 July 2021 and 9 August 2023, wastewater N gene concentrations from CR2C ranged from not detected to 2.4 x 10 6 gc/g (mean: 1.3 x 10 5 gc/g, median: 4.4 x 10 4 gc/g) ( Fig 1A ). PMMoV-normalized wastewater concentrations ranged from not detected to 5.0 x 10 −3 (mean: 2.4 10 −4 , median: 6.4 x 10 −5 ) ( Fig 1B ). Reported daily incident COVID-19 cases within the CR2C subsewershed ranged from 0 cases to 420 cases (mean: 52 cases, median: 15 cases) ( Fig 1C ). Over the entire analysis period (the week ending on 31 July 2021 through the week ending on 12 August 2023), weekly average wastewater SARS-CoV-2 RNA concentrations were positively and significantly correlated with weekly average incident COVID-19 cases using unnormalized N gene concentrations but not significantly when using normalized N gene concentrations ( Table 2 ). The subgroup analyses suggest the correlation between wastewater SARS-CoV-2 RNA concentrations and incident COVID-19 cases changed over time.

(A) N gene concentrations in gene copies per dry gram dry weight (gc/g), (B) N/PMMoV concentrations, and (C) incident COVID-19 cases over time. Gray circles represent measurements; error bars are one standard deviation. Gray triangles indicate measurements outside of the range shown on the plot. Black lines connect weekly average values. The shaded area corresponds to the nonacademic year. The dashed lines correspond to the date the surveillance testing requirement was suspended (7 April 2022) and the date of widespread availability of self-administered COVID-19 antigen tests in the region (1 May 2022).

https://doi.org/10.1371/journal.pwat.0000242.g001

https://doi.org/10.1371/journal.pwat.0000242.t002

Weekly average wastewater SARS-CoV-2 RNA concentrations were positively and significantly correlated with weekly average incident COVID-19 cases during the academic year using both unnormalized and normalized N gene concentrations; this correlation was not statistically significant during the nonacademic portion of the year ( Table 2 ). The decrease in students on campus and increase in nonresidential visitors during the nonacademic portion of the year may explain the lack of a statistically significant correlation during the nonacademic year. The COVID-19 case data only include reported student cases residing within the CR2C subsewershed, but infected, nonresidential visitors may still contribute viral RNA to the wastewater that flows to CR2C.

Weekly average wastewater SARS-CoV-2 RNA concentrations were positively and significantly correlated with weekly average incident COVID-19 cases before the suspension of surveillance testing using both unnormalized and normalized N gene concentrations; this correlation was not statistically significant after the suspension of surveillance testing using normalized N gene concentrations only ( Table 2 ). The required, laboratory-based surveillance testing program intended to capture both symptomatic and asymptomatic cases through routine testing. Thus, fewer asymptomatic cases may have been captured in the case data after surveillance testing was suspended which may explain the lack of a statistically significant correlation after this policy change.

Lastly, weekly average wastewater SARS-CoV-2 RNA concentrations were positively and significantly correlated with weekly average incident COVID-19 cases before the widespread availability of self-administered antigen tests using both unnormalized and normalized N gene concentrations; this correlation was not statistically significant after the widespread availability of self-administered antigen tests ( Table 2 ). Positive, laboratory-based PCR tests are reportable under state-disease reporting laws [ 72 ]; however, self-reporting of self-administered antigen test results is voluntary. The widespread availability of self-administered antigen tests may have contributed to underreporting of cases which may explain the lack of a statistically significant correlation after the change in testing options.

It is not possible to deduce the main driver for the change in correlation between wastewater SARS-CoV-2 RNA concentrations and incident COVID-19 cases over time, but we suspect the change is due to several factors including changes in routine COVID-19 surveillance testing requirements, changes in test reporting, and overall decreases in PCR test-seeking behaviors as the pandemic continues [ 19 , 73 – 75 ]. Studies suggest that virus shedding patterns differ among SARS-CoV-2 variants [ 76 – 79 ], so changes in SARS-CoV-2 variants over time could be another reason for the change in correlation over time. We also did not consider lead-lag time effects between wastewater monitoring and case surveillance data as done in other studies [ 16 , 80 ], so future work could investigate how lead-lag time effects between wastewater monitoring and case surveillance data have changed over the course of the pandemic. Nonetheless, wastewater monitoring data are independent of test-seeking behaviors or test reporting patterns so may be a less biased tool for monitoring public health, particularly in periods characterized by low test-seeking and reporting rates.

3.2 Association between campus COVID-19 public health policies and changes in wastewater concentrations of SARS-CoV-2 RNA

Because the reliability of campus COVID-19 case data changed over the course of the study period at Stanford University, we used campus wastewater monitoring data from CR2C to evaluate the impact of COVID-19 public health policies at Stanford University using a DiD approach. Table 3 summarizes the average treatment effect on the treated (ATT) and associated p-value for each unique date associated with a change in campus COVID-19 public health policies as estimated using Eq 1 . The two policies that were also implemented by the greater Santa Clara County were omitted from the analysis. Dates associated with a significant change (p ≤ 0.05) in wastewater N/PMMoV concentrations are shaded (red if ATT > 0 and blue if ATT < 0). A depiction of the DiD approach using the date when indoor events and gatherings were allowed to resume (28 January 2022) as an example is shown in Fig 2 . In total, we analyzed 13 unique dates on which at least one change in campus COVID-19 public health policies went into effect. Most policy change dates (n = 8) were not associated with a significant change in wastewater N/PMMoV concentrations at CR2C. Five policy change dates were associated with a significant change in wastewater N/PMMoV concentrations ( Table 3 and Fig B in S1 Text ). These five dates included policies from all categories (masking, mobility, testing); three dates corresponded to policy relaxations, one corresponded to a policy restriction, and one corresponded to both a policy relaxation and restriction.

(A) Daily N/PMMoV concentrations at the Codiga Resource Recovery Center (CR2C) and Palo Alto Regional Water Quality Control Plant (RWQCP) over the 14 days before and after the policy change (denoted by the dotted line). Concentrations are displayed on a log 10 scale. (B) Average log 10 (N/PMMoV) concentration at CR2C and RWQCP across the 14 days before and after the policy change. The counterfactual average log 10 (N/PMMoV) concentration at CR2C post-policy was estimated based on the time trend observed at RWQCP. The difference between the observed and counterfactual average log 10 (N/PMMoV) concentration at CR2C post-policy represents the average treatment effect on the treated (ATT). Here, a positive ATT value suggests that the policy change was associated with an increase in wastewater N/PMMoV concentrations at CR2C.

https://doi.org/10.1371/journal.pwat.0000242.g002

https://doi.org/10.1371/journal.pwat.0000242.t003

We did not expect policy relaxations to be associated with a significant change in wastewater N/PMMoV concentrations because these policy types are not intended to curb virus transmission. Eight dates exclusively corresponded to a policy relaxation, and five of them were not associated with a change in N/PMMoV concentrations. However, three of these dates were associated with a significant change in N/PMMoV concentrations. There was a significant increase in N/PMMoV concentrations associated with allowing indoor gatherings to resume (28 January 2022), which suggests that indoor gatherings are high-risk activities for SARS-CoV-2 transmission on campus. Indoor gatherings are known to promote SARS-CoV-2 transmission [ 81 ]. There was a significant decrease in N/PMMoV concentrations associated with suspending the surveillance testing requirement for students (7 April 2022), and then a significant increase in N/PMMoV concentrations associated with ending optional, free, laboratory-based PCR testing for employees (24 March 2023). These results are difficult to reconcile with expectations.

We expected policy restrictions to be associated with a significant decrease in wastewater N/PMMoV concentrations because these policy types are intended to curb virus transmission. Four dates exclusively corresponded to a policy restriction, but only one of these dates was associated with a significant change in N/PMMoV concentrations (recommending face coverings outdoors and prohibiting indoor parties on 2 September 2021). This date was associated with a significant increase rather than decrease in N/PMMoV concentrations, which could suggest these restrictive policies did not curb virus transmission on campus. Both a policy relaxation (revised travel guidelines) and policy restriction (surveillance testing required for all faculty, staff, and postdoctoral scholars) were implemented on the remaining date associated with a significant change in N/PMMoV concentrations (20 September 2021). This date was associated with a significant decrease in N/PMMoV concentrations; it is not possible to disentangle the individual causal effects of different policy types implemented on the same day.

Limitations of the DiD analysis may impact the interpretation of results and explain why some results did not align with expectations. First, policies may not be associated with immediate effects on outcomes [ 82 ]. The policy restrictions we considered may be associated with long-term effects on N/PMMoV concentrations despite being associated with null short-term effects. We determined that 14 days preceding and succeeding a policy was the most justified time interval for the DiD design given the maximum incubation period for SARS-CoV-2 is 14 days and people who shed SARS-CoV-2 RNA generally do so at the start of infection [ 62 – 66 ]. When using 14 days, the pre- or post-treatment period of one policy sometimes overlapped part of the pre- or post-treatment period of another policy for policies implemented close together which may lead to cumulative impacts on N/PMMoV concentrations that are not possible to disentangle. Second, there were sometimes campus announcements or national news headlines about COVID-19 preceding the implementation of policies, which could impact peoples’ behaviors leading up to the actual policy change date [ 82 ]. Peoples’ knowledge about the gravity of the COVID-19 pandemic has been shown to influence the effectiveness of lockdown policies [ 39 ]. We used official dates associated with changes to campus policies, but peoples’ behaviors may have started changing before these dates. Alternatively, peoples’ behaviors may have never changed if campus policies were ignored. Third, while the DiD design accounts for non-policy events that affect both CR2C and RWQCP, some non-policy events that affect the CR2C population and not the RWQCP population may have occurred. Co-occurrence of such events with policy changes is unaccounted for in the DiD design. For example, starts of quarters and university commencements may influence N/PMMoV concentrations at CR2C because these events result in large influxes of students and visitors to Stanford’s campus. We conducted the DiD analysis for dates associated with commencements and the first day of classes each quarter (Table B in the S1 Text ), and two starts of quarters were associated with a significant change in N/PMMoV concentrations at CR2C (decrease at the start of autumn 2021 and increase at the start of spring 2022). Lastly, the proportion of people with immunity, either from prior infection or vaccination, changes over time. Potential impacts of public health policies on virus transmission may depend on the susceptible fraction of the population; however, the DiD design does not account for changing levels of susceptibility in either population. Importantly though, COVID-19 vaccination rates were similar and high among the Stanford University and greater Santa Clara County populations at the start of the analysis period [ 69 , 83 ].

Notwithstanding these limitations, we compared our results to those of other studies that also assessed COVID-19 public health policies among a vaccinated university population. Yang et al. similarly found that large gatherings are potentially high-risk events on campus [ 84 ]. Niu and Scarciotti concluded that mask wearing and social distancing measures were most effective at reducing new infections [ 33 ]. Motta et al. [ 85 ] and Paltiel and Schwartz [ 86 ] determined that routine surveillance testing was associated with a reduction in infections, even as vaccine effectiveness or coverage decreased. These other studies all used modeling approaches to assess COVID-19 public health policies; models are useful tools to evaluate public health measures although they often simplify real-world circumstances.

To our knowledge, there are no other published studies that empirically evaluate public health policies using wastewater monitoring data and a quasi-experimental approach, particularly among a vaccinated university population. The few other empirical studies using wastewater monitoring data for policy impact evaluation, which were conducted at large geographic scales and the beginning of the pandemic, used before-and-after descriptive approaches [ 87 , 88 ] or regression modeling and changepoint analysis [ 89 ]. The DiD design used herein aimed to account for co-occurring factors that may also affect the trajectory of N/PMMoV concentrations, such as changing SARS-CoV-2 variants, by using a nearby sewershed as a comparison group. We further considered both policy restrictions and policy relaxations during a period when COVID-19 vaccines were widely available. Previous studies that empirically assessed the impact of NPIs on health-related outcomes generally only focused on restrictions and were most commonly conducted at the start of the pandemic when economies were not fully opened and vaccines were not available. It is not only important to evaluate the implementation of policies but also whether policies are eventually relaxed appropriately, especially because early or rapid relaxation of NPIs may lessen the anticipated benefits of vaccine rollout efforts [ 90 – 94 ]. The quasi-experimental approach demonstrated herein could be useful in other epidemic situations triggering policy interventions, provided the pathogen is shed in human excretions that contribute to wastewater and there exists a reasonable comparison sewershed for the DiD design (e.g., a sewershed in a different state that did not roll out a given intervention).

Causal effects of COVID-19 public health policies are inherently challenging to study given the inability to conduct randomized controlled experiments and concurrence of policy and non-policy events [ 95 ]. Policymakers often need to make decisions despite having robust evidence. Quasi-experiments, which are growing in recognition in the health sciences, are a practical alternative to randomized controlled experiments that can still generate causal evidence [ 34 ]. In the DiD quasi-experimental design used herein, RWQCP represents a reasonable comparison group for CR2C because both sewersheds are in the same geographic area and policies implemented by Stanford University were only applicable to the CR2C population. We also implemented the DiD analysis using wastewater data from another, similar comparison sewershed because wastewater monitoring data from CR2C and RWQCP are not truly independent—although CR2C comprises only a very small proportion of RWQCP. Using wastewater data from the San José-Santa Clara Regional Wastewater Facility [ 96 ], which also services portions of Santa Clara County, as a comparison sewershed generated similar results as provided in Table 3 (Table B in the S1 Text ). Similar findings using a different comparison group further strengthens the credibility of our DiD design and affirms the plausibility of the parallel trends assumption [ 97 ]. Still, uncertainties regarding wastewater monitoring data affect interpretation of data from any sewershed. Limited knowledge exists about SARS-CoV-2 RNA fecal shedding quantity and duration, especially differences in fecal shedding patterns among demographic groups and vaccination statuses [ 64 ]. Studies suggest that SARS-CoV-2 RNA shedding quantity and duration in human excretions that contribute to wastewater differs among SARS-CoV-2 variants, which may affect the interpretation of wastewater monitoring data over time [ 76 – 79 ]. Wastewater monitoring data can also exhibit high day-to-day variability; potential mechanisms for this variability remain yet to be systematically understood but could be due to heterogeneity of the wastestream [ 98 ]. Future studies using longitudinal wastewater monitoring data for causal inference may consider analyzing changes in a computed outcome variable, such as a wastewater-based estimation of the effective reproductive number [ 54 ] or wastewater-based measure of trend [ 99 ], rather than changes in raw wastewater concentrations. Ultimately, the performance of such computed outcomes still depends on understanding the raw wastewater concentration data that are used to generate computed outcomes. Continued work investigating sources of uncertainty and variability in wastewater monitoring data—and particularly the effect size of these sources—is necessary for better interpretation of these data for public health use cases [ 100 , 101 ].

4. Conclusions

We assessed the correlation between wastewater concentrations of SARS-CoV-2 RNA and incident, reported COVID-19 cases at a university and evaluated changes to this correlation over time. Consistent with other studies, we provide evidence that the correlation between wastewater SARS-CoV-2 RNA concentrations and incident COVID-19 cases has changed over time. We further investigated the use of longitudinal wastewater monitoring data for policy impact evaluation. Using a DiD approach, we observed that most campus COVID-19 public health policy changes were not associated with a significant change in wastewater SARS-CoV-2 RNA concentrations on campus. The quasi-experimental design presented herein demonstrates how longitudinal wastewater monitoring of viruses at a small geographic scale may be used for causal inference when randomized controlled experiments are not possible to conduct.

Supporting information

S1 text. supporting information..

https://doi.org/10.1371/journal.pwat.0000242.s001

Acknowledgments

We acknowledge feedback on study design, implementation, and interpretation from James Jacobs, Julie Parsonnet, Russell Furr, Rich Wittman, Robyn Tepper, Jorge Salinas, Bonnie Maldonado, Christina Kong, and Stephanie Kalfayan. We acknowledge Palo Alto and San Jose wastewater treatment plant staff and the CR2C Student Operators team for wastewater sample collection.

• 1. U.S. Centers for Disease Control and Prevention. Nonpharmaceutical Interventions (NPIs). 4 Oct 2022 [cited 18 Jul 2023]. Available: https://www.cdc.gov/nonpharmaceutical-interventions/index.html .
• 2. Kaiser Family Foundation. State Actions to Mitigate the Spread of COVID-19. In: State Health Facts [Internet]. 4 Aug 2021 [cited 18 Jul 2023]. Available: https://www.kff.org/other/state-indicator/state-actions-to-mitigate-the-spread-of-covid-19/ .
• View Article
• PubMed/NCBI
• Google Scholar
• 8. Poeschl J, Larsen RB. How do non-pharmaceutical interventions affect the spread of COVID-19? A literature review. Danmarks Nationalbank; 2021 Apr. Report No.: No. 4. Available: https://www.nationalbanken.dk/en/news-and-knowledge/publications-and-speeches/archive-publications/2021/how-do-non-pharmaceutical-interventions-affect-the-spread-of-covid-19-a-literature-review .
• 15. World Health Organization. Environmental surveillance for SARS-COV-2 to complement public health surveillance: interim guidance. World Health Organization; 2022 Apr. Report No.: WHO/HEP/ECH/WSH/2022.1. Available: https://apps.who.int/iris/bitstream/handle/10665/353158/WHO-HEP-ECH-WSH-2022.1-eng.pdf?sequence=1 .
• 36. Gertler PJ, Martinez S, Premand P, Rawlings LB, Vermeersch CMJ. Impact Evaluation in Practice, Second Edition. Washington, DC: Inter-American Development Bank and World Bank; 2016. https://doi.org/10.1596/978-1-4648-0779-4
• 45. Stanford University. Codiga Resource Recovery Center at Stanford. In: Stanford Engineering [Internet]. [cited 24 Feb 2023]. Available: https://cr2c.stanford.edu/ .
• 46. Stanford University. Wastewater Analysis. In: Stanford COVID-19 Health Alerts [Internet]. [cited 17 Mar 2023]. Available: https://healthalerts.stanford.edu/covid-19/wastewater-dashboard/ .
• 47. City of Palo Alto. Regional Water Quality Control Plant. In: cleanbay.org [Internet]. [cited 24 Feb 2023]. Available: https://cleanbay.org/our-programs/regional-water-quality-control-plant/ .
• 48. Metcalf & Eddy Inc., Tchobanoglous G, Burton FL, Tsuchihashi R, Stensel HD. Processing and Treatment of Sludges. 5th ed. Wastewater Engineering: Treatment and Resource Recovery. 5th ed. New York, NY, USA: McGraw-Hill Professional; 2013. p. 1487.
• 59. Stanford University. COVID-19 resources for the Stanford community. In: Stanford COVID-19 Health Alerts [Internet]. [cited 24 Feb 2023]. Available: https://healthalerts.stanford.edu/covid-19 .
• 61. Stanford University. Academic Dates. In: Stanford Student Services [Internet]. [cited 24 Feb 2023]. Available: https://studentservices.stanford.edu/calendar/academic-dates .
• 63. U.S. Centers for Disease Control and Prevention. Symptoms of COVID-19. 26 Oct 2022 [cited 22 Feb 2023]. Available: https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html .
• 67. County of Santa Clara. Success in Vaccinating Most County Residents and Low COVID-19 Case Rates Allow Health Officer to Adopt New, More Focused Local Order. In: Santa Clara County Public Health [Internet]. 18 May 2021 [cited 3 Aug 2023]. Available: https://covid19.sccgov.org/news-releases/pr-05-18-2021-scc-adopts-more-focused-local-health-order .
• 68. California Department of Public Health. Blueprint for a Safer Economy. In: California Department of Public Health [Internet]. 15 Jun 2021 [cited 3 Aug 2023]. Available: https://www.cdph.ca.gov/Programs/CID/DCDC/Pages/COVID-19/COVID19CountyMonitoringOverview.aspx .
• 69. County of Santa Clara. Bay Area Health Officials Urge Immediate Vaccination and Issue Orders Requiring Use of Face Coverings Indoors to Prevent the Spread of COVID-19 ‐ Emergency Operations Center ‐ County of Santa Clara. In: Santa Clara County Public Health [Internet]. 2 Aug 2021 [cited 3 Aug 2023]. Available: https://covid19.sccgov.org/news-releases/pr-08-02-2021-health-officials-urge-immediate-vaccination-and-require-face-covering-indoors .
• 70. County of Santa Clara. County of Santa Clara Universal Indoor Mask Requirement to Transition to a Recommendation March 2. In: Santa Clara County Public Health [Internet]. 1 Mar 2022 [cited 3 Aug 2023]. Available: https://covid19.sccgov.org/news-releases/pr-03-01-2022-indoor-mask-requirement-transitions-to-recommendation .
• 72. U.S. Centers for Disease Control and Prevention. What is Case Surveillance? 27 Mar 2023 [cited 31 Aug 2023]. Available: https://www.cdc.gov/nndss/about/index.html .
• 83. Stanford University. Update for Stanford community on unvaccinated employee testing and face coverings. In: Stanford COVID-19 Health Alerts [Internet]. 28 Jul 2021 [cited 22 Aug 2023]. Available: https://healthalerts.stanford.edu/covid-19/2021/07/28/update-for-stanford-community-on-unvaccinated-employee-testing-and-face-coverings/ .
• 96. San José-Santa Clara Regional Wastewater Facility. In: City of San Jose [Internet]. [cited 16 Jun 2022]. Available: https://www.sanjoseca.gov/your-government/environment/water-utilities/regional-wastewater-facility .

This website may not work correctly because your browser is out of date. Please update your browser .

Using case studies to do program evaluation

• Using case studies to do program evaluation File type PDF File size 79.49 KB

This paper, authored by Edith D. Balbach for the California Department of Health Services is designed to help evaluators decide whether to use a case study evaluation approach.

It also offers guidance on how to conduct a case study evaluation.

This resource was suggested to BetterEvaluation by Benita Williams.

• Using a Case Study as an Evaluation Tool 3
• When to Use a Case Study 4
• How to Do a Case Study 6
• Unit Selection 6
• Data Collection 7
• Data Analysis and Interpretation 12

Balbach, E. D. 9 California Department of Health Services, (1999).  Using case studies to do program evaluation . Retrieved from website: http://www.case.edu/affil/healthpromotion/ProgramEvaluation.pdf

'Using case studies to do program evaluation' is referenced in:

Back to top

© 2022 BetterEvaluation. All right reserved.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 05 June 2024

In-situ tool wear condition monitoring during the end milling process based on dynamic mode and abnormal evaluation

• Min Chen 1   na1 ,
• Jianwei Mao 2   na1 ,
• Yu Fu 2   na1 ,
• Xin Liu 2   na1 ,
• Yuqing Zhou 3 &
• Weifang Sun 2  

Scientific Reports volume  14 , Article number:  12888 ( 2024 ) Cite this article

Metrics details

• Electrical and electronic engineering
• Information theory and computation
• Mechanical engineering

Rapid tool wear conditions during the manufacturing process are crucial for the enhancement of product quality. As an extension of our recent works, in this research, a generic in-situ tool wear condition monitoring during the end milling process based on dynamic mode and abnormal evaluation is proposed. With the engagement of dynamic mode decomposition, the real-time response of the sensing physical quantity during the end milling process can be predicted. Besides, by constructing the graph structure of the time series and calculating the difference between the predicted signal and the real-time signal, the anomaly can be acquired. Meanwhile, the tool wear state during the end milling process can be successfully evaluated. The proposed method is validated in milling tool wear experiments and received positive results (the mean relative error is recorded as 0.0507). The research, therefore, paves a new way to realize the in-situ tool wear condition monitoring.

Introduction

Condition monitoring and fault diagnosis for computer numerical control (CNC) machines have been widely investigated in recent years and achieved great progress 1 , 2 . As a crucial component used to remove materials from the workpiece, the cutting tool’s running state will inevitably influence the surface quality of the final part, as well as the cutting process stability 3 , 4 . Therefore, rapid tool operating state estimation is important to maintaining the machining performance of the cutting system, preventing workpiece scrap and operator injury.

To realize rapid tool operating state estimation, considerable research efforts have been devoted. In such works, based on the physical location and measurement object of the sensor, those methods can be divided into direct methods and indirect methods 5 . As can be seen in Fig.  1 , direct methods can directly acquire the digital image of cutting edges and evaluate the tool wear state accordingly. For the indirect methods, dynamic signals during the manufacturing process can be sampled across the sensor mounted on the workpiece, spindles, or other components 6 , 7 . The tool wear state can be estimated indirectly based on the acquired signals.

Tool condition monitoring methods.

Benefiting from the implementation convenience, direct methods were successfully demonstrated in a number of studies, and the robustness of the methods is also testified. To realize the tool condition monitoring, considerable attention has been paid to evolution mechanism exploration and attempted to identify the service state based on their characteristic information. Among them, feature extraction based on sparse measure optimization has emerged as an interesting candidate for identifying the health state of mechanical systems. Based on the specific requirement, via the feature extraction methods, the mathematical model and the response characteristics can be investigated. After that, the optimal filter bank is obtained through iterative or non-iterative methods to achieve explicit representation of features. To address the problem that traditional tool wear prediction methods rely on the experience and knowledge of experts, Yang et al. 8 proposed a new tool wear prediction method based on local features and global dependencies. Focus on the weak fault detection of the rolling bearing in strong noise conditions, Deng et al. 9 propose a novel fault diagnosis method with an improved empirical wavelet transform (EWT) and the maximum correlated kurtosis deconvolution (MCKD). To address the low efficiency of iterative solutions during the MCKD process, Mcdonald et al. 10 proposed a non-iterative deconvolution method to directly acquire the optimal filter coefficient and successfully apply it in related scenarios. These researches provide the theoretical basis for system state identification. However, with the increasing complexity and systematization of mechanical equipment, the failure modes also become complex and variable, which leads to the instability of the proposed methods. In addition, traditional sparse measure optimization methods strongly rely on the prior knowledge of professional technicians and diagnostic experts in the diagnostic process (such as system structure, fault frequency, etc.) 11 , which restricts the applicability of these excellent methods in a wider range of engineering application scenarios.

With the rapid development of machine learning technology, artificial intelligence (AI) based fault diagnosis and prediction have increasingly become an important strategy for equipment safety and service monitoring 12 . Via related intelligent algorithms, the data-driven diagnostic method can adaptively identify equipment operation status information from existing data without the need of prior knowledge for professional technicians 13 . With an edge-labeling graph neural network method, Zhi et al. 14 propose a tool for wear condition monitoring using wear images which suitable for small sample conditions. Mishra et al. 15 developed a tool condition estimation method during the precision machining process with the unsupervised approach. However, data-driven methods are inevitably influenced by the distribution of training data, which may lead to data bias in the training model. Combining the sparrow search algorithm, Li et al. 16 developed a CNN-BiLSTM-based neural network to effectively predict sea level heights. Therefore, if the equipment status characteristics can be effectively re-characterized through a simple method, it is expected to overcome related shortcomings and achieve robust identification of its service performance.

Recently, time-domain-based CM-FD methods have been intensively investigated and successfully applied in some application scenarios 17 . Among them, the graph-based method enjoys the merits of anomalies quantitative evaluation and approximate shift-invariance 18 , 19 . However, it remains challenging to establish the adjacency matrix in a short time which might threaten the online evaluation reliability 20 . To overcome the potential drawbacks, as an extension of our works, Shiliang Feng et al. 21 proposed a time-domain signal-driven mechanical system state description method and validated in some typical mechanical experiments.

During the manufacturing process, rapid tool wear might unpredictably occur, especially for hard-to-machining materials (e.g. nickel-based alloy or titanium alloy). The rapid tool wear will greatly affect the durability of cutting tools and the integrity of machining surfaces. To effectively trace the tool wear dynamic variation and avoid rapid deterioration of surface integrity, it is crucial to predict the short-term time-series response and estimate the tool wear status in advance. However, very little optimization work has been carried out on the dynamic evaluation of the tool wear state based on the predicted short-term time series.

Focus on the drawbacks and the existing research gap mentioned above, inspired by the time-domain-based CM-FD methods, in this research, a tool service state evaluation method that does not rely on any prior knowledge is proposed. With dynamic mode decomposition, the time series in the next snapshot can be predicted. Furthermore, based on graph structure, the anomalies of the tool wear state can be identified. The proposed strategy enjoys the merit of short-time time series prediction and offers exciting opportunities for rapid monitoring of tool wear state. The main structure of the manuscript is summarized as follows. A brief description of the proposed data prediction based on the dynamic mode decomposition method is listed in “ Data prediction based on dynamic mode decomposition method ” section. “ Anomalies identification for time-series ” section presents the proposed anomaly identification for time series. The In-situ tool wear condition monitoring is summarized in “ The proposed in-situ tool wear condition monitoring method ” section. The validation experiment of the proposed tool wear estimation method is listed in “ Experimental investigation ” section. The conclusion of the manuscript is listed in “ Conclusion ” section.

Data prediction based on dynamic mode decomposition method

As a typical fluid dynamics analysis method, benefiting from the extraordinary spatiotemporal feature presentation ability (decomposing complex flow processes into low-rank spatiotemporal features), dynamic mode decomposition has lately received great attention. Because the decomposition does not rely on any given dynamic model, the method is suitable for dynamic process description. In this section, dynamic mode decomposition is employed for short-term time-series prediction.

Model establishment

After equally resampling from the temporal signals, a multivariate time series can be acquired. Assuming the combined multivariate time series is composed of M temporal signals with a length of T , the expression at time t can be expressed as 22 :

where A is the Koopman matrix (coefficient matrix in vector autoregression process) with a dimension of M  ×  M , and ε is the residual term.

Similar, by arranging the T snapshots into two large data matrices:

The expression of the dynamic mode decomposition can be represented as:

Equation ( 3 ) can be regarded as a vector autoregressive problem. If A can be regarded as the Koopman matrix in dynamic mode decomposition, a low-rank structure can be used for the approximation. For autoregressive problems, if it is necessary to calculate the coefficient matrix A , by minimizing the squared residual, the matrix can be acquired.

Model solution

To decrease the model calculation complexity, intrinsic orthogonal decomposition methods (e.g. singular value decomposition) are widely employed to map the high-dimensional variables to low dimension.

With singular value decomposition, the matrix X 1 can be decomposed by 23 :

where U is a m -order unitary matrix, V is a n -order unitary matrix, Σ is a non-negative real diagonal matrix with dimension of m  ×  n . Generally, each eigenvector in V is called the right singular vector of M , each eigenvector in U is called the left singular vector, and the elements on the diagonal of D are called the singular values of M . When the singular values are arranged in descending order, a unique D can be determined.

If the matrix X 1 is truncated for singular value decomposition with a rank of r , the Koopman matrix A can be approximated using the following matrix:

where matrix U r   ∈   ℝ M × r , V r   ∈   ℝ ( T −1)× r , Σ r   ∈   ℝ r× r are the truncation matrixes of the unitary matrix U , the unitary matrix V , the non-negative real diagonal matrix, respectively.

Data prediction

If it is necessary to solve the modal of a matrix and analyze its spatiotemporal characteristics using it, the matrix can be decomposed into eigenvalues 24 :

where \(\Phi\) is a diagonal matrix (the diagonal elements are the corresponding eigenvalues), the matrix Q is composed by the eigenvectors. Therefore, eigenvalues and eigenvectors can be used to analyze and predict the complex spatiotemporal characteristics of the system.

The mode of the dynamic can be defined as:

Therefore, the dynamic prediction of data can be represented as:

where the symbol † indicates the Moore Penrose generalized inverse operation.

Anomalies identification for time-series

A graph mechanism based on temporal signals is proposed to identify anomalies in temporal data.

Graph structure description

In recent years, a novel abnormal health status of equipment evaluation method using time-domain signals has been proposed and aroused wide concerns in mechanical systems 25 , 26 . Based on the graph structure in computer science, the internal feature structure of signals can be evaluated, and the health state of the equipment can also be evaluated accordingly.

According to the graph theoretical, a graph structure can be expressed by G  = { N , E }, where N is the set of nodes and E is the set of connections. Among them, the node set can be used to describe different sampling points, and the connection set E is used to describe the connection strength between different nodes. The connection strength between different nodes is reversible, so the set of connections is clearly a symmetric matrix.

For any temporal signal (as shown in Fig.  2 a), the adjacency matrix of its data segments can be expressed as a symmetric matrix (as shown in Fig.  2 c). In this study, the connection strength between different nodes is described by the Euclidean distance between nodes (as shown in Fig.  2 b). Therefore, the collected temporal signals can be reprojected into a set of adjacency matrices (detail of the process can be seen in Ref. 25 ):

where Xn is the n -th symmetric matrix that constitutes the set of adjacency matrices.

Adjacency matrix construction. ( a ) Node determination ( b ) Graph illustration ( c ) Adjacency matrix.

Anomalies identification

Based on the definition of graph structure, related researchers found that if the equipment state (i.e. operating state) is changed, the internal structure or parameters of its corresponding adjacency matrix will also change. Therefore, by evaluating the differences between the corresponding adjacency matrices, the dynamic characteristic of the equipment operation status can be monitored. The methods for anomaly identification can be summarized as follows.

The “standard template” X (in normal state) can be established from the sampled time series based on the traditional graph structure. The standard template matrix can be decomposed by 27 :

where \(\Lambda\) is the eigenvalue matrix (diagonal elements are eigenvalues), \({{\varvec{\Gamma}}}\) is the eigenvector matrix (each column is the eigenvector of matrix X ).

For a given test signal y , the corresponding adjacency matrix Y can be established accordingly. Similarity, the adjacency matrix Y can be decomposed as 28 :

where the symbol diag [.] represents the diagonal elements of the adjacency matrix, and non–diag [.] represents the non-diagonal elements of the adjacency matrix.

By evaluating the non-diagonal components, the similarity between the two signals can be evaluated. For in-situ tool condition monitoring problems, the Frobeniu norm of the non-diagonal can be directly employed for the tool wear evaluation.

The proposed in-situ tool wear condition monitoring method

Combining the dynamic mode decomposition and real-time prediction signal anomaly identification, a method for evaluating the wear status of machining tools without relying on any given prior knowledge is proposed in this research. The main process of this method can be described in Fig.  3 , the specific step is listed as follows:

Step 1 Based on the sensor (near the cutting area) and the digital signal acquisition device, the dynamic signal which can reflect the tool service status information is recorded.

Step 2 By using the dynamic mode decomposition method established in “ Data prediction based on dynamic mode decomposition method ” section, the time-series signal in the next moment can be predicted.

Step 3 With the graph establishment approach (as shown in Eq. ( 10 )), the graph structures of the current moment and the predicted signal can be constructed.

Step 4 Taking the acquired signal as the “standard template”, as shown in Eq. ( 12 ), the graph structure of the predicted signal can be decomposed.

Step 5 The Frobeniu norm of the acquired non-diagonal elements can be used to evaluate the similarity (the tool wear state) between the two signals.

Flowchart of the proposed method.

Experimental investigation

To verify the proposed method, an open-source database, and actual milling experiments are employed for the effectiveness verification of the proposed method.

Investigation in NASA database

As a typical dataset, the NASA Ames and UC Berkeley milling dataset is widely used in the research on the tool condition monitoring of general machining 29 . To investigate the applicability of the proposed method, the NASA dataset is employed in this section. As listed in Table 1 , the acoustic emission signals acquired during the experiment (Mstsuura MC-510 V machine center is employed for the experiment) under the spindle speed of 826 rev/min, depth of cut is 0.75 mm, feed speed of 0.25 mm/rev. A 70 mm face mill with 6 inserts is employed for the processing.

According to the presentation above, appropriate graph structure construction is crucial for tool wear evaluation. The collected time-domain signals contain a significant amount of dynamic information, which can reflect the state of the machining process. Long sampling points can better preserve dynamic information but inevitably affect the timeliness of calculations. Too few sampling points result in the generated graph structure being unable to accurately describe the state information of the machining process. To investigate the performance of tool wear situation in NASA database, setting 800 as the data length of the research, the first sample (data length is 800) is considered as the reference signal (or healthy signal), other samples in the database can be considered as testing signals. Based on the adjacency establish equation before, the calculated reference adjacency is shown in Fig.  4 a. Accordingly, the corresponding eigenvalue and eigenvectors (Fig.  4 b), columns are the corresponding eigenvectors) are acquired after the diagonalization of the reference adjacency.

Signal diagonalization of NASA signal. ( a ) Adjacency. ( b ) Eigenvector.

Based on the proposed method and the acquired eigenvectors, the fluctuation value of the corresponding signal can be calculated. To evaluate the performance quantitatively, the measured tool wear area and the anomaly are normalized as [0, 1]. The normalization can be represented as:

where z indicates the tool wear area or the anomalies sequence, z n is the normalized sequence, min( ⋅ ), max( ⋅ ) are the minimum and maximum value of the sequence respectively. The evaluated tool wear condition is shown in Table 2 . As illustrated in the table, in the whole 23 continuous samples, the corresponding error is ranging from 0 to 0.3280. Accordingly, the mean error between the evaluated tool wear state and the measured tool wear values is calculated as 0.1482. Figure  5 plots the normalized similarity (evaluated tool wear state based on the proposed method, red solid curve in the figure) and the normalized tool wear values (measured tool wear value, blue solid curve in the figure). As shown in Fig.  5 , during the milling process, with the deterioration of the tool state, the Frobeniu norm (anomalies) also increased. The two variables had significant simultaneous change trends.

Tool wear evaluation of NASA signal.

Milling experiment

Experiment setup.

The experiment setup is shown in Fig.  6 . In this experiment, the end milling experiment is conducted on a vertical machining center (Dalian Machine Tool Group DMTG VDL 850A). During the experiment, a kind of uncoated tungsten steel end milling cutter (diameter of 10 mm, detail of the milling cutter can be seen in Fig.  7 , Table 3 ) was employed to cut the workpiece (45 steel, with dimension of 300 × 100 × 80 mm, the chemical properties of the workpiece material is shown in Table 4 ). Related literatures 30 , 31 have shown that there is a certain correlation between the sound pressure signals and the tool wear status during the manufacturing process. To minimize the impact of sensor installation on the machining environment, in this experiment, a non-contact sound pressure sensor is employed to collect dynamic signals during the milling process. During the whole process, the sound pressure sensor (GRAS 46AE, the sensitivity is 50 mV/Pa) is mounted on the table of the machining center near the workpiece (approximately 100 mm away from the workpiece) and used for the acquisition of sound signal. The dynamic signals in this experiment are recorded by a data acquisition instrument (Econ MI-7016 Avant) with a 12 kHz sampling frequency.

Experiment setup. ( a ) Machine tool. ( b ) Magnification of the marked rectangle area.

Diagram of the milling cutter.

To accurately evaluate the actual tool wear state, a direct measuring instrument (Ksgaopin precision instrument GP-300C, as shown in Fig.  8 ) is employed for the estimation. Obviously, the three teeth are independent, the evaluation process of each tooth should perform separately. In the experiment process, the workpiece is manufactured layer upon layer. There are three forward and two backward cuts in each layer, as described in Fig.  9 . After finishing one layer of the workpiece, the tool holder is taken off to evaluate the tool wear state. Generally, according to ISO3685-1977, the tool wear state is presented as the tool flank wear VB. However, as mentioned in related literature, the one-dimensional evaluation parameter cannot fully reflect the tool wear state. In this research, the mean flank wear area of the three flanks is indicated for the tool wear state evaluation.

Direct tool wear detection.

Machining path.

Tool wear evaluation

In order to validate the reliability in actual milling experiments, two milling tests were conducted with the experimental setup. The experimental conditions are shown in Table 5 .

Figure  10 plots the waveforms of the acquired acoustic signals via the sound sensor, as well as the frequency domain. The sampled signals in the 1st layer, 3rd layer, and 5th layer are described in Fig.  10 a,c,e, respectively. The corresponding frequency spectrums are listed in Fig.  10 b,d,f, respectively. As can be seen in the figure, there is almost no obvious variation law or characteristics between the two samples either in time-domain waveforms or frequency-domain. The local magnification of the frequency spectrum (green dashed rectangle in Fig.  10 f) of the 5th layer is shown in Fig.  10 g. The spindle speed during the milling process is 2400 rev/min. Therefore, the spindle rotating frequency (40 Hz) and its harmonics can be observed (orange dashed lines). Caused by the distributed three teeth, the cutting frequency (120 Hz) and its harmonics can also be monitored (green dashed lines).

Signal samples in the milling process. ( a ) Time-domain signal of the 1st layer. ( b ) Frequency spectrum of the 1st layer. ( c ) Time-domain signal of the 3rd layer. ( d ) Frequency spectrum of the 3rd layer. ( e ) Time-domain signal of the 5th layer. ( f ) Frequency spectrum of the 5th layer. ( g ) Local magnification of ( f ).

With the mentioned direct measuring instrument, the actual tool wear state during the manufacturing process can be monitored. Generally, the evolution of wear in tool experiments is a continuous process. The variation process of the wear area and its average value of three cutting edges is shown in Fig.  11 , as well as the tool wear images. According to Fig.  11 , caused by direct contact with the cutting material, generally, a triangular wear band will occur at the cutter edge tip. Besides, the maximum flank wear will also tend to appear around the tooltip. The specific information during the milling tool wear evolution process is shown in Table 6 . As can be seen in Fig.  11 and Table 6 , with the increase in cutting time, the tool wear area grew to mm 2 (mean tool wear area).

Tool wear variation in milling Case I.

By considering the first sample (still setting the data length as 800) is considered as the reference signal (or healthy signal), the other samples can be considered as testing signals. Based on the adjacency establish equation before, the calculated reference adjacency is shown in Fig.  12 a. Accordingly, the corresponding eigenvalue and eigenvectors (Fig.  12 b, columns are the corresponding eigenvectors) are acquired after the diagonalization of the reference adjacency.

Signal diagonalization of the first sample in Case I. ( a ) Adjacency. ( b ) Eigenvector.

Based on the proposed method and the acquired eigenvectors, the fluctuation can be calculated for the similarity evaluation according to Eq. ( 12 ). The measured tool wear states, the anomalies, and their errors are summarized in Table 7 . As illustrated in the table, in the whole 6 continuous samples, the corresponding error is ranging from 0 to 0.2350. Accordingly, the mean error between the evaluated tool wear state and the measured tool wear values is calculated as 0.0881. Figure  13 plots the normalized similarity (evaluated tool wear state based on the proposed method, blue solid curve in the figure) and the normalized tool wear values (measured tool wear value, red solid curve in the figure). The slimier simultaneous trend indicates the potential mapping relationship between the Frobeniu norm (anomalies) and the tool wear.

Tool wear evaluation in Case I.

The variation process of the wear area and its average value of three cutting edges is shown in Fig.  14 , as well as the tool wear images. The specific information during the milling tool wear evolution process is shown in Table 8 . As can be seen in Fig.  14 and Table 8 , with the increase in cutting time, the tool wear area grew to mm 2 (mean tool wear area).

Tool wear variation in milling Case II.

With the same investigation method mentioned before, the measured tool wear states, the anomalies, and their errors can be calculated, as listed in Table 9 . As illustrated in the table, in the whole 6 continuous samples, the corresponding error ranges from 0 to 0.1266 (the measured tool wear values is calculated as 0.0484). Figure  15 plots the normalized similarity (evaluated tool wear state based on the proposed method, blue solid curve in the figure) and the normalized tool wear values (measured tool wear value, red solid curve in the figure).

Tool wear evaluation in Case II.

Tool wear evaluation in different data length

As presented above, data length during the data processing process is crucial for the tool wear evaluation. To investigate the influence of data length in the evaluation process, repeat measuring experiments are conducted. The results in different data lengths are shown in Table 10 . As can be conducted in the table, when the data length varies from 100 to 1000, the evaluation error distribution in the two experiments has no obvious change regular pattern. According to the results, the mean error for the two experiments reaches its minimum value when the data length is 800. Therefore, in this research, the data length is set as 800.

Aiming to the quantitative evaluation of tool wear state, in this research, authors proposed a two-stage method to estimate the tool running condition directly from the time series. In the prediction stage, with the engagement of dynamic mode decomposition, the real-time response of the end milling process can be predicted. In the estimation process, by constructing the graph structure of the time series and calculating the difference between the predicted signal and the real-time signal, the tool wear state during the end milling process can be successfully evaluated. To further confirm the effectiveness of the proposed method, investigations in an open source are presented and achieve a preferable effect. The results were also confirmed by an actual milling experiment from our laboratory. Accordingly, the combination of dynamic mode decomposition and anomalies evaluation method presents a wide range of possibilities for the further development of condition monitoring and fault detection techniques via time series. Besides, in this research, it is assumed that there is a linear relationship between the anomalies and the real tool wear. The nonlinear factors caused by environmental factors such as process parameters and tools have not been considered. Further mechanistic studies and the development of the proposed method are still ongoing in our team.

Data availability

The data that support the findings of this study are included and will be available from the corresponding author upon reasonable request.

Pimenov, D. Y. et al. Artificial intelligence systems for tool condition monitoring in machining: Analysis and critical review. J. Intell. Manuf. 34 , 2078–2121. https://doi.org/10.1007/s10845-022-01923-2 (2023).

Article   Google Scholar  

Lin, W. J. et al. Integrating object detection and image segmentation for detecting the tool wear area on stitched image. Sci. Rep. 11 (1), 19938. https://doi.org/10.1038/s41598-021-97610-y (2021).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Wojciechowski, S. & Twardowski, P. Tool life and process dynamics in high speed ball end milling of hardened steel. Procedia CIRP 1 , 2012. https://doi.org/10.1016/j.procir.2012.04.052 (2012).

Wojciechowski, S. et al. Study on ploughing phenomena in tool flank face—Workpiece interface including tool wear effect during ball-end milling. Tribol. Int. 181 , 108313. https://doi.org/10.1016/j.triboint.2023.108313 (2023).

Article   CAS   Google Scholar  

Zhou, Y., Sun, B. & Sun, W. A tool condition monitoring method based on two-layer angle kernel extreme learning machine and binary differential evolution for milling. Measurement 166 , 108186. https://doi.org/10.1016/j.measurement.2020.108186 (2020).

Lei, Z. et al. A GAPSO-enhanced extreme learning machine method for tool wear estimation in milling processes based on vibration signals. Int. J. Precis. Eng. Manuf. Green Technol. 8 , 745–759. https://doi.org/10.1007/s40684-021-00353-4 (2021).

Zi, X., Gao, S. & Xie, Y. An online monitoring method of milling cutter wear condition driven by digital twin. Sci. Rep. 14 (1), 4956. https://doi.org/10.1038/s41598-024-55551-2 (2024).

Article   CAS   PubMed   Google Scholar  

Yang, C., Zhou, J., Li, E., Wang, M. & Ting, J. Local-feature and global-dependency based tool wear prediction using deep learning. Sci. Rep. 12 (1), 14574. https://doi.org/10.1038/s41598-022-18235-3 (2022).

Deng, W., Zhang, S., Zhao, S. & Yang, X. A novel fault diagnosis method based on improved empirical wavelet transform and maximum correlated kurtosis deconvolution for rolling element bearing. J. Mech. Eng. 55 (23), 136–146. https://doi.org/10.3901/JME.2019.23.136 (2019).

McDonald, G. L. & Zhao, Q. Multipoint optimal minimum entropy deconvolution and convolution fix: Application to vibration fault detection. Mech. Syst. Signal Process. 82 , 461–477. https://doi.org/10.1016/j.ymssp.2016.05.036 (2017).

Article   ADS   Google Scholar  

Liu, Y., Xiang, H., Jiang, Z. & Xiang, J. Iterative synchrosqueezing-based general linear chirplet transform for time-frequency feature extraction. IEEE Trans. Instrum. Meas. 72 , 1–11. https://doi.org/10.1109/TIM.2022.3232090 (2023).

Ruan, D., Han, J., Yan, J. & Gühmann, C. Light convolutional neural network by neural architecture search and model pruning for bearing fault diagnosis and remaining useful life prediction. Sci. Rep. 13 (1), 5484. https://doi.org/10.1038/s41598-023-31532-9 (2023).

Wu, T. et al. Remaining useful life prediction of circuit breaker operating mechanisms based on wavelet-enhanced dual-tree residual networks. J. Power Electron. 24 (1), 78–91. https://doi.org/10.1007/s43236-023-00706-z (2024).

Article   ADS   CAS   Google Scholar  

Zhi, G. et al. An edge-labeling graph neural network method for tool wear condition monitoring using wear image with small samples. Meas. Sci. Technol. 32 (6), 064006. https://doi.org/10.1088/1361-6501/abe0d9 (2021).

Mishra, D., Awasthi, U., Pattipati, K. R. & Bollas, G. M. Tool wear classification in precision machining using distance metrics and unsupervised machine learning. J. Intell. Manuf. https://doi.org/10.1007/s10845-023-02239-5 (2023).

Li, X., Zhou, S., Wang, F. & Fu, L. An improved sparrow search algorithm and CNN-BiLSTM neural network for predicting sea level height. Sci. Rep. 14 (1), 4560. https://doi.org/10.1038/s41598-024-55266-4 (2024).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dutta, A., McKay, M., Kopsaftopoulos, F. & Gandhi, F. Statistical residual-based time series methods for multicopter fault detection and identification. Aerosp. Sci. Technol. 112 , 106649. https://doi.org/10.1016/j.ast.2021.106649 (2021).

Wang, T., Liu, Z., Lu, G. & Liu, J. Temporal-spatio graph based spectrum analysis for bearing fault detection and diagnosis. IEEE Trans. Ind. Electron. 68 (3), 2598–2607. https://doi.org/10.1109/TIE.2020.2975499 (2020).

Yang, C., Liu, J., Zhou, K. & Li, X. Dynamic spatial-temporal graph-driven machine remaining useful life prediction method using graph data augmentation. J. Intell. Manuf. 35 (1), 355–366. https://doi.org/10.1007/s10845-022-02052-6 (2024).

Sun, W., Zhou, Y., Xiang, J., Chen, B. & Feng, W. Hankel matrix-based condition monitoring of rolling element bearings: An enhanced framework for time-series analysis. IEEE Trans. Instrum. Meas. 70 , 1–10. https://doi.org/10.1109/TIM.2021.3062194 (2021).

Feng, S. et al. A time-series driven mechanical system state description method and its application in condition monitoring. IEEE Sens. J. 23 (9), 9677–9684. https://doi.org/10.1109/JSEN.2023.3260921 (2023).

Kutz, J. N., Brunton, S. L., Brunton, B. W. & Proctor, J. L. Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems (Society for Industrial and Applied Mathematics, 2016).

Book   Google Scholar  

Dai, L., Cao, W., Yi, S. & Wang, L. Damage identification of concrete structure based on WPT-SVD and GA-BPNN. J. Zhejiang Univ. (Eng. Sci.) 57 (1), 100–110. https://doi.org/10.3785/j.issn.1008-973X.2023.01.011 (2023).

Yang, J., Shen, L., Zheng, Z., Li, T. & Yang, Y. Transmission tower looseness detection based on dynamic mode decomposition. J. Vib. Shock 42 (19), 204–211. https://doi.org/10.13465/j.cnki.jvs.2023.19.027 (2023).

Zhang, F. et al. A health condition assessment and prediction method of Francis turbine units using heterogeneous signal fusion and graph-driven health benchmark model. Eng. Appl. Artif. Intell. 126 , 106974. https://doi.org/10.1016/j.engappai.2023.106974 (2023).

Liu, J., Zhou, K., Yang, C. & Lu, G. Imbalanced fault diagnosis of rotating machinery using autoencoder-based SuperGraph feature learning. Front. Mech. Eng. 16 (4), 829–839. https://doi.org/10.1007/s11465-021-0652-4 (2021).

Wen, X., Lu, G., Liu, J. & Yan, P. Graph modeling of singular values for early fault detection and diagnosis of rolling element bearings. Mech. Syst. Signal Process. 145 , 106956. https://doi.org/10.1016/j.ymssp.2020.106956 (2020).

Sun, W. et al. A two-stage method for bearing fault detection using graph similarity evaluation. Measurement 165 , 108138. https://doi.org/10.1016/j.measurement.2020.108138 (2020).

Goebel, A. A. A. K. Best Lab, UC Berkeley Milling Data Set (NASA Ames Research Center).

Liu, M., Tseng, Y. & Tran, M. Tool wear monitoring and prediction based on sound signal. Int. J. Adv. Manuf. Technol. 103 , 3361–3373. https://doi.org/10.1007/s00170-019-03686-2 (2019).

Li, Z. et al. A novel ensemble deep learning model for cutting tool wear monitoring using audio sensors. J. Manuf. Process. 79 , 233–249. https://doi.org/10.1016/j.jmapro.2022.04.066 (2022).

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 52205122.

Author information

These authors contributed equally: Min Chen, Jianwei Mao, Yu Fu and Xin Liu.

Authors and Affiliations

Zhejiang Dewei Cemented Carbide Manufacturing Co., Ltd., Wenzhou, 325699, China

College of Mechanical and Electrical Engineering, Wenzhou University, Wenzhou, 325035, China

Jianwei Mao, Yu Fu, Xin Liu & Weifang Sun

College of Mechanical and Electrical Engineering, Jiaxing Nanhu University, Jiaxing, 314001, China

Yuqing Zhou

You can also search for this author in PubMed   Google Scholar

Contributions

M.C. conceived the experiment, and together with Y.F. and X.L. carried it out; J.M. designed and carried out the data analysis; M.C. and W.S. co-wrote the paper; All authors reviewed the manuscript.

Corresponding author

Correspondence to Weifang Sun .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chen, M., Mao, J., Fu, Y. et al. In-situ tool wear condition monitoring during the end milling process based on dynamic mode and abnormal evaluation. Sci Rep 14 , 12888 (2024). https://doi.org/10.1038/s41598-024-63865-4

Download citation

Received : 10 March 2024

Accepted : 03 June 2024

Published : 05 June 2024

DOI : https://doi.org/10.1038/s41598-024-63865-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Dynamic mode decomposition
• Condition monitoring
• Abnormal evaluation
• Graph similarity

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Content Search

Targeting in protracted crises: nigeria case study.

This country case study focuses on Nigeria and the specific challenge of conflict, violence, and insecurity. Using four waves of General Household Survey data covering the period 2010 to 2019, we analyse trends in poverty, food insecurity, shocks, and coping strategies among different population groups, differentiated according to where they reside in the country and the degree to which those areas are affected by violence, in particular as a result of the militant Islamist Boko Haram insurgency and conflicts between herders and farmers. The survey data is then used to model the notional performance of different potential targeting approaches across a range of targeting performance indicators, to indicate the types of choices and trade-offs entailed when selecting different targeting criteria for either routine or humanitarian social assistance programmes in the context of Nigeria. We also consider the status of enabling conditions for implementing different targeting approaches in the form of key infrastructure. We conclude with a discussion of the interrelated considerations social assistance programmes have to contend with when selecting appropriate targeting criteria.

Read the full report

Related Content

Nigeria + 2 more

Protection Monitoring Report, UNHCR Maiduguri Sub-Office, March - April 2024

Nigeria key message update may 2024: atypical staple prices and below-average dry harvest led to high food assistance needs, 2024.

Nigeria + 5 more

West Africa Key Message Update April 2024: La persistance de l’insécurité et la hausse des prix continuent de limiter l’accès alimentaire des ménages dans les zones de conflits, 2024

Nigeria - increased violence (dg echo, dg echo partners, media) (echo daily flash of 29 may 2024).

Influence of tectonic effects on the formation and characteristics of landslide dams on the NE Tibetan Plateau: a case study in the Bailong River Basin, China

• Original Paper
• Published: 04 June 2024

Cite this article

• Guan Chen 1 , 2 , 4 ,
• Jiacheng Jin 1 , 2 , 4 ,
• Xingmin Meng 1 , 2 , 4 ,
• Tianjun Qi 1 , 2 , 4 ,
• Wei Shi 2 , 3 , 4 ,
• Yan Chong 2 , 3 , 4 ,
• Yunpeng Yang 1 , 3 , 4 &
• Shiqiang Bian 2 , 3 , 4  

Hazards created by the landslide damming of rivers have become common in tectonically active mountainous areas. However, it remains unclear how tectonic effects may influence the formation and characteristics of landslide dams. The purpose of this paper is to explore how tectonic effects impact the drivers, geomorphic features, and activity characteristics of landslide dams along a fault zone. We investigated 83 landslide dams clustered along a fault zone in the Bailong River Basin. Most of the dams are located in areas of high tectonic stress, resulting from the rapid river incision and destruction of slope structure caused by intense tectonic activities in these areas. Statistical analysis, InSAR monitoring, and field investigation revealed that different tectonic effects were associated with significant differences in the geomorphic features, activity characteristics, and controlling factors of the landslide dams. Thus, we identified three distinct patterns of landslide dams in tectonically active mountainous areas: (1) Topography-driven landslide dams are caused by rapid rock uplift and river incision. Here, the steep terrain enhances the development of small landslides, the narrowness of the channels favors river damming, and the residual deposits on the hillslope remain active. (2) Tectonic activities promote the development of structural planes in the rock mass and reduce its strength, ultimately forming structural plane-controlled landslide dams. Although their volumes are not very large, the strong erosion resistance of rockslides can cause river damming and maintain the stability of deposits. (3) Fractured rock mass-controlled landslide dams are composed of broken rock and fault gouge. The extremely low strength of these materials allows them to form very large landslides that can easily dam the river, and maintain a slow-moving state. Through a geomorphological and geological model, our study offers new insights and enhances the understanding of the formation and characteristics of landslide dams induced by tectonic activity in mountainous regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Ambrosi C, Crosta GB (2011) Valley shape influence on deformation mechanisms of rock slopes. Geological Society, London, Special Publications 351:215–233. https://doi.org/10.1144/SP351.12

Article   Google Scholar  

Bai S, Cheng C, Wang J, Thiebes B, Zhang Z (2013) Regional scale rainfall- and earthquake-triggered landslide susceptibility assessment in Wudu County. China J Mt Sci 10(5):743–753. https://doi.org/10.1007/s11629-013-2432-z

Berardino P, Fornaro G, Lanari R, Sansosti E (2002) A new algorithm for surface deformation monitoring based on small baseline differential SAR interferograms. Geosci Remote Sen, IEEE Transactions on 40(11):2375–2383. https://doi.org/10.1109/TGRS.2002.803792

Borrelli L, Gullà G (2017) Tectonic constraints on a deep-seated rock slide in weathered crystalline rocks. Geomorphology 290:288–316. https://doi.org/10.1016/j.geomorph.2017.04.025

Bull WB, McFadden LD (1977) Tectonic geomorphology north and south of the Garlock fault, California. In: Doehring DO (ed) Geomorphology in Arid Regions. Proceedings of the Eighth Annual Geomorphology Symposium. State University of New York, Binghamton, pp 115–138

Google Scholar  

Carlini M, Chelli A, Vescovi P, Artoni A, Clemenzi L, Tellini C, Torelli L (2016) Tectonic control on the development and distribution of large landslides in the Northern Apennines (Italy). Geomorphology 253:425–437. https://doi.org/10.1016/j.geomorph.2015.10.028

Carlini M, Chelli A, Francese R, Giacomelli S, Giorgi M, Quagliarini A, Carpena A, Tellini C (2017) Landslides types controlled by tectonics-induced evolution of valley slopes (Northern Apennines, Italy). Landslides 15(2):283–296. https://doi.org/10.1007/s10346-017-0871-2

Casagli N, Ermini L (1999) Geomorphic Analysis of Landslide Dams in the Northern Apennine. Trans Jpn Geomorphl 20(3):219–249

Chang Z, Wang J, Bai S, Zhang Z (2014) Appraisal of active tectonic in Bailongjiang basin on DEM data. Quat Sci 34(2):292–301 (in Chinese)

Chen W, Zhao Z, Liu G, Liang S, Yang Z, Han W, Da S (2006) Research on engineering geology of Lanzhou-Haikou highway in Gansu. Lanzhou University Press, Lanzhou (in Chinese)

Chen X, Cui P, You Y, Cheng Z, Khan A, Ye C, Zhang S (2016) Dam-break risk analysis of the Attabad landslide dam in Pakistan and emergency countermeasures. Landslides 14(2):675–683. https://doi.org/10.1007/s10346-016-0721-7

Chigira M (1992) Long-term gravitational deformation of rocks bymass rock creep. Eng Geol 32(3):157–184

Colesanti C, Wasowski J (2004) Satellite SAR interferometry forwide-area slope hazard detection and site-specific monitoring of slow landslides. Proceedings of the ninth International Symposium on Landslides, Rio de Janeiro (Brazil), 28 June– 2 July 2004, pp. 795–802.

Corominas J (1996) The angle of reach as a mobility index for small and large landslides. Can Geotech J 33(2):260–271. https://doi.org/10.1139/t96-005

Corominas J, Moya J (2008) A review of assessing landslide frequency for hazard zoning purposes. Eng Geol 102(3–4):193–213. https://doi.org/10.1016/j.enggeo.2008.03.018

Costa JE, Schuster RL (1988) The formation and failure of natural dams. Geol Soc Am Bull 100(7):1054–1068. https://doi.org/10.1130/0016-7606(1988)100%3c1054:TFAFON%3e2.3.CO;2

Cui S, Pei X, Huang R (2017) Effects of geological and tectonic characteristics on the earthquake-triggered Daguangbao landslide. China Landslides 15(4):649–667. https://doi.org/10.1007/s10346-017-0899-3

Cui S, Wu H, Pei X, Yang Q, Huang R, Guo B (2022) Characterizing the spatial distribution, frequency, geomorphological and geological controls on landslides triggered by the 1933 Mw 7.3 Diexi Earthquake, Sichuan. China Geomorphology 403:108177. https://doi.org/10.1016/j.geomorph.2022.108177

Dai FC, Lee CF, Deng JH, Tham LG (2005) The 1786 earthquake-triggered landslide dam and subsequent dam-break flood on the Dadu River, southwestern China. Geomorphology 65(3–4):205–221. https://doi.org/10.1016/j.geomorph.2004.08.011

Dai L, Fan X, Jansen JD, Xu Q (2021) Landslides and fluvial response to landsliding induced by the 1933 Diexi earthquake, Minjiang River, eastern Tibetan Plateau. Landslides 18(9):3011–3025. https://doi.org/10.1007/s10346-021-01717-2

Delgado F, Zerathe S, Audin L, Schwartz S, Benavente C, Carcaillet J, Bourlès DL, Team A (2020) Giant landslide triggerings and paleoprecipitations in the Central Western Andes: The aricota rockslide dam (South Peru). Geomorphology 350:106932. https://doi.org/10.1016/j.geomorph.2019.106932

Dijkstra T, Chandler J, Wackrow R, Meng X, Ma D, Gibson A, Whitworth M, Foster C, Lee K, Hobbs P (2012) Geomorphic controls and debris flows—the 2010 Zhouqu disaster, China. Proceedings of the 11th International Symposium on Landslides (ISL) and the 2nd North American Symposium on Landslides. CRC press Inc.

Dong JJ, Tung YH, Chen CC, Liao JJ, Pan YW (2009) Discriminant analysis of the geomorphic characteristics and stability of landslide dams. Geomorphology 110(3–4):162–171. https://doi.org/10.1016/j.geomorph.2009.04.004

Dong H, Su X, Dou X, Dong K (2021) Cause for and mechanism of the reactivation of Yahuokou landslide in Zhouqu County. Gansu Province Journal of Lanzhou University (natural Sciences) 57(6):760–766 (in Chinese)

El Hamdouni R, Irigaray C, Fernández T, Chacón J, Keller EA (2008) Assessment of relative active tectonics, southwest border of the Sierra Nevada (southern Spain). Geomorphology 96(1–2):150–173. https://doi.org/10.1016/j.geomorph.2007.08.004

Ermini L, Casagli N (2003) Prediction of the behaviour of landslide dams using a geomorphological dimensionless index. Earth Surf Processes Landforms 28(1):31–47. https://doi.org/10.1002/esp.424

Fan X, van Westen CJ, Korup O, Gorum T, Xu Q, Dai F, Huang R, Wang G (2012) Transient water and sediment storage of the decaying landslide dams induced by the 2008 Wenchuan earthquake, China. Geomorphology 171–172:58–68. https://doi.org/10.1016/j.geomorph.2012.05.003

Fan X, Rossiter DG, van Westen CJ, Xu Q, Görüm T (2014) Empirical prediction of coseismic landslide dam formation. Earth Surf Processes Landforms 39(14):1913–1926. https://doi.org/10.1002/esp.3585

Fan X, Scaringi G, Korup O, West AJ, van Westen CJ, Tanyas H, Hovius N, Hales TC, Jibson RW, Allstadt KE, Zhang L, Evans SG, Xu C, Li G, Pei X, Xu Q, Huang R (2019) Earthquake-induced chains of geologic hazards: Patterns, Mechanisms, and Impacts. Rev Geophys. https://doi.org/10.1029/2018RG000626

Fan X, Dufresne A, Siva Subramanian S, Strom A, Hermanns R, Tacconi Stefanelli C, Hewitt K, Yunus AP, Dunning S, Capra L, Geertsema M, Miller B, Casagli N, Jansen JD, Xu Q (2020) The formation and impact of landslide dams – State of the art. Earth Sci Rev 203:103116. https://doi.org/10.1016/j.earscirev.2020.103116

Fan X, Xu Q, van Westen CJ, Huang R, Tang R (2017) Characteristics and classification of landslide dams associated with the 2008 Wenchuan earthquake. Geoenvironmental Disasters. https://doi.org/10.1186/s40677-017-0079-8

Ferrero AM, Migliazza M, Roncella R, Rabbi E (2010) Rock slopes risk assessment based on advanced geostructural survey techniques. Landslides 8(2):221–231. https://doi.org/10.1007/s10346-010-0246-4

Finnegan NJ, Roe G, Montgomery DR, Hallet B (2005) Controls on the channel width of rivers: Implications for modeling fluvial incision of bedrock. Geology 33(3):229. https://doi.org/10.1130/g21171.1

Guo C, Zhang Y, Li X, Ren S, Yang Z, Wu R, Jin J (2019) Reactivation of giant Jiangdingya ancient landslide in Zhouqu County, Gansu Province. China Landslides 17(1):179–190. https://doi.org/10.1007/s10346-019-01266-9

Guzzetti F, Ardizzone F, Cardinali M, Rossi M, Valigi D (2009) Landslide volumes and landslide mobilization rates in Umbria, central Italy. Earth Planet Sci Lett 279(3–4):222–229. https://doi.org/10.1016/j.epsl.2009.01.005

Article   CAS   Google Scholar  

Hermanns RL, Hewitt K, Strom A, Evans SG, Dunning SA, Scarascia-Mugnozza G (2011) The Classification of rockslide dams. Natural and artificial rockslide dams. Lecture series in earth sciences, Evans SG, Hermanns RL, Strom A. Scarascia Mugnozza G (eds) 133:581–593. https://doi.org/10.1007/978-3-642-04764-0_24

Hou Y, Chigira M, Tsou CY (2014) Numerical study on deep-seated gravitational slope deformation in a shale-dominated dip slope due to river incision. Eng Geol 179:59–75. https://doi.org/10.1016/j.enggeo.2014.06.020

Huang R, Wang Y, Wang S, Li Y (2011) High geo-stress distribution and high geo-stress concentration area models for eastern margin of Qinghai-Tibet plateau. SCIENCE CHINA Technol Sci 54(S1):154–166. https://doi.org/10.1007/s11431-011-4652-1

Huang D, Ma H, Huang R (2022) Deep-seated toppling deformations of rock slopes in western China. Landslides. https://doi.org/10.1007/s10346-021-01829-9

Huggel C, Gruber S, Korup O (2013) Landslide hazards and climate change in high mountains. In: Shroder J, James LA, Harden CP, Clague JJ (eds) Treatise on Geomorphology. Geomorphology of Human Disturbances, Climate Change, and Natural Hazards, vol 13. Academic Press, San Diego, CA, pp 288–301. https://doi.org/10.1016/B978-0-12-374739-6.00367-5

Chapter   Google Scholar  

Hungr O, Leroueil S, Picarelli L (2013) The Varnes classification of landslide types, an update. Landslides 11(2):167–194. https://doi.org/10.1007/s10346-013-0436-y

IAEG (1990) Suggested nomenclature for landslides. Bull Eng Geol Environ 41:13–16

Intrieri E, Frodella W, Raspini F, Bardi F, Tofani V (2020) Using satellite interferometry to infer landslide sliding surface depth and geometry. Remote Sens 12(9):1462. https://doi.org/10.3390/rs12091462

Jin J, Chen G, Meng X, Zhang Y, Shi W, Li Y, Yang Y, Jiang W (2022) Prediction of river damming susceptibility by landslides based on a logistic regression model and InSAR techniques: A case study of the Bailong River Basin. China Eng Geol 299:106562. https://doi.org/10.1016/j.enggeo.2022.106562

Keller EA, Pinter N (2002) Active Tectonics: Earthquakes, Uplift, and Landscape. Second Edition Prentice Hall Earth Science Series, pp 361.

Korup O (2002) Recent research on landslide dams-a literature review with special attention to New Zealand. Prog Phys Geogr 26(2):206–235. https://doi.org/10.1191/0309133302pp333ra

Korup O (2004) Geomorphometric characteristics of New Zealand landslide dams. Eng Geol 73(1–2):13–35. https://doi.org/10.1016/j.enggeo.2003.11.003

Korup O (2005) Geomorphic imprint of landslides on alpine river systems, southwest New Zealand. Earth Surf Processes Landforms 30(7):783–800. https://doi.org/10.1002/esp.1171

Kumar V, Gupta V, Jamir I, Chattoraj SL (2019) Evaluation of potential landslide damming: Case study of Urni landslide, Kinnaur, Satluj valley. India Geosci Front 10(2):753–767. https://doi.org/10.1016/j.gsf.2018.05.004

Kumar V, Jamir I, Gupta V, Bhasin RK (2021) Inferring potential landslide damming using slope stability, geomorphic constraints, and run-out analysis: a case study from the NW Himalaya. Earth Surf Dyn 9(2):351–377. https://doi.org/10.5194/esurf-9-351-2021

Kuo YS, Tsang YC, Chen KT, Shieh CL (2011) Analysis of landslide dam geometries. J Mt Sci 8(4):544–550. https://doi.org/10.1007/s11629-011-2128-1

Larsen IJ, Montgomery DR (2012) Landslide erosion coupled to tectonics and river incision. Nat Geosci 5(7):468–473. https://doi.org/10.1038/ngeo1479

Li M, Zhang L, Ding C, Li W, Luo H, Liao M, Xu Q (2020) Retrieval of historical surface displacements of the Baige landslide from time-series SAR observations for retrospective analysis of the collapse event. Remote Sens Environ 240:111695. https://doi.org/10.1016/j.rse.2020.111695

Liao H, Yang X, Lu G, Tao J, Zhou J (2019) Experimental study on the river blockage and landslide dam formation induced by rock slides. Eng Geol 261:105269. https://doi.org/10.1016/j.enggeo.2019.105269

Ling S, Sun C, Li X, Ren Y, Xu J, Huang T (2020) Characterizing the distribution pattern and geologic and geomorphic controls on earthquake-triggered landslide occurrence during the 2017 Ms 7.0 Jiuzhaigou earthquake, Sichuan. China Landslides 18(4):1275–1291. https://doi.org/10.1007/s10346-020-01549-6

Liu BY (2012) Study on Seismologic Tectonic and Mechanism on Two M8 Historical Earthquakes at the Southeastern Area, Gansu Province (Master thesis). Lanzhou Institute of Seismology, China Earthquake Administration, Lanzhou (in Chinese with English abstract)

Montgomery DR, Brandon MT (2002) Topographic controls on erosion rates in tectonically active mountain ranges. Earth Planet Sci Lett 201(3–4):481–489. https://doi.org/10.1016/S0012-821X(02)00725-2

Nian T, Wu H, Takara K, Li D, Zhang Y (2021) Numerical investigation on the evolution of landslide-induced river blocking using coupled DEM-CFD. Comput Geotech 134:104101. https://doi.org/10.1016/j.compgeo.2021.104101

Nicoletti PG, Sorriso-Valvo M (1991) Geomorphic controls of the shape and mobility of rock avalanches. Geol Soc Am Bull 103(10):1365–1373. https://doi.org/10.1130/0016-7606(1991)103%3c1365:GCOTSA%3e2.3.CO;2

Peng M, Zhang LM (2011) Breaching parameters of landslide dams. Landslides 9(1):13–31. https://doi.org/10.1007/s10346-011-0271-y

Qi T, Meng X, Qing F, Zhao Y, Shi W, Chen G, Zhang Y, Li Y, Yue D, Su X, Guo F, Zeng R, Dijkstra T (2021a) Distribution and characteristics of large landslides in a fault zone: A case study of the NE Qinghai-Tibet Plateau. Geomorphology 379:107592. https://doi.org/10.1016/j.geomorph.2021.107592

Qi T, Zhao Y, Meng X, Shi W, Qing F, Chen G, Zhang Y, Yue D, Guo F (2021b) Distribution Modeling and Factor Correlation Analysis of Landslides in the Large Fault Zone of the Western Qinling Mountains: A Machine Learning Algorithm. Remote Sens 13(24):4990. https://doi.org/10.3390/rs13244990

Qi S, Xu Q, Lan H, Zhang B, Liu J (2010) Spatial distribution analysis of landslides triggered by 2008.5.12 Wenchuan Earthquake. China. Eng Geol 116(1–2):95–108. https://doi.org/10.1016/j.enggeo.2010.07.011

Qin SW, Liu CZ, Li GJ (2006) Neotectonic stress field in Three Gorges region inversed against river system by utilizing the improved GIS method. Global Geology 25(2):160–163 (in Chinese)

Roering J (2012) Landslides limit mountain relief. Nat Geosci 5(7):446–447. https://doi.org/10.1038/ngeo1511

Schäbitz M, Janssen C, Wenk HR, Wirth R, Schuck B, Wetzel HU, Meng X, Dresen G (2018) Microstructures in landslides in northwest China – Implications for creeping displacements? J Struct Geol 106:70–85. https://doi.org/10.1016/j.jsg.2017.11.009

Scheidegger AE (1973) On the prediction of the reach and velocity of catastrophic landslides. Rock Mech Rock Eng 5:231–236. https://doi.org/10.1007/BF01301796

Sharafi S, Ehteshami-Moinabadi M, Darvishi S (2021) Mechanism of landslide damming in Morzarrin Valley and its consequences, Lorestan Province, western Iran. Landslides 18(5):1823–1832. https://doi.org/10.1007/s10346-021-01641-5

Swanson FJ, Oyagi N, Tominaga M (1986) Landslide dam in Japan. In Landslide Dam: Processes Risk and Mitigation, Schuster, R.L., Ed., Geotechnical Special Publication, American Society of Civil Engineers, New York, USA, Volume 3, pp. 131–145.

Tacconi Stefanelli C, Segoni S, Casagli N, Catani F (2016) Geomorphic indexing of landslide dams evolution. Eng Geol 208:1–10. https://doi.org/10.1016/j.enggeo.2016.04.024

Tacconi Stefanelli C, Catani F, Casagli N (2015) Geomorphological investigations on landslide dams. Geoenvironmental Disasters. https://doi.org/10.1186/s40677-015-0030-9

Tang R, Fan X, Scaringi G, Xu Q, van Westen CJ, Ren J, Havenith HB (2018) Distinctive controls on the distribution of river-damming and non-damming landslides induced by the 2008 Wenchuan earthquake. Bull Eng Geol Environ 78(6):4075–4093. https://doi.org/10.1007/s10064-018-1381-8

Tang R, Xu Q, Fan X (2021) Influence of tectonic stress field on formation and evolution of sub-horizontal translational landslides in Sichuan basin. J Eng Geol 29(5):1437–1451 (in Chinese)

Tapponnier P, Zhiqin X, Roger F, Meyer B, Arnaud N, Wittlinger G, Jingsui Y (2001) Oblique stepwise rise and growth of the Tibet plateau. Science 294(5547):1671–1677. https://doi.org/10.1126/science.105978

Tian Y, Xu C, Chen J, Zhou Q, Shen L (2017) Geometrical characteristics of earthquake-induced landslides and correlations with control factors: a case study of the 2013 Minxian, Gansu, China, Mw 5.9 event. Landslides 14(6):1915–1927. https://doi.org/10.1007/s10346-017-0835-6

Tsou CY, Chigira M, Matsushi Y, Chen SC (2014) Fluvial incision history that controlled the distribution of landslides in the Central Range of Taiwan. Geomorphology 226:175–192. https://doi.org/10.1016/j.geomorph.2014.08.015

Tsou CY, Chigira M, Matsushi Y, Hiraishi N, Arai N (2017) Coupling fluvial processes and landslide distribution toward geomorphological hazard assessment: a case study in a transient landscape in Japan. Landslides 14(6):1901–1914. https://doi.org/10.1007/s10346-017-0838-3

Tu G, Deng H (2020) Formation and evolution of a successive landslide dam by the erosion of river: a case study of the Gendakan landslide dam on the Lancang River. China Bull Eng Geol Environ 79(6):2747–2761. https://doi.org/10.1007/s10064-020-01743-9

Tu G, Deng H, Shang Q, Zhang Y, Luo X (2021) Effect of unloading in two directions on the formation of a deep-seated flexural toppling failure. Int J Rock Mech Min Sci 142:104790. https://doi.org/10.1016/j.ijrmms.2021.104790

Turowski JM, Lague D, Hovius N (2009) Response of bedrock channel width to tectonic forcing: Insights from a numerical model, theoretical considerations, and comparison with field data. J Geophys Res. https://doi.org/10.1029/2008jf001133

Varnes DJ (1978) Slope movement types and processes. In: Schuster RL, Krizek RJ (eds) Landslides-Analysis and Control, vol 176. Nat Acad Sci Transport Res Board Spec, pp 11–33

Wang Y, Zhao B, Li J (2017) Mechanism of the catastrophic June 2017 landslide at Xinmo Village, Songping River, Sichuan Province. China Landslides 15(2):333–345. https://doi.org/10.1007/s10346-017-0927-3

Wang X, Clague JJ, Crosta GB, Sun J, Stead D, Qi S, Zhang L (2021) Relationship between the spatial distribution of landslides and rock mass strength, and implications for the driving mechanism of landslides in tectonically active mountain ranges. Eng Geol 292:106281. https://doi.org/10.1016/j.enggeo.2021.106281

Wang H, Cui P, Zhou L, Liu W, Yang A, Yao S, Jiang Z (2022) Spatial and temporal distribution of landslide-dammed lakes in Purlung Tsangpo. Eng Geol 308:106802. https://doi.org/10.1016/j.enggeo.2022.106802

Wu LZ, Zhao DJ, Zhu JD, Peng JB, Zhou Y (2019) A Late Pleistocene river-damming landslide, Minjiang River. China Landslides 17(2):433–444. https://doi.org/10.1007/s10346-019-01305-5

Wu H, Nian TK, Chen GQ, Zhao W, Li DY (2020) Laboratory-scale investigation of the 3-D geometry of landslide dams in a U-shaped valley. Eng Geol 265:105428. https://doi.org/10.1016/j.enggeo.2019.105428

Wu R, Guo C, Ni J, Song D, Zhan W, Zhong N, Yang Z, Li X, Yan Y (2022) Ancient landslide river damming event in the Batang fault zone, Tibetan Plateau. Bull Eng Geol Environ. https://doi.org/10.1007/s10064-022-03048-5

Xu Q, Fan X, Huang R, Westen CV (2009) Landslide dams triggered by the Wenchuan Earthquake, Sichuan Province, south west China. Bull Eng Geol Environ 68(3):373–386. https://doi.org/10.1007/s10064-009-0214-1

Yang H, Chen G, Chong Y, Jin J, Shi W (2021) Quantitative prediction of outburst flood hazard of the Zhouqu “8.8” debris flow-barrier dam in Western China. Water 13(5):639. https://doi.org/10.3390/w13050639

Yu JX, Zheng WJ, Yuan DY, Pang JZ, Liu XW, Liu BY (2012) Late Quaternary active characeristics and slip rate of Pingding-Huama fault, the eastern segment of Guanggaishan-Dieshan fault zone (west Qinling Mountain). Quat Sci 32(5):957–967 (in Chinese)

Zhang Y, Meng X, Chen G, Qiao L, Zeng R, Chang J (2015) Detection of geohazards in the Bailong River Basin using synthetic aperture radar interferometry. Landslides 13(5):1273–1284. https://doi.org/10.1007/s10346-015-0660-8

Zhang Y, Meng X, Jordan C, Novellino A, Dijkstra T, Chen G (2018) Investigating slow-moving landslides in the Zhouqu region of China using InSAR time series. Landslides 15(7):1299–1315. https://doi.org/10.1007/s10346-018-0954-8

Zhang L, Xiao T, He J, Chen C (2019) Erosion-based analysis of breaching of Baige landslide dams on the Jinsha River, China, in 2018. Landslides 16:1965–1979. https://doi.org/10.1007/s10346-019-01247-y

Zhang Y, Meng XM, Dijkstra TA, Jordan CJ, Chen G, Zeng RQ, Novellino A (2020) Forecasting the magnitude of potential landslides based on InSAR techniques. Remote Sens Environ 241:111738. https://doi.org/10.1016/j.rse.2020.111738

Zhang Y, Meng X, Novellino A, Dijkstra T, Chen G, Jordan C, Li Y, Su X (2021) Characterization of pre-failure deformation and evolution of a large earthflow using InSAR monitoring and optical image interpretation. Landslides. https://doi.org/10.1007/s10346-021-01744-z

Zhao S, Chigira M, Wu X (2019) Gigantic rockslides induced by fluvial incision in the Diexi area along the eastern margin of the Tibetan Plateau. Geomorphology 338:27–42. https://doi.org/10.1016/j.geomorph.2019.04.008

Zhao Y, Meng X, Qi T, Chen G, Li Y, Yue D, Qing F (2021) Modeling the spatial distribution of debris flows and analysis of the controlling factors: a machine learning approach. Remote Sens 13(23):4813. https://doi.org/10.3390/rs13234813

Zheng W, Liu X, Yu J, Yuan D, Zhang P, Ge W, Pang J, Liu B (2016) Geometry and late pleistocene slip rates of the liangdang-jiangluo fault in the western qinling mountains, NW China. Tectonophysics 687:1–13. https://doi.org/10.1016/j.tecto.2016.08.021

Zou Q, Cui P, Jiang H, Wang J, Li C, Zhou B (2020) Analysis of regional river blocking by debris flows in response to climate change. Sci Total Environ 741:140262. https://doi.org/10.1016/j.scitotenv.2020.140262

Download references

This study was supported by the Second Tibetan Plateau Scientific Expedition and Research Program (STEP) (Grant No. 2021QZKK0201), National Key Research and Development Program of China (Grant No. 2017YFC1501005), Key Research and Development Program of Gansu Province (Grant No. 20YF8FA074), National Natural Science Foundation of China (Grant No. 42377193 and 42130709), Science and Technology Major Project of Gansu Province (Grant No. 22ZD6FA051), Fundamental Research Funds for the Central Universities (lzujbky-2021-6), and Geohazard prevention project of Gansu Province (Grant No. CNPC-B-FS2021012). We would like to acknowledge Wangcai Liu, Linxin Lin, Weiwei Guo, and Xiaojun Su for their help with the field investigation.

Author information

Authors and affiliations.

School of Earth Sciences, Lanzhou University, Lanzhou, 730000, People’s Republic of China

Guan Chen, Jiacheng Jin, Xingmin Meng, Tianjun Qi & Yunpeng Yang

Technology & Innovation Centre for Environmental Geology and Geohazards Prevention, Lanzhou University, Lanzhou, 730000, People’s Republic of China

Guan Chen, Jiacheng Jin, Xingmin Meng, Tianjun Qi, Wei Shi, Yan Chong & Shiqiang Bian

College of Earth and Environmental Sciences, Lanzhou University, Lanzhou, 730000, People’s Republic of China

Wei Shi, Yan Chong, Yunpeng Yang & Shiqiang Bian

Gansu Geohazards Field Observation and Research Station, Lanzhou University, Lanzhou, 730000, People’s Republic of China

Guan Chen, Jiacheng Jin, Xingmin Meng, Tianjun Qi, Wei Shi, Yan Chong, Yunpeng Yang & Shiqiang Bian

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jiacheng Jin .

Ethics declarations

Competing interest.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Chen, G., Jin, J., Meng, X. et al. Influence of tectonic effects on the formation and characteristics of landslide dams on the NE Tibetan Plateau: a case study in the Bailong River Basin, China. Landslides (2024). https://doi.org/10.1007/s10346-024-02273-1

Download citation

Received : 11 May 2023

Accepted : 11 January 2024

Published : 04 June 2024

DOI : https://doi.org/10.1007/s10346-024-02273-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Landslide dams
• Fluvial incision
• Tectonically active mountainous areas
• Find a journal
• Publish with us
• Track your research