what is qualitative research quality

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Qualitative Research : Definition

Qualitative research is the naturalistic study of social meanings and processes, using interviews, observations, and the analysis of texts and images.  In contrast to quantitative researchers, whose statistical methods enable broad generalizations about populations (for example, comparisons of the percentages of U.S. demographic groups who vote in particular ways), qualitative researchers use in-depth studies of the social world to analyze how and why groups think and act in particular ways (for instance, case studies of the experiences that shape political views).   

Events and Workshops

• Introduction to NVivo Have you just collected your data and wondered what to do next? Come join us for an introductory session on utilizing NVivo to support your analytical process. This session will only cover features of the software and how to import your records. Please feel free to attend any of the following sessions below: April 25th, 2024 12:30 pm - 1:45 pm Green Library - SVA Conference Room 125 May 9th, 2024 12:30 pm - 1:45 pm Green Library - SVA Conference Room 125 May 30th, 2024 12:30 pm - 1:45 pm Green Library - SVA Conference Room 125
• Next: Choose an approach >>
• Choose an approach
• Find studies
• Learn methods
• Get software
• Get data for secondary analysis
• Network with researchers

• Last Updated: Apr 2, 2024 10:41 AM
• URL: https://guides.library.stanford.edu/qualitative_research

Dr Karen Lumsden

trainer / coach / consultant / researcher

Assessing the ‘Quality’ of Qualitative Research

One of the questions that comes up regularly in training courses on qualitative methods is how we should assess the quality of a qualitative study. At some point in their research career, qualitative researchers will inevitably experience the ‘apples versus oranges’ phenomenon, whereby our qualitative research is evaluated based on quantitative principles and criteria, instead of qualitative principles. The quality standards used in quantitative research do not directly translate to qualitative studies.

Should We Use Standardized Criteria to Evaluate Qualitative Research?

Over the years, many qualitative scholars have proposed frameworks and criteria for assessing qualitative research (see Guba and Lincoln 1989; Lather 1993; Schwandt 1996; Bochner 2000; Ritchie et al. 2003; Tracy 2010; Altheide and Johnson 2011). Some have also argued that standardized criteria in are unhelpful in qualitative inquiry (i.e. see Schwandt 1996; Altheide and Johnson 2011). For example, Bochner (2000) argues that ‘traditional empiricist criteria’ are ‘unhelpful’ when applied to new ethnographic approaches (cited in Tracy 2010: 838). As Altheide and Johnson (2011: 582) argue:

“There are many ways to use, practice, promote, and claim qualitative research, and in each there is a proposed or claimed relationship between some field of human experience, a form of representation, and an audience. Researchers and scholars in each of these areas have been grappling with issues of truth, validity, verisimilitude, credibility, trustworthiness, dependability, confirmability, and so on. What is valid for clinical studies or policy studies may not be adequate or relevant for ethnography or autoethnography or performance ethnography.”

Qualitative research is conducted within different research paradigms, which complicates the assessment of the quality of a particular study.

As Tracy (2010) notes, many of these critiques result in the development of new quality standards and criteria for evaluating qualitative inquiry which are seen as more flexible than quantitative standard and of more sensitive to the context bound nature of qualitative research. Below, we explore the main criteria proposed for assessing qualitative research:

Criteria for Assessing Qualitative Research

• Trustworthiness

In the 1980s, Guba and Lincoln (1989 see also Krefting 1991) developed criteria which can be used to determine rigor in a qualitative inquiry. Instead of ‘rigor’, they focus on the development of trustworthiness in qualitative inquiry through determining: credibility, transferability, reliability and confirmability .

• Credibility

Credibility asks us to consider if the research findings are plausible and convincing. Questions to consider include:

• How well does the study capture and portray the world it is trying to describe?
• How well backed up are the claims made by the research?
• What is the evidential base for the research?
• How plausible are the findings?

As Stenfors et al. (2020) point out, there should be alignment between ‘theory, research question, data collection, analysis and results’ while the ‘sampling strategy, the depth and volume of data, and the analytical steps taken’ must be appropriate within that framework.

• Transferability

Here, we are interested in how clear the basis is for drawing wider inference (Ritchie et al. 2003) from our study. Can the findings of our study be transferred to another group, context or setting?

As Ritchie et al. (2003) argue, the findings of qualitative research can be generalized but the framework within which this can occur needs greater clarification. Instead, we refer to the transferability of findings in a qualitative study. For example, in an empirical sense: can findings from qualitative research studies be applied to populations or settings beyond the particular sample of the study? We can also explore the generation of theoretical concepts or propositions which are deemed to be of wider, or universal, application from a qualitative study.

When attempting to extrapolate from a qualitative study we should be conscious that meanings and behaviours are context bound. Therefore extrapolation may be possible if offered as a working hypothesis to help us to make sense of findings in other contexts.

Questions to consider include:

• Sample coverage: did the sample frame contain any known bias; were the criteria used for selection inclusive of the constituencies thought to be of importance?
• Capture of the phenomena: was the environment, quality of questioning effective for participants to fully express their views?
• Identification or labelling: have the phenomena been identified, categorised and named in ways that reflect the meanings assigned by participants?
• Interpretation: is there sufficient internal evidence for the explanatory accounts that have been developed?
• Display: have the findings been portrayed in a way that remains true to the original data and allows others to see the analytic constructions which have occurred? (see Ritchie et al. 2003)
• Dependability

Dependability is ‘the extent to which the research could be replicated in similar conditions’ (Stenfors et al. 2020). The researcher should have provided enough information on the design and conduct of their study that another researcher could follow these and take the same steps in their study. Given the context specific nature of qualitative research, it can be difficult to demonstrate which features of the qualitative data should be expected to be consistent, dependable or reliable.

Questions to consider for reliability include:

• Was the sample design/selection without bias, ‘symbolically’ representative of the target population, comprehensive of all known constituencies; was there any known feature of non-response or attrition within the sample?
• Was the fieldwork carried out consistently, did it allow respondents sufficient opportunities to cover relevant ground, to portray their experiences?
• Was the analysis carried out systematically and comprehensively, were classifications, typologies confirmed by multiple assessment?
• Is the interpretation well supported by the evidence?
• Did the design/conduct allow equal opportunity for all perspectives to be identified or were there features that led to selective, or missing, coverage? (see Ritchie et al. 2003).
• Confirmability

Here, we are looking for a clear link between the data and the findings. For example, researchers should evidence their claims with the use of quotes/excerpts of data. Qualitative researchers should avoid the temptation to quantify findings with claims such as ‘70% of participants felt that xxx…’ It is also important in the Discussion to demonstrate how the research findings relate to the wider body of literature and to answer the research question. Any limitations of the study should also be flagged up.

• Reflexivity

Stenfors et al. (2020) draw attention to reflexivity as another important criteria in assessing qualitative inquiry. For Guba and Lincoln (1989) the reflexive journal is a further means of helping to assess qualitative inquiry. A reflexive approach helps us to be aware of the social, ethical and political impact of our research, the central, fluid and changing nature/s of power relations (with participants, gatekeepers, research funders, etc.) and our relationships with the researched (Lumsden 2019).

We can ask whether the researcher has stepped back and critically reflected on their role in the research process, their relationships with the researched, and their social position? It should be clear how reflexivity has been embedded in the research process (Stenfors et al. 2020). As Altheide and Johnson (2011: 581) write:

‘Good qualitative research—and particularly ethnographies—shows the hand of the ethnographer. The effort may not always be successful, but there should be clear “tracks” that the attempt has been made.’

Additional Criteria: Ethics

Tracy (2010) also provides a useful overview of 8 key criteria for excellent qualitative research: worthy topic, rich rigor, sincerity, credibility, resonance, significant contribution, ethical, meaningful coherence (p.840). There is overlap with the above criteria and some elements could be said to be already subsumed in the above discussion, therefore I will not delve into them all here. However, it is important to draw attention to ethical considerations in qualitative studies. As Tracy notes, the research should consider:

• Procedural ethics (such as human subjects);
• Situational and culturally specific ethics;
• Relational ethics;
• Exiting ethics (leaving the scene and sharing the research) (see Tracy 2010: 840).

Strategies for Determining Trustworthiness (Rigor)

The strategies adopted in order to determine the trustworthiness of a qualitative study depend on a variety of factors including: research paradigm, the specifics of each research design, the research methods utilised (i.e. interviews, ethnography, observation, focus groups, creative methods, visual methods, secondary data analysis, narratives etc.) and the type of qualitative analysis being conducted.

Moore (2015) provides a useful evaluation of the use of various strategies for ensuring rigor in qualitative studies. Strategies which she evaluates as typically used in attempts to ensure validity and reliability include:

• Prolonged engagement in ethnographic research via time spent in the field to reduce researcher effect;
• Prolonged observation in ethnographic research reduces researcher effect;
• Thick description;
• Triangulation;
• Development of a coding system and inter-rater reliability in semi-structured interviews;
• Researcher bias;
• Negative case analysis;
• Peer review debriefing (in team research);
• Member checks;
• External audits (viewed as problematic and not routinely used) (see pages 1217-1220).

She provides a useful evaluation of the appropriateness and success of these strategies for ensuring rigor, for those who wish to explore this further. Interestingly, through her critique of these strategies, Moore also suggests that ‘qualitative researchers return to the terminology of social sciences, using rigor, reliability, validity, and generalizability’ (p.1212) instead of those proposed in the 1980s by Guba and Lincoln (1989).

Awareness of the criteria used when assessing the quality of qualitative research is key for anyone conducting qualitative research. As we have seen these criteria typically include: trustworthiness, credibility, transferability, dependability, confirmability, reflexivity and ethics.

However the strategies which each researcher adopts in order to ensure the trustworthiness (rigor) of their study, will depend on a variety of factors specific to each qualitative research project including the research method they adopt and the research paradigm. As Moore (2019: 1219) writes: ‘…rigor, comprising both validity and reliability, is achieved primarily by researchers in the process of data collection and analysis’. In addition, the assessment criteria which are valid when assessing fields such as clinical studies may not be relevant for those working in areas such as ethnography or narrative studies (see Altheide and Johnson 2011). There is no easy route or ‘one size fits all’ approach for assessing the quality of qualitative research, but the above criteria give us a good starting point which we can refer to when designing and conducting our qualitative inquiries.

References and further reading

Altheide, D.L. and Johnson, J.M. (2011) ‘Reflections on Interpretive Adequacy in Qualitative Research.’ In N.K. Denzin and Y.S. Lincoln (eds) Handbook of Qualitative Research, Fifth Edition (pp. 581-594). London: Sage.

Bochner, A. (2000) ‘Criteria Against Ourselves.’ Qualitative Inquiry , 6: 266-272.

Braun, V. and Clarke, V. (2013) Successful Qualitative Research . London: Sage.

Guba, E. and Lincoln, Y. (1989) Fourth Generation Evaluation . Newbury Park, CA: Sage.

Krefting, L. (1991) ‘Rigor in Qualitative Research: The Assessment of Trustworthiness.’ American Journal of Occupational Therapy , 45: 214–222.

Lather, P. (1993) ‘Fertile Obsession: Validity after Poststructuralism.’ Sociological Quarterly , 34: 673-693.

Lingard L. (2015) ‘Joining a Conversation: The Problem/Gap/Hook Heuristic.’ Perspectives on Medical Education , 4(5): 252–253.

Lumsden, K. (2019) Reflexivity: Theory, Method and Practice . London: Routledge.

Morse, J.M. (2015) ‘Critical Analysis of Strategies for Determining Rigor in Qualitative Inquiry.’ Qualitative Health Research , 25(9): 1212-1222.

Schwandt, T.A. (1996) ‘Farewell to Criteriology.’ Qualitative Inquiry , 2: 58-72.

Spencer, L., Ritchie, J., Lewis, J., and Dillon, L. (2003) Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence , GCSRO.  Available at: www.policyhub.gov.uk/publications

Stenfors, T., Kajamaa, A. and Bennett, D. (2020) ‘How to… Assess the Quality of Qualitative Research.’ The Clinical Teacher , https://doi.org/10.1111/tct.13242

Tracy, S.J. (2010) ‘Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research.’ Qualitative Inquiry , 16: 837–851.

Share this:

Published by Dr Karen Lumsden

trainer / coach / consultant / researcher I am a social scientist with expertise in qualitative research methods. I have a passion for delivering qualitative methods training and coaching to clients. I am also Assistant Professor in Criminology at the University of Nottingham, UK and have experience of delivering qualitative methods training via the Social Research Association and to various universities and organisations. View all posts by Dr Karen Lumsden

Leave a Reply Cancel reply

Discover more from dr karen lumsden.

Subscribe now to keep reading and get access to the full archive.

Type your email…

Continue reading

• Reviews / Why join our community?
• For companies
• Frequently asked questions

Qualitative Research

What is qualitative research.

Qualitative research is the methodology researchers use to gain deep contextual understandings of users via non-numerical means and direct observations. Researchers focus on smaller user samples—e.g., in interviews—to reveal data such as user attitudes, behaviors and hidden factors: insights which guide better designs.

“ There are also unknown unknowns, things we don’t know we don’t know.” — Donald Rumsfeld, Former U.S. Secretary of Defense

• Transcript loading…

See how you can use qualitative research to expose hidden truths about users and iteratively shape better products.

Qualitative Research Focuses on the “Why”

Qualitative research is a subset of user experience (UX) research and user research . By doing qualitative research, you aim to gain narrowly focused but rich information about why users feel and think the ways they do. Unlike its more statistics-oriented “counterpart”, quantitative research , qualitative research can help expose hidden truths about your users’ motivations, hopes, needs, pain points and more to help you keep your project’s focus on track throughout development. UX design professionals do qualitative research typically from early on in projects because—since the insights they reveal can alter product development dramatically—they can prevent costly design errors from arising later. Compare and contrast qualitative with quantitative research here:

Qualitative research

Quantitative Research

You Aim to Determine

The “why” – to get behind how users approach their problems in their world

The “what”, “where” & “when” of the users’ needs & problems – to help keep your project’s focus on track during development

Loosely structured (e.g., contextual inquiries) – to learn why users behave how they do & explore their opinions

Highly structured (e.g., surveys) – to gather data about what users do & find patterns in large user groups

Number of Representative Users

Often around 5

Ideally 30+

Level of Contact with Users

More direct & less remote (e.g., usability testing to examine users’ stress levels when they use your design)

Less direct & more remote (e.g., analytics)

Statistically

You need to take great care with handling non-numerical data (e.g., opinions), as your own opinions might influence findings

Reliable – given enough test users

Regarding care with opinions, it’s easy to be subjective about qualitative data, which isn’t as comprehensively analyzable as quantitative data. That’s why design teams also apply quantitative research methods, to reinforce the “why” with the “what”.

Qualitative Research Methods You Can Use to Get Behind Your Users

You have a choice of many methods to help gain the clearest insights into your users’ world – which you might want to complement with quantitative research methods. In iterative processes such as user-centered design , you/your design team would use quantitative research to spot design problems, discover the reasons for these with qualitative research, make changes and then test your improved design on users again. The best method/s to pick will depend on the stage of your project and your objectives. Here are some:

Diary studies – You ask users to document their activities, interactions, etc. over a defined period. This empowers users to deliver context-rich information. Although such studies can be subjective—since users will inevitably be influenced by in-the-moment human issues and their emotions—they’re helpful tools to access generally authentic information.

Structured – You ask users specific questions and analyze their responses with other users’.

Semi-structured – You have a more free-flowing conversation with users, but still follow a prepared script loosely.

Ethnographic – You interview users in their own environment to appreciate how they perform tasks and view aspects of tasks.

Usability testing

Moderated – In-person testing in, e.g., a lab.

Unmoderated – Users complete tests remotely: e.g., through a video call.

Guerrilla – “Down-the-hall”/“down-and-dirty” testing on a small group of random users or colleagues.

User observation – You watch users get to grips with your design and note their actions, words and reactions as they attempt to perform tasks.

Qualitative research can be more or less structured depending on the method.

Qualitative Research – How to Get Reliable Results

Some helpful points to remember are:

Participants – Select a number of test users carefully (typically around 5). Observe the finer points such as body language. Remember the difference between what they do and what they say they do.

Moderated vs. unmoderated – You can obtain the richest data from moderated studies, but these can involve considerable time and practice. You can usually conduct unmoderated studies more quickly and cheaply, but you should plan these carefully to ensure instructions are clear, etc.

Types of questions – You’ll learn far more by asking open-ended questions. Avoid leading users’ answers – ask about their experience during, say, the “search for deals” process rather than how easy it was. Try to frame questions so users respond honestly: i.e., so they don’t withhold grievances about their experience because they don’t want to seem impolite. Distorted feedback may also arise in guerrilla testing, as test users may be reluctant to sound negative or to discuss fine details if they lack time.

Location – Think how where users are might affect their performance and responses. If, for example, users’ tasks involve running or traveling on a train, select the appropriate method (e.g., diary studies for them to record aspects of their experience in the environment of a train carriage and the many factors impacting it).

Overall, no single research method can help you answer all your questions. Nevertheless, The Nielsen Norman Group advise that if you only conduct one kind of user research, you should pick qualitative usability testing, since a small sample size can yield many cost- and project-saving insights. Always treat users and their data ethically. Finally, remember the importance of complementing qualitative methods with quantitative ones: You gain insights from the former; you test those using the latter.

Learn More about Qualitative Research

Take our course on User Research to see how to get the most from qualitative research.

Read about the numerous considerations for qualitative research in this in-depth piece.

This blog discusses the importance of qualitative research , with tips.

Explore additional insights into qualitative research here .

Literature on Qualitative Research

Here’s the entire UX literature on Qualitative Research by the Interaction Design Foundation, collated in one place:

Learn more about Qualitative Research

Take a deep dive into Qualitative Research with our course User Research – Methods and Best Practices .

How do you plan to design a product or service that your users will love , if you don't know what they want in the first place? As a user experience designer, you shouldn't leave it to chance to design something outstanding; you should make the effort to understand your users and build on that knowledge from the outset. User research is the way to do this, and it can therefore be thought of as the largest part of user experience design .

In fact, user research is often the first step of a UX design process—after all, you cannot begin to design a product or service without first understanding what your users want! As you gain the skills required, and learn about the best practices in user research, you’ll get first-hand knowledge of your users and be able to design the optimal product—one that’s truly relevant for your users and, subsequently, outperforms your competitors’ .

This course will give you insights into the most essential qualitative research methods around and will teach you how to put them into practice in your design work. You’ll also have the opportunity to embark on three practical projects where you can apply what you’ve learned to carry out user research in the real world . You’ll learn details about how to plan user research projects and fit them into your own work processes in a way that maximizes the impact your research can have on your designs. On top of that, you’ll gain practice with different methods that will help you analyze the results of your research and communicate your findings to your clients and stakeholders—workshops, user journeys and personas, just to name a few!

By the end of the course, you’ll have not only a Course Certificate but also three case studies to add to your portfolio. And remember, a portfolio with engaging case studies is invaluable if you are looking to break into a career in UX design or user research!

We believe you should learn from the best, so we’ve gathered a team of experts to help teach this course alongside our own course instructors. That means you’ll meet a new instructor in each of the lessons on research methods who is an expert in their field—we hope you enjoy what they have in store for you!

All open-source articles on Qualitative Research

How to do a thematic analysis of user interviews.

• 1.2k shares
• 3 years ago

How to Visualize Your Qualitative User Research Results for Maximum Impact

• 2 years ago

Creating Personas from User Research Results

Best Practices for Qualitative User Research

Card Sorting

Contextual Interviews and How to Handle Them

Understand the User’s Perspective through Research for Mobile UX

• 10 mths ago

Ethnography

7 simple ways to get better results from ethnographic research.

Semi-structured qualitative studies

Pros and cons of conducting user interviews.

Workshops to Establish Empathy and Understanding from User Research Results

How to Moderate User Interviews

• 4 years ago

Question Everything

Adding Quality to Your Design Research with an SSQS Checklist

• 8 years ago

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this page , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share the knowledge!

Share this content on:

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this page.

New to UX Design? We’re Giving You a Free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

Qualitative vs Quantitative Research Methods & Data Analysis

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

What is the difference between quantitative and qualitative?

The main difference between quantitative and qualitative research is the type of data they collect and analyze.

Quantitative research collects numerical data and analyzes it using statistical methods. The aim is to produce objective, empirical data that can be measured and expressed in numerical terms. Quantitative research is often used to test hypotheses, identify patterns, and make predictions.

Qualitative research , on the other hand, collects non-numerical data such as words, images, and sounds. The focus is on exploring subjective experiences, opinions, and attitudes, often through observation and interviews.

Qualitative research aims to produce rich and detailed descriptions of the phenomenon being studied, and to uncover new insights and meanings.

Quantitative data is information about quantities, and therefore numbers, and qualitative data is descriptive, and regards phenomenon which can be observed but not measured, such as language.

What Is Qualitative Research?

Qualitative research is the process of collecting, analyzing, and interpreting non-numerical data, such as language. Qualitative research can be used to understand how an individual subjectively perceives and gives meaning to their social reality.

Qualitative data is non-numerical data, such as text, video, photographs, or audio recordings. This type of data can be collected using diary accounts or in-depth interviews and analyzed using grounded theory or thematic analysis.

Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Denzin and Lincoln (1994, p. 2)

Interest in qualitative data came about as the result of the dissatisfaction of some psychologists (e.g., Carl Rogers) with the scientific study of psychologists such as behaviorists (e.g., Skinner ).

Since psychologists study people, the traditional approach to science is not seen as an appropriate way of carrying out research since it fails to capture the totality of human experience and the essence of being human.  Exploring participants’ experiences is known as a phenomenological approach (re: Humanism ).

Qualitative research is primarily concerned with meaning, subjectivity, and lived experience. The goal is to understand the quality and texture of people’s experiences, how they make sense of them, and the implications for their lives.

Qualitative research aims to understand the social reality of individuals, groups, and cultures as nearly as possible as participants feel or live it. Thus, people and groups are studied in their natural setting.

Some examples of qualitative research questions are provided, such as what an experience feels like, how people talk about something, how they make sense of an experience, and how events unfold for people.

Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon, or behavior, operates as it does in a particular context. It can be used to generate hypotheses and theories from the data.

Qualitative Methods

There are different types of qualitative research methods, including diary accounts, in-depth interviews , documents, focus groups , case study research , and ethnography.

The results of qualitative methods provide a deep understanding of how people perceive their social realities and in consequence, how they act within the social world.

The researcher has several methods for collecting empirical materials, ranging from the interview to direct observation, to the analysis of artifacts, documents, and cultural records, to the use of visual materials or personal experience. Denzin and Lincoln (1994, p. 14)

Here are some examples of qualitative data:

Interview transcripts : Verbatim records of what participants said during an interview or focus group. They allow researchers to identify common themes and patterns, and draw conclusions based on the data. Interview transcripts can also be useful in providing direct quotes and examples to support research findings.

Observations : The researcher typically takes detailed notes on what they observe, including any contextual information, nonverbal cues, or other relevant details. The resulting observational data can be analyzed to gain insights into social phenomena, such as human behavior, social interactions, and cultural practices.

Unstructured interviews : generate qualitative data through the use of open questions.  This allows the respondent to talk in some depth, choosing their own words.  This helps the researcher develop a real sense of a person’s understanding of a situation.

Diaries or journals : Written accounts of personal experiences or reflections.

Notice that qualitative data could be much more than just words or text. Photographs, videos, sound recordings, and so on, can be considered qualitative data. Visual data can be used to understand behaviors, environments, and social interactions.

Qualitative Data Analysis

Qualitative research is endlessly creative and interpretive. The researcher does not just leave the field with mountains of empirical data and then easily write up his or her findings.

Qualitative interpretations are constructed, and various techniques can be used to make sense of the data, such as content analysis, grounded theory (Glaser & Strauss, 1967), thematic analysis (Braun & Clarke, 2006), or discourse analysis.

For example, thematic analysis is a qualitative approach that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded.

Key Features

• Events can be understood adequately only if they are seen in context. Therefore, a qualitative researcher immerses her/himself in the field, in natural surroundings. The contexts of inquiry are not contrived; they are natural. Nothing is predefined or taken for granted.
• Qualitative researchers want those who are studied to speak for themselves, to provide their perspectives in words and other actions. Therefore, qualitative research is an interactive process in which the persons studied teach the researcher about their lives.
• The qualitative researcher is an integral part of the data; without the active participation of the researcher, no data exists.
• The study’s design evolves during the research and can be adjusted or changed as it progresses. For the qualitative researcher, there is no single reality. It is subjective and exists only in reference to the observer.
• The theory is data-driven and emerges as part of the research process, evolving from the data as they are collected.

Limitations of Qualitative Research

• Because of the time and costs involved, qualitative designs do not generally draw samples from large-scale data sets.
• The problem of adequate validity or reliability is a major criticism. Because of the subjective nature of qualitative data and its origin in single contexts, it is difficult to apply conventional standards of reliability and validity. For example, because of the central role played by the researcher in the generation of data, it is not possible to replicate qualitative studies.
• Also, contexts, situations, events, conditions, and interactions cannot be replicated to any extent, nor can generalizations be made to a wider context than the one studied with confidence.
• The time required for data collection, analysis, and interpretation is lengthy. Analysis of qualitative data is difficult, and expert knowledge of an area is necessary to interpret qualitative data. Great care must be taken when doing so, for example, looking for mental illness symptoms.

Advantages of Qualitative Research

• Because of close researcher involvement, the researcher gains an insider’s view of the field. This allows the researcher to find issues that are often missed (such as subtleties and complexities) by the scientific, more positivistic inquiries.
• Qualitative descriptions can be important in suggesting possible relationships, causes, effects, and dynamic processes.
• Qualitative analysis allows for ambiguities/contradictions in the data, which reflect social reality (Denscombe, 2010).
• Qualitative research uses a descriptive, narrative style; this research might be of particular benefit to the practitioner as she or he could turn to qualitative reports to examine forms of knowledge that might otherwise be unavailable, thereby gaining new insight.

What Is Quantitative Research?

Quantitative research involves the process of objectively collecting and analyzing numerical data to describe, predict, or control variables of interest.

The goals of quantitative research are to test causal relationships between variables , make predictions, and generalize results to wider populations.

Quantitative researchers aim to establish general laws of behavior and phenomenon across different settings/contexts. Research is used to test a theory and ultimately support or reject it.

Quantitative Methods

Experiments typically yield quantitative data, as they are concerned with measuring things.  However, other research methods, such as controlled observations and questionnaires , can produce both quantitative information.

For example, a rating scale or closed questions on a questionnaire would generate quantitative data as these produce either numerical data or data that can be put into categories (e.g., “yes,” “no” answers).

Experimental methods limit how research participants react to and express appropriate social behavior.

Findings are, therefore, likely to be context-bound and simply a reflection of the assumptions that the researcher brings to the investigation.

There are numerous examples of quantitative data in psychological research, including mental health. Here are a few examples:

Another example is the Experience in Close Relationships Scale (ECR), a self-report questionnaire widely used to assess adult attachment styles .

The ECR provides quantitative data that can be used to assess attachment styles and predict relationship outcomes.

Neuroimaging data : Neuroimaging techniques, such as MRI and fMRI, provide quantitative data on brain structure and function.

This data can be analyzed to identify brain regions involved in specific mental processes or disorders.

For example, the Beck Depression Inventory (BDI) is a clinician-administered questionnaire widely used to assess the severity of depressive symptoms in individuals.

The BDI consists of 21 questions, each scored on a scale of 0 to 3, with higher scores indicating more severe depressive symptoms. 

Quantitative Data Analysis

Statistics help us turn quantitative data into useful information to help with decision-making. We can use statistics to summarize our data, describing patterns, relationships, and connections. Statistics can be descriptive or inferential.

Descriptive statistics help us to summarize our data. In contrast, inferential statistics are used to identify statistically significant differences between groups of data (such as intervention and control groups in a randomized control study).

• Quantitative researchers try to control extraneous variables by conducting their studies in the lab.
• The research aims for objectivity (i.e., without bias) and is separated from the data.
• The design of the study is determined before it begins.
• For the quantitative researcher, the reality is objective, exists separately from the researcher, and can be seen by anyone.
• Research is used to test a theory and ultimately support or reject it.

Limitations of Quantitative Research

• Context: Quantitative experiments do not take place in natural settings. In addition, they do not allow participants to explain their choices or the meaning of the questions they may have for those participants (Carr, 1994).
• Researcher expertise: Poor knowledge of the application of statistical analysis may negatively affect analysis and subsequent interpretation (Black, 1999).
• Variability of data quantity: Large sample sizes are needed for more accurate analysis. Small-scale quantitative studies may be less reliable because of the low quantity of data (Denscombe, 2010). This also affects the ability to generalize study findings to wider populations.
• Confirmation bias: The researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on the theory of hypothesis generation.

Advantages of Quantitative Research

• Scientific objectivity: Quantitative data can be interpreted with statistical analysis, and since statistics are based on the principles of mathematics, the quantitative approach is viewed as scientifically objective and rational (Carr, 1994; Denscombe, 2010).
• Useful for testing and validating already constructed theories.
• Rapid analysis: Sophisticated software removes much of the need for prolonged data analysis, especially with large volumes of data involved (Antonius, 2003).
• Replication: Quantitative data is based on measured values and can be checked by others because numerical data is less open to ambiguities of interpretation.
• Hypotheses can also be tested because of statistical analysis (Antonius, 2003).

Antonius, R. (2003). Interpreting quantitative data with SPSS . Sage.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics . Sage.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology . Qualitative Research in Psychology , 3, 77–101.

Carr, L. T. (1994). The strengths and weaknesses of quantitative and qualitative research : what method for nursing? Journal of advanced nursing, 20(4) , 716-721.

Denscombe, M. (2010). The Good Research Guide: for small-scale social research. McGraw Hill.

Denzin, N., & Lincoln. Y. (1994). Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications Inc.

Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing research, 17(4) , 364.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Punch, K. (1998). Introduction to Social Research: Quantitative and Qualitative Approaches. London: Sage

Further Information

• Designing qualitative research
• Methods of data collection and analysis
• Introduction to quantitative and qualitative research
• Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
• Qualitative research in health care: Analysing qualitative data
• Qualitative data analysis: the framework approach
• Using the framework method for the analysis of
• Qualitative data in multi-disciplinary health research
• Content Analysis
• Grounded Theory
• Thematic Analysis

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved April 16, 2024, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, what is your plagiarism score.

AI in Qualitative Research: What have we learned, and where do we go from here?

The growth of AI in research

Over the course of just one year, artificial intelligence (AI) tools have been vastly improved to be more powerful, flexible, and accessible to researchers around the world. This has opened exciting possibilities as well as important questions regarding how AI can be responsibly and usefully utilized.

ATLAS.ti has been at the forefront of creating innovative AI tools, embarking on a journey to bring AI to qualitative researchers everywhere. Through this journey, we have learned a great deal about addressing researchers’ most pressing concerns and supporting their core needs. ATLAS.ti serves as a cockpit for researchers, offering a comprehensive overview and control, ensuring that researchers can always check the intentions guiding AI analyses, review and edit any coded results, and instantly see any data quotation in its original document. This ability to check the exact data and full context behind any AI analysis results is crucial.

In this article, we reflect on the past year of AI in qualitative research to take stock of what we’ve learned and share key insights on how researchers can harness AI for rigorous research, with ATLAS.ti acting as their command center.

AI tools for qualitative researchers

There are various AI tools integrated into ATLAS.ti that cannot be found elsewhere, such as Intentional AI Coding and Conversational AI. With Intentional AI Coding, researchers can guide how the AI will code their data by specifying their research questions and providing key contextual information. With Conversational AI, researchers freely ask any questions about their data and receive answers in a natural language chat. As always, researchers can instantly access the exact data and context supporting the AI results. Let Conversational AI become your assistant with ATLAS.ti! Imagine having a friendly chat with your documents and effortlessly extracting key insights through natural conversation.

ATLAS.ti was the first qualitative research software to offer AI-driven coding, and we have been developing and integrating AI tools for over a year now. The development of these tools was not a simple process. Before they became what they are today, hundreds of research intentions were transformed into numerous prompts and sent individually to the AI. Each response was checked for its quality and appropriateness, and the prompt was then adjusted and sent again to closely related areas of the AI's n-dimensional "brain". This process was repeated until we achieved optimal results. We then implemented this in the software to obtain the best outcome with every query, leading to the precise and unique analytical possibilities that ATLAS.ti offers today.

Researchers can also summarize any text (e.g., all quotations associated with one or several codes) and receive AI-driven code suggestions when highlighting any segment of text (i.e., creating a quotation) in ATLAS.ti. In fact, generating AI summaries and code suggestions is already widely available and accessible with the basic resources of ChatGPT. While other qualitative research software has also implemented AI summaries and code suggestions, they have done little more than add these low hanging fruits of AI into their software.

As already mentioned above, in contrast, the ATLAS.ti AI Lab has carefully crafted a whole host of AI-driven tools that extend far beyond the default capabilities of ChatGPT by pushing ourselves to meet the challenges posed by qualitative research. We have achieved the development of both general and Intentional AI coding, including the fine-tuning of AI through categories and Conversational AI, as well as the implementation of AI data residency regions. This progress is attributed to our state-of-the-art architecture and the substantial efforts and expertise of our developer-researchers. They possess a deep understanding of and dedication to fulfilling research needs and are continuously enhancing the performance of our AI algorithms. At ATLAS.ti, our team’s unwavering commitment and spirit fuel our mission to deliver software that significantly benefits every user and thus makes a significant difference. And more is to come.

As ATLAS.ti has been continuously innovating AI tools for qualitative research, the time is ripe to take the next steps in developing AI tools that can support rigorous research. Our extensive experience with developing AI tools and rich conversations with users have taught us that AI can help researchers engage with their data. Still, a key concern for ensuring rigor and precise individual analysis tailored to the researcher is maintaining control over the process and having transparency in how the AI analyzes the data. The foundational functionality of ATLAS.ti, combined with Conversational AI and Intentional AI Coding, exemplifies this focus. Integrating these AI tools into ATLAS.ti enables researchers to unpack the black box of AI.

Giving researchers access to the data behind any AI results

A core benefit of qualitative research software is that it maintains fluid connections between all the parts of a research project. Researchers can add codes in their documents, write notes in memos, visualize connections in networks, and more. In ATLAS.ti, all entities are dynamically interrelated and accessible to researchers. Importantly, no matter where in their project researchers are working, the data behind any part of the analysis can quickly be accessed. All data always remains embedded in its context: By simply double-clicking on a data quotation, researchers can open the original document.

Thus, a key element of ATLAS.ti is access to the data behind the AI results because every entity is an own object in ATLAS.ti, which ensures a seamless connection between the individual parts of a research project. Every data quotation and every code is directly accessible and can be contextualized in the original document. This functionality in ATLAS.ti was originally programmed by Thomas Muhr, the inventor and owner of the software, and this allows researchers to view each element – whether it's a code, a quotation, or a memo – in its original context, exactly where it was created.

This interconnectedness makes it easy for researchers to access and review any results from AI analyses. Researchers can edit any of the results, including combining codes and updating code categories. The results can also be saved in different copies or outputs so that researchers can explore various paths in their data.

Integrating AI-driven tools into ATLAS.ti allows researchers to open the black box of AI. Instead of wondering where AI results come from, researchers can easily see all the data and context behind any codings. ATLAS.ti thus places researchers firmly in the cockpit of their research, with a clear view of everything happening in their analysis and full control over the journey of their research.

Ensuring data residency and privacy with AI Regions

As AI capabilities become more powerful, data privacy and residency considerations are increasingly important for researchers. At ATLAS.ti, we understand these concerns, which is why we offer users and multi-user license administrators the option to disable AI features: With one switch of a button, any user or license admin can deactivate OpenAI integration across all AI features in ATLAS.ti – ensuring no data gets submitted externally. We have also pioneered a solution to give users full control over their data. With our new AI Regions option for ATLAS.ti Desktop, researchers can choose where their data is processed for AI operations. Our AI Data Residency Program enables the selection of either the United States or Europe as the preferred region.

This empowers researchers to easily control where their personal research data is managed and processed for AI features. Their data remains protected in the chosen region with advanced encryption for both data in transit and at rest. Moreover, optimized server proximity in the selected region ensures faster response times during AI processing. By giving researchers the ability to designate a geographic jurisdiction for their AI data handling, we provide an additional layer of privacy assurance. This way, the full power of ATLAS.ti's cutting-edge AI capabilities can be leveraged while data residency is secured according to the researcher's preference.

Focusing on making AI analyses understandable to researchers

We are continuing to focus on making each aspect of AI-based analyses understandable to researchers because this is essential to ensuring researchers stay in full control. ATLAS.ti is a dutiful assistant that researchers can guide and supervise to ensure the analysis is being carried out correctly and rigorously.

AI features are embedded in the entire world of unique functionalities in ATLAS.ti, and coupled with ATLAS.ti's interconnected objects and intuitive interface, researchers can unpack any part of the analysis. Codes and data quotations can be visually displayed in networks to draw out the overarching picture of the study, and any quotation can be simply clicked on to view the data in its original document. Many other AI-driven tools can also facilitate focused analyses, such as automatically coding for sentiments, entities, concepts, and more. As always, all data quotations are displayed together with an easy-to-view margin area that shows which codes were attached, and these margin areas are interactive so that researchers can continue to code their data and edit their codes on the go. In other words, it is always easy to see the origins of a code, quotation, or memo, the meanings and interpretations captured in these entities, and the context from which they emerged. Researchers can thus form, control, and review every aspect of the analyses carried out by AI.

By giving researchers the possibility to double-check the insights that have been generated by AI, each person can get a better and better feeling over time about how good the insights created by AI are. This will then lead to a deeper understanding of the results created by AI, which will help researchers with large amounts of data, as they can then just review samples of the AI analysis to assess the quality of the AI-generated insights. In the end, when writing up the analysis report and when the data has already been filtered and condensed by the AI analysis, the researcher can, of course, double-check and edit all the insights found by AI to create their individual report. Importantly, though, this entire process is streamlined and sped up by AI but always steered by the researcher and any user of ATLAS.ti, such as the growing number of students and people we have seen who are simply seeking to gain insights about a broad range of text documents.

Moving forward in dialogue with researchers

We believe that making AI results understandable is an important next step that research software needs to take to support researchers’ efforts to conduct rigorous research. AI has been developing so quickly that we see it in a range of research tools today, such as literature search platforms, writing programs, and visual display software. For AI to truly support researchers as a dutiful assistant, we call for all AI-driven tools to make the analyses and results understandable and traceable for their users. Researchers need to be able to see the data or sources on which any results are based and be able to make any adjustments to the inputs and outputs of AI. In this way, researchers can guide how AI works through their data and review any aspects of the resulting analysis.

Our mission is to help make qualitative research easier for everyone, everywhere. This is why we have long championed project exchange capabilities, allowing researchers to seamlessly move their projects between ATLAS.ti Windows, Mac, and Web. ATLAS.ti remains the only qualitative research software that provides this level of freedom for researchers to work on their projects wherever they prefer. In addition, ATLAS.ti ensures that licenses are used with utmost efficiency: A single license gives access to ATLAS.ti Windows, Mac, and Web, and license admins can invite as many other people as they wish to their license. There is no limit to the number of installations. Rather, the ATLAS.ti license systems simply keeps track of who is currently using the software, so if one person logs out, their seat in the license is automatically freed up and available for anyone else to log in and use the software.

We also place great importance in always being there for our users, which is why we offer support 24 hours a day, 5 days a week, via live chat, telephone, and email. Other qualitative research software tend to offer email support or premium support options in specific countries, but all researchers using ATLAS.ti can always call or chat with us, at no extra cost, to receive help from experts in ATLAS.ti and qualitative research. We strive to help everyone with any types of questions they have, going beyond giving basic instructions found in manuals to offering tailored advice on how ATLAS.ti can be harnessed according to each person's research questions and methodology.

We are passionate about qualitative research and helping others succeed in their research. We are always looking for ways to meet qualitative researchers' needs, which is why we offer all these things and more to our users. ATLAS.ti is the only qualitative research software to offer flexible access to a broad range of powerful tools along with sophisticated license management and free user support. We are grateful to be in a position to research and develop powerful tools for qualitative research and engage in dialogue with qualitative researchers all around the world. This has taught us that being able to understand where AI results come from is crucial for rigorous research. Our goal at ATLAS.ti is thus to streamline the analysis process by bringing AI to researchers in a way that they can fully understand the origins of all AI results. As I said in my interview nearly a year ago, this is just the beginning! We warmly invite everyone to join ATLAS.ti at the forefront of the newest technologies to help you get better, more tailored, and faster insights for your research.

Ready to try out ATLAS.ti?

Intuitive AI tools to help you with your research projects. Check them out with a free trial of ATLAS.ti.

• Open access
• Published: 10 April 2024

“So at least now I know how to deal with things myself, what I can do if it gets really bad again”—experiences with a long-term cross-sectoral advocacy care and case management for severe multiple sclerosis: a qualitative study

• Anne Müller   ORCID: orcid.org/0000-0002-2456-2492 1 ,
• Fabian Hebben   ORCID: orcid.org/0009-0003-6401-3433 1 ,
• Kim Dillen 1 ,
• Veronika Dunkl 1 ,
• Yasemin Goereci 2 ,
• Raymond Voltz 1 , 3 , 4 ,
• Peter Löcherbach 5 ,
• Clemens Warnke   ORCID: orcid.org/0000-0002-3510-9255 2 &
• Heidrun Golla   ORCID: orcid.org/0000-0002-4403-630X 1

on behalf of the COCOS-MS trial group represented by Martin Hellmich

BMC Health Services Research volume  24 , Article number:  453 ( 2024 ) Cite this article

121 Accesses

Metrics details

Persons with severe Multiple Sclerosis (PwsMS) face complex needs and daily limitations that make it challenging to receive optimal care. The implementation and coordination of health care, social services, and support in financial affairs can be particularly time consuming and burdensome for both PwsMS and caregivers. Care and case management (CCM) helps ensure optimal individual care as well as care at a higher-level. The goal of the current qualitative study was to determine the experiences of PwsMS, caregivers and health care specialists (HCSs) with the CCM.

In the current qualitative sub study, as part of a larger trial, in-depth semi-structured interviews with PwsMS, caregivers and HCSs who had been in contact with the CCM were conducted between 02/2022 and 01/2023. Data was transcribed, pseudonymized, tested for saturation and analyzed using structuring content analysis according to Kuckartz. Sociodemographic and interview characteristics were analyzed descriptively.

Thirteen PwsMS, 12 caregivers and 10 HCSs completed interviews. Main categories of CCM functions were derived deductively: (1) gatekeeper function, (2) broker function, (3) advocacy function, (4) outlook on CCM in standard care. Subcategories were then derived inductively from the interview material. 852 segments were coded. Participants appreciated the CCM as a continuous and objective contact person, a person of trust (92 codes), a competent source of information and advice (on MS) (68 codes) and comprehensive cross-insurance support (128 codes), relieving and supporting PwsMS, their caregivers and HCSs (67 codes).

Conclusions

Through the cross-sectoral continuous support in health-related, social, financial and everyday bureaucratic matters, the CCM provides comprehensive and overriding support and relief for PwsMS, caregivers and HCSs. This intervention bears the potential to be fine-tuned and applied to similar complex patient groups.

Trial registration

The study was approved by the Ethics Committee of the University of Cologne (#20–1436), registered at the German Register for Clinical Studies (DRKS00022771) and in accordance with the Declaration of Helsinki.

Peer Review reports

Introduction

Multiple sclerosis (MS) is the most frequent and incurable chronic inflammatory and degenerative disease of the central nervous system (CNS). Illness awareness and the number of specialized MS clinics have increased since the 1990s, paralleled by the increased availability of disease-modifying therapies [ 1 ]. There are attempts in the literature for the definition of severe MS [ 2 , 3 ]. These include a high EDSS (Expanded disability Status Scale [ 4 ]) of ≥ 6, which we took into account in our study. There are also other factors to consider, such as a highly active disease course with complex therapies that are associated with side effects. These persons are (still) less disabled, but may feel overwhelmed with regard to therapy, side effects and risk monitoring of therapies [ 5 , 6 ].

Persons with severe MS (PwsMS) develop individual disease trajectories marked by a spectrum of heterogeneous symptoms, functional limitations, and uncertainties [ 7 , 8 ] manifesting individually and unpredictably [ 9 ]. This variability can lead to irreversible physical and mental impairment culminating in complex needs and daily challenges, particularly for those with progressive and severe MS [ 5 , 10 , 11 ]. Such challenges span the spectrum from reorganizing biographical continuity and organizing care and everyday live, to monitoring disease-specific therapies and integrating palliative and hospice care [ 5 , 10 ]. Moreover, severe MS exerts a profound of social and economic impact [ 9 , 12 , 13 , 14 ]. PwsMS and their caregivers (defined in this manuscript as relatives or closely related individuals directly involved in patients’ care) often find themselves grappling with overwhelming challenges. The process of organizing and coordinating optimal care becomes demanding, as they contend with the perceived unmanageability of searching for, implementing and coordinating health care and social services [ 5 , 15 , 16 , 17 ].

Case management (CM) proved to have a positive effect on patients with neurological disorders and/or patients with palliative care needs [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 ]. However, a focus on severe MS has been missed so far Case managers primarily function as: (1) gatekeeper involving the allocation of necessary and available resources to a case, ensuring the equitable distribution of resources; as (2) broker assisting clients in pursuing their interests, requiring negotiation to provide individualized assistance that aligns as closely as possible with individual needs and (3) advocate working to enhance clients’ individual autonomy, to advocate for essential care offers, and to identify gaps in care [ 25 , 26 , 27 , 28 , 29 ].

Difficulties in understanding, acting, and making decisions regarding health care-related aspects (health literacy) poses a significant challenge for 54% of the German population [ 30 ]. Additionally acting on a superordinate level as an overarching link, a care and case management (CCM) tries to reduce disintegration in the social and health care system [ 31 , 32 ]. Our hypothesis is that a CCM allows PwsMS and their caregivers to regain time and resources outside of disease management and to facilitate the recovery and establishment of biographical continuity that might be disrupted due to severe MS [ 33 , 34 ].

Health care specialists (HCSs) often perceive their work with numerous time and economic constraints, especially when treating complex and severely ill individuals like PwsMS and often have concerns about being blamed by patients when expectations could not be met [ 35 , 36 ]. Our hypothesis is that the CCM will help to reduce time constraints and free up resources for specialized tasks.

To the best of our knowledge there is no long-term cross-sectoral and outreaching authority or service dedicated to assisting in the organization and coordination of the complex care concerns of PwsMS within the framework of standard care addressing needs in health, social, financial, every day and bureaucratic aspects. While some studies have attempted to design and test care programs for persons with MS (PwMS), severely affected individuals were often not included [ 37 , 38 , 39 ]. They often remain overlooked by existing health and social care structures [ 5 , 9 , 15 ].

The COCOS-MS trial developed and applied a long-term cross-sectoral CCM intervention consisting of weekly telephone contacts and monthly re-assessments with PwsMS and caregivers, aiming to provide optimal care. Their problems, resources and (unmet) needs were assessed holistically including physical health, mental health, self-sufficiency and social situation and participation. Based on assessed (unmet) needs, individual care plans with individual actions and goals were developed and constantly adapted during the CCM intervention. Contacts with HCSs were established to ensure optimal care. The CCM intervention was structured through and documented in a CCM manual designed for the trial [ 40 , 41 ].

Our aim was to find out how PwsMS, caregivers and HCSs experienced the cross-sectoral long-term, outreaching patient advocacy CCM.

This study is part of a larger phase II, randomized, controlled clinical trial “Communication, Coordination and Security for people with severe Multiple Sclerosis (COCOS-MS)” [ 41 ]. This explorative clinical trial, employing a mixed-method design, incorporates a qualitative study component with PwsMS, caregivers and HCSs to enrich the findings of the quantitative data. This manuscript focuses on the qualitative data collected between February 2022 and January 2023, following the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines [ 42 ].

Research team

Three trained authors AM, KD and FH (AM, female, research associate, M.A. degree in Rehabilitation Sciences; KD, female, researcher, Dr. rer. medic.; FH, male, research assistant, B.Sc. degree in Health Care Management), who had no prior relationship with patients, caregivers or HCSs conducted qualitative interviews. A research team, consisting of clinical experts and health services researchers, discussed the development of the interview guides and the finalized category system.

Theoretical framework

Interview data was analyzed with the structuring content analysis according to Kuckartz. This method enables a deductive structuring of interview material, as well as the integration of new aspects found in the interview material through the inductive addition of categories in an iterative analysis process [ 43 ].

Sociodemographic and interview characteristics were analyzed descriptively (mean, median, range, SD). PwsMS, caregivers and HCSs were contacted by the authors AM, KD or FH via telephone or e-mail after providing full written informed consent. Participants had the option to choose between online interviews conducted via the GoToMeeting 10.19.0® Software or face-to-face. Peasgood et al. (2023) found no significant differences in understanding questions, engagement or concentration between face-to-face and online interviews [ 44 , 45 ]. Digital assessments were familiar to participants due to pandemic-related adjustments within the trial.

Out of 14 PwsMS and 14 caregivers who were approached to participate in interviews, three declined to complete interviews, resulting in 13 PwsMS (5 male, 8 female) and 12 caregiver (7 male, 5 female) interviews, respectively (see Fig.  1 ). Thirty-one HCSs were contacted of whom ten (2 male, 8 female) agreed to be interviewed (see Fig.  2 ).

Flowchart of PwsMS and caregiver participation in the intervention group of the COCOS-MS trial. Patients could participate with and without a respective caregiver taking part in the trial. Therefore, number of caregivers does not correspond to patients. For detailed inclusion criteria see also Table  1 in Golla et al. [ 41 ]

Flowchart of HCSs interview participation

Setting and data collection

Interviews were carried out where participants preferred, e.g. at home, workplace, online, and no third person being present. In total, we conducted 35 interviews whereof 7 interviews face-to-face (3 PwsMS, 3 caregivers, 1 HCS).

The research team developed a topic guide which was meticulously discussed with research and clinical staff to enhance credibility. It included relevant aspects for the evaluation of the CCM (see Tables  1 and 2 , for detailed topic guides see Supplementary Material ). Patient and caregiver characteristics (covering age, sex, marital status, living situation, EDSS (patients only), subgroup) were collected during the first assessment of the COCOS-MS trial and HCSs characteristics (age, sex, profession) as well as interview information (length and setting) were collected during the interviews. The interview guides developed for this study addressed consistent aspects both for PwsMS and caregivers (see Supplementary Material ):

For HCSs it contained the following guides:

Probing questions were asked to get more specific and in-depth information. Interviews were carried out once and recorded using a recording device or the recording function of the GoToMeeting 10.19.0® Software. Data were pseudonymized (including sensitive information, such as personal names, dates of birth, or addresses), audio files were safely stored in a data protection folder. The interview duration ranged from 11 to 56 min (mean: 23.9 min, SD: 11.1 min). Interviews were continued until we found that data saturation was reached. Audio recordings were transcribed verbatim by an external source and not returned to participants.

Data analysis

Two coders (AM, FH) coded the interviews. Initially, the first author (AM) thoroughly reviewed the transcripts to gain a sense of the interview material. Using the topic guide and literature, she deductively developed a category system based on the primary functions of CM [ 25 , 26 , 27 , 28 , 29 ]. Three interviews were coded repeatedly for piloting, and inductive subcategories were added when new themes emerged in the interview material. This category system proved suitable for the interview material. The second coder (FH) familiarized himself with the interview material and category system. Both coders (AM, FH) independently coded all interviews, engaging in discussions and adjusting codes iteratively. The finalized category system was discussed and consolidated in a research workshop and within the COCOS-MS trial group and finally we reached an intercoder agreement of 90% between the two coders AM and FH, computed by the MAXQDA Standard 2022® software.

We analyzed sociodemographic and interview characteristics using IBM SPSS Statistics 27® and Excel 2016®. Transcripts were managed and analyzed using MAXQDA Standard 2022®.

Participants were provided with oral and written information about the trial and gave written informed consent. Ethical approvals were obtained from the Ethics Committee of the University of Cologne (#20–1436). The trial is registered in the German Register for Clinical Studies (DRKS) (DRKS00022771) and is conducted under the Declaration of Helsinki.

Characteristics of participants and interviews

PwsMS participating in an interview were mainly German (84.6%), had a mean EDSS of 6.8 (range: 6–8) and MS for 13.5 years (median: 14; SD: 8.1). For detailed characteristics see Table  3 .

Most of the interviewed caregivers (9 caregivers) were the partners of the PwsMS with whom they lived in the same household. For further details see Table  3 .

HCSs involved in the study comprised various professions, including MS-nurse (3), neurologist (2), general physician with further training in palliative care (1), physician with further training in palliative care and pain therapist (1), housing counselling service (1), outpatient nursing service manager (1), participation counselling service (1).

Structuring qualitative content analysis

The experiences of PwsMS, caregivers and HCSs were a priori deductively assigned to four main categories: (1) gatekeeper function, (2) broker function, (3) advocacy function [ 25 , 26 , 27 , 28 , 29 ] and (4) Outlook on CCM in standard care, whereas the subcategories were developed inductively (see Fig.  3 ).

Category system including main and subcategories of the qualitative thematic content analysis

The most extensive category, housing the highest number of codes and subcodes, was the “ Outlook on CCM in standard care ” (281 codes). Following this, the category “ Advocacy Function ” contained 261 codes. The “ Broker Function ” (150 codes) and the “ Gatekeeper Function ” (160 codes) constituted two smaller categories. The majority of codes was identified in the caregivers’ interviews, followed by those of PwsMS (see Table  4 ). Illustrative quotes for each category and subcategory can be found in Table  5 .

Persons with severe multiple sclerosis

In the gatekeeper function (59 codes), PwsMS particularly valued the CCM as a continuous contact person . They appreciated the CCM as a person of trust who was reliably accessible throughout the intervention period. This aspect, with 41 codes, held significant importance for PwsMS.

Within the broker function (44 codes), establishing contact was most important for PwsMS (22 codes). This involved the CCM as successfully connecting PwsMS and caregivers with physicians and therapists, as well as coordinating and arranging medical appointments, which were highly valued. Assistance in authority and health and social insurance matters (10 codes) was another subcategory, where the CCM encompassed support in communication with health insurance companies, such as improving the level of care, assisting with retirement pension applications, and facilitating rehabilitation program applications. Optimized care (12 codes) resulted in improved living conditions and the provision of assistive devices through the CCM intervention.

The advocacy function (103 codes) emerged as the most critical aspect for PwsMS, representing the core of the category system. PwsMS experienced multidimensional, comprehensive, cross-insurance system support from the CCM. This category, with 43 statements, was the largest within all subcategories. PwsMS described the CCM as addressing their concerns, providing help, and assisting with the challenges posed by the illness in everyday life. The second-largest subcategory, regaining, maintaining and supporting autonomy (25 codes), highlighted the CCM’s role in supporting self-sufficiency and independence. Reviving personal wellbeing (17 codes) involved PwsMSs’ needs of regaining positive feelings, improved quality of life, and a sense of support and acceptance, which could be improved by the CCM. Temporal relief (18 codes) was reported, with the CCM intervention taking over or reducing tasks.

Within the outlook on CCM in standard care (84 codes), eight subcategories were identified. Communications was described as friendly and open (9 codes), with the setting of communication (29 codes) including the frequency of contacts deemed appropriate by the interviewed PwsMS, who preferred face-to-face contact over virtual or telephone interactions. Improvement suggestions for CCM (10 codes) predominantly revolved around the desire for the continuation of the CCM beyond the trial, expressing intense satisfaction with the CCM contact person and program. PwsMS rarely wished for better cooperation with the CCM. With respect to limitations (7 codes), PwsMS distinguished between individual limitations (e.g. when not feeling ready for using a wheelchair) and overriding structural limitations (e.g. unsuccessful search for an accessible apartment despite CCM support). Some PwsMS mentioned needing the CCM earlier in the course of the disease and believed it would beneficial for anyone with a chronic illness (6 codes).

In the gatekeeper function (75 codes), caregivers highly valued the CCM as a continuous contact partner (33 codes). More frequently than among the PwsMS interviewed, caregivers valued the CCM as a source of consultation/ information on essential individual subjects (42 codes). The need for basic information about the illness, its potential course, treatment and therapy options, possible supportive equipment, and basic medical advice/ information could be met by the CCM.

Within the broker function (63 codes), caregivers primarily experienced the subcategory establish contacts (24 codes). They found the CCM as helpful in establishing and managing contact with physicians, therapists and especially with health insurance companies. In the subcategory assistance in authority and health and social insurance matters (22 codes), caregivers highlighted similar aspects as the PwsMS interviewed. However, there was a particular emphasis on assistance with patients' retirement matters. Caregivers also valued the optimization of patients’ care and living environment (17 codes) in various life areas during the CCM intervention, including improved access to assistive devices, home modification, and involvement of a household support and/ or nursing services.

The advocacy function, with 115 codes, was by far the broadest category . The subcategory multidimensional, comprehensive, cross-insurance system support represented the largest subcategory of caregivers, with 70 statements. In summary, caregivers felt supported by the CCM in all domains of life. Regaining, maintaining and supporting autonomy (11 codes) and reviving personal wellbeing (8 codes) in the form of an improved quality of life played a role not only for patients but also for caregivers, albeit to a lower extend. Caregivers experienced temporal relief (26 codes) as the CCM undertook a wide range of organizational tasks, freeing up more needed resources for their own interests.

For the Outlook on CCM in standard care , caregivers provided various suggestions (81 codes). Similar to PwsMS, caregivers felt that setting (home based face-to-face, telephone, virtual) and frequency of contact were appropriate (10 codes, communication setting ) and communications (7 codes) were recognized as open and friendly. However, to avoid conflicts between caregiver and PwsMS, caregivers preferred meeting the CCM separately from the PwsMS in the future. Some caregivers wished the CCM to specify all services it might offer at the beginning, while others emphasized not wanting this. Like PwsMS, caregivers criticized the CCM intervention being (trial-related) limited to one year, regardless of whether further support was needed or processes being incomplete (13 codes, improvement suggestions ). After the CCM intervention time had expired, the continuous contact person and assistance were missed and new problems had arisen and had to be managed with their own resources again (9 codes, effects of CCM discontinuation ), which was perceived as an exhausting or unsolvable endeavor. Caregivers identified analogous limitations (8 codes), both individual and structural. However, the largest subcategory, was the experienced potential of CCM (27 codes), reflected in extremely high satisfaction with the CCM intervention. Like PwsMS, caregivers regarded severe chronically ill persons in general as target groups for a CCM (7 codes) and would implement it even earlier, starting from the time of diagnosis. They considered a CCM to be particularly helpful for patients without caregivers or for caregivers with limited (time) resources, as it was true for most caregivers.

Health care specialists

In the gatekeeper function (26 codes) HCSs particularly valued the CCM as a continuous contact partner (18 codes). They primarily described their valuable collaboration with the CCM, emphasizing professional exchange between the CCM and HCSs.

Within the broker function (43 codes), the CCM was seen as a connecting link between patients and HCSs, frequently establishing contacts (18 codes). This not only improved optimal care on an individual patient level (case management) but also at a higher, superordinate care level (care management). HCSs appreciated the optimized care and living environment (18 codes) for PwsMS, including improved medical and therapeutic access and the introduction of new assistive devices. The CCM was also recognized as providing assistance in authority and health and social matters (7 codes) for PwsMS and their caregivers.

In the advocacy function (43 codes), HCSs primarily reported temporal relief through CCM intervention (23 codes). They experienced this relief, especially as the CCM provided multidimensional, comprehensive, and cross-insurance system support (15 codes) for PwsMS and their caregivers. Through this support, HCSs felt relieved from time intensive responsibilities that may not fall within their area of expertise, freeing up more time resources for their actual professional tasks.

The largest category within the HCSs interviews was the outlook on CCM in standard care (116 codes). In the largest subcategory, HCSs made suggestions for further patient groups who could benefit (38 codes) from a CCM. Chronic neurological diseases like neurodegenerative diseases (e.g. amyotrophic lateral sclerosis), typical and atypical Parkinson syndromes were mentioned. HCSs considered the enrollment of the CCM directly after the diagnosis of these complex chronic diseases. Additionally, chronic progressive diseases in general or oncological diseases, which may also run chronically, were regarded worthwhile for this approach. HCSs also provided suggestions regarding improvement (21 codes). They wished e.g. for information or contact when patients were enrolled to the CCM, regular updates, exchange and collaborative effort. On the other hand, HCSs reported, that their suggestions for improvement would hardly be feasible due to their limited time resources. Similar to patients and caregivers, HCSs experienced structural limits (13 codes), which a CCM could not exceed due to overriding structural limitations (e.g. insufficient supply of (household) aids, lack of outreach services like psychotherapists, and long processing times on health and pension insurers' side). HCSs were also asked about their opinions on financial resources (14 codes) of a CCM in standard care. All interviewed HCSs agreed that CCM would initially cause more costs for health and social insurers, but they were convinced of cost savings in the long run. HCSs particularly perceived the potential of the CCM (20 codes) through the feedback of PwsMS, highlighting the trustful relationship enabling individualized help for PwsMS and their caregivers.

Persons with severe multiple sclerosis and their caregivers

The long-term cross-sectoral CCM intervention implemented in the COCOS-MS trial addressed significant unmet needs of PwsMS and their caregivers which previous research revealed as burdensome and hardly or even not possible to improve without assistance [ 5 , 6 , 9 , 10 , 33 , 35 , 46 ]. Notably, the CCM service met the need for a reliable, continuous contact partner, guiding patients through the complexities of regulations, authorities and the insurance system. Both, PwsMS and their caregivers highly valued the professional, objective perspective provided by the CCM, recognizing it as a source of relief, support and improved care in line with previous studies [ 37 , 47 ]. Caregivers emphasized the CCM’s competence in offering concrete assistance and information on caregiving and the fundamentals of MS, including bureaucratic, authority and insurances matters. On the other hand, PwsMS particularly appreciated the CCMs external reflective and advisory function, along with empathic social support tailored to their individual concerns. Above all, the continuous partnership of trust, available irrespective of the care sector, was a key aspect that both PwsMS and their caregivers highlighted. This consistent support was identified as one of the main components in the care of PwsMS in previous studies [ 5 , 33 , 35 ].

As the health literacy is inadequate or problematic for 54% of the German population and disintegration in the health and social care system is high [ 30 , 31 , 32 ], the CCM approach serves to enhance health literacy and reduce disintegration of PwsMS and their caregivers by providing cross-insurance navigational guidance in the German health and social insurance sector on a superordinate level. Simultaneously PwsMS and caregivers experienced relief and gained more (time) resources for all areas of life outside of the disease and its management, including own interests and establishing biographical continuity. This empowerment enables patients to find a sense of purpose beyond their illness, regain autonomy, and enhance social participation, reducing the feeling of being a burden to those closest to them. Such feelings are often experienced as burdensome and shameful by PwsMS [ 6 , 48 , 49 , 50 ]. Finding a sense of purpose beyond the illness also contributes to caregivers perceiving their loved ones not primarily as patient but as individuals outside of the disease, reinforcing valuable relationships such as partners, siblings, or children, strengthening emotional bonds. These factors are also highly relevant and well-documented in a suicide-preventive context, as the suicide rate is higher in persons diagnosed with neurological disorders [ 19 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ] and the feeling of being a burden to others, loss of autonomy, and perceived loss of dignity are significant factors in patients with severe chronic neurological diseases for suicide [ 50 , 57 ].

The temporal relief experienced by the CCM was particularly significant for HCSs and did not only improve the satisfaction of HCSs but also removed unfulfilled expectations and concerns about being blamed by patients when expectations could not be met, which previous studied elaborated [ 35 , 36 ]. Moreover, the CCM alleviated the burden on HCSs by addressing patients’ concerns, allowing them to focus on their own medical responsibilities. This aspect probably reduced the dissatisfaction that arises when HCSs are expected to address issues beyond their medical expertise, such as assistive devices, health and social insurance, and the organization and coordination of supplementary therapies, appointments, and contacts [ 35 , 36 , 61 ]. Consequently, the CCM reduced difficulties of HCSs treating persons with neurological or chronical illnesses, which previous research identified as problematic.

HCSs perceive their work as increasingly condensed with numerous time and economic constraints, especially when treating complex and severely ill individuals like PwsMS [ 36 ]. This constraint was mentioned by HCSs in the interviews and was one of the main reasons why they were hesitant to participate in interviews and may also be an explanation for a shorter interview duration than initially planned in the interview guides. The CCM’s overarching navigational competence in the health and social insurance system was particularly valued by HCSs. The complex and often small-scale specialties in the health and social care system are not easily manageable or well-known even for HCSs, and dealing with them can exceed their skills and time capacities [ 61 ]. The CCM played a crucial role in keeping (temporal) resources available for what HCSs are professionally trained and qualified to work on. However, there remains a challenge in finding solutions to the dilemma faced by HCSs regarding their wish to be informed about CCM procedures and linked with each other, while also managing the strain of additional requests and contact with the CCM due to limited (time) resources [ 62 ]. Hudon et al. (2023) suggest that optimizing time resources and improving exchange could involve meetings, information sharing via fax, e-mail, secure online platforms, or, prospectively, within the electronic patient record (EPR). The implementation of an EPR has shown promise in improving the quality of health care and time resources, when properly implemented [ 63 , 64 ]. The challenge lies ineffective information exchange between HCSs and CCM for optimal patient care. The prospect of time saving in the long run and at best for a financial incentive, e.g., when anchoring in the Social Security Code, will help best to win over the HCSs.If this crucial factor can be resolved, there is a chance that HCSs will thoroughly accept the CCM as an important pillar, benefiting not only PwsMS but also other complex patient groups, especially those with long-term neurological or complex oncological conditions that might run chronically.

Care and case management and implications for the health care system

The results of our study suggest that the cross-sectoral long-term advocacy CCM in the COCOS-MS trial, with continuous personal contacts at short intervals and constant reevaluation of needs, problems, resources and goals, is highly valued by PwsMS, caregivers, and HCSs. The trial addresses several key aspects that may have been overlooked in previous studies which have shown great potential for the integration of case management [ 17 , 47 , 62 , 65 , 66 ]. However, they often excluded the overriding care management, missed those patient groups with special severity and complexity who might struggle to reach social and health care structures independently or the interventions were not intended for long-term [ 22 , 37 ]. Our results indicate that the CCM intervention had a positive impact on PwsMS and caregivers as HCSs experienced them with benefits such as increased invigoration, reduced demands, and enhanced self-confidence. However, there was a notable loss experienced by PwsMS and caregivers after the completion of the CCM intervention, even if they had stabilized during the intervention period. The experiences of optimized social and health care for the addressed population, both at an individual and superordinate care level, support the integration of this service into standard care. Beyond the quantitatively measurable outcomes and economic considerations reported elsewhere [ 16 , 20 , 21 ], our results emphasize the importance of regaining control, self-efficacy, self-worth, dignity, autonomy, and social participation. These aspects are highlighted as preventive measures in suicidal contexts, which is particularly relevant for individuals with severe and complex illnesses [ 19 , 50 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ]. Our findings further emphasize the societal responsibilities to offer individuals with severe and complex illnesses the opportunity to regain control and meaningful aspects of life, irrespective of purely economic considerations. This underscores the need for a comprehensive evaluation that not only takes into account quantitative measures but also the qualitative aspects of well-being and quality of life when making recommendations of a CCM in standard care.

The study by J. Y. Joo and Huber (2019) highlighted that CM interventions aligned with the standards of the Case Management Society of America varied in duration, ranging from 1 month to 15.9 years, and implemented in community- or hospital-based settings. However, they noted a limitation in understanding how CM processes unfold [ 67 ]. In contrast, our trial addressed this criticism by providing transparent explanations of the CCM process, which also extends to a superordinate care management [ 40 , 41 ]. Our CCM manual [ 40 ] outlines a standardized and structured procedure for measuring and reevaluating individual resources, problems, and unmet needs on predefined dimensions. It also identifies goals and actions at reducing unmet needs and improving the individual resources of PwsMS and caregivers. Importantly, the CCM manual demonstrates that the CCM process can be structured and standardized, while accounting for the unique aspects of each individual’s serious illness, disease courses, complex needs, available resources, and environmental conditions. Furthermore, the adaptability of the CCM manual to other complex chronically ill patient groups suggests the potential for a standardized approach in various health care settings. This standardized procedure allows for consistency in assessing and addressing the individual needs of patients, ensuring that the CCM process remains flexible while maintaining a structured and goal-oriented framework.

The discussion about the disintegration in the social and health care system and the increasing specialization dates back to 2009 [ 31 , 32 ]. Three strategies were identified to address this issue: (a) “driver-minimizing” [Treiberminimierende], (b) “effect-modifying” [Effektmodifizierende] and (c) “disintegration-impact-minimizing” [Desintegrationsfolgenminimierende] strategies. “Driver-minimizing strategies” involve comprehensive and radical changes within the existing health and social care system, requiring political and social pursuit. “Disintegration-impact-minimizing strategies” are strategies like quality management or tele-monitoring, which are limited in scope and effectiveness. “Effect-modifying strategies”, to which CCM belongs, acknowledges the segmentation within the system but aims to overcome it through cooperative, communicative, and integrative measures. CCM, being an “effect-modifying strategy”, operates the “integrated segmentation model” [Integrierte Segmentierung] rather than the “general contractor model” [Generalunternehmer-Modell] or “total service provider model” [Gesamtdienstleister-Modell] [ 31 , 32 ]. In this model, the advantage lies in providing an overarching and coordinating service to link different HCSs and services cross-sectorally. The superordinate care management aspect of the CCM plays a crucial role in identifying gaps in care, which is essential for future development strategies within the health and social care system. It aims to find or develop (regional) alternatives to ensure optimal care [ 17 , 23 , 24 , 68 , 69 ], using regional services of existing health and social care structures. Therefore, superordinate care management within the CCM process is decisive for reducing disintegration in the system.

Strengths and limitations

The qualitative study results of the explorative COCOS-MS clinical trial, which employed an integrated mixed-method design, provide valuable insights into the individual experiences of three leading stakeholders: PwsMS, caregivers and HCSs with a long-term cross-sectoral CCM. In addition to in-depth interviews, patient and caregiver reported outcome measurements were utilized and will be reported elsewhere. The qualitative study’s strengths include the inclusion of patients who, due to the severity of their condition (e.g. EDSS mean: 6.8, range: 6–8, highly active MS), age (mean: 53.9 years, range: 36–73 years) family constellations, are often underrepresented in research studies and often get lost in existing social and health care structures. The study population is specific to the wider district region of Cologne, but the broad inclusion criteria make it representative of severe MS in Germany. The methodological approach of a deductive and inductive structuring content analysis made it possible to include new findings into an existing theoretical framework.

However, the study acknowledges some limitations. While efforts were made to include more HCSs, time constraints on their side limited the number of interviews conducted and might have biased the results. Some professions are underrepresented in the interviews. Complex symptoms (e.g. fatigue, ability to concentrate), medical or therapeutic appointments and organization of the everyday live may have been reasons for the patients’ and caregivers’ interviews lasting shorter than initially planned.

The provision of functions of a CCM, might have pre-structured the answers of the participants.

At current, there is no support system for PwsMS, their caregivers and HCSs that addresses their complex and unmet needs comprehensively and continuously. There are rare qualitative insights of the three important stakeholders: PwsMS, caregivers and HCSs in one analysis about a supporting service like a CCM. In response to this gap, we developed and implemented a long-term cross-sectoral advocacy CCM and analyzed it qualitatively. PwsMS, their caregivers and HCSs expressed positive experiences, perceiving the CCM as a source of relief and support that improved care across various aspects of life. For patients, the CCM intervention resulted in enhanced autonomy, reviving of personal wellbeing and new established contacts with HCSs. Caregivers reported a reduced organizational burden and felt better informed, and HCSs experienced primarily temporal relief, allowing them to concentrate on their core professional responsibilities. At a higher level of care, the study suggests that the CCM contributed to a reduction in disintegration within the social and health care system.

The feedback from participants is seen as valuable for adapting the CCM intervention and the CCM manual for follow-up studies, involving further complex patient groups such as neurological long-term diseases apart from MS and tailoring the duration of the intervention depending on the complexity of evolving demands.

Availability of data and materials

Generated and/or analyzed datasets of participants are available from the corresponding author on reasonable request to protect participants. Preliminary partial results have been presented as a poster during the EAPC World Congress in June 2023 and the abstract has been published in the corresponding abstract booklet [ 70 ].

Abbreviations

Amyotrophic lateral sclerosis

• Care and case management

Case management

Central nervous system

Communication, Coordination and security for people with multiple sclerosis

Consolidated criteria for reporting qualitative research

German register for clinical studies

Extended disability status scale

Electronic patient record

Quality of life

Multiple sclerosis

Koch-Henriksen N, Magyari M. Apparent changes in the epidemiology and severity of multiple sclerosis. Nat Rev Neurol. 2021;17:676–88. https://doi.org/10.1038/s41582-021-00556-y .

Article   PubMed   Google Scholar  

Ellenberger D, Flachenecker P, Fneish F, Frahm N, Hellwig K, Paul F, et al. Aggressive multiple sclerosis: a matter of measurement and timing. Brain. 2020;143:e97. https://doi.org/10.1093/brain/awaa306 .

Article   PubMed   PubMed Central   Google Scholar  

Edmonds P, Vivat B, Burman R, Silber E, Higginson IJ. Loss and change: experiences of people severely affected by multiple sclerosis. Palliat Med. 2007;21:101–7. https://doi.org/10.1177/0269216307076333 .

Kurtzke JF. Rating neurologic impairment in multiple rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS). Neurology. 1983;33(11):1444–52.

Article   CAS   PubMed   Google Scholar  

Galushko M, Golla H, Strupp J, Karbach U, Kaiser C, Ernstmann N, et al. Unmet needs of patients feeling severely affected by multiple sclerosis in Germany: a qualitative study. J Palliat Med. 2014;17:274–81. https://doi.org/10.1089/jpm.2013.0497 .

Borreani C, Bianchi E, Pietrolongo E, Rossi I, Cilia S, Giuntoli M, et al. Unmet needs of people with severe multiple sclerosis and their carers: qualitative findings for a home-based intervention. PLoS One. 2014:e109679. https://doi.org/10.1371/journal.pone.0109679 .

Yamout BI, Alroughani R. Multiple Sclerosis. Semin Neurol. 2018;38:212–25. https://doi.org/10.1055/s-0038-1649502 .

Nissen N, Lemche J, Reestorff CM, Schmidt M, Skjerbæk AG, Skovgaard L, et al. The lived experience of uncertainty in everyday life with MS. Disabil Rehabil. 2022;44:5957–63. https://doi.org/10.1080/09638288.2021.1955302 .

Strupp J, Hartwig A, Golla H, Galushko M, Pfaff H, Voltz R. Feeling severely affected by multiple sclerosis: what does this mean? Palliat Med. 2012;26:1001–10. https://doi.org/10.1177/0269216311425420 .

Strupp J, Voltz R, Golla H. Opening locked doors: Integrating a palliative care approach into the management of patients with severe multiple sclerosis. Mult Scler J. 2016;22:13–8.

Article   CAS   Google Scholar  

Kraft AK, Berger K. Kernaspekte einer bedarfsgerechten Versorgung von Patienten mit Multipler Sklerose : Inanspruchnahme ambulanter Leistungen und „shared decision making“ [Core aspects of a needs-conform care of patients with multiple sclerosis : Utilization of outpatient services and shared decision making]. Nervenarzt. 2020;91:503–10. https://doi.org/10.1007/s00115-020-00906-z .

Doshi A, Chataway J. Multiple sclerosis, a treatable disease. Clin Med (Lond). 2017;17:530–6. https://doi.org/10.7861/clinmedicine.17-6-530 .

Kobelt G, Thompson A, Berg J, Gannedahl M, Eriksson J. New insights into the burden and costs of multiple sclerosis in Europe. Mult Scler. 2017;23:1123–36. https://doi.org/10.1177/1352458517694432 .

Conradsson D, Ytterberg C, Engelkes C, Johansson S, Gottberg K. Activity limitations and participation restrictions in people with multiple sclerosis: a detailed 10-year perspective. Disabil Rehabil. 2021;43:406–13. https://doi.org/10.1080/09638288.2019.1626919 .

Sorensen PS, Giovannoni G, Montalban X, Thalheim C, Zaratin P, Comi G. The Multiple Sclerosis Care Unit. Mult Scler J. 2019;5:627–36.

Article   Google Scholar  

Tan H, Yu J, Tabby D, Devries A, Singer J. Clinical and economic impact of a specialty care management program among patients with multiple sclerosis: a cohort study. Mult Scler. 2010;16:956–63. https://doi.org/10.1177/1352458510373487 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Oeseburg B, Wynia K, Middel B, Reijneveld SA. Effects of case management for frail older people or those with chronic illness: a systematic review. Nurs Res. 2009;58:201–10.

Aiken LS, Butner J, Lockhart CA, Volk-Craft BE, Hamilton G, Williams FG. Outcome evaluation of a randomized trial of the PhoenixCare intervention: program of case management and coordinated care for the seriously chronically ill. J Palliat Med. 2006;9:111–26. https://doi.org/10.1089/jpm.2006.9.111 .

Kuhn U, Düsterdiek A, Galushko M, Dose C, Montag T, Ostgathe C, Voltz R. Identifying patients suitable for palliative care—a descriptive analysis of enquiries using a Case Management Process Model approach. BMC Res Notes. 2012;5:611. https://doi.org/10.1186/1756-0500-5-611 .

Leary A, Quinn D, Bowen A. Impact of proactive case management by multiple sclerosis specialist nurses on use of unscheduled care and emergency presentation in multiple sclerosis: a case study. Int J MS Care. 2015;17:159–63. https://doi.org/10.7224/1537-2073.2014-011 .

Strupp J, Dose C, Kuhn U, Galushko M, Duesterdiek A, Ernstmann N, et al. Analysing the impact of a case management model on the specialised palliative care multi-professional team. Support Care Cancer. 2018;26:673–9. https://doi.org/10.1007/s00520-017-3893-3 .

Wynia K, Annema C, Nissen H, de Keyser J, Middel B. Design of a Randomised Controlled Trial (RCT) on the effectiveness of a Dutch patient advocacy case management intervention among severely disabled Multiple Sclerosis patients. BMC Health Serv Res. 2010;10:142. https://doi.org/10.1186/1472-6963-10-142 .

Ewers M, Schaeffer D, editors. Case Management in Theorie und Praxis. Bern: Huber; 2005.

Google Scholar  

Neuffer M. Case Management: Soziale Arbeit mit Einzelnen und Familien. 5th ed. Weinheim, Basel: Beltz Juventa; 2013.

Case Management Society of America. The standards of practice for case management. 2022.

Deutsche Gesellschaft für Care und Case Management e.V., editor. Case Management Leitlinien: Rahmenempfehlung, Standards und ethische Grundlagen. 2nd ed. Heidelberg: Medhochzwei; 2020.

Monzer M. Case Management Grundlagen. 2nd ed. Heidelberg: Medhochzwei; 2018.

Wissert M. Grundfunktionen und fachliche Standards des Unterstützungsmanagements. Z Gerontol Geriat. 1998;31(5):331–7.

Wissert M. Tools und Werkzeuge beim Case Management: Die Hilfeplanung. Case Manag. 2007;1:35–7.

Schaeffer D, Berens E-M, Vogt D. Health literacy in the German population. Dtsch Arztebl Int. 2017;114:53–60. https://doi.org/10.3238/arztebl.2017.0053 .

Pfaff H, Schulte H. Der onkologische Patient der Zukunft. Onkologe. 2012;18:127–33. https://doi.org/10.1007/s00761-011-2201-y .

Pfaff H, Kowalski C, Ommen O. Modelle zur Analyse von Integration und Koordination im Versorgungssystem. In: Ameldung, Sydow, Windeler, editor. Vernetzung im Gesundheitswesen: Wettbewerb und Kooperation. Stuttgart: Kohlhammer Verlag; 2009. p. 75–90.

Golla H, Mammeas S, Galushko M, Pfaff H, Voltz R. Unmet needs of caregivers of severely affected multiple sclerosis patients: A qualitative study. Palliat Support Care. 2015;13(6):1685–93.

Golla H, Galushko M, Pfaff H, Voltz R. Multiple sclerosis and palliative care - perceptions of severely affected multiple sclerosis patients and their health professionals: a qualitative study. BMC Palliat Care. 2014;13:11. https://doi.org/10.1186/1472-684x-13-11 .

Golla H, Galushko M, Pfaff H, Voltz R. Unmet needs of severely affected multiple sclerosis patients: the health professionals’ view. Palliat Med. 2012;26:139–51. https://doi.org/10.1177/0269216311401465 .

Methley AM, Chew-Graham CA, Cheraghi-Sohi S, Campbell SM. A qualitative study of patient and professional perspectives of healthcare services for multiple sclerosis: implications for service development and policy. Health Soc Care Community. 2017;25:848–57. https://doi.org/10.1111/hsc.12369 .

Kalb R, Costello K, Guiod L. Case management services to meet the complex needs of patients with multiple sclerosis in the community—the successes and challenges of a unique program from the national multiple sclerosis society. US Neurology. 2019;15:27–31.

Krüger K, Fricke LM, Dilger E-M, Thiele A, Schaubert K, Hoekstra D, et al. How is and how should healthcare for people with multiple sclerosis in Germany be designed?-The rationale and protocol for the mixed-methods study Multiple Sclerosis-Patient-Oriented Care in Lower Saxony (MS-PoV). PLoS One. 2021;16:e0259855. https://doi.org/10.1371/journal.pone.0259855 .

Ivancevic S, Weegen L, Korff L, Jahn R, Walendzik A, Mostardt S, et al. Effektivität und Kosteneffektivät von Versorgungsmanagement-Programmen bei Multipler Sklerose in Deutschland – Eine Übersichtsarbeit. Akt Neurol. 2015;42:503–8. https://doi.org/10.1055/s-0035-1564111 .

Müller A, Dillen K, Dojan T, Ungeheuer S, Goereci Y, Dunkl V, et al. Development of a long-term cross-sectoral case and care management manual for patients with severe multiple sclerosis and their caregivers. Prof Case Manag. 2023;28:183–93. https://doi.org/10.1097/NCM.0000000000000608 .

Golla H, Dillen K, Hellmich M, Dojan T, Ungeheuer S, Schmalz P, et al. Communication, Coordination, and Security for People with Multiple Sclerosis (COCOS-MS): a randomised phase II clinical trial protocol. BMJ Open. 2022;12:e049300. https://doi.org/10.1136/bmjopen-2021-049300 .

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. https://doi.org/10.1093/intqhc/mzm042 .

Kuckartz U. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung. 4th ed. Weinheim: Beltz Juventa; 2018.

Akyirem S, Ekpor E, Aidoo-Frimpong GA, Salifu Y, Nelson LE. Online interviews for qualitative health research in Africa: a scoping review. Int Health. 2023. https://doi.org/10.1093/inthealth/ihad010 .

Peasgood T, Bourke M, Devlin N, Rowen D, Yang Y, Dalziel K. Randomised comparison of online interviews versus face-to-face interviews to value health states. Soc Sci Med. 2023;323:115818. https://doi.org/10.1016/j.socscimed.2023.115818 .

Giordano A, Cimino V, Campanella A, Morone G, Fusco A, Farinotti M, et al. Low quality of life and psychological wellbeing contrast with moderate perceived burden in carers of people with severe multiple sclerosis. J Neurol Sci. 2016;366:139–45. https://doi.org/10.1016/j.jns.2016.05.016 .

Joo JY, Liu MF. Experiences of case management with chronic illnesses: a qualitative systematic review. Int Nurs Rev. 2018;65(1):102–1113.

Freeman J, Gorst T, Gunn H, Robens S. “A non-person to the rest of the world”: experiences of social isolation amongst severely impaired people with multiple sclerosis. Disabil Rehabil. 2020;42:2295–303. https://doi.org/10.1080/09638288.2018.1557267 .

National Institute for Health and Care Excellence. Multiple sclerosis: Management of multiple sclerosis in primary and secondary care. 2014.

Erdmann A, Spoden C, Hirschberg I, Neitzke G. The wish to die and hastening death in amyotrophic lateral sclerosis: A scoping review. BMJ Support Palliat Care. 2021;11:271–87. https://doi.org/10.1136/bmjspcare-2020-002640 .

Erlangsen A, Stenager E, Conwell Y, Andersen PK, Hawton K, Benros ME, et al. Association between neurological disorders and death by suicide in Denmark. JAMA. 2020;323:444–54. https://doi.org/10.1001/jama.2019.21834 .

Kalb R, Feinstein A, Rohrig A, Sankary L, Willis A. Depression and suicidality in multiple sclerosis: red flags, management strategies, and ethical considerations. Curr Neurol Neurosci Rep. 2019;19:77. https://doi.org/10.1007/s11910-019-0992-1 .

Feinstein A, Pavisian B. Multiple sclerosis and suicide. Mult Scler. 2017;23:923–7. https://doi.org/10.1177/1352458517702553 .

Marrie RA, Salter A, Tyry T, Cutter GR, Cofield S, Fox RJ. High hypothetical interest in physician-assisted death in multiple sclerosis. Neurology. 2017;88:1528–34. https://doi.org/10.1212/WNL.0000000000003831 .

Gauthier S, Mausbach J, Reisch T, Bartsch C. Suicide tourism: a pilot study on the Swiss phenomenon. J Med Ethics. 2015;41:611–7. https://doi.org/10.1136/medethics-2014-102091 .

Fischer S, Huber CA, Imhof L, MahrerImhof R, Furter M, Ziegler SJ, Bosshard G. Suicide assisted by two Swiss right-to-die organisations. J Med Ethics. 2008;34:810–4. https://doi.org/10.1136/jme.2007.023887 .

Strupp J, Ehmann C, Galushko M, Bücken R, Perrar KM, Hamacher S, et al. Risk factors for suicidal ideation in patients feeling severely affected by multiple sclerosis. J Palliat Med. 2016;19:523–8. https://doi.org/10.1089/jpm.2015.0418 .

Spence RA, Blanke CD, Keating TJ, Taylor LP. Responding to patient requests for hastened death: physician aid in dying and the clinical oncologist. J Oncol Pract. 2017;13:693–9. https://doi.org/10.1200/JOP.2016.019299 .

Monforte-Royo C, Villavicencio-Chávez C, Tomás-Sábado J, Balaguer A. The wish to hasten death: a review of clinical studies. Psychooncology. 2011;20:795–804. https://doi.org/10.1002/pon.1839 .

Blanke C, LeBlanc M, Hershman D, Ellis L, Meyskens F. Characterizing 18 years of the death with dignity act in Oregon. JAMA Oncol. 2017;3:1403–6. https://doi.org/10.1001/jamaoncol.2017.0243 .

Methley A, Campbell S, Cheraghi-Sohi S, Chew-Graham C. Meeting the mental health needs of people with multiple sclerosis: a qualitative study of patients and professionals. Disabil Rehab. 2017;39(11):1097-105. https://doi.org/10.1080/09638288.2016.1180547 .

Hudon C, Bisson M, Chouinard M-C, Delahunty-Pike A, Lambert M, Howse D, et al. Implementation analysis of a case management intervention for people with complex care needs in primary care: a multiple case study across Canada. BMC Health Serv Res. 2023;23:377. https://doi.org/10.1186/s12913-023-09379-7 .

Beckmann M, Dittmer K, Jaschke J, Karbach U, Köberlein-Neu J, Nocon M, et al. Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol. BMC Health Serv Res. 2021;21:377. https://doi.org/10.1186/s12913-021-06377-5 .

Campanella P, Lovato E, Marone C, Fallacara L, Mancuso A, Ricciardi W, Specchia ML. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. Eur J Public Health. 2016;26:60–4. https://doi.org/10.1093/eurpub/ckv122 .

García-Hernández M, González de León B, Barreto-Cruz S, Vázquez-Díaz JR. Multicomponent, high-intensity, and patient-centered care intervention for complex patients in transitional care: SPICA program. Front Med (Lausanne). 2022;9:1033689. https://doi.org/10.3389/fmed.2022.1033689 .

Meisinger C, Stollenwerk B, Kirchberger I, Seidl H, Wende R, Kuch B, Holle R. Effects of a nurse-based case management compared to usual care among aged patients with myocardial infarction: results from the randomized controlled KORINNA study. BMC Geriatr. 2013. https://doi.org/10.1186/1471-2318-13-115 .

Joo JY, Huber DL. Case management effectiveness on health care utilization outcomes: a systematic review of reviews. West J Nurs Res. 2019;41:111–33. https://doi.org/10.1177/0193945918762135 .

Stergiopoulos V, Gozdzik A, Misir V, Skosireva A, Connelly J, Sarang A, et al. Effectiveness of housing first with intensive case management in an ethnically diverse sample of homeless adults with mental illness: a randomized controlled trial. PLoS One. 2015;10:e0130281. https://doi.org/10.1371/journal.pone.0130281 .

Löcherbach P, Wendt R, editors. Care und Case Management: Transprofessionelle Versorgungsstrukturen und Netzwerke. 1st ed. Stuttgart: Kohlhammer; 2020.

EAPC2023 Abstract Book. Palliat Med. 2023;37:1–302. https://doi.org/10.1177/02692163231172891 .

Download references

Acknowledgements

We would like to thank all the patients, caregivers and health care specialists who volunteered their time to participate in an interview and the trial, Carola Janßen for transcribing the interviews, Fiona Brown for translating the illustrative quotes and Beatrix Münzberg, Kerstin Weiß and Monika Höveler for data collection in the quantitative study part.

COCOS-MS Trial Group

Anne Müller 1 , Fabian Hebben 1 , Kim Dillen 1 , Veronika Dunkl 1 , Yasemin Goereci 2 , Raymond Voltz 1,3,4 , Peter Löcherbach 5 , Clemens Warnke 2 , Heidrun Golla 1 , Dirk Müller 6 , Dorthe Hobus 1 , Eckhard Bonmann 7 , Franziska Schwartzkopff 8 , Gereon Nelles 9 , Gundula Palmbach 8 , Herbert Temmes 10 , Isabel Franke 1 , Judith Haas 10 , Julia Strupp 1 , Kathrin Gerbershagen 7 , Laura Becker-Peters 8 , Lothar Burghaus 11 , Martin Hellmich 12 , Martin Paus 8 , Solveig Ungeheuer 1 , Sophia Kochs 1 , Stephanie Stock 6 , Thomas Joist 13 , Volker Limmroth 14

1 Department of Palliative Medicine, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

2 Department of Neurology, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

3 Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), University of Cologne, Cologne, Germany

4 Center for Health Services Research (ZVFK), University of Cologne, Cologne, Germany

5 German Society of Care and Case Management e.V. (DGCC), Münster, Germany

6 Institute for Health Economics and Clinical Epidemiology (IGKE), Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

7 Department of Neurology, Klinikum Köln, Cologne, Germany

8 Clinical Trials Centre Cologne (CTCC), Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

9 NeuroMed Campus, MedCampus Hohenlind, Cologne, Germany

10 German Multiple Sclerosis Society Federal Association (DMSG), Hannover, Germany

11 Department of Neurology, Heilig Geist-Krankenhaus Köln, Cologne, Germany

12 Institute of Medical Statistics and Computational Biology (IMSB), Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

13 Academic Teaching Practice, University of Cologne, Cologne, Germany

14 Department of Neurology, Klinikum Köln-Merheim, Cologne, Germany

Open Access funding enabled and organized by Projekt DEAL. This work was supported by the Innovation Funds of the Federal Joint Committee (G-BA), grant number: 01VSF19029.

Author information

Authors and affiliations.

Department of Palliative Medicine, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

Anne Müller, Fabian Hebben, Kim Dillen, Veronika Dunkl, Raymond Voltz & Heidrun Golla

Department of Neurology, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany

Yasemin Goereci & Clemens Warnke

Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), University of Cologne, Cologne, Germany

Raymond Voltz

Center for Health Services Research, University of Cologne, Cologne, Germany

German Society of Care and Case Management E.V. (DGCC), Münster, Germany

Peter Löcherbach

You can also search for this author in PubMed   Google Scholar

• Anne Müller
• , Fabian Hebben
• , Kim Dillen
• , Veronika Dunkl
• , Yasemin Goereci
• , Raymond Voltz
• , Peter Löcherbach
• , Clemens Warnke
• , Heidrun Golla
• , Dirk Müller
• , Dorthe Hobus
• , Eckhard Bonmann
• , Franziska Schwartzkopff
• , Gereon Nelles
• , Gundula Palmbach
• , Herbert Temmes
• , Isabel Franke
• , Judith Haas
• , Julia Strupp
• , Kathrin Gerbershagen
• , Laura Becker-Peters
• , Lothar Burghaus
• , Martin Hellmich
• , Martin Paus
• , Solveig Ungeheuer
• , Sophia Kochs
• , Stephanie Stock
• , Thomas Joist
•  & Volker Limmroth

Contributions

HG, KD, CW designed the trial. HG, KD obtained ethical approvals. HG, KD developed the interview guidelines with help of the CCM (SU). AM was responsible for collecting qualitative data, developing the code system, coding, analysis of the data and writing the first draft of the manuscript, thoroughly revised and partly rewritten by HG. FH supported in collecting qualitative data, coding and analysis of the interviews. KD supported in collecting qualitative data. AM, FH, KD, VD, YG, RV, PL, CW, HG discussed and con-solidated the finalized category system. AM, FH, KD, VD, YG, RV, PL, CW, HG read and commented on the manuscript and agreed to the final version.

Authors’ information

Not applicable.

Corresponding author

Correspondence to Anne Müller .

Ethics declarations

Ethics approval and consent to participate.

Participants were provided with oral and written information about the trial and provided written informed consent. Ethical approval was obtained from the Ethics Committee of the University of Cologne (#20–1436). The trial is registered in the German Register for Clinical Studies (DRKS) (DRKS00022771) and is conducted under the Declaration of Helsinki.

Consent for publication

Competing interests.

Clemens Warnke has received institutional support from Novartis, Alexion, Sanofi Genzyme, Janssen, Biogen, Merck and Roche. The other authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Müller, A., Hebben, F., Dillen, K. et al. “So at least now I know how to deal with things myself, what I can do if it gets really bad again”—experiences with a long-term cross-sectoral advocacy care and case management for severe multiple sclerosis: a qualitative study. BMC Health Serv Res 24 , 453 (2024). https://doi.org/10.1186/s12913-024-10851-1

Download citation

Received : 23 November 2023

Accepted : 11 March 2024

Published : 10 April 2024

DOI : https://doi.org/10.1186/s12913-024-10851-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Cross-sectoral
• Qualitative research
• Health care specialist
• Severe multiple sclerosis

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Volume 20 Supplement 2

The Healthy Birth study: an evaluative research of the Adequate Childbirth Program

• Open access
• Published: 17 April 2024

Incorporation, adaptation and rejection of obstetric practices during the implementation of the “Adequate Childbirth Program” in Brazilian private hospitals: a qualitative study

• Débora Cecília Chaves de Oliveira   ORCID: orcid.org/0000-0003-1550-743X 1 ,
• Maysa Luduvice Gomes   ORCID: orcid.org/0000-0003-4557-4377 1 ,
• Andreza Rodrigues   ORCID: orcid.org/0000-0002-1873-5828 2 ,
• Thamires Soares   ORCID: orcid.org/0000-0003-3693-393X 2 ,
• Lucia Regina de Azevedo Nicida   ORCID: orcid.org/0000-0002-6517-463X 3 ,
• Jacqueline Alves Torres   ORCID: orcid.org/0000-0002-0567-2952 4 &
• Elyne Montenegro Engstrom   ORCID: orcid.org/0000-0001-6149-3396 5  

Reproductive Health volume  20 , Article number:  189 ( 2022 ) Cite this article

Metrics details

The “Adequate Childbirth Program” (PPA) is a quality improvement project that aims to reduce the high rates of unnecessary cesarean section in Brazilian private hospitals. This study aimed to analyze labor and childbirth care practices after the first phase of PPA implementation.

This study uses a qualitative approach. Eight hospitals were selected. At each hospital, during the period of 5 (five) days, from July to October 2017, the research team conducted face to face interviews with doctors ( n  = 21) and nurses ( n  = 28), using semi-structured scripts. For the selection of professionals, the Snowball technique was used. The interviews were transcribed, and the data submitted to Thematic Content Analysis, using the MaxQda software.

The three analytical dimensions of the process of change in the care model: (1) Incorporation of care practices: understood as the practices that have been included since PPA implementation; (2) Adaptation of care practices: understood as practices carried out prior to PPA implementation, but which underwent modifications with the implementation of the project; (3) Rejection of care practices: understood as those practices that were abandoned or questioned whether or not they should be carried out by hospital professionals.

Conclusions

After the PPA, changes were made in hospitals and in the way, women were treated. Birth planning, prenatal hospital visits led by experts (for expecting mothers and their families), diet during labor, pharmacological analgesia for vaginal delivery, skin-to-skin contact, and breastfeeding in the first hour of life are all included. To better monitor labor and vaginal birth and to reduce CS without a clinical justification, hospitals adjusted their present practices. Finally, the professionals rejected the Kristeller maneuver since research has demonstrated that using it’s harmful.

Plain English summary

Brazil has high Cesarean Section (CS) rates, with rates far from the ideal recommended by the World Health Organization and a model of care that does not favor women’s autonomy and empowerment. In 2015, a quality improvement project, called “Projeto Parto Adequado” (PPA), was implemented in Brazilian private hospitals to reduce unnecessary cesarean section, in addition to encouraging the process of natural and safe childbirth. One of the components of this project was to reorganize the model of care in hospitals to prepare professionals for humanized and safe care. The data were collected in 8 hospitals with interviews with 49 professionals, approximately two years after the beginning of the project in the hospitals. There were changes in the hospital routine and in the care of women after the project. The professionals incorporated practices such as skin-to-skin contact and breastfeeding; diet during labor; non-invasive care technologies, especially to relieve pain during labor; birth plan; pregnancy courses with guided tours in hospitals (for pregnant women and family); and analgesia for vaginal labor. There was adaptation of existing practices in hospitals to reduce CS that had no clinical indication; better monitoring of labor, favoring vaginal delivery. And finally, the professionals rejected the practice that presses the uterine fundus, for not having shown efficacy in recent studies. We can conclude that the hospitals that participated in this study have made an effort to change their obstetric model. However, specific aspects of each hospital, the organization of the health system in Brazil, and the incentive of the local administration influenced the implementation of these changes by professionals in practice.

Introduction

With global rates of 21.1% and an increase of 12.1% in fifteen years, the use of cesarean section (CS) as an important resource to save lives transcends clinical aspects. The high rates of CS can be influenced by the different political, economic and governmental contexts of each country [ 1 ]. In Brazil, even with several investments in the field of obstetric and neonatal care within the Brazilian Unified Health System (SUS), few programs to improve maternal care have been created or expanded in partnership with the private sector [ 2 ], although the latter participates in a complementary manner to SUS [ 3 ]. As a result of this distance and of a specific cultural context, in 2016 the rate in the country was of 55.44%, with different proportions for the public sector (41%) and the private sector (83%) [ 4 ]. Such rates are far from the ideal recommended by WHO (World Health Organization), which is of 10% to 15% [ 5 , 6 ].

In this sense, the principles guided by the humanization of care and autonomy of women during the process of childbirth and birth [ 7 ] has become a struggle for the women’s social movement in Brazil, which through the Public Ministry filed a complaint of Public Civil Action [ 8 ] to the National Supplementary Health Agency (ANS), linked to the Ministry of Health (MH) and responsible for creating rules, controlling and inspecting the health insurance market in the country [ 9 ], so that there are effective measures for the reduction of unjustified CS in the private health sector.

In response, in 2015, the “Adequate Childbirth Program” (Projeto Parto Adequado—PPA), was launched with the objective of improving the quality of obstetric and neonatal care in the supplementary service, through institutional, scientific and methodological support to hospitals that wished to implement a reorganization regarding prenatal, delivery and puerperium care [ 10 ]. In addition to promoting the reduction of CS with no clinical indications, the project promotes a physiological and safe labor and birth process.

To make the intervention feasible, one of the guiding components included the reorganization of the care model, a challenge that carried out the modification and reinvention of care practices, paying attention to the best evidence in obstetrics. Such reorganization depends not only on the adherence of maternity hospitals to the intervention, but also on the adoption or rejection [ 11 ] of the implementation by professionals of women’s health care. This article aims to analyze the incorporation, adaptation and rejection of obstetric practices during the implementation of PPA, identified by the care professionals.

Recognizing the importance of the intervention PPA to women and newborns health care in Brazil, the National School of Public Health (ENSP) − Oswaldo Cruz Foundation (Fiocruz) proposed an evaluation entitled “Healthy Birth: a prospective study to assess the implementation and effects of multifaceted intervention to improve the quality of care during childbirth and births in hospitals in Brazil” in order to analyze the implementation and the effects of PPA in a sample of 12 hospitals, using mixed methods of analysis [ 10 ].

Integrated with this evaluation, this article develops an exploratory study with a qualitative data approach in an intentional subsample of eight hospitals (Hosp01; Hosp02; Hosp04; Hosp05; Hosp06; Hosp09; Hosp10; Hosp12). Inclusion criteria for this subsample were hospital location according to geographic macro-region (South/Southeast/Midwest and North/Northeast), type of hospital (hospitals owned or not owned by health insurance companies), and hospital performance according to administrative data provided by the PPA coordination board. Four hospitals (Hosp03; Hosp07; Hosp08; Hosp11) were excluded due to similarities in geographic location and type of management [ 10 ]. More details on data collection, contextual aspects, and protocols established by the “Healthy Birth” survey can be found in Torres et al. [ 10 ].

The training of the interviewers who carried out the data collection was carried out in two ways: in person for 02 (two) coaches and remotely for another 02 (two) coaches. In these trainings, there was a reading of the instrument, presentation of techniques for conducting interviews, field observations, and the importance of records.

Regarding the characterization of the 04 (four) interviewers who carried out the data collection: they were all women with academic training in Nursing, Midwifery, and History. The historians were researchers in public health; the nurse and the Midwife were researchers in women’s health; all had previous experience with data collection for research. These interviewees were drawn to the field mainly because they were already involved in women’s health research or care practice. They were presented as such to the interviewees.

Before entering the field for data collection, pilot tests of the scripts were carried out in a maternity hospital in Rio de Janeiro/Brazil, part of the Adequate Childbirth Project. However, this maternity hospital was excluded from the sample of hospitals in the Nascer Saudável evaluation. This phase made it possible to make adjustments and validate the interview scripts. Furthermore, the research team established a previous contact connection with the management of the institutions to agree on the dates on which the interviewers would be placed in the field for data collecting and any ethical issues that would be released.

The immersion for data collection in the hospitals took place during a period of five days, from July to October 2017, in each hospital, in which the research team conducted interviews with the professionals of direct assistance to women. Key informants from the clinical staff (doctors and nurses) were interviewed; they were indicated by the project leader in the hospital among those who were the most integrated and the least integrated to the PPA.

The selection process started with the managers, who nominated leaders and who later nominated the professionals who had, at the time, more and less participation in the project at the hospital (professionals who participated in the PPA alignment meetings and professionals who did not). Physicians and nurses who worked in labor and delivery care at each institution were considered eligible. This selection is called snowball sampling [ 12 ] a type of non-probabilistic sampling that focuses on reference chains and is commonly used in exploratory investigations, was used to select participants. A total of 49 interviewees were selected, being 21 doctors (26% women and 18% men) and 28 nurses (all women). The number of respondents was different for each hospital, as it considered the characteristics of each institution. There was no refusal or withdrawal from participating in the research by the interviewees and no need to repeat interviews.

The interviews were carried out in the hospitals, face to face, in a place that guaranteed the participant’s privacy and did not have a pre-established length of time, which varied according to the engagement in the PPA and the subjective characteristics of the interviewees. The interviews were recorded, transcribed by an independent professional, and reviewed by type of sampling by the research team. These transcripts weren’t given back to the participants for review, feedback, or edits. However, for the order of internal validation of the transcripts, there was a review by the research team. Field notes were taken at the end of all interviews in each hospital.

The interviews were guided by a semi-structured script, which included the following axes: decision and participation process; strategies; care practice; and results. This script was developed for Healthy Birth research [ 10 ]. With a focus on the care practice axis, the following aspects were explored: the change in the work routine and the way of assisting women in labor, after the beginning of the PPA implementation.

The term ‘saturation’ of qualitative data is incorporated when there are repetitions about the object studied, and it can then interrupt data collection [ 13 ]. This term brought ambivalence to researchers, some revealed the feeling of practicality, and others questioned it [ 14 ]. It is worth noting that qualitative research, unlike quantitative research, is not based on how many individuals should be heard, but on the intensity of the phenomenon and scope to which the actors are linked to the researched object. This reinforces the importance of refinement in the selection of respondents [ 15 , 16 ]. Therefore, it is believed that this thesis achieved what the two poles of scholars portray, as it reveals the refinement of the interviewees selected for the interviews, and at the same time, it used the minimum required by many authors, which is the contemplation of at least 20 to 30 interviews for any type of qualitative investigation [ 17 ].

Data were submitted to Thematic Content Analysis [ 18 ], using the MAXQDA software, version 2020.3 [ 19 ]. The interviews were imported into the software, organized and encrypted according to their respective hospital and professional category. The encryption used was identified only in the research dictionary, for greater security with regard to the anonymity of the interviewees. The open categorization was then carried out, generating broad segments. With the refinement of these segments, a list of codes was generated, a step called axial coding. Starting from this list of codes, an inductive association was made to create the categories. The coding tree is available in Table  1 .

Although there was no feedback from the participants regarding the findings, for the interpretation of the data, the interviewees’ speeches and their respective coding were validated by members of the research group, in which there were also reflexive co-participations on the interpretive procedures of the entire analytical phase [ 18 ]. All citations in this article only include the professional position held by the interviewee, to avoid possible identification.

As a methodological guide for qualitative research, it used the consolidated criteria for reporting qualitative research (COREQ) [ 20 ].

This research was approved by the Research Ethics Committee, of the National School of Public Health Sérgio Arouca, of the Oswaldo Cruz Foundation (Escola Nacional de Saúde Pública Sérgio Arouca – ENSP/Fiocruz), CAAE opinion: 1.761. 027, on January 16, 2017. The research participants were informed and confirmed their interest in participating by signing the Free and Informed Consent Form.

The analysis of topics “change in the work routine” and “way of assisting women in labor” after the beginning of the PPA produced three analytical dimensions of the process of change in the care model, described below and synthesized in Fig.  1 :

Incorporation of care practices: understood as the practices that were included from the implementation of the PPA;

Adaptation of the care practices: understood as practices carried out prior to the implementation of the PPA, but which underwent modifications with the implementation of the project;

Rejection of care practices: understood as those practices that were abandoned or called into question whether or not they should be carried out by hospital professionals.

Synthesis of the process of changing the care model in private hospitals in Brazil, after the PPA

Incorporation of care practices

In this category, we describe the incorporation of care practices by maternity hospitals into the PPA implementation process. Moreover, according to the report of the professionals interviewed, these incorporations signified innovation.

Among the practices incorporated is the provision of: information, communication, and education; birth plan; courses for pregnant women and families with guided visits to maternity hospitals; diet during childbirth; skin-to-skin contact and breastfeeding; non-invasive care technologies; and analgesia for women’s vaginal labor.

Several strategies were used to offer information, communication, and education to women by different professionals and at various moments of their care, which tends to provide subsidies for the strengthening of women’s autonomy and shared decisions, including the reduction of CS for professional convenience.

“When the patient was indicated by her doctor, then it was much more difficult. Today she comes, we already know about the Project, we try to guide her the best possible way. Many come with the idea of having a CS, because their doctor tried to induce it.” (Hosp05_Doctors)

“So much so, that it is not for nothing that we have increased our number of vaginal births. Also because pregnant women are so much better informed. With this Adequate Childbirth Program they already get there with the birth plan, they already give it to the doctor. So the doctor will not arrive at the door and will indicate cesarean because he already sees that she is informed. And the doctor’s posture changes a lot when he sees that the pregnant woman is already informed, that she is empowered, it is different than the one that comes without information. So it has certainly changed.” (Hosp05_Nurses)

Courses for pregnant women with guided visits to the maternity hospitals was another strategy, because it provides subsidies for the recognition by the woman and family of the hospital environment and the health team, in an approach prior to delivery. In addition, the professionals use this space in the course to publicize the maternity hospital’s participation in the PPA, also using it as a marketing strategy to attract customers.

“[…] Our course is not a one-day course. It is a very dynamic course, rich in information. It ends with the guided visit [to the maternity hospital]. It is a monthly course.” (Hosp02_Nurses)

“We also have a monthly pregnancy course, and in this pregnancy course I take the opportunity to publicize the project. So, before finishing the course, I always talk about the PPA, […] what it is and what its purpose is.” (Hosp02_Nurses)

Another strategy carried out by hospitals was the incorporation, encouragement and acceptance of the birth plan made by women. The understanding of the importance of women’s choices during labor and delivery tends to change the focus of attention in obstetrics within the private health sector in Brazil. Currently, the power relationship between medical professionals and women still prevails, which ends up largely disregarding the decisions and planning of women during the gestational and parturient process.

“Obviously, we realize that it is improving. The birth plans are being more respected, in the past they did not exist, and now there is the birth plan for this project. So the least possible intervention is beneficial for the evolution of this labor.” (Hosp 05_Doctors)

“We have a birth plan, which we deliver to patients when they are hospitalized […]. So, when the patient goes into labor and when she is able to answer our questionnaire, we already have this possibility. Every hour, at any time, if we have to do any procedure that escapes our routine a little, the patient will be guided, will be asked at all times. At no time do we take any approach that is not beneficial to the patient, unless there is some risk for the mother and the baby. Then we say: you have to do this procedure, because of this, this and that. But the patient is approached at all times in relation to this.” (Hosp 12 _Nurses)

The diet during labor was described as incorporated by only one professional from one of the eight maternity hospitals, which suggests, considering the context of the country, that there is already an institutionalization of this practice by the other hospitals.

“[…] diet, for example, used to leave every patient in labor on zero diet. Not today. A bland diet is released. So, I can see that this is changing.” (Hosp 02_Doctors)

There was also the introduction of non-invasive care technologies by obstetric nurses, strengthened by the adaptation of the environment and purchase of supplies to be offered during labor and delivery, in addition to being encouraged by medical professionals. These adaptations of the birth environment even modify the logic of control of the bodies for a new model of care, centered on women’s wishes.

“It changed, in the sense that we have more structure to conduct this delivery. So, we have the ball, we have the horse, which they didn’t have before, so when it comes to conducting more effective labor, I think it is favored too.” (Hosp10_Nurses)

“Bath therapy, which previously was almost not talked about, the usual well-being maneuvers in terms of pain.” (Hosp 05_Doctors)

“[…] there are some [women] who just lie down on the stretcher to induce with medication, but then they don’t even lie down, they keep walking, on the ball, on the stool, so it’s different, very different.” (Hosp 02_Nurses)

Another practice adopted was skin-to-skin contact and breastfeeding. Although already well established in Brazilian public institutions, in the private health sector scenario these practices were still not widespread and experienced by women.

“[…] I come from a school where we picked up the baby right away and then, I had it at the time, you know, I aspirated, until the baby had contact with the mother again it would take one hour, at least. So, I completely changed my approach, adapting myself precisely to the rules of PPA. The baby is born, he was born well, let his mother in skin contact. It often goes to the breast before we do the care. So, without any doubt, it completely changed the approach.” (Hosp 02_Doctors)

“So now, this mother / baby contact, breastfeeding is especially important, we are orienting them to breastfeeding, positioning the baby. This is very important. When I came, in the beginning, for pre-delivery, we didn’t have that much about breastfeeding, you know, for an hour.” (Hosp 09_Nurses)

Birth analgesia is between the process of incorporation and adaptation of the practice (Fig.  1 ), since it had already been incorporated in some maternity hospitals and in others only after the PPA. The availability of anesthesiologists on duty was a major change indicated by professionals to promote the increase of vaginal deliveries in the institutions. Before the PPA, it seemed to be up to the obstetrician to bring or not the anesthesiologist to his team, and consequently to make this practice available to women.

“I think what changed the most was this possibility of analgesia, which is still not possible so soon, but with seven (centimeters of cervical dilation) yes, that already helped a lot. Maybe this analgesia also generates a little more stress, because we know that sometimes you can have fetal bradycardia, so this gives a certain discomfort, but that we keep following.” (Hosp 12_Doctors)

Adaptation of care practices

In hospitals where analgesia for vaginal childbirth was already incorporated, the need for adaptations was noted in the professionals’ speeches. For labor analgesia, the most frequently mentioned adaptation related the professionals’ difficulty in valuing the pain threshold and the mother’s desire to indicate labor analgesia, while the cervical dilation score was what substantiated the indication. This strengthens the hypothesis that adaptations to practices may be more difficult to be modified because they are already part of the daily routine and the intrinsic way of doing things of each professional.

“[…] most [professionals] do analgesia only with total dilation, or when it’s very close to giving birth. Obviously, there are some cases that do with seven (centimeters of cervical dilation), but this is the minority of cases. So, when she does analgesia, the patient is already in the room to get a baby. Usually, they put Oxytocin, even if she was not conducting the delivery with Oxytocin before, usually they put it in the room, because when she does analgesia, the contraction is lowered and she has this difficulty to feel because of the block [..].” (Hosp 09_Nurses)

“So, we also have a lot of work, because it is like this: at first it is the obstetrician who decides the analgesia. It is not the woman. It’s not the woman’s pain, it’s the obstetrician. […] This is being changed too.” (Hosp 05_Nurses)

Regarding the use of the partograph, a tool that promotes a follow-up of labor, it has been incorporated more frequently during the PPA. However, at the time of the implementation of the PPA, the recommendation used was not the most recent in the literature. This allows us to reflect on the benefits of the insertion of this practice, but with the need to adapt to the best evidence. This reflection on adaptation is raised by the professionals.

“The use of the partograph for assistance in labor was significantly higher.” (Hosp 04_Doctors)

“[…] we still have a tool that is obligatory, and we must fill in, which is the partograph that I think is a great advance and is one of the tools of PPA, but it is still in the model of the WHO, which I think should be updated […]. Because today is it like this: every two hours you have to play it. It is a guidance. There are already some guidelines that guide it for every four hours, but less than four hours there is no need. I joke with the nurses: the finger doesn’t have an eye.” (Hosp 05_Doctors)

The perspective of CS indications was also modified after the PPA. It included mechanisms such as indicating elective CS to be performed only above 39 weeks of gestational age, requiring professional justification for performing elective CS below 39 weeks, and the adoption of a consent form for women to be informed and authorize the surgery.

“[…] there is a whole training for this. And it’s barred since the call, when the doctor calls to schedule [the CS], always ask: elective CS, how many weeks? But thirty-seven weeks, why? Why are you asking for a vacancy (to perform the surgery)?” (Hosp 01_Doctors)

“There has been a change. There is a CS form that the patient has to sign, or the husband.” (Hosp 02_Nurses)

In order to reduce the number of CS and promote vaginal birth there were consequently more cases of induction of labor. Therefore, it has been necessary, for example, to acquire specific drugs for induction, such as Misoprostol.

“[…] I can mention the increase in the rate of labor induction for patients, instead of CS.” (Hosp 04_Doctors)

“A patient who arrived with an unfavorable Bishop score […] outside of labor, sometimes ended up in a CS, because we didn’t have this device [Misoprostol]. Now, having it, improved.” (Hosp 09_Doctors)

In this sense, the routinely use of synthetic Oxytocin also changed. Professionals of most hospitals reported a change in culture, for use only in cases of induction of labor, in the postpartum, and for cases of postpartum hemorrhage. However, two professionals of Mat06, of all the interviewees, expressed that this practice has not changed after the PPA in the institution where they work.

“A lot of changes. Today Oxytocin is used is for an induction, if it’s a postpartum, or something. […] Use Oxytocin to accelerate, there’s no more of that.” (Hosp 05_Nurses)

“But then, when i was hired by the hospital, Oxytocin use was routine. Does not do a dynamic and it already starts with Oxytocin. I didn’t have the right protocols to follow, you know? But you realize that this is already changing.” (Hosp 05_Nurses)

“Oxytocin, […] I believe it hasn’t really changed, because I think that those who used it continue to use it.” (Hosp 06_Doctors)

The practice of episiotomy is still performed in all hospitals. However, according to the reports, there has been a decrease in its use, with most professionals, according to them, using it in a “selective” way. Even if slowly, professionals began to question the use of episiotomy, since there is no scientific evidence of its effectiveness. However, it is still far from a unanimous acceptance.

“Episiotomy I was already using, selective episiotomy. When I was doing residency, we did seventy percent episiotomy. Now, today, we do twenty to thirty percent of episiotomy. So there is already a selection.” (Hosp 06_Doctors)

“Episiotomy, it is always at the discretion and when we see that there is a need, we ask the patient: do you authorize me to do the episiotomy? This is very controversial, in fact, because currently there are people who say they don’t really need it, anyway, but when it is done, it is done with discretion and asking: do you authorize?” (Hosp 09_Doctors)

“I no longer see a doctor doing an episiotomy.” (Hosp 04_Nurses)

“Performing episiotomy, is a maneuver no longer used by our entire team.” (Hosp 04_Doctors)

Rejection of care practices

Most professionals have reported abandoning the practice of Kristeller. However, for the few who still mention the use of this practice, it has remnants of a need to take possession of the female body. This category demonstrates how difficult it is to let go of obsolete practices, which still conform a violent obstetric care, less woman-centered and exercised without scientific base. Therefore, we understand that the (de)construction of practices seems to be daily and continuous in hospitals.

“Kristeller has been abolished. We still used it in some situations, before the project and today it has been abolished. I never saw it again […]. We started talking, like this, slowly, asking not to do it, in a nice way.” (Hosp 06_Nurses)

“Kristteller, I think I only saw it once. Because help was needed. But he (the doctor) talked to the woman.” (Hosp 05_Nurses)

Below (Fig.  1 ) we present a synthesis of the results with the analytical categories of incorporation, adaptation and rejection of practices for the change of the obstetric model in private hospitals after the implementation of PPA.

The model of obstetric care in force in Brazil is still predominantly marked by obsolete interventionist care practices that, for the most part, do not value women’s wishes and expectations. An epidemiological survey conducted in the Brazilian territory, entitled “Born in Brazil”, in 2011/2012, found an excess of unnecessary obstetric practices during labor and delivery, with only 5% of women at usual risk without any intervention during the birth process [ 21 ]. This finding reinforces the need to change the obstetric care model in force in the country, especially in the private sector. Thus, the Adequate Childbirth Program encourages and promotes the incorporation, adaptation, and rejection of practices based on the best evidence in private hospitals in Brazil.

Thus, the intrinsic innovation of PPA is in the promotion of this new model of care, which corresponds to the transformation of the scenario, using technologies, processes, ways of caring and managing [ 22 ] in obstetric health institutions. Innovations can be carried out incrementally to what already existed in the context, or they can be implemented radically, excluding what existed before [ 23 ]. In the case of hospitals inserted in the PPA, we note that the change of model, still in a continuous process, occurs incrementally, as shown in Fig.  1 .

In the context of each hospital inserted in the PPA study presented in this work, there was the incorporation of practices that were not yet institutionalized, and one of them was the promotion of information, communication, and education in health [ 24 ]. WHO’s guidelines recommends that there is clear communication between health professionals and parturient, thus ensuring the recognition of their rights regarding the entire birth process [ 25 ]. However, a study conducted in Brazil in 2020 revealed that women still remain uninformed about the benefits and harms of obstetric care practices, which ends up hindering their autonomy at the time of labor and the claims of their rights [ 26 ].

Corroborating the WHO guidelines [ 25 ] and keeping in mind that this is an ongoing process, the initiative of hospitals to offer health education, such as courses for pregnant women, maternity tours, and stimulus to the construction of the birth plan, exemplifies strategies to increase information, knowledge of women and better prepare them for the birthing process, and this includes inhibiting medically unjustified CS.

The incorporation of the use of practices with Non-Invasive Technologies of Care [ 27 ] goes through a change in professional behavior, in which the care relationship is modified and becomes shared, centered on the woman and the family. Such incorporation modifies the whole perspective of the current model, because in this new model, the skills and competences of the professional who assists the woman enhance her femininity in the scenario of labor and birth. In this context, in a sensitive and individual way, the development of non-invasive technologies of care involves relational attitudes, professional knowledge in obstetrics and the offer of objects and procedures such as the ball, the horse, therapeutic bath, music therapy, among others [ 27 , 28 ].

This corroborates the results of this study, which shows that professionals perceive the woman’s satisfaction when using these technologies. This path of change must be built daily in women’s care settings, because many are still unaware of the importance of freedom of position during labor [ 29 ], besides the possibility of giving birth in vertical positions, as well as the possibility of feeding during labor.

Proper monitoring of labor is essential for safe maternal and neonatal outcomes. The use of partographs as a labor monitoring tool is a strategy encouraged by health agencies worldwide. However, in recent years, the partograph’s warning and action lines have been questioned as to their effectiveness because not all women progress 1 cm/hour during the first stage of labor. Some women progress more slowly. Recognizing the limitations of this tool and proposing the creation of an assessment that can better individualize women, the WHO recommendation is to continue to use the partograph’s warning and action lines and its other aspects of labor assessment, but for women with suspected slower labor progression, it should be ensured that their physical and psychological needs are being met, to avoid misdiagnosis of cephalo-pelvic disproportion [ 25 ].

In a review on the use of partography during work, the lack of training of professionals to use the instrument, the lack of time to execute it properly, as well as the lack of institutional protocols to guide the conduct on the results of the partography were highlighted [ 30 ]. Given this and the reports of professionals, clinical updates will be necessary for the use of partography in hospitals inserted in the PPA.

Another recommended practice for pain relief, in healthy women who request its use, is pharmacological labor analgesia [ 25 ]. In Brazil, this right is guaranteed by the ordinances of the Ministry of Health NR2815/1998 and NR572/2000 [ 31 , 32 ]. Regional analgesia by continuous or combined epidural technique with reduced doses is considered the gold standard [ 32 ], mainly because it provides effective pain relief, without removing the woman’s participation during labor, it avoids maternal hyperventilation and increases maternal satisfaction [ 33 ].

Studies on the association of pharmacological analgesia for vaginal delivery with unfavorable neonatal outcomes showed that it can increase by about 3 times the chance of Apgar score of the newborn in the first minute to less than 7. However, in the Apgar score of the fifth minute there was no such association, which may be related to the preparation of the team in relation to resuscitation maneuvers [ 32 , 34 ]. Thus, the concern of the professionals interviewed is valid, when they understand that there is maternal satisfaction due to pain relief; however, there is also the need to prepare the team for neonatal care, when necessary.

Globally, CS rates tend to be higher in countries with more doctors in the territory; and in women living in large urban centers, with higher levels of economic development, with higher education and with lower fertility rates. Comparing wealthy women, especially those who give birth in private maternity hospitals, one notices higher rates of CS compared to poorer women [ 1 ]. In a local study, women more likely to have CS are those with higher levels of education, who live in places with better living conditions, who attend prenatal care, who are over 20 years old, and who have partners. In Brazil, this happens not only because of issues related to access to health services, but also because of contextual aspects of the health system organization, the obstetric care model, and sociocultural factors [ 4 ].

In this sense, even with the adjustment to the Brazilian population of the CS rate to about 25–30%, through the WHO C-Model [ 32 ], the development of various strategies in the field of obstetrics is still necessary to reduce the high rates of CS [ 35 ]. Understanding this, the hospitals inserted in the PPA started with good strategies to decrease CS rates that can serve as an example for other institutions.

Skin-to-skin contact regulates and maintains the baby’s body temperature and cardiorespiratory stability soon after birth. This practice provides opportunities for breastfeeding in the first postpartum hour and tends to strengthen the emotional bond between mother and baby, and short episodes of crying and signs of stress in the newborn [ 36 , 37 , 38 ]. Moreover, skin-to-skin contact also stimulates the newborn to spontaneously seek the breast and start early breastfeeding.

This last practice is encouraged by the World Health Organization and the United Nations Children’s Fund (UNICEF) when it establishes the Ten Steps for Successful Breastfeeding as the basis for the Baby-Friendly Hospital Initiative [ 39 ], by promoting maternal and neonatal benefits. Among the immediate benefits are the stimulation of maternal endogenous oxytocin release, which favors uterine contraction, reducing postpartum bleeding, and consequently, reducing maternal postpartum hemorrhage [ 36 , 38 ]. The incorporation of the practices of skin-to-skin contact and breastfeeding by the professionals of hospitals inserted in the PPA, also reinforces the change in the care model, centering care on the woman and the newborn.

With regard to episiotomy, the literature recommends its selective use in women with spontaneous vaginal delivery. However, as there is no evidence to support the effectiveness of its use, even if selectively, the most recent WHO manual preferred to emphasize that its routinely use is not recommended. Moreover, it reinforces that in cases where the procedure must be performed, local anesthesia and the woman’s consent are essential [ 25 ]. In Brazil, the use of episiotomy is still extremely high, varying from institution to institution, but there is a higher frequency of this practice especially in primiparous women and more frequently in the private sector, even if the woman is considered of usual risk [ 21 ].

WHO’s recommendations [ 25 ] highlight the evidence that women are afraid of interventions such as episiotomy and that when there is a need for this practice to be “indicated”, such women would like to have more information about the procedure, in addition to it being performed by a competent professional. Furthermore, most women do not have the necessary information about why the procedure should be performed and what care should be taken afterwards, and they rarely have their permission requested [ 25 ]. The speech of hospital professionals included in the PPA reinforces WHO’s proposal, but still in a transition phase between the adequacy of the practice and its rejection. This path of change is arduous and difficult to achieve due to the fact that this practice in considered as normal in Brazilian institutions, since the idea that episiotomy facilitates childbirth still exists.

The use of the Kristeller maneuver at the time of fetal expulsion is contrary to the best evidence, suggesting that there is little or no difference in the time of fetal expulsion when pressure is applied or not to the uterine fundus. In addition, low certainty evidence suggests that women who receive the Kristeller maneuver may experience more pain after birth, and that birth trauma, including fractures and bruising, may occur. Furthermore, concerns regarding the practice of uterine pressure are due to the possibility of serious harm such as uterine and other organ rupture, and maternal and perinatal death. The use of such practice without seeking the woman’s consent may be considered an abuse of human rights [ 25 ].

Between 2011 and 2017, there was a considerable increase in the proportion of women who had access to technologies for labor and delivery (presence of a companion, attendance by an obstetric nurse, use of partogram, use of non-pharmacological methods, ambulation during labor and delivery, feeding and position for delivery) and decline in practices such as peripheral venous catheter use, episiotomy, and Kristeller maneuver. The private sector also observed declining cesarean rates and increasing gestational age at birth. The study’s findings, which compared two improvement programs in the Brazilian obstetric industry, demonstrate that well-designed public policies can change how care is provided during labor and delivery, helping to reduce adverse outcomes for mothers and newborns [ 40 ].

Initiatives to increase the information and participation of women and to modify hospital routines can be strategies to reduce cesarean sections, used mainly in high-income countries, such as changing the form of reimbursement to health institutions and organizations; and creating or strengthening mechanisms for legislatures and politicians to bar medically unjustified elective cesareans and stimulus to the construction of the birth plan [ 40 , 41 , 42 , 43 , 44 ].

The lack of proportion of participants in “more integrated” and “less integrated” to the PPA can be considered a methodological limitation of the study. In addition, not known whether such changes presented in the results also happened due to other political, legislative, and women’s movement interventions that may have occurred concomitantly in the country or health institutions. Future studies that reflect the implementation of the PPA from other perspectives, such as the voices of maternity heads and women, are necessary. Many of these articles are already in the process of being published by the research team.

After the PPA, there were adjustments made to the hospital’s routine and the care given to women. Skin-to-skin contact, breastfeeding, food during labor, non-invasive care technology, birth plans, prenatal with guided hospital visits (for expectant mothers and their families), and analgesia for vaginal delivery included by professionals. Hospitals modified their current procedures to decrease CS without a clinical reason and to better monitor labor and vaginal birth. And lastly, because recent research has shown that the practice of applying the Kristeller maneuver is ineffective, the professionals rejected it. We can draw the conclusion that the hospitals included in this study have tried to alter their obstetric model.

It is expected that these practices, which are constantly changing, will produce positive impacts on the obstetric care model and, consequently, on the safety and satisfaction of women in the labor and birth process. The contextual aspects of each hospital, the organization of the health system, and the management stimulus has influenced the process of change through the PPA implementation.

Therefore, the sustainability of such change transitions in a long-term culture modification, professional training, continuous monitoring and evaluation of clinical practice, in addition to the influence of hospital management in the face of innovation. It is concluded that the professionals interviewed reported a reorganization of the ways to know and care for obstetrics after the implementation of the PPA.

Availability of data and materials

The datasets used during the current study are available upon request at: Leal, Maria do Carmo (Coord.), 2023, "Nascer Saudável: estudo prospectivo de avaliação da implantação e dos efeitos de intervenção multifacetada para melhoria da qualidade da atenção ao parto e nascimento", https://doi.org/10.35078/C1PSMZ , Arca Dados, V2, UNF:6:LQBoC/LmVRpwUe/UvaYCfQ== [fileUNF]

Abbreviations

National Supplementary Health Agency

Cesarean Section

National School of Public Health

Institute for Healthcare Improvement

Ministry of Health

Adequate Childbirth Program (“Projeto Parto Adequado”)

Unified Health System

World Health Organization

Boerma T, Ronsmans C, Melesse DY, Barros AJD, Barros FC, Juan L, et al. Global epidemiology of use of and disparities in caesarean sections. Lancet. 2018;392(10155):1341–8. https://doi.org/10.1016/S0140-6736(18)31928-7 .

Article   PubMed   Google Scholar  

ANS AN de SS. Histórico - Linha do tempo. Available at: http://www.ans.gov.br/aans/quem-somos/historico . Citado 13 de fevereiro de 2021.

Paim J, Travassos C, Almeida C, Bahia L, MacInko J. The Brazilian health system: History, advances, and challenges. Lancet. 2011;377(9779):1778–97. https://doi.org/10.1016/S0140-6736(11)60054-8 .

Occhi GM, de Lamare Franco Netto T, Neri MA, Rodrigues EAB, de Lourdes Vieira Fernandes A. Strategic measures to reduce the caesarean section rate in Brazil. Lancet. 2018;392(10155):1290–1. https://doi.org/10.1016/S0140-6736(18)32407-3 .

Appropriate technology for birth. Lancet. 1985;2(8452):436–7.

Betran AP, Torloni MR, Zhang J, et al. What is the optimal rate of caesarean section at population level? A systematic review of ecologic studies. Reprod Health. 2015;12:57. Published 2015 Jun 21. https://doi.org/10.1186/s12978-015-0043-6 .

Google Scholar  

Núcleo Técnico da Política Nacional de Humanização. HumanizaSUS - Política Nacional de Humanização: a humanização como eixo norteador das práticas de atenção e gestão em todas as instâncias do SUS. Bras Minist da Saúde. Brasília: Ministério da Saúde; 2004. p. 20.

BRASIL MPF. Justiça Federal da 3 a Região PJe - Número:5005407–46.2019.4.03.6100. 24 a Vara Cível Fed São Paulo. 2010. p. 2–5.

Pietrobon L, Lenise M, Caetano JC. Saúde suplementar no Brasil: o papel da Agência Nacional de Saúde Suplementar na regulação do setor Suplemental health in Brazil: the role of the National Agency of Suplemental Health in the sector’s regulation. Physis. 2008;18(4):767–83.

Article   Google Scholar  

Torres JA, Leal M do C, Domingues RMSM, Esteves-Pereira AP, Nakano AR, Gomes ML, et al. Evaluation of a quality improvement intervention for labour and birth care in Brazilian private hospitals: a protocol. Reprod Health. 2018l;15:1–11.

Rogers Everett M. Diffusion of innovations. New York. 1995;12:576

Vinuto J. A amostragem em bola de neve na pesquisa qualitativa: um debate em aberto. Temáticas. 2014;22(44):203–20.

Glaser BG, Strauss AL, Strutzel E. The discovery of grounded theory; strategies for qualitative research. Nurs Res. 1968;17(4):364.

Fontanella BJB, Ricas J, Turato ER. Amostragem por saturação em pesquisas qualitativas em saúde: contribuições teóricas. Cad Saude Publica. 2008;24:17–27.

de Souza Minayo MC. Amostragem e saturação em pesquisa qualitativa: consensos e controvérsias. Rev Pesqui Qual. 2017;5(7):1–12.

Onwuegbuzie AJ, Leech NL. Sampling designs in qualitative research: making the sampling process more public. Qual Rep. 2007;12(2):238–54.

Creswell JW, Poth CN. Qualitative inquiry and research design: choosing among five approaches. New York: Sage publications; 2016.

Flick U. An introduction to qualitative fourth edition. New York: SAGE Publ; 2009.

MAXQDA. MAXQDA - software for qualitative data analyses. VERBI software. Consult. Sozialforschung GmbH; 2020. Available from maxqda.com .

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042 .

Leal MC, Pereira APE, Domingues RMSM, Filha MMT, Dias MAB, Nakamura-Pereira M, et al. Intervenções obstétricas durante o trabalho de parto e parto em mulheres Brasileiras de risco habitual. Cad Saude Publica. 2014;30(SUPPL1):17–32.

Gadelha CAG, Temporão JG. Development, innovation and health: the theoretical and political perspective of the health economic-industrial complex. Cien Saude Colet. 2018;23(6):1891–902.

Learning M, Cookbook R. Relatório de Primavera 2008 - Sistema de Saúde Português: riscos e incertezas. Obs Port Sist Saúde. 2008.

Cardoso IS de AJM. Comunicação e Saúde. 1 a reimpre. Brazil: Editora Fiocruz; 2007. 152 (30–32).

World Health Organization. Intrapartum care for a positive childbirth experience. 2018. p. 212. Available at: http://apps.who.int/iris/bitstream/10665/260178/1/9789241550215-eng.pdf?ua=1%0Ahttp://www.who.int/reproductivehealth/publications/intrapartum-care-guidelines/en/ .

Vidal ÁT, Barreto JOM, Rattner D. Barreiras à implementação de recomendações ao parto normal no Brasil: a perspectiva das mulheres. Rev Panam Salud Pública. 2020;44:1.

do Nascimento NM, Progianti JM, Novoa RI, de Oliveira TR, Vargens OMC. Tecnologias não invasivas de cuidado no parto realizadas por enfermeiras: a percepção de mulheres. Esc Anna Nery. 2010;14(3):456–61.

Prata JA, Ares LPM, Vargens OMC, Reis CSC, Pereira AL, Progianti JM. Non-invasive care technologies: nurses’ contributions to the demedicalization of health care in a high-risk maternity hospital. Esc Anna Nery. 2019;23(2):1–8.

Niederauer C, Pedroso S, López LC. À margem da humanização? Experiências de parto de usuárias de uma maternidade pública de Porto Alegre-RS. 2017.

Ollerhead E, Osrin D. Barriers to and incentives for achieving partograph use in obstetric practice in low- and middle-income countries: a systematic review. BMC Pregnancy Childbirth. 2014;1–7;14:281. Available at: http://www.biomedcentral.com/1471-2393/14/281 .

Ministério da Saúde. Diretrizes Nacionais de Assistência ao Parto Normal. 2017. p. 53.

Brasil. Ministério da Saúde. Secretaria de Políticas de Saúde. Área Técnica de Saúde da Mulher. Parto, Aborto e Puerpério: Assistência Humanizada à Mulher. Rev Bras Ginecol Obstet. 2010;13:44–55. Available at: http://www.sciencedirect.com/science/article/pii/S0104423011704055%5Cn ,  http://www.revistas.usp.br/rlae/article/view/2392%5Cn , http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-72032014000400152&lng=en&nrm=iso&tlng=pt%5Cn ,  http://pepsic.bvsalud.org/s .

de Almeida Cunha A, da Costa Gribel GP, Palmiro A. Analgesia e anestesia farmacológica em Obstetrícia. Fem Publicação ofi cial da Fed Bras das Assoc Ginecol e Obs. 2020;48:555–60.

Silva YAP, et al. Obstetric analgesia in labor and its association with neonatal outcomes. Rev Bras Enferm. 2020;73(5):e20180757. https://doi.org/10.1590/0034-7167-2018-0757 . Epub 01 July 2020. ISSN 1984–0446. Accessed 10 Jul 2022.

Souza JP, Betran AP, Dumont A, De Mucio B, Gibbs Pickens CM, Deneux-Tharaux C, et al. A global reference for caesarean section rates (C-Model): a multicountry cross-sectional study. BJOG An Int J Obstet Gynaecol. 2016;123(3):427–36.

Article   CAS   Google Scholar  

Organization PAH. Beyond survival: integrated delivery care practices for long-term maternal and infant nutrition, health and development. 2nd ed. Washington, DC: Pan American Health Organization. https://iris.paho.org/handle/10665.2/3464 .

Abdala LG, da Cunha MLC. Contato pele a pele entre mãe e recém-nascido e amamentação na primeira hora de vida. Clin Biomed Res. 2018;38(4):356–60.

Widström AM, Brimdyr K, Svensson K, Cadwell K, Nissen E. Skin-to-skin contact the first hour after birth, underlying implications and clinical practice. Acta Paediatr Int J Paediatr. 2019;108(7):1192–204.

Organization WH, (UNICEF) the UNCF. Protecting, promoting and supporting breastfeeding in facilities providing maternity and newborn services: implementing the revised Baby-friendly Hospital Initiative 2018. Vol. 48, Pediatric Clinics of North America. Geneva; 2018. p. 475–483.

Leal MC, et al. Avanços na assistência ao parto no Brasil: resultados preliminares de dois estudos avaliativos. Cad Saúde Pública. 2019;35(7):e00223018. https://doi.org/10.1590/0102-311X00223018 . Epub 22 Jul 2019. ISSN 1678–4464. Acessado 10 Jul 2022.

Opiyo N, Young C, Requejo JH, Erdman J, Bales S, Betrán AP. Reducing unnecessary caesarean sections: scoping review of financial and regulatory interventions. Reprod Health. 2020;17(1):133. https://doi.org/10.1186/s12978-020-00983-y . PMID: 32867791; PMCID: PMC7457477.

Article   PubMed   PubMed Central   Google Scholar  

Chen I, Opiyo N, Tavender E, Mortazhejri S, Rader T, Petkovic J, Yogasingam S, Taljaard M, Agarwal S, Laopaiboon M, Wasiak J, Khunpradit S, Lumbiganon P, Gruen RL, Betran AP. Non-clinical interventions for reducing unnecessary caesarean section. Cochrane Database Syst Rev. 2018;9(9):CD005528. https://doi.org/10.1002/14651858.CD005528.pub3 . PMID: 30264405; PMCID: PMC6513634.

Santos FSR, Souza PA, Lansky S, Oliveira BJ, Matozinhos FP, Abreu ALN, Souza KV, Pena ED. Os significados e sentidos do plano de parto para as mulheres que participaram da Exposição Sentidos do Nascer. Cad Saúde Pública. 2019;35(6):ISSN 1678-4464. https://doi.org/10.1590/0102-311X00143718 .

Gomes RPC, Silva RDS, Oliveira DCC, Manzo BF, Guimarães GDL, Souza K VD. Plano de parto em rodas de conversa: escolhas das mulheres. Rev Mineira Enferm. 2017;21(1). https://doi.org/10.5935/1415-2762.20170043 .

Download references

Acknowledgements

Not applicable.

About this supplement

This article has been published as part of Reproductive Health Volume 20 Supplement 2, 2023: The Healthy Birth study: an evaluative research of the Adequate Childbirth Program. The full contents of the supplement are available online at https://reproductive-health-journal.biomedcentral.com/articles/supplements/volume-20-supplement-2 .

This work was supported by fundings from Centro Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Bill & Melinda Gates Foundation, Edital MCTI/CNPq/MS/SCTIE/Decit/Fundação Bill e Melinda Gates N ° 47/2014 and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES—Brazil) – Finance Code 001. The funders had no influence in the identification, design, conduct, and reporting of the analysis.

Author information

Authors and affiliations.

Faculty of Nursing, State University of Rio de Janeiro (UERJ), Rio de Janeiro, RJ, Brazil

Débora Cecília Chaves de Oliveira & Maysa Luduvice Gomes

Anna Nery School of Nursing, Federal University of Rio de Janeiro (UFRJ), Rio de Janeiro, RJ, Brazil

Andreza Rodrigues & Thamires Soares

Foundation Oswaldo Cruz (Fiocruz), Rio de Janeiro, RJ, Brazil

Lucia Regina de Azevedo Nicida

Institute for Healthcare Improvement, Rio de Janeiro, RJ, Brazil

Jacqueline Alves Torres

Department of Social Sciences at the National School of Public Health, Foundation Oswaldo Cruz (Fiocruz), Rio de Janeiro, RJ, Brazil

Elyne Montenegro Engstrom

You can also search for this author in PubMed   Google Scholar

Contributions

JAT, AR, MLG conceived and designed the study. AR, DCCO, LRAN collected the data. DCCO, MLG, TS, AR, LRAN, JAT, EME analyzed and interpreted the data. DCCO, drafted the manuscript in collaboration with MLG, TS, AR, LRAN, JAT, EME. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Débora Cecília Chaves de Oliveira .

Ethics declarations

Ethics approval and consent to participate.

The “Healthy Birth” study was approved by the research ethics committee of the Escola Nacional de Saúde Pública Sérgio Arouca/Fiocruz (CAAE research protocol: 1.761.027, approved on January 16, 2017). All participants received and signed the Free and Informed Consent Form prior to the interview and all precautions were taken to maintain the confidentiality of the information.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

de Oliveira, D.C.C., Gomes, M.L., Rodrigues, A. et al. Incorporation, adaptation and rejection of obstetric practices during the implementation of the “Adequate Childbirth Program” in Brazilian private hospitals: a qualitative study. Reprod Health 20 (Suppl 2), 189 (2022). https://doi.org/10.1186/s12978-024-01772-7

Download citation

Received : 21 May 2021

Accepted : 11 March 2024

Published : 17 April 2024

DOI : https://doi.org/10.1186/s12978-024-01772-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health evaluation
• Healthcare models
• Quality improvement

Reproductive Health

ISSN: 1742-4755

• General enquiries: [email protected]

• Research article
• Open access
• Published: 15 April 2024

What is quality in long covid care? Lessons from a national quality improvement collaborative and multi-site ethnography

• Trisha Greenhalgh   ORCID: orcid.org/0000-0003-2369-8088 1 ,
• Julie L. Darbyshire 1 ,
• Cassie Lee 2 ,
• Emma Ladds 1 &
• Jenny Ceolta-Smith 3  

BMC Medicine volume  22 , Article number:  159 ( 2024 ) Cite this article

822 Accesses

55 Altmetric

Metrics details

Long covid (post covid-19 condition) is a complex condition with diverse manifestations, uncertain prognosis and wide variation in current approaches to management. There have been calls for formal quality standards to reduce a so-called “postcode lottery” of care. The original aim of this study—to examine the nature of quality in long covid care and reduce unwarranted variation in services—evolved to focus on examining the reasons why standardizing care was so challenging in this condition.

In 2021–2023, we ran a quality improvement collaborative across 10 UK sites. The dataset reported here was mostly but not entirely qualitative. It included data on the origins and current context of each clinic, interviews with staff and patients, and ethnographic observations at 13 clinics (50 consultations) and 45 multidisciplinary team (MDT) meetings (244 patient cases). Data collection and analysis were informed by relevant lenses from clinical care (e.g. evidence-based guidelines), improvement science (e.g. quality improvement cycles) and philosophy of knowledge.

Participating clinics made progress towards standardizing assessment and management in some topics; some variation remained but this could usually be explained. Clinics had different histories and path dependencies, occupied a different place in their healthcare ecosystem and served a varied caseload including a high proportion of patients with comorbidities. A key mechanism for achieving high-quality long covid care was when local MDTs deliberated on unusual, complex or challenging cases for which evidence-based guidelines provided no easy answers. In such cases, collective learning occurred through idiographic (case-based) reasoning , in which practitioners build lessons from the particular to the general. This contrasts with the nomothetic reasoning implicit in evidence-based guidelines, in which reasoning is assumed to go from the general (e.g. findings of clinical trials) to the particular (management of individual patients).

Not all variation in long covid services is unwarranted. Largely because long covid’s manifestations are so varied and comorbidities common, generic “evidence-based” standards require much individual adaptation. In this complex condition, quality improvement resources may be productively spent supporting MDTs to optimise their case-based learning through interdisciplinary discussion. Quality assessment of a long covid service should include review of a sample of individual cases to assess how guidelines have been interpreted and personalized to meet patients’ unique needs.

Study registration

NCT05057260, ISRCTN15022307.

Peer Review reports

The term “long covid” [ 1 ] means prolonged symptoms following SARS-CoV-2 infection not explained by an alternative diagnosis [ 2 ]. It embraces the US term “post-covid conditions” (symptoms beyond 4 weeks) [ 3 ], the UK terms “ongoing symptomatic covid-19” (symptoms lasting 4–12 weeks) and “post covid-19 syndrome” (symptoms beyond 12 weeks) [ 4 ] and the World Health Organization’s “post covid-19 condition” (symptoms occurring beyond 3 months and persisting for at least 2 months) [ 5 ]. Long covid thus defined is extremely common. In UK, for example, 1.8 million of a population of 67 million met the criteria for long covid in early 2023 and 41% of these had been unwell for more than 2 years [ 6 ].

Long covid is characterized by a constellation of symptoms which may include breathlessness, fatigue, muscle and joint pain, chest pain, memory loss and impaired concentration (“brain fog”), sleep disturbance, depression, anxiety, palpitations, dizziness, gastrointestinal problems such as diarrhea, skin rashes and allergy to food or drugs [ 2 ]. These lead to difficulties with essential daily activities such as washing and dressing, impaired exercise tolerance and ability to work, and reduced quality of life [ 2 , 7 , 8 ]. Symptoms typically cluster (e.g. in different patients, long covid may be dominated by fatigue, by breathlessness or by palpitations and dizziness) [ 9 , 10 ]. Long covid may follow a fairly constant course or a relapsing and remitting one, perhaps with specific triggers [ 11 ]. Overlaps between fatigue-dominant subtypes of long covid, myalgic encephalomyelitis and chronic fatigue syndrome have been hypothesized [ 12 ] but at the time of writing remain unproven.

Long covid has been a contested condition from the outset. Whilst long-term sequelae following other coronavirus (SARS and MERS) infections were already well-documented [ 13 ], SARS-CoV-2 was originally thought to cause a short-lived respiratory illness from which the patient either died or recovered [ 14 ]. Some clinicians dismissed protracted or relapsing symptoms as due to anxiety or deconditioning, especially if the patient had not had laboratory-confirmed covid-19. People with long covid got together in online groups and shared accounts of their symptoms and experiences of such “gaslighting” in their healthcare encounters [ 15 , 16 ]. Some groups conducted surveys on their members, documenting the wide range of symptoms listed in the previous paragraph and showing that whilst long covid is more commonly a sequel to severe acute covid-19, it can (rarely) follow a mild or even asymptomatic acute infection [ 17 ].

Early publications on long covid depicted a post-pneumonia syndrome which primarily affected patients who had been hospitalized (and sometimes ventilated) [ 18 , 19 ]. Later, covid-19 was recognized to be a multi-organ inflammatory condition (the pneumonia, for example, was reclassified as pneumonitis ) and its long-term sequelae attributed to a combination of viral persistence, dysregulated immune response (including auto-immunity), endothelial dysfunction and immuno-thrombosis, leading to damage to the lining of small blood vessels and (thence) interference with transfer of oxygen and nutrients to vital organs [ 20 , 21 , 22 , 23 , 24 ]. But most such studies were highly specialized, laboratory-based and written primarily for an audience of fellow laboratory researchers. Despite demonstrating mean differences in a number of metabolic variables, they failed to identify a reliable biomarker that could be used routinely in the clinic to rule a diagnosis of long covid in or out. Whilst the evidence base from laboratory studies grew rapidly, it had little influence on clinical management—partly because most long covid clinics had been set up with impressive speed by front-line clinical teams to address an immediate crisis, with little or no input from immunologists, virologists or metabolic specialists [ 25 ].

Studies of the patient experience revealed wide geographical variation in whether any long covid services were provided and (if they were) which patients were eligible for these and what tests and treatments were available [ 26 ]. An interim UK clinical guideline for long covid had been produced at speed and published in December 2020 [ 27 ], but it was uncertain about diagnostic criteria, investigations, treatments and prognosis. Early policy recommendations for long covid services in England, based on wide consultation across UK, had proposed a tiered service with “tier 1” being supported self-management, “tier 2” generalist assessment and management in primary care, “tier 3” specialist rehabilitation or respiratory follow-up with oversight from a consultant physician and “tier 4” tertiary care for patients with complications or complex needs [ 28 ]. In 2021, ring-fenced funding was allocated to establish 90 multidisciplinary long covid clinics in England [ 29 ]; some clinics were also set up with local funding in Scotland and Wales. These clinics varied widely in eligibility criteria, referral pathways, staffing mix (some had no doctors at all) and investigations and treatments offered. A further policy document on improving long covid services was published in 2022 [ 30 ]; it recommended that specialist long covid clinics should continue, though the long-term funding of these services remains uncertain [ 31 ]. To build the evidence base for delivering long covid services, major programs of publicly funded research were commenced in both UK [ 32 ] and USA [ 33 ].

In short, at the time this study began (late 2021), there appeared to be much scope for a program of quality improvement which would capture fast-emerging research findings, establish evidence-based standards and ensure these were rapidly disseminated and consistently adopted across both specialist long covid services and in primary care.

Quality improvement collaboratives

The quality improvement movement in healthcare was born in the early 1980s when clinicians and policymakers US and UK [ 34 , 35 , 36 , 37 ] began to draw on insights from outside the sector [ 38 , 39 , 40 ]. Adapting a total quality management approach that had previously transformed the Japanese car industry, they sought to improve efficiency, reduce waste, shift to treating the upstream causes of problems (hence preventing disease) and help all services approach the standards of excellence achieved by the best. They developed an approach based on (a) understanding healthcare as a complex system (especially its key interdependencies and workflows), (b) analysing and addressing variation within the system, (c) learning continuously from real-world data and (d) developing leaders who could motivate people and help them change structures and processes [ 41 , 42 , 43 , 44 ].

Quality improvement collaboratives (originally termed “breakthrough collaboratives” [ 45 ]), in which representatives from different healthcare organizations come together to address a common problem, identify best practice, set goals, share data and initiate and evaluate improvement efforts [ 46 ], are one model used to deliver system-wide quality improvement. It is widely assumed that these collaboratives work because—and to the extent that—they identify, interpret and implement high-quality evidence (e.g. from randomized controlled trials).

Research on why quality improvement collaboratives succeed or fail has produced the following list of critical success factors: taking a whole-system approach, selecting a topic and goal that fits with organizations’ priorities, fostering a culture of quality improvement (e.g. that quality is everyone’s job), engagement of everyone (including the multidisciplinary clinical team, managers, patients and families) in the improvement effort, clearly defining people’s roles and contribution, engaging people in preliminary groundwork, providing organizational-level support (e.g. chief executive endorsement, protected staff time, training and support for teams, resources, quality-focused human resource practices, external facilitation if needed), training in specific quality improvement techniques (e.g. plan-do-study-act cycle), attending to the human dimension (including cultivating trust and working to ensure shared vision and buy-in), continuously generating reliable data on both processes (e.g. current practice) and outcomes (clinical, satisfaction) and a “learning system” infrastructure in which knowledge that is generated feeds into individual, team and organizational learning [ 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ].

The quality improvement collaborative approach has delivered many successes but it has been criticized at a theoretical level for over-simplifying the social science of human motivation and behaviour and for adopting a somewhat mechanical approach to the study of complex systems [ 55 , 56 ]. Adaptations of the original quality improvement methodology (e.g. from Sweden [ 57 , 58 ]) have placed greater emphasis on human values and meaning-making, on the grounds that reducing the complexities of a system-wide quality improvement effort to a set of abstract and generic “success factors” will miss unique aspects of the case such as historical path dependencies, personalities, framing and meaning-making and micropolitics [ 59 ].

Perhaps this explains why, when the abovementioned factors are met, a quality improvement collaborative’s success is more likely but is not guaranteed, as a systematic review demonstrated [ 60 ]. Some well-designed and well-resourced collaboratives addressing clear knowledge gaps produced few or no sustained changes in key outcome measures [ 49 , 53 , 60 , 61 , 62 ]. To identify why this might be, a detailed understanding of a service’s history, current challenges and contextual constraints is needed. This explains our decision, part-way through the study reported here, to collect rich contextual data on participating sites so as to better explain success or failure of our own collaborative.

Warranted and unwarranted variation in clinical practice

A generation ago, Wennberg described most variation in clinical practice as “unwarranted” (which he defined as variation in the utilization of health care services that cannot be explained by variation in patient illness or patient preferences) [ 63 ]. Others coined the term “postcode lottery” to depict how such variation allegedly impacted on health outcomes [ 64 ]. Wennberg and colleagues’ Atlas of Variation , introduced in 1999 [ 65 ], and its UK equivalent, introduced in 2010 [ 66 ], described wide regional differences in the rates of procedures from arthroscopy to hysterectomy, and were used to prompt services to identify and address examples of under-treatment, mis-treatment and over-treatment. Numerous similar initiatives, mostly based on hospital activity statistics, have been introduced around the world [ 66 , 67 , 68 , 69 ]. Sutherland and Levesque’s proposed framework for analysing variation, for example, has three domains: capacity (broadly, whether sufficient resources are allocated at organizational level and whether individuals have the time and headspace to get involved), evidence (the extent to which evidence-based guidelines exist and are followed), and agency (e.g. whether clinicians are engaged with the issue and the effect of patient choice) [ 70 ].

Whilst it is clearly a good idea to identify unwarranted variation in practice, it is also important to acknowledge that variation can be warranted . The very act of measuring and describing variation carries great rhetorical power, since revealing geographical variation in any chosen metric effectively frames this as a problem with a conceptually simple solution (reducing variation) that will appeal to both politicians and the public [ 71 ]. The temptation to expose variation (e.g. via visualizations such as maps) and address it in mechanistic ways should be resisted until we have fully understood the reasons why it exists, which may include perverse incentives, insufficient opportunities to discuss cases with colleagues, weak or absent feedback on practice, unclear decision processes, contested definitions of appropriate care and professional challenges to guidelines [ 72 ].

Research question, aims and objectives

Research question.

What is quality in long covid care and how can it best be achieved?

To identify best practice and reduce unwarranted variation in UK long covid services.

To explain aspects of variation in long covid services that are or may be warranted.

Our original objectives were to:

Establish a quality improvement collaborative for 10 long covid clinics across UK.

Use quality improvement methods in collaboration with patients and clinic staff to prioritize aspects of care to improve. For each priority topic, identify best (evidence-informed) clinical practice, measure performance in each clinic, compare performance with a best practice benchmark and improve performance.

Produce organizational case studies of participating long covid clinics to explain their origins, evolution, leadership, ethos, population served, patient pathways and place in the wider healthcare ecosystem.

Examine these case studies to explain variation in practice, especially in topics where the quality improvement cycle proves difficult to follow or has limited impact.

The LOCOMOTION study

LOCOMOTION (LOng COvid Multidisciplinary consortium Optimising Treatments and services across the NHS) was a 30-month multi-site case study of 10 long covid clinics (8 in England, 1 in Wales and 1 in Scotland), beginning in 2021, which sought to optimise long covid care. Each clinic offered multidisciplinary care to patients referred from primary or secondary care (and, in some cases, self-referred), and held regular multidisciplinary team (MDT) meetings, mostly online via Microsoft Teams, to discuss cases. A study protocol for LOCOMOTION, with details of ethical approvals, management, governance and patient involvement has been published [ 25 ]. The three main work packages addressed quality improvement, technology-supported patient self-management and phenotyping and symptom clustering. This paper reports on the first work package, focusing mainly on qualitative findings.

Setting up the quality improvement collaborative

We broadly followed standard methodology for “breakthrough” quality improvement collaboratives [ 44 , 45 ], with two exceptions. First, because of geographical distance, continuing pandemic precautions and developments in videoconferencing technology, meetings were held online. Second, unlike in the original breakthrough model, patients were included in the collaborative, reflecting the cultural change towards patient partnerships since the model was originally proposed 40 years ago.

Each site appointed a clinical research fellow (doctor, nurse or allied health professional) funded partly by the LOCOMOTION study and partly with clinical sessions; some were existing staff who were backfilled to take on a research role whilst others were new appointments. The quality improvement meetings were held approximately every 8 weeks on Microsoft Teams and lasted about 2 h; there was an agenda and a chair, and meetings were recorded with consent. The clinical research fellow from each clinic attended, sometimes joined by the clinical lead for that site. In the initial meeting, the group proposed and prioritized topics before merging their consensus with the list of priority topics generated separately by patients (there was much overlap but also some differences).

In subsequent meetings, participants attempted to reach consensus on how to define, measure and achieve quality for each priority topic in turn, implement this approach in their own clinic and monitor its impact. Clinical leads prepared illustrative clinical cases and summaries of the research evidence, which they presented using Microsoft Powerpoint; the group then worked towards consensus on the implications for practice through general discussion. Clinical research fellows assisted with literature searches, collected baseline data from their own clinic, prepared and presented anonymized case examples, and contributed to collaborative goal-setting for improvement. Progress on each topic was reviewed at a later meeting after an agreed interval.

An additional element of this work package was semi-structured interviews with 29 patients, recruited from 9 of the 10 participating sites, about their clinic experiences with a view to feeding into service improvement (in the other site, no patient volunteered).

Our patient advisory group initially met separately from the quality improvement collaborative. They designed a short survey of current practice and sent it to each clinic; the results of this informed a prioritization exercise for topics where they considered change was needed. The patient-generated list was tabled at the quality improvement collaborative discussions, but patients were understandably keen to join these discussions directly. After about 9 months, some patient advisory group members joined the regular collaborative meetings. This dynamic was not without its tensions, since sharing performance data requires trust and there were some concerns about confidentiality when real patient cases were discussed with other patients present.

How evidence-informed quality targets were set

At the time the study began, there were no published large-scale randomized controlled trials of any interventions for long covid. We therefore followed a model used successfully in other quality improvement efforts where research evidence was limited or absent or it did not translate unambiguously into models for current services. In such circumstances, the best evidence may be custom and practice in the best-performing units. The quality improvement effort becomes oriented to what one group of researchers called “potentially better practices”—that is, practices that are “developed through analysis of the processes of care, literature review, and site visits” (page 14) [ 73 ]. The idea was that facilitated discussion among clinical teams, drawing on published research where available but also incorporating clinical experience, established practice and systematic analysis of performance data across participating clinics would surface these “potentially better practices”—an approach which, though not formally tested in controlled trials, appears to be associated with improved outcomes [ 46 , 73 ].

Adding an ethnographic component

Following limited progress made on some topics that had been designated high priority, we interviewed all 10 clinical research fellows (either individually or, in two cases, with a senior clinician present) and 18 other clinic staff (five individually plus two groups of 5 and 8), along with additional informal discussions, to explore the challenges of implementing the changes that had been agreed. These interviews were not audiotaped but detailed notes were made and typed up immediately afterwards. It became evident that some aspects of what the collaborative had deemed “evidence-informed” care were contested by front-line clinic staff, perceived as irrelevant to the service they were delivering, or considered impossible to implement. To unpack these issues further, the research protocol was amended to include an ethnographic component.

TG and EL (academic general practitioners) and JLD (a qualitative researcher with a PhD in the patient experience) attended a total of 45 MDT meetings in participating clinics (mostly online or hybrid). Staff were informed in advance that there would be an observer present; nobody objected. We noted brief demographic and clinical details of cases discussed (but no identifying data), dilemmas and uncertainties on which discussions focused, and how different staff members contributed.

TG made 13 in-person visits to participating long covid clinics. Staff were notified in advance; all were happy to be observed. Visits lasted between 5 and 8 h (54 h in total). We observed support staff booking patients in and processing requests and referrals, and shadowed different clinical staff in turn as they saw patients. Patients were informed of our presence and its purpose beforehand and given the opportunity to decline (three of 53 patients approached did). We discussed aspects of each case with the clinician after the patient left. When invited, we took breaks with staff and used these as an opportunity to ask them informally what it was like working in the clinic.

Ethnographic observation, analysis and reporting was geared to generating a rich interpretive account of the clinical, operational and interpersonal features of each clinic—what Van Maanen calls an “impressionist tales” [ 74 ]. Our work was also guided by the principles set out by Golden-Biddle and Locke, namely authenticity (spending time in the field and basing interpretations on these direct observations), plausibility (creating a plausible account through rich persuasive description) and criticality (e.g. reflexively examining our own assumptions) [ 75 ]. Our collection and analysis of qualitative data was informed by our own professional backgrounds (two general practitioners, one physical therapist, two non-clinicians).

In both MDTs and clinics, we took contemporaneous notes by hand and typed these up immediately afterwards.

Data management and analysis

Typed interview notes and field notes from clinics were collated in a set of Word documents, one for each clinic attended. They were analysed thematically [ 76 ] with attention to the literature on quality improvement and variation (see “ Background ”). Interim summaries were prepared on each clinic, setting out the narrative of how it had been established, its ethos and leadership, setting and staffing, population served and key links with other parts of the local healthcare ecosystem.

Minutes and field notes from the quality improvement collaborative meetings were summarized topic by topic, including initial data collected by the researchers-in-residence, improvement actions taken (or attempted) in that clinic, and any follow-up data shared. Progress or lack of it was interpreted in relation to the contextual case summary for that clinic.

Patient cases seen in clinic, and those discussed by MDTs, were summarized as brief case narratives in Word documents. Using the constant comparative method [ 77 ], we produced an initial synthesis of the clinical picture and principles of management based on the first 10 patient cases seen, and refined this as each additional case was added. Demographic and brief clinical and social details were also logged on Excel spreadsheets. When writing up clinical cases, we used the technique of composite case construction (in which we drew on several actual cases to generate a fictitious one, thereby protecting anonymity whilst preserving key empirical findings [ 78 ]); any names reported in this paper are pseudonyms.

Member checking

A summary was prepared for each clinic, including a narrative of the clinic’s own history and a summary of key quality issues raised across the ten clinics. These summaries included examples from real cases in our dataset. These were shared with the clinical research fellow and a senior clinician from the clinic, and amended in response to feedback. We also shared these summaries with representatives from the patient advisory group.

Overview of dataset

This study generated three complementary datasets. First, the video recordings, minutes, and field notes of 12 quality improvement collaborative meetings, along with the evidence summaries prepared for these meetings and clinic summaries (e.g. descriptions of current practice, audits) submitted by the clinical research fellows. This dataset illustrated wide variation in practice, and (in many topics) gaps or ambiguities in the evidence base.

Second, interviews with staff ( n  = 30) and patients ( n  = 29) from the clinics, along with ethnographic field notes (approximately 100 pages) from 13 in-person clinic visits (54 h), including notes on 50 patient consultations (40 face-to-face, 6 telephone, 4 video). This dataset illustrated the heterogeneity among the ten participating clinics.

Third, field notes (approximately 100 pages), including discussions on 244 clinical cases from the 45 MDT meetings (49 h) that we observed. This dataset revealed further similarities and contrasts among clinics in how patients were managed. In particular, it illustrated how, for the complex patients whose cases were presented at these meetings, teams made sense of, and planned for, each case through multidisciplinary dialogue. This dialogue typically began with one staff member presenting a detailed clinical history along with a narrative of how it had affected the patient’s life and what was at stake for them (e.g. job loss), after which professionals from various backgrounds (nursing, physical therapy, occupational therapy, psychology, dietetics, and different medical specialties) joined in a discussion about what to do.

The ten participating sites are summarized in Table  1 .

In the next two sections, we explore two issues—difficulty defining best practice and the heterogeneous nature of the clinics—that were key to explaining why quality, when pursued in a 10-site collaborative, proved elusive. We then briefly summarize patients’ accounts of their experience in the clinics and give three illustrative examples of the elusiveness of quality improvement using selected topics that were prioritized in our collaborative: outcome measures, investigation of palpitations and management of fatigue. In the final section of the results, we describe how MDT deliberations proved crucial for local quality improvement. Further detail on clinical priority topics will be presented in a separate paper.

“Best practice” in long covid: uncertainty and conflict

The study period (September 2021 to December 2023) corresponded with an exponential increase in published research on long covid. Despite this, the quality improvement collaborative found few unambiguous recommendations for practice. This gap between what the research literature offered and what clinical practice needed was partly ontological (relating what long covid is ). One major bone of contention between patients and clinicians (also evident in discussions with our patient advisory group), for example, was how far (and in whom) clinicians should look for and attempt to treat the various metabolic abnormalities that had been documented in laboratory research studies. The literature on this topic was extensive but conflicting [ 20 , 21 , 22 , 23 , 24 , 79 , 80 , 81 , 82 ]; it was heavy on biological detail but light on clinical application.

Patients were often aware of particular studies that appeared to offer plausible molecular or cellular explanations for symptom clusters along with a drug (often repurposed and off-label) whose mechanism of action appeared to be a good fit with the metabolic chain of causation. In one clinic, for example, we were shown an email exchange between a patient (not medically qualified) and a consultant, in which the patient asked them to reconsider their decision not to prescribe low-dose naltrexone, an opioid receptor antagonist with anti-inflammatory properties. The request included a copy of a peer-reviewed academic paper describing a small, uncontrolled pre-post study (i.e. a weak study design) in which this drug appeared to improve symptoms and functional performance in patients with long covid, as well as a mechanistic argument explaining why the patient felt this drug was a plausible choice in their own case.

This patient’s clinician, in common with most clinicians delivering front-line long covid services, considered that the evidence for such mechanism-based therapies was weak. Clinicians generally felt that this evidence, whilst promising, did not yet support routine measurement of clotting factors, antibodies, immune cells or other biomarkers or the prescription of mechanism-based therapies such as antivirals, anti-inflammatories or anticoagulants. Low-dose naltroxone, for example, is currently being tested in at least one randomized controlled trial (see National Clinical Trials Registry NCT05430152), which had not reported at the time of our observations.

Another challenge to defining best practice was the oft-repeated phrase that long covid is a “diagnosis by exclusion”, but the high prevalence of comorbidities meant that the “pure” long covid patient untainted by other potential explanations for their symptoms was a textbook ideal. In one MDT, for example, we observed a discussion about a patient who had had both swab-positive covid-19 and erythema migrans (a sign of Lyme disease) in the weeks before developing fatigue, yet local diagnostic criteria for each condition required the other to be excluded.

The logic of management in most participating clinics was pragmatic: prompt multidisciplinary assessment and treatment with an emphasis on obtaining a detailed clinical history (including premorbid health status), excluding serious complications (“red flags”), managing specific symptom clusters (for example, physical therapy for breathing pattern disorder), treating comorbidities (for example, anaemia, diabetes or menopause) and supporting whole-person rehabilitation [ 7 , 83 ]. The evidentiary questions raised in MDT discussions (which did not include patients) addressed the practicalities of the rehabilitation model (for example, whether cognitive therapy for neurocognitive complications is as effective when delivered online as it is when delivered in-person) rather than the molecular or cellular mechanisms of disease. For example, the question of whether patients with neurocognitive impairment should be tested for micro-clots or treated with anticoagulants never came up in the MDTs we observed, though we did visit a tertiary referral clinic (the tier 4 clinic in site H), whose lead clinician had a research interest in inflammatory coagulopathies and offered such tests to selected patients.

Because long covid typically produces dozens of symptoms that tend to be uniquely patterned in each patient, the uncertainties on which MDT discussions turned were rarely about general evidence of the kind that might be found in a guideline (e.g. how should fatigue be managed?). Rather they concerned particular case-based clinical decisions (e.g. how should this patient’s fatigue be managed, given the specifics of this case?). An example from our field notes illustrates this:

Physical therapist presents the case of a 39-year-old woman who works as a cleaner on an overnight ferry. Has had long covid for 2 years. Main symptoms are shortness of breath and possible anxiety attacks, especially when at work. She has had a course of physical therapy to teach diaphragmatic breathing but has found that focusing on her breathing makes her more anxious. Patient has to do a lot of bending in her job (e.g. cleaning toilets and under seats), which makes her dizzy, but Active Stand Test was normal. She also has very mild tricuspid incompetence [someone reads out a cardiology report—not hemodynamically significant].

Rehabilitation guidelines (e.g. WHO) recommend phased return to work (e.g. with reduced hours) and frequent breaks. “Tricky!” says someone. The job is intense and busy, and the patient can’t afford not to work. Discussion on whether all her symptoms can be attributed to tension and anxiety. Physical therapist who runs the breathing group says, “No, it’s long covid”, and describes severe initial covid-19 episode and results of serial chest X-rays which showed gradual clearing of ground glass shadows. Team discussion centers on how to negotiate reduced working hours in this particular job, given the overnight ferry shifts. --MDT discussion, Site D

This example raises important considerations about the nature of clinical knowledge in long covid. We return to it in the final section of the “ Results ” and in the “ Discussion ”.

Long covid clinics: a heterogeneous context for quality improvement

Most participating clinics had been established in mid-2020 to follow up patients who had been hospitalized (and perhaps ventilated) for severe acute covid-19. As mass vaccination reduced the severity of acute covid-19 for most people, the patient population in all clinics progressively shifted to include fewer “post-ICU [intensive care unit]” patients (in whom respiratory symptoms almost always dominated), and more people referred by their general practitioners or other secondary care specialties who had not been hospitalized for their acute covid-19 infection, and in whom fatigue, brain fog and palpitations were often the most troubling symptoms. Despite these similarities, the ten clinics had very different histories, geographical and material settings, staffing structures, patient pathways and case mix, as Table  1 illustrates. Below, we give more detail on three example sites.

Site C was established as a generalist “assessment-only” service by a general practitioner with an interest in infectious diseases. It is led jointly by that general practitioner and an occupational therapist, assisted by a wide range of other professionals including speech and language therapy, dietetics, clinical psychology and community-based physical therapy and occupational therapy. It has close links with a chronic fatigue service and a pain clinic that have been running in the locality for over 20 years. The clinic, which is entirely virtual (staff consult either from home or from a small side office in the community trust building), is physically located in a low-rise building on the industrial outskirts of a large town, sharing office space with various community-based health and social care services. Following a 1-h telephone consultation by one of the clinical leads, each patient is discussed at the MDT and then either discharged back to their general practitioner with a detailed management plan or referred on to one of the specialist services. This arrangement evolved to address a particular problem in this locality—that many patients with long covid were being referred by their general practitioner to multiple specialties (e.g. respiratory, neurology, fatigue), leading to a fragmented patient experience, unnecessary specialist assessments and wasteful duplication. The generalist assessment by telephone is oriented to documenting what is often a complex illness narrative (including pre-existing physical and mental comorbidities) and working with the patient to prioritize which symptoms or problems to pursue in which order.

Site E, in a well-regarded inner-city teaching hospital, had been set up in 2020 by a respiratory physician. Its initial ethos and rationale had been “respiratory follow-up”, with strong emphasis on monitoring lung damage via repeated imaging and lung function tests and in ensuring that patients received specialist physical therapy to “re-learn” efficient breathing techniques. Over time, this site has tried to accommodate a more multi-system assessment, with the introduction of a consultant-led infectious disease clinic for patients without a dominant respiratory component, reflecting the shift towards a more fatigue-predominant case mix. At the time of our fieldwork, each patient was seen in turn by a physician, psychologist, occupational therapist and respiratory physical therapist (half an hour each) before all four staff reconvened in a face-to-face MDT meeting to form a plan for each patient. But whilst a wide range of patients with diverse symptoms were discussed at these meetings, there remained a strong focus on respiratory pathology (e.g. tracking improvements in lung function and ensuring that coexisting asthma was optimally controlled).

Site F, one of the first long covid clinics in UK, was set up by a rehabilitation consultant who had been drafted to work on the ICU during the first wave of covid-19 in early 2020. He had a longstanding research interest in whole-patient rehabilitation, especially the assessment and management of chronic fatigue and pain. From the outset, clinic F was more oriented to rehabilitation, including vocational rehabilitation to help patients return to work. There was less emphasis on monitoring lung function or pursuing respiratory comorbidities. At the time of our fieldwork, clinic F offered both a community-based service (“tier 2”) led by an occupational therapist, supported by a respiratory physical therapist and psychologist, and a hospital-based service (“tier 3”) led by the rehabilitation consultant, supported by a wider MDT. Staff in both tiers emphasized that each patient needs a full physical and mental assessment and help to set and work towards achievable goals, whilst staying within safe limits so as to avoid post-exertional symptom exacerbation. Because of the research interest of the lead physician, clinic F adapted well to the growing numbers of patients with fatigue and quickly set up research studies on this cohort [ 84 ].

Details of the other seven sites are shown in Table  1 . Broadly speaking, sites B, E, G and H aligned with the “respiratory follow-up” model and sites F and I aligned with the “rehabilitation” model. Sites A and J had a high-volume, multi-tiered service whose community tier aligned with the “holistic GP assessment” model (site C above) and which also offered a hospital-based, rehabilitation-focused tier. The small service in Scotland (site D) had evolved from an initial respiratory focus to become part of the infectious diseases (ME/CFS) service; Lyme disease (another infectious disease whose sequelae include chronic fatigue) was also prevalent in this region.

The patient experience

Whilst the 10 participating clinics were very diverse in staffing, ethos and patient flows, the 29 patient interviews described remarkably consistent clinic experiences. Almost all identified the biggest problem to be the extended wait of several months before they were seen and the limited awareness (when initially referred) of what long covid clinics could provide. Some talked of how they cried with relief when they finally received an appointment. When the quality improvement collaborative was initially established, waiting times and bottlenecks were patients’ the top priority for quality improvement, and this ranking was shared by clinic staff, who were very aware of how much delays and uncertainties in assessment and treatment compounded patients’ suffering. This issue resolved to a large extent over the study period in all clinics as the referral backlog cleared and the incidence of new cases of long covid fell [ 85 ]; it will be covered in more detail in a separate publication.

Most patients in our sample were satisfied with the care they received when they were finally seen in clinic, especially how they finally felt “heard” after a clinician took a full history. They were relieved to receive affirmation of their experience, a diagnosis of what was wrong and reassurance that they were believed. They were grateful for the input of different members of the multidisciplinary teams and commented on the attentiveness, compassion and skill of allied professionals in particular (“she was wonderful, she got me breathing again”—patient BIR145 talking about a physical therapist). One or two patient participants expressed confusion about who exactly they had seen and what advice they had been given, and some did not realize that a telephone assessment had been an actual clinical consultation. A minority expressed disappointment that an expected investigation had not been ordered (one commented that they had not had any blood tests at all). Several had assumed that the help and advice from the long covid clinic would continue to be offered until they were better and were disappointed that they had been discharged after completing the various courses on offer (since their clinic had been set up as an “assessment only” service).

In the next sections, we give examples of topics raised in the quality improvement collaborative and how they were addressed.

Example quality topic 1: Outcome measures

The first topic considered by the quality improvement collaborative was how (that is, using which measures and metrics) to assess and monitor patients with long covid. In the absence of a validated biomarker, various symptom scores and quality of life scales—both generic and disease-specific—were mooted. Site F had already developed and validated a patient-reported outcome measure (PROM), the C19-YRS (Covid-19 Yorkshire Rehabilitation Scale) and used it for both research and clinical purposes [ 86 ]. It was quickly agreed that, for the purposes of generating comparative research findings across the ten clinics, the C19-YRS should be used at all sites and completed by patients three-monthly. A commercial partner produced an electronic version of this instrument and an app for patient smartphones. The quality improvement collaborative also agreed that patients should be asked to complete the EUROQOL EQ5D, a widely used generic health-related quality of life scale [ 87 ], in order to facilitate comparisons between long covid and other chronic conditions.

In retrospect, the discussions which led to the unopposed adoption of these two measures as a “quality” initiative in clinical care were somewhat aspirational. A review of progress at a subsequent quality improvement meeting revealed considerable variation among clinics, with a wide variety of measures used in different clinics to different degrees. Reasons for this variation were multiple. First, although our patient advisory group were keen that we should gather as much data as possible on the patient experience of this new condition, many clinic patients found the long questionnaires exhausting to complete due to cognitive impairment and fatigue. In addition, whilst patients were keen to answer questions on symptoms that troubled them, many had limited patience to fill out repeated surveys on symptoms that did not trouble them (“it almost felt as if I’ve not got long covid because I didn’t feel like I fit the criteria as they were laying it out”—patient SAL001). Staff assisted patients in completing the measures when needed, but this was time-consuming (up to 45 min per instrument) and burdensome for both staff and patients. In clinics where a high proportion of patients required assistance, staff time was the rate-limiting factor for how many instruments got completed. For some patients, one short instrument was the most that could be asked of them, and the clinician made a judgement on which one would be in their best interests on the day.

The second reason for variation was that the clinical diagnosis and management of particular features, complications and comorbidities of long covid required more nuance than was provided by these relatively generic instruments, and the level of detail sought varied with the specialist interest of the clinic (and the clinician). The modified C19-YRS [ 88 ], for example, contained 19 items, of which one asked about sleep quality. But if a patient had sleep difficulties, many clinicians felt that these needed to be documented in more detail—for example using the 8-item Epworth Sleepiness Scale, originally developed for conditions such as narcolepsy and obstructive sleep apnea [ 89 ]. The “Epworth score” was essential currency for referrals to some but not all specialist sleep services. Similarly, the C19-YRS had three items relating to anxiety, depression and post-traumatic stress disorder, but in clinics where there was a strong focus on mental health (e.g. when there was a resident psychologist), patients were usually invited to complete more specific tools (e.g. the Patient Health Questionnaire 9 [ 90 ], a 9-item questionnaire originally designed to assess severity of depression).

The third reason for variation was custom and practice. Ethnographic visits revealed that paper copies of certain instruments were routinely stacked on clinicians’ desks in outpatient departments and also (in some cases) handed out by administrative staff in waiting areas so that patients could complete them before seeing the clinician. These familiar clinic artefacts tended to be short (one-page) instruments that had a long tradition of use in clinical practice. They were not always fit for purpose. For example, the Nijmegen questionnaire was developed in the 1980s to assess hyperventilation; it was validated against a longer, “gold standard” instrument for that condition [ 91 ]. It subsequently became popular in respiratory clinics to diagnose or exclude breathing pattern disorder (a condition in which the normal physiological pattern of breathing becomes replaced with less efficient, shallower breathing [ 92 ]), so much so that the researchers who developed the instrument published a paper to warn fellow researchers that it had not been validated for this purpose [ 93 ]. Whilst a validated 17-item instrument for breathing pattern disorder (the Self-Evaluation of Breathing Questionnaire [ 94 ]) does exist, it is not in widespread clinical use. Most clinics in LOCOMOTION used Nijmegen either on all patients (e.g. as part of a comprehensive initial assessment, especially if the service had begun as a respiratory follow-up clinic) or when breathing pattern disorder was suspected.

In sum, the use of outcome measures in long covid clinics was a compromise between standardization and contingency. On the one hand, all clinics accepted the need to use “validated” instruments consistently. On the other hand, there were sometimes good reasons why they deviated from agreed practice, including mismatch between the clinic’s priorities as a research site, its priorities as a clinical service, and the particular clinical needs of a patient; the clinic’s—and the clinician’s—specialist focus; and long-held traditions of using particular instruments with which staff and patients were familiar.

Example quality topic 2: Postural orthostatic tachycardia syndrome (POTS)

Palpitations (common in long covid) and postural orthostatic tachycardia syndrome (POTS, a disproportionate acceleration in heart rate on standing, the assumed cause of palpitations in many long covid patients) was the top priority for quality improvement identified by our patient advisory group. Reflecting discussions and evidence (of various kinds) shared in online patient communities, the group were confident that POTS is common in long covid patients and that many cases remain undetected (perhaps misdiagnosed as anxiety). Their request that all long covid patients should be “screened” for POTS prompted a search for, and synthesis of, evidence (which we published in the BMJ [ 95 ]). In sum, that evidence was sparse and contested, but, combined with standard practice in specialist clinics, broadly supported the judicious use of the NASA Lean Test [ 96 ]. This test involves repeated measurements of pulse and blood pressure with the patient first lying and then standing (with shoulders resting against a wall).

The patient advisory group’s request that the NASA Lean Test should be conducted on all patients met with mixed responses from the clinics. In site F, the lead physician had an interest in autonomic dysfunction in chronic fatigue and was keen; he had already published a paper on how to adapt the NASA Lean Test for self-assessment at home [ 97 ]. Several other sites were initially opposed. Staff at site E, for example, offered various arguments:

The test is time-consuming, labor-intensive, and takes up space in the clinic which has an opportunity cost in terms of other potential uses;

The test is unvalidated and potentially misleading (there is a high incidence of both false negative and false positive results);

There is no proven treatment for POTS, so there is no point in testing for it;

It is a specialist test for a specialist condition, so it should be done in a specialist clinic where its benefits and limitations are better understood;

Objective testing does not change clinical management since what we treat is the patient’s symptoms (e.g. by a pragmatic trial of lifestyle measures and medication);

People with symptoms suggestive of dysautonomia have already been “triaged out” of this clinic (that is, identified in the initial telephone consultation and referred directly to neurology or cardiology);

POTS is a manifestation of the systemic nature of long covid; it does not need specific treatment but will improve spontaneously as the patient goes through standard interventions such as active pacing, respiratory physical therapy and sleep hygiene;

Testing everyone, even when asymptomatic, runs counter to the ethos of rehabilitation, which is to “de-medicalize” patients so as to better orient them to their recovery journey.

When clinics were invited to implement the NASA Lean Test on a consecutive sample of patients to resolve a dispute about the incidence of POTS (from “we’ve only seen a handful of people with it since the clinic began” to “POTS is common and often missed”), all but one site agreed to participate. The tertiary POTS centre linked to site H was already running the NASA Lean Test as standard on all patients. Site C, which operated entirely virtually, passed the work to the referring general practitioner by making this test a precondition for seeing the patient; site D, which was largely virtual, sent instructions for patients to self-administer the test at home.

The NASA Lean Test study has been published separately [ 98 ]. In sum, of 277 consecutive patients tested across the eight clinics, 20 (7%) had a positive NASA Lean Test for POTS and a further 28 (10%) a borderline result. Six of 20 patients who met the criteria for POTS on testing had no prior history of orthostatic intolerance. The question of whether this test should be used to “screen” all patients was not answered definitively. But the experience of participating in the study persuaded some sceptics that postural changes in heart rate could be severe in some long covid patients, did not appear to be fully explained by their previously held theories (e.g. “functional”, anxiety, deconditioning), and had likely been missed in some patients. The outcome of this particular quality improvement cycle was thus not a wholescale change in practice (for which the evidence base was weak) but a more subtle increase in clinical awareness, a greater willingness to consider testing for POTS and a greater commitment to contribute to research into this contested condition.

More generally, the POTS audit prompted some clinicians to recognize the value of quality improvement in novel clinical areas. One physician who had initially commented that POTS was not seen in their clinic, for example, reflected:

“ Our clinic population is changing. […] Overall there’s far fewer post-ICU patients with ECMO [extra-corporeal membrane oxygenation] issues and far more long covid from the community, and this is the bit our clinic isn’t doing so well on. We’re doing great on breathing pattern disorder; neuro[logists] are helping us with the brain fogs; our fatigue and occupational advice is ok but some of the dysautonomia symptoms that are more prevalent in the people who were not hospitalized – that’s where we need to improve .” -Respiratory physician, site G (from field visit 6.6.23)

Example quality topic 3: Management of fatigue

Fatigue was the commonest symptom overall and a high priority among both patients and clinicians for quality improvement. It often coexisted with the cluster of neurocognitive symptoms known as brain fog, with both conditions relapsing and remitting in step. Clinicians were keen to systematize fatigue management using a familiar clinical framework oriented around documenting a full clinical history, identifying associated symptoms, excluding or exploring comorbidities and alternative explanations (e.g. poor sleep patterns, depression, menopause, deconditioning), assessing how fatigue affects physical and mental function, implementing a program of physical and cognitive therapy that was sensitive to the patient’s condition and confidence level, and monitoring progress using validated patient-reported outcome measures and symptom diaries.

The underpinning logic of this approach, which broadly reflected World Health Organization guidance [ 99 ], was that fatigue and linked cognitive impairment could be a manifestation of many—perhaps interacting—conditions but that a whole-patient (body and mind) rehabilitation program was the cornerstone of management in most cases. Discussion in the quality improvement collaborative focused on issues such as whether fatigue was so severe that it produced safety concerns (e.g. in a person’s job or with childcare), the pros and cons of particular online courses such as yoga, relaxation and mindfulness (many were viewed positively, though the evidence base was considered weak), and the extent to which respiratory physical therapy had a crossover impact on fatigue (systematic reviews suggested that it may do, but these reviews also cautioned that primary studies were sparse, methodologically flawed, and heterogeneous [ 100 , 101 ]). They also debated the strengths and limitations of different fatigue-specific outcome measures, each of which had been developed and validated in a different condition, with varying emphasis on cognitive fatigue, physical fatigue, effect on daily life, and motivation. These instruments included the Modified Fatigue Impact Scale; Fatigue Severity Scale [ 102 ]; Fatigue Assessment Scale; Functional Assessment Chronic Illness Therapy—Fatigue (FACIT-F) [ 103 ]; Work and Social Adjustment Scale [ 104 ]; Chalder Fatigue Scale [ 105 ]; Visual Analogue Scale—Fatigue [ 106 ]; and the EQ5D [ 87 ]. In one clinic (site F), three of these scales were used in combination for reasons discussed below.

Some clinicians advocated melatonin or nutritional supplements (such as vitamin D or folic acid) for fatigue on the grounds that many patients found them helpful and formal placebo-controlled trials were unlikely ever to be conducted. But neurostimulants used in other fatigue-predominant conditions (e.g. brain injury, stroke), which also lacked clinical trial evidence in long covid, were viewed as inappropriate in most patients because of lack of evidence of clear benefit and hypothetical risk of harm (e.g. adverse drug reactions, polypharmacy).

Whilst the patient advisory group were broadly supportive of a whole-patient rehabilitative approach to fatigue, their primary concern was fatiguability , especially post-exertional symptom exacerbation (PESE, also known as “crashes”). In these, the patient becomes profoundly fatigued some hours or days after physical or mental exertion, and this state can last for days or even weeks [ 107 ]. Patients viewed PESE as a “red flag” symptom which they felt clinicians often missed and sometimes caused. They wanted the quality improvement effort to focus on ensuring that all clinicians were aware of the risks of PESE and acted accordingly. A discussion among patients and clinicians at a quality improvement collaborative meeting raised a new research hypothesis—that reducing the number of repeated episodes of PESE may improve the natural history of long covid.

These tensions around fatigue management played out differently in different clinics. In site C (the GP-led virtual clinic run from a community hub), fatigue was viewed as one manifestation of a whole-patient condition. The lead general practitioner used the metaphor of untangling a skein of wool: “you have to find the end and then gently pull it”. The underlying problem in a fatigued patient, for example, might be an undiagnosed physical condition such as anaemia, disturbed sleep, or inadequate pacing. These required (respectively) the chronic fatigue service (comprising an occupational therapist and specialist psychologist and oriented mainly to teaching the techniques of goal-setting and pacing), a “tiredness” work-up (e.g. to exclude anaemia or menopause), investigation of poor sleep (which, not uncommonly, was due to obstructive sleep apnea), and exploration of mental health issues.

In site G (a hospital clinic which had evolved from a respiratory service), patients with fatigue went through a fatigue management program led by the occupational therapist with emphasis on pacing, energy conservation, avoidance of PESE and sleep hygiene. Those without ongoing respiratory symptoms were often discharged back to their general practitioner once they had completed this; there was no consultant follow-up of unresolved fatigue.

In site F (a rehabilitation clinic which had a longstanding interest in chronic fatigue even before the pandemic), active interdisciplinary management of fatigue was commenced at or near the patient’s first visit, on the grounds that the earlier this began, the more successful it would be. In this clinic, patients were offered a more intensive package: a similar occupational therapy-led fatigue course as those in site G, plus input from a dietician to advise on regular balanced meals and caffeine avoidance and a group-based facilitated peer support program which centred on fatigue management. The dietician spoke enthusiastically about how improving diet in longstanding long covid patients often improved fatigue (e.g. because they had often lost muscle mass and tended to snack on convenience food rather than make meals from scratch), though she agreed there was no evidence base from trials to support this approach.

Pursuing local quality improvement through MDTs

Whilst some long covid patients had “textbook” symptoms and clinical findings, many cases were unique and some were fiendishly complex. One clinician commented that, somewhat paradoxically, “easy cases” were often the post-ICU follow-ups who had resolving chest complications; they tended to do well with a course of respiratory physical therapy and a return-to-work program. Such cases were rarely brought to MDT meetings. “Difficult cases” were patients who had not been hospitalized for their acute illness but presented with a months- or years-long history of multiple symptoms with fatigue typically predominant. Each one was different, as the following example (some details of which have been fictionalized to protect anonymity) illustrates.

The MDT is discussing Mrs Fermah, a 65-year-old homemaker who had covid-19 a year ago. She has had multiple symptoms since, including fluctuating fatigue, brain fog, breathlessness, retrosternal chest pain of burning character, dry cough, croaky voice, intermittent rashes (sometimes on eating), lips going blue, ankle swelling, orthopnoea, dizziness with the room spinning which can be triggered by stress, low back pain, aches and pains in the arms and legs and pins and needles in the fingertips, loss of taste and smell, palpitations and dizziness (unclear if postural, but clear association with nausea), headaches on waking, and dry mouth. She is somewhat overweight (body mass index 29) and admits to low mood. Functionally, she is mostly confined to the house and can no longer manage the stairs so has begun to sleep downstairs. She has stumbled once or twice but not fallen. Her social life has ceased and she rarely has the energy to see her grandchildren. Her 70-year-old husband is retired and generally supportive, though he spends most evenings at his club. Comorbidities include glaucoma which is well controlled and overseen by an ophthalmologist, mild club foot (congenital) and stage 1 breast cancer 20 years ago. Various tests, including a chest X-ray, resting and exercise oximetry and a blood panel, were normal except for borderline vitamin D level. Her breathing questionnaire score suggests she does not have breathing pattern disorder. ECG showed first-degree atrioventricular block and left axis deviation. No clinician has witnessed the blue lips. Her current treatment is online group respiratory physical therapy; a home visit is being arranged to assess her climbing stairs. She has declined a psychologist assessment. The consultant asks the nurse who assessed her: “Did you get a feel if this is a POTS-type dizziness or an ENT-type?” She sighs. “Honestly it was hard to tell, bless her.”—Site A MDT

This patient’s debilitating symptoms and functional impairments could all be due to long covid, yet “evidence-based” guidance for how to manage her complex suffering does not exist and likely never will exist. The question of which (if any) additional blood or imaging tests to do, in what order of priority, and what interventions to offer the patient will not be definitively answered by consulting clinical trials involving hundreds of patients, since (even if these existed) the decision involves weighing this patient’s history and the multiple factors and uncertainties that are relevant in her case. The knowledge that will help the MDT provide quality care to Mrs Fermah is case-based knowledge—accumulated clinical experience and wisdom from managing and deliberating on multiple similar cases. We consider case-based knowledge further in the “ Discussion ”.

Summary of key findings

This study has shown that a quality improvement collaborative of UK long covid clinics made some progress towards standardizing assessment and management in some topics, but some variation remained. This could be explained in part by the fact that different clinics had different histories and path dependencies, occupied a different place in the local healthcare ecosystem, served different populations, were differently staffed, and had different clinical interests. Our patient advisory group and clinicians in the quality improvement collaborative broadly prioritized the same topics for improvement but interpreted them somewhat differently. “Quality” long covid care had multiple dimensions, relating to (among other things) service set-up and accessibility, clinical provision appropriate to the patient’s need (including options for referral to other services locally), the human qualities of clinical and support staff, how knowledge was distributed across (and accessible within) the system, and the accumulated collective wisdom of local MDTs in dealing with complex cases (including multiple kinds of specialist expertise as well as relational knowledge of what was at stake for the patient). Whilst both staff and patients were keen to contribute to the quality improvement effort, the burden of measurement was evident: multiple outcome measures, used repeatedly, were resource-intensive for staff and exhausting for patients.

Strengths and limitations of this study

To our knowledge, we are the first to report both a quality improvement collaborative and an in-depth qualitative study of clinical work in long covid. Key strengths of this work include the diverse sampling frame (with sites from three UK jurisdictions and serving widely differing geographies and demographics); the use of documents, interviews and reflexive interpretive ethnography to produce meaningful accounts of how clinics emerged and how they were currently organized; the use of philosophical concepts to analyse data on how MDTs produced quality care on a patient-by-patient basis; and the close involvement of patient co-researchers and coauthors during the research and writing up.

Limitations of the study include its exclusive UK focus (the external validity of findings to other healthcare systems is unknown); the self-selecting nature of participants in a quality improvement collaborative (our patient advisory group suggested that the MDTs observed in this study may have represented the higher end of a quality spectrum, hence would be more likely than other MDTs to adhere to guidelines); and the particular perspective brought by the researchers (two GPs, a physical therapist and one non-clinical person) in ethnographic observations. Hospital specialists or organizational scholars, for example, may have noticed different things or framed what they observed differently.

Explaining variation in long covid care

Sutherland and Levesque’s framework mentioned in the “ Background ” section does not explain much of the variation found in our study [ 70 ]. In terms of capacity, at the time of this study most participating clinics benefited from ring-fenced resources. In terms of evidence, guidelines existed and were not greatly contested, but as illustrated by the case of Mrs Fermah above, many patients were exceptions to the guideline because of complex symptomatology and relevant comorbidities. In terms of agency, clinicians in most clinics were passionately engaged with long covid (they were pioneers who had set up their local clinic and successfully bid for national ring-fenced resources) and were generally keen to support patient choice (though not if the patient requested tests which were unavailable or deemed not indicated).

Astma et al.’s list of factors that may explain variation in practice (see “ Background ”) includes several that may be relevant to long covid, especially that the definition of appropriate care in this condition remains somewhat contested. But lack of opportunity to discuss cases was not a problem in the clinics in our sample. On the contrary, MDT meetings in each locality gave clinicians multiple opportunities to discuss cases with colleagues and reflect collectively on whether and how to apply particular guidelines.

The key problem was not that clinicians disputed the guidelines for managing long covid or were unaware of them; it was that the guidelines were not self-interpreting . Rather, MDTs had to deliberate on the balance of benefits and harms in different aspects of individual cases. In patients whose symptoms suggested a possible diagnosis of POTS (or who suspected themselves of having POTS), for example, these deliberations were sometimes lengthy and nuanced. Should a test result that is not technically in the abnormal range but close to it be treated as diagnostic, given that symptoms point to this diagnosis? If not, should the patient be told that the test excludes POTS or that it is equivocal? If a cardiology opinion has stated firmly that the patient does not have POTS but the cardiologist is not known for their interest in this condition, should a second specialist opinion be sought? If the gold standard “tilt test” [ 108 ] for POTS (usually available only in tertiary centres) is not available locally, does this patient merit a costly out-of-locality referral? Should the patient’s request for a trial of off-label medication, reflecting discussions in an online support group, be honoured? These are the kinds of questions on which MDTs deliberated at length.

The fact that many cases required extensive deliberation does not necessarily justify variation in practice among clinics. But taking into account the clinics’ very different histories, set-up, and local referral pathways, the variation begins to make sense. A patient who is being assessed in a clinic that functions as a specialist chronic fatigue centre and attracts referrals which reflect this interest (e.g. site F in our sample) will receive different management advice from one that functions as a telephone-only generalist assessment centre and refers on to other specialties (site C in our sample). The wide variation in case mix, coupled with the fact that a different proportion of these cases were highly complex in each clinic (and in different ways), suggests that variation in practice may reflect appropriate rather than inappropriate care.

Our patient advisory group affirmed that many of the findings reported here resonated with their own experience, but they raised several concerns. These included questions about patient groups who may have been missed in our sample because they were rarely discussed in MDTs. The decision to take a case to MDT discussion is taken largely by a clinician, and there was evidence from online support groups that some patients’ requests for their case to be taken to an MDT had been declined (though not, to our knowledge, in the clinics participating in the LOCOMOTION study).

We began this study by asking “what is quality in long covid care?”. We initially assumed that this question referred to a generalizable evidence base, which we felt we could identify, and we believed that we could then determine whether long covid clinics were following the evidence base through conventional audits of structure, process, and outcome. In retrospect, these assumptions were somewhat naïve. On the basis of our findings, we suggest that a better (and more individualized) research question might be “to what extent does each patient with long covid receive evidence-based care appropriate to their needs?”. This question would require individual case review on a sample of cases, tracking each patient longitudinally including cross-referrals, and also interviewing the patient.

Nomothetic versus idiographic knowledge

In a series of lectures first delivered in the 1950s and recently republished [ 109 ], psychiatrist Dr Maurice O’Connor Drury drew on the later philosophy of his friend and mentor Ludwig Wittgenstein to challenge what he felt was a concerning trend: that the nomothetic (generalizable, abstract) knowledge from randomized controlled trials (RCTs) was coming to over-ride the idiographic (personal, situated) knowledge about particular patients. Based on Wittgenstein’s writings on the importance of the particular, Drury predicted—presciently—that if implemented uncritically, RCTs would result in worse, not better, care for patients, since it would go hand-in-hand with a downgrading of experience, intuition, subjective judgement, personal reflection, and collective deliberation.

Much conventional quality improvement methodology is built on an assumption that nomothetic knowledge (for example, findings from RCTs and systematic reviews) is a higher form of knowing than idiographic knowledge. But idiographic, case-based reasoning—despite its position at the very bottom of evidence-based medicine’s hierarchy of evidence [ 110 ]—is a legitimate and important element of medical practice. Bioethicist Kathryn Montgomery, drawing on Aristotle’s notion of praxis , considers clinical practice to be an example of case-based reasoning [ 111 ]. Medicine is governed not by hard and fast laws but by competing maxims or rules of thumb ; the essence of judgement is deciding which (if any) rule should be applied in a particular circumstance. Clinical judgement incorporates science (especially the results of well-conducted research) and makes use of available tools and technologies (including guidelines and decision-support algorithms that incorporate research findings). But rather than being determined solely by these elements, clinical judgement is guided both by the scientific evidence and by the practical and ethical question “what is it best to do, for this individual, given these circumstances?”.

In this study, we observed clinical management of, and MDT deliberations on, hundreds of clinical cases. In the more straightforward ones (for example, recovering pneumonitis), guideline-driven care was not difficult to implement and such cases were rarely brought to the MDT. But cases like Mrs Fermah (see last section of “ Results ”) required much discussion on which aspects of which guideline were in the patient’s best interests to bring into play at any particular stage in their illness journey.

Conclusions

One systematic review on quality improvement collaboratives concluded that “ [those] reporting success generally addressed relatively straightforward aspects of care, had a strong evidence base and noted a clear evidence-practice gap in an accepted clinical pathway or guideline” (page 226) [ 60 ]. The findings from this study suggest that to the extent that such collaboratives address clinical cases that are not straightforward, conventional quality improvement methods may be less useful and even counterproductive.

The question “what is quality in long covid care?” is partly a philosophical one. Our findings support an approach that recognizes and values idiographic knowledge —including establishing and protecting a safe and supportive space for deliberation on individual cases to occur and to value and draw upon the collective learning that occurs in these spaces. It is through such deliberation that evidence-based guidelines can be appropriately interpreted and applied to the unique needs and circumstances of individual patients. We suggest that Drury’s warning about the limitations of nomothetic knowledge should prompt a reassessment of policies that rely too heavily on such knowledge, resulting in one-size-fits-all protocols. We also cautiously hypothesize that the need to centre the quality improvement effort on idiographic rather than nomothetic knowledge is unlikely to be unique to long covid. Indeed, such an approach may be particularly important in any condition that is complex, unpredictable, variable in presentation and clinical course, and associated with comorbidities.

Availability of data and materials

Selected qualitative data (ensuring no identifiable information) will be made available to formal research teams on reasonable request to Professor Greenhalgh at the University of Oxford, on condition that they have research ethics approval and relevant expertise. The quantitative data on NASA Lean Test have been published in full in a separate paper [ 98 ].

Abbreviations

Chronic fatigue syndrome

Intensive care unit

Jenny Ceolta-Smith

Julie Darbyshire

LOng COvid Multidisciplinary consortium Optimising Treatments and services across the NHS

Multidisciplinary team

Myalgic encephalomyelitis

Middle East Respiratory Syndrome

National Aeronautics and Space Association

Occupational therapy/ist

Post-exertional symptom exacerbation

Postural orthostatic tachycardia syndrome

Speech and language therapy

Severe Acute Respiratory Syndrome

Trisha Greenhalgh

United Kingdom

United States

World Health Organization

Perego E, Callard F, Stras L, Melville-JÛhannesson B, Pope R, Alwan N. Why the Patient-Made Term “Long Covid” is needed. Wellcome Open Res. 2020;5:224.

Article   Google Scholar  

Greenhalgh T, Sivan M, Delaney B, Evans R, Milne R: Long covid—an update for primary care. bmj 2022;378:e072117.

Centers for Disease Control and Prevention (US): Long COVID or Post-COVID Conditions (updated 16th December 2022). Atlanta: CDC. Accessed 2nd June 2023 at https://www.cdc.gov/coronavirus/2019-ncov/long-term-effects/index.html ; 2022.

National Institute for Health and Care Excellence (NICE) Scottish Intercollegiate Guidelines Network (SIGN) and Royal College of General Practitioners (RCGP): COVID-19 rapid guideline: managing the long-term effects of COVID-19, vol. Accessed 30th January 2022 at https://www.nice.org.uk/guidance/ng188/resources/covid19-rapid-guideline-managing-the-longterm-effects-of-covid19-pdf-51035515742 . London: NICE; 2022.

Organization WH: Post Covid-19 Condition (updated 7th December 2022), vol. Accessed 2nd June 2023 at https://www.who.int/europe/news-room/fact-sheets/item/post-covid-19-condition#:~:text=It%20is%20defined%20as%20the,months%20with%20no%20other%20explanation . Geneva: WHO; 2022.

Office for National Statistics: Prevalence of ongoing symptoms following coronavirus (COVID-19) infection in the UK: 31st March 2023. London: ONS. Accessed 30th May 2023 at https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datasets/alldatarelatingtoprevalenceofongoingsymptomsfollowingcoronaviruscovid19infectionintheuk ; 2023.

Crook H, Raza S, Nowell J, Young M, Edison P: Long covid—mechanisms, risk factors, and management. bmj 2021;374.

Sudre CH, Murray B, Varsavsky T, Graham MS, Penfold RS, Bowyer RC, Pujol JC, Klaser K, Antonelli M, Canas LS. Attributes and predictors of long COVID. Nat Med. 2021;27(4):626–31.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Reese JT, Blau H, Casiraghi E, Bergquist T, Loomba JJ, Callahan TJ, Laraway B, Antonescu C, Coleman B, Gargano M: Generalisable long COVID subtypes: findings from the NIH N3C and RECOVER programmes. EBioMedicine 2023;87.

Thaweethai T, Jolley SE, Karlson EW, Levitan EB, Levy B, McComsey GA, McCorkell L, Nadkarni GN, Parthasarathy S, Singh U. Development of a definition of postacute sequelae of SARS-CoV-2 infection. JAMA. 2023;329(22):1934–46.

Brown DA, O’Brien KK. Conceptualising Long COVID as an episodic health condition. BMJ Glob Health. 2021;6(9): e007004.

Article   PubMed   Google Scholar  

Tate WP, Walker MO, Peppercorn K, Blair AL, Edgar CD. Towards a Better Understanding of the Complexities of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and Long COVID. Int J Mol Sci. 2023;24(6):5124.

Ahmed H, Patel K, Greenwood DC, Halpin S, Lewthwaite P, Salawu A, Eyre L, Breen A, Connor RO, Jones A. Long-term clinical outcomes in survivors of severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome coronavirus (MERS) outbreaks after hospitalisation or ICU admission: a systematic review and meta-analysis. J Rehabil Med. 2020;52(5):1–11.

Google Scholar  

World Health Organisation: Clinical management of severe acute respiratory infection (SARI) when COVID-19 disease is suspected: Interim guidance (13th March 2020). Geneva: WHO. Accessed 3rd January 2023 at https://t.co/JpNdP8LcV8?amp=1 ; 2020.

Rushforth A, Ladds E, Wieringa S, Taylor S, Husain L, Greenhalgh T: Long Covid – the illness narratives. Under review for Sociology of Health and Illness 2021.

Russell D, Spence NJ. Chase J-AD, Schwartz T, Tumminello CM, Bouldin E: Support amid uncertainty: Long COVID illness experiences and the role of online communities. SSM-Qual Res Health. 2022;2: 100177.

Article   PubMed   PubMed Central   Google Scholar  

Ziauddeen N, Gurdasani D, O’Hara ME, Hastie C, Roderick P, Yao G, Alwan NA. Characteristics and impact of Long Covid: Findings from an online survey. PLoS ONE. 2022;17(3): e0264331.

Evans RA, McAuley H, Harrison EM, Shikotra A, Singapuri A, Sereno M, Elneima O, Docherty AB, Lone NI, Leavy OC. Physical, cognitive, and mental health impacts of COVID-19 after hospitalisation (PHOSP-COVID): a UK multicentre, prospective cohort study. Lancet Respir Med. 2021;9(11):1275–87.

Sykes DL, Holdsworth L, Jawad N, Gunasekera P, Morice AH, Crooks MG. Post-COVID-19 symptom burden: what is long-COVID and how should we manage it? Lung. 2021;199(2):113–9.

Altmann DM, Whettlock EM, Liu S, Arachchillage DJ, Boyton RJ: The immunology of long COVID. Nat Rev Immunol 2023:1–17.

Klein J, Wood J, Jaycox J, Dhodapkar RM, Lu P, Gehlhausen JR, Tabachnikova A, Greene K, Tabacof L, Malik AA et al : Distinguishing features of Long COVID identified through immune profiling. Nature 2023.

Chen B, Julg B, Mohandas S, Bradfute SB. Viral persistence, reactivation, and mechanisms of long COVID. Elife. 2023;12: e86015.

Wang C, Ramasamy A, Verduzco-Gutierrez M, Brode WM, Melamed E. Acute and post-acute sequelae of SARS-CoV-2 infection: a review of risk factors and social determinants. Virol J. 2023;20(1):124.

Cervia-Hasler C, Brüningk SC, Hoch T, Fan B, Muzio G, Thompson RC, Ceglarek L, Meledin R, Westermann P, Emmenegger M et al Persistent complement dysregulation with signs of thromboinflammation in active Long Covid Science 2024;383(6680):eadg7942.

Sivan M, Greenhalgh T, Darbyshire JL, Mir G, O’Connor RJ, Dawes H, Greenwood D, O’Connor D, Horton M, Petrou S. LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS (LOCOMOTION): protocol for a mixed-methods study in the UK. BMJ Open. 2022;12(5): e063505.

Rushforth A, Ladds E, Wieringa S, Taylor S, Husain L, Greenhalgh T. Long covid–the illness narratives. Soc Sci Med. 2021;286: 114326.

National Institute for Health and Care Excellence: COVID-19 rapid guideline: managing the long-term effects of COVID-19, vol. Accessed 4th October 2023 at https://www.nice.org.uk/guidance/ng188/resources/covid19-rapid-guideline-managing-the-longterm-effects-of-covid19-pdf-51035515742 . London: NICE 2020.

NHS England: Long COVID: the NHS plan for 2021/22. London: NHS England. Accessed 2nd August 2022 at https://www.england.nhs.uk/coronavirus/documents/long-covid-the-nhs-plan-for-2021-22/ ; 2021.

NHS England: NHS to offer ‘long covid’ sufferers help at specialist centres. London: NHS England. Accessed 10th October 2020 at https://www.england.nhs.uk/2020/10/nhs-to-offer-long-covid-help/ ; 2020 (7th October).

NHS England: The NHS plan for improving long COVID services, vol. Acessed 4th February 2024 at https://www.england.nhs.uk/publication/the-nhs-plan-for-improving-long-covid-services/ .London: Gov.uk; 2022.

NHS England: Commissioning guidance for post-COVID services for adults, children and young people, vol. Accessed 6th February 2024 at https://www.england.nhs.uk/long-read/commissioning-guidance-for-post-covid-services-for-adults-children-and-young-people/ . London: gov.uk; 2023.

National Institute for Health Research: Researching Long Covid: Adressing a new global health challenge, vol. Accessed 9.8.23 at https://evidence.nihr.ac.uk/collection/researching-long-covid-addressing-a-new-global-health-challenge/ . London: NIHR; 2022.

Subbaraman N. NIH will invest $1 billion to study long COVID. Nature. 2021;591(7850):356–356.

Article   CAS   PubMed   Google Scholar  

Donabedian A. The definition of quality and approaches to its assessment and monitoring. Ann Arbor: Michigan; 1980.

Laffel G, Blumenthal D. The case for using industrial quality management science in health care organizations. JAMA. 1989;262(20):2869–73.

Maxwell RJ. Quality assessment in health. BMJ. 1984;288(6428):1470.

Berwick DM, Godfrey BA, Roessner J. Curing health care: New strategies for quality improvement. The Journal for Healthcare Quality (JHQ). 1991;13(5):65–6.

Deming WE. Out of the Crisis. Cambridge, MA: MIT Press; 1986.

Argyris C: Increasing leadership effectiveness: New York: J. Wiley; 1976.

Juran JM: A history of managing for quality: The evolution, trends, and future directions of managing for quality: Asq Press; 1995.

Institute of Medicine (US): Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

McNab D, McKay J, Shorrock S, Luty S, Bowie P. Development and application of ‘systems thinking’ principles for quality improvement. BMJ Open Qual. 2020;9(1): e000714.

Sampath B, Rakover J, Baldoza K, Mate K, Lenoci-Edwards J, Barker P. ​Whole-System Quality: A Unified Approach to Building Responsive, Resilient Health Care Systems. Boston: Institute for Healthcare Immprovement; 2021.

Batalden PB, Davidoff F: What is “quality improvement” and how can it transform healthcare? In . , vol. 16: BMJ Publishing Group Ltd; 2007: 2–3.

Baker G. Collaborating for improvement: the Institute for Healthcare Improvement’s breakthrough series. New Med. 1997;1:5–8.

Plsek PE. Collaborating across organizational boundaries to improve the quality of care. Am J Infect Control. 1997;25(2):85–95.

Ayers LR, Beyea SC, Godfrey MM, Harper DC, Nelson EC, Batalden PB. Quality improvement learning collaboratives. Qual Manage Healthcare. 2005;14(4):234–47.

Brandrud AS, Schreiner A, Hjortdahl P, Helljesen GS, Nyen B, Nelson EC. Three success factors for continual improvement in healthcare: an analysis of the reports of improvement team members. BMJ Qual Saf. 2011;20(3):251–9.

Dückers ML, Spreeuwenberg P, Wagner C, Groenewegen PP. Exploring the black box of quality improvement collaboratives: modelling relations between conditions, applied changes and outcomes. Implement Sci. 2009;4(1):1–12.

Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–94.

Shortell SM, Marsteller JA, Lin M, Pearson ML, Wu S-Y, Mendel P, Cretin S, Rosen M: The role of perceived team effectiveness in improving chronic illness care. Medical Care 2004:1040–1048.

Wilson T, Berwick DM, Cleary PD. What do collaborative improvement projects do? Experience from seven countries. Joint Commission J Qual Safety. 2004;30:25–33.

Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008;336(7659):1491–4.

Hulscher ME, Schouten LM, Grol RP, Buchan H. Determinants of success of quality improvement collaboratives: what does the literature show? BMJ Qual Saf. 2013;22(1):19–31.

Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.

Bate P, Mendel P, Robert G: Organizing for quality: the improvement journeys of leading hospitals in Europe and the United States: CRC Press; 2007.

Andersson-Gäre B, Neuhauser D. The health care quality journey of Jönköping County Council. Sweden Qual Manag Health Care. 2007;16(1):2–9.

Törnblom O, Stålne K, Kjellström S. Analyzing roles and leadership in organizations from cognitive complexity and meaning-making perspectives. Behav Dev. 2018;23(1):63.

Greenhalgh T, Russell J. Why Do Evaluations of eHealth Programs Fail? An Alternative Set of Guiding Principles. PLoS Med. 2010;7(11): e1000360.

Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226–40.

Landon BE, Wilson IB, McInnes K, Landrum MB, Hirschhorn L, Marsden PV, Gustafson D, Cleary PD. Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004;140(11):887–96.

Mittman BS. Creating the evidence base for quality improvement collaboratives. Ann Intern Med. 2004;140(11):897–901.

Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. BMJ. 2002;325(7370):961–4.

Bungay H. Cancer and health policy: the postcode lottery of care. Soc Policy Admin. 2005;39(1):35–48.

Wennberg JE, Cooper MM: The Quality of Medical Care in the United States: A Report on the Medicare Program: The Dartmouth Atlas of Health Care 1999: The Center for the Evaluative Clinical Sciences [Internet]. 1999.

DaSilva P, Gray JM. English lessons: can publishing an atlas of variation stimulate the discussion on appropriateness of care? Med J Aust. 2016;205(S10):S5–7.

Gray WK, Day J, Briggs TW, Harrison S. Identifying unwarranted variation in clinical practice between healthcare providers in England: Analysis of administrative data over time for the Getting It Right First Time programme. J Eval Clin Pract. 2021;27(4):743–50.

Wabe N, Thomas J, Scowen C, Eigenstetter A, Lindeman R, Georgiou A. The NSW Pathology Atlas of Variation: Part I—Identifying Emergency Departments With Outlying Laboratory Test-Ordering Practices. Ann Emerg Med. 2021;78(1):150–62.

Jamal A, Babazono A, Li Y, Fujita T, Yoshida S, Kim SA. Elucidating variations in outcomes among older end-stage renal disease patients on hemodialysis in Fukuoka Prefecture, Japan. PLoS ONE. 2021;16(5): e0252196.

Sutherland K, Levesque JF. Unwarranted clinical variation in health care: definitions and proposal of an analytic framework. J Eval Clin Pract. 2020;26(3):687–96.

Tanenbaum SJ. Reducing variation in health care: The rhetorical politics of a policy idea. J Health Polit Policy Law. 2013;38(1):5–26.

Atsma F, Elwyn G, Westert G. Understanding unwarranted variation in clinical practice: a focus on network effects, reflective medicine and learning health systems. Int J Qual Health Care. 2020;32(4):271–4.

Horbar JD, Rogowski J, Plsek PE, Delmore P, Edwards WH, Hocker J, Kantak AD, Lewallen P, Lewis W, Lewit E. Collaborative quality improvement for neonatal intensive care. Pediatrics. 2001;107(1):14–22.

Van Maanen J: Tales of the field: On writing ethnography: University of Chicago Press; 2011.

Golden-Biddle K, Locke K. Appealing work: An investigation of how ethnographic texts convince. Organ Sci. 1993;4(4):595–616.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Glaser BG. The constant comparative method of qualitative analysis. Soc Probl. 1965;12:436–45.

Willis R. The use of composite narratives to present interview findings. Qual Res. 2019;19(4):471–80.

Vojdani A, Vojdani E, Saidara E, Maes M. Persistent SARS-CoV-2 Infection, EBV, HHV-6 and other factors may contribute to inflammation and autoimmunity in long COVID. Viruses. 2023;15(2):400.

Choutka J, Jansari V, Hornig M, Iwasaki A. Unexplained post-acute infection syndromes. Nat Med. 2022;28(5):911–23.

Connors JM, Ariëns RAS. Uncertainties about the roles of anticoagulation and microclots in postacute sequelae of severe acute respiratory syndrome coronavirus 2 infection. J Thromb Haemost. 2023;21(10):2697–701.

Patel MA, Knauer MJ, Nicholson M, Daley M, Van Nynatten LR, Martin C, Patterson EK, Cepinskas G, Seney SL, Dobretzberger V. Elevated vascular transformation blood biomarkers in Long-COVID indicate angiogenesis as a key pathophysiological mechanism. Mol Med. 2022;28(1):122.

Greenhalgh T, Sivan M, Delaney B, Evans R, Milne R: Long covid—an update for primary care. bmj 2022, 378.

Parkin A, Davison J, Tarrant R, Ross D, Halpin S, Simms A, Salman R, Sivan M. A multidisciplinary NHS COVID-19 service to manage post-COVID-19 syndrome in the community. J Prim Care Commun Health. 2021;12:21501327211010990.

NHS England: COVID-19 Post-Covid Assessment Service, vol. Accessed 5th March 2024 at https://www.england.nhs.uk/statistics/statistical-work-areas/covid-19-post-covid-assessment-service/ . London: NHS England; 2024.

Sivan M, Halpin S, Gee J, Makower S, Parkin A, Ross D, Horton M, O'Connor R: The self-report version and digital format of the COVID-19 Yorkshire Rehabilitation Scale (C19-YRS) for Long Covid or Post-COVID syndrome assessment and monitoring. Adv Clin Neurosci Rehabil 2021;20(3).

The EuroQol Group. EuroQol-a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208.

Sivan M, Preston NJ, Parkin A, Makower S, Gee J, Ross D, Tarrant R, Davison J, Halpin S, O’Connor RJ, et al. The modified COVID-19 Yorkshire Rehabilitation Scale (C19-YRSm) patient-reported outcome measure for Long Covid or Post-COVID syndrome. J Med Virol. 2022;94(9):4253–64.

Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–5.

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.

Van Dixhoorn J, Duivenvoorden H. Efficacy of Nijmegen Questionnaire in recognition of the hyperventilation syndrome. J Psychosom Res. 1985;29(2):199–206.

Evans R, Pick A, Lardner R, Masey V, Smith N, Greenhalgh T: Breathing difficulties after covid-19: a guide for primary care. BMJ 2023;381.

Van Dixhoorn J, Folgering H: The Nijmegen Questionnaire and dysfunctional breathing. In . , vol. 1: Eur Respiratory Soc; 2015.

Courtney R, Greenwood KM. Preliminary investigation of a measure of dysfunctional breathing symptoms: The Self Evaluation of Breathing Questionnaire (SEBQ). Int J Osteopathic Med. 2009;12(4):121–7.

Espinosa-Gonzalez A, Master H, Gall N, Halpin S, Rogers N, Greenhalgh T. Orthostatic tachycardia after covid-19. BMJ (Clinical Research ed). 2023;380:e073488–e073488.

PubMed   Google Scholar  

Bungo M, Charles J, Johnson P Jr. Cardiovascular deconditioning during space flight and the use of saline as a countermeasure to orthostatic intolerance. Aviat Space Environ Med. 1985;56(10):985–90.

CAS   PubMed   Google Scholar  

Sivan M, Corrado J, Mathias C. The Adapted Autonomic Profile (Aap) Home-Based Test for the Evaluation of Neuro-Cardiovascular Autonomic Dysfunction. Adv Clin Neurosci Rehabil. 2022;3:10–13. https://doi.org/10.47795/QKBU46715 .

Lee C, Greenwood DC, Master H, Balasundaram K, Williams P, Scott JT, Wood C, Cooper R, Darbyshire JL, Gonzalez AE. Prevalence of orthostatic intolerance in long covid clinic patients and healthy volunteers: A multicenter study. J Med Virol. 2024;96(3): e29486.

World Health Organization: Clinical management of covid-19 - living guideline. Geneva: WHO. Accessed 4th October 2023 at https://www.who.int/publications/i/item/WHO-2019-nCoV-clinical-2021-2 ; 2023.

Ahmed I, Mustafaoglu R, Yeldan I, Yasaci Z, Erhan B: Effect of pulmonary rehabilitation approaches on dyspnea, exercise capacity, fatigue, lung functions and quality of life in patients with COVID-19: A Systematic Review and Meta-Analysis. Arch Phys Med Rehabil 2022.

Dillen H, Bekkering G, Gijsbers S, Vande Weygaerde Y, Van Herck M, Haesevoets S, Bos DAG, Li A, Janssens W, Gosselink R, et al. Clinical effectiveness of rehabilitation in ambulatory care for patients with persisting symptoms after COVID-19: a systematic review. BMC Infect Dis. 2023;23(1):419.

Learmonth Y, Dlugonski D, Pilutti L, Sandroff B, Klaren R, Motl R. Psychometric properties of the fatigue severity scale and the modified fatigue impact scale. J Neurol Sci. 2013;331(1–2):102–7.

Webster K, Cella D, Yost K. The Functional Assessment of Chronic Illness T herapy (FACIT) Measurement System: properties, applications, and interpretation. Health Qual Life Outcomes. 2003;1(1):1–7.

Mundt JC, Marks IM, Shear MK, Greist JM. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry. 2002;180(5):461–4.

Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, Wallace E. Development of a fatigue scale. J Psychosom Res. 1993;37(2):147–53.

Shahid A, Wilkinson K, Marcu S, Shapiro CM: Visual analogue scale to evaluate fatigue severity (VAS-F). In: STOP, THAT and one hundred other sleep scales . edn.: Springer; 2011:399–402.

Parker M, Sawant HB, Flannery T, Tarrant R, Shardha J, Bannister R, Ross D, Halpin S, Greenwood DC, Sivan M. Effect of using a structured pacing protocol on post-exertional symptom exacerbation and health status in a longitudinal cohort with the post-COVID-19 syndrome. J Med Virol. 2023;95(1): e28373.

Kenny RA, Bayliss J, Ingram A, Sutton R. Head-up tilt: a useful test for investigating unexplained syncope. The Lancet. 1986;327(8494):1352–5.

Drury MOC: Science and Psychology. In: The selected writings of Maurice O’Connor Drury: On Wittgenstein, philosophy, religion and psychiatry. edn.: Bloomsbury Publishing; 2017.

Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887–92.

Mongtomery K: How doctors think: Clinical judgment and the practice of medicine: Oxford University Press; 2005.

Download references

Acknowledgements

We are grateful to clinic staff for allowing us to study their work and to patients for allowing us to sit in on their consultations. We also thank the funder of LOCOMOTION (National Institute for Health Research) and the patient advisory group for lived experience input.

This research is supported by National Institute for Health Research (NIHR) Long Covid Research Scheme grant (Ref COV-LT-0016).

Author information

Authors and affiliations.

Nuffield Department of Primary Care Health Sciences, University of Oxford, Woodstock Rd, Oxford, OX2 6GG, UK

Trisha Greenhalgh, Julie L. Darbyshire & Emma Ladds

Imperial College Healthcare NHS Trust, London, UK

LOCOMOTION Patient Advisory Group and Lived Experience Representative, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

TG conceptualized the overall study, led the empirical work, supported the quality improvement meetings, conducted the ethnographic visits, led the data analysis, developed the theorization and wrote the first draft of the paper. JLD organized and led the quality improvement meetings, supported site-based researchers to collect and analyse data on their clinic, collated and summarized data on quality topics, and liaised with the patient advisory group. CL conceptualized and led the quality topic on POTS, including exploring reasons for some clinics’ reluctance to conduct testing and collating and analysing the NASA Lean Test data across all sites. EL assisted with ethnographic visits, data analysis, and theorization. JCS contributed lived experience of long covid and also clinical experience as an occupational therapist; she liaised with the wider patient advisory group, whose independent (patient-led) audit of long covid clinics informed the quality improvement prioritization exercise. All authors provided extensive feedback on drafts and contributed to discussions and refinements. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Trisha Greenhalgh .

Ethics declarations

Ethics approval and consent to participate.

LOng COvid Multidisciplinary consortium Optimising Treatments and servIces acrOss the NHS study is sponsored by the University of Leeds and approved by Yorkshire & The Humber—Bradford Leeds Research Ethics Committee (ref: 21/YH/0276) and subsequent amendments.

Patient participants in clinic were approached by the clinician (without the researcher present) and gave verbal informed consent for a clinically qualified researcher to observe the consultation. If they consented, the researcher was then invited to sit in. A written record was made in field notes of this verbal consent. It was impractical to seek consent from patients whose cases were discussed (usually with very brief clinical details) in online MDTs. Therefore, clinical case examples from MDTs presented in the paper are fictionalized cases constructed from multiple real cases and with key clinical details changed (for example, comorbidities were replaced with different conditions which would produce similar symptoms). All fictionalized cases were checked by our patient advisory group to check that they were plausible to lived experience experts.

Consent for publication

No direct patient cases are reported in this manuscript. For details of how the fictionalized cases were constructed and validated, see “Consent to participate” above.

Competing interests

TG was a member of the UK National Long Covid Task Force 2021–2023 and on the Oversight Group for the NICE Guideline on Long Covid 2021–2022. She is a member of Independent SAGE.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Greenhalgh, T., Darbyshire, J.L., Lee, C. et al. What is quality in long covid care? Lessons from a national quality improvement collaborative and multi-site ethnography. BMC Med 22 , 159 (2024). https://doi.org/10.1186/s12916-024-03371-6

Download citation

Received : 04 December 2023

Accepted : 26 March 2024

Published : 15 April 2024

DOI : https://doi.org/10.1186/s12916-024-03371-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Post-covid-19 syndrome
• Quality improvement
• Breakthrough collaboratives
• Warranted variation
• Unwarranted variation
• Improvement science
• Ethnography
• Idiographic reasoning
• Nomothetic reasoning

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

What is Qualitative in Qualitative Research

• Open access
• Published: 27 February 2019
• Volume 42 , pages 139–160, ( 2019 )

Cite this article

You have full access to this open access article

• Patrik Aspers 1 , 2 &
• Ugo Corte 3  

582k Accesses

283 Citations

24 Altmetric

Explore all metrics

What is qualitative research? If we look for a precise definition of qualitative research, and specifically for one that addresses its distinctive feature of being “qualitative,” the literature is meager. In this article we systematically search, identify and analyze a sample of 89 sources using or attempting to define the term “qualitative.” Then, drawing on ideas we find scattered across existing work, and based on Becker’s classic study of marijuana consumption, we formulate and illustrate a definition that tries to capture its core elements. We define qualitative research as an iterative process in which improved understanding to the scientific community is achieved by making new significant distinctions resulting from getting closer to the phenomenon studied. This formulation is developed as a tool to help improve research designs while stressing that a qualitative dimension is present in quantitative work as well. Additionally, it can facilitate teaching, communication between researchers, diminish the gap between qualitative and quantitative researchers, help to address critiques of qualitative methods, and be used as a standard of evaluation of qualitative research.

Similar content being viewed by others

Qualitative Research: Ethical Considerations

Criteria for Good Qualitative Research: A Comprehensive Review

Drishti Yadav

Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations

Gordon W. Cheung, Helena D. Cooper-Thomas, … Linda C. Wang

Avoid common mistakes on your manuscript.

If we assume that there is something called qualitative research, what exactly is this qualitative feature? And how could we evaluate qualitative research as good or not? Is it fundamentally different from quantitative research? In practice, most active qualitative researchers working with empirical material intuitively know what is involved in doing qualitative research, yet perhaps surprisingly, a clear definition addressing its key feature is still missing.

To address the question of what is qualitative we turn to the accounts of “qualitative research” in textbooks and also in empirical work. In his classic, explorative, interview study of deviance Howard Becker ( 1963 ) asks ‘How does one become a marijuana user?’ In contrast to pre-dispositional and psychological-individualistic theories of deviant behavior, Becker’s inherently social explanation contends that becoming a user of this substance is the result of a three-phase sequential learning process. First, potential users need to learn how to smoke it properly to produce the “correct” effects. If not, they are likely to stop experimenting with it. Second, they need to discover the effects associated with it; in other words, to get “high,” individuals not only have to experience what the drug does, but also to become aware that those sensations are related to using it. Third, they require learning to savor the feelings related to its consumption – to develop an acquired taste. Becker, who played music himself, gets close to the phenomenon by observing, taking part, and by talking to people consuming the drug: “half of the fifty interviews were conducted with musicians, the other half covered a wide range of people, including laborers, machinists, and people in the professions” (Becker 1963 :56).

Another central aspect derived through the common-to-all-research interplay between induction and deduction (Becker 2017 ), is that during the course of his research Becker adds scientifically meaningful new distinctions in the form of three phases—distinctions, or findings if you will, that strongly affect the course of his research: its focus, the material that he collects, and which eventually impact his findings. Each phase typically unfolds through social interaction, and often with input from experienced users in “a sequence of social experiences during which the person acquires a conception of the meaning of the behavior, and perceptions and judgments of objects and situations, all of which make the activity possible and desirable” (Becker 1963 :235). In this study the increased understanding of smoking dope is a result of a combination of the meaning of the actors, and the conceptual distinctions that Becker introduces based on the views expressed by his respondents. Understanding is the result of research and is due to an iterative process in which data, concepts and evidence are connected with one another (Becker 2017 ).

Indeed, there are many definitions of qualitative research, but if we look for a definition that addresses its distinctive feature of being “qualitative,” the literature across the broad field of social science is meager. The main reason behind this article lies in the paradox, which, to put it bluntly, is that researchers act as if they know what it is, but they cannot formulate a coherent definition. Sociologists and others will of course continue to conduct good studies that show the relevance and value of qualitative research addressing scientific and practical problems in society. However, our paper is grounded in the idea that providing a clear definition will help us improve the work that we do. Among researchers who practice qualitative research there is clearly much knowledge. We suggest that a definition makes this knowledge more explicit. If the first rationale for writing this paper refers to the “internal” aim of improving qualitative research, the second refers to the increased “external” pressure that especially many qualitative researchers feel; pressure that comes both from society as well as from other scientific approaches. There is a strong core in qualitative research, and leading researchers tend to agree on what it is and how it is done. Our critique is not directed at the practice of qualitative research, but we do claim that the type of systematic work we do has not yet been done, and that it is useful to improve the field and its status in relation to quantitative research.

The literature on the “internal” aim of improving, or at least clarifying qualitative research is large, and we do not claim to be the first to notice the vagueness of the term “qualitative” (Strauss and Corbin 1998 ). Also, others have noted that there is no single definition of it (Long and Godfrey 2004 :182), that there are many different views on qualitative research (Denzin and Lincoln 2003 :11; Jovanović 2011 :3), and that more generally, we need to define its meaning (Best 2004 :54). Strauss and Corbin ( 1998 ), for example, as well as Nelson et al. (1992:2 cited in Denzin and Lincoln 2003 :11), and Flick ( 2007 :ix–x), have recognized that the term is problematic: “Actually, the term ‘qualitative research’ is confusing because it can mean different things to different people” (Strauss and Corbin 1998 :10–11). Hammersley has discussed the possibility of addressing the problem, but states that “the task of providing an account of the distinctive features of qualitative research is far from straightforward” ( 2013 :2). This confusion, as he has recently further argued (Hammersley 2018 ), is also salient in relation to ethnography where different philosophical and methodological approaches lead to a lack of agreement about what it means.

Others (e.g. Hammersley 2018 ; Fine and Hancock 2017 ) have also identified the treat to qualitative research that comes from external forces, seen from the point of view of “qualitative research.” This threat can be further divided into that which comes from inside academia, such as the critique voiced by “quantitative research” and outside of academia, including, for example, New Public Management. Hammersley ( 2018 ), zooming in on one type of qualitative research, ethnography, has argued that it is under treat. Similarly to Fine ( 2003 ), and before him Gans ( 1999 ), he writes that ethnography’ has acquired a range of meanings, and comes in many different versions, these often reflecting sharply divergent epistemological orientations. And already more than twenty years ago while reviewing Denzin and Lincoln’ s Handbook of Qualitative Methods Fine argued:

While this increasing centrality [of qualitative research] might lead one to believe that consensual standards have developed, this belief would be misleading. As the methodology becomes more widely accepted, querulous challengers have raised fundamental questions that collectively have undercut the traditional models of how qualitative research is to be fashioned and presented (1995:417).

According to Hammersley, there are today “serious treats to the practice of ethnographic work, on almost any definition” ( 2018 :1). He lists five external treats: (1) that social research must be accountable and able to show its impact on society; (2) the current emphasis on “big data” and the emphasis on quantitative data and evidence; (3) the labor market pressure in academia that leaves less time for fieldwork (see also Fine and Hancock 2017 ); (4) problems of access to fields; and (5) the increased ethical scrutiny of projects, to which ethnography is particularly exposed. Hammersley discusses some more or less insufficient existing definitions of ethnography.

The current situation, as Hammersley and others note—and in relation not only to ethnography but also qualitative research in general, and as our empirical study shows—is not just unsatisfactory, it may even be harmful for the entire field of qualitative research, and does not help social science at large. We suggest that the lack of clarity of qualitative research is a real problem that must be addressed.

Towards a Definition of Qualitative Research

Seen in an historical light, what is today called qualitative, or sometimes ethnographic, interpretative research – or a number of other terms – has more or less always existed. At the time the founders of sociology – Simmel, Weber, Durkheim and, before them, Marx – were writing, and during the era of the Methodenstreit (“dispute about methods”) in which the German historical school emphasized scientific methods (cf. Swedberg 1990 ), we can at least speak of qualitative forerunners.

Perhaps the most extended discussion of what later became known as qualitative methods in a classic work is Bronisław Malinowski’s ( 1922 ) Argonauts in the Western Pacific , although even this study does not explicitly address the meaning of “qualitative.” In Weber’s ([1921–-22] 1978) work we find a tension between scientific explanations that are based on observation and quantification and interpretative research (see also Lazarsfeld and Barton 1982 ).

If we look through major sociology journals like the American Sociological Review , American Journal of Sociology , or Social Forces we will not find the term qualitative sociology before the 1970s. And certainly before then much of what we consider qualitative classics in sociology, like Becker’ study ( 1963 ), had already been produced. Indeed, the Chicago School often combined qualitative and quantitative data within the same study (Fine 1995 ). Our point being that before a disciplinary self-awareness the term quantitative preceded qualitative, and the articulation of the former was a political move to claim scientific status (Denzin and Lincoln 2005 ). In the US the World War II seem to have sparked a critique of sociological work, including “qualitative work,” that did not follow the scientific canon (Rawls 2018 ), which was underpinned by a scientifically oriented and value free philosophy of science. As a result the attempts and practice of integrating qualitative and quantitative sociology at Chicago lost ground to sociology that was more oriented to surveys and quantitative work at Columbia under Merton-Lazarsfeld. The quantitative tradition was also able to present textbooks (Lundberg 1951 ) that facilitated the use this approach and its “methods.” The practices of the qualitative tradition, by and large, remained tacit or was part of the mentoring transferred from the renowned masters to their students.

This glimpse into history leads us back to the lack of a coherent account condensed in a definition of qualitative research. Many of the attempts to define the term do not meet the requirements of a proper definition: A definition should be clear, avoid tautology, demarcate its domain in relation to the environment, and ideally only use words in its definiens that themselves are not in need of definition (Hempel 1966 ). A definition can enhance precision and thus clarity by identifying the core of the phenomenon. Preferably, a definition should be short. The typical definition we have found, however, is an ostensive definition, which indicates what qualitative research is about without informing us about what it actually is :

Qualitative research is multimethod in focus, involving an interpretative, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Qualitative research involves the studied use and collection of a variety of empirical materials – case study, personal experience, introspective, life story, interview, observational, historical, interactional, and visual texts – that describe routine and problematic moments and meanings in individuals’ lives. (Denzin and Lincoln 2005 :2)

Flick claims that the label “qualitative research” is indeed used as an umbrella for a number of approaches ( 2007 :2–4; 2002 :6), and it is not difficult to identify research fitting this designation. Moreover, whatever it is, it has grown dramatically over the past five decades. In addition, courses have been developed, methods have flourished, arguments about its future have been advanced (for example, Denzin and Lincoln 1994) and criticized (for example, Snow and Morrill 1995 ), and dedicated journals and books have mushroomed. Most social scientists have a clear idea of research and how it differs from journalism, politics and other activities. But the question of what is qualitative in qualitative research is either eluded or eschewed.

We maintain that this lacuna hinders systematic knowledge production based on qualitative research. Paul Lazarsfeld noted the lack of “codification” as early as 1955 when he reviewed 100 qualitative studies in order to offer a codification of the practices (Lazarsfeld and Barton 1982 :239). Since then many texts on “qualitative research” and its methods have been published, including recent attempts (Goertz and Mahoney 2012 ) similar to Lazarsfeld’s. These studies have tried to extract what is qualitative by looking at the large number of empirical “qualitative” studies. Our novel strategy complements these endeavors by taking another approach and looking at the attempts to codify these practices in the form of a definition, as well as to a minor extent take Becker’s study as an exemplar of what qualitative researchers actually do, and what the characteristic of being ‘qualitative’ denotes and implies. We claim that qualitative researchers, if there is such a thing as “qualitative research,” should be able to codify their practices in a condensed, yet general way expressed in language.

Lingering problems of “generalizability” and “how many cases do I need” (Small 2009 ) are blocking advancement – in this line of work qualitative approaches are said to differ considerably from quantitative ones, while some of the former unsuccessfully mimic principles related to the latter (Small 2009 ). Additionally, quantitative researchers sometimes unfairly criticize the first based on their own quality criteria. Scholars like Goertz and Mahoney ( 2012 ) have successfully focused on the different norms and practices beyond what they argue are essentially two different cultures: those working with either qualitative or quantitative methods. Instead, similarly to Becker ( 2017 ) who has recently questioned the usefulness of the distinction between qualitative and quantitative research, we focus on similarities.

The current situation also impedes both students and researchers in focusing their studies and understanding each other’s work (Lazarsfeld and Barton 1982 :239). A third consequence is providing an opening for critiques by scholars operating within different traditions (Valsiner 2000 :101). A fourth issue is that the “implicit use of methods in qualitative research makes the field far less standardized than the quantitative paradigm” (Goertz and Mahoney 2012 :9). Relatedly, the National Science Foundation in the US organized two workshops in 2004 and 2005 to address the scientific foundations of qualitative research involving strategies to improve it and to develop standards of evaluation in qualitative research. However, a specific focus on its distinguishing feature of being “qualitative” while being implicitly acknowledged, was discussed only briefly (for example, Best 2004 ).

In 2014 a theme issue was published in this journal on “Methods, Materials, and Meanings: Designing Cultural Analysis,” discussing central issues in (cultural) qualitative research (Berezin 2014 ; Biernacki 2014 ; Glaeser 2014 ; Lamont and Swidler 2014 ; Spillman 2014). We agree with many of the arguments put forward, such as the risk of methodological tribalism, and that we should not waste energy on debating methods separated from research questions. Nonetheless, a clarification of the relation to what is called “quantitative research” is of outmost importance to avoid misunderstandings and misguided debates between “qualitative” and “quantitative” researchers. Our strategy means that researchers, “qualitative” or “quantitative” they may be, in their actual practice may combine qualitative work and quantitative work.

In this article we accomplish three tasks. First, we systematically survey the literature for meanings of qualitative research by looking at how researchers have defined it. Drawing upon existing knowledge we find that the different meanings and ideas of qualitative research are not yet coherently integrated into one satisfactory definition. Next, we advance our contribution by offering a definition of qualitative research and illustrate its meaning and use partially by expanding on the brief example introduced earlier related to Becker’s work ( 1963 ). We offer a systematic analysis of central themes of what researchers consider to be the core of “qualitative,” regardless of style of work. These themes – which we summarize in terms of four keywords: distinction, process, closeness, improved understanding – constitute part of our literature review, in which each one appears, sometimes with others, but never all in the same definition. They serve as the foundation of our contribution. Our categories are overlapping. Their use is primarily to organize the large amount of definitions we have identified and analyzed, and not necessarily to draw a clear distinction between them. Finally, we continue the elaboration discussed above on the advantages of a clear definition of qualitative research.

In a hermeneutic fashion we propose that there is something meaningful that deserves to be labelled “qualitative research” (Gadamer 1990 ). To approach the question “What is qualitative in qualitative research?” we have surveyed the literature. In conducting our survey we first traced the word’s etymology in dictionaries, encyclopedias, handbooks of the social sciences and of methods and textbooks, mainly in English, which is common to methodology courses. It should be noted that we have zoomed in on sociology and its literature. This discipline has been the site of the largest debate and development of methods that can be called “qualitative,” which suggests that this field should be examined in great detail.

In an ideal situation we should expect that one good definition, or at least some common ideas, would have emerged over the years. This common core of qualitative research should be so accepted that it would appear in at least some textbooks. Since this is not what we found, we decided to pursue an inductive approach to capture maximal variation in the field of qualitative research; we searched in a selection of handbooks, textbooks, book chapters, and books, to which we added the analysis of journal articles. Our sample comprises a total of 89 references.

In practice we focused on the discipline that has had a clear discussion of methods, namely sociology. We also conducted a broad search in the JSTOR database to identify scholarly sociology articles published between 1998 and 2017 in English with a focus on defining or explaining qualitative research. We specifically zoom in on this time frame because we would have expect that this more mature period would have produced clear discussions on the meaning of qualitative research. To find these articles we combined a number of keywords to search the content and/or the title: qualitative (which was always included), definition, empirical, research, methodology, studies, fieldwork, interview and observation .

As a second phase of our research we searched within nine major sociological journals ( American Journal of Sociology , Sociological Theory , American Sociological Review , Contemporary Sociology , Sociological Forum , Sociological Theory , Qualitative Research , Qualitative Sociology and Qualitative Sociology Review ) for articles also published during the past 19 years (1998–2017) that had the term “qualitative” in the title and attempted to define qualitative research.

Lastly we picked two additional journals, Qualitative Research and Qualitative Sociology , in which we could expect to find texts addressing the notion of “qualitative.” From Qualitative Research we chose Volume 14, Issue 6, December 2014, and from Qualitative Sociology we chose Volume 36, Issue 2, June 2017. Within each of these we selected the first article; then we picked the second article of three prior issues. Again we went back another three issues and investigated article number three. Finally we went back another three issues and perused article number four. This selection criteria was used to get a manageable sample for the analysis.

The coding process of the 89 references we gathered in our selected review began soon after the first round of material was gathered, and we reduced the complexity created by our maximum variation sampling (Snow and Anderson 1993 :22) to four different categories within which questions on the nature and properties of qualitative research were discussed. We call them: Qualitative and Quantitative Research, Qualitative Research, Fieldwork, and Grounded Theory. This – which may appear as an illogical grouping – merely reflects the “context” in which the matter of “qualitative” is discussed. If the selection process of the material – books and articles – was informed by pre-knowledge, we used an inductive strategy to code the material. When studying our material, we identified four central notions related to “qualitative” that appear in various combinations in the literature which indicate what is the core of qualitative research. We have labeled them: “distinctions”, “process,” “closeness,” and “improved understanding.” During the research process the categories and notions were improved, refined, changed, and reordered. The coding ended when a sense of saturation in the material arose. In the presentation below all quotations and references come from our empirical material of texts on qualitative research.

Analysis – What is Qualitative Research?

In this section we describe the four categories we identified in the coding, how they differently discuss qualitative research, as well as their overall content. Some salient quotations are selected to represent the type of text sorted under each of the four categories. What we present are examples from the literature.

Qualitative and Quantitative

This analytic category comprises quotations comparing qualitative and quantitative research, a distinction that is frequently used (Brown 2010 :231); in effect this is a conceptual pair that structures the discussion and that may be associated with opposing interests. While the general goal of quantitative and qualitative research is the same – to understand the world better – their methodologies and focus in certain respects differ substantially (Becker 1966 :55). Quantity refers to that property of something that can be determined by measurement. In a dictionary of Statistics and Methodology we find that “(a) When referring to *variables, ‘qualitative’ is another term for *categorical or *nominal. (b) When speaking of kinds of research, ‘qualitative’ refers to studies of subjects that are hard to quantify, such as art history. Qualitative research tends to be a residual category for almost any kind of non-quantitative research” (Stiles 1998:183). But it should be obvious that one could employ a quantitative approach when studying, for example, art history.

The same dictionary states that quantitative is “said of variables or research that can be handled numerically, usually (too sharply) contrasted with *qualitative variables and research” (Stiles 1998:184). From a qualitative perspective “quantitative research” is about numbers and counting, and from a quantitative perspective qualitative research is everything that is not about numbers. But this does not say much about what is “qualitative.” If we turn to encyclopedias we find that in the 1932 edition of the Encyclopedia of the Social Sciences there is no mention of “qualitative.” In the Encyclopedia from 1968 we can read:

Qualitative Analysis. For methods of obtaining, analyzing, and describing data, see [the various entries:] CONTENT ANALYSIS; COUNTED DATA; EVALUATION RESEARCH, FIELD WORK; GRAPHIC PRESENTATION; HISTORIOGRAPHY, especially the article on THE RHETORIC OF HISTORY; INTERVIEWING; OBSERVATION; PERSONALITY MEASUREMENT; PROJECTIVE METHODS; PSYCHOANALYSIS, article on EXPERIMENTAL METHODS; SURVEY ANALYSIS, TABULAR PRESENTATION; TYPOLOGIES. (Vol. 13:225)

Some, like Alford, divide researchers into methodologists or, in his words, “quantitative and qualitative specialists” (Alford 1998 :12). Qualitative research uses a variety of methods, such as intensive interviews or in-depth analysis of historical materials, and it is concerned with a comprehensive account of some event or unit (King et al. 1994 :4). Like quantitative research it can be utilized to study a variety of issues, but it tends to focus on meanings and motivations that underlie cultural symbols, personal experiences, phenomena and detailed understanding of processes in the social world. In short, qualitative research centers on understanding processes, experiences, and the meanings people assign to things (Kalof et al. 2008 :79).

Others simply say that qualitative methods are inherently unscientific (Jovanović 2011 :19). Hood, for instance, argues that words are intrinsically less precise than numbers, and that they are therefore more prone to subjective analysis, leading to biased results (Hood 2006 :219). Qualitative methodologies have raised concerns over the limitations of quantitative templates (Brady et al. 2004 :4). Scholars such as King et al. ( 1994 ), for instance, argue that non-statistical research can produce more reliable results if researchers pay attention to the rules of scientific inference commonly stated in quantitative research. Also, researchers such as Becker ( 1966 :59; 1970 :42–43) have asserted that, if conducted properly, qualitative research and in particular ethnographic field methods, can lead to more accurate results than quantitative studies, in particular, survey research and laboratory experiments.

Some researchers, such as Kalof, Dan, and Dietz ( 2008 :79) claim that the boundaries between the two approaches are becoming blurred, and Small ( 2009 ) argues that currently much qualitative research (especially in North America) tries unsuccessfully and unnecessarily to emulate quantitative standards. For others, qualitative research tends to be more humanistic and discursive (King et al. 1994 :4). Ragin ( 1994 ), and similarly also Becker, ( 1996 :53), Marchel and Owens ( 2007 :303) think that the main distinction between the two styles is overstated and does not rest on the simple dichotomy of “numbers versus words” (Ragin 1994 :xii). Some claim that quantitative data can be utilized to discover associations, but in order to unveil cause and effect a complex research design involving the use of qualitative approaches needs to be devised (Gilbert 2009 :35). Consequently, qualitative data are useful for understanding the nuances lying beyond those processes as they unfold (Gilbert 2009 :35). Others contend that qualitative research is particularly well suited both to identify causality and to uncover fine descriptive distinctions (Fine and Hallett 2014 ; Lichterman and Isaac Reed 2014 ; Katz 2015 ).

There are other ways to separate these two traditions, including normative statements about what qualitative research should be (that is, better or worse than quantitative approaches, concerned with scientific approaches to societal change or vice versa; Snow and Morrill 1995 ; Denzin and Lincoln 2005 ), or whether it should develop falsifiable statements; Best 2004 ).

We propose that quantitative research is largely concerned with pre-determined variables (Small 2008 ); the analysis concerns the relations between variables. These categories are primarily not questioned in the study, only their frequency or degree, or the correlations between them (cf. Franzosi 2016 ). If a researcher studies wage differences between women and men, he or she works with given categories: x number of men are compared with y number of women, with a certain wage attributed to each person. The idea is not to move beyond the given categories of wage, men and women; they are the starting point as well as the end point, and undergo no “qualitative change.” Qualitative research, in contrast, investigates relations between categories that are themselves subject to change in the research process. Returning to Becker’s study ( 1963 ), we see that he questioned pre-dispositional theories of deviant behavior working with pre-determined variables such as an individual’s combination of personal qualities or emotional problems. His take, in contrast, was to understand marijuana consumption by developing “variables” as part of the investigation. Thereby he presented new variables, or as we would say today, theoretical concepts, but which are grounded in the empirical material.

Qualitative Research

This category contains quotations that refer to descriptions of qualitative research without making comparisons with quantitative research. Researchers such as Denzin and Lincoln, who have written a series of influential handbooks on qualitative methods (1994; Denzin and Lincoln 2003 ; 2005 ), citing Nelson et al. (1992:4), argue that because qualitative research is “interdisciplinary, transdisciplinary, and sometimes counterdisciplinary” it is difficult to derive one single definition of it (Jovanović 2011 :3). According to them, in fact, “the field” is “many things at the same time,” involving contradictions, tensions over its focus, methods, and how to derive interpretations and findings ( 2003 : 11). Similarly, others, such as Flick ( 2007 :ix–x) contend that agreeing on an accepted definition has increasingly become problematic, and that qualitative research has possibly matured different identities. However, Best holds that “the proliferation of many sorts of activities under the label of qualitative sociology threatens to confuse our discussions” ( 2004 :54). Atkinson’s position is more definite: “the current state of qualitative research and research methods is confused” ( 2005 :3–4).

Qualitative research is about interpretation (Blumer 1969 ; Strauss and Corbin 1998 ; Denzin and Lincoln 2003 ), or Verstehen [understanding] (Frankfort-Nachmias and Nachmias 1996 ). It is “multi-method,” involving the collection and use of a variety of empirical materials (Denzin and Lincoln 1998; Silverman 2013 ) and approaches (Silverman 2005 ; Flick 2007 ). It focuses not only on the objective nature of behavior but also on its subjective meanings: individuals’ own accounts of their attitudes, motivations, behavior (McIntyre 2005 :127; Creswell 2009 ), events and situations (Bryman 1989) – what people say and do in specific places and institutions (Goodwin and Horowitz 2002 :35–36) in social and temporal contexts (Morrill and Fine 1997). For this reason, following Weber ([1921-22] 1978), it can be described as an interpretative science (McIntyre 2005 :127). But could quantitative research also be concerned with these questions? Also, as pointed out below, does all qualitative research focus on subjective meaning, as some scholars suggest?

Others also distinguish qualitative research by claiming that it collects data using a naturalistic approach (Denzin and Lincoln 2005 :2; Creswell 2009 ), focusing on the meaning actors ascribe to their actions. But again, does all qualitative research need to be collected in situ? And does qualitative research have to be inherently concerned with meaning? Flick ( 2007 ), referring to Denzin and Lincoln ( 2005 ), mentions conversation analysis as an example of qualitative research that is not concerned with the meanings people bring to a situation, but rather with the formal organization of talk. Still others, such as Ragin ( 1994 :85), note that qualitative research is often (especially early on in the project, we would add) less structured than other kinds of social research – a characteristic connected to its flexibility and that can lead both to potentially better, but also worse results. But is this not a feature of this type of research, rather than a defining description of its essence? Wouldn’t this comment also apply, albeit to varying degrees, to quantitative research?

In addition, Strauss ( 2003 ), along with others, such as Alvesson and Kärreman ( 2011 :10–76), argue that qualitative researchers struggle to capture and represent complex phenomena partially because they tend to collect a large amount of data. While his analysis is correct at some points – “It is necessary to do detailed, intensive, microscopic examination of the data in order to bring out the amazing complexity of what lies in, behind, and beyond those data” (Strauss 2003 :10) – much of his analysis concerns the supposed focus of qualitative research and its challenges, rather than exactly what it is about. But even in this instance we would make a weak case arguing that these are strictly the defining features of qualitative research. Some researchers seem to focus on the approach or the methods used, or even on the way material is analyzed. Several researchers stress the naturalistic assumption of investigating the world, suggesting that meaning and interpretation appear to be a core matter of qualitative research.

We can also see that in this category there is no consensus about specific qualitative methods nor about qualitative data. Many emphasize interpretation, but quantitative research, too, involves interpretation; the results of a regression analysis, for example, certainly have to be interpreted, and the form of meta-analysis that factor analysis provides indeed requires interpretation However, there is no interpretation of quantitative raw data, i.e., numbers in tables. One common thread is that qualitative researchers have to get to grips with their data in order to understand what is being studied in great detail, irrespective of the type of empirical material that is being analyzed. This observation is connected to the fact that qualitative researchers routinely make several adjustments of focus and research design as their studies progress, in many cases until the very end of the project (Kalof et al. 2008 ). If you, like Becker, do not start out with a detailed theory, adjustments such as the emergence and refinement of research questions will occur during the research process. We have thus found a number of useful reflections about qualitative research scattered across different sources, but none of them effectively describe the defining characteristics of this approach.

Although qualitative research does not appear to be defined in terms of a specific method, it is certainly common that fieldwork, i.e., research that entails that the researcher spends considerable time in the field that is studied and use the knowledge gained as data, is seen as emblematic of or even identical to qualitative research. But because we understand that fieldwork tends to focus primarily on the collection and analysis of qualitative data, we expected to find within it discussions on the meaning of “qualitative.” But, again, this was not the case.

Instead, we found material on the history of this approach (for example, Frankfort-Nachmias and Nachmias 1996 ; Atkinson et al. 2001), including how it has changed; for example, by adopting a more self-reflexive practice (Heyl 2001), as well as the different nomenclature that has been adopted, such as fieldwork, ethnography, qualitative research, naturalistic research, participant observation and so on (for example, Lofland et al. 2006 ; Gans 1999 ).

We retrieved definitions of ethnography, such as “the study of people acting in the natural courses of their daily lives,” involving a “resocialization of the researcher” (Emerson 1988 :1) through intense immersion in others’ social worlds (see also examples in Hammersley 2018 ). This may be accomplished by direct observation and also participation (Neuman 2007 :276), although others, such as Denzin ( 1970 :185), have long recognized other types of observation, including non-participant (“fly on the wall”). In this category we have also isolated claims and opposing views, arguing that this type of research is distinguished primarily by where it is conducted (natural settings) (Hughes 1971:496), and how it is carried out (a variety of methods are applied) or, for some most importantly, by involving an active, empathetic immersion in those being studied (Emerson 1988 :2). We also retrieved descriptions of the goals it attends in relation to how it is taught (understanding subjective meanings of the people studied, primarily develop theory, or contribute to social change) (see for example, Corte and Irwin 2017 ; Frankfort-Nachmias and Nachmias 1996 :281; Trier-Bieniek 2012 :639) by collecting the richest possible data (Lofland et al. 2006 ) to derive “thick descriptions” (Geertz 1973 ), and/or to aim at theoretical statements of general scope and applicability (for example, Emerson 1988 ; Fine 2003 ). We have identified guidelines on how to evaluate it (for example Becker 1996 ; Lamont 2004 ) and have retrieved instructions on how it should be conducted (for example, Lofland et al. 2006 ). For instance, analysis should take place while the data gathering unfolds (Emerson 1988 ; Hammersley and Atkinson 2007 ; Lofland et al. 2006 ), observations should be of long duration (Becker 1970 :54; Goffman 1989 ), and data should be of high quantity (Becker 1970 :52–53), as well as other questionable distinctions between fieldwork and other methods:

Field studies differ from other methods of research in that the researcher performs the task of selecting topics, decides what questions to ask, and forges interest in the course of the research itself . This is in sharp contrast to many ‘theory-driven’ and ‘hypothesis-testing’ methods. (Lofland and Lofland 1995 :5)

But could not, for example, a strictly interview-based study be carried out with the same amount of flexibility, such as sequential interviewing (for example, Small 2009 )? Once again, are quantitative approaches really as inflexible as some qualitative researchers think? Moreover, this category stresses the role of the actors’ meaning, which requires knowledge and close interaction with people, their practices and their lifeworld.

It is clear that field studies – which are seen by some as the “gold standard” of qualitative research – are nonetheless only one way of doing qualitative research. There are other methods, but it is not clear why some are more qualitative than others, or why they are better or worse. Fieldwork is characterized by interaction with the field (the material) and understanding of the phenomenon that is being studied. In Becker’s case, he had general experience from fields in which marihuana was used, based on which he did interviews with actual users in several fields.

Grounded Theory

Another major category we identified in our sample is Grounded Theory. We found descriptions of it most clearly in Glaser and Strauss’ ([1967] 2010 ) original articulation, Strauss and Corbin ( 1998 ) and Charmaz ( 2006 ), as well as many other accounts of what it is for: generating and testing theory (Strauss 2003 :xi). We identified explanations of how this task can be accomplished – such as through two main procedures: constant comparison and theoretical sampling (Emerson 1998:96), and how using it has helped researchers to “think differently” (for example, Strauss and Corbin 1998 :1). We also read descriptions of its main traits, what it entails and fosters – for instance, an exceptional flexibility, an inductive approach (Strauss and Corbin 1998 :31–33; 1990; Esterberg 2002 :7), an ability to step back and critically analyze situations, recognize tendencies towards bias, think abstractly and be open to criticism, enhance sensitivity towards the words and actions of respondents, and develop a sense of absorption and devotion to the research process (Strauss and Corbin 1998 :5–6). Accordingly, we identified discussions of the value of triangulating different methods (both using and not using grounded theory), including quantitative ones, and theories to achieve theoretical development (most comprehensively in Denzin 1970 ; Strauss and Corbin 1998 ; Timmermans and Tavory 2012 ). We have also located arguments about how its practice helps to systematize data collection, analysis and presentation of results (Glaser and Strauss [1967] 2010 :16).

Grounded theory offers a systematic approach which requires researchers to get close to the field; closeness is a requirement of identifying questions and developing new concepts or making further distinctions with regard to old concepts. In contrast to other qualitative approaches, grounded theory emphasizes the detailed coding process, and the numerous fine-tuned distinctions that the researcher makes during the process. Within this category, too, we could not find a satisfying discussion of the meaning of qualitative research.

Defining Qualitative Research

In sum, our analysis shows that some notions reappear in the discussion of qualitative research, such as understanding, interpretation, “getting close” and making distinctions. These notions capture aspects of what we think is “qualitative.” However, a comprehensive definition that is useful and that can further develop the field is lacking, and not even a clear picture of its essential elements appears. In other words no definition emerges from our data, and in our research process we have moved back and forth between our empirical data and the attempt to present a definition. Our concrete strategy, as stated above, is to relate qualitative and quantitative research, or more specifically, qualitative and quantitative work. We use an ideal-typical notion of quantitative research which relies on taken for granted and numbered variables. This means that the data consists of variables on different scales, such as ordinal, but frequently ratio and absolute scales, and the representation of the numbers to the variables, i.e. the justification of the assignment of numbers to object or phenomenon, are not questioned, though the validity may be questioned. In this section we return to the notion of quality and try to clarify it while presenting our contribution.

Broadly, research refers to the activity performed by people trained to obtain knowledge through systematic procedures. Notions such as “objectivity” and “reflexivity,” “systematic,” “theory,” “evidence” and “openness” are here taken for granted in any type of research. Next, building on our empirical analysis we explain the four notions that we have identified as central to qualitative work: distinctions, process, closeness, and improved understanding. In discussing them, ultimately in relation to one another, we make their meaning even more precise. Our idea, in short, is that only when these ideas that we present separately for analytic purposes are brought together can we speak of qualitative research.

Distinctions

We believe that the possibility of making new distinctions is one the defining characteristics of qualitative research. It clearly sets it apart from quantitative analysis which works with taken-for-granted variables, albeit as mentioned, meta-analyses, for example, factor analysis may result in new variables. “Quality” refers essentially to distinctions, as already pointed out by Aristotle. He discusses the term “qualitative” commenting: “By a quality I mean that in virtue of which things are said to be qualified somehow” (Aristotle 1984:14). Quality is about what something is or has, which means that the distinction from its environment is crucial. We see qualitative research as a process in which significant new distinctions are made to the scholarly community; to make distinctions is a key aspect of obtaining new knowledge; a point, as we will see, that also has implications for “quantitative research.” The notion of being “significant” is paramount. New distinctions by themselves are not enough; just adding concepts only increases complexity without furthering our knowledge. The significance of new distinctions is judged against the communal knowledge of the research community. To enable this discussion and judgements central elements of rational discussion are required (cf. Habermas [1981] 1987 ; Davidsson [ 1988 ] 2001) to identify what is new and relevant scientific knowledge. Relatedly, Ragin alludes to the idea of new and useful knowledge at a more concrete level: “Qualitative methods are appropriate for in-depth examination of cases because they aid the identification of key features of cases. Most qualitative methods enhance data” (1994:79). When Becker ( 1963 ) studied deviant behavior and investigated how people became marihuana smokers, he made distinctions between the ways in which people learned how to smoke. This is a classic example of how the strategy of “getting close” to the material, for example the text, people or pictures that are subject to analysis, may enable researchers to obtain deeper insight and new knowledge by making distinctions – in this instance on the initial notion of learning how to smoke. Others have stressed the making of distinctions in relation to coding or theorizing. Emerson et al. ( 1995 ), for example, hold that “qualitative coding is a way of opening up avenues of inquiry,” meaning that the researcher identifies and develops concepts and analytic insights through close examination of and reflection on data (Emerson et al. 1995 :151). Goodwin and Horowitz highlight making distinctions in relation to theory-building writing: “Close engagement with their cases typically requires qualitative researchers to adapt existing theories or to make new conceptual distinctions or theoretical arguments to accommodate new data” ( 2002 : 37). In the ideal-typical quantitative research only existing and so to speak, given, variables would be used. If this is the case no new distinction are made. But, would not also many “quantitative” researchers make new distinctions?

Process does not merely suggest that research takes time. It mainly implies that qualitative new knowledge results from a process that involves several phases, and above all iteration. Qualitative research is about oscillation between theory and evidence, analysis and generating material, between first- and second -order constructs (Schütz 1962 :59), between getting in contact with something, finding sources, becoming deeply familiar with a topic, and then distilling and communicating some of its essential features. The main point is that the categories that the researcher uses, and perhaps takes for granted at the beginning of the research process, usually undergo qualitative changes resulting from what is found. Becker describes how he tested hypotheses and let the jargon of the users develop into theoretical concepts. This happens over time while the study is being conducted, exemplifying what we mean by process.

In the research process, a pilot-study may be used to get a first glance of, for example, the field, how to approach it, and what methods can be used, after which the method and theory are chosen or refined before the main study begins. Thus, the empirical material is often central from the start of the project and frequently leads to adjustments by the researcher. Likewise, during the main study categories are not fixed; the empirical material is seen in light of the theory used, but it is also given the opportunity to kick back, thereby resisting attempts to apply theoretical straightjackets (Becker 1970 :43). In this process, coding and analysis are interwoven, and thus are often important steps for getting closer to the phenomenon and deciding what to focus on next. Becker began his research by interviewing musicians close to him, then asking them to refer him to other musicians, and later on doubling his original sample of about 25 to include individuals in other professions (Becker 1973:46). Additionally, he made use of some participant observation, documents, and interviews with opiate users made available to him by colleagues. As his inductive theory of deviance evolved, Becker expanded his sample in order to fine tune it, and test the accuracy and generality of his hypotheses. In addition, he introduced a negative case and discussed the null hypothesis ( 1963 :44). His phasic career model is thus based on a research design that embraces processual work. Typically, process means to move between “theory” and “material” but also to deal with negative cases, and Becker ( 1998 ) describes how discovering these negative cases impacted his research design and ultimately its findings.

Obviously, all research is process-oriented to some degree. The point is that the ideal-typical quantitative process does not imply change of the data, and iteration between data, evidence, hypotheses, empirical work, and theory. The data, quantified variables, are, in most cases fixed. Merging of data, which of course can be done in a quantitative research process, does not mean new data. New hypotheses are frequently tested, but the “raw data is often the “the same.” Obviously, over time new datasets are made available and put into use.

Another characteristic that is emphasized in our sample is that qualitative researchers – and in particular ethnographers – can, or as Goffman put it, ought to ( 1989 ), get closer to the phenomenon being studied and their data than quantitative researchers (for example, Silverman 2009 :85). Put differently, essentially because of their methods qualitative researchers get into direct close contact with those being investigated and/or the material, such as texts, being analyzed. Becker started out his interview study, as we noted, by talking to those he knew in the field of music to get closer to the phenomenon he was studying. By conducting interviews he got even closer. Had he done more observations, he would undoubtedly have got even closer to the field.

Additionally, ethnographers’ design enables researchers to follow the field over time, and the research they do is almost by definition longitudinal, though the time in the field is studied obviously differs between studies. The general characteristic of closeness over time maximizes the chances of unexpected events, new data (related, for example, to archival research as additional sources, and for ethnography for situations not necessarily previously thought of as instrumental – what Mannay and Morgan ( 2015 ) term the “waiting field”), serendipity (Merton and Barber 2004 ; Åkerström 2013 ), and possibly reactivity, as well as the opportunity to observe disrupted patterns that translate into exemplars of negative cases. Two classic examples of this are Becker’s finding of what medical students call “crocks” (Becker et al. 1961 :317), and Geertz’s ( 1973 ) study of “deep play” in Balinese society.

By getting and staying so close to their data – be it pictures, text or humans interacting (Becker was himself a musician) – for a long time, as the research progressively focuses, qualitative researchers are prompted to continually test their hunches, presuppositions and hypotheses. They test them against a reality that often (but certainly not always), and practically, as well as metaphorically, talks back, whether by validating them, or disqualifying their premises – correctly, as well as incorrectly (Fine 2003 ; Becker 1970 ). This testing nonetheless often leads to new directions for the research. Becker, for example, says that he was initially reading psychological theories, but when facing the data he develops a theory that looks at, you may say, everything but psychological dispositions to explain the use of marihuana. Especially researchers involved with ethnographic methods have a fairly unique opportunity to dig up and then test (in a circular, continuous and temporal way) new research questions and findings as the research progresses, and thereby to derive previously unimagined and uncharted distinctions by getting closer to the phenomenon under study.

Let us stress that getting close is by no means restricted to ethnography. The notion of hermeneutic circle and hermeneutics as a general way of understanding implies that we must get close to the details in order to get the big picture. This also means that qualitative researchers can literally also make use of details of pictures as evidence (cf. Harper 2002). Thus, researchers may get closer both when generating the material or when analyzing it.

Quantitative research, we maintain, in the ideal-typical representation cannot get closer to the data. The data is essentially numbers in tables making up the variables (Franzosi 2016 :138). The data may originally have been “qualitative,” but once reduced to numbers there can only be a type of “hermeneutics” about what the number may stand for. The numbers themselves, however, are non-ambiguous. Thus, in quantitative research, interpretation, if done, is not about the data itself—the numbers—but what the numbers stand for. It follows that the interpretation is essentially done in a more “speculative” mode without direct empirical evidence (cf. Becker 2017 ).

Improved Understanding

While distinction, process and getting closer refer to the qualitative work of the researcher, improved understanding refers to its conditions and outcome of this work. Understanding cuts deeper than explanation, which to some may mean a causally verified correlation between variables. The notion of explanation presupposes the notion of understanding since explanation does not include an idea of how knowledge is gained (Manicas 2006 : 15). Understanding, we argue, is the core concept of what we call the outcome of the process when research has made use of all the other elements that were integrated in the research. Understanding, then, has a special status in qualitative research since it refers both to the conditions of knowledge and the outcome of the process. Understanding can to some extent be seen as the condition of explanation and occurs in a process of interpretation, which naturally refers to meaning (Gadamer 1990 ). It is fundamentally connected to knowing, and to the knowing of how to do things (Heidegger [1927] 2001 ). Conceptually the term hermeneutics is used to account for this process. Heidegger ties hermeneutics to human being and not possible to separate from the understanding of being ( 1988 ). Here we use it in a broader sense, and more connected to method in general (cf. Seiffert 1992 ). The abovementioned aspects – for example, “objectivity” and “reflexivity” – of the approach are conditions of scientific understanding. Understanding is the result of a circular process and means that the parts are understood in light of the whole, and vice versa. Understanding presupposes pre-understanding, or in other words, some knowledge of the phenomenon studied. The pre-understanding, even in the form of prejudices, are in qualitative research process, which we see as iterative, questioned, which gradually or suddenly change due to the iteration of data, evidence and concepts. However, qualitative research generates understanding in the iterative process when the researcher gets closer to the data, e.g., by going back and forth between field and analysis in a process that generates new data that changes the evidence, and, ultimately, the findings. Questioning, to ask questions, and put what one assumes—prejudices and presumption—in question, is central to understand something (Heidegger [1927] 2001 ; Gadamer 1990 :368–384). We propose that this iterative process in which the process of understanding occurs is characteristic of qualitative research.

Improved understanding means that we obtain scientific knowledge of something that we as a scholarly community did not know before, or that we get to know something better. It means that we understand more about how parts are related to one another, and to other things we already understand (see also Fine and Hallett 2014 ). Understanding is an important condition for qualitative research. It is not enough to identify correlations, make distinctions, and work in a process in which one gets close to the field or phenomena. Understanding is accomplished when the elements are integrated in an iterative process.

It is, moreover, possible to understand many things, and researchers, just like children, may come to understand new things every day as they engage with the world. This subjective condition of understanding – namely, that a person gains a better understanding of something –is easily met. To be qualified as “scientific,” the understanding must be general and useful to many; it must be public. But even this generally accessible understanding is not enough in order to speak of “scientific understanding.” Though we as a collective can increase understanding of everything in virtually all potential directions as a result also of qualitative work, we refrain from this “objective” way of understanding, which has no means of discriminating between what we gain in understanding. Scientific understanding means that it is deemed relevant from the scientific horizon (compare Schütz 1962 : 35–38, 46, 63), and that it rests on the pre-understanding that the scientists have and must have in order to understand. In other words, the understanding gained must be deemed useful by other researchers, so that they can build on it. We thus see understanding from a pragmatic, rather than a subjective or objective perspective. Improved understanding is related to the question(s) at hand. Understanding, in order to represent an improvement, must be an improvement in relation to the existing body of knowledge of the scientific community (James [ 1907 ] 1955). Scientific understanding is, by definition, collective, as expressed in Weber’s famous note on objectivity, namely that scientific work aims at truths “which … can claim, even for a Chinese, the validity appropriate to an empirical analysis” ([1904] 1949 :59). By qualifying “improved understanding” we argue that it is a general defining characteristic of qualitative research. Becker‘s ( 1966 ) study and other research of deviant behavior increased our understanding of the social learning processes of how individuals start a behavior. And it also added new knowledge about the labeling of deviant behavior as a social process. Few studies, of course, make the same large contribution as Becker’s, but are nonetheless qualitative research.

Understanding in the phenomenological sense, which is a hallmark of qualitative research, we argue, requires meaning and this meaning is derived from the context, and above all the data being analyzed. The ideal-typical quantitative research operates with given variables with different numbers. This type of material is not enough to establish meaning at the level that truly justifies understanding. In other words, many social science explanations offer ideas about correlations or even causal relations, but this does not mean that the meaning at the level of the data analyzed, is understood. This leads us to say that there are indeed many explanations that meet the criteria of understanding, for example the explanation of how one becomes a marihuana smoker presented by Becker. However, we may also understand a phenomenon without explaining it, and we may have potential explanations, or better correlations, that are not really understood.

We may speak more generally of quantitative research and its data to clarify what we see as an important distinction. The “raw data” that quantitative research—as an idealtypical activity, refers to is not available for further analysis; the numbers, once created, are not to be questioned (Franzosi 2016 : 138). If the researcher is to do “more” or “change” something, this will be done by conjectures based on theoretical knowledge or based on the researcher’s lifeworld. Both qualitative and quantitative research is based on the lifeworld, and all researchers use prejudices and pre-understanding in the research process. This idea is present in the works of Heidegger ( 2001 ) and Heisenberg (cited in Franzosi 2010 :619). Qualitative research, as we argued, involves the interaction and questioning of concepts (theory), data, and evidence.

Ragin ( 2004 :22) points out that “a good definition of qualitative research should be inclusive and should emphasize its key strengths and features, not what it lacks (for example, the use of sophisticated quantitative techniques).” We define qualitative research as an iterative process in which improved understanding to the scientific community is achieved by making new significant distinctions resulting from getting closer to the phenomenon studied. Qualitative research, as defined here, is consequently a combination of two criteria: (i) how to do things –namely, generating and analyzing empirical material, in an iterative process in which one gets closer by making distinctions, and (ii) the outcome –improved understanding novel to the scholarly community. Is our definition applicable to our own study? In this study we have closely read the empirical material that we generated, and the novel distinction of the notion “qualitative research” is the outcome of an iterative process in which both deduction and induction were involved, in which we identified the categories that we analyzed. We thus claim to meet the first criteria, “how to do things.” The second criteria cannot be judged but in a partial way by us, namely that the “outcome” —in concrete form the definition-improves our understanding to others in the scientific community.

We have defined qualitative research, or qualitative scientific work, in relation to quantitative scientific work. Given this definition, qualitative research is about questioning the pre-given (taken for granted) variables, but it is thus also about making new distinctions of any type of phenomenon, for example, by coining new concepts, including the identification of new variables. This process, as we have discussed, is carried out in relation to empirical material, previous research, and thus in relation to theory. Theory and previous research cannot be escaped or bracketed. According to hermeneutic principles all scientific work is grounded in the lifeworld, and as social scientists we can thus never fully bracket our pre-understanding.

We have proposed that quantitative research, as an idealtype, is concerned with pre-determined variables (Small 2008 ). Variables are epistemically fixed, but can vary in terms of dimensions, such as frequency or number. Age is an example; as a variable it can take on different numbers. In relation to quantitative research, qualitative research does not reduce its material to number and variables. If this is done the process of comes to a halt, the researcher gets more distanced from her data, and it makes it no longer possible to make new distinctions that increase our understanding. We have above discussed the components of our definition in relation to quantitative research. Our conclusion is that in the research that is called quantitative there are frequent and necessary qualitative elements.

Further, comparative empirical research on researchers primarily working with ”quantitative” approaches and those working with ”qualitative” approaches, we propose, would perhaps show that there are many similarities in practices of these two approaches. This is not to deny dissimilarities, or the different epistemic and ontic presuppositions that may be more or less strongly associated with the two different strands (see Goertz and Mahoney 2012 ). Our point is nonetheless that prejudices and preconceptions about researchers are unproductive, and that as other researchers have argued, differences may be exaggerated (e.g., Becker 1996 : 53, 2017 ; Marchel and Owens 2007 :303; Ragin 1994 ), and that a qualitative dimension is present in both kinds of work.

Several things follow from our findings. The most important result is the relation to quantitative research. In our analysis we have separated qualitative research from quantitative research. The point is not to label individual researchers, methods, projects, or works as either “quantitative” or “qualitative.” By analyzing, i.e., taking apart, the notions of quantitative and qualitative, we hope to have shown the elements of qualitative research. Our definition captures the elements, and how they, when combined in practice, generate understanding. As many of the quotations we have used suggest, one conclusion of our study holds that qualitative approaches are not inherently connected with a specific method. Put differently, none of the methods that are frequently labelled “qualitative,” such as interviews or participant observation, are inherently “qualitative.” What matters, given our definition, is whether one works qualitatively or quantitatively in the research process, until the results are produced. Consequently, our analysis also suggests that those researchers working with what in the literature and in jargon is often called “quantitative research” are almost bound to make use of what we have identified as qualitative elements in any research project. Our findings also suggest that many” quantitative” researchers, at least to some extent, are engaged with qualitative work, such as when research questions are developed, variables are constructed and combined, and hypotheses are formulated. Furthermore, a research project may hover between “qualitative” and “quantitative” or start out as “qualitative” and later move into a “quantitative” (a distinct strategy that is not similar to “mixed methods” or just simply combining induction and deduction). More generally speaking, the categories of “qualitative” and “quantitative,” unfortunately, often cover up practices, and it may lead to “camps” of researchers opposing one another. For example, regardless of the researcher is primarily oriented to “quantitative” or “qualitative” research, the role of theory is neglected (cf. Swedberg 2017 ). Our results open up for an interaction not characterized by differences, but by different emphasis, and similarities.

Let us take two examples to briefly indicate how qualitative elements can fruitfully be combined with quantitative. Franzosi ( 2010 ) has discussed the relations between quantitative and qualitative approaches, and more specifically the relation between words and numbers. He analyzes texts and argues that scientific meaning cannot be reduced to numbers. Put differently, the meaning of the numbers is to be understood by what is taken for granted, and what is part of the lifeworld (Schütz 1962 ). Franzosi shows how one can go about using qualitative and quantitative methods and data to address scientific questions analyzing violence in Italy at the time when fascism was rising (1919–1922). Aspers ( 2006 ) studied the meaning of fashion photographers. He uses an empirical phenomenological approach, and establishes meaning at the level of actors. In a second step this meaning, and the different ideal-typical photographers constructed as a result of participant observation and interviews, are tested using quantitative data from a database; in the first phase to verify the different ideal-types, in the second phase to use these types to establish new knowledge about the types. In both of these cases—and more examples can be found—authors move from qualitative data and try to keep the meaning established when using the quantitative data.

A second main result of our study is that a definition, and we provided one, offers a way for research to clarify, and even evaluate, what is done. Hence, our definition can guide researchers and students, informing them on how to think about concrete research problems they face, and to show what it means to get closer in a process in which new distinctions are made. The definition can also be used to evaluate the results, given that it is a standard of evaluation (cf. Hammersley 2007 ), to see whether new distinctions are made and whether this improves our understanding of what is researched, in addition to the evaluation of how the research was conducted. By making what is qualitative research explicit it becomes easier to communicate findings, and it is thereby much harder to fly under the radar with substandard research since there are standards of evaluation which make it easier to separate “good” from “not so good” qualitative research.

To conclude, our analysis, which ends with a definition of qualitative research can thus both address the “internal” issues of what is qualitative research, and the “external” critiques that make it harder to do qualitative research, to which both pressure from quantitative methods and general changes in society contribute.

Åkerström, Malin. 2013. Curiosity and serendipity in qualitative research. Qualitative Sociology Review 9 (2): 10–18.

Google Scholar  

Alford, Robert R. 1998. The craft of inquiry. Theories, methods, evidence . Oxford: Oxford University Press.

Alvesson, Mats, and Dan Kärreman. 2011. Qualitative research and theory development. Mystery as method . London: SAGE Publications.

Book   Google Scholar  

Aspers, Patrik. 2006. Markets in Fashion, A Phenomenological Approach. London Routledge.

Atkinson, Paul. 2005. Qualitative research. Unity and diversity. Forum: Qualitative Social Research 6 (3): 1–15.

Becker, Howard S. 1963. Outsiders. Studies in the sociology of deviance . New York: The Free Press.

Becker, Howard S. 1966. Whose side are we on? Social Problems 14 (3): 239–247.

Article   Google Scholar  

Becker, Howard S. 1970. Sociological work. Method and substance . New Brunswick: Transaction Books.

Becker, Howard S. 1996. The epistemology of qualitative research. In Ethnography and human development. Context and meaning in social inquiry , ed. Jessor Richard, Colby Anne, and Richard A. Shweder, 53–71. Chicago: University of Chicago Press.

Becker, Howard S. 1998. Tricks of the trade. How to think about your research while you're doing it . Chicago: University of Chicago Press.

Becker, Howard S. 2017. Evidence . Chigaco: University of Chicago Press.

Becker, Howard, Blanche Geer, Everett Hughes, and Anselm Strauss. 1961. Boys in White, student culture in medical school . New Brunswick: Transaction Publishers.

Berezin, Mabel. 2014. How do we know what we mean? Epistemological dilemmas in cultural sociology. Qualitative Sociology 37 (2): 141–151.

Best, Joel. 2004. Defining qualitative research. In Workshop on Scientific Foundations of Qualitative Research , eds . Charles, Ragin, Joanne, Nagel, and Patricia White, 53-54. http://www.nsf.gov/pubs/2004/nsf04219/nsf04219.pdf .

Biernacki, Richard. 2014. Humanist interpretation versus coding text samples. Qualitative Sociology 37 (2): 173–188.

Blumer, Herbert. 1969. Symbolic interactionism: Perspective and method . Berkeley: University of California Press.

Brady, Henry, David Collier, and Jason Seawright. 2004. Refocusing the discussion of methodology. In Rethinking social inquiry. Diverse tools, shared standards , ed. Brady Henry and Collier David, 3–22. Lanham: Rowman and Littlefield.

Brown, Allison P. 2010. Qualitative method and compromise in applied social research. Qualitative Research 10 (2): 229–248.

Charmaz, Kathy. 2006. Constructing grounded theory . London: Sage.

Corte, Ugo, and Katherine Irwin. 2017. “The Form and Flow of Teaching Ethnographic Knowledge: Hands-on Approaches for Learning Epistemology” Teaching Sociology 45(3): 209-219.

Creswell, John W. 2009. Research design. Qualitative, quantitative, and mixed method approaches . 3rd ed. Thousand Oaks: SAGE Publications.

Davidsson, David. 1988. 2001. The myth of the subjective. In Subjective, intersubjective, objective , ed. David Davidsson, 39–52. Oxford: Oxford University Press.

Denzin, Norman K. 1970. The research act: A theoretical introduction to Ssociological methods . Chicago: Aldine Publishing Company Publishers.

Denzin, Norman K., and Yvonna S. Lincoln. 2003. Introduction. The discipline and practice of qualitative research. In Collecting and interpreting qualitative materials , ed. Norman K. Denzin and Yvonna S. Lincoln, 1–45. Thousand Oaks: SAGE Publications.

Denzin, Norman K., and Yvonna S. Lincoln. 2005. Introduction. The discipline and practice of qualitative research. In The Sage handbook of qualitative research , ed. Norman K. Denzin and Yvonna S. Lincoln, 1–32. Thousand Oaks: SAGE Publications.

Emerson, Robert M., ed. 1988. Contemporary field research. A collection of readings . Prospect Heights: Waveland Press.

Emerson, Robert M., Rachel I. Fretz, and Linda L. Shaw. 1995. Writing ethnographic fieldnotes . Chicago: University of Chicago Press.

Esterberg, Kristin G. 2002. Qualitative methods in social research . Boston: McGraw-Hill.

Fine, Gary Alan. 1995. Review of “handbook of qualitative research.” Contemporary Sociology 24 (3): 416–418.

Fine, Gary Alan. 2003. “ Toward a Peopled Ethnography: Developing Theory from Group Life.” Ethnography . 4(1):41-60.

Fine, Gary Alan, and Black Hawk Hancock. 2017. The new ethnographer at work. Qualitative Research 17 (2): 260–268.

Fine, Gary Alan, and Timothy Hallett. 2014. Stranger and stranger: Creating theory through ethnographic distance and authority. Journal of Organizational Ethnography 3 (2): 188–203.

Flick, Uwe. 2002. Qualitative research. State of the art. Social Science Information 41 (1): 5–24.

Flick, Uwe. 2007. Designing qualitative research . London: SAGE Publications.

Frankfort-Nachmias, Chava, and David Nachmias. 1996. Research methods in the social sciences . 5th ed. London: Edward Arnold.

Franzosi, Roberto. 2010. Sociology, narrative, and the quality versus quantity debate (Goethe versus Newton): Can computer-assisted story grammars help us understand the rise of Italian fascism (1919- 1922)? Theory and Society 39 (6): 593–629.

Franzosi, Roberto. 2016. From method and measurement to narrative and number. International journal of social research methodology 19 (1): 137–141.

Gadamer, Hans-Georg. 1990. Wahrheit und Methode, Grundzüge einer philosophischen Hermeneutik . Band 1, Hermeneutik. Tübingen: J.C.B. Mohr.

Gans, Herbert. 1999. Participant Observation in an Age of “Ethnography”. Journal of Contemporary Ethnography 28 (5): 540–548.

Geertz, Clifford. 1973. The interpretation of cultures . New York: Basic Books.

Gilbert, Nigel. 2009. Researching social life . 3rd ed. London: SAGE Publications.

Glaeser, Andreas. 2014. Hermeneutic institutionalism: Towards a new synthesis. Qualitative Sociology 37: 207–241.

Glaser, Barney G., and Anselm L. Strauss. [1967] 2010. The discovery of grounded theory. Strategies for qualitative research. Hawthorne: Aldine.

Goertz, Gary, and James Mahoney. 2012. A tale of two cultures: Qualitative and quantitative research in the social sciences . Princeton: Princeton University Press.

Goffman, Erving. 1989. On fieldwork. Journal of Contemporary Ethnography 18 (2): 123–132.

Goodwin, Jeff, and Ruth Horowitz. 2002. Introduction. The methodological strengths and dilemmas of qualitative sociology. Qualitative Sociology 25 (1): 33–47.

Habermas, Jürgen. [1981] 1987. The theory of communicative action . Oxford: Polity Press.

Hammersley, Martyn. 2007. The issue of quality in qualitative research. International Journal of Research & Method in Education 30 (3): 287–305.

Hammersley, Martyn. 2013. What is qualitative research? Bloomsbury Publishing.

Hammersley, Martyn. 2018. What is ethnography? Can it survive should it? Ethnography and Education 13 (1): 1–17.

Hammersley, Martyn, and Paul Atkinson. 2007. Ethnography. Principles in practice . London: Tavistock Publications.

Heidegger, Martin. [1927] 2001. Sein und Zeit . Tübingen: Max Niemeyer Verlag.

Heidegger, Martin. 1988. 1923. Ontologie. Hermeneutik der Faktizität, Gesamtausgabe II. Abteilung: Vorlesungen 1919-1944, Band 63, Frankfurt am Main: Vittorio Klostermann.

Hempel, Carl G. 1966. Philosophy of the natural sciences . Upper Saddle River: Prentice Hall.

Hood, Jane C. 2006. Teaching against the text. The case of qualitative methods. Teaching Sociology 34 (3): 207–223.

James, William. 1907. 1955. Pragmatism . New York: Meredian Books.

Jovanović, Gordana. 2011. Toward a social history of qualitative research. History of the Human Sciences 24 (2): 1–27.

Kalof, Linda, Amy Dan, and Thomas Dietz. 2008. Essentials of social research . London: Open University Press.

Katz, Jack. 2015. Situational evidence: Strategies for causal reasoning from observational field notes. Sociological Methods & Research 44 (1): 108–144.

King, Gary, Robert O. Keohane, S. Sidney, and S. Verba. 1994. Designing social inquiry. In Scientific inference in qualitative research . Princeton: Princeton University Press.

Chapter   Google Scholar  

Lamont, Michelle. 2004. Evaluating qualitative research: Some empirical findings and an agenda. In Report from workshop on interdisciplinary standards for systematic qualitative research , ed. M. Lamont and P. White, 91–95. Washington, DC: National Science Foundation.

Lamont, Michèle, and Ann Swidler. 2014. Methodological pluralism and the possibilities and limits of interviewing. Qualitative Sociology 37 (2): 153–171.

Lazarsfeld, Paul, and Alan Barton. 1982. Some functions of qualitative analysis in social research. In The varied sociology of Paul Lazarsfeld , ed. Patricia Kendall, 239–285. New York: Columbia University Press.

Lichterman, Paul, and Isaac Reed I (2014), Theory and Contrastive Explanation in Ethnography. Sociological methods and research. Prepublished 27 October 2014; https://doi.org/10.1177/0049124114554458 .

Lofland, John, and Lyn Lofland. 1995. Analyzing social settings. A guide to qualitative observation and analysis . 3rd ed. Belmont: Wadsworth.

Lofland, John, David A. Snow, Leon Anderson, and Lyn H. Lofland. 2006. Analyzing social settings. A guide to qualitative observation and analysis . 4th ed. Belmont: Wadsworth/Thomson Learning.

Long, Adrew F., and Mary Godfrey. 2004. An evaluation tool to assess the quality of qualitative research studies. International Journal of Social Research Methodology 7 (2): 181–196.

Lundberg, George. 1951. Social research: A study in methods of gathering data . New York: Longmans, Green and Co..

Malinowski, Bronislaw. 1922. Argonauts of the Western Pacific: An account of native Enterprise and adventure in the archipelagoes of Melanesian New Guinea . London: Routledge.

Manicas, Peter. 2006. A realist philosophy of science: Explanation and understanding . Cambridge: Cambridge University Press.

Marchel, Carol, and Stephanie Owens. 2007. Qualitative research in psychology. Could William James get a job? History of Psychology 10 (4): 301–324.

McIntyre, Lisa J. 2005. Need to know. Social science research methods . Boston: McGraw-Hill.

Merton, Robert K., and Elinor Barber. 2004. The travels and adventures of serendipity. A Study in Sociological Semantics and the Sociology of Science . Princeton: Princeton University Press.

Mannay, Dawn, and Melanie Morgan. 2015. Doing ethnography or applying a qualitative technique? Reflections from the ‘waiting field‘. Qualitative Research 15 (2): 166–182.

Neuman, Lawrence W. 2007. Basics of social research. Qualitative and quantitative approaches . 2nd ed. Boston: Pearson Education.

Ragin, Charles C. 1994. Constructing social research. The unity and diversity of method . Thousand Oaks: Pine Forge Press.

Ragin, Charles C. 2004. Introduction to session 1: Defining qualitative research. In Workshop on Scientific Foundations of Qualitative Research , 22, ed. Charles C. Ragin, Joane Nagel, Patricia White. http://www.nsf.gov/pubs/2004/nsf04219/nsf04219.pdf

Rawls, Anne. 2018. The Wartime narrative in US sociology, 1940–7: Stigmatizing qualitative sociology in the name of ‘science,’ European Journal of Social Theory (Online first).

Schütz, Alfred. 1962. Collected papers I: The problem of social reality . The Hague: Nijhoff.

Seiffert, Helmut. 1992. Einführung in die Hermeneutik . Tübingen: Franke.

Silverman, David. 2005. Doing qualitative research. A practical handbook . 2nd ed. London: SAGE Publications.

Silverman, David. 2009. A very short, fairly interesting and reasonably cheap book about qualitative research . London: SAGE Publications.

Silverman, David. 2013. What counts as qualitative research? Some cautionary comments. Qualitative Sociology Review 9 (2): 48–55.

Small, Mario L. 2009. “How many cases do I need?” on science and the logic of case selection in field-based research. Ethnography 10 (1): 5–38.

Small, Mario L 2008. Lost in translation: How not to make qualitative research more scientific. In Workshop on interdisciplinary standards for systematic qualitative research, ed in Michelle Lamont, and Patricia White, 165–171. Washington, DC: National Science Foundation.

Snow, David A., and Leon Anderson. 1993. Down on their luck: A study of homeless street people . Berkeley: University of California Press.

Snow, David A., and Calvin Morrill. 1995. New ethnographies: Review symposium: A revolutionary handbook or a handbook for revolution? Journal of Contemporary Ethnography 24 (3): 341–349.

Strauss, Anselm L. 2003. Qualitative analysis for social scientists . 14th ed. Chicago: Cambridge University Press.

Strauss, Anselm L., and Juliette M. Corbin. 1998. Basics of qualitative research. Techniques and procedures for developing grounded theory . 2nd ed. Thousand Oaks: Sage Publications.

Swedberg, Richard. 2017. Theorizing in sociological research: A new perspective, a new departure? Annual Review of Sociology 43: 189–206.

Swedberg, Richard. 1990. The new 'Battle of Methods'. Challenge January–February 3 (1): 33–38.

Timmermans, Stefan, and Iddo Tavory. 2012. Theory construction in qualitative research: From grounded theory to abductive analysis. Sociological Theory 30 (3): 167–186.

Trier-Bieniek, Adrienne. 2012. Framing the telephone interview as a participant-centred tool for qualitative research. A methodological discussion. Qualitative Research 12 (6): 630–644.

Valsiner, Jaan. 2000. Data as representations. Contextualizing qualitative and quantitative research strategies. Social Science Information 39 (1): 99–113.

Weber, Max. 1904. 1949. Objectivity’ in social Science and social policy. Ed. Edward A. Shils and Henry A. Finch, 49–112. New York: The Free Press.

Download references

Acknowledgements

Financial Support for this research is given by the European Research Council, CEV (263699). The authors are grateful to Susann Krieglsteiner for assistance in collecting the data. The paper has benefitted from the many useful comments by the three reviewers and the editor, comments by members of the Uppsala Laboratory of Economic Sociology, as well as Jukka Gronow, Sebastian Kohl, Marcin Serafin, Richard Swedberg, Anders Vassenden and Turid Rødne.

Author information

Authors and affiliations.

Department of Sociology, Uppsala University, Uppsala, Sweden

Patrik Aspers

Seminar for Sociology, Universität St. Gallen, St. Gallen, Switzerland

Department of Media and Social Sciences, University of Stavanger, Stavanger, Norway

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Patrik Aspers .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Aspers, P., Corte, U. What is Qualitative in Qualitative Research. Qual Sociol 42 , 139–160 (2019). https://doi.org/10.1007/s11133-019-9413-7

Download citation

Published : 27 February 2019

Issue Date : 01 June 2019

DOI : https://doi.org/10.1007/s11133-019-9413-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Epistemology
• Philosophy of science
• Phenomenology
• Find a journal
• Publish with us
• Track your research