case study direct observation

Encyclopedia of Autism Spectrum Disorders pp 980–981 Cite as

Direct Observation

Anne Holmes M.S., C.C.C., B.C.B.A. 2
Reference work entry

4273 Accesses

3 Citations

Direct observation, also known as observational study, is a method of collecting evaluative information in which the evaluator watches the subject in his or her usual environment without altering that environment. Direct observation is used when other data collection procedures, such as surveys, questionnaires, etc., are not effective; when the goal is to evaluate an ongoing behavior process, event, or situation; or when there are physical outcomes that can be readily seen.

Direct observation can be overt, when the subject and individuals in the environment know the purpose of the observation, or covert, when the subject and individuals in the environment are unaware of the purpose of the observation.

Structured direct observations are most appropriate when standardized information needs to be gathered, and result in quantitative data. Unstructured direct observation looks at natural occurrence and provides qualitative data, such as that used when administering the Childhood...

This is a preview of subscription content, log in via an institution .

Buying options

Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF

Tax calculation will be finalised at checkout

Purchases are for personal use only

References and Readings

Barnhill, G. P. (2002). Behavioral, social and emotional assessment of students with asd. Assessment for Effective Intervention, 27 (n1–2), 47–55.

Article Google Scholar

Carr, E. G., Ladd, M. V., & Schulte, C. F. (2008). Validation of the contextual assessment inventory for problem behavior. Journal of Positive Behavior Interventions, 10 , 91–104.

Drury, C. G. (1995). Methods for direct observation of performance. In J. R. Wilson & E. N. Corlett (Eds.), Evolution of human work; a practical ergonomics methodology (2nd ed., pp. 45–68). Philadelphia: Taylor and Francis.

Google Scholar

Matson, J. L., & Wilkins, J. (2007). A critical review of assessment targets and methods for social skill excesses and deficits for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 1 , 28–37.

Noterdaeme, M., Mildenberger, K., Sitter, S., & Amorosa, H. (2002). Parent information and direct observation in the diagnosis of pervasive and specific developmental disorders. Autism , 6 (2), 159–168. Abstract retrieved from http://www.online.sagepub.com

Download references

Author information

Authors and affiliations.

Eden Autism Services, 2 Merwick Road, 08540, Princeton, NJ, USA

Anne Holmes M.S., C.C.C., B.C.B.A. ( Chief Clinical Officer )

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anne Holmes M.S., C.C.C., B.C.B.A. .

Editor information

Editors and affiliations.

Irving B. Harris Professor of Child Psychiatry, Pediatrics and Psychology Yale University School of Medicine, Chief, Child Psychiatry Children's Hospital at Yale-New Haven Child Study Center, New Haven, CT, USA

Fred R. Volkmar ( Director, Child Study Center ) ( Director, Child Study Center )

Rights and permissions

Reprints and permissions

Copyright information

About this entry

Cite this entry.

Holmes, A. (2013). Direct Observation. In: Volkmar, F.R. (eds) Encyclopedia of Autism Spectrum Disorders. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-1698-3_1758

Download citation

DOI : https://doi.org/10.1007/978-1-4419-1698-3_1758

Publisher Name : Springer, New York, NY

Print ISBN : 978-1-4419-1697-6

Online ISBN : 978-1-4419-1698-3

eBook Packages : Behavioral Science

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

Direct Observation Methods: a Practical Guide for Health Researchers

Affiliations.

1 VA Center for Healthcare Organization and Implementation Research, Boston and Bedford, MA, USA.
2 Boston University School of Medicine, Boston, MA, USA.
3 Department of Psychiatry, Harvard Medical School, Cambridge, MA, USA.
4 Department of Psychology, University of Maine, Orono, ME, USA.
5 Department of Public Health, University of Massachusetts Lowell, Lowell, MA, USA.
PMID: 36406296
PMCID: PMC9670254
DOI: 10.1016/j.pecinn.2022.100036

Objective: To provide health research teams with a practical, methodologically rigorous guide on how to conduct direct observation.

Methods: Synthesis of authors' observation-based teaching and research experiences in social sciences and health services research.

Results: This article serves as a guide for making key decisions in studies involving direct observation. Study development begins with determining if observation methods are warranted or feasible. Deciding what and how to observe entails reviewing literature and defining what abstract, theoretically informed concepts look like in practice. Data collection tools help systematically record phenomena of interest. Interdisciplinary teams--that include relevant community members-- increase relevance, rigor and reliability, distribute work, and facilitate scheduling. Piloting systematizes data collection across the team and proactively addresses issues.

Conclusion: Observation can elucidate phenomena germane to healthcare research questions by adding unique insights. Careful selection and sampling are critical to rigor. Phenomena like taboo behaviors or rare events are difficult to capture. A thoughtful protocol can preempt Institutional Review Board concerns.

Innovation: This novel guide provides a practical adaptation of traditional approaches to observation to meet contemporary healthcare research teams' needs.

Keywords: Direct Observation; Ethnography; Health Services Research; Methods; Qualitative Methods.

Grants and funding

IK2 HX001783/HX/HSRD VA/United States

Designing and Conducting Case Studies

This guide examines case studies, a form of qualitative descriptive research that is used to look at individuals, a small group of participants, or a group as a whole. Researchers collect data about participants using participant and direct observations, interviews, protocols, tests, examinations of records, and collections of writing samples. Starting with a definition of the case study, the guide moves to a brief history of this research method. Using several well documented case studies, the guide then looks at applications and methods including data collection and analysis. A discussion of ways to handle validity, reliability, and generalizability follows, with special attention to case studies as they are applied to composition studies. Finally, this guide examines the strengths and weaknesses of case studies.

Definition and Overview

Case study refers to the collection and presentation of detailed information about a particular participant or small group, frequently including the accounts of subjects themselves. A form of qualitative descriptive research, the case study looks intensely at an individual or small participant pool, drawing conclusions only about that participant or group and only in that specific context. Researchers do not focus on the discovery of a universal, generalizable truth, nor do they typically look for cause-effect relationships; instead, emphasis is placed on exploration and description.

Case studies typically examine the interplay of all variables in order to provide as complete an understanding of an event or situation as possible. This type of comprehensive understanding is arrived at through a process known as thick description, which involves an in-depth description of the entity being evaluated, the circumstances under which it is used, the characteristics of the people involved in it, and the nature of the community in which it is located. Thick description also involves interpreting the meaning of demographic and descriptive data such as cultural norms and mores, community values, ingrained attitudes, and motives.

Unlike quantitative methods of research, like the survey, which focus on the questions of who, what, where, how much, and how many, and archival analysis, which often situates the participant in some form of historical context, case studies are the preferred strategy when how or why questions are asked. Likewise, they are the preferred method when the researcher has little control over the events, and when there is a contemporary focus within a real life context. In addition, unlike more specifically directed experiments, case studies require a problem that seeks a holistic understanding of the event or situation in question using inductive logic--reasoning from specific to more general terms.

In scholarly circles, case studies are frequently discussed within the context of qualitative research and naturalistic inquiry. Case studies are often referred to interchangeably with ethnography, field study, and participant observation. The underlying philosophical assumptions in the case are similar to these types of qualitative research because each takes place in a natural setting (such as a classroom, neighborhood, or private home), and strives for a more holistic interpretation of the event or situation under study.

Unlike more statistically-based studies which search for quantifiable data, the goal of a case study is to offer new variables and questions for further research. F.H. Giddings, a sociologist in the early part of the century, compares statistical methods to the case study on the basis that the former are concerned with the distribution of a particular trait, or a small number of traits, in a population, whereas the case study is concerned with the whole variety of traits to be found in a particular instance" (Hammersley 95).

Case studies are not a new form of research; naturalistic inquiry was the primary research tool until the development of the scientific method. The fields of sociology and anthropology are credited with the primary shaping of the concept as we know it today. However, case study research has drawn from a number of other areas as well: the clinical methods of doctors; the casework technique being developed by social workers; the methods of historians and anthropologists, plus the qualitative descriptions provided by quantitative researchers like LePlay; and, in the case of Robert Park, the techniques of newspaper reporters and novelists.

Park was an ex-newspaper reporter and editor who became very influential in developing sociological case studies at the University of Chicago in the 1920s. As a newspaper professional he coined the term "scientific" or "depth" reporting: the description of local events in a way that pointed to major social trends. Park viewed the sociologist as "merely a more accurate, responsible, and scientific reporter." Park stressed the variety and value of human experience. He believed that sociology sought to arrive at natural, but fluid, laws and generalizations in regard to human nature and society. These laws weren't static laws of the kind sought by many positivists and natural law theorists, but rather, they were laws of becoming--with a constant possibility of change. Park encouraged students to get out of the library, to quit looking at papers and books, and to view the constant experiment of human experience. He writes, "Go and sit in the lounges of the luxury hotels and on the doorsteps of the flophouses; sit on the Gold Coast settees and on the slum shakedowns; sit in the Orchestra Hall and in the Star and Garter Burlesque. In short, gentlemen [sic], go get the seats of your pants dirty in real research."

But over the years, case studies have drawn their share of criticism. In fact, the method had its detractors from the start. In the 1920s, the debate between pro-qualitative and pro-quantitative became quite heated. Case studies, when compared to statistics, were considered by many to be unscientific. From the 1930's on, the rise of positivism had a growing influence on quantitative methods in sociology. People wanted static, generalizable laws in science. The sociological positivists were looking for stable laws of social phenomena. They criticized case study research because it failed to provide evidence of inter subjective agreement. Also, they condemned it because of the few number of cases studied and that the under-standardized character of their descriptions made generalization impossible. By the 1950s, quantitative methods, in the form of survey research, had become the dominant sociological approach and case study had become a minority practice.

Educational Applications

The 1950's marked the dawning of a new era in case study research, namely that of the utilization of the case study as a teaching method. "Instituted at Harvard Business School in the 1950s as a primary method of teaching, cases have since been used in classrooms and lecture halls alike, either as part of a course of study or as the main focus of the course to which other teaching material is added" (Armisted 1984). The basic purpose of instituting the case method as a teaching strategy was "to transfer much of the responsibility for learning from the teacher on to the student, whose role, as a result, shifts away from passive absorption toward active construction" (Boehrer 1990). Through careful examination and discussion of various cases, "students learn to identify actual problems, to recognize key players and their agendas, and to become aware of those aspects of the situation that contribute to the problem" (Merseth 1991). In addition, students are encouraged to "generate their own analysis of the problems under consideration, to develop their own solutions, and to practically apply their own knowledge of theory to these problems" (Boyce 1993). Along the way, students also develop "the power to analyze and to master a tangled circumstance by identifying and delineating important factors; the ability to utilize ideas, to test them against facts, and to throw them into fresh combinations" (Merseth 1991).

In addition to the practical application and testing of scholarly knowledge, case discussions can also help students prepare for real-world problems, situations and crises by providing an approximation of various professional environments (i.e. classroom, board room, courtroom, or hospital). Thus, through the examination of specific cases, students are given the opportunity to work out their own professional issues through the trials, tribulations, experiences, and research findings of others. An obvious advantage to this mode of instruction is that it allows students the exposure to settings and contexts that they might not otherwise experience. For example, a student interested in studying the effects of poverty on minority secondary student's grade point averages and S.A.T. scores could access and analyze information from schools as geographically diverse as Los Angeles, New York City, Miami, and New Mexico without ever having to leave the classroom.

The case study method also incorporates the idea that students can learn from one another "by engaging with each other and with each other's ideas, by asserting something and then having it questioned, challenged and thrown back at them so that they can reflect on what they hear, and then refine what they say" (Boehrer 1990). In summary, students can direct their own learning by formulating questions and taking responsibility for the study.

Types and Design Concerns

Researchers use multiple methods and approaches to conduct case studies.

Types of Case Studies

Under the more generalized category of case study exist several subdivisions, each of which is custom selected for use depending upon the goals and/or objectives of the investigator. These types of case study include the following:

Illustrative Case Studies These are primarily descriptive studies. They typically utilize one or two instances of an event to show what a situation is like. Illustrative case studies serve primarily to make the unfamiliar familiar and to give readers a common language about the topic in question.

Exploratory (or pilot) Case Studies These are condensed case studies performed before implementing a large scale investigation. Their basic function is to help identify questions and select types of measurement prior to the main investigation. The primary pitfall of this type of study is that initial findings may seem convincing enough to be released prematurely as conclusions.

Cumulative Case Studies These serve to aggregate information from several sites collected at different times. The idea behind these studies is the collection of past studies will allow for greater generalization without additional cost or time being expended on new, possibly repetitive studies.

Critical Instance Case Studies These examine one or more sites for either the purpose of examining a situation of unique interest with little to no interest in generalizability, or to call into question or challenge a highly generalized or universal assertion. This method is useful for answering cause and effect questions.

Identifying a Theoretical Perspective

Much of the case study's design is inherently determined for researchers, depending on the field from which they are working. In composition studies, researchers are typically working from a qualitative, descriptive standpoint. In contrast, physicists will approach their research from a more quantitative perspective. Still, in designing the study, researchers need to make explicit the questions to be explored and the theoretical perspective from which they will approach the case. The three most commonly adopted theories are listed below:

Individual Theories These focus primarily on the individual development, cognitive behavior, personality, learning and disability, and interpersonal interactions of a particular subject.

Organizational Theories These focus on bureaucracies, institutions, organizational structure and functions, or excellence in organizational performance.

Social Theories These focus on urban development, group behavior, cultural institutions, or marketplace functions.

Two examples of case studies are used consistently throughout this chapter. The first, a study produced by Berkenkotter, Huckin, and Ackerman (1988), looks at a first year graduate student's initiation into an academic writing program. The study uses participant-observer and linguistic data collecting techniques to assess the student's knowledge of appropriate discourse conventions. Using the pseudonym Nate to refer to the subject, the study sought to illuminate the particular experience rather than to generalize about the experience of fledgling academic writers collectively.

For example, in Berkenkotter, Huckin, and Ackerman's (1988) study we are told that the researchers are interested in disciplinary communities. In the first paragraph, they ask what constitutes membership in a disciplinary community and how achieving membership might affect a writer's understanding and production of texts. In the third paragraph they state that researchers must negotiate their claims "within the context of his sub specialty's accepted knowledge and methodology." In the next paragraph they ask, "How is literacy acquired? What is the process through which novices gain community membership? And what factors either aid or hinder students learning the requisite linguistic behaviors?" This introductory section ends with a paragraph in which the study's authors claim that during the course of the study, the subject, Nate, successfully makes the transition from "skilled novice" to become an initiated member of the academic discourse community and that his texts exhibit linguistic changes which indicate this transition. In the next section the authors make explicit the sociolinguistic theoretical and methodological assumptions on which the study is based (1988). Thus the reader has a good understanding of the authors' theoretical background and purpose in conducting the study even before it is explicitly stated on the fourth page of the study. "Our purpose was to examine the effects of the educational context on one graduate student's production of texts as he wrote in different courses and for different faculty members over the academic year 1984-85." The goal of the study then, was to explore the idea that writers must be initiated into a writing community, and that this initiation will change the way one writes.

The second example is Janet Emig's (1971) study of the composing process of a group of twelfth graders. In this study, Emig seeks to answer the question of what happens to the self as a result educational stimuli in terms of academic writing. The case study used methods such as protocol analysis, tape-recorded interviews, and discourse analysis.

In the case of Janet Emig's (1971) study of the composing process of eight twelfth graders, four specific hypotheses were made:

Twelfth grade writers engage in two modes of composing: reflexive and extensive.
These differences can be ascertained and characterized through having the writers compose aloud their composition process.
A set of implied stylistic principles governs the writing process.
For twelfth grade writers, extensive writing occurs chiefly as a school-sponsored activity, or reflexive, as a self-sponsored activity.

In this study, the chief distinction is between the two dominant modes of composing among older, secondary school students. The distinctions are:

The reflexive mode, which focuses on the writer's thoughts and feelings.
The extensive mode, which focuses on conveying a message.

Emig also outlines the specific questions which guided the research in the opening pages of her Review of Literature , preceding the report.

Designing a Case Study

After considering the different sub categories of case study and identifying a theoretical perspective, researchers can begin to design their study. Research design is the string of logic that ultimately links the data to be collected and the conclusions to be drawn to the initial questions of the study. Typically, research designs deal with at least four problems:

What questions to study
What data are relevant
What data to collect
How to analyze that data

In other words, a research design is basically a blueprint for getting from the beginning to the end of a study. The beginning is an initial set of questions to be answered, and the end is some set of conclusions about those questions.

Because case studies are conducted on topics as diverse as Anglo-Saxon Literature (Thrane 1986) and AIDS prevention (Van Vugt 1994), it is virtually impossible to outline any strict or universal method or design for conducting the case study. However, Robert K. Yin (1993) does offer five basic components of a research design:

A study's questions.
A study's propositions (if any).
A study's units of analysis.
The logic that links the data to the propositions.
The criteria for interpreting the findings.

In addition to these five basic components, Yin also stresses the importance of clearly articulating one's theoretical perspective, determining the goals of the study, selecting one's subject(s), selecting the appropriate method(s) of collecting data, and providing some considerations to the composition of the final report.

Conducting Case Studies

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of approaches and methods. These approaches, methods, and related issues are discussed in depth in this section.

Method: Single or Multi-modal?

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of methods. Some common methods include interviews , protocol analyses, field studies, and participant-observations. Emig (1971) chose to use several methods of data collection. Her sources included conversations with the students, protocol analysis, discrete observations of actual composition, writing samples from each student, and school records (Lauer and Asher 1988).

Berkenkotter, Huckin, and Ackerman (1988) collected data by observing classrooms, conducting faculty and student interviews, collecting self reports from the subject, and by looking at the subject's written work.

A study that was criticized for using a single method model was done by Flower and Hayes (1984). In this study that explores the ways in which writers use different forms of knowing to create space, the authors used only protocol analysis to gather data. The study came under heavy fire because of their decision to use only one method.

Participant Selection

Case studies can use one participant, or a small group of participants. However, it is important that the participant pool remain relatively small. The participants can represent a diverse cross section of society, but this isn't necessary.

For example, the Berkenkotter, Huckin, and Ackerman (1988) study looked at just one participant, Nate. By contrast, in Janet Emig's (1971) study of the composition process of twelfth graders, eight participants were selected representing a diverse cross section of the community, with volunteers from an all-white upper-middle-class suburban school, an all-black inner-city school, a racially mixed lower-middle-class school, an economically and racially mixed school, and a university school.

Often, a brief "case history" is done on the participants of the study in order to provide researchers with a clearer understanding of their participants, as well as some insight as to how their own personal histories might affect the outcome of the study. For instance, in Emig's study, the investigator had access to the school records of five of the participants, and to standardized test scores for the remaining three. Also made available to the researcher was the information that three of the eight students were selected as NCTE Achievement Award winners. These personal histories can be useful in later stages of the study when data are being analyzed and conclusions drawn.

Data Collection

There are six types of data collected in case studies:

Archival records.
Interviews.
Direct observation.
Participant observation.

In the field of composition research, these six sources might be:

A writer's drafts.
School records of student writers.
Transcripts of interviews with a writer.
Transcripts of conversations between writers (and protocols).
Videotapes and notes from direct field observations.
Hard copies of a writer's work on computer.

Depending on whether researchers have chosen to use a single or multi-modal approach for the case study, they may choose to collect data from one or any combination of these sources.

Protocols, that is, transcriptions of participants talking aloud about what they are doing as they do it, have been particularly common in composition case studies. For example, in Emig's (1971) study, the students were asked, in four different sessions, to give oral autobiographies of their writing experiences and to compose aloud three themes in the presence of a tape recorder and the investigator.

In some studies, only one method of data collection is conducted. For example, the Flower and Hayes (1981) report on the cognitive process theory of writing depends on protocol analysis alone. However, using multiple sources of evidence to increase the reliability and validity of the data can be advantageous.

Case studies are likely to be much more convincing and accurate if they are based on several different sources of information, following a corroborating mode. This conclusion is echoed among many composition researchers. For example, in her study of predrafting processes of high and low-apprehensive writers, Cynthia Selfe (1985) argues that because "methods of indirect observation provide only an incomplete reflection of the complex set of processes involved in composing, a combination of several such methods should be used to gather data in any one study." Thus, in this study, Selfe collected her data from protocols, observations of students role playing their writing processes, audio taped interviews with the students, and videotaped observations of the students in the process of composing.

It can be said then, that cross checking data from multiple sources can help provide a multidimensional profile of composing activities in a particular setting. Sharan Merriam (1985) suggests "checking, verifying, testing, probing, and confirming collected data as you go, arguing that this process will follow in a funnel-like design resulting in less data gathering in later phases of the study along with a congruent increase in analysis checking, verifying, and confirming."

It is important to note that in case studies, as in any qualitative descriptive research, while researchers begin their studies with one or several questions driving the inquiry (which influence the key factors the researcher will be looking for during data collection), a researcher may find new key factors emerging during data collection. These might be unexpected patterns or linguistic features which become evident only during the course of the research. While not bearing directly on the researcher's guiding questions, these variables may become the basis for new questions asked at the end of the report, thus linking to the possibility of further research.

Data Analysis

As the information is collected, researchers strive to make sense of their data. Generally, researchers interpret their data in one of two ways: holistically or through coding. Holistic analysis does not attempt to break the evidence into parts, but rather to draw conclusions based on the text as a whole. Flower and Hayes (1981), for example, make inferences from entire sections of their students' protocols, rather than searching through the transcripts to look for isolatable characteristics.

However, composition researchers commonly interpret their data by coding, that is by systematically searching data to identify and/or categorize specific observable actions or characteristics. These observable actions then become the key variables in the study. Sharan Merriam (1988) suggests seven analytic frameworks for the organization and presentation of data:

The role of participants.
The network analysis of formal and informal exchanges among groups.
Historical.
Thematical.
Ritual and symbolism.
Critical incidents that challenge or reinforce fundamental beliefs, practices, and values.

There are two purposes of these frameworks: to look for patterns among the data and to look for patterns that give meaning to the case study.

As stated above, while most researchers begin their case studies expecting to look for particular observable characteristics, it is not unusual for key variables to emerge during data collection. Typical variables coded in case studies of writers include pauses writers make in the production of a text, the use of specific linguistic units (such as nouns or verbs), and writing processes (planning, drafting, revising, and editing). In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, researchers coded the participant's texts for use of connectives, discourse demonstratives, average sentence length, off-register words, use of the first person pronoun, and the ratio of definite articles to indefinite articles.

Since coding is inherently subjective, more than one coder is usually employed. In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, three rhetoricians were employed to code the participant's texts for off-register phrases. The researchers established the agreement among the coders before concluding that the participant used fewer off-register words as the graduate program progressed.

Composing the Case Study Report

In the many forms it can take, "a case study is generically a story; it presents the concrete narrative detail of actual, or at least realistic events, it has a plot, exposition, characters, and sometimes even dialogue" (Boehrer 1990). Generally, case study reports are extensively descriptive, with "the most problematic issue often referred to as being the determination of the right combination of description and analysis" (1990). Typically, authors address each step of the research process, and attempt to give the reader as much context as possible for the decisions made in the research design and for the conclusions drawn.

This contextualization usually includes a detailed explanation of the researchers' theoretical positions, of how those theories drove the inquiry or led to the guiding research questions, of the participants' backgrounds, of the processes of data collection, of the training and limitations of the coders, along with a strong attempt to make connections between the data and the conclusions evident.

Although the Berkenkotter, Huckin, and Ackerman (1988) study does not, case study reports often include the reactions of the participants to the study or to the researchers' conclusions. Because case studies tend to be exploratory, most end with implications for further study. Here researchers may identify significant variables that emerged during the research and suggest studies related to these, or the authors may suggest further general questions that their case study generated.

For example, Emig's (1971) study concludes with a section dedicated solely to the topic of implications for further research, in which she suggests several means by which this particular study could have been improved, as well as questions and ideas raised by this study which other researchers might like to address, such as: is there a correlation between a certain personality and a certain composing process profile (e.g. is there a positive correlation between ego strength and persistence in revising)?

Also included in Emig's study is a section dedicated to implications for teaching, which outlines the pedagogical ramifications of the study's findings for teachers currently involved in high school writing programs.

Sharan Merriam (1985) also offers several suggestions for alternative presentations of data:

Prepare specialized condensations for appropriate groups.
Replace narrative sections with a series of answers to open-ended questions.
Present "skimmer's" summaries at beginning of each section.
Incorporate headlines that encapsulate information from text.
Prepare analytic summaries with supporting data appendixes.
Present data in colorful and/or unique graphic representations.

Issues of Validity and Reliability

Once key variables have been identified, they can be analyzed. Reliability becomes a key concern at this stage, and many case study researchers go to great lengths to ensure that their interpretations of the data will be both reliable and valid. Because issues of validity and reliability are an important part of any study in the social sciences, it is important to identify some ways of dealing with results.

Multi-modal case study researchers often balance the results of their coding with data from interviews or writer's reflections upon their own work. Consequently, the researchers' conclusions become highly contextualized. For example, in a case study which looked at the time spent in different stages of the writing process, Berkenkotter concluded that her participant, Donald Murray, spent more time planning his essays than in other writing stages. The report of this case study is followed by Murray's reply, wherein he agrees with some of Berkenkotter's conclusions and disagrees with others.

As is the case with other research methodologies, issues of external validity, construct validity, and reliability need to be carefully considered.

Commentary on Case Studies

Researchers often debate the relative merits of particular methods, among them case study. In this section, we comment on two key issues. To read the commentaries, choose any of the items below:

Strengths and Weaknesses of Case Studies

Most case study advocates point out that case studies produce much more detailed information than what is available through a statistical analysis. Advocates will also hold that while statistical methods might be able to deal with situations where behavior is homogeneous and routine, case studies are needed to deal with creativity, innovation, and context. Detractors argue that case studies are difficult to generalize because of inherent subjectivity and because they are based on qualitative subjective data, generalizable only to a particular context.

Flexibility

The case study approach is a comparatively flexible method of scientific research. Because its project designs seem to emphasize exploration rather than prescription or prediction, researchers are comparatively freer to discover and address issues as they arise in their experiments. In addition, the looser format of case studies allows researchers to begin with broad questions and narrow their focus as their experiment progresses rather than attempt to predict every possible outcome before the experiment is conducted.

Emphasis on Context

By seeking to understand as much as possible about a single subject or small group of subjects, case studies specialize in "deep data," or "thick description"--information based on particular contexts that can give research results a more human face. This emphasis can help bridge the gap between abstract research and concrete practice by allowing researchers to compare their firsthand observations with the quantitative results obtained through other methods of research.

Inherent Subjectivity

"The case study has long been stereotyped as the weak sibling among social science methods," and is often criticized as being too subjective and even pseudo-scientific. Likewise, "investigators who do case studies are often regarded as having deviated from their academic disciplines, and their investigations as having insufficient precision (that is, quantification), objectivity and rigor" (Yin 1989). Opponents cite opportunities for subjectivity in the implementation, presentation, and evaluation of case study research. The approach relies on personal interpretation of data and inferences. Results may not be generalizable, are difficult to test for validity, and rarely offer a problem-solving prescription. Simply put, relying on one or a few subjects as a basis for cognitive extrapolations runs the risk of inferring too much from what might be circumstance.

High Investment

Case studies can involve learning more about the subjects being tested than most researchers would care to know--their educational background, emotional background, perceptions of themselves and their surroundings, their likes, dislikes, and so on. Because of its emphasis on "deep data," the case study is out of reach for many large-scale research projects which look at a subject pool in the tens of thousands. A budget request of $10,000 to examine 200 subjects sounds more efficient than a similar request to examine four subjects.

Ethical Considerations

Researchers conducting case studies should consider certain ethical issues. For example, many educational case studies are often financed by people who have, either directly or indirectly, power over both those being studied and those conducting the investigation (1985). This conflict of interests can hinder the credibility of the study.

The personal integrity, sensitivity, and possible prejudices and/or biases of the investigators need to be taken into consideration as well. Personal biases can creep into how the research is conducted, alternative research methods used, and the preparation of surveys and questionnaires.

A common complaint in case study research is that investigators change direction during the course of the study unaware that their original research design was inadequate for the revised investigation. Thus, the researchers leave unknown gaps and biases in the study. To avoid this, researchers should report preliminary findings so that the likelihood of bias will be reduced.

Concerns about Reliability, Validity, and Generalizability

Merriam (1985) offers several suggestions for how case study researchers might actively combat the popular attacks on the validity, reliability, and generalizability of case studies:

Prolong the Processes of Data Gathering on Site: This will help to insure the accuracy of the findings by providing the researcher with more concrete information upon which to formulate interpretations.
Employ the Process of "Triangulation": Use a variety of data sources as opposed to relying solely upon one avenue of observation. One example of such a data check would be what McClintock, Brannon, and Maynard (1985) refer to as a "case cluster method," that is, when a single unit within a larger case is randomly sampled, and that data treated quantitatively." For instance, in Emig's (1971) study, the case cluster method was employed, singling out the productivity of a single student named Lynn. This cluster profile included an advanced case history of the subject, specific examination and analysis of individual compositions and protocols, and extensive interview sessions. The seven remaining students were then compared with the case of Lynn, to ascertain if there are any shared, or unique dimensions to the composing process engaged in by these eight students.
Conduct Member Checks: Initiate and maintain an active corroboration on the interpretation of data between the researcher and those who provided the data. In other words, talk to your subjects.
Collect Referential Materials: Complement the file of materials from the actual site with additional document support. For example, Emig (1971) supports her initial propositions with historical accounts by writers such as T.S. Eliot, James Joyce, and D.H. Lawrence. Emig also cites examples of theoretical research done with regards to the creative process, as well as examples of empirical research dealing with the writing of adolescents. Specific attention is then given to the four stages description of the composing process delineated by Helmoltz, Wallas, and Cowley, as it serves as the focal point in this study.
Engage in Peer Consultation: Prior to composing the final draft of the report, researchers should consult with colleagues in order to establish validity through pooled judgment.

Although little can be done to combat challenges concerning the generalizability of case studies, "most writers suggest that qualitative research should be judged as credible and confirmable as opposed to valid and reliable" (Merriam 1985). Likewise, it has been argued that "rather than transplanting statistical, quantitative notions of generalizability and thus finding qualitative research inadequate, it makes more sense to develop an understanding of generalization that is congruent with the basic characteristics of qualitative inquiry" (1985). After all, criticizing the case study method for being ungeneralizable is comparable to criticizing a washing machine for not being able to tell the correct time. In other words, it is unjust to criticize a method for not being able to do something which it was never originally designed to do in the first place.

Annotated Bibliography

Armisted, C. (1984). How Useful are Case Studies. Training and Development Journal, 38 (2), 75-77.

This article looks at eight types of case studies, offers pros and cons of using case studies in the classroom, and gives suggestions for successfully writing and using case studies.

Bardovi-Harlig, K. (1997). Beyond Methods: Components of Second Language Teacher Education . New York: McGraw-Hill.

A compilation of various research essays which address issues of language teacher education. Essays included are: "Non-native reading research and theory" by Lee, "The case for Psycholinguistics" by VanPatten, and "Assessment and Second Language Teaching" by Gradman and Reed.

Bartlett, L. (1989). A Question of Good Judgment; Interpretation Theory and Qualitative Enquiry Address. 70th Annual Meeting of the American Educational Research Association. San Francisco.

Bartlett selected "quasi-historical" methodology, which focuses on the "truth" found in case records, as one that will provide "good judgments" in educational inquiry. He argues that although the method is not comprehensive, it can try to connect theory with practice.

Baydere, S. et. al. (1993). Multimedia conferencing as a tool for collaborative writing: a case study in Computer Supported Collaborative Writing. New York: Springer-Verlag.

The case study by Baydere et. al. is just one of the many essays in this book found in the series "Computer Supported Cooperative Work." Denley, Witefield and May explore similar issues in their essay, "A case study in task analysis for the design of a collaborative document production system."

Berkenkotter, C., Huckin, T., N., & Ackerman J. (1988). Conventions, Conversations, and the Writer: Case Study of a Student in a Rhetoric Ph.D. Program. Research in the Teaching of English, 22, 9-44.

The authors focused on how the writing of their subject, Nate or Ackerman, changed as he became more acquainted or familiar with his field's discourse community.

Berninger, V., W., and Gans, B., M. (1986). Language Profiles in Nonspeaking Individuals of Normal Intelligence with Severe Cerebral Palsy. Augmentative and Alternative Communication, 2, 45-50.

Argues that generalizations about language abilities in patients with severe cerebral palsy (CP) should be avoided. Standardized tests of different levels of processing oral language, of processing written language, and of producing written language were administered to 3 male participants (aged 9, 16, and 40 yrs).

Bockman, J., R., and Couture, B. (1984). The Case Method in Technical Communication: Theory and Models. Texas: Association of Teachers of Technical Writing.

Examines the study and teaching of technical writing, communication of technical information, and the case method in terms of those applications.

Boehrer, J. (1990). Teaching With Cases: Learning to Question. New Directions for Teaching and Learning, 42 41-57.

This article discusses the origins of the case method, looks at the question of what is a case, gives ideas about learning in case teaching, the purposes it can serve in the classroom, the ground rules for the case discussion, including the role of the question, and new directions for case teaching.

Bowman, W. R. (1993). Evaluating JTPA Programs for Economically Disadvantaged Adults: A Case Study of Utah and General Findings . Washington: National Commission for Employment Policy.

"To encourage state-level evaluations of JTPA, the Commission and the State of Utah co-sponsored this report on the effectiveness of JTPA Title II programs for adults in Utah. The technique used is non-experimental and the comparison group was selected from registrants with Utah's Employment Security. In a step-by-step approach, the report documents how non-experimental techniques can be applied and several specific technical issues can be addressed."

Boyce, A. (1993) The Case Study Approach for Pedagogists. Annual Meeting of the American Alliance for Health, Physical Education, Recreation and Dance. (Address). Washington DC.

This paper addresses how case studies 1) bridge the gap between teaching theory and application, 2) enable students to analyze problems and develop solutions for situations that will be encountered in the real world of teaching, and 3) helps students to evaluate the feasibility of alternatives and to understand the ramifications of a particular course of action.

Carson, J. (1993) The Case Study: Ideal Home of WAC Quantitative and Qualitative Data. Annual Meeting of the Conference on College Composition and Communication. (Address). San Diego.

"Increasingly, one of the most pressing questions for WAC advocates is how to keep [WAC] programs going in the face of numerous difficulties. Case histories offer the best chance for fashioning rhetorical arguments to keep WAC programs going because they offer the opportunity to provide a coherent narrative that contextualizes all documents and data, including what is generally considered scientific data. A case study of the WAC program, . . . at Robert Morris College in Pittsburgh demonstrates the advantages of this research method. Such studies are ideal homes for both naturalistic and positivistic data as well as both quantitative and qualitative information."

---. (1991). A Cognitive Process Theory of Writing. College Composition and Communication. 32. 365-87.

No abstract available.

Cromer, R. (1994) A Case Study of Dissociations Between Language and Cognition. Constraints on Language Acquisition: Studies of Atypical Children . Hillsdale: Lawrence Erlbaum Associates, 141-153.

Crossley, M. (1983) Case Study in Comparative and International Education: An Approach to Bridging the Theory-Practice Gap. Proceedings of the 11th Annual Conference of the Australian Comparative and International Education Society. Hamilton, NZ.

Case study research, as presented here, helps bridge the theory-practice gap in comparative and international research studies of education because it focuses on the practical, day-to-day context rather than on the national arena. The paper asserts that the case study method can be valuable at all levels of research, formation, and verification of theories in education.

Daillak, R., H., and Alkin, M., C. (1982). Qualitative Studies in Context: Reflections on the CSE Studies of Evaluation Use . California: EDRS

The report shows how the Center of the Study of Evaluation (CSE) applied qualitative techniques to a study of evaluation information use in local, Los Angeles schools. It critiques the effectiveness and the limitations of using case study, evaluation, field study, and user interview survey methodologies.

Davey, L. (1991). The Application of Case Study Evaluations. ERIC/TM Digest.

This article examines six types of case studies, the type of evaluation questions that can be answered, the functions served, some design features, and some pitfalls of the method.

Deutch, C. E. (1996). A course in research ethics for graduate students. College Teaching, 44, 2, 56-60.

This article describes a one-credit discussion course in research ethics for graduate students in biology. Case studies are focused on within the four parts of the course: 1) major issues, 2 )practical issues in scholarly work, 3) ownership of research results, and 4) training and personal decisions.

DeVoss, G. (1981). Ethics in Fieldwork Research. RIE 27p. (ERIC)

This article examines four of the ethical problems that can happen when conducting case study research: acquiring permission to do research, knowing when to stop digging, the pitfalls of doing collaborative research, and preserving the integrity of the participants.

Driscoll, A. (1985). Case Study of a Research Intervention: the University of Utah’s Collaborative Approach . San Francisco: Far West Library for Educational Research Development.

Paper presented at the annual meeting of the American Association of Colleges of Teacher Education, Denver, CO, March 1985. Offers information of in-service training, specifically case studies application.

Ellram, L. M. (1996). The Use of the Case Study Method in Logistics Research. Journal of Business Logistics, 17, 2, 93.

This article discusses the increased use of case study in business research, and the lack of understanding of when and how to use case study methodology in business.

Emig, J. (1971) The Composing Processes of Twelfth Graders . Urbana: NTCE.

This case study uses observation, tape recordings, writing samples, and school records to show that writing in reflexive and extensive situations caused different lengths of discourse and different clusterings of the components of the writing process.

Feagin, J. R. (1991). A Case For the Case Study . Chapel Hill: The University of North Carolina Press.

This book discusses the nature, characteristics, and basic methodological issues of the case study as a research method.

Feldman, H., Holland, A., & Keefe, K. (1989) Language Abilities after Left Hemisphere Brain Injury: A Case Study of Twins. Topics in Early Childhood Special Education, 9, 32-47.

"Describes the language abilities of 2 twin pairs in which 1 twin (the experimental) suffered brain injury to the left cerebral hemisphere around the time of birth and1 twin (the control) did not. One pair of twins was initially assessed at age 23 mo. and the other at about 30 mo.; they were subsequently evaluated in their homes 3 times at about 6-mo intervals."

Fidel, R. (1984). The Case Study Method: A Case Study. Library and Information Science Research, 6.

The article describes the use of case study methodology to systematically develop a model of online searching behavior in which study design is flexible, subject manner determines data gathering and analyses, and procedures adapt to the study's progressive change.

Flower, L., & Hayes, J. R. (1984). Images, Plans and Prose: The Representation of Meaning in Writing. Written Communication, 1, 120-160.

Explores the ways in which writers actually use different forms of knowing to create prose.

Frey, L. R. (1992). Interpreting Communication Research: A Case Study Approach Englewood Cliffs, N.J.: Prentice Hall.

The book discusses research methodologies in the Communication field. It focuses on how case studies bridge the gap between communication research, theory, and practice.

Gilbert, V. K. (1981). The Case Study as a Research Methodology: Difficulties and Advantages of Integrating the Positivistic, Phenomenological and Grounded Theory Approaches . The Annual Meeting of the Canadian Association for the Study of Educational Administration. (Address) Halifax, NS, Can.

This study on an innovative secondary school in England shows how a "low-profile" participant-observer case study was crucial to the initial observation, the testing of hypotheses, the interpretive approach, and the grounded theory.

Gilgun, J. F. (1994). A Case for Case Studies in Social Work Research. Social Work, 39, 4, 371-381.

This article defines case study research, presents guidelines for evaluation of case studies, and shows the relevance of case studies to social work research. It also looks at issues such as evaluation and interpretations of case studies.

Glennan, S. L., Sharp-Bittner, M. A. & Tullos, D. C. (1991). Augmentative and Alternative Communication Training with a Nonspeaking Adult: Lessons from MH. Augmentative and Alternative Communication, 7, 240-7.

"A response-guided case study documented changes in a nonspeaking 36-yr-old man's ability to communicate using 3 trained augmentative communication modes. . . . Data were collected in videotaped interaction sessions between the nonspeaking adult and a series of adult speaking."

Graves, D. (1981). An Examination of the Writing Processes of Seven Year Old Children. Research in the Teaching of English, 15, 113-134.

Hamel, J. (1993). Case Study Methods . Newbury Park: Sage. .

"In a most economical fashion, Hamel provides a practical guide for producing theoretically sharp and empirically sound sociological case studies. A central idea put forth by Hamel is that case studies must "locate the global in the local" thus making the careful selection of the research site the most critical decision in the analytic process."

Karthigesu, R. (1986, July). Television as a Tool for Nation-Building in the Third World: A Post-Colonial Pattern, Using Malaysia as a Case-Study. International Television Studies Conference. (Address). London, 10-12.

"The extent to which Television Malaysia, as a national mass media organization, has been able to play a role in nation building in the post-colonial period is . . . studied in two parts: how the choice of a model of nation building determines the character of the organization; and how the character of the organization influences the output of the organization."

Kenny, R. (1984). Making the Case for the Case Study. Journal of Curriculum Studies, 16, (1), 37-51.

The article looks at how and why the case study is justified as a viable and valuable approach to educational research and program evaluation.

Knirk, F. (1991). Case Materials: Research and Practice. Performance Improvement Quarterly, 4 (1 ), 73-81.

The article addresses the effectiveness of case studies, subject areas where case studies are commonly used, recent examples of their use, and case study design considerations.

Klos, D. (1976). Students as Case Writers. Teaching of Psychology, 3.2, 63-66.

This article reviews a course in which students gather data for an original case study of another person. The task requires the students to design the study, collect the data, write the narrative, and interpret the findings.

Leftwich, A. (1981). The Politics of Case Study: Problems of Innovation in University Education. Higher Education Review, 13.2, 38-64.

The article discusses the use of case studies as a teaching method. Emphasis is on the instructional materials, interdisciplinarity, and the complex relationships within the university that help or hinder the method.

Mabrito, M. (1991, Oct.). Electronic Mail as a Vehicle for Peer Response: Conversations of High and Low Apprehensive Writers. Written Communication, 509-32.

McCarthy, S., J. (1955). The Influence of Classroom Discourse on Student Texts: The Case of Ella . East Lansing: Institute for Research on Teaching.

A look at how students of color become marginalized within traditional classroom discourse. The essay follows the struggles of one black student: Ella.

Matsuhashi, A., ed. (1987). Writing in Real Time: Modeling Production Processes Norwood, NJ: Ablex Publishing Corporation.

Investigates how writers plan to produce discourse for different purposes to report, to generalize, and to persuade, as well as how writers plan for sentence level units of language. To learn about planning, an observational measure of pause time was used" (ERIC).

Merriam, S. B. (1985). The Case Study in Educational Research: A Review of Selected Literature. Journal of Educational Thought, 19.3, 204-17.

The article examines the characteristics of, philosophical assumptions underlying the case study, the mechanics of conducting a case study, and the concerns about the reliability, validity, and generalizability of the method.

---. (1988). Case Study Research in Education: A Qualitative Approach San Francisco: Jossey Bass.

Merry, S. E., & Milner, N. eds. (1993). The Possibility of Popular Justice: A Case Study of Community Mediation in the United States . Ann Arbor: U of Michigan.

". . . this volume presents a case study of one experiment in popular justice, the San Francisco Community Boards. This program has made an explicit claim to create an alternative justice, or new justice, in the midst of a society ordered by state law. The contributors to this volume explore the history and experience of the program and compare it to other versions of popular justice in the United States, Europe, and the Third World."

Merseth, K. K. (1991). The Case for Cases in Teacher Education. RIE. 42p. (ERIC).

This monograph argues that the case method of instruction offers unique potential for revitalizing the field of teacher education.

Michaels, S. (1987). Text and Context: A New Approach to the Study of Classroom Writing. Discourse Processes, 10, 321-346.

"This paper argues for and illustrates an approach to the study of writing that integrates ethnographic analysis of classroom interaction with linguistic analysis of written texts and teacher/student conversational exchanges. The approach is illustrated through a case study of writing in a single sixth grade classroom during a single writing assignment."

Milburn, G. (1995). Deciphering a Code or Unraveling a Riddle: A Case Study in the Application of a Humanistic Metaphor to the Reporting of Social Studies Teaching. Theory and Research in Education, 13.

This citation serves as an example of how case studies document learning procedures in a senior-level economics course.

Milley, J. E. (1979). An Investigation of Case Study as an Approach to Program Evaluation. 19th Annual Forum of the Association for Institutional Research. (Address). San Diego.

The case study method merged a narrative report focusing on the evaluator as participant-observer with document review, interview, content analysis, attitude questionnaire survey, and sociogram analysis. Milley argues that case study program evaluation has great potential for widespread use.

Minnis, J. R. (1985, Sept.). Ethnography, Case Study, Grounded Theory, and Distance Education Research. Distance Education, 6.2.

This article describes and defines the strengths and weaknesses of ethnography, case study, and grounded theory.

Nunan, D. (1992). Collaborative language learning and teaching . New York: Cambridge University Press.

Included in this series of essays is Peter Sturman’s "Team Teaching: a case study from Japan" and David Nunan’s own "Toward a collaborative approach to curriculum development: a case study."

Nystrand, M., ed. (1982). What Writers Know: The Language, Process, and Structure of Written Discourse . New York: Academic Press.

Owenby, P. H. (1992). Making Case Studies Come Alive. Training, 29, (1), 43-46. (ERIC)

This article provides tips for writing more effective case studies.

---. (1981). Pausing and Planning: The Tempo of Writer Discourse Production. Research in the Teaching of English, 15 (2),113-34.

Perl, S. (1979). The Composing Processes of Unskilled College Writers. Research in the Teaching of English, 13, 317-336.

"Summarizes a study of five unskilled college writers, focusing especially on one of the five, and discusses the findings in light of current pedagogical practice and research design."

Pilcher J. and A. Coffey. eds. (1996). Gender and Qualitative Research . Brookfield: Aldershot, Hants, England.

This book provides a series of essays which look at gender identity research, qualitative research and applications of case study to questions of gendered pedagogy.

Pirie, B. S. (1993). The Case of Morty: A Four Year Study. Gifted Education International, 9 (2), 105-109.

This case study describes a boy from kindergarten through third grade with above average intelligence but difficulty in learning to read, write, and spell.

Popkewitz, T. (1993). Changing Patterns of Power: Social Regulation and Teacher Education Reform. Albany: SUNY Press.

Popkewitz edits this series of essays that address case studies on educational change and the training of teachers. The essays vary in terms of discipline and scope. Also, several authors include case studies of educational practices in countries other than the United States.

---. (1984). The Predrafting Processes of Four High- and Four Low Apprehensive Writers. Research in the Teaching of English, 18, (1), 45-64.

Rasmussen, P. (1985, March) A Case Study on the Evaluation of Research at the Technical University of Denmark. International Journal of Institutional Management in Higher Education, 9 (1).

This is an example of a case study methodology used to evaluate the chemistry and chemical engineering departments at the University of Denmark.

Roth, K. J. (1986). Curriculum Materials, Teacher Talk, and Student Learning: Case Studies in Fifth-Grade Science Teaching . East Lansing: Institute for Research on Teaching.

Roth offers case studies on elementary teachers, elementary school teaching, science studies and teaching, and verbal learning.

Selfe, C. L. (1985). An Apprehensive Writer Composes. When a Writer Can't Write: Studies in Writer's Block and Other Composing-Process Problems . (pp. 83-95). Ed. Mike Rose. NMY: Guilford.

Smith-Lewis, M., R. and Ford, A. (1987). A User's Perspective on Augmentative Communication. Augmentative and Alternative Communication, 3, 12-7.

"During a series of in-depth interviews, a 25-yr-old woman with cerebral palsy who utilized augmentative communication reflected on the effectiveness of the devices designed for her during her school career."

St. Pierre, R., G. (1980, April). Follow Through: A Case Study in Metaevaluation Research . 64th Annual Meeting of the American Educational Research Association. (Address).

The three approaches to metaevaluation are evaluation of primary evaluations, integrative meta-analysis with combined primary evaluation results, and re-analysis of the raw data from a primary evaluation.

Stahler, T., M. (1996, Feb.) Early Field Experiences: A Model That Worked. ERIC.

"This case study of a field and theory class examines a model designed to provide meaningful field experiences for preservice teachers while remaining consistent with the instructor's beliefs about the role of teacher education in preparing teachers for the classroom."

Stake, R. E. (1995). The Art of Case Study Research. Thousand Oaks: Sage Publications.

This book examines case study research in education and case study methodology.

Stiegelbauer, S. (1984) Community, Context, and Co-curriculum: Situational Factors Influencing School Improvements in a Study of High Schools. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Discussion of several case studies: one looking at high school environments, another examining educational innovations.

Stolovitch, H. (1990). Case Study Method. Performance And Instruction, 29, (9), 35-37.

This article describes the case study method as a form of simulation and presents guidelines for their use in professional training situations.

Thaller, E. (1994). Bibliography for the Case Method: Using Case Studies in Teacher Education. RIE. 37 p.

This bibliography presents approximately 450 citations on the use of case studies in teacher education from 1921-1993.

Thrane, T. (1986). On Delimiting the Senses of Near-Synonyms in Historical Semantics: A Case Study of Adjectives of 'Moral Sufficiency' in the Old English Andreas. Linguistics Across Historical and Geographical Boundaries: In Honor of Jacek Fisiak on the Occasion of his Fiftieth Birthday . Berlin: Mouton de Gruyter.

United Nations. (1975). Food and Agriculture Organization. Report on the FAO/UNFPA Seminar on Methodology, Research and Country: Case Studies on Population, Employment and Productivity . Rome: United Nations.

This example case study shows how the methodology can be used in a demographic and psychographic evaluation. At the same time, it discusses the formation and instigation of the case study methodology itself.

Van Vugt, J. P., ed. (1994). Aids Prevention and Services: Community Based Research . Westport: Bergin and Garvey.

"This volume has been five years in the making. In the process, some of the policy applications called for have met with limited success, such as free needle exchange programs in a limited number of American cities, providing condoms to prison inmates, and advertisements that depict same-sex couples. Rather than dating our chapters that deal with such subjects, such policy applications are verifications of the type of research demonstrated here. Furthermore, they indicate the critical need to continue community based research in the various communities threatened by acquired immuno-deficiency syndrome (AIDS) . . . "

Welch, W., ed. (1981, May). Case Study Methodology in Educational Evaluation. Proceedings of the Minnesota Evaluation Conference. Minnesota. (Address).

The four papers in these proceedings provide a comprehensive picture of the rationale, methodology, strengths, and limitations of case studies.

Williams, G. (1987). The Case Method: An Approach to Teaching and Learning in Educational Administration. RIE, 31p.

This paper examines the viability of the case method as a teaching and learning strategy in instructional systems geared toward the training of personnel of the administration of various aspects of educational systems.

Yin, R. K. (1993). Advancing Rigorous Methodologies: A Review of 'Towards Rigor in Reviews of Multivocal Literatures.' Review of Educational Research, 61, (3).

"R. T. Ogawa and B. Malen's article does not meet its own recommended standards for rigorous testing and presentation of its own conclusions. Use of the exploratory case study to analyze multivocal literatures is not supported, and the claim of grounded theory to analyze multivocal literatures may be stronger."

---. (1989). Case Study Research: Design and Methods. London: Sage Publications Inc.

This book discusses in great detail, the entire design process of the case study, including entire chapters on collecting evidence, analyzing evidence, composing the case study report, and designing single and multiple case studies.

Observation Method in Psychology: Naturalistic, Participant and Controlled

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

The observation method in psychology involves directly and systematically witnessing and recording measurable behaviors, actions, and responses in natural or contrived settings without attempting to intervene or manipulate what is being observed.

Used to describe phenomena, generate hypotheses, or validate self-reports, psychological observation can be either controlled or naturalistic with varying degrees of structure imposed by the researcher.

There are different types of observational methods, and distinctions need to be made between:

1. Controlled Observations 2. Naturalistic Observations 3. Participant Observations

In addition to the above categories, observations can also be either overt/disclosed (the participants know they are being studied) or covert/undisclosed (the researcher keeps their real identity a secret from the research subjects, acting as a genuine member of the group).

In general, conducting observational research is relatively inexpensive, but it remains highly time-consuming and resource-intensive in data processing and analysis.

The considerable investments needed in terms of coder time commitments for training, maintaining reliability, preventing drift, and coding complex dynamic interactions place practical barriers on observers with limited resources.

Controlled Observation

Controlled observation is a research method for studying behavior in a carefully controlled and structured environment.

The researcher sets specific conditions, variables, and procedures to systematically observe and measure behavior, allowing for greater control and comparison of different conditions or groups.

The researcher decides where the observation will occur, at what time, with which participants, and in what circumstances, and uses a standardized procedure. Participants are randomly allocated to each independent variable group.

Rather than writing a detailed description of all behavior observed, it is often easier to code behavior according to a previously agreed scale using a behavior schedule (i.e., conducting a structured observation).

The researcher systematically classifies the behavior they observe into distinct categories. Coding might involve numbers or letters to describe a characteristic or the use of a scale to measure behavior intensity.

The categories on the schedule are coded so that the data collected can be easily counted and turned into statistics.

For example, Mary Ainsworth used a behavior schedule to study how infants responded to brief periods of separation from their mothers. During the Strange Situation procedure, the infant’s interaction behaviors directed toward the mother were measured, e.g.,

Proximity and contact-seeking
Contact maintaining
Avoidance of proximity and contact
Resistance to contact and comforting

The observer noted down the behavior displayed during 15-second intervals and scored the behavior for intensity on a scale of 1 to 7.

Sometimes participants’ behavior is observed through a two-way mirror, or they are secretly filmed. Albert Bandura used this method to study aggression in children (the Bobo doll studies ).

A lot of research has been carried out in sleep laboratories as well. Here, electrodes are attached to the scalp of participants. What is observed are the changes in electrical activity in the brain during sleep ( the machine is called an EEG ).

Controlled observations are usually overt as the researcher explains the research aim to the group so the participants know they are being observed.

Controlled observations are also usually non-participant as the researcher avoids direct contact with the group and keeps a distance (e.g., observing behind a two-way mirror).

Controlled observations can be easily replicated by other researchers by using the same observation schedule. This means it is easy to test for reliability .
The data obtained from structured observations is easier and quicker to analyze as it is quantitative (i.e., numerical) – making this a less time-consuming method compared to naturalistic observations.
Controlled observations are fairly quick to conduct which means that many observations can take place within a short amount of time. This means a large sample can be obtained, resulting in the findings being representative and having the ability to be generalized to a large population.

Limitations

Controlled observations can lack validity due to the Hawthorne effect /demand characteristics. When participants know they are being watched, they may act differently.

Naturalistic Observation

Naturalistic observation is a research method in which the researcher studies behavior in its natural setting without intervention or manipulation.

It involves observing and recording behavior as it naturally occurs, providing insights into real-life behaviors and interactions in their natural context.

Naturalistic observation is a research method commonly used by psychologists and other social scientists.

This technique involves observing and studying the spontaneous behavior of participants in natural surroundings. The researcher simply records what they see in whatever way they can.

In unstructured observations, the researcher records all relevant behavior with a coding system. There may be too much to record, and the behaviors recorded may not necessarily be the most important, so the approach is usually used as a pilot study to see what type of behaviors would be recorded.

Compared with controlled observations, it is like the difference between studying wild animals in a zoo and studying them in their natural habitat.

With regard to human subjects, Margaret Mead used this method to research the way of life of different tribes living on islands in the South Pacific. Kathy Sylva used it to study children at play by observing their behavior in a playgroup in Oxfordshire.

Collecting Naturalistic Behavioral Data

Technological advances are enabling new, unobtrusive ways of collecting naturalistic behavioral data.

The Electronically Activated Recorder (EAR) is a digital recording device participants can wear to periodically sample ambient sounds, allowing representative sampling of daily experiences (Mehl et al., 2012).

Studies program EARs to record 30-50 second sound snippets multiple times per hour. Although coding the recordings requires extensive resources, EARs can capture spontaneous behaviors like arguments or laughter.

EARs minimize participant reactivity since sampling occurs outside of awareness. This reduces the Hawthorne effect, where people change behavior when observed.

The SenseCam is another wearable device that passively captures images documenting daily activities. Though primarily used in memory research currently (Smith et al., 2014), systematic sampling of environments and behaviors via the SenseCam could enable innovative psychological studies in the future.

By being able to observe the flow of behavior in its own setting, studies have greater ecological validity.
Like case studies , naturalistic observation is often used to generate new ideas. Because it gives the researcher the opportunity to study the total situation, it often suggests avenues of inquiry not thought of before.
The ability to capture actual behaviors as they unfold in real-time, analyze sequential patterns of interactions, measure base rates of behaviors, and examine socially undesirable or complex behaviors that people may not self-report accurately.
These observations are often conducted on a micro (small) scale and may lack a representative sample (biased in relation to age, gender, social class, or ethnicity). This may result in the findings lacking the ability to generalize to wider society.
Natural observations are less reliable as other variables cannot be controlled. This makes it difficult for another researcher to repeat the study in exactly the same way.
Highly time-consuming and resource-intensive during the data coding phase (e.g., training coders, maintaining inter-rater reliability, preventing judgment drift).
With observations, we do not have manipulations of variables (or control over extraneous variables), meaning cause-and-effect relationships cannot be established.

Participant Observation

Participant observation is a variant of the above (natural observations) but here, the researcher joins in and becomes part of the group they are studying to get a deeper insight into their lives.

If it were research on animals , we would now not only be studying them in their natural habitat but be living alongside them as well!

Leon Festinger used this approach in a famous study into a religious cult that believed that the end of the world was about to occur. He joined the cult and studied how they reacted when the prophecy did not come true.

Participant observations can be either covert or overt. Covert is where the study is carried out “undercover.” The researcher’s real identity and purpose are kept concealed from the group being studied.

The researcher takes a false identity and role, usually posing as a genuine member of the group.

On the other hand, overt is where the researcher reveals his or her true identity and purpose to the group and asks permission to observe.

It can be difficult to get time/privacy for recording. For example, researchers can’t take notes openly with covert observations as this would blow their cover. This means they must wait until they are alone and rely on their memory. This is a problem as they may forget details and are unlikely to remember direct quotations.
If the researcher becomes too involved, they may lose objectivity and become biased. There is always the danger that we will “see” what we expect (or want) to see. This problem is because they could selectively report information instead of noting everything they observe. Thus reducing the validity of their data.

Recording of Data

With controlled/structured observation studies, an important decision the researcher has to make is how to classify and record the data. Usually, this will involve a method of sampling.

In most coding systems, codes or ratings are made either per behavioral event or per specified time interval (Bakeman & Quera, 2011).

The three main sampling methods are:

Event-based coding involves identifying and segmenting interactions into meaningful events rather than timed units.

For example, parent-child interactions may be segmented into control or teaching events to code. Interval recording involves dividing interactions into fixed time intervals (e.g., 6-15 seconds) and coding behaviors within each interval (Bakeman & Quera, 2011).

Event recording allows counting event frequency and sequencing while also potentially capturing event duration through timed-event recording. This provides information on time spent on behaviors.

Coding Systems

The coding system should focus on behaviors, patterns, individual characteristics, or relationship qualities that are relevant to the theory guiding the study (Wampler & Harper, 2014).

Codes vary in how much inference is required, from concrete observable behaviors like frequency of eye contact to more abstract concepts like degree of rapport between a therapist and client (Hill & Lambert, 2004). More inference may reduce reliability.

Macroanalytic coding systems

Macroanalytic coding systems involve rating or summarizing behaviors using larger coding units and broader categories that reflect patterns across longer periods of interaction rather than coding small or discrete behavioral acts.

For example, a macroanalytic coding system may rate the overall degree of therapist warmth or level of client engagement globally for an entire therapy session, requiring the coders to summarize and infer these constructs across the interaction rather than coding smaller behavioral units.

These systems require observers to make more inferences (more time-consuming) but can better capture contextual factors, stability over time, and the interdependent nature of behaviors (Carlson & Grotevant, 1987).

Microanalytic coding systems

Microanalytic coding systems involve rating behaviors using smaller, more discrete coding units and categories.

For example, a microanalytic system may code each instance of eye contact or head nodding during a therapy session. These systems code specific, molecular behaviors as they occur moment-to-moment rather than summarizing actions over longer periods.

Microanalytic systems require less inference from coders and allow for analysis of behavioral contingencies and sequential interactions between therapist and client. However, they are more time-consuming and expensive to implement than macroanalytic approaches.

Mesoanalytic coding systems

Mesoanalytic coding systems attempt to balance macro- and micro-analytic approaches.

In contrast to macroanalytic systems that summarize behaviors in larger chunks, mesoanalytic systems use medium-sized coding units that target more specific behaviors or interaction sequences (Bakeman & Quera, 2017).

For example, a mesoanalytic system may code each instance of a particular type of therapist statement or client emotional expression. However, mesoanalytic systems still use larger units than microanalytic approaches coding every speech onset/offset.

The goal of balancing specificity and feasibility makes mesoanalytic systems well-suited for many research questions (Morris et al., 2014). Mesoanalytic codes can preserve some sequential information while remaining efficient enough for studies with adequate but limited resources.

For instance, a mesoanalytic couple interaction coding system could target key behavior patterns like validation sequences without coding turn-by-turn speech.

In this way, mesoanalytic coding allows reasonable reliability and specificity without requiring extensive training or observation. The mid-level focus offers a pragmatic compromise between depth and breadth in analyzing interactions.

Preventing Coder Drift

Coder drift results in a measurement error caused by gradual shifts in how observations get rated according to operational definitions, especially when behavioral codes are not clearly specified.

This type of error creeps in when coders fail to regularly review what precise observations constitute or do not constitute the behaviors being measured.

Preventing drift refers to taking active steps to maintain consistency and minimize changes or deviations in how coders rate or evaluate behaviors over time. Specifically, some key ways to prevent coder drift include:

Operationalize codes : It is essential that code definitions unambiguously distinguish what interactions represent instances of each coded behavior.
Ongoing training : Returning to those operational definitions through ongoing training serves to recalibrate coder interpretations and reinforce accurate recognition. Having regular “check-in” sessions where coders practice coding the same interactions allows monitoring that they continue applying codes reliably without gradual shifts in interpretation.
Using reference videos : Coders periodically coding the same “gold standard” reference videos anchors their judgments and calibrate against original training. Without periodic anchoring to original specifications, coder decisions tend to drift from initial measurement reliability.
Assessing inter-rater reliability : Statistical tracking that coders maintain high levels of agreement over the course of a study, not just at the start, flags any declines indicating drift. Sustaining inter-rater agreement requires mitigating this common tendency for observer judgment change during intensive, long-term coding tasks.
Recalibrating through discussion : Having meetings for coders to discuss disagreements openly explores reasons judgment shifts may be occurring over time. Consensus on the application of codes is restored.
Adjusting unclear codes : If reliability issues persist, revisiting and refining ambiguous code definitions or anchors can eliminate inconsistencies arising from coder confusion.

Essentially, the goal of preventing coder drift is maintaining standardization and minimizing unintentional biases that may slowly alter how observational data gets rated over periods of extensive coding.

Through the upkeep of skills, continuing calibration to benchmarks, and monitoring consistency, researchers can notice and correct for any creeping changes in coder decision-making over time.

Reducing Observer Bias

Observational research is prone to observer biases resulting from coders’ subjective perspectives shaping the interpretation of complex interactions (Burghardt et al., 2012). When coding, personal expectations may unconsciously influence judgments. However, rigorous methods exist to reduce such bias.

Coding Manual

A detailed coding manual minimizes subjectivity by clearly defining what behaviors and interaction dynamics observers should code (Bakeman & Quera, 2011).

High-quality manuals have strong theoretical and empirical grounding, laying out explicit coding procedures and providing rich behavioral examples to anchor code definitions (Lindahl, 2001).

Clear delineation of the frequency, intensity, duration, and type of behaviors constituting each code facilitates reliable judgments and reduces ambiguity for coders. Application risks inconsistency across raters without clarity on how codes translate to observable interaction.

Coder Training

Competent coders require both interpersonal perceptiveness and scientific rigor (Wampler & Harper, 2014). Training thoroughly reviews the theoretical basis for coded constructs and teaches the coding system itself.

Multiple “gold standard” criterion videos demonstrate code ranges that trainees independently apply. Coders then meet weekly to establish reliability of 80% or higher agreement both among themselves and with master criterion coding (Hill & Lambert, 2004).

Ongoing training manages coder drift over time. Revisions to unclear codes may also improve reliability. Both careful selection and investment in rigorous training increase quality control.

Blind Methods

To prevent bias, coders should remain unaware of specific study predictions or participant details (Burghardt et al., 2012). Separate data gathering versus coding teams helps maintain blinding.

Coders should be unaware of study details or participant identities that could bias coding (Burghardt et al., 2012).

Separate teams collecting data versus coding data can reduce bias.

In addition, scheduling procedures can prevent coders from rating data collected directly from participants with whom they have had personal contact. Maintaining coder independence and blinding enhances objectivity.

Bakeman, R., & Quera, V. (2017). Sequential analysis and observational methods for the behavioral sciences. Cambridge University Press.

Burghardt, G. M., Bartmess-LeVasseur, J. N., Browning, S. A., Morrison, K. E., Stec, C. L., Zachau, C. E., & Freeberg, T. M. (2012). Minimizing observer bias in behavioral studies: A review and recommendations. Ethology, 118 (6), 511-517.

Hill, C. E., & Lambert, M. J. (2004). Methodological issues in studying psychotherapy processes and outcomes. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 84–135). Wiley.

Lindahl, K. M. (2001). Methodological issues in family observational research. In P. K. Kerig & K. M. Lindahl (Eds.), Family observational coding systems: Resources for systemic research (pp. 23–32). Lawrence Erlbaum Associates.

Mehl, M. R., Robbins, M. L., & Deters, F. G. (2012). Naturalistic observation of health-relevant social processes: The electronically activated recorder methodology in psychosomatics. Psychosomatic Medicine, 74 (4), 410–417.

Morris, A. S., Robinson, L. R., & Eisenberg, N. (2014). Applying a multimethod perspective to the study of developmental psychology. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd ed., pp. 103–123). Cambridge University Press.

Smith, J. A., Maxwell, S. D., & Johnson, G. (2014). The microstructure of everyday life: Analyzing the complex choreography of daily routines through the automatic capture and processing of wearable sensor data. In B. K. Wiederhold & G. Riva (Eds.), Annual Review of Cybertherapy and Telemedicine 2014: Positive Change with Technology (Vol. 199, pp. 62-64). IOS Press.

Traniello, J. F., & Bakker, T. C. (2015). The integrative study of behavioral interactions across the sciences. In T. K. Shackelford & R. D. Hansen (Eds.), The evolution of sexuality (pp. 119-147). Springer.

Wampler, K. S., & Harper, A. (2014). Observational methods in couple and family assessment. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd ed., pp. 490–502). Cambridge University Press.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

QUALITATIVE METHODS

9 Direct Observation and Ethnography

Nicolas Fischer

Direct observation or ethnography is a qualitative method that consists in directly observing the social situation under study – for example, the implementation of a public policy – implying a physical presence of the researcher in the situation at hand. It is a demanding method in terms of the commitment it requires (long-term physical presence in the field, systematic note-taking). It is particularly useful to account for the reality of practices and interactions, at a distance from official discourse.

Keywords: Qualitative methods, ethnography, direct observation, policy implementation, semi-structured interview, interactions, case study

I. What does this method consist of?

Direct observation derives from the practice of ethnographic observation, which is an old tradition in the social sciences, particularly in anthropology. It is part of qualitative evaluation methods. It thus aims to overcome the limitations of quantitative surveys, which are based solely on statistical analyses: the latter provide an overall numerical picture of the results of a policy, but they say nothing about how it is implemented and the concrete difficulties that are responsible for its failures or unexpected effects. Direct observation, on the other hand, allows us to grasp the practical situations that constitute policy implementation on the ground: we then have a first-hand description of the implementation of a given programme, but also of the material conditions of its success or failure.

The direct observation of social practices has a long history. First of all, it is inseparable from anthropology and ethnology: when these disciplines fully constituted themselves as sciences during the 19th century, they progressively theorised ethnography as their main method of data collection. At the time, the aim was to study populations that were geographically and culturally distant. Observation made it possible to reduce the social distance with the subjects of the investigation through immersive research, which involved prolonged stays in the field, learning local languages, and a series of methodological precautions designed to avoid any ethnocentric judgement on the part of the ethnographer. At the end of the 19th century, and in a perspective closer to evaluation, the social surveys conducted in Europe among working-class or marginalised populations also used observation, which again was intended to reduce the social distance separating the ethnographer from the environment he or she was observing. Finally, in the 20th century, observation was used in sociology, and later in political science, to study ‘close’ objects (public services, political parties, organisations). The challenge is to ‘unfamiliarise’ these known practices, as the observer’s position invites us to decentralise our gaze and to question the causes and social mechanisms of activities that are taken for granted.

Within the qualitative family, observation is often combined with semi-structured interviews (see dedicated chapter on semi-structured interviews ), both with administrative agents and with the publics they encounter. Here again, observation makes it possible to reconstruct what these interviews cannot say: first of all, it makes it possible to circumvent the self-censorship that informants often impose on themselves in interviews, particularly when it comes to talking about the quality of their work and the performance of their missions. It also makes it possible to describe precisely certain aspects of public policies that the evaluators and the evaluated would not think of mentioning in an interview. Local routines and habits, the practical organisation of work, postures and attitudes or non-verbal communication with users – and all that they reveal about the social relations and inequalities involved in the relationship between civil servants and their publics – are then made directly visible (Perret, 2008). This type of approach can be particularly useful when the policies evaluated target sensitive populations (precarious or socially marginalised people, people with disabilities, etc.), with whom interactions require specific skills on the part of civil servants: self-presentation, the ability to explain the administrative process or to manage the anxiety or anger of the public encountered.

Conducting ethnography requires special preparation (Becker, 2002). While it may seem easy to go to a place to observe it, it is necessary for the observer’s outlook to be informed, and thus to constitute the space(s) studied as a scene of observation. A great deal of theoretical and documentary work is therefore essential to identify the relevant observation sites: which offices to observe, in which location (rural, urban, rich or disadvantaged municipality)? What activities and dimensions should be focused on? Should one try to compare the same moment of policy implementation in different places, or on the contrary analyse the different stages of a single administrative chain? After answering these questions, the ethnographer must go to the field and confront the inevitable tension between closeness and distance from the respondents. Observation implies sharing the daily life of the people being surveyed over a long period of time, while minimising the distance that potentially separates one from them. It is therefore necessary to align one’s appearance, speech and body language as much as possible with that of the people being observed. Conversely, it is also advisable to regularly leave the field of observation in order to “retreat” into a space specific to the reflection on the activities observed: in this case, it is a matter of avoiding too strong an immersion in the practice, and thus of reinstating the external position of observation.

Throughout the observation, the observed activities are regularly recorded in a fieldwork diary, in written or recorded form. Although there is no standardised form or method for writing it, this diary must combine not only the description (of the places observed, with plans and sketches, and of the activities taking place there), but also the ethnographer’s reactions: surprise, indignation or sympathy in the face of the phenomena observed provide information on the sensitivity of the observer, but also on the divergent sensitivity of the people being observed: it highlights the production of local representations of what is ‘normal’, ‘acceptable’ or ‘problematic’, representations that are not (yet) shared by an outsider who discovers the situation. From a methodological point of view, recording one’s reactions during the observation also makes it possible to objectify them in order to analyse them, thus limiting the impact of the ethnographer’s subjectivity on their observations.

II. How is this method useful for policy evaluation?

As Stéphane Beaud and Florence Weber (2012) note, the adoption of the ethnographic method results from dissatisfaction with the discourse that a group – in this case an administration – holds about itself: it is a question of going beyond the official presentation of an activity, what the legal rules, instructions or presentation brochures say about it, to analyse the reality of its practice. Such direct observation can therefore take place ex post , at the stage of implementation of public policies, which we know often corresponds to a real re-elaboration of the policy by administrative agents. It is particularly justified when it comes to evaluating a policy format that is difficult to quantify (reception at an administration counter, for example, see next section). Such an approach makes it possible to observe the diversity of local investments in the same policy, and its adaptation to the local conditions of its implementation (specificity of the public, of the socio-economic or political context) or of the actors who carry it out (legacy of local routines specific to a department, an office or a municipality). Such a perspective opens up two potential evaluative logics: highlighting the local innovations of which street-level bureaucrats are capable in order to deal with situations not provided for by the texts, and also considering the multiple logics that can possibly cause a public policy to deviate from its stated objective. Typically, this involves evaluating how a policy and the material resources allocated to it adjust with the realities encountered on the ground, identifying the issues neglected during its design, and isolating the practices that need to be modified to enable public action to produce its full effects.

III. An example of the use of this method: the evaluation of the reception policy in public services

Although it is already old, the report submitted to the Prime Minister in 1993 on Les services publics et les populations défavorisées: évaluation de la politique d’accueil (Paris: la Documentation française, 1993) [Public services and disadvantaged populations: an evaluation of reception policies] is a good example of the usefulness of the ethnographic method for evaluation. It illustrates first of all the interest of observation in order to carry out a detailed approach to the question initially posed in 1990 by the Interministerial Evaluation Committee: in a context marked by the development of the theme of the modernisation of public services, the challenge was to evaluate the capacity of local public service counters to effectively deal with the difficulties encountered on a daily basis by the most precarious populations. Such an analysis could not be carried out through a purely quantitative evaluation, nor by a simple interview survey: the objective was indeed to take an interest in interactions – that of state services located on the ‘front line’ with the publics who most depend on the benefit they allow – and to try to evaluate their quality – in particular to judge the capacity of users to effectively assert their rights. The aim was to examine the implementation of reception services, the quality of information provided to the public, the impact on the effectiveness of their rights, the possibility of implementing satisfaction indicators and, ultimately, the appropriateness of adopting selective reception policies, some of which would be adapted to disadvantaged groups.

This report also highlights the fact that observation is often combined with other methods to shed light on ethnographic findings and to connect them to more general statements on the observed administration: in this case, the qualitative survey is combined with a quantitative component (questionnaires sent to users to select them according to their socio-demographic characteristics). Within the qualitative component, the observations made at the counter were supplemented by qualitative interviews with users, reception staff and ‘social intermediaries’ (associations or civil servants from the social services who facilitate access to public services).

The research required the joint work of the administration’s inspection services and consulting firms or academic research centres (3 private firms and a university centre), and a preliminary work of identifying the relevant observation scenes: each fieldwork was prepared by a mapping of all urban services, which made it possible to identify eight public services considered central to the problem of reception (police, hospital emergencies, town hall, etc.) The localities surveyed were selected because of their pre-existing classification as “disadvantaged areas”.

These methodological choices are not without bias and illustrate in passing one of the difficulties of ethnographic research and the joint importance of the initial question, and of the observation protocol designed to answer it. In this case, the report concludes that it is necessary to adapt reception policies to disadvantaged populations, in particular by creating platforms or “public service centres” that bring together in the same place, within marginalised neighbourhoods, the offices of different public services (post office, town hall, etc.). These conclusions have been criticised by academics who have conducted their own ethnographic studies of precarious counter users (see Siblot, 2005; also Dubois, 2003): by focusing solely on the dependence of users on public services, the evaluation remains blind, in their view, to the multiple ‘coping’ strategies that precarious populations are able to develop in order to assert their rights, and which an in-depth ethnographic survey reveals. Similarly, the evaluation is accused of making an abusive generalisation by asserting the dominated nature of the users, whereas they are unequally endowed with different sorts of capital, particularly educational capital, and some of them may be in a position to interact on an equal footing with the reception staff.

IV. What are the criteria for judging the quality of the mobilisation of this method?

Ethnographic observation will be all the more useful if the observers have been able to carry out ‘casework’: in other words, to constitute the always singular situations observed in the field into ‘cases’ that can corroborate or invalidate a theory. The challenge is then to ’empirically delimit what is a problematic relationship between ideas and evidence, between theory and data’ (Hamidi, 2012). Maintaining this relationship requires ethnographers to pay constant attention to the practices observed in the field: they regularly bring up unexpected logics or themes, which must lead to enriching or modifying the initial theoretical question. This is an important issue in policy evaluation, where the initial design of the evaluation mission may be modified to avoid neglecting certain realities in the field (a problem raised in particular in the case of the reception of underprivileged populations in public services, cf. previous section).

The complexity of the ethnographic exercise then lies in the ability of observers to articulate, in the same research, cases of different status (Hamidi, 2012, referring to the extended case theory of the Manchester School). We can thus associate ‘exemplary’ cases for which we can expect, given the context and the populations concerned, that the theoretical hypotheses will be fully validated (to keep the previous example: a post office counter in a working-class neighbourhood of a neglected urban area), and ‘borderline’ cases in which they will only be partially confirmed (another counter located in a less isolated neighbourhood, or located in an area with closer community solidarity or a narrower network of associations). The various factors that can influence policy implementation are unevenly present in these different cases: bringing them together therefore makes it possible to identify with precision those that have a full impact on public action and those that are more secondary.

V. What are the strengths and limitations of this method compared to others?

As we have seen, direct observation makes it possible to grasp ex post the material conditions of the implementation of a policy on the ground, away from official presentations. The identification of observation scenes that illustrate different configurations of implementation of the same policy can allow for a particularly detailed evaluation of the effects of a given policy.

As we have also seen, observation is most often intended to be combined with other methods and complementary approaches. A classic criticism of direct observation concerns the possibility of generalising its results (external validity): observations, carried out in a specific area and necessarily situated, would only concern the local context they describe and would not make it possible to move from the micro-sociological scale to the macro-scale, that of a more global evaluation of the public policy under study. This objection has been partly overcome in recent work, which has emphasised the need to supplement ethnography with other methods, in order to connect the practices observed locally with their institutional framework and its history. This link can be established differently depending on the approach: in Vincent Dubois (2003)’s research on family benefit offices (CAF), the interviews conducted with the staff make it possible to link the observation of interactions at the counter with the career paths of the civil servants, and beyond that with the institutional conditions of their recruitment (absence of a clear definition of the counter staff’s mission and job description, etc.). On the same theme, Jean-Marc Weller’s research (1999) focuses on the material organisation of reception in administrations and what it reveals (budget cuts, withdrawal of the welfare state and a new managerial conception that turns users into ‘clients’) in order to link the interactions observed in the field to the global reforms of public action, of which they are the reflection.

Another limitation of the ethnographic method is the investment in time and personnel that it requires. While observation is technically inexpensive – it requires neither recording equipment nor computer processing of the data collected – it does require the presence of an observer, or more often a group of observers working in a concerted manner on several scenes and for long observation sequences (several months), alternating periods of ‘withdrawal’ and then ‘return’ to the field. The aim is to capture changes in practices (particularly when evaluating the implementation of a recent reform, which field officials are discovering and then gradually appropriating), but also, as we have seen, to allow the evaluators to regularly withdraw from fieldwork in order to compare their conclusions during the course of the survey and to clarify or modify the general observations they intend to make about the policy being evaluated. Although this long investigation period may therefore seem time-consuming, it is clear that it does not only refer to “field” work and observation: it also corresponds to a period of (re)drafting the final evaluation report and the general conclusions it will propose.

Some bibliographical references to go further

Beaud, Stéphane. 2010. Guide de l’enquête de terrain: produire et analyser des données ethnographiques . Paris: La Découverte, Grands Repères Guides.

Dubois, Vincent. 2003. La vie au guichet. Relation administrative et traitement de la misère . Paris: Economica.

Hamidi, Camille. 2012. “De quoi un cas est-il le cas? Penser les cas limites.” Politix , n°100, vol. 4: 85-98.

Jeannot, Gilles. 2008. “Les fonctionnaires travaillent-ils de plus en plus? Un double inventaire des recherches sur l’activité des agents publics.” Revue française de science politique 58, n°1: 123-40.

Siblot, Yasmine. 2005. « “Adapter” les services publics aux habitants des “quartiers difficiles”. Diagnostics misérabilistes et réformes libérales », Actes de la recherche en sciences sociales , 159, n°4: 70-87.

Weller, Jean-Marc. 1999. L’État au guichet. Sociologie cognitive du travail et modernisation administrative des services publics . Paris: Desclée de Brouwer.

Share This Book

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

How to observe business operations: An empirical study of family business

Contributed equally to this work with: Tsu-Cheng Chou, Hsi-Peng Lu

Roles Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Graduate Institute of Management, National Taiwan University of Science and Technology, Taipei, Taiwan, ROC

Roles Conceptualization, Supervision, Validation, Writing – review & editing

Affiliation Department of Information Management, National Taiwan University of Science and Technology, Taipei, Taiwan, ROC

Tsu-Cheng Chou,
Hsi-Peng Lu

Published: April 21, 2022
https://doi.org/10.1371/journal.pone.0267223
Reader Comments

The direct observation method is commonly used for data collection in family business case studies. Nevertheless, in research on family business succession that is mainly based on retrospective data, it is difficult to directly observe an event or situation at a given time. This paper thus first explores the application of observation data in the published research results of some family business case studies through a literature review. It then describes our observation methodology, i.e., sampling process, method, observation process, and reevaluation of our interview data, through a case study. Finally, the conclusion offers suggestions for using these observation methods, i.e., employing different types of observation, by considering running time and financial cost, familiarizing observers with various observation occasions, and seeking the assistance of relevant professionals for a research topic.

Citation: Chou T-C, Lu H-P (2022) How to observe business operations: An empirical study of family business. PLoS ONE 17(4): e0267223. https://doi.org/10.1371/journal.pone.0267223

Editor: Mehmet Serkan Kirgiz, Hacettepe Universitesi, TURKEY

Received: November 30, 2021; Accepted: April 5, 2022; Published: April 21, 2022

Copyright: © 2022 Chou, Lu. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within Supporting information files.

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The case study is the most commonly used qualitative research method in family business research [ 1 ]. There are six types of sources of evidence for case studies: documents, archival records, interviews, direct observations, participant observations, and physical artifacts. No single source has comprehensive advantages, and different sources are complementary [ 2 ]. Interviews are the most commonly used data source for family business case studies [ 3 , 4 ]. They are considered the most appropriate method when researchers are trying to understand what respondents consider essential or when research topics focus on deep-rooted values or beliefs and the complex dynamics of family and business [ 1 ]. However, family business owners and managers are often reluctant to provide sensitive information about their company and family and may be less willing to participate openly in research projects without higher levels of researcher engagement [ 5 ]. Narrators may gloss over stories, provide an embellished account of what happened or exaggerate the events and facts of intended and realized strategies [ 6 ]. It is thus difficult to prevent memory distortion [ 7 ] or inaccuracy [ 4 ], as well as any deviation caused by social image and self-esteem [ 3 ].

Family businesses are often privately owned, which restricts the type of secondary data access that is readily available for publicly listed corporations [ 5 ]. rthermore, files and records are often highly variable in quality, with great detail in some cases and virtually none for other programmatic components [ 8 ]. One way to reduce data inaccuracy is to calibrate the data sources of multiple respondents. Another essential method is to combine interviews with direct observation [ 1 ] to form an evidence database for research.

Family business succession has been one of the hot topics in family business research since the 1950s [ 9 – 11 ]. Hence, Le Breton-Miller et al. [ 10 ] have proposed an integrated, four-stage model for the succession process in a family business: defining the ground rules, training potential successors, selecting the successor, and passing the baton at the successor’s establishment. Because succession is a process involving numerous activities over a protracted period [ 12 ], its temporal factors are an inherently important consideration [ 13 ]. The critical approach to evaluating narratives in family business entails a rather traditional qualitative analysis, where scholars inductively analyze and interpret a narrative to extend, understand, or develop theoretical frameworks [ 14 ].

Family business succession is a long-term process involving three elements: the predecessor, the successor, and the succession process [ 12 , 15 ]. Undoubtedly, the source of evidence for its analysis should mainly be the retrospective interview data of crucial persons [ 3 , 4 , 16 ]. Accordingly, since direct observation is a record of real-time phenomena, conducting it while studying family business succession amid the mutual calibration of data accuracy is this paper’s focus. Based on our direct observation during our research on the social capital succession of family enterprises in the Taiwanese machinery industry, this paper discusses the following: (1) the application of the direct observation method in a case study and (2) how to calibrate observation data and interview data.

The research period was from May 2017 to August 2020. The case study method was used to observe the operation of 8 family businesses, interview 12 incumbents and successors, and examine the relevant documents of these family businesses. Although we obtained some exemplary papers on observational execution methods and analytical methods, there was a shortage of papers that met the transparency criteria for qualitative research. Therefore, this article offers some suggestions on the application of observational analysis in the future, including the research design of direct observation, the implementation steps, and the use of mixed-research methods, which are expected to help researchers improve the transparency and rigor of their case studies in their future research and publications. Therefore, this article first describes the transparency criteria for qualitative research and the implementation of direct observation methods. Next, it reviews some early and recent salient journal papers and then explains the process of our empirical research. Finally, it presents our findings, conclusions, and recommendations.

Literature review

Quality of observation research.

Regarding the desirability of replicability, this is a potentially contentious issue in qualitative research, in contrast to quantitative research [ 17 ]. Reay [ 18 ] offers several suggestions for publishing high-quality qualitative research. Her recommendations include (1) ensuring access to sufficient, high-quality data, (2) composing an appropriate research question to guide the article, (3) grounding the study in the relevant literature, (4) explaining the methods in detail, (5) telling an intriguing empirical story, (6) articulating a convincing theoretical story, and (7) providing a clear contribution to the family business literature.

For case study methodology, Maxwell [ 19 ] identifies the following four measures to improve a study’s validity: intensive and long-term involvement; creation of rich data, i.e., complete transcription of interviews; triangulation, involving various informants and methods; and a multiple-case design to allow comparison across cases.

Concerning the practical application of research procedures, Aguinis and Solarino [ 17 ] offer best-practice recommendations—approximately 12 transparency criteria that authors can use when conducting their work. These requirements include research design (i.e., qualitative method, research setting, position of researcher on the insider–outsider continuum, sampling procedures, relative importance of participants/cases), measurement (i.e., documenting interactions with participants; saturation point; unexpected opportunities, challenges, and other events; management of power imbalance), data analysis (i.e., data coding and first-order codes, data analysis and second-and higher-order codes), and data disclosure (i.e., raw material availability). They emphasize that transparency is a continuous variable and a matter of degree. Therefore, the more fully met criteria there are, the better [ 17 ].

Observation study execution procedure

When researchers visit their case study site, opportunities for direct observation are created [ 2 ]. Direct observation requires them to spend time observing and experiencing the operations of a company or organization, and these observations can be time-consuming and expensive. This kind of data source is particularly suitable for studying all aspects of organizational culture. Only by experiencing corporate activities can the fundamental values and ideas commonly held by the members of an organization be understood [ 1 ].

Direct observations of phenomena, behaviors, or objects’ occurrence processes that lack an oral report by interviewees can prevent the influence of interviewees’ screening of information or incomplete expression. Thus, the data obtained are highly reliable, and the distortion rate of data conversion is minimized [ 5 ]. Furthermore, direct observation can record actual real-time events and the contexts of their occurrence [ 2 ], which is its most significant advantage.

There are few research objects suitable for direct observation in family business research. Observations must be made of specific behaviors or objects [ 8 , 20 ]. Typically, however, only performed or explicit behaviors can be observed, but one cannot understand the internal thinking process of the relevant research subjects. Therefore, to address a research question, the observer may have to infer and interpret the observation results, and such interpretations may be entirely subjective [ 21 ]. In addition, researchers must witness the occurrence process of an event as it happens. Predicting the timing of events makes it difficult to observe them. The research subjects and events that can be observed simultaneously are limited, making the time cost of observation very high [ 2 , 8 ].

Therefore, the observation method is more suitable for studying nonhistorical situations. As Yin [ 2 ] has pointed out, assuming that the phenomenon to be studied is not entirely historical, some information about appropriate behaviors and environmental conditions can be obtained through observations. Such observations can then be used as another data source in a case study.

For reliability, Yin [ 2 ] suggests creating a case study protocol that details the exact procedure that the cases have been approached with. Such a protocol should include field procedures, e.g., case sites, credentials, and schedules. Observation studies are conducted in steps to achieve the regularity and accuracy required for direct observation [ 20 , 22 ]. These steps include (1) selecting a site based on its reflection of a theoretical issue, not because it more or less represents an issue of immediate concern or is simply convenient; (2) gaining entrée—when the subject of observation is unfriendly, the gatekeepers should be contacted to obtain their approval and support; (3) starting the observation and spending some time becoming familiar with the local environment and how to interact within it; (4) using the recording method that can best help the researcher to retrieve and analyze the collected data; (5) finding some patterns in the observation for further observation or extended study; and (6) continuing to observe until theoretical saturation is reached.

Moreover, researchers’ actual strategies for observation include entering the observation scene or contacting individuals through network referrals, concisely stating the purpose of the study and the intent of the observations, having sufficient knowledge of the research field and subject, showing the ability to conduct research, and graciously asking for cooperation and negotiating a convenient time and place for observation [ 2 , 20 , 21 ].

Literature on observation studies

A well-rounded study should clearly state its observation procedure to confirm the reliability and validity of its case study. Aguinis and Solarino [ 17 ] have searched and analyzed 52 articles involving “interviews with elite informants” that were published in Strategy Management Journal from 2000 to 2017. Overall, and across the 12 criteria, none of the 52 studies are sufficiently transparent to facilitate exact replication. Furthermore, few studies list all the sources they used, while only one-third of the total sample identify the nature of their sources.

Leppäaho et al. [ 23 ] have provided similar research findings. They conducted an in-depth qualitative content analysis of 75 qualitative family business case studies, published between 2000 and 2014 in high-quality academic journals. Overall, these applications of the case study method appeared to lack transparency.

Accordingly, in the present study, we first reviewed 26 qualitative research papers on family businesses by Fletcher et al. [ 24 ]. Among them, ten used direct observation for collecting data, and only two provided specific explanations on their practical application of direct observation. For example, Kotlar and De Massis [ 25 ] investigate goal setting among 19 family businesses, having obtained their research evidence through interviews, observations, and documents, and employ a table to detail the participation of different research subjects in family and business meetings (board meetings, family meetings, and unscheduled meetings). In total, 114 meetings were observed across 19 companies, the least of which was three meetings and the most 12.

Additionally, Nordqvist and Melin [ 26 ] investigate strategic planning practices in three medium-sized and multigenerational family firms and explain how to observe family businesses as follows:

The empirical material of the overall project involves 98 face-to-face interviews (with key actors , such as owners , managers , consultants , board members , family members , accountants , former managers) , observations of 10 meetings where strategic issues were treated (e . g ., board meetings , top management team meetings , strategic planning meetings , strategy away days) , casual conversations , site visits and secondary sources such as company reports , minutes from board and top management team meetings and strategic plans . All interviews were audiotaped and transcribed verbatim . Detailed notes were taken during all observed meetings [ 26 , p. 18].

The internal meetings of enterprises are the most observed occasions in case studies of enterprises. However, most studies provide only a brief description of the implementation process of these practical sessions, with limited descriptions of the meeting occasions, frequencies, observation focus, and collected data:

The fieldwork comprised 95 interviews , observations of several meetings , and many site visits to the firms where we interacted informally with family and firm-related individuals [ 27 , pp. 55–56]. We conducted these two case studies over two years , with more than ten interviews in each firm , supplemented with several informal talks with key actors and participation in several meetings . Thus , each study covers a period of more than ten years , where we both reconstructed the near history and observed the ongoing change processes in real time . Together with documents and notes from observations and informal talks , the transcribed interviews represent the basis for our interpretations of the family business culture and its impact on entrepreneurial processes [ 28 , p. 198].

Case observation is often carried out simultaneously with interviews. For example, a case study on the social network of eight Vietnamese nail salons in London mentions that its authors had observed the cases during their discussions without specifying the frequency and manner of the observed cases:

The interviews with the owner-managers took place in the nail shops , enabling observations of the business to be made . By interviewing and observing these different aspects of the supply chain , it was possible to cross-check and triangulate data from various sources and build up a picture of how the sector operated [ 29 , p. 384].

Several studies only mention case observations and do not explain how to conduct observations; that is, they lack any discussion of observation subject, location, number, or other research processes:

his research arises from a self-enriching process of reading , analysis , observation , interviewing , and writing [ 30 , p. 268] Data were collected over two years (2003 and 2004) from a range of sources , including interviews of senior managers from each firm (34 in total) , observations , notes from 22 field visits , questionnaires , and firm documents [ 31 , p. 155].

Second, we examined whether the application of observational methods in the family business research literature of the last five years has improved compared to earlier periods. We found several studies that once again only mention case observations without explaining how they conducted their observations:

We took field notes , documenting our company visits and observations . The research output of this stage is not presented here but was extremely useful for the research team to contextualize the participants’ narratives of the succession process collected in the interviews [ 32 , p. 583]. The authors started by independently examining the data in interviews , observation notes , and documents . Then , a coding process was carried out by reading and rereading transcripts , notes , and documents and then using codes for sentences or paragraphs in order to organize data [ 33 , p. 173].

We also found that observational methods are often used as an incidental data source and that some papers describe the interview process in far more detail than the observational process entails:

We integrated data collected from a wide range of sources : interviews , historical records , financial data , sustainability data , and direct observations to triangulate (to adopt different angles in observing the same phenomenon) . We chose to use the semistructured interview because of its high degree of flexibility and because it offers the opportunity to address themes that can come to light during the semistructured interviews . Data collection was conducted from March 2015 to April 2015 . The interview transcriptions , field notes , and documentary analysis were coded into key themes [ 34 , p. 977].

Some studies briefly account for the frequency of observations but do not explain how they conducted their observations, omitting their observation subject, location, number, or other research processes:

Our single-case study framework allowed us to conduct participant observations . During this period , one researcher systematically attended all meetings organized by the consulting firm with the different stakeholders (approximately one day every two months) [ 11 , p. 260]. Data processing was performed using a handwritten copy of our observations , interviews , and secondary data sources [ 11 , p. 261].

Third, we also obtained published articles with high transparency that could serve as examples for future researchers. For instance, Wielsma and Brubbinge [ 35 ] conducted a longitudinal single case study. They detail their procedures for data collection as follows:

The data were collected between September 2015 and September 2017 . During this time , the company was visited in intervals ranging from one week to four months . Interviews were recorded and transcribed , while note-taking was used for the informal talks . The participants in the project included family members (six individuals from three generations) and nonfamily employees (twelve individuals) . Additional data were obtained through nonparticipant observation of staff meetings , review of company documents (including the website) , and review of publications from third parties . The use of multiple sources during data collection enabled the triangulation of information [ 35 , p. 41].

Especially the way the authors use the table [ 35 , p. 42], which lists the data collection methods, periods, data sources, and durations, including nonparticipatory observations and other research methods. The research procedures and data are thus highly transparent and reproducible.

The following problems are found in the above research that has implemented the observation method to collect family business case data. First, the collection and design of observation data are accompanied mainly by interviews, and only a few works actively plan the occasions, frequencies, and focus of observations. When determining an observation site, although directly related to a research topic, most observations are based on an interview location. Second, the most frequent observation context is the internal meeting of a company. However, the interaction between a company and the outside world cannot be observed within the company. Third, only some of the observations clearly state the frequency and nature of the observation meetings, and most of them lack an explanation. Fourth, the observation recording method does not explain the use of an open structure or a closed record table. Fifth, it is impossible to grasp how most studies’ observation data are combined with other interviews and documents to form the evidence chain of their information. Finally, we have not discerned any discussion of how to overcome difficulties and obtain the support of an observation object. Accordingly, family business case studies seem to be dominated by interview data, while observation data are only incidental.

For a researcher interested in the direct observation of family business cases, it is unfortunate that he or she cannot refer to the existing empirical research literature to learn the relevant application steps. These include establishing the precautions of the observer, selecting a site, obtaining the support of the subject of observation, and recording the observation(s).

Materials and methods

The methodology for this project was the case study approach. A case study is an appropriate data collection method if the goal is to understand how or why a phenomenon occurs [ 23 , 36 ]. In the context of family firms, case studies are helpful because data come from various sources, allowing scholars to capture multiple perspectives [ 1 ]. In addition, this approach enables a deeper and more meaningful understanding of real-life events in a business context [ 2 ].

To increase our understanding of succession in family business, a qualitative multiple-case study methodology was identified as the most appropriate means to answer our how and why questions [ 1 ]. The research period was from May 2017 to August 2020. The case study method was used to observe the operation of 8 family businesses, where we interviewed 12 predecessors and successors.

Theoretical sampling

In this study, we selected cases with theoretical sampling by choosing the predecessors and successors of representative family businesses. All the chosen family businesses were directly contacted by the researcher. Hence, the researcher was able to determine whether the cases would meet the research purposes by gauging the family businesses’ internal functions and how their two relevant generations interact.

Purpose sampling should be conducted in qualitative research because how much data are collected from how many subjects are irrelevant [ 37 ]. Instead, it is more important to collect correct information that is sufficiently rich to enable a better understanding of the case phenomenon’s meaning [ 4 ]. This study therefore entailed the following preselection criteria: first, the cases’ capital or business volume should be relatively large and unique in their market to show a value of succession. The interviewed managers should be responsible for all or most of their company’s operations to affirm the significance of the research. Therefore, the sampling criteria for this study included two levels: organizational and individual.

The conditions regarding organization level were mechanical industry, small-and-medium enterprise (SMEs), and family business. In Taiwan, the robotic industry is defined by cases that involve members of the Taiwan Association of Machinery Industry (TAMI). According to the Taiwanese government, SMEs must contribute capital below NT $100 million (US $3.57 million) or have fewer than 200 employees. Finally, a family business is an enterprise where family members obtain ownership and are responsible for operations.

At the individual level, we believed it was worthwhile to interview the predecessors and successors in the same enterprises to conduct data cross-validation. In addition, the successors needed to have more than simply a title or position; they needed to be among the top management team that is responsible for all or most operations in their company.

There is no accepted ideal number of cases. Eisenhardt [ 38 ] suggests that it is challenging to generate theory from fewer than four cases, but more than ten make it challenging to manage and analyze data. The appropriate number of cases will therefore be determined by how each case incrementally contributes data [ 39 ].

The machinery industry is one of the three most significant industries in Taiwan. Most companies in this industry are family-owned SMEs. Before formal invitations were sent, the researcher asked the chairman of TAMI to solicit support for this study. This allowed the researcher to select the case study companies with purpose sampling.

Hence, we selected eight family businesses with successful succession and spoke with 12 interviewees. All eight companies are market leaders or uniquely positioned in the marketplace. Passing the torch to the next generation is worthwhile for these companies. The capitalization of these companies ranges from US $357,000 to US $6.6 million. Their number of employees ranges from 30 to 200 people. The youngest company is 18 years old, and its founder is passing the business to the second generation. The oldest company has an operating history of 70 years, and the third generation has already inherited it from the second generation. All the participants met the research criteria for a multiple-case study. Table 1 summarizes the background of these eight companies, which are sizable, in the same sector, and align with the standards suggested by Miller & Le Breton-Miller [ 40 ] in their review of research on family businesses.

PPT PowerPoint slide
PNG larger image
TIFF original image

https://doi.org/10.1371/journal.pone.0267223.t001

Data collection

This study involves three data sources: interviews, direct observations, and documents. Family business succession is a long-term process [ 12 ], and the interaction between predecessors and successors may start during family life instruction before the successor matures. These complex, long-term interactions have many details that are unknown to outsiders. Information can be obtained through retrospective interviews. In addition, researchers often have close contact with the interviewed companies and interviewees in their work, offering researchers many direct observation opportunities to collect information beyond interviews. These include observing the interaction of family members in a case enterprise, evaluating the division of labor among family members and the interaction among peers at the exhibition site, and monitoring how a predecessor leads the next generation to expand horizontal and vertical social relationship capital during fellowship activities. This study also uses document data from the Economic Daily News (Taiwan) database, books, and magazines.

This article mainly discusses the use of observational methods, and thus the steps we used to conduct our observational data collection are described below and illustrated in Fig 1 .

https://doi.org/10.1371/journal.pone.0267223.g001

Site selection.

In this study, interviews for seven of the eight case enterprises were conducted in their factory to facilitate the observation of factory operations. Nevertheless, the observation field was not limited to the inside of each case company. We also needed to collect the external social capital succession data of the subjects according to the research framework. Therefore, we also conducted observations at golf clubs, a meeting of TAMI’s board of directors and supervisors, industry dinner parties, domestic and foreign exhibition sites, industry symposiums, and other forms of external enterprise cooperation.

Obtaining support.

Researchers must attain a high level of trust when studying family businesses to potentially improve response rates and accuracy when eliciting information through direct means, such as surveys [ 35 , 41 ]. The critical factors for the successful interviews and observations in this study were that the researcher had preexisting friendships within the machinery industry and the support of the chairman of TAMI.

Preparation in advance.

We searched the relevant reports of the case businesses through the databases of Economic Daily News , Global Views Monthly , and Business Weekly . In addition, we learned the backgrounds of the cases and interviewees to enrich our preparation work before the interviews.

Starting observation.

All cases were contacted directly by the researcher via telephone, who called to explain the purpose of the study and confirmed whether they agreed before making formal visits. During the first visits, the researcher indicated the research relationship with the interviewees, the research purpose, and the research outline. Research interviews and observations were conducted after the second visits. The intervening period from the first formal visit to the enterprise observation was often more than three weeks. The research period was from May 2017 to August 2020.

Observation records.

In this study, unstructured observations and open observation records were used to plan the field or direction to be observed, but the detailed items were not established in advance. After determining whether the observed information was helpful for the study, the key points were recorded in a notebook.

Extended observation.

Once we discerned some patterns in our observations, we asked whether there were other suitable places for extending our observations through activities, itinerary visits, industry seminars, golf parties, or other events. There were thus a total of eight factory visits, three golf parties, two industry seminars, three exhibitions, one foreign visit, one dinner party, and one board of directors and supervisors meeting. Hence, there were seven different observation types, 19 observation occasions, and 26 observation records (see S1 Appendix ).

Observation guide

The observation guide is a conceptual framework that we developed via the research literature and research objectives. It includes (1) the size of the case enterprise, including the production equipment and products that are used to confirm the enterprise’s research value; (2) the roles played by the predecessor and successor in the current stage of family business succession; (3) the succession of the internal social capital of a family business (i.e., how to build a human resources team, internal trust relationship, shared vision, and standard language); and (4) the succession of the external social capital of a family business, i.e., vertical social capital, horizontal social capital, and social relationship capital, such as its relationships with customers and suppliers, interaction with peers, inheritance of industrial contacts, and relationships with industry associations, governments, and media.

Research ethics

Observational research pays special attention to ethical issues because it involves entering the private realm of the case study. Therefore, the following guidelines were strictly followed during the observation period [ 8 , 22 ]: (1) showing identity—at any site, the researcher stated that the goal of the research is the composition of a doctoral thesis and clearly defined the researcher’s role. Thus, the identity and purpose of the study were known to any case under observation, both in the public and private spheres. (2) Informed consent—with the permission of the case enterprise operator, the researcher entered the enterprise for observation. In the first meeting, the purpose and implementation of the study were clearly stated, and permission was obtained. After the second visit, the researcher began to conduct observations. (3) Commitment to privacy and confidentiality—all the case enterprises and interviewees in the study were treated anonymously. All data will be used for research purposes only and will not be disclosed. (4) Equal and mutually beneficial research relationships—observational research requires researchers and participants to establish a deep bond that is connected, shared, and equal. The cases thus provided a wealth of information to facilitate discussion in this study. In addition, after completing the study, the researcher promised to provide a reference for the respondents.

Results: Combining observations with other data

Following Neergard and Leitch [ 42 ], the authors started by independently examining the data from interviews, observation notes, and documents. Multiple data sources are a strength of any case study [ 1 , 2 ]. Data collected from various sources can assist researchers in organizing data sources by establishing a case study database comprising interview results, observation notes, and files to improve the reliability of data and assist researchers with subsequent analysis. Using different data sources, researchers can triangulate data and adopt different angles to observe the same phenomenon [ 2 ], thus making their findings more credible and accurate [ 8 , 43 ]. Accordingly, the present study can be used as an example to illustrate the results of mutual calibration and rechecking via direct observation and other data.

Succession stage

Case B is the current chairman and the second generation of his family business. He has been in control for more than 30 years. Five years ago, he made his younger brother the general manager and his younger sister the business manager. The observation data show that the interviewee did not receive a single phone call, document, or request during the entire three-hour morning interview in his office. Indeed, the interviewee claimed that he had finally completed the business succession.

Case F stated that his elder brother is the chairman of the board of directors but he is the general manager responsible for the company’s operation. Therefore, he would only ask his brother questions if necessary. According to the observation data, the researcher previously met the chairman of the case enterprise. The chairman stated that his younger brother should be interviewed as a representative, confirming the younger brother’s representativeness of the family business.

Internal social capital: Family human resources

Case B stated that the second and third generations of the family are the primary sources of human resources, accounting for approximately 10% of the company’s employees. According to the observation data, during a visit to the factory, the interviewee introduced the colleagues on the site who are involved in each production process, including the cousin in charge of factory affairs, the nephew managing business development, and the son-in-law directing the department of research and development, demonstrating that numerous family members work in the enterprise.

Case B stated that he began to reserve human resources for future succession when he was in high school. After his university graduation, he agreed to allow ten classmates to work in his family’s enterprise. They then became essential team members who aided his takeover of the management of the enterprise. According to the observation data, when visiting the factory, the researcher met with two deputy general managers, both of whom were high school classmates of the interviewee and had worked in the company for 40 years, confirming the statement of Case B. The interviews and observations in Case B are the same as those presented in a book [ 44 ].

External social capital: Supplier relationships

Case G (founder) emphasized that outsiders were aware that he paid the most attention to quality, acted according to principles, and never accepted commissions or gifts. The observational data showed that Case G refused to accept the researcher’s gift during the first visit, indicating that Case G’s words and actions were consistent.

Case G stated that the second generation is responsible for business development. According to the observation data, the second generation visited the Taichung Automation Industrial Equipment Exhibition and suppliers and customers, confirming the second generation’s role in the enterprise.

Case C (predecessor) is the second-generation successor, and there are currently eight people in the third generation who are department heads in the enterprise. The third-generation successor led four cousins to visit suppliers and customers at the Plastic Machinery Equipment Exhibition held in Guangzhou according to the observation data. After arriving at the exhibition center, the successor greeted the president and secretary-general of the Machinery Association in the office of the exhibition venue and introduced the work that each person was responsible for in the family business. This therefore confirms the division of labor among Case C’s family members in the enterprise and the firm’s practice of maintaining external social capital with, e.g., suppliers and customers.

External social capital: Industry social network

Case F stated that his brother serves as the chairman of the board while he serves as the general manager and participates in industrial activities on behalf of the company. In addition, according to the observation data, the younger brother represents the company by serving on TAMI’s board of directors and participating in golf activities.

Case G (predecessor) stated that the two generations of the family business divide the labor. He is responsible for internal production and manufacturing, and his eldest son is liable for business development and the maintenance of external relations. According to the observation data, the son of Case G attended the industry symposium sponsored by the Economic Daily News on behalf of the company. The son of Case H also represented his company by participating in the industry symposium held by Taichung City Government. This confirms that the second generation is responsible for maintaining the social relation capital of external social capital.

The machinery industry has a close relationship with people in the financial and economic industry media. When senior news media executives visited the case enterprises, the predecessor typically brought the successors together to receive them and introduce them to the next generation. According to the observation data, this occurred at Cases A, C, G, and H.

Cases D and E are the second-generation successors of enterprises with close relations. They share ideas and resources. According to the observation data, Cases D and E play golf, have dinner, and exchange industrial information almost every week.

Discussion and conclusions

Based on direct observations of cases of family business succession, this study provides the following discussions regarding the execution method of collecting direct observation data in a case study:

First, different data can be obtained from different observation contexts. Observations of the internal operations of enterprises often accompany interviews. We could therefore confirm whether a company’s operating scale and product attributes met the criteria for sample selection. We could also observe the succession of internal social capital, including the division of a workforce within a company, the trust relationship between members of an organization, a shared vision, and a common language. The external field observations of the case enterprises were mainly conducted to evaluate the following: horizontal social capital of the family businesses, including their interactions with the industry and friendships via the second generation of enterprise successors; vertical social capital, including on-site interactions with suppliers and customers; and social relationship capital, including interactions with government departments, industry unions, and industry media.

Second, there are many business activities; hence, it is difficult to systematically implement case observations. This study was therefore conducted over three years, inside and outside of enterprises in Taiwan, at internal production lines, external exhibitions, golf courses, and dinner parties. The costs of conducting so many case observations are very high, and their implementations are also difficult. However, the data we collected are extremely valuable and cannot be replaced with interview data.

Third, it is necessary to clearly state the recording method for obtaining observation data and how the data will be collated and rechecked with other data sources to form a transparent chain of evidence, which helps strengthen the reliability and validity of the research. Observation is not a random glance. Scientific inquiry entails using observational methods that require disciplined training and rigorous preparation [ 8 ]. Such training includes learning how to write descriptively, practicing the disciplined recording of field notes, separating useful details from trivia to obtain the former without being overwhelmed by the latter, and using rigorous methods to validate observations. Moreover, proper preparation has mental, physical, intellectual, and psychological dimensions that must be learned to effectively concentrate during an observation [ 8 ].

Accordingly, this article provides five suggestions for applying the observation method in family business case studies, which derive from our findings. (1) Observation design should be included in the research design stage. The researcher should select observation objects, determine the occasion and frequency of observation, design the observation outline, and compose records according to the research question. Observation methods are not incidental and should have a thorough and complete design. (2) Although occasions for observation vary according to research topic, one should pay attention to internal and external observations of the research case. External observation facilitates obtaining multiple pieces of information and can account for any deficiency of inner observation. (3) Long-term observation is necessary. Since corporate activities are very diverse, the time and cost of observations are very high, and it is challenging to implement systematically, it is difficult to obtain sufficient research data in a short period. (4) Researchers should be familiar with various observation environments, have sufficient industry knowledge, and exhibit considerable social skills to harness every occasion to perform an observation. For example, formal industry events, such as association meetings, exhibitions, or forums, and informal events, such as fellowship parties, galas, and dinner parties, are more complex than company meetings. Thus, it is challenging to familiarize oneself with the specific environments of various observation venues. (5) Researchers should "get a ticket" to an observation and attempt to bridge the gaps between academia and industry. Finding professionals with rich connections within an industry can help the observer attain the support and trust of research cases and the opportunity to observe family businesses’ internal and external operations in depth.

Fourth, observations can be both qualitative quantitative. According to the literature and this study, an observation is often accompanied by interviews [ 2 ]. If interviews are conducted in respondents’ company or factory, the number of employees in the case company that are observed on site can confirm the company’s size. For example, the brand, quantity, and depreciation of factory production equipment can also be used to evaluate investment funds. The number of container trucks in a factory parking lot can be used to reference shipment turnover. Whether a factory regularly requires overtime can affirm a company’s revenue. Observations at an exhibition site may also have quantitative data. For instance, the size of the booth at an exhibition is often an indicator of a company’s size. The number of visiting buyers can also be used as a reference for revenue. Overall, this approach represents a mixed-method design.

Mixed-methods research utilizes both qualitative and quantitative perspectives in the same study. Here, quantitative survey data were collected for empirical information (existing contextual, relational, and business structure facts), while qualitative data were collected through interviews [ 45 ]. Mixed-methods research can therefore add to the knowledge of and generalizability of a case topic. Specifically, researchers can capture the distinct complexities of family business research [ 46 ].

Mixed-methods research offers the capability to make the generalizable observations that usually result from quantitative approaches and to combine them with the rich and "thick" descriptions that typically result from qualitative techniques [ 46 ]. Moreover, the combination of qualitative and quantitative perspectives can help uncover previously unobserved relationships, generate interesting new insights to advance theory development [ 47 ], and help resolve theoretical complexities [ 46 ].

Family business researchers face challenges concerning data collection and the availability of specific data sets and are expected to employ greater rigor to ensure uniform applications of methods [ 5 , 46 ]. Direct observation is an important data source for a case study. However, such observation requires more time and costs than conducting interviews or accessing archives. Hence, its implementation should be more prudent to prevent the failure of research observations to produce useful evidence. On the other hand, if the difficulties of observation implementation can be overcome and observation sites are sufficiently diverse and complete, then data completeness will be fostered and accuracy will be increased, facilitating the construction of a comprehensive research database involving other data sources, which will form a solid evidentiary basis for case studies.

Conclusions

Qualitative studies are a crucial for researchers to not only answer important research questions but also develop new questions [ 48 ]. The case study is a robust methodology that can be employed in an inventive and rigorous way to generate a more fine-grained contextual understanding of family business phenomena and advance research in the field [ 23 ].

Interviews are the primary method for conducting case studies. However, family business case researchers could use data other than pure interviews when developing their accounts [ 23 ]. Since researchers want to avoid an overreliance on any one type of sample or data, more creative approaches to data gathering and methods are needed. Furthermore, we should encourage future studies on the nonfinancial performance of family firms to use primary data or draw from a combination of different data sources [ 49 ].

In this article, we have analyzed how observation studies have been pursued in the family business literature and have offered suggestions on how they may be used in the future to more effectively capture the idiosyncrasies, dynamics, and processes of family businesses. A family business researcher’s predilection is to rely on past literature as a source of inspiration, and thus such sources should be numerous and of varying types [ 50 ]. Where researchers find inspiration and clarification can be an essential factor in the success of their theorizing efforts. Observation can and should be a central component when theorizing. Accordingly, we expect family business research to be more analytical and less subjective than most extant studies.

Supporting information

S1 appendix. case observation records..

https://doi.org/10.1371/journal.pone.0267223.s001

View Article
Google Scholar
2. Yin RK. Applications of case study research. London: Sage; 2014.
PubMed/NCBI
8. Patton MQ. Qualitative research & evaluation methods. Thousand Oaks, CA: Sage; 2015.
12. Seetoo DH. The governance, sustainability, and succession of family business. Taipei, Taiwan: Commonwealth; 2021.
16. Floris M, Dessì C, Dettori A. Mapping narratives in family business studies: a guideline for researchers. In: De Massis A, Kammerlander N, editors. Handbook of qualitative research methods for family business. Cheltenham, Glos, UK: Edward Elgar; 2020. pp. 224–247.
19. Maxwell JA. Qualitative research design: an interactive approach. Thousand Oaks, CA: Sage; 2012.
20. Ye ZT. The observation method: a wonderful journey into the world of senses. In: Wang XH, Kuo LW, editors. Communication research methods. Beijing: Higher Education Press; 2022. pp. 261–277.
21. Lofland J, Snow DA, Aderson L, Lofland LH. Analyzing social settings: a guide to qualitative observation and analysis. Belmont, CA: Wadsworth; 2006.
22. Angrosino M. Doing ethnographic and observational research. Thousand Oaks, CA: Sage; 2007.
36. De Massis A, Kammerlander N. Handbook of qualitative research methods for family business. Cheltenham, UK: Edward Elgar; 2020.
42. Neergaard H, Leitch CM. Handbook of qualitative research: techniques and analysis in entrepreneurship. Cheltenham and Northampton, MA: Edward Elgar; 2015.
44. Wei L. Create a brilliant life with gratitude. In: Tsai CY, et al., editors. Climb the top: 10 super passwords of EMBA elites from Cheng-Kung University. Taipei, Taiwan: Chin-Chung Press; 2018. pp. 36–59.
48. Reay T, Zhang Z. Qualitative methods in family business research. In: Melin L, Nordqvist M, Sharma P, editors. SAGE handbook of family business. Thousand Oaks, CA: Sage; 2014. pp. 573–593.

What Is Direct Observation?

Tuul & Bruno Morandi/Getty Images

Research, Samples, and Statistics
Key Concepts
Major Sociologists
News & Issues
Recommended Reading
Archaeology

There are many different kinds of field research in which researchers can take any number of roles. They can participate in the settings and situations they wish to study or they can simply observe without participating; they can immerse themselves in the setting and live among those being studied or they can come and go from the setting for short periods of time; they can go "undercover" and not disclose their real purpose for being there or they can disclose their research agenda to those in the setting. This article discusses direct observation with no participation.

Direct Observation With No Participation

Being a complete observer means studying a social process without becoming a part of it in any way. It is possible that, because of the researcher’s low profile, the subjects of the study might not even realize that they are being studied. For example, if you were sitting at a bus stop and observing jaywalkers at a nearby intersection, people would likely not notice you watching them. Or if you were sitting on a bench at a local park observing the behavior of a group of young men playing hacky sack, they probably would not suspect you were studying them.

Fred Davis, a sociologist who taught at the University of California, San Diego, characterized this role of the complete observer as "the Martian." Imagine you were sent to observe some newfound life on Mars. You would likely feel obviously separate and different from the Martians. This is how some social scientists feel when they observe cultures and social groups that are different from their own. It is easier and more comfortable to sit back, observe, and not interact with anyone when you are "the Martian."

How to Decide What Type of Field Research to Use?

In choosing between direct observation, participant observation , immersion, or any form of field research in between, the choice ultimately comes down to the research situation . Different situations require different roles for the researcher. While one setting might call for direct observation, another might be better with immersion. There are no clear guidelines for making the choice on which method to use. The researcher must rely on his or her own understanding of the situation and use his or her own judgment. Methodological and ethical considerations must also come into play as a part of the decision. These things can often conflict, so the decision might be a difficult one and the researcher could find that his or her role limits the study.

Babbie, E. (2001). The Practice of Social Research: 9th Edition. Belmont, CA: Wadsworth/Thomson Learning.

What Is Naturalistic Observation? Definition and Examples
An Overview of Qualitative Research Methods
Immersion Definition: Cultural, Language, and Virtual
An Introduction to Medical Anthropology
What Is Participant Observation Research?
An Introduction to Cultural Anthropology
What Is a Cohort Effect? Definition and Examples
What Is Ethnography?
What Is Ethnomusicology? Definition, History, and Methods
What Is a Primary Source?
Understanding the Use of Language Through Discourse Analysis
What Is Environmental Science?
What Is a Population in Statistics?
What Is a Quota Sample in Sociology?
Conducting Case Study Research in Sociology
Deductive Versus Inductive Reasoning

An official website of the Department of Health & Human Services

Search All AHRQ Sites
Email Updates

1. Use quotes to search for an exact match of a phrase.

2. Put a minus sign just before words you don't want.

3. Enter any important keywords in any order to find entries where all these terms appear.

The PSNet Collection
All Content
Perspectives
Current Weekly Issue
Past Weekly Issues
Curated Libraries
Clinical Areas
Patient Safety 101
The Fundamentals
Training and Education
Continuing Education
WebM&M: Case Studies
Training Catalog
Submit a Case
Improvement Resources
Innovations
Submit an Innovation
About PSNet
Editorial Team
Technical Expert Panel

Benefits of direct observation in medication administration to detect errors.

Diaz-Navarlaz T, Pronovost P, Beortegui E, et al. Benefits of Direct Observation in Medication Administration to Detect Errors. J Patient Saf. 2009;3(4). doi:10.1097/pts.0b013e31815b4cc3.

In this study, investigators observed several potential sources of medication administration errors at a university hospital. The authors outline specific strategies for change that are currently being piloted on their ward in response to their findings.

Technology-induced errors associated with computerized provider order entry software for older patients. August 23, 2017

Impact of a national multimodal intervention to prevent catheter-related bloodstream infection in the ICU: the Spanish experience. September 18, 2013

Using Healthcare Failure Mode and Effect Analysis to reduce medication errors in the process of drug prescription, validation and dispensing in hospitalised patients. October 3, 2012

Healthcare failure mode and effect analysis in the chemotherapy preparation process. November 17, 2021

Nurses' perceptions and demands regarding COVID-19 care delivery in critical care units and hospital emergency services. December 2, 2020

Impact of simulation-based closed-loop communication training on medical errors in a pediatric emergency department. May 6, 2020

Systematic review of computerized prescriber order entry and clinical decision support. February 6, 2019

Medication errors in an intensive care unit. May 20, 2009

Introduction of checklists at daily progress notes improves patient care among the gynecological oncology service. December 5, 2012

Cognitive processes involved in blame and blame-like judgments and in forgiveness and forgiveness-like judgments. May 9, 2007

Learning accountability for patient outcomes. July 21, 2010

We need leaders: the 48th Annual Rovenstine Lecture. May 19, 2010

Enhancing physicians' use of clinical guidelines. January 8, 2014

An interview with Peter Pronovost November 2, 2005

Communication on safe caregiving between community nurse case managers and family caregivers. April 7, 2021

Assessing the impact of real-time random safety audits through full propensity score matching on reliable data from the clinical information system. March 20, 2024

Balancing "no blame" with accountability in patient safety. October 7, 2009

Rethinking rapid response teams. September 29, 2010

Building a highway to quality health care. September 7, 2016

"Never events" and the quest to reduce preventable harm. June 3, 2015

A piece of my mind. From shame to guilt to love. January 6, 2016

Progress in patient safety: a glass fuller than it seems. August 28, 2013

Measuring preventable harm: helping science keep pace with policy. April 1, 2009

Diagnostic errors, health disparities, and artificial intelligence: a combination for health or harm. September 1, 2021

Forum: The 100,000 Lives Campaign: a scientific and policy review [with IHI response]. November 1, 2006

Shepherding change: how the market, healthcare providers, and public policy can deliver quality care for the 21st century. March 1, 2006

Identifying adverse events in patients hospitalized in isolation or quarantine due to COVID-19. May 10, 2023

Nursing peer review: developing a framework for patient safety. November 26, 2008

Higher incidence of adverse events in isolated patients compared with non-isolated patients: a cohort study. November 18, 2020

Mitigating the July effect. July 7, 2021

Intervention study for the reduction of medication errors in elderly trauma patients. September 2, 2020

Underreporting of patient safety incidents reduces health care's ability to quantify and accurately measure harm reduction. November 17, 2010

Viewing health care delivery as science: challenges, benefits, and policy implications. October 13, 2010

The need for systems integration in health care. March 9, 2011

Setting priorities for patient safety: ethics, accountability, and public engagement. September 2, 2009

Toward a safer health care system: the critical need to improve measurement. April 27, 2016

Patient safety and the problem of many hands. March 30, 2016

Physician motivation: listening to what pay-for-performance programs and quality improvement collaboratives are telling us. November 11, 2015

Organising a manuscript reporting quality improvement or patient safety research. August 21, 2013

Rapid response systems: should we still question their implementation? May 8, 2013

Science-based training in patient safety and quality. June 27, 2012

Improving healthcare quality through organisational peer-to-peer assessment: lessons from the nuclear power industry. June 6, 2012

Did hospital engagement networks actually improve care? September 3, 2014

Misdiagnosis in the emergency department: time for a system solution. February 8, 2023

Physician autonomy and informed decision making: finding the balance for patient safety and quality. January 14, 2009

The checklist--a tool for error management and performance improvement. November 29, 2006

Safety culture and the positive association of being a primary care training practice during COVID-19: the results of the multi-country European PRICOV-19 Study. November 16, 2022

Medication safety and the administration of intravenous vincristine: international survey of oncology pharmacists. February 26, 2014

Preventing patient harms through systems of care. August 22, 2012

Remote patient monitoring during COVID-19: an unexpected patient safety benefit. March 23, 2022

Diagnostic errors--The next frontier for patient safety. March 18, 2009

Medication reconciliation for patients after their discharge from intensive care unit to the hospital ward. July 19, 2023

The frequency of diagnostic errors in radiologic reports depends on the patient's age. October 13, 2010

Responsibility for quality improvement and patient safety: hospital board and medical staff leadership challenges. July 21, 2010

A public health approach to patient safety reporting systems is urgently needed. June 8, 2011

Fostering transparency in outcomes, quality, safety, and costs. October 12, 2016

Re-examining high reliability: actively organising for safety. May 18, 2016

Cardiac surgical ICU care: eliminating "preventable" complications. September 11, 2013

Strategies to improve patient safety: the evidence base matures. March 6, 2013

Time to accelerate integration of human factors and ergonomics in patient safety. January 11, 2012

Time for transparent standards in quality reporting by health care organizations. August 16, 2017

What to do with healthcare incident reporting systems. January 29, 2014

Health system leaders' role in addressing racism: time to prioritize eliminating health care disparities. February 10, 2021

The wisdom and justice of not paying for "preventable complications." May 14, 2008

Tracking progress in patient safety: an elusive target. August 16, 2006

Patient-reported safety incidents in older patients with long-term conditions: a large cross-sectional study. June 14, 2017

Hospital board checklist to improve culture and reduce central line–associated bloodstream infections. November 3, 2010

Why don't we know whether care is safe? April 10, 2013

A framework for encouraging patient engagement in medical decision making. August 29, 2012

Using a logic model to design and evaluate quality and patient safety improvement programs. July 25, 2012

From heroism to safe design: leveraging technology. February 12, 2014

Learning from defects to enhance morbidity and mortality conferences. March 25, 2009

Developing a measure of value in health care. June 1, 2016

Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. July 16, 2008

Rapid response teams—walk, don't run. October 11, 2006

Ethical and practical aspects of disclosing adverse events in the emergency department. October 4, 2006

Medicare payment for selected adverse events: building the business case for investing in patient safety. September 27, 2006

A framework for health care organizations to develop and evaluate a safety scorecard. November 7, 2007

Variations by state in physician disciplinary actions by US medical licensure boards. April 13, 2016

Surgery is in itself a risk factor for the patient. June 1, 2022

Feeling safe in the context of digitalization in healthcare: a scoping review. April 10, 2024

Establishing an ambulatory medicine quality and safety oversight structure: leveraging the fractal model. March 9, 2016

Fifteen years after To Err Is Human: a success story to learn from. January 13, 2016

Changing the narratives for patient safety. June 28, 2017

Data-driven implementation of alarm reduction interventions in a cardiovascular surgical ICU. February 8, 2017

Integrating CUSP and TRIP to improve patient safety. November 24, 2010

Patient-assisted incident reporting: including the patient in patient safety. June 1, 2011

Disclosing medical errors to patients: it's not what you say, it's what they hear. July 22, 2009

Impact of attending physician workload on patient care: a survey of hospitalists. February 6, 2013

Measurement as a performance driver: the case for a national measurement system to improve patient safety. April 26, 2017

On the CUSP: Stop BSI: evaluating the relationship between central line–associated bloodstream infection rate and patient safety climate profile. November 12, 2014

How safe is my intensive care unit? Methods for monitoring and measurement. November 21, 2007

The combined effect of psychological and social capital in registered nurses experiencing second victimization: a structural equation model. December 1, 2021

Multimorbidity and patient safety incidents in primary care: a systematic review and meta-analysis. October 14, 2015

The clinical safety of disabled patients: proposal for a methodology for analysis of health care risks and specific measures for improvement. March 27, 2013

Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. April 30, 2014

Predictive power of the "trigger tool" for the detection of adverse events in general surgery: a multicenter observational validation study. March 9, 2022

Gender biases and diagnostic delay in inflammatory bowel disease: multicenter observational study. February 15, 2023

Frequency of prescribing errors by medical residents in various training programs. June 10, 2015

Overview of progress on patient safety. January 12, 2011

Observational evidence of the prevalence and association of polypharmacy and drug administration errors in hospitalized adult patients. May 18, 2022

Annual Perspective

Patient Safety Primers

Medication Administration Errors

Nurse burnout predicts self-reported medication administration errors in acute care hospitals. January 20, 2021

Do falls and other safety issues occur more often during handovers when nurses are away from patients? Findings from a retrospective study design. November 11, 2020

Risk of medication errors and nurses' quality of sleep: a national cross-sectional web survey study. September 16, 2020

Factors associated with workarounds in barcode-assisted medication administration in hospitals. August 26, 2020

Differences between methods of detecting medication errors: a secondary analysis of medication administration errors using incident reports, the Global Trigger Tool method, and observations. June 24, 2020

WebM&M Cases

Reasons for drug administration problems and perceived needs for assistance of patients, family caregivers, and nurses: a qualitative study. June 17, 2020

Preventing medication errors at small and rural hospitals. May 20, 2020

The contribution of staffing to medication administration errors: a text mining analysis of incident report data. December 18, 2019

Effectiveness of double checking to reduce medication administration errors: a systematic review. September 18, 2019

Patient participation in patient safety—an exploration of promoting factors. February 13, 2019

IV push medications survey results—part 1 and part 2. November 28, 2018

Medication administration and interruptions in nursing homes: a qualitative observational study. July 11, 2018

Making an infusion error: the second victims of infusion therapy-related medication errors. May 30, 2018

Patient involvement for improved patient safety: a qualitative study of nurses' perceptions and experiences. November 22, 2017

Factors associated with barcode medication administration technology that contribute to patient safety: an integrative review. July 26, 2017

Impact of interruptions, distractions, and cognitive load on procedure failures and medication administration errors. July 19, 2017

Nursing skill mix in European hospitals: cross-sectional study of the association with mortality, patient ratings, and quality of care. June 28, 2017

Nurses' perceived skills and attitudes about updated safety concepts: impact on medication administration errors and practices. June 28, 2017

Teaching students to administer medications safely. April 19, 2017

Effectiveness of a 'Do not interrupt' bundled intervention to reduce interruptions during medication administration: a cluster randomised controlled feasibility study. March 15, 2017

Separate medication preparation rooms reduce interruptions and medication errors in the hospital setting: a prospective observational study. February 15, 2017

Medication double-checking procedures in clinical practice: a cross-sectional survey of oncology nurses' experiences. September 7, 2016

RN assessments of excellent quality of care and patient safety are associated with significantly lower odds of 30-day inpatient mortality: a national cross-sectional study of acute-care hospitals. July 20, 2016

Is technology the best medicine? Three practice theoretical perspectives on medication administration technologies in nursing. June 15, 2016

Transformational leadership in nursing and medication safety education: a discussion paper. June 1, 2016

The relationship between nursing experience and education and the occurrence of reported pediatric medication administration errors. April 27, 2016

Connect With Us

Sign up for Email Updates

To sign up for updates or to access your subscriber preferences, please enter your email address below.

Agency for Healthcare Research and Quality

5600 Fishers Lane Rockville, MD 20857 Telephone: (301) 427-1364

Accessibility
Disclaimers
Electronic Policies
HHS Digital Strategy
HHS Nondiscrimination Notice
Inspector General
Plain Writing Act
Privacy Policy
Viewers & Players
U.S. Department of Health & Human Services
The White House
Don't have an account? Sign up to PSNet

Submit Your Innovations

Please select your preferred way to submit an innovation.

Continue as a Guest

Track and save your innovation

in My Innovations

Edit your innovation as a draft

Continue Logged In

Please select your preferred way to submit an innovation. Note that even if you have an account, you can still choose to submit an innovation as a guest.

Continue logged in

New users to the psnet site.

Access to quizzes and start earning

CME, CEU, or Trainee Certification.

Get email alerts when new content

matching your topics of interest

in My Innovations.

A Promising HIP Intervention — Electronic Directly Observed Therapy for Active TB Disease

Prevention Case Study 2

TB is the leading cause of infectious disease death worldwide. The bacterium that causes TB, Mycobacterium tuberculosis , is spread when a person with TB disease of the lungs or throat coughs. In 2020, a total of 7,174 persons received a diagnosis for TB disease in the United States. 1 The majority of persons with TB disease can be treated by taking certain medications for ≥6 months. Effective case management supports patients’ adherence to medication and treatment completion.

Directly observed therapy (DOT) is a key element of TB case management. Endorsed by the World Health Organization and CDC, DOT entails a trained observer watching as the patient swallows the anti-TB medications. As the percentage of the population that owns a mobile phone, computer, or tablet with video capability has increased, potential for having the trained observer use an electronic device to enable DOT also has increase. 2,3 One form of electronic DOT, or eDOT, allows a member of the health care team to watch the patient swallow the anti-TB medications by using live or real-time video, or recorded video that is submitted for later review. Multiple pilot studies have documented that eDOT is acceptable to patients and feasible to implement; these studies also identified potential cost savings from reduced staff time and travel expenditures for either patients or medical staff. 4-15 These studies have also shown adherence to attending scheduled sessions and treatment completion with eDOT has been comparable with in-person DOT. For example, in a study based in New York City, 61 (16%) patients were on eDOT, and 329 (84%) were on in-person DOT. Adherence to scheduled eDOT sessions was 95%, compared with 91% for in-person DOT, and treatment completion was similar between the 2 groups (96% versus 97%, respectively). Among pilot studies that collected cost data, the amount of money saved by using eDOT ranged from $1,811 (2016 US$)/patient 10,16 to $14,355 (2016US$)/patient. 14,16 These studies selected participants for eDOT based on patients having demonstrated good adherence to initial doses of medication and that their TB cases were uncomplicated. Additional evaluations with large patient samples in different situations will add useful information about eDOT’s feasibility, effectiveness, and cost.

Selected Pilots and Small Case Studies with Cost Data Regarding eDOT

Suburban illinois pilot 14.

Summary: The DuPage County Health Department, Illinois, piloted an eDOT program from January 2013 to December 2014, enrolling 11 patients with active TB who did not have multidrug-resistant TB and who were considered likely to have high adherence (e.g., in stable housing). Using teleconferencing technology, staff connected with 9 patients using smartphones and 2 using their personal computers.

Results: Patients received 1,083 eDOT doses, with a 97% adherence rate (defined as the percentage of scheduled sessions the patient attended). Throughout the treatment course, eDOT patients saved an average of 9,499 travel miles. In total, the eDOT program saved 579 hours in staff time, representing $144,750 and $13,159 in travel miles/expenses (or $147,288 and $13,390, respectively, in 2016US$, or an average of $14,355/patient). 16

Urban Washington State/Seattle-Tacoma Metropolitan Area Pilot 15

Summary: During 2002–2006, two local health jurisdictions in Pierce and Snohomish Counties, Washington, conducted a retrospective chart review of 57 patients with TB disease who used videophones for medication monitoring.

Results: Medication adherence was documented 95.6% of the time. The average patient in the eDOT program saved $2,449 in travel expenses, compared with in-person DOT patients. 5 The savings in staff salaries and, benefits, and travel costs was $139,576 (or $169,441 and $2,973, respectively, in 2016US$). Staff saved 2,994 hours of travel time and 103,632 miles in travel.

Note: Monetary values were inflated from the year of publication to 2016US$ by using the Bureau of Economic Analysis, Personal Consumption Expenditure Index, Health Care component. 16

Recent Studies of eDOT

More recent studies have validated the findings of the initial pilot and case studies of eDOT. A randomized controlled trial to evaluate in-person DOT and eDOT for TB, funded by CDC in collaboration with the New York City (NYC) Department of Health and Mental Hygiene and Columbia University, demonstrated that when implemented by a program with a history of successfully providing in-person DOT, eDOT is noninferior to in-person DOT across multiple modes of statistical analysis. The study, which observed patients from July 2017 through October 2019, found that:

The percentage of completed doses with in-person DOT was 87.2% vs 89.8% with eDOT.
Issues affecting medication observations (such as operating software, schedule interferences, or unscheduled staff absences) occurred with both electronic and in-person DOT. Community-based in-person DOT observations had the greatest percentage of issues affecting the DOT session with 19% of observations having issues compared with 10% for live-video electronic DOT observations, 8% for recorded-video electronic DOT, and 6% for clinic-based in-person DOT.
After experiencing both in-person and eDOT, 84% of the participants reported they preferred to continue treatment with eDOT. 17

An economic evaluation that assessed health department and patient costs of eDOTand traditional in-person DOT in low-, medium-, and high-TB incidence settings in the United States was conducted between August 2017 and June 2018. Findings from this study indicated that patient costs were lowest for live video DOT ($1.01) and highest for clinic-based DOT ($34.53). The societal (health department + patient) costs of both video DOT types (live and recorded; $6.65 and $12.64, respectively) were less than both in-person DOT types (field-based and clinic-based; $21.40 and $46.11, respectively). Among the 4 different DOT types, both types of video DOT were associated with lower patient and health department costs when compared with traditional forms of DOT. 18 Notably, when the data were examined by setting, the cost of video DOT with recorded videos was significantly lower than cost of community-based DOT for both the medium incidence and high-incidence settings. However, the cost of video DOT with recorded videos was not statistically different from cost of community DOT in the low-incident setting. This finding demonstrates an economy of scale, in that when there are fewer patients to spread fixed costs, the cost per dose is higher.

An eDOT Resource

As of 2015, 42% (47/113) of TB programs surveyed used eDOT in some form, and 36% were planning to implement an eDOT program in the next year. 9 To support TB programs, CDC recently published a resource for programs that are considering initiating an eDOT program. The toolkit, “Implementing an Electronic Directly Observed Therapy (eDOT) Program: A Toolkit for Tuberculosis (TB) Programs” is designed to help TB programs review the potential benefits and drawbacks of an eDOT program, gain insights related to management and regulation concerns, and use available resources.

CDC. NCHHSTP AtlasPlus . Atlanta, GA: US Department of Health and Human Services, CDC; 2022.
World Health Organization (WHO). Digital health in the TB response: Scaling up the TB response through information and communication technologies . Geneva, Switzerland: WHO; 2014.
World Health Organization (WHO). Digital health for the end TB Strategy: an agenda for action . Geneva, Switzerland: WHO; 2015.
Chuck C, Robinson E, Macaraig M, Alexander M, Burzynski J. Enhancing management of tuberculosis treatment with video directly observed therapy in New York City. Int J Tuberc Lung Dis 2016;20:588–93.
Garfein RS, Collins K, Muñoz F, et al. Feasibility of tuberculosis treatment monitoring by video directly observed therapy: a binational pilot study. Int J Tuberc Lung Dis 2015;19:1057–64.
Gassanov MA, Feldman LJ, Sebastian A, Kraguljac MJ, Rea E, Yaffe B. The use of videophone for directly observed therapy for the treatment of tuberculosis. Can J Public Health 2013;104:e272.
Hoffman JA, Cunningham JR, Suleh AJ, et al. Mobile direct observation treatment for tuberculosis patients: a technical feasibility pilot using mobile phones in Nairobi, Kenya. Am J Prev Med 2010;39:78–80.
Holzschuh EL, Province S, Johnson K, et al. Use of video directly observed therapy for treatment of latent tuberculosis infection—Johnson County, Kansas, 2015. MMWR Morb Mortal Wkly Rep 2017;66:387–9.
Macaraig, M, Lobato MN, McGinnis Pilote K, Wegener D. A national survey on the use of electronic directly observed therapy for treatment of tuberculosis. J Public Health Manag Pract 2017. Epub ahead of print.
DeMaio J, Sharma D. Tuberculosis therapy and telemedicine. Expert Opin Pharmacother 2002;3:1283–8.
Molton JS, Pang Y, Wang Z, et al. Prospective single-arm interventional pilot study to assess a smartphone-based system for measuring and supporting adherence to medication. BMJ Open 2016;6:e014194.
Story A, Garfein RS, Hayward A, et al. Monitoring therapy compliance of tuberculosis patients by using video-enabled electronic devices. Emerg Infect Dis 2016;22:538–40.
Wade VA, Karnon J, Eliott JA, Hiller JE. Home videophones improve direct observation in tuberculosis treatment: a mixed methods evaluation. PLoS One 2012;7:e50155.
Mirsaeidi M, Farshidpour M, Banks-Tripp D, Hashmi S, Kujoth C, Schraufnagel D. Video directly observed therapy for treatment of tuberculosis is patient-oriented and cost-effective. Eur Respir J 2015;46:871–4.
Krueger K, Ruby D, Cooley P, et al. Videophone utilization as an alternative to directly observed therapy for tuberculosis. Int J Tuberc Lung Dis 2010;14:779–81.
Bureau of Economic Analysis. Consumer spending. Table 2.3.4U. In: Price Indexes for Personal Consumption Expenditures by Major Type of Product and by Major Function . Washington, DC: US Department of Commerce; 2017.
Burzynski J, Mangan JM, Lam CK, et al. In-Person vs Electronic Directly Observed Therapy for Tuberculosis Treatment Adherence: A Randomized Noninferiority Trial. JAMA Netw Open. 2022;5(1):e2144210. doi:10.1001/jamanetworkopen.2021.44210
Beeler Asay GR, Lam CK, Stewart B, Mangan JM, Romo L, Marks SM, Morris SB, Gummo CL, Keh CE, Hill AN, Thomas A, Macaraig M, St John K, J Ampie T, Chuck C, Burzynski J. Cost of Tuberculosis Therapy Directly Observed on Video for Health Departments and Patients in New York City; San Francisco, California; and Rhode Island (2017-2018). Am J Public Health. 2020 Nov;110(11):1696-1703. doi: 10.2105/AJPH.2020.305877. Epub 2020 Sep 17. PMID: 32941064; PMCID: PMC7542290.

Get Tested - Find a Testing Site Near You

To receive email updates about this page, enter your email address:

Adolescent and School Health
Sexually Transmitted Diseases
Tuberculosis
Viral Hepatitis

Exit Notification / Disclaimer Policy

The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
You will be subject to the destination website's privacy policy when you follow the link.
CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
Front Psychol

Indirect Observation in Everyday Contexts: Concepts and Methodological Guidelines within a Mixed Methods Framework

M. teresa anguera.

1 Faculty of Psychology, Institute of Neurosciences, University of Barcelona, Barcelona, Spain

Mariona Portell

2 Faculty of Psychology, Department of Psychobiology and Methodology of Health Sciences, Universitat Autònoma de Barcelona, Barcelona, Spain

Salvador Chacón-Moscoso

3 Facultad de Psicología, Universidad de Sevilla, Seville, Spain

4 Departamento de Psicología, Universidad Autónoma de Chile, Santiago, Chile

Susana Sanduvete-Chaves

Indirect observation is a recent concept in systematic observation. It largely involves analyzing textual material generated either indirectly from transcriptions of audio recordings of verbal behavior in natural settings (e.g., conversation, group discussions) or directly from narratives (e.g., letters of complaint, tweets, forum posts). It may also feature seemingly unobtrusive objects that can provide relevant insights into daily routines. All these materials constitute an extremely rich source of information for studying everyday life, and they are continuously growing with the burgeoning of new technologies for data recording, dissemination, and storage. Narratives are an excellent vehicle for studying everyday life, and quantitization is proposed as a means of integrating qualitative and quantitative elements. However, this analysis requires a structured system that enables researchers to analyze varying forms and sources of information objectively. In this paper, we present a methodological framework detailing the steps and decisions required to quantitatively analyze a set of data that was originally qualitative. We provide guidelines on study dimensions, text segmentation criteria, ad hoc observation instruments, data quality controls, and coding and preparation of text for quantitative analysis. The quality control stage is essential to ensure that the code matrices generated from the qualitative data are reliable. We provide examples of how an indirect observation study can produce data for quantitative analysis and also describe the different software tools available for the various stages of the process. The proposed method is framed within a specific mixed methods approach that involves collecting qualitative data and subsequently transforming these into matrices of codes (not frequencies) for quantitative analysis to detect underlying structures and behavioral patterns. The data collection and quality control procedures fully meet the requirement of flexibility and provide new perspectives on data integration in the study of biopsychosocial aspects in everyday contexts.

Introduction

Psychological science has shown a growing interest in the study of everyday life. New methodologies have been proposed for the within-person study of real-time biopsychosocial aspects in their natural settings (Bolger et al., 2003 ; Conner and Lehman, 2013 ; Reis, 2013 ; Portell et al., 2015b , c ). New technologies have made it possible to record spontaneous behavior—that is, behavior that is not elicited by a researcher but forms part of the subject's behavioral repertoire in his or her natural context (see e.g., Mehl et al., 2001 ). Compared with elicited behavior, spontaneous behavior has the advantage of occurring in a natural context and natural situation, so it is not influenced by extraneous variables such as a non-natural context or social desirability based on researchers' expectations. However, this area of study remains highly complex, particularly when it comes to obtaining quantitative indicators that make it possible to reconstruct the “who,” “what,” “how,” and “when” of events of interest and position these events in the individuals' ecological niche. The difficult task of obtaining quantitative indicators of spontaneous behavior in everyday contexts has been further complicated by the long-standing clash between the qualitative and quantitative paradigms in psychology. Mixed methods research (Johnson et al., 2007 ) has provided valuable resources for combining qualitative data derived from non-spontaneous behavior (e.g., questionnaire responses) and narrative data derived from natural settings (e.g., a life story). Nevertheless, the merging of qualitative and quantitative perspectives in cases where only spontaneous behavior is of interest has been little explored. In this article, we describe a mixed methods approach grounded in observational methodology (Anguera, 2003 ) that we believe fills this gap. The proposed approach combines the strengths and offsets the weaknesses of the qualitative and quantitative perspectives.

We present a methodological framework for studying everyday behavior using a rigorous scientific approach based on indirect observation that involves “liquefying” transcribed verbal material or texts from original settings. The process involves the quantification of qualitative data using techniques that are based on order or sequence of events rather than on traditional frequency measures. The approach proposed is perfectly compatible with any guiding theoretical framework whatsoever; this method is not linked to any concrete theoretical model, it offers numerous methodological opportunities, and it has the potential to lead to significant developments in the field of studying everyday behaviors. This approach differs from previous work in this area (Sandelowski, 2001 ; Sandelowski et al., 2009 ; Seltzer-Kelly et al., 2012 ; Bell et al., 2016 ) in that it analyzes the order and sequence of events. The parameters of frequency (which only indicates the number of occurrences), order (which also provides information about sequence), and duration (which, in addition to the aforementioned information, also indicates the time in conventional units) provide a higher degree of data consistency (Bakeman, 1978 ). The use of the order parameter, with the introduction of sequentiality, entails added value of extraordinary importance (Sackett, 1980 , 1987 ; Bakeman and Gottman, 1987 ; Magnusson, 1996 , 2000 , 2005 , 2016 ; Sánchez-Algarra and Anguera, 2013 ; Portell et al., 2015a ).

The presented liquefying method enables the systematic analysis of minor details that arise in a multitude of situations involving text (e.g., conversations, speeches, diary, or blog entries) with a level of granularity (Schegloff, 2000 ) that enables these “natural texts” to be analyzed in combination with other contextual data. The approach is applicable to both conventional and new forms of communication (e.g., WhatsApp messages), regardless of format or source. The source may be verbal behavior (informal conversations, focus group discussions, etc.) or documentary material (diaries, narratives, etc.), including in some cases graphic material, such as photographs and drawings.

Most of the solutions proposed to date for transforming text into quantitative data are either qualitative (e.g., ethnographic methods) or quantitative. Our proposal, however, takes a mixed methods approach in which spontaneously generated qualitative material is transformed into quantifiable code matrices.

In this article, we discuss key aspects of our proposed system. We analyze the concepts and meaning of systematic observation and one of its two branches, indirect observation, alongside key concepts of mixed methods research. We also look at types of qualitative data used in indirect observation and describe a methodological framework for building ad hoc observation instruments, creating matrices of codes for the data collected, and analyzing data and checking their reliability. Finally, we present a protocol specifically designed for indirect observation with examples from each of the stages in the process.

From systematic to indirect observation

Psychologists work in a wide range of fields and subfields that correspond to everyday life situations. To name just a few examples, they are involved in health education programs in nurseries and nursing homes, prosocial programs in primary schools, exercise programs for the elderly, social support programs in neighborhoods, or communities with families of multiple nationalities, AIDS prevention programs for adolescents, support programs for families with a history of child abuse or negligence or families of young car crash victims, relaxation programs for athletes, and social programs in prisons or juvenile correctional institutions. Systematic observation can make important contributions to the study of spontaneous behavior in a vast range of everyday contexts.

Observation is a useful method for collecting, processing, and analyzing information that cannot be studied in the artificial setting of a laboratory. It enables a largely unbiased analysis of everyday behaviors and interactions that occur naturally (Anguera, 2010 ). Although systematic observation dates back to the 1970s, it has taken on an identity of its own in the last two decades (Anguera, 1979 , 2003 ; Anguera and Izquierdo, 2006 ; Sánchez-Algarra and Anguera, 2013 ). It offers both flexibility and rigor as it is built on sound scientific principles, and this combination makes it ideal for use in many fields (Portell et al., 2015b ).

Systematic observation differs from ethnography in that its purpose is not to obtain a narrative account of subjective experiences in a process that requires the participation of the researcher or person being studied. Ethnographic studies require a qualitative approach, but unlike systematic observation, they do not require quantitative analysis and rigorous data quality control. Systematic observation, by contrast, is characterized by highly systematic data collection and analysis, stringent data quality controls, and the merging of qualitative and quantitative methods.

Systematic observation follows the four fundamental stages of scientific research: formulation of a research question, collection of data, analysis of data, and interpretation of results. The wealth of data collected in an observational study provides researchers with the opportunity to capture valuable chunks or snippets of everyday realities, without having to specifically ask for the information (there are no interviews, questionnaires, or psychological tests). In addition, it allows the researcher to study spontaneous behavior in a natural, uncontrolled environment.

Everyday activity in context is the cornerstone of observational studies. It is the source of a rich fabric of information that the psychologist/researcher needs to tap into in order to extract relevant information that is subsequently processed systematically to produce a set of “net” data that can be analyzed both qualitatively and quantitatively.

The study of everyday activity provides insights into the diverse behaviors and events that occur throughout a person's life. It provides thus a privileged vantage point from which to observe changes, but everyday life is a highly complex, dynamic process replete with information that is often not even known to exist (Anguera, 2001 ). Its study requires the examination of diverse phenomena at different levels of a pyramid-like structure. At the top of the pyramid, psychologists analyze how individuals go about their lives and gradually become familiar with what has shaped their life course. As they move down the pyramid, they discover everyday realities at different levels (family, career, social relationships, hobbies, etc.) and come to understand how these are influenced by interacting factors, such as health, satisfaction of needs, and conflicts.

According to Mucchielli ( 1974 ) observation equation, O = P+I+Pk−B, observation equals perception plus interpretation plus previous knowledge minus bias. Observation thus is not possible unless what is being observed is perceivable. Perceptibility is a key concept when it comes to differentiating between direct and indirect observation (Anguera, 1979 , 2003 ). In indirect observation, it is always incomplete, and Mucchielli's equation is only partially fulfilled.

In direct observation, perceptibility is considered to be complete when what is being observed (whether in situ or through video or audio recordings) can be captured by visual or auditory senses. In anthropology, for example, the subfield concerned with the study of visual representations is known as visual rather than observational anthropology. Modern-day technology permits maximum levels of precision in visual and auditory perception (Escalera et al., 2009 ; Bautista et al., 2015 ) and minimizes the need for interpretation.

Although everyday contexts can take countless shapes and forms, the levels of response (or criteria or dimensions) that can be directly observed are similar. Facial expressions, for example, can be analyzed by software such as Face Reader, which can distinguish between facial and emotional mimicry. Gestures, in turn, which also have an important role in human communication (Holle et al., 2012 ; Mashal et al., 2012 ), even in children (Lederer and Battaglia, 2015 ) can be effectively analyzed using programs such as NEUROGES+ELAN (Lausberg and Sloetjes, 2009 , 2016 ). Finally, vocal behavior (Russ et al., 2008 ) can be analyzed using sound analysis software. Non-verbal manifestations, or “expressiveness,” are interesting external indicators of a person's emotional state (Rodriguez et al., 2014 ), although adequate quality control is needed to reduce bias.

While aspects of human communication such as facial expressions, gestures, posture, and voice tone are fully perceivable through visual or auditory channels, they are frequently accompanied by verbal behavior, which has very different characteristics in terms of perceptibility. Indirect observation is an appropriate method for studying both verbal behavior and textual material, whether in the form of transcripts or original material produced by the participants in a study.

Verbal behavior transmits messages and both these and the channels through which they are transmitted can take many shapes and forms. Messages are analyzed differently depending on whether they are spoken or written. Written forms of expression (e.g., self-reports, diaries, biographies) are largely considered to be narratives. Narrative studies have been used in qualitative methodology for many years and have both strengths and shortcomings. One of their main strengths is their adaptability to very different situations and contexts. Narrative studies provide insights into a person's true nature and help to understand their experiences and needs (Riva et al., 2006 ). They have been used, for example, in a wide range of settings, such as secondary schools (García-Fariña, 2015 ; García-Fariña et al., 2016 ), high schools (Tronchoni et al., 2018 ), family gatherings (Gimeno et al., 2006 ), support groups for patients (Roustan et al., 2013 ), therapeutic interaction (Blanchet et al., 2005 ), and group therapy for adolescents (Arias-Pujol and Anguera, 2017 ). One of their shortcomings is that perceptibility is limited by the documentary nature of the texts, and it is not uncommon for different researchers to draw different conclusions from the same text.

Human communication does not simply refer to the transmission of information. It involves numerous aspects that vary according to content, the people transmitting or receiving the message, their relationship (hierarchy, previous interactions, etc.), the flow of data or metadata, and the interpretative context. In addition, changing lifestyle habits and new technologies have led to new forms of human communication (Bavelas and Chovil, 2000 ), such as WhatsApp messages and blog posts, extending the traditional dichotomy between verbal and non-verbal behavior established by the classical sociologist Weick ( 1968 , 1985 ). In a recent study, for example, Radzikowski et al. ( 2016 ) analyzed Twitter messages in a quantitative study on the rubella vaccine, and as stated by Hardley ( 2014 , p. 34), “Over many decades, surveillance methods (often termed “indicator based” methods) have been developed and refined to provide disciplined, standardized approaches to acquiring and recording important information. More recently, ubiquitous and unstandardized data collected from the Internet have been used to gain insight into emerging disease events.”

Indirect observation can be considered a valid scientific method (Webb et al., 1966 ; Anguera, 1991 , 2017 , in press ; Behar, 1993 ; Morales-Ortiz, 1999 ; Morales-Sánchez et al., 2014 ). It uses similar techniques to systematic observation, and as a procedure, it is structurally identical, although there are important differences dictated by the nature of the source data (verbal behavior and text).

Indirect observation involves the analysis of textual material generated either indirectly from transcriptions of audio recordings of verbal behavior in natural settings (e.g., conversation, group discussions) or directly from narratives (e.g., letters of complaint, tweets, forum posts). The addition of seemingly unobtrusive objects can also provide important insights into daily routines. All these materials constitute an extremely rich source of information for studying everyday life, and they are continuously growing with the burgeoning of new technologies for data recording, dissemination, and storage (Morales-Ortiz, 1999 ; Morales-Sánchez et al., 2014 ).

Narratives are an excellent vehicle for studying everyday life through indirect observation, and one option for studying them is to apply a procedure for systematizing and structuring the information through quantitization . This approach makes it possible to integrate qualitative and quantitative elements.

The data used in indirect observation invariably start out as qualitative and the source material varies according to the level of participation of the person being observed and the nature of the source (textual or non-textual).

Common sources of material used in indirect observation studies include:

Recordings of verbal behavior as it occurs (normally in mp3 files). There may be single or multiple dialogues and it is essential to clearly distinguish between the different “voices” recorded.
Transcripts of audio recordings of verbal behavior in a natural setting (Krueger and Casey, 2009 ). These may involve an individual (speaking, for example, in person or on the telephone), or a group (dyad, triad, focus group, etc.), in which each person can be clearly identified.
Written texts produced by the participants in a research study. These include texts produced by the participants or those close to them (e.g., letters of complaint, letters to a newspaper, tweets, ads, messages on a mural, instant text messages). A variety of communication channels are possible (e.g., paper, e-mail, WhatsApp).
Texts transmitted through the Internet, such as e-mails (Björk et al., 2014 ) and forum posts (Vaimberg, 2010 ). These constitute an extremely rich source of information and are particularly relevant to psychological interventions.
Everyday objects related to the research question(s). While objects may appear to have a secondary role in communication, they can provide relevant insights into everyday life as they evoke or facilitate the expression of emotions through micro-valences (Lebrecht et al., 2012 ). Examples are graphs, paintings, models, and clay figures. Technological advances have also opened up new opportunities in this area in recent years.
Graphic material, particularly photographs. These can constitute an extremely rich source of information (Zaros, 2016 ). A single photograph captures a moment, something static, but a gallery of photographs separated in time can capture the dynamics of an episode or successive episodes in the life of a person, or even a group or institution. This material can be primary (the only source available) or secondary (complementing other sources).
Unobtrusive objects, also referred to as aggregates (Webb et al., 1966 ). These may simply be anecdotal, but in some cases they can reveal the existence of certain behaviors, but only after a process of inference involving variable risk. Examples are fingerprints and objects such as cigarette butts or a napkin with notes or drawings left behind in a café.

The above sources of information give rise to a varied set of data that provides empirical evidence and can position specific events and everyday behaviors along a continuum of time. Finally, the information available becomes progressively richer as one gains access to several sources of documentary material.

As mentioned, the material used to collect data in indirect observation is only partly perceivable (Anguera, 1991 ) and any conclusions made need to be inferred by a researcher drawing from a theoretical framework or taking a position. This is the main challenge in indirect observation. In the system we propose, rigorous application of a carefully designed observation instrument by duly trained observers offers the necessary guarantees of data reliability. Although direct and indirect observation may vary in terms of source material, level of interpretation, and level of participation, the two methods share a scientific procedure that when properly applied can provide quantitative indicators of the processes underlying everyday behavior.

The challenges of mixed methods research

Mixed methods research has been increasingly embraced by the scientific community over the past 15 years (Creswell et al., 2003 ; Johnson et al., 2007 ; Tashakkori and Teddlie, 2010 ; Onwuegbuzie and Hitchcock, 2015 ). The mixed methods approach involves the collection, analysis, and interpretation of qualitative and quantitative data for the same purpose and within the framework of the same study; some authors have even raised the approach to the rank of paradigm. Molina-Azorín and Cameron ( 2015 ) acknowledge that mixed methods research is not easy to conduct and requires considerable time and resources. Nonetheless, it is a movement that is gradually gaining supporters. As stated by Leech and Onwuegbuzie ( 2009 ) and Onwuegbuzie ( 2003 ), mixed methods research lies on a continuum between single-method and fully mixed studies, although the scientific community has yet to agree on which position it holds along this continuum. That said, it is generally agreed that the position will depend on the research objective and the nature of the data, analyses, and level of inference.

Overall, mixed research is largely understood as “a synthesis that includes ideas from qualitative and quantitative research” (Johnson et al., 2007 , p. 113). However, this is a very broad framework in which many gaps need to be filled. In the case of indirect observation, the methodological approach must be extremely rigorous as we are dealing with situations in which substantive areas merge with the multiple realities of everyday life.

The exponential growth of mixed methods research in recent decades has generated certain inconsistencies in terms of terminology and definitions. We therefore believe that it is first necessary to clarify the meaning of method/methodology and to discuss the multiple meanings attached to the term “mixed method” before we present our methodological framework for indirect observation.

Greene ( 2006 , p. 93) proposed a broad description of the term “methodology,” understood as an inquiry logic that admits different forms of data collection (questionnaires, interviews, observational datasets, etc.), methods of research (experimental, ethnographic, etc.), and related philosophical issues (ontology, epistemology, axiology, etc.). Greene also refers to specific guidelines for practice, which distinguish between methods that obviously vary in terms of design, sampling, data gathering, analysis, etc. We consider that systematic observation fits with Greene's definition of methodology (Anguera, 2003 ), although we have not always used the term. We also agree with the following statement by Johnson et al. ( 2007 , p. 118): “It is important to keep in one's mind, however, that the word methods should be viewed broadly.” Accordingly, in the approach we describe in this article, we also consider indirect observation to be a method in the broad sense of the word.

Johnson et al. ( 2007 , p. 123) defined mixed methods research as “the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration” (Johnson et al., 2007 , p. 123). They formulated this definition after asking 19 renowned researchers in the field (Pat Bazeley, Valerie Caracelli, Huey Chen, John Creswell, Steve Currall, Marvin Formosa, Jennifer Greene, Al Hunter, Burke Johnsson and Anthony Onwuegbuzie, Udo Kelle, Donna Mertens, Steven Miller, Janice Morse, Isadore Newman, Michael Q. Patton, Hallie Preskill, Margarete Sandelowski, Lyn Shulha, Abbas Tashakkori, and Charles Teddlie) to send in their definition of the term “mixed methods” by e-mail.

We fully agree with the definition proposed by Johnson et al. ( 2007 ) and it provided us with the necessary elements to draw up our methodological framework for indirect observation. The success of any mixed methods approach depends on the adequate mixing or integration of qualitative and quantitative elements. Numerous authors have analyzed the term “mixing” in an attempt to provide guidance on the processes required to achieve a seamless result (Bazeley, 2009 ; O'Cathain et al., 2010 ; Fetters and Freshwater, 2015 ). Qualitative and quantitative data can be mixed in three different ways, aptly summed up by Creswell and Plano Clark ( 2007 , p. 7): “There are three ways in which mixing occurs: merging or converging the two datasets by actually bringing them together, connecting the two datasets by having one build on the other, or embedding one data set within the other so that one type of data provides a supportive role for the other data set.” For our proposal, we chose the second form: connecting two databases by having one build on the other. According to Sandelowski et al. ( 2009 ), this connection can be achieved through transformation, i.e., by quantitizing qualitative data or qualitizing quantitative data. In our indirect observation framework, we transform non-systematic qualitative data into a format suitable for quantitative analysis.

Mixed methods research is marked by a persistent scientific gap that requires powerful solutions rooted in two key challenges in the field of indirect observation. These two challenges, discussed in this article, are (a) how to rigorously transform qualitative textual material derived largely from everyday human communication into matrices of codes, and (b) how to subsequently analyze these codes using quantitative methods suited to the categorical nature of the data in order to uncover the underlying structure. The proposed transformation system breaks away from the classical theoretical framework of mixed methods, which simply involves integrating qualitative and quantitative elements. The key difference is that it contemplates systematic observation, and hence indirect observation, to be a mixed method in itself (Anguera and Hernández-Mendo, 2016 ; Anguera et al., 2017a ).

Integration of qualitative and quantitative elements is the key to any mixed methods approach (Creswell and Plano Clark, 2007 ; Bazeley, 2009 ; O'Cathain et al., 2010 ; Maxwell et al., 2015 ). Our approach adds another element: the liquefaction of verbal behavior and texts. This process consists of schematically transforming “solid” textual material into “liquid” matrices of codes apt for quantitative analysis (Anguera et al., 2017b ; Anguera, in press ). The quantitative processing of originally qualitative data with the aim of detecting hidden behavioral patterns or underlying structures, for example, adds an element of robustness to the integration of qualitative and quantitative data, particularly in the case of everyday life events and behaviors.

Talkativeness and text, for example, can now be analyzed within the framework of mixed methods research using frequency counts (Poitras et al., 2015 ) thanks to the development of reliable—and extremely useful—measures of verbal productivity and the multiple opportunities offered by modern-day technology (Bazeley, 2003 , 2006 , 2009 ). Frequency counts, however, are weak and insufficient measures. Considering that “methodological plenitude” (Love, 2006 , p. 455) is not always attainable in applied research, the mixed method framework offers new and interesting possibilities for indirect observation.

The combined use of qualitative and quantitative approaches has been tried and tested in multiple studies and has also been analyzed in several systematic reviews (Elvish et al., 2013 ). In the following sections, we show that it is necessary to start with qualitative inputs and to then quantify these in a process that ensures reliability throughout the various stages.

Qualitative datasets in indirect observation

The empirical process in indirect observation starts with the collection of qualitative data. While the characteristics and standards that guarantee quality are perfectly outlined in the literature on quantitative methodology, the same cannot be said of qualitative methodology. Qualitative methodology offers enormous flexibility, but interpretations on content and form vary and are not free of controversy. Content provides personal and interpersonal information, which stems from experiences that are temporally unstable and highly influenced by the context and versatility of the moment. As for form, the tools used to support indirect observation (narratives, biographies, self-reports, life stories, in-depth interviews, etc.) cause doubt and distrust in many researchers, who, in the absence of standardized tools, question their stability and consistency.

Much has been written about the forms used to structure narratives (e.g., Hurwitz et al., 2004 ; De Fina and Georgakopoulo, 2015 ; Riessman, 2015 ), and qualitative data can be gathered using many tools, including interviews (e.g., Riera et al., 2015 ), biographies (e.g., Lindqvist et al., 2014 ), children's vignettes (e.g., Jackson et al., 2015 ), focus group vignettes (e.g., Brondani et al., 2008 ), telephone interviews (e.g., Björk et al., 2014 ), self-reports (e.g., Coutinho et al., 2014 ), focus group recordings (e.g., McLean et al., 2011 ), and participant observation (e.g., Caddick et al., 2015 ). In our case we are specifically interested in qualitative datasets within the framework of indirect observation. Although systematic observation dates back to the 1970s, it has taken on an identity of its own in the last two decades (Anguera, 2003 ; Anguera and Izquierdo, 2006 ; Sánchez-Algarra and Anguera, 2013 ; Anguera et al., 2017a ). Indirect observation shares many of the characteristics previously described for systematic observation, namely, highly systematic data collection and analysis, strict data quality controls, and an approach that requires the merging of qualitative and quantitative techniques.

A methodological framework for liquefying text

In these next sections, we are going to describe, and illustrate with examples, the stages and sub-stages involved in an indirect observation study. We will focus largely on the extraction and transformation of information from textual material produced using conventional or newer channels of communication in a variety of formats (handwritten letters, reports, transcriptions of group meetings, and interviews, etc.), irrespective of origin (e.g., informal conversations or focus group discussions or documentary material).

Extracting information on human behavior from text and transforming it into suitably systematized and organized categorical data, without loss of key information, is a major challenge in the Behavioral Sciences. In addition, the process must offer sufficient scientific and ethical guarantees and produce results in a format that can be rigorously processed using any of a range of quantitative techniques available for analyzing categorical data.

Our text-liquefying process consists of six stages: (1) specification of study dimensions, (2) establishment of segmentation criteria to divide the text into meaningful units, (3) building of a purpose-designed observation instrument, (4) coding of information, (5) data quality control, and (6) quantitative analysis of data. Table 9 presents detailed steps and guidelines for the “liquefication” of indirect observations. Each of the steps will be explicated within the following sections.

Specification of study dimensions

In systematic observation, and by extension, indirect observation, the term “dimension,” also known as level of response (Weick, 1968 ) or criterion, refers to a distinguishable facet related to the research objective. Dimensions are generally derived from a theoretical framework (e.g., the seminal work of (Weick, 1985 ) in the field of social interaction), but they can also be created ex novo based on experience or expertise. In the latter case, they must always be justified.

Studies can be one-dimensional or multidimensional. It is not uncommon for researchers to start off with a single dimension and then gradually add others as they delve deeper into the theoretical framework. Below are examples of dimensions and theoretical frameworks used in three indirect observation studies. In the first case, a study of disruptive behavior and communication difficulties in adolescents participating in group communication therapy, Arias-Pujol and Anguera ( 2004 ) proposed the dimensions verbal and non-verbal behavior, derived from the corresponding interpersonal theoretical framework (Danzinger, 1982 ; Gale, 1991 ; Poyatos, 1993 ). In the second case, Vaimberg ( 2010 ), on studying a psychotherapy group in which participants were able to write what they wanted on an online forum at any time over 3 years, chose the following dimensions: in-person, otherness, emotionality, thoughtfulness, positivity, and realism. The theoretical framework was built from work by various authors (e.g., Winniccott, 1979 ; Bion, 1985 ; McDougall, 1991 ; Lévy, 1995 ). In the third case, which was a recent study of teacher-led discourse in physical education built on the theoretical framework of the Teaching Games for Understanding model (originally proposed by Bunker and Thorpe, 1982 ) and work on discourse strategies by Coll and Onrubia ( 2001 ), García-Fariña et al. ( 2016 ) proposed nine dimensions: exploration and activation of previous knowledge, attribution of positive meaning by students, progressive establishment of increasingly expert and complex representations of subject matter, interactivity segment, message structure, extralinguistic resources, task type, destination of message, and location of session.

Specification of segmentation criteria to create textual units

The second step toward liquefying a text is to define the segmentation criteria to divide the text into meaningful units. This process is known as “unitizing.” Although initially proposed by Dickman ( 1963 ) and Birdwhistell ( 1970 ), Krippendorff ( 2013 , p. 84) defined unitizing as “the systematic distinctions with a continuum of otherwise undifferentiated text—documents, images, voices, websites, and other observables—that are of interest to an analysis, omitting irrelevant matter but keeping together what cannot be divided without loss of meaning.” This definition suggests that it would be logical to first segment the text into primary criteria within the main study dimension and then establish secondary criteria for the other dimensions (e.g., voices, gestural behavior, etc.).

Krippendorff ( 2013 ) suggested segmenting text using orthographic, syntactic, contextual, and inter-speaker criteria. In this last case, each intervention by an individual is considered a unit. This is a very useful approach for analyzing interactions between various people. We propose using the inter-speaker criterion as the primary criterion and subsequently establishing secondary criteria (subunits) for verbal or written interventions containing various syntactic elements (phrases).

In cases with several dimensions, such as verbal behavior accompanied by gestures, postures, or exchange of looks, verbal behavior, as the most perceivable behavior, could be established as the primary criterion. The other behaviors could then be segmented into subunits as appropriate. In very specialized cases, however, we consider that the above level of segmentation is insufficient. The initial segmentation stage is crucial as the categories that will be created in the next stage will directly determine the content of the dataset for analysis. Where possible, test runs or pilot studies should be performed first. Table Table1 1 shows how a conversation between two anonymous speakers is segmented into units.

Vignette showing the segmentation of a text (transcribed from a conversation) into units.

Building an indirect observation instrument

Indirect observation studies, like systematic observation studies (Anguera, 2003 ; Anguera and Izquierdo, 2006 ; Sánchez-Algarra and Anguera, 2013 ; Portell et al., 2015a ) require a purpose-built observation instrument to systematically code the information that will form the subsequent datasets.

Observation instruments can be built using category systems, a field format system, a combination of these systems, or rating scales (Anguera et al., 2007 ). One-dimensional studies use category systems and rating scales, while multidimensional studies use field formats or field formats combined with category systems. To build a category system, there must be a theoretical framework, and to build a rating scale, it must be possible to grade the corresponding dimensions ordinally. In addition, the category system must fulfill the requirements of exhaustivity and mutual exclusion, and each category must be accurately defined.

The field format is built by creating a catalog of mutually exclusive behaviors for each dimension. As it is not exhaustive, the catalog is left open and is therefore considered to be in a permanent state of construction. While not required, a theoretical framework is recommendable for field format systems.

Observation instruments combining a field format system with category systems are becoming increasingly common. This combination is possible when some or all of the dimensions in the field format have a theoretical framework and the object of research is atemporal (i.e., it is not a process).

To simplify matters, it is highly recommendable to code both categories and dimensions using letters, numbers, or symbols. If A, B, C, and D are categories in a category system, i.e., fulfilling the requirements of exhaustivity and mutual exclusion (e.g., A = XX, B = XX, C = XX, and D = XX), then the notation would be CS (category system) = {A B C D}. If A, B, C, and D are behaviors in an open catalog, i.e., they are mutually exclusive but not exhaustive (e.g., A = XX, B = XX, C = XX, and D = XX), the notation would be Catalogue = A B C D…

Guidelines for coding information

Observational datasets created from narratives (Crawford, 1992 ; Gabriel, 2004 ; Tuttas, 2015 ) have wide applications in many everyday life situations. However, before qualitative inputs from human communication can be transformed into quantitative data, it is first necessary to decide how to organize the heterogeneous information available. This process can be extremely complex as it is necessary to bring together data from very different sources, and very possibly, different points in time (Duran et al., 2007 ). The first step is to correctly record and code the data, and this is where the ad hoc observation instrument becomes invaluable. As started by Bradley et al. ( 2007 , p. 1,761), “coding provides the analyst with a formal system to organize the data, uncovering and documenting additional links within and between concepts and experiences described in the data.”

If the sources have been carefully selected, they will all contribute to creating a stockpile of information on the behaviors or actions of all those involved in the communication process being analyzed (e.g., therapists, participants, supervisors…).

The system for processing narratives or bodies of texts is quite similar to that used in discourse analysis (Calsamiglia and Tusón, 1999 ), although the information retrieved is richer and more diverse. Once the necessary quality controls are in place, the information can be managed and processed systematically within an empirical research setting that ensures replicability. Examples of texts used for this purpose are interviews, speeches, and conversations (Sidnell and Stivers, 2013 ). These may be a specific audience, a single speaker or several (with turn-taking), words in isolation, or, when direct and indirect observation are combined, words accompanied by tone/pitch, gestures, facial expressions, posture, objects, etc (Fischer et al., 2012 ).

Once the study dimensions have been selected (section Specification of Study Dimensions) and the text has been segmented into units (section Specification of Segmentation Criteria to Create Textual Units) and the behaviors coded using the ad hoc observation instrument (section Building an Indirect Observation Instrument), the data can be transformed into a series of complete or incomplete code matrices containing purely qualitative information (Anguera, 2017 , in press ; Anguera et al., 2017b ). This transformation is achieved by organizing the dimensions into columns and adding the behavioral units to the corresponding rows, achieving thus a “liquid” text, ready for quantitative analysis (Table (Table2 2 contains an example).

Tabular structure for creating a code matrix.

Each row of the matrix contains a series of boxes that are completed with codes corresponding to each textual unit (fragments of text from indirect observation). The columns, in turn, contain the different dimensions, or criteria, of the observation instrument. The codes come from the ad hoc observation instrument and may correspond to behaviors from a field format catalog or to categories in an observation instrument based on category systems only or on category systems combined with a field format. By way of illustration, we have added in brackets the first two dimensions (verbal behavior and vocal behavior), a simulation of the first units and an indication of the behaviors produced (which will be coded) .

Table Table3a 3a shows a hypothetical example of data extracted from a text in a one-dimensional study using an observation instrument with category systems, using a simulated example of the diary of a patient with endogenous depression. Table Table3b, 3b , in turn, shows the results for a combined field format-category system instrument from a multidimensional study, using a simulated example of an oral mediation situation involving a conflict between the parties A and B, with the assistance of the mediator C. These matrices of codes (Table (Table3a 3a is atypical as it has just one column due to the single dimension analyzed) show how the qualitative data have been structured.

(a,b) Hypothetical examples of a code matrix derived from a text.

E = {E1 E2 E3 E4 E5}

F = F1 F2 F3 F4 F5 F6 F7 F8 …
G = {G1 G2 G3}
H = H1 H2 H3 H4 …

E1: Facilitating elements (greeting, courtesy routines, etc.)

E2: Focused on the crux of the issue

E3: Related to secondary aspects

E4: Neutral sentences not related to the conflict

E5: Conflictive elements (insults, mockery, etc.)

[This is an exhaustive and mutually exclusive system of categories]

F1: Shouting

F2: Speaking in an annoyed tone

F3: Speaking loudly

F4: Speaking while crying

F5: Speaking normally

F6: Speaking softly

F7: Whispering

F8: Silence

[This is a catalog of behaviors; as such, it is an open list and additional codes can be added]

G1: Party A

G2: Party B

G3: Mediator

H1: Shaking head to indicate “no”

H2: H1 plus hands clasped

H3: H2 plus bulging eyes

H4: H3 plus clenched jaw

The columns correspond to the dimensions and the rows to the units into which they were segmented. Codes on the same row reflect concurrent behaviors. The codes are defined in the ad hoc instrument designed for the study .

Additional sources of information, such as drawings, sounds, or photographs can be incorporated simply by adding new dimensions. Although this is still a relatively new concept, it is perfectly feasible with today's advanced coding systems (Saldaña, 2013 ) and technological possibilities (e.g., Bazeley, 2003 , 2006 , 2009 ; Crutcher, 2003 , 2007 ; Holtgraves and Han, 2007 ; Romero et al., 2007 ; Dam and Kaufmann, 2008 ; Taylor et al., 2015 ). In the ATLAS.ti (v.7) qualitative data analysis program, for example, the text coding feature can be used to supplement the information entered with an object or an audio or video recording.

Researchers now have access to a multitude of software programs that facilitate their work. For those working with indirect observation, the CAQDAS platform (AQUAD6, ATLAS.ti, MAXqda2, NUDIST, NVivo, etc.) offers numerous programs for segmenting and coding text, and there are also open-access programs, such as T-LAB ( http://tlab.it/en/presentation.php ), IRAMUTEQ ( www.iramuteq.org ), and those created by the Italian group GIAT ( www.giat.org ). Numerous considerations are necessary when extracting information from text using content analysis techniques. Content analysis programs have traditionally favored the processing of large, mostly qualitative, bodies of texts, graphs, and audio and video material. The analysis uncovers relational structures (families, networks, etc.) that are relatively stable, or at least appear to be, and are always determined by the choices of the researcher. Nowadays, however, powerful software programs can analyze multiple sources of information to produce code matrices (Vaimberg, 2010 ) that are of enormous value for analyzing human communication in many fields.

Two programs can be used for both direct and indirect observation. These are HOISAN (Hernández-Mendo et al., 2012 ) ( http://www.menpas.com ), which is open-access and is available in several languages (English, Spanish, Portuguese, French) that can be selected from the tab Archivos (Files) , and TRANSANA ( http://www.transana.com ).

Quantitative processing of code matrices

Rigorous data quality control.

The issue of data quality in indirect observation has been widely debated in the literature, with a particular focus on reliability and validity, and concerns have led many psychologists and researchers working in this area to modify their approaches. Both intraobserver and interobserver agreement are important measures of reliability, but they are not the only ones. While reliability is necessary, it alone does not guarantee the validity of a dataset (Krippendorff, 2013 ).

Krippendorff ( 2013 ) was the first author to insist on rigorous data quality control as a requirement for the quantification of data resulting from indirect observation. Thanks to his contributions in this area, there are now methodological tools in place to demonstrate the quality of such data. The two main quantitative measures for testing the reliability of data from direct observation (behaviors) and indirect observation (texts) are (a) coefficients of agreement between two observers who separately code behaviors using the same dataset and observation instrument and (b) coefficients of agreement based on correlation. Numerous coefficients exist for quantitatively verifying the quality of data in a wide range of situations. One widely used measure in indirect observation is Krippendorff's canonical agreement coefficient, which is an adaptation of Cohen's kappa coefficient for analyzing three or more datasets. It can be calculated in HOISAN. Another option for use in situations with different sources of variation is generalizability theory (Blanco-Villaseñor, 2001 ; Escolano-Pérez et al., 2017 ).

A more qualitative method, the consensus agreement method (Anguera, 1990 ), is gaining increasing recognition in indirect observation and other studies. In this method, at least three observers work together to discuss and agree on the most suitable code for each unit from the observation unit. This method has obvious advantages, as it produces a single dataset and frequently results in a better observation instrument thanks to the detection of possible gaps and shortcomings. While it offers significant guarantees of quality, however, it also carries risks. An observer may defer to the decisions of a more senior or “expert” colleague, for example, and the need to agree can also give rise to frictions or conflicts. The results of the consensus agreement method can be complemented by quantitative measures of agreement (Arana et al., 2016 ).

There has been much debate in the field of psychology about the extent to which adherence to a particular theoretical framework may influence agreement between observers. To overcome this potential problem, Pope et al. ( 2000 ) proposed using observers from different backgrounds to analyze the data. Such an approach, however, would require even more rigorous quality control measures given the greater difficulty of reaching agreement.

Table Table4 4 shows the canonical agreement coefficient calculated in HOISAN for the data in Table Table3b, 3b , combined with two other sets of data recorded for the same section of text by the same observer and with the same instrument, but at different moments.

Example of datasets used to calculate intraobserver canonical agreement.

In such cases, the same verbal behavior or textual material must be coded by the same observer, using the same indirect observation instrument, on three separate occasions, separated by at least a week. The data in the first column are from Table Table3b 3b .

Once the text has been liquefied and the necessary data controls performed, the researcher now has access to a series of code matrices perfectly suited for analysis using different techniques.

The novel nature of our proposal is that we do not study frequency counts, which, despite their serious limitations, were the only measure of quantification used in observation studies for decades.

Over the last 15 years, our group has prioritized three analytical techniques that are particularly well-suited to processing qualitative data in both systematic observation (Blanco-Villaseñor et al., 2003 ) and indirect observation studies. These are lag sequential analysis, polar coordinate analysis, and T-pattern detection. All three techniques are based on statistical calculations and therefore provide the necessary guarantees of replicability and robustness.

Lag sequential analysis

Lag sequential analysis, which works with code matrices (see example in Table Table5), 5 ), is used to detect behavioral patterns that show the structure of interactive episodes (Bakeman, 1978 , 1991 ; Bakeman and Gottman, 1987 ; Bakeman and Quera, 1996 , 2011 ). The analysis can be performed prospectively (looking forward in time from a given moment) or retrospectively (looking backwards) using positive or negative lag counts. A behavior, for example, with a lag count of +2 would correspond to a behavior that occurs 2 positions after the behavior(s) of interest, while one with a lag count of −2 would correspond to a behavior that occurs 2 positions before the behavior(s) of interest.

(a) The first row shows the simple frequency counts for the data from Table Table3a. 3a . The matrix below shows the transition frequencies for the given behavior A with the conditional behaviors shown at the head of each column. The different lags are shown by rows. (b) The first row shows the unconditional probabilities while the rows below show the conditional probabilities.

Bold values are significative (upper that respective unconditional probabilities) .

The analysis can be applied to part of a session, to a complete session, to parts of different sessions (e.g., the first few minutes of a series of sessions), or to series of complete sessions. The technique thus offers enormous flexibility in terms of addressing different research questions. Two types of data can be used: data for which only the order of occurrence of concurrent behaviors has been recorded, using any of the free software programs available SDIS-GSEQ v. 4.1.2 (Bakeman and Quera, 2011 ), GSEQ5 (Bakeman and Quera, 2011 ), or HOISAN v. 1.6.3.3 (Hernández-Mendo et al., 2012 ), and data for which both order and duration have been recorded (SDIS-QSEQ and GSEQ5). Lag sequential analysis has been successfully applied in many indirect observation studies conducted over the past 25 years (e.g., Martínez del Pozo, 1993 ; Arias-Pujol and Anguera, 2004 ; Cuervo, 2014 ).

Using the data from Table Table3a 3a again, we illustrate how to manually calculate the results for the first, and simple, part of the lag sequential analysis process. The first step is to create tables for the matching frequencies and probabilities (Tables 5a,b ) for category A (in our example, expressions of sorrow or sadness), which, according to the hypothesis applied, is the given behavior (the behavior of interest). In row 1, for example, A has a frequency count of 0 because this code does not occur again; B (expressions of self-perceived improvement) has a count of 5 because it occurs after A on five occasions (units 2, 7, 12, 15, and 18); C (expressions of self-perceived worsening) has a count of 1 because it only occurs after A on one occasion (unit 5), similarly to D (expressions of joy at having overcome the problem) (unit 10). In row 2, in turn, A has a count of 2 because it occurs on two occasions (units 6 and 11) in the second position after the given behaviors (units 4 and 9, respectively); B has a count of 0 because it does not occur in the second position after the given behavior; and C has a count of 1 because it occurs just once (unit 13) in the second position after the given behavior (unit 11), and so on.

The data are analyzed to search for behavioral patterns, with consideration of some or all of the other behaviors, known as target behaviors, to see if they form part of the pattern(s) detected.

The information for each of the categories is shown on a graph with the lags on the X-axis and the probability values (ranging between 0 and 1) on the Y-axis. Each of the four Figures 1A–D , shows the value of the unconditional probability (the line parallel to the Y-axis) and the points corresponding to the conditional probability of each lag.

An external file that holds a picture, illustration, etc.
Object name is fpsyg-09-00013-g0001.jpg

(A–D) The lags are shown on the X-axis and the probabilities on the Y-axis. Based on the results from Table Table5b, 5b , the values corresponding to the unconditional probabilities (first row) are indicated by the horizontal line parallel to the X-axis (e.g., 0.35 for category A). Also shown are the values for each of the conditional probabilities for each category and lag. These values are linked by a (generally uneven) line for each category. The horizontal line parallel to the X-axis represents the upper limit for the effect of chance. Accordingly, any conditional probabilities in the subsequent lags that are higher than the unconditional probability for the corresponding category are significant and hence form part of the behavioral pattern.

Based on this simple visual output and considering all the statistically significant categories at each lag (i.e., the categories with a conditional probability value greater than that of the unconditional probability), we extracted the behavioral pattern shown in Figure Figure2. 2 . The strength of patterns is assessed using interpretative rules (Bakeman and Gottman, 1987 ). In the example provided, the first lag that is followed by another lag containing significant categories is considered to be the last lag (max lag) in the pattern (lag 3 in the example).

An external file that holds a picture, illustration, etc.
Object name is fpsyg-09-00013-g0002.jpg

Behavioral pattern extracted after assigning significant conditional behaviors (behaviors with a conditional probability greater than the unconditional probability) to each lag. The behavior pattern extracted from the presented illustration exhibits a regularity consisting of expressions of sorrow or sadness being followed by expressions of self-perceived improvement and these expressions, in turn, being followed by joy at having overcome the problem. From there, the pattern bifurcates, leading either to the initial situation of sorrow and sadness or to expressions of self-perceived worsening.

The robustness of the pattern must then be further strengthened by building a confidence interval around the conditional probabilities, for which only the upper limit is needed. This upper limit is used to determine whether a given category will form part of the pattern at the lag being analyzed, as the conditional probability obtained has to be higher than unconditional probability. The lower limit, by contrast, will always be lower than the unconditional probability and as such, will never be significant. Application of this confidence interval increases the requirements for statistical significance for the categories at each lag, resulting in a more robust corrected pattern.

The results obtained by applying the formula corresponding to the corrected expected or unconditional probability (shown in Table Table6a) 6a ) are presented in Table Table6b, 6b , which is an extension of Table Table5b 5b .

(a) Formula for calculating the corrected unconditional (expected) probability. (b) Table showing the probabilities from Table Table5b 5b with the addition of the corrected conditional probabilities in the second row (bold values).

These correspond to the upper limit of the confidence interval built around the unconditional probability values, with p < 0.05 .

A second optimization step involving the calculation of adjusted residuals or hypergeometric Z -values (Allison and Liker, 1982 ) is also possible but cannot be done manually.

Figure 4 shows the corrected behavioral pattern extracted from the data in Table Table6b. 6b . As shown, it is different to the uncorrected pattern shown in Figure Figure3. 3 . Note that in both cases, A, the given behavior, is statistically associated with B at the first lag and D at the second lag.

An external file that holds a picture, illustration, etc.
Object name is fpsyg-09-00013-g0003.jpg

Optimized corrected behavioral pattern following construction of a confidence interval around the unconditional probabilities. The corrected pattern reveals the typical alternation seen in patients with endogenous depression.

Lag sequential analysis is the first of the three key techniques we use in our text-liquefying approach to indirect observation. It has been widely used in systematic observation studies from a range of areas published in journals listed in the Journal Citations Report (JCR) (e.g., Gimeno et al., 2006 ; Lapresa et al., 2013 ; Roustan et al., 2013 ).

Polar coordinate analysis

Polar coordinate analysis, which was proposed by Sackett ( 1980 ), combines adjusted residuals from lag sequential analysis and the Z sum statistic (Cochran, 1954 ). This statistic provides a representative value for a series of independent values (adjusted residuals at different prospective or retrospective negative lags) to produce prospective and retrospective Z sum values. Sackett ( 1980 ) recommended using the same number of prospective and retrospective lags. Based on experience to date (Sackett, 1987 ; Anguera and Losada, 1999 ), we suggest analyzing at least five prospective lags and five retrospective lags (−5 to +5).

The results of the computation determine the quadrant in which the different vectors are located and indicate their respective lengths and angles (Sackett, 1980 ). Vectors provide information on the nature of the relationship (prospective/retrospective activation/inhibition) between a focal behavior, which is equivalent to a given behavior in lag sequential analysis, and other categories of interest, known as conditional behaviors. The concept of genuine retrospectivity (Anguera, 1997 ) was introduced at a later stage to improve the classic concept of retrospectivity. The genuine retrospective approach considers negative lags from a backwards rather than a forwards perspective, i.e., it looks at what happened from lag 0 back to lag −5 rather than from lag −5 to lag 0.

Adjusted residuals, Z -values, and vector length and angles can all be computed in the open-access software program HOISAN (v. 1.6.3.3) (Hernández-Mendo et al., 2012 ), which also includes a feature to produce the results in graph form.

The meaning of the vectors (see below) varies according to the quadrant in which they are located, and the position of a vector in one quadrant or another is determined by the combination of positive or negative signs on the prospective and retrospective Z sum values. In quadrant I (+ +), the focal and conditional behaviors activate each other; in quadrant II (− +), the focal behavior inhibits and is activated by the conditional behavior; in quadrant III (− −), the focal and conditional behaviors inhibit each other; and in quadrant IV (+ −), the focal behavior activates and is inhibited by the conditional behavior. The length of the vectors indicates the strength (statistical significance) of the association between the focal and conditional behaviors.

To illustrate briefly how the technique works, we used the data from Table Table3a 3a to produce a vector map showing the relationships between A, the focal behavior (in our example, expressions of sorrow or sadness), and categories B (expressions of self-perceived improvement), C (expressions of self-perceived worsening), and D (expressions of joy at having overcome the problem), the conditional behaviors. Table Table7 7 shows the values for the adjusted residuals and corresponding Z sum values, while Table Table8 8 shows the length and angle of the vectors for each of the conditional behaviors. The corresponding vectors are shown in Figure Figure4 4 .

Adjusted residuals and corresponding Z -values from the polar coordinate analysis with A as the focal behavior or category and B, C, and D as the conditional behaviors.

The analysis was performed in HOISAN .

Polar coordinate analysis results showing the length and angle of the different vectors, the quadrant in which each vector is located, and the Z sum values (Cochran, 1954 ) from the prospective and retrospective perspectives.

In the presented situation, A is the focal behavior, so the results show how expressions of sorrow or sadness activate expressions of self-perceived improvement (Quadrant IV) or joy at having overcome the problem (Quadrant I). The focal behavior is not self-generating (Quadrant III). Additionally, expressions of sorrow or sadness do not generate self-perceived worsening (Quadrant II), although self-perceived worsening does generate the focal behavior .

An external file that holds a picture, illustration, etc.
Object name is fpsyg-09-00013-g0004.jpg

Polar coordinate map showing the vectors for the categories A (focal category), B, C, and D. As indicated in the legend of Table Table8, 8 , A is the focal behavior and expressions of sorrow or sadness activate expressions of self-perceived improvement (Quadrant IV) and joy at having overcome the problem (Quadrant 1). The focal behavior is not self-generating (Quadrant III). Additionally, expressions of sorrow or sadness do not generate self-perceived worsening (Quadrant II), although self-perceived worsening does generate the focal behavior.

The strongest association detected for the focal behavior A (apart from with itself) was with B (in quadrant IV, with a vector length of 0.54), followed by D (quadrant I, with a vector length of 0.41). Although A and C have the longest vector (0.65), the fact that C is located in quadrant II (because its angle is 125.79°) means that A inhibits rather than activates C. C does not appear because its excitatory activity was insignificant.

Readers can find numerous examples of the application of polar coordinate analysis in a wide range of fields in direct observation (e.g., Gorospe and Anguera, 2000 ; Herrero Nivela, 2000 ; Anguera et al., 2003 ; Castañer et al., 2016 , 2017 ; López et al., 2016 ; Aragón et al., 2017 ; Morillo et al., 2017 ; Santoyo et al., 2017 ; Suárez et al., 2018 ), and more recently indirect observation (e.g. Arias-Pujol and Anguera, 2017 ).

T-pattern detection

T-pattern detection was proposed and developed by Magnusson ( 1996 , 2000 , 2005 , 2016 ). It involves the use of an algorithm that calculates the temporal distances between behaviors and analyzes the extent to which the critical interval remains invariant relative to the null hypothesis that each behavior is independently and randomly distributed over time. It needs data, in the form of code matrices, for which the duration of each co-occurrence has been recorded. Microanalyses of data are also possible and very useful (Anguera, 2005 ). The software program, Theme (v. 6 Edu), features different settings that can be modified to obtain complementary results that, analyzed together, can provide a greater understanding of interactive transitions over time. Theme is an open-access software program that provides all the necessary features for analyzing data and presenting the results graphically as dendrograms or tree diagrams.

As with lag sequential and polar coordinate analysis, we have also used the data from Table Table3a 3a to illustrate the use of T-pattern detection. It should be noted that the method applied is rather unconventional, as the temporal distance parameter was set at 1 in all cases.

Figure Figure5 5 shows the first of the 13 T-patterns obtained ( p < 0.05). Note that despite the small size of the dataset, Theme detected a primary relationship between A and B (between expressions of sorrow or sadness and expressions of self-perceived improvement) and A and D (between expressions of sorrow or sadness and expressions of joy at having overcome the problem), as shown graphically in Figure Figure5 5 .

An external file that holds a picture, illustration, etc.
Object name is fpsyg-09-00013-g0005.jpg

First of the 13 T-patterns detected in the data from Table Table3a 3a ( p < 0.05).

Examples of the application of T-pattern detection can be found in studies by Castañer et al. ( 2013 ), Diana et al. ( 2017 ), Lapresa et al. ( 2013 ), and Sarmento et al. ( 2015 ) in direct observation and by Blanchet et al. ( 2005 ) and Baraud et al. ( 2016 ) in indirect observation.

Complementary use of techniques

Although the specifics of lag sequential analysis, polar coordinate analysis, and T-pattern detection differ, all three techniques serve to analyze and increase understanding of the internal structure of verbal or textual material derived from indirect observation. In addition, they can be applied to the same data to provide complementary insights and unveil invisible structures hidden within data. Their relevance is even greater in indirect observation studies where data have traditionally been analyzed from a purely qualitative perspective.

The convergence of results from three different quantitative approaches is a cause for celebration in a field such as indirect observation, where studies to date have largely relied on frequency counts or on qualitative approaches, which of course have their merits but are prone to considerable subjectivity bias.

There is growing interest in combining these techniques to gain a greater understanding of behavioral patterns that remain hidden to the naked eye. Two recent examples can be found in the studies of Santoyo et al. ( 2017 ) and Tarragó et al. ( 2017 ).

Adapted methodological procedure for conducting an indirect observation study based on text liquefaction

We have presented a structured procedure detailing the successive stages of the method we propose for studying verbal behavior and/or textual material in an indirect observation study (Table (Table9). 9 ). Our aim was not to offer a general approach to systematic observation from the perspective of indirect observation, as guidelines already exist for the reporting of systematic studies within observational methodology (Portell et al., 2015a ). Our aim rather was to introduce the reader to the key concepts of indirect observation studies and provide step-by-step guidance on how to perform such a study. The procedure we propose is summarized in Table Table9 9 and has already been applied in studies from different fields (Vaimberg, 2010 ; García-Fariña et al., 2016 ; Arias-Pujol and Anguera, 2017 ).

Procedure for conducting an indirect observation study based on liquefying a text.

Conclusions and limitations

Within the broad framework of mixed methods, we have presented indirect observation as a structured method consisting of different steps designed to guarantee scientific rigor. The method consists of the quantitization of qualitative data derived from verbal or textual material to produce code matrices which, following appropriate organization and rigorous quality control procedures, can be analyzed using robust, rigorous, and objective techniques. In a sense, we liquefy the text into a form suitable for quantitative analysis.

Although the materials that support direct and indirect observation are different, the methodological proposal described in this paper shows that both forms of observation share a systematic procedure in which adequately trained observers apply a robust, reliable purpose-designed observation instrument to produce quantitative indicators of the many processes underlying everyday behavior. The main strengths of our approach are that it enables the merging of data from different sources and offers the possibility of taking advantage of the continuous advances in information and communication technologies to study aspects of biopsychosocial behavior in everyday contexts. There are two main limitations. On the one hand, the dimensions in an indirect observation study depend largely on a theoretical framework and a conceptual framework, and these may be lacking. On the other hand, observation instruments comprising category systems, either alone or combined with a field format, also require a theoretical framework. However, the proposed approach has the advantage of allowing all data obtained from narratives to be included in the study, even those which do not fit with the theoretical framework or are contradictory. In fact, the validation of the coding process entails, among other things, checking that no new information has been added, that no information has been eliminated, and that the meaning of the information has not been altered. In this way, there is no omission of information that could lead to bias. This information can be included using bottom-up or top-down processes (Anguera, 1991 ; Anguera et al., 2007 ), in other words, the narratives are categorized on the basis of the chosen theoretical framework (top-down) and the theoretical framework is adapted on the basis of the narratives given (bottom-up). An exclusively quantitative study would entail the loss of sensitive and relevant information about the spontaneous behavior, as it would require excluding all variables not envisaged in the chosen theoretical framework. Hence our insistence on the enormous potential of mixed methods research, which suitably integrates both qualitative and quantitative elements.

This work presented a novel approach, based on sequence of occurrence, for transforming qualitative data into quantitative data that can be analyzed using robust quantitative techniques. Additionally, it is important to note that it is possible, at any time during the analysis, to return from the quantitative data to the narrative data. As a result, this approach presents advantages of both qualitative and quantitative methods, at the same time it covers weaknesses of both methods.

Author contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the reviewers whose suggestions and comments greatly helped to improve and clarify this manuscript.

Funding. We gratefully acknowledge the support of the Spanish government (Ministerio de Economía y Competitividad) within the Projects Avances metodológicos y tecnológicos en el estudio observacional del comportamiento deportivo [Grant PSI2015-71947-REDT; MINECO/FEDER, UE] (2015-2017), and La actividad f í sica y el deporte como potenciadores del estilo de vida saludable: evaluación del comportamiento deportivo desde metodolog í as no intrusivas [Grant DEP2015-66069-P; MINECO/FEDER, UE] (2016-2018). We gratefully acknowledge the support of the Generalitat de Catalunya Research Group (GRUP DE RECERCA E INNOVACIÓ EN DISSENYS [GRID]). Tecnolog í a i aplicació multimedia i digital als dissenys observacionals , [Grant 2014 SGR 971]. This research was also funded by the project Methodological quality and effectiveness from evidence (Chilean National Fund of Scientific and Technological Development -FONDECYT-, reference number 1150096). Lastly, first author also acknowledge the support of University of Barcelona (Vice-Chancellorship of Doctorate and Research Promotion), and second author also acknowledge the support of Universitat Autònoma de Barcelona.

Allison P. D., Liker J. K. (1982). Analyzing sequential categorical data on dyadic interaction: a comment on Gottman . Psychol. Bull . 91 , 393–403. 10.1037/0033-2909.91.2.393 [ CrossRef ] [ Google Scholar ]
Altimir C., Krause M., de la Parra G., Dagnino P., Tomicic A., Valdés N., et al.. (2010). Clients', therapists', and observers' agreement on the amount, temporal location, and content of psychotherapeutic change and its relation to outcome . Psychother. Res. 20 , 472–487. 10.1080/10503301003705871 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Anguera M. T. (1979). Observational typology . Qual. Quant . 13 , 449–484. [ Google Scholar ]
Anguera M. T. (1990). Metodología observacional [Observational methodology], in Metodología de la Investigación en Ciencias del Comportamiento , eds Arnau J., Anguera M. T., Gómez J. (Murcia: Universidad de Murcia; ), 125–238. [ Google Scholar ]
Anguera M. T. (1991). La metodología observacional en evaluación de programas [Observational methodology in program evaluation] . Rev. Mex. Anal. Conducta 17 , 121–145. [ Google Scholar ]
Anguera M. T. (1997). From prospective patterns in behavior to joint analysis with a retrospective perspective, in Colloque sur Invitation Méthodologie d'Analyse des Interactions Sociales . Paris: University of Paris V, Sorbonne. [ Google Scholar ]
Anguera M. T. (2001). Hacia una evaluación de la actividad cotidiana y su contexto: presente o futuro para la metodología? [Towards an evaluation of everyday activity and its context: present or future for methodology? Lecture of admission to the Royal Europan Academy of Doctors in 1999], in Estrategias de Evaluación y Medición del Comportamiento en Psicología [Strategies of Evaluation and Measurement of the Behavior in Psychology] , eds Bazán Ramírez A., Arce Ferrer A. (México: Instituto Tecnológico de Sonora y Universidad Autónoma de Yucatán; ), 11–86. [ Google Scholar ]
Anguera M. T. (2003). Observational methods (general), in Encyclopedia of Psychological Assessment , Vol. 2 , ed Fernández-Ballesteros R. (London: Sage; ), 632–637. [ Google Scholar ]
Anguera M. T. (2005). Microanalysis of T-patterns. Analysis of symmetry/asymmetry in social interaction, in The Hidden Structure of Social Interaction. From Genomics to Culture Patterns , eds Anolli L., Duncan S., Magnusson M., Riva G. (Amsterdam: IOS Press; ), 51–70. [ Google Scholar ]
Anguera M. T. (2010). Posibilidades y relevancia de la observación sistemática por el profesional de la Psicología [Possibilities and relevance of systematic observation for pyschology professionals] . Papeles Psicol . 31 , 122–130. [ Google Scholar ]
Anguera M. T. (2017). Transiciones interactivas a lo largo de un proceso de desarrollo: complementariedad de análisis [Interactive transitions throughout a development process. A complementary analysis approach], in Mecanismos Básicos de Toma de Decisiones: Perspectivas Desde las Ciencias del Comportamiento y Del Desarrollo , ed Santoyo C. (México: CONACYT 178383/UNAM; ), 179–213. [ Google Scholar ]
Anguera M. T. (in press). Is it possible to perform ”Liquefying“ actions in conversational analysis? The detection of structures in indirect observation, in The Temporal Structure of Multimodal Communication , ed Hunyadi L. (New York, NY: Springer; ). [ Google Scholar ]
Anguera M. T., Camerino O., Castañer M., Sánchez-Algarra P., Onwuegbuzie A. J. (2017a). The specificity of observational studies in physical activity and sports sciences: moving forward in mixed methods research and proposals for achieving quantitative and qualitative symmetry . Front. Psychol. 8 :2196. 10.3389/fpsyg.2017.02196 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Anguera M. T., Hernández-Mendo A. (2016). Avances en estudios observacionales en Ciencias del Deporte desde los mixed methods [Advances in observational studies in Sport Sciences from a mixed methods approach] . Cuad. Psicol. Deporte 16 , 17–30. [ Google Scholar ]
Anguera M. T., Izquierdo C. (2006). Methodological approaches in human communication. From complexity of situation to data analysis, in From Communication to Presence. Cognition, Emotions and Culture towards the Ultimate Communicative Experience , eds Riva G., Anguera M. T., Wiederhold B. K., Mantovani F. (Amsterdam: IOS Press; ), 203–222. [ Google Scholar ]
Anguera M. T., Jonsson G. K., Sánchez-Algarra P. (2017b). Liquefying text from human communication processes: a methodological proposal based on t-pattern detection . J. Multimodal. Comm. Stud. 4 , 10–15. [ Google Scholar ]
Anguera M. T., Losada J. L. (1999). Reducción de datos en marcos de conducta mediante la técnica de coordenadas polares [Data reduction in behavioral frameworks through polar coordinate analysis], in Observación de la Conducta Interactiva en Situaciones Naturales: Aplicaciones , ed Anguera M. T. (Barcelona: E.U.B; ), 163–188. [ Google Scholar ]
Anguera M. T., Magnusson M. S., Jonsson G. K. (2007). Instrumentos no estándar [Non-standard instruments] . Avances Med. 5 , 63–82. [ Google Scholar ]
Anguera M. T., Santoyo C., Espinosa M. C. (2003). Evaluating links intensity in social networks in a school context through observational designs, in Culture, Environmental Action and Sustainability , eds García Mira R., Sabucedo Cameselle J. M., Romay Martínez J. (Göttingen: Hogrefe and Huber; ), 286–298. [ Google Scholar ]
Aragón S., Lapresa D., Arana J., Anguera M. T., Garzón B. (2017). An example of the informative potential of polar coordinate analysis: Sprint tactics in elite 1500 m track events . Meas. Phys. Educ. Exerc. Sci. 21 , 26–33. 10.1080/1091367X.2016.1245192 [ CrossRef ] [ Google Scholar ]
Arana J., Lapresa D., Anguera M. T., Garzón B. (2016). Ad hoc procedure for optimising agreement between observational records . Anal. Psicol. 32 , 589–595. 10.6018/analesps.32.2.213551 [ CrossRef ] [ Google Scholar ]
Arias-Pujol E., Anguera M. T. (2004). Detección de patrones de conducta comunicativa en un grupo terapéutico de adolescentes [Detection of communicative behavior patterns in an adolescent group therapy group] . Acción Psicol . 3 , 199–206. 10.5944/ap.3.3.513 [ CrossRef ] [ Google Scholar ]
Arias-Pujol E., Anguera M. T. (2017). Observation of interactions in adolescent group therapy: a mixed methods study . Front. Psychol. 8 :1188. 10.3389/fpsyg.2017.01188 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bakeman R. (1978). Untangling streams of behavior: sequential analysis of observation data, in Observing Behavior, Vol. 2: Data Collection and Analysis Methods , ed Sackett G. P. (Baltimore, MD: University of Park Press; ), 63–78. [ Google Scholar ]
Bakeman R. (1991). From lags to logs: advances in sequential analysis . Rev. Mex. Anal. Conducta 17 , 65–83. [ Google Scholar ]
Bakeman R., Gottman J. M. (1987). Applying observational methods: a systematic view, in Handbook of Infant Development , ed Osofsky J. D. (New York, NY: Wiley; ), 818–853. [ Google Scholar ]
Bakeman R., Quera V. (1996). Análisis de la Interacción. Análisis Secuencial con SDIS y GSEQ [Analysis of Interaction. Sequential Analysis with SDIS and GSEQ ]. Madrid: Ra-Ma. [ Google Scholar ]
Bakeman R., Quera V. (2011). Sequential Analysis and Observational Methods for the Behavioral Sciences . Cambridge: Cambridge University Press. [ Google Scholar ]
Baraud I., Deputte B. L., Pierre J.-S., Blois-Heulin C. (2016). Informative value of vocalizations during multimodal interactions in red-capped mangabeys, in Discovering Hidden Temporal Patterns in Behavior and Interactions: T-Pattern Detection and Analysis with THEME , eds Magnusson M. S., Burgoon J. K., Casarrubea M., McNeill D. (New York, NY: Springer; ), 255–278. [ Google Scholar ]
Bautista M. A., Hernández-Vela A., Escalera S., Igual L., Pujol O., Mora J., et al.. (2015). A gesture recognition system for detecting behavioral patterns of ADHD . IEEE Trans. Cybern . 46 , 136–147. 10.1109/TCYB.2015.2396635 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bavelas J. B., Chovil N. (2000). Visible acts of meaning. An integrated message model of language in face-to-face dialogue . J. Lang. Soc. Psychol. 19 , 163–194. 10.1177/0261927X00019002001 [ CrossRef ] [ Google Scholar ]
Bazeley P. (2003). Computerized data analysis for mixed methods research, in Handbook of Mixed Methods in Social and Behavioral Research , eds Tashakkori A., Teddlie C. (Thousand Oaks, CA: Sage; ), 385–422. [ Google Scholar ]
Bazeley P. (2006). The contribution of computer software to integrating qualitative and quantitative data and analyses . Res. Sch . 13 , 64–74. [ Google Scholar ]
Bazeley P. (2009). Integrating data analyses in mixed methods research . J. Mix. Methods Res . 3 , 203–207. 10.1177/1558689809334443 [ CrossRef ] [ Google Scholar ]
Behar J. (1993). Observación y análisis de la producción verbal de la conducta [Observation and analysis of the verbal production of behavior], in Metodología Observacional en la Investigación Psicológica. Vol. 1. Fundamentación , ed Anguera M. T. (Barcelona: P.P.U.), 331–389. [ Google Scholar ]
Bell K., Fahmy E., Gordon D. (2016). Quantitative conversations: the importance of developing rapport in standardised interviewing . Qual. Quant. 50 , 193–212. 10.1007/s11135-014-0144-2 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bion W. R. (1985). Experiencias en Grupos [Experiences in Groups] . Buenos Aires: Paidós. [ Google Scholar ]
Birdwhistell R. L. (1970). Kinesics and Context: Essays and Body Motion Communication . Philadelphia, PA: University of Pennsylvania Press. [ Google Scholar ]
Björk A. B., Sjöström M., Johansson E. E., Samuelson E., Umefjord G. (2014). Women's experiences of Internet-based or postal treatment for stress urinary incontinence . Qual. Health Res . 24 , 484–493. 10.1177/1049732314524486 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Blanchet A., Batt M., Trognon A., Masse L. (2005). Language and behaviour patterns in a therapeutic interaction sequence, in The Hidden Structure of Social Interaction. From Genomics to Culture Patterns , eds Anolli L., Duncan S., Magnusson M. S., Riva G. (Amsterdam: IOS Press; ), 124–140. [ Google Scholar ]
Blanco-Villaseñor A. (2001). Generalizabilidad de observaciones uni y multifaceta: estimadores LS y ML [Generalizability of mono and multifaceted observations: LS and ML estimators] . Metodol. Cienc. Comport . 3 , 161–193. [ Google Scholar ]
Blanco-Villaseñor A., Losada J. L., Anguera M. T. (2003). Analytic techniques in observational designs in environment behavior relation . Medio Ambient. Comport. Hum. 4 , 111–126. [ Google Scholar ]
Bolger N., Davis A., Rafaeli E. (2003). Diary methods: capturing life as it is lived . Annu. Rev. Psychol. 54 , 579–616. 10.1146/annurev.psych.54.101601.145030 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bradley E. H., Curry L. A., Devers K. J. (2007). Qualitative data analysis for health services research: developing taxonomy, themes, and theory . Health Serv. Res . 42 , 1758–1772. 10.1111/j.1475-6773.2006.00684.x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Brondani M. A., MacEntee M. I., Bryant S. R., O'Neill B. (2008). Using written vignettes in focus groups among older adults to discuss oral health as a sensitive topic . Qual. Health Res . 18 , 1145–1153. 10.1177/1049732308320114 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bunker D., Thorpe R. (1982). A model for the teaching of games in secondary schools . Bull. Phys. Educ . 18 , 5–8. [ Google Scholar ]
Caddick N., Smith B., Phoenix C. (2015). Male combat veterans' narratives of PTSD, masculinity, and health . Sociol. Health Illn . 37 , 97–111. 10.1111/1467-9566.12183 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Calsamiglia H., Tusón A. (1999). Las Cosas del Decir. Manual de Análisis del Discurso [Things About Saying. A Discourse Analysis Manual] . Barcelona: Ariel. [ Google Scholar ]
Castañer M., Barreira D., Camerino O., Anguera M. T., Canton A., Hileno R. (2016). Goal scoring in soccer: a polar coordinate analysis of motor skills used by Lionel Messi . Front. Psychol . 7 :806. 10.3389/fpsyg.2016.00806 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Castañer M., Barreira D., Camerino O., Anguera M. T., Fernandes T., Hileno R. (2017). Mastery in goal scoring, T-pattern detection and polar coordinate analysis of motor skills used by Lionel Messi and Cristiano Ronaldo . Front. Psychol. 8 :741. 10.3389/fpsyg.2017.00741 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Castañer M., Camerino O., Anguera M. T., Jonsson G. K. (2013). Kinesics and proxemics communication of expert and novice PE teachers . Qual. Quant . 47 , 1813–1829. 10.1007/s11135-011-9628-5 [ CrossRef ] [ Google Scholar ]
Cochran W. G. (1954). Some methods for strengthening the common χ2 tests . Biometrics 10 , 417–451. 10.2307/3001616 [ CrossRef ] [ Google Scholar ]
Coll C., Onrubia J. (2001). Estrategias discursivas y recursos semióticos en la construcción de sistemas de significados compartidos entre profesor y alumnos [Discursive strategies and semiotic resources in the construction of shared meaning systems between teachers and students] . Invest. Esc . 45 , 21–31. [ Google Scholar ]
Conner T. S., Lehman B. J. (2013). Getting started. Lauching a study in daily life, in Handbook of Research Methods for Studying Daily Life , eds Mehl M. R., Conner T. S. (New York, NY: The Guilford Press; ), 89–105. [ Google Scholar ]
Coutinho J., Ribeiro E., Sousa I., Safran J. D. (2014). Comparing two methods of identifying alliance rupture events . Psychotherapy 51 , 434–442. 10.1037/a0032171 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Crawford T. H. (1992). The politics of narrative form . Lit. Med . 11 , 147–162. 10.1353/lm.2011.0234 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Creswell J. W., Plano Clark V. L. (2007). Designing and Conducting Mixed Methods Research, 3rd Edn 2017 . Thousand Oaks, CA: Sage. [ Google Scholar ]
Creswell J. W., Plano Clark V. L., Gutmann M. L., Hanson W. E. (2003). Advanced mixed methods research designs, in Handbook of Mixed Methods in Social and Behavioral Research , eds Tashakkori A., Teddlie C. (Thousand Oaks, CA: Sage; ), 209–240. [ Google Scholar ]
Crutcher R. J. (2003). A computer-aided digital audio recording and encoding system for improving the encoding of verbal reports . Behav. Res. Methods 35 , 263–268. 10.3758/BF03202551 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Crutcher R. J. (2007). CAPAS 2.0: a computer tool for coding transcribed and digitally recorded verbal reports . Behav. Res. Methods 39 , 167–174. 10.3758/BF03193145 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Cuervo J. J. (2014). Comunicación y Perdurabilidad en Parejas Viables. Estudio Observacional de Caso Múltiple [Communication and Durability in Viable Couples. A Multiple Observational Case Study]. Ph.D. thesis, Universidad Autónoma de Barcelona: Barcelona. [ Google Scholar ]
Dagnino P., Krause M., Pérez J. C., Valdés N., Tomicic A. (2012). The evolution of communicative intentions during change episodes and throughout the therapeutic process . Res. Psychother. 15 , 75–86. [ Google Scholar ]
Dam G., Kaufmann S. (2008). Computer assessment of interview data using latent semantic analysis . Behav. Res. Methods 40 , 8–20. 10.3758/BRM.40.1.8 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Danzinger K. (1982). Comunicación Interpersonal [Interpersonal Communication] . México: Manual Moderno. [ Google Scholar ]
De Fina A., Georgakopoulo A. (eds.). (2015). The handbook of narrative analysis. Malden: Wiley-Blackwell. [ Google Scholar ]
Diana B., Zurloni V., Elia M., Cavalera C. M., Jonsson G. K., Anguera M. T. (2017). How game location affects soccer performance: T-pattern analysis of attack actions . Front. Psychol. 8 :1415. 10.3389/fpsyg.2017.01415 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Dickman H. R. (1963). The perception of behavioral units, in The Stream of Behavior , ed Barker R. (New York, NY: Appleton-Century-Crofts; ), 23–41. [ Google Scholar ]
Duran N. D., McCarthy P. M., Graesser A. C., McNamara D. S. (2007). Using temporal cohesion to predict temporal coherence in narrative and expository texts . Behav. Res. Methods 39 , 212–223. 10.3758/BF03193150 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Elvish R., Lever S. J., Johnstone J., Cawley R., Keady J. (2013). Psychological interventions for careers of people with dementia: a systematic review of quantitative and qualitative evidence . Counsell. Psychother. Res. J . 13 , 106–125. 10.1080/14733145.2012.739632 [ CrossRef ] [ Google Scholar ]
Escalera S., Martínez R. M., Vitriá J., Radeva P., Anguera M. T. (2009). Automatic dominance detection in dyadic conversations . Escr. Psicol . 3 , 41–45. [ Google Scholar ]
Escolano-Pérez E., Herrero-Nivela M. L., Blanco-Villaseñor A., Anguera M. T. (2017). Systematic observation: relevance of this approach in preschool executive function assessment and association with later academic skills . Front. Psychol. 8 :2031 10.3389/fpsyg.2017.02031 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Fetters M. D., Freshwater D. (2015). The 1 + 1 = 3 integration challenge . J. Mix. Methods Res . 9 , 115–117. 10.1177/1558689815581222 [ CrossRef ] [ Google Scholar ]
Fischer A. H., Becker D., Veenstra L. (2012). Emotional mimicry in social context: the case of disgust and pride . Front. Psychol . 3 :475. 10.3389/fpsyg.2012.00475 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Gabriel Y. (2004). The voice of experience and the voice of the expert - Can they speak to each other?, in Narrative Research in Health and Illness , eds Hurwitz B., Greenhalgh T., Skultans V. (Malden, MA: Blackwell; ), 168–185. [ Google Scholar ]
Gale J. E. (1991). Conversation Analysis of Therapeutic Discourse . Norwood, OH: Ablex Publishing Corporation. [ Google Scholar ]
García-Fariña A. (2015). Análisis del Discurso Docente Como Recurso Metodológico del Profesorado de Educación Física en la Etapa de Educación Primaria [Analysis of Teacher-Led Discourse as a Methodological Resource for Primary School Physical Education Teachers] . Ph.D. thesis, Universidad de La Laguna, Tenerife.
García-Fariña A., Jiménez Jiménez F., Anguera M. T. (2016). Análisis observacional del discurso docente del profesorado de educación física en formación a través de patrones comunicativos [Observational analysis of teaching discourse physical education training teachers through communicative patterns] . Cuad. Psicol. Deporte 16 , 171–182. [ Google Scholar ]
Gimeno A., Anguera M. T., Berzosa A., Ramírez L. (2006). Detección de patrones interactivos en la comunicación de familias con hijos adolescentes [Detection of interactive communication patterns in families with adolescent children] . Psicothema 18 , 785–790. [ PubMed ] [ Google Scholar ]
Gorospe G., Anguera M. T. (2000). Modificación de la técnica clásica de coordenadas polares mediante un desarrollo distinto de la retrospectividad: aplicación al tenis [Modification to the classic polar coordinate technique using a distinct concept of retrospectivity: an application to tennis] . Psicothema 12 ( Suppl. 2 ), 279–282. [ Google Scholar ]
Greene J. C. (2006). Toward a methodology of mixed methods social inquiry . Res. Sch. 13 , 93–98. [ Google Scholar ]
Hardley D. M. (2014). Using social media and Internet data for public health surveillance: the importance of talking . Milbank Q . 92 , 34–39. 10.1111/1468-0009.12039 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Hernández-Mendo A., López-López J. A., Castellano J., Morales-Sánchez V., Pastrana J. L. (2012). Hoisan 1.2: programa informático para uso en metodología observacional [Hoisan 1.2: software for observational methodology] . Cuad. Psicol. Deporte 12 , 55–78. 10.4321/S1578-84232012000100006 [ CrossRef ] [ Google Scholar ]
Herrero Nivela M. L. (2000). Utilización de la técnica de coordenadas polares en el estudio de la interacción infantil en el marco escolar [Use of the technic of polar coordinates in the study of the children interaction in the school room] . Psicothema 12 , 292–297. [ Google Scholar ]
Holle H., Obermeier C., Schmidt-Kassow M., Friederici A. D., Ward J., Gunter T. C. (2012). Gesture facilitates the syntactic analysis of speech . Front. Psychol . 3 :74. 10.3389/fpsyg.2012.00074 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Holtgraves T., Han T. L. (2007). A procedure for studying online conversational processing using a chat bot . Behav. Res. Methods 39 , 156–163. 10.3758/BF03192855 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Hurwitz B., Greenhalgh T., Skultans V. (eds.). (2004). Narrative Research in Health and Illness . London: British Medical Journal Books and Blackwell. [ Google Scholar ]
Jackson M., Harrison P., Swinburn B., Lawrence M. (2015). Using a qualitative vignette to explore a complex public health issue . Qual. Health Res . 25 , 1395–1409. 10.1177/1049732315570119 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Johnson R. B., Onwuegbuzie A. J., Turner L. A. (2007). Toward a definition of mixed methods research . J. Mix. Methods Res . 1 , 112–133. 10.1177/1558689806298224 [ CrossRef ] [ Google Scholar ]
Krause M., Altimir C., Pérez J. C., Echávarri O., Valdés N., Strasser K. (2016). Therapeutic verbal communication in change episodes: a comparative microanalysis of linguistic basic forms . Estud. Psicol. 37 , 514–547. 10.1080/02109395.2016.1227575 [ CrossRef ] [ Google Scholar ]
Krippendorff K. (2013). Content Analysis. An Introduction to Its Methodology, 3rd Edn. Thousand Oaks, CA: Sage. [ Google Scholar ]
Krueger R. A., Casey M. A. (2009). Focus Groups. A Practical Guide for Applied Research . Los Angeles, CA: Sage. [ Google Scholar ]
Lapresa D., Arana J., Anguera M. T., Garzón B. (2013). Comparative analysis of the sequentiality using SDIS-GSEQ and THEME: a concrete example in soccer . J. Sports Sci. 31 , 1687–1695. 10.1080/02640414.2013.796061 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lausberg H., Sloetjes H. (2009). Coding gestural behavior with the NEUROGES–ELAN system . Behav. Res. Methods 41 , 841–849. 10.3758/BRM.41.3.841 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lausberg H., Sloetjes H. (2016). The revised NEUROGES-ELAN system: an objective and reliable interdisciplinary analysis tool for nonverbal behavior and gesture . Behav. Res. Methods 48 , 973–993. 10.3758/s13428-015-0622-z [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lebrecht S., Bar M., Barrett L. F., Tarr M. J. (2012). Micro-valences. Perceiving affective valence in everyday objects . Front. Psychol. 3 :107. 10.3389/fpsyg.2012.00107 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lederer S. H., Battaglia D. (2015). Using signs to facilitate vocabulary in chldren with language delays . Infants Young Child . 28 , 18–31. 10.1097/IYC.0000000000000025 [ CrossRef ] [ Google Scholar ]
Leech N. L., Onwuegbuzie A. J. (2009). A typology of mixed methods research designs . Qual. Quant . 43 , 265–275. 10.1007/s11135-007-9105-3 [ CrossRef ] [ Google Scholar ]
Lévy P. (1995). Qué es lo Virtual? [What is Virtual?]. Barcelona: Paidós. [ Google Scholar ]
Lindqvist O., Threlkeld G., Street A. F., Tishelman C. (2014). Reflections on using biographical approaches in end-of-life-care: dignity therapy as example . Qual. Health. Res . 25 , 40–50. 10.1177/1049732314549476 [ PubMed ] [ CrossRef ] [ Google Scholar ]
López J., Valero A., Anguera M. T., Díaz A. (2016). Disruptive behavior among elementary students in physical education . Springerplus 5 :1154 10.1186/s40064-016-2764-6 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Love A. C. (2006). History, scientific methodology, and the “squishy” sciences . Perspect. Biol. Med . 49 , 452–456. 10.1353/pbm.2006.0042 [ CrossRef ] [ Google Scholar ]
Magnusson M. S. (1996). Hidden real-time patterns in intra- and inter-individual behavior . Eur. J. Psychol. Assess . 12 , 112–123. 10.1027/1015-5759.12.2.112 [ CrossRef ] [ Google Scholar ]
Magnusson M. S. (2000). Discovering hidden time patterns in behavior: T-patterns and their detection . Behav. Res. Methods Instrum. Comput . 32 , 93–110. 10.3758/BF03200792 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Magnusson M. S. (2005). Understanding social interaction: discovering hidden structure with model and algorithms, in The Hidden Structure of Social Interaction. From Genomics to Culture Patterns , eds Anolli L., Duncan S., Magnusson M. S., Riva G. (Amsterdam: IOS Press; ), 4–22. [ Google Scholar ]
Magnusson M. S. (2016). Time and self-similar structure in behavior and interactions: from sequences to symmetry and fractals, in Discovering Hidden Temporal Patterns in Behavior and Interactions: T-Pattern Detection and Analysis with THEME , eds Magnusson M. S., Burgoon J. K., Casarrubea M., McNeill D. (New York, NY: Springer; ), 3–35. [ Google Scholar ]
Martínez del Pozo M. (1993). Análisis del ”Proceso de Elaboración del Duelo“ en el Hipertenso Esencial: Estudio Empírico de Sus Fases Mediante Observación Sistemática [Analysis of the ”Elaboration of Mourning“ in Individuals with Essential Hypertension: An Empirical Study of Elaboration Phases through Systematic Observation] . Ph.D. thesis, University of Barcelona, Barcelona.
Mashal N., Solodkin A., Dick A. S., Chen E. E., Small S. L. (2012). A network model of observation and imitation if speech . Front. Psychol . 3 :84. 10.3389/fpsyg.2012.00084 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Maxwell J. A., Chmiel M., Rogers S. E. (2015). Designing integration in multimethod and mixed methods research, in The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry , eds Hesse-Biber S. N., Johnson R. B. (New York, NY: Oxford University Press; ), 223–239. [ Google Scholar ]
McDougall J. (1991). Teatros del Cuerpo [Body Theaters] . Madrid: Julián Yébenes. [ Google Scholar ]
McLean M., Cleland J. A., Worrell M., Vögele C. (2011). What am I going to say here?” The experiences of doctors and nurses communicating with patients in a cancer unit . Front. Psychol. 2 :339 10.3389/fpsyg.2011.00339 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Mehl M. R., Pennebaker J. W., Crow D. M., Dabbs J., Price J. H. (2001). The Electronically Activated Recorder (EAR): a device for sampling naturalistic daily activities and conversations . Behav. Res. Methods Instrum. Comput . 33 , 517–523. 10.3758/BF03195410 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Molina-Azorín J. F., Cameron R. (2015). History and emergent practices of mixed and multiple methods in business research, in The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry , eds Hesse-Biber S., Johnson R. B. (New York, NY: Oxford University Press; ), 466–485. [ Google Scholar ]
Morales-Ortiz M. (1999). La observación indirecta: una aplicación al análisis de textos [Indirect observation applied to text analysis], in Observación en la Escuela: Aplicaciones , ed Anguera M. T. (Barcelona: Ediciones Universidad de Barcelona; ), 299–307. [ Google Scholar ]
Morales-Sánchez V., Pérez-López V., Anguera M. T. (2014). Tratamiento metodológico de la observación indirecta en la gestión de organizaciones deportivas [Indirect observational methodology in managing sports services] . Rev. Psicol. Deport . 23 , 201–207. [ Google Scholar ]
Morillo J. P., Reigal R. E., Hernández-Mendo A., Montaña A., Morales-Sánchez V. (2017). Decision-making by handball referees: design of an ad hoc observation instrument and polar coordinate analysis . Front. Psychol. 8 :1842. 10.3389/fpsyg.2017.01842 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Mucchielli R. (1974). L'observation Psychologique et Psychosociologique [Psychological and Psychosociological Observation] . Paris: E.S.F. [ Google Scholar ]
O'Cathain A., Murphy E., Nicholl J. (2010). Three techniques for integrating data in mixed methods studies . BMJ 341 :c4587. 10.1136/bmj.c4587 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Onwuegbuzie A. J. (2003). Effect sizes in qualitative research: a prolegomenon . Qual. Quant . 37 , 393–409. 10.1023/A:1027379223537 [ CrossRef ] [ Google Scholar ]
Onwuegbuzie A. J., Hitchcock J. H. (2015). Advanced mixed analysis approaches, in The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry , eds Hesse-Biber S., Johnson R. B. (New York, NY: Oxford University Press; ), 275–295. [ Google Scholar ]
Poitras J., Hill K., Hamel V., Pelletier F. B. (2015). Managerial mediation competency: a mixed-method study . Negot. J . 31 , 105–115. 10.1111/nejo.12085 [ CrossRef ] [ Google Scholar ]
Pope C., Ziebland S., Mays N. (2000). Qualitative research in health care. Analysing qualitative data . BMJ 320 , 114–116. 10.1136/bmj.320.7227.114 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Portell M., Anguera M. T., Chacón-Moscoso S., Sanduvete-Chaves S. (2015a). Guidelines for reporting evaluations based on observational methodology . Psicothema 27 , 283–289. 10.7334/psicothema2014.276 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Portell M., Anguera M. T., Hernández-Mendo A., Jonsson G. K. (2015b). Quantifying biopsychosocial aspects in everyday contexts: an integrative methodological approach from the behavioral sciences . Psychol. Res. Behav. Manag . 8 , 153–160. 10.2147/PRBM.S82417 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Portell M., Anguera M. T., Hernández-Mendo A., Jonsson G. K. (2015c). The legacy of Brunswik's representative design in the 21st century: methodological innovations for studying everyday life . Brunswik Soc. Newslett. 30 , 38–40. [ Google Scholar ]
Poyatos F. (1993). Paralanguage. A Linguistic and Interdisciplinary Approach to Interactive Speech and Sound. Amsterdam: John Benjamins Publishing Company. [ Google Scholar ]
Radzikowski J., Stefanidis A., Jacobsen K. H., Croitoru A., Crooks A., Delamater P. L. (2016). The measles vaccination narrative in Twiter: a quantitative analysis . JMIR Public Health Surveill . 2 :e1. 10.2196/publichealth.5059 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Reis H. T. (2013). Why researchers should think “Real-World.” A conceptual rationale, in Handbook of Research Methods for Studying Daily Life , eds Mehl M. R., Conner T. S. (New York, NY: The Guilford Press; ), 3–21. [ Google Scholar ]
Riera A., Ocasio A., Tiyyagura G., Krumeich L., Ragins K., Thomas A., et al.. (2015). Latino caregiver experiencies with asthma health communication . Qual. Health Res . 25 , 16–26. 10.1177/1049732314549474 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Riessman C. K. (2015). Ruptures and sutures: time, audience and identity in an illness narrative . Sociol. Health Illn . 37 , 1055–1071. 10.1111/1467-9566.12281 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Riva G. Anguera M. T. Wiederhold B. K. Mantovani F. (eds.). (2006). From Communication to Presence. Cognition, Emotions and Culture Towards the Ultimate Communicative Experience. Amsterdam: IOS Press. [ Google Scholar ]
Rodriguez C. S., Spring H. J., Rowe M. (2014). Nurse's experiences of communicating with hospitalized, suddenly speechless patients . Qual. Health. Res . 25 , 168–178. 10.1177/1049732314550206 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Romero P., Cox R., du Boulay B., Lutz R., Bryant S. (2007). A methodology for the capture and analysis of hybrid data. A case study of program debugging . Behav. Res. Methods 39 , 309–317. 10.3758/BF03193162 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Roustan M., Izquierdo Rodríguez C., Anguera Argilaga M. T. (2013). Sequential analysis of an interactive peer support group . Psicothema 25 , 396–401. 10.7334/psicothema2012.93 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Russ J. B., Gur R. C., Bilker W. B. (2008). Validation of affective and neutral sentence content for prosodic testing . Behav. Res. Methods 40 , 935–939. 10.3758/BRM.40.4.935 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Sackett G. P. (1980). Lag sequential analysis as a data reduction technique in social interaction research, in Exceptional Infant. Psychosocial Risks in Infant-Environment Transactions , eds Sawin D. B., Hawkins R. C., Walker L. O., Penticuff J. H. (New York, NY: Brunner/Mazel; ), 300–340. [ Google Scholar ]
Sackett G. P. (1987). Analysis of sequential social interaction data: some issues, recent developments, and a causal inference model, in Handbook of Infant Development, 2nd Edn , ed Osofsky J. D. (New York, NY: Wiley; ), 855–878. [ Google Scholar ]
Saldaña J. (2013). The Coding Manual for Qualitative Researchers . Los Angeles, CA: Sage. [ Google Scholar ]
Sánchez-Algarra P., Anguera M. T. (2013). Qualitative/quantitative integration in the inductive observational study of interactive behaviour: impact of recording and coding among predominating perspectives . Qual. Quant . 47 , 1237–1257. 10.1007/s11135-012-9764-6 [ CrossRef ] [ Google Scholar ]
Sandelowski M. (2001). Real qualitative researchers do not count: the use of numbers in qualitative research . Res. Nurs. Health 24 , 230–240. 10.1002/nur.1025 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Sandelowski M., Voils C. I., Knafl G. (2009). On quantitizing . J. Mix. Methods Res . 3 , 208–222. 10.1177/1558689809334210 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Santoyo C., Jonsson G. K., Anguera M. T., López-López J. A. (2017). Observational analysis of the organization of on-task behavior in the classroom using complementary data analyses . Anal. Psicol. 33 , 497–514. 10.6018/analesps.33.3.271061 [ CrossRef ] [ Google Scholar ]
Sarmento H., Bradley P. S., Anguera M. T., Polido T., Resende R., Campaniço J. (2015). Quantifying the offensive sequences that result in goals in elite futsal matches . J. Sports Sci . 34 , 621–629. 10.1080/02640414.2015.1066024 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Schegloff E. (2000). On granularity . Annu. Rev. Sociol . 26 , 715–720. 10.1146/annurev.soc.26.1.715 [ CrossRef ] [ Google Scholar ]
Seltzer-Kelly D., Westwood S. J., Pena-Guzmán D. M. (2012). A methodological self-study of quantitizing: negotiating meaning and revealing multiplicity . J. Mix. Methods Res . 6 , 258–274. 10.1177/1558689811425798 [ CrossRef ] [ Google Scholar ]
Sidnell J., Stivers T. (eds.). (2013). The Handbook of Conversation Analysis . New York, NY: Wiley and Sons. [ Google Scholar ]
Suárez N., Sánchez-López C. R., Jiménez J. E., Anguera M. T. (2018). Is reading instruction evidence-based? Analyzing teaching practices using T-Patterns . Front. Psychol. 9 :7 10.3389/fpsyg.2018.00007 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Tarragó R., Iglesias X., Lapresa D., Anguera M. T., Ruiz-Sanchís L., Arana J. (2017). Analysis of diachronic relationships in successful and unsuccessful behaviors by world fencing champions using three complementary techniques . Anal. Psicol. 33 , 471–485. 10.6018/analesps.33.3.271041 [ CrossRef ] [ Google Scholar ]
Tashakkori A., Teddlie C. (eds.). (2010). The Sage Handbook of Mixed Methods in Social and Behavioral Research, 2nd Edn . Thousand Oaks, CA: Sage. [ Google Scholar ]
Taylor K., Thorne S., Oliffe J. L. (2015). It's a sentence, not a word. Insights from a keyword analysis in cancer communication . Qual. Health Res. 25 , 110–121. 10.1177/1049732314549606 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Tronchoni H., Izquierdo C., Anguera M. T. (2018). Interacción Participativa en las Clases Magistrales: Fundamentacion y Construcción de un Instrumento de Observación [Participatory Interaction in Lectures: Theoretical Framework and Construction of an Observation Instrument] . Publicaciones. Facultad de Educación y Humanidades del Campus de Melilla.
Tuttas C. A. (2015). Lessons learned using web conference technology for online focus group interviews . Qual. Health Res . 25 , 122–133. 10.1177/1049732314549602 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Vaimberg R. (2010). Psicoterapias Tecnológicamente Mediadas [Technology-Mediated Psychotherapy] . Ph.D. thesis, Universidad de Barcelona, Barcelona. [ Google Scholar ]
Webb E. T., Campbell D. T., Schwartz R. D., Sechrest L., Grove J. B. (1966). Nonreactive Measures in the Social Sciences . Boston, MA: Houghton Mifflin. [ Google Scholar ]
Weick K. E. (1968). Systematic observational methods, in Handbook of Social Psychology, Vol. 2, 2nd Edn. , eds Lindzey G., Aronson E. (Reading, MA: Addison-Wesley; ), 357–451. [ Google Scholar ]
Weick K. E. (1985). Systematic observational methods, in Handbook of Social Psychology, Vol. 1, 3rd Edn. , eds Lindzey G., Aronson E. (Reading, MA: Addison-Wesley; ), 567–634. [ Google Scholar ]
Winniccott D. W. (1979). Realidad y Juego [Reality and Play] . Barcelona: Gedisa. [ Google Scholar ]
Zaros A. A. (2016). Retratos de una comunidad religiosa: sobre la memoria y las fotos familiares de la comunidad armenia en Padua [Portraits of a religious community. About the memory and family photos of Padua's Armenian community] . Rev. Cult. Relig. 10 , 88–106. [ Google Scholar ]

IMAGES

Direct Observation (A Research Method)
types of observation case study
Direct Observation
[PDF] Case Study Observational Research: A Framework for Conducting
Discuss the types and disadvantages of direct observation
Direct observation template

VIDEO

##Let's study||Direct Indirect speech||Change of Tense & helping verbs by RG CLASSES 👍
A priori Meaning in English
OBSERVATION FILE ( कक्षा-अवलोकन) Of B.E.D Observation-1 / Classroom observation [email protected]_info
Difference between observational studies and randomized experiments?
16 III) Observational studies (2) Case control study
Advantages and Disadvantages of Direct Observation

COMMENTS

Direct observation methods: A practical guide for health researchers
This article serves as a guide for making key decisions in studies involving direct observation. Study development begins with determining if observation methods are warranted or feasible. ... Ethnography (RAPICE) in Pragmatic clinical trials of mental health services implementation: methods and applied case study. Admin Pol Ment Health, 46 (2 ...
Direct observation methods: A practical guide for health researchers
Health research study designs benefit from observations of behaviors and contexts. •. Direct observation methods have a long history in the social sciences. •. Social science approaches should be adapted for health researchers' unique needs. •. Health research observations should be feasible, well-defined and piloted.
PDF Case Study Observational Research: A Framework for Conducting Case
the accuracy and completeness of the case study, strength-ening the credibility of the research findings (Cronin, 2014; Yin, 2014). Sources of data collected vary depending on the research question. Commonly used methods include interviews, observation of archival records, and direct observation of study participants (Yin, 1994).
Case Study Observational Research: A Framework for Conducting Case
Observation methods have the potential to reach beyond other methods that rely largely or solely on self-report. This article describes the distinctive characteristics of case study observational research, a modified form of Yin's 2014 model of case study research the authors used in a study exploring interprofessional collaboration in primary ...
What Is an Observational Study?
Revised on June 22, 2023. An observational study is used to answer a research question based purely on what the researcher observes. There is no interference or manipulation of the research subjects, and no control and treatment groups. These studies are often qualitative in nature and can be used for both exploratory and explanatory research ...
Direct Observation
Direct observation, also known as observational study, is a method of collecting evaluative information in which the evaluator watches the subject in his or her usual environment without altering that environment. Direct observation is used when other data collection procedures, such as surveys, questionnaires, etc., are not effective; when the ...
Direct Observation Methods: a Practical Guide for Health ...
Objective: To provide health research teams with a practical, methodologically rigorous guide on how to conduct direct observation. Methods: Synthesis of authors' observation-based teaching and research experiences in social sciences and health services research. Results: This article serves as a guide for making key decisions in studies involving direct observation.
Study Design, Precision, and Validity in Observational Studies
Observational research involves the direct observation of individuals in their natural setting. As such, who does or does not receive an intervention is determined by individual preferences, practice patterns, ... For case-control studies, the general concern is that cases will be more likely than controls to recall past exposures because they ...
Guide: Designing and Conducting Case Studies
Designing and Conducting Case Studies. This guide examines case studies, a form of qualitative descriptive research that is used to look at individuals, a small group of participants, or a group as a whole. Researchers collect data about participants using participant and direct observations, interviews, protocols, tests, examinations of ...
Framework for direct observation of performance and safety in
Any direct observation study needs to ensure that the observer has the appropriate support to address the range of personal and ethical challenges that they will face. Encouragingly, a growing cadre of publications and experienced observers are available for support that was not available a decade ago. 40 , 41
What Is Qualitative Observation?
Qualitative observation is a type of observational study, often used in conjunction with other types of research through triangulation. It is often used in fields like social sciences, education, healthcare, marketing, and design. This type of study is especially well suited for gaining rich and detailed insights into complex and/or subjective ...
Observation Methods: Naturalistic, Participant and Controlled
Controlled observations are also usually non-participant as the researcher avoids direct contact with the group and keeps a distance (e.g., observing behind a two-way mirror). ... Like case studies, naturalistic observation is often used to generate new ideas. Because it gives the researcher the opportunity to study the total situation, it ...
The Qualitative Report
aspect of deciding if case study design is a fit is to ascertain whether sufficient data is obtainable to support findings and conclusions. Six sources of data typically are used in qualitative case studies: documentation, archival records, interviews, direct observation, participant observation, and physical artifacts
Case Study Methodology of Qualitative Research: Key Attributes and
A case study is one of the most commonly used methodologies of social research. This article attempts to look into the various dimensions of a case study research strategy, the different epistemological strands which determine the particular case study type and approach adopted in the field, discusses the factors which can enhance the effectiveness of a case study research, and the debate ...
Direct Observation and Ethnography
Abstract. Direct observation or ethnography is a qualitative method that consists in directly observing the social situation under study - for example, the implementation of a public policy - implying a physical presence of the researcher in the situation at hand. It is a demanding method in terms of the commitment it requires (long-term ...
How to observe business operations: An empirical study of family ...
The direct observation method is commonly used for data collection in family business case studies. Nevertheless, in research on family business succession that is mainly based on retrospective data, it is difficult to directly observe an event or situation at a given time. This paper thus first explores the application of observation data in the published research results of some family ...
Observe Before You Leap: Why Observation Provides Critical Insights for
Four case studies show how observation can uncover issues critical to making a health intervention succeed or, sometimes, reveal reasons why it is likely to fail. ... Ram R, Gnywali T. Estimating reactivity in direct observation studies of health behaviors. Hum Organ. 1997; 56 (2):182-189. 10.17730/humo.56.2.c7x0532q2u86m207 [Google Scholar] 79.
What is Direct Observation?
In choosing between direct observation, participant observation, immersion, or any form of field research in between, the choice ultimately comes down to the research situation. Different situations require different roles for the researcher. While one setting might call for direct observation, another might be better with immersion.
Direct Observation Definition, Steps & Examples
Direct observation is a method of collecting data in which a researcher watches or listens to research participants rather than conducting an interview or manipulating variables through ...
Benefits of direct observation in medication administration to ...
In this study, investigators observed several potential sources of medication administration errors at a university hospital. The authors outline specific strategies for change that are currently being piloted on their ward in response to their findings. Diaz-Navarlaz T, Pronovost P, Beortegui E, et al. Benefits of Direct Observation in ...
Distinguishing case study as a research method from case reports as a
Case studies should also provide multiple sources of data, a case study database, and a clear chain of evidence among the questions asked, the data collected, and the conclusions drawn . Sources of evidence for case studies include interviews, documentation, archival records, direct observations, participant-observation, and physical artifacts.
Electronic Directly Observed Therapy for Active TB Disease
Directly observed therapy (DOT) is a key element of TB case management. Endorsed by the World Health Organization and CDC, DOT entails a trained observer watching as the patient swallows the anti-TB medications. As the percentage of the population that owns a mobile phone, computer, or tablet with video capability has increased, potential for ...
Indirect Observation in Everyday Contexts: Concepts and Methodological
Below are examples of dimensions and theoretical frameworks used in three indirect observation studies. In the first case, a study of disruptive behavior and communication difficulties in adolescents participating in group communication therapy, Arias-Pujol and Anguera proposed the dimensions verbal and non-verbal behavior, derived from the ...

Direct Observation

Buying options

References and Readings

Author information

Corresponding author

Editor information

Rights and permissions

Copyright information

About this entry

Download citation

Share this entry

Direct Observation Methods: a Practical Guide for Health Researchers

Grants and funding

Designing and Conducting Case Studies

Definition and Overview

Educational Applications

Types and Design Concerns

Types of Case Studies

Identifying a Theoretical Perspective

Designing a Case Study

Conducting Case Studies

Method: Single or Multi-modal?

Participant Selection

Data Collection

Data Analysis

Composing the Case Study Report

Issues of Validity and Reliability

Commentary on Case Studies

Strengths and Weaknesses of Case Studies

Concerns about Reliability, Validity, and Generalizability

Annotated Bibliography

Related Links

Observation Method in Psychology: Naturalistic, Participant and Controlled

Controlled Observation

Limitations

Naturalistic Observation

Collecting Naturalistic Behavioral Data

Participant Observation

Recording of Data

Coding Systems

Macroanalytic coding systems

Microanalytic coding systems

Mesoanalytic coding systems

Preventing Coder Drift

Reducing Observer Bias

Coding Manual

Coder Training

Blind Methods

9 Direct Observation and Ethnography

I. What does this method consist of?

II. How is this method useful for policy evaluation?

III. An example of the use of this method: the evaluation of the reception policy in public services

IV. What are the criteria for judging the quality of the mobilisation of this method?

V. What are the strengths and limitations of this method compared to others?

Some bibliographical references to go further

Share This Book

How to observe business operations: An empirical study of family business

Introduction

Literature review

Observation study execution procedure

Literature on observation studies

Materials and methods

Theoretical sampling

Data collection

Site selection.

Obtaining support.

Preparation in advance.

Starting observation.

Observation records.

Extended observation.

Observation guide

Research ethics

Results: Combining observations with other data

Succession stage

Internal social capital: Family human resources

External social capital: Supplier relationships

External social capital: Industry social network

Discussion and conclusions

Conclusions

Supporting information