study research difference

Research vs. Study

What's the difference.

Research and study are two essential components of the learning process, but they differ in their approach and purpose. Research involves a systematic investigation of a particular topic or issue, aiming to discover new knowledge or validate existing theories. It often involves collecting and analyzing data, conducting experiments, and drawing conclusions. On the other hand, study refers to the process of acquiring knowledge or understanding through reading, memorizing, and reviewing information. It is typically focused on a specific subject or discipline and aims to deepen one's understanding or mastery of that subject. While research is more exploratory and investigative, study is more focused on acquiring and retaining information. Both research and study are crucial for intellectual growth and expanding our knowledge base.

Further Detail

Introduction.

Research and study are two fundamental activities that play a crucial role in acquiring knowledge and understanding. While they share similarities, they also have distinct attributes that set them apart. In this article, we will explore the characteristics of research and study, highlighting their differences and similarities.

Definition and Purpose

Research is a systematic investigation aimed at discovering new knowledge, expanding existing knowledge, or solving specific problems. It involves gathering and analyzing data, formulating hypotheses, and drawing conclusions based on evidence. Research is often conducted in a structured and scientific manner, employing various methodologies and techniques.

On the other hand, study refers to the process of acquiring knowledge through reading, memorizing, and understanding information. It involves examining and learning from existing materials, such as textbooks, articles, or lectures. The purpose of study is to gain a comprehensive understanding of a particular subject or topic.

Approach and Methodology

Research typically follows a systematic approach, involving the formulation of research questions or hypotheses, designing experiments or surveys, collecting and analyzing data, and drawing conclusions. It often requires a rigorous methodology, including literature review, data collection, statistical analysis, and peer review. Research can be qualitative or quantitative, depending on the nature of the investigation.

Study, on the other hand, does not necessarily follow a specific methodology. It can be more flexible and personalized, allowing individuals to choose their own approach to learning. Study often involves reading and analyzing existing materials, taking notes, summarizing information, and engaging in discussions or self-reflection. While study can be structured, it is generally less formalized compared to research.

Scope and Depth

Research tends to have a broader scope and aims to contribute to the overall body of knowledge in a particular field. It often involves exploring new areas, pushing boundaries, and generating original insights. Research can be interdisciplinary, involving multiple disciplines and perspectives. The depth of research is often extensive, requiring in-depth analysis, critical thinking, and the ability to synthesize complex information.

Study, on the other hand, is usually more focused and specific. It aims to gain a comprehensive understanding of a particular subject or topic within an existing body of knowledge. Study can be deep and detailed, but it is often limited to the available resources and materials. While study may not contribute directly to the advancement of knowledge, it plays a crucial role in building a solid foundation of understanding.

Application and Output

Research is often driven by the desire to solve real-world problems or contribute to practical applications. The output of research can take various forms, including scientific papers, patents, policy recommendations, or technological advancements. Research findings are typically shared with the academic community and the public, aiming to advance knowledge and improve society.

Study, on the other hand, focuses more on personal development and learning. The application of study is often seen in academic settings, where individuals acquire knowledge to excel in their studies or careers. The output of study is usually reflected in improved understanding, enhanced critical thinking skills, and the ability to apply knowledge in practical situations.

Limitations and Challenges

Research faces several challenges, including limited resources, time constraints, ethical considerations, and the potential for bias. Conducting research requires careful planning, data collection, and analysis, which can be time-consuming and costly. Researchers must also navigate ethical guidelines and ensure the validity and reliability of their findings.

Study, on the other hand, may face challenges such as information overload, lack of motivation, or difficulty in finding reliable sources. It requires self-discipline, time management, and the ability to filter and prioritize information. Without proper guidance or structure, study can sometimes lead to superficial understanding or misconceptions.

In conclusion, research and study are both essential activities in the pursuit of knowledge and understanding. While research focuses on generating new knowledge and solving problems through a systematic approach, study aims to acquire and comprehend existing information. Research tends to be more formalized, rigorous, and contributes to the advancement of knowledge, while study is often more flexible, personalized, and focused on individual learning. Both research and study have their unique attributes and challenges, but together they form the foundation for intellectual growth and development.

Comparisons may contain inaccurate information about people, places, or facts. Please report any issues.

Study vs. Research — What's the Difference?

Difference Between Study and Research

Table of contents, key differences, comparison chart, methodology, compare with definitions, common curiosities, what is the main difference between study and research, is study less rigorous than research, can you research without studying, what's the goal of research, is research always scientific, can you study and research at the same time, what's the goal of study, is study always academic, who typically conducts research, can you study by yourself, who typically studies, is research always formal, do research findings affect general populations, is research always published, can study be informal, share your discovery.

Author Spotlight

Popular Comparisons

Trending Comparisons

New Comparisons

Trending Terms

•      Language Editing     For Manuscripts    For Response Letter new    For LaTeX    For Annual Review and Tenure    For Books new
•      Scientific Editing     For Manuscripts    For Response Letter new
•      Grant Editing 
•      Translation 
• Publication Support  Journal Recommendation  Manuscript Formatting  Figure Formatting  Data Analysis new  Plagiarism Check  Conference Poster  Plain Language Summary
• Scientific Illustration  Journal Cover Design  Graphical Abstract  Infographic  Custom Illustration
• Scientific Videos  Video Abstract  Explainer Video  Scientific Animation
• Ethics and Confidentiality
• Editorial Certificate
• Testimonials
• Design Gallery
• Institutional Provider
• Publisher Portal
• Brand Localization
• Journal Selector Tool
• Learning Nexus

Research vs. study

The confusion about these words is that they can both be either nouns or verbs. If you ask someone, "Does 'studies' mean the same as 'researches'?" you may hear "Yes," but it is only true if they are used as verbs. As nouns, they have subtly different meanings.

"This team has done a lot of good research. I just read their latest study, which they wrote about calcium in germinating soybeans. It described several interesting experiments."

research 1. to perform a systematic investigation

1. "What kind of scientist is he? He's a botanist. He researches plants."

study 1. to perform a systematic investigation; 2. to actively learn or memorize academic material

1. "What kind of scientist is he? He's a botanist. He studies plants."

2. "Mindy studies every day. That is why she gets such excellent grades. She wants to go to college to study math."

Some authors say "research" when they mean "study." "Research," as a verb, means "to perform a study or studies," but "research" as a noun refers to the sum of many studies. "Chemical research" means the sum of all chemical studies. If you find yourself writing "a research" or "in this research," change it to "a study" or "in this study."

research The act of performing research. Also, the results of research. Note that "research" is a mass noun. It is already plural in meaning but grammatically singular. If you want to indicate more than one type, say "bodies of research" or "pieces of research," not "researches."

"Dr. Lee was a prolific scientist. She performed a great deal of research over her long career."

study A single research project or paper.

"Dr. Lee was a prolific scientist. She performed a great many studies over her long career."

The noun "study" refers to a single paper or project. You can replace "paper" with "study" in almost all cases (but not always the other way around), to the point where you can say "I wrote a study." The noun "research" means more like a whole body of research including many individual studies: The research of a field. The lifetime achievements of a scientist or research team.

Quick Links

• Language Editing Service
• Expert Scientific Editing Service
• Professional Translation Service
• Formatting Service
• Figure Preparation Service

Intentional Space Tag

Contact us  

Your name *

Your email *

Your message *

Please fill in all fields and provide a valid email.

© 2010-2024  ACCDON LLC 400 5 th Ave, Suite 530, Waltham, MA 02451, USA Privacy • Terms of Service

© 2010-2024 United States: ACCDON LLC Tel: 1-781-202-9968 Email: [email protected]

Address: 400 5 th Ave, Suite 530, Waltham, Massachusetts 02451, United States

• En español – ExME
• Em português – EME

An introduction to different types of study design

Posted on 6th April 2021 by Hadi Abbas

Study designs are the set of methods and procedures used to collect and analyze data in a study.

Broadly speaking, there are 2 types of study designs: descriptive studies and analytical studies.

Descriptive studies

• Describes specific characteristics in a population of interest
• The most common forms are case reports and case series
• In a case report, we discuss our experience with the patient’s symptoms, signs, diagnosis, and treatment
• In a case series, several patients with similar experiences are grouped.

Analytical Studies

Analytical studies are of 2 types: observational and experimental.

Observational studies are studies that we conduct without any intervention or experiment. In those studies, we purely observe the outcomes.  On the other hand, in experimental studies, we conduct experiments and interventions.

Observational studies

Observational studies include many subtypes. Below, I will discuss the most common designs.

Cross-sectional study:

• This design is transverse where we take a specific sample at a specific time without any follow-up
• It allows us to calculate the frequency of disease ( p revalence ) or the frequency of a risk factor
• This design is easy to conduct
• For example – if we want to know the prevalence of migraine in a population, we can conduct a cross-sectional study whereby we take a sample from the population and calculate the number of patients with migraine headaches.

Cohort study:

• We conduct this study by comparing two samples from the population: one sample with a risk factor while the other lacks this risk factor
• It shows us the risk of developing the disease in individuals with the risk factor compared to those without the risk factor ( RR = relative risk )
• Prospective : we follow the individuals in the future to know who will develop the disease
• Retrospective : we look to the past to know who developed the disease (e.g. using medical records)
• This design is the strongest among the observational studies
• For example – to find out the relative risk of developing chronic obstructive pulmonary disease (COPD) among smokers, we take a sample including smokers and non-smokers. Then, we calculate the number of individuals with COPD among both.

Case-Control Study:

• We conduct this study by comparing 2 groups: one group with the disease (cases) and another group without the disease (controls)
• This design is always retrospective
•  We aim to find out the odds of having a risk factor or an exposure if an individual has a specific disease (Odds ratio)
•  Relatively easy to conduct
• For example – we want to study the odds of being a smoker among hypertensive patients compared to normotensive ones. To do so, we choose a group of patients diagnosed with hypertension and another group that serves as the control (normal blood pressure). Then we study their smoking history to find out if there is a correlation.

Experimental Studies

• Also known as interventional studies
• Can involve animals and humans
• Pre-clinical trials involve animals
• Clinical trials are experimental studies involving humans
• In clinical trials, we study the effect of an intervention compared to another intervention or placebo. As an example, I have listed the four phases of a drug trial:

I:  We aim to assess the safety of the drug ( is it safe ? )

II: We aim to assess the efficacy of the drug ( does it work ? )

III: We want to know if this drug is better than the old treatment ( is it better ? )

IV: We follow-up to detect long-term side effects ( can it stay in the market ? )

• In randomized controlled trials, one group of participants receives the control, while the other receives the tested drug/intervention. Those studies are the best way to evaluate the efficacy of a treatment.

Finally, the figure below will help you with your understanding of different types of study designs.

References (pdf)

You may also be interested in the following blogs for further reading:

An introduction to randomized controlled trials

Case-control and cohort studies: a brief overview

Cohort studies: prospective and retrospective designs

Prevalence vs Incidence: what is the difference?

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

No Comments on An introduction to different types of study design

you are amazing one!! if I get you I’m working with you! I’m student from Ethiopian higher education. health sciences student

Very informative and easy understandable

You are my kind of doctor. Do not lose sight of your objective.

Wow very erll explained and easy to understand

I’m Khamisu Habibu community health officer student from Abubakar Tafawa Balewa university teaching hospital Bauchi, Nigeria, I really appreciate your write up and you have make it clear for the learner. thank you

well understood,thank you so much

Well understood…thanks

Simply explained. Thank You.

Thanks a lot for this nice informative article which help me to understand different study designs that I felt difficult before

That’s lovely to hear, Mona, thank you for letting the author know how useful this was. If there are any other particular topics you think would be useful to you, and are not already on the website, please do let us know.

it is very informative and useful.

thank you statistician

Fabulous to hear, thank you John.

Thanks for this information

Thanks so much for this information….I have clearly known the types of study design Thanks

That’s so good to hear, Mirembe, thank you for letting the author know.

Very helpful article!! U have simplified everything for easy understanding

I’m a health science major currently taking statistics for health care workers…this is a challenging class…thanks for the simified feedback.

That’s good to hear this has helped you. Hopefully you will find some of the other blogs useful too. If you see any topics that are missing from the website, please do let us know!

Hello. I liked your presentation, the fact that you ranked them clearly is very helpful to understand for people like me who is a novelist researcher. However, I was expecting to read much more about the Experimental studies. So please direct me if you already have or will one day. Thank you

Dear Ay. My sincere apologies for not responding to your comment sooner. You may find it useful to filter the blogs by the topic of ‘Study design and research methods’ – here is a link to that filter: https://s4be.cochrane.org/blog/topic/study-design/ This will cover more detail about experimental studies. Or have a look on our library page for further resources there – you’ll find that on the ‘Resources’ drop down from the home page.

However, if there are specific things you feel you would like to learn about experimental studies, that are missing from the website, it would be great if you could let me know too. Thank you, and best of luck. Emma

Great job Mr Hadi. I advise you to prepare and study for the Australian Medical Board Exams as soon as you finish your undergrad study in Lebanon. Good luck and hope we can meet sometime in the future. Regards ;)

You have give a good explaination of what am looking for. However, references am not sure of where to get them from.

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.

Related Articles

Cluster Randomized Trials: Concepts

This blog summarizes the concepts of cluster randomization, and the logistical and statistical considerations while designing a cluster randomized controlled trial.

Expertise-based Randomized Controlled Trials

This blog summarizes the concepts of Expertise-based randomized controlled trials with a focus on the advantages and challenges associated with this type of study.

A well-designed cohort study can provide powerful results. This blog introduces prospective and retrospective cohort studies, discussing the advantages, disadvantages and use of these type of study designs.

Designing Difference in Difference Studies: Best Practices for Public Health Policy Research

Affiliations.

• 1 School of Public and Environmental Affairs, Indiana University, Bloomington, Indiana 47405, USA; email: [email protected] , [email protected].
• 2 School of Public and Environmental Affairs, Indiana University, Bloomington, Indiana 47405, USA, and National Bureau of Economic Research; email: [email protected].
• PMID: 29328877
• DOI: 10.1146/annurev-publhealth-040617-013507

The difference in difference (DID) design is a quasi-experimental research design that researchers often use to study causal relationships in public health settings where randomized controlled trials (RCTs) are infeasible or unethical. However, causal inference poses many challenges in DID designs. In this article, we review key features of DID designs with an emphasis on public health policy research. Contemporary researchers should take an active approach to the design of DID studies, seeking to construct comparison groups, sensitivity analyses, and robustness checks that help validate the method's assumptions. We explain the key assumptions of the design and discuss analytic tactics, supplementary analysis, and approaches to statistical inference that are often important in applied research. The DID design is not a perfect substitute for randomized experiments, but it often represents a feasible way to learn about casual relationships. We conclude by noting that combining elements from multiple quasi-experimental techniques may be important in the next wave of innovations to the DID approach.

Keywords: causal inference; difference in difference; policy analysis; quasi-experiments; research design.

• Data Interpretation, Statistical
• Health Policy*
• Policy Making*
• Public Health*
• Public Policy
• Research Design*

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

• University Libraries
• Research Guides
• Topic Guides
• Research Methods Guide
• Research Design & Method

Research Methods Guide: Research Design & Method

• Introduction
• Survey Research
• Interview Research
• Data Analysis
• Resources & Consultation

Tutorial Videos: Research Design & Method

Research Methods (sociology-focused)

Qualitative vs. Quantitative Methods (intro)

Qualitative vs. Quantitative Methods (advanced)

FAQ: Research Design & Method

What is the difference between Research Design and Research Method?

Research design is a plan to answer your research question.  A research method is a strategy used to implement that plan.  Research design and methods are different but closely related, because good research design ensures that the data you obtain will help you answer your research question more effectively.

Which research method should I choose ?

It depends on your research goal.  It depends on what subjects (and who) you want to study.  Let's say you are interested in studying what makes people happy, or why some students are more conscious about recycling on campus.  To answer these questions, you need to make a decision about how to collect your data.  Most frequently used methods include:

• Observation / Participant Observation
• Focus Groups
• Experiments
• Secondary Data Analysis / Archival Study
• Mixed Methods (combination of some of the above)

One particular method could be better suited to your research goal than others, because the data you collect from different methods will be different in quality and quantity.   For instance, surveys are usually designed to produce relatively short answers, rather than the extensive responses expected in qualitative interviews.

What other factors should I consider when choosing one method over another?

Time for data collection and analysis is something you want to consider.  An observation or interview method, so-called qualitative approach, helps you collect richer information, but it takes time.  Using a survey helps you collect more data quickly, yet it may lack details.  So, you will need to consider the time you have for research and the balance between strengths and weaknesses associated with each method (e.g., qualitative vs. quantitative).

• << Previous: Introduction
• Next: Survey Research >>
• Last Updated: Aug 21, 2023 10:42 AM

A Primer for Interpreting and Designing Difference-in-Differences Studies in Higher Education Research

• Reference work entry
• First Online: 26 April 2020
• Cite this reference work entry

• Fernando Furquim 3 ,
• Daniel Corral 4 &
• Nicholas Hillman 4  

Part of the book series: Higher Education: Handbook of Theory and Research ((HATR,volume 35))

2117 Accesses

14 Citations

Though randomized control trials continue to serve as the “gold standard” of causal inference, they are neither feasible nor desirable in numerous instances. Even in the absence of randomized trials, higher education researchers have at their disposal several statistical tools for estimating causal relationships. One such method is difference-in-differences, a powerful and intuitive approach to causal evaluation that exploits variation in the timing and coverage of policies. The method lends itself well to studying higher education policies and initiatives, as these frequently diffuse over time and across space in ways that may permit for causal inference. Difference-in-differences has become one of the most widely used methods for causal inference in higher education research. We use this chapter to introduce new researchers to this method with an overview of difference-in-differences models, common threats to their validity, and robustness checks. We then present extensions of the method, including event study models and variation in treatment timing. We illustrate these methods throughout the chapter by analyzing the effect of hurricanes on enrollment at affected colleges using data from the Integrated Postsecondary Education Data System and provide Stata code for replication of the analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105 (490), 493–505.

Article   Google Scholar  

Andrews, R. J., DesJardins, S., & Ranchhod, V. (2010). The effects of the Kalamazoo promise on college choice. Economics of Education Review, 29 (5), 722–737.

Angrist, J. (2004). American education research changes tack. Oxford Review of Economic Policy, 20 (2), 198–212.

Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist's companion . Princeton: Princeton University Press.

Book   Google Scholar  

Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91 (434), 444–455.

Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2019). Synthetic difference in differences (No. w25532). National Bureau of Economic Research.

Google Scholar  

Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. The Review of Economics and Statistics, 60 (1), 47–57.

Athey, S., & Imbens, G. W. (2018). Design-based analysis in difference-in-differences settings with staggered adoption (No. w24963). National Bureau of Economic Research.

Baker, D. J. (2019). Pathways to racial equity in higher education: Modeling the antecedents of state affirmative action bans. American Educational Research Journal . Advance online publication. https://doi.org/10.3102/0002831219833918 .

Belasco, A. S., Rosinger, K. O., & Hearn, J. C. (2015). The test-optional movement at America’s selective liberal arts colleges: A boon for equity or something else? Educational Evaluation and Policy Analysis, 37 (2), 206–223.

Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in- differences estimates? The Quarterly Journal of Economics, 119 (1), 249–275.

Betts, J. R., & McFarland, L. L. (1995). Safe port in a storm: The impact of labor market conditions on community college enrollments. Journal of Human Resources, 30 (4), 741–765.

Bielby, R. M., House, E., Flaster, A., & DesJardins, S. L. (2013). Instrumental variables: Conceptual issues and an application considering high school course taking. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXVIII, pp. 263–321). Dordrecht: Springer.

Chapter   Google Scholar  

Brewer, M., Crossley, T. F., & Joyce, R. (2018). Inference with difference-in-differences revisited. Journal of Econometric Methods, 7 (1), 1–16.

Bumphus, W. G., & Royal, A. M. (2007). In our own words: Disaster recovery post-Katrina. In A. W. Villadsen & G. E. De los Santos (Eds.), Hoping for the best while preparing for the worst: Disasters, emergencies and the community college (pp. 24–30). Phoenix: League for Innovation in the Community College.

Charles, K., Hurst, E., & Notowidigdo, M. (2018). Housing booms and busts, labor market opportunities, and college attendance. American Economic Review, 108 (10), 2947–2994.

Conley, T. G., & Taber, C. R. (2011). Inference with “`difference in differences” with a small number of policy changes. The Review of Economics and Statistics, 93 (1), 113–125.

Cortes, K. E. (2010). Do bans on affirmative action hurt minority students? Evidence from the Texas Top 10% Plan. Economics of Education Review, 29 (6), 1110–1124.

Dache-Gerbino, A. (2018). College desert and oasis: A critical geographic analysis of local college access. Journal of Diversity in Higher Education, 11 (2), 97–116.

Daw, J. R., & Hatfield, L. A. (2018). Matching and regression to the mean in difference-in-differences analysis. Health Services Research, 53 (6), 4138–4156.

Deryugina, T., Kawano, L., & Levitt, S. (2018). The economic impact of Hurricane Katrina on its victims: Evidence from individual tax returns. American Economic Journal: Applied Economics, 10 (2), 202–233.

DeWaard, J., Curtis, K., & Fussell, E. (2016). Population recovery in New Orleans after Hurricane Katrina: Exploring the potential role of stage migration in migration systems. Population and Environment, 37 (4), 449–463.

Dynarski, S. M. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review, 93 (1), 279–288.

Foote, A., & Grosz, M. (2019). The effect of local labor market downturns on postsecondary enrollment and program choice. Education Finance and Policy . Advance online publication.

Freyaldenhoven, S., Hansen, C., & Shapiro, J. M. (2018). Pre-event trends in the panel event-study design (No. w24565). National Bureau of Economic Research.

Furquim, F., & Glasener, K. M. (2017). A quest for equity? Measuring the effect of QuestBridge on economic diversity at selective institutions. Research in Higher Education, 58 (6), 646–671.

Furuta, J. (2017). Rationalization and student/school personhood in US college admissions: The rise of test-optional policies, 1987 to 2015. Sociology of Education, 90 (3), 236–254.

Fussell, E. (2015). The long-term recovery of New Orleans’ population after Hurricane Katrina. American Behavioral Scientist, 59 (10), 1231–1245.

Galea, S., Brewin, C., Gruber, M., Jones, R., King, D., King, L., McNally, R., Ursano, R., Petukhova, M., & Kessler, R. (2007). Exposure to hurricane-related stressors and mental illness after Hurricane Katrina. Archives of General Psychiatry, 64 (12), 1427–1434.

Garces, L. M. (2013). Understanding the impact of affirmative action bans in different graduate fields of study. American Educational Research Journal, 50 (2), 251–284.

Garces, L. M., & Mickey-Pabello, D. (2015). Racial diversity in the medical profession: The impact of affirmative action bans on underrepresented student of color matriculation in medical schools. The Journal of Higher Education, 86 (2), 264–294.

Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. (2016). Impact evaluation in practice . Washington, DC: The World Bank.

Goodman-Bacon, A. (2018). Difference-in-differences with variation in treatment timing (No. w25018). National Bureau of Economic Research.

Groen, J., & Polivka, A. (2008). The effect of Hurricane Katrina on the labor market outcomes of evacuees. American Economic Review, 98 (2), 43–48.

Heckman, J. J., & Smith, J. A. (1999). The pre-programme earnings dip and the determinants of participation in a social programme: Implications for simple programme evaluation strategies. The Economic Journal, 109 (457), 313–348.

Heller, D. E. (1999). The effects of tuition and state financial aid on public college enrollment. The Review of Higher Education, 23 (1), 65–89.

Hemelt, S., & Marcotte, D. (2011). The impact of tuition increases on enrollment at public colleges and universities. Educational Evaluation and Policy Analysis, 33 (4), 435–457.

Hillman, N. W. (2017). Geospatial analysis in higher education research. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. XXXII, pp. 529–575).

Hillman, N. W., & Orians, E. L. (2013). Community colleges and labor market conditions: How does enrollment demand change relative to local unemployment rates? Research in Higher Education, 54 (7), 765–780.

Hillman, N. W., Tandberg, D. A., & Gross, J. P. K. (2014). Performance funding in higher education: Do financial incentives impact college completions? The Journal of Higher Education, 85 (6), 826–857.

Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37 (4), 501–519.

Hillman, N. W., Fryar, A. H., & Crespín-Trujillo, V. (2018). Evaluating the impact of performance funding in Ohio and Tennessee. American Educational Research Journal, 55 (1), 144–170.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81 (396), 945–960.

Hubbard, D. (2018). The impact of local labor market shocks on college choice: Evidence from plant closings in Michigan. Unpublished manuscript. Retrieved from https://www.aeaweb.org/conference/2019/preliminary/paper/Q67dtN77

Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20 (1), 1–24.

Jaquette, O., Kramer, D. A., & Curs, B. R. (2018). Growing the pie? The effect of responsibility center management on tuition revenue. The Journal of Higher Education, 89 (5), 637–676.

Jarvie, J., & Landsberg, M. (2018, September 17). When a Category None hurricane is worse than a Category 5. Los Angeles Times. Retrieved from https://www.latimes.com

Johnson, G., & Rainey, S. (2007). Hurricane Katrina’s impact on three historically black colleges and universities (HBCUs): Voices from displaced students. Race, Gender & Class, 14 (1), 100–119.

Kahn-Lang, A., & Lang, K. (2019). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications. Journal of Business & Economic Statistics , 1–14.

Kates, R. W., Colten, C. E., Laska, S., & Leatherman, S. P. (2006). Reconstruction of New Orleans after Hurricane Katrina: a research perspective. Proceedings of the National Academy of Sciences, 103 (40), 14653–14660.

Kothari, S. P., & Warner, J. B. (2007). Econometrics of event studies. In B. E. Eckbo (Ed.), Handbook of empirical corporate finance (pp. 3–36). North Holland: Elsevier.

Ladd, A., Gill, D., & Marszalek, J. (2007). Riders from the storm: Narratives of relocated New Orleans college students in the aftermath of Hurricane Katrina. Journal of Public Management & Social Policy, 13 (2), 51–80.

Lafortune, J., Rothstein, J., & Schanzenbach, D. W. (2018). School finance reform and the distribution of student achievement. American Economic Journal: Applied Economics, 10 (2), 1–26.

Lall, R. (2016). How multiple imputation makes a difference. Political Analysis, 24 (4), 414–433.

Lapid, P. (2018). Expanding college access: The impact of new universities on local enrollment. Unpublished manuscript. Retrieved from https://www.ocf.berkeley.edu/~palapid/wp-content/uploads/2019/03/newU-SEA-18_11_05.pdf

Lechner, M. (2010). The estimation of causal effects by difference-in-difference methods. Foundations and Trends in Econometrics, 4 (3), 165–224.

Li, A. Y., & Kennedy, A. I. (2018). Performance funding policy effects on community college outcomes: Are short-term certificates on the rise? Community College Review, 46 (1), 3–39.

Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73 (1), 13–22.

Long, B. T. (2011). The new financial aid policies: Their impact on access and equity for low-income students. In L. M. Stulberg & S. L. Weinberg (Eds.), Diversity in American higher education: Toward a more comprehensive approach . New York: Routledge Books.

Lowe, S., & Rhodes, J. (2012). Community college re-enrollment after Hurricane Katrina. Journal of College Student Retention, 14 (2), 227–249.

Mangan, K., & O’Leary, B. (2010). New Orleans colleges 5 years after Katrina. The Chronicle of Higher Education . https://www.chronicle.com/article/New-Orleans-Colleges-5-Years/124107 . Accessed 29 May 2019.

McCall, B. P., & Bielby, R. M. (2012). Regression discontinuity design: Recent developments and a guide to practice for researchers in higher education. In J. C. Smart & M. B. Paulsen (Eds.), Higher education: Handbook of theory and research (Vol. XXVII, pp. 249–290). Dordrecht: Springer.

Murnane, R., & Willett, J. (2011). Methods matter: Improving causal inference in educational and social science research . Oxford, UK: Oxford University Press.

Ngo, F., & Astudillo, S. (2019). California DREAM: The impact of financial aid for undocumented community college students. Educational Researcher, 48 (1), 5–18.

Reynolds, C. L., & DesJardins, S. L. (2009). The use of matching methods in higher education research: Answering whether attendance at a 2-year institution results in differences in educational attainment. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXIV, pp. 47–97). Dordrecht: Springer.

Rios-Aguilar, C., & Titus, M. (2018). Spatial research in higher education: Expanding understanding and identifying inequities. New Directions in Institutional Research, 2018 (180), 5–10.

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66 (5), 688.

Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100 (469), 322–331.

Sacerdote, B. (2012). When the saints go marching out: Long-term outcomes for student evacuees from Hurricanes Katrina and Rita. American Economic Journal: Applied Economics, 4 (1), 109–135.

Savoie, E. J., O’Neil, R. M., & Rabban, D. M. (2007). Report of an AAUP special committee: Hurricane Katrina and New Orleans universities. Academe, 93 (3), 59–126.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference . Boston: Houghton Mifflin.

Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103 (484), 1334–1344.

St. Clair, T., & Cook, T. D. (2015). Difference-in-differences methods in public finance. National Tax Journal, 68 (2), 319–338.

Stange, K. (2015). Differential pricing in undergraduate education: Effects on degree production by field. Journal of Policy Analysis and Management, 34 (1), 107–135.

Strauss, V. (2018, September 4). The real story of New Orleans and its charter schools. The Washington Post. Retrieved from https://www.washingtonpost.com

Strezhnev, A. (2018). Semiparametric weighting estimators for multi-period difference-in-differences designs . Working paper. Retried from https://www.antonstrezhnev.com/research

Tandberg, D. A., & Hillman, N. W. (2014). State higher education performance funding: Data, outcomes, and policy implications. Journal of Education Finance, 29 (3), 222–243.

Turley, R. (2009). College proximity: Mapping access to opportunity. Sociology of Education, 82 (2), 126–146.

U.S. Census Bureau. (2018a). Metropolitan and micropolitan: About. Retrieved from https://www.census.gov/programs-surveys/metro-micro/about.html

U.S. Census Bureau. (2018b). Metropolitan and micropolitan: Guidance for data users. Retrieved from https://www.census.gov/programs-surveys/metro-micro/guidance.html

U.S. Census Bureau. (2019). Resident population in Orleans Parish, LA [LAORLE0POP]. Retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/LAORLE0POP . 13 Apr 2019.

U.S. Department of Education. (2019). Integrated postsecondary education data system. https://nces.ed.gov/ipeds/datacenter/Data.aspx

U.S. Department of Homeland Security. (2019). OpenFEMA dataset: Disaster declarations summaries – V1. Retrieved from https://www.fema.gov/openfema-dataset-disaster-declarations-summaries-v1

Vigdor, J. (2008). The economic aftermath of Hurricane Katrina. Journal of Economic Perspectives, 22 (4), 135–154.

Wolfers, J. (2006). Did unilateral divorce laws raise divorce rates? A reconciliation and new results. American Economic Review, 96 (5), 1802–1820.

Zhang, L. (2010). The use of panel data models in higher education policy studies. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXV, pp. 307–349). Dordrecht: Springer.

Download references

Author information

Authors and affiliations.

University of Michigan, Ann Arbor, MI, USA

Fernando Furquim

University of Wisconsin-Madison, Madison, MI, USA

Daniel Corral & Nicholas Hillman

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Fernando Furquim .

Editor information

Editors and affiliations.

Graduate School of Education, University of Pennsylvania, Philadelphia, PA, USA

Laura W. Perna

Section Editor information

No affiliation provided

Linda Eisenmann

Pamela Eddy

Adrianna Kezar

Anne-Marie Nunez

Shouping Hu

Anna Neumann

Nicholas A Bowman

Marvin Titus

Nicholas Hillman

Appendix Code

The following Stata code produced many of the tables and charts included in this chapter. Because of space constraints, we have omitted various parts of the code such as data management steps and much of the analysis beyond Hurricane Katrina; we are happy to share complete .do files by request.

// open data for analysis use C:\Desktop\did_handbook_data.dta // install the following user-written programs: ssc install lgraph, replace ssc install blindschemes, replace ssc install estout, replace ssc install coefplot, replace // figure 1: fall enrollment for colleges in new orleans lgraph total_enroll year if treat==1, scheme(plottig) ylabel(0(500)3500) /// legend(off) loptions(1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("") // figure 2: fall enrollment for colleges in new orleans and other southern states lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) /// legend(on order(1 "Southern cities" 2 "New Orleans") position(6)) /// loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) /// lpat(line) mcolor(black)) ytitle("Mean fall enrollment") xtitle("") // table 3: did means table table treat post if (comparison==1 | treat==1), c(mean total_enroll) f(%10.0fc) // table 4: canonical did regression with different standard errors and covariates ∗ top panel: canonical did regression with different standard errors reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1) estimates store table4_a // no se adjustment reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), robust estimates store table4_b // robust s.e. reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table4_c // cluster s.e. ∗ bottom panel: canonical did regression (cluster s.e.) with covariates and fe global controls "tuition1 metro_ue_rate" reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1), /// cluster(unitid) estimates store table4_d // controls only areg total_enroll i.treat i.post i.treat#i.post i.year if _est_table4_d==1 & /// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_e // fixed effects only areg total_enroll i.treat i.post i.treat#i.post $controls i.year if _est_table4_d==1 &/// (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table4_f // controls and fe estout table4_a table4_b table4_c table4_d table4_e table4_f, cells(b(star fmt(2) /// label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) /// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) // table 5: canonical did with multiple comparison groups, without and with controls ∗ create regional neighbor gen neighbor = 1 if inlist(stabbr,"AR","MS","TX") & sreb==1 ∗ create ps matched logit treat total_enroll if sreb==1 & year<2005 predict double ps ssc install psmatch2, replace psmatch2 treat, outcome(total_enroll) pscore(ps) egen ps_match = min(_weight), by(unitid) ∗ top panel (without controls) reg total_enroll i.treat i.post i.treat#i.post if (comparison==1 | treat==1), /// cluster(unitid) estimates store table5_a // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post, cluster(unitid) estimates store table5_b // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), /// cluster(unitid) estimates store table5_c // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), /// cluster(unitid) estimates store table5_d // new orleans vs matched ∗ bottom panel (with controls) reg total_enroll i.treat i.post i.treat#i.post $controls if (comparison==1|treat==1),/// cluster(unitid) estimates store table5_e // new orleans vs sreb reg total_enroll i.treat i.post i.treat#i.post $controls, cluster(unitid) estimates store table5_f // new orleans vs nationwide reg total_enroll i.treat i.post i.treat#i.post $controls if (neighbor==1|treat==1),/// cluster(unitid) estimates store table5_g // new orleans vs neighbor reg total_enroll i.treat i.post i.treat#i.post $controls if (ps_match==1|treat==1),/// cluster(unitid) estimates store table5_h // new orleans vs matched estout table5_a table5_b table5_c table5_d table5_e table5_f table5_g table5_h, /// cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels/// ( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) // figure 6: enrollment trend for multiple comparison groups lgraph total_enroll year treat if (sreb==1 | treat==1), scheme(plottig) /// ylabel(0(500)3500) name(sreb, replace) legend(on order(1 "SREB" 2 "New Orleans") /// position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)/// mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat, scheme(plottig) ylabel(0(500)3500) name(us, replace) /// legend(on order(1 "U.S." 2 "New Orleans") position(6)) loptions(0 lcolor(black) /// lpat(dash) m(square); 1 lcolor(black) lpat(line) mcolor(black)) ytitle("Mean fall /// enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (neighbor==1 | treat==1), scheme(plottig) /// ylabel(0(500)3500) name(neigh, replace) legend(on order(1 "Neighbors" 2 "New Orleans")/// position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)/// mcolor(black)) ytitle("Mean fall enrollment") xtitle("") lgraph total_enroll year treat if sreb==1 & (ps_match==1 | treat==1), scheme(plottig)/// ylabel(0(500)4500) name(psm, replace) legend(on order(1 "PS Matched" 2 "New Orleans")/// position(6)) loptions(0 lcolor(black) lpat(dash) m(square); 1 lcolor(black) lpat(line)/// mcolor(black)) ytitle("Mean fall enrollment") xtitle("") graph combine sreb us neigh psm, name(combined, replace) // table 6: did regression with state-specific trends encode stabbr, gen(stn) areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year if (comparison==1 | treat==1), absorb(stn) cluster(unitid) estimates store table6_a // state x year trends areg total_enroll i.treat i.post i.treat#i.post i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_b // fe only areg total_enroll i.treat i.post i.treat#i.post i.stn##c.year i.year if (comparison==1 | treat==1), absorb(unitid) cluster(unitid) estimates store table6_c //state x years trends and fe estout table6_a table6_b table6_c, cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) //table 7: placebo test for change to treatment timing ∗generate placebo years gen placebo_2003 = 1 if year>=2003 gen placebo_2004 = 1 if year>=2004 gen placebo_2005 = 1 if year>=2005 gen placebo_2006 = 1 if year>=2006 gen placebo_2007 = 1 if year>=2007 recode placebo_2003-placebo_2007 (.=0) ∗ top panel (without controls) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect estimates store p_2003_noco reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 if year<2005, /// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_noco reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005, cluster(unitid) est sto p_2005_noco reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_noco reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 if year>2005, /// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_noco ∗ bottom panel (with control) reg total_enroll i.treat i.placebo_2003 i.treat#i.placebo_2003 $controls if year<2005,/// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2003_co reg total_enroll i.treat i.placebo_2004 i.treat#i.placebo_2004 $controls if year<2005,/// cluster(unitid) // analysis stops at 2005 to avoid picking up Katrina effect est sto p_2004_co reg total_enroll i.treat i.placebo_2005 i.treat#i.placebo_2005 $controls, cluster(unitid) est sto p_2005_co reg total_enroll i.treat i.placebo_2006 i.treat#i.placebo_2006 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2006_co reg total_enroll i.treat i.placebo_2007 i.treat#i.placebo_2007 $controls if year>2005,/// cluster(unitid) // analysis starts at 2006 to avoid picking up Katrina effect est sto p_2007_co estout p_2003_noco p_2004_noco p_2005_noco p_2006_noco p_2007_noco,cells(b(star fmt(2)/// label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)/// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) estout p_2003_co p_2004_co p_2005_co p_2006_co p_2007_co, cells(b(star fmt(2) /// label(Coef.)) se(par fmt(2) label(std.errors))) starlevels( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010)/// stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) // table 8: placebo test for non-equivalent outcome ∗ top panel (in-state tuition) reg tuition1 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid) estimates store table8_a // new orleans vs sreb reg tuition1 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_b // new orleans vs nationwide reg tuition1 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_c // new orleans vs neighbor reg tuition1 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_d // new orleans vs matched ∗ bottom panel (out-of-state tuition) reg tuition3 i.treat i.post i.treat#i.post if (comparison==1 | treat==1), cluster(unitid) estimates store table8_e // new orleans vs sreb reg tuition3 i.treat i.post i.treat#i.post, cluster(unitid) estimates store table8_f // new orleans vs nationwide reg tuition3 i.treat i.post i.treat#i.post if (neighbor==1 | treat==1), cluster(unitid) estimates store table8_g // new orleans vs neighbor reg tuition3 i.treat i.post i.treat#i.post if (ps_match==1 | treat==1), cluster(unitid) estimates store table8_h // new orleans vs matched estout table8_a table8_b table8_c table8_d table8_e table8_f table8_g table8_h,/// cells(b(star fmt(2) label(Coef.)) se(par fmt(2) label(std.errors))) starlevels/// ( ∗ 0.10 ∗∗ 0.05 ∗∗∗ 0.010) stats(N r2, labels ("No. of Obs.""R-Squared") fmt(2)) // table 9: did regression for multiple hurricanes areg total_enroll i.treat_mh i.year if inc==1, vce(cluster unitid) absorb(unitid) // table 10: event study results (hurricane katrina) ∗ create adoption year of treatment and limit to five-year pre/post period gen adopt_delta=2005-year if treat==1 gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" areg total_enroll predelta_5 predelta_4 predelta_3 predelta_2 postdelt∗ i.year if /// within_5==1 & year<=2010, vce(cluster unitid) absorb(unitid) estimates store fig7 // figure 7: event study estimates (hurricane katrina) coefplot fig7, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 /// postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel /// (, angle(vertical)) xtitle("Years since hurricane (0)")ytitle("Estimated effect")/// yline(0, lcolor(black)) scheme(plottig) // table 11: event study results (all hurricanes) ∗ create new within_5 & pre/post indicators because they should be relative to each hurricane cap drop adopt_delta within_5 predelta∗ postdelta∗ gen adopt_delta=year-first_hurr if hurr_ever==1 ∗create event study lag (here’s how to do it in a loop) forvalues i=1(1)5 { gen predelta_`i'=(adopt_delta==`i') label var predelta_`i' "-`i'" } ∗ create event study lead (here’s how to do it one by one) gen postdelta_0=(adopt_delta==0) label var postdelta_0 "0" gen postdelta_1=(adopt_delta==-1) label var postdelta_1 "1" gen postdelta_2=(adopt_delta==-2) label var postdelta_2 "2" gen postdelta_3=(adopt_delta==-3) label var postdelta_3 "3" gen postdelta_4=(adopt_delta<=-4) label var postdelta_4 "4" gen postdelta_5=(adopt_delta<=-5) label var postdelta_5 "5" ∗limit to obs within 5 yrs of a hurricane. gen within_5=(adopt_delta==. | inrange(adopt_delta, -5, 5)) ∗ coefficient only areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year if /// within_5==1 & inc==1, vce(cluster unitid) absorb(unitid) ∗ adding state-specific trends areg total_enroll predelta_2 predelta_3 predelta_4 predelta_5 postdelt∗ i.year /// i.year##c.stn if within_5==1 & inc==1, vce(cluster unitid) absorb(unitid) // figure 8: event study estimates (all hurricanes) coefplot fig8a, keep ( predelta_5 predelta_4 predelta_3 predelta_2 postdelta_0 /// postdelta_1 postdelta_2 postdelta_3 postdelta_4 postdelta_5) vertical xlabel /// (, angle(vertical)) xtitle("Years since hurricane (0)") ytitle("Estimated effect")/// yline(0, lcolor(black)) scheme(plotplain)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Cite this entry.

Furquim, F., Corral, D., Hillman, N. (2020). A Primer for Interpreting and Designing Difference-in-Differences Studies in Higher Education Research. In: Perna, L. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 35. Springer, Cham. https://doi.org/10.1007/978-3-030-31365-4_5

Download citation

DOI : https://doi.org/10.1007/978-3-030-31365-4_5

Published : 26 April 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-31364-7

Online ISBN : 978-3-030-31365-4

eBook Packages : Education Reference Module Humanities and Social Sciences Reference Module Education

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Quantitative vs. Qualitative Research in Psychology

Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

• Key Differences

Quantitative Research Methods

Qualitative research methods.

• How They Relate

In psychology and other social sciences, researchers are faced with an unresolved question: Can we measure concepts like love or racism the same way we can measure temperature or the weight of a star? Social phenomena⁠—things that happen because of and through human behavior⁠—are especially difficult to grasp with typical scientific models.

At a Glance

Psychologists rely on quantitative and quantitative research to better understand human thought and behavior.

• Qualitative research involves collecting and evaluating non-numerical data in order to understand concepts or subjective opinions.
• Quantitative research involves collecting and evaluating numerical data. 

This article discusses what qualitative and quantitative research are, how they are different, and how they are used in psychology research.

Qualitative Research vs. Quantitative Research

In order to understand qualitative and quantitative psychology research, it can be helpful to look at the methods that are used and when each type is most appropriate.

Psychologists rely on a few methods to measure behavior, attitudes, and feelings. These include:

• Self-reports , like surveys or questionnaires
• Observation (often used in experiments or fieldwork)
• Implicit attitude tests that measure timing in responding to prompts

Most of these are quantitative methods. The result is a number that can be used to assess differences between groups.

However, most of these methods are static, inflexible (you can't change a question because a participant doesn't understand it), and provide a "what" answer rather than a "why" answer.

Sometimes, researchers are more interested in the "why" and the "how." That's where qualitative methods come in.

Qualitative research is about speaking to people directly and hearing their words. It is grounded in the philosophy that the social world is ultimately unmeasurable, that no measure is truly ever "objective," and that how humans make meaning is just as important as how much they score on a standardized test.

Used to develop theories

Takes a broad, complex approach

Answers "why" and "how" questions

Explores patterns and themes

Used to test theories

Takes a narrow, specific approach

Answers "what" questions

Explores statistical relationships

Quantitative methods have existed ever since people have been able to count things. But it is only with the positivist philosophy of Auguste Comte (which maintains that factual knowledge obtained by observation is trustworthy) that it became a "scientific method."

The scientific method follows this general process. A researcher must:

• Generate a theory or hypothesis (i.e., predict what might happen in an experiment) and determine the variables needed to answer their question
• Develop instruments to measure the phenomenon (such as a survey, a thermometer, etc.)
• Develop experiments to manipulate the variables
• Collect empirical (measured) data
• Analyze data

Quantitative methods are about measuring phenomena, not explaining them.

Quantitative research compares two groups of people. There are all sorts of variables you could measure, and many kinds of experiments to run using quantitative methods.

These comparisons are generally explained using graphs, pie charts, and other visual representations that give the researcher a sense of how the various data points relate to one another.

Basic Assumptions

Quantitative methods assume:

• That the world is measurable
• That humans can observe objectively
• That we can know things for certain about the world from observation

In some fields, these assumptions hold true. Whether you measure the size of the sun 2000 years ago or now, it will always be the same. But when it comes to human behavior, it is not so simple.

As decades of cultural and social research have shown, people behave differently (and even think differently) based on historical context, cultural context, social context, and even identity-based contexts like gender , social class, or sexual orientation .

Therefore, quantitative methods applied to human behavior (as used in psychology and some areas of sociology) should always be rooted in their particular context. In other words: there are no, or very few, human universals.

Statistical information is the primary form of quantitative data used in human and social quantitative research. Statistics provide lots of information about tendencies across large groups of people, but they can never describe every case or every experience. In other words, there are always outliers.

Correlation and Causation

A basic principle of statistics is that correlation is not causation. Researchers can only claim a cause-and-effect relationship under certain conditions:

• The study was a true experiment.
• The independent variable can be manipulated (for example, researchers cannot manipulate gender, but they can change the primer a study subject sees, such as a picture of nature or of a building).
• The dependent variable can be measured through a ratio or a scale.

So when you read a report that "gender was linked to" something (like a behavior or an attitude), remember that gender is NOT a cause of the behavior or attitude. There is an apparent relationship, but the true cause of the difference is hidden.

Pitfalls of Quantitative Research

Quantitative methods are one way to approach the measurement and understanding of human and social phenomena. But what's missing from this picture?

As noted above, statistics do not tell us about personal, individual experiences and meanings. While surveys can give a general idea, respondents have to choose between only a few responses. This can make it difficult to understand the subtleties of different experiences.

Quantitative methods can be helpful when making objective comparisons between groups or when looking for relationships between variables. They can be analyzed statistically, which can be helpful when looking for patterns and relationships.

Qualitative data are not made out of numbers but rather of descriptions, metaphors, symbols, quotes, analysis, concepts, and characteristics. This approach uses interviews, written texts, art, photos, and other materials to make sense of human experiences and to understand what these experiences mean to people.

While quantitative methods ask "what" and "how much," qualitative methods ask "why" and "how."

Qualitative methods are about describing and analyzing phenomena from a human perspective. There are many different philosophical views on qualitative methods, but in general, they agree that some questions are too complex or impossible to answer with standardized instruments.

These methods also accept that it is impossible to be completely objective in observing phenomena. Researchers have their own thoughts, attitudes, experiences, and beliefs, and these always color how people interpret results.

Qualitative Approaches

There are many different approaches to qualitative research, with their own philosophical bases. Different approaches are best for different kinds of projects. For example:

• Case studies and narrative studies are best for single individuals. These involve studying every aspect of a person's life in great depth.
• Phenomenology aims to explain experiences. This type of work aims to describe and explore different events as they are consciously and subjectively experienced.
• Grounded theory develops models and describes processes. This approach allows researchers to construct a theory based on data that is collected, analyzed, and compared to reach new discoveries.
• Ethnography describes cultural groups. In this approach, researchers immerse themselves in a community or group in order to observe behavior.

Qualitative researchers must be aware of several different methods and know each thoroughly enough to produce valuable research.

Some researchers specialize in a single method, but others specialize in a topic or content area and use many different methods to explore the topic, providing different information and a variety of points of view.

There is not a single model or method that can be used for every qualitative project. Depending on the research question, the people participating, and the kind of information they want to produce, researchers will choose the appropriate approach.

Interpretation

Qualitative research does not look into causal relationships between variables, but rather into themes, values, interpretations, and meanings. As a rule, then, qualitative research is not generalizable (cannot be applied to people outside the research participants).

The insights gained from qualitative research can extend to other groups with proper attention to specific historical and social contexts.

Relationship Between Qualitative and Quantitative Research

It might sound like quantitative and qualitative research do not play well together. They have different philosophies, different data, and different outputs. However, this could not be further from the truth.

These two general methods complement each other. By using both, researchers can gain a fuller, more comprehensive understanding of a phenomenon.

For example, a psychologist wanting to develop a new survey instrument about sexuality might and ask a few dozen people questions about their sexual experiences (this is qualitative research). This gives the researcher some information to begin developing questions for their survey (which is a quantitative method).

After the survey, the same or other researchers might want to dig deeper into issues brought up by its data. Follow-up questions like "how does it feel when...?" or "what does this mean to you?" or "how did you experience this?" can only be answered by qualitative research.

By using both quantitative and qualitative data, researchers have a more holistic, well-rounded understanding of a particular topic or phenomenon.

Qualitative and quantitative methods both play an important role in psychology. Where quantitative methods can help answer questions about what is happening in a group and to what degree, qualitative methods can dig deeper into the reasons behind why it is happening. By using both strategies, psychology researchers can learn more about human thought and behavior.

Gough B, Madill A. Subjectivity in psychological science: From problem to prospect . Psychol Methods . 2012;17(3):374-384. doi:10.1037/a0029313

Pearce T. “Science organized”: Positivism and the metaphysical club, 1865–1875 . J Hist Ideas . 2015;76(3):441-465.

Adams G. Context in person, person in context: A cultural psychology approach to social-personality psychology . In: Deaux K, Snyder M, eds. The Oxford Handbook of Personality and Social Psychology . Oxford University Press; 2012:182-208.

Brady HE. Causation and explanation in social science . In: Goodin RE, ed. The Oxford Handbook of Political Science. Oxford University Press; 2011. doi:10.1093/oxfordhb/9780199604456.013.0049

Chun Tie Y, Birks M, Francis K. Grounded theory research: A design framework for novice researchers .  SAGE Open Med . 2019;7:2050312118822927. doi:10.1177/2050312118822927

Reeves S, Peller J, Goldman J, Kitto S. Ethnography in qualitative educational research: AMEE Guide No. 80 . Medical Teacher . 2013;35(8):e1365-e1379. doi:10.3109/0142159X.2013.804977

Salkind NJ, ed. Encyclopedia of Research Design . Sage Publishing.

Shaughnessy JJ, Zechmeister EB, Zechmeister JS.  Research Methods in Psychology . McGraw Hill Education.

By Anabelle Bernard Fournier Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

Which Master? Postgraduate Taught vs Research (Differences)

Find Master ’ s degrees in Europe now

💡 Taught Masters vs Research Masters:

There are many types of Master’s degrees, and most of these are Taught Masters . In the United Kingdom, such programmes are also called postgraduate taught or PGT for short. They typically require completing a set number of modules and a thesis (also called dissertation), plus sometimes work experience through a placement. The type of dissertation you will undertake will depend on the type of Master’s degree you are enrolled in, and might for example include your own small research project. Most postgraduate taught degrees have these elements of independent work and research to varying extents.

In contrast, a Research Master ’ s degree will focus on, you guessed it, research. In the UK, such programmes are also called postgraduate research or PGR (although this term may also refer to doctorate programmes). Rather than attending classes every semester and completing module assignments, you will need to focus on an independent research project – under supervision, of course. The course will still include a few taught modules, most often on research methodologies, but it will require you to work independently for most of the time.

Master’s degrees in the UK are usually one year full-time or two years part-time, but in other countries the duration may differ.

Remember: A final dissertation will be compulsory for all Master ’ s degrees. However, a dissertation for an MRes will typically be longer than that for an MA or MSc.

Typically, Research Masters will lead to an MRes degree. At some universities, however, you'll instead be awarded an MPhil (Master of Philosophy) or MLitt (Master of Letters). For more information, have a look at our detailed glossary.

🤔 Is a Research Master’s the same as a PhD?

No, a Research Master’s degree is not the same as a PhD. Although for both degrees you will need to complete a dissertation based on an independent research project, there are notable differences:

• The first difference is the duration : A Master’s degree will typically last one to two years, while a PhD usually takes up about three to five years. The research project you’ll undertake during a doctorate degree will therefore be longer and broader than one you would pursue in a Master’s degree.
• As a PhD student, you’re expected to publish research papers in journals before you are awarded your degree. MRes students might occasionally do that during or after their studies, but it’s rarely obligatory.
• As a PhD student, you’ll most often be expected to take on other duties , such as teaching.

If you wish to pursue doctoral research and a career in academia, a research Master’s degree could be a great option for you as it will allow you to get to grips with and gather valuable experience and training on independent research early on in your studies.

👀 Overview: What’s the difference?

There are a few differences between Taught Masters and Research Masters , and not all of them are obvious.

The table below outlines some of the main elements to consider when choosing which of the two degrees to pursue after your Bachelor’s degree:

Study in Europe: Find your Master ’ s degrees

🏛️ Which should you choose?

The choice between a taught Master’s and a research Master’s depends on a few factors.

• First of all, do you enjoy research more than coursework? Then an MRes may be more suitable – but remember that any Master’s degree, especially an MSc, will have a research component.
• Then, it’s crucial to understand how you like to work and study. Do you particularly enjoy working independently? Perhaps then you can consider an MRes. In a taught Master’s, you’ll have a more solid structure, timetables and regular deadlines to keep you on track, but these may not be as readily available during an MRes, so consider which environment you are more likely to thrive in.  
• Another important consideration is what you want to do after your Master’s degree. If you want to enter the labour market immediately, and you are not particularly interested in focusing on research training, then perhaps a taught Master’s degree is more suited.

💸 Is there a difference in fees between Taught and Research Masters?

No, normally, you won’t find a huge difference in tuition fees between taught and research Master’s degrees. Only in some instances, Postgraduate Research Masters tend to be cheaper.

📝 Can I do a PhD after taking a Taught Master’s Degree?

Yes, you can pursue a PhD after any type of Master’s course, provided that you have a degree in a relevant subject. All taught postgraduate degrees involve some independent work and research, especially for your dissertation, which will prepare you for further research should you choose to pursue a PhD.

Some taught Masters require more independent research work than others, particularly when it comes to the dissertation after completing the taught modules. Consult the curriculum or ask admissions staff to get a better idea of what to expect.

While a taught Master’s degree won’t prevent you from doing a PhD further down the line, it’s vital that you have a good idea of what requirements you will have to fulfil in order to be admitted to the PhD, and how you can best prepare.

If you already have a clear idea of what field you’d like to conduct your doctoral research in, you could take advantage of the joint Master’s – PhD programmes on offer at some universities.

These four-year programmes – also called “combined” or “integrated” degrees – offer the chance to complete a Master’s degree in the first year and to progress seamlessly to PhD research in the next three.

Looking for Masters in Europe? Have a look at these English-taught degrees 👀

Author: Claudia Civinini

Claudia has many years of experience as a reporter and writer on international education and student mobility. Originally from Italy, she holds a BA in Communication and Media Studies from the University of Genova; a Graduate Diploma in Education, Secondary Education and Teaching from the Australian Catholic University; and a joint MSc in Educational Neuroscience from UCL and Birkbeck, University of London. Claudia has previously worked as Chief Reporter for the English Language Gazette, as Senior Reporter for the PIE News (Professionals in International Education), and as Reporter for Tes.

You might be interested in this:

Qualitative vs Quantitative Research Methods & Data Analysis

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

What is the difference between quantitative and qualitative?

The main difference between quantitative and qualitative research is the type of data they collect and analyze.

Quantitative research collects numerical data and analyzes it using statistical methods. The aim is to produce objective, empirical data that can be measured and expressed in numerical terms. Quantitative research is often used to test hypotheses, identify patterns, and make predictions.

Qualitative research , on the other hand, collects non-numerical data such as words, images, and sounds. The focus is on exploring subjective experiences, opinions, and attitudes, often through observation and interviews.

Qualitative research aims to produce rich and detailed descriptions of the phenomenon being studied, and to uncover new insights and meanings.

Quantitative data is information about quantities, and therefore numbers, and qualitative data is descriptive, and regards phenomenon which can be observed but not measured, such as language.

What Is Qualitative Research?

Qualitative research is the process of collecting, analyzing, and interpreting non-numerical data, such as language. Qualitative research can be used to understand how an individual subjectively perceives and gives meaning to their social reality.

Qualitative data is non-numerical data, such as text, video, photographs, or audio recordings. This type of data can be collected using diary accounts or in-depth interviews and analyzed using grounded theory or thematic analysis.

Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Denzin and Lincoln (1994, p. 2)

Interest in qualitative data came about as the result of the dissatisfaction of some psychologists (e.g., Carl Rogers) with the scientific study of psychologists such as behaviorists (e.g., Skinner ).

Since psychologists study people, the traditional approach to science is not seen as an appropriate way of carrying out research since it fails to capture the totality of human experience and the essence of being human.  Exploring participants’ experiences is known as a phenomenological approach (re: Humanism ).

Qualitative research is primarily concerned with meaning, subjectivity, and lived experience. The goal is to understand the quality and texture of people’s experiences, how they make sense of them, and the implications for their lives.

Qualitative research aims to understand the social reality of individuals, groups, and cultures as nearly as possible as participants feel or live it. Thus, people and groups are studied in their natural setting.

Some examples of qualitative research questions are provided, such as what an experience feels like, how people talk about something, how they make sense of an experience, and how events unfold for people.

Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon, or behavior, operates as it does in a particular context. It can be used to generate hypotheses and theories from the data.

Qualitative Methods

There are different types of qualitative research methods, including diary accounts, in-depth interviews , documents, focus groups , case study research , and ethnography.

The results of qualitative methods provide a deep understanding of how people perceive their social realities and in consequence, how they act within the social world.

The researcher has several methods for collecting empirical materials, ranging from the interview to direct observation, to the analysis of artifacts, documents, and cultural records, to the use of visual materials or personal experience. Denzin and Lincoln (1994, p. 14)

Here are some examples of qualitative data:

Interview transcripts : Verbatim records of what participants said during an interview or focus group. They allow researchers to identify common themes and patterns, and draw conclusions based on the data. Interview transcripts can also be useful in providing direct quotes and examples to support research findings.

Observations : The researcher typically takes detailed notes on what they observe, including any contextual information, nonverbal cues, or other relevant details. The resulting observational data can be analyzed to gain insights into social phenomena, such as human behavior, social interactions, and cultural practices.

Unstructured interviews : generate qualitative data through the use of open questions.  This allows the respondent to talk in some depth, choosing their own words.  This helps the researcher develop a real sense of a person’s understanding of a situation.

Diaries or journals : Written accounts of personal experiences or reflections.

Notice that qualitative data could be much more than just words or text. Photographs, videos, sound recordings, and so on, can be considered qualitative data. Visual data can be used to understand behaviors, environments, and social interactions.

Qualitative Data Analysis

Qualitative research is endlessly creative and interpretive. The researcher does not just leave the field with mountains of empirical data and then easily write up his or her findings.

Qualitative interpretations are constructed, and various techniques can be used to make sense of the data, such as content analysis, grounded theory (Glaser & Strauss, 1967), thematic analysis (Braun & Clarke, 2006), or discourse analysis.

For example, thematic analysis is a qualitative approach that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded.

Key Features

• Events can be understood adequately only if they are seen in context. Therefore, a qualitative researcher immerses her/himself in the field, in natural surroundings. The contexts of inquiry are not contrived; they are natural. Nothing is predefined or taken for granted.
• Qualitative researchers want those who are studied to speak for themselves, to provide their perspectives in words and other actions. Therefore, qualitative research is an interactive process in which the persons studied teach the researcher about their lives.
• The qualitative researcher is an integral part of the data; without the active participation of the researcher, no data exists.
• The study’s design evolves during the research and can be adjusted or changed as it progresses. For the qualitative researcher, there is no single reality. It is subjective and exists only in reference to the observer.
• The theory is data-driven and emerges as part of the research process, evolving from the data as they are collected.

Limitations of Qualitative Research

• Because of the time and costs involved, qualitative designs do not generally draw samples from large-scale data sets.
• The problem of adequate validity or reliability is a major criticism. Because of the subjective nature of qualitative data and its origin in single contexts, it is difficult to apply conventional standards of reliability and validity. For example, because of the central role played by the researcher in the generation of data, it is not possible to replicate qualitative studies.
• Also, contexts, situations, events, conditions, and interactions cannot be replicated to any extent, nor can generalizations be made to a wider context than the one studied with confidence.
• The time required for data collection, analysis, and interpretation is lengthy. Analysis of qualitative data is difficult, and expert knowledge of an area is necessary to interpret qualitative data. Great care must be taken when doing so, for example, looking for mental illness symptoms.

Advantages of Qualitative Research

• Because of close researcher involvement, the researcher gains an insider’s view of the field. This allows the researcher to find issues that are often missed (such as subtleties and complexities) by the scientific, more positivistic inquiries.
• Qualitative descriptions can be important in suggesting possible relationships, causes, effects, and dynamic processes.
• Qualitative analysis allows for ambiguities/contradictions in the data, which reflect social reality (Denscombe, 2010).
• Qualitative research uses a descriptive, narrative style; this research might be of particular benefit to the practitioner as she or he could turn to qualitative reports to examine forms of knowledge that might otherwise be unavailable, thereby gaining new insight.

What Is Quantitative Research?

Quantitative research involves the process of objectively collecting and analyzing numerical data to describe, predict, or control variables of interest.

The goals of quantitative research are to test causal relationships between variables , make predictions, and generalize results to wider populations.

Quantitative researchers aim to establish general laws of behavior and phenomenon across different settings/contexts. Research is used to test a theory and ultimately support or reject it.

Quantitative Methods

Experiments typically yield quantitative data, as they are concerned with measuring things.  However, other research methods, such as controlled observations and questionnaires , can produce both quantitative information.

For example, a rating scale or closed questions on a questionnaire would generate quantitative data as these produce either numerical data or data that can be put into categories (e.g., “yes,” “no” answers).

Experimental methods limit how research participants react to and express appropriate social behavior.

Findings are, therefore, likely to be context-bound and simply a reflection of the assumptions that the researcher brings to the investigation.

There are numerous examples of quantitative data in psychological research, including mental health. Here are a few examples:

Another example is the Experience in Close Relationships Scale (ECR), a self-report questionnaire widely used to assess adult attachment styles .

The ECR provides quantitative data that can be used to assess attachment styles and predict relationship outcomes.

Neuroimaging data : Neuroimaging techniques, such as MRI and fMRI, provide quantitative data on brain structure and function.

This data can be analyzed to identify brain regions involved in specific mental processes or disorders.

For example, the Beck Depression Inventory (BDI) is a clinician-administered questionnaire widely used to assess the severity of depressive symptoms in individuals.

The BDI consists of 21 questions, each scored on a scale of 0 to 3, with higher scores indicating more severe depressive symptoms. 

Quantitative Data Analysis

Statistics help us turn quantitative data into useful information to help with decision-making. We can use statistics to summarize our data, describing patterns, relationships, and connections. Statistics can be descriptive or inferential.

Descriptive statistics help us to summarize our data. In contrast, inferential statistics are used to identify statistically significant differences between groups of data (such as intervention and control groups in a randomized control study).

• Quantitative researchers try to control extraneous variables by conducting their studies in the lab.
• The research aims for objectivity (i.e., without bias) and is separated from the data.
• The design of the study is determined before it begins.
• For the quantitative researcher, the reality is objective, exists separately from the researcher, and can be seen by anyone.
• Research is used to test a theory and ultimately support or reject it.

Limitations of Quantitative Research

• Context: Quantitative experiments do not take place in natural settings. In addition, they do not allow participants to explain their choices or the meaning of the questions they may have for those participants (Carr, 1994).
• Researcher expertise: Poor knowledge of the application of statistical analysis may negatively affect analysis and subsequent interpretation (Black, 1999).
• Variability of data quantity: Large sample sizes are needed for more accurate analysis. Small-scale quantitative studies may be less reliable because of the low quantity of data (Denscombe, 2010). This also affects the ability to generalize study findings to wider populations.
• Confirmation bias: The researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on the theory of hypothesis generation.

Advantages of Quantitative Research

• Scientific objectivity: Quantitative data can be interpreted with statistical analysis, and since statistics are based on the principles of mathematics, the quantitative approach is viewed as scientifically objective and rational (Carr, 1994; Denscombe, 2010).
• Useful for testing and validating already constructed theories.
• Rapid analysis: Sophisticated software removes much of the need for prolonged data analysis, especially with large volumes of data involved (Antonius, 2003).
• Replication: Quantitative data is based on measured values and can be checked by others because numerical data is less open to ambiguities of interpretation.
• Hypotheses can also be tested because of statistical analysis (Antonius, 2003).

Antonius, R. (2003). Interpreting quantitative data with SPSS . Sage.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics . Sage.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology . Qualitative Research in Psychology , 3, 77–101.

Carr, L. T. (1994). The strengths and weaknesses of quantitative and qualitative research : what method for nursing? Journal of advanced nursing, 20(4) , 716-721.

Denscombe, M. (2010). The Good Research Guide: for small-scale social research. McGraw Hill.

Denzin, N., & Lincoln. Y. (1994). Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications Inc.

Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing research, 17(4) , 364.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Punch, K. (1998). Introduction to Social Research: Quantitative and Qualitative Approaches. London: Sage

Further Information

• Designing qualitative research
• Methods of data collection and analysis
• Introduction to quantitative and qualitative research
• Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
• Qualitative research in health care: Analysing qualitative data
• Qualitative data analysis: the framework approach
• Using the framework method for the analysis of
• Qualitative data in multi-disciplinary health research
• Content Analysis
• Grounded Theory
• Thematic Analysis

• Share full article

Advertisement

Supported by

New Study Bolsters Idea of Athletic Differences Between Men and Trans Women

Research financed by the International Olympic Committee introduced new data to the unsettled and fractious debate about bans on transgender athletes.

By Jeré Longman

A new study financed by the International Olympic Committee found that transgender female athletes showed greater handgrip strength — an indicator of overall muscle strength — but lower jumping ability, lung function and relative cardiovascular fitness compared with women whose gender was assigned female at birth.

That data, which also compared trans women with men, contradicted a broad claim often made by proponents of rules that bar transgender women from competing in women’s sports. It also led the study’s authors to caution against a rush to expand such policies, which already bar transgender athletes from a handful of Olympic sports.

The study’s most important finding, according to one of its authors, Yannis Pitsiladis, a member of the I.O.C.’s medical and scientific commission, was that, given physiological differences, “Trans women are not biological men.”

Alternately praised and criticized, the study added an intriguing data set to an unsettled and often politicized debate that may only grow louder with the Paris Olympics and a U.S. presidential election approaching.

The authors cautioned against the presumption of immutable and disproportionate advantages for transgender female athletes who compete in women’s sports, and they advised against “precautionary bans and sport eligibility exclusions” that were not based on sport-specific research.

Outright bans, though, continue to proliferate. Twenty-five U.S. states now have laws or regulations barring transgender athletes from competing in girls and women’s sports, according to the Movement Advancement Project , a nonprofit that focuses on gay, lesbian, bisexual and transgender parity. And the National Association of Intercollegiate Athletics , the governing body for smaller colleges, this month barred transgender athletes from competing in women’s sports unless their sex was assigned female at birth and they had not undergone hormone therapy.

Two of the most visible sports at this summer’s Paris Games — swimming and track and field — along with cycling have effectively barred transgender female athletes who went through puberty as males. Rugby has instituted a total ban on trans female athletes, citing safety concerns, and those permitted to participate in other sports often face stricter requirements in suppressing their levels of testosterone.

The International Olympic Committee has left eligibility rules for transgender female athletes up to the global federations that govern individual sports. And while the Olympic committee provided financing for the study — as it does on a variety of topics through a research fund — Olympic officials had no input or influence on the results, Dr. Pitsiladis said.

In general, the argument for the bans has been that profound advantages gained from testosterone-fueled male puberty — broader shoulders, bigger hands, longer torsos, and greater muscle mass, strength, bone density and heart and lung capacity — give transgender female athletes an inequitable and largely irreversible competitive edge.

The new laboratory-based, peer-reviewed and I.O.C.-funded study at the University of Brighton, published this month in the British Journal of Sports Medicine , tested 19 cisgender men (those whose gender identity matches the sex they were assigned at birth) and 12 trans men, along with 23 trans women and 21 cisgender women.

All of the participants played competitive sports or underwent physical training at least three times a week. And all of the trans female athletes had undergone at least a year of treatment suppressing their testosterone levels and taking estrogen supplementation, the researchers said. None of the participants were athletes competing at the national or international level.

The study found that transgender female participants showed greater handgrip strength than cisgender female participants but lower lung function and relative VO2 max, the amount of oxygen used when exercising. Transgender female athletes also scored below cisgender women and men on a jumping test that measured lower-body power.

The study acknowledged some limitations, including its small sample size and the fact that the athletes were not followed over the long term as they transitioned. And, as previous research has indicated, it found that transgender female athletes did retain at least one advantage over cisgender female athletes — a measurement of handgrip strength .

But it is a combination of factors, not a single parameter, that determines athletic performance, said Dr. Pitsiladis, a professor of sport and exercise science.

Athletes who grow taller and heavier after going through puberty as males must “carry this big skeleton with a smaller engine” after transitioning, he said. He cited volleyball as an example, saying that, for transgender female athletes, “the jumping and blocking will not be to the same height as they were doing before. And they may find that, overall, their performance is less good.”

But Michael J. Joyner, a doctor at the Mayo Clinic who studies the physiology of male and female athletes, said that, based on his research and the research of others, science supports the bans in elite sports, where events can be decided by the smallest of margins.

“We know testosterone is performance enhancing,” Dr. Joyner said. “And we know testosterone has residual effects.” Additionally, he added, declines in performance by trans women after taking drugs to suppress their testosterone levels do not fully reduce the typical differences in athletic performance between men and women.

Supporters of transgender athletes, and some scientists who disagree with bans, have accused governing bodies and lawmakers of enacting solutions for a problem that doesn’t exist. There are few elite trans female athletes, they have noted. And there has been limited scientific study of presumed unalterable advantages in strength, power and aerobic capacity gained by experiencing puberty as a male.

For those who have competed in the Olympics, results have varied widely. At the 2021 Tokyo Games, Quinn , a soccer player who is trans nonbinary and was assigned female at birth, helped Canada’s team win a gold medal. But Laurel Hubbard , a transgender weight lifter from New Zealand, failed to complete a lift in her event.

“The idea that trans women are going to take over women’s sports is ludicrous,” said Joanna Harper, a leading researcher of trans athletes and a postdoctoral scholar at Oregon Health & Science University.

Dr. Harper, who is transgender, said it was important for sports to consider physiological differences between transgender women and cisgender women and that she supported certain restrictions, such as requiring the suppression of testosterone levels. But she called blanket bans “unnecessary and unjustified” and said she welcomed the I.O.C.-funded study.

“This fear that trans women aren’t really women, that they’re men who are invading women’s sports, and that trans women will carry all of their male athleticism, their athletic capabilities, into women’s sports — neither of those things are true,” Dr. Harper said.

Sebastian Coe, the president of World Athletics, which governs global track and field, acknowledged that the science remains unresolved. But the organization decided to bar transgender female athletes from international track and field, he said, because “I’m not going to take a risk on this.”

“We think this is in the best interest of preserving the female category,” Mr. Coe said.

In at least two prominent cases, the fight over transgender bans has moved to the courts. The former University of Pennsylvania swimmer Lia Thomas is challenging a ban imposed by World Aquatics, swimming’s global governing body, after she won the 500-yard freestyle race at the 2022 N.C.A.A. championships. That victory made Thomas, who had been among the best men’s swimmers in the Ivy League, the first known trans athlete to win a women’s championship event in college sports’ top division.

Thomas did not dominate all of her races, though, finishing tied for fifth in a second race and eighth in a third. Her winning time in the 500 was more than nine seconds slower than the N.C.A.A. record. Her case, filed at the Swiss-based Court of Arbitration for Sport, is not expected to be resolved before the Paris Olympics begin in July.

Meanwhile, more than a dozen current and former U.S. college athletes, including at least one who competed against Thomas, sued the N.C.A.A. last month . They claimed that, by letting Thomas participate in the national championships, the organization had violated their rights under Title IX, the law that prohibits sex discrimination at institutions that receive federal funding. (Title IX has also been relied upon to argue in favor of transgender female athletes.)

Outsports , a website that reports on L.G.B.T.Q. issues, hailed the I.O.C.-funded study as a “landmark” that concluded that “blanket sports bans are a mistake.” But some scientists and athletes called the study deeply flawed in an article in The Telegraph , which labeled the suggestion that transgender women are at a disadvantage in sports a “new low” for the I.O.C.

So heated is the debate that Dr. Pitsiladis said he and his research team have received threats. That, he warned, could lead other scientists to shy away from pursuing research on the topic.

“Why would any scientist do this if you’re going to get totally slammed and character-assassinated?” he said. “This is no longer a science matter. Unfortunately, it’s become a political matter.”

Jeré Longman covers international sports, focusing on competitive, social, cultural and political issues around the world. More about Jeré Longman

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

How Pew Research Center will report on generations moving forward

Journalists, researchers and the public often look at society through the lens of generation, using terms like Millennial or Gen Z to describe groups of similarly aged people. This approach can help readers see themselves in the data and assess where we are and where we’re headed as a country.

Pew Research Center has been at the forefront of generational research over the years, telling the story of Millennials as they came of age politically and as they moved more firmly into adult life . In recent years, we’ve also been eager to learn about Gen Z as the leading edge of this generation moves into adulthood.

But generational research has become a crowded arena. The field has been flooded with content that’s often sold as research but is more like clickbait or marketing mythology. There’s also been a growing chorus of criticism about generational research and generational labels in particular.

Recently, as we were preparing to embark on a major research project related to Gen Z, we decided to take a step back and consider how we can study generations in a way that aligns with our values of accuracy, rigor and providing a foundation of facts that enriches the public dialogue.

A typical generation spans 15 to 18 years. As many critics of generational research point out, there is great diversity of thought, experience and behavior within generations.

We set out on a yearlong process of assessing the landscape of generational research. We spoke with experts from outside Pew Research Center, including those who have been publicly critical of our generational analysis, to get their take on the pros and cons of this type of work. We invested in methodological testing to determine whether we could compare findings from our earlier telephone surveys to the online ones we’re conducting now. And we experimented with higher-level statistical analyses that would allow us to isolate the effect of generation.

What emerged from this process was a set of clear guidelines that will help frame our approach going forward. Many of these are principles we’ve always adhered to , but others will require us to change the way we’ve been doing things in recent years.

Here’s a short overview of how we’ll approach generational research in the future:

We’ll only do generational analysis when we have historical data that allows us to compare generations at similar stages of life. When comparing generations, it’s crucial to control for age. In other words, researchers need to look at each generation or age cohort at a similar point in the life cycle. (“Age cohort” is a fancy way of referring to a group of people who were born around the same time.)

When doing this kind of research, the question isn’t whether young adults today are different from middle-aged or older adults today. The question is whether young adults today are different from young adults at some specific point in the past.

To answer this question, it’s necessary to have data that’s been collected over a considerable amount of time – think decades. Standard surveys don’t allow for this type of analysis. We can look at differences across age groups, but we can’t compare age groups over time.

Another complication is that the surveys we conducted 20 or 30 years ago aren’t usually comparable enough to the surveys we’re doing today. Our earlier surveys were done over the phone, and we’ve since transitioned to our nationally representative online survey panel , the American Trends Panel . Our internal testing showed that on many topics, respondents answer questions differently depending on the way they’re being interviewed. So we can’t use most of our surveys from the late 1980s and early 2000s to compare Gen Z with Millennials and Gen Xers at a similar stage of life.

This means that most generational analysis we do will use datasets that have employed similar methodologies over a long period of time, such as surveys from the U.S. Census Bureau. A good example is our 2020 report on Millennial families , which used census data going back to the late 1960s. The report showed that Millennials are marrying and forming families at a much different pace than the generations that came before them.

Even when we have historical data, we will attempt to control for other factors beyond age in making generational comparisons. If we accept that there are real differences across generations, we’re basically saying that people who were born around the same time share certain attitudes or beliefs – and that their views have been influenced by external forces that uniquely shaped them during their formative years. Those forces may have been social changes, economic circumstances, technological advances or political movements.

When we see that younger adults have different views than their older counterparts, it may be driven by their demographic traits rather than the fact that they belong to a particular generation.

The tricky part is isolating those forces from events or circumstances that have affected all age groups, not just one generation. These are often called “period effects.” An example of a period effect is the Watergate scandal, which drove down trust in government among all age groups. Differences in trust across age groups in the wake of Watergate shouldn’t be attributed to the outsize impact that event had on one age group or another, because the change occurred across the board.

Changing demographics also may play a role in patterns that might at first seem like generational differences. We know that the United States has become more racially and ethnically diverse in recent decades, and that race and ethnicity are linked with certain key social and political views. When we see that younger adults have different views than their older counterparts, it may be driven by their demographic traits rather than the fact that they belong to a particular generation.

Controlling for these factors can involve complicated statistical analysis that helps determine whether the differences we see across age groups are indeed due to generation or not. This additional step adds rigor to the process. Unfortunately, it’s often absent from current discussions about Gen Z, Millennials and other generations.

When we can’t do generational analysis, we still see value in looking at differences by age and will do so where it makes sense. Age is one of the most common predictors of differences in attitudes and behaviors. And even if age gaps aren’t rooted in generational differences, they can still be illuminating. They help us understand how people across the age spectrum are responding to key trends, technological breakthroughs and historical events.

Each stage of life comes with a unique set of experiences. Young adults are often at the leading edge of changing attitudes on emerging social trends. Take views on same-sex marriage , for example, or attitudes about gender identity .

Many middle-aged adults, in turn, face the challenge of raising children while also providing care and support to their aging parents. And older adults have their own obstacles and opportunities. All of these stories – rooted in the life cycle, not in generations – are important and compelling, and we can tell them by analyzing our surveys at any given point in time.

When we do have the data to study groups of similarly aged people over time, we won’t always default to using the standard generational definitions and labels. While generational labels are simple and catchy, there are other ways to analyze age cohorts. For example, some observers have suggested grouping people by the decade in which they were born. This would create narrower cohorts in which the members may share more in common. People could also be grouped relative to their age during key historical events (such as the Great Recession or the COVID-19 pandemic) or technological innovations (like the invention of the iPhone).

By choosing not to use the standard generational labels when they’re not appropriate, we can avoid reinforcing harmful stereotypes or oversimplifying people’s complex lived experiences.

Existing generational definitions also may be too broad and arbitrary to capture differences that exist among narrower cohorts. A typical generation spans 15 to 18 years. As many critics of generational research point out, there is great diversity of thought, experience and behavior within generations. The key is to pick a lens that’s most appropriate for the research question that’s being studied. If we’re looking at political views and how they’ve shifted over time, for example, we might group people together according to the first presidential election in which they were eligible to vote.

With these considerations in mind, our audiences should not expect to see a lot of new research coming out of Pew Research Center that uses the generational lens. We’ll only talk about generations when it adds value, advances important national debates and highlights meaningful societal trends.

• Age & Generations
• Demographic Research
• Generation X
• Generation Z
• Generations
• Greatest Generation
• Methodological Research
• Millennials
• Silent Generation

Kim Parker is director of social trends research at Pew Research Center

How Teens and Parents Approach Screen Time

Who are you the art and science of measuring identity, u.s. centenarian population is projected to quadruple over the next 30 years, older workers are growing in number and earning higher wages, teens, social media and technology 2023, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

2-FOR-1 GA TICKETS WITH OUTSIDE+

Don’t miss Thundercat, Fleet Foxes, and more at the Outside Festival.

GET TICKETS

BEST WEEK EVER

Try out unlimited access with 7 days of Outside+ for free.

Start Your Free Trial

Research Suggests Trans Women Don’t Have a Complete Athletic Advantage

In a recent study, transgender women athletes performed better at some strength and fitness metrics than cisgender women—and worse at others

Heading out the door? Read this article on the Outside app available now on iOS devices for members! >","name":"in-content-cta","type":"link"}}'>Download the app .

In the last few years, women’s sports have been a highly visible arena for the broader cultural and political debate over the rights and inclusion of transgender people. As transgender athletes come out, pursue gender-affirming medical care like hormone therapies, and try to compete in the division that aligns with their gender identity, many sports’ governing bodies have developed policies restricting participation on the basis of sex assigned at birth or other biological markers. Sometimes, these policies amount to outright bans on transgender women competing in the women’s category.

Even though proponents of bans often argue using the language of biology—men have more testosterone, women are less muscular—there’s actually very little scientific research into how hormone therapies commonly used in transition-related care affect athletic performance. Most of these bans are based on the assumption that the physical traits resulting from boys’ testosterone-saturated puberty, like increased muscle mass, strength, and height, are retained by someone assigned male at birth when they choose to transition.

But a new study  published in the British Journal of Sports Medicine , which compared athletic performance between cis- and transgender men and women, suggests it’s a lot more complicated than that. “Trans women are not biological men,” Yannis Pitsiladis, one of the study’s authors, told the  New York Times . 

The only measurement in this study in which transgender women categorically outperformed cisgender women was grip strength , which can be an indicator of overall strength. But the trans women who were surveyed had lower lung capacity, VO2 max (a measure of how efficiently oxygen is transported throughout the body, a marker of endurance capacity), and jump height.

This suggests that a transgender woman competing at, say, volleyball or long-distance running, could actually be at a disadvantage compared to her cisgender counterparts. This is likely because once someone assigned male at birth starts hormone replacement therapy, their strength and muscle mass relative to their frame declines, leaving them to “carry this big skeleton with a smaller engine,” as Pitsiladis puts it.

These findings are in line with real-world performances. The few trans athletes out there have been competitive without being head-and-shoulders superior to their cisgender peers. Transgender gravel cyclist Austin Killips won a couple of races in 2023—and finished somewhere in the top ten in a few others—which led the Union Cycliste Internationale, the sport’s governing body, to institute a sweeping ban of transgender women racing in the women’s category. Laurel Hubbard, a transgender weight lifter from New Zealand, qualified for the Tokyo Olympics but couldn’t execute a lift in her event.

More than anything, this study highlights just how little we actually know about how transition-related medical care affects an athlete’s body and performance. It’s not a definitive piece of research by itself—the sample size is very small, with only 23 trans women, 21 cisgender women, 19 cisgender men, and 12 trans men. All of the transgender athletes included were at least one year into hormone replacement therapy, but the researchers suggest collecting long-term data, following athletes over the course of their transition, to confirm that the measured differences are caused by gender-affirming care.

“This fear that trans women aren’t really women, that they’re men who are invading women’s sports, and that trans women will carry all of their male athleticism, their athletic capabilities, into women’s sports—neither of those things are true,” Joanna Harper, who researches transgender athletes at Oregon Health and Science University and was not involved with the study, told the New York Times. 

The International Olympic Committee, who funded the study, has gone through several different iterations of their gender inclusion policy. They presently defer to the rules of the international governing bodies of respective sports, after replacing fairly strict guidelines that required medical examinations and a cap on women’s testosterone levels in 2021.

• paris olympics

Popular on Outside Online

Enjoy coverage of racing, history, food, culture, travel, and tech with access to unlimited digital content from Outside Network's iconic brands.

Healthy Living

• Clean Eating
• Vegetarian Times
• Yoga Journal
• Fly Fishing Film Tour
• National Park Trips
• Warren Miller
• Fastest Known Time
• Trail Runner
• Women's Running
• Bicycle Retailer & Industry News
• FinisherPix
• Outside Events Cycling Series
• Outside Shop

© 2024 Outside Interactive, Inc