regression analysis in research proposal

• Privacy Policy

Buy Me a Coffee

Home » Regression Analysis – Methods, Types and Examples

Regression Analysis – Methods, Types and Examples

Table of Contents

Regression Analysis

Regression analysis is a set of statistical processes for estimating the relationships among variables . It includes many techniques for modeling and analyzing several variables when the focus is on the relationship between a dependent variable and one or more independent variables (or ‘predictors’).

Regression Analysis Methodology

Here is a general methodology for performing regression analysis:

• Define the research question: Clearly state the research question or hypothesis you want to investigate. Identify the dependent variable (also called the response variable or outcome variable) and the independent variables (also called predictor variables or explanatory variables) that you believe are related to the dependent variable.
• Collect data: Gather the data for the dependent variable and independent variables. Ensure that the data is relevant, accurate, and representative of the population or phenomenon you are studying.
• Explore the data: Perform exploratory data analysis to understand the characteristics of the data, identify any missing values or outliers, and assess the relationships between variables through scatter plots, histograms, or summary statistics.
• Choose the regression model: Select an appropriate regression model based on the nature of the variables and the research question. Common regression models include linear regression, multiple regression, logistic regression, polynomial regression, and time series regression, among others.
• Assess assumptions: Check the assumptions of the regression model. Some common assumptions include linearity (the relationship between variables is linear), independence of errors, homoscedasticity (constant variance of errors), and normality of errors. Violation of these assumptions may require additional steps or alternative models.
• Estimate the model: Use a suitable method to estimate the parameters of the regression model. The most common method is ordinary least squares (OLS), which minimizes the sum of squared differences between the observed and predicted values of the dependent variable.
• I nterpret the results: Analyze the estimated coefficients, p-values, confidence intervals, and goodness-of-fit measures (e.g., R-squared) to interpret the results. Determine the significance and direction of the relationships between the independent variables and the dependent variable.
• Evaluate model performance: Assess the overall performance of the regression model using appropriate measures, such as R-squared, adjusted R-squared, and root mean squared error (RMSE). These measures indicate how well the model fits the data and how much of the variation in the dependent variable is explained by the independent variables.
• Test assumptions and diagnose problems: Check the residuals (the differences between observed and predicted values) for any patterns or deviations from assumptions. Conduct diagnostic tests, such as examining residual plots, testing for multicollinearity among independent variables, and assessing heteroscedasticity or autocorrelation, if applicable.
• Make predictions and draw conclusions: Once you have a satisfactory model, use it to make predictions on new or unseen data. Draw conclusions based on the results of the analysis, considering the limitations and potential implications of the findings.

Types of Regression Analysis

Types of Regression Analysis are as follows:

Linear Regression

Linear regression is the most basic and widely used form of regression analysis. It models the linear relationship between a dependent variable and one or more independent variables. The goal is to find the best-fitting line that minimizes the sum of squared differences between observed and predicted values.

Multiple Regression

Multiple regression extends linear regression by incorporating two or more independent variables to predict the dependent variable. It allows for examining the simultaneous effects of multiple predictors on the outcome variable.

Polynomial Regression

Polynomial regression models non-linear relationships between variables by adding polynomial terms (e.g., squared or cubic terms) to the regression equation. It can capture curved or nonlinear patterns in the data.

Logistic Regression

Logistic regression is used when the dependent variable is binary or categorical. It models the probability of the occurrence of a certain event or outcome based on the independent variables. Logistic regression estimates the coefficients using the logistic function, which transforms the linear combination of predictors into a probability.

Ridge Regression and Lasso Regression

Ridge regression and Lasso regression are techniques used for addressing multicollinearity (high correlation between independent variables) and variable selection. Both methods introduce a penalty term to the regression equation to shrink or eliminate less important variables. Ridge regression uses L2 regularization, while Lasso regression uses L1 regularization.

Time Series Regression

Time series regression analyzes the relationship between a dependent variable and independent variables when the data is collected over time. It accounts for autocorrelation and trends in the data and is used in forecasting and studying temporal relationships.

Nonlinear Regression

Nonlinear regression models are used when the relationship between the dependent variable and independent variables is not linear. These models can take various functional forms and require estimation techniques different from those used in linear regression.

Poisson Regression

Poisson regression is employed when the dependent variable represents count data. It models the relationship between the independent variables and the expected count, assuming a Poisson distribution for the dependent variable.

Generalized Linear Models (GLM)

GLMs are a flexible class of regression models that extend the linear regression framework to handle different types of dependent variables, including binary, count, and continuous variables. GLMs incorporate various probability distributions and link functions.

Regression Analysis Formulas

Regression analysis involves estimating the parameters of a regression model to describe the relationship between the dependent variable (Y) and one or more independent variables (X). Here are the basic formulas for linear regression, multiple regression, and logistic regression:

Linear Regression:

Simple Linear Regression Model: Y = β0 + β1X + ε

Multiple Linear Regression Model: Y = β0 + β1X1 + β2X2 + … + βnXn + ε

In both formulas:

• Y represents the dependent variable (response variable).
• X represents the independent variable(s) (predictor variable(s)).
• β0, β1, β2, …, βn are the regression coefficients or parameters that need to be estimated.
• ε represents the error term or residual (the difference between the observed and predicted values).

Multiple Regression:

Multiple regression extends the concept of simple linear regression by including multiple independent variables.

Multiple Regression Model: Y = β0 + β1X1 + β2X2 + … + βnXn + ε

The formulas are similar to those in linear regression, with the addition of more independent variables.

Logistic Regression:

Logistic regression is used when the dependent variable is binary or categorical. The logistic regression model applies a logistic or sigmoid function to the linear combination of the independent variables.

Logistic Regression Model: p = 1 / (1 + e^-(β0 + β1X1 + β2X2 + … + βnXn))

In the formula:

• p represents the probability of the event occurring (e.g., the probability of success or belonging to a certain category).
• X1, X2, …, Xn represent the independent variables.
• e is the base of the natural logarithm.

The logistic function ensures that the predicted probabilities lie between 0 and 1, allowing for binary classification.

Regression Analysis Examples

Regression Analysis Examples are as follows:

• Stock Market Prediction: Regression analysis can be used to predict stock prices based on various factors such as historical prices, trading volume, news sentiment, and economic indicators. Traders and investors can use this analysis to make informed decisions about buying or selling stocks.
• Demand Forecasting: In retail and e-commerce, real-time It can help forecast demand for products. By analyzing historical sales data along with real-time data such as website traffic, promotional activities, and market trends, businesses can adjust their inventory levels and production schedules to meet customer demand more effectively.
• Energy Load Forecasting: Utility companies often use real-time regression analysis to forecast electricity demand. By analyzing historical energy consumption data, weather conditions, and other relevant factors, they can predict future energy loads. This information helps them optimize power generation and distribution, ensuring a stable and efficient energy supply.
• Online Advertising Performance: It can be used to assess the performance of online advertising campaigns. By analyzing real-time data on ad impressions, click-through rates, conversion rates, and other metrics, advertisers can adjust their targeting, messaging, and ad placement strategies to maximize their return on investment.
• Predictive Maintenance: Regression analysis can be applied to predict equipment failures or maintenance needs. By continuously monitoring sensor data from machines or vehicles, regression models can identify patterns or anomalies that indicate potential failures. This enables proactive maintenance, reducing downtime and optimizing maintenance schedules.
• Financial Risk Assessment: Real-time regression analysis can help financial institutions assess the risk associated with lending or investment decisions. By analyzing real-time data on factors such as borrower financials, market conditions, and macroeconomic indicators, regression models can estimate the likelihood of default or assess the risk-return tradeoff for investment portfolios.

Importance of Regression Analysis

Importance of Regression Analysis is as follows:

• Relationship Identification: Regression analysis helps in identifying and quantifying the relationship between a dependent variable and one or more independent variables. It allows us to determine how changes in independent variables impact the dependent variable. This information is crucial for decision-making, planning, and forecasting.
• Prediction and Forecasting: Regression analysis enables us to make predictions and forecasts based on the relationships identified. By estimating the values of the dependent variable using known values of independent variables, regression models can provide valuable insights into future outcomes. This is particularly useful in business, economics, finance, and other fields where forecasting is vital for planning and strategy development.
• Causality Assessment: While correlation does not imply causation, regression analysis provides a framework for assessing causality by considering the direction and strength of the relationship between variables. It allows researchers to control for other factors and assess the impact of a specific independent variable on the dependent variable. This helps in determining the causal effect and identifying significant factors that influence outcomes.
• Model Building and Variable Selection: Regression analysis aids in model building by determining the most appropriate functional form of the relationship between variables. It helps researchers select relevant independent variables and eliminate irrelevant ones, reducing complexity and improving model accuracy. This process is crucial for creating robust and interpretable models.
• Hypothesis Testing: Regression analysis provides a statistical framework for hypothesis testing. Researchers can test the significance of individual coefficients, assess the overall model fit, and determine if the relationship between variables is statistically significant. This allows for rigorous analysis and validation of research hypotheses.
• Policy Evaluation and Decision-Making: Regression analysis plays a vital role in policy evaluation and decision-making processes. By analyzing historical data, researchers can evaluate the effectiveness of policy interventions and identify the key factors contributing to certain outcomes. This information helps policymakers make informed decisions, allocate resources effectively, and optimize policy implementation.
• Risk Assessment and Control: Regression analysis can be used for risk assessment and control purposes. By analyzing historical data, organizations can identify risk factors and develop models that predict the likelihood of certain outcomes, such as defaults, accidents, or failures. This enables proactive risk management, allowing organizations to take preventive measures and mitigate potential risks.

When to Use Regression Analysis

• Prediction : Regression analysis is often employed to predict the value of the dependent variable based on the values of independent variables. For example, you might use regression to predict sales based on advertising expenditure, or to predict a student’s academic performance based on variables like study time, attendance, and previous grades.
• Relationship analysis: Regression can help determine the strength and direction of the relationship between variables. It can be used to examine whether there is a linear association between variables, identify which independent variables have a significant impact on the dependent variable, and quantify the magnitude of those effects.
• Causal inference: Regression analysis can be used to explore cause-and-effect relationships by controlling for other variables. For example, in a medical study, you might use regression to determine the impact of a specific treatment while accounting for other factors like age, gender, and lifestyle.
• Forecasting : Regression models can be utilized to forecast future trends or outcomes. By fitting a regression model to historical data, you can make predictions about future values of the dependent variable based on changes in the independent variables.
• Model evaluation: Regression analysis can be used to evaluate the performance of a model or test the significance of variables. You can assess how well the model fits the data, determine if additional variables improve the model’s predictive power, or test the statistical significance of coefficients.
• Data exploration : Regression analysis can help uncover patterns and insights in the data. By examining the relationships between variables, you can gain a deeper understanding of the data set and identify potential patterns, outliers, or influential observations.

Applications of Regression Analysis

Here are some common applications of regression analysis:

• Economic Forecasting: Regression analysis is frequently employed in economics to forecast variables such as GDP growth, inflation rates, or stock market performance. By analyzing historical data and identifying the underlying relationships, economists can make predictions about future economic conditions.
• Financial Analysis: Regression analysis plays a crucial role in financial analysis, such as predicting stock prices or evaluating the impact of financial factors on company performance. It helps analysts understand how variables like interest rates, company earnings, or market indices influence financial outcomes.
• Marketing Research: Regression analysis helps marketers understand consumer behavior and make data-driven decisions. It can be used to predict sales based on advertising expenditures, pricing strategies, or demographic variables. Regression models provide insights into which marketing efforts are most effective and help optimize marketing campaigns.
• Health Sciences: Regression analysis is extensively used in medical research and public health studies. It helps examine the relationship between risk factors and health outcomes, such as the impact of smoking on lung cancer or the relationship between diet and heart disease. Regression analysis also helps in predicting health outcomes based on various factors like age, genetic markers, or lifestyle choices.
• Social Sciences: Regression analysis is widely used in social sciences like sociology, psychology, and education research. Researchers can investigate the impact of variables like income, education level, or social factors on various outcomes such as crime rates, academic performance, or job satisfaction.
• Operations Research: Regression analysis is applied in operations research to optimize processes and improve efficiency. For example, it can be used to predict demand based on historical sales data, determine the factors influencing production output, or optimize supply chain logistics.
• Environmental Studies: Regression analysis helps in understanding and predicting environmental phenomena. It can be used to analyze the impact of factors like temperature, pollution levels, or land use patterns on phenomena such as species diversity, water quality, or climate change.
• Sports Analytics: Regression analysis is increasingly used in sports analytics to gain insights into player performance, team strategies, and game outcomes. It helps analyze the relationship between various factors like player statistics, coaching strategies, or environmental conditions and their impact on game outcomes.

Advantages and Disadvantages of Regression Analysis

About the author.

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Cluster Analysis – Types, Methods and Examples

Discriminant Analysis – Methods, Types and...

MANOVA (Multivariate Analysis of Variance) –...

Documentary Analysis – Methods, Applications and...

ANOVA (Analysis of variance) – Formulas, Types...

Graphical Methods – Types, Examples and Guide

Regression Analysis

Regression analysis is a quantitative research method which is used when the study involves modelling and analysing several variables, where the relationship includes a dependent variable and one or more independent variables. In simple terms, regression analysis is a quantitative method used to test the nature of relationships between a dependent variable and one or more independent variables.

The basic form of regression models includes unknown parameters (β), independent variables (X), and the dependent variable (Y).

Regression model, basically, specifies the relation of dependent variable (Y) to a function combination of independent variables (X) and unknown parameters (β)

                                    Y  ≈  f (X, β)   

Regression equation can be used to predict the values of ‘y’, if the value of ‘x’ is given, and both ‘y’ and ‘x’ are the two sets of measures of a sample size of ‘n’. The formulae for regression equation would be

Do not be intimidated by visual complexity of correlation and regression formulae above. You don’t have to apply the formula manually, and correlation and regression analyses can be run with the application of popular analytical software such as Microsoft Excel, Microsoft Access, SPSS and others.

Linear regression analysis is based on the following set of assumptions:

1. Assumption of linearity . There is a linear relationship between dependent and independent variables.

2. Assumption of homoscedasticity . Data values for dependent and independent variables have equal variances.

3. Assumption of absence of collinearity or multicollinearity . There is no correlation between two or more independent variables.

4. Assumption of normal distribution . The data for the independent variables and dependent variable are normally distributed

My e-book,  The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance  offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

Multiple Regression Analysis Example with Conceptual Framework

Data analysis using multiple regression analysis is a fairly common tool used in statistics. Many graduate students find this too complicated to understand. However, this is not that difficult to do, especially with computers as everyday household items nowadays. You can now quickly analyze more than just two sets of variables in your research using multiple regression analysis. 

How is multiple regression analysis done? This article explains this handy statistical test when dealing with many variables, then provides an example of a research using multiple regression analysis to show how it works. It explains how research using multiple regression analysis is conducted.

Multiple regression is often confused with multivariate regression. Multivariate regression, while also using several variables, deals with more than one dependent variable . Karen Grace-Martin clearly explains the difference in her post on the difference between the Multiple Regression Model and Multivariate Regression Model .

Table of Contents

Statistical software applications used in computing multiple regression analysis.

Multiple regression analysis is a powerful statistical test used to find the relationship between a given dependent variable and a set of independent variables .

Using multiple regression analysis requires a dedicated statistical software like the popular  Statistical Package for the Social Sciences (SPSS) , Statistica, Microstat, and open-source statistical software applications like SOFA statistics and Jasp, among other sophisticated statistical packages.

Two decades ago, it will be near impossible to do the calculations using the obsolete simple calculator replaced by smartphones. 

However, a standard spreadsheet application like Microsoft Excel can help you compute and model the relationship between the dependent variable and a set of predictor or independent variables. But you cannot do this without activating first the setting of statistical tools that ship with MS Excel.

Activating MS Excel

To activate the add-in for multiple regression analysis in MS Excel, you may view the two-minute Youtube tutorial below. If you already have this installed on your computer, you may proceed to the next section.

Multiple Regression Analysis Example

I will illustrate the use of multiple regression analysis by citing the actual research activity that my graduate students undertook two years ago.

The study pertains to identifying the factors predicting a current problem among high school students, the long hours they spend online for a variety of reasons. The purpose is to address many parents’ concerns about their difficulty of weaning their children away from the lures of online gaming, social networking, and other engaging virtual activities.

Review of Literature on Internet Use and Its Effect on Children

Upon reviewing the literature, the graduate students discovered that very few studies were conducted on the subject. Studies on problems associated with internet use are still in its infancy as the Internet has just begun to influence everyone’s life.

Hence, with my guidance, the group of six graduate students comprising school administrators, heads of elementary and high schools, and faculty members proceeded with the study.

Given that there is a need to use a computer to analyze multiple variable data, a principal who is nearing retirement was “forced” to buy a laptop, as she had none. Anyhow, she is very much open-minded and performed the class activities that require data analysis with much enthusiasm.

The Research on High School Students’ Use of the Internet

The brief research using multiple regression analysis is a broad study or analysis of the reasons or underlying factors that significantly relate to the number of hours devoted by high school students in using the Internet. The regression analysis is broad because it only focuses on the total number of hours devoted by high school students to activities online.

They correlated the time high school students spent online with their profile. The students’ profile comprised more than two independent variables, hence the term “multiple.” The independent variables are age, gender, relationship with the mother, and relationship with the father.

The statement of the problem in this study is:

“Is there a significant relationship between the total number of hours spent online and the students’ age, gender, relationship with their mother, and relationship with their father?”

Their parents’ relationship was gauged using a scale of 1 to 10, 1 being a poor relationship, and 10 being the best experience with parents. The figure below shows the paradigm of the study.

Notice that in research using multiple regression studies such as this, there is only one dependent variable involved. That is the total number of hours spent by high school students online.

Although many studies have identified factors that influence the use of the internet, it is standard practice to include the respondents’ profile among the set of predictor or independent variables. Hence, the standard variables age and gender are included in the multiple regression analysis.

Also, among the set of variables that may influence internet use, only the relationship between children and their parents was tested. The intention of this research using multiple regression analysis is to determine if parents spend quality time establishing strong emotional bonds between them and their children.

Findings of the Research Using Multiple Regression Analysis

What are the findings of this exploratory study? This quickly done example of a research using multiple regression analysis revealed an interesting finding.

The number of hours spent online relates significantly to the number of hours spent by a parent, specifically the mother, with her child. These two factors are inversely or negatively correlated.

The relationship means that the greater the number of hours spent by the mother with her child to establish a closer emotional bond, the fewer hours spent by her child using the internet. The number of hours spent by the children online relates significantly to the mother’s number of hours interacting with their children.

The number of hours spent by the children online relates significantly to the mother’s number of hours interacting with their children.

While this example of a research using multiple regression analysis may be a significant finding, the mother-child bond accounts for only a small percentage of the variance in total hours spent by the child online. This observation means that other factors need to be addressed to resolve long waking hours and abandonment of serious study of lessons by children.

But establishing a close bond between mother and child is a good start. Undertaking more investigations along this research concern will help strengthen the findings of this study.

The above example of a research using multiple regression analysis shows that the statistical tool is useful in predicting dependent variables’ behavior. In the above case, this is the number of hours spent by students online.

The identification of significant predictors can help determine the correct intervention to resolve the problem. Using multiple regression approaches prevents unnecessary costs for remedies that do not address an issue or a question.

Thus, this example of a research using multiple regression analysis streamlines solutions and focuses on those influential factors that must be given attention.

Once you become an expert in using multiple regression in analyzing data, you can try your hands on multivariate regression where you will deal with more than one dependent variable.

©2012 November 11 Patrick Regoniel Updated: 14 November 2020

Related Posts

Analyzing Data in Qualitative Research

Willingness to Pay Example: Drivers’ WTP to Reduce Air Pollution

5 ways on how to generate ideas even when you are not inspired, about the author, patrick regoniel.

Dr. Regoniel, a faculty member of the graduate school, served as consultant to various environmental research and development projects covering issues and concerns on climate change, coral reef resources and management, economic valuation of environmental and natural resources, mining, and waste management and pollution. He has extensive experience on applied statistics, systems modelling and analysis, an avid practitioner of LaTeX, and a multidisciplinary web developer. He leverages pioneering AI-powered content creation tools to produce unique and comprehensive articles in this website.

mostly in monasteries.

Manuscript is a collective name for texts

the example is good but lacks the table of regression results. With the tables, a student could learn more on how to interpret regression results

this is so enlightening,hope it reaches most of the parents…

nice; but it is not good enough for reference

This is an action research Daniel. And I have updated it here. It can set off other studies. And please take note that blogs nowadays are already recognized sources of information. Please read my post here on why this is so: https://simplyeducate.me/wordpress_Y//2019/09/26/using-blogs-in-education/

Was this study published? It may have important implications

Dear Gabe, this study was presented by one of my students in a conference. I am just unsure if she was able to publish it in a journal.

How to conduct a meta-analysis in eight steps: a practical guide

• Open access
• Published: 30 November 2021
• Volume 72 , pages 1–19, ( 2022 )

Cite this article

You have full access to this open access article

• Christopher Hansen 1 ,
• Holger Steinmetz 2 &
• Jörn Block 3 , 4 , 5  

139k Accesses

43 Citations

157 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

1 Introduction

“Scientists have known for centuries that a single study will not resolve a major issue. Indeed, a small sample study will not even resolve a minor issue. Thus, the foundation of science is the cumulation of knowledge from the results of many studies.” (Hunter et al. 1982 , p. 10)

Meta-analysis is a central method for knowledge accumulation in many scientific fields (Aguinis et al. 2011c ; Kepes et al. 2013 ). Similar to a narrative review, it serves as a synopsis of a research question or field. However, going beyond a narrative summary of key findings, a meta-analysis adds value in providing a quantitative assessment of the relationship between two target variables or the effectiveness of an intervention (Gurevitch et al. 2018 ). Also, it can be used to test competing theoretical assumptions against each other or to identify important moderators where the results of different primary studies differ from each other (Aguinis et al. 2011b ; Bergh et al. 2016 ). Rooted in the synthesis of the effectiveness of medical and psychological interventions in the 1970s (Glass 2015 ; Gurevitch et al. 2018 ), meta-analysis is nowadays also an established method in management research and related fields.

The increasing importance of meta-analysis in management research has resulted in the publication of guidelines in recent years that discuss the merits and best practices in various fields, such as general management (Bergh et al. 2016 ; Combs et al. 2019 ; Gonzalez-Mulé and Aguinis 2018 ), international business (Steel et al. 2021 ), economics and finance (Geyer-Klingeberg et al. 2020 ; Havranek et al. 2020 ), marketing (Eisend 2017 ; Grewal et al. 2018 ), and organizational studies (DeSimone et al. 2020 ; Rudolph et al. 2020 ). These articles discuss existing and trending methods and propose solutions for often experienced problems. This editorial briefly summarizes the insights of these papers; provides a workflow of the essential steps in conducting a meta-analysis; suggests state-of-the art methodological procedures; and points to other articles for in-depth investigation. Thus, this article has two goals: (1) based on the findings of previous editorials and methodological articles, it defines methodological recommendations for meta-analyses submitted to Management Review Quarterly (MRQ); and (2) it serves as a practical guide for researchers who have little experience with meta-analysis as a method but plan to conduct one in the future.

2 Eight steps in conducting a meta-analysis

2.1 step 1: defining the research question.

The first step in conducting a meta-analysis, as with any other empirical study, is the definition of the research question. Most importantly, the research question determines the realm of constructs to be considered or the type of interventions whose effects shall be analyzed. When defining the research question, two hurdles might develop. First, when defining an adequate study scope, researchers must consider that the number of publications has grown exponentially in many fields of research in recent decades (Fortunato et al. 2018 ). On the one hand, a larger number of studies increases the potentially relevant literature basis and enables researchers to conduct meta-analyses. Conversely, scanning a large amount of studies that could be potentially relevant for the meta-analysis results in a perhaps unmanageable workload. Thus, Steel et al. ( 2021 ) highlight the importance of balancing manageability and relevance when defining the research question. Second, similar to the number of primary studies also the number of meta-analyses in management research has grown strongly in recent years (Geyer-Klingeberg et al. 2020 ; Rauch 2020 ; Schwab 2015 ). Therefore, it is likely that one or several meta-analyses for many topics of high scholarly interest already exist. However, this should not deter researchers from investigating their research questions. One possibility is to consider moderators or mediators of a relationship that have previously been ignored. For example, a meta-analysis about startup performance could investigate the impact of different ways to measure the performance construct (e.g., growth vs. profitability vs. survival time) or certain characteristics of the founders as moderators. Another possibility is to replicate previous meta-analyses and test whether their findings can be confirmed with an updated sample of primary studies or newly developed methods. Frequent replications and updates of meta-analyses are important contributions to cumulative science and are increasingly called for by the research community (Anderson & Kichkha 2017 ; Steel et al. 2021 ). Consistent with its focus on replication studies (Block and Kuckertz 2018 ), MRQ therefore also invites authors to submit replication meta-analyses.

2.2 Step 2: literature search

2.2.1 search strategies.

Similar to conducting a literature review, the search process of a meta-analysis should be systematic, reproducible, and transparent, resulting in a sample that includes all relevant studies (Fisch and Block 2018 ; Gusenbauer and Haddaway 2020 ). There are several identification strategies for relevant primary studies when compiling meta-analytical datasets (Harari et al. 2020 ). First, previous meta-analyses on the same or a related topic may provide lists of included studies that offer a good starting point to identify and become familiar with the relevant literature. This practice is also applicable to topic-related literature reviews, which often summarize the central findings of the reviewed articles in systematic tables. Both article types likely include the most prominent studies of a research field. The most common and important search strategy, however, is a keyword search in electronic databases (Harari et al. 2020 ). This strategy will probably yield the largest number of relevant studies, particularly so-called ‘grey literature’, which may not be considered by literature reviews. Gusenbauer and Haddaway ( 2020 ) provide a detailed overview of 34 scientific databases, of which 18 are multidisciplinary or have a focus on management sciences, along with their suitability for literature synthesis. To prevent biased results due to the scope or journal coverage of one database, researchers should use at least two different databases (DeSimone et al. 2020 ; Martín-Martín et al. 2021 ; Mongeon & Paul-Hus 2016 ). However, a database search can easily lead to an overload of potentially relevant studies. For example, key term searches in Google Scholar for “entrepreneurial intention” and “firm diversification” resulted in more than 660,000 and 810,000 hits, respectively. Footnote 1 Therefore, a precise research question and precise search terms using Boolean operators are advisable (Gusenbauer and Haddaway 2020 ). Addressing the challenge of identifying relevant articles in the growing number of database publications, (semi)automated approaches using text mining and machine learning (Bosco et al. 2017 ; O’Mara-Eves et al. 2015 ; Ouzzani et al. 2016 ; Thomas et al. 2017 ) can also be promising and time-saving search tools in the future. Also, some electronic databases offer the possibility to track forward citations of influential studies and thereby identify further relevant articles. Finally, collecting unpublished or undetected studies through conferences, personal contact with (leading) scholars, or listservs can be strategies to increase the study sample size (Grewal et al. 2018 ; Harari et al. 2020 ; Pigott and Polanin 2020 ).

2.2.2 Study inclusion criteria and sample composition

Next, researchers must decide which studies to include in the meta-analysis. Some guidelines for literature reviews recommend limiting the sample to studies published in renowned academic journals to ensure the quality of findings (e.g., Kraus et al. 2020 ). For meta-analysis, however, Steel et al. ( 2021 ) advocate for the inclusion of all available studies, including grey literature, to prevent selection biases based on availability, cost, familiarity, and language (Rothstein et al. 2005 ), or the “Matthew effect”, which denotes the phenomenon that highly cited articles are found faster than less cited articles (Merton 1968 ). Harrison et al. ( 2017 ) find that the effects of published studies in management are inflated on average by 30% compared to unpublished studies. This so-called publication bias or “file drawer problem” (Rosenthal 1979 ) results from the preference of academia to publish more statistically significant and less statistically insignificant study results. Owen and Li ( 2020 ) showed that publication bias is particularly severe when variables of interest are used as key variables rather than control variables. To consider the true effect size of a target variable or relationship, the inclusion of all types of research outputs is therefore recommended (Polanin et al. 2016 ). Different test procedures to identify publication bias are discussed subsequently in Step 7.

In addition to the decision of whether to include certain study types (i.e., published vs. unpublished studies), there can be other reasons to exclude studies that are identified in the search process. These reasons can be manifold and are primarily related to the specific research question and methodological peculiarities. For example, studies identified by keyword search might not qualify thematically after all, may use unsuitable variable measurements, or may not report usable effect sizes. Furthermore, there might be multiple studies by the same authors using similar datasets. If they do not differ sufficiently in terms of their sample characteristics or variables used, only one of these studies should be included to prevent bias from duplicates (Wood 2008 ; see this article for a detection heuristic).

In general, the screening process should be conducted stepwise, beginning with a removal of duplicate citations from different databases, followed by abstract screening to exclude clearly unsuitable studies and a final full-text screening of the remaining articles (Pigott and Polanin 2020 ). A graphical tool to systematically document the sample selection process is the PRISMA flow diagram (Moher et al. 2009 ). Page et al. ( 2021 ) recently presented an updated version of the PRISMA statement, including an extended item checklist and flow diagram to report the study process and findings.

2.3 Step 3: choice of the effect size measure

2.3.1 types of effect sizes.

The two most common meta-analytical effect size measures in management studies are (z-transformed) correlation coefficients and standardized mean differences (Aguinis et al. 2011a ; Geyskens et al. 2009 ). However, meta-analyses in management science and related fields may not be limited to those two effect size measures but rather depend on the subfield of investigation (Borenstein 2009 ; Stanley and Doucouliagos 2012 ). In economics and finance, researchers are more interested in the examination of elasticities and marginal effects extracted from regression models than in pure bivariate correlations (Stanley and Doucouliagos 2012 ). Regression coefficients can also be converted to partial correlation coefficients based on their t-statistics to make regression results comparable across studies (Stanley and Doucouliagos 2012 ). Although some meta-analyses in management research have combined bivariate and partial correlations in their study samples, Aloe ( 2015 ) and Combs et al. ( 2019 ) advise researchers not to use this practice. Most importantly, they argue that the effect size strength of partial correlations depends on the other variables included in the regression model and is therefore incomparable to bivariate correlations (Schmidt and Hunter 2015 ), resulting in a possible bias of the meta-analytic results (Roth et al. 2018 ). We endorse this opinion. If at all, we recommend separate analyses for each measure. In addition to these measures, survival rates, risk ratios or odds ratios, which are common measures in medical research (Borenstein 2009 ), can be suitable effect sizes for specific management research questions, such as understanding the determinants of the survival of startup companies. To summarize, the choice of a suitable effect size is often taken away from the researcher because it is typically dependent on the investigated research question as well as the conventions of the specific research field (Cheung and Vijayakumar 2016 ).

2.3.2 Conversion of effect sizes to a common measure

After having defined the primary effect size measure for the meta-analysis, it might become necessary in the later coding process to convert study findings that are reported in effect sizes that are different from the chosen primary effect size. For example, a study might report only descriptive statistics for two study groups but no correlation coefficient, which is used as the primary effect size measure in the meta-analysis. Different effect size measures can be harmonized using conversion formulae, which are provided by standard method books such as Borenstein et al. ( 2009 ) or Lipsey and Wilson ( 2001 ). There also exist online effect size calculators for meta-analysis. Footnote 2

2.4 Step 4: choice of the analytical method used

Choosing which meta-analytical method to use is directly connected to the research question of the meta-analysis. Research questions in meta-analyses can address a relationship between constructs or an effect of an intervention in a general manner, or they can focus on moderating or mediating effects. There are four meta-analytical methods that are primarily used in contemporary management research (Combs et al. 2019 ; Geyer-Klingeberg et al. 2020 ), which allow the investigation of these different types of research questions: traditional univariate meta-analysis, meta-regression, meta-analytic structural equation modeling, and qualitative meta-analysis (Hoon 2013 ). While the first three are quantitative, the latter summarizes qualitative findings. Table 1 summarizes the key characteristics of the three quantitative methods.

2.4.1 Univariate meta-analysis

In its traditional form, a meta-analysis reports a weighted mean effect size for the relationship or intervention of investigation and provides information on the magnitude of variance among primary studies (Aguinis et al. 2011c ; Borenstein et al. 2009 ). Accordingly, it serves as a quantitative synthesis of a research field (Borenstein et al. 2009 ; Geyskens et al. 2009 ). Prominent traditional approaches have been developed, for example, by Hedges and Olkin ( 1985 ) or Hunter and Schmidt ( 1990 , 2004 ). However, going beyond its simple summary function, the traditional approach has limitations in explaining the observed variance among findings (Gonzalez-Mulé and Aguinis 2018 ). To identify moderators (or boundary conditions) of the relationship of interest, meta-analysts can create subgroups and investigate differences between those groups (Borenstein and Higgins 2013 ; Hunter and Schmidt 2004 ). Potential moderators can be study characteristics (e.g., whether a study is published vs. unpublished), sample characteristics (e.g., study country, industry focus, or type of survey/experiment participants), or measurement artifacts (e.g., different types of variable measurements). The univariate approach is thus suitable to identify the overall direction of a relationship and can serve as a good starting point for additional analyses. However, due to its limitations in examining boundary conditions and developing theory, the univariate approach on its own is currently oftentimes viewed as not sufficient (Rauch 2020 ; Shaw and Ertug 2017 ).

2.4.2 Meta-regression analysis

Meta-regression analysis (Hedges and Olkin 1985 ; Lipsey and Wilson 2001 ; Stanley and Jarrell 1989 ) aims to investigate the heterogeneity among observed effect sizes by testing multiple potential moderators simultaneously. In meta-regression, the coded effect size is used as the dependent variable and is regressed on a list of moderator variables. These moderator variables can be categorical variables as described previously in the traditional univariate approach or (semi)continuous variables such as country scores that are merged with the meta-analytical data. Thus, meta-regression analysis overcomes the disadvantages of the traditional approach, which only allows us to investigate moderators singularly using dichotomized subgroups (Combs et al. 2019 ; Gonzalez-Mulé and Aguinis 2018 ). These possibilities allow a more fine-grained analysis of research questions that are related to moderating effects. However, Schmidt ( 2017 ) critically notes that the number of effect sizes in the meta-analytical sample must be sufficiently large to produce reliable results when investigating multiple moderators simultaneously in a meta-regression. For further reading, Tipton et al. ( 2019 ) outline the technical, conceptual, and practical developments of meta-regression over the last decades. Gonzalez-Mulé and Aguinis ( 2018 ) provide an overview of methodological choices and develop evidence-based best practices for future meta-analyses in management using meta-regression.

2.4.3 Meta-analytic structural equation modeling (MASEM)

MASEM is a combination of meta-analysis and structural equation modeling and allows to simultaneously investigate the relationships among several constructs in a path model. Researchers can use MASEM to test several competing theoretical models against each other or to identify mediation mechanisms in a chain of relationships (Bergh et al. 2016 ). This method is typically performed in two steps (Cheung and Chan 2005 ): In Step 1, a pooled correlation matrix is derived, which includes the meta-analytical mean effect sizes for all variable combinations; Step 2 then uses this matrix to fit the path model. While MASEM was based primarily on traditional univariate meta-analysis to derive the pooled correlation matrix in its early years (Viswesvaran and Ones 1995 ), more advanced methods, such as the GLS approach (Becker 1992 , 1995 ) or the TSSEM approach (Cheung and Chan 2005 ), have been subsequently developed. Cheung ( 2015a ) and Jak ( 2015 ) provide an overview of these approaches in their books with exemplary code. For datasets with more complex data structures, Wilson et al. ( 2016 ) also developed a multilevel approach that is related to the TSSEM approach in the second step. Bergh et al. ( 2016 ) discuss nine decision points and develop best practices for MASEM studies.

2.4.4 Qualitative meta-analysis

While the approaches explained above focus on quantitative outcomes of empirical studies, qualitative meta-analysis aims to synthesize qualitative findings from case studies (Hoon 2013 ; Rauch et al. 2014 ). The distinctive feature of qualitative case studies is their potential to provide in-depth information about specific contextual factors or to shed light on reasons for certain phenomena that cannot usually be investigated by quantitative studies (Rauch 2020 ; Rauch et al. 2014 ). In a qualitative meta-analysis, the identified case studies are systematically coded in a meta-synthesis protocol, which is then used to identify influential variables or patterns and to derive a meta-causal network (Hoon 2013 ). Thus, the insights of contextualized and typically nongeneralizable single studies are aggregated to a larger, more generalizable picture (Habersang et al. 2019 ). Although still the exception, this method can thus provide important contributions for academics in terms of theory development (Combs et al., 2019 ; Hoon 2013 ) and for practitioners in terms of evidence-based management or entrepreneurship (Rauch et al. 2014 ). Levitt ( 2018 ) provides a guide and discusses conceptual issues for conducting qualitative meta-analysis in psychology, which is also useful for management researchers.

2.5 Step 5: choice of software

Software solutions to perform meta-analyses range from built-in functions or additional packages of statistical software to software purely focused on meta-analyses and from commercial to open-source solutions. However, in addition to personal preferences, the choice of the most suitable software depends on the complexity of the methods used and the dataset itself (Cheung and Vijayakumar 2016 ). Meta-analysts therefore must carefully check if their preferred software is capable of performing the intended analysis.

Among commercial software providers, Stata (from version 16 on) offers built-in functions to perform various meta-analytical analyses or to produce various plots (Palmer and Sterne 2016 ). For SPSS and SAS, there exist several macros for meta-analyses provided by scholars, such as David B. Wilson or Andy P. Field and Raphael Gillet (Field and Gillett 2010 ). Footnote 3 Footnote 4 For researchers using the open-source software R (R Core Team 2021 ), Polanin et al. ( 2017 ) provide an overview of 63 meta-analysis packages and their functionalities. For new users, they recommend the package metafor (Viechtbauer 2010 ), which includes most necessary functions and for which the author Wolfgang Viechtbauer provides tutorials on his project website. Footnote 5 Footnote 6 In addition to packages and macros for statistical software, templates for Microsoft Excel have also been developed to conduct simple meta-analyses, such as Meta-Essentials by Suurmond et al. ( 2017 ). Footnote 7 Finally, programs purely dedicated to meta-analysis also exist, such as Comprehensive Meta-Analysis (Borenstein et al. 2013 ) or RevMan by The Cochrane Collaboration ( 2020 ).

2.6 Step 6: coding of effect sizes

2.6.1 coding sheet.

The first step in the coding process is the design of the coding sheet. A universal template does not exist because the design of the coding sheet depends on the methods used, the respective software, and the complexity of the research design. For univariate meta-analysis or meta-regression, data are typically coded in wide format. In its simplest form, when investigating a correlational relationship between two variables using the univariate approach, the coding sheet would contain a column for the study name or identifier, the effect size coded from the primary study, and the study sample size. However, such simple relationships are unlikely in management research because the included studies are typically not identical but differ in several respects. With more complex data structures or moderator variables being investigated, additional columns are added to the coding sheet to reflect the data characteristics. These variables can be coded as dummy, factor, or (semi)continuous variables and later used to perform a subgroup analysis or meta regression. For MASEM, the required data input format can deviate depending on the method used (e.g., TSSEM requires a list of correlation matrices as data input). For qualitative meta-analysis, the coding scheme typically summarizes the key qualitative findings and important contextual and conceptual information (see Hoon ( 2013 ) for a coding scheme for qualitative meta-analysis). Figure  1 shows an exemplary coding scheme for a quantitative meta-analysis on the correlational relationship between top-management team diversity and profitability. In addition to effect and sample sizes, information about the study country, firm type, and variable operationalizations are coded. The list could be extended by further study and sample characteristics.

Exemplary coding sheet for a meta-analysis on the relationship (correlation) between top-management team diversity and profitability

2.6.2 Inclusion of moderator or control variables

It is generally important to consider the intended research model and relevant nontarget variables before coding a meta-analytic dataset. For example, study characteristics can be important moderators or function as control variables in a meta-regression model. Similarly, control variables may be relevant in a MASEM approach to reduce confounding bias. Coding additional variables or constructs subsequently can be arduous if the sample of primary studies is large. However, the decision to include respective moderator or control variables, as in any empirical analysis, should always be based on strong (theoretical) rationales about how these variables can impact the investigated effect (Bernerth and Aguinis 2016 ; Bernerth et al. 2018 ; Thompson and Higgins 2002 ). While substantive moderators refer to theoretical constructs that act as buffers or enhancers of a supposed causal process, methodological moderators are features of the respective research designs that denote the methodological context of the observations and are important to control for systematic statistical particularities (Rudolph et al. 2020 ). Havranek et al. ( 2020 ) provide a list of recommended variables to code as potential moderators. While researchers may have clear expectations about the effects for some of these moderators, the concerns for other moderators may be tentative, and moderator analysis may be approached in a rather exploratory fashion. Thus, we argue that researchers should make full use of the meta-analytical design to obtain insights about potential context dependence that a primary study cannot achieve.

2.6.3 Treatment of multiple effect sizes in a study

A long-debated issue in conducting meta-analyses is whether to use only one or all available effect sizes for the same construct within a single primary study. For meta-analyses in management research, this question is fundamental because many empirical studies, particularly those relying on company databases, use multiple variables for the same construct to perform sensitivity analyses, resulting in multiple relevant effect sizes. In this case, researchers can either (randomly) select a single value, calculate a study average, or use the complete set of effect sizes (Bijmolt and Pieters 2001 ; López-López et al. 2018 ). Multiple effect sizes from the same study enrich the meta-analytic dataset and allow us to investigate the heterogeneity of the relationship of interest, such as different variable operationalizations (López-López et al. 2018 ; Moeyaert et al. 2017 ). However, including more than one effect size from the same study violates the independency assumption of observations (Cheung 2019 ; López-López et al. 2018 ), which can lead to biased results and erroneous conclusions (Gooty et al. 2021 ). We follow the recommendation of current best practice guides to take advantage of using all available effect size observations but to carefully consider interdependencies using appropriate methods such as multilevel models, panel regression models, or robust variance estimation (Cheung 2019 ; Geyer-Klingeberg et al. 2020 ; Gooty et al. 2021 ; López-López et al. 2018 ; Moeyaert et al. 2017 ).

2.7 Step 7: analysis

2.7.1 outlier analysis and tests for publication bias.

Before conducting the primary analysis, some preliminary sensitivity analyses might be necessary, which should ensure the robustness of the meta-analytical findings (Rudolph et al. 2020 ). First, influential outlier observations could potentially bias the observed results, particularly if the number of total effect sizes is small. Several statistical methods can be used to identify outliers in meta-analytical datasets (Aguinis et al. 2013 ; Viechtbauer and Cheung 2010 ). However, there is a debate about whether to keep or omit these observations. Anyhow, relevant studies should be closely inspected to infer an explanation about their deviating results. As in any other primary study, outliers can be a valid representation, albeit representing a different population, measure, construct, design or procedure. Thus, inferences about outliers can provide the basis to infer potential moderators (Aguinis et al. 2013 ; Steel et al. 2021 ). On the other hand, outliers can indicate invalid research, for instance, when unrealistically strong correlations are due to construct overlap (i.e., lack of a clear demarcation between independent and dependent variables), invalid measures, or simply typing errors when coding effect sizes. An advisable step is therefore to compare the results both with and without outliers and base the decision on whether to exclude outlier observations with careful consideration (Geyskens et al. 2009 ; Grewal et al. 2018 ; Kepes et al. 2013 ). However, instead of simply focusing on the size of the outlier, its leverage should be considered. Thus, Viechtbauer and Cheung ( 2010 ) propose considering a combination of standardized deviation and a study’s leverage.

Second, as mentioned in the context of a literature search, potential publication bias may be an issue. Publication bias can be examined in multiple ways (Rothstein et al. 2005 ). First, the funnel plot is a simple graphical tool that can provide an overview of the effect size distribution and help to detect publication bias (Stanley and Doucouliagos 2010 ). A funnel plot can also support in identifying potential outliers. As mentioned above, a graphical display of deviation (e.g., studentized residuals) and leverage (Cook’s distance) can help detect the presence of outliers and evaluate their influence (Viechtbauer and Cheung 2010 ). Moreover, several statistical procedures can be used to test for publication bias (Harrison et al. 2017 ; Kepes et al. 2012 ), including subgroup comparisons between published and unpublished studies, Begg and Mazumdar’s ( 1994 ) rank correlation test, cumulative meta-analysis (Borenstein et al. 2009 ), the trim and fill method (Duval and Tweedie 2000a , b ), Egger et al.’s ( 1997 ) regression test, failsafe N (Rosenthal 1979 ), or selection models (Hedges and Vevea 2005 ; Vevea and Woods 2005 ). In examining potential publication bias, Kepes et al. ( 2012 ) and Harrison et al. ( 2017 ) both recommend not relying only on a single test but rather using multiple conceptionally different test procedures (i.e., the so-called “triangulation approach”).

2.7.2 Model choice

After controlling and correcting for the potential presence of impactful outliers or publication bias, the next step in meta-analysis is the primary analysis, where meta-analysts must decide between two different types of models that are based on different assumptions: fixed-effects and random-effects (Borenstein et al. 2010 ). Fixed-effects models assume that all observations share a common mean effect size, which means that differences are only due to sampling error, while random-effects models assume heterogeneity and allow for a variation of the true effect sizes across studies (Borenstein et al. 2010 ; Cheung and Vijayakumar 2016 ; Hunter and Schmidt 2004 ). Both models are explained in detail in standard textbooks (e.g., Borenstein et al. 2009 ; Hunter and Schmidt 2004 ; Lipsey and Wilson 2001 ).

In general, the presence of heterogeneity is likely in management meta-analyses because most studies do not have identical empirical settings, which can yield different effect size strengths or directions for the same investigated phenomenon. For example, the identified studies have been conducted in different countries with different institutional settings, or the type of study participants varies (e.g., students vs. employees, blue-collar vs. white-collar workers, or manufacturing vs. service firms). Thus, the vast majority of meta-analyses in management research and related fields use random-effects models (Aguinis et al. 2011a ). In a meta-regression, the random-effects model turns into a so-called mixed-effects model because moderator variables are added as fixed effects to explain the impact of observed study characteristics on effect size variations (Raudenbush 2009 ).

2.8 Step 8: reporting results

2.8.1 reporting in the article.

The final step in performing a meta-analysis is reporting its results. Most importantly, all steps and methodological decisions should be comprehensible to the reader. DeSimone et al. ( 2020 ) provide an extensive checklist for journal reviewers of meta-analytical studies. This checklist can also be used by authors when performing their analyses and reporting their results to ensure that all important aspects have been addressed. Alternative checklists are provided, for example, by Appelbaum et al. ( 2018 ) or Page et al. ( 2021 ). Similarly, Levitt et al. ( 2018 ) provide a detailed guide for qualitative meta-analysis reporting standards.

For quantitative meta-analyses, tables reporting results should include all important information and test statistics, including mean effect sizes; standard errors and confidence intervals; the number of observations and study samples included; and heterogeneity measures. If the meta-analytic sample is rather small, a forest plot provides a good overview of the different findings and their accuracy. However, this figure will be less feasible for meta-analyses with several hundred effect sizes included. Also, results displayed in the tables and figures must be explained verbally in the results and discussion sections. Most importantly, authors must answer the primary research question, i.e., whether there is a positive, negative, or no relationship between the variables of interest, or whether the examined intervention has a certain effect. These results should be interpreted with regard to their magnitude (or significance), both economically and statistically. However, when discussing meta-analytical results, authors must describe the complexity of the results, including the identified heterogeneity and important moderators, future research directions, and theoretical relevance (DeSimone et al. 2019 ). In particular, the discussion of identified heterogeneity and underlying moderator effects is critical; not including this information can lead to false conclusions among readers, who interpret the reported mean effect size as universal for all included primary studies and ignore the variability of findings when citing the meta-analytic results in their research (Aytug et al. 2012 ; DeSimone et al. 2019 ).

2.8.2 Open-science practices

Another increasingly important topic is the public provision of meta-analytical datasets and statistical codes via open-source repositories. Open-science practices allow for results validation and for the use of coded data in subsequent meta-analyses ( Polanin et al. 2020 ), contributing to the development of cumulative science. Steel et al. ( 2021 ) refer to open science meta-analyses as a step towards “living systematic reviews” (Elliott et al. 2017 ) with continuous updates in real time. MRQ supports this development and encourages authors to make their datasets publicly available. Moreau and Gamble ( 2020 ), for example, provide various templates and video tutorials to conduct open science meta-analyses. There exist several open science repositories, such as the Open Science Foundation (OSF; for a tutorial, see Soderberg 2018 ), to preregister and make documents publicly available. Furthermore, several initiatives in the social sciences have been established to develop dynamic meta-analyses, such as metaBUS (Bosco et al. 2015 , 2017 ), MetaLab (Bergmann et al. 2018 ), or PsychOpen CAMA (Burgard et al. 2021 ).

3 Conclusion

This editorial provides a comprehensive overview of the essential steps in conducting and reporting a meta-analysis with references to more in-depth methodological articles. It also serves as a guide for meta-analyses submitted to MRQ and other management journals. MRQ welcomes all types of meta-analyses from all subfields and disciplines of management research.

Gusenbauer and Haddaway ( 2020 ), however, point out that Google Scholar is not appropriate as a primary search engine due to a lack of reproducibility of search results.

One effect size calculator by David B. Wilson is accessible via: https://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-Home.php .

The macros of David B. Wilson can be downloaded from: http://mason.gmu.edu/~dwilsonb/ .

The macros of Field and Gillet ( 2010 ) can be downloaded from: https://www.discoveringstatistics.com/repository/fieldgillett/how_to_do_a_meta_analysis.html .

The tutorials can be found via: https://www.metafor-project.org/doku.php .

Metafor does currently not provide functions to conduct MASEM. For MASEM, users can, for instance, use the package metaSEM (Cheung 2015b ).

The workbooks can be downloaded from: https://www.erim.eur.nl/research-support/meta-essentials/ .

Aguinis H, Dalton DR, Bosco FA, Pierce CA, Dalton CM (2011a) Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact. J Manag 37(1):5–38

Google Scholar  

Aguinis H, Gottfredson RK, Joo H (2013) Best-practice recommendations for defining, identifying, and handling outliers. Organ Res Methods 16(2):270–301

Article   Google Scholar  

Aguinis H, Gottfredson RK, Wright TA (2011b) Best-practice recommendations for estimating interaction effects using meta-analysis. J Organ Behav 32(8):1033–1043

Aguinis H, Pierce CA, Bosco FA, Dalton DR, Dalton CM (2011c) Debunking myths and urban legends about meta-analysis. Organ Res Methods 14(2):306–331

Aloe AM (2015) Inaccuracy of regression results in replacing bivariate correlations. Res Synth Methods 6(1):21–27

Anderson RG, Kichkha A (2017) Replication, meta-analysis, and research synthesis in economics. Am Econ Rev 107(5):56–59

Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM (2018) Journal article reporting standards for quantitative research in psychology: the APA publications and communications BOARD task force report. Am Psychol 73(1):3–25

Aytug ZG, Rothstein HR, Zhou W, Kern MC (2012) Revealed or concealed? Transparency of procedures, decisions, and judgment calls in meta-analyses. Organ Res Methods 15(1):103–133

Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50(4):1088–1101. https://doi.org/10.2307/2533446

Bergh DD, Aguinis H, Heavey C, Ketchen DJ, Boyd BK, Su P, Lau CLL, Joo H (2016) Using meta-analytic structural equation modeling to advance strategic management research: Guidelines and an empirical illustration via the strategic leadership-performance relationship. Strateg Manag J 37(3):477–497

Becker BJ (1992) Using results from replicated studies to estimate linear models. J Educ Stat 17(4):341–362

Becker BJ (1995) Corrections to “Using results from replicated studies to estimate linear models.” J Edu Behav Stat 20(1):100–102

Bergmann C, Tsuji S, Piccinini PE, Lewis ML, Braginsky M, Frank MC, Cristia A (2018) Promoting replicability in developmental research through meta-analyses: Insights from language acquisition research. Child Dev 89(6):1996–2009

Bernerth JB, Aguinis H (2016) A critical review and best-practice recommendations for control variable usage. Pers Psychol 69(1):229–283

Bernerth JB, Cole MS, Taylor EC, Walker HJ (2018) Control variables in leadership research: A qualitative and quantitative review. J Manag 44(1):131–160

Bijmolt TH, Pieters RG (2001) Meta-analysis in marketing when studies contain multiple measurements. Mark Lett 12(2):157–169

Block J, Kuckertz A (2018) Seven principles of effective replication studies: Strengthening the evidence base of management research. Manag Rev Quart 68:355–359

Borenstein M (2009) Effect sizes for continuous data. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis. Russell Sage Foundation, pp 221–235

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. John Wiley, Chichester

Book   Google Scholar  

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2010) A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 1(2):97–111

Borenstein M, Hedges L, Higgins J, Rothstein H (2013) Comprehensive meta-analysis (version 3). Biostat, Englewood, NJ

Borenstein M, Higgins JP (2013) Meta-analysis and subgroups. Prev Sci 14(2):134–143

Bosco FA, Steel P, Oswald FL, Uggerslev K, Field JG (2015) Cloud-based meta-analysis to bridge science and practice: Welcome to metaBUS. Person Assess Decis 1(1):3–17

Bosco FA, Uggerslev KL, Steel P (2017) MetaBUS as a vehicle for facilitating meta-analysis. Hum Resour Manag Rev 27(1):237–254

Burgard T, Bošnjak M, Studtrucker R (2021) Community-augmented meta-analyses (CAMAs) in psychology: potentials and current systems. Zeitschrift Für Psychologie 229(1):15–23

Cheung MWL (2015a) Meta-analysis: A structural equation modeling approach. John Wiley & Sons, Chichester

Cheung MWL (2015b) metaSEM: An R package for meta-analysis using structural equation modeling. Front Psychol 5:1521

Cheung MWL (2019) A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychol Rev 29(4):387–396

Cheung MWL, Chan W (2005) Meta-analytic structural equation modeling: a two-stage approach. Psychol Methods 10(1):40–64

Cheung MWL, Vijayakumar R (2016) A guide to conducting a meta-analysis. Neuropsychol Rev 26(2):121–128

Combs JG, Crook TR, Rauch A (2019) Meta-analytic research in management: contemporary approaches unresolved controversies and rising standards. J Manag Stud 56(1):1–18. https://doi.org/10.1111/joms.12427

DeSimone JA, Köhler T, Schoen JL (2019) If it were only that easy: the use of meta-analytic research by organizational scholars. Organ Res Methods 22(4):867–891. https://doi.org/10.1177/1094428118756743

DeSimone JA, Brannick MT, O’Boyle EH, Ryu JW (2020) Recommendations for reviewing meta-analyses in organizational research. Organ Res Methods 56:455–463

Duval S, Tweedie R (2000a) Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56(2):455–463

Duval S, Tweedie R (2000b) A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 95(449):89–98

Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315(7109):629–634

Eisend M (2017) Meta-Analysis in advertising research. J Advert 46(1):21–35

Elliott JH, Synnot A, Turner T, Simmons M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review: 1. Introduction—the why, what, when, and how. J Clin Epidemiol 91:2330. https://doi.org/10.1016/j.jclinepi.2017.08.010

Field AP, Gillett R (2010) How to do a meta-analysis. Br J Math Stat Psychol 63(3):665–694

Fisch C, Block J (2018) Six tips for your (systematic) literature review in business and management research. Manag Rev Quart 68:103–106

Fortunato S, Bergstrom CT, Börner K, Evans JA, Helbing D, Milojević S, Petersen AM, Radicchi F, Sinatra R, Uzzi B, Vespignani A (2018) Science of science. Science 359(6379). https://doi.org/10.1126/science.aao0185

Geyer-Klingeberg J, Hang M, Rathgeber A (2020) Meta-analysis in finance research: Opportunities, challenges, and contemporary applications. Int Rev Finan Anal 71:101524

Geyskens I, Krishnan R, Steenkamp JBE, Cunha PV (2009) A review and evaluation of meta-analysis practices in management research. J Manag 35(2):393–419

Glass GV (2015) Meta-analysis at middle age: a personal history. Res Synth Methods 6(3):221–231

Gonzalez-Mulé E, Aguinis H (2018) Advancing theory by assessing boundary conditions with metaregression: a critical review and best-practice recommendations. J Manag 44(6):2246–2273

Gooty J, Banks GC, Loignon AC, Tonidandel S, Williams CE (2021) Meta-analyses as a multi-level model. Organ Res Methods 24(2):389–411. https://doi.org/10.1177/1094428119857471

Grewal D, Puccinelli N, Monroe KB (2018) Meta-analysis: integrating accumulated knowledge. J Acad Mark Sci 46(1):9–30

Gurevitch J, Koricheva J, Nakagawa S, Stewart G (2018) Meta-analysis and the science of research synthesis. Nature 555(7695):175–182

Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods 11(2):181–217

Habersang S, Küberling-Jost J, Reihlen M, Seckler C (2019) A process perspective on organizational failure: a qualitative meta-analysis. J Manage Stud 56(1):19–56

Harari MB, Parola HR, Hartwell CJ, Riegelman A (2020) Literature searches in systematic reviews and meta-analyses: A review, evaluation, and recommendations. J Vocat Behav 118:103377

Harrison JS, Banks GC, Pollack JM, O’Boyle EH, Short J (2017) Publication bias in strategic management research. J Manag 43(2):400–425

Havránek T, Stanley TD, Doucouliagos H, Bom P, Geyer-Klingeberg J, Iwasaki I, Reed WR, Rost K, Van Aert RCM (2020) Reporting guidelines for meta-analysis in economics. J Econ Surveys 34(3):469–475

Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, Orlando

Hedges LV, Vevea JL (2005) Selection methods approaches. In: Rothstein HR, Sutton A, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment, and adjustments. Wiley, Chichester, pp 145–174

Hoon C (2013) Meta-synthesis of qualitative case studies: an approach to theory building. Organ Res Methods 16(4):522–556

Hunter JE, Schmidt FL (1990) Methods of meta-analysis: correcting error and bias in research findings. Sage, Newbury Park

Hunter JE, Schmidt FL (2004) Methods of meta-analysis: correcting error and bias in research findings, 2nd edn. Sage, Thousand Oaks

Hunter JE, Schmidt FL, Jackson GB (1982) Meta-analysis: cumulating research findings across studies. Sage Publications, Beverly Hills

Jak S (2015) Meta-analytic structural equation modelling. Springer, New York, NY

Kepes S, Banks GC, McDaniel M, Whetzel DL (2012) Publication bias in the organizational sciences. Organ Res Methods 15(4):624–662

Kepes S, McDaniel MA, Brannick MT, Banks GC (2013) Meta-analytic reviews in the organizational sciences: Two meta-analytic schools on the way to MARS (the Meta-Analytic Reporting Standards). J Bus Psychol 28(2):123–143

Kraus S, Breier M, Dasí-Rodríguez S (2020) The art of crafting a systematic literature review in entrepreneurship research. Int Entrepreneur Manag J 16(3):1023–1042

Levitt HM (2018) How to conduct a qualitative meta-analysis: tailoring methods to enhance methodological integrity. Psychother Res 28(3):367–378

Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C (2018) Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA publications and communications board task force report. Am Psychol 73(1):26

Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage Publications, Inc.

López-López JA, Page MJ, Lipsey MW, Higgins JP (2018) Dealing with effect size multiplicity in systematic reviews and meta-analyses. Res Synth Methods 9(3):336–351

Martín-Martín A, Thelwall M, Orduna-Malea E, López-Cózar ED (2021) Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations. Scientometrics 126(1):871–906

Merton RK (1968) The Matthew effect in science: the reward and communication systems of science are considered. Science 159(3810):56–63

Moeyaert M, Ugille M, Natasha Beretvas S, Ferron J, Bunuan R, Van den Noortgate W (2017) Methods for dealing with multiple outcomes in meta-analysis: a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. Int J Soc Res Methodol 20(6):559–572

Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 6(7):e1000097

Mongeon P, Paul-Hus A (2016) The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics 106(1):213–228

Moreau D, Gamble B (2020) Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychol Methods. https://doi.org/10.1037/met0000351

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev 4(1):1–22

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A (2016) Rayyan—a web and mobile app for systematic reviews. Syst Rev 5(1):1–10

Owen E, Li Q (2021) The conditional nature of publication bias: a meta-regression analysis. Polit Sci Res Methods 9(4):867–877

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E,McDonald S,McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372. https://doi.org/10.1136/bmj.n71

Palmer TM, Sterne JAC (eds) (2016) Meta-analysis in stata: an updated collection from the stata journal, 2nd edn. Stata Press, College Station, TX

Pigott TD, Polanin JR (2020) Methodological guidance paper: High-quality meta-analysis in a systematic review. Rev Educ Res 90(1):24–46

Polanin JR, Tanner-Smith EE, Hennessy EA (2016) Estimating the difference between published and unpublished effect sizes: a meta-review. Rev Educ Res 86(1):207–236

Polanin JR, Hennessy EA, Tanner-Smith EE (2017) A review of meta-analysis packages in R. J Edu Behav Stat 42(2):206–242

Polanin JR, Hennessy EA, Tsuji S (2020) Transparency and reproducibility of meta-analyses in psychology: a meta-review. Perspect Psychol Sci 15(4):1026–1041. https://doi.org/10.1177/17456916209064

R Core Team (2021). R: A language and environment for statistical computing . R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ .

Rauch A (2020) Opportunities and threats in reviewing entrepreneurship theory and practice. Entrep Theory Pract 44(5):847–860

Rauch A, van Doorn R, Hulsink W (2014) A qualitative approach to evidence–based entrepreneurship: theoretical considerations and an example involving business clusters. Entrep Theory Pract 38(2):333–368

Raudenbush SW (2009) Analyzing effect sizes: Random-effects models. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis, 2nd edn. Russell Sage Foundation, New York, NY, pp 295–315

Rosenthal R (1979) The file drawer problem and tolerance for null results. Psychol Bull 86(3):638

Rothstein HR, Sutton AJ, Borenstein M (2005) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester

Roth PL, Le H, Oh I-S, Van Iddekinge CH, Bobko P (2018) Using beta coefficients to impute missing correlations in meta-analysis research: Reasons for caution. J Appl Psychol 103(6):644–658. https://doi.org/10.1037/apl0000293

Rudolph CW, Chang CK, Rauvola RS, Zacher H (2020) Meta-analysis in vocational behavior: a systematic review and recommendations for best practices. J Vocat Behav 118:103397

Schmidt FL (2017) Statistical and measurement pitfalls in the use of meta-regression in meta-analysis. Career Dev Int 22(5):469–476

Schmidt FL, Hunter JE (2015) Methods of meta-analysis: correcting error and bias in research findings. Sage, Thousand Oaks

Schwab A (2015) Why all researchers should report effect sizes and their confidence intervals: Paving the way for meta–analysis and evidence–based management practices. Entrepreneurship Theory Pract 39(4):719–725. https://doi.org/10.1111/etap.12158

Shaw JD, Ertug G (2017) The suitability of simulations and meta-analyses for submissions to Academy of Management Journal. Acad Manag J 60(6):2045–2049

Soderberg CK (2018) Using OSF to share data: A step-by-step guide. Adv Methods Pract Psychol Sci 1(1):115–120

Stanley TD, Doucouliagos H (2010) Picture this: a simple graph that reveals much ado about research. J Econ Surveys 24(1):170–191

Stanley TD, Doucouliagos H (2012) Meta-regression analysis in economics and business. Routledge, London

Stanley TD, Jarrell SB (1989) Meta-regression analysis: a quantitative method of literature surveys. J Econ Surveys 3:54–67

Steel P, Beugelsdijk S, Aguinis H (2021) The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews. J Int Bus Stud 52(1):23–44

Suurmond R, van Rhee H, Hak T (2017) Introduction, comparison, and validation of Meta-Essentials: a free and simple tool for meta-analysis. Res Synth Methods 8(4):537–553

The Cochrane Collaboration (2020). Review Manager (RevMan) [Computer program] (Version 5.4).

Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C, Glasziou P, Shemilt I, Synnot A, Turner T, Elliot J (2017) Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol 91:31–37

Thompson SG, Higgins JP (2002) How should meta-regression analyses be undertaken and interpreted? Stat Med 21(11):1559–1573

Tipton E, Pustejovsky JE, Ahmadi H (2019) A history of meta-regression: technical, conceptual, and practical developments between 1974 and 2018. Res Synth Methods 10(2):161–179

Vevea JL, Woods CM (2005) Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychol Methods 10(4):428–443

Viechtbauer W (2010) Conducting meta-analyses in R with the metafor package. J Stat Softw 36(3):1–48

Viechtbauer W, Cheung MWL (2010) Outlier and influence diagnostics for meta-analysis. Res Synth Methods 1(2):112–125

Viswesvaran C, Ones DS (1995) Theory testing: combining psychometric meta-analysis and structural equations modeling. Pers Psychol 48(4):865–885

Wilson SJ, Polanin JR, Lipsey MW (2016) Fitting meta-analytic structural equation models with complex datasets. Res Synth Methods 7(2):121–139. https://doi.org/10.1002/jrsm.1199

Wood JA (2008) Methodology for dealing with duplicate study effects in a meta-analysis. Organ Res Methods 11(1):79–95

Download references

Open Access funding enabled and organized by Projekt DEAL. No funding was received to assist with the preparation of this manuscript.

Author information

Authors and affiliations.

University of Luxembourg, Luxembourg, Luxembourg

Christopher Hansen

Leibniz Institute for Psychology (ZPID), Trier, Germany

Holger Steinmetz

Trier University, Trier, Germany

Erasmus University Rotterdam, Rotterdam, The Netherlands

Wittener Institut Für Familienunternehmen, Universität Witten/Herdecke, Witten, Germany

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jörn Block .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table 1 .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Hansen, C., Steinmetz, H. & Block, J. How to conduct a meta-analysis in eight steps: a practical guide. Manag Rev Q 72 , 1–19 (2022). https://doi.org/10.1007/s11301-021-00247-4

Download citation

Published : 30 November 2021

Issue Date : February 2022

DOI : https://doi.org/10.1007/s11301-021-00247-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Dermatol
• v.62(5); Sep-Oct 2017

Summary and Synthesis: How to Present a Research Proposal

Maninder singh setia.

From the MGM Institute of Health Sciences, Navi Mumbai, Maharashtra, India

Saumya Panda

1 Department of Dermatology, KPC Medical College, Kolkata, West Bengal, India

This concluding module attempts to synthesize the key learning points discussed during the course of the previous ten sets of modules on methodology and biostatistics. The objective of this module is to discuss how to present a model research proposal, based on whatever was discussed in the preceding modules. The lynchpin of a research proposal is the protocol, and the key component of a protocol is the study design. However, one must not neglect the other areas, be it the project summary through which one catches the eyes of the reviewer of the proposal, or the background and the literature review, or the aims and objectives of the study. Two critical areas in the “methods” section that cannot be emphasized more are the sampling strategy and a formal estimation of sample size. Without a legitimate sample size, none of the conclusions based on the statistical analysis would be valid. Finally, the ethical parameters of the study should be well understood by the researchers, and that should get reflected in the proposal.

As we reach the end of an exhaustive module encompassing research methods and biostatistics, we need to summarize and synthesize the key learning points, to demonstrate how one may utilize the different sections of the module to undertake research projects of different kinds. After all, the practical purpose behind publishing such a module is to facilitate the preparation of high quality research proposals and protocols. This concluding part will make an attempt to provide a window to the different sections of the module, underlining the various aspects of design and analysis needed to formulate protocols applicable to different kinds of clinical research in dermatology.

Components of a Research Proposal

The goal of a research proposal is to present and justify the need to study a research problem and to present the practical ways in which the proposed study should be conducted. A research proposal is generally meant to be presented by an investigator to request an agency or a body to support research work in the form of grants. The vast majority of research proposals, in India, however, are not submitted to agency or body for grants, simply because of the paucity of such agencies, bodies, and research grants. Most are academic research proposals, self-financed, and submitted to scientific and ethics committee of an institution. The parts of a proposal include the title page, abstract/project summary, table of contents, introduction, background and review of literature, and the research protocol.

The title page should contain the personal data pertaining to the investigators, and title of the project, which should be concise and comprehensive at the same time. The table of contents, strictly speaking, is not necessary for short proposals. The introduction includes a statement of the problem, purpose, and significance of the research.

The protocol is the document that specifies the research plan. It is the single most important quality control tool for all aspects of a clinical research. It is the instrument where the researcher explains how data will be collected, including the calculation for estimating sample size, and what outcome variables to measure.

A complete clinical research protocol includes the following:

Study design

• Precise definition of the disease or problem
• Completely defined prespecified primary and secondary outcome measures, including how and when these will be assessed
• Clear description of variables
• Well-defined inclusion and exclusion criteria
• Efficacy and safety parameters
• Whenever applicable, stopping guidelines and parameters of interim analyses
• Sample size calculation
• Randomization details
• Plan of statistical analysis
• Detailed description of interventions
• A chronogram of research flow (Gantt chart)
• Informed consent document
• Clinical research form
• Details of budget; and
• References.

(Modified from: Bagatin et al ., 2013).

Project Summary

The project summary is a brief document that consists of an overview, and discusses the intellectual merits, and broader impacts of the research project. Each of these three sections is required to be present and must be clearly defined. The project summary is one of the most important parts of the proposal. It is likely the first thing a reviewer will read, and is the investigators’ best chance to grab their interest, and convince them of the importance, and quality, of their research before they even read the proposal. Though it is the first proposal element in order, many applicants prefer to write the project summary last, after writing the protocol. This allows the writer to better avoid any inconsistencies between the two.

The overview specifies the research goal and it should demonstrate that this goal fits with the principal investigator's long-term research goals. It should specify the proposed research approach and the educational goal of the research project.

The intellectual merits (the contribution your research will make to your field) should specify the current state of knowledge in the field, and where it is headed. It should also clarify what your research will add to the state of knowledge in the field. Furthermore, important to state is what your research will do to enhance or enable other researches in the field. Finally, one should answer why your research is important for the advancement of the field.

The broader impacts (the contribution the research will make to the society) should answer the questions on the benefit to the society at large from the research, and the possible applications of the research, and why the general public would care. It should also clarify how the research can benefit the site of research (medical college or university, etc.) and the funding agency.

Background and Review of Literature

This is an important component of the research protocol. The review should discuss all the relevant literature, the method used in the literature, the lacunae in the literature, and justify the proposed research. We have provided a list of the useful databases in the section on systematic reviews and meta-analysis (Setia, 2017). Some of these are PubMed, Cochrane database, EMBASE, and LILACS.

Provide a critical analysis of the literature

The researcher should not provide a descriptive analysis of literature. For instance, the literature reviews should not be a list of one article followed by the next article. It should be a critical analysis of literature.

A study by XXXX et al . found that the prevalence of psoriasis was 20%. It was a hospital-based study conducted in North India. The prevalence was 35% in males and 12% in females.

Another study by YYYYY et al . found that the prevalence of psoriasis was 14%. The study was conducted in a private clinic in North India. The prevalence was 8% in males and 18% in females.

A third study by ZZZZZ et al . found that the prevalence of psoriasis was 5%. This study was a community-based study. The prevalence was 7% in males and 3% in females.

In this type of review, the researcher has described all the studies. However, it is useful to understand the findings of these three studies and summarize them in researcher's own words.

A possible option can be “ The reported prevalence of psoriasis in the Indian population varied from 5% to 20%. In general, it was higher in hospital-based studies and lower in community-based studies. There was no consistent pattern in the prevalence of psoriasis in males and females. Though some studies found the prevalence to be higher in males, others reported that females had a higher prevalence .”

Discuss the limitations and lacunae of these studies

The researcher should discuss the limitations of the studies. These could be the limitations that the authors have presented in the manuscript or the ones that the researcher has identified. Usually, the current research proposal should try to address the limitations of a previous study.

A study by BBBB et al : “ One of the main limitations of our study was the lack of objective criteria for assessing anemia in patients presenting with psoriasis. We classified the patients based on clinical assessment of pallor .”

The present proposal can mention “ Though previous studies have assessed the association between anemia and psoriasis, they have not used any objective criteria (such as hemoglobin or serum ferritin levels). Furthermore, pallor was evaluated by three clinicians; the authors have not described the agreement between these clinicians .”

In the above example, the authors have stated the limitation of their research in the manuscript. However, in the review of literature, the researcher has added another limitation. It is important to convince the reviewers that the researcher has read and understood the literature. It is also important that some or most of these lacunae should be addressed in the present proposal as far as possible.

Justify the present proposal by review

The researcher should adequately justify the present proposal based on the review of literature. The justification should not only be for the research question, but also the methods, study design, variables of interest, study instruments or measurements, and statistical methods of choice. Sometimes, the justification can be purely statistical. For example, all the previous studies have used cross-sectional data or cross-sectional analysis of longitudinal data in their manuscripts. The present proposal will use methods used for longitudinal data analysis. The researcher should justify the benefit of these methods over the previous statistical methods.

In short, the review should not be a “laundry list” of all the articles. The review should be able to convince the reader that the present research is required and it builds on the existing literature (either as a novel research question, new measurement of the outcome, a better study design, or advanced and appropriate statistical methods).

Kindly try to avoid this justification: “ It has not been done in our center .”

Aims and Objectives

The “aim” of the study is an overarching goal of the study. The objectives are measurable and help the researcher achieve the overall aim.

For example, the overall aim of our study is to assess the long-term health of patients of psoriasis.

The specific objectives are:

• To record the changes in Psoriasis Area and Severity Index (PASI) score in patients with psoriasis over a period of 5 years
• To study the side effects of medications in these patients over a period of 5 years.

It is important to clearly state the objectives, since the research proposal should be designed to achieve these objectives.

For example, the methods should describe the following:

• How will the researcher answer the first objective?
• Where will the researcher recruit the study participants (study site and population)?
• Which patients of psoriasis will be recruited (inclusion and exclusion criteria)?
• What will be the design of the study (cohort, etc.)?
• What are all the variables to be measured to achieve the study outcomes (exposure and outcome variables)?
• How will the researcher measure these variables (clinical evaluation, history, serological examination, etc.)?
• How will the researcher record these data (clinical forms, etc.)?
• How will the researcher analyze the data that have been collected?
• Are there any limitations of these methods? If so, what has the researcher done to minimize the limitations?

All the ten modules on research methodology have to be read and grasped to plan and design any kind of research applicable to one's chosen field. However, some key areas have been outlined below with examples to appreciate the same in an easier manner.

The study setting must be specified. This should include both the geographical location and the population from which the study sample would be recruited.

“The study took place at the antiretroviral therapy clinic of Queen Elizabeth Central Hospital in Blantyre, Malawi, from January 2006 to April 2007. Blantyre is the major commercial city of Malawi, with a population of 1,000,000 and an estimated HIV prevalence of 27% in adults in 2004” (Ndekha et al ., 2009).

This is a perfect example of description of a study setting which underscores the importance of planning it in detail a priori .

Study population, sampling strategy, and sample size

Study population has to be clearly and precisely defined. For example, a study on atopic dermatitis may be conducted upon patients defined according to the UK Working Party's modified diagnostic criteria, or the Hanifin and Rajka's criteria, or some other criteria defined by the investigators. However, it should always be prespecified within the protocol.

Similarly, the eligibility criteria of the participants for the study must be explicit. One truism that is frequently forgotten is that the inclusion and exclusion criteria are mutually exclusive, and one is not the negative image of the other. Eligible cases are included according to a set of inclusion criteria, and this is followed by administration of the exclusion criteria. Thus, in fact, they can never be the negative image of each other.

“Eligible participants were all adults aged 18 or over with HIV who met the eligibility criteria for antiretroviral therapy according to the Malawian national HIV treatment guidelines (WHO clinical stage III or IV or any WHO stage with a CD4 count < 250/mm 3 ) and who were starting treatment with a BMI < 18.5. Exclusion criteria were pregnancy and lactation or participation in another supplementary feeding program” (Ndekha et al ., 2009).

To put in perspective the point we made about inclusion and exclusion criteria, in the above example, “age above 18 years” or “CD4 count >250/mm 3 ” cannot be exclusion criteria, as these have already been excluded.

Sampling strategy has been adequately discussed in the Module 5 of the Methodology series (Setia, 2016). A few points are worth repeating:

• The sampling strategy should never be misrepresented. Example: If you have not done random sampling, no big deal. There are other legitimate sampling strategies available for your study. But once you have mentioned “random sampling” in your protocol, you cannot resort to purposive sampling
• Sometimes, the researcher might want to know the characteristics of a certain problem within a specific population, without caring for generalizability of results. In such a scenario, purposive sampling may be resorted to
• Nonprobability sampling methods such as consecutive consenting sampling or any such convenience sampling are perfectly legitimate and easy to do, particularly in case of dissertations where time and resources are limited.

Sample size is one of the most misunderstood, yet fundamentally important, issues among clinicians and has to be addressed once the study objectives have been set and the design has been finalized. Too small a sample means that there would be a failure to detect change following test intervention. A sample larger than necessary may also result in bad quality data. In either case, there would be ethical problems and wastage of resources. The researcher needs just enough samples to draw accurate inferences, which would be adequately powered (Panda, 2015).

Estimation of sample size has been dealt with adequately in the Module 5 biostatistics series (Hazra et al ., 2016), including the different mathematical derivations and the available software. Sample size determination is a statistical exercise based on the probability of errors in testing of hypothesis, power of the sample, and effect size. Although, relatively speaking, these are simple concepts to grasp, a large number of different study designs and analytical methods lead to a bewilderingly large number of formulae for determining sample size. Thus, the software are really handy and are becoming increasingly popular.

The study design defines the objectives and end points of the study, the type and manner of data collection, and the strategy of data analysis (Panda 2015). The different types of clinical studies have been depicted in Figure 1 . The suitability of various study designs vis-à-vis different types of research questions is summarized in Table 1 .

Types of study (Source: Panda, 2015)

Research questions vis-a-vis study designs

In our previous series of ten modules on methodology, we have discussed all these different kinds of studies and more. Some key issues that require reiteration are given below:

• The control of a case–control study and that of a randomized controlled trial is more different from each other than chalk is from cheese. The former is an observational study, while the latter is an interventional one. Every study with a control group is not a case–control study. For a study to be classified as a case–control study, the study should be an observational study and the participants should be recruited based on their outcome status (Setia, 2016). Apparently, this is not so difficult to understand, yet even now we have publications which confuse between the different kinds of controls (Bhanja et al ., 2015)
• Due to the fact that the outcome and exposure are assessed at the same time point in a cross-sectional study, it is pretty difficult, if not impossible, to derive causal relationships from such a study. At most, one may establish statistical association between exposures and outcomes by calculating the odds ratio. However, these associations must not be confused with causation.
• It is generally said that a cohort design may not be efficient for rare outcomes. However, if the rare outcome is common in some exposures, it may be useful to follow a cohort design. For example, melanoma is a rare condition in India. Hence, if we follow individuals to study the incidence of melanoma, it may not be efficient. However, if we know that, in India, acral lentiginous melanoma is the most commonly reported variant, we should follow a cohort of individuals with acral lentiginous and study the incidence of melanoma in this group (Setia, 2016).

Clinical researchers should also be accustomed with observational designs beyond case–control, cohort, and cross-sectional studies. Sometimes, the unit of analysis has to be a group or aggregate rather than the individual. Consider the following example:

The government introduced the supplementation of salt with iodine for about 20 years. However, not all states have used the same level of iodine in salt. Certain hilly states have used higher quantities compared with other states. Incidentally, you read a report that high iodine levels are associated with psoriasis. You are intrigued to find if introduction of iodine has altered the picture of psoriasis in the country. You feel compelled to design a study to answer this question .

It is obvious that here the unit of study cannot be individuals, but a large population distributed in a certain geographical area. This is the domain of ecologic studies. An allied category of observational studies is named “natural experiments,” where the exposure is not assigned by the investigator (as in an interventional study), but through “natural processes.” These may be through changes in the existing regulations or public policies or, may be, through introduction of new laws (Setia, 2017).

Another category of research questions that cannot be satisfactorily captured by all the quantitative methods described earlier, like social stigma experienced by patients or their families with, say, vitiligo, leprosy, or sexually transmitted infections, are best dealt with by qualitative research. As can be seen by the examples given above, this is a type of research which is very relevant to medical research, yet to which the regular medical researcher has got a very poor exposure, if any. We shall encourage interested researchers to take a look at the 10 th Module of the Methodology series that specifically deals with qualitative research (Setia, 2017).

Clinical studies are experiments that are not conducted in laboratories but in controlled real-life settings on human subjects with some disease. Hence, designing a study involves many pragmatic considerations aside pure methodology. Thus, factors to consider when selecting a study design are objectives of the study, time frame, treatment duration, carryover effects, cost and logistics, patient convenience, statistical considerations, sample size, etc. (Panda, 2015).

Certain truisms regarding study designs should always be remembered: a study design has to be tailored to objectives. The same question may be answered by different designs. The optimum design has to be based on workforce, budgetary allocation, infrastructure, and clinical material that may be commanded by the researchers. Finally, no design is perfect, and there is no design to provide a perfect answer to all research questions relevant to a particular problem (Panda, 2015).

Variables of interest and collection of these variables

Data structure depends on the characteristics of the variables [ Figure 2 ]. A variable refers to a particular character on which a set of data are recorded. Data are thus the values of a variable (Hazra et al ., 2016).

Types of data and variables (Source: Panda, 2015)

Quantitative data always have a proportional scale among values, and can be either discrete (e.g., number of moles) or continuous (e.g., age). Qualitative data can be either nominal (e.g., blood groups) or ordinal (e.g., Fitzpatrick's phototypes I-VI). Variables can be binary or dichotomous (male/female) or multinomial or polychotomous (homosexual/bisexual/heterosexual) (Panda, 2015).

Changing data scales is possible so that numerical data may become ordinal and ordinal data may become nominal. This may be done when the researcher is not confident about the accuracy of the measuring instrument, is unconcerned about the loss of fine detail, or where group numbers are not large enough to adequately represent a variable of interest. It may also make clinical interpretation easier (Hazra et al ., 2016).

The variables whose effects are observed on other variables are known as independent variables (e.g., risk factors). The latter kind of variables that change as a result of independent variables are known as dependent variables (i.e., outcome). Confounders are those variables that influence the relation between independent and dependent variables (e.g., the clinical effect of sunscreen used as part of a test intervention regimen in melasma). If the researcher fails to control or eliminate the confounder, it will damage the internal validity of an experiment (Panda, 2015).

Biostatistics begins with descriptive statistics that implies summarizing a collection of data from a sample or population. An excellent overview of descriptive statistics has been given in the Module 1 of the Biostatistics series (Hazra et al ., 2016). We would encourage every researcher to embark on designing and collecting data on their own to go through this particular module to have a clear idea on how to proceed further.

Statistical methods

As briefly discussed earlier, the “methods” section should also include a detailed description of statistical methods. It is best to describe the methods for each objective.

For example: Which statistical methods will the researcher use to study the changes in PASI score over time?

It is important to first identify the nature of the outcome – will it be linear or categorical?

• It may be noticed that the PASI is a score and can range from 0 to 72. The researcher can measure the actual score and assess the changes in score. Thus, the researcher will use methods for statistical analysis of continuous data (such as means, standard deviations, t -test, or linear regressions)
• However, the researcher may choose to cut off the PASI score at 60 (of course, there has to be justification!) and call it severe psoriasis. Thus, the researcher will have an outcome variable with two outcomes (Yes: >60 PASI, and No: <60 PASI). Thus, in this case, the researcher will use methods for statistical analysis of categorical data (proportions, Chi-square test, or logistic regression models).

The statistical methods have been described in detail in the Biostatistics section of the series. The reader is encouraged to read all the sections to understand these methods. However, the key points to remember are:

• Identify the nature of the outcome for each objective
• Describe the statistical methods separately for each objective
• Identify the methods to handle confounding and describe them in the statistical methods
• If the researcher is using advanced statistical methods or specific tools, please provide reference to these methods
• Provide the name of the statistical software (including the version) that will be used for data analysis in the present study
• Do not provide a laundry list of all the statistical methods. It just shows that the researcher has not understood the relevance of statistics in the study design.

Multivariate models

In general, multivariate analyses are used in studies and research proposals. These analyses are useful to adjust for confounding (though these are also useful to test for interaction, we shall discuss confounding in this section). For example, we propose to compare two different types of medications in psoriasis. We have used secondary clinical data for this study. The outcome of interest is PASI score. We have collected data on the type of medication, age, sex, and alcohol use. When we compare the PASI score in these two groups, we will use t -test (if linear comparison) or Chi-square test (if PASI is categorized – as described earlier). However, it is possible that age, sex, and alcohol use may also play a role in the clinical progression of psoriasis (which is measured as PASI score). Thus, the researcher would like to account for differences in these variables in the two groups. This can be done using multivariate analytical methods (such as linear regression for continuous variables and logistic regression for categorical dichotomous variables). This is a type of mathematical model in which we include multiple variables: the main explanatory variable (type of drug in this study) and potential confounders (age, sex, and alcohol use in this study). Thus, the outcome (PASI score) after multivariate analyses will be “adjusted” for age, sex, and alcohol use after multivariate analysis. We would like to encourage the readers to consult a statistician for these methods.

TRIVIA: The singular for “data” is “datum,” just as “stratum” is the singular for “strata.” Thus, “ data were analyzed …,” “ data were collected …,” and “ data have been ….”

Clinical Record Forms

We have discussed designing of questionnaires and clinical record forms (CRFs) in detail in two modules. We shall just highlight the most important aspects in this part. The CRF is an important part of the research protocol. The CRF should include all the variables of interest in the study. Thus, it is important to make a list of all parameters of interest before working on the CRF. This can be done by a thorough review of literature and discussion with experts. Once the questionnaire/CRF has been designed, the researcher should pilot it and change according to the feedback from the participants and one's own experience while administering the questionnaire or recording data in the CRF. The CRF should use coded responses (for close-ended questions), this will help in data entry and analysis. If the researcher has developed a scale, the reliability and validity should be tested (methods have been discussed in earlier sections). The CRF can be paper based or computer based (it will depend on the resources).

It is very important to describe the ethics for the present study. It should not be restricted to “ The study will be evaluated by an Institutional Review Committee …” The researcher should demonstrate that s/he has understood the various ethical issues in the present study. The three core principles for ethics are: autonomy (the participants have a right to decide whether to participate in the study or opt out), beneficence/nonmaleficence (the study should not be harmful to participants and the risk–benefit ratio should be adequately understood and described), and justice (all the risks and benefits of the present study should be equally distributed).

The researcher should try to address these issues in the section of “Ethics.” Currently, the National Institutes of Health has proposed the following seven principles of “Ethics in Clinical Research:” social and clinical value, scientific validity, fair subject selection, favorable risk–benefit ratio, independent review, informed consent, and respect for potential and enrolled subjects. The Indian Council of Medical Research has also published guidelines to conduct biomedical research in India. We strongly encourage the readers to be familiar with these guidelines. Furthermore, the researchers should keep themselves updated with changes in these regulations. If it is a clinical trial, the researcher should also be familiar with Schedule Y and Consent form requirements for these types of clinical trials.

Concluding Remarks

This module has been designed as a comprehensive guide for a dermatologist to enable him/her to embark on the exciting journey of designing studies of almost any kind that can be thought to be of relevance to clinical dermatology. There has been a conscious attempt to customize the discussion on design and analysis keeping not only dermatology, but also Indian conditions in mind. However, the module can be of help to any medical doctor embarking on the path to medical research. As contributors, it is our ardent hope that this module might act as a catalyst of good-quality research in the field of dermatology and beyond in India and elsewhere.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

Bibliography

An official website of the United States government

Here’s how you know

The .gov means it’s official.

Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure.

The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Fact Sheets Fiscal Year 2025 Medicare Inpatient Psychiatric Facilities Prospective Payment System (IPF PPS) and Quality Reporting (IPFQR) Updates Proposed Rule (CMS-1806-P)

• Medicare Parts A & B

On March 28, 2024, the Centers for Medicare & Medicaid Services (CMS) issued a proposed rule to update Medicare payment policies and rates for the Inpatient Psychiatric Facilities Prospective Payment System (IPF PPS) for fiscal year (FY) 2025. CMS is publishing this proposed rule consistent with the legal requirements to update Medicare payment policies for IPFs on an annual basis. 

Changes included in this proposed rule would support the provision of high-quality behavioral health treatment in inpatient psychiatric facilities, consistent with the  Biden Administration’s Unity Agenda and focus on addressing the mental health crisis . This fact sheet discusses the major provisions of the proposed rule, including proposed annual updates to the prospective payment rates, the outlier threshold, the wage index, and associated impact analysis. In addition, the rule includes a proposal to revise patient-level adjustment factors, as well as a proposal to increase the per-treatment amount for Electroconvulsive Therapy (ECT). Additionally, the rule proposes to update the wage index using the Core‑Based Statistical Area (CBSA) Labor Market Areas defined in the Office of Management and Budget (OMB)  Bulletin 23-01 . CMS believes these policies will improve or maintain individual access to high-quality care by ensuring that payment rates reflect the best available data on the resources involved in inpatient psychiatric care and the costs of these resources. CMS is proposing a three-year budget-neutral phase-out of the rural adjustment for IPFs located in the 54 rural counties that will become urban under the new OMB delineations. We will also provide a 5% cap on any decrease to the provider’s wage index from its wage index in the prior year, as finalized in previous rulemaking. 

In addition, this rule includes a clarification of the eligibility criteria for the option to elect to file an all-inclusive cost report . CMS will issue revised cost reporting guidance and make operational changes to improve the quality of ancillary charge data on IPF claims by enforcing these eligibility criteria, resulting in the appropriate collection of more cost data from IPFs that have been erroneously filing an “all-inclusive” rate cost report. For cost reporting periods beginning on or after October 1, 2024, only government-owned and tribally owned IPFs will be permitted to file an “all-inclusive” rate cost report.  CMS believes these operational changes will support its ongoing efforts to analyze variation in costs between patients with different characteristics and will increase the accuracy of future payment refinements to the IPF PPS while also aligning with the President’s Executive Order on Promoting Competition in the American Economy. [1]

This proposed rule also includes two requests for information on future revisions to the IPF PPS facility-level adjustment factors and development of the new standardized IPF Patient Assessment Instrument (IPF-PAI), required by the Consolidated Appropriations Act, 2023 (CAA, 2023), which IPFs participating in the IPF Quality Reporting (IPFQR) Program will be required to report for Rate Year 2028.

For the IPFQR Pro gram, CMS proposes adopting the 30-Day Risk-Standardized All-Cause Emergency Department Visit Following an Inpatient Psychiatric Facility Discharge measure and to require IPFs to submit data on a quarterly basis for patient-level measures.

This fact sheet discusses the provisions of the proposed rule. The FY 2025 Inpatient Psychiatric Facilities Prospective Payment System proposed rule (CMS-1806-P) can be downloaded from the Federal Register at  https://www.federalregister.gov/public-inspection/2024-06764/medicare-program-fy-2025-inpatient-psychiatric-facilities-prospective-payment-system---rate-update

Proposed Changes to Payments Under the IPF PPS 

Proposed Updates to IPF Payment Rates

For FY 2025, CMS is proposing to update the IPF PPS payment rates by 2.7%, based on the proposed 2021-based IPF market basket increase of 3.1% less a proposed 0.4 percentage point productivity adjustment. CMS is proposing that if more recent data becomes available (for example, a more recent estimate of the market basket update or productivity adjustment), CMS would use this data, if appropriate, to determine the FY 2025 market basket update percentage increase and the productivity adjustment in the final rule. Additionally, CMS proposes to update the outlier threshold so that estimated outlier payments remain at 2.0% of total payments. CMS estimates that this would result in a 0.1% decrease in aggregate payments due to updating the outlier threshold. Total estimated payments to IPFs are estimated to increase by 2.6%, or $70 million, in FY 2025 relative to IPF payments in FY 2024.

Proposed Revisions to IPF PPS Patient-Level Adjustment Factors

CMS is proposing revisions to the methodology for determining the payment rates under the IPF PPS for psychiatric hospitals and psychiatric units based on a review of the data and information collected in prior years in accordance with section 1886(s)(5)(A) of the Social Security Act, as added by the Consolidated Appropriations Act, 2023. CMS is proposing revisions to the IPF patient-level adjustment factors. The patient-level adjustments include Medicare Severity Diagnosis Related Groups (MS–DRGs) assignment of the patient’s principal diagnosis, selected comorbidities, patient age, and the variable per diem adjustments.

The IPF PPS uses the patient and facility-level adjustment factors derived from the regression model implemented in 2005. In this proposed rule, we have updated the regression model used to determine IPF PPS payment adjustments to reflect costs and claims data for 2019, 2020, and 2021. Based on our analysis of the more recent claims and costs data, as well as public comments received in the FY 2022 and FY 2023 IPF PPS rules, we are proposing to change the patient-level adjustments for which we adjust payment. We are proposing to implement these revisions in a budget-neutral manner (that is, estimated payments to IPFs for FY 2025 would be the same with or without the proposed revisions). 

Proposed Increase to the Electroconvulsive Therapy Payment per Treatment

In addition, analysis of the latest IPF PPS claims and cost data found that ancillary costs for stays that include electroconvulsive therapy (ECT) treatments have increased since 2005 by a greater amount than the current ECT per treatment payment under the IPF PPS. Therefore, we are proposing to increase the IPF PPS ECT per treatment amount. The current (FY 2024) ECT payment per treatment is $385.58, and the proposed FY 2025 ECT payment per treatment is $660.30. We believe this increase would help ensure that the patients who need ECT are more able to access it. 

FY 2025 Wage Index Update for Revised Census Data  

IPF regulations require CMS to use the best Medicare data available to estimate the average inpatient operating and capital-related costs per day, including an appropriate wage index to adjust for wage differences. We update the wage index annually based on the most recent available acute care hospital wage index, without any floors or reclassifications applicable under the Medicare Inpatient Prospective Payment System. Historically, we have also updated the Core‑Based Statistical Area (CBSA) delineations in accordance with the latest available Office of Management and Budget (OMB) Bulletin. For FY 2025, we are proposing to adopt the CBSA Labor Market Areas for the IPF PPS wage index as defined in the OMB Bulletin 23-01. We are also proposing that providers transitioning from rural to urban based on these CBSA revisions would receive two-thirds of the rural adjustment in FY 2025, one-third of the rural adjustment in FY 2026, and no rural adjustment in FY 2027. This proposed approach is consistent with how we implemented this policy in previous years.   

Clarification of Requirements for Reporting Ancillary Charges and All-Inclusive Status Eligibility Under the IPF PPS

The Consolidated Appropriations Act, 2023 (CAA, 2023) requires the collection of data and information, such as charges related to ancillary services, to revise the IPF PPS. Currently, CMS expects IPFs with a charge structure to report ancillary costs and charges on cost reports, while IPFs without this cost structure have the option to use an alternative method of cost reporting by filing all-inclusive cost reports. All-inclusive cost reporting accommodates these hospitals’ inability to allocate costs to charges and allows them to use an alternative cost allocation method. Historically, there have been a small number of hospitals that file all-inclusive cost reports, which consistently do not include or have very minimal ancillary cost data. These have mostly included Indian Health Service (IHS) hospitals and government-owned psychiatric and acute care hospitals. However, CMS has observed a notable increase in the number of IPFs erroneously filing all-inclusive cost reports.  

CMS is clarifying the eligibility criteria for the option to elect to file an all-inclusive cost report and will make operational changes to ensure that only government-owned or tribally owned IPF hospitals are permitted to file an all-inclusive cost report for cost reporting periods beginning on or after October 1, 2024. By improving the reporting of ancillary costs and charges, CMS would be able to increase accuracy of future payment refinements to the IPF PPS, which would further advance behavioral health treatment and support IPFs that provide care to beneficiaries with more complex and costlier conditions. 

Request for Information (RFI) about IPF PPS Patient Assessment Instrument Required by the Consolidated Appropriations Act, 2023 (CAA, 2023) 

The CAA, 2023, requires IPFs to collect and submit standardized patient assessment data on specified categories. This data will enable CMS to propose future revisions to the IPF PPS that would more accurately pay for care, monitor quality, and assess for disparities in behavioral health care. Therefore, CMS is including an RFI to solicit comments with the goal of engaging the public to identify meaningful data elements for collection that are appropriate for the acute inpatient psychiatric care setting and potential criteria for the development and implementation of the instrument. In addition, we are seeking to understand the burden on IPFs that this additional data collection would impose and soliciting comment on ways we might minimize this burden by evaluating whether any data that is currently collected through one or more existing assessment instruments in other settings, or collected as part of IPFs’ existing processes, could be collected as standardized patient assessment data elements for the IPF-PAI.

Request for Information (RFI) about IPF PPS Facility-Level Adjustment Factors

The CAA, 2023, requires CMS to revise the IPF PPS methodology for determining payment rates for FY 2025 and subsequent years. CMS is seeking input on potential revisions to the IPF PPS facility-level adjustments in the future based on the results of more recent IPF cost and claim analysis. An analysis of 2019 through 2021 IPF claims and costs data identified potential changes in the regression factors for IPFs with a rural location and teaching status and suggested there may be value in including a new facility-level variable for safety net patient population. We also analyzed a potential adjustment based on the Medicare Safety Net Index (MSNI), developed by MedPAC as a recommended alternative to the current statutorily required methodology for disproportionate share payments to IPPS hospitals. In this proposed rule, we discuss considerations related to applicability and modeling that demonstrate the effect of revising the rural and teaching status adjustment factors, as well as the inclusion of a new facility-level variable for safety net patient populations. In future rulemaking, updating these facility-level adjustment factors could more accurately pay for care, support psychiatry residency training, and support IPFs in rural and underserved areas. We welcome feedback on this RFI. 

Proposed Updates to the Inpatient Psychiatric Facilities Quality Reporting (IPFQR) Program

The IPFQR Program requires that all IPFs paid under the IPF PPS submit certain specified quality data to CMS in a form and manner and within the timeframes that CMS prescribes. IPFs that do not submit the specified data on quality measures as required by the IPFQR Program receive a 2.0 percentage point reduction to their annual payment update. The IPFQR Program aims to assess and foster improvement in the quality of care provided to patients in IPFs. By requiring IPFs to submit quality data to CMS and by CMS publicly reporting these data under the IPFQR Program, CMS ensures that patients are able to make more informed decisions about their healthcare options.

In this proposed rule, CMS is proposing to adopt one new measure, the 30-Day Risk-Standardized All-Cause Emergency Department Visit Following an Inpatient Psychiatric Facility Discharge measure (also referred to as the IPF ED Visit measure). This claims-based measure would assess the proportion of patients 18 and older with an emergency department visit, including observation stays, within 30 days of discharge from an IPF without subsequent admission. Patients who are subsequently admitted to an acute care hospital or IPF are represented under the Thirty-Day All-Cause Unplanned Readmission Following Psychiatric Hospitalization in an Inpatient Psychiatric Facility measure, which is already in the IPFQR Program. By adopting the IPF ED Visit measure, the IPFQR Program would provide a more complete assessment of post-discharge acute care and encourage improvements in discharge planning and care coordination. 

Additionally, CMS is proposing to require IPFs to submit patient-level quality data on a quarterly basis (as opposed to the current annual basis). This would align the IPFQR Program with other quality reporting programs that require patient-level data submission on a quarterly basis and would reduce data strains on IPF systems.

[1] https://www.whitehouse.gov/briefing-room/presidential-actions/2021/07/09/executive-order-on-promoting-competition-in-the-american-economy/

• Previous Newsroom article
• Next Newsroom article

CMS News and Media Group Catherine Howden, Director Media Inquiries Form 202-690-6145