impact of covid 19 case study

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Public health impact...

Public health impact of covid-19 vaccines in the US: observational study

Linked editorial.

The benefits of large scale covid-19 vaccination

• Related content
• Peer review
• Amitabh Bipin Suthar , epidemiologist ,
• Jing Wang , epidemiologist ,
• Victoria Seffren , epidemiologist ,
• Ryan E Wiegand , statistician ,
• Sean Griffing , epidemiologist ,
• Elizabeth Zell , statistician
• Coronavirus Disease (COVID-19) Response, Centers for Disease Control and Prevention, Atlanta, GA, USA
• Correspondence to: A B Suthar icf4{at}cdc.gov (or @AmitabhSuthar on Twitter)
• Accepted 11 March 2022

Objective To evaluate the impact of vaccine scale-up on population level covid-19 mortality and incidence in the United States.

Design Observational study.

Setting US county level case surveillance and vaccine administration data reported from 14 December 2020 to 18 December 2021.

Participants Residents of 2558 counties from 48 US states.

Main outcome measures The primary outcome was county covid-19 mortality rates (deaths/100 000 population/county week). The secondary outcome was incidence of covid-19 (cases/100 000 population/county week). Incidence rate ratios were used to compare rates across vaccination coverage levels. The impact of a 10% improvement in county vaccination coverage (defined as at least one dose of a covid-19 vaccine among adults ≥18 years of age) was estimated During the eras of alpha and delta variant predominance, the impact of very low (0-9%), low (10-39%), medium (40-69%), and high (≥70%) vaccination coverage levels was compared.

Results In total, 30 643 878 cases of covid-19 and 439 682 deaths associated with covid-19 occurred over 132 791 county weeks. A 10% improvement in vaccination coverage was associated with an 8% (95% confidence interval 8% to 9%) reduction in mortality rates and a 7% (6% to 8%) reduction in incidence. Higher vaccination coverage levels were associated with reduced mortality and incidence rates during the eras of alpha and delta variant predominance.

Conclusions Higher vaccination coverage was associated with lower rates of population level covid-19 mortality and incidence in the US.

Introduction

As of 11 April 2022, 497 960 492 cases covid-19 and 6 181 850 covid-19 related deaths had been reported globally, and 80 260 092 covid-19 cases and 983 237 covid-19 related deaths had been reported in the United States. 1 2 The US death toll recently surpassed the 1918 Spanish flu as the deadliest pandemic in recent history. 3 In addition to covid-19 related deaths, the pandemic has also had indirect effects on other health conditions. These effects are captured in excess mortality and reduced life expectancy estimates. Domestically, life expectancy decreased by 1.5 years from 2019 to 2020, representing the largest reduction since the second world war. 4

Messenger RNA (mRNA) covid-19 vaccines developed by Pfizer-BioNTech and Moderna and an adenovirus covid-19 vaccine developed by Johnson & Johnson have become valuable tools to combat this pandemic. Clinical trials evaluating efficacy against symptomatic infection found that the Pfizer-BioNTech vaccine was 95.0% effective, the Moderna vaccine was 94.1% effective, and the Janssen vaccine (Johnson & Johnson) was 66.3% effective. 5 6 7 The US Food and Drug Administration (FDA) granted emergency use authorization for mRNA vaccines in December 2020 and the Janssen vaccine in February 2021. FDA approval for the Pfizer and Moderna vaccines was granted in August 2021 and January 2022, respectively. 8 Emergency use authorization was further granted to additional doses of the mRNA vaccines for certain populations. 8 As of 11 April 2022, nearly 570 million vaccine doses have been administered in the US and more than 11 billion vaccine doses have been administered globally. 1 2 The World Health Organization’s target is to vaccinate 70% of the world’s population by mid-2022. 9

Across countries, the real world effectiveness of the covid-19 vaccines has largely been consistent with estimates of efficacy observed in clinical trials. 10 11 12 In addition to the individual level effect on disease risk and progression, vaccines may also have secondary benefits of slowing spread and reducing onward transmission and its associated morbidity and mortality. 13 Population level data and analyses have been limited. 14 15 We aimed to estimate how increasing county coverage of vaccines affected population level mortality and incidence of covid-19.

Study design

Our observational study of the US population used national, county level surveillance data. In the US, counties are a geographic administrative unit below states and territories and include the nation’s capital, Washington DC. The US Centers for Disease Control and Prevention (CDC) receives surveillance data from 3224 US counties (or county equivalents). We included and analyzed county covid-19 cases, deaths, and vaccinations reported to the CDC. We tracked mortality as our primary outcome and incidence (using reported probable and confirmed covid-19 cases) as our secondary outcome. We calculated county level incidence by standardizing reported county cases and deaths per 100 000 population over a week. 2

Study definitions

We defined a case as one that met the Council of State and Territorial Epidemiologists’ surveillance case definitions as confirmed or probable covid-19 and a death as those that were related to covid-19, as determined or reported by jurisdictions. 16 17 Each vaccine dose administered was attributed to the county in which the person resided. 18 We defined the county vaccination coverage as the number of people aged ≥18 years who received at least one dose of covid-19 vaccine among the total number of people aged ≥18 years old residing in that county. 2

Data sources

For case and death counts disaggregated by county and week, we used the CDC’s managed case surveillance dataset, which includes the most recent numbers reported by states, territories, and other jurisdictions. This dataset is populated by routine reporting from jurisdictions to the CDC. 16 To document new cases, jurisdictions may use the date that a case was reported to the health department, a person took a covid-19 test, a laboratory confirmed a covid-19 test as positive, or a person was diagnosed as having covid-19 by a clinician. For death reporting, jurisdictions may use the date when the death was reported to the health department or the date of covid-19 associated death. 2 We retrieved counts of covid-19 vaccine doses administered by week and county from the CDC’s managed vaccine dataset. This dataset includes covid-19 vaccination data (including the date of vaccine administration, the number of doses administered, and county of residence, among other variables) reported to the CDC by jurisdictions, pharmacies, and federal entities through Immunization Information Systems, the Vaccine Administration Management System, or direct submission of vaccination records. 19 The population data, used for denominators to measure vaccination coverage, came from the vintage 2019 US population estimates. 20

Inclusion criteria

We included case surveillance and vaccine administration data from 14 December 2020 to 18 December 2021. We included people at least 18 years of age with a valid county of residence in one of the states or territories who received at least one covid-19 vaccination. Given that population benefits may extend beyond the primary vaccine recipient, we included case and mortality data across all ages. Data completeness was an inclusion criterion for analysis. We used a 70% threshold for data completeness of reporting county of residence across all data sources. Specifically, we excluded a jurisdiction if more than 30% of the case, death, and/or vaccination data for the jurisdiction were contributed by unspecified or unknown counties of residence. We excluded Texas and Hawaii because vaccination data were unavailable at county level. We excluded county equivalents in territories except for Puerto Rico and Guam, either because the county level population data of adults ≥18 years old were unavailable (US Virgin Islands) or because the county equivalent vaccination data were unavailable (all other territories). In addition, we excluded eight counties in California with a population of fewer than 20 000 people, as California does not report the vaccination data of counties with under 20 000 people. We excluded the Kusilvak Census Area in Alaska owing to unavailable vaccination data and the Valdez-Cordova Census Area in Alaska because the case and mortality data were unavailable. We excluded the District of Columbia, villages in Guam, and municipalities in Puerto Rico because of a lack of mobility data. Finally, we excluded Rio Arriba County in New Mexico because the social vulnerability index was missing. In addition, we excluded any county week missing covariate information used in regression models.

Data analysis

County of residence case and first dose covid-19 vaccination data were aggregated by Morbidity and Mortality Weekly Report (MMWR) week beginning with MMWR week 2020-51 (13-19 December 2020) and ending with MMWR week 2021-50 (12-18 December 2021). 21 The CDC and Agency for Toxic Substances and Disease Registry’s social vulnerability index encompasses socioeconomic status (that is, poverty rates, unemployment rates, income levels, and education levels), household composition and disability (that is, ages, disability, and single parent households), minority status, language capability, and housing type and transportation (that is, multi-unit structures, mobile homes, crowding levels, vehicle ownership, and group housing) into a single measure. 22 Google’s community mobility reports help to measure changes in community mobility related to covid-19. 23 To prevent confounding related to the social vulnerability and mobility of communities, we included these variables in the model. 24 25

We used generalized linear mixed models assuming a negative binomial outcome distribution to assess associations between vaccination coverage and rates of deaths and cases by using continuous estimates. 26 We used a first order autoregressive correlation structure to account for multiple observations per county and for potential autocorrelation. We included county level population as an offset and included social vulnerability index categorized into quarters and retail and work mobility data as covariates. To account for cases occurring during the period of developing immunity, a county remained in the lower vaccination category for two weeks before moving to the next vaccination category.

We calculated estimates during the period of alpha variant predominance and the period of delta variant predominance (starting when the national prevalence of delta was estimated to be at least 50%—that is, the week of 20 June 2021 onward) categorically. 27 28 We compared four different categories for county vaccination coverage: very low (0-9% of the county had been vaccinated), low (10-39% of the county had been vaccinated), medium (40-69% of the county had been vaccinated), and high (≥70% of the county had been vaccinated) during the era of alpha variant predominance. As with the continuous analyses, to account for cases occurring during the period of developing immunity, a county remained in the lower vaccination category for two weeks before moving to the next vaccination category. Moreover, we included county level population as an offset and included social vulnerability index categorized by quarter and retail and work mobility data as covariates. Given the inadequate number of county weeks accrued with very low and low vaccination coverage, we compared the mortality and incidence rates for medium and high coverage during the era of delta variant predominance.

Sensitivity analyses

We did three sensitivity analyses with the continuous analyses. The first sensitivity analysis was to compare definitions of vaccination being at least one dose with including only fully vaccinated people (that is, at least two mRNA doses or a single adenovirus dose). The second was to compare use of a stringency level for data completeness of 70% and 90%. The third was to compare estimates with and without the two week lag period.

Patient and public involvement

We used routinely generated covid-19 vaccine and case surveillance data. No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results.

First year of vaccine roll-out

We included data from 2558 counties in 48 US states ( fig 1 ). In total, we observed 30 643 878 cases of covid-19 and 439 682 covid-19 related deaths over 132 791 county weeks ( table 1 ). Every 10% improvement in vaccination coverage was associated with an 8% (95% confidence interval 8% to 9%) reduction in mortality rates ( fig 2 ) and with a 7% (6% to 8% reduction in case incidence ( fig 2 ).

County scale-up of vaccinations from 13 December 2020 to 18 December 2021. Light purple lines represent individual counties; dark purple line is aggregated estimate

• Download figure
• Open in new tab
• Download powerpoint

Characteristics of included counties. Values are median (range) unless stated otherwise

• View inline

Effect of vaccination coverage on county covid-19 related mortality (top) and incidence (bottom) during first year of vaccine roll-out. Analyses are from 2558 counties in 48 US jurisdictions. Model controlled for county population size, social vulnerability index, and mobility changes

Era of alpha variant predominance

In total, we observed 15 493 299 cases of covid-19 and 263 873 covid-19 related deaths over 70 189 county weeks during the era of alpha variant predominance. Compared with very low coverage, low (incidence rate ratio 0.40, 95% confidence interval 0.39 to 0.42), medium (0.25, 0.23 to 0.26), and high (0.19, 0.16 to 0.22) vaccination coverage categories had lower rates of mortality ( fig 3 ). Compared with very low coverage, low (incidence rate ratio 0.43, 0.41 to 0.44), medium (0.30, 0.29 to 0.32), and high (0.20, 0.18 to 0.22) vaccination coverage categories had lower incidence rates ( fig 3 ).

Effect of vaccination coverage on county covid-19 mortality and incidence during era of alpha variant predominance. Analyses are from 2557 counties in 48 US jurisdictions. Model controlled for county population size, social vulnerability index, and mobility changes

Era of delta variant predominance

In total, we observed 15 150 579 cases of covid-19 and 175 809 covid-19 related deaths over 62 602 county weeks during the era of delta variant predominance. When comparing high and medium coverage, we observed similar mortality effects during the era of alpha variant predominance (incidence rate ratio 0.77, 0.66 to 0.90) and the era of delta variant predominance (0.75, 0.71 to 0.79). When comparing high and medium coverage, we observed smaller incidence effect sizes during the era of delta variant predominance (incidence rate ratio 0.90, 0.87 to 0.94) compared with the era of alpha variant predominance (0.66, 0.60 to 0.73).

We observed sustained reductions in county mortality and incidence rates when we included only fully vaccinated people in the vaccination coverage categories, when we increased our data stringency level, and when we removed the two week immunity lag period ( fig 4 ).

Sensitivity analyses of including only fully vaccinated people, increasing data stringency requirements, and removing two week immunity lag period. *In baseline group, vaccination coverage refers to coverage of at least one dose of vaccine, 2558 counties and 48 US jurisdictions included had ≥70% completeness rates of reporting county of residence, and study period was 14 December 2020 to 18 December 2021. †Vaccination coverage refers to coverage of fully vaccinated people. ‡2164 counties and 42 US jurisdictions included had ≥90% completeness rates of reporting county of residence. §Two week immunity period was removed

Using data from 2558 counties—representing nearly 300 million people and 80% of the US population—we found that increasing the vaccination coverage in counties was associated with a reduced incidence of covid-19 related mortality and cases. We observed decreasing trends in mortality and case incidence associated with higher levels of vaccination coverage across the eras of both alpha and delta variant predominance. This effect was robust to various sensitivity analyses, which improves prediction and confidence in these findings.

Covid-19 associated mortality remains one of the most important clinical outcomes to guide public health decision making, measure pandemic severity, and evaluate mitigation efforts. It was our primary outcome. In the US, death registration rates, and cause of death ascertainment, remain high. This suggests that US mortality surveillance systems have been, and will continue to be, useful for covid-19 mortality surveillance. Previous vaccine studies have shown survival benefits at the individual level. 29 We observed that these benefits may extend to the population level; counties with high coverage had a greater than 80% reduction in mortality rates compared with largely unvaccinated counties. Given that infection fatality rates for covid-19 increase with age, counties with a higher proportion of older people may have more covid-19 related mortality and stand to benefit from high coverage of covid-19 vaccines. 30

We used reported cases as a proxy for incidence for our secondary outcome. Although reliable, available across jurisdictions, and reported continuously, reported cases may not reflect true transmission rates because of variation in when people seek out testing. 31 For example, people without symptoms may not actively seek out testing of their own accord but may be important to test for gauging disease transmission. Owing to more recent reopening requirements for workplaces, restaurants, entertainment venues, schools, and outgoing international air travel, more people without symptoms may be seeking out testing. 32 These requirements, and their uptake, may vary across states and counties. Nevertheless, the reduction in incidence observed with increasing vaccination coverage is consistent with surveillance data from other countries that have achieved high vaccination coverage and emerging evidence on transmission from contact tracing programs. 1 33

Comparison with other studies

Increasing vaccination coverage may play a role in mitigating the effects of the delta and omicron variants and reduce the emergence of future variants. 34 35 By 27 June 2021 the delta variant made up more than 50% of circulating variants in the US, increasing to almost 100% by 21 September 2021. 2 More recently, the omicron variant was first reported on 1 December 2021 and comprised 95% of circulating variants by 2 January 2022. 2 The delta variant had increased transmissibility and possible increased virulence compared with earlier SARS-CoV-2 strains. 36 37 In our study, by the time the delta variant predominated, counties with lower levels of vaccination coverage (that is, 0-39%) were rare. Nevertheless, our findings of continued population level protection against death and reductions in population level protection against infection during the period of delta variant predominance seem to be consistent with clinical literature on vaccine effectiveness. 29 38 39 40 Additional studies aimed at assessing the population impact of vaccines during the period of delta variant predominance merit consideration for validating our observations. Although our study period did not include the period of omicron variant predominance, data suggesting reduced vaccine effectiveness and the importance of staying up to date on covid-19 vaccinations are emerging and may lead to changes in population level vaccine impact that merit exploration. 41 42 43 Continuing to monitor the delta and omicron variants, and the emergence of other variants of interest, is critical and will require ongoing genomic surveillance.

Clinical studies indicate that a single dose of an mRNA vaccine provides a lower level of protection than two doses. 44 Furthermore, two mRNA doses seem to be more effective than a single adenovirus dose against symptomatic infection. 5 6 7 We defined people with at least one dose of vaccine as being vaccinated for the purposes of vaccination coverage. Given that our study design used population surveillance data, changing our coverage definition to include only fully vaccinated people would place people with a single dose of mRNA vaccine in our referent, the very low coverage category. This may introduce bias in the incidence and mortality estimates. When we changed our definition of vaccination coverage to being fully dosed during sensitivity analysis, we did not find increased effect sizes, as would be expected from clinical studies. 44 Ongoing vaccine studies continue to evaluate the comparative effectiveness of vaccines by manufacturer. 10

Given that only people aged 18 and older were eligible for vaccination across vaccines during most of our study period, we used this age threshold to define vaccination coverage. Pediatric studies will be a welcome contribution to understanding the effects of vaccines on younger age groups, when feasible. As of September 2021 the FDA began recommending a third dose for specific populations. 8 Owing to waning immunity and variant induced changes in vaccine effectiveness, additional doses may be needed in specific populations and scenarios. Further studies may benefit from evaluating the population impact of vaccination coverage by using different definitions and eligibility scenarios.

Limitations of study

Several limitations should be considered when interpreting these data. We chose vaccination coverage thresholds based on programmatic experience; exploring coverage thresholds above 70% may be worth examining in future research once more counties have achieved these levels for extended periods of time. We excluded some jurisdictions because they did not have county level information on immunizations, cases, and deaths for at least 70% of their counties. Additional markers of disease severity, such as hospital admissions, were not explored in this study owing to possible differences in ascertainment and reporting coverage across jurisdictions. Given the limited number of variables that were known to affect mortality and incidence, collected at the county level, and available on a weekly basis, we did not control for masking, physical distancing, or other similar potential confounding variables in this study. Furthermore, given the limited number of county weeks, we lacked power to stratify by time periods and cannot rule out the possibility of temporal confounding. Finally, given that we used aggregate case surveillance data to have the most complete case and death data available, other characteristics of cases, such as demographics and comorbidities, were not available. States, territories, and jurisdictions adapt national guidance on which date to use for case reporting. 17 In this study we collated county data across these geographic areas. A time difference may be present depending on which date a health department uses; however, this is unlikely to be substantial enough to affect which week a case or death occurs. Naturally acquired immunity resulting from SARS-CoV-2 infection may have affected the reduced case incidence observed during the study period. These are limitations of our study. Nevertheless, reductions in incidence and death observed in emerging US data using alternative data sources and study designs give us confidence in the directionality and magnitude of our estimates. 45 46

Conclusions and policy implications

In addition to individual level benefits, we observed that vaccines protect communities against severe disease and infection. Higher coverage of vaccines seemed to confer greater levels of community benefits. Given that community benefits are rooted in individual benefits, for which vaccine effectiveness has been established in countries around the world, these data may be generalizable to other countries. Future research may benefit from evaluating macroeconomic effects of improving population health, such as changes in employment rates and gross domestic product resulting from reopening society. Vaccines should be deployed strategically with public health and social measures based on ongoing levels of transmission.

What is already known on this topic

The public health impact of scaling up covid-19 vaccination remains largely uncharacterized

What this study adds

Higher vaccination coverage was associated with lower rates of population level incidence of covid-19 and mortality related to covid-19

This community level benefit complements the large body of evidence indicating individual level benefits of covid-19 vaccination

Ethics statements

Ethical approval.

Not applicable.

Data availability statement

Centers for Disease Control and Prevention covid-19 vaccination, case, and death data are available at data.cdc.gov .

Acknowledgments

JW is an Oakridge Institute for Science and Education fellow and SG is a lieutenant with the United States Public Health Service. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Use of trade names and commercial sources is for identification only and does not imply endorsement by the US Department of Health and Human Services.

Contributors: ABS, JW, VS, REW, SG, and EZ conceived and designed the study. JW, VS, REW, and EZ analyzed the data. ABS, JW, VS, REW, SG, and EZ wrote the manuscript. ABS, JW, VS, REW, SG, and EZ agree with manuscript’s results and conclusions. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. ABS is the guarantor.

Funding: None.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained

Dissemination to participants and related patient and public communities: This publication will be shared on appropriate websites and social media platforms and at meetings.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

• ↵ World Health Organization. WHO Coronavirus (COVID-19) Dashboard. https://covid19.who.int/ .
• ↵ Centers for Disease Control and Prevention. COVID Data Tracker. https://covid.cdc.gov/covid-data-tracker/#datatracker-home .
• ↵ Centers for Disease Control and Prevention. 1918 Pandemic (H1N1 virus). 2019. https://www.cdc.gov/flu/pandemic-resources/1918-pandemic-h1n1.html .
• ↵ Arias E, Tejada-Vera B, Ahmad F, Kochanek KD. Provisional Life Expectancy Estimates for 2020. Vital Statistics Rapid Release. Report No. 015. 2021. https://stacks.cdc.gov/view/cdc/107201 .
• Polack FP ,
• Thomas SJ ,
• Kitchin N ,
• C4591001 Clinical Trial Group
• El Sahly HM ,
• COVE Study Group
• Vandebosch A ,
• ENSEMBLE Study Group
• ↵ US Food and Drug Administration. COVID-19 Vaccines. https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/covid-19-vaccines .
• ↵ World Health Organization. Strategy to Achieve Global Covid-19 Vaccination by mid-2022. https://cdn.who.int/media/docs/default-source/immunization/covid-19/strategy-to-achieve-global-covid-19-vaccination-by-mid-2022.pdf .
• Tenforde MW ,
• Rhoads JP ,
• IVY Network
• Foulkes S ,
• SIREN Study Group
• Angulo FJ ,
• McLaughlin JM ,
• ↵ COVID-19 Scenario Modeling Hub Team. COVID-19 Scenario Modeling Hub. https://covid19scenariomodelinghub.org/viz.html .
• McNamara LA ,
• Wiegand RE ,
• ↵ Centers for Disease Control and Prevention. About CDC COVID-19 Data. 2021. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/about-us-cases-deaths.html .
• ↵ Council of State and Territorial Epidemiologists. Update to the standardized surveillance case definition and national notification for 2019 novel coronavirus disease (COVID-19). 2021. https://cdn.ymaws.com/www.cste.org/resource/resmgr/21-ID-01_COVID-19_updated_Au.pdf .
• ↵ Centers for Disease Control and Prevention. Reporting COVID-19 Vaccinations in the United States. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/reporting-vaccinations.html .
• ↵ Centers for Disease Control and Prevention. About COVID-19 Vaccine Delivered and Administration Data. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/distributing/about-vaccine-data.html .
• ↵ United States Census Bureau. Datasets. https://www2.census.gov/programs-surveys/popest/datasets/2010-2019/ .
• ↵ Centers for Disease Control and Prevention. MMWR weeks ending log 2020-2021. https://stacks.cdc.gov/view/cdc/84011/ .
• ↵ Centers for Disease Control and Prevention, Agency for Toxic Substances and Disease Registry. CDC SVI 2018 Documentation – 1/31/2020. https://www.atsdr.cdc.gov/placeandhealth/svi/documentation/SVI_documentation_2018.html .
• ↵ Google. COVID-19 Community Mobility Reports. https://www.google.com/covid19/mobility/ .
• Wellenius GA ,
• Vispute S ,
• Espinosa V ,
• France AM ,
• Lambrou AS ,
• Steele MK ,
• Strain Surveillance and Emerging Variants Bioinformatic Working Group ,
• Strain Surveillance and Emerging Variants NS3 Working Group
• Scobie HM ,
• Johnson AG ,
• Suthar AB ,
• Hanage WP ,
• Owusu-Boaitey N ,
• Cochran KB ,
• Meyerowitz-Katz G
• Schubert S ,
• Couture A ,
• ↵ Centers for Disease Control and Prevention. Community, Work, and School: Information for Where You Live, Work, Learn, and Play. 2021. https://www.cdc.gov/coronavirus/2019-ncov/community/index.html .
• Christie A ,
• Brooks JT ,
• Sauber-Schatz EK ,
• Honein MA ,
• CDC COVID-19 Response Team
• Walensky RP ,
• Twohig KA ,
• COVID-19 Genomics UK (COG-UK) consortium
• Rosenberg ES ,
• Holtgrave DR ,
• Dorabawila V ,
• Fowlkes A ,
• Gaglani M ,
• Groover K ,
• Thiese MS ,
• Ellingson K ,
• HEROES-RECOVER Cohorts
• Lopez Bernal J ,
• Andrews N ,
• Ackerson BK ,
• Lauring AS ,
• Chappell JD ,
• Influenza and Other Viruses in the Acutely Ill (IVY) Network
• Pilishvili T ,
• Fleming-Dutra KE ,
• Farrar JL ,
• Vaccine Effectiveness Among Healthcare Personnel Study Team
• Clemens JD ,
• Zhang Z-F ,
• Swerdlow DL ,

• Research article
• Open access
• Published: 04 June 2021

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

• Israel Júnior Borges do Nascimento 1 , 2 ,
• Dónal P. O’Mathúna 3 , 4 ,
• Thilo Caspar von Groote 5 ,
• Hebatullah Mohamed Abdulazeem 6 ,
• Ishanka Weerasekara 7 , 8 ,
• Ana Marusic 9 ,
• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 10 ,
• Vinicius Tassoni Civile 11 ,
• Irena Zakarija-Grkovic 9 ,
• Tina Poklepovic Pericic 9 ,
• Alvaro Nagib Atallah 11 ,
• Santino Filoso 12 ,
• Nicola Luigi Bragazzi 13 &
• Milena Soriano Marcolino 1

On behalf of the International Network of Coronavirus Disease 2019 (InterNetCOVID-19)

BMC Infectious Diseases volume  21 , Article number:  525 ( 2021 ) Cite this article

15k Accesses

28 Citations

13 Altmetric

Metrics details

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as “critically low”. Identified symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

Conclusions

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Peer Review reports

The spread of the “Severe Acute Respiratory Coronavirus 2” (SARS-CoV-2), the causal agent of COVID-19, was characterized as a pandemic by the World Health Organization (WHO) in March 2020 and has triggered an international public health emergency [ 1 ]. The numbers of confirmed cases and deaths due to COVID-19 are rapidly escalating, counting in millions [ 2 ], causing massive economic strain, and escalating healthcare and public health expenses [ 3 , 4 ].

The research community has responded by publishing an impressive number of scientific reports related to COVID-19. The world was alerted to the new disease at the beginning of 2020 [ 1 ], and by mid-March 2020, more than 2000 articles had been published on COVID-19 in scholarly journals, with 25% of them containing original data [ 5 ]. The living map of COVID-19 evidence, curated by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), contained more than 40,000 records by February 2021 [ 6 ]. More than 100,000 records on PubMed were labeled as “SARS-CoV-2 literature, sequence, and clinical content” by February 2021 [ 7 ].

Due to publication speed, the research community has voiced concerns regarding the quality and reproducibility of evidence produced during the COVID-19 pandemic, warning of the potential damaging approach of “publish first, retract later” [ 8 ]. It appears that these concerns are not unfounded, as it has been reported that COVID-19 articles were overrepresented in the pool of retracted articles in 2020 [ 9 ]. These concerns about inadequate evidence are of major importance because they can lead to poor clinical practice and inappropriate policies [ 10 ].

Systematic reviews are a cornerstone of today’s evidence-informed decision-making. By synthesizing all relevant evidence regarding a particular topic, systematic reviews reflect the current scientific knowledge. Systematic reviews are considered to be at the highest level in the hierarchy of evidence and should be used to make informed decisions. However, with high numbers of systematic reviews of different scope and methodological quality being published, overviews of multiple systematic reviews that assess their methodological quality are essential [ 11 , 12 , 13 ]. An overview of systematic reviews helps identify and organize the literature and highlights areas of priority in decision-making.

In this overview of systematic reviews, we aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Methodology

Research question.

This overview’s primary objective was to summarize and critically appraise systematic reviews that assessed any type of primary clinical data from patients infected with SARS-CoV-2. Our research question was purposefully broad because we wanted to analyze as many systematic reviews as possible that were available early following the COVID-19 outbreak.

Study design

We conducted an overview of systematic reviews. The idea for this overview originated in a protocol for a systematic review submitted to PROSPERO (CRD42020170623), which indicated a plan to conduct an overview.

Overviews of systematic reviews use explicit and systematic methods for searching and identifying multiple systematic reviews addressing related research questions in the same field to extract and analyze evidence across important outcomes. Overviews of systematic reviews are in principle similar to systematic reviews of interventions, but the unit of analysis is a systematic review [ 14 , 15 , 16 ].

We used the overview methodology instead of other evidence synthesis methods to allow us to collate and appraise multiple systematic reviews on this topic, and to extract and analyze their results across relevant topics [ 17 ]. The overview and meta-analysis of systematic reviews allowed us to investigate the methodological quality of included studies, summarize results, and identify specific areas of available or limited evidence, thereby strengthening the current understanding of this novel disease and guiding future research [ 13 ].

A reporting guideline for overviews of reviews is currently under development, i.e., Preferred Reporting Items for Overviews of Reviews (PRIOR) [ 18 ]. As the PRIOR checklist is still not published, this study was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 statement [ 19 ]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [ 14 ].

Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.

Eligibility criteria

Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19 including, but not limited to, those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings, or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.

No restrictions were imposed on the design of the primary studies included within the systematic reviews, the last search date, whether the review included meta-analyses or language. Reviews related to SARS-CoV-2 and other coronaviruses were eligible, but from those reviews, we analyzed only data related to SARS-CoV-2.

No consensus definition exists for a systematic review [ 20 ], and debates continue about the defining characteristics of a systematic review [ 21 ]. Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion [ 14 ]. That is supported by a recent scoping review about guidance for overviews of systematic reviews [ 22 ].

Thus, for this study, we defined a systematic review as a research report which searched for primary research studies on a specific topic using an explicit search strategy, had a detailed description of the methods with explicit inclusion criteria provided, and provided a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Cochrane and non-Cochrane systematic reviews were considered eligible for inclusion, with or without meta-analysis, and regardless of the study design, language restriction and methodology of the included primary studies. To be eligible for inclusion, reviews had to be clearly analyzing data related to SARS-CoV-2 (associated or not with other viruses). We excluded narrative reviews without those characteristics as these are less likely to be replicable and are more prone to bias.

Scoping reviews and rapid reviews were eligible for inclusion in this overview if they met our pre-defined inclusion criteria noted above. We included reviews that addressed SARS-CoV-2 and other coronaviruses if they reported separate data regarding SARS-CoV-2.

Information sources

Nine databases were searched for eligible records published between December 1, 2019, and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.

The comprehensive search strategy for each database is provided in Additional file 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported into the Covidence platform [ 23 ]. In addition to database searches, we screened reference lists of reviews included after screening records retrieved via databases.

Study selection

All searches, screening of titles and abstracts, and record selection, were performed independently by two investigators using the Covidence platform [ 23 ]. Articles deemed potentially eligible were retrieved for full-text screening carried out independently by two investigators. Discrepancies at all stages were resolved by consensus. During the screening, records published in languages other than English were translated by a native/fluent speaker.

Data collection process

We custom designed a data extraction table for this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.

We extracted the following data: article identification data (authors’ name and journal of publication), search period, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA), we extracted journal rank (quartile) and Journal Impact Factor (JIF).

We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no), and length of stay in the intensive care unit.

The following outcomes were categorized as exploratory: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).

We also collected data on reporting guidelines and requirements for the publication of systematic reviews and meta-analyses from journal websites where included reviews were published.

Quality assessment in individual reviews

Two researchers independently assessed the reviews’ quality using the “A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2)”. We acknowledge that the AMSTAR 2 was created as “a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both” [ 24 ]. However, since AMSTAR 2 was designed for systematic reviews of intervention trials, and we included additional types of systematic reviews, we adjusted some AMSTAR 2 ratings and reported these in Additional file 2 .

Adherence to each item was rated as follows: yes, partial yes, no, or not applicable (such as when a meta-analysis was not conducted). The overall confidence in the results of the review is rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains, which are items 2, 4, 7, 9, 11, 13, 15 as defined by AMSTAR 2 authors [ 24 ]. We reported our adherence ratings for transparency of our decision with accompanying explanations, for each item, in each included review.

One of the included systematic reviews was conducted by some members of this author team [ 25 ]. This review was initially assessed independently by two authors who were not co-authors of that review to prevent the risk of bias in assessing this study.

Synthesis of results

For data synthesis, we prepared a table summarizing each systematic review. Graphs illustrating the mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths, and limitations.

For analysis of the prevalence of clinical outcomes, we extracted data on the number of events and the total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9–6), using the “metaprop” function for reviews that did not perform a meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform a meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ 2 . We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. We created forest plots using the RStudio© software, with the “metafor” package (version 2.1–0) and “forest” function.

Managing overlapping systematic reviews

Some of the included systematic reviews that address the same or similar research questions may include the same primary studies in overviews. Including such overlapping reviews may introduce bias when outcome data from the same primary study are included in the analyses of an overview multiple times. Thus, in summaries of evidence, multiple-counting of the same outcome data will give data from some primary studies too much influence [ 14 ]. In this overview, we did not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic [ 14 ]. To avoid any bias in summary estimates associated with overlapping reviews, we generated forest plots showing data from individual systematic reviews, but the results were not pooled because some primary studies were included in multiple reviews.

Our search retrieved 1063 publications, of which 175 were duplicates. Most publications were excluded after the title and abstract analysis ( n = 860). Among the 28 studies selected for full-text screening, 10 were excluded for the reasons described in Additional file 3 , and 18 were included in the final analysis (Fig. 1 ) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Reference list screening did not retrieve any additional systematic reviews.

PRISMA flow diagram

Characteristics of included reviews

Summary features of 18 systematic reviews are presented in Table 1 . They were published in 14 different journals. Only four of these journals had specific requirements for systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology, and Clinical Research in Cardiology . Two journals reported that they published only invited reviews ( Journal of Medical Virology and Clinica Chimica Acta ). Three systematic reviews in our study were published as letters; one was labeled as a scoping review and another as a rapid review (Table 2 ).

All reviews were published in English, in first quartile (Q1) journals, with JIF ranging from 1.692 to 6.062. One review was empty, meaning that its search did not identify any relevant studies; i.e., no primary studies were included [ 36 ]. The remaining 17 reviews included 269 unique studies; the majority ( N = 211; 78%) were included in only a single review included in our study (range: 1 to 12). Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts, and other observational studies. We found only one review that included randomized clinical trials [ 38 ]. In the included reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020. Ten systematic reviews included meta-analyses. The list of primary studies found in the included systematic reviews is shown in Additional file 4 , as well as the number of reviews in which each primary study was included.

Population and study designs

Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS), or any other correlated complication. One review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [ 36 ], one review was focused on pediatric patients [ 34 ], and one review investigated COVID-19 in pregnant women [ 37 ]. Most reviews assessed clinical symptoms, laboratory findings, or radiological results.

Systematic review findings

The summary of findings from individual reviews is shown in Table 2 . Overall, all-cause mortality ranged from 0.3 to 13.9% (Fig. 2 ).

A meta-analysis of the prevalence of mortality

Clinical symptoms

Seven reviews described the main clinical manifestations of COVID-19 [ 26 , 28 , 29 , 34 , 35 , 39 , 41 ]. Three of them provided only a narrative discussion of symptoms [ 26 , 34 , 35 ]. In the reviews that performed a statistical analysis of the incidence of different clinical symptoms, symptoms in patients with COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figs. 3 , 4 , 5 , 6 , 7 , 8 and 9 ). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).

A meta-analysis of the prevalence of fever

A meta-analysis of the prevalence of cough

A meta-analysis of the prevalence of dyspnea

A meta-analysis of the prevalence of fatigue or myalgia

A meta-analysis of the prevalence of headache

A meta-analysis of the prevalence of gastrointestinal disorders

A meta-analysis of the prevalence of sore throat

Diagnostic aspects

Three reviews described methodologies, protocols, and tools used for establishing the diagnosis of COVID-19 [ 26 , 34 , 38 ]. The use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic method mentioned in the included studies. These diagnostic tests have been widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study with clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least two symptoms suggestive of COVID-19, together with laboratory and chest radiography abnormalities) [ 34 ].

Therapeutic possibilities

Pharmacological and non-pharmacological interventions (supportive therapies) used in treating patients with COVID-19 were reported in five reviews [ 25 , 27 , 34 , 35 , 38 ]. Antivirals used empirically for COVID-19 treatment were reported in seven reviews [ 25 , 27 , 34 , 35 , 37 , 38 , 41 ]; most commonly used were protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analogs (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in two studies [ 25 , 35 ]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation) [ 25 ]. The use of antibiotics, both empirically and to treat secondary pneumonia, was reported in six studies [ 25 , 26 , 27 , 34 , 35 , 38 ]. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [ 27 ]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. The use of mesenchymal stem cells, antifungals, and glucocorticoids were described in four reviews [ 25 , 34 , 35 , 38 ].

Laboratory and radiological findings

Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [ 25 , 29 , 30 , 32 , 33 , 34 , 35 , 39 ]; elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described in those eight reviews. Lippi et al. assessed cardiac troponin I (cTnI) [ 25 ], procalcitonin [ 32 ], and platelet count [ 33 ] in COVID-19 patients. Elevated levels of procalcitonin [ 32 ] and cTnI [ 30 ] were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections [ 33 ].

Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [ 25 , 34 , 35 , 39 , 40 , 41 ]. Those six reviews showed that septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs, and pneumothorax were observed in patients suffering from COVID-19.

Quality of evidence in individual systematic reviews

Table 3 shows the detailed results of the quality assessment of 18 systematic reviews, including the assessment of individual items and summary assessment. A detailed explanation for each decision in each review is available in Additional file 5 .

Using AMSTAR 2 criteria, confidence in the results of all 18 reviews was rated as “critically low” (Table 3 ). Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of an explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).

Risk of bias assessment, based on a reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method, were reported only in one review [ 25 ]. Five reviews presented a table summarizing bias, using various risk of bias tools [ 25 , 29 , 39 , 40 , 41 ]. One review analyzed “study quality” [ 37 ]. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis [ 28 ].

This overview of systematic reviews analyzed the first 18 systematic reviews published after the onset of the COVID-19 pandemic, up to March 24, 2020, with primary studies involving more than 60,000 patients. Using AMSTAR-2, we judged that our confidence in all those reviews was “critically low”. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings, and interventions. We found no systematic reviews on the utility of diagnostic tests.

Symptoms were reported in seven reviews; most of the patients had a fever, cough, dyspnea, myalgia or muscle fatigue, and gastrointestinal disorders such as diarrhea, nausea, or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been described in patients infected with COVID-19 [ 43 ]; however, this was not reported in any of the reviews included in this overview. During the SARS outbreak in 2002, there were reports of impairment of the sense of smell associated with the disease [ 44 , 45 ].

The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control, and treatment facilities; variability in the mortality rate fits the pattern of emerging infectious diseases [ 46 ]. Furthermore, the reported cases did not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries had limited access to diagnostic tests or have implemented testing policies later than the others. Considering the lack of reviews assessing diagnostic testing (sensitivity, specificity, and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, considerable imprecision around the calculated mortality rates existed in the early stage of the COVID-19 pandemic.

Few reviews included treatment data. Those reviews described studies considered to be at a very low level of evidence: usually small, retrospective studies with very heterogeneous populations. Seven reviews analyzed laboratory parameters; those reviews could have been useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, such as assessing which patients need to be reassessed more frequently.

All systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Most of the original studies included in the reviews were case series and case reports, impacting the quality of evidence. Such evidence has major implications for clinical practice and the use of these reviews in evidence-based practice and policy. Clinicians, patients, and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.

We acknowledge that there are numerous challenges associated with analyzing COVID-19 data during a pandemic [ 47 ]. High-quality evidence syntheses are needed for decision-making, but each type of evidence syntheses is associated with its inherent challenges.

The creation of classic systematic reviews requires considerable time and effort; with massive research output, they quickly become outdated, and preparing updated versions also requires considerable time. A recent study showed that updates of non-Cochrane systematic reviews are published a median of 5 years after the publication of the previous version [ 48 ].

Authors may register a review and then abandon it [ 49 ], but the existence of a public record that is not updated may lead other authors to believe that the review is still ongoing. A quarter of Cochrane review protocols remains unpublished as completed systematic reviews 8 years after protocol publication [ 50 ].

Rapid reviews can be used to summarize the evidence, but they involve methodological sacrifices and simplifications to produce information promptly, with inconsistent methodological approaches [ 51 ]. However, rapid reviews are justified in times of public health emergencies, and even Cochrane has resorted to publishing rapid reviews in response to the COVID-19 crisis [ 52 ]. Rapid reviews were eligible for inclusion in this overview, but only one of the 18 reviews included in this study was labeled as a rapid review.

Ideally, COVID-19 evidence would be continually summarized in a series of high-quality living systematic reviews, types of evidence synthesis defined as “ a systematic review which is continually updated, incorporating relevant new evidence as it becomes available ” [ 53 ]. However, conducting living systematic reviews requires considerable resources, calling into question the sustainability of such evidence synthesis over long periods [ 54 ].

Research reports about COVID-19 will contribute to research waste if they are poorly designed, poorly reported, or simply not necessary. In principle, systematic reviews should help reduce research waste as they usually provide recommendations for further research that is needed or may advise that sufficient evidence exists on a particular topic [ 55 ]. However, systematic reviews can also contribute to growing research waste when they are not needed, or poorly conducted and reported. Our present study clearly shows that most of the systematic reviews that were published early on in the COVID-19 pandemic could be categorized as research waste, as our confidence in their results is critically low.

Our study has some limitations. One is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. In three reviews, the methodological quality appraisal was challenging because they were published as letters, or labeled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as “not reported”, resulting in low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low-quality review methods. To make their review available more rapidly and more concisely, the authors may have omitted methodological details. A general issue during a crisis is that speed and completeness must be balanced. However, maintaining high standards requires proper resourcing and commitment to ensure that the users of systematic reviews can have high confidence in the results.

Furthermore, we used adjusted AMSTAR 2 scoring, as the tool was designed for critical appraisal of reviews of interventions. Some reviews may have received lower scores than actually warranted in spite of these adjustments.

Another limitation of our study may be the inclusion of multiple overlapping reviews, as some included reviews included the same primary studies. According to the Cochrane Handbook, including overlapping reviews may be appropriate when the review’s aim is “ to present and describe the current body of systematic review evidence on a topic ” [ 12 ], which was our aim. To avoid bias with summarizing evidence from overlapping reviews, we presented the forest plots without summary estimates. The forest plots serve to inform readers about the effect sizes for outcomes that were reported in each review.

Several authors from this study have contributed to one of the reviews identified [ 25 ]. To reduce the risk of any bias, two authors who did not co-author the review in question initially assessed its quality and limitations.

Finally, we note that the systematic reviews included in our overview may have had issues that our analysis did not identify because we did not analyze their primary studies to verify the accuracy of the data and information they presented. We give two examples to substantiate this possibility. Lovato et al. wrote a commentary on the review of Sun et al. [ 41 ], in which they criticized the authors’ conclusion that sore throat is rare in COVID-19 patients [ 56 ]. Lovato et al. highlighted that multiple studies included in Sun et al. did not accurately describe participants’ clinical presentations, warning that only three studies clearly reported data on sore throat [ 56 ].

In another example, Leung [ 57 ] warned about the review of Li, L.Q. et al. [ 29 ]: “ it is possible that this statistic was computed using overlapped samples, therefore some patients were double counted ”. Li et al. responded to Leung that it is uncertain whether the data overlapped, as they used data from published articles and did not have access to the original data; they also reported that they requested original data and that they plan to re-do their analyses once they receive them; they also urged readers to treat the data with caution [ 58 ]. This points to the evolving nature of evidence during a crisis.

Our study’s strength is that this overview adds to the current knowledge by providing a comprehensive summary of all the evidence synthesis about COVID-19 available early after the onset of the pandemic. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews.

In conclusion, in this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all the reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic could be categorized as research waste. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards to provide patients, clinicians, and decision-makers trustworthy evidence.

Availability of data and materials

All data collected and analyzed within this study are available from the corresponding author on reasonable request.

World Health Organization. Timeline - COVID-19: Available at: https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 1 June 2021.

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available at: https://coronavirus.jhu.edu/map.html . Accessed 1 June 2021.

Anzai A, Kobayashi T, Linton NM, Kinoshita R, Hayashi K, Suzuki A, et al. Assessing the Impact of Reduced Travel on Exportation Dynamics of Novel Coronavirus Infection (COVID-19). J Clin Med. 2020;9(2):601.

Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020;368(6489):395–400. https://doi.org/10.1126/science.aba9757 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fidahic M, Nujic D, Runjic R, Civljak M, Markotic F, Lovric Makaric Z, et al. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med Res Methodol. 2020;20(1):161. https://doi.org/10.1186/s12874-020-01047-2 .

EPPI Centre . COVID-19: a living systematic map of the evidence. Available at: http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx . Accessed 1 June 2021.

NCBI SARS-CoV-2 Resources. Available at: https://www.ncbi.nlm.nih.gov/sars-cov-2/ . Accessed 1 June 2021.

Gustot T. Quality and reproducibility during the COVID-19 pandemic. JHEP Rep. 2020;2(4):100141. https://doi.org/10.1016/j.jhepr.2020.100141 .

Article   PubMed   PubMed Central   Google Scholar  

Kodvanj, I., et al., Publishing of COVID-19 Preprints in Peer-reviewed Journals, Preprinting Trends, Public Discussion and Quality Issues. Preprint article. bioRxiv 2020.11.23.394577; doi: https://doi.org/10.1101/2020.11.23.394577 .

Dobler CC. Poor quality research and clinical practice during COVID-19. Breathe (Sheff). 2020;16(2):200112. https://doi.org/10.1183/20734735.0112-2020 .

Article   Google Scholar  

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 .

Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/10.1186/s13643-017-0617-1 .

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. 2020. Available from www.training.cochrane.org/handbook .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.1 (updated September 2020). Cochrane. 2020; Available from www.training.cochrane.org/handbook .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18. https://doi.org/10.1186/s13643-018-0914-3 .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):29. https://doi.org/10.1186/s13643-018-0768-8 .

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/10.1186/s13643-018-0695-8 .

Pollock M, Fernandes RM, Pieper D, Tricco AC, Gates M, Gates A, et al. Preferred reporting items for overviews of reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):335. https://doi.org/10.1186/s13643-019-1252-9 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0 .

Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5. https://doi.org/10.1016/j.jclinepi.2017.08.002 .

Article   PubMed   Google Scholar  

Gates M, Gates A, Guitard S, Pollock M, Hartling L. Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review. Syst Rev. 2020;9(1):254. https://doi.org/10.1186/s13643-020-01509-0 .

Covidence - systematic review software. Available at: https://www.covidence.org/ . Accessed 1 June 2021.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Borges do Nascimento IJ, et al. Novel Coronavirus Infection (COVID-19) in Humans: A Scoping Review and Meta-Analysis. J Clin Med. 2020;9(4):941.

Article   PubMed Central   Google Scholar  

Adhikari SP, Meng S, Wu YJ, Mao YP, Ye RX, Wang QZ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty. 2020;9(1):29. https://doi.org/10.1186/s40249-020-00646-x .

Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020;57:279–83. https://doi.org/10.1016/j.jcrc.2020.03.005 .

Li B, Yang J, Zhao F, Zhi L, Wang X, Liu L, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol. 2020;109(5):531–8. https://doi.org/10.1007/s00392-020-01626-9 .

Article   CAS   PubMed   Google Scholar  

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(6):577–83. https://doi.org/10.1002/jmv.25757 .

Lippi G, Lavie CJ, Sanchis-Gomar F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): evidence from a meta-analysis. Prog Cardiovasc Dis. 2020;63(3):390–1. https://doi.org/10.1016/j.pcad.2020.03.001 .

Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107–8. https://doi.org/10.1016/j.ejim.2020.03.014 .

Lippi G, Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–1. https://doi.org/10.1016/j.cca.2020.03.004 .

Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145–8. https://doi.org/10.1016/j.cca.2020.03.022 .

Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270 .

Lupia T, Scabini S, Mornese Pinna S, di Perri G, de Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: a new challenge. J Glob Antimicrob Resist. 2020;21:22–7. https://doi.org/10.1016/j.jgar.2020.02.021 .

Marasinghe, K.M., A systematic review investigating the effectiveness of face mask use in limiting the spread of COVID-19 among medically not diagnosed individuals: shedding light on current recommendations provided to individuals not medically diagnosed with COVID-19. Research Square. Preprint article. doi : https://doi.org/10.21203/rs.3.rs-16701/v1 . 2020 .

Mullins E, Evans D, Viner RM, O’Brien P, Morris E. Coronavirus in pregnancy and delivery: rapid review. Ultrasound Obstet Gynecol. 2020;55(5):586–92. https://doi.org/10.1002/uog.22014 .

Pang J, Wang MX, Ang IYH, Tan SHX, Lewis RF, Chen JIP, et al. Potential Rapid Diagnostics, Vaccine and Therapeutics for 2019 Novel coronavirus (2019-nCoV): a systematic review. J Clin Med. 2020;9(3):623.

Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, Villamizar-Peña R, Holguin-Rivera Y, Escalera-Antezana JP, et al. Clinical, laboratory and imaging features of COVID-19: a systematic review and meta-analysis. Travel Med Infect Dis. 2020;34:101623. https://doi.org/10.1016/j.tmaid.2020.101623 .

Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. https://doi.org/10.2214/AJR.20.23034 .

Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612–7. https://doi.org/10.1002/jmv.25735 .

Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis. 2020;94:91–5. https://doi.org/10.1016/j.ijid.2020.03.017 .

Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: challenges for fighting the storm. Eur J Clin Investig. 2020;50(3):e13209. https://doi.org/10.1111/eci.13209 .

Article   CAS   Google Scholar  

Hwang CS. Olfactory neuropathy in severe acute respiratory syndrome: report of a case. Acta Neurol Taiwanica. 2006;15(1):26–8.

Google Scholar  

Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, et al. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272–7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e .

Rajgor DD, Lee MH, Archuleta S, Bagdasarian N, Quek SC. The many estimates of the COVID-19 case fatality rate. Lancet Infect Dis. 2020;20(7):776–7. https://doi.org/10.1016/S1473-3099(20)30244-9 .

Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol. 2020;20(1):81. https://doi.org/10.1186/s12874-020-00972-6 .

Rombey T, Lochner V, Puljak L, Könsgen N, Mathes T, Pieper D. Epidemiology and reporting characteristics of non-Cochrane updates of systematic reviews: a cross-sectional study. Res Synth Methods. 2020;11(3):471–83. https://doi.org/10.1002/jrsm.1409 .

Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114–21. https://doi.org/10.1016/j.jclinepi.2019.08.010 .

Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13(1):224. https://doi.org/10.1186/s12916-015-0465-6 .

COVID-19 Rapid Reviews: Cochrane’s response so far. Available at: https://training.cochrane.org/resource/covid-19-rapid-reviews-cochrane-response-so-far . Accessed 1 June 2021.

Cochrane. Living systematic reviews. Available at: https://community.cochrane.org/review-production/production-resources/living-systematic-reviews . Accessed 1 June 2021.

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T. Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019;8(1):325. https://doi.org/10.1186/s13643-019-1248-5 .

Babic A, Poklepovic Pericic T, Pieper D, Puljak L. How to decide whether a systematic review is stable and not in need of updating: analysis of Cochrane reviews. Res Synth Methods. 2020;11(6):884–90. https://doi.org/10.1002/jrsm.1451 .

Lovato A, Rossettini G, de Filippis C. Sore throat in COVID-19: comment on “clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis”. J Med Virol. 2020;92(7):714–5. https://doi.org/10.1002/jmv.25815 .

Leung C. Comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1431–2. https://doi.org/10.1002/jmv.25912 .

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. Response to Char’s comment: comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1433. https://doi.org/10.1002/jmv.25924 .

Download references

Acknowledgments

We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from the Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Additional file 6 . We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.

This research received no external funding.

Author information

Authors and affiliations.

University Hospital and School of Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

Israel Júnior Borges do Nascimento & Milena Soriano Marcolino

Medical College of Wisconsin, Milwaukee, WI, USA

Israel Júnior Borges do Nascimento

Helene Fuld Health Trust National Institute for Evidence-based Practice in Nursing and Healthcare, College of Nursing, The Ohio State University, Columbus, OH, USA

Dónal P. O’Mathúna

School of Nursing, Psychotherapy and Community Health, Dublin City University, Dublin, Ireland

Department of Anesthesiology, Intensive Care and Pain Medicine, University of Münster, Münster, Germany

Thilo Caspar von Groote

Department of Sport and Health Science, Technische Universität München, Munich, Germany

Hebatullah Mohamed Abdulazeem

School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Callaghan, Australia

Ishanka Weerasekara

Department of Physiotherapy, Faculty of Allied Health Sciences, University of Peradeniya, Peradeniya, Sri Lanka

Cochrane Croatia, University of Split, School of Medicine, Split, Croatia

Ana Marusic, Irena Zakarija-Grkovic & Tina Poklepovic Pericic

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Cochrane Brazil, Evidence-Based Health Program, Universidade Federal de São Paulo, São Paulo, Brazil

Vinicius Tassoni Civile & Alvaro Nagib Atallah

Yorkville University, Fredericton, New Brunswick, Canada

Santino Filoso

Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada

Nicola Luigi Bragazzi

You can also search for this author in PubMed   Google Scholar

Contributions

IJBN conceived the research idea and worked as a project coordinator. DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP, ANA, SF, NLB and MSM were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. All authors revised the manuscript critically for the content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

Not required as data was based on published studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1..

Search strategies used in the study.

Additional file 2: Appendix 2.

Adjusted scoring of AMSTAR 2 used in this study for systematic reviews of studies that did not analyze interventions.

Additional file 3: Appendix 3.

List of excluded studies, with reasons.

Additional file 4: Appendix 4.

Table of overlapping studies, containing the list of primary studies included, their visual overlap in individual systematic reviews, and the number in how many reviews each primary study was included.

Additional file 5: Appendix 5.

A detailed explanation of AMSTAR scoring for each item in each review.

Additional file 6: Appendix 6.

List of members and affiliates of International Network of Coronavirus Disease 2019 (InterNetCOVID-19).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Borges do Nascimento, I.J., O’Mathúna, D.P., von Groote, T.C. et al. Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews. BMC Infect Dis 21 , 525 (2021). https://doi.org/10.1186/s12879-021-06214-4

Download citation

Received : 12 April 2020

Accepted : 19 May 2021

Published : 04 June 2021

DOI : https://doi.org/10.1186/s12879-021-06214-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus
• Evidence-based medicine
• Infectious diseases

BMC Infectious Diseases

ISSN: 1471-2334

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Browse All Articles
• Newsletter Sign-Up

• 12 Dec 2023
• Research & Ideas

• 15 Aug 2023
• Cold Call Podcast

• 10 Feb 2023

• 01 Feb 2023
• What Do You Think?

• 12 Oct 2022

• 04 Oct 2022

• 06 Sep 2022

• 21 Jul 2022

• 12 Jul 2022

• 09 Jun 2022

• 14 Apr 2022

• 14 Mar 2022

• 24 Feb 2022

• 10 Feb 2022

• 08 Feb 2022

• 20 Jan 2022

• 11 Jan 2022

• 02 Nov 2021

• 22 Oct 2021
• 19 Oct 2021

NIHR Oxford Biomedical Research Centre

Enabling translational research through partnership

COVID-19 Research Case Studies

Development of a covid-19 vaccine.

When the new SARS-CoV-2 virus emerged in China at the end of 2019, the BRC Vaccines theme was already working on human coronavirus vaccines and was in a unique position to respond rapidly to the pandemic. The Oxford team – led by Prof Sarah Gilbert, Prof Andrew Pollard, Prof Teresa Lambe, Dr Sandy Douglas and Prof Adrian Hill – identified a vaccine candidate and began testing in human volunteers in April 2020. In December 2020, the vaccine was found to be safe and effective, according to the peer-reviewed findings of the Phase III trial. The efficacy data were based on 11,636 volunteers across the United Kingdom and Brazil who took part in the trial.

In January 2021, the NHS launched a roll-out of the Oxford AstraZeneca vaccine, with patients at Oxford University Hospitals NHS Foundation Trust the very first to get the life-saving jab. The Oxford team has now launched the first study to assess the safety and immune responses in children and young adults of the vaccine.

The University of Oxford is working with the UK-based global biopharmaceutical company AstraZeneca for the development, large-scale manufacture and distribution of the COVID-19 vaccine.

Read more .

The RECOVERY Trial

The RECOVERY Trial is the world’s biggest clinical trial looking at whether existing therapies can help to treat people hospitalised with COVID-19. The trial was launched as an emergency response to the pandemic, with the first patient enrolled just nine days after the protocol was first drafted and the first result announced after 12 weeks.

The trial has found effective treatments: dexamethasone , a cheap and widely available steroid, which was found to cut the risk of death for COVID-19 patients on ventilators by a third; tocilizumab an anti-inflammatory treatment; and Ronapreve, a monoclocal antibody treatment developed by Regeneron. The discovery that dexamethasone is an effective treatment is estimated to have saved over a million lives worldwide, including 22,000 in the UK.

Since it began, RECOVERY (Randomised Evaluation of COVid-19 thERapY) has recruited more than 45,000 patients across 188 NHS hospitals. The trial is also being conducted in four other countries , with a view to finding treatments that are appropriate for a wide range of patients and healthcare systems.

The trial, which is jointly led by the Oxford BRC’s Co-them Lead for Clinical Informatics and Big Data, Professor Martin Landray, has received widespread praise for its efficient, streamlined design and clear results, based on reliable, large-scale data. It is being seen as a promising basis for future trials in other acute infections.

Watch Prof Landray talk on RECOVERY Trial

Covid-19 UK Infection Survey and antibody test

Senior NIHR Oxford BRC researcher Professor Sarah Walker was chosen to lead a major UK population-level infection study with the Office for National Statistics (ONS), which has proved important in helping scientists and the government manage the pandemic.

The aim of the Covid-19 infection study is to find out how many people from a representative cross-section of the UK population have COVID-19 at a given time and how many have had COVID-19 in the past. The ongoing survey has delivered important findings , which has been especially important as vaccinations are rolled out across the country.

The study, launched in April 2020 had a total of around 300,000 participants during its first 12 months. The study has helped to improve understanding around current rates of infection and how many people are likely to have developed antibodies to the virus through the use of an antibody test developed by researchers supported by the Oxford BRC. 

Professor Derrick Crook from the BRC’s Antimicrobial Resistance and Modernising Microbiology Theme co-led the development of this high-throughput test to detect COVID-19 antibodies from blood samples. The test, initially developed in Professor Gavin Screaton’s lab, is based on the commonly used enzyme-linked immunosorbent assay (ELISA) and uses a COVID-19-specific protein to capture COVID-19 antibodies that are present in the blood and so indicate which individuals are likely to have come into contact with the virus previously and developed an immune response.

Investigating the long-term impacts of COVID-19 on multiple organs

Prof Stefan Neubauer and Dr Betty Raman, from the BRC’s Imaging theme, led the C-MORE study, which found that a significant proportion of COVID-19 patients discharged from hospital reported symptoms of breathlessness, fatigue and depression and had limited exercise capacity several weeks after leaving hospital.

The researchers used advanced state-of-the art imaging to investigate the impact of the virus on patients’ organs, and to assess its effects on exercise tolerance, quality of life and mental health. They found that at two to three months after the onset of the disease, 64% of patients continued to experience breathlessness and 55% reported fatigue. MRI scans revealed abnormalities in the lungs of 60% of participants, in the hearts of 26%, in the livers of 10% and in the kidneys of 29% of patients.

The C-MORE study is a key part of the national PHOSP-COVID platform, led by the University of Leicester, which is investigating the long-term effects of COVID-19 on hospitalised patients. Around 10,000 patients are expected to take part, making it one of the largest comprehensive studies in the world to understand and improve the health of survivors after hospitalisation from COVID-19.

Creating a COVID-19 risk prediction model

Professor Julia Hippisley-Cox, of the University of Oxford’s Nuffield Department of Primary Care Health Sciences, led the development of a risk assessment tool to identify people at highest risk of serious illness from COVID-19.

The risk prediction model, called QCovid , was developed using anonymised data from more than 8 million adults. The researchers found that there are several health and personal factors which, when combined, could mean someone is at a higher risk from COVID-19. These include characteristics like age, ethnicity and BMI, as well as certain medical conditions and treatments.

NHS Digital has now used this model to develop a population risk assessment. The risk assessment predicts on a population basis whether registered patients with a combination of risk factors may be at more serious risk from COVID-19, enabling the government to prioritise them for vaccination, and provide appropriate advice and support.

The STOIC study: Treating COVID-19 with asthma inhalers

Researchers led by Professor Mona Bafadhel, from the University of Oxford’s Nuffield Department of Medicine, found that early treatment with a common asthma medication appears to significantly reduce the need for urgent care and hospitalisation in people with COVID-19. The STOIC study found that inhaled budesonide given to patients with COVID-19 within seven days of the onset of symptoms also reduced recovery time. Budesonide is a corticosteroid used in the long-term management of asthma and chronic obstructive pulmonary disease (COPD).

The trial was inspired by the fact that, in the early days of the pandemic, patients with chronic respiratory disease, who are often prescribed inhaled steroids, were significantly under-represented among those admitted to hospital with COVID-19.

Subscribe to the Oxford BRC Newsletter

Keep informed about the work of the Oxford BRC by subscribing to our Mailchimp e-newsletter. It is produced several times a year and delivers news and information about upcoming events straight to your inbox.

Subscribe Now

Oxford BRC on Social Media

We’d love to hear your feedback. Please contact us at [email protected]

• Open access
• Published: 06 April 2024

Adapting care provision in family practice during the COVID-19 pandemic: a qualitative study exploring the impact of primary care reforms in four Canadian regions

• Maria Mathews 1 ,
• Lindsay Hedden 2 ,
• Julia Lukewich 3 ,
• Emily Gard Marshall 4 ,
• Leslie Meredith 1 ,
• Lauren Moritz 4 ,
• Dana Ryan 1 ,
• Sarah Spencer 2 ,
• Judith B. Brown 1 ,
• Paul S. Gill 1 &
• Eric K. W. Wong 1  

BMC Primary Care volume  25 , Article number:  109 ( 2024 ) Cite this article

Metrics details

Over the past two decades, Canadian provinces and territories have introduced a series of primary care reforms in an attempt to improve access to and quality of primary care services, resulting in diverse organizational structures and practice models. We examine the impact of these reforms on family physicians’ (FPs) ability to adapt their roles during the COVID-19 pandemic, including the provision of routine primary care.

As part of a larger case study, we conducted semi-structured qualitative interviews with FPs in four Canadian regions: British Columbia, Newfoundland and Labrador, Nova Scotia, and Ontario. During the interviews, participants were asked about their personal and practice characteristics, the pandemic-related roles they performed over different stages of the pandemic, the facilitators and barriers they experienced in performing these roles, and potential roles FPs could have filled. Interviews were transcribed and a thematic analysis approach was applied to identify recurring themes in the data.

Sixty-eight FPs completed an interview across the four regions. Participants described five areas of primary care reform that impacted their ability to operate and provide care during the pandemic: funding models, electronic medical records (EMRs), integration with regional entities, interdisciplinary teams, and practice size. FPs in alternate funding models experienced fewer financial constraints than those in fee-for-service practices. EMR access enhanced FPs’ ability to deliver virtual care, integration with regional entities improved access to personal protective equipment and technological support, and team-based models facilitated the implementation of infection prevention and control protocols. Lastly, larger group practices had capacity to ensure adequate staffing and cover additional costs, allowing FPs more time to devote to patient care.

Conclusions

Recent primary care system reforms implemented in Canada enhanced FPs’ ability to adapt to the uncertain and evolving environment of providing primary care during the pandemic. Our study highlights the importance of ongoing primary care reforms to enhance pandemic preparedness and advocates for further expansion of these reforms.

Peer Review reports

In Canada, individual provinces have responsibility for the organization and delivery of health care. Primary care is the first point-of-contact in the healthcare system and involves the delivery of comprehensive, accessible, longitudinal, and coordinated patient-centered care [ 1 ]. Primary care is largely delivered by family physicians (FPs) who are independent business owners or sub-contractors in the health system and who have traditionally been paid by fee-for-service (FFS) through province-run universal health insurance programs. Over the past two decades, provinces and territories have introduced a series of primary care reforms on an incremental, voluntary basis, resulting in a variety of organizational structures and practice models, with substantial variability across provincial jurisdictions [ 2 , 3 , 4 ]. Implemented reforms include changes to payment (or billing) models such as alternate models to FFS remuneration (collectively called alternative payment plans [APP]), formal patient enrolment or rostering, and performance incentives. In addition, there were changes to practice models such as the creation of larger group practices, interdisciplinary teams, linked and/or integrated practices, and expansion of primary care professionals, as well as the implementation of information technology including electronic medical records (EMRs) [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ]. These reforms were framed as ways to improve access to and quality of primary care services.

Over the course of the COVID-19 pandemic, primary care practices in Canada had to adapt to a range of challenges that impacted their ability to safely deliver ongoing patient care. In March 2020, the volume of in-person primary care visits dropped as part of a larger closure of non-essential services amidst shortages of personal protective equipment (PPE) [ 10 ]. In addition, FPs had to manage patient care with restricted access to routine laboratory and diagnostic testing, specialist visits, or hospital-based services [ 11 ]. Out of necessity, FPs rapidly adopted virtual (primarily telephone) visits as public health insurance programs introduced or modified virtual care fee codes [ 10 , 12 ]. Primary care practices implemented infection prevention and control (IPAC) procedures that allowed for the safe delivery of care by limiting the number of providers in office and altering patient flow to provide care when in-person was truly needed [ 13 ]. Over subsequent waves of COVID-19 infections (and closures of non-essential services), family practices provided a mix of virtual and in-person visits.

In this paper, we examine the impact of underlying primary care reforms on FPs’ ability to adapt to pandemic roles, including the provision of routine primary care. This study is part of a series of papers from a larger project exploring the pandemic from the perspective of FPs in Canada. The larger project is a multiple case study with cases consisting of: the Ontario (ON) Health West Region, the Vancouver Coastal health region in British Columbia (BC), the Eastern Health region in Newfoundland and Labrador (NL), and the province of Nova Scotia (NS). These regions, while pragmatically chosen as the locations of our pre-existing research team, vary in the organization and funding of primary care and reflect a variety of primary care reforms implemented across Canada [ 14 ].

The protocol for the larger case study has been published elsewhere [ 14 ]. Using a case study approach [ 15 ], we conducted semi-structured qualitative interviews with FPs from October 2020 to June 2021. We invited FPs representing a wide range of personal characteristics (i.e., maximum variation sampling [ 16 ]) to participate in an interview until we had reached data saturation (i.e., sufficient data to allow for rigorous analysis and interpretation) [ 16 , 17 ] based on post-interview debriefings and review of field notes during the data collection phase of the project. Throughout our recruitment, we considered characteristics such as having hospital and/or academic affiliations, genders, varied practice and funding models (e.g., FFS, APP), different community sizes, and different primary care practice settings (e.g., family practice clinics, long-term care facilities, and hospitals). To be eligible, FPs had to be licensed to practice independently at the time of the interview. We excluded FPs on temporary licenses or who held exclusively academic, research, or administrative roles. In each region, research assistants consulted faculty practice, team, and privileging lists, as well as the physician search portals of provincial medical regulators to identify potential participants. Research assistants posted recruitment notices in medical organizations’ newsletters, social media posts, and used snowball sampling (where permitted by local research ethics boards).

In each interview, we asked FPs to describe their personal and practice characteristics including their practice setting, primary funding model, and whether they work in an interdisciplinary team. With this context, we then asked them about the pandemic-related roles they performed over different stages of the pandemic and the facilitators and barriers to performing these roles. In the interviews, FPs organically discussed the impact of their organizational and funding models on their ability to adapt to the changing circumstances of providing patient care during the pandemic. Our questions and FP responses were based on the response to the pandemic at the time that we held the interviews (e.g., availability of vaccines, closure of schools, etc.). We carried out interviews using Zoom (Zoom Video Communications Inc.) or telephone as per the preference of the participant. A research assistant transcribed the recording of each interview.

Working independently, two or more members of the research team from each region read two to three transcripts (along with field notes from the interview) to identify key words and create codes, which were then organized into a preliminary coding scheme following discussion, in line with thematic analysis. The teams from each case compared the coding of four transcripts (one from each region) and, through discussion, developed a uniform coding template with code labels and descriptions that applied across all cases. We discussed disagreements in coding and coding descriptions until we reached an acceptable compromise. We used the unified coding template to code all transcripts and field notes, with the assistance of the NVivo 12 (QSR International) software. We used counts and proportions to describe participant characteristics. This paper presents findings from the codes related to primary care organization, funding, and integration with other health system organizations. We have also described the province and funding model (FFS, APP, or mixed) of each quoted participant.

In each region, research ethics boards approved the study. We obtained informed consent from participants before we scheduled and conducted interviews. We password protected files and securely stored recordings. We used participant identification numbers to help conceal the identity of individual participants.

Positionality

We took a pragmatic approach in our research and made efforts to improve the quality of our data collection and analyses [ 16 , 17 , 18 ]. These efforts included pre-testing the interview questions with the FP members of our research team, documenting steps in the collection and analysis of the data, having experienced research assistants conduct interviews, and summarising responses back to participants to confirm meaning. We provided description of the context and meaning of quotes and looked for examples of data that presented opposing experiences or views. Our research team included FPs and public health officials, as well as researchers with extensive knowledge of primary care reforms in Canada, allowing us to draw on prior expert knowledge in the development of our interview guide, the development of the uniform coding template, and the interpretation of our results [ 15 ].

Among the 68 participants in the study, 41 (60.3%) were women, 49 (73.5%) had hospital privileges or affiliations, and 44 (64.7%) practiced in urban settings (Table  1 ).

FPs identified reforms in five key areas that directly impacted their practices’ ability to operate and provide ongoing care during the pandemic: funding model, EMRs, integration with regional entities, interdisciplinary teams, and number of FPs in a group practice. All participants described their funding or practice model and referred to at least one of these key areas in their responses to interview questions.

Table  2 summarizes the state of reforms for each area of reform in each study site prior to the COVID-19 pandemic. As shown in Table  2 , overall, FFS remained the predominant form of payment in BC and NL. Access to EMRs was high in all study provinces, except among FPs in NL. There was limited integration of FP practices with regional entities (such as a regional health authority, network, or local hospital), except in NS. With the exception of ON, team-based models of care were limited in BC, NS, and NL, where FPs predominately practiced in small groups of one or two physicians. In ON, team-based care was more widely integrated; teams were encouraged in Family Health Teams and reforms were conditional on forming co-located or virtual group practices of three or more physicians.

Funding model

Pandemic-related closures resulted in a sudden decrease to the volume of in-person patient visits and a rapid transition to virtual care. Participants noted that clinics funded by APP models were better able to financially navigate the sudden changes related to virtual care than FFS practices: “Patient volumes went down. Fortunately, because we’re capitation, the income stayed relatively the same. … if I was a fee-for-service physician, yeah, I would have ate my shirt” [ON10 APP]. The introduction of previously limited fee codes for virtual care were vital for FFS physicians: “having that ability to bill for a virtual visit, just from a practical financial standpoint, that was at least something, right? Because for a purely fee-for-service physician, if you didn’t have that, then you have zero, you have nothing” [ON16 FFS]. The introduction of the virtual fee codes had less impact on FPs who were funded through APPs: “we quickly adapted, as I mentioned because of our population-based funding model, an encounter is an encounter, it doesn’t matter if it’s by phone or a refill straight to the pharmacy or a discussion with a specialist colleague” [BC7 APP]. However, while provincial insurance plans quickly expanded the availability of virtual care billing codes, billing criteria in some provinces made it difficult for FFS FPs to generate sufficient revenue:

We were going on the assumption that in each day we can get paid for a certain number of phone calls to patients to discuss more in-depth topics. But we get paid – we get $10 per phone call…You know, a regular in-office visit is $37. To go from getting paid a maximum capped per day of $10 per patient, … from a busy fee-for-service [practice], is pretty nerve-racking . [NL7 FFS]

Moreover, participants noted that FFS remuneration for a virtual appointment with patients who had complex needs did not fully recognize the time required to provide care. There was considerable variation, even within the same region, in the adequacy of virtual care fees for complex care. While generally satisfied with the new fee codes, the participant explained:

I was very pleased with how responsive the [provincial health insurance program] was in rolling out the new billing codes. …because my complex patients that take half an hour, I can bill a primary care HIV code, which is much higher paid than a primary care basic code. [BC2 FFS]

However, the participant went on to state that slower changes to fee codes for addictions care resulted in uncompensated services: “We scaled-up pandemic withdrawal management, like the safe supply prescribing. … and there are no billing codes to match it, so we did it all for free” [BC2 FFS].

Participants also described the greater operating costs associated with providing care during the pandemic, which none of the funding models directly addressed. Of note, physicians paid through APP were better able to manage due to income stability. As summarized by a participant, “there needs to be clear provisions in a physician services agreement for pandemic care and for pandemic periods” [ON10 APP]. Additional costs stemmed from the IPAC requirements, such as the screening of patients: “I think the government needs to, or needed to, recognize that the costs of [providing care during a pandemic] are certainly much greater… when I look at the amount of time my staff spend on the phones screening patients” [ON13 APP]. They also needed to purchase PPE and cleaning supplies: “we had to purchase additional PPE, a whole bunch of different cleaning stuff and then we didn’t hire any additional staff, we just took on a lot of the tasks ourselves. … So, a lot more, certainly, work demands, which led to inefficiencies in seeing in-person people for sure” [ON10 APP]. Participants noted that they personally took on the additional cleaning requirements, which reduced their capacity to provide care because they could not afford to hire additional staff to perform these tasks: “at a time when we were earning less, we couldn’t afford to hire another person to come in just to do cleaning” [NL6 FFS].

The rapid shift to virtual care was facilitated by the existing use of EMRs: “so 70–80% of family docs now have access to the EMR” [NL8 APP] and “we already had remote access to our EMRs” [BC4 mixed]. However, participants noted that the move to virtual care required additional investment in infrastructure:

There’s a cost involved with [virtual care] … to be able to provide virtual care with messaging and video and all that. There’s a supplemental cost. It works out to a few thousand dollars a year. I’m not sure as physicians whether we should say, ‘Well, that’s just the cost of business and we’re going to have to [absorb] that’ or whether there’s a role for the government… [NS5 APP] .

Practices also incurred costs facilitating EMR access to all clinic staff who were working from home during the pandemic: “All my nurses, because they work from home, we have to set up an EMR for them at home. We have to make extra payment to the EMR people to give them access from home” [ON18 APP]. Without these additional changes to the EMR, staff would not have been able to provide virtual care during pandemic restrictions.

Integration with regional entities

Participants described how integration (or, conversely, lack of integration) with a regional health authority, network, or local hospitals affected access to pandemic-related communication, PPE, and information and technology supports. For example, a participant whose primary care practice was one of the sites of the regional health authority commented that “we were in good communication with the [regional health authority] and they were very supportive” [NL3 APP]. In contrast, a participant without affiliation to the regional health authority recalled that for many months into the pandemic, “there was no formal communication between … the regional health authority and the community-based physicians” [NL11 FFS].

Participants who were integrated with the regional health authorities had better access to PPE and did not have to bear this additional cost (“I know there was a lot of discontent from other physicians about lack of PPE; I’m fortunate our clinic is funded by [the regional health authority], so we did have access to it … and it wasn’t an issue of having to pay for it” [NL4 APP]) unlike unaffiliated practices (“they didn’t provide personal protective equipment for us at all until this fall, maybe November, October, when they started giving it to us. So, we had nothing. And you couldn’t order it if you tried” [NS15 APP]).

Participants who belonged to a regional network also had access to computing support to help transition to online meetings (“[the network] rolled out like, an IT [information technology] support… It was like a virtual help desk …they have an IT guy specific for trying to get a Zoom up and running” [BC4 mixed]) and EMR upgrades to support virtual care:

we were quite fortunate as a [regional health authority] site to, as opposed to an independent practice/fee-for-service practice … to have a relatively ready supply of PPE and then the [EMR], which we already had in place, and we were privy to the changes that they brought in [NL2 APP].

In most provinces, COVID-related duties were first offered to providers who had pre-existing links with the regional entity that organized the assessment or vaccination clinics. In some provinces, performing pandemic-related duties was a condition of receiving income support and consequently, as one participant explained, FFS FPs in the region (who generally are not integrated with the regional health authority) were unlikely to benefit from the income supplement program:

the income supplementation seemed to be helpful, but honestly, it was not good for most of us fee-for-service family physicians. We only got it if we did COVID work and this is where I feel that [our] regional health authority did not protect us very well … I know other [regional health authorities in the province] protected their family doctors by getting them to do COVID work …so that they would get the [income] supplementation [NL6 FFS].

The participant noted that other regional health authorities in the province prioritized FFS physicians for COVID-related work to ensure that their incomes would be protected.

Interdisciplinary team

Interdisciplinary practices were able to adapt more easily to IPAC guidelines, with specific team members taking the lead on updating and implementing processes for the practice: “it’s a collaborative clinic, which has an amazing family practice nurse, who’s well, well up on the latest [regional] policies and she’s very good to guide us through what’s required for personal protection equipment and sanitizing and those kinds of things” [NS11 APP]. In contrast, practices without teams had greater difficulty taking on these additional roles: “those practices where they really are fee-for-service, they don’t have the team, they don’t have the ability to respond in the way that we [a team-based practice] have” [ON3 APP].

While newer models of care include team-based care, non-physician health professionals are often employees of the local hospital or regional health authority assigned to work in primary care practices, rather than employees of the practice. As a result, a participant noted that individual practices had little control over the redeployment of members of their teams, leaving the practice short-staffed: “most of our nursing staff were redeployed and [our clinic] is really a nursing-run clinic, so that had a major impact” [BC5 APP].

Number of FPs in a group practice

Participants also highlighted the benefits of belonging to a group practice with a larger number of FPs, which could afford to hire additional staff to take on some of the new roles required by IPAC protocols: “I’m in a large group, we have the availability of hiring on extra staff to screen everybody on the way in” [NS3 FFS]. Another participant noted that clinicians could devote more time to providing patient care while others took on leadership roles on behalf of the entire group:

I felt lucky that we had a scale, enough of a size of our clinics that organizationally we had some people in place in leadership roles to take on doing that on our behalf … And the rest of us lowly clinicians could just keep going seeing patients while they were figuring that out for us. If you were a 1 or 2-person GP [general practitioner] practice with a nurse and an office in a building in wherever, you had nobody to do that, right? [NS1 APP]

A participant in BC noted that the large number of FPs in their group practice meant they were better able to afford the extra costs associated with IPAC: “We’re a bit more fortunate because we are a group of nine practices now that collectively contribute to overhead. There was a pool of money that we were able to utilize” [BC7 APP].

Interviews with FPs during the COVID-19 pandemic suggest that practices that had adopted primary care reforms introduced in the past 20 years (namely APP models, EMRs, integration with regional entities, interdisciplinary teams, and increased number of FPs in a group practice; Table  2 ) were better able to implement pandemic-related adaptations in care. These reforms include both funding and practice model changes which are often linked (i.e., funding model changes that are tied to or facilitate changes in practice model). While none of the funding models fully covered the costs of providing care during the pandemic, belonging to an APP, large practice group, and/or an interdisciplinary team provided both financial and human resource capacity to absorb additional pandemic-related roles such as leadership, planning, and implementing IPAC protocols. Practices with prospective payment models (i.e., salaried, capitation, or blended capitation models) did not experience the same degree of cash flow restrictions as FFS practices. Across all case study regions, traditional solo or small group FFS practices had greater difficulty adapting the provision of routine primary care to the circumstances presented by the COVID-19 pandemic. Similar findings were reported among FFS primary care providers in other regions of Canada, the United States, and Australia [ 32 , 33 , 34 , 35 , 36 , 37 ]. An international study examining the use of virtual care in the first year of the pandemic found that the three countries (Canada [specifically the province of Ontario], Sweden, and the United Kingdom) with the highest rates of virtual primary care use in the pandemic period used capitation funding models; however, the study noted that funding model alone did not explain the higher utilization of virtual care [ 38 ]

Our findings highlight the value of greater integration of primary care practices with regional health entities [ 33 ]. Prior to the pandemic, practice networks in Ontario provided a critical mass to enable quality improvement, after-hours access, and economies of scales [ 5 ]. In addition to practical supports such as PPE, integration of family practices with regional entities facilitated planning and coordination and bi-directional communication between FPs and decision-makers in the four regions in our study [ 12 , 39 , 40 ], echoing findings from Alberta [ 33 ] and Australia [ 37 ].

The impact of the pandemic on primary care practices demonstrates the need for continued primary care reforms, including the expansion of alternate payment approaches, supports for virtual care, interdisciplinary teams, integration of primary care practices into regional entities, and greater numbers of FPs in a group practice. These reforms align with practice models favoured by FPs, especially recent graduates, [ 12 , 32 , 34 , 41 , 42 , 43 ], are touted to reduce FP burnout [ 32 , 44 ], and are reflected in new reforms in three of the four study regions. In 2023, BC announced increases in physician payment linked to time and patient complexity, and patient rostering [ 26 ], while NS’s recent reforms promote blended capitation payment, EMR use, minimum team size, and commitment to comprehensive primary care [ 26 , 45 ]. The Health Accord NL [ 46 ] outlined efforts to integrate “collaborative care models” (now referred to as family care teams clinics) through interdisciplinary team-based care, rostering of patients to salaried physicians and/or nurse practitioners, and blended capitation funding models.

Limitations

Our study is based on interviews conducted with the primary purpose of identifying FP pandemic roles and their supports and barriers. The interview guide did not explicitly ask about specific reforms in each region; nonetheless, the impact of primary healthcare reforms in each study site were readily apparent in the data as participants shared their stories. Moreover, unlike most FPs in Canada, the majority of FPs in the study were paid by APPs and had hospital affiliations; thus, our findings may under-represent the experiences of FFS and unaffiliated FPs. Primary care reforms also vary by province, so our findings may not represent the experiences of FPs outside our four case study provinces. Our study is based on self-reported data and may have been impacted by social desirability and recall bias [ 47 , 48 ]. For example, participants may have been hesitant to discuss financial difficulties or appear to consider financial motives above community and patient needs.

Primary care system reforms implemented in Canada over the last 20 years, namely APP models, EMRs, integration with regional health care entities, team-based models of care, and large group practices, enhanced FPs’ ability to adapt to the uncertain and evolving environment of providing primary care during the COVID-19 pandemic. Our study findings strengthen calls to expand these reforms within Canada.

Data availability

The datasets analysed for this study are not publicly available due to the need to maintain participant confidentiality; however, a portion of these data may be available from the corresponding author on reasonable request.

Abbreviations

• Family physician

Fee-for-service

Alternative payment plan

Electronic medical record

Personal protective equipment

Infection prevention and control

British Columbia

Newfoundland and Labrador

Nova Scotia

National Academies of Sciences, Engineering, and Medicine. Implementing high-quality primary care: rebuilding the foundation of health care. The National Academies Press. 2021. https://www.nap.edu/read/25983 . Accessed 08 Sept 23.

Marchildon G, Hutchison B. Primary care in Ontario, Canada: new proposals after 15 years of reform. Health Policy. 2016;120:732–8. https://doi.org/10.1016/j.healthpol.2016.04.010 .

Article   PubMed   Google Scholar  

McKay M, Lavergne MR, Prince LA, Le M, Grudniewicz A, Blackie D, et al. Government policies targeting primary care physician practice from 1998–2018 in three Canadian provinces: a jurisdictional scan. Health Policy. 2022;19:e431–11.

Google Scholar  

Tan S, Farmer J, Roerig M, Allin S. Primary care governance and financing: models and approaches. Rapid Review (No. 37). North American Observatory on Health Systems and Policies; 2023.

Hutchison B, Glazier R. Ontario’s primary care reforms have transformed the local care landscape, but a plan is needed for ongoing improvement. Health Aff. 2013;32(4):695–703. https://doi.org/10.1377/hlthaff.2012.1087 .

Article   Google Scholar  

Aggarwal M, Williams AP. Tinkering at the margins: evaluating the pace and direction of primary care reforms in Ontario, Canada. BMC Fam Pract. 2019;20:128:e1–14. https://doi.org/10.1186/s12875-019-1014-8 .

Blomqvist A, Wyonch R. Health teams and primary care reform in Ontario: staying the course (commentary No. 551). CD Howe Institute; 2019.

Martin-Misener R, Wong ST, Johnston S, Blackman S, Scott C, Hogg W, et al. Regional variation in primary care improvement strategies and policy: case studies that consider qualitative contextual data for performance measurement in three Canadian provinces. BMJ Open. 2019;9e029622:e1–11. https://doi.org/10.1136/bmjopen-2019-029622 .

Hutchison B, Levesques J-F, Strumpf E, Coyle N. Primary health care in Canada: systems in motion. Milbank Q. 2011;89(2):256–88.

Article   PubMed   PubMed Central   Google Scholar  

Glazier RH, Green ME, Wu FC, Frymire E, Kopp A, Kiran T. Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario. CMAJ. 2021;193(6):e200–10. https://doi.org/10.1503/cmaj.202303 .

Mathews M, Meredith L, Ryan D, Hedden L, Lukewich J, Marshall EG, Buote R, et al. The roles of family physicians during a pandemic. Healthc Manage Forum. 2023;36(1):30–5. https://doi.org/10.1177/08404704221112311 .

Hedden L, Spencer S, Mathews M, Marshall EG, Lukewich J, Asghari S, et al. There’s nothing like a good crisis for innovation: a qualitative study of family physicians’ experiences with virtual care during the COVID-19 pandemic. BMC Health Serv Res. 2023;23(338):e1–10. https://doi.org/10.1186/s12913-023-09256-3 .

Mathews M, Meredith L, Ryan D et al. Practice and system-based interventions to reduce the risk of COVID 19 transmission in primary care settings. Healthc Policy. 2023;19(2):63–78. https://doi.org/10.12927/hcpol.2023.27232 .

Mathews M, Spencer S, Hedden H, Marshall EG, Lukewich J, Buote R, et al. Development of a primary care pandemic plan informed by in-depth policy analysis and qualitative case studies with family physicians across Canada during COVID-19: a qualitative case study protocol. BMJ Open. 2021;11:e048209. https://doi.org/10.1136/bmjopen-2020-048209 .

Yin RK. Case study research design and methods. 5th ed. Thousand Oaks: SAGE Publications Ltd.; 2014.

Creswell JW. Research design - qualitative, quantitative and mixed methods approaches. 4th ed. Thousand Oaks: SAGE Publications Ltd.; 2014.

Berg BL. Qualitative research methods for the social sciences. 2nd ed. Boston: Allyn and Bacon; 1995.

Guest G, MacQueen KM, Namey EE. Applied thematic analysis. Los Angeles: SAGE Publications Ltd.; 2012.

Book   Google Scholar  

Canadian Institute for Health Information. National Physician Database – payments data, 2019–2020. CIHI. 2021. https://secure.cihi.ca/estore/productSeries.htm?pc=PCC476 . Accessed Nov 23, 2023.

Marchildon GP, Allin S, Merkur S. Canada: health system review. Health Syst Transit. 2020;22(3):1–194.

PubMed   Google Scholar  

Lukey A, Maiwald K, Wankah P, Breton M, Hirschkorn P, Oelke ND. Implementing team-based innovation in primary health care in British Columbia. HRO-ORS. 2022;10(1). https://doi.org/10.13162/hro-ors.v10i1.4696 .

Misfeldt R, Suter E, Mallinson S, Boakye O, Wong S, Nasmith L. Exploring context and the factors shaping team-based primary healthcare policies in three Canadian provinces: a comparative analysis. Healthc Policy. 2017;13(1):74–93. https://doi.org/10.12927/hcpol.2017.25190 .

British Columbia Ministry of Health. Supportive policy direction – Patient Medical Homes. Government of British Columbia; 2017.

Mazowita G, Cavers W. Reviving full-service family practice in British Columbia. Issue Brief Commonw Fund. 2011;19:1–20. PMID:22616139.

Ontario Ministry of Health and Ministry of Long-Term Care. Primary care payment models in Ontario. Government of Ontario. 2020. https://www.health.gov.on.ca/en/pro/programs/pcpm/ . Accessed Nov 23, 2023.

Martin-Misener R, McNab J, Skertis IS, Edwards L. Collaborative practice in health system change: the Nova Scotia experience with the strengthening primary care initiative. Nurs Leadersh. 2004;17(2):33–46. https://doi.org/10.12927/cjnl.2004.16345 .

College of Family Physicians of Canada. The Patient’s Medical Home. CFPC. 2013. https://www.cfpc.ca/en/policy-innovation/health-policy-goverment-relations/the-patient-s-medical-home . Accessed Nov 23, 2023.

Collier R. National Physician Survey: EMR use at 75%. CMAJ. 2015;187(1):e18. https://doi.org/10.1503/cmaj.109-4957 .

Zeidenberg J. Newfoundland and Labrador on verge of province-wide EHR. Canadian Healthcare Technology. 2015. https://www.canhealth.com/2016/11/02/newfoundland-and-labrador-on-the-verge-of-a-province-wide-ehr/ . Accessed Nov 23, 2023.

Newfoundland and Labrador Medical Association, Memorial University Faculty of Medicine, Newfoundland Labrador College of Family Physicians. 2018. Family medicine in Newfoundland and Labrador: a ten year vision. CFPC. 2018. https://nl.cfpc.ca/wp-content/uploads/2018/06/Family_Medicine_in_NL_-_A_Ten_Year_Vision.pdf . Accessed Nov 23, 2023.

Peckham A, Ho J, Marchildon G. Policy innovations in primary care across Canada. Rapid Review (No. 1). North American Observatory on Health Systems and Policies. 2018.

Ivers N, Newbery S, Eissa A, Bayoumi I, Kiran T, Pinto A, et al. Brief on primary care part 3: lessons learned for strengthened primary care in the next phase of the COVID-19 pandemic. Science Briefs of the Ontario COVID-19. Sci Advisory Table 2022;3(69). https://doi.org/10.47326/ocsat.2022.03.69.1.0 .

Leslie M, Fadaak R, Pinto N, Davies J, Green L, Seidel J, et al. A shock test to primary care integration: COVID-19 lessons from Alberta. Healthc Policy. 2021;17(2):38–53. https://doi.org/10.12927/hcpol.2021.26658 .

Glauser W. Pandemic amplifies calls for alternative payment models. CMAJ. 2020;192(23):e638–9. https://doi.org/10.1503/cmaj.1095874 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Horstman C, Lewis C, Abrams MK. Strengthening primary health care: the importance of payment reform. Commonwealth Fund. 2021. https://www.commonwealthfund.org/blog/2021/strengthening-primary-health-care-importance-payment-reform . Accessed Nov 23, 2023.

The Larry A, Green Centre. Quick COVID-19 primary care survey. Series 30. Oregon Rural Practice-Based Research Network. 2021. https://www.ohsu.edu/sites/default/files/2021-08/C19%20Series%2030%20Oregon%20x%20National%20Data.pdf . Accessed Nov 23, 2023.

Duckett S. What should primary care look like after the COVID-19 pandemic. Aust J Prim Care. 2020;26:207–11. https://doi.org/10.1071/PY20095 .

Tu K, Kristiansson SR, Gronsbell J, de Lusignan S, Flottorp S, Goh LH, et al. Changes in primary care visits arising from the COVID-19 pandemic: an international comparative study by the International Consortium of Primary Care Big Data Researchers (INTRePID). BMJ Open. 2022;12:e059130. https://doi.org/10.1136/bmjopen-2021-059130 .

Young G, Mathews M, Hedden L, Lukewich J, Marshall EG, Gill P, et al. Swamped with information: a qualitative study of family physicians’ experiences of managing and applying pandemic-related information. Front Commun. 2023;8:1186678. https://doi.org/10.3389/fcomm.2023.1186678 .

Mathews M, Ryan D, Hedden L, Lukewich J, Marshall EG, Brown JB, et al. Family physician leadership during the COVID19 pandemic: roles, functions and key supports. Leadersh Health Serv. 2022;35(4):559–75. https://doi.org/10.1108/LHS-03-2022-0030 .

Lavergne MR, Scott I, Mitra G, Snadden D, Blackie D, Goldsmith LJ, et al. Regional differences in where and how family medicine residents intend to practice: a cross-sectional survey analysis. CMAJ Open. 2019;7:e124–30. https://doi.org/10.9778/cmajo.20180152 .

Gisler LB, Bachofner M, Moser-Bucher CN, Scherz N, Streit S. From practice employee to (co-)owner: young GPs predict their future careers: a cross-sectional survey. BMC Fam Pract. 2017;18(1):12. https://doi.org/10.1186/s12875-017-0591-7 .

Brcic V, McGregor M, Kaczorowski J, Dharamsi S, Verma S. Practice and payment preferences of newly practicing family physicians in British Columbia. Can Fam Physician. 2012;58:e275–81.

PubMed   PubMed Central   Google Scholar  

Mathews M, Idrees S, Ryan D, Hedden L, Lukewich J, Marshall EG et al. System-based interventions to address physician burnout: a qualitative study of Canadian family physicians’ experiences during the COVID-19 pandemic. IJHPM [submitted June 15, 2023].

Primary Health Care, Nova Scotia Health. Current state assessment of the primary health care system in Nova Scotia: the primary health care 2020-21 system performance report. Primary Health Care, Nova Scotia Health. 2022. https://www.nshealth.ca/sites/nshealth.ca/files/nsh_phc_system_performance_update_2022_executive_summary_-_final.pdf . Accessed Nov 23, 2023.

Health Accord NL. Our province. Our health. Our future. A 10-year health transformation: the report. 2022. https://healthaccordnl.ca/final-reports/ . Accessed Nov 23, 2023.

Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol. 1990;43(1):87–91. https://doi.org/10.1016/0895-4356(90)90060-3 .

Article   CAS   PubMed   Google Scholar  

Bergen N, Labonté R. Everything is perfect, and we have no problems: detecting and limiting social desirability bias in qualitative research. Qual Health Res. 2020;30(5):783–92. https://doi.org/10.1177/1049732319889354 .

Download references

Acknowledgements

Not applicable.

This study was funded the Canadian Institutes for Health Research (VR41 72756). The funding agency had no role in the research process.

Author information

Authors and affiliations.

Department of Family Medicine, Schulich School of Medicine & Dentistry, Western University, 1151 Richmond Street, London, ON, N6A 5C1, Canada

Maria Mathews, Leslie Meredith, Dana Ryan, Judith B. Brown, Paul S. Gill & Eric K. W. Wong

Faculty of Health Sciences, Simon Fraser University, 8888 University Drive, Burnaby, BC, V5A 1S6, Canada

Lindsay Hedden & Sarah Spencer

Faculty of Nursing, Memorial University, 300 Prince Philip Drive, St. John’s, NL, A1B 3V6, Canada

Julia Lukewich

Department of Family Medicine Primary Care Research Unit, Dalhousie University, 1465 Brenton Street, Suite 402, Halifax, NS, B3J 3T4, Canada

Emily Gard Marshall & Lauren Moritz

You can also search for this author in PubMed   Google Scholar

Contributions

Writing–Original Draft: MM; Writing–Review and Editing: MM, LH, JL, EGM, LMe, LMo, DR, SS, JBB, PSG, EKWW; Methodology: MM, LH, JL, EGM, LMe, LMo, DR, SS; Supervision: MM, LH, JL, EGM; Project Administration: MM, LMe, LMo; LH, SS, EGM, JL, DR; Funding Acquisition: MM, LH, EGM, JL. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Maria Mathews .

Ethics declarations

Ethics approval and consent to participate.

We obtained approval from the research ethics boards at Simon Fraser University and the University of British Columbia (through the harmonised research ethics platform provided by Research Ethics British Columbia), the Health Research Ethics Board of Newfoundland and Labrador, Nova Scotia Health, and Western University. Participants provided informed consent before interviews were scheduled.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mathews, M., Hedden, L., Lukewich, J. et al. Adapting care provision in family practice during the COVID-19 pandemic: a qualitative study exploring the impact of primary care reforms in four Canadian regions. BMC Prim. Care 25 , 109 (2024). https://doi.org/10.1186/s12875-024-02356-x

Download citation

Received : 25 November 2023

Accepted : 28 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s12875-024-02356-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Primary care
• Primary care reforms
• Pandemic response
• Policy planning
• Qualitative research

BMC Primary Care

ISSN: 2731-4553

• General enquiries: [email protected]

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Online education and its effect on teachers during COVID-19—A case study from India

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Area of Humanities and Social Sciences, Indian Institute of Management Indore, Indore, Madhya Pradesh, India

• Surbhi Dayal

• Published: March 2, 2023
• https://doi.org/10.1371/journal.pone.0282287
• Reader Comments

COVID pandemic resulted in an initially temporary and then long term closure of educational institutions, creating a need for adapting to online and remote learning. The transition to online education platforms presented unprecedented challenges for the teachers. The aim of this research was to investigate the effects of the transition to online education on teachers’ wellbeing in India.

The research was conducted on 1812 teachers working in schools, colleges, and coaching institutions from six different Indian states. Quantitative and qualitative data was collected via online survey and telephone interviews.

The results show that COVID pandemic exacerbated the existing widespread inequality in access to internet connectivity, smart devices, and teacher training required for an effective transition to an online mode of education. Teachers nonetheless adapted quickly to online teaching with the help of institutional training as well as self-learning tools. However, respondents expressed dissatisfaction with the effectiveness of online teaching and assessment methods, and exhibited a strong desire to return to traditional modes of learning. 82% respondents reported physical issues like neck pain, back pain, headache, and eyestrain. Additionally, 92% respondents faced mental issues like stress, anxiety, and loneliness due to online teaching.

As the effectiveness of online learning perforce taps on the existing infrastructure, not only has it widened the learning gap between the rich and the poor, it has also compromised the quality of education being imparted in general. Teachers faced increased physical and mental health issues due to long working hours and uncertainty associated with COVID lockdowns. There is a need to develop a sound strategy to address the gaps in access to digital learning and teachers’ training to improve both the quality of education and the mental health of teachers.

Citation: Dayal S (2023) Online education and its effect on teachers during COVID-19—A case study from India. PLoS ONE 18(3): e0282287. https://doi.org/10.1371/journal.pone.0282287

Editor: Lütfullah Türkmen, Usak University College of Education, TURKEY

Received: November 13, 2021; Accepted: January 27, 2023; Published: March 2, 2023

Copyright: © 2023 Surbhi Dayal. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data apart from manuscript has been submitted as supporting information .

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

As of November 4, 2021, the spread of novel coronavirus had reached 219 countries and territories of the world, infecting a total of 248 million people and resulting in five million deaths [ 1 ]. In March 2020, several countries including India declared a mandatory lockdown, resulting in the temporary closure of many institutions, not least educational ones. Since then, various restrictions and strategies have been implemented to counter the spread of the virus. These include wearing masks, washing hands frequently, maintaining social and physical distance, and avoiding public gatherings. The pandemic has greatly disrupted all aspects of human life and forced new ways of functioning, notably in work and education, much of which has been restricted to the household environment. The closure for over a year of many schools and colleges across the world has shaken the foundations of the traditional structures of education. Due to widespread restrictions, employees have been forced to carve out working spaces in the family home; likewise, students and teachers have been compelled to bring classes into homes [ 2 ]. Nearly 1.6 billion learners in more than 190 countries have been physically out of school due to the pandemic. In total, 94 percent of the world’s student population has been affected by school closures, and up to 99 percent of this student population come from low-to middle-income countries [ 3 ].

According to the World Economic Forum, the pandemic has changed how people receive and impart education [ 4 ]. Physical interaction between students and teachers in traditional classrooms has been replaced by exchanges on digital learning platforms, such as online teaching and virtual education systems, characterized by an absence of face-to-face connection [ 5 ]. Online education has thus emerged as a viable option for education from preschool to university level, and governments have used tools such as radio, television, and social media to support online teaching and training [ 6 ]. Various stakeholders, including government and private institutions, have collaborated to provide teachers with resources and training to teach effectively on digital platforms. New digital learning platforms like Zoom, Google Classroom, Canvas, and Blackboard have been used extensively to create learning material and deliver online classes; they have also allowed teachers to devise training and skill development programs [ 7 ]. Many teachers and students were initially hesitant to adopt online education. However indefinite closure of institutions required educational facilities to find new methods to impart education and forced teachers to learn new digital skills. Individuals have experienced different levels of difficulty in doing this; for some, “it has resulted in tears, and for some, it is a cup of tea” [ 8 ].

Teachers have reported finding it difficult to use online teaching as a daily mode of communication, and enabling students’ cognitive activation has presented a significant challenge in the use of distance modes of teaching and learning. Teachers have also expressed concerns about administering tests with minimal student interaction [ 9 ]. Lack of availability of smart devices, combined with unreliable internet access, has led to dissatisfaction with teacher-student interaction. Under pressure to select the appropriate tools and media to reach their students, some teachers have relied on pre-recorded videos, which further discouraged interaction. In locations where most teaching is done online, teachers in tier 2 and tier 3 cities (i.e., semi-urban areas) have had to pay extra to secure access to high-speed internet, digital devices, and reliable power sources [ 10 ]. Teachers in India, in particular, have a huge gap in digital literacy caused by a lack of training and access to reliable electricity supply, and internet services. In rural or remote areas, access to smart devices, the internet, and technology is limited and inconsistent [ 6 ]. In cities, including the Indian capital Delhi, even teachers who are familiar with the required technology do not necessarily have the pedagogical skills to meet the demands of online education. The absence of training, along with local factors (for example, stakeholders’ infrastructure and socio-economic standing), contributes to difficulties in imparting digital education successfully [ 10 ]. The gap in digital education across Indian schools is striking. For example, only 32.5% of school children are in a position to pursue online classes. Only 11% of children can take online classes in private and public schools, and more than half can only view videos or other recorded content. Only 8.1% of children in government schools have access to online classes in the event of a pandemic-related restrictions [ 11 ].

The adverse effects of COVID-19 on education must therefore be investigated and understood, particularly the struggles of students and teachers to adapt to new technologies. Significant societal effects of the pandemic include not only serious disruption of education but also isolation caused by social distancing. Various studies [ 7 , 12 , 13 ] have suggested that online education has caused significant stress and health problems for students and teachers alike; health issues have also been exacerbated by the extensive use of digital devices. Several studies [ 6 , 11 , 14 ] have been conducted to understand the effects of the COVID lockdown on digital access to education, students’ physical and emotional well-being, and the effectiveness of online education. However, only a few studies [ 13 , 15 – 17 ] have touched the issues that teachers faced due to COVID lockdown.

In this context, this study is trying to fill existing gaps and focuses on the upheavals that teachers went through to accommodate COVID restrictions and still impart education. It also provides an in-depth analysis of consequences for the quality of education imparted from the teachers’ perspective. It discusses geographical inequalities in access to the infrastructure required for successful implementation of online education. In particular, it addresses the following important questions: (1) how effectively have teachers adapted to the new virtual system? (2) How has online education affected the quality of teaching? (3) How has online education affected teachers’ overall health?

Because of lockdown restrictions, data collection for this study involved a combination of qualitative and quantitative methods in the form of online surveys and telephonic interviews. A questionnaire for teachers was developed consisting of 41 items covering a variety of subjects: teaching styles, life-work balance, and how working online influences the mental and physical well-being of teachers. In the interviews, participants were asked about their experiences of online teaching during the pandemic, particularly in relation to physical and mental health issues. A pilot study was conducted with thirty respondents, and necessary changes to the items were made before the data collection. The survey tool was created using google forms and disseminated via email, Facebook, and WhatsApp. A total of 145 telephonic interviews were also conducted to obtain in-depth information from the respondents.

The data were collected between December 2020 and June 2021. The Research Advisory Committee on Codes of Ethics for Research of Aggrawal College, Ballabhgarh, Haryana, reviewed and approved this study. A statement included in the google survey form as a means of acquiring written consent from the participants. Information was gathered from 1,812 Indian teachers in six Indian states (Assam, Haryana, Karnataka, Madhya Pradesh, New Delhi, and Rajasthan) working in universities, schools, and coaching institutions. Nearly three-quarters of the total sample population was women. All participants were between the ages of 18 and 60, with an average age of 34 and a clear majority being 35 or younger. Nearly three-quarters of participants work in private institutions (25% in semi-government entities and the remainder in government entities). In terms of education, 52% of participants have a graduate degree, 34% a postgraduate degree, and 14% a doctorate. Table 1 summarizes the demographic characteristics of the participants.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0282287.t001

Results & discussion

Upon analyzing the survey responses, three crucial areas were identified for a better understanding of the effect of COVID-19 on the Indian education system and its teachers: how effectively teachers have adapted, how effective teaching has been, and how teachers’ health has been affected.

1. How effectively have teachers adapted to the new virtual system?

The first research question concerns how willing teachers were to embrace the changes brought about by the online teaching system and how quickly they were able to adapt to online modes of instruction. This information was gathered from December 2020 to June 2021, at which point teachers had been dealing with school lockdowns for months and therefore had some time to become conversant with online teaching.

While 93.82% of respondents were involved in online teaching during the pandemic, only 16% had previously taught online. These results were typically different from the results of a similar study conducted in Jordon where most of the faculty (60%) had previous experience with online teaching and 68% of faculty had also received formal training [ 16 ]. Since the spread of COVID-19 was rapid and the implementation of the lockdown was sudden, government and educational institutions were not prepared for alternative modes of learning, and teachers needed some time for adjustment. Several other factors also affected the effectiveness of the transition to online education, namely access to different types of resources and training [ 18 ].

a. Access to smart devices.

Online teaching requires access to smart devices. A surprising number of teachers stated that they had internet access at home via laptops, smartphones, or tablets. A more pertinent question, however, was whether they had sole access to the smart device, or it was shared with family members. Only 37.25% of those surveyed had a device for their exclusive use while others shared a device with family members, due to lack of access to additional devices and affordability of new devices. During the lockdown, an increase in demand led to a scarcity of smart devices, so that even people who could afford to buy a device could not necessarily find one available for purchase. With children attending online classes, and family members working from home, households found it difficult to manage with only a few devices, and access to a personal digital device became an urgent matter for many. Respondents admitted to relying on their smartphones to teach courses since they lacked access to other devices. Teachers on independent-school rosters were significantly better equipped to access smart devices than those employed at other types of schools. The data also indicates that teachers in higher education and at coaching centers had relatively better access to laptops and desktop computers through their institutions, whereas teachers in elementary and secondary schools had to scramble for securing devices for their own use.

b. Internet access.

Internet access is crucial for effective delivery of online education. However, our survey shows that teachers often struggled to stay connected because of substantial differences between states in the availability of internet. Of the respondents, 52% reported that their internet was stable and reliable, 32% reported it to be satisfactory and the rest reported it to be poor. Internet connectivity was better in the states of Karnataka, New Delhi, and Rajasthan than in Assam, Haryana, and Madhya Pradesh. Internet connectivity in Assam was particularly poor. Consequently, many teachers with access to advanced devices were unable to use them due to inadequate internet connection.

The following comments from a teacher in Assam capture relevant situational challenges: “I do not have an internet modem at home, and teaching over the phone is difficult. My internet connection is exhausted, and I am unable to see or hear the students.” Another teacher from Haryana reported similar difficulties: “During the lockdown, I moved to my hometown, and I do not have internet access here, so I go to a nearby village and send videos to students every three days.” Another teacher from Madhya Pradesh working at a premier institution reported experiencing somewhat different concerns: “I am teaching in one of the institute’s semi-smart classrooms, and while I have access to the internet, my students do not, making it difficult to hear what they are saying.”

These responses indicates clearly that it is not only teachers living in states where connectivity was poor who experienced difficulties in imparting education to students; even those who had good internet connectivity experiences problems caused by the poor internet connections of their students.

c. Tools for remote learning.

Teachers made use of a variety of remote learning tools, but access to these tools varied depending on the educator’s affiliation. Teachers at premier institutions and coaching centers routinely used the Zoom and Google Meet apps to conduct synchronous lessons. Teachers at state colleges used pre-recorded videos that were freely available on YouTube. Teachers in government schools used various platforms, including WhatsApp for prepared material and YouTube for pre-recorded videos. To deliver the content, private school teachers used pre-recorded lectures and Google Meet. In addition to curriculum classes, school teachers offered life skill classes (for example, cooking, gardening, and organizing) to help students become more independent and responsible in these difficult circumstances. In addition to online instruction, 16% of teachers visited their students’ homes to distribute books and other materials. Furthermore, of this 36% visited students’ homes once a week, 29% visited twice a week, 18% once every two weeks, and the rest once a month. Additionally, a survey done on 6435 respondents across six states in India reported that 21% teachers in schools conducted home visits for teaching children [ 19 ].

d. Knowledge and training for the use of information and communication technologies.

With the onset of the pandemic, information and communication technology (ICT) became a pivotal point for the viability of online education. The use of ICT can facilitate curriculum coverage, application of pedagogical practices and assessment, teacher’s professional development, and streamlining school organization [ 20 ]. However, the effective adoption and implementation of ICT necessitated delivery of appropriate training and prolonged practice. Also the manner in which teachers use ICT is crucial to successful implementation of online education [ 21 ]. While countries such as Germany, Japan, Turkey, the United Kingdom, and the United States recognized the importance of ICT by integrating it into their respective teacher training programmes [ 22 ], this has not been case in India. However, there are some training programmes available to teachers once they commence working. In accordance with our survey results, the vast majority of respondents (94%) lacked any ICT training or experience. In the absence of appropriate tools and support, these teachers self-experimented with online platforms, with equal chances of success and failure.

The transition from offline to online or remote learning was abrupt, and teachers had to adapt quickly to the new systems. Our data indicate that teachers in professional colleges and coaching centers received some training to help them adapt to the new online system, whereas teachers in urban areas primarily learned on their own from YouTube videos, and school teachers in rural areas received no support at all. Overall, teachers had insufficient training and support to adjust to this completely new situation. Policy research conducted on online and remote learning systems following COVID-19 has found similar results, namely that teachers implemented distance learning modalities from the start of the pandemic, often without adequate guidance, training, or resources [ 23 ]. Similar trends have been found in the Caribbean, where the unavailability of smart learning devices, lack of or poor internet access, and lack of prior training for teachers and students hampered online learning greatly. Furthermore, in many cases the curriculum was not designed for online teaching, which was a key concern for teachers [ 24 ]. Preparing online lectures as well as monitoring, supervising and providing remote support to students also led to stress and anxiety. Self-imposed perfectionism further exacerbated these issues while delivering online education [ 15 ]. A study conducted on 288 teachers from private and government schools in Delhi and National Capital Region area, also found that transition to online education has further widened the gap between pupils from government and private schools. It was more difficult to reach students from economically weaker sections of the society due to the digital divide in terms of access, usage, and skills gap. The study also found that even when teachers were digitally savvy, it did not mean that they know how to prepare for and take online classes [ 10 ].

2. How has online education affected the quality of teaching?

Once teachers had acquired some familiarity with the online system, new questions arose concerning how online education affected the quality of teaching in terms of learning and assessment, and how satisfied teachers were with this new mode of imparting education. To address these questions, specific questionnaire items about assessment and effectiveness of teaching has been included.

a. Effectiveness of online education.

Respondents agreed unanimously that online education impeded student-teacher bonding. They reported several concerns, including the inattentiveness of the majority of the students in the class, the physical absence of students (who at times logged in but then went elsewhere), the inability to engage students online, and the difficulty of carrying out any productive discussion given that only a few students were participating. Another significant concern was the difficulty in administrating online tests in light of widespread cheating. In the words of one teacher: “I was teaching a new class of students with whom I had never interacted in person. It was not easy because I could not remember the names of the students or relate to them. Students were irritated when I called out their names. It had a significant impact on my feedback. I would like us to return to class so I do not have to manage four screens and can focus on my students and on solving their problems.”

For these reasons, 85.65% of respondents stated that the quality of education had been significantly compromised in the online mode. As a result, only 33% reported being interested in continuing with online teaching after COVID-19. The results show slightly higher dissatisfaction in comparison to another study conducted in India that reported 67% of teachers feeling dissatisfied with online teaching [ 25 ]. Findings of this study were similar to the findings of a survey of lecturers in Ukraine assessing the effectiveness of online education. Lower quality student work was cited as the third most mentioned problem among the problems cited by instructors in their experience with online teaching, right behind unreliable internet connectivity and the issues related with software and hardware. Primary reasons for lower quality student work were drop in the number of assignments and work quality as well as cheating. Almost half (48.7%) of the participants expressed their disapproval of online work and would not like to teach online [ 26 ].

Due to the nature of the online mode, teachers were also unable to use creative methods to teach students. Some were accustomed to using physical objects and role-playing to engage students in the classroom, but they found it extremely difficult to make learning exciting and to engage their students in virtual space. Similar trends have been reported in Australia, where schoolteachers in outback areas did not find online education helpful or practical for children, a majority of whom came from low-income families. The teachers were used to employing innovative methods to keep the students engaged in the classroom. However, in online teaching, they could not connect with their students using those methods, which significantly hampered their students’ progress. Some teachers mentioned difficulties with online teaching caused by not being able to use physical and concrete objects to improve their instructions [ 27 ].

b. Online evaluation.

Of our respondents, 81% said that they had conducted online assessments of their students. Teachers used various online assessment methods, including proctored closed/open book exams and quizzes, assignment submissions, class exercises, and presentations. Teachers who chose not to administer online assessments graded their students’ performance based on participation in class and previous results.

Almost two-thirds of teachers who had administered online assessments were dissatisfied with the effectiveness and transparency of those assessments, given the high rates of cheating and internet connectivity issues. They also reported that family members had been helping students to cheat in exams because they wanted their children to get higher grades by any means necessary. In response, the teachers had tried to devise methods to discourage students and their families from cheating, but they still felt powerless to prevent widespread cheating.

As one respondent stated: “We are taking many precautions to stop cheating, such as asking to install a mirror behind the student and doing online proctoring, but students have their ways out for every matter. They disconnect the internet cable or turn it off and reconnect it later. When we question them, they have a connectivity reason ready”.

Teachers are also concerned about the effects of the digital skills gap on their creation of worksheets, assessments, and other teaching materials. As a result, some private companies have been putting together teacher training programs. The main challenge pertains to be implementation of a type of specialized education that many teachers are unfamiliar with and unwilling to adopt [ 28 ]. Because of the lack of effective and transparent online assessments, school teachers have reported that students were promoted to the next level regardless of their performance. Thus, only time will tell how successful online education has been in terms of its effects on the lives of learners.

3. How has online education affected teacher’s overall health?

The onset of the COVID-19 pandemic brought about a situation that few people had experienced or even imagined living through. Governments and individuals tried their best to adjust to the new circumstances, but sudden lockdown, confinement to the household periphery, and working from home had adverse effects on the mental and physical health of many people, including educators and students. To clarify the effects of online education on teachers’ overall health, a number of questionnaire items were focused on respondents’ feelings during the lockdown, the physical and mental health issues they experienced, and their concerns about the future given the uncertainty of the present situation.

a. Physical health issues.

COVID-19 brought a multitude of changes to the lives of educators. Confinement to the household, working from home, and an increased burden of household and caregiving tasks due to the absence of paid domestic assistants increased physical workload and had corresponding adverse effects on the physical health of educators.

Of the study participants, 82% reported an increase in physical health issues since the lockdown ( Fig 1 ). Notably, 47% of those who were involved in digital mode of learning for less than 3 hours per day reported experiencing some physical discomfort daily, rising to 51% of teachers who worked online for 4–6 hours per day and 55% of teachers who worked more than 6 hours per day. Respondents reported a variety of physical health issues, including headaches, eye strain, back pain, and neck pain.

https://doi.org/10.1371/journal.pone.0282287.g001

The number of hours worked showed a positive correlation with the physical discomfort or health issues experienced. A chi-square test was applied to determine the relationship between the number of online working hours and the frequency of physical issues experienced by the participants and found it to be significant at the 0.05 level ( Table 2 ).

https://doi.org/10.1371/journal.pone.0282287.t002

As Fig 2 shows, 28% respondents’ complaint about experiencing giddiness, headaches; 59% complain of having neck and back pain. The majority of the participants had eye-strain problems most of the time; 32% faced eye problems sometimes, and 18% reported never having any eye issue. In addition, 49% had experienced two issues at the same time and 20% reported experiencing more than 2 physical issues at the same time.

https://doi.org/10.1371/journal.pone.0282287.g002

The data in this study indicates a link between bodily distresses and hours worked. As working hours increased, so did reports of back and neck pain. 47% respondents reported back and neck pain after working for 3 hours or less, 60% after working for 3–6 hours, and nearly 70% after working for 6 hours or more.

The analysis also indicates link between physical issues experienced and the educator’s gender. Women experienced more physical discomfort than men, with 51% reporting frequent discomfort, compared to only 46% of men. Only 14% of female educators reported never experiencing physical discomfort, against 30% of male educators.

In terms of types of discomfort, 76% of female teachers and 51% of male teachers reported eye strain; 62% of female teacher and 43% of male teachers reported back and neck pain; 30% of female teachers and 18% of male teachers said they had experienced dizziness and headaches. The gender differences may be caused by the increase in household and childcare responsibilities falling disproportionately on female educators compared to their male counterparts. Several studies [ 17 , 29 – 31 ] have reported similar results, indicating that the gender gap widened during the pandemic period. The social expectations of women to take care of children increased the gender gap during the pandemic by putting greater responsibilities on women in comparison to men [ 29 ]. Women in academics were affected more in comparison to the men. Working from home burdened female educators with additional household duties and childcare responsibilities. A study done [ 32 ] in France, Germany, Italy, Norway, Sweden, the United States and the United Kingdom discovered that women were immensely affected by lockdown in comparison to men. On top of this, women with children are affected more than women without children.

No effect of age on physical discomfort was observed in this study but increasing use of online tools (such as class websites) for content creation and delivery and extended working periods were major contributors to health problems.

b. Mental health issues.

The psychological effects of the COVID-19 pandemics have also proved difficult to manage. Being at home all day with limited social interaction, not to mention other pandemic-related sources of stress, affected the mental health of many people. The majority of the participants in this study admitted experiencing mental health issues including anxious feelings, low mood, restlessness, hopelessness, and loneliness. According to UNESCO [ 33 ], due to the sudden closure of schools and adaptability to new systems, teachers across the world are suffering from stress. Studies conducted in various parts of the world confirmed similar trends [ 34 , 35 ]. In Israel, teachers reported psychological stress due to online teaching. 30.4% teachers reported being stressed in comparison to 6.1% teachers in traditional classroom settings [ 34 ]. In Spain, teachers experienced various kinds of mental health issues like anxiety, stress, and depression [ 36 ]. An Arabian study found an increased number of cases related to anxiety, depression, and violence during the pandemic [ 37 ]. In New Zealand teachers in Higher education reported being overwhelmed due to the online teaching [ 15 ].

Online teaching appears to have negatively affected the mental health of all the study participants. Women (94%) reported more mental health issues than men (91%), as shown in Fig 3 . Nearly two-thirds of participants said they had been dealing with mental health issues regularly and a third occasionally; only 7% said they never dealt with them. Findings of this study are in line with other studies which found that female teachers had higher levels of stress and anxiety in comparison to men [ 36 ]. Studies conducted in China reported that teachers developed mental health issues due to online classes [ 37 , 38 ].

https://doi.org/10.1371/journal.pone.0282287.g003

Our analysis indicated a positive relationship between the number of working hours and the frequency of mental health issues. Of the respondents who worked online for less than 3 hours, 55% experienced some kind of mental health issue; this rose to 60% of participants who worked online for 3–6 hours, and 66% of those who worked more than 6 hours every day. A chi-square test was applied to determine the relationship between the number of online working hours and the frequency of mental issues experienced by the participants and found it to be significant at the 0.05 level ( Table 3 ).

https://doi.org/10.1371/journal.pone.0282287.t003

In terms of types of mental health issues, respondents reported restlessness, anxious feelings, and a sense of powerlessness, along with feelings of hopelessness, low mood, and loneliness as shown in Fig 4 . The stress of adapting to a new online working environment, the extended hours of work required to prepare content in new formats, the trial-and-error nature of learning and adopting new practices, uncertainty caused by lockdown, and an overall feeling of having no control were some of the contributing factors.

https://doi.org/10.1371/journal.pone.0282287.g004

Mental health issues were more common among those under the age of 35, with 64% reporting a problem most of the time compared to 53% of those over 35. It has been found that job uncertainty is one of the primary causes of a higher prevalence of mental health concerns among younger respondents than among older respondents. These findings are in line with other studies which found higher levels of stress among the young people in comparison to older one [ 36 , 39 ]. Feelings of loneliness and a sense of no control were reported by 30% of respondents under the age of 35, with these feelings occurring constantly or most of the time; only 12% of respondent over the age of 35 reported experiencing these feelings always or most of the time. Of respondents under 35 years of age 61% felt lonely at some point during the COVID-19 pandemic, compared to only 40% of those age 35 or older.

This study also found gender-based differences in the frequency of mental health issues experienced, with 62% of male respondents and 52% of female respondents reporting that they had always experienced mental health issues. The types of issues also differed by gender, with men more likely to report restlessness and loneliness and women more likely to report feeling anxious or helpless. More female respondents reported feelings of hopelessness than male respondents (76% compared to 69%), and they were also more anxious (66%).

The uncertainty of the pandemic seems to have caused helplessness and anxious feelings for female teachers in particular, perhaps because a lack of paid domestic help increased the burden of household and caregiving tasks disproportionately for women at a time when the pressure to adapt to new online platforms was particularly acute. In some cases, respondents left their jobs to accommodate new family dynamics, since private employers offered no assistance or flexibility. Deterioration of mental health also led to the increased number of suicides in Japan during COVID-19 [ 39 ].

However, female teachers fared better than their male counterparts on some measures of mental health. Although half of the respondents (men and women equally) reported low mood during the pandemic, the men reported more restlessness (53%) and loneliness (59%) than the women (50% and 49%, respectively). Restrictions on eating and drinking outside the household may have had a disproportionate effect on male respondents, making them more likely to feel restless or lonely than their female counterparts, who may have handled COVID-related isolation better by being more involved in household work and caregiving.

Number of hours worked online was also a factor contributing to mental health issues. Just as respondents had more physical complaints (including eye strain, back and neck pain, and headaches) the more hours they worked online, respondents who worked longer hours online reported more mental health issues.

One of the major drawbacks of online education is the widespread occurrence of physical and mental health issues, and the results of this study corroborate concerns on this point. This study found that online teaching causes more mental and physical problems for teachers than another study, which only found that 52.7% of respondents had these problems [ 12 ].

A report by the University of Melbourne has also indicated that online teaching and learning have a negative effect on the physical and mental well-being of individuals. Teachers working from home, in particular, have reported isolation, excessive screen time, inability to cope with additional stress, and exhaustion due to increased workload; despite being wary of the risks of exposure to COVID-19, they were eager to return to the campus [ 27 ].

c. Support mechanisms.

In general, teachers experienced good support from family and colleagues during the pandemic, with 45.64% of teachers reported receiving strong support, 29.64 percent moderate support (although the remainder claimed to have received no or only occasional support from family and colleagues). 9.39% of male respondents reported that they have never received any support in comparison to 4.36% females. Female respondents reported receiving more support than male respondents perhaps because they have access to a more extensive network of family members and coworkers. Children, parents, and siblings were cited as the provider of a robust support system by most female respondents. For example, maternal relatives called or texted children to keep them engaged and helped them with homework, and female participants said their peers helped them to prepare lectures and materials. A link was also found between age and support; the older the respondent, the stronger the support system. A possible explanation for this difference is that older people have had time to develop stronger and longer-lasting professional and personal ties than younger people.

This study explored the effects of the COVID-19 pandemic on the Indian education system and teachers working across six Indian states. The effectiveness of online education methods varied significantly by geographical location and demographics based on internet connectivity, access to smart devices, and teachers’ training. While premier higher education institutions and some private institutions had provided teachers with the necessary infrastructure and training to implement effective successful online learning with relatively few challenges, teachers at schools and community colleges have more often been left to adopt a trial-and-error approach to the transition to an online system. Further, it indicates that online education has had a significant effect on the quality of education imparted and the lives and wellbeing of teachers. While online learning has enabled teachers to reach out to students and maintain some normalcy during a time of uncertainty, it has also had negative consequences. Owing to the lack of in-person interaction with and among students in digital classes, the absence of creative learning tools in the online environment, glitches and interruptions in internet services, widespread cheating in exams, and lack of access to digital devices, online learning adversely affected the quality of education. Teachers experienced mounting physical and mental health issues due to stress of adjusting to online platforms without any or minimal ICT training and longer working hours to meet the demands of shifting responsibilities. A positive correlation was found between working hours and mental and physical health problems.

The long-term impact of COVID-19 pandemic on both the education system and the teachers would become clear only with time. Meanwhile, this study sheds light on some of the issues that teachers are facing and needs to be addressed without further ado. These findings will provide direction to the policy makers to develop sound strategies to address existing gaps for the successful implementation of digital learning. However, researchers should continue to investigate the longer-term effects of COVID pandemic on online education.

Supporting information

S1 file. supplementary material..

https://doi.org/10.1371/journal.pone.0282287.s001

• 1. WHO Coronavirus (COVID-19) Dashboard. [cited 14 Jan 2022]. Available: https://covid19.who.int
• View Article
• Google Scholar
• 3. unesco_covid-19_response_in_cambodia.pdf. Available: https://en.unesco.org/sites/default/files/unesco_covid-19_response_in_cambodia.pdf
• PubMed/NCBI
• 11. aser2020wave1report_feb1.pdf. Available: https://img.asercentre.org/docs/ASER%202021/ASER%202020%20wave%201%20-%20v2/aser2020wave1report_feb1.pdf
• 19. India Case Study.pdf. Available: https://www.unicef.org/rosa/media/16511/file/India%20Case%20Study.pdf
• 23. UNSDG | Policy Brief: Education during COVID-19 and beyond. [cited 21 Jan 2022]. Available: https://unsdg.un.org/resources/policy-brief-education-during-covid-19-and-beyond , https://unsdg.un.org/resources/policy-brief-education-during-covid-19-and-beyond
• 25. Report on rapid assessment of learning during school closures in context of COVID-19.pdf. Available: https://www.unicef.org/india/media/6121/file/Report%20on%20rapid%20assessment%20of%20learning%20during%20school%20closures%20in%20context%20of%20COVID-19.pdf
• 31. Reflections on motherhood and the impact of COVID 19 pandemic on women’s scientific careers—Guatimosim—2020—Journal of Neurochemistry—Wiley Online Library. [cited 30 Jun 2022]. Available: https://onlinelibrary.wiley.com/doi/10.1111/jnc.15158
• 33. https://plus.google.com/+UNESCO . Adverse consequences of school closures. In: UNESCO [Internet]. 10 Mar 2020 [cited 30 Jun 2022]. Available: https://en.unesco.org/covid19/educationresponse/consequences

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 06 April 2023

A global analysis of the effectiveness of policy responses to COVID-19

• Kwadwo Agyapon-Ntra 1 &
• Patrick E. McSharry 1 , 2 , 3  

Scientific Reports volume  13 , Article number:  5629 ( 2023 ) Cite this article

7382 Accesses

10 Citations

19 Altmetric

Metrics details

• Health policy
• Infectious diseases
• Psychology and behaviour
• Respiratory tract diseases
• Viral infection

Governments implemented many non-pharmaceutical interventions (NPIs) to suppress the spread of COVID-19 with varying results. In this paper, country-level daily time series from Our World in Data facilitates a global analysis of the propagation of the virus, policy responses and human mobility patterns. High death counts and mortality ratios influence policy compliance levels. Evidence of long-term fatigue was found with compliance dropping from over 85% in the first half of 2020 to less than 40% at the start of 2021, driven by factors such as economic necessity and optimism coinciding with vaccine effectiveness. NPIs ranged from facial coverings to restrictions on mobility, and these are compared using an empirical assessment of their impact on the growth rate of case numbers. Masks are the most cost-effective NPI currently available, delivering four times more impact than school closures, and approximately double that of other mobility restrictions. Gathering restrictions were the second most effective. International travel controls and public information campaigns had negligible effects. Literacy rates and income support played key roles in maintaining compliance. A 10% increase in literacy rate was associated with a 3.2% increase in compliance, while income support of greater than half of previous earnings increased compliance by 4.8%.

Similar content being viewed by others

A comprehensive evaluation of COVID-19 policies and outcomes in 50 countries and territories

Hsiao-Hui Tsou, Shu-Chen Kuo, … Shan-Chwen Chang

Effects of government policies on the spread of COVID-19 worldwide

Hye Won Chung, Catherine Apio, … Taesung Park

Non-pharmaceutical interventions to combat COVID-19 in the Americas described through daily sub-national data

Michael Touchton, Felicia Marie Knaul, … Valentina Vargas Enciso

Introduction

COVID-19 is generally believed to have originated in Wuhan, China in late 2019 1 . By the 11th of March 2020, the virus had been detected in 114 countries across the world and was officially declared a pandemic by the World Health Organization (WHO) 2 . Government responses to the pandemic have varied dramatically with time 3 , 4 . For federal governments like that of the United States of America, these variations in responses are present even at the state level 5 . Following the declaration of a pandemic by WHO, there were widespread global lockdowns in March of 2020. More disparate measures were implemented in countries across the globe over the following months. These varying policy responses produced different outcomes not just in terms of how the virus spread, but also with regard to how different individuals reacted to restrictions, adverse effects on national economies 6 , and decreasing the personal well-being of billions of people across the planet, especially with respect to the declining mental health of people during lockdowns 7 , 8 .

The wide variation in policy responses to COVID-19 across countries can largely be attributed to a global lack of experience in dealing with pandemics at this scale. COVID-19 forced many countries to adopt various non-pharmaceutical interventions (NPIs) as community mitigation strategies. NPIs are actions taken, besides the use of medicine and vaccines, to flatten the disease transmission curve 9 . While NPIs have proved to be important in mitigating the spread of the virus and ensuing deaths over time, and across waves 10 , 11 , certain policies had terrible effects on national economies, from which some nations are still yet to recover 12 , 13 , 14 .

In this paper, we perform a global analysis of various policies rolled out by governments. We study the effect that these policies had on the spread of the virus while taking into account the socioeconomic factors at play. We seek to provide models and evidence for guiding effective government policies in the early (pre-vaccination) stages of a pandemic to yield the highest compliance from citizens without severely compromising the economy and quality of life of people needlessly.

In this paper, we seek to answer the following four research questions.

How did COVID-19 (cases and deaths) and stringent policies affect mobility?

Which policy measures are most effective for managing COVID-19?

What are the influences of socioeconomic and demographic factors?

How has compliance changed with the emergence of variants?

Literature review

COVID-19 has had far-reaching global effects, leading to a worldwide concerted effort to curate relevant data and generate research necessary to understand the virus, responses, and eventual outcomes.

As Berger et al. 15 note, governments all over the world were faced with the arduous task of making policy decisions under high levels of uncertainty in an attempt to halt the spread of the virus, and when that failed, they had to make more tough decisions on the timing, duration and intensity of interventions to minimize case counts and mortality, primarily through attempts to enforce social distancing and consequently reduce transmission rates 6 . This is commonly referred to as “flattening the curve”, and according to a forecasting study on the United States of America 16 , could have potentially resulted in benefits valued at about $5.2 trillion if implemented effectively.

In the pre-vaccination stage of the pandemic, the policies implemented by governments were NPIs rolled out at country and state levels with different levels of enforcement 7 , 11 . One of the policies most advocated for was the use of facial coverings, and primarily, nose masks 17 , 18 . According to Mitze et al. 19 not only did face masks considerably reduce COVID-19 cases in Germany, achieving a 45% reduction in a span of 20 days, but they also have negligible economic costs when compared to other public health measures.

With what is hopefully the worst of the pandemic behind us, we, as a planet, are trying to adapt to life with COVID-19; what many have called a “new normal” 20 , 21 . However, while we have learned to live with COVID-19, experts still warn of the possibility of future pandemics 22 , 23 . It is imperative that we learn as much as we can about the optimal blend of policies for addressing a pandemic based on our experience with the novel coronavirus during the last two years. It is especially important to understand the efficacy of different policies that are meant to address the effects of the virus in the period before vaccines are available. This allows us to prepare ourselves for the possibility of future pandemics.

Retrospective studies performed at a country level have helped to assess the effectiveness of certain policies against COVID-19. For example, the use of facial coverings has been found to be most effective and cost-effective in Germany 19 . Azman and Luquero 24 suggest that China’s extreme lockdowns, active case surveillance, and other rapid control measures led to substantial reductions in transmission as of late March 2020, although this was at the expense of the social and emotional well-being of many and has resulted in slower economic growth. In Rwanda, in addition to early action and effective use of social media for information dissemination, robots were employed for patient monitoring in hospitals 25 .

Considering that the operational contexts of every country are different and that certain policies cannot be readily transferred, it becomes necessary to take a holistic look at the policies on a multinational scale to test empirically which ones perform adequately regardless of the country in question. The Organisation for Economic Co-operation and Development (OECD) attempted such an analysis 26 but was limited to just 18 out of its 38 member states. Their paper acknowledged the role of income support and debt relief as motivating factors for people required to stay at home but raised the issue of poor targeting in terms of who needed this kind of support the most, and whether they had the right access. This is in line with assertions that an understanding of socioeconomic variations in government and citizen responses is required for pandemic governance 27 . The OECD finally concedes the need for extra investigations to make the right policy recommendations for member states. A study of 37 countries examined the effectiveness of policies in response to the first COVID-19 outbreak and found that the greater the strength of government interventions at an early stage, the more effective these are in slowing down or reversing the growth rate of deaths 28 .

Contribution

In this paper, we move past learnings from single countries and international economic organisations like the OECD to pursue a truly global view of the effects of COVID-19 and the effectiveness of NPI mitigation policies using data available through the Our World In Data (OWID) platform.

Through our investigations, we provide an empirical analysis of the effects of government policies on the spread of the virus, while taking into account the effects of socio-economic factors on the general compliance of citizens, especially with regard to mobility-limiting restrictions, which can have severe economic consequences.

We quantify compliance as the correlation between stringency and relative residential mobility; the relative amount of time people spend at home, as provided by Google mobility trends. Stringency, as used in this paper, refers to policy response combinations adopted by governments to reduce and manage the spread of COVID-19 by limiting contact between citizens. A stringency index, recording the strictness of 'lockdown style' policies that primarily restrict people's behaviour is computed by the Oxford Coronavirus Government Response Tracker (OxCGRT) project 29 , 30 . We give further details in the data sub-section of our methodology section. Our focus on quantifying compliance is of particular interest to policymakers as failure to effectively change mobility patterns weakens the impact of the intervention.

The following subsections detail our methods of analysis, including visualisations and regressions performed to generate insights and explain the contributions of certain features to both case numbers and compliance. We then inspect and draw out insights from our results, followed by data-driven recommendations to serve as a policy guide in the event of future pandemics.

Data and definitions

Through a comprehensive aggregation of datasets relevant to COVID-19 on the open-source Our World in Data (OWID) platform 29 , we had access to COVID-19 case data beginning from 28th January 2020 and government policy responses within the same period 30 . Besides a few clearly indicated exceptions, all data referenced in this paper was obtained from OWID. Published data on cases and deaths attributed to COVID-19 are subject to under-reporting 31 . In particular, reported cases are likely to be a substantial underestimate of the prevalence of the disease in a population given that most people with COVID-19 are asymptomatic, and even among those who are symptomatic, not all are tested. While acknowledging these data limitations, our study focuses on visualising the evolution of the pandemic across several variables, quantifying compliance in terms of reducing mobility and assessing whether or not policies successfully reduced the number of cases.

Our first objective was to assess the effects of COVID-19 on society, policy responses and subsequent changes in mobility. A visual representation is given in Fig.  1 . To quantify these effects, we analysed relevant data focusing on five variables of interest.

New deaths smoothed per million: provides a snapshot of deaths as an intensity metric and how this changes over time across the globe.

Mortality ratio (derived as the number of deaths smoothed per million divided by the number of cases smoothed per million): measures how deadly the virus is at any point in time, but strictly in relation to the percentage of people confirmed to have died from COVID-19 at that time.

Stringency index: a composite measure of stringency responses by governments based on the OxCGRT index developed to quantify the extent to which restrictions were applied at a governmental level across the nations of the world.

Google residential mobility: quantifies the time people spent at home relative to the median value over the five‑week period from January 3rd to February 6th 2020, as captured by Google mobility trends data. This data serves as a measure of the human response to government policies and is our most trustworthy means of empirically determining compliance with mobility restrictions 32 , 33 .

Vaccination percentage: a snapshot of the percentage of the global population that, in line with the approach of Hallas et al. 5 , has received at least one dose of any COVID-19 vaccine.

Global evolution of COVID-19 showing the temporal variation in new cases, mortality ratio, mean stringency ratio, mean residential mobility index and vaccination administration.

In addition to these variables, we also had to identify when major variants of the virus were first detected. This data was collected from the official WHO records 34 , however, only the month and year were provided so we assumed the 15th day of the given month as the date of first detection. Another important date is the 6th of June 2020, when masks were mandated by WHO 35 .

Our next agenda was to determine the effectiveness of stringency measures implemented by governments. As mentioned earlier, the stringency index is a composite measure, and it aggregates nine different metrics: (1) school closures; (2) workplace closures; (3) cancellation of public events; (4) restrictions on public gatherings; (5) closures of public transport; (6) stay-at-home requirements; (7) public information campaigns; (8) restrictions on internal movements; and (9) international travel controls 29 . All these stringency metrics are primarily meant to reduce mobility and facilitate social distancing. Masks and facial coverings have also been heavily advocated for and therefore data on facial covering policies is included 19 . The impact of these policies over time was defined as the negative correlation coefficient between each policy and the percentage change in the number of cases smoothed per million over a given time period.

Finally, we addressed the issue of stringency policy compliance, defined as the correlation between the composite stringency index and the Google residential mobility index. We investigated how compliance could be explained by socioeconomic and demographic factors, taking the following into account: (1) GDP per capita; (2) Life expectancy; (3) Median age 29 ; (4) Literacy rate 36 ; (5) Corruption index 37 ; and (6) Freedom of expression 38 .

We applied a log transformation to the GDP per capita, which is a standard normalization method to make the distribution more normal. This is also in line with the popular United Nations human development index 39 , which is a combination of the log of GDP per capita, life expectancy and an education index 40 .

Lockdown restrictions are economically costly both to states and to individuals and could have a bearing on how well people comply with directives to stay at home or avoid workplaces. As such, we consider the effects of (1) income support and (2) debt relief policies on the predictability of compliance.

The Pearson correlation coefficient is a statistical standard for measuring the strength of a relationship between two variables. We used it extensively in this paper for calculations of (1) policy compliance, which is assessed using the correlation between stringency indices and Google residential mobility trends, and (2) policy impact, defined as the negative correlation between stringency sub-indices, including policy on facial coverings, and the percentage change in normalised case counts.

Considering that the COVID-19 pandemic has been ongoing for over two years, a certain amount of restriction fatigue 5 is expected to have developed over time across different countries, which is consistent with the findings of Petherick et al. 41 . Fatigue is visually recognized as a dip in compliance over a period of time. Compliance can be measured and tracked over time by applying a rolling average window to calculate the correlation coefficient between stringency and residential mobility. This window needs to be long enough to provide sufficient data for reliable statistics but short enough to capture changes in behaviour. We experimented with different window sizes and found that three months was most effective for this purpose.

We divided the pandemic window into three major time periods: the uncertainty period (January–March 2020) which represents a break between the normalcy of 2019 and the declaration of COVID-19 as a pandemic by WHO 2 ; the pre-vaccination period (April–December 2020); and the vaccination period (January 2021 to the present). The analysis below focuses predominantly on the period from 1st April 2020 through 31st December 2020. This was the period before the first major rollout of vaccinations and within which much of the uncertainty associated with the pandemic was decreasing. Essentially, this pre-vaccine period is the most relevant period within which we can assess the effectiveness of different policies. Once the vaccines were rolled out, there was a marked change in behaviour with many people becoming much less risk-averse than they had been previously. It can be argued that this period is therefore a stable period of time for quantifying policy impact and compliance.

To quantify the average policy impact, I(k), for a period of the following k days, we pooled daily country-level policy index data, x(t), for the pre-vaccination period (1st April through 31st December 2020) and computed the correlation with the relative change, g(t,k), in normalised case counts smoothed per million, y(t), over a period of k days. These calculations were performed on a range of horizons, k, from 0 to 84 days (12 weeks):

To assess the influence of socioeconomic and demographic factors on compliance, we ran regressions on a variety of country-level factors within the pre-vaccination period against the average compliance of that country within the same period. We took into account the income support and debt relief indices so as to quantify the effects of these policies on the compliance levels. The results are studied across the globe and on a continent level during the pre-vaccination period. The adjusted R-squared is used as a measure of the goodness of fit for each model.

In order to ensure the robustness of our regression analyses to potential overfitting we calculated the cross-validated R-squared value by leaving out one country in each fold of our cross-validation technique. Leave-one-out essentially provides an out-of-sample result for each country by estimating the model on the remaining N − 1 countries. Hence this is more computationally intensive than the traditional in-sample approach because instead of fitting a single model, it is now necessary to construct a separate model for each country that is left out. All the results are then combined to provide the final cross-validated R-squared result.

Effects of COVID-19 and policy responses on mobility

Figure  1 gives a global long-term temporal view of the spread of the virus (blue) and the associated mortality rate (red) as well as the average stringency index (green) and the human response in terms of the Google residential mobility index (orange), indicating the relative change in the amount of time people spend at home, compared to the median value for the 5‑week period from 3rd January to 6th February 2020. Super-imposed on all of this is the percentage of the global population with at least one dose of the vaccine (brown). We also divide the graph into separate sections using specific dates of interest, particularly the day on which WHO recommended mask mandates and the dates on which the major variants were estimated to have first been detected.

The period before April 2020 was marked by a lot of uncertainty about the virus, its transmission, treatment, and prevention, among other things. The first wave of deaths in this period was met by lockdowns and extreme restrictions on mobility, with the global average stringency index reaching a high of 80.83% on 18th April 2020. Just six days after this peak in restrictions, the highest mortality ratio of 6.35% was recorded on 24th April 2020. Eventually, concerted global efforts of restricting human mobility followed by a WHO recommendation for facial coverings (masks) had the desired effect of reducing the mortality rate and keeping it under 2.0%.

After the initial lockdowns, there have been more waves of the virus, and while not synchronized across countries, these have been marked by the emergence of different variants: Beta, Delta, and Omicron (Fig.  1 ). According to WHO 34 , these variants were estimated to have emerged in May 2020 (Beta), October 2021 (Delta), and November 2021 (Omicron), with Alpha being the originally sequenced variant. Omicron proved to be the most contagious, with cases reaching a record high of over 3.5 million cases in January of 2022.

From Tables 1 and 2 we can accurately track the levels and changes that occurred between key dates for these important variables. For example, during the period from when the virus was declared a pandemic on 11th March 2020 until the peak of the mortality ratio of 6.35% experienced on 24th April 2020, the normalized case count grew by 157.85%. This was within the period of the first global lockdown in which more than 100 countries instituted a full or partial lockdown 42 . The lockdowns led to an 8.21% reduction in the normalised case count and a 2.65% drop in the mortality ratio by the time the Beta variant was detected in May 2020. Throughout this period, compliance was consistently above 80%, indicating that people were actually staying at home and following recommended guidelines.

On 6th June 2020, WHO officially recommended the use of masks, and in the four months between then and when the Delta variant was first detected in October, the mortality ratio more than halved from 1.99 to 0.92%. Nevertheless, with a drop in compliance from 77.94 to 31.52%, we see a 314% increase in the number of cases in the same period. With the advent of Delta, and moving into the end of 2020 we observe an 18.82% increase in compliance as the mortality ratio climbed back up to 1.76%, suggesting that people’s behaviour, in terms of risk aversion, is largely motivated by the perceived deadliness of the virus at a given time.

In the eleven months following vaccination rollouts, the mortality ratio dropped back down to 1.09% and compliance saw a corresponding drop to 24.24% despite three waves of the Delta variant. At this point, in November 2021, the Omicron variant emerged, and with it, a 94% surge in the normalised case counts by the end of 2021. However, the mortality ratio dropped to 0.36% and compliance dropped to 21.3% in the same period. It is believed that the Omicron variant is more contagious but less deadly than the Delta variant 43 .

Omicron went on to reach an all-time high normalised case count of 1158 new cases smoothed per million on 26th January 2022, marking an almost 200% increase in just 26 days. The mortality ratio, however, dropped to 0.19% in that same period. With the continuous drop in mortality ratio throughout 2021 and going into 2022 we see that even though many people became infected and tested positive, the risk of fatality was successfully managed downwards as time progressed.

With the drop in the mortality ratio, we also see a drop in stringency and a corresponding drop in residential mobility (people staying at home). For example, at the height of the first wave on 24th April 2020, the global average stringency is 79.78% and residential mobility is 20.31%. However, by the last day of 2021 stringency has dropped to 46.29% and residential mobility has declined to 5.46%. The correlation between stringency and residential mobility provides a useful measure of compliance, and the changes in compliance over time demonstrate how human behaviour has varied. The factors influencing compliance are many and of course vary from one person to the next. For example, a UK study 44 highlighted increased symptoms of fatigue based among males, the divorced, part-time employees, and/or parents of more than two children during periods of warmer temperature. By considering a number of variables, we can, however, offer some insights about how populations respond in aggregate.

Using a moving three-month window, we plotted the evolution of compliance over the pre-vaccination period of 2020, using a moving average window of three months, comparing the global evolution of compliance to the weekly percentage changes in new deaths smoothed per million and new cases smoothed per million (Fig.  2 ). The detrimental effect of non-compliance is evident in this graph. As compliance decreased from almost 75% in July 2020 to less than 30% in October 2020, the week-on-week death rate increased from a low of − 8% in July 2020 to eventually peak at 12% in November 2020.

Average compliance levels for the globe using rolling windows of three months from April 2020 to December 2020 (the pre-vaccination period) superimposed on the weekly percentage change in deaths smoothed per million and the weekly percentage change in case counts smoothed per million.

A similar plot was made using a three-month window to show the evolution of compliance over the 2-year period of varying restrictions from April 2020 to April 2022 (Fig.  3 ). The plots were made for countries grouped by continent, with global compliance superimposed.

Average compliance levels for the globe and each continent using correlations estimated using rolling windows of three months between April 2020 and April 2022.

From Figs.  2 and 3 we see an obvious downward trend in compliance over time; a clear indicator of restriction fatigue. Figure  2 gives us a sense of how compliance is affected by changes in case and death counts over time, and under the influence of new variants. Figure  3 gives us a longer-term sense of the fatigue trends in the various continents throughout the pandemic, with occasional peaks shortly following a sharp dip in compliance or the emergence of a new variant.

Fitting an exponential decay curve to compliance in the pre-vaccination period (April 1, 2020 to December 31, 2020), we can estimate the half-life of compliance. Suppose that c(t) is compliance at time t, then exponential decay with a decay constant ɑ is represented as:

Fitting our compliance time series to the equation, we estimate a decay constant ɑ of 0.004. The half-life is then given by t = − ln(2)/− ɑ. Our estimates found that during the pre-vaccination period, it takes 173 days, a little less than 6 months, for compliance levels to drop to half of the initial value.

From Fig.  2 we see that global compliance up until the middle of 2020 was above 65 percent with a 3-month window, showing a high level of willingness from most citizens in the early days to follow the rules imposed to restrict mobility and successfully manage the pandemic. However, towards the end of 2020, we see a sharp decline in compliance. The largest dip in compliance culminates in a less than 30% compliance level seen in early October 2020. This dip in compliance was followed by the emergence of the Delta variant, after which we see a slight rise in compliance followed by another dip in December around the holiday period. The year 2020 ended with a compliance level of 37.45%. This aligns well with the findings of Ganslmeier et al. 44 , where compliance is shown to be modulated both by weather and social patterns; compliance dips in the typically hotter months and known periods of socialisation.

From Fig.  3 , it can be seen that compliance was highest in early 2020 and the disparity between different continents was small, demonstrating a concerted effort to contain the virus using mobility restrictions in the absence of vaccines. As the different continents experienced different waves of the pandemic we begin to see clearer temporal differences in compliance levels. For example, after the emergence of the Delta variant, Europe generally had the highest peaks in compliance (greater than 70% in two cases), while Africa’s compliance was generally low (within the range of 0–20% in most cases). By 2022, compliance had dropped so far down in Oceania and North America that compliance levels were actually in the negative based on the Google residential mobility data.

The observations in the pre-vaccine period throughout 2020 are still the most interesting since these are the responses solely affected by NPIs. During this period, the average compliance levels by continent were, from highest to lowest: 68.17%, 67.22%, 61.35%, 59.37%, 54.25%, and 50.52% for the continents of Europe, Oceania, South America, North America, Asia and Africa respectively. At the country level (Fig.  4 ) we see a trend of increasing compliance with increasing GDP per capita. The nations of South Korea, Nicaragua, Mongolia, and Tajikistan proved to be outliers with average compliance rates of less than 20% and were thus excluded in subsequent regressions to avoid skewing the global analysis.

Scatter plot of GDP per capita against compliance for each country in the pre-vaccination period (1st April 2020 through 31st December 2020).

Major differences are expected across nations based on government support mechanisms that allow people to stay at home. In addition, some countries are more likely to have a larger number of people that already work from home, with many engaged in the digital economy. It is therefore not surprising that wealthier countries can afford to have higher compliance levels, potentially explaining why Europe and Oceania are the most compliant continents.

Effectiveness of policy measures

With the exception of public information campaigns which maintain a high level from April 2020 to the present, the disaggregated stringency measures (Fig.  5 ) all follow a similar trend, with a spike between March and April 2020 and a slow consistent descent over time. In contrast, policies on facial coverings took off with a much slower start but eventually remained relatively constant around the 70% mark towards the end of the year 2020.

Global temporal variation in mean stringency sub-index, including facial covering index (right), and number of new cases smoothed per million (left).

Case numbers generally continued to increase, but the Omicron variant, known to be extremely transmissible, introduced a sharp spike, starting in December 2021 and carried through to April 2022. Fortunately, there was no commensurate spike in the number of deaths in the early part of 2022.

The impact analysis of the government NPI policies investigated how different responses achieve the desired outcome of lowering cases over different time horizons (Fig.  6 ). For each policy, the maximum impact and corresponding horizon were identified (Table 3 ). The aim is to identify policies that reduce cases and therefore have a negative correlation with a large absolute value.

Policy impact quantified using the correlation between government policies and relative changes in normalised case counts for various horizons.

Facial coverings have the greatest impact by successfully driving down the percentage of new COVID-19 case counts with an optimal horizon of 31 days. With the exception of public information campaigns and international travel controls, all the other stringency sub-indices have a positive impact on the percentage change in the number of cases smoothed per million with their optimal horizons in the range of 12–31 days.

Influence of socioeconomic and demographic factors on compliance

Country-level socioeconomic and demographic factors were used to investigate the variability in compliance across countries. Compliance was positively correlated with all the considered factors, which intuitively makes sense as an increase in any of the factors should result in better compliance. Note that the corruption index is an inverted feature by definition, and a higher corruption index is indicative of less perceived corruption in a country. The square of the coefficients revealed the following order of feature importance (Fig.  7 ).

Feature importance of demographic and socioeconomic factors based on the square of correlation coefficients (R-squared) against compliance.

Literacy Rate is the most highly correlated variable with compliance. This demonstrates that the presence of a highly literate population is a major factor in determining how willing people are to stay at home during lockdowns. Older populations, the ability of a government to provide income support, wealthier populations, and countries with lower corruption, are also more likely to be compliant with restriction measures.

Co-linearity is an obvious problem with the features highlighted in Fig.  7 , so after identifying a model with backward stepwise regression, Literacy Rate and the Income Support index were selected as statistically significant features (p < 0.001 and p < 0.05 respectively). The model had an adjusted R-squared value of 0.293 (Table 4 ).

While it is clear that income support is effective in driving compliance to stay-at-home requirements, it is also true that many countries do not have the means to implement this. According to a World Bank report 45 ninety percent of countries reported a decline in GDP per capita in 2020 and an estimated 120 million people were pushed into extreme poverty. The richer countries of the world would need to help developing economies in order to use income support as a means of improving policy compliance in the face of contagious disease outbreaks.

An increase of 1% in the literacy rate of a country is associated with a 0.32% increase in compliance (Table 4 ), meaning that a country with an 80% literacy rate is likely to see 3.2% more compliance than one with a 70% literacy rate. This is in line with Rodon et al. 46 who assert that people with a higher COVID-19 health literacy adopt more protective behaviours.

Income support index, on the other hand, takes on the values 0, 1 or 2 depending, respectively, on whether zero financial support is available, up to half, or greater than half of one’s previously earned income was provided by the government. This regression suggests that providing income support of more than half of previous earnings is associated with an increase of 4.76% in the compliance level (Table 4 ).

Univariate linear regressions on the selected features estimate even more gains in compliance for each feature with an estimated 0.41% increase in compliance per unit percentage increase in literacy rate (Fig.  8 ) and an estimated 8.82% increase in compliance per unit percentage increase in income support (Fig.  9 ).

Linear regression model using the literacy rate as the selected independent variable and compliance as the dependent variable.

Linear regression model using income support as the selected independent variable and compliance as the dependent variable.

Compliance was defined as the correlation between the stringency index and residential mobility (time spent in residential locations). The high positive values of compliance found in the first half of 2020, averaging over a three-month window (Fig.  2 ), is proof that people are generally compliant with COVID-19 policies, but fatigue tends to set in once the number of deaths begins to decline (Fig.  2 ). In the first half of 2020, global citizens maintained a compliance level higher than 65% on average. Towards the end of the first year of the pandemic, however, there was a significant drop in compliance across the entire globe, which is evidence of further fatigue. Vaccinations have not solved the problem of increasing COVID-19 case counts, but have been strongly associated with the falling mortality rate 47 , 48 . Lower mortality rates offer people the confidence to resume life as normal, making vaccines a likely motivator for people’s return to a semblance of normalcy. Nevertheless, vaccines are not the only plausible reason for further drops in compliance after 2020.

Economies are sustained by the working population, and eventually, countries had to reopen workplaces to keep their economies afloat. The effectiveness of policies aimed at reducing mobility was studied by regressing compliance on country-level variables. Literacy Rate was the most highly correlated variable that explained when the citizens of a country were able to maintain a high level of compliance. It can be inferred that with a high COVID-19 health literacy rate in a population, people tend to adopt better protective measures. The second most significant variable is the income support index. To ensure that citizens have enough resources to stay at home, income support is clearly a recommended mechanism, since citizens are more likely to comply with stay-at-home directives if they have an alternative means of making ends meet. Median age and the log of GDP per capita are also well-featured values, supporting the assertions that older populations are more compliant due to their vulnerability 49 , and that countries need to have adequate wealth in order to offer resources to sustain workers and families at home.

It currently takes at least a year to fully develop and test a vaccine to meet international standards 50 . As such, in the event of any new pandemic, policies implemented in the pre-vaccination period are critical for suppressing the spread of infections. Determining how citizens respond to different policies until a more permanent solution can be developed and applied at a national or global scale is therefore of great importance.

There are two major considerations for such policies: the first is the effectiveness of the policy in controlling the spread of the disease, and the second is the effect of the policy on the livelihoods of people. Any extended negative impact on the livelihoods of people will weaken the effectiveness of a policy due to fatigue and eventually a lack of compliance.

Our study using information about the stringency of a particular policy and the number of cases for each day and country allowed for the quantification of the impact over different temporal horizons. It provided a comparison of the impact of various policies and the horizon over which they take maximum effect. It is evident that most policies require well over 20 days to yield any effect, and that the impact is marginal for some of these policies. It is also worth noting that it takes at least 12 days for any single policy to effectively contribute to a decline in case counts as seen in the case of Gathering Restrictions and Closure of Public Transport (Fig.  6 ).

Facial coverings have the highest impact at 8.8% and play an important role in reducing the number of COVID-19 cases within a period of approximately one month, while also being the most cost-effective method, as confirmed by Mitze et al. 19 . Gathering restrictions are the most useful for achieving a short-term impact of 5.9%. Workplace Closures, Cancellation of Public Events, Stay Home Requirements, School Closures and Internal Movement Restrictions all operate over a horizon of around 25 days and had decreasing impacts of 4.5%, 3.4%, 3.1%, 2.1% and 1.9% respectively. Closure of Public Transport was also short-term at 12 days but delivered a small impact of 1.0%. Both public information campaigns and international travel controls are found to deliver negligible impact and are therefore difficult to justify based on the global evidence from this study. It is therefore recommended that facial coverings are introduced immediately when a new airborne pandemic emerges as this is both an effective and relatively cheap policy with no adverse effects on mobility or economic growth. School closures had a relatively small impact of 2.1% and it is recommended to focus on the most impactful policies before resorting to this restriction which can have serious long-term effects on the education of children. For example, school closures in Uganda lasted for 82 weeks 45 adversely affecting many students. The minor impact of school closures was previously echoed in the findings of the heavily cited study by Viner et al. 51 which concluded that “school closures alone would prevent only 2–4% of deaths, much less than other social distancing interventions”.

The greatest influence on compliance is the literacy rate, followed by income support. Both variables are selected in backward stepwise regression at a significance level of p < 0.05 (p < 0.001 for literacy rate), with an adjusted R-squared value of 29.3%. This finding suggests that these variables are statistically significant in explaining the compliant behaviour of those citizens that stayed at home under tight stringency measures.

A cross-validation evaluation was used to test for overfitting and establish the robustness of the results. The objective here is to train and test on different samples, thereby ensuring that the evaluation is fully out-of-sample. This was achieved using leave-one-out cross-validation, whereby one country is left out in each regression fold. This approach yielded a cross-validated R-squared value of 26.2%. As this value is close to our initial adjusted R-squared value of 29.3% we can safely conclude that overfitting is not an issue and therefore confirm a high level of confidence in our analyses.

Conclusions

In this paper we have empirically analysed the effects of COVID-19 case counts and deaths on mobility. We observed how increased numbers of deaths increased the mortality ratio in the pre-vaccination period leading to lockdowns and other high stringency measures, causing people to stay at home in line with policies. We also observed the effects of policy fatigue as compliance waned with every reduction in the number of deaths.

With the variants of concern observed in this paper, we see a trend where a new variant leads to a spike either in deaths or in cases (the former being more seriously impactful). With this increase comes more reactive stringency measures from governments, leading to compliance for as long as the wave lasts. By the end of each wave, measures are relaxed and life returns to some semblance of normalcy. With each new variant, however, people appear to have become less risk-averse, especially with the downward trends of the mortality ratio and the successful administration of vaccinations. Effectively, based on a multiplicity of factors, compliance has reduced despite the emergence of new variants.

We came to the conclusion, in line with existing research 17 , 18 , that face coverings are the most effective intervention as well as the most cost-effective, associated with the highest reductions by percentage in the number of COVID-19 cases after approximately a month.

Recommendations

Based on the relative efficacy of masks, we recommend the mandatory use of face masks as the best first step in the pre-vaccination stage of any airborne disease. Gathering restrictions are also advisable in the short term, achieving maximum impact over a short period of just 12 days. Workplace closures, cancellation of public events and stay home requirements are also advisable in countries with a large digital economy. Where there is the ability to work remotely and organize events online without having to leave home, there is an opportunity to curb the effects of the pandemic without delivering a crippling shock to the economy. In countries with low levels of digitization and poor internet penetration, the options may be limited and it is necessary to carefully balance public health with economic considerations. There already exists evidence that COVID-19 has increased global income inequality, partly undoing two decades of progress in lowering inequality and disproportionately affecting vulnerable groups and Emerging Markets and Developing Economies (EMDEs), where income inequality is considerably higher than in advanced economies 52 .

We do not advise school closures except as a last resort because of the terrible implications for students and their future 53 , with further evidence in the case of Uganda 45 , and the particularly harsh effects school closure has on girl-child education 54 . Restrictions on internal motion and public transport are likely to have an effect on the economy because of how critical mobility is to business. Public information campaigns have been so consistent since the start of the pandemic that they no longer offer any explainability. International travel restrictions are only useful in the very early days of a pandemic, but our analysis suggests that once cases are recorded internally in several countries, they become unnecessary as a mitigation measure.

Policies for managing a pandemic are only as effective as the citizens are cooperative, meaning that governments should put measures in place to keep fatigue at a minimum and avoid risking the livelihoods of citizens for extended periods of time. Income support and GDP per capita are strong indicators of likely compliance. Essentially, the financial standing of a nation explains its ability to keep citizens comfortable enough to abide with restrictions. If income support is not an option based on available finances, lockdowns will likely fail, and it is more advisable for poorer nations to enforce the use of masks and implement other social distancing measures.

Limitations

The data available only permitted us to calculate compliance based on mobility. While we know what policies are being rolled out by governments we have no way of knowing how people comply with non-mobility related policies, like the wearing of face coverings.

Another limitation was accurate information about variants. The information on emergence of variants often lacked temporal accuracy due to few countries having the required technical capacity to detect variants. This implies that reasonable assumptions have to be made about when variants were actually first discovered. There is also a lack of information on the percentages of variants circulating at any point in time, forcing us to disregard the possibility that certain previous variants might still be circulating with significant impact.

We also encountered a lack of data on the monetary valuation of policy implementation costs. A lot of the existing analysis focuses on the social and wellness costs of policies. Unfortunately, without any explicit financial data on the spend for policy implementations, there is no practical way of undertaking an extensive cost–benefit analysis across the globe to understand which policies make the most financial sense, and what the actual monetary cost is when implementing such a policy.

Data availability

The datasets analysed during the current study are available in the Oxford COVID-19 government response tracker (OxCGRT) repository, https://ourworldindata.org/coronavirus . The Google Mobility Trends datasets can be accessed from Google’s COVID-19 Community Mobility Reports, https://www.google.com/covid19/mobility/ . Finally, the data used to estimate the emergence of COVID-19 variants was found on the Word Health Organization’s official web page, https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/ .

Hadi, A., Kadhom, M., Hairunisa, N., Yousif, E. & Mohammed, S. A review on COVID-19: Origin, spread, symptoms, treatment, and prevention. Biointerface Res. Appl. Chem. 10 , 7234–7242 (2020).

Article   CAS   Google Scholar  

Cucinotta, D. and Vanelli, M. WHO declares COVID-19 a pandemic. PubMed . https://pubmed.ncbi.nlm.nih.gov/32191675/ (Accessed 1 March 2022).

Cameron-Blake, E. et al. Variation in the response to COVID-19 across the four nations of the United Kingdom. Blavatnik School of Government, University of Oxford. https://www.bsg.ox.ac.uk/sites/default/files/2020-10/BSG-WP-2020-035-v1_0.pdf (Accessed 8 March 2022).

Hale, T., Webster, S., Petherick, A., Phillips, T. and Kira, B. Oxford COVID-19 government response tracker (OxCGRT). Blavatnik School of Government, University of Oxford . https://www.bsg.ox.ac.uk/research/research-projects/covid-19-government-response-tracker (Accessed 8 March 2022).

Hallas, L., Hatibie, A., Majumdar, S., Pyarali, M. and Hale, T. Variation in US states' responses to COVID-19. University of Oxford . https://www.bsg.ox.ac.uk/research/publications/variation-us-states-responses-covid-19 (Accessed 8 March 2022).

Cameron-Blake, E. et al. What have we learned from tracking every government policy on COVID-19 for the past two years? Blavatnik School of Government, University of Oxford . https://www.bsg.ox.ac.uk/research/publications/what-have-we-learned-tracking-every-government-policy-covid-19-past-two-years (Accessed 1 September 2022).

Desvars-Larrive, A. et al. A structured open dataset of government interventions in response to COVID-19. Sci. Data 7 , 285 (2020).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fore, H. A wake-up call: COVID-19 and its impact on children’s health and wellbeing. Lancet Glob. Health 8 , e861–e862 (2020).

Article   PubMed   PubMed Central   Google Scholar  

Center for Disease Control and Prevention (CDC) . https://www.cdc.gov/nonpharmaceutical-interventions/index.html (Accessed 1 March 2022).

Hale, T. et al. Global assessment of the relationship between government response measures and COVID-19 deaths. https://doi.org/10.1101/2020.07.04.20145334v1 (2020).

Hale, T. et al. Government responses and COVID-19 deaths: Global evidence across multiple pandemic waves. PLoS ONE 16 , e0253116 (2021).

Jena, P., Majhi, R., Kalli, R., Managi, S. & Majhi, B. Impact of COVID-19 on GDP of major economies: Application of the artificial neural network forecaster. Econ. Anal. Policy 69 , 324–339 (2021).

Article   Google Scholar  

Makridis, C. and Hartley, J. The cost of COVID-19: A rough estimate of the 2020 US GDP impact. SSRN Electron. J. Special Edition Policy Brief . https://doi.org/10.2139/ssrn.3570731 (2020).

Wren-Lewis, S. The economic effects of a pandemic in Economics in the time of COVID-19 . 109–112 (Centre for Economic Policy Research press, 2020).

Berger, L. et al. Rational policymaking during a pandemic. in Proceedings of the National Academy of Sciences , 118 (2021).

Thunström, L., Newbold, S. C., Finnoff, D., Ashworth, M. & Shogren, J. F. The benefits and costs of using social distancing to flatten the curve for COVID-19. J. Benefit-Cost Anal . 11 , 179–195. https://doi.org/10.1017/bca.2020.12 (Cambridge University Press, 2020).

Van der Westhuizen, H., Kotze, K., Tonkin-Crine, S., Gobat, N. and Greenhalgh, T. Face coverings for covid-19: from medical intervention to social practice. BMJ m3021 (2020).

Howard, J. et al . An evidence review of face masks against COVID-19. In Proceedings of the National Academy of Sciences 118 , (2021).

Mitze, T., Kosfeld, R., Rode, J. & Wälde, K. Face masks considerably reduce COVID-19 cases in Germany. Proc. Natl. Acad. Sci. 117 , 32293–32301 (2020).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Corpuz, J. Adapting to the culture of ‘new normal’: An emerging response to COVID-19. J. Public Health 43 , e344–e345 (2021).

Bonacini, L., Gallo, G. & Scicchitano, S. Working from home and income inequality: Risks of a ‘new normal’ with COVID-19. J. Popul. Econ. 34 , 303–360 (2020).

Thoradeniya, T. & Jayasinghe, S. COVID-19 and future pandemics: A global systems approach and relevance to SDGs. Globalization Health. 17 , 59. https://doi.org/10.1186/s12992-021-00711-6 (2021).

Behl, A. et al. Threat, challenges, and preparedness for future pandemics: A descriptive review of phylogenetic analysis based predictions. Infect. Genet. Evol. 98 , 105217 (2022).

Article   CAS   PubMed   Google Scholar  

Azman, A. & Luquero, F. From China: hope and lessons for COVID-19 control. Lancet. Infect. Dis 20 , 756–757 (2020).

Karim, N. et al. Lessons learned from Rwanda: innovative strategies for prevention and containment of COVID-19. Ann. Glob. Health. 87 , 23. https://doi.org/10.5334/aogh.3172 (2021).

The Organisation for Economic Co-operation and Development (OECD). First lessons from government evaluations of COVID-19 responses: a synthesis. https://www.oecd.org/coronavirus/policy-responses/first-lessons-from-government-evaluations-of-covid-19-responses-a-synthesis-483507d6/#biblio-d1e4920 (Accessed 2 September 2022).

Hale, T. et al. Pandemic governance requires understanding socioeconomic variation in government and citizen responses to COVID-19. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3641927 (Accessed 2 March 2022).

Dergiades, T., Milas, C., Mossialos, E. & Panagiotidis, T. Effectiveness of government policies in response to the first COVID-19 outbreak. PLOS Glob. Public Health 2 , 1–19 (2022).

Ritchie, H. et al. Coronavirus pandemic (COVID-19). https://ourworldindata.org/coronavirus (Accessed 2 February 2022).

Hale, T. et al. A global panel database of pandemic policies (Oxford COVID-19 government response tracker). Nat. Hum. Behav. 5 , 529–538 (2021).

Article   PubMed   Google Scholar  

Raftery, A. E., Currie, J., Bassett, M. T. & Groves, R. Evaluating data types: A guide for decision makers using data to understand the extent and spread of COVID-19. Societal Experts Action Network (SEAN), The National Academies of Sciences, Engineering, and Medicine . https://doi.org/10.17226/25826 (2020).

Google. COVID-19 community mobility report. https://www.google.com/covid19/mobility/ (Accessed 2 February 2022).

Yilmazkuday, H. Stay-at-home works to fight against COVID-19: International evidence from Google mobility data. J. Hum. Behav. Soc. Environ. 31 , 210–220 (2021).

The World Health Organization (WHO). Tracking SARS-CoV-2 variants. https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/ (Accessed 2 March 2022).

British Broadcasting Corporation (BBC). Coronavirus: WHO advises to wear masks in public areas. https://www.bbc.com/news/health-52945210 (Accessed 2 March 2022).

Roser, M. and Ortiz-Ospina, E. Literacy. https://ourworldindata.org/literacy (Accessed 18 May 2022).

Herre, B. and Roser, M. Human rights. https://ourworldindata.org/human-rights (Accessed 18 May 2022).

Ortiz-Ospina, E. and Roser, M. Corruption. https://ourworldindata.org/corruption (Accessed 18 May 2022).

Roser, M. Human development index (HDI). https://ourworldindata.org/human-development-index (Accessed 18 May 2022).

Adeleke, O. & McSharry, P. E. Female enrollment, child mortality and corruption are good predictors of a country’s UN education index. Int. J. Educ. Dev. 90 , 102561 (2022).

Petherick, A. et al. A worldwide assessment of changes in adherence to COVID-19 protective behaviours and hypothesized pandemic fatigue. Nat. Hum. Behav. 5 , 1145–1160 (2021).

British Broadcasting Corporation (BBC). Coronavirus: The world in lockdown in maps and charts. BBC News. https://www.bbc.com/news/world-52103747 (Accessed 2 March 2022).

Mohapatra, R. K. et al. Omicron (B. 1.1. 529) variant of SARS-CoV-2: Concerns, challenges, and recent updates. J. Med. Virol. 94 , 2336–2342. https://doi.org/10.1002/jmv.27633 (2022).

Ganslmeier, M., Van Parys, J. & Vlandas, T. Compliance with the first UK COVID-19 lockdown and the compounding effects of weather. Sci. Rep. 12 , 3821 (2022).

Olaberria, E. and Reinhart, C. The reversal problem: development going backwards. World Bank Blogs. https://blogs.worldbank.org/developmenttalk/reversal-problem-development-going-backwards (Accessed 6 July 2022).

Rodon, C., Chin, J. & Chevalier, A. Assessing COVID-19 health literacy (CoHL) and its relationships with sociodemographic features, locus of control and compliance with social distancing rules during the first lockdown in France. Health Educ. Res. 37 , 143–154 (2022).

Ledford, H., Cyranoski, D. & Van Noorden, R. The UK has approved a COVID vaccine—Here’s what scientists now want to know. Nature 588 , 205–206 (2020).

Article   ADS   CAS   PubMed   Google Scholar  

Callaway, E. COVID vaccine excitement builds as Moderna reports third positive result. Nature 587 , 337–339 (2020).

De Pue, S. et al. The impact of the COVID-19 pandemic on wellbeing and cognitive functioning of older adults. Sci. Rep. 11 , 4636 (2021).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Cleve, M. What the lightning-fast quest for COVID vaccines means for other diseases. Nature 589 , 16–18 (2021).

Viner, R. M. et al. School closure and management practices during coronavirus outbreaks including covid-19: A rapid systematic review. Lancet Child Adolesc. Health 4 , 397–404 (2020).

World Bank. Global income inequality and the COVID-19 pandemic in three charts. World Bank Blogs . https://blogs.worldbank.org/developmenttalk/global-income-inequality-and-covid-19-pandemic-three-charts (2022).

Hammerstein, S., König, C., Dreisörner, T. & Frey, A. Effects of COVID-19-related school closures on student achievement—A systematic review. Front. Psychol. 12 , 746289. https://doi.org/10.3389/fpsyg.2021.746289 (2021).

Burzynska, K. & Contreras, G. Gendered effects of school closures during the COVID-19 pandemic. Lancet 395 , 1968 (2020).

Download references

Author information

Authors and affiliations.

Carnegie Mellon University Africa, Kigali, Rwanda

Kwadwo Agyapon-Ntra & Patrick E. McSharry

African Centre of Excellence in Data Science, University of Rwanda, Kigali, Rwanda

Patrick E. McSharry

Oxford MAN Institute of Quantitative Finance, University of Oxford, Oxford, UK

You can also search for this author in PubMed   Google Scholar

Contributions

K.A.N. and P.E.M. designed the experiments, analysed the results, and reviewed the manuscript. K.A.N. implemented the experiments in code, wrote the manuscript text, and prepared the figures.

Corresponding authors

Correspondence to Kwadwo Agyapon-Ntra or Patrick E. McSharry .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Agyapon-Ntra, K., McSharry, P.E. A global analysis of the effectiveness of policy responses to COVID-19. Sci Rep 13 , 5629 (2023). https://doi.org/10.1038/s41598-023-31709-2

Download citation

Received : 15 September 2022

Accepted : 16 March 2023

Published : 06 April 2023

DOI : https://doi.org/10.1038/s41598-023-31709-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Efficacy and safety of risankizumab for active psoriatic arthritis: 100-week results from the keepsake 2 randomized clinical trial.

• Andrew Östör
• Filip Van den Bosch
• Alan Kivitz

Rheumatology and Therapy (2024)

Efficacy and Safety of Risankizumab for Active Psoriatic Arthritis: 100-Week Results from the Phase 3 KEEPsAKE 1 Randomized Clinical Trial

• Lars Erik Kristensen
• Mauro Keiserman
• Frank Behrens

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

The impact and management of internet-based public opinion dissemination during emergencies: A case study of Baidu News during the first wave of coronavirus disease 2019 (COVID-19)

Affiliation.

• 1 School of Business Administration, Shandong University of Finance and Economics, Jinan, Shandong Province, China.
• PMID: 38573976
• PMCID: PMC10994342
• DOI: 10.1371/journal.pone.0299374

Background and aims: The coronavirus disease 2019 (COVID-19) public health emergency has had a huge impact worldwide. We analyzed news headlines and keywords from the initial period of COVID-19, and explored the dissemination timeline of news related to the epidemic, and the impact of Internet-based media on the public using lifecycle theory and agenda-setting theory. We aimed to explore the impact of Baidu news headlines on public attention during the first wave of COVID-19, as well as the management mechanism of regulatory departments for social public opinion.

Methods: We searched Baidu News using the keywords "Novel Coronavirus" and "COVID-19" from 8 January to 21 February 2020, a total of 45 days, and used Python V3.6 to extract news samples during the first wave of the epidemic. We used text analysis software to structurally process captured news topics and content summaries, applied VOSviewer V6.19 and Ucinet V6.0 to examine key aspects of the data.

Results: We analyzed the impact of Baidu News headlines on social opinion during the first wave of COVID-19 in the budding, spread, and outbreak stage of the information lifecycle. From clustering visualization and social network analysis perspectives, we explored the characteristics of Baidu News during the initial stage of the COVID-19. The results indicated that agenda-setting coverage through online media helped to mitigate the negative impact of COVID-19. The findings revealed that news reporting generated a high level of public attention toward a specific emergency event.

Conclusions: The public requires accurate and objective information on the progress of COVID-19 through Baidu News headlines to inform their planning for the epidemic. Meanwhile, government can enhance the management mechanism of news dissemination, correct false and inaccurate news, and guide public opinion in a positive direction. In addition, timely official announcements on the progress of the COVID-19 outbreak and responses to matters of public concern can help calm tensions and maintain social stability.

Copyright: © 2024 Su, Wang. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Grants and funding

Impact of aerosol concentration changes on carbon sequestration potential of rice in a temperate monsoon climate zone during the COVID-19: a case study on the Sanjiang Plain, China

• Research Article
• Published: 06 April 2024

Cite this article

• Xiaokang Zuo 1 &
• Hanxi Wang   ORCID: orcid.org/0000-0003-4130-6981 1 , 2  

The emission reduction of atmospheric pollutants during the COVID-19 caused the change in aerosol concentration. However, there is a lack of research on the impact of changes in aerosol concentration on carbon sequestration potential. To reveal the impact mechanism of aerosols on rice carbon sequestration, the spatial differentiation characteristics of aerosol optical depth (AOD), gross primary productivity (GPP), net primary productivity (NPP), leaf area index (LAI), fraction of absorbed photosynthetically active radiation (FPAR), and meteorological factors were compared in the Sanjiang Plain. Pearson correlation analysis and geographic detector were used to analyze the main driving factors affecting the spatial heterogeneity of GPP and NPP. The study showed that the spatial distribution pattern of AOD in the rice-growing area during the epidemic was gradually decreasing from northeast to southwest with an overall decrease of 29.76%. Under the synergistic effect of multiple driving factors, both GPP and NPP increased by more than 5.0%, and the carbon sequestration capacity was improved. LAI and FPAR were the main driving factors for the spatial differentiation of rice GPP and NPP during the epidemic, followed by potential evapotranspiration and AOD. All interaction detection results showed a double-factor enhancement, which indicated that the effects of atmospheric environmental changes on rice primary productivity were the synergistic effect result of multiple factors, and AOD was the key factor that indirectly affected rice primary productivity. The synergistic effects between aerosol-radiation-meteorological factor-rice primary productivity in a typical temperate monsoon climate zone suitable for rice growth were studied, and the effects of changes in aerosol concentration on carbon sequestration potential were analyzed. The study can provide important references for the assessment of carbon sequestration potential in this climate zone.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

Data used in this research are available upon request from the corresponding author.

Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/a:1010933404324

Article   Google Scholar  

Cao S, Sanchez-Azofeifa GA, Duran SM, Calvo-Rodriguez S (2016) Estimation of aboveground net primary productivity in secondary tropical dry forests using the Carnegie–Ames–Stanford approach (CASA) model. Environ Res Lett 11(7):075004. https://doi.org/10.1029/2007JG000603

Article   CAS   Google Scholar  

Chandra MA, Bedi SS (2021) Survey on SVM and their application in image classification. Int J Inform Technol 13:1–11. https://doi.org/10.1007/s41870-017-0080-1

Chang Q, Xiao X, Jiao W, Wu X, Doughty R, Wang J, Qin Y (2019) Assessing consistency of spring phenology of snow-covered forests as estimated by vegetation indices, gross primary production, and solar-induced chlorophyll fluorescence. Agr Forest Meteorol 275:305–316. https://doi.org/10.1016/j.agrformet.2019.06.002

Chen J, He L, Wen Z, Liao H, Wang B, Cui L, Li G (2017) Carbon sequestration potential of reed swamp wetland vegetation in the estuary of the Liaohe Delta. J Ecol 37(16):5402–5410. https://doi.org/10.5846/stxb201605241004

Cheng Y, Yu Q, Liu J, Cao X, Zhong Y, Du Z, Liang L, Geng G, Ma W, Qi H, Zhang Q, He K (2021) Dramatic changes in Harbin aerosol during 2018–2020: the roles of open burning policy and secondary aerosol formation. Atmo Chem Phys 21(19):15199–15211. https://doi.org/10.5194/acp-21-15199-2021

Chipman HA, George EI, McCulloch RE (1998) Bayesian CART model search. J Am Stat Assoc 93(443):935–948. https://doi.org/10.1080/01621459.1998.10473750

Duan J, Ju T, Wang Q, Li F, Fan J, Huang R, Liang Z, Zhang G, Geng T (2021) Absorbable aerosols based on OMI data: a case study in three provinces of Northeast China. Environ Monit Assess 193:479. https://doi.org/10.1007/s10661-021-09249-x

Ezhova E, Ylivinkka I, Kuusk J, Komsaare K, Vana M, Krasnova A, Noe S, Arshinov M, Belan B, Park S, Lavrič JV, Heimann M, Petäjä T, Vesala T, Mammarella I, Kolari P, Bäck J, Rannik Ü, Kerminen V, Kulmala M (2018) Direct effect of aerosols on solar radiation and gross primary production in boreal and hemiboreal forests. Atmos Chem Phys 18(24):17863–17881. https://doi.org/10.5194/acp-18-17863-2018

Fan J, Wang Y, Rosenfeld D, Liu X (2016) Review of aerosol–cloud interactions: mechanisms, significance, and challenges. J Atmos Sci 73(11):4221–4252. https://doi.org/10.1175/JAS-D-16-0037.1

Flamant C, Gaetani M, Chaboureau JP, Chazette P, Cuesta J, Piketh SJ, Formenti P (2022) Smoke in the river: an aerosols, radiation, and clouds in southern Africa (AEROCLO-SA) case study. Atmos Chem Physi 22(8):5701–5724. https://doi.org/10.5194/acp-22-5701-2022

Gao M (2020) Environmental effect condition (air temperature) of aerosols on gross primary productivity of vegetation. Int J Ecol 9(2):210–222. https://doi.org/10.12677/IJE.2020.92027

Gao X, Gu F, Mei X, Hao W, Li H, Gong D, Li X (2018) Light and water use efficiency as influenced by clouds and/or aerosols in a rainfed spring maize cropland on the loess plateau. Crop Sci 58(2):853–862. https://doi.org/10.2135/cropsci2017.06.0341

Ge W, Deng L, Wang F, Han J (2021) Quantifying the contributions of human activities and climate change to vegetation net primary productivity dynamics in China from 2001 to 2016. Sci Total Environ 773:145648. https://doi.org/10.1016/j.scitotenv.2021.145648

Greenwald R, Bergin MH, Xu J, Cohan D, Hoogenboom G, Chameides WL (2006) The influence of aerosols on crop production: a study using the CERES crop model. Agr Syst 89(2-3):390–413. https://doi.org/10.1016/j.agsy.2005.10.004

Gu L, Baldocchi DD, Wofsy SC, Munger JW, Michalsky JJ, Urbanski SP, Boden TA (2003) Response of a deciduous forest to the Mount Pinatubo eruption: enhanced photosynthesis. Science 299(5615):2035–2038. https://doi.org/10.1126/science.1078366

Haywood JM, Abel SJ, Barrett PA, Bellouin N, Blyth A, Bower KN, Brooks M, Carslaw K, Che HC, Coe H, Cotterell MI, Crawford I, Cui Z, Davies N, Dingley B, Field P, Formenti P, Gordon H, Graaf MD et al (2021) The cloud–aerosol–radiation interaction and forcing: the year 2017 (CLARIFY-2017) measurement campaign. Atmos Chem Phys 21(2):1049–1084. https://doi.org/10.5194/acp-21-1049-2021

Ho T (1998) The random subspace method for constructing decision forests. IEEE Tran Pattern Anal Mach Intell 20(8):832–844. https://doi.org/10.1109/34.709601

Huang S, Cai N, Pacheco PP, Narrandes S, Wang Y, Xu W (2018) Applications of support vector machine (SVM) learning in cancer genomics. Cancer Genom Proteom 15(1):41–51. https://doi.org/10.21873/cgp.20063

Jiang L, Chen X, Zhu H (2021) Spatial distribution characteristics of urban nursing homes in China and their divergent causes. J Geogr 76(8):1951–1964. https://doi.org/10.11821/dlxb202108010

Jiang S, Huang Y, Zhao L, Cui N, Wang Y, Hu X, Zheng S, Zou Q, Feng Y, Guo L (2022) Effects of clouds and aerosols on ecosystem exchange, water and light use efficiency in a humid region orchard. Sci Total Environ 811:152377. https://doi.org/10.1016/j.scitotenv.2021.152377

Kang S, Hao X, Du T, Tong L, Su X, Lu H, Li X, Huo Z, Li S, Ding R (2017) Improving agricultural water productivity to ensure food security in China under changing environment: from research to practice. Agr Water Manag 179:5–17. https://doi.org/10.1016/j.agwat.2016.05.007

Kong X, Zhao J, Xu H, Xu J (2019) Assessment of atmospheric aerosol direct radiation effect on maize yield in China based on APSIM model. Chin J Ecol Agr 27(7):994–1003. https://doi.org/10.13930/j.cnki.cjea.181071

Kumar N, Middey A (2022) Interaction of aerosol with meteorological parameters and its effect on the cash crop in the Vidarbha region of Maharashtra, India. Int J Biometeorol 66(7):1473–1485. https://doi.org/10.1007/s00484-022-02296-0

Lau WKM, Kim KM, Leung LR (2017) Changing circulation structure and precipitation characteristics in Asian monsoon regions: greenhouse warming vs. aerosol effects. Geosci Lett 4:1–11. https://doi.org/10.1186/s40562-017-0094-3

Le TH, Wang Y, Liu L, Yang J, Yung YL, Li G, Seinfeld JH (2020) Unexpected air pollution with marked emission reductions during the COVID-19 outbreak in China. Science 369(6504):702–706. https://doi.org/10.1126/science.abb7431

Li B, Liu Z, Huang F, Yang XG, Liu ZJ, Wan W, Wang J, Xu Y, Li Z, Ren T (2021b) Ensuring national food security by strengthening high-productivity black soil granary in Northeast China. Bull Chin Acad Sci 36(10):1184–1193. https://doi.org/10.16418/j.issn.1000-3045.20210706003

Li M, Zhang R, Luo H, Gu S, Qin Z (2022) Crop mapping in the Sanjiang Plain using an improved object-oriented method based on google earth engine and combined growth period attributes. Remote Sens 14:273. https://doi.org/10.3390/rs14020273

Li X, Liang H, Cheng W (2021a) Evaluation and comparison of light use efficiency models for their sensitivity to the diffuse PAR fraction and aerosol loading in China. Int J Appl Earth Obs Geoinf 95:102269. https://doi.org/10.1016/j.jag.2020.102269

Li Y, Shiraiwa M (2019) Timescales of secondary organic aerosols to reach equilibrium at various temperatures and relative humidities. Atmos Chem Physi 19(9):5959–5971. https://doi.org/10.5194/acp-19-5959-2019

Liu X, Ning J, Dong F, Yu J, Du G, Kuang W (2017) Spatial-temporal variation characteristics of vegetation NPP of northern Sanjiang plain from 2000 to 2013. J Northeast Agr Univ 48(7):63–71. https://doi.org/10.19720/j.cnki.issn.1005-9369.2017.07.007

Liu X, Xu J, Yang S, Zhang J, Wang Y (2018) Vapor condensation in rice fields and its contribution to crop evapotranspiration in the subtropical monsoon climate of China. J Hydrometeorol 19(6):1043–1057. https://doi.org/10.1175/JHM-D-17-0201.1

Lv F, Deng L, Zhang Z, Wang Z, Wu Q, Qiao J (2022) Multiscale analysis of factors affecting food security in China, 1980–2017. Environ Sci Pollut Res 29(5):6511–6525. https://doi.org/10.1007/s11356-021-16125-1

Ma W, Ding J, Wang J, Zhang J (2022) Effects of aerosol on terrestrial gross primary productivity in Central Asia. Atmos Environ 288:119294. https://doi.org/10.1016/j.atmosenv.2022.119294

Miao C, He X, Gao Z, Chen W, He B (2023) Assessing the vertical synergies between outdoor thermal comfort and air quality in an urban street canyon based on field measurements. Build Environ 227(109810):109810. https://doi.org/10.1016/j.buildenv.2022.109810

Mo X, Chen X, Hu S, Liu S, Xia J (2017) Attributing regional trends of evapotranspiration and gross primary productivity with remote sensing: a case study in the North China Plain. Hydrol Earth Syst Sci 21(1):295–310. https://doi.org/10.5194/hess-21-295-2017

Moazenzadeh R, Mohammadi B, Shamshirband S, Chau KW (2018) Coupling a firefly algorithm with support vector regression to predict evaporation in northern Iran. Eng Appl Comput Fluid Mech 12(1):584–597. https://doi.org/10.1080/19942060.2018.1482476

Paul A, Mukherjee DP, Das P, Gangopadhyay A, Chintha AR, Kundu S (2018) Improved random forest for classification. IEEE Tran Image Process 27(8):4012–4024. https://doi.org/10.1109/TIP.2018.2834830

Pei Y, Dong J, Zhang Y, Yang J, Zhang Y, Jiang C, Xiao X (2020) Performance of four state-of-the-art GPP products (VPM, MOD17, BESS, and PML) for grasslands in drought years. Ecol Inform 56:101052. https://doi.org/10.1016/j.ecoinf.2020.101052

Ren Y, Wang C, Zhao Y (2010) Review on impact of atmospheric aerosol radiation effect on crops and ecological system. China Agr Weather 31(4):533–540. https://doi.org/10.3969/j.issn.1000-6362.2010.04.009

Rosenfeld D, Sherwood S, Wood R, Donner L (2014) Climate effects of aerosol-cloud interactions. Science 343(6169):379–380. https://doi.org/10.1126/science.1247490

Shu Y, Liu S, Wang Z, Xiao J, Shi Y, Peng X, Gao H, Wang Y, Yuan W, Yan W, Ning Y, Li Q (2022) Effects of aerosols on gross primary production from ecosystems to the globe. Remote Sens 14(12):2759. https://doi.org/10.3390/rs14122759

Singh P, Vaishya A, Rastogi S, Babu S (2020) Seasonal heterogeneity in aerosol optical properties over the subtropical humid region of northern India. J Atmos Solar-Terr Phy 201:105246. https://doi.org/10.1016/j.jastp.2020.105246

Sun Q, Lu C, Guo H, Yan L, He X, Wu C (2021) Impact of land use change on water balance in the Sanjiang Plain. Adv Water Sci 32(5):694–706. https://doi.org/10.14042/j.cnki.32.1309.2021.05.005

Sun Y, Wang Z, Fu P, Jiang Q, Yang T, Li J, Ge X (2013) The impact of relative humidity on aerosol composition and evolution processes during wintertime in Beijing, China. Atmos Environ 77:927–934. https://doi.org/10.1016/j.atmo-senv.2013.06.019

Sun Z, Wang X, Zhang X, Tani H, Guo E, Yin S, Zhang T (2019) Evaluating and comparing remote sensing terrestrial GPP models for their response to climate variability and CO 2 trends. Sci Total Environ 668:696–713. https://doi.org/10.1016/j.scitotenv.2019.03.025

Thorsen TJ, Ferrare RA, Kato S, Winker DM (2020) Aerosol direct radiative effect sensitivity analysis. J Clim 33(14):6119–6139. https://doi.org/10.1175/JCLI-D-19-0669.1

Tian J, Wang Q, Zhang Y, Yan M, Liu H, Zhang N, Ran W, Cao J (2021) Impacts of primary emissions and secondary aerosol formation on air pollution in an urban area of China during the COVID-19 lockdown. Environ Int 150:106426. https://doi.org/10.1016/j.envint.2021.106426

Tie X, Huang R, Dai W, Cao J, Long X, Su X, Zhao S, Wang Q, Li G (2016) Effect of heavy haze and aerosol pollution on rice and wheat productions in China. Sci Rep 6:29612. https://doi.org/10.1038/srep29612

Velavan TP, Meyer CG (2020) The COVID-19 epidemic. Trop Med Int Health 25(3):278. https://doi.org/10.1111/tmi.13383

Wang H, Liang H, Gao D (2017) Occurrence and risk assessment of phthalate esters (PAEs) in agricultural soils of the Sanjiang Plain, northeast China. Environ Sci Pollut Res 24:19723–19732. https://doi.org/10.1007/s11356-017-9646-5

Wang J, Xu C (2017) Geodetectors: principles and prospects. J Geogr 72(1):116–134. https://doi.org/10.11821/dlxb201701010

Wang W, He B (2023) Co-occurrence of urban heat and the COVID-19: impacts, drivers, methods, and implications for the post-pandemic era. Sustain Cities Soc 90:104387. https://doi.org/10.1016/j.scs.2022.104387

Wei S, Yi C, Fang W, Hendrey G (2017) A global study of GPP focusing on light-use efficiency in a random forest regression model. Ecosphere 8(5):e01724. https://doi.org/10.1002/ecs2.1724

Xiao Z, Miao Y, Zhu S, Yu Y, Du X, Che H (2022) Relationship between aerosol pollution and different types of precipitation in autumn and winter in North China. J Meteorol 80(6):986–998. https://doi.org/10.11676/qxxb2022.066

Xu Y, Lin L (2017) Pattern scaling based projections for precipitation and potential evapotranspiration: sensitivity to the composition of GHGs and aerosols forcing. Climatic Change 140(3-4):635–647. https://doi.org/10.1007/s10584-016-1879-7

Yang H, Zhong X, Deng S, Xu H (2021) Assessment of the impact of LUCC on NPP and its influencing factors in the Yangtze River basin, China. Catena 206:105542. https://doi.org/10.1016/j.catena.2021.105542

Zhang H, Zhang YD, Huang Y, Huan G, Bai W (2023) Study on water consumption and growth characteristics of rice under different irrigation modes. Water-saving Irrig 48(4):25–31. https://doi.org/10.12396/jsgg.2022339

Zhang L, Wang Z, Du G, Chen Z (2022) Analysis of the climatic basis for the change of cultivated land area in Sanjiang Plain of China. Front Earth Sci 10:590. https://doi.org/10.3389/feart.2022.862141

Zhang Y, Boucher O, Ciais P, Li L, Bellouin N (2020) How to reconstruct diffuse radiation scenario for simulating GPP in land surface models? Geosci Model Dev 2020:1–15. https://doi.org/10.5194/gmd-2020-267

Zhang Y, Li W, Zhu Q, Chen H, Fang X, Zhang T, Zhao P, Peng C (2015) Monitoring the impact of aerosol contamination on the drought-induced decline of gross primary productivity. Int J Appl Earth Obs Geoinf 36:30–40. https://doi.org/10.1016/j.jag.2014.11.006

Zhang Y, Xiao X, Wu X, Zhou S, Zhang G, Qin Y, Dong J (2017) A global moderate resolution dataset of gross primary production of vegetation for 2000–2016. Sci Data 4(1):1–13. https://doi.org/10.1038/sdata.2017.165

Zhang Z, Liu Q, Ruan Y, Tan Y (2021) Estimation of aerosol radiative effects on terrestrial gross primary productivity and water use efficiency using the process-based model and satellite data. Atmos Res 247:105245. https://doi.org/10.1016/j.atmosres.2020.105245

Zheng C, Zhao C, Zhu Y, Shi XQ, Wu XL, Chen TM, Wu F, Qiu YM (2017) Analysis of influential factors for the relationship between PM 2.5 and AOD in Beijing. Atmos Chem Phys 17(21):13473–13489. https://doi.org/10.5194/acp-17-13473-2017

Zhou H, Yue X, Lei Y, Tian C, Zhu J, Ma Y, Cao Y, Yin X, Zhang Z (2022) Distinguishing the impacts of natural and anthropogenic aerosols on global gross primary productivity through diffuse fertilization effect. Atmos Chem Phys 22(1):693–709. https://doi.org/10.5194/acp-22-693-2022

Zimmerman RK, Balasubramani GK, Norwalk MP, Eng H, Urbanski L, Jackson LA, Mclean HQ, Belongia EA, Monto AS, Malosh RE, Gaglani M, Clipper L, Flannery B, Wisniewski SR (2016) Classification and regression tree (CART) analysis to predict influenza in primary care patients. BMC Infect Dis 16(1):1–11. https://doi.org/10.1186/s12879-016-1839-x

Download references

Acknowledgements

The authors thank the reviewers for their valuable comments, and the authors thank the editor for his efforts in this paper.

This research was funded by the High-level Talent Foundation Project of Harbin Normal University (No. 1305123005).

Author information

Authors and affiliations.

Heilongjiang Province Key Laboratory of Geographical Environment Monitoring and Spatial Information Service in Cold Regions/School of Geographical Sciences, Harbin Normal University, Harbin, 150025, China

Xiaokang Zuo & Hanxi Wang

Heilongjiang Province Collaborative Innovation Center of Cold Region Ecological Safety, Harbin, 150025, China

You can also search for this author in PubMed   Google Scholar

Contributions

Xiaokang Zuo: Ms. Huang drafted the initial manuscript, analyzed data, reviewed the manuscript, and approved the final manuscript as submitted. Hanxi Wang: Mr. Wang conceptualized and designed the study, reviewed and revised the manuscript, and approved the final manuscript as submitted. All authors approved the final manuscript as submitted and agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Hanxi Wang .

Ethics declarations

Ethics approval.

Not applicable

Consent to participate

Consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Responsible Editor: Zhihong Xu

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

(DOCX 20 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Zuo, X., Wang, H. Impact of aerosol concentration changes on carbon sequestration potential of rice in a temperate monsoon climate zone during the COVID-19: a case study on the Sanjiang Plain, China. Environ Sci Pollut Res (2024). https://doi.org/10.1007/s11356-024-33149-5

Download citation

Received : 24 December 2023

Accepted : 26 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1007/s11356-024-33149-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Aerosol optical depth (AOD)
• Gross primary productivity (GPP)
• Net primary productivity (NPP)
• Driving factors
• Carbon sequestration
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

Evaluation and prediction of COVID-19 in India: A case study of worst hit states

In this manuscript, system modeling and identification techniques are applied in developing a prognostic yet deterministic model to forecast the spread of COVID-19 in India. The model is verified with the historical data and a forecast of the spread for 30-days is presented in the 10 most affected states of India. The major results suggest that our model can very well capture the disease variations with high accuracy. The results also show a steep rise in the total cumulative cases and deaths in the coming weeks.

1. Introduction

The advent and spread of 2019 novel coronavirus (SARS-CoV-2) has posed a global health crisis with a sharp rise in cases and deaths since its first detection in Wuhan, China, in December 2019. The infection causes illness ranging from common cold to extreme respiratory disease and death [1] . Currently, the prime epidemiological risk factor for 2019 novel coronavirus disease includes close contact with infected individuals with an incubation period of 2–14 days [2] . The case mortality rate is projected to range from 2 to 3% [3] . Various drugs are being assessed in line with previous researches into therapeutic treatments for SARS and MERS, however, there is no robust evidence for any significantly improved clinical outcome [4] . Apparent risk of acquiring the disease has led many governments to institute a variety of control procedures like quarantine, isolation and lock-down measures. Despite rigorous global containment measures, the frequency of the novel coronavirus disease continues to rise, with over 4.5 million confirmed cases and over 300,000 deaths worldwide as on 17 th May, 2020 [5] . Although countries around the world have enhanced capacity building of the laboratory systems and response procedures, yet, there is a need for proper disease surveillance systems. Comprehending the initial transmission of the virus and analyzing the effectiveness of control measures are crucial in assessing the prospects for continued transmission in newer locations. This necessitates tracking the course of the pandemic to be able to foresee its emergence for a better response.

Prospective studies on modeling and forecasting of the epidemic have been carried out to provide analytical predictions on the size and end phase of the spread. Wu et al. [6] have used a susceptible exposed infectious recovered (SEIR) meta-population model to simulate the epidemic across all major cities in China. Early dynamics of transmission and control of COVID-19 within and outside Wuhan has also been studied using a stochastic transmission dynamic model [7] . Another study used the SEIR compartmental model to predict the feasibility for conducting the summer Olympics of 2020 in Japan [8] . Similarly, Abdullah et al. [9] presented a stochastic SIR model to predict the spread of COVID-19 in Kuwait. A classical SEIR type mathematical model is also presented by Mandal et al. [10] to study the qualitative dynamics of COVID-19 in India. Further work has been carried out by Ndairou et al. [11] , with special focus on the transmissibility of super-spreader individuals in Wuhan, China.

Besides the above mentioned compartmental models, some other methods have been used to model and forecast the COVID-19 spread. For example, in Tomar and Gupta [12] , a data-driven estimation method like long short-term memory (LSTM) is used for the prediction of total number of COVID-19 cases in India for a 30-days ahead prediction window. In addition to this, global epidemic and mobility model (GLEAM), an agent-based mechanistic model has also been used for daily forcasts of COVID-19 activity [13] . Harun, et al. [14] have used Box-Jenkins (ARIMA) and Brown/Holt linear exponential smoothing methods to estimate and forecast the number of COVID-19 cases in the G8 countries. Furthermore, Al-qaness et al. [15] have incorporated a modified version of flower pollination algorithm (FPA) coupled with the salp swarm algorithm (SSA) to forecast the number of cases of COVID-19 for ten days in China.

As on 17 th May 2020, India has observed a total cases of 90,927 with 2872 deaths [16] , [17] . The very first case was reported on 30 th January 2020, in a coastal state of Kerela (southern India) when a student returned from Wuhan, China. Subsequently, the number of positive cases in India rose rapidly due to the arrival of many passengers via airways [18] . An overview of the spread of COVID-19 in India is shown in Fig. 1 . It can be easily seen that the virus has spread to entire country with the worst hit states being Maharashtra (30,706 cases), Gujarat (10,988), Tamil Nadu (10,588), Delhi (9333), Rajasthan (4960), and Madhya Pradesh (4789). Figs. 2 and ​ and3 show 3 show the trend of rising new cases and deaths in India.

Heat map of COVID-19 in Indian (as of 17 May 2020).

(Top:) cumulative cases in India till 17 May 2020, (bottom:) daily new cases till 17 May 2020.

(Top:) cumulative deaths in India till 17 May 2020, (bottom:) daily new deaths till 17 May 2020.

This manuscript demonstrates a control-theoretic, data-driven estimation technique to derive a time-series model from the historical data collected from [5] , [16] up-to 17 th May 2020. The model is then used for the prediction of the total number of cases and deaths in most affected states of India for the next 30 days. The paper is sectioned as follows: Section 2 describes the system identification method employed. Section 3 presents the predicted cases and deaths along-with some discussions. Finally, conclusions are presented in Section 4 .

2. Data driven forecasting of COVID-19 in India

To estimate the spread of COVID-19 in India, we used a predictive error minimization (PEM) based system identification technique to identify a discrete-time, single-input, single-output (SISO) model [19] , [20] , [21] . Different models were identified for different states based on the data collected. The models were then verified on the testing data and upon validation, the models were used to predict the total number of cases and deaths for the next 30-days in the 10 worst hit states in India.

2.1. Model development

The discrete-time, identified model can be realized in the state-space from given as:

where the y ( t ) represents total number of cases or deaths of a particular area which is proportional to system state vector x ( t ) ∈ R n , u ( t ) is the time series input and T s is the sampling interval. Here, the unknowns to be identified are A ∈ R n × n , K ∈ R n × 1 and C ∈ R 1 × n which are in canonical form. Also, n is the dimension of the state-space model.

The identification problem can thus be posed as to selecting a model set M ( θ ) (indexed by a finite dimensional parameter vector θ) and evaluating a member from the set which best describes the recorded input-output relation according to a given criterion. One such criteria is given by Ljung [22] which is defined as :

where ϵ ( t , θ ) = ( y 0 − y ^ 0 , … , y N − y ^ N ) is referred as the prediction error, l ( . ) is a scalar measure of fit, z ( t ) = [ y T ( t ) , u T ( t ) ] and N is length of data-set. Typical choices of l ( t , θ , ϵ ) can be seen in Ljung [22] .

The identified model thus minimizes the 1-step ahead prediction and the error ϵ ( t , θ ) between the measured y ( t ) and predicted values y ^ ( t ) is used to make the future prediction about the system. The prediction error identification estimate is thus given as:

Here, we have taken:

and the least-square problem has been solved iteratively via the Levenberg-Marquardt method [23] , [24] , [25] .

The choice of model structure and its size is of crucial importance as it dictates the quality of long-term prediction and parameter estimation. The selection of model size n was made on the basis of the decay of the Hankel singular values of the system (1) [26] , [27] .

3. Results and discussions

Fig. 4 , Fig. 5 , Fig. 6 , Fig. 7 , Fig. 8 , Fig. 9 , Fig. 10 , Fig. 11 , Fig. 12 , Fig. 13 show the dynamics of the forecasted response for the most infected states of India along-with a 10-step predicted response comparison with the validation data. Further results are presented in Table 1 . As seen from Table 1 , Maharashtra has recorded the highest number of COVID-19 cases accounting for 36% of the total country’s caseload. It has also witnessed the sharpest rise in COVID-19 deaths with Mumbai being the epicenter of the pandemic in India. The constant influx of tourists, reliance on public transportation and population destiny have cumulatively made the metropolitan city hospitable for corona virus. Even though the state is conducting more tests, the violation of physical distancing rules by individuals particularly in containment zones result in the mixing of infected with healthy population. Moreover, unlike other red zones of Maharashtra, Mumbai faces shortage of ICU beds and dedicated COVID-19 hospitals. According to the prediction made herein, it would be inevitable that Mumbai and its suburbs would continue to see an upsurge in the number of cases and deaths for at least up to 17 th June 2020.

(Top): 30-day prediction for number of cases in Maharashtra, (bottom): 30-day prediction for the number of deaths in Maharashtra. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Gujarat, (bottom): 30-day prediction for the number of deaths in Gujarat. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Tamil Nadu, (bottom): 30-day prediction for the number of deaths in Tamil Nadu. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Delhi, (bottom): 30-day prediction for the number of deaths in Delhi. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Rajasthan, (bottom): 30-day prediction for the number of deaths in Rajasthan. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Madhya Pradesh, (bottom): 30-day prediction for the number of deaths in Madhya Pradesh. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Uttar Pradesh, (bottom): 30-day prediction for the number of deaths in Uttar Pradesh. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Andhra Pradesh, (bottom): 30-day prediction for the number of deaths in Andhra Pradesh. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Punjab, (bottom): 30-day prediction for the number of deaths in Punjab. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

(Top): 30-day prediction for number of cases in Telangana, (bottom): 30-day prediction for the number of deaths in Telangana. Red line shows the start of prediction window, dark blue:  ± 3 std. deviation, light blue:  ± 5 std. deviation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

COVID-19 scenario in worst hit states of India upto 17 May 2020 along-with predicted values.

Gujarat has recorded the second highest COVID-19 mortality rate in the country in spite of reporting its first case as late as March 20. The COVID-19 mortality rate of Ahmedabad city is 6.8%, which is double the national average. Officials acknowledge that while Gujarat had its guard up sufficiently fast, there was a delay in testing. Even by mid of March, the daily average was as less as 15 tests per day, going up to 200/day by the end of March. According to the data driven identification scheme employed herein, the mortality rate in Gujarat may increase as high as 15.2% up to 17 th June 2020.

Tamil Nadu, although being the third worst hit Indian state in terms of COVID-19 cases has witnessed the least number of mortalities with 1 among 143 positive cases succumbing to the disease (see Fig. 6 ). This is attributed to its credibility as a trusted medical center of the country. Chennai has the highest medical tourism in India with the state’s average being above the national average in the health sector. This may be the reason that the predictable mortality rate of Tamil Nadu projected in this study is least among the rest of the states in consideration (see Table 1 ).

As per our prediction based on data up to 17th May 2020, Delhi along with other states would continue to see marginal surge in the number of COVID-19 cases owing to the relaxations in lock-down measures. The impact of removing the curbs will be more evident by the mid of June 2020. The under-funding of the healthcare system, paucity of testing labs, violations of the lock-down protocols and inadequate quarantine facilities arranged by states and union territories are the biggest hurdles in combating the spread.

4. Conclusions

The study concerns the spread of COVID-19 in India. A control-theoretic approach is used to develop an epidemic model to simulate and predict the disease variations in 10 most affected states of India. Results depict a rapid increase in the number of cases in the coming days. However, it is pertinent to mention that the future estimation provided, is subject to certain system parameters and can vary based on the external inputs like lock-down measures, social-distancing, vaccine/drug development, rapid testing, etc. Information provided by our model could help establish a realistic assessment of the situation for the time-being and in the near future in order to apply the appropriate public health measures.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The Doctoral fellowship of Author 1 and 2 from Ministry of Human Resource Development (MHRD/2017PHAELE006/009), New Delhi, India, is duly acknowledged. Author 1 would like to thank Asiya Batool for fruitful discussions.