phd thesis outcome measures

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Patient-Reported Outcome Measures—Challenges and Opportunities for China

• 1 Department of Internal Medicine, Rush Medical College, Rush University, Chicago, Illinois
• 2 Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, Pennsylvania
• 3 Institute for Healthcare Informatics, University of Minnesota, Minneapolis
• Original Investigation Application of Patient-Reported Outcome Measurements in Clinical Trials in China Hui Zhou, MSc; Mi Yao, MD; Xiaodan Gu, MBBS; Mingrui Liu, BP; Ruifeng Zeng, MBBS; Qin Li, MNS; Tingjia Chen, MD; Wen He, MD; Xiao Chen, PhD; Gang Yuan, MD, PhD JAMA Network Open

In 2016, The People’s Republic of China (PRC) formally passed the blueprint of Healthy China 2030, working toward the national goal of reaching a health standard on par with high-income countries by 2030. 1 In 2021, the Chinese government approved its 14th Five-Year Plan for National Economic and Social Development of the PRC, which includes a comprehensive strategy for advancing the quality of health care delivered by its national health system. 2 Yet, achieving this goal for China’s diverse population of 1.4 billion people is often complex, depending on employment, insurance type, geodemographic location, socioeconomic status, health care workforce supply, and many other variables. 3 In addition, PRC overall health care expenditures as a percentage of gross domestic product have increased by more than 42% since 2010, with the most currently available data showing 7.1% in 2020. 4

Over the past several years, patient-reported outcomes (PROs) and derivative standardized patient-reported outcome measures (PROMs) have come into widespread use in other countries, including use, for example, as part of industry-sponsored clinical trials, population outcomes and comparative effectiveness research, health care delivery system program evaluation, and health insurance coverage determinations. 5 PROs are intended to provide objective and subjective assessments of a variety of dimensions, for example, health-related quality of life (eg, I do not socialize with friends much anymore), physical capacity (eg, I have difficulty walking 3 city blocks), mental and cognitive changes (eg, I sometimes have trouble concentrating), functional status (eg, I am unable to lift more than 5 pounds on the job), symptoms (eg, I experience moderate pain on most days), and overall well-being (eg, I am in poor health). Data on PROs are usually collected via standardized, psychometrically developed, and validated survey-type instruments that are often used by clinicians and researchers to evaluate health care delivery from the perspectives of individual patients. A PROM is often then generated, which is most typically a summary composite score of the individual PRO item response scores captured by the survey instrument.

Using a cross-sectional survey of interventional clinical trials using data from the Chinese Clinical Trial Registry and the ClinicalTrials.gov databases, Chinese researchers have evaluated the current applications of PROMs in clinical trials in the PRC. 6 As the authors note, this study documents the major increase from 2010 to 2020 in the number of clinical trials originating in the PRC that include the application and characteristics of PRO instruments and PROMs as primary and secondary outcomes in clinical trials across China. Only 29.7% of the selected 10 093 eligible PRO-related trials were categorized according to those that precisely listed PRO tools as outcomes, and 70.3% did not incorporate PROMs into the analyses. Also documented was a striking use imbalance by regional provincial locations, sponsors, clinical phases, and a relative lack of diversity of PROMs deployed. Most trials were in phase 4, performed in hospitals, and located in the most populous eastern Chinese provinces. The authors accurately conclude that there is a need for more widespread, robust, and correctly targeted use of standardized PROMs in clinical trials across the PRC.

In the important context of achieving its goals outlined in Healthy China 2030, the PRC has been aggressively evaluating the new improvements to the Chinese health care delivery system. Much is required to generate better quality of evidence and measurement of cost-effectiveness for guideline-directed medical therapies indicated for major chronic conditions and other, less common diseases. As such, the discovery of new insights reported by Zhou et al 6 requires more widespread and consistent deployment of well-constructed PROMs that provide the best understanding of which PROs matter most to patients and other subsequent relevant stakeholders interested in the resultant data.

The benefit of this well-done study by Zhou et al 6 is that it exposes both challenges and opportunities for the all-important future of PROM development and implementation—not just for the PRC, but for the entire international field. Fortunately, several reliable and publicly available resources and repositories for PROMs now exist widely, such as Patient-Reported Outcomes Measurement Information System, Patient-Reported Outcomes and Quality-of-Life Instrument database, and Online Guide to Quality-of-Life Assessment. 5 , 6 The joint initiative between the Consensus-Based Standards for the Selection of Health Measurement Instruments initiative and the Core Outcome Measures in Effectiveness Trials initiative helps facilitate agreement among PROMs-focused stakeholders with regard to the selection of outcomes (eg, constructs or domains) and outcome measurement instruments for clinical trials. 7

Measure development and evaluation standards are also in use by consensus-driven groups (eg, the National Quality Forum) for the various and complex design domains of true PROMs, including content validity (including face validity), structural validity, internal consistency, reliability, measurement error, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness. Initial PROM development and subsequent evaluation should include a formal evidence review and standardized quality of evidence grading process for each PROM item property, taking into account the number of studies, the methodologic quality of the studies, and the consistency of the results of the measurement properties. 5 Several sophisticated statistical-based methods are also often required, such as systematic review, meta-analysis, interrater reliability, internal consistency, factor analysis, analysis of variance, bayesian estimation, risk adjustment, and exclusion and missing data analysis. From empirical data generated through PROMs, it is now also becoming important to determine causal relationships between changes in PROs to generalizable improvements in at least 1 health care delivery system structure, process, intervention, or service.

Data for PROMs generated through traditional methods, such as self-reported questionnaires, structured interviews, and clinical assessments during patient encounters, is rapidly moving toward electronic data capture and storage in patient records. PROM data are now often collected through digital health interfaces that automate accurate and complete response capture and interoperable data transmission to electronic health records, clinical registries, and data analytic platforms. Translations of country- and language-specific PRO instruments may require additional psychometric testing that is appropriately sensitive to racial, ethnic, and cultural nuances of different target populations. As such, the generation of PROMs is becoming more resource intensive in terms of development, field testing, and ongoing evaluation, such as detection of changes consistent with improvement or worsening in specific and aggregate patient health outcomes.

Most recently, for-profit digital health firms have aggressively entered this market, touting sophisticated and parsimonious measure development and field-testing capability; robust interoperable data science-driven storage and access platforms; advanced data analytical expertise (eg, supervised machine learning and artificial intelligence); patient-centered interfaces for meaningful, team-based shared decision-making; and cost-effectiveness evaluation methods for assessing precision and multifactorial health outcomes. Hence, the standards generated today by more traditional stakeholders, such as regulators, policy-makers, health technology assessment authorities, and researchers may become less relevant and outdated for future PROM developments.

All of these diverse groups are necessary for the emergence of new tenets for a global learning health system necessary to achieve the worldwide goal of improving personalized population health. Building on and expanding these extensive advances should not, therefore, require the invention of a new wheel.

Published: May 11, 2022. doi:10.1001/jamanetworkopen.2022.11652

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2022 Casey DE Jr. JAMA Network Open .

Corresponding Author: Donald E. Casey Jr, MD, MPH, MBA, Department of Internal Medicine, Rush Medical College, 1717 W Congress Pkwy, 10th Floor, Chicago, IL 60612 ( [email protected] ).

Conflict of Interest Disclosures: Dr Casey reported being an active member of National Quality Forum Patient Experience and Function Committee and does not receive any financial remuneration for its activities.

Disclaimer: The views expressed in this commentary represent those of the author alone and should not be interpreted as policy of the National Quality Forum.

See More About

Casey DE. Patient-Reported Outcome Measures—Challenges and Opportunities for China. JAMA Netw Open. 2022;5(5):e2211652. doi:10.1001/jamanetworkopen.2022.11652

Manage citations:

© 2024

Select Your Interests

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council (US) Committee to Examine the Methodology for the Assessment of Research-Doctorate Programs; Ostriker JP, Kuh CV, editors. Assessing Research-Doctorate Programs: A Methodology Study. Washington (DC): National Academies Press (US); 2003.

Assessing Research-Doctorate Programs: A Methodology Study.

• Hardcopy Version at National Academies Press

4 Quantitative Measures

This chapter proposes and describes the quantitative measures relevant to the assessment of research-doctorate programs. These measures are valuable because they

• Permit comparisons across programs,
• Allow analyses of the correlates of the qualitative reputational measure,
• Provide potential students with a variety of dimensions along which to compare program characteristics, and
• Are easily updateable so that, even if assessing reputation is an expensive and time-intensive process, updated quantitative measures will allow current comparisons of programs.

Of course, quantitative measures can be subject to distortion just as reputational measures can be. An example would be a high citation count generated by a faulty result, but these distortions are different from and may be more easily identified and corrected than those involving reputational measures. Each quantitative measure reflects a dimension of the quality of a program, while reputational measures are more holistic and reflect the weighting of a variety of factors depending on rater preferences.

The Panel on Quantitative Measures recommended to the Committee several new data-collection approaches to address concerns about the 1995 Study. Evidence from individuals and organizations that corresponded with the Committee and the reactions to the previous study both show that the proposed study needs to provide information to potential students concerning the credentials required for admission to programs and the context within which graduate education occurs at each institution. It is important to present evidence on educational conditions for students as well as data on faculty quality. Data on post-Ph.D. plans are collected by the National Science Foundation and, although inadequate for those biological sciences in which postdoctoral study is expected to follow the receipt of a degree, they do differentiate among programs in other fields and should be reported in this context. It is also important to collect data to provide a quantitative basis for the assessment of scholarly work in the graduate programs.

With these purposes in mind, the Panel focused on quantitative data that could be obtained from four different groups of respondents in universities that are involved in doctoral education:

University-wide. These data reflect resources available to, and characteristics of, doctoral education at the university level. Examples include: library resources, health care, child care, on-campus housing, laboratory space (by program), and interdisciplinary centers. Program-specific. These data describe the characteristics of program faculty and students. Examples include: characteristics of students offered admission, information on program selectivity, support available to students, completion rates, time to degree, and demographic characteristics of faculty. Faculty-related. These data cover the disciplinary subfield, doctoral program connections, Ph.D. institution, and prior employment for each faculty member as well as tenure status and rank. Currently enrolled students. These data cover professional development, career plans and guidance, research productivity, research infrastructure, and demographic characteristics for students who have been admitted to candidacy in selected fields.

In addition to these data, which would be collected through surveys, data on research funding, citations, publications, and awards would be gathered from awarding agencies and the Institute for Scientific Information (ISI), as was done in the 1995 Study.

The mechanics of collecting these data have been greatly simplified since 1993 by the development of questionnaires and datasets that can be made available on the Web as well as software that permits easy analysis of large datasets. This technology makes it possible to expand the pool of potential raters of doctoral programs.

• MEASURABLE CHARACTERISTICS OF DOCTORAL PROGRAMS

The 1995 Study presented data on 17 characteristics of doctoral programs and their students beyond reputational measures. These are shown in Table 4–1 . Although these measures are interesting and useful, it is now possible to gather data that will paint a far more nuanced picture of doctoral programs. Indicators of what data would be especially useful have been pointed out in a number of recent discussions and surveys of doctoral education.

TABLE 4–1

Data Recommended for Inclusion in the Next Assessment of Research-Doctorate Programs. Bolded Elements Were Not Collected for the 1995 Study.

Institutional Variables

In the 1995 Study, data were presented on size, type of control, level of research and development funding, size of the graduate school, and library characteristics (total volumes and serials). These variables paint a general picture of the environment in which a doctoral program exists. Does it reside in a big research university? Does the graduate school loom large in its overall educational mission? The Committee added to these measures that were specifically related to doctoral education. Does the institution contribute to health care for doctoral students and their families? Does it provide graduate student housing? Are day care facilities provided on campus? All these variables are relevant to the quality of life of the doctoral student, who is often married and subsisting on a limited stipend.

The Committee took an especially hard look at the quantitative measures of library resources. The number of books and serials is not an adequate measure in the electronic age. Many universities participate in library consortia and digital material is a growing portion of their acquisitions. The Committee revised the library measures by asking for budget data on print serials, electronic serials, and other electronic media as well as for the size of library staff.

An addition to the institutional data collection effort is the question about laboratory space. Although this is a program characteristic, information about laboratory space is provided to the National Science Foundation and to government auditors at the institutional level. This is a measure of considerable interest for the laboratory sciences and engineering, and the Committee agreed that it should be collected as a possible correlate of quality.

Program Characteristics

The 1995 Study included data about faculty, students, and graduates gathered through institutional coordinators, Institute for Scientific Information (ISI) and the NSF Doctorate Records File (DRF). For the humanities, it gathered data on honors and awards from the granting organizations. Most of the institutional coordinators did a conscientious and thorough job, but the Committee believes that it would be helpful to pursue a more complex data-collection strategy that would include a program data collector (usually the director of graduate studies) in addition to the key institutional coordinator, a questionnaire to faculty, and questionnaires to students in selected programs. This approach was tested with the help of the pilot institutions. The institutional coordinator sent the NRC e-mail addresses of respondents for each program. The NRC then provided the respondent a password and the Web address of the program questionnaire. A similar procedure was followed for faculty whose names were provided by the program respondents. Copies of the questionnaires may be found in Appendix D .

In 1995, programs were asked for the number of faculty engaged in doctoral education and the percentage of faculty who were full professors. They were also asked for the numbers of Ph.D.s granted in the previous 3 years, their graduate enrollment both full-time and part-time, and the percentage of females in their total enrollment. Data on doctoral recipients, such as time to degree and demographic characteristics, came entirely from the DRF and represented only those who had completed their degrees.

The Committee believed that more informative data could be collected directly from the program respondents. Following the 1995 Study, a number of questions had been raised about the DRF data on time to degree. More generally, the Committee observed that data on graduates alone gave a possibly biased picture of the composition and funding of students enrolled in the program. The program questionnaire contains questions that are directly relevant to these concerns.

In the area of faculty characteristics, the program questionnaire requests the name, e-mail address, rank, tenure status, and demographic characteristics (gender, race/ ethnicity, and citizenship status) of each faculty member associated with the program. Student data requested include characteristics of students offered admission, information on program selectivity, support available to students, completion rates, and time to degree. It also asks whether the program requires a master's degree prior to admission to the doctoral program, since this is a crucial consideration affecting the measurement of time to degree. The questionnaire also permits construction of a detailed profile of the percentage of students receiving financial aid and the nature of that aid. Finally, the questionnaire asks a variety of questions related to program support of doctoral education: whether student teaching is mentored, whether students are provided with their own workspaces, whether professional development is encouraged through travel grants, and whether excellence in the mentoring of graduate students by faculty is rewarded. These are all “yes/no” questions that impose little respondent burden.

Faculty Characteristics

In the 1995 Study, a brief faculty questionnaire was administered to the raters who produced the reputational rankings. These raters were drawn from a sample of faculty nominated by their institutional coordinators. The sample size reflected the number of programs in each field. The brief questionnaire asked raters the year, institution, and date of their highest degree as well as their current field of specialization. The Committee believes that the faculty questionnaire should be modified to collect certain other data. For example, the university origins of current faculty are a direct measure of which graduate programs are training Ph.D.s who become faculty at research universities. Data on date of degree would also permit a comparison of origins of recently hired faculty as compared to faculty hired, for example, more than 20 years ago. Although subfield data were collected for the 1995 Study, they were not used. They could be useful in improving program descriptions for potential graduate students and for assuring that specialist programs are rated by knowledgeable peers in the same specialty.

The Committee also believes that additional questions asked of faculty could permit a richer description of interdisciplinarity. For example, faculty could list all programs in which they have participated, either by teaching or serving on dissertation committees. Many faculty would be listed as members of more than one graduate program, and for the purposes of the reputational survey, the Committee recommends that they be listed as program faculty for all programs with which they are associated. To avoid the possibility of double counting the output of productive faculty, objective measures should be attributed pro rata among the various programs in which they are listed. The decision as to how to prorate an effort should be made by the faculty member with guidance that they should try to describe how time devoted to doctoral education (teaching and student mentoring) has been allocated among the programs for the past 3-year period.

The Committee was concerned that programs might want to associate a well-known faculty member with as many programs as possible in order to boost its rating, even if he or she were not involved with the program. Allocation of publications should serve to discourage this behavior.

Student Characteristics and Views

Student observations have not been a part of past assessments of research-doctorate programs. Past studies have included data about demographic characteristics and about sources of financial support of Ph.D. recipients drawn from the DRF and about graduate student enrollment collected from the doctoral institutions. Another student measure was “educational effectiveness of the doctoral program,” and for reasons discussed in Chapter 6 , the Committee is recommending the elimination of this measure. The approach for measuring student processes and outcomes is discussed in Chapter 5 .

• PILOT TRIAL FINDINGS

The pilot trials were conducted over a 3-month period. The most important finding was that 3 months was barely sufficient for dealing with the study questionnaires. The full study should probably allow at least 4–6 months for data submission. The answers to many of the questions are prepared for other data collection efforts, but additional time is needed to customize answers to fit the taxonomy and to permit time for follow-up with nonrespondents.

All institutions carried out the trial through a single point of contact for the campus. This single point of contact worked with institutional research offices and program contacts to answer questions as well as interacted with NRC staff to assure that data definitions were uniform.

Electronic data collection worked well for institutions, programs, and faculty. We learned that it was better not to provide a hard copy alternative (as contrasted to Web response), since hard copy data simply had to be re-entered in databases once it was received by the NRC. All the pilot institutions store and access institutional and program data electronically. E-mail is the standard mode of communication with faculty and the rates of faculty response (60 percent) were high for a one-wave administration.

The Committee also learned that more precise definitions are needed to guide respondents. For example, when asking for data about “first-year doctoral students” a distinction may be needed about whether the students have a master's in the field. Care needs to be taken not to include terminal master's students, and precise definitions of “full-time” and “part-time” should be included.

The Committee learned the following from the questionnaire responses:

Institutional Questionnaire

• Library expenditures: Not all institutions separate e-media expenditures from print expenditures.
• Space: The questionnaire needs to provide guidance about how to allocate shared space. Answers to space questions also depend on how well the institution's programs fit the taxonomy. If the fit is poor, the allocation of space is arbitrary.
• Graduate student awards and support are more appropriately queried at the program level.

Program questionnaire

• Programs had difficulty filling out the inception cohort matrix but believed they could have done it if they had had more lead time.
• Programs knew who their competitors were for doctoral students.
• Programs that required GREs knew the averages and minima. For programs that do not require GREs, it would be helpful to ask what percentage of applicants submit GRE scores as well as report averages and minima only for those programs that are above a certain level (e.g., 80 percent).
• Requests for faculty lists and faculty data should be separate from requests for other program data.

Faculty questionnaire

• E-mail notifications must have a sufficiently informative subject heading so that they are not mistaken for spam.
• Questionnaires should contain a due date.
• Faculty associated with more than one program should be asked to fill out only one questionnaire. The NRC needs to develop procedures to duplicate information for the other programs with which a faculty member is associated.
• Some faculty identified their program by a name other than that of the program that submitted their name. A procedure must be developed to resolve this problem.

Each pilot institution was asked to provide comments on the questionnaires. These comments, some of which are reported above, will be used as background material for the committee that conducts the full study. Draft questionnaires for the full study should be reviewed by a number of institutional researchers from a diverse set of institutions as well as by survey researchers.

Data Collected from Other Sources

The Committee recommends that most of the quantitative data presented in the 1995 Study from other sources be collected again. These include: publication and citation data from ISI, data on research grants from government agencies and large private foundations, data on books from the Arts and Humanities Citation Index, and data on awards and honors from a large set of foundations and professional societies. Student data from the Doctorate Record File should be considered for inclusion but checked for inconsistencies against institutional and program records. In the case of inconsistencies, a validation process should be designed.

• RECOMMENDATIONS

The Committee recommends that the data listed in Bold type in Table 4–1 be added to the quantitative measures that were collected for the 1995 Study.

• Cite this Page National Research Council (US) Committee to Examine the Methodology for the Assessment of Research-Doctorate Programs; Ostriker JP, Kuh CV, editors. Assessing Research-Doctorate Programs: A Methodology Study. Washington (DC): National Academies Press (US); 2003. 4, Quantitative Measures.
• PDF version of this title (7.5M)

In this Page

Other titles in this collection.

• The National Academies Collection: Reports funded by National Institutes of Health

Recent Activity

• Quantitative Measures - Assessing Research-Doctorate Programs Quantitative Measures - Assessing Research-Doctorate Programs

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Do successful PhD outcomes reflect the research environment rather than academic ability?

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Visualization, Writing – original draft

* E-mail: [email protected] , [email protected]

Affiliation Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Deakin University, Geelong, Victoria, Australia

Roles Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing

Roles Conceptualization, Data curation, Methodology, Resources, Writing – review & editing

Affiliation Faculty of Health, Office of Faculty of Health, Deakin University, Geelong, Victoria, Australia

• Daniel L. Belavy, 
• Patrick J. Owen, 
• Patricia M. Livingston

• Published: August 5, 2020
• https://doi.org/10.1371/journal.pone.0236327
• Peer Review
• Reader Comments

Maximising research productivity is a major focus for universities world-wide. Graduate research programs are an important driver of research outputs. Choosing students with the greatest likelihood of success is considered a key part of improving research outcomes. There has been little empirical investigation of what factors drive the outcomes from a student's PhD and whether ranking procedures are effective in student selection. Here we show that, the research environment had a decisive influence: students who conducted research in one of the University's priority research areas and who had experienced, research-intensive, supervisors had significantly better outcomes from their PhD in terms of number of manuscripts published, citations, average impact factor of journals published in, and reduced attrition rates. In contrast, students’ previous academic outcomes and research training was unrelated to outcomes. Furthermore, students who received a scholarship to support their studies generated significantly more publications in higher impact journals, their work was cited more often and they were less likely to withdraw from their PhD. The findings suggest that experienced supervisors researching in a priority research area facilitate PhD student productivity. The findings question the utility of assigning PhD scholarships solely on the basis of student academic merit, once minimum entry requirements are met. Given that citations, publication numbers and publications in higher ranked journals drive university rankings, and that publications from PhD student contribute approximately one-third of all research outputs from universities, strengthening research infrastructure and supervision teams may be more important considerations for maximising the contribution of PhD students to a university’s international standing.

Citation: Belavy DL, Owen PJ, Livingston PM (2020) Do successful PhD outcomes reflect the research environment rather than academic ability? PLoS ONE 15(8): e0236327. https://doi.org/10.1371/journal.pone.0236327

Editor: Sergi Lozano, Universitat de Barcelona, SPAIN

Received: October 3, 2019; Accepted: July 3, 2020; Published: August 5, 2020

Copyright: © 2020 Belavy et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Participants did not give consent for their data to be published in online databanks and data are accessible with appropriate ethical approvals. Interested parties may contact the authors and/or the Deakin University Human Research Ethics Committee [email protected] to gain access to the data.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

A research doctorate degree comprises a process of independent research that produces an original contribution to knowledge [ 1 ]. The Australian Commonwealth Government supports [ 2 ] both domestic and overseas students undertaking research doctorate degrees, known as PhDs. These scholarships, which comprise a stipend for three years, are competitive. For this reason, when students apply for scholarships for their PhD studies, prior academic performance and research training play a key role in deciding whether the applicant receives a scholarship. However, is assigning scholarships predominately on the basis of academic grades and previous research experience effective in determining who will succeed?

A university’s international and national ranking is important for its reputation and marketing to prospective students [ 3 ]. Citation rates, number of publications and impact factor of journals faculty publish in, influence the ranking of a university. The Quacquarelli Symonds University Rank [ 4 ] is weighted 30% by the number of citations per faculty member, the Times Higher Education World Ranking [ 5 ] 30% by the number of citations and 6% by the number of publications per academic, and the Academic Ranking of World Universities [ 6 ] 20% by number of highly cited researchers, 20% by number of papers published in Nature or Science and 20% by the number of publications in total.

PhD students are important drivers of research outputs from universities, with one analysis [ 7 ] showing that one-third of research publications was from doctoral students. It is important to consider to what extent the procedures by which universities select students who go on to produce higher numbers of highly cited publications in high impact journals. We are not aware of any prior research that has examined this topic.

Waldinger [ 8 ] showed that the quality of academic staff (in departments of mathematics at German universities in the 1930s) influenced the likelihood of whether a doctoral student would become a full professor later in their career. Waldinger also showed that the amount of citations the scientific work of a doctoral student received through their entire subsequent scientific career was influenced by the status of their supervisor. Other factors, such as, the reputation of a department [ 9 ], the reputation the group leader [ 10 ], and access to resources and equipment [ 11 ], the number of full-professors on staff [ 12 ] influenced the research output of the academics involved in that group. Less information is available on the impact of student academic ability or prior research training on PhD outcomes: one analysis found that the reputation of a given department was more important for employment outcomes post-PhD than the accomplishments of the student during their studies [ 13 ]. Overall, the evidence available implies that the research environment may have an inordinate impact on the PhD student outcomes (e.g. citations, number of publications, impact factor of journals of those publications).

Here we examine the relationship between information known about applicants and their proposed supervisory teams at the time of scholarship application with the subsequent research outputs, as measured by number of citations, number of publications and the impact of journals of those publications.

Materials and methods

Deakin University Human Research Ethics Committee reviewed this project (2019–191) and found it to be compliant with the Ethical Considerations in Quality Assurance and Evaluation Activities guidelines of the National Health and Medical Research Council of Australia and determined that no further ethics review was required. Consent was not obtained and the data analysed anonymously.

Over a four year period, 2010–2013, 324 PhD scholarship applications were submitted to the Faculty of Health at one university in Australia ( Fig 1 ). In these applications, data were collated on:

• the grade the student achieved for their prior research training degree and their rank in this degree (top, middle, bottom third of first class honours or second class honours; or their equivalency to this),
• the grade point average achieved in their undergraduate degree (ranked on a scale of 1 to 5 with 5 = high distinction grade point average plus prizes awarded, 4 = high distinction grade point average, 3 = distinction, 2 = credit, 1 = pass).
• whether the applicant had published in a scientific journal (‘yes’ or ‘no’)
• research environment: whether the primary supervisor was located in a strategic research centre or institute within the university (‘yes’ or ‘no’).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

In 2010 to 2013, applications were submitted for PhD scholarships and in July 2018 data on publication outputs and completion of degree were obtained. Overall, 11 students did not enrol in PhD despite an offer with scholarship being made and 37 withdrew from their studies after starting.

https://doi.org/10.1371/journal.pone.0236327.g001

At the time of ranking for scholarships, the review panel scored each application on the basis of their academic merit and the research experience, alignment of the proposed research with the strategic research goals of the Faculty and university, and the experience of the supervisory team (as expressed by prior PhD completions, student progress, external grants, previous student publications, supervisor track record). In July 2018, these scores were reviewed by two independent assessors experienced in the scholarship ranking process and consensus was attained. Subsequent to this, following variables were generated:

• quartile of the academic merit scores in which each student was located.
• strategic alignment score achieved maximum points (‘yes’ or ‘no’). The presence or absence of a maximum score was taken for this variable as there were few instances of low scores on this criterion and data were skewed to the maximum score.
• supervisor team scores achieved maximum points (‘yes’ or ‘no’). The presence or absence of maximum score was taken for this variable as there were few instances of low scores on this criterion and data were skewed to the maximum score.
• level of academic appointment of the primary supervisor (lecturer/senior lecturer, associate professor, or full professor)

Data on whether the applicant subsequently enrolled (if ‘no’ they were excluded from further analysis; Fig 1 ), whether they completed their studies (‘yes’ or ‘no’), and whether the student received a scholarship to support his/her study (‘yes’ or ‘no’) obtained from another university database.

The university tracks publication outputs of its faculty and students. In July 2018, these data were obtained to link the number of publications by the student with their primary supervisor, the impact factor of the journals in which these publications appeared, and the number of citations received by the publications in Web of Science by the cut-off data of data access. Publications were matched on the basis of student name and primary supervisor name. If a change of primary supervisor occurred during student candidature, publication matches with the new primary supervisor were included as well. If the student had enrolled in a PhD but achieved no publications within the time-period examined, data were coded as zero publications, zero citations and zero average impact factor. Datasets were merged in using custom written code implemented in the 'R' statistical environment (version 3.4.0 https://www.r-project.org/ ). Where repeat applications were submitted in subsequent years by the same person, only the data available at the first application was used in further analysis. Prior to statistical analysis, all identifying information was removed.

Statistical analyses

All analyses were conducted using Stata statistical software version 15 (College Station TX, USA). Univariate associations between continuous dependent variables (number of publications, number of citations, number of citations per publication, average publication impact factor) and explanatory variables were assessed by the Kruskal-Wallis H test or Mann-Whitney U test (both non-parametric tests), as well as one-way analysis of variance and t-tests (both parametric tests). Univariate associations between withdrawal (yes/no) and independent variables were assessed by penalized maximum likelihood [ 14 , 15 ] logistic regression. We categorised the explanatory variables as follows: student specific factors (student research degree rank, student undergraduate rank, student prior publication, student academic merit), supervisor specific factors (supervisor located in a strategic research centre, supervisor academic level, supervisor team scores achieved maximum points), research topic related factors (strategic alignment score achieved maximum points), and whether a scholarship was awarded. To investigate which variables were more important than others for PhD student outcome metrics, factorial analysis of variance (ANOVA) as well as stepwise multiple linear regression models with both forward and backward selection were used to assess the association between the dependent variables and the independent variables. We further conducted factorial ANOVA to assess the association between the dependent variables and independent variables. Stepwise penalized maximum likelihood logistic regression models were used to predict withdrawal from PhD (yes/no) based on independent variables. An adjusted alpha level of 0.10 to enter and 0.20 to remove were used for all step-wise regression models. An alpha-level of 0.05 was adopted for all other statistical tests, including the assessment of the final step-wise regression models.

Primary analyses involved 198 students who enrolled in PhD (61% of 324 applications; Fig 1 ). The descriptive data on the characteristics of the students are shown in Table 1 . In the whole cohort, median (25 th percentile, 75 th percentile) and mean (standard deviation; SD) number of publications were 1.0 (0.0, 3.0) and 2.8 (4.4), impact factor 0.86 (0.00, 2.61) and 1.59 (2.36), citations per publication 0.0 (0.0, 4.5) and 3.5 (7.4) and total citations 0.0 (0.0, 17.0) and 19.6 (49.8). S1 Table presents the stability of the explanatory variables across each year of student applications. The relationship between ranking criteria and PhD student output metrics are shown in Table 2 (non-parametric analyses) and Table 3 (parametric analyses). Findings of both non-parametric and parametric analyses were similar. Non-parametric ( S1 Table ) and parametric ( S2 Table ) effect sizes as well as variability among variables by year of application ( S3 Table ) are reported in the data supplement.

https://doi.org/10.1371/journal.pone.0236327.t001

https://doi.org/10.1371/journal.pone.0236327.t002

https://doi.org/10.1371/journal.pone.0236327.t003

Number of publications

On univariate analysis (Tables 2 and 3 , Fig 2 ), primary supervisor being located in a strategic research centre (non-parametric and parametric both: P≤0.014), supervisory teams who received a maximum score (both: P≤0.014), being awarded a scholarship (both: P<0.001), student academic merit score (non-parametric: P = 0.017, parametric: P = 0.758) were associated with this outcome, but student undergraduate performance (both: P≥0.588), student research training degree outcome (e.g. first-class honours upper band; both: P≥0.262), research topic (both: P≥0.347), primary supervisor academic level (both: P≥0.107) were not.

Data are non-parametric effect sizes (95% confidence interval) for each parameter. See S1 Table for more detail and Tables 2 and 3 for more detail on each parameter. Student academic merit score from scholarship panel ranking showed moderate effect sizes, yet these students received 46% of all scholarships and multivariate analyses showed that receiving a scholarship was more important than the student's academic merit (see Results for more detail). Other markers of student ability and prior research training were unrelated to outcomes from the PhD. The score assigned by the panel to the alignment of the research topic with research priorities was unrelated to outcomes.

https://doi.org/10.1371/journal.pone.0236327.g002

Step-wise regression models ( Table 4 ) showed that receiving a scholarship (P = 0.001), primary supervisor being located in a strategic research centre (P = 0.018) remained in final model for number of publications, and whilst 'research topic' remained in the final model, it was not significant (P = 0.076). Factorial ANOVA ( S4 Table ) yielded similar results (having a scholarship, supervisory teams who received a maximum score, primary supervisor being located in a strategic research centre were associated, but not student related variables).

https://doi.org/10.1371/journal.pone.0236327.t004

Number of citations

On univariate analysis (Tables 2 and 3 , Fig 2 ), primary supervisor being located in a strategic research centre (non-parametric and parametric P both≤0.010), supervisory teams who received a maximum score (both: P≤0.012), being awarded a scholarship (both: P<0.001) were associated with this outcome, but student undergraduate performance (both: P≥0.668), student research training degree outcome (e.g. first-class honours upper band; both: P≥0.237), student academic merit score (both: P≥0.080), research topic (both: P≥0.202), primary supervisor academic level (both: P≥0.482) were not.

Step-wise regression models ( Table 4 ) showed that supervisory team who received a maximum score (P = 0.039) and the receiving a scholarship (P = 0.053), but in this case the scholarship award was not significant. Factorial ANOVA ( S4 Table ) yielded similar results (having a scholarship and supervisory teams who received a maximum score were associated, but not student related variables).

Citations per publications

On univariate analysis (Tables 2 and 3 , Fig 2 ), primary supervisor being located in a strategic research centre (non-parametric and parametric P both P≤0.009), supervisory teams who received a maximum score (non-parametric: P<0.001, parametric: P = 0.159), being awarded a scholarship (both: P≤0.048) were associated with this outcome, but student undergraduate performance (both: P≥0.640), student research training degree outcome (e.g. first-class honours upper band; both: P≥0.668), student academic merit score (both: P≥0.082), research topic (both: P≥0.185), primary supervisor academic level (both: P≥0.160) were not.

Step-wise regression models ( Table 4 ) showed that primary supervisor being located in a strategic research centre (P = 0.079) and supervisory team achieving maximum score (P = 0.087) remained in the final model, but neither terms were significant. Factorial ANOVA ( S4 Table ) yielded similar results (having a scholarship and supervisory teams who received a maximum score approached, but did not reach, significance).

Average impact factor

On univariate analysis (Tables 2 and 3 , Fig 2 ), primary supervisor being located in a strategic research centre (non-parametric and parametric P both P≤0.001), supervisory teams who received a maximum score (both: P≤0.005), being awarded a scholarship (both: P<0.001), student academic merit score (both: P≤0.005), were associated with this outcome, but student undergraduate performance (both: P≥0.077), student research training degree outcome (e.g. first-class honours upper band; both: P≥0.238), research topic (both: P≥0.161), primary supervisor academic level (both: P≥0.125) were not.

Step-wise regression models ( Table 4 ) showed that receiving a scholarship (P<0.001) and primary supervisor being located in a strategic research centre (P = 0.051) remained in the final model, with the latter not achieving statistical significance. Factorial ANOVA ( S4 Table ) yielded similar results (having a scholarship was significant, but supervisor related variables approached, but did not reach, significance; student related variables were not significant).

Drop-out from PhD

Odds ratios for student attrition is shown in Table 1 . Students were more than two times more likely to withdraw from their PhD when the supervisory team did not achieve maximum score (odds ratio [95% confidence interval] 2.88[1.39, 5.93], P = 0.004) or a scholarship was not awarded (odds ratio [95% confidence interval] 3.04[1.37, 6.73], P = 0.006). No other independent variables significantly predicted the likelihood of withdrawal.

The final multiple logistic regression model (χ 2 = 13.80, df = 3, P = 0.003) for predicting withdrawal from PhD included maximum supervisory team score (OR = 3.29, P = 0.013; i.e. lower risk of withdrawal when the supervisor score was maximum), student undergraduate degree grades (OR = 0.58, P = 0.047; i.e. reduced risk for each GPA rank lower) and receiving a scholarship (OR = 2.30, P = 0.090; i.e. lower risk when scholarship received), albeit the latter was not significant.

Associations between explanatory variables

Students in the highest quartile of academic merit received the most (42%) of all scholarships awarded. Of those in the highest quartile of academic merit, 79% received scholarships, compared to 62% in the second quartile, 20% in the third quartile and 22% in the lowest quartile.

Students who received a scholarship were more often supervised by strong supervisory teams (χ 2 = 9.346, P = 0.002; Table 5 ) and by supervisors who were located in a strategic research centre (χ 2 = 8.225, P = 0.004; Table 5 ). Supervisors who were in a strategic research centre were more likely to attract students in the highest quartile of academic merit (χ 2 = 3.899, P = 0.048; Table 6 ). Supervisory teams who received a maximum score were more likely to attract students in the highest quartile of academic merit (χ 2 = 10.147, P = 0.001; Table 6 ).

https://doi.org/10.1371/journal.pone.0236327.t005

https://doi.org/10.1371/journal.pone.0236327.t006

To the best of our knowledge, this is the first analysis of PhD student outcomes in relation to their research environment, their academic abilities and prior research training. The key finding was that the 'research environment', such as whether the supervisor was in a research centre or institute and the research experience of the supervision team, were most significant predictors of, with the largest effect sizes for, student outcomes. In contrast, the students' previous academic outcomes and previous research training were not predictors. Receiving a PhD scholarship had a significant influence on positive student outcomes and was more important than students being judged as having the highest academic merit. Receiving a scholarship occurred more frequently in students tied to stronger supervisory teams and supervisors in strategic research centres.

Entry to a PhD is typically restricted to those students with a minimum grade in a prior Masters or Honours degree [ 16 ]. At our university, prospective PhD students are required to have completed a research project with a dissertation of at least 25% of one year full-time study at Honours or Masters level and their grade needs to have been at least 70%. Our findings suggest that once students meet the minimum academic ability for entry into PhD, any further ability or research training above that does not influence the outcome of their PhD. This is in line with findings that scientist’s intelligence quotient does not correlate with their citation rates [ 17 ].

By contrast, it is the research environment in which the student is embedded that is decisive for the outcomes of their PhD; including the strength of their supervisory team. This is in line with the hypothesis of “accumulative advantage”, also known as “Matthew effects” in science [ 18 ] where differences between scientists at an early stage of their career become reinforced over time [ 19 ]. The standing of a PhD supervisor directly influences [ 8 ] the future career trajectory, and number of citations, their students receive throughout their career. Also, the standing of a department influences the future employment chances of its PhD graduates, on average, more than the individual achievements of those students [ 13 ]. The impact of teacher quality is seen in other areas of education [ 20 , 21 ], although ‘PhD supervisor quality’ is assessed differently to teacher quality in school and undergraduate education.

There are other factors known to impact the number and impact of publication outputs. Research collaboration has clearly been shown to lead to higher impact publications [ 22 – 25 ]. In the health-sciences field, publications of higher levels of evidence [ 26 ] are more likely to be cited. Similarly interventional (rather than observational) and prospective (rather than retrospective) studies [ 25 , 27 ], as well as randomised controlled trials and basic science papers [ 28 ] are more likely to be cited. Papers published in high impact factor journals will be more often cited simply for that reason [ 23 , 25 ]. We argue these factors are more likely to be determined by the research culture in which the student are embedded, as opposed to being determined by the student alone.

We also showed that receiving a PhD scholarship contributed to the students’ outcomes, in particular with more publications arising, more citations higher impact factor journals. In step-wise regression, we found that impact of the scholarship persisted for the number of publications and average impact factor of the journals in which the students published. This finding is in line with prior work [ 29 ] that showed PhD students receiving scholarships to support their studies published more peer reviewed papers. Similar to prior work [ 29 ], our results showed that receiving a scholarship was also associated with lower withdrawal rates.

Students were awarded scholarships based on their prior academic performance [ 30 ]. At this university, whilst the student’s academic merit contributed to 60% of their total ranking score, in practice this was the most decisive factor in determining which applicants were offered scholarships first. We show here, however, that the most significant attributes for PhD success were research environment and the performance metrics of the supervision team. How these attributes may influence employment opportunities post PhD also warrants further investigation.

Strengthening the research environment is also worthy of further investigation. Prior work [ 12 ] has shown that very few university departments rely solely on a small number of high-performing researchers for its research productivity. We show here that supervisor team quality has a key impact on the PhD student’s outcomes. Therefore, having more highly trained researchers is likely to lead to overall higher research student productivity, such as in having a higher percentage of faculty members who are at full-professor level [ 12 ]. Strategies for strengthening the research capacity of academic staff and potential supervisors include [ 31 ] structured research mentoring of academic staff, formal requirements for further academic research training.

The strengths of this analysis include being a prospective analysis of outcomes based on data that were known at the time of student selection. The limitations of the analysis were that it was focussed on one faculty at one university. It was not possible to conduct this analysis more widely at our university or at other universities as not all faculties and universities collate the same data on their PhD applicants. It would be relevant to examine such patterns at a wider range of universities, however obtaining such data from other universities is further complicated by data from scholarship ranking being confidential internal university information. Whilst this study was comprised one university, we believe its findings can easily be extrapolated to other regions of Australia and/or the world. Furthermore, we focussed on outcomes from PhDs that relate to university ranking procedures. Other outcomes, such as employment achieved post-PhD, student satisfaction, mental health are important to consider more widely.

Conclusions

In conclusion, to best of our knowledge, our study is the first to examine the relative importance of the environment versus student ability in the allocation and outcomes of their PhD. Our key finding was that the research environment is likely more important for supporting PhD students to produce larger numbers of highly cited publications in higher impact journals. Once the minimum level of academic ability and research training is met for entry to PhD, working with a strong research focussed supervisory team, being embedded in a research intensive institute, and receiving a scholarship are also important factors for publication and citation outcomes.

Supporting information

S1 table. non-parametric effect sizes between the ranking criteria of the 198 unique phd applications and researcher metrics..

Data are Cohen’s d. Bold = P<0.05. GPA: Grade point average.

https://doi.org/10.1371/journal.pone.0236327.s001

S2 Table. Parametric effect sizes between the ranking criteria of the 198 unique PhD applications and researcher metrics.

https://doi.org/10.1371/journal.pone.0236327.s002

S3 Table. Variability among variables by year of application.

Dependent variables are mean (standard deviation), expect withdrawing from PhD which are number (percentage within year). Explanatory variables are number (percentage within year). GPA: Grade point average.

https://doi.org/10.1371/journal.pone.0236327.s003

S4 Table. Results from factorial ANOVA.

Data are F-value (corresponding P-value). ANOVA fits explanatory variables sequentially to the dependent variables. Explanatory variables were fitted to the dependent variables in the order above (i.e. top variable at left fitted first, followed by the second to top variable). This therefore accounted for potential association of student related factors first to PhD outcomes, with then having a scholarship and then supervisor related factors considered. Despite accounting for student related variables first, having a scholarship and supervisor quality were most consistently associated with outcomes from a student’s PhD.

https://doi.org/10.1371/journal.pone.0236327.s004

Acknowledgments

The authors thank Grant Michie, Rachelle DeBrito and their teams for assistance with access to enrolment and publication output data, Steve Sawyer for assistance in reviewing and accessing the scholarship application data and biostatistician A/Prof Steven Bowe for statistical advice.

• View Article
• Google Scholar
• 2. Australian Government Department of Education. Research Training Program. Research Training Program https://www.education.gov.au/research-training-program (2019).
• 4. QS World University Rankings Methodology https://www.topuniversities.com/qs-world-university-rankings/methodology (2018).
• 5. World University Rankings 2019: methodology https://www.timeshighereducation.com/world-university-rankings/world-university-rankings-2019-methodology (2018).
• 6. Academic Ranking of World Universities Methodology http://www.shanghairanking.com/ARWU-Methodology-2018.html (2018).
• PubMed/NCBI
• 16. Haidar, H. PhD admission requirements. What is a PhD? https://www.topuniversities.com/blog/what-phd .
• 21. Rowe, K. The importance of teacher quality as a key determinant of students’ experiences and outcomes of schooling. in 15–23 (2003).
• 30. The University of Melbourne. How are graduate research scholarships awarded? https://education.unimelb.edu.au/study/scholarships/scholarship-support .

The Strathclyde inventory as a measure of outcome in person-centred therapy

Downloadable content.

• Stephen, Susan.
• Strathclyde Thesis Copyright
• University of Strathclyde
• Doctoral (Postgraduate)
• Doctor of Philosophy (PhD)
• Counselling Unit.
• School of Psychological Sciences and Health.
• Person-centred therapy, like other humanistic therapies, proposes a potentiality model in which psychological growth, not simply the reduction of symptoms, is the anticipated outcome of therapy. Although substantial evidence of the effectiveness of person-centred therapy using medical model concepts exists, there is a need to develop measures that test outcome in therapy according to the person-centred theory of change. The Strathclyde Inventory (SI) is a brief self-report instrument designed to measure congruent functioning (described elsewhere as Rogers' fully functioning person, or congruence) for use as an outcome measure in therapy. The main purpose of this innovative three-part mixed method study was to investigate the validity of the SI as an outcome measure from multiple perspectives using data collected from a large UK-based clinical population. The first study evaluated the internal structure and reliability/precision of the instrument using the Rasch model. The second study investigated patterns of change in SI scores over the course of therapy seeking evidence of sensitivity to change, as well as convergent and construct validity. The third study tested the validity of change in SI scores as a measure of congruent functioning via a meta-synthesis of a series of eight systematic case studies examining client improvement and deterioration in therapy identified by pre-post change in SI scores. The results supported the validity of the SI as an internally consistent and precise unidimensional instrument that is able to identify meaningful change in congruent functioning within a UK-based clinical population. A brief 12-item version of the instrument was produced. An evidence-based, theoretically coherent, developmental pathway for congruent functioning was proposed, identifying self-acceptance as a pivot point. Overall, the results of this three-part study established that change in participants' scores during therapy demonstrated a high degree of variation and proposed an explanation for different post-therapy outcomes in congruent functioning.
• Dixon, Diane
• Elliott, Robert, 1950-
• Doctoral thesis
• 10.48730/0bxy-dv30
• 9912876793402996

Student Learning Outcomes: Ph.D.

Examples of student learning outcomes for phd programs.

Source: Adapted from student learning outcomes in a range of graduate programs at Brigham Young University .

All Disciplines

All graduates will be able to:

• Critically apply theories, methodologies, and knowledge to address fundamental questions in their primary area of study. (Research, Critical Thinking, Content Knowledge)
• Pursue research of significance in the discipline or an interdisciplinary or creative project. Students plan and conduct this research or implement this project under the guidance of an advisor while developing the intellectual independence that typifies true scholarship. (Research, Critical and Creative Thinking)
• Demonstrate skills in oral and written communication sufficient to publish and present work in their field and to prepare grant proposals. (Communication)
• Follow the principles of ethics in their field and in academia. (Ethics)
• Demonstrate, through service, the value of their discipline to the academy and community at large. (Service, Content Knowledge)
• Demonstrate a mastery of skills and knowledge at a level required for college and university undergraduate teaching in their discipline and assessment of student learning. (Content Knowledge, Teaching)
• Interact productively with people from diverse backgrounds as both leaders/mentors and team members with integrity and professionalism. (Communication, Leadership)

Discipline or Degree-specific Skills, Knowledge, and Values

Graduates in [discipline or degree] will be able to:

• as defined by the program

Last updated 8/22/2013

• Open access
• Published: 26 August 2020

Understanding the mental health of doctoral researchers: a mixed methods systematic review with meta-analysis and meta-synthesis

• Cassie M. Hazell   ORCID: orcid.org/0000-0001-5868-9902 1 ,
• Laura Chapman 2 ,
• Sophie F. Valeix 3 ,
• Paul Roberts 4 ,
• Jeremy E. Niven 5 &
• Clio Berry 6  

Systematic Reviews volume  9 , Article number:  197 ( 2020 ) Cite this article

13k Accesses

57 Citations

74 Altmetric

Metrics details

Data from studies with undergraduate and postgraduate taught students suggest that they are at an increased risk of having mental health problems, compared to the general population. By contrast, the literature on doctoral researchers (DRs) is far more disparate and unclear. There is a need to bring together current findings and identify what questions still need to be answered.

We conducted a mixed methods systematic review to summarise the research on doctoral researchers’ (DRs) mental health. Our search revealed 52 articles that were included in this review.

The results of our meta-analysis found that DRs reported significantly higher stress levels compared with population norm data. Using meta-analyses and meta-synthesis techniques, we found the risk factors with the strongest evidence base were isolation and identifying as female. Social support, viewing the PhD as a process, a positive student-supervisor relationship and engaging in self-care were the most well-established protective factors.

Conclusions

We have identified a critical need for researchers to better coordinate data collection to aid future reviews and allow for clinically meaningful conclusions to be drawn.

Systematic review registration

PROSPERO registration CRD42018092867

Peer Review reports

Student mental health has become a regular feature across media outlets in the United Kingdom (UK), with frequent warnings in the media that the sector is facing a ‘mental health crisis’ [ 1 ]. These claims are largely based on the work of regulatory authorities and ‘grey’ literature. Such sources corroborate an increase in the prevalence of mental health difficulties amongst students. In 2013, 1 in 5 students reported having a mental health problem [ 2 ]. Only 3 years later, however, this figure increased to 1 in 4 [ 3 ]. In real terms, this equates to 21,435 students disclosing mental health problems in 2013 rising to 49,265 in 2017 [ 4 ]. Data from the Higher Education Statistics Agency (HESA) demonstrates a 210% increase in the number of students terminating their studies reportedly due to poor mental health [ 5 ], while the number of students dying by suicide has consistently increased in the past decade [ 6 ].

This issue is not isolated to the UK. In the United States (US), the prevalence of student mental health problems and use of counselling services has steadily risen over the past 6 years [ 7 ]. A large international survey of more than 14,000 students across 8 countries (Australia, Belgium, Germany, Mexico, Northern Ireland, South Africa, Spain and the United States) found that 35% of students met the diagnostic criteria for at least one common mental health condition, with highest rates found in Australia and Germany [ 8 ].

The above figures all pertain to undergraduate students. Finding equivalent information for postgraduate students is more difficult, and where available tends to combine data for postgraduate taught students and doctoral researchers (DRs; also known as PhD students or postgraduate researchers) (e.g. [ 4 ]). The latest trend analysis based on data from 36 countries suggests that approximately 2.3% of people will enrol in a PhD programme during their lifetime [ 9 ]. The countries with the highest number of DRs are the US, Germany and the UK [ 10 ]. At present, there are more than 281,360 DRs currently registered across these three countries alone [ 11 , 12 ], making them a significant part of the university population. The aim of this systematic review is to bring attention specifically to the mental health of DRs by summarising the available evidence on this issue.

Using a mixed methods approach, including meta-analysis and meta-synthesis, this review seeks to answer three research questions: (1) What is the prevalence of mental health difficulties amongst DRs? (2) What are the risk factors associated with poor mental health in DRs? And (3) what are the protective factors associated with good mental health in DRs?

Literature search

We conducted a search of the titles and abstracts of all article types within the following databases: AMED, BNI, CINAHL, Embase, HBE, HMIC, Medline, PsycInfo, PubMed, Scopus and Web of Science. The same search terms were used within all of the databases, and the search was completed on the 13th April 2018. Our search terms were selected to capture the variable terms used to describe DRs, as well as the terms used to describe mental health, mental health problems and related constructs. We also reviewed the reference lists of all the papers included in this review. Full details of the search strategy are provided in the supplementary material .

Inclusion criteria

Articles meeting the following criteria were considered eligible for inclusion: (1) the full text was available in English; (2) the article presented empirical data; (3) all study participants, or a clearly delineated sub-set, were studying at the doctoral level for a research degree (DRs or equivalent); and (4) the data collected related to mental health constructs. The last of these criteria was operationalised (a) for quantitative studies as having at least one mental health-related outcome measure, and (b) for qualitative studies as having a discussion guide that included questions related to mental health. We included university-published theses and dissertations as these are subjected to a minimum level of peer-review by examiners.

Exclusion criteria

In order to reduce heterogeneity and focus the review on doctoral research as opposed to practice-based training, we excluded articles where participants were studying at the doctoral level, but their training did not focus on research (e.g. PsyD doctorate in Clinical Psychology).

Screening articles

Papers were screened by one of the present authors at the level of title, then abstract, and finally at full text (Fig. 1 ). Duplicates were removed after screening at abstract. At each level of screening, a random 20% sub-set of articles were double screened by another author, and levels of agreement were calculated (Cohen’s kappa [ 13 ]). Where disagreements occurred between authors, a third author was consulted to decide whether the paper should or should not be included. All kappa values evidence at least moderate agreement between authors [ 14 ]—see Fig. 1 for exact kappa values.

PRISMA diagram of literature review process

Data extraction

This review reports on both quantitative and qualitative findings, and separate extraction methods were used for each. Data extraction was performed by authors CH, CB, SV and LC.

Quantitative data extraction

The articles in this review used varying methods and measures. To accommodate this heterogeneity, multiple approaches were used to extract quantitative data. Where available, we extracted (a) descriptive statistics, (b) correlations and (c) a list of key findings. For all mental health outcome measures, we extracted the means and standard deviations for the DR participants, and where available for the control group (descriptive statistics). For studies utilising a within-subjects study design, we extracted data where a mental health outcome measure was correlated with another construct (correlations). Finally, to ensure that we did not lose important findings that did not use descriptive statistics or correlations, we extracted the key findings from the results sections of each paper (list of key findings). Key findings were identified as any type of statistical analysis that included at least one mental health outcome.

Qualitative data extraction

In line with the meta-ethnographic method [ 15 ] and our interest in the empirical data as well as the authors’ interpretations thereof, i.e. the findings of each article [ 16 ], the data extracted from the articles comprised both results/findings and discussion/conclusion sections. For articles reporting qualitative findings, we extracted the results and discussion sections from articles verbatim. Where articles used mixed methods, only the qualitative section of the results was extracted. Methodological and setting details from each article were also extracted and provided (see Appendix A) in order to contextualise the studies.

Data analysis

Quantitative data analysis, descriptive statistics.

We present frequencies and percentages of the constructs measured, the tools used and whether basic descriptive statistics ( M and SD ) were reported. The full data file is available from the first author upon request.

Effect sizes

Where studies had a control group, we calculated a between-group effect size (Cohen’s d ) using the formula reported by Wilson [ 17 ], and interpreted using the standard criteria [ 13 ]. For all other studies, we sought to compare results with normative data where the following criteria were satisfied: (a) at least three studies reported data using the same mental health assessment tool; (b) empirical normative data were available; and (c) the scale mean/total had been calculated following original authors’ instructions. Only the Perceived Stress Scale (PSS) 10- [ 18 ] and 14-item versions [ 19 ] met these criteria. Normative data were available from a sample of adults living in the United States: collected in 2009 for the 10-item version ( n = 2000; M = 15.21; SD = 7.28) [ 20 ] and in 1983 for the 14-item version ( n = 2355; M = 19.62; SD = 7.49) [ 18 ].

The meta-analysis of PSS data was conducted using MedCalc [ 21 ], and based on a random effects model, as recommended by [ 22 ]. The between-group effect sizes (DRs versus US norms) were calculated comparing PSS means and standard deviations in the respective groups. The effect sizes were weighted using the variable variances [ 23 ].

Correlations

Where at least three studies reported data reflecting a bivariate association between a mental health and another variable, we summarised this data into a meta-analysis using the reported r coefficients and sample sizes. Again, we used MedCalc [ 21 ] to conduct the analysis using a random effects model, based on the procedure outlined by Borenstein, Hedges, Higgins and Rothstein [ 24 ]. This analysis approach involves converting correlation coefficients into Fisher’s z values [ 25 ], calculating the summary of Fisher’s z , and then converting this to a summary correlation coefficient ( r ). The effect sizes were weighted in line with the Hedges and Ollkin [ 23 ] method. Heterogeneity was assessed using the Q statistic, and I 2 value—both were interpreted according to the GRADE criteria [ 26 ]. Where correlations could not be summarised within a meta-analysis, we have reported these descriptively.

Due to the heterogenous nature of the studies, the above methods could not capture all of the quantitative data. Therefore, additional data (e.g. frequencies, statistical tests) reported in the identified articles was collated into a single document, coded as relating to prevalence, risk or protective factors and reported as a narrative review.

Qualitative data analysis

We used thematic analytic methods to analyse the qualitative data. We followed the thematic synthesis method [ 16 , 27 ] and were informed by a thematic analysis approach [ 28 , 29 ]. We took a critical realist epistemological stance [ 30 , 31 ] and aimed to bring together an analysis reflecting meaningful patterns amongst the data [ 29 ] or demi-regularities, and identifying potential social mechanisms that might influence the experience of such phenomena [ 31 ]. The focus of the meta-synthesis is interpretative rather than aggregative [ 32 ].

Coding was line by line, open and complete. Following line-by-line coding of all articles, a thematic map was created. Codes were entered on an article-by-article basis and then grouped and re-grouped into meaningful patterns. Comparisons were made across studies to attempt to identify demi-regularities or patterns and contradictions or points of departure. The thematic map was reviewed in consultation with other authors to inductively create and refine themes. Thematic summaries were created and brought together into a first draft of the thematic structure. At this point, each theme was compared against the line-by-line codes and the original articles in order to check its fit and to populate the written account with illustrative quotations.

Research rigour

The qualitative analysis was informed by independent coding by authors CB and SV, and analytic discussions with CH, SV and LC. Our objective was not to capture or achieve inter-rater reliability, rather the analysis was strengthened through involvement of authors from diverse backgrounds including past and recent PhD completion, experiences of mental health problems during PhD completion, PhD supervision experience, experience as employees in a UK university doctoral school and different nationalities. In order to enhance reflexivity, CB used a journal throughout the analytic process to help notice and bracket personal reflections on the data and the ways in which these personal reflections might impact on the interpretation [ 29 , 33 ]. The ENTREQ checklist [ 34 ] was consulted in the preparation of this report to improve the quality of reporting.

Quality assessment

Quantitative data.

The quality of the quantitative papers was assessed using the STROBE combined checklist [ 35 ]. A random 20% sub-sample of these studies were double-coded and inter-rater agreement was 0.70, indicating ‘substantial’ agreement [ 14 ]. The maximum possible quality score was 23, with a higher score indicating greater quality, with the mean average of 15.97, and a range from 0 to 22. The most frequently low-scoring criteria were incomplete reporting regarding the management of missing data, and lack of reported efforts to address potential causes of bias.

Qualitative data

There appeared to be no discernible pattern in the perceived quality of studies; the highest [ 36 , 37 , 38 , 39 , 40 ] and lowest scoring [ 41 , 42 , 43 , 44 , 45 , 46 ] studies reflected both theses and journal publications, a variety of locations and settings and different methodologies. The most frequent low-scoring criteria were relating to the authors’ positions and reflections thereof (i.e. ‘Qualitative approach and research paradigm’, ‘Researcher characteristics and reflexivity’, ‘Techniques to enhance trustworthiness’, ‘Limitations’, ‘Conflict of interest and Funding’). Discussions of ethical issues and approval processes was also frequently absent. We identified that we foregrounded higher quality studies in our synthesis in that these studies appeared to have greater contributions reflected in the shape and content of the themes developed and were more likely to be the sources of the selected illustrative quotes.

Mixed methods approach

The goal of this review is to answer the review questions by synthesising the findings from both quantitative and/or qualitative studies. To achieve our goal, we adopted an integrated approach [ 47 ], whereby we used both quantitative and qualitative methods to answer the same review question, and draw a synthesised conclusion. Different analysis approaches were used for the quantitative and qualitative data and are therefore initially reported separately within the methods. A separate synthesised summary of the findings is then provided.

Overview of literature

Of the 52 papers included in this review (Table 1 ), 7 were qualitative, 29 were quantitative and 16 mixed methods. Most articles (35) were peer-reviewed papers, and the minority were theses (17). Only four of the articles included a control group; in three instances comprising students (but not DRs) and in the other drawn from the general population.

Quantitative results

Thirty-five papers reported quantitative data, providing 52 reported sets of mental health related data (an average of 1.49 measures per study): 24 (68.57%) measured stress, 10 (28.57%) anxiety, 9 (25.71%) general wellbeing, 5 (14.29%) social support, 3 (8.57%) depression and 1 (2.86%) self-esteem. Five studies (9.62%) used an unvalidated scale created for the purposes of the study. Fifteen studies (28.85%) did not report descriptive statistics.

Of the four studies that included a control group, only two of these reported descriptive statistics for both groups on a mental health outcome [ 66 , 69 ]. There is a small (Cohen’s d = 0.27) and large between-group effect (Cohen’s d = 1.15) when DRs were compared to undergraduate and postgraduate clinical psychology students respectively in terms of self-reported stress.

The meta-analysis of DR scores on the PSS (both 10- and 14-item versions) compared to population normative data produced a large and significant between-group effect size ( d = 1.12, 95% CI [0.52, 1.73]) in favour of DRs scoring higher on the PSS than the general population (Fig. 2 ), suggesting DRs experience significantly elevated stress. However, these findings should be interpreted in light of the significant between-study heterogeneity that can be classified as ‘considerable’ [ 26 ].

A meta-analysis of between-group effect sizes (Cohen’s d ) comparing PSS scores (both 10- and 14-item versions) from DRs and normative population data. *Studies using the 14 item version of the PSS; a positive effect size indicates DRs had a higher score on the PSS; a negative effect size indicates that the normative data produced a higher score on the PSS; black diamond = total effect size (based on random effects model); d = Cohen’s d ; Q = heterogeneity; Z = z score; I 2 = proportion of variance due to between-study heterogeneity; p = exact p value

To explore this heterogeneity, we re-ran the meta-analysis separately for the 10- and 14-item versions. The effect size remained large and significant when looking only at the studies using the 14-item version ( k = 6; d = 1.41, 95% CI [0.63, 2.19]), but was reduced and no longer significant when looking at the 10-item version only ( k = 3; d = 0.57, 95% CI [− 0.51, 1.64]). However, both effect sizes were still marred by significant heterogeneity between studies (10-item: Q = 232.02, p < .001; 14-item: Q = 356.76, p < .001).

Studies reported sufficient correlations for two separate meta-analyses; the first assessing the relationship between stress (PSS [ 18 , 19 ]) and perceived support, and the second between stress (PSS) and academic performance.

Stress x support

We included all measures related to support irrespective of whom that support came from (e.g. partner support, peer support, mentor support). The overall effect size suggests a small and significant negative correlation between stress and support ( r = − .24, 95% CI [− 0.34, − 0.13]) (see Fig. 3 ), meaning that low support is associated with greater perceived stress. However, the results should be interpreted in light of the significant heterogeneity between studies. The I 2 value quantifies this heterogeneity as almost 90% of the variance being explained by between-study heterogeneity, which is classified as ‘substantial’ (26).

Forest plot and meta-analysis of correlation coefficients testing the relationship between stress and perceived support. Black diamond = total effect size (based on random effects model); r = Pearson’s r ; Q = heterogeneity; Z = z score; I 2 = proportion of variance due to between-study heterogeneity; p = exact p value

Stress x performance

The overall effect size suggests that there is no relationship between stress and performance in their studies ( r = − .07, 95% CI [− 0.19, 0.05]) (see Fig. 4 ), meaning that DRs perception of their progress was not associated with their perceived stress This finding suggests that the amount of progress that DRs were making during their studies was not associated with stress levels.

Forest plot and meta-analysis of correlation coefficients testing the relationship between stress and performance. Black diamond = total effect size (based on random effects model); r = Pearson’s r ; Q = heterogeneity; Z = z score; I 2 = proportion of variance due to between-study heterogeneity; p = exact p value

Other correlations

Correlations reported in less than three studies are summarised in Fig. 5 . Again, stress was the most commonly tested mental health variable. Self-care and positive feelings towards the thesis were consistently found to negatively correlate with mental health constructs. Negative writing habits (e.g. perfectionism, blocks and procrastination) were consistently found to positively correlate with mental health constructs. The strongest correlations were found between stress, and health related quality of life ( r = − .62) or neuroticism ( r = .59), meaning that lower stress was associated with greater quality of life and reduced neuroticism. The weakest relationships ( r < .10) were found between mental health outcomes and: faculty concern, writing as knowledge transformation, innate writing ability (stress and anxiety), years married, locus of control, number of children and openness (stress only).

Correlation coefficients testing the relationship between a mental health outcome and other construct. Correlation coefficients are given in brackets ( r ); * p < .05; each correlation coefficient reflects the results from a single study

Several studies reported DR mental health problem prevalence and this ranged from 36.30% [ 54 ] to 55.9% [ 67 ]. Using clinical cut-offs, 32% were experiencing a common psychiatric disorder [ 64 ]; with another study finding that 53.7% met the questionnaire cut-off criteria for depression, and 41.9% for anxiety [ 67 ]. One study compared prevalence amongst DRs and the general population, employees and other higher education students; in all instances, DRs had higher levels of psychological distress (non-clinical), and met criteria for a clinical psychiatric disorder more frequently [ 64 ].

Risk factors

Demographics Two studies reported no significant difference between males and females in terms of reported stress [ 57 , 73 ], but the majority suggested female DRs report greater clinical [ 80 ], and non-clinical problems with their mental health [ 37 , 64 , 79 , 83 , 89 ].

Several studies explored how mental health difficulties differed in relation to demographic variables other than gender, suggesting that being single or not having children was associated with poorer mental health [ 64 ] as was a lower socioeconomic status [ 71 ]. One study found that mental health difficulties did not differ depending on DRs’ ethnicity [ 51 ], but another found that Black students attending ‘historically Black universities’ were significantly more anxious [ 87 ]. The majority of the studies were conducted in the US, but only one study tested for cross-cultural differences: reporting that DRs in France were more psychologically distressed than those studying in the UK [ 67 ].

Work-life balance Year of study did not appear to be associated with greater subjective stress in a study involving clinical psychology DRs (Platt and Schaefer [ 75 ]), although other studies suggested greater stress reported by those in the latter part of their studies [ 89 ], who viewed their studies as a burden [ 81 ], or had external contracts, i.e. not employed by their university [ 85 ]. Regression analyses revealed that a common predictor of poor mental health was uncertainty in DR studies; whether in relation to uncertain funding [ 64 ] or uncertain progress [ 80 ]. More than two-thirds of DRs reported general academic pressure as a cause of stress, and a lack of time as preventing them from looking after themselves [ 58 ]. Being isolated was also a strong predictor of stress [ 84 ].

Protective factors

DRs who more strongly endorsed all of the five-factor personality traits (openness, conscientiousness, extraversion, agreeableness and neuroticism) [ 66 ], self-reported higher academic achievement [ 40 ] and viewed their studies as a learning process (rather than a means to an end) [ 82 ] reported fewer mental health problems. DRs were able to mitigate poor mental health by engaging in self-care [ 72 ], having a supervisor with an inspirational leadership style [ 64 ] and building coping strategies [ 56 ]. The most frequently reported coping strategy was seeking support from other people [ 37 , 58 ].

Qualitative results

Meta-synthesis.

Four higher-order themes were identified: (1) Always alone in the struggle, (2) Death of personhood, (3) The system is sick and (4) Seeing, being and becoming. The first two themes reflect individual risk/vulnerability factors and the processes implicated in the experience of mental distress, the third represents systemic risk and vulnerability factors and the final theme reflects individual and systemic protective mechanisms and transformative influences. See Table 2 for details of the full thematic structure with illustrative quotes.

Always alone in the struggle

‘Always alone in the struggle’ reflects the isolated nature of the PhD experience. Two subthemes reflect different aspects of being alone; ‘Invisible, isolated and abandoned’ represents DRs’ sense of physical and psychological separation from others and ‘It’s not you, it’s me’ represents DRs’ sense of being solely responsible for their PhD process and experience.

Invisible, isolated and abandoned

Feeling invisible and isolated both within and outside of the academic environment appears a core DR experience [ 39 , 43 , 81 ]. Isolation from academic peers seemed especially salient for DRs with less of a physical presence on campus, e.g. part-time and distance students, those engaging in extensive fieldwork, outside employment and those with no peer research or lab group [ 36 , 52 , 68 ]. Where DRs reported relationships with DR peers, these were characterised as low quality or ‘not proper friendships’ and this appeared linked to a sense of essential and obvious competition amongst DRs with respect to current and future resources, support and opportunities [ 39 ], in which a minority of individuals were seen to receive the majority share [ 36 , 74 ]. Intimate sharing with peers thus appeared to feel unsafe. This reflected the competitive environment but also a sense of peer relationships being predicated on too shared an experience [ 39 ].

In addition to poor peer relations, a mismatch between the expected and observed depth of supervisor interest, engagement and was evident [ 40 , 81 ]. This mismatch was clearly associated with disappointment and anger, and a sense of abandonment, which appeared to impact negatively on DR mental health and wellbeing [ 42 ] (p. 182). Moreover, DRs perceived academic departments as complicit in their isolation; failing to offer adequate opportunities for academic and social belonging and connections [ 42 , 81 ] and including PGRs only in a fleeting or ‘hollow’ sense [ 37 ]. DRs identified this isolation as sending a broader message about academia as a solitary and unsupported pursuit; a message that could lead some DRs to self-select out of planning for future in academia [ 37 , 42 ]. DRs appeared to make sense of their lack of belonging in their department as related to their sense of being different, and that this difference might suggest they did not ‘fit in’ with academia more broadly [ 74 ]. In the short-term, DRs might expend more effort to try and achieve a social and/or professional connection and equitable access to support, opportunities and resources [ 74 ]. However, over the longer-term, the continuing perception of being professionally ‘other’ also seemed to undermine DRs’ sense of meaning and purpose [ 81 ] and could lead to opting out of an academic career [ 62 , 74 ].

Isolation within the PhD was compounded by isolation from one’s personal relationships. This personal isolation was first physical, in which the laborious nature of the PhD acted as a catalyst for the breakdown of pre-existing relationships [ 76 ]. Moreover, DRs also experienced a sense of psychological detachment [ 45 , 74 ]. Thus, the experience of isolation appeared to be extremely pervasive, with DRs feeling excluded and isolated physically and psychologically and across both their professional and personal lives.

It’s not you, it’s me

‘It’s not you, it’s me’ reflects DRs’ perfectionism as a central challenge of their PhD experience and a contributor to their sense of psychological isolation from other people. DRs’ perfectionism manifested in four key ways; firstly, in the overwhelming sense of responsibility experienced by DRs; secondly, in the tendency to position themselves as inadequate and inferior; thirdly, in cycles of perfectionist paralysis; and finally, in the tendency to find evidence which confirms their assumed inferiority.

DRs positioned themselves as solely responsible for their PhD and for the creation of a positive relationship with their supervisor [ 36 , 52 , 81 ]. DRs expressed a perceived need to capture their supervisors’ interest and attention [ 36 , 52 , 74 ], feeling that they needed to identify and sell to their supervisors some shared characteristic or interest in order to scaffold a meaningful relationship. DRs appeared to feel it necessary to assume sole responsibility for their personal lives and to prohibit any intrusion of the personal in to the professional, even in incredibly distressing circumstances [ 42 ].

DRs appeared to compare themselves against an ideal or archetypal DR and this comparison was typically unfavourable [ 37 ], with DRs contrasting the expected ideal self with their actual imperfect and fallible self [ 37 , 42 , 52 ]. DRs’ sense of inadequacy appeared acutely and frequently reflected back to them by supervisors in the form of negative or seemingly disdainful feedback and interactions [ 41 , 76 ]. DRs framed negative supervisor responses as a cue to work harder, meaning they were continually striving, but never reaching, the DR ideal [ 76 ]. This ideal-actual self-discrepancy was associated with a tendency towards punitive self-talk with clear negative valence [ 38 ].

DRs appear to commonly use self-castigation as a necessary (albeit insufficient) means to motivate themselves to improve their performance in line with perfectionistic standards [ 38 , 41 ]. The oscillation between expectation and actuality ultimately resulted in increased stress and anxiety and reduced enjoyment and motivation. Low motivation and enjoyment appeared to cause procrastination and avoidance, which lead to a greater discrepancy between the ideal and actual self; in turn, this caused more stress and anxiety and further reduced enjoyment and motivation leading to a sense of stuckness [ 76 ].

The internalisation of perceived failure was such that DRs appeared to make sense of their place, progress and possible futures through a lens of inferiority, for example, positioning themselves as less talented and successful compared to their peers [ 37 ]. Thus, instances such as not being offered a job, not receiving funding, not feeling connected to supervisors, feeling excluded by academics and peers were all made sense of in relation to DRs’ perceived relative inadequacy [ 36 ].

Death of personhood

The higher-order theme ‘Death of personhood’ reflects DRs’ identity conflict during the PhD process; a sense that DRs’ engage in a ‘Sacrifice of personal identity’ in which they feel they must give up their pre-existing self-identity, begin to conceive of themselves as purely ‘takers’ personally and professionally, thus experiencing the ‘Self as parasitic’, and ultimately experience a ‘Death of self-agency’ in relation to the thesis, the supervisor and other life roles and activities.

A sacrifice of personal identity

The sacrifice of personal identity first manifests as an enmeshment with the PhD and consequent diminishment of other roles, relationships and activities that once were integral to the DRs’ sense of self [ 59 , 76 ]. DRs tended to prioritise PhD activities to the extent that they engaged in behaviours that were potentially damaging to their personal relationships [ 76 ]. DRs reported a sense of never being truly free; almost physically burdened by the weight of their PhD and carrying with them a constant ambient guilt [ 37 , 38 , 44 , 76 ]. Time spent on non-PhD activities was positioned as selfish or indulgent, even very basic activities of living [ 76 ].

The seeming incompatibility of aspects of prior personal identity and the PhD appears to result in a sense of internal conflict or identity ‘collision’ [ 59 ]. Friends and relatives often provided an uncomfortable reflection of the DR’s changing identity, leaving DRs feeling hyper-visible and carrying the burden of intellect or trailblazer status [ 74 ]; providing further evidence for the incompatibility of their personal and current and future professional identities. Some DRs more purposefully pruned their relationships and social activities; to avoid identity dissonance, to conserve precious time and energy for their PhD work, or as an acceptance of total enmeshment with academic work as necessary (although not necessarily sufficient) for successful continuation in academia [ 40 , 52 , 77 ]. Nevertheless, the diminishment of the personal identity did not appear balanced by the development of a positive professional identity. The professional DR identity was perceived as unclear and confusing, and the adoption of an academic identity appeared to require DRs to have a greater degree of self-assurance or self-belief than was often the case [ 37 , 81 ].

Self as parasitic

Another change in identity manifested as DRs beginning to conceive of themselves as parasitic. DRs spoke of becoming ‘takers’, feeling that they were unable to provide or give anything to anyone. For some DRs, being ‘parasitic’ reflected them being on the bottom rung of the professional ladder or the ‘bottom of the pile’; thus, professionally only able to receive support and assistance rather than to provide for others. Other DRs reported more purposefully withdrawing from activities in which they were a ‘giver’, for example voluntary work, as providing or caring for others required time or energy that they no longer had [ 38 , 44 ]. Furthermore, DRs appeared to conceive of themselves as also causing difficulty or harm to others [ 81 ], as problems in relation to their PhD could lead them to unwillingly punishing close others, for example, through reducing the duration or quality of time spent together [ 38 ].

Feeling that close others were offering support appeared to heighten the awareness of the toll of the PhD on the individual and their close relationships, emphasising the huge undertaking and the often seemingly slow progress, and actually contributing to the sense of ambient guilt, shame, anger and failure [ 38 ]. Moreover, DRs spoke of feeling extreme guilt in perceiving that they had possibly sacrificed their own, and possibly family members’, current wellbeing and future financial security [ 49 ].

Death of self-agency

In addition to their sense of having to sacrifice their personal identity, DRs also expressed a loss of their sense of themselves as agentic beings. DRs expressed feeling powerless in various domains of their lives. First, DRs positioned the thesis as a powerful force able to overwhelm or swallow them [ 46 , 52 , 59 ]. Secondly, DRs expressed a sense of futility in trying to retain any sense of personal power in the climate of academia. An acute feeling of powerlessness especially in relation to supervisors was evident, with many examples provided of being treated as means to an end, as opposed to ends in themselves [ 39 , 42 , 62 ]. Supervisors did not interact with DRs in a holistic way that recognised their personhood and instead were perceived as prioritising their own will, or the will of other academics, above that of the DR [ 39 , 62 ].

Furthermore, DRs reported feeling as if they were used as a means for research production or furthering their supervisors’ reputations or careers [ 62 ]. DRs perceived that holding on to a sense of personal agency sometimes felt incompatible with having a positive supervisor relationship [ 42 ]. Thus whilst emotional distress, anger, disappointment, sadness, jealousy and resentment were clearly evident in relation to feeling excluded, used or over-powered by supervisors [ 37 , 42 , 52 , 62 ], DRs usually felt unable to change supervisor irrespective of how seriously this relationship had degraded [ 37 , 62 ]. Instead, DRs appeared to take on a position of resignation or defensive pessimism, in which they perceived their supervisors as thwarting their personhood, personal goals and preferences, but typically felt compelled to accept this as the status quo and focus on finishing their PhDs [ 42 ]. DRs resignation was such that they internalised this culture of silence and silenced themselves; tending to share litanies of problems with supervisors whilst prefacing or ending the statements with some contradictory or undermining phrase such as ‘but that’s okay’ [ 42 , 52 ].

The apparent lack of self-agency extended outward from the PhD into DRs not feeling able to curate positive life circumstances more generally [ 76 ]. A lack of time was perhaps the key struggle across both personal and professional domains, yet DRs paradoxically reported spending a lot of time procrastinating and rarely (if ever) mentioned time management as a necessary or desired coping strategy for the problem of having too little time [ 46 ]. The lack of self-agency was not only current but also felt in reference to a bleak and uncertain future; DRs lack of surety in a future in academia and the resultant sense of futility further undermined their motivation to engage currently with PhD tasks [ 38 , 40 ].

The system is sick

The higher-order theme ‘The system is sick’ represents systemic influences on DR mental health. First, ‘Most everyone’s mad here’ reflects the perceived ubiquity mental health problems amongst DRs. ‘Emperor’s new clothes’ reflects the DR experience of engaging in a performative piece in which they attempt to live in accordance with systemic rather than personal values. Finally, ‘Beware the invisible and visible walls’ reflects concerns with being caught between ephemeral but very real institutional divides.

Most everyone’s mad here

No studies focused explicitly on experiences of DRs who had been given diagnoses of mental health problems. Some study participants self-disclosed mental health problems and emphasised their pervasive impact [ 50 ]. Further lived experiences of mental distress in the absence of explicit disclosure were also clearly identifiable. The ‘typical’ presentation of DRs with respect to mental health appeared characterised as almost unanimous [ 39 ] accounts of chronic stress, anxiety and depression, emotional distress including frustration, anger and irritability, lack of mental and physical energy, somatic problems including appetite problems, headaches, physical pain, nausea and problems with drug and alcohol abuse [ 39 , 46 , 59 , 76 ]. Health anxiety, concerns regarding perceived new and unusual bodily sensations and perceived risks of developing stress-related illnesses were also common [ 46 , 59 , 76 ]. A PhD-specific numbness and hypervigilance was also reported, in which DRs might be less responsive to personal life stressors but develop an extreme sensitivity and reactivity to PhD-relevant stimuli [ 39 ].

An interplay of trait and state factors were suggested to underlie the perceived ubiquity of mental health problems amongst DRs. Etiological factors associated with undertaking a PhD specifically included the high workload, high academic standards, competing personal and professional demands, social isolation, poor resources in the university, poor living conditions and poverty, future and career uncertainty [ 36 , 41 , 43 , 46 , 49 , 76 ]. The ‘nexus’ of these factors was such that the PhD itself acted as a crucible; a process of such intensity that developing mental health problems was perhaps inevitable [ 39 ].

The perceived inevitability of mental health problems was such that DRs described people who did not experience mental health problems during a PhD as ‘lucky’ [ 39 ]. Supervisors and the wider academic system were seen to promote an expectation of suffering, for example, with academics reportedly normalising drug and alcohol problems and encouraging unhealthy working practices [ 39 ]. Furthermore, DRs felt that academics were uncaring with respect to the mental challenge of doing a PhD [ 39 ]. Nevertheless, academics were suggested to deny any culpability or accountability for mental health problems amongst DRs [ 39 , 59 , 74 ]. The cycle of indigenousness was further maintained by a lack of mental health literacy and issues with awareness, availability and access to help-seeking and treatment options amongst DRs and academics more widely [ 39 ]. Thus, DRs appeared to feel they were being let down by a system that was almost set up to cause mental distress, but within which there was a widespread denial of the size and scope of the problem and little effort put into identifying and providing solutions [ 39 , 59 ]. DRs ultimately felt that the systemic encouragement of unhealthy lifestyles in pursuit of academic success was tantamount to abuse [ 62 ].

A performance of optimum suffering

Against a backdrop of expected mental distress, DRs expressed their PhD as a performative piece. DRs first had to show just the right amount of struggle and difficulty; feeling that if they did not exhibit enough stress, distress and ill-health, their supervisors or the wider department might not believe they were taking their PhD seriously enough [ 40 ]. At the same time, DRs felt that their ‘researcher mettle’ was constantly being tested and they must rise to this challenge. This included first guarding against presenting oneself as intellectually inferior [ 36 ]. Yet it also seemed imperative not to show vulnerability more broadly [ 74 ]. Disclosing mental or physical health problems might lead not only to changed perceptions of the DR but to material disadvantage [ 74 ]. The poor response to mental health disclosures suggested to some DRs that universities might be purposefully trying to dissuade or discourage DRs with mental health problems or learning disabilities from continuing [ 74 ]. The performative piece is thus multi-layered, in that DRs must experience extreme internal psychological struggles, exhibit some lower-level signs of stress and fatigue for peer and faculty observance, yet avoid expressing any real academic or interpersonal weakness or the disclosure of any diagnosable disability or disease.

Emperor’s new clothes

DRs described feeling beholden to the prevailing culture in which it was expected to prioritise above all else developing into a competitive, self-promoting researcher in a high-performing research-active institution [ 39 , 42 ]. Supervisors often appeared the conduit for transmission of this academic ideal [ 74 ]. DRs felt reticent to act in any way which suggested that they did not personally value the pursuit of a leading research career above all else. For example, DRs felt that valuing teaching was non-conformist and could endanger their continuing success within their current institution [ 55 ]. Many DRs thus exhibited a sense of dissonance as their personal values often did not align with the institutional values they identified [ 74 ]. Yet DRs expressed a sense of powerlessness and a feeling of being ‘caught up’ in the values of the institution even when such values were personally incongruent [ 74 ]. The psychological toll of this sense of inauthenticity seemed high [ 55 ]. Where DRs acted in ways which ostensibly suggested values other than prioritising a research career, for example becoming pregnant, they sensed disapproval [ 76 ]. DRs also felt unable to challenge other ‘institutional myths’ for example, the perceived institutional denial of the duration of and financial struggle involved in completing a PhD [ 49 ]. There was a perceived tendency of academics to locate problems within DRs as opposed to acknowledging institutional or systemic inequalities [ 49 ]. DRs expressed strongly a sense in which there is inequity in support, resources and opportunities, yet universities were perceived as ignoring such inequity or labelling such divisions as based on meritocracy [ 36 , 74 ].

Beware the invisible and visible walls

DRs described the reality of working in academia as needing to negotiate a maze of invisible and visible walls. In the former case, ‘invisible walls’ reflect ephemeral norms and rules that govern academia. DRs felt that a big part of their continuing success rested upon being able to negotiate such rules [ 39 ]. Where rules were violated and explicit or implicit conflicts occurred, DRs were seen to be vulnerable to being caught in the ‘crossfire’ [ 36 ]. DRs identified academic groups and departments as being poor in explicitly identifying, discussing and resolving conflicts [ 37 ]. The intangibility of the ‘invisible walls’ gave rise to a sense of ambient anxiety about inadvertently transgressing norms and divides, such that some DRs reported behaving in ways that surprised even themselves [ 37 ].

Gendered and racial micropolitics of academic institutions were seen to manifest as more visible walls between people, with institutions privileging those with ‘insider’ status [ 36 ]. Women and people of colour typically felt excluded or disadvantaged in a myriad of observable and unobservable ways, with individuals able to experience both insider and outsider statuses simultaneously [ 36 , 37 ], for example when a male person of colour [ 36 ]. Female DRs suggested that not only must women prove themselves to a greater extent than men to receive equal access to resources, opportunities and acclaim but also are typically under additional pressure in both their professional and personal lives [ 37 , 52 , 76 ]. Women also felt that they had to take on more additional roles and responsibilities and encountered more conflicts in their personal lives compared to men [ 52 ]. Examples of professionally successful women in DRs’ departments were described as those who had crossed the divide and adopted a more traditionally male role [ 40 ]. Thus, being female or non-White were considered visible characteristics that would disadvantage people in the competitive academic environment and could give rise to a feeling of increased stress, pressure, role conflicts, and a feeling of being unsafe.

Seeing, being and becoming

The higher-order theme of ‘Seeing, being and becoming’ reflects protective and transformative influences on DR mental health. ‘De-programming’ refers to the DRs disentangling their personal beliefs and values from systemic values and also from their own tendency towards perfectionism. ‘The power of being seen’ reflects the positive impact on DR mental health afforded by feeling visible to personal and professional others. ‘Finding hope, meaning and authenticity’ refers to processes by which DRs can find or re-locate their own self-agency, purpose and re/establish a sense of living in accordance with their values. ‘The importance of multiple goals, roles and groups’ represents the beneficial aspects of accruing and sustaining multiple aspects to one’s identity and connections with others and activities outside the PhD. Finally, ‘The PhD as a process of transcendence’ reflects how the struggles involved in completing a PhD can be transformative and self-actualising.

De-programming

DRs reported being able to protect their mental health by ‘de-programming’ and disentangling their attitudes and practices from social and systemic values and norms. This disentangling helped negate DRs’ adopting unhealthy working practices and offered some protection against experiencing inauthenticity and dissonance between personal and systemic values.

First, DRs spoke of rejecting the belief that they should sacrifice or neglect personal relationships, outside interests and their self-identity in pursuit of academic achievement. DRs could opt-out entirely by choosing a ‘user-friendly’ programme [ 44 ] which encouraged balance between personal and professional goals, or else could psychologically reject the prevailing institutional discourse [ 40 ]. Rather than halting success, de-programming from the prioritisation of academia above all else was seen to be associated not only with reduced stress but greater confidence, career commitment and motivation [ 40 , 50 ]. It was also suggested possible to ‘de-programme’ in the sense of choosing not to be preoccupied by the ‘invisible walls’ of academia and psychologically ‘opt out’ of being concerned by potential conflicts, norms and rules governing academic workplace conduct [ 36 ]. Interaction with people outside of academia was seen to scaffold de-programming, by helping DRs to stay ‘grounded’ and offering a model what ‘normal’ life looks like. People outside of academia could also help DRs to see the truth by providing unbiased opinions regarding systemic practices [ 39 ].

A further way in which de-programming manifested was in DRs challenging their perfectionist beliefs. This include re-framing the goal as not trying to be the archetype of a perfect DR, and accepting that multiple demands placed on one individual invariably requires compromise [ 40 , 76 ]. DRs spoke of the need to conceptualise the PhD as a process, rather than just a product [ 46 , 82 ]. The process orientation facilitated framing of the PhD as just one-step in the broader process of becoming an academic as opposed to providing discrete evidence of worth [ 82 ]. Within this perspective, uncertainty itself could be conceived as a privilege [ 81 ]. The PhD was then seen as an opportunity rather than a test [ 37 , 46 ]. Moreover, the process orientation facilitated viewing the PhD as a means of growing into a contributing member of the research community, as opposed to needing to prove oneself to be accepted [ 82 ]. Remembering the temporary nature of the PhD was advised [ 45 ] as was holding on to a sense that not completing the PhD was also a viable life choice [ 76 ]. DRs also expressed, implicitly or explicitly, a decision to change their conceptualisation of themselves and their progress; choosing not to perceive themselves as stuck, but planning, learning and progressing [ 38 , 39 , 81 , 82 ]. This new perspective appeared to be helpful in reducing mental distress.

The power of being seen

DRs described powerful benefits to feeling seen by other people, including a sense of belonging and mattering, increased self-confidence and a sense of positive progress [ 37 ]. Being seen by others seems to provoke the genesis of an academic identity; it brings DRs into existence as academics. Being seen within the academic institution also supports mental health and can buffer emotional exhaustion [ 37 , 52 , 55 , 81 ]. DRs expressed a need to feel that supervisors, academics and peers were interested in them as people, their values, goals, struggles and successes; yet they also needed to feel that they and their research mattered and made a difference within and outside of the institution [ 42 , 52 , 81 ]. It was clear that DRs could find in their disciplinary communities the sense of belonging that often eluded them within their immediate departments [ 42 ]. Feeling a sense of belonging to the academic community seemed to buffer disengagement and amotivation during the PhD [ 81 ]. Positive engagement with the broader community was scaffolded by a sense of trust in the supervisor [ 81 ]. DRs often felt seen and supported by postdocs, especially where supervisors appeared absent or unsupportive [ 50 ].

Spending time with peers could be beneficial when there was a sense of shared experience and walking alongside each other [ 39 ]. Friendship was seen to buffer stress and protect against mental health problems through the provision of social and emotional support and help in identifying struggles [ 39 , 43 ]. In addition to relational aspects, the provision of designated physical spaces on campus or in university buildings also seemed important to being seen [ 37 ]. Peers in the university could provide DRs with further physical embodiments of being seen, for example, gift-giving in response to their birthdays or returning from leave [ 37 , 50 ]. Outside of the academic institution, DRs described how being seen by close others could support DRs to be their authentic selves, providing an antidote to the invisible walls of academia [ 50 ]. Good quality friendships within or outside academia could be life-changing, providing a visceral sense of connection, belonging and authenticity that can scaffold positive mental health outcomes during the PhD [ 39 ]. Pets could also serve to help DRs feel seen but without needing to extend too much energy into maintaining social relationships [ 50 ].

Finally, DRs also needed to see themselves, i.e. to begin to see themselves as burgeoning academics as opposed to ‘just students’ [ 81 ]. Feeling that the supervisor and broader academic community were supportive, developing one’s own network of process collaborators and successfully obtaining grant funding seemed tangible markers that helped DRs to see themselves as academics [ 37 , 81 ]. Seeing their own work published was also helpful in providing a boost in confidence and being a joyful experience [ 42 ]. Moreover, with sufficient self-agency, DRs can not only see themselves but render themselves visible to other people [ 37 ].

Multiple goals, roles and groups

In antidote to the diminished personal identity and enmeshment with the PhD, DRs benefitted from accruing and sustaining multiple goals, roles, occupations, activities and social group memberships. Although ‘costly’ in terms of increased stress and role conflicts, sustaining multiple roles and activities appeared essential for protecting against mental health problems [ 50 , 68 ].

Leisure activities appeared to support mental health through promoting physical health, buffering stress, providing an uplift to DRs’ mood and through the provision of another identity other than as an academic [ 44 , 50 , 76 ]. Furthermore, engagement in activities helped DRs to find a sense of freedom, allowing them to carve up leisure and work time and psychologically detach from their PhD [ 68 , 76 ]. Competing roles, especially family, forced DRs to distance themselves from the PhD physically which reinforced psychological separation [ 50 , 59 ]. Engaging in self-care and enjoyable activities provided a sense of balance and normalcy [ 39 , 44 , 68 ]. This normalcy was a needed antidote to abnormal pressure [ 59 ]. Even in the absence of fiercely competing roles and priorities, DRs still appeared to benefit from treating their PhD as if it is only one aspect of life [ 59 ]. Additional roles and activities reduced enmeshment with the PhD to the extent that considering not completing the PhD was less averse [ 40 ]. This position appeared to help DRs to be less overwhelmed and less sensitive to perceived and anticipated failures.

Finding hope, meaning and authenticity

Finding hopefulness and meaning within the PhD can scaffold a sense of living a purposeful, enjoyable, important and authentic life. Hopefulness is predicated on the ability to identify a goal, i.e. to visualise and focus on the desired outcome and to experience both self-agency and potential pathways towards the goal. Hopefulness was enhanced by the ability to break down tasks into smaller goals and progress in to ‘baby steps’ [ 38 , 59 ]. In addition, DRs benefitted from finding explicit milestones against which they can compare their progress [ 59 ], as this appeared to feed back into the cycle of hopeful thinking and spur further self-agency and goal pursuit.

The experience of meaning manifested in two main ways; first as the more immediate lived experience of passion in action [ 76 ]. Secondly, DRs found meaning in feeling that in their PhD and lives more broadly they were living in accordance with their values, for example, experiencing their own commitment in action through continuing to work on their PhD even when it was difficult to do so [ 76 ]. DRs who were able to locate their PhD within a broader sense of purpose appeared to derive wellbeing benefits. There was a need to ensure that values were in alignment, for example, finding homeostasis between emotional, intellectual, social and spiritual parts of the self [ 46 , 59 , 90 ].

The processes of finding hopefulness and meaning appear to be largely relational. Frequent contact with supervisors in person and social and academic contact with other DRs were basic scaffolds for hope and meaning [ 52 ]. DRs spoke of how a sense that their supervisors believed in them inspired their self-agency and motivation [ 42 , 62 , 76 ]. Partners, friends and family could also inspire motivation for continuing in PhD tasks [ 44 , 76 ]. Other people also could help instil a sense of motivation to progress and complete the PhD; a sense of being seen is to be beholden to finish [ 39 ]. Meaning appeared to be scaffolded by a sense of contribution, belonging and mattering [ 81 ] and could arise from the perception of putting something into the collective pot, inspiring hopefulness and helping others [ 39 , 42 ]. Moreover, hopefulness, meaning and authenticity also appeared mutually reinforcing [ 81 ]. Finding meaning and working on a project which is in accordance with personal values, preferences and interests is also helpful in completing the PhD and provides a feedback loop into hope, motivation and agentic thinking [ 39 , 81 ]. Furthermore, DRs could use agentic action to source a community of people who share their values, enabling them to engage in collective authenticity [ 39 ].

The PhD as a process of transcendence

The immense challenge of the PhD could be a catalyst for growth, change and self-actualisation, involving empowerment through knowledge, self-discovery, and developing increased confidence, maturity, capacity for self-direction and use of one’s own autonomy [ 44 , 82 ]. The PhD acted as a forge in which DRs were tested and became remoulded into something greater than they had been before [ 44 , 82 , 90 ]. The struggles endured during the PhD caused DRs to reconsider their sense of their own capacities, believing themselves to be more able than they previously would have thought [ 50 ]. The struggles endured added to the sense of accomplishment. A trusted and trusting supervisor appears to aid in the PhD being a process of transcendence [ 62 ].

More broadly, the PhD also helped DRs to transcend personal tragedy, allowing immersion in a meaningful activity which begins as a means of coping and becomes something completely [ 39 ]. The PhD could also serve as a transformative selection process for DRs’ social relationships, with some relationships cast aside and yet others formed anew [ 39 ]. Overall, therefore, the very aspects of the PhD which were challenging, and distressing could allow DRs to transcend their former selves and, through the struggle, become something more.

Summation of results

The findings regarding the risk and protective factors associated with DR mental health have been summarised in Table 3 in relation to (1) the type of research evidencing the factor (i.e. whether the evidence is quantitative only, part of the meta-synthesis only, or evident in both results sections); and (2) the volume of evidence (i.e. whether the factor was found in a single study or across multiple studies). The factors in the far-right column (i.e. the factors found across multiple research studies utilising both qualitative and quantitative methods) are the ones with the strongest evidence at present.

This systematic review summarises a heterogeneous research area, with the aim of understanding the mental health of DRs, including possible risk and protective factors. The qualitative and quantitative findings presented here suggest that poor mental health is a pertinent problem facing DRs; stress appears to be a key issue and significantly in excess of that experienced in the general population. Several risk and protective factors at the individual, interpersonal and systemic levels emerged as being important in determining the mental health of DRs. The factors with the strongest evidence-base (i.e. those supported by multiple studies using qualitative and quantitative findings) denote that being female and isolated increases the risk of the mental health problems, whereas seeing the PhD as a process, feeling socially supported, having a positive supervisor relationship and engaging in self-care is protective.

Results in context

Stress can be defined as (1) the extent to which a stimulus exerts pressure on an individual, and their propensity to bear the load; (2) the duration of the response to an aversive stimuli, from initial alert to exhaustion; or (3) a dynamic (im)balance between the demands and personal resource to manage those demands [ 91 ]. The Perceived Stress Scale (PSS) [ 18 , 19 ] used in our meta-analysis is aligned with the third of these definitions. As elaborated upon within the Transactional Model of Stress [ 92 ], stress is conceptualised as a persons’ appraisal of the internal and external demands put upon them, and whether these exceed their available resources. Thus, our results suggest that, when compared to the general population, PhD students experience a greater maladaptive imbalance between their available resources and the demands placed upon them. Stress in itself is not a diagnosable mental health problem, yet chronic stress is a common precipitant to mental health difficulties such as depression and posttraumatic stress disorder [ 93 , 94 ]. Therefore, interventions should seek to bolster DRs’ resources and limit demands placed on them to minimise the risks associated with acute stress and limit its chronicity.

Individual factors

Female DRs were identified as being at particular risk of developing mental health difficulties. This may result from additional hurdles when studying in a male-dominated profession [ 95 , 96 , 97 ], and the expectation that in addition to their doctoral studies, females should retain sole or majority responsibility for the domestic and/or caring duties within their family [ 52 , 76 ]. It may also be that females are more willing to disclose and seek help for mental health difficulties [ 98 ]. Nevertheless, the World Health Organisation (WHO) mental health survey results indicate that whilst anxiety and mood disorders are more prevalent amongst females, externalising disorders are more common in males [ 99 ]. As the vast majority of studies in this review focussed on internalising problems (e.g. stress, anxiety and depression) [ 37 , 64 , 79 , 80 , 83 , 89 ], this may explain the gender differences found in this review. Further research is needed to explore which perspective, if any, may explain gender gap in our results.

Perhaps unsurprisingly, self-care was associated with reduced mental health problems. The quantitative findings suggest that all types of self-care are likely to be protective of mental health (i.e. physical, emotional, professional and spiritual self-care). Self-care affords DRs the opportunity to take time away from their studies and nurture their non-PhD identities. However, the results from our meta-synthesis suggest that DRs are not attending to their most basic needs much less engaging in self-care behaviours that correspond to psychological and/or self-fulfilment needs [ 100 ]. Consequently, an important area for future enquiry will be identifying the barriers preventing DRs from engaging in self-care.

Interpersonal factors

Across both quantitative and qualitative studies, interpersonal factors emerged as the most salient correlate of DR mental health. That is, isolation was a risk factor, whereas connectedness to others was a protective factor. There was some variability in how these constructs were conceptualised across studies, i.e. (1) isolation: a lack of social support, having fewer social connections, feeling isolated or being physically separate from others; and (2) social connectedness: multiple group membership, academic relationships or non-academic relationships; but there was no indication that effects varied between concepts. The relationship between isolation and negative health consequences is well-established, for example both physical and mental health problems [ 101 ], and even increased mortality [ 102 ]. Conversely, social support is associated with reduced stress in the workplace [ 103 , 104 ]. Reducing isolation is therefore a promising interventional target for improving DRs’ mental health.

The findings regarding isolation are even more alarming when considered alongside the findings from several studies that PhD studies are consistently reported to dominate the lives of DRs, resulting in poor ‘work-life balance’ and losing non-PhD aspects of their identities. The negative impact of having fewer identities [ 105 ] can be mitigated by having a strong support network [ 106 ], and increasing multiple group memberships [ 107 ]. But for DRs, it is perhaps the absence of this social support, combined with identity impoverishment, which can explain the higher than average prevalence of stress found in our meta-analysis.

Systemic factors

DRs’ attitudes towards their studies may be a product of top-down systemic issues in academia more broadly. Experiencing mental health problems was reported as being the ‘norm’, but also appeared to be understood as a sign of weakness. The meta-synthesis results suggest that DRs believed their respective universities prioritise academic success over workplace wellbeing and encourage unhealthy working habits. Working in an unsupportive and pressured environment is strongly associated with negative psychological outcomes, including increased depression, anxiety and burnout [ 108 ]. The supervisory relationship appeared a particularly important aspect of the workplace environment. The quantitative analysis found a negative correlation between inspirational supervision and mental health problems. Meta-synthetic finding suggested toxic DR-supervisor relationships characterised by powerlessness and neglect, as well as relationships where DRs felt valued and respected—the former of these being associated with poor mental health, and the latter being protective. The association between DR-supervisor relationship characteristics needs to be verified using quantitative methods. Furthermore, DRs’ sense that they needed to exhibit ‘optimum suffering’, which appears to reflect a PhD-specific aspect of a broader academic performativity [ 109 ], is an important area for consideration. An accepted narrative around DRs needing to experience a certain level of dis/stress would likely contribute to poor mental health and as an impediment to the uptake and effectiveness of proffered interventions. Although further research is needed, it is apparent that individual interventions alone are not sufficient to improve DR mental health, and that a widespread culture shift is needed in order to prevent the transmission of unhealthy work attitudes and practices.

Limitations of the literature

Although we found a respectable number of articles in this area, the focus and measures used varied to the extent that typical review analysis procedures could not be used. That is, there was much heterogeneity in terms of how mental health was conceptualised and measured, as well as the range of risk and protective factors explored. Similarly, the quality of the studies was hugely variable. Common flaws amongst the literature include small sample sizes, the use of unvalidated tools and the incomplete reporting of results. Furthermore, for qualitative studies specifically, there appeared to be a focus on breadth instead of depth, particularly in relation to studies using mixed methods.

The generalisability of our findings is limited largely due to the lack of research conducted outside of the US, but also because we limited our review to papers written in English only. The nature of doctoral studies varies in important ways between studies. For example, in Europe, PhD studies usually apply for funding to complete their thesis within 3–4 years and must know their topic of interest at the application stage. Whereas in the US, PhD studies usually take between 5 and 6 years, involve taking classes and completing assignments, and the thesis topic evolves over the course of the PhD. These factors, as well as any differences in the academic culture, are likely to affect the prevalence of mental health problems amongst DRs and the associated risk and protective factors. More research is needed outside of the US.

‘Mental health’ in this review was largely conceptualised as a type of general wellbeing rather than a clinically meaningful construct. None of the studies were ostensibly focused on sampling DRs who were currently experiencing or had previously experienced mental health problems per se, meaning the relevance of the risk, vulnerability and protective factors identified in the meta-synthesis may be more limited in this group. Few studies used clinically meaningful measures. Where clinical measures were used, they captured data on common mental health problems only (i.e. anxiety and depression). Due to these limitations, we are unable to make any assertions about the prevalence of clinical-level mental health problems amongst DRs.

Limitations of this review

As a result of the heterogeneity in this research area, some of the results presented within this review are based on single studies (e.g. correlation data; see Fig. 5 ) rather than the amalgamation of several studies (e.g. meta-analysis/synthesis). To aid clarity when interpreting the results of this review, we have (Table 3 ) summarised the volume of evidence supporting risk and protective factors. Moreover, due to the small number of studies eligible for inclusion in this review, we were unable to test whether any of our findings are related to the study characteristics (e.g. year of publication, country of origin, methodology).

We were able to conduct three meta-analyses, one of which aimed to calculate the between-group effect size on the PSS [ 18 , 19 ] between DRs and normative population data. Comparing these data allowed us to draw some initial conclusions about the prevalence of stress amongst DRs, yet we were unable to control for other group differences which might moderate stress levels. For example, the population data was from people in the United States (US) in 1 year, whereas the DR data was multi-national at a variety of time points; and self-reported stress levels may vary with nationality [ 110 ] or by generation [ 111 , 112 ]. Moreover, two of the three meta-analyses showed significant heterogeneity. This heterogeneity could be explained by differences in the sample characteristics (e.g. demographics, country of origin), doctoral programme characteristics (e.g. area of study, funding status, duration of course) or research characteristics (e.g. study design, questionnaires used). However, due to the small number of studies included in these meta-analyses, we were unable to test any of these hypotheses and are therefore unable to determine the cause of this heterogeneity. As more research is conducted on the mental health of DRs, we will be able to conduct larger and more robust meta-analyses that have sufficient power and variance to statistically explore the causes of this heterogeneity. At present, our findings should be interpreted in light of this limitation.

Practice recommendations

Although further research is clearly needed, we assert that this review has identified sufficient evidence in support of several risk and protective factors to the extent that they could inform prevention and intervention strategies. Several studies have evidenced that isolation is toxic for DRs, and that social support can protect against poor mental health. Initiatives that provide DRs with the opportunity to network and socialise both in and outside of their studies are likely to be beneficial. Moreover, there is support for psychoeducation programmes that introduce DRs to a variety of self-care strategies, allow them to find the strategies that work for them and encourage DRs to make time to regularly enact their chosen strategies. Finally, the supervisory relationship was identified as an important correlate of DR mental health. Positive supervision was characterised as inspirational and inclusive, whereas negative supervision productised DRs or neglected them altogether. Supervisor training programmes should be reviewed in light of these findings to inform how institutions shape supervisory practices. Moreover, the initial findings reported here evidence a culture of normalising and even celebrating suffering in academia. It is imperative therefore that efforts to improve and protect the mental health of DRs are endorsed by the whole institution.

Research recommendations

First, we encourage further large-scale mental health prevalence studies that include a non-PhD comparison group and use validated clinical tools. None of the existing studies focused on the presence of serious mental health problems—this should be a priority for future studies in this area. Mixed-methods explorations of the experiences of DRs who have mental health problems, including serious problems, and in accessing mental health support services would be a welcome addition to the literature. More qualitative studies involving in-depth data collection, for example interview and focus group techniques, would be useful in further supplementing findings from qualitative surveys. Our review highlights a need for better communication and collaboration amongst researchers in this field with the goal of creating a level of consistency across studies to strengthen any future reviews on this subject.

The results from this systematic review, meta-analysis and meta-synthesis suggest that DRs reported greater levels of stress than the general population. Research regarding the risk and protective factors associated with the mental health of DRs is heterogenous and disparate. Based on available evidence, robust risk factors appear to include being isolated and being female, and robust protective factors include social support, viewing the PhD as a process, a positive DR-supervisor relationship and engaging in self-care. Further high-quality, controlled research is needed before any firm statements can be made regarding the prevalence of clinically relevant mental health problems in this population.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Confidence intervals

Doctoral researchers

Higher Education Statistics Agency

Perceived Stress Scale

Standard deviation

United Kingdom

United States

Baron E. Eleven sketches inspired by the university mental health crisis—in pictures. The Guardian. 2017. Available from: https://www.theguardian.com/education/gallery/2017/jun/27/eleven-sketches-university-mental-health-crisis . Cited 2017 Oct 6.

Google Scholar  

National Union of Students. 20 per cent of students consider themselves to have a mental health problem: National Union of Students; 2013. Available from: https://www.nus.org.uk/en/news/20-per-cent-of-students-consider-themselves-to-have-a-mental-health-problem/ . Cited 2017 Oct 6.

YouGov. One in four students suffer from mental health problems. 2016. Available from: https://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/obtomdatp4/Survey_Results.pdf . Cited 2017 Oct 6.

Universities UK. Minding Our Future: starting a conversation about the support of student mental health. London: Universities UK; 2017. Available from: https://www.universitiesuk.ac.uk/minding-our-future .

The Guardian. Number of university dropouts due to mental health problems trebles. The Guardian. 2017. Available from: https://www.theguardian.com/society/2017/may/23/number-university-dropouts-due-to-mental-health-problems-trebles . Cited 2017 Oct 6.

Thorley C. Not By Degrees: Improving student mental health in the UK’s universities. London; 2017. Available from: www.ippr.org . Cited 2017 Oct 6.

Oswalt SB, Lederer AM, Chestnut-Steich K, Day C, Halbritter A, Ortiz D. Trends in college students’ mental health diagnoses and utilization of services, 2009–2015. J Am Coll Health. 2018;68:41–51.

Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO world mental health surveys international college student project: prevalence and distribution of mental disorders. J Abnorm Psychol. 2018;127(7):623–38.

PubMed   PubMed Central   Google Scholar  

OECD. Education at a Glance 2019. Education at a Glance: OECD Indicators: OECD; 2019. (Education at a Glance). Available from: http://gpseducation.oecd.org/Content/EAGCountryNotes/BRA.pdf . Cited 2020 Mar 26.

OECD. OECD Science, Technology and Innovation Outlook 2016. OECD Science, technology and innovation outlook 2016. 2016. Available from: http://www.oecd.org/sti/STIO 10 key technology trends for the future.pdf. Cited 2020 Mar 26.

Higher Education Statistics Agency (HESA). Higher education student statistics: UK, 2016/17: HESA; 2018. Available from: https://www.hesa.ac.uk/news/11-01-2018/sfr247-higher-education-student-statistics/numbers .

NCES. Number of doctoral degrees earned in the United States from 1949/50 to 2028/29, by gender. 2019. Available from: https://www.statista.com/statistics/185167/number-of-doctoral-degrees-by-gender-since-1950/ .

Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46 Available from: http://psycnet.apa.org/index.cfm?fa=search.displayRecord&uid=1960-06759-001 .

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159 Available from: http://www.jstor.org/stable/2529310?origin=crossref . Cited 2017 Mar 8.

CAS   PubMed   Google Scholar  

Noblit GW, Hare RD. Meta-ethnography: Synthesizing qualitative studies: Sage Publications; 1988. p. 88.

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45.

Wilson DB. Meta-analysis stuff. 2011. Available from: http://mason.gmu.edu/~dwilsonb/ma.html . Cited 2017 Dec 3.

Cohen S, Williamson GM. Perceived stress in a probability sample of the U.S. In: Spacapam S, Oskamp S, editors. The social psychology of health: Claremont Symposium on Applied Social Psychology. Newbury Park, CA: Sage; 1988.

Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–96.

Cohen S, Janicki-Deverts D. Who’s Stressed? Distributions of psychological stress in the United States in probability samples from 1983, 2006, and 2009. J Appl Soc Psychol. 2012;42(6):1320–34.

MedCalc Software bvba. MedCalc Statistical Software. Belgium: Ostend; 2016.

Viechtbauer W. Bias and efficiency of Meta-analytic variance estimators in the random-effects model. J Educ Behav Stat. 2005;30(3):261–93 Available from: http://jeb.sagepub.com/cgi/doi/10.3102/10769986030003261 . Cited 2017 Mar 8.

Hedges LV, Ollkin L. Statistical methods for meta-analysis. New York: Academic Press; 1985. Available from:. https://doi.org/10.1002/9780470743386.refs .

Book   Google Scholar  

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: Wiley; 2009. Cited 2019 Jan 10. Available from. https://doi.org/10.1002/9780470743386 .

Fisher RA. Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika. 1915;10(4):507–21 Available from: https://www.statista.com/statistics/185167/number-of-doctoral-degrees-by-gender-since-1950/ .

Schünemann H, Brożek J, Guyatt G, Oxman A. Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach: GRADE Working Group; 2013. Available from: gdt.guidelinedevelopment.org/app/handbook/handbook.html .

Lachal J, Revah-Levy A, Orri M, Moro MR. Metasynthesis: an original method to synthesize qualitative literature in psychiatry. Front Psychiatry. 2017;8:269.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: Sage Publications Ltd; 2013.

Bhaskar R. The possibility of naturalism: a philosophical critique of the contemporary human sciences. London: Routledge; 2014.

Fletcher AJ. Applying critical realism in qualitative research: methodology meets method. Int J Soc Res Methodol. 2017;20(2):181–94.

Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs. 2005;50(2):204–11.

PubMed   Google Scholar  

Maton K. Reflexivity, relationism, &amp; research: Pierre Bourdieu and the epistemic conditions of social scientific knowledge. Sp Cult. 2003;6(1):52–65.

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12(1):181.

von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for Reporting Observational Studies. Ann Intern Med. 2007;147(8):573. Cited 2019 Sep 27. Available from. https://doi.org/10.7326/0003-4819-147-8-200710160-00010 .

Article   Google Scholar  

Acker S, Haque E. The struggle to make sense of doctoral study. High Educ Res Dev. 2015;34(2):229–41. Available from. https://doi.org/10.1080/07294360.2014.956699 .

Appel ML, Dahlgren LG. Swedish doctoral students’ experiences on their journey towards a PhD: obstacles and opportunities inside and outside the academic building. Scand J Educ Res. 2003;47(1):89–110. Available from. https://doi.org/10.1080/00313830308608 .

Devonport TJ, Lane AM. In it together: Dyadic coping among doctoral students and partners. J Hosp Leis Sport Tour Educ. 2014;15:124–34. Available from. https://doi.org/10.1016/j.jhlste.2014.08.002 .

Enzor J. Friendship, mental health, and doctoral education: a generic qualitative thematic analysis: Capella University; 2017.

Kurtz-Costes B, Helmke LA, Ülkü-Steiner B. Gender and doctoral studies: the perceptions of Ph.D. students in an American university. Gend Educ. 2006;18(2):137–55.

Bazrafkan L, Shokrpour N, Yousefi A, Yamani N. Management of stress and anxiety among PhD students during thesis writing: a qualitative study. The Health Care Manager. 2016;35:231–40.

Cotterall S. More than just a brain: emotions and the doctoral experience. High Educ Res Dev. 2013;32(2):174–87.

Kaufman JA. Personal perceptions of stress and self-perceived need for social support among doctoral psychology students in a distance education university sample: Capella University; 2004.

Kenty JR. Stress management strategies for women doctoral students. Nurse Educ. 2000;25(5):251–4 Available from: http://pesquisa.bvsalud.org/portal/resource/pt/mdl-16646205 .

Scrubb MM. An examination of the doctoral student stress survey (DSSS) as an instrument for measuring the effects of stress as perceived by doctoral students in a distance learning university: Walden University; 1997.

Usman Yousaf S, Akram M, Usman B. Exploring the causes of stress and coping with it amongst doctoral level students: highlighting the importance of information collection and management. Pakistan J Inf Manag Libr. 2016;18(2):12–25 Available from: https://search.proquest.com/openview/68246cbba8a9da0ec067d56631b479f1/1?pq-origsite=gscholar&cbl=54989 .

Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis Studies. Res Sch. 2006;13(1):29 Available from: http://www.ncbi.nlm.nih.gov/pubmed/20098638%0A http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC2809982 .

Bauer J. Personality factors, self-care, and perceived stress levels on counselor education and counseling psychology doctoral students. Dissertations: Western Michigan University; 2016.

Begun AL, Carter JR. Career Implications of Doctoral Social Work Student Debt Load. J Soc Work Educ. 2017;53(2):161–73. Available from. https://doi.org/10.1080/10437797.2016.1243500 .

Benjamin S, Williams J, Maher MA. Focusing the lens to share the story: using photographs and interviews to explore doctoral students’ sense of well-being. Int J Dr Stud. 2017;12:197–217.

Benesek JP. Stress and coping among psychology doctoral students: a comparison of self-reported stress levels and coping styles of PhD and PsyD students: University of Hartford; 1998.

Bireda AD. Challenges to the doctoral journey: a case of female doctoral students from Ethiopia. Open Prax. 2015;7(4):287–97.

Bolliger DU, Halupa C. Student perceptions of satisfaction and anxiety in an online doctoral program. Distance Educ. 2012;33(1):81–98.

Cole LJ. Academic worry and frequent mental distress among online doctoral students: Walden University; 2008.

Devine K, Hunter KH. PhD student emotional exhaustion: the role of supportive supervision and self-presentation behaviours. Innov Educ Teach Int. 2017;54(4):335–44.

Drake KL. Psychology graduate student well-being: the relationship between stress, coping, and health outcomes: University of Cincinnati; 2010.

Dumitrescu GA. Self-efficacy, locus of control , perceived stress and student satisfaction as correlates of dissertation completion: Andrews University; 2016.

El-Ghoroury NH, Galper DI, Sawaqdeh A, Bufka LF. Stress, coping, and barriers to wellness among psychology graduate students. Train Educ Prof Psychol. 2012;6(2):122–34.

Haynes C, Bulosan M, Citty J, Grant-Harris M, Hudson J, Koro-Ljungberg M. My world is not my doctoral program…Or is it?: Female students’ perceptions of well-being. Int J Dr Stud. 2012;7:001–17.

Hill LM. Perceived stress, academic support, social support, and professional support factors as predictors of student success in distributed-learning doctoral education: Fielding Graduate University; 2010.

Holahan CK. Stress experienced by women doctoral students, need for support, and occupational sex typing: An interactional view. Sex Roles. 1979;5(4):425–36.

Hunter KH, Devine K. Doctoral students’ emotional exhaustion and intentions to leave academia. Int J Doctoral Stud. 2016;11.

Kaufman JA. Stress and social support among online doctoral psychology students. J College Stud Psychother. 2006;20(3):79–88 Available from: http://search.proquest.com/docview/57185202?accountid=12253%5Cn http://man-fe.hosted.exlibrisgroup.com/openurl/44MAN/44MAN_services_page?url_ver=Z39.88-2004&rft_val_fmt=journal&genre=unknown&sid=ProQ:ProQ%3Aeducation&atitle=Stress+and+Social+Support+among +.

Levecque K, Anseel F, De Beuckelaer A, Van Der Heyden J, Gisle L. Work organization and mental health problems in PhD students. Res Policy. 2017;46:868–79.

Lonka K, Chow A, Keskinen J, Hakkarainen K, Sandström N, Pyhältö K. How to measure PhD students’ conceptions of academic writing - and are they related to wellbeing? J Writ Res. 2014;5(3):245–69.

Lowe RL. The relationship between personality, self-care, stress, and perceived wellness in psychology doctoral students: Tennessee University; 2015.

Marais GAB, Shankland R, Haag P, Fiault R, Juniper B. A survey and a positive psychology intervention on French PhD student well-being. Int J Dr Stud. 2018;13:109–38.

Martinez E, Ordu C, Sala MRD, McFarlane A. Striving to obtain a school-work-life balance: The full-time doctoral student. Int J Dr Stud. 2013;8:39–59.

McGregor BA, Antoni MH, Ceballos R, communication BBBS. very low CD19+ B-lymphocyte percentage is associated with high levels of academic stress among healthy graduate students. Stress Heal. 2008;24(5):413–8.

Nelson K. Academic progress in doctoral students: Levels of hope, subjective well-being, and stress: Walden University; 2014. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc12&NEWS=N&AN=2015-99151-189 .

Nottingham K. A predictive analysis of the psychosocial outcomes of doctoral students: Capella University; 2017.

Orozco AB. Self-care effects on anxiety in doctoral students: Fielding Graduate University; 2014. p. 35–61.

Peters BM. The relationships among physiological and perceived stress, quality of life, self-care and impairment in doctoral students: State University of New York at Buffalo; 2007.

Pifer MJ, Baker VL. “It could be just because I’m different”: Otherness and its outcomes in doctoral education. J Divers High Educ. 2014;7(1):14–30.

Platt J, Schaefer C. Clinical psychological students’ subjective stress ratings during their doctoral training. Psychol Rep. 1995;76:994 Available from: http://www.ncbi.nlm.nih.gov/pubmed/7568619 .

Pychyl TA. Personal projects, subjective well-being and the lives of doctoral students. Ottawa: Carleton University; 1995.

Pychyl TA, Little BR. Dimensional specificity in the prediction of subjective well-being: Personal projects in pursuit of the PhD. Soc Indic Res. 1998;45(1–3):423–73 Available from: file://d/d/Texte/Guenter/Literatu/Original/Journals/SIR98-45-423-473.pdf.

Rocha-Singh IA. Perceived stress among graduate students: development and validation of the Graduate Stress Inventory. Educ Psychol Meas. 1994;54(3):714–27.

Scheidler JA. Effects of perceived stress and perceived social support on marital satisfaction in doctoral students: Walden University. 2008;20.

Sekas G, Wile MZ. Stress-related illnesses and sources of stress: comparing M.D., Ph.D., M.D. and Ph.D. students. J Med Educ. 1980;55:440–6 Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L10000589%5Cn http://sfx.library.uu.nl/utrecht?sid=EMBASE&issn=00222577&id=doi:&atitle=Stress-related+illnesses+and+sources+of+stress%3A+comparing+M.D.-Ph.D.%2C+M.D.+and+Ph.D.+student .

Stubb J, Pyhältö K, Lonka K. Balancing between inspiration and exhaustion: PhD students’ experienced socio-psychological well-being. Stud Contin Educ. 2011;33(1):33–50.

Stubb J, Pyhältö K, Lonka K. The experienced meaning of working with a PhD thesis. Scand J Educ Res. 2012;56(4):439–56.

Ülkü-Steiner B, Kurtz-Costes B, Kinlaw CR. Doctoral student experiences in gender-balanced and male-dominated graduate programs. J Educ Psychol. 2000;92(2):296–307.

Volkert D, Candela L, Bernacki M. Student motivation, stressors, and intent to leave nursing doctoral study: a national study using path analysis. Nurse Educ Today. 2018;61:210–5. Available from. https://doi.org/10.1016/j.nedt.2017.11.033 .

Article   PubMed   Google Scholar  

Waaijer CJF, Heyer A, Kuli S. Effects of appointment types on the availability of research infrastructure, work pressure, stress, and career attitudes of PhD candidates of a Dutch university. Res Eval. 2016;25(4):349–57.

Wang C-H, Chen Y-W, Wu T-Y. Self-guided bibliotherapy: a case study of a Taiwanese doctoral student. Int J Humanties. 2010;8(1):413–22.

Williams MD. HBCU vs. PWI: institutional integration at PWIs and Black doctoral student depression, anxiety, and stress: University of Minnesota; 2014.

Wright T. Issues in brief counselling with postgraduate research students. Couns Psychol Q. 2006;19(4):357–72.

Scrubb MM. An examination of the Doctoral Student Stress Survey (DSSS) as an instrument for measuring the effects of stress as perceived by doctoral students in a distance learning university, vol. 58: Walden University; 1998. p. 3041. Dissertation Abstracts International Section A: Humanities and Social Sciences. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc3&NEWS=N&AN=1998-95003-070 .

Hadden BW, Smith CV. I gotta say, today was a good (and meaningful) day: daily meaning in life as a potential basic psychological need. J Happiness Stud. 2017:1–18.

Butler G. Definitions of stress. Occas Pap R Coll Gen Pract. 1993;(61):1–5 Available from: http://www.ncbi.nlm.nih.gov/pubmed/8199583%0A http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC2560943 .

Lazarus RS, Folkman S. Stress, Appraisal, and Coping. New York: Springer Publishing Company; 1984.

Siegrist J. Chronic psychosocial stress at work and risk of depression: evidence from prospective studies. Eur Arch Psychiatry Clin Neurosci. 2008;258(SUPPL. 5):115–9.

Marin MF, Lord C, Andrews J, Juster RP, Sindi S, Arsenault-Lapierre G, et al. Chronic stress, cognitive functioning and mental health. Neurobiol Learn Mem. 2011;96(4):583–95. Available from. https://doi.org/10.1016/j.nlm.2011.02.016 .

Ward M. The gender salary gap in British academia. Appl Econ. 2001;33(13):1669–81. Cited 2019 May 23. Available from. https://doi.org/10.1080/00036840010014445 .

Howe-Walsh L, Turnbull S. Barriers to women leaders in academia: tales from science and technology. Stud High Educ. 2016;41(3):415–28. Cited 2019 May 23. Available from. https://doi.org/10.1080/03075079.2014.929102 .

Gardiner M, Tiggemann M. Gender differences in leadership style, job stress and mental health in male- and female-dominated industries. J Occup Organ Psychol. 1999;72(3):301–15.

Mackenzie CS, Gekoski WL, Knox VJ. Age, gender, and the underutilization of mental health services: the influence of help-seeking attitudes. Aging Ment Health. 2006;10(6):574–82 Cited 2017 May 12. Available from: http://www.tandfonline.com/action/journalInformation?journalCode=camh20 .

Seedat S, Scott KM, Sampson NA, Williams D, Kessler RC. Cross-national associations between gender and mental disorders in the World Health Organization World Mental Health Surveys. Arch Gen Pschiatry. 2013;66(7):785–95.

Maslow AH. A theory of human motivation. Psychol Rev. 1943;50(4):370–96 Available from: http://content.apa.org/journals/rev/50/4/370 . Cited 2019 Jun 10.

Leigh-Hunt N, Bagguley D, Bash K, Turner V, Turnbull S, Valtorta N, et al. An overview of systematic reviews on the public health consequences of social isolation and loneliness. Public Health. 2017;152:157–71 Available from: https://doi.org/10.1016/j.puhe.2017.07.035 .

Holt-Lunstad J, Smith TB, Baker M, Harris T, Stephenson D. Loneliness and social isolation as risk factors for mortality. Perspect Psychol Sci. 2015;10(2):227–37. Available from. https://doi.org/10.1177/1745691614568352 .

Viswesvaran C, Sanchez JI, Fisher J. The role of social support in the process of work stress: a meta-analysis. J Vocat Behav. 1999;54(2):314–34.

Michie S, Williams S. Reducing work related psychological ill health and sickness absence: a systematic literature review. Occup Environ Med. 2003;60(1):3–9.

CAS   PubMed   PubMed Central   Google Scholar  

Brook AT, Garcia J, Fleming M. The effects of multiple identities on psychological well-being. Personal Soc Psychol Bull. 2008;34(12):1588–600.

Thoits PA. Self, Identity, Stress, and Mental Health. In: Aneshensel CS, Phelan JC, Bierman A, editors. Handbook of the Sociology of Mental Health. Dordrecht: Springer; 2012. p. 357–77.

Haslam C, Jetten J, Cruwys T, Dingle G, Haslam A. The new psychology of health: Unlocking the social cure. New York: Routledge; 2018.

Faragher EB, Cass M, Cooper CL. The relationship between job satisfaction and health: a meta-analysis. Occup Environ Med. 2005;62(2):105–12.

Macfarlane B. Student performativity in higher education: converting learning as a private space into a public performance. High Educ Res Dev. 2015;34(2):338–50.

Daniels K. Perceived risk from occupational stress: a survey of 15 European countries. Occup Environ Med. 2004;61(5):467–70.

American Psychiatric Association (APA). Stress in America: missing the health care connection. Washington: American Psychiatric Association (APA). 2013.

Twenge JM. Time period and birth cohort differences in depressive symptoms in the U.S., 1982–2013. Soc Indic Res. 2015;121(2):437–54.

Download references

Acknowledgements

Thank you to the Office for Students for their funding to support this work; and thank you to the University of Sussex Doctoral school and our steering group for championing and guiding the ‘Understanding the mental health of Doctoral Researchers (U-DOC)’ project.

The present project was supported by the Office for Students Catalyst Award. The funder had no involvement in the design of the study, the collection, analysis or interpretation of the data, nor the writing of this manuscript.

Author information

Authors and affiliations.

School of Social Sciences, University of Westminster, 115 New Cavendish Street, London, W1W 6UW, UK

Cassie M. Hazell

School of Psychology, University of Sussex, Falmer, Brighton, BN1 9QJ, UK

Laura Chapman

Research and Enterprise, University of Sussex, Falmer, Brighton, BN1 9RH, UK

Sophie F. Valeix

Centre for Higher Education and Equity Research, University of Sussex, Falmer, Brighton, BN1 9RH, UK

Paul Roberts

School of Life Sciences, University of Sussex, Falmer, Brighton, BN1 9QG, UK

Jeremy E. Niven

Primary Care and Public Health, Brighton and Sussex Medical School and School of Psychology, University of Sussex, Falmer, Brighton, BN1 9PH, UK

You can also search for this author in PubMed   Google Scholar

Contributions

CH contributed to the conceptualisation, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, supervision, validation, visualisation, writing—original draft preparation and writing—review and editing of this paper. LC contributed to the data curation, investigation, project administration, validation and writing—review and editing of this paper. SV contributed to the data curation, formal analysis, investigation, project administration, validation and writing—review and editing of this paper. PR contributed to the funding acquisition, project administration, supervision and writing—review and editing of this paper. JN contributed to the conceptualisation, funding acquisition, methodology, project administration, supervision, validation, writing—original draft preparation and writing—review and editing of this paper. CB contributed to the conceptualisation, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, supervision, validation, visualisation, writing—original draft preparation and writing—review and editing of this paper. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Clio Berry .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was not needed for the present study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hazell, C.M., Chapman, L., Valeix, S.F. et al. Understanding the mental health of doctoral researchers: a mixed methods systematic review with meta-analysis and meta-synthesis. Syst Rev 9 , 197 (2020). https://doi.org/10.1186/s13643-020-01443-1

Download citation

Received : 15 June 2020

Accepted : 31 July 2020

Published : 26 August 2020

DOI : https://doi.org/10.1186/s13643-020-01443-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mental health
• PhD students
• doctoral researchers
• Postgraduate researchers
• Systematic review

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• 12 March 2024

Bring PhD assessment into the twenty-first century

You have full access to this article via your institution.

Innovation in PhD education has not reached how doctoral degrees are assessed. Credit: Dan Dunkley/Science Photo Library

Research and teaching in today’s universities are unrecognizable compared with what they were in the early nineteenth century, when Germany and later France gave the world the modern research doctorate. And yet significant aspects of the process of acquiring and assessing a doctorate have remained remarkably constant. A minimum of three years of independent study mentored by a single individual culminates in the production of the doctoral thesis — often a magisterial, book-length piece of work that is assessed in an oral examination by a few senior academic researchers. In an age in which there is much research-informed innovation in teaching and learning, the assessment of the doctoral thesis represents a curious throwback that is seemingly impervious to meaningful reform.

But reform is needed. Some doctoral candidates perceive the current assessment system to lack transparency, and examiners report concerns of falling standards ( G. Houston A Study of the PhD Examination: Process, Attributes and Outcomes . PhD thesis, Oxford Univ.; 2018 ). Making the qualification more structured would help — and, equally importantly, would bring the assessment of PhD education in line with education across the board. PhD candidates with experience of modern assessment methods will become better researchers, wherever they work. Indeed, most will not be working in universities: the majority of PhD holders find employment outside academia.

Collection: Career resources for PhD students

It’s not that PhD training is completely stuck in the nineteenth century. Today’s doctoral candidates can choose from a range of pathways. Professional doctorates, often used in engineering, are jointly supervised by an employer and an academic, and are aimed at solving industry-based problems. Another innovation is PhD by publication, in which, instead of a final thesis on one or more research questions, the criterion for an award is a minimum number of papers published or accepted for publication. In some countries, doctoral students are increasingly being trained in cohorts, with the aim of providing a less isolating experience than that offered by the conventional supervisor–student relationship. PhD candidates are also encouraged to acquire transferable skills — for example, in data analysis, public engagement, project management or business, economics and finance. The value of such training would be even greater if these skills were to be formally assessed alongside a dissertation rather than seen as optional.

And yet, most PhDs are still assessed after the production of a final dissertation, according to a format that, at its core, has not changed for at least half a century, as speakers and delegates noted at an event in London last month on PhD assessment, organized by the Society for Research in Higher Educatio n. Innovations in assessment that are common at other levels of education are struggling to find their way into the conventional doctoral programme.

Take the concept of learning objectives. Intended to aid consistency, fairness and transparency, learning objectives are a summary of what a student is expected to know and how they will be assessed, and are given at the start of a course of study. Part of the ambition is also to help tutors to keep track of their students’ learning and take remedial action before it is too late.

PhD training is no longer fit for purpose — it needs reform now

Formative assessment is another practice that has yet to find its way into PhD assessment consistently. Here, a tutor evaluates a student’s progress at the mid-point of a course and gives feedback or guidance on what students need to do to improve ahead of their final, or summative, assessment. It is not that these methods are absent from modern PhDs; a conscientious supervisor will not leave candidates to sink or swim until the last day. But at many institutions, such approaches are not required of PhD supervisors.

Part of the difficulty is that PhD training is carried out in research departments by people who do not need to have teaching qualifications or awareness of innovations based on education research. Supervisors shouldn’t just be experts in their field, they should also know how best to convey that subject knowledge — along with knowledge of research methods — to their students.

It is probably not possible for universities to require all doctoral supervisors to have teaching qualifications. But there are smaller changes that can be made. At a minimum, doctoral supervisors should take the time to engage with the research that exists in the field of PhD education, and how it can apply to their interactions with students.

There can be no one-size-fits-all solution to improving how a PhD is assessed, because different subjects often have bespoke needs and practices ( P. Denicolo Qual. Assur. Educ. 11 , 84–91; 2003 ). But supervisors and representatives of individual subject communities must continue to discuss what is most appropriate for their disciplines.

All things considered, there is benefit to adopting a more structured approach to PhD assessment. It is high time that PhD education caught up with changes that are now mainstream at most other levels of education. That must start with a closer partnership between education researchers, PhD supervisors and organizers of doctoral-training programmes in universities. This partnership will benefit everyone — PhD supervisors and doctoral students coming into the research workforce, whether in universities or elsewhere.

Education and training in research has entered many secondary schools, along with undergraduate teaching, which is a good thing. In the spirit of mutual learning, research doctoral supervisors, too, will benefit by going back to school.

Nature 627 , 244 (2024)

doi: https://doi.org/10.1038/d41586-024-00718-0

Reprints and permissions

Related Articles

• Scientific community

Overcoming low vision to prove my abilities under pressure

Career Q&A 28 MAR 24

How a spreadsheet helped me to land my dream job

Career Column 28 MAR 24

Maple-scented cacti and pom-pom cats: how pranking at work can lift lab spirits

Career Feature 27 MAR 24

Cuts to postgraduate funding threaten Brazilian science — again

Correspondence 26 MAR 24

Don’t underestimate the rising threat of groundwater to coastal cities

‘Exhausted and insulted’: how harsh visa-application policies are hobbling global research

World View 26 MAR 24

The corpse of an exploded star and more — March’s best science images

News 28 MAR 24

Expression of Interest – Marie Skłodowska-Curie Actions – Postdoctoral Fellowships 2024 (MSCA-PF)

Academic institutions in Brittany are looking for excellent postdoctoral researchers willing to apply for a Marie S. Curie Postdoctoral Fellowship.

France (FR)

Plateforme projets européens (2PE) -Bretagne

Tenure-track Assistant Professor in Ecological and Evolutionary Modeling

Tenure-track Assistant Professor in Ecosystem Ecology linked to IceLab’s Center for modeling adaptive mechanisms in living systems under stress

Umeå, Sweden

Umeå University

Faculty Positions in Westlake University

Founded in 2018, Westlake University is a new type of non-profit research-oriented university in Hangzhou, China, supported by public a...

Hangzhou, Zhejiang, China

Westlake University

Postdoctoral Fellowships-Metabolic control of cell growth and senescence

Postdoctoral positions in the team Cell growth control by nutrients at Inst. Necker, Université Paris Cité, Inserm, Paris, France.

Paris, Ile-de-France (FR)

Inserm DR IDF Paris Centre Nord

Zhejiang Provincial Hospital of Chinese Medicine on Open Recruitment of Medical Talents and Postdocs

Director of Clinical Department, Professor, Researcher, Post-doctor

The First Affiliated Hospital of Zhejiang Chinese Medical University

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• PhD Viva Voces – A Complete Guide
• Doing a PhD
• A PhD viva involves defending your thesis in an oral examination with at least two examiners.
• The aim of a PhD viva is to confirm that the work is your own , that you have a deep understanding of your project and, overall, that you are a competent researcher .
• There are no standard durations, but they usually range from one to three hours, with most lasting approximately two hours .
• There are six outcomes of a PhD viva: (1) pass without corrections (2) pass subject to minor corrections, (3) pass subject to major corrections, (4) downgrade to MPhil with no amendments, (5) downgrade to MPhil subject to amendments, (6) immediate fail.
• Almost all students who sit their viva pass it, with the most common outcome being ‘(2) – pass subject to minor corrections’.

What Is a PhD Viva?

A viva voce , more commonly referred to as ‘viva’, is an oral examination conducted at the end of your PhD and is essentially the final hurdle on the path to a doctorate. It is the period in which a student’s knowledge and work are evaluated by independent examiners.

In order to assess the student and their work around their research question, a viva sets out to determine:

• you understand the ideas and theories that you have put forward,
• you can answer questions about elements of your work that the examiners have questions about,
• you understand the broader research in your field and how your work contributes to this,
• you are aware of the limitations of your work and understand how it can be developed further,
• your work makes an original contribution, is your own and has not been plagiarised.

Note: A viva is a compulsory procedure for all PhD students, with the only exception being when a PhD is obtained through publication as opposed to the conventional route of study.

Who Will Attend a Viva?

In the UK, at least two examiners must take part in all vivas. Although you could have more than two examiners, most will not in an attempt to facilitate a smoother questioning process.

One of the two examiners will be internal, i.e. from your university, and the other will be external, i.e. from another university. Regardless, both will be knowledgeable in your research field and have read your thesis beforehand.

In addition to your two examiners, two other people may be present. The first is a chairperson. This is an individual who will be responsible for monitoring the interview and for ensuring proper conduct is followed at all times. The need for an external chairperson will vary between universities, as one of the examiners can also take on this role. The second is your supervisor, whose attendance is decided upon by you in agreement with your examiners. If your supervisor attends, they are prohibited from asking questions or from influencing the outcome of the viva.

To avoid any misunderstandings, we have summarised the above in a table:

Note: In some countries, such as in the United States, a viva is known as a ‘PhD defense’ and is performed publicly in front of a panel or board of examiners and an open audience. In these situations, the student presents their work in the form of a lecture and then faces questions from the examiners and audience which almost acts as a critical appraisal.

How Long Does a Viva Last?

Since all universities have different guidelines , and since all PhDs are unique, there are no standard durations. Typically, however, the duration ranges from one to three hours, with most lasting approximately two hours.

Your examiners will also influence the duration of your viva as some will favour a lengthy discussion, while others may not. Usually, your university will consult your examiners in advance and notify you of the likely duration closer to the day of your viva.

What Happens During a Viva?

Regardless of the subject area, all PhD vivas follow the same examination process format as below.

Introductions

You will introduce yourselves to each other, with the internal examiner normally introducing the external examiner. If an external chairperson is present, they too are introduced; otherwise, this role will be assumed by one of the examiners.

Procedure Explained

After the introductions, the appointed chair will explain the viva process. Although it should already be known to everyone, it will be repeated to ensure the viva remains on track during the forthcoming discussion.

Warm-Up Questions

The examiners will then begin the questioning process. This usually starts with a few simple opening questions, such as asking you to summarise your PhD thesis and what motivated you to carry out the research project.

In-Depth Questions

The viva questions will then naturally increase in difficulty as the examiners go further into the details of your thesis. These may include questions such as “What was the most critical decision you made when determining your research methodology ?”, “Do your findings agree with the current published work?” and “How do your findings impact existing theories or literature? ”. In addition to asking open-ended questions, they will also ask specific questions about the methodology, results and analysis on which your thesis is based.

Closing the Viva

Once the examiners are satisfied that they have thoroughly evaluated your knowledge and thesis, they will invite you to ask any questions you may have, and then bring the oral examination to a close.

What Happens After the Viva?

Once your viva has officially ended, your examiners will ask you to leave the room so that they can discuss your performance. Once a mutual agreement has been reached, which can take anywhere from 10 minutes to an hour, you will be invited back inside and informed of your outcome.

PhD Viva Outcomes

There are six possible outcomes to a viva:

• Immediate award of degree: A rare recommendation – congratulations, you are one of the few people who completely satisfied your examiners the first time around. You do not have to do anything further at this point.
• Minor amendments required: The most common recommendation – you obtain a pass on the condition that you make a number of minor amendments to your thesis, such as clarifying certain points and correcting grammatical errors. The time you have to make these changes depends on the number of them, but is usually one to six months.
• Major amendments required: A somewhat uncommon recommendation – you are requested to make major amendments to your thesis, ranging from further research to collecting more data or rewriting entire sections. Again, the time you have to complete this will depend on the number of changes required, but will usually be six months to one year. You will be awarded your degree once your amended thesis has been reviewed and accepted.
• Immediate award of MPhil: An uncommon recommendation – your examiners believe your thesis does not meet the standard for a doctoral degree but meets the standard for an MPhil (Master of Philosophy), a lower Master’s degree.
• Amendments required for MPhil: A rare recommendation – your examiners believe your thesis does not meet the standard for a doctoral degree, but with several amendments will meet the standard for an MPhil.
• Immediate fail: A very rare recommendation – you are given an immediate fail without the ability to resubmit and without entitlement to an MPhil.

Finding a PhD has never been this easy – search for a PhD by keyword, location or academic area of interest.

What Is the Pass Rate for Vivas?

Based on an  analysis of 26,076 PhD students  who took their viva exam between 2006 and 2017, the PhD viva pass rate in the UK is 96%; of those who passed, about 80% were required to make minor amendments to their thesis. The reason for this high pass rate is that supervisors will only put their students forward for a viva once they confidently believe they are ready for it. As a result, most candidates who sit a viva are already well-versed in their PhD topic before they even start preparing for the exam.

How Do I Arrange a Viva?

Your viva will be arranged either by the examiners or by the chairperson. The viva will be arranged at least one to two months after you have submitted your thesis and will arrange a viva date and venue that is suitable for all participants.

Can I Choose My Examiners?

At most universities, you and your supervisor will choose the internal and external examiners yourselves. This is because the examiners must have extensive knowledge of the thesis topic in order to be able to examine you and, as the author of the thesis in question, who else could better determine who they might be than you and your supervisor. The internal examiner is usually quite easy to find given they will be from your institution, but the external examiner may end up being your second or third preference depending on availability.

Can I Take Notes Into a Viva?

A viva is about testing your competence, not your memory. As such, you are allowed to take notes and other supporting material in with you. However, keep in mind that your examiners will not be overly impressed if you constantly have to refer to your notes to answer each question. Because of this, many students prefer to take an annotated copy of their thesis, with important points already highlighted and key chapters marked with post-it notes.

In addition to an annotated copy of a thesis, some students also take:

• a list of questions they would like to ask the examiners,
• notes that were created during their preparation,
• a list of minor corrections they have already identified from their viva prep work.

How Do I Prepare for a PhD Viva?

There are several ways to prepare for a PhD viva, one of the most effective being a mock viva voce examination . This allows you to familiarise yourself with the type of viva questions you will be asked and identify any weak areas you need to improve. They also give you the opportunity to practise without the pressure, giving you more time to think about your answers which will help to make sure that you know your thesis inside out. However, a mock viva exam is just one of many methods available to you – some of the other viva preparation methods can be found on our “ How to Prepare for a PhD Viva ” page.

Browse PhDs Now

Join thousands of students.

Join thousands of other students and stay up to date with the latest PhD programmes, funding opportunities and advice.