Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

in literature review articles authors should

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved February 19, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

  • Undergraduate courses
  • Postgraduate courses
  • January starts
  • Foundation courses
  • Apprenticeships
  • Part-time and short courses
  • Apply undergraduate
  • Apply postgraduate

Search for a course

Search by course name, subject, and more

  • Undergraduate
  • Postgraduate
  • (suspended) - Available in Clearing Not available in Clearing location-sign UCAS

Fees and funding

  • Tuition fees
  • Scholarships
  • Funding your studies
  • Student finance
  • Cost of living support

Why study at Kent

Student life.

  • Careers and employability
  • Student support and wellbeing
  • Our locations
  • Placements and internships
  • Year abroad
  • Student stories
  • Schools and colleges
  • International
  • International students
  • Your country
  • Applicant FAQs
  • International scholarships
  • University of Kent International College
  • Campus Tours
  • Applicant Events
  • Postgraduate events
  • Maps and directions
  • Research strengths
  • Research centres
  • Research impact

Research institutes

  • Durrell Institute of Conservation and Ecology
  • Institute of Cyber Security for Society
  • Institute of Cultural and Creative Industries
  • Institute of Health, Social Care and Wellbeing

Research students

  • Graduate and Researcher College
  • Research degrees
  • Find a supervisor
  • How to apply

Popular searches

  • Visits and Open Days
  • Jobs and vacancies
  • Accommodation
  • Student guide
  • Library and IT
  • Research highlights
  • Signature themes
  • Partner with us
  • Student Guide
  • Student Help
  • Health & wellbeing
  • Student voice
  • Living at Kent
  • Careers & volunteering
  • Diversity at Kent
  • Finance & funding
  • Life after graduation

Literature reviews

Writing a literature review.

The following guide has been created for you by the  Student Learning Advisory Service . For more detailed guidance and to speak to one of our advisers, please book an  appointment  or join one of our  workshops . Alternatively, have a look at our  SkillBuilder  skills videos.   

Preparing a literature review involves:

  • Searching for reliable, accurate and up-to-date material on a topic or subject
  • Reading and summarising the key points from this literature
  • Synthesising these key ideas, theories and concepts into a summary of what is known
  • Discussing and evaluating these ideas, theories and concepts
  • Identifying particular areas of debate or controversy
  • Preparing the ground for the application of these ideas to new research

Finding and choosing material

Ensure you are clear on what you are looking for. ask yourself:.

  • What is the specific question, topic or focus of my assignment?
  • What kind of material do I need (e.g. theory, policy, empirical data)?
  • What type of literature is available (e.g. journals, books, government documents)?

What kind of literature is particularly authoritative in this academic discipline (e.g. psychology, sociology, pharmacy)?

How much do you need?

This will depend on the length of the dissertation, the nature of the subject, and the level of study (undergraduate, Masters, PhD). As a very rough rule of thumb – you may choose 8-10 significant pieces (books and/or articles) for an 8,000 word dissertation, up to 20 major pieces of work for 12-15,000 words, and so on. Bear in mind that if your dissertation is based mainly around an interaction with existing scholarship you will need a longer literature review than if it is there as a prelude to new empirical research. Use your judgement or ask your supervisor for guidance.

Where to find suitable material

Your literature review should include a balance between substantial academic books, journal articles and other scholarly publications. All these sources should be as up-to-date as possible, with the exception of ‘classic texts’ such as major works written by leading scholars setting out formative ideas and theories central to your subject. There are several ways to locate suitable material:

Module bibliography: for undergraduate dissertations, look first at the bibliography provided with the module documentation. Choose one or two likely looking books or articles and then scan through the bibliographies provided by these authors. Skim read some of this material looking for clues: can you use these leads to identify key theories and authors or track down other appropriate material?

Library catalogue search engine: enter a few key words to capture a range of items, but avoid over-generalisations; if you type in something as broad as ‘social theory’ you are likely to get several thousand results. Be more specific: for example, ‘Heidegger, existentialism’. Ideally, you should narrow the field to obtain just a few dozen results. Skim through these quickly to identity texts which are most likely to contribute to your study.

Library bookshelves: browse the library shelves in the relevant subject area and examine the books that catch your eye. Check the contents and index pages, or skim through the introductions (or abstracts, in the case of journal articles) to see if they contain relevant material, and replace them if not. Don’t be afraid to ask one of the subject librarians for further help. Your supervisor may also be able to point you in the direction of some of the important literature , but remember this is your literature search, not theirs.

Online: for recent journal articles you will almost certainly need to use one of the online search engines. These can be found on the ‘Indexing Services’ button on the Templeman Library website. Kent students based at Medway still need to use the Templeman pages to access online journals, although you can get to these pages through the Drill Hall Library catalogue. Take a look as well at the Subject Guides on both the Templeman and DHL websites.

Check that you have made the right selection by asking:

  • Has my search been wide enough to ensure that I have identified all the relevant material, but narrow enough to exclude irrelevant material?
  • Is there a good enough sample of literature for the level (PhD, Masters, undergraduate) of my dissertation or thesis?
  • Have I considered as many alternative points of view as possible?
  • Will the reader find my literature review relevant and useful?

Assessing the literature

Read the material you have chosen carefully, considering the following:

  • The key point discussed by the author: is this clearly defined
  • What evidence has the author produced to support this central idea?
  • How convincing are the reasons given for the author’s point of view?
  • Could the evidence be interpreted in other ways?
  • What is the author's research method (e.g. qualitative, quantitative, experimental, etc.)?
  • What is the author's theoretical framework (e.g. psychological, developmental, feminist)?
  • What is the relationship assumed by the author between theory and practice?
  • Has the author critically evaluated the other literature in the field?
  • Does the author include literature opposing their point of view?
  • Is the research data based on a reliable method and accurate information?
  • Can you ‘deconstruct’ the argument – identify the gaps or jumps in the logic?
  • What are the strengths and limitations of this study?
  • What does this book or article contribute to the field or topic?
  • What does this book or article contribute to my own topic or thesis?

As you note down the key content of each book or journal article (together with the reference details of each source) record your responses to these questions. You will then be able to summarise each piece of material from two perspectives:     

Content: a brief description of the content of the book or article. Remember, an author will often make just one key point; so, what is the point they are making, and how does it relate to your own research project or assignment?

Critical analysis: an assessment of the relative strengths and weaknesses of the evidence used, and the arguments presented. Has anything conveniently been left out or skated over? Is there a counter-argument, and has the author dealt with this adequately? Can the evidence presented be interpreted another way? Does the author demonstrate any obvious bias which could affect their reliability? Overall, based on the above analysis of the author’s work, how do you evaluate its contribution to the scholarly understanding and knowledge surrounding the topic?    

Structuring the literature review

In a PhD thesis, the literature review typically comprises one chapter (perhaps 8-10,000 words), for a Masters dissertation it may be around 2-3,000 words, and for an undergraduate dissertation it may be no more than 2,000 words. In each case the word count can vary depending on a range of factors and it is always best, if in doubt, to ask your supervisor.

The overall structure of the section or chapter should be like any other: it should have a beginning, middle and end. You will need to guide the reader through the literature review, outlining the strategy you have adopted for selecting the books or articles, presenting the topic theme for the review, then using most of the word limit to analyse the chosen books or articles thoroughly before pulling everything together briefly in the conclusion.

Some people prefer a less linear approach. Instead of simply working through a list of 8-20 items on your book review list, you might want to try a thematic approach, grouping key ideas, facts, concepts or approaches together and then bouncing the ideas off each other. This is a slightly more creative (and interesting) way of producing the review, but a little more risky as it is harder to establish coherence and logical sequencing.

Whichever approach you adopt, make sure everything flows smoothly – that one idea or book leads neatly to the next. Take your reader effortlessly through a sequence of thought that is clear, accurate, precise and interesting. 

Writing up your literature review

As with essays generally, only attempt to write up the literature review when you have completed all the reading and note-taking, and carefully planned its content and structure. Find an appropriate way of introducing the review, then guide the reader through the material clearly and directly, bearing in mind the following:

  • Be selective in the number of points you draw out from each piece of literature; remember that one of your objectives is to demonstrate that you can use your judgement to identify what is central and what is secondary.
  • Summarise and synthesise – use your own words to sum up what you think is important or controversial about the book or article.
  • Never claim more than the evidence will support. Too many dissertations and theses are let down by sweeping generalisations. Be tentative and careful in the way you interpret the evidence.
  • Keep your own voice – you are entitled to your own point of view provided it is based on evidence and clear argument.
  • At the same time, aim to project an objective and tentative tone by using the 3rd person, (for example, ‘this tends to suggest’, ‘it could be argued’ and so on).
  • Even with a literature review you should avoid using too many, or overlong, quotes. Summarise material in your own words as much as possible. Save the quotes for ‘punch-lines’ to drive a particular point home.
  • Revise, revise, revise: refine and edit the draft as much as you can. Check for fluency, structure, evidence, criticality and referencing, and don’t forget the basics of good grammar, punctuation and spelling.
  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: Feb 8, 2024 1:57 PM
  • URL: https://libguides.usc.edu/writingguide

University of Texas

  • University of Texas Libraries

Literature Reviews

  • What is a literature review?
  • Steps in the Literature Review Process
  • Define your research question
  • Determine inclusion and exclusion criteria
  • Choose databases and search
  • Review Results
  • Synthesize Results
  • Analyze Results
  • Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

  • Provide a comprehensive and updated review of the literature;
  • Explain why this review has taken place;
  • Articulate a position or hypothesis;
  • Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

  • Demonstrate how a study fills a gap in research
  • Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

  • Organizes or describes a topic
  • Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

  • It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
  • It may be too focused. Some niche studies may miss the bigger picture.
  • It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
  • It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

  • October 26, 2022 recording
  • Last Updated: Oct 26, 2022 2:49 PM
  • URL: https://guides.lib.utexas.edu/literaturereviews

Creative Commons License

  • Search Menu
  • Advance Articles
  • Editor's Choice
  • CME Reviews
  • Best of 2021 collection
  • Abbreviated Breast MRI Virtual Collection
  • Contrast-enhanced Mammography Collection
  • Author Guidelines
  • Submission Site
  • Open Access
  • Self-Archiving Policy
  • Accepted Papers Resource Guide
  • About Journal of Breast Imaging
  • About the Society of Breast Imaging
  • Guidelines for Reviewers
  • Resources for Reviewers and Authors
  • Editorial Board
  • Advertising Disclaimer
  • Advertising and Corporate Services
  • Journals on Oxford Academic
  • Books on Oxford Academic

Society of Breast Imaging

Article Contents

  • Introduction
  • Selection of a Topic
  • Scientific Literature Search and Analysis
  • Structure of a Scientific Review Article
  • Tips for Success
  • Acknowledgments
  • Conflict of Interest Statement
  • < Previous

A Step-by-Step Guide to Writing a Scientific Review Article

  • Article contents
  • Figures & tables
  • Supplementary Data

Manisha Bahl, A Step-by-Step Guide to Writing a Scientific Review Article, Journal of Breast Imaging , Volume 5, Issue 4, July/August 2023, Pages 480–485, https://doi.org/10.1093/jbi/wbad028

  • Permissions Icon Permissions

Scientific review articles are comprehensive, focused reviews of the scientific literature written by subject matter experts. The task of writing a scientific review article can seem overwhelming; however, it can be managed by using an organized approach and devoting sufficient time to the process. The process involves selecting a topic about which the authors are knowledgeable and enthusiastic, conducting a literature search and critical analysis of the literature, and writing the article, which is composed of an abstract, introduction, body, and conclusion, with accompanying tables and figures. This article, which focuses on the narrative or traditional literature review, is intended to serve as a guide with practical steps for new writers. Tips for success are also discussed, including selecting a focused topic, maintaining objectivity and balance while writing, avoiding tedious data presentation in a laundry list format, moving from descriptions of the literature to critical analysis, avoiding simplistic conclusions, and budgeting time for the overall process.

  • narrative discourse

Email alerts

Citing articles via.

  • Recommend to your Librarian
  • Journals Career Network

Affiliations

  • Online ISSN 2631-6129
  • Print ISSN 2631-6110
  • Copyright © 2024 Society of Breast Imaging
  • About Oxford Academic
  • Publish journals with us
  • University press partners
  • What we publish
  • New features  
  • Open access
  • Institutional account management
  • Rights and permissions
  • Get help with access
  • Accessibility
  • Advertising
  • Media enquiries
  • Oxford University Press
  • Oxford Languages
  • University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

  • Copyright © 2024 Oxford University Press
  • Cookie settings
  • Cookie policy
  • Privacy policy
  • Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

  • UConn Library
  • Literature Review: The What, Why and How-to Guide
  • Introduction

Literature Review: The What, Why and How-to Guide — Introduction

  • Getting Started
  • How to Pick a Topic
  • Strategies to Find Sources
  • Evaluating Sources & Lit. Reviews
  • Tips for Writing Literature Reviews
  • Writing Literature Review: Useful Sites
  • Citation Resources
  • Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

  • To develop a theory or evaluate an existing theory
  • To summarize the historical or existing state of a research topic
  • Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

  • A research paper assigned in a course
  • A thesis or dissertation
  • A grant proposal
  • An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

  • Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

  • Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

  • Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

  • Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

  • UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
  • << Previous: Getting Started
  • Next: How to Pick a Topic >>
  • Last Updated: Sep 21, 2022 2:16 PM
  • URL: https://guides.lib.uconn.edu/literaturereview

Creative Commons

Rasmussen University Flame logo

  • General Education Courses
  • School of Business
  • School of Design
  • School of Education
  • School of Health Sciences
  • School of Justice Studies
  • School of Nursing
  • School of Technology
  • CBE Student Guide
  • Online Library
  • Ask a Librarian
  • Learning Express Library
  • Interlibrary Loan Request Form
  • Library Staff
  • Databases A-to-Z
  • Articles by Subject
  • Discovery Search
  • Publication Finder
  • Video Databases
  • NoodleTools
  • Library Guides
  • Course Guides
  • Writing Lab
  • Rasmussen Technical Support (PSC)
  • Copyright Toolkit
  • Faculty Toolkit
  • Suggest a Purchase
  • Refer a Student Tutor
  • Live Lecture/Peer Tutor Scheduler
  • Faculty Interlibrary Loan Request Form
  • Professional Development Databases
  • Publishing Guide
  • Professional Development Guides (AAOPD)
  • Literature Review Guide

The Literature Review

  • What is a Literature Review?
  • Plan Your Literature Review
  • Identify a Research Gap
  • Define Your Research Question
  • Search the Literature
  • Analyze Your Research Results
  • Manage Research Results
  • Write the Literature Review

in literature review articles authors should

What is a Literature Review?  What is its purpose?

The purpose of a literature review is to offer a  comprehensive review of scholarly literature on a specific topic along with an  evaluation of the strengths and weaknesses of authors' arguments . In other words, you are summarizing research available on a certain topic and then drawing conclusions about researchers' findings. To make gathering research easier, be sure to start with a narrow/specific topic and then widen your topic if necessary.

A thorough literature review provides an accurate description of current knowledge on a topic and identifies areas for future research.  Are there gaps or areas that require further study and exploration? What opportunities are there for further research? What is missing from my collection of resources? Are more resources needed?

It is important to note that conclusions described in the literature you gather may contradict each other completely or in part.  Recognize that knowledge creation is collective and cumulative.  Current research is built upon past research findings and discoveries.  Research may bring previously accepted conclusions into question.  A literature review presents current knowledge on a topic and may point out various academic arguments within the discipline.

What a Literature Review is not

  • A literature review is not an annotated bibliography .  An annotated bibliography provides a brief summary, analysis, and reflection of resources included in the bibliography.  Often it is not a systematic review of existing research on a specific subject.  That said, creating an annotated bibliography throughout your research process may be helpful in managing the resources discovered through your research.
  • A literature review is not a research paper .  A research paper explores a topic and uses resources discovered through the research process to support a position on the topic.  In other words, research papers present one side of an issue.  A literature review explores all sides of the research topic and evaluates all positions and conclusions achieved through the scientific research process even though some conclusions may conflict partially or completely.

From the Online Library

Cover Art

  • SAGE Research Methods This link opens in a new window SAGE Research Methods is a web-based research methods tool that covers quantitative, qualitative and mixed methods. Researchers can explore methods and concepts to help design research projects, understand a particular method or identify a new method, and write up research. Sage Research Methods focuses on methodology rather than disciplines, and is of potential use to researchers from the social sciences, health sciences and other research areas.
  • Sage Research Methods Project Planner - Reviewing the Literature View the resources and videos for a step-by-step guide to performing a literature review.

The Literature Review: Step by Step

Follow this step-by-step process by using the related tabs in this Guide.

  • Define your Research question
  • Analyze the material you’ve found
  • Manage the results of your research
  • Write your Review

Getting Started

Consider the following questions as you develop your research topic, conduct your research, and begin evaluating the resources discovered in the research process:

  • What is known about the subject?
  • Are there any gaps in the knowledge of the subject?
  • Have areas of further study been identified by other researchers that you may want to consider?
  • Who are the significant research personalities in this area?
  • Is there consensus about the topic?
  • What aspects have generated significant debate on the topic?
  • What methods or problems were identified by others studying in the field and how might they impact your research?
  • What is the most productive methodology for your research based on the literature you have reviewed?
  • What is the current status of research in this area?
  • What sources of information or data were identified that might be useful to you?
  • How detailed? Will it be a review of ALL relevant material or will the scope be limited to more recent material, e.g., the last five years.
  • Are you focusing on methodological approaches; on theoretical issues; on qualitative or quantitative research?

What is Academic Literature?

What is the difference between popular and scholarly literature?

To better understand the differences between popular and scholarly articles, comparing characteristics and purpose of the publications where these articles appear is helpful.

Popular Article (Magazine)

  • Articles are shorter and are written for the general public
  • General interest topics or current events are covered
  • Language is simple and easy to understand
  • Source material is not cited
  • Articles often include glossy photographs, graphics, or visuals
  • Articles are written by the publication's staff of journalists
  • Articles are edited and information is fact checked

Examples of magazines that contain popular articles:

in literature review articles authors should

Scholarly Article (Academic Journal)

  • Articles are written by scholars and researchers for academics, professionals, and experts in the field
  • Articles are longer and report original research findings
  • Topics are narrower in focus and provide in-depth analysis
  • Technical or scholarly language is used
  • Source material is cited
  • Charts and graphs illustrating research findings are included
  • Many are  "peer reviewed"  meaning that panels of experts review articles submitted for publication to ensure that proper research methods were used and research findings are contributing something new to the field before selecting for publication.

Examples of academic journals that contain scholarly articles:

in literature review articles authors should

Define your research question

Selecting a research topic can be overwhelming.  Consider following these steps:

1.  Brainstorm  research topic ideas

      - Free write: Set a timer for five minutes and write down as many ideas as you can in the allotted time

      -  Mind-Map  to explore how ideas are related

2.  Prioritize  topics based on personal interest and curiosity

3.  Pre-research

      - Explore encyclopedias and reference books for background information on the topic

      - Perform a quick database or Google search on the topic to explore current issues. 

4.  Focus the topic  by evaluating how much information is available on the topic

         - Too much information?  Consider narrowing the topic by focusing on a specific issue 

         - Too little information?  Consider broadening the topic 

5.  Determine your purpose  by considering whether your research is attempting to:

         - further the research on this topic

         - fill a gap in the research

         - support existing knowledge with new evidence

         - take a new approach or direction

         - question or challenge existing knowledge

6.  Finalize your research question

NOTE:  Be aware that your initial research question may change as you conduct research on your topic.

Searching the Literature

Research on your topic should be conducted in the academic literature.  The  Rasmussen University Online Library contains subject-focused databases that contain the leading academic journals in your programmatic area.

Consult the  Using the Online Library video tutorials  for information about how to effectively search library databases.

Watch the video below for tips on how to create a search statement that will provide relevant results

Need help starting your research?  Make a  research appointment with a Rasmussen Librarian .

in literature review articles authors should

TIP:  Document as you research.  Begin building your references list using the citation managers in one of these resources:

  • APA Academic Writer

Recommended programmatic databases include:

Data Science

Coverage includes computer engineering, computer theory & systems, research and development, and the social and professional implications of new technologies. Articles come from more than 1,900 academic journals, trade magazines, and professional publications.

Provides access to full-text peer-reviewed journals, transactions, magazines, conference proceedings, and published standards in the areas of electrical engineering, computer science, and electronics. It also provides access to the IEEE Standards Dictionary Online. Full-text available.

Computing, telecommunications, art, science and design databases from ProQuest.

Healthcare Management

Articles from scholarly business journals back as far as 1886 with content from all disciplines of business, including marketing, management, accounting, management information systems, production and operations management, finance, and economics. Contains 55 videos from the Harvard Faculty Seminar Series, on topics such as leadership, sustaining competitive advantage, and globalization. To access the videos, click "More" in the blue bar at the top. Select "Images/ Business Videos." Uncheck "Image Quick View Collection" to indicate you only wish to search for videos. Enter search terms.

Provides a truly comprehensive business research collection. The collection consists of the following databases and more: ABI/INFORM Complete, ProQuest Entrepreneurship, ProQuest Accounting & Tax, International Bibliography of Social Sciences (IBSS), ProQuest Asian Business and Reference, and Banking Information Source.

The definitive research tool for all areas of nursing and allied health literature. Geared towards the needs of nurses and medical professionals. Covers more than 750 journals from 1937 to present.

HPRC provides information on the creation, implementation and study of health care policy and the health care system. Topics covered include health care administration, economics, planning, law, quality control, ethics, and more.

PolicyMap is an online mapping site that provides data on demographics, real estate, health, jobs, and other areas across the U.S. Access and visualize data from Census and third-party records.

Human Resources

Articles from all subject areas gathered from more than 11,000 magazines, journals, books and reports. Subjects include astronomy, multicultural studies, humanities, geography, history, law, pharmaceutical sciences, women's studies, and more. Coverage from 1887 to present. Start your research here.

Cochrane gathers and summarizes the best evidence from research to help you make informed choices about treatments. Whether a doctor or nurse, patient, researcher or student, Cochrane evidence provides a tool to enhance your healthcare knowledge and decision making on topics ranging from allergies, blood disorders, and cancer, to mental health, pregnancy, urology, and wounds.

Health sciences, biology, science, and pharmaceutical information from ProQuest. Includes articles from scholarly, peer-reviewed journals, practical and professional development content from professional journals, and general interest articles from magazines and newspapers.

Joanna Briggs Institute Academic Collection contains evidence-based information from across the globe, including evidence summaries, systematic reviews, best practice guidelines, and more. Subjects include medical, nursing, and healthcare specialties.

Comprehensive source of full-text articles from more than 1,450 scholarly medical journals.

Articles from more than 35 nursing journals in full text, searchable as far back as 1995.

Analyzing Your Research Results

You have completed your research and discovered many, many academic articles on your topic.  The next step involves evaluating and organizing the literature found in the research process.

As you review, keep in mind that there are three types of research studies:

  • Quantitative
  • Qualitative 
  • Mixed Methods

Consider these questions as you review the articles you have gathered through the research process:

1. Does the study relate to your topic?

2. Were sound research methods used in conducting the study?

3. Does the research design fit the research question? What variables were chosen? Was the sample size adequate?

4. What conclusions were drawn?  Do the authors point out areas for further research?

Reading Academic Literature

Academic journals publish the results of research studies performed by experts in an academic discipline.  Articles selected for publication go through a rigorous peer-review process.  This process includes a thorough evaluation of the research submitted for publication by journal editors and other experts or peers in the field.  Editors select articles based on specific criteria including the research methods used, whether the research contributes new findings to the field of study, and how the research fits within the scope of the academic journal.  Articles selected often go through a revision process prior to publication.

Most academic journal articles include the following sections:

  • Abstract    (An executive summary of the study)
  • Introduction  (Definition of the research question to be studied)
  • Literature Review  (A summary of past research noting where gaps exist)
  • Methods  (The research design including variables, sample size, measurements)
  • Data   (Information gathered through the study often displayed in tables and charts)
  • Results   (Conclusions reached at the end of the study)
  • Conclusion   (Discussion of whether the study proved the thesis; may suggest opportunities for further research)
  • Bibliography  (A list of works cited in the journal article)

TIP:  To begin selecting articles for your research, read the   highlighted sections   to determine whether the academic journal article includes information relevant to your research topic.

Step 1: Skim the article

When sorting through multiple articles discovered in the research process, skimming through these sections of the article will help you determine whether the article will be useful in your research.

1.  Article title   and subject headings assigned to the article

2.   Abstract

3.   Introduction

4.  Conclusion

If the article fits your information need, go back and  read the article thoroughly.

TIP:  Create a folder on your computer to save copies of articles you plan to use in your thesis or research project.  Use  NoodleTools  or  APA Academic Writer  to save APA references.

Step 2: Determine Your Purpose

Think about how you will evaluate the academic articles you find and how you will determine whether to include them in your research project.  Ask yourself the following questions to focus your search in the academic literature:

  • ​Are you looking for an overview of a topic? an explanation of a specific concept, idea, or position?
  • Are you exploring gaps in the research to identify a new area for academic study?
  • Are you looking for research that supports or disagrees with your thesis or research question?
  • Are you looking for examples of a research design and/or research methods you are considering for your own research project?

Step 3: Read Critically

Before reading the article, ask yourself the following:

  • What is my research question?  What position am I trying to support?
  • What do I already know about this topic?  What do I need to learn?
  • How will I evaluate the article?  Author's reputation? Research design? Treatment of topic? 
  • What are my biases about the topic?

As you read the article make note of the following:

  • Who is the intended audience for this article?
  • What is the author's purpose in writing this article?
  • What is the main point?
  • How was the main point proven or supported?  
  • Were scientific methods used in conducting the research?
  • Do you agree or disagree with the author? Why?
  • How does this article compare or connect with other articles on the topic?
  • Does the author recommend areas for further study?
  • How does this article help to answer your research question?

Managing your Research

Tip:  Create APA references for resources as you discover them in the research process

Use APA Academic Writer or NoodleTools to generate citations and manage your resources.  Find information on how to use these resources in the Citation Tools Guide .

in literature review articles authors should

Writing the Literature Review

Once research has been completed, it is time to structure the literature review and begin summarizing and synthesizing information.  The following steps may help with this process:

  • Chronological
  • By research method used
  • Explore contradictory or conflicting conclusions
  • Read each study critically
  • Critique methodology, processes, and conclusions
  • Consider how the study relates to your topic

Writing Lab

  • Description of public health nursing nutrition assessment and interventions for home‐visited women. This article provides a nice review of the literature in the article introduction. You can see how the authors have used the existing literature to make a case for their research questions. more... less... Horning, M. L., Olsen, J. M., Lell, S., Thorson, D. R., & Monsen, K. A. (2018). Description of public health nursing nutrition assessment and interventions for home‐visited women. Public Health Nursing, 35(4), 317–326. https://doi.org/10.1111/phn.12410
  • Improving Diabetes Self-Efficacy in the Hispanic Population Through Self-Management Education Doctoral papers are a good place to see how literature reviews can be done. You can learn where they searched, what search terms they used, and how they decided which articles were included. Notice how the literature review is organized around the three main themes that came out of the literature search. more... less... Robles, A. N. (2023). Improving diabetes self-efficacy in the hispanic population through self-management education (Order No. 30635901). Available from ProQuest Dissertations & Theses Global: The Sciences and Engineering Collection. https://www.proquest.com/dissertations-theses/improving-diabetes-self-efficacy-hispanic/docview/2853708553/se-2
  • Exploring mediating effects between nursing leadership and patient safety from a person-centred perspective: A literature review Reading articles that publish the results of a systematic literature review is a great way to see in detail how a literature review is conducted. These articles provide an article matrix, which provides you an example of how you can document information about the articles you find in your own search. To see more examples, include "literature review" or "systematic review" as a search term. more... less... Wang, M., & Dewing, J. (2021). Exploring mediating effects between nursing leadership and patient safety from a person‐centred perspective: A literature review. Journal of Nursing Management, 29(5), 878–889. https://doi.org/10.1111/jonm.13226

Database Search Tips

  • Boolean Operators
  • Keywords vs. Subjects
  • Creating a Search String
  • Library databases are collections of resources that are searchable, including full-text articles, books, and encyclopedias.
  • Searching library databases is different than searching Google. Best results are achieved when using Keywords linked with Boolean Operators . 
  • Applying Limiters such as full-text, publication date, resource type, language, geographic location, and subject help to refine search results.
  • Utilizing Phrases or Fields , in addition to an awareness of Stop Words , can focus your search and retrieve more useful results.
  • Have questions? Ask a Librarian

Boolean Operators connect keywords or concepts logically to retrieve relevant articles, books, and other resources.  There are three Boolean Operators:

Using AND 

  • Narrows search results
  • Connects two or more keywords/concepts
  • All keywords/concepts connected with "and" must be in an article or resource to appear in the search results list

in literature review articles authors should

Venn diagram of the AND connector

Example: The result list will include resources that include both keywords -- "distracted driving" and "texting" -- in the same article or resource, represented in the shaded area where the circles intersect (area shaded in purple).

  • Broadens search results ("OR means more!")
  • Connects two or more synonyms or related keywords/concepts
  • Resources appearing in the results list will include any of the terms connected with the OR connector

in literature review articles authors should

Venn diagram of the OR connector

Example:  The result list will include resources that include the keyword "texting" OR the keyword "cell phone" (entire area shaded in blue); either is acceptable.

  • Excludes keywords or concepts from the search
  • Narrows results by removing resources that contain the keyword or term connected with the NOT connector
  • Use sparingly

in literature review articles authors should

Venn diagram of the NOT connector

Example: The result list will include all resources that include the term "car" (green area) but will exclude any resource that includes the term "motorcycle" (purple area) even though the term car may be present in the resource.

A library database searches for keywords throughout the entire resource record including the full-text of the resource, subject headings, tags, bibliographic information, etc.

  • Natural language words or short phrases that describe a concept or idea
  • Can retrieve too few or irrelevant results due to full-text searching (What words would an author use to write about this topic?)
  • Provide flexibility in a search
  • Must consider synonyms or related terms to improve search results
  • TIP: Build a Keyword List

in literature review articles authors should

Example:  The keyword list above was developed to find resources that discuss how texting while driving results in accidents.  Notice that there are synonyms (texting and "text messaging"), related terms ("cell phones" and texting), and spelling variations ("cell phone" and cellphone).  Using keywords when searching full text requires consideration of various words that express an idea or concept.

  • Subject Headings
  • Predetermined "controlled vocabulary" database editors apply to resources to describe topical coverage of content
  • Can retrieve more precise search results because every article assigned that subject heading will be retrieved.
  • Provide less flexibility in a search
  • Can be combined with a keyword search to focus search results.
  • TIP: Consult database subject heading list or subject headings assigned to relevant resources

in literature review articles authors should

Example 1: In EBSCO's Academic Search Complete, clicking on the "Subject Terms" tab provides access to the entire subject heading list used in the database.  It also allows a search for specific subject terms.

in literature review articles authors should

Example 2:  A subject term can be incorporated into a keyword search by clicking on the down arrow next to "Select a Field" and selecting "Subject Terms" from the dropdown list.  Also, notice how subject headings are listed below the resource title, providing another strategy for discovering subject headings used in the database.

When a search term is more than one word, enclose the phrase in quotation marks to retrieve more precise and accurate results.  Using quotation marks around a term will search it as a "chunk," searching for those particular words together in that order within the text of a resource. 

"cell phone"

"distracted driving"

"car accident"

TIP: In some databases, neglecting to enclose phrases in quotation marks will insert the AND Boolean connector between each word resulting in unintended search results.

Truncation provides an option to search for a root of a keyword in order to retrieve resources that include variations of that word.  This feature can be used to broaden search results, although some results may not be relevant.  To truncate a keyword, type an asterisk (*) following the root of the word.

For example:

in literature review articles authors should

Library databases provide a variety of tools to limit and refine search results.  Limiters provide the ability to limit search results to resources having specified characteristics including:

  • Resource type
  • Publication date
  • Geographic location

In both the EBSCO and ProQuest databases, the limiting tools are located in the left panel of the results page.

                                                 EBSCO                                                     ProQuest

in literature review articles authors should

The short video below provides a demonstration of how to use limiters to refine a list of search results.

Each resource in a library database is stored in a record.  In addition to the full-text of the resources, searchable Fields are attached that typically include:

  • Journal title
  • Date of Publication

Incorporating Fields into your search can assist in focusing and refining search results by limiting the results to those resources that include specific information in a particular field.

In both EBSCO and ProQuest databases, selecting the Advanced Search option will allow Fields to be included in a search.

For example, in the Advanced Search option in EBSCO's Academic Search Complete database, clicking on the down arrow next to "Select a Field" provides a list of fields that can be searched within that database.  Select the field and enter the information in the text box to the left to use this feature.

in literature review articles authors should

Stop words are short, commonly used words--articles, prepositions, and pronouns-- that are automatically dropped from a search.  Typical stop words include:

In library databases, a stop word will not be searched even if it is included in a phrase enclosed in quotation marks.  In some instances, a word will be substituted for the stop word to allow for the other words in the phrase to be searched in proximity to one another within the text of the resource.

For example, if you searched company of America, your result list will include these variatons:

  • company in America
  • company of America
  • company for America

Creating an Search String

This short video demonstrates how to create a search string -- keywords connected with Boolean operators -- to use in a library database search to retrieve relevant resources for any research assignment.

  • Database Search Menu Template Use this search menu template to plan a database search.
  • Last Updated: Feb 16, 2024 10:01 AM
  • URL: https://guides.rasmussen.edu/LitReview

Literature reviews as independent studies: guidelines for academic practice

  • Original Paper
  • Open access
  • Published: 14 October 2022
  • Volume 16 , pages 2577–2595, ( 2022 )

Cite this article

You have full access to this open access article

  • Sascha Kraus   ORCID: orcid.org/0000-0003-4886-7482 1 , 2 ,
  • Matthias Breier 3 ,
  • Weng Marc Lim 4 , 8 , 22 ,
  • Marina Dabić 5 , 6 ,
  • Satish Kumar 7 , 8 ,
  • Dominik Kanbach 9 , 10 ,
  • Debmalya Mukherjee 11 ,
  • Vincenzo Corvello 12 ,
  • Juan Piñeiro-Chousa 13 ,
  • Eric Liguori 14 ,
  • Daniel Palacios-Marqués 15 ,
  • Francesco Schiavone 16 , 17 ,
  • Alberto Ferraris 18 , 21 ,
  • Cristina Fernandes 19 , 20 &
  • João J. Ferreira 19  

55k Accesses

239 Citations

4 Altmetric

Explore all metrics

Review articles or literature reviews are a critical part of scientific research. While numerous guides on literature reviews exist, these are often limited to the philosophy of review procedures, protocols, and nomenclatures, triggering non-parsimonious reporting and confusion due to overlapping similarities. To address the aforementioned limitations, we adopt a pragmatic approach to demystify and shape the academic practice of conducting literature reviews. We concentrate on the types, focuses, considerations, methods, and contributions of literature reviews as independent, standalone studies. As such, our article serves as an overview that scholars can rely upon to navigate the fundamental elements of literature reviews as standalone and independent studies, without getting entangled in the complexities of review procedures, protocols, and nomenclatures.

Similar content being viewed by others

in literature review articles authors should

How to Write and Publish a Research Paper for a Peer-Reviewed Journal

Clara Busse & Ella August

in literature review articles authors should

What is Qualitative in Qualitative Research

Patrik Aspers & Ugo Corte

in literature review articles authors should

Criteria for Good Qualitative Research: A Comprehensive Review

Drishti Yadav

Avoid common mistakes on your manuscript.

1 Introduction

A literature review – or a review article – is “a study that analyzes and synthesizes an existing body of literature by identifying, challenging, and advancing the building blocks of a theory through an examination of a body (or several bodies) of prior work (Post et al. 2020 , p. 352). Literature reviews as standalone pieces of work may allow researchers to enhance their understanding of prior work in their field, enabling them to more easily identify gaps in the body of literature and potential avenues for future research. More importantly, review articles may challenge established assumptions and norms of a given field or topic, recognize critical problems and factual errors, and stimulate future scientific conversations around that topic. Literature reviews Footnote 1 come in many different formats and purposes:

Some review articles conduct a critical evaluation of the literature, whereas others elect to adopt a more exploratory and descriptive approach.

Some reviews examine data, methodologies, and findings, whereas others look at constructs, themes, and theories.

Some reviews provide summaries by holistically synthesizing the existing research on a topic, whereas others adopt an integrative approach by assessing related and interdisciplinary work.

The number of review articles published as independent or standalone studies has been increasing over time. According to Scopus (i.e., search database ), reviews (i.e., document type ) were first published in journals (i.e., source type ) as independent studies in 1945, and they subsequently appeared in three digits yearly from the late 1980s to the late 1990s, four digits yearly from the early 2000s to the late 2010s, and five digits in the year 2021 (Fig.  1 ). This increase is indicative that reviewers and editors in business and management research alike see value and purpose in review articles to such a level that they are now commonly accepted as independent, standalone studies. This development is also reflected in the fact that some academic journals exclusively publish review articles (e.g., the Academy of Management Annals , or the  International Journal of Management Reviews ), and journals publishing in various fields often have special issues dedicated to literature reviews on certain topic areas (e.g., the Journal of Management and the Journal of International Business Studies ).

figure 1

Full-year publication trend of review articles on Scopus (1945–2021)

One of the most important prerequisites of a high-quality review article is that the work follows an established methodology, systematically selects and analyzes articles, and periodically covers the field to identify latest developments (Snyder 2019 ). Additionally, it needs to be reproducible, well-evidenced, and transparent, resulting in a sample inclusive of all relevant and appropriate studies (Gusenbauer and Haddaway 2020; Hansen et al. 2021 ). This observation is in line with Palmatier et al. ( 2018 ), who state that review articles provide an important synthesis of findings and perspectives in a given body of knowledge. Snyder ( 2019 ) also reaffirmed this rationale, pointing out that review articles have the power to answer research questions beyond that which can be achieved in a single study. Ultimately, readers of review articles stand to gain a one-stop, state-of-the-art synthesis (Lim et al. 2022a ; Popli et al. 2022) that encapsulates critical insights through the process of re-interpreting, re-organizing, and re-connecting a body knowledge (Fan et al. 2022 ).

There are many reasons to conduct review articles. Kraus et al. ( 2020 ) explicitly mention the benefits of conducting systematic reviews by declaring that they often represent the first step in the context of larger research projects, such as doctoral dissertations. When carrying out work of this kind, it is important that a holistic overview of the current state of literature is achieved and embedded into a proper synthesis. This allows researchers to pinpoint relevant research gaps and adequately fit future conceptual or empirical studies into the state of the academic discussion (Kraus et al., 2021 ). A review article as an independent or standalone study is a viable option for any academic – especially young scholars, such as doctoral candidates – who wishes to delve into a specific topic for which a (recent) review article is not available.

The process of conducting a review article can be challenging, especially for novice scholars (Boell and Cecez-Kecmanovic 2015 ). Therefore, it is not surprising that numerous guides have been written in an attempt to improve the quality of review studies and support emerging scholars in their endeavors to have their work published. These guides for conducting review articles span a variety of academic fields, such as engineering education (Borrego et al. 2014 ), health sciences (Cajal et al. 2020 ), psychology (Laher and Hassem 2020 ), supply chain management (Durach et al. 2017 ), or business and entrepreneurship (Kraus et al. 2020 ; Tranfield et al. 2003 ) – the latter were among the first scholars to recognize the need to educate business/management scholars on the roles of review studies in assembling, ascertaining, and assessing the intellectual territory of a specific knowledge domain. Furthermore, they shed light on the stages (i.e., planning the review, conducting the review, reporting, and dissemination) and phases (i.e., identifying the need for a review, preparation of a proposal for a review, development of a review protocol, identification of research, selection of studies, study quality assessment, data extraction and monitoring progress, data synthesis, the report and recommendations, and getting evidence into practice) of conducting a systematic review. Other scholars have either adapted and/or developed new procedures (Kraus et al. 2020 ; Snyder 2019 ) or established review protocols such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram (Moher et al. 2015 ). The latter provides a checklist that improves transparency and reproducibility, thus reducing questionable research practices. The declarative and procedural knowledge of a checklist allows users to derive value from (and, in some cases, produce) methodological literature reviews.

Two distinct and critical gaps or issues provide impetus for our article. First, while the endeavors of the named scholars are undoubtedly valuable contributions, they often encourage other scholars to explain the methodology of their review studies in a non-parsimonious way ( 1st issue ). This can become problematic if this information distracts and deprives scholars from providing richer review findings, particularly in instances in which publication outlets impose a strict page and/or word limit. More often than not, the early parts (i.e., stages/phases, such as needs, aims, and scope) of these procedures or protocols are explained in the introduction, but they tend to be reiterated in the methodology section due to the prescription of these procedures or protocols. Other parts of these procedures or protocols could also be reported more parsimoniously, for example, by filtering out documents, given that scientific databases (such as Scopus or Web of Science ) have since been upgraded to allow scholars to select and implement filtering criteria when conducting a search (i.e., criterion-by-criterion filtering may no longer be necessary). More often than not, the procedures or protocols of review studies can be signposted (e.g., bracket labeling) and disclosed in a sharp and succinct manner while maintaining transparency and replicability.

Other guides have been written to introduce review nomenclatures (i.e., names/naming) and their equivalent philosophical underpinnings. Palmatier et al. ( 2018 ) introduced three clearly but broadly defined nomenclatures of literature reviews as independent studies: domain-based reviews, theory-based reviews, and method-based reviews. However, such review nomenclatures can be confusing due to their overlapping similarities ( 2nd issue ). For example, Lim et al. ( 2022a ) highlighted their observation that the review nomenclatures associated with domain-based reviews could also be used for theory-based and method-based reviews.

The two aforementioned issues – i.e., the lack of a parsimonious understanding and the reporting of the review methodology , and the confusion emerging from review nomenclatures – are inarguably the unintended outcomes of diving into an advanced (i.e., higher level) understanding of literature review procedures, protocols, and nomenclatures from a philosophical perspective (i.e., underpinnings) without a foundational (i.e., basic level) understanding of the fundamental (i.e., core) elements of literature reviews from a pragmatic perspective. Our article aims to shed light on these issues and hopes to provide clarity for future scholarly endeavors.

Having a foundational understanding of literature reviews as independent studies is (i) necessary when addressing the aforementioned issues; (ii) important in reconciling and scaffolding our understanding, and (iii) relevant and timely due to the proliferation of literature reviews as independent studies. To contribute a solution toward addressing this gap , we aim to demystify review articles as independent studies from a pragmatic standpoint (i.e., practicality). To do so, we deliberately (i) move away from review procedures, protocols, and nomenclatures, and (ii) invest our attention in developing a parsimonious, scaffolded understanding of the fundamental elements (i.e., types, focuses, considerations, methods, and contributions) of review articles as independent studies.

Three contributions distinguish our article. It is worth noting that pragmatic guides (i.e., foundational knowledge), such as the present one, are not at odds with extant philosophical guides (i.e., advanced knowledge), but rather they complement them. Having a foundational knowledge of the fundamental elements of literature reviews as independent studies is valuable , as it can help scholars to (i) gain a good grasp of the fundamental elements of literature reviews as independent studies ( 1st contribution ), and (ii) mindfully adopt or adapt existing review procedures, protocols, and nomenclatures to better suit the circumstances of their reviews (e.g., choosing and developing a well-defined review nomenclature, and choosing and reporting on review considerations and steps more parsimoniously) ( 2nd contribution ). Therefore, this pragmatic guide serves as (iii) a foundational article (i.e., preparatory understanding) for literature reviews as independent studies ( 3rd contribution ). Following this, extant guides using a philosophical approach (i.e., advanced understanding) could be relied upon to make informed review decisions (e.g., adoption, adaptation) in response to the conventions of extant review procedures, protocols, and nomenclatures (Fig.  2 ).

figure 2

Foundational and advanced understanding of literature reviews as independent studies

2 Fundamental elements of literature reviews as independent studies

A foundational understanding of literature reviews as independent studies can be acquired through the appreciation of five fundamental elements – i.e., types, focuses, considerations, methods, and contributions – which are illustrated in Fig.  3 and summarized in the following sections.

figure 3

Fundamental elements of literature reviews as independent studies

There are two types of literature reviews as independent studies: systematic literature reviews ( SLRs ) and non-systematic literature reviews ( non-SLRs ). It is important to recognize that SLRs and non-SLRs are not review nomenclatures (i.e., names/naming) but rather review types (i.e., classifications).

In particular, SLRs are reviews carried out in a systematic way using an adopted or adapted procedure or protocol to guide data curation and analysis, thus enabling transparent disclosure and replicability (Lim et al. 2022a ; Kraus et al. 2020 ). Therefore, any review nomenclature guided by a systematic methodology is essentially an SLR. The origin of this type of literature review can be traced back to the evidence-based medicine movement in the early 1990s, with the objective being to overcome the issue of inconclusive findings in studies for medical treatments (Boell and Cecez-Kecmanovic 2015 ).

In contrast, non-SLRs are reviews conducted without any systematic procedure or protocol; instead, they weave together relevant literature based on the critical evaluations and (subjective) choices of the author(s) through a process of discovery and critique (e.g., pointing out contradictions and questioning assertions or beliefs); they are shaped by the exposure, expertise, and experience (i.e., the “3Es” in judgement calls) of the author(s). Therefore, non-SLRs are essentially critical reviews of the literature (Lim and Weissmann 2021 ).

2.2 Focuses

Unlike Palmatier et al. ( 2018 ) who considered domain-based reviews, theory-based reviews, and method-based reviews as review nomenclatures, we consider domain , theory , and method as three substantive focuses that can take center stage in literature reviews as independent studies. This is in line with our attempt to move away from review nomenclatures when providing a foundational understanding of literature reviews as independent studies.

A review that is domain-focused can examine: (i) a  concept (e.g., customer engagement; Lim et al. 2022b ; digital transformation; Kraus et al. 2021 ; home sharing; Lim et al. 2021 ; sharing economy; Lim 2020 ), (ii) a context (e.g., India; Mukherjee et al. 2022a ), (iii) a discipline (e.g., entrepreneurship; Ferreira et al. 2015 ; international business; Ghauri et al. 2021 ), (iv) a field (e.g., family business; Lahiri et al. 2020 ; Rovelli et al. 2021 ; female entrepreneurship; Ojong et al. 2021 ), or (v) an outlet (e.g., Journal of Business Research ; Donthu et al. 2020 ; Management International Review ; Mukherjee et al. 2021 ; Review of Managerial Science ; Mas-Tur et al. 2020 ), which typically offer broad, overarching insights.

Domain-focused hybrids , such as the between-domain hybrid (e.g., concept-discipline hybrid, such as digital transformation in business and management; Kraus et al. 2022 ; religion in business and entrepreneurship; Kumar et al. 2022a ; personality traits in entrepreneurship; Salmony and Kanbach 2022 ; and policy implications in HR and OB research; Aguinis et al., 2022 ) and the within-domain hybrid (e.g., the concept-concept hybrid, such as customer engagement and social media; Lim and Rasul 2022 ; and global business and organizational excellence; Lim 2022 ; and the discipline-discipline hybrid, such as neuromarketing; Lim 2018 ) are also common as they can provide finer-grained insights.

A review that is theory-focused can explore a standalone theory (e.g., theory of planned behavior; Duan and Jiang 2008 ), as well as a theory in conjunction with a domain , such as the concept-theory hybrid (e.g., behavioral control and theory of planned behavior; Lim and Weissmann 2021 ) and the theory-discipline hybrid (e.g., theory of planned behavior in hospitality, leisure, and tourism; Ulker-Demirel and Ciftci 2020 ), or a theory in conjunction with a method (e.g., theory of planned behavior and structural equation modeling).

A review that is method-focused can investigate a standalone method (e.g., structural equation modeling; Deng et al. 2018 ) or a method in conjunction with a domain , such as the method-discipline hybrid (e.g., fsQCA in business and management; Kumar et al. 2022b ).

2.3 Planning the review, critical considerations, and data collection

The considerations required for literature reviews as independent studies depend on their type: SLRs or non-SLRs.

For non-SLRs, scholars often rely on the 3Es (i.e., exposure, expertise, and experience) to provide a critical review of the literature. Scholars who embark on non-SLRs should be well versed with the literature they are dealing with. They should know the state of the literature (e.g., debatable, underexplored, and well-established knowledge areas) and how it needs to be deciphered (e.g., tenets and issues) and approached (e.g., reconciliation proposals and new pathways) to advance theory and practice. In this regard, non-SLRs follow a deductive reasoning approach, whereby scholars initially develop a set of coverage areas for reviewing a domain, theory, or method and subsequently draw on relevant literature to shed light and support scholarly contentions in each area.

For SLRs, scholars often rely on a set of criteria to provide a well-scoped (i.e., breadth and depth), structured (i.e., organized aspects), integrated (i.e., synthesized evidence) and interpreted/narrated (i.e., describing what has happened, how and why) systematic review of the literature. Footnote 2 In this regard, SLRs follow an inductive reasoning approach, whereby a set of criteria is established and implemented to develop a corpus of scholarly documents that scholars can review. They can then deliver a state-of-the-art overview, as well as a future agenda for a domain, theory, or method. Such criteria are often listed in philosophical guides on SLR procedures (e.g., Kraus et al. 2020 ; Snyder 2019 ) and protocols (e.g., PRISMA), and they may be adopted/adapted with justifications Footnote 3 . Based on their commonalities they can be summarized as follows:

Search database (e.g., “Scopus” and/or “Web of Science”) can be defined based on justified evidence (e.g., by the two being the largest scientific databases of scholarly articles that can provide on-demand bibliographic data or records; Pranckutė 2021 ). To avoid biased outcomes due to the scope covered by the selected database, researchers could utilize two or more different databases (Dabić et al. 2021 ).

Search keywords may be developed by reading scholarly documents and subsequently brainstorming with experts. The expanding number of databases, journals, periodicals, automated approaches, and semi-automated procedures that use text mining and machine learning can offer researchers the ability to source new, relevant research and forecast the citations of influential studies. This enables them to determine further relevant articles.

Boolean operators (e.g., AND, OR) should be strategically used in developing the  string   of search keywords (e.g., “engagement” AND “customer” OR “consumer” OR “business”). Furthermore, the correct and precise application of quotation marks is important but is very frequently sidestepped, resulting in incorrect selection processes and differentiated results.

Search period (e.g., between a specified period [e.g., 2000 to 2020] or up to the latest full year at the time or writing [e.g., up to 2021]) can be defined based on the justified scope of study (e.g., contemporary evolution versus historical trajectory).

Search field (e.g., “article title, abstract, keywords”) can be defined based on justified assumptions (e.g., it is assumed that the focus of relevant documents will be mentioned in the article title, abstract, and/or keywords).

Subject area (e.g., “business, management, and accounting”) can be defined based on justified principles (e.g., the focus of the review is on the marketing discipline, which is located under the “business, management, and accounting” subject area in Scopus).

Publication stage (e.g., “final”) can be defined based on justified grounds (e.g., enabling greater accuracy in replication).

Document type (e.g., “article” and/or “review”), which reflects the type of scientific/practical contributions (e.g., empirical, synthesis, thought), can be defined based on justified rationales (e.g., articles selected because they are peer-reviewed; editorials not selected because they are not peer-reviewed).

Source type (e.g., “journal”) can be defined based on justified reasons (e.g., journals selected because they publish finalized work; conference proceedings not selected because they are work in progress, and in business/management, they are usually not being considered as full-fledged “publications”).

Language (e.g., “English”) can be determined based on justified limitations (e.g., nowadays, there are not many reasons to use another language besides the academic lingua franca English). Different spellings should also be considered, as the literature may contain both American and British spelling variants (e.g., organization and organisation). Truncation and wildcards in searches are recommended to capture both sets of spellings. It is important to note that each database varies in its symbology.

Quality filtering (e.g., “A*” and “A” or “4*”, “4”, and “3”) can be defined based on justified motivations (e.g., the goal is to unpack the most originally and rigorously produced knowledge, which is the hallmark of premier journals, such as those ranked “A*” and “A” by the Australian Business Deans Council [ABDC] Journal Quality List [JQL] and rated “4*”, “4”, and “3” by the Chartered Association of Business Schools [CABS] Academic Journal Guide [AJG]).

Document relevance (i.e., within the focus of the review) can be defined based on justified judgement (e.g., for a review focusing on customer engagement, articles that mention customer engagement as a passing remark without actually investigating it would be excluded).

Others: Screening process should be accomplished by beginning with the deduction of duplicate results from other databases, tracked using abstract screening to exclude unfitting studies, and ending with the full-text screening of the remaining documents.

Others: Exclusion-inclusion criteria interpretation of the abstracts/articles is obligatory when deciding whether or not the articles dealt with the matter. This step could involve removing a huge percentage of initially recognized articles.

Others: Codebook building pertains to the development of a codebook of the main descriptors within a specific field. An inductive approach can be followed and, in this case, descriptors are not established beforehand. Instead, they are established through the analysis of the articles’ content. This procedure is made up of several stages: (i) the extraction of important content from titles, abstracts, and keywords; (ii) the classification of this content to form a reduced list of the core descriptors; and (iii) revising the codebook in iterations and combining similar categories, thus developing a short list of descriptors (López-Duarte et al. 2016 , p. 512; Dabić et al. 2015 ; Vlacic et al. 2021 ).

2.4 Methods

Various methods are used to analyze the pertinent literature. Often, scholars choose a method for corpus analysis before corpus curation. Knowing the analytical technique beforehand is useful, as it allows researchers to acquire and prepare the right data in the right format. This typically occurs when scholars have decided upon and justified pursuing a specific review nomenclature upfront (e.g., bibliometric reviews) based on the problem at hand (e.g., broad domain [outlet] with a large corpus [thousands of articles], such as a premier journal that has been publishing for decades) (Donthu et al. 2021 ). However, this may not be applicable in instances where (i) scholars do not curate a corpus of articles (non-SLRs), and (ii) scholars only know the size of the corpus of articles once that corpus is curated (SLRs). Therefore, scholars may wish to decide on a method of analyzing the literature depending on (i) whether they rely on a corpus of articles (i.e., yes or no), and (ii) the size of the corpus of articles that they rely on to review the literature (i.e., n  = 0 to ∞).

When analytical techniques (e.g., bibliometric analysis, critical analysis, meta-analysis) are decoupled from review nomenclatures (e.g., bibliometric reviews, critical reviews, meta-analytical reviews), we uncover a toolbox of the following methods for use when analyzing the literature:

Bibliometric analysis measures the literature and processes data by using algorithm, arithmetic, and statistics to analyze, explore, organize, and investigate large amounts of data. This enables scholars to identify and recognize potential “hidden patterns” that could help them during the literature review process. Bibliometrics allows scholars to objectively analyze a large corpus of articles (e.g., high hundreds or more) using quantitative techniques (Donthu et al. 2021 ). There are two overarching categories for bibliometric analysis: performance analysis and science mapping. Performance analysis enables scholars to assess the productivity (publication) and impact (citation) of the literature relating to a domain, method, or theory using various quantitative metrics (e.g., average citations per publication or year, h -index, g -index, i -index). Science mapping grants scholars the ability to map the literature in that domain, method, or theory based on bibliographic data (e.g., bibliographic coupling generates thematic clusters based on similarities in shared bibliographic data [e.g., references] among citing articles; co-citation analysis generates thematic clusters based on commonly cited articles; co-occurrence analysis generates thematic clusters based on bibliographic data [e.g., keywords] that commonly appear together; PageRank analysis generates thematic clusters based on articles that are commonly cited in highly cited articles; and topic modeling generates thematic clusters based on the natural language processing of bibliographic data [e.g., article title, abstract, and keywords]). Footnote 4 Given the advancement in algorithms and technology, reviews using bibliometric analysis are considered to be smart (Kraus et al. 2021 ) and technologically-empowered (Kumar et al. 2022b ) SLRs, in which a review has harnessed the benefits of (i) the machine learning of the bibliographic data of scholarly research from technologically-empowered scientific databases, and (ii) big data analytics involving various science mapping techniques (Kumar et al. 2022c ).

Content analysis allows scholars to analyze a small to medium corpus of articles (i.e., tens to low hundreds) using quantitative and qualitative techniques. From a quantitative perspective , scholars can objectively carry out a content analysis by quantifying a specific unit of analysis . A useful method of doing so involves adopting, adapting, or developing an organizing framework . For example, Lim et al. ( 2021 ) employed an organizing (ADO-TCM) framework to quantify content in academic literature based on: (i) the categories of knowledge; (ii) the relationships between antecedents, decisions, and outcomes; and (iii) the theories, contexts, and methods used to develop the understanding for (i) and (ii). The rapid evolution of software for content analysis allows scholars to carry out complex elaborations on the corpus of analyzed articles, so much so that the most recent software enables the semi-automatic development of an organizing framework (Ammirato et al. 2022 ). From a qualitative perspective , scholars can conduct a content analysis or, more specifically, a thematic analysis , by subjectively organizing the content into themes. For example, Creevey et al. ( 2022 ) reviewed the literature on social media and luxury, providing insights on five core themes (i.e., luxury brand strategy, luxury brand social media communications, luxury consumer attitudes and perceptions, engagement, and the influence of social media on brand performance-related outcomes) generated through a content (thematic) analysis. Systematic approaches for inductive concept development through qualitative research are similarly applied in literature reviews in an attempt to reduce the subjectivity of derived themes. Following the principles of the approach by Gioia et al. ( 2012 ), Korherr and Kanbach ( 2021 ) develop a taxonomy of human-related capabilities in big data analytics. Building on a sample of 75 studies for the literature review, 33 first-order concepts are identified. These are categorized into 15 second-order themes and are finally merged into five aggregate dimensions. Using the same procedure, Leemann and Kanbach ( 2022 ) identify 240 idiosyncratic dynamic capabilities in a sample of 34 studies for their literature review. They then categorize these into 19 dynamic sub-capabilities. The advancement of technology also makes it possible to conduct content analysis using computer assisted qualitative data analysis (CAQDA) software (e.g., ATLAS.ti, Nvivo, Quirkos) (Lim et al. 2022a ).

Critical analysis allows scholars to subjectively use their 3Es (i.e., exposure, expertise, and experience) to provide a critical evaluation of academic literature. This analysis is typically used in non-SLRs, and can be deployed in tandem with other analyses, such as bibliometric analysis and content analysis in SLRs, which are used to discuss consensual, contradictory, and underexplored areas of the literature. For SLRs, scholars are encouraged to engage in critical evaluations of the literature so that they can truly contribute to advancing theory and practice (Baker et al. 2022 ; Lim et al. 2022a ; Mukherjee et al. 2022b ).

Meta-analysis allows scholars to objectively establish a quantitative estimate of commonly studied relationships in the literature (Grewal et al. 2018 ). This analysis is typically employed in SLRs intending to reconcile a myriad of relationships (Lim et al. 2022a ). The relationships established are often made up of conflicting evidence (e.g., a positive or significant effect in one study, but a negative or non-significant effect in another study). However, through meta-analysis, scholars are able to identify potential factors (e.g., contexts or sociodemographic information) that may have led to the conflict.

Others: Multiple correspondence analysis helps to map the field, assessing the associations between qualitative content within a matrix of variables and cases. Homogeneity Analysis by Means of Alternating Least Squares ( HOMALS ) is also considered useful in allowing researchers to map out the intellectual structure of a variety of research fields (Gonzalez-Loureiro et al. 2015 ; Gonzalez-Louriero 2021; Obradović et al. 2021 ). HOMALS can be performed in R or used along with a matrix through SPSS software. In summary, the overall objective of this analysis is to discover a low dimensional representation of the original high dimensional space (i.e., the matrix of descriptors and articles). To measure the goodness of fit, a loss function is used. This function is used minimally, and the HOMALS algorithm is applied to the least squares loss functions in SPSS. This analysis provides a proximity map, in which articles and descriptors are shown in low-dimensional spaces (typically on two axes). Keywords are paired and each couple that appears together in a large number of articles is shown to be closer on the map and vice-versa.

When conducting a literature review, software solutions allow researchers to cover a broad range of variables, from built-in functions of statistical software packages to software orientated towards meta-analyses, and from commercial to open-source solutions. Personal preference plays a huge role, but the decision as to which software will be the most useful is entirely dependent on how complex the methods and the dataset are. Of all the commercial software providers, we have found the built-in functions of (i) R and VOSviewer most useful in performing bibliometric analysis (Aria and Cuccurullo 2017 ; R Core Team 2021 ; Van Eck and Waltman 2014 ) and (ii) Stata most useful in performing meta-analytical tasks.

Many different analytical tools have been used. These include simple document counting, citation analysis, word frequency analysis, cluster analysis, co-word analysis, and cooperation analysis (Daim et al. 2006 ). Software has also been produced for bibliometric analysis, such as the Thomson Data Analyzer (TDA), which Thomson Reuters created, and CiteSpace developed by Chen ( 2013 ). VOSviewer helps us to construct and visualize bibliometric networks, which can include articles, journals, authors, countries, and institutions, among others (Van Eck and Waltman 2014 ). These can be organized based on citations, co-citations, bibliographic coupling, or co-authorship relations. In addition, VOSviewer provides text mining functions, which can be used to facilitate a better understanding of co-occurrence networks with regards to the key terms taken from a body of scientific literature (Donthu et al. 2021 ; Wong 2018 ). Other frequently used tools include for bibliometric analysis include Bibliometrix/Biblioshiny in R, CitNetExplorer, and Gephi, among others.

2.5 Contributions

Well-conducted literature reviews may make multiple contributions to the literature as standalone, independent studies.

Generally, there are three primary contributions of literature reviews as independent studies: (i) to provide an overview of current knowledge in the domain, method, or theory, (ii) to provide an evaluation of knowledge progression in the domain, method, or theory, including the establishment of key knowledge, conflicting or inconclusive findings, and emerging and underexplored areas, and (iii) to provide a proposal for potential pathways for advancing knowledge in the domain, method, or theory (Lim et al. 2022a , p. 487). Developing theory through literature reviews can take many forms, including organizing and categorizing the literature, problematizing the literature, identifying and exposing contradictions, developing analogies and metaphors, and setting out new narratives and conceptualizations (Breslin and Gatrell 2020 ). Taken collectively, these contributions offer crystalized, evidence-based insights that both ‘mine’ and ‘prospect’ the literature, highlighting extant gaps and how they can be resolved (e.g., flags paradoxes or theoretical tensions, explaining why something has not been done, what the challenges are, and how these challenges can be overcome). These contributions can be derived through successful bibliometric analysis, content analysis, critical analysis, and meta-analysis.

Additionally, the deployment of specific methods can bring in further added value. For example, a performance analysis in a bibliometric analysis can contribute to: (i) objectively assessing and reporting research productivity and impact ; (ii) ascertaining reach for coverage claims ; (iii) identifying social dominance and hidden biases ; (iv) detecting anomalies ; and (v) evaluating ( equitable ) relative performance ; whereas science mapping in bibliometric analysis can contribute to: (i) objectively discovering thematic clusters of knowledge ; (ii) clarifying nomological networks ; (iii) mapping social patterns ; (iv) tracking evolutionary nuances ; and (v) recognizing knowledge gaps (Mukherjee et al. 2022b , p. 105).

3 Conclusion

Independent literature reviews will continue to be written as a result of their necessity, importance, relevance, and urgency when it comes to advancing knowledge (Lim et al. 2022a ; Mukherjee et al. 2022b ), and this can be seen in the increasing number of reviews being published over the last several years. Literature reviews advance academic discussion. Journal publications on various topics and subject areas are becoming more frequent sites for publication. This trend will only heighten the need for literature reviews. This article offers directions and control points that address the needs of three different stakeholder groups: producers (i.e., potential authors), evaluators (i.e., journal editors and reviewers), and users (i.e., new researchers looking to learn more about a particular methodological issue, and those teaching the next generation of scholars). Future producers will derive value from this article’s teachings on the different fundamental elements and methodological nuances of literature reviews. Procedural knowledge (i.e., using control points to assist in decision-making during the manuscript preparation phase) will also be of use. Evaluators will be able to make use of the procedural and declarative knowledge evident in control points as well. As previously outlined, the need to cultivate novelty within research on business and management practices is vital. Scholars must also be supported to choose not only safe mining approaches; they should also be encouraged to attempt more challenging and risky ventures. It is important to note that abstracts often seem to offer a lot of potential, stating that authors intend to make large conceptual contributions, broadening the horizons of the field.

Our article offers important insights also for practitioners. Noteworthily, our framework can support corporate managers in decomposing and better understanding literature reviews as ad-hoc and independent studies about specific topics that matter for their organization. For instance, practitioners can understand more easily what are the emerging trends within their domain of interest and make corporate decisions in line with such trends.

This article arises from an intentional decoupling from philosophy, in favor of adopting a more pragmatic approach. This approach can assist us in clarifying the fundamental elements of literature reviews as independent studies. Five fundamental elements must be considered: types, focuses, considerations, methods, and contributions. These elements offer a useful frame for scholars starting to work on a literature review. Overview articles (guides) such as ours are thus invaluable, as they equip scholars with a solid foundational understanding of the integral elements of a literature review. Scholars can then put these teachings into practice, armed with a better understanding of the philosophy that underpins the procedures, protocols, and nomenclatures of literature reviews as independent studies.

Data availability

Our manuscript has no associate data.

Our focus here is on standalone literature reviews in contrast with literature reviews that form the theoretical foundation for a research article.

Scoping reviews, structured reviews, integrative reviews, and interpretive/narrative reviews are commonly found in review nomenclature. However, the philosophy of these review nomenclatures essentially reflects what constitutes a good SLR. That is to say, a good SLR should be well scoped, structured, integrated, and interpreted/narrated. This observation reaffirms our position and the value of moving away from review nomenclatures to gain a foundational understanding of literature reviews as independent studies.

Given that many of these considerations can be implemented simultaneously in contemporary versions of scientific databases, scholars may choose to consolidate them into a single (or a few) step(s), where appropriate, so that they can be reported more parsimoniously. For a parsimonious but transparent and replicable exemplar, see Lim ( 2022 ).

Where keywords are present (e.g., author keywords or keywords derived from machine learning [e.g., natural language processing]), it is assumed that each keyword represents a specific meaning (e.g., topic [concept, context], method), and that a collection of keywords grouped under the same cluster represents a specific theme.

Aguinis H, Jensen SH, Kraus S (2022) Policy implications of organizational behavior and human resource management research. Acad Manage Perspect 36(3):1–22

Article   Google Scholar  

Ammirato S, Felicetti AM, Rogano D, Linzalone R, Corvello V (2022) Digitalising the systematic literature review process: The My SLR platform. Knowl Manage Res Pract. doi: https://doi.org/10.1080/14778238.2022.2041375

Aria M, Cuccurullo C (2017) bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetrics 11(4):959–975

Baker WE, Mukherjee D, Perin MG (2022) Learning orientation and competitive advantage: A critical synthesis and future directions. J Bus Res 144:863–873

Boell SK, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews. J Inform Technol 30:161–173

Borrego M, Foster MJ, Froyd JE (2014) Systematic literature reviews in engineering education and other developing interdisciplinary fields. J Eng Educ 103(1):45–76

Breslin D, Gatrell C (2020) Theorizing through literature reviews: The miner-prospector continuum. Organizational Res Methods. https://doi.org/10.1177/1094428120943288 (in press)

Cajal B, Jiménez R, Gervilla E, Montaño JJ (2020) Doing a systematic review in health sciences. Clínica y Salud 31(2):77–83

Chen C (2013) Mapping scientific frontiers: The quest for knowledge visualization. Springer Science & Business Media

Creevey D, Coughlan J, O’Connor C (2022) Social media and luxury: A systematic literature review. Int J Manage Reviews 24(1):99–129

Dabić M, González-Loureiro M, Harvey M (2015) Evolving research on expatriates: what is ‘known’after four decades (1970–2012). Int J Hum Resource Manage 26(3):316–337

Dabić M, Vlačić B, Kiessling T, Caputo A, Pellegrini M(2021) Serial entrepreneurs: A review of literature and guidance for future research.Journal of Small Business Management,1–36

Daim TU, Rueda G, Martin H, Gerdsri P (2006) Forecasting emerging technologies: Use of bibliometrics and patent analysis. Technol Forecast Soc Chang 73(8):981–1012

Deng L, Yang M, Marcoulides KM (2018) Structural equation modeling with many variables: A systematic review of issues and developments. Front Psychol 9:580

Donthu N, Kumar S, Pattnaik D (2020) Forty-five years of Journal of Business Research: A bibliometric analysis. J Bus Res 109:1–14

Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM (2021) How to conduct a bibliometric analysis: An overview and guidelines. J Bus Res 133:285–296

Duan W, Jiang G (2008) A review of the theory of planned behavior. Adv Psychol Sci 16(2):315–320

Google Scholar  

Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manage 53(4):67–85

Fan D, Breslin D, Callahan JL, Szatt-White M (2022) Advancing literature review methodology through rigour, generativity, scope and transparency. Int J Manage Reviews 24(2):171–180

Ferreira MP, Reis NR, Miranda R (2015) Thirty years of entrepreneurship research published in top journals: Analysis of citations, co-citations and themes. J Global Entrepreneurship Res 5(1):1–22

Ghauri P, Strange R, Cooke FL (2021) Research on international business: The new realities. Int Bus Rev 30(2):101794

Gioia DA, Corley KG, Hamilton AL (2012) Seeking qualitative rigor in inductive research: Notes on the gioia methodology. Organizational Res Methods 16(1):15–31

Gonzalez-Loureiro M, Dabić M, Kiessling T (2015) Supply chain management as the key to a firm’s strategy in the global marketplace: Trends and research agenda. Int J Phys Distribution Logistics Manage 45(1/2):159–181. https://doi.org/10.1108/IJPDLM-05-2013-0124

Grewal D, Puccinelli N, Monroe KB (2018) Meta-analysis: Integrating accumulated knowledge. J Acad Mark Sci 46(1):9–30

Hansen C, Steinmetz H, Block J(2021) How to conduct a meta-analysis in eight steps: a practical guide.Management Review Quarterly,1–19

Korherr P, Kanbach DK (2021) Human-related capabilities in big data analytics: A taxonomy of human factors with impact on firm performance. RMS. https://doi.org/10.1007/s11846-021-00506-4 (in press)

Kraus S, Breier M, Dasí-Rodríguez S (2020) The art of crafting a systematic literature review in entrepreneurship research. Int Entrepreneurship Manage J 16(3):1023–1042

Kraus S, Durst S, Ferreira J, Veiga P, Kailer N, Weinmann A (2022) Digital transformation in business and management research: An overview of the current status quo. Int J Inf Manag 63:102466

Kraus S, Jones P, Kailer N, Weinmann A, Chaparro-Banegas N, Roig-Tierno N (2021) Digital transformation: An overview of the current state of the art of research. Sage Open 11(3):1–15

Kraus S, Mahto RV, Walsh ST (2021) The importance of literature reviews in small business and entrepreneurship research. J Small Bus Manage. https://doi.org/10.1080/00472778.2021.1955128 (in press)

Kumar S, Sahoo S, Lim WM, Dana LP (2022a) Religion as a social shaping force in entrepreneurship and business: Insights from a technology-empowered systematic literature review. Technol Forecast Soc Chang 175:121393

Kumar S, Sahoo S, Lim WM, Kraus S, Bamel U (2022b) Fuzzy-set qualitative comparative analysis (fsQCA) in business and management research: A contemporary overview. Technol Forecast Soc Chang 178:121599

Kumar S, Sharma D, Rao S, Lim WM, Mangla SK (2022c) Past, present, and future of sustainable finance: Insights from big data analytics through machine learning of scholarly research. Ann Oper Res. https://doi.org/10.1007/s10479-021-04410-8 (in press)

Laher S, Hassem T (2020) Doing systematic reviews in psychology. South Afr J Psychol 50(4):450–468

Leemann N, Kanbach DK (2022) Toward a taxonomy of dynamic capabilities – a systematic literature review. Manage Res Rev 45(4):486–501

Lahiri S, Mukherjee D, Peng MW (2020) Behind the internationalization of family SMEs: A strategy tripod synthesis. Glob Strategy J 10(4):813–838

Lim WM (2018) Demystifying neuromarketing. J Bus Res 91:205–220

Lim WM (2020) The sharing economy: A marketing perspective. Australasian Mark J 28(3):4–13

Lim WM (2022) Ushering a new era of Global Business and Organizational Excellence: Taking a leaf out of recent trends in the new normal. Global Bus Organizational Excellence 41(5):5–13

Lim WM, Rasul T (2022) Customer engagement and social media: Revisiting the past to inform the future. J Bus Res 148:325–342

Lim WM, Weissmann MA (2021) Toward a theory of behavioral control. J Strategic Mark. https://doi.org/10.1080/0965254X.2021.1890190 (in press)

Lim WM, Kumar S, Ali F (2022a) Advancing knowledge through literature reviews: ‘What’, ‘why’, and ‘how to contribute’. Serv Ind J 42(7–8):481–513

Lim WM, Rasul T, Kumar S, Ala M (2022b) Past, present, and future of customer engagement. J Bus Res 140:439–458

Lim WM, Yap SF, Makkar M (2021) Home sharing in marketing and tourism at a tipping point: What do we know, how do we know, and where should we be heading? J Bus Res 122:534–566

López-Duarte C, González-Loureiro M, Vidal-Suárez MM, González-Díaz B (2016) International strategic alliances and national culture: Mapping the field and developing a research agenda. J World Bus 51(4):511–524

Mas-Tur A, Kraus S, Brandtner M, Ewert R, Kürsten W (2020) Advances in management research: A bibliometric overview of the Review of Managerial Science. RMS 14(5):933–958

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Reviews 4(1):1–9

Mukherjee D, Kumar S, Donthu N, Pandey N (2021) Research published in Management International Review from 2006 to 2020: A bibliometric analysis and future directions. Manage Int Rev 61:599–642

Mukherjee D, Kumar S, Mukherjee D, Goyal K (2022a) Mapping five decades of international business and management research on India: A bibliometric analysis and future directions. J Bus Res 145:864–891

Mukherjee D, Lim WM, Kumar S, Donthu N (2022b) Guidelines for advancing theory and practice through bibliometric research. J Bus Res 148:101–115

Obradović T, Vlačić B, Dabić M (2021) Open innovation in the manufacturing industry: A review and research agenda. Technovation 102:102221

Ojong N, Simba A, Dana LP (2021) Female entrepreneurship in Africa: A review, trends, and future research directions. J Bus Res 132:233–248

Palmatier RW, Houston MB, Hulland J (2018) Review articles: Purpose, process, and structure. J Acad Mark Sci 46(1):1–5

Post C, Sarala R, Gatrell C, Prescott JE (2020) Advancing theory with review articles. J Manage Stud 57(2):351–376

Pranckutė R (2021) Web of Science (WoS) and Scopus: The titans of bibliographic information in today’s academic world. Publications 9(1):12

R Core Team (2021) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ Accessed 20th July 2022

Rovelli P, Ferasso M, De Massis A, Kraus S(2021) Thirty years of research in family business journals: Status quo and future directions.Journal of Family Business Strategy,100422

Salmony FU, Kanbach DK (2022) Personality trait differences across types of entrepreneurs: a systematic literature review. RMS 16:713–749

Snyder H (2019) Literature review as a research methodology: An overview and guidelines. J Bus Res 104:333–339

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222

Ulker-Demirel E, Ciftci G (2020) A systematic literature review of the theory of planned behavior in tourism, leisure and hospitality management research. J Hospitality Tourism Manage 43:209–219

Van Eck NJ, Waltma L (2014) CitNetExplorer: A new software tool for analyzing and visualizing citation networks. J Informetrics 8(4):802–823

Vlačić B, Corbo L, Silva e, Dabić M (2021) The evolving role of artificial intelligence in marketing: A review and research agenda. J Bus Res 128:187–203

Wong D (2018) VOSviewer. Tech Serv Q 35(2):219–220. https://doi.org/10.1080/07317131.2018.1425352

Download references

Open access funding provided by Libera Università di Bolzano within the CRUI-CARE Agreement.

Author information

Authors and affiliations.

Faculty of Economics & Management, Free University of Bozen-Bolzano, Bolzano, Italy

Sascha Kraus

Department of Business Management, University of Johannesburg, Johannesburg, South Africa

School of Business, Lappeenranta University of Technology, Lappeenranta, Finland

Matthias Breier

Sunway University Business School, Sunway University, Sunway City, Malaysia

Weng Marc Lim

Faculty of Economics and Business, University of Zagreb, Zagreb, Croatia

Marina Dabić

School of Economics and Business, University of Ljubljana, Ljubljana, Slovenia

Department of Management Studies, Malaviya National Institute of Technology Jaipur, Jaipur, India

Satish Kumar

Faculty of Business, Design and Arts, Swinburne University of Technology, Kuching, Malaysia

Weng Marc Lim & Satish Kumar

Chair of Strategic Management and Digital Entrepreneurship, HHL Leipzig Graduate School of Management, Leipzig, Germany

Dominik Kanbach

School of Business, Woxsen University, Hyderabad, India

College of Business, The University of Akron, Akron, USA

Debmalya Mukherjee

Department of Engineering, University of Messina, Messina, Italy

Vincenzo Corvello

Department of Finance, Santiago de Compostela University, Santiago de Compostela, Spain

Juan Piñeiro-Chousa

Rowan University, Rohrer College of Business, Glassboro, NJ, USA

Eric Liguori

School of Engineering Design, Universidad Politécnica de Valencia, Valencia, Spain

Daniel Palacios-Marqués

Department of Management and Quantitative Studies, Parthenope University, Naples, Italy

Francesco Schiavone

Paris School of Business, Paris, France

Department of Management, University of Turin, Turin, Italy

Alberto Ferraris

Department of Management and Economics & NECE Research Unit in Business Sciences, University of Beira Interior, Covilha, Portugal

Cristina Fernandes & João J. Ferreira

Centre for Corporate Entrepreneurship and Innovation, Loughborough University, Loughborough, UK

Cristina Fernandes

Laboratory for International and Regional Economics, Graduate School of Economics and Management, Ural Federal University, Yekaterinburg, Russia

School of Business, Law and Entrepreneurship, Swinburne University of Technology, Hawthorn, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sascha Kraus .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Kraus, S., Breier, M., Lim, W.M. et al. Literature reviews as independent studies: guidelines for academic practice. Rev Manag Sci 16 , 2577–2595 (2022). https://doi.org/10.1007/s11846-022-00588-8

Download citation

Received : 15 August 2022

Accepted : 07 September 2022

Published : 14 October 2022

Issue Date : November 2022

DOI : https://doi.org/10.1007/s11846-022-00588-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Literature reviews
  • Bibliometrics
  • Meta Analysis
  • Contributions

JEL classification

  • Find a journal
  • Publish with us
  • Track your research

Loading metrics

Open Access

Ten Simple Rules for Writing a Literature Review

* E-mail: [email protected]

Affiliations Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France, Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

  • Marco Pautasso

PLOS

Published: July 18, 2013

  • https://doi.org/10.1371/journal.pcbi.1003149
  • Reader Comments

Figure 1

Citation: Pautasso M (2013) Ten Simple Rules for Writing a Literature Review. PLoS Comput Biol 9(7): e1003149. https://doi.org/10.1371/journal.pcbi.1003149

Editor: Philip E. Bourne, University of California San Diego, United States of America

Copyright: © 2013 Marco Pautasso. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Competing interests: The author has declared that no competing interests exist.

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

  • interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
  • an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
  • a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

  • keep track of the search items you use (so that your search can be replicated [10] ),
  • keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
  • use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
  • define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
  • do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

thumbnail

  • PPT PowerPoint slide
  • PNG larger image
  • TIFF original image

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

https://doi.org/10.1371/journal.pcbi.1003149.g001

  • discussing in your review the approaches, limitations, and conclusions of past reviews,
  • trying to find a new angle that has not been covered adequately in the previous reviews, and
  • incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

  • be thorough,
  • use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
  • look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

  • the major achievements in the reviewed field,
  • the main areas of debate, and
  • the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

  • 1. Rapple C (2011) The role of the critical review article in alleviating information overload. Annual Reviews White Paper. Available: http://www.annualreviews.org/userimages/ContentEditor/1300384004941/Annual_Reviews_WhitePaper_Web_2011.pdf . Accessed May 2013.
  • View Article
  • Google Scholar
  • 7. Budgen D, Brereton P (2006) Performing systematic literature reviews in software engineering. Proc 28th Int Conf Software Engineering, ACM New York, NY, USA, pp. 1051–1052. doi: https://doi.org/10.1145/1134285.1134500 .
  • 16. Eco U (1977) Come si fa una tesi di laurea. Milan: Bompiani.
  • 17. Hart C (1998) Doing a literature review: releasing the social science research imagination. London: SAGE.
  • 21. Ridley D (2008) The literature review: a step-by-step guide for students. London: SAGE.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Perspective
  • Published: 12 October 2020

Eight problems with literature reviews and how to fix them

  • Neal R. Haddaway   ORCID: orcid.org/0000-0003-3902-2234 1 , 2 , 3 ,
  • Alison Bethel 4 ,
  • Lynn V. Dicks 5 , 6 ,
  • Julia Koricheva   ORCID: orcid.org/0000-0002-9033-0171 7 ,
  • Biljana Macura   ORCID: orcid.org/0000-0002-4253-1390 2 ,
  • Gillian Petrokofsky 8 ,
  • Andrew S. Pullin 9 ,
  • Sini Savilaakso   ORCID: orcid.org/0000-0002-8514-8105 10 , 11 &
  • Gavin B. Stewart   ORCID: orcid.org/0000-0001-5684-1544 12  

Nature Ecology & Evolution volume  4 ,  pages 1582–1589 ( 2020 ) Cite this article

11k Accesses

80 Citations

388 Altmetric

Metrics details

  • Conservation biology
  • Environmental impact

An Author Correction to this article was published on 19 October 2020

This article has been updated

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a ‘critical friend’ role in supporting would-be systematic reviews and avoiding individual responses to police use of the ‘systematic review’ label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

in literature review articles authors should

Change history

19 october 2020.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Grant, M. J. & Booth, A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr. J. 26 , 91–108 (2009).

PubMed   Google Scholar  

Haddaway, N. R. & Macura, B. The role of reporting standards in producing robust literature reviews. Nat. Clim. Change 8 , 444–447 (2018).

Google Scholar  

Pullin, A. S. & Knight, T. M. Science informing policy–a health warning for the environment. Environ. Evid. 1 , 15 (2012).

Haddaway, N., Woodcock, P., Macura, B. & Collins, A. Making literature reviews more reliable through application of lessons from systematic reviews. Conserv. Biol. 29 , 1596–1605 (2015).

CAS   PubMed   Google Scholar  

Pullin, A., Frampton, G., Livoreil, B. & Petrokofsky, G. Guidelines and Standards for Evidence Synthesis in Environmental Management (Collaboration for Environmental Evidence, 2018).

White, H. The twenty-first century experimenting society: the four waves of the evidence revolution. Palgrave Commun. 5 , 47 (2019).

O’Leary, B. C. et al. The reliability of evidence review methodology in environmental science and conservation. Environ. Sci. Policy 64 , 75–82 (2016).

Woodcock, P., Pullin, A. S. & Kaiser, M. J. Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biol. Conserv. 176 , 54–62 (2014).

Campbell Systematic Reviews: Policies and Guidelines (Campbell Collaboration, 2014).

Higgins, J. P. et al. Cochrane Handbook for Systematic Reviews of Interventions (John Wiley & Sons, 2019).

Shea, B. J. et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358 , j4008 (2017).

PubMed   PubMed Central   Google Scholar  

Haddaway, N. R., Land, M. & Macura, B. “A little learning is a dangerous thing”: a call for better understanding of the term ‘systematic review’. Environ. Int. 99 , 356–360 (2017).

Freeman, R. E. Strategic Management: A Stakeholder Approach (Cambridge Univ. Press, 2010).

Haddaway, N. R. et al. A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environ. Evid. 6 , 11 (2017).

Land, M., Macura, B., Bernes, C. & Johansson, S. A five-step approach for stakeholder engagement in prioritisation and planning of environmental evidence syntheses. Environ. Evid. 6 , 25 (2017).

Oliver, S. & Dickson, K. Policy-relevant systematic reviews to strengthen health systems: models and mechanisms to support their production. Evid. Policy 12 , 235–259 (2016).

Savilaakso, S. et al. Systematic review of effects on biodiversity from oil palm production. Environ. Evid. 3 , 4 (2014).

Savilaakso, S., Laumonier, Y., Guariguata, M. R. & Nasi, R. Does production of oil palm, soybean, or jatropha change biodiversity and ecosystem functions in tropical forests. Environ. Evid. 2 , 17 (2013).

Haddaway, N. R. & Crowe, S. Experiences and lessons in stakeholder engagement in environmental evidence synthesis: a truly special series. Environ. Evid. 7 , 11 (2018).

Sánchez-Bayo, F. & Wyckhuys, K. A. Worldwide decline of the entomofauna: a review of its drivers. Biol. Conserv. 232 , 8–27 (2019).

Agarwala, M. & Ginsberg, J. R. Untangling outcomes of de jure and de facto community-based management of natural resources. Conserv. Biol. 31 , 1232–1246 (2017).

Gurevitch, J., Curtis, P. S. & Jones, M. H. Meta-analysis in ecology. Adv. Ecol. Res. 32 , 199–247 (2001).

CAS   Google Scholar  

Haddaway, N. R., Macura, B., Whaley, P. & Pullin, A. S. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid. 7 , 7 (2018).

Lwasa, S. et al. A meta-analysis of urban and peri-urban agriculture and forestry in mediating climate change. Curr. Opin. Environ. Sustain. 13 , 68–73 (2015).

Pacifici, M. et al. Species’ traits influenced their response to recent climate change. Nat. Clim. Change 7 , 205–208 (2017).

Owen-Smith, N. Ramifying effects of the risk of predation on African multi-predator, multi-prey large-mammal assemblages and the conservation implications. Biol. Conserv. 232 , 51–58 (2019).

Prugh, L. R. et al. Designing studies of predation risk for improved inference in carnivore-ungulate systems. Biol. Conserv. 232 , 194–207 (2019).

Li, Y. et al. Effects of biochar application in forest ecosystems on soil properties and greenhouse gas emissions: a review. J. Soil Sediment. 18 , 546–563 (2018).

Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. G., The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6 , e1000097 (2009).

Bernes, C. et al. What is the influence of a reduction of planktivorous and benthivorous fish on water quality in temperate eutrophic lakes? A systematic review. Environ. Evid. 4 , 7 (2015).

McDonagh, M., Peterson, K., Raina, P., Chang, S. & Shekelle, P. Avoiding bias in selecting studies. Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet] (Agency for Healthcare Research and Quality, 2013).

Burivalova, Z., Hua, F., Koh, L. P., Garcia, C. & Putz, F. A critical comparison of conventional, certified, and community management of tropical forests for timber in terms of environmental, economic, and social variables. Conserv. Lett. 10 , 4–14 (2017).

Min-Venditti, A. A., Moore, G. W. & Fleischman, F. What policies improve forest cover? A systematic review of research from Mesoamerica. Glob. Environ. Change 47 , 21–27 (2017).

Bramer, W. M., Giustini, D. & Kramer, B. M. R. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst. Rev. 5 , 39 (2016).

Bramer, W. M., Giustini, D., Kramer, B. M. R. & Anderson, P. F. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst. Rev. 2 , 115 (2013).

Gusenbauer, M. & Haddaway, N. R. Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 11 , 181–217 (2020).

Livoreil, B. et al. Systematic searching for environmental evidence using multiple tools and sources. Environ. Evid. 6 , 23 (2017).

Mlinarić, A., Horvat, M. & Šupak Smolčić, V. Dealing with the positive publication bias: why you should really publish your negative results. Biochem. Med. 27 , 447–452 (2017).

Lin, L. & Chu, H. Quantifying publication bias in meta‐analysis. Biometrics 74 , 785–794 (2018).

Haddaway, N. R. & Bayliss, H. R. Shades of grey: two forms of grey literature important for reviews in conservation. Biol. Conserv. 191 , 827–829 (2015).

Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36 , 1–48 (2010).

Bilotta, G. S., Milner, A. M. & Boyd, I. On the use of systematic reviews to inform environmental policies. Environ. Sci. Policy 42 , 67–77 (2014).

Englund, G., Sarnelle, O. & Cooper, S. D. The importance of data‐selection criteria: meta‐analyses of stream predation experiments. Ecology 80 , 1132–1141 (1999).

Burivalova, Z., Şekercioğlu, Ç. H. & Koh, L. P. Thresholds of logging intensity to maintain tropical forest biodiversity. Curr. Biol. 24 , 1893–1898 (2014).

Bicknell, J. E., Struebig, M. J., Edwards, D. P. & Davies, Z. G. Improved timber harvest techniques maintain biodiversity in tropical forests. Curr. Biol. 24 , R1119–R1120 (2014).

Damette, O. & Delacote, P. Unsustainable timber harvesting, deforestation and the role of certification. Ecol. Econ. 70 , 1211–1219 (2011).

Blomley, T. et al. Seeing the wood for the trees: an assessment of the impact of participatory forest management on forest condition in Tanzania. Oryx 42 , 380–391 (2008).

Haddaway, N. R. et al. How does tillage intensity affect soil organic carbon? A systematic review. Environ. Evid. 6 , 30 (2017).

Higgins, J. P. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343 , d5928 (2011).

Stewart, G. Meta-analysis in applied ecology. Biol. Lett. 6 , 78–81 (2010).

Koricheva, J. & Gurevitch, J. Uses and misuses of meta‐analysis in plant ecology. J. Ecol. 102 , 828–844 (2014).

Vetter, D., Ruecker, G. & Storch, I. Meta‐analysis: a need for well‐defined usage in ecology and conservation biology. Ecosphere 4 , 1–24 (2013).

Stewart, G. B. & Schmid, C. H. Lessons from meta-analysis in ecology and evolution: the need for trans-disciplinary evidence synthesis methodologies. Res. Synth. Methods 6 , 109–110 (2015).

Macura, B. et al. Systematic reviews of qualitative evidence for environmental policy and management: an overview of different methodological options. Environ. Evid. 8 , 24 (2019).

Koricheva, J. & Gurevitch, J. in Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J. et al.) Ch. 1 (Princeton Scholarship Online, 2013).

Britt, M., Haworth, S. E., Johnson, J. B., Martchenko, D. & Shafer, A. B. The importance of non-academic coauthors in bridging the conservation genetics gap. Biol. Conserv. 218 , 118–123 (2018).

Graham, L., Gaulton, R., Gerard, F. & Staley, J. T. The influence of hedgerow structural condition on wildlife habitat provision in farmed landscapes. Biol. Conserv. 220 , 122–131 (2018).

Delaquis, E., de Haan, S. & Wyckhuys, K. A. On-farm diversity offsets environmental pressures in tropical agro-ecosystems: a synthetic review for cassava-based systems. Agric. Ecosyst. Environ. 251 , 226–235 (2018).

Popay, J. et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: A Product from the ESRC Methods Programme Version 1 (Lancaster Univ., 2006).

Pullin, A. S. et al. Human well-being impacts of terrestrial protected areas. Environ. Evid. 2 , 19 (2013).

Waffenschmidt, S., Knelangen, M., Sieben, W., Bühn, S. & Pieper, D. Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med. Res. Methodol. 19 , 132 (2019).

Rallo, A. & García-Arberas, L. Differences in abiotic water conditions between fluvial reaches and crayfish fauna in some northern rivers of the Iberian Peninsula. Aquat. Living Resour. 15 , 119–128 (2002).

Glasziou, P. & Chalmers, I. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. BMJ 363 , k4645 (2018).

Haddaway, N. R. Open Synthesis: on the need for evidence synthesis to embrace Open Science. Environ. Evid. 7 , 26 (2018).

Download references

Acknowledgements

We thank C. Shortall from Rothamstead Research for useful discussions on the topic.

Author information

Authors and affiliations.

Mercator Research Institute on Climate Change and Global Commons, Berlin, Germany

Neal R. Haddaway

Stockholm Environment Institute, Stockholm, Sweden

Neal R. Haddaway & Biljana Macura

Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa

College of Medicine and Health, Exeter University, Exeter, UK

Alison Bethel

Department of Zoology, University of Cambridge, Cambridge, UK

Lynn V. Dicks

School of Biological Sciences, University of East Anglia, Norwich, UK

Department of Biological Sciences, Royal Holloway University of London, Egham, UK

Julia Koricheva

Department of Zoology, University of Oxford, Oxford, UK

Gillian Petrokofsky

Collaboration for Environmental Evidence, UK Centre, School of Natural Sciences, Bangor University, Bangor, UK

  • Andrew S. Pullin

Liljus ltd, London, UK

Sini Savilaakso

Department of Forest Sciences, University of Helsinki, Helsinki, Finland

Evidence Synthesis Lab, School of Natural and Environmental Sciences, University of Newcastle, Newcastle-upon-Tyne, UK

Gavin B. Stewart

You can also search for this author in PubMed   Google Scholar

Contributions

N.R.H. developed the manuscript idea and a first draft. All authors contributed to examples and edited the text. All authors have read and approve of the final submission.

Corresponding author

Correspondence to Neal R. Haddaway .

Ethics declarations

Competing interests.

S.S. is a co-founder of Liljus ltd, a firm that provides research services in sustainable finance as well as forest conservation and management. The other authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary table.

Examples of literature reviews and common problems identified.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Haddaway, N.R., Bethel, A., Dicks, L.V. et al. Eight problems with literature reviews and how to fix them. Nat Ecol Evol 4 , 1582–1589 (2020). https://doi.org/10.1038/s41559-020-01295-x

Download citation

Received : 24 March 2020

Accepted : 31 July 2020

Published : 12 October 2020

Issue Date : December 2020

DOI : https://doi.org/10.1038/s41559-020-01295-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A review of the necessity of a multi-layer land-use planning.

  • Hashem Dadashpoor
  • Leyla Ghasempour

Landscape and Ecological Engineering (2024)

Parasitism in viviparous vertebrates: an overview

  • Juan J. Palacios-Marquez
  • Palestina Guevara-Fiore

Parasitology Research (2024)

Environmental evidence in action: on the science and practice of evidence synthesis and evidence-based decision-making

  • Steven J. Cooke
  • Carly N. Cook

Environmental Evidence (2023)

Greenhouse gas emissions and mitigation in rice agriculture

  • Xiangchen Zhu

Nature Reviews Earth & Environment (2023)

Broadening scientific engagement and inclusivity in IPCC reports through collaborative technology platforms

  • Anthony Jude De-Gol
  • Corinne Le Quéré
  • Marianne Aubin Le Quéré

npj Climate Action (2023)

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

in literature review articles authors should

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • Current issue
  • Write for Us
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Volume 24, Issue 2
  • Five tips for developing useful literature summary tables for writing review articles
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0003-0157-5319 Ahtisham Younas 1 , 2 ,
  • http://orcid.org/0000-0002-7839-8130 Parveen Ali 3 , 4
  • 1 Memorial University of Newfoundland , St John's , Newfoundland , Canada
  • 2 Swat College of Nursing , Pakistan
  • 3 School of Nursing and Midwifery , University of Sheffield , Sheffield , South Yorkshire , UK
  • 4 Sheffield University Interpersonal Violence Research Group , Sheffield University , Sheffield , UK
  • Correspondence to Ahtisham Younas, Memorial University of Newfoundland, St John's, NL A1C 5C4, Canada; ay6133{at}mun.ca

https://doi.org/10.1136/ebnurs-2021-103417

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Literature reviews offer a critical synthesis of empirical and theoretical literature to assess the strength of evidence, develop guidelines for practice and policymaking, and identify areas for future research. 1 It is often essential and usually the first task in any research endeavour, particularly in masters or doctoral level education. For effective data extraction and rigorous synthesis in reviews, the use of literature summary tables is of utmost importance. A literature summary table provides a synopsis of an included article. It succinctly presents its purpose, methods, findings and other relevant information pertinent to the review. The aim of developing these literature summary tables is to provide the reader with the information at one glance. Since there are multiple types of reviews (eg, systematic, integrative, scoping, critical and mixed methods) with distinct purposes and techniques, 2 there could be various approaches for developing literature summary tables making it a complex task specialty for the novice researchers or reviewers. Here, we offer five tips for authors of the review articles, relevant to all types of reviews, for creating useful and relevant literature summary tables. We also provide examples from our published reviews to illustrate how useful literature summary tables can be developed and what sort of information should be provided.

Tip 1: provide detailed information about frameworks and methods

  • Download figure
  • Open in new tab
  • Download powerpoint

Tabular literature summaries from a scoping review. Source: Rasheed et al . 3

The provision of information about conceptual and theoretical frameworks and methods is useful for several reasons. First, in quantitative (reviews synthesising the results of quantitative studies) and mixed reviews (reviews synthesising the results of both qualitative and quantitative studies to address a mixed review question), it allows the readers to assess the congruence of the core findings and methods with the adapted framework and tested assumptions. In qualitative reviews (reviews synthesising results of qualitative studies), this information is beneficial for readers to recognise the underlying philosophical and paradigmatic stance of the authors of the included articles. For example, imagine the authors of an article, included in a review, used phenomenological inquiry for their research. In that case, the review authors and the readers of the review need to know what kind of (transcendental or hermeneutic) philosophical stance guided the inquiry. Review authors should, therefore, include the philosophical stance in their literature summary for the particular article. Second, information about frameworks and methods enables review authors and readers to judge the quality of the research, which allows for discerning the strengths and limitations of the article. For example, if authors of an included article intended to develop a new scale and test its psychometric properties. To achieve this aim, they used a convenience sample of 150 participants and performed exploratory (EFA) and confirmatory factor analysis (CFA) on the same sample. Such an approach would indicate a flawed methodology because EFA and CFA should not be conducted on the same sample. The review authors must include this information in their summary table. Omitting this information from a summary could lead to the inclusion of a flawed article in the review, thereby jeopardising the review’s rigour.

Tip 2: include strengths and limitations for each article

Critical appraisal of individual articles included in a review is crucial for increasing the rigour of the review. Despite using various templates for critical appraisal, authors often do not provide detailed information about each reviewed article’s strengths and limitations. Merely noting the quality score based on standardised critical appraisal templates is not adequate because the readers should be able to identify the reasons for assigning a weak or moderate rating. Many recent critical appraisal checklists (eg, Mixed Methods Appraisal Tool) discourage review authors from assigning a quality score and recommend noting the main strengths and limitations of included studies. It is also vital that methodological and conceptual limitations and strengths of the articles included in the review are provided because not all review articles include empirical research papers. Rather some review synthesises the theoretical aspects of articles. Providing information about conceptual limitations is also important for readers to judge the quality of foundations of the research. For example, if you included a mixed-methods study in the review, reporting the methodological and conceptual limitations about ‘integration’ is critical for evaluating the study’s strength. Suppose the authors only collected qualitative and quantitative data and did not state the intent and timing of integration. In that case, the strength of the study is weak. Integration only occurred at the levels of data collection. However, integration may not have occurred at the analysis, interpretation and reporting levels.

Tip 3: write conceptual contribution of each reviewed article

While reading and evaluating review papers, we have observed that many review authors only provide core results of the article included in a review and do not explain the conceptual contribution offered by the included article. We refer to conceptual contribution as a description of how the article’s key results contribute towards the development of potential codes, themes or subthemes, or emerging patterns that are reported as the review findings. For example, the authors of a review article noted that one of the research articles included in their review demonstrated the usefulness of case studies and reflective logs as strategies for fostering compassion in nursing students. The conceptual contribution of this research article could be that experiential learning is one way to teach compassion to nursing students, as supported by case studies and reflective logs. This conceptual contribution of the article should be mentioned in the literature summary table. Delineating each reviewed article’s conceptual contribution is particularly beneficial in qualitative reviews, mixed-methods reviews, and critical reviews that often focus on developing models and describing or explaining various phenomena. Figure 2 offers an example of a literature summary table. 4

Tabular literature summaries from a critical review. Source: Younas and Maddigan. 4

Tip 4: compose potential themes from each article during summary writing

While developing literature summary tables, many authors use themes or subthemes reported in the given articles as the key results of their own review. Such an approach prevents the review authors from understanding the article’s conceptual contribution, developing rigorous synthesis and drawing reasonable interpretations of results from an individual article. Ultimately, it affects the generation of novel review findings. For example, one of the articles about women’s healthcare-seeking behaviours in developing countries reported a theme ‘social-cultural determinants of health as precursors of delays’. Instead of using this theme as one of the review findings, the reviewers should read and interpret beyond the given description in an article, compare and contrast themes, findings from one article with findings and themes from another article to find similarities and differences and to understand and explain bigger picture for their readers. Therefore, while developing literature summary tables, think twice before using the predeveloped themes. Including your themes in the summary tables (see figure 1 ) demonstrates to the readers that a robust method of data extraction and synthesis has been followed.

Tip 5: create your personalised template for literature summaries

Often templates are available for data extraction and development of literature summary tables. The available templates may be in the form of a table, chart or a structured framework that extracts some essential information about every article. The commonly used information may include authors, purpose, methods, key results and quality scores. While extracting all relevant information is important, such templates should be tailored to meet the needs of the individuals’ review. For example, for a review about the effectiveness of healthcare interventions, a literature summary table must include information about the intervention, its type, content timing, duration, setting, effectiveness, negative consequences, and receivers and implementers’ experiences of its usage. Similarly, literature summary tables for articles included in a meta-synthesis must include information about the participants’ characteristics, research context and conceptual contribution of each reviewed article so as to help the reader make an informed decision about the usefulness or lack of usefulness of the individual article in the review and the whole review.

In conclusion, narrative or systematic reviews are almost always conducted as a part of any educational project (thesis or dissertation) or academic or clinical research. Literature reviews are the foundation of research on a given topic. Robust and high-quality reviews play an instrumental role in guiding research, practice and policymaking. However, the quality of reviews is also contingent on rigorous data extraction and synthesis, which require developing literature summaries. We have outlined five tips that could enhance the quality of the data extraction and synthesis process by developing useful literature summaries.

  • Aromataris E ,
  • Rasheed SP ,

Twitter @Ahtisham04, @parveenazamali

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

  • Open access
  • Published: 21 February 2024

Mapping study for health emergency and disaster risk management competencies and curricula: literature review and cross-sectional survey

  • Kevin K. C. Hung 1 , 2 ,
  • Makiko K. MacDermot 1 ,
  • Theresa S. I. Hui 1 ,
  • Suet Yi Chan 1 ,
  • Sonoe Mashino 3 ,
  • Catherine P. Y. Mok 1 ,
  • Pak Ho Leung 1 ,
  • Ryoma Kayano 4 ,
  • Jonathan Abrahams 5 ,
  • Chi Shing Wong 2 ,
  • Emily Y. Y. Chan 1 , 2 &
  • Colin A. Graham   ORCID: orcid.org/0000-0002-4381-7470 1 , 2  

Globalization and Health volume  20 , Article number:  15 ( 2024 ) Cite this article

Metrics details

With the increasing threat of hazardous events at local, national, and global levels, an effective workforce for health emergency and disaster risk management (Health EDRM) in local, national, and international communities is urgently needed. However, there are no universally accepted competencies and curricula for Health EDRM. This study aimed to identify Health EDRM competencies and curricula worldwide using literature reviews and a cross-sectional survey.

Literature reviews in English and Japanese languages were performed. We searched MEDLINE, EMBASE, CINAHL (English), and the ICHUSHI (Japanese) databases for journal articles published between 1990 and 2020. Subsequently, a cross-sectional survey was sent to WHO Health EDRM Research Network members and other recommended experts in October 2021 to identify competency models and curricula not specified in the literature search.

Nineteen studies from the searches were found to be relevant to Health EDRM competencies and curricula. Most of the competency models and curricula were from the US. The domains included knowledge and skills, emergency response systems (including incident management principles), communications, critical thinking, ethical and legal aspects, and managerial and leadership skills. The cross-sectional survey received 65 responses with an estimated response rate of 25%. Twenty-one competency models and 20 curricula for managers and frontline personnel were analyzed; managers' decision-making and leadership skills were considered essential.

An increased focus on decision-making and leadership skills should be included in Health EDRM competencies and curricula to strengthen the health workforce.

Introduction

Disasters have been defined by the World Health Organization (WHO) as “serious disruptions of the functioning of a community or a society at any scale due to hazardous events interacting with conditions of exposure, vulnerability and capacity, leading to one or more of the following: human, material, economic and environmental losses and impacts” [ 1 ]. The consequences of disasters are often devastating, leading to high mortality and imposing extreme burdens on health systems and national economies [ 2 , 3 , 4 , 5 , 6 , 7 ]. During the 2010s, an average of approximately 45,000 people died annually from disasters associated with natural hazards globally, around 0.1% of total deaths [ 2 ].

The latest coronavirus disease (COVID-19) pandemic demonstrated how biological hazards could overwhelm health systems in both developing and developed countries. As of March 2023, over 758 million people had been infected globally, resulting in more than 6.8 million deaths [ 4 ].

With the increasing threat of hazardous events at local, national, and global levels, an effective workforce for health emergency and disaster risk management (Health EDRM) in local, national, and international communities is urgently needed [ 8 ]. Health EDRM emphasizes assessing, communicating, and reducing risks across all phases of the disaster cycle and building the resilience of communities, countries, and health systems. Health EDRM requires collective action by health systems, communities, and partners across society to reduce health risks and the consequences of all types of emergencies and disasters [ 8 ].

Enhancing health workforce capacity and competencies has long been an international goal in disaster risk reduction. According to the 2015 Sendai Framework for Disaster Risk Reduction, strengthening the training capacities in disaster medicine and training the healthcare workforce in disaster risk reduction are important ways to achieve effective disaster risk reduction and enhance resilience [ 9 ]. The WHO recognizes that a skilled and trained health workforce is imperative for countries to effectively implement disaster risk management, including emergency preparedness measures [ 10 ]. In the WHO Global Strategy on Human Resources for Health 2030, technical support to health system capacities and workforce competency is considered one of the Secretariat's core activities [ 10 ].

Studies have shown that receiving training and having relevant knowledge and skills for roles in emergencies and disasters are significantly related to increased willingness to work in an emergency and higher confidence in performing duties or functions in disaster situations [ 11 , 12 ]. The global competency framework for universal health coverage has also included “developing preparedness for health emergencies and disasters, including disease outbreaks” and “responding to health emergencies and disasters, including disease outbreaks” in the list of core functions of health practice, which require relevant training in Health EDRM core competencies [ 13 ].

While the importance of disaster health education and training is well recognized, there is a need for an evidence-based approach to the training contents and their delivery for the wide range of roles in the health workforce. Defining the core competencies in knowledge and skills in Health EDRM for this health workforce is challenging. Previous literature reviews failed to identify an agreed set of competencies for disaster healthcare providers [ 14 , 15 , 16 , 17 , 18 ]. There were also wide variations in training delivery modalities and evaluation methods. However, for chemical, biological, radiological, and nuclear (CBRN) incident response, scenario-based training appeared to be more effective than other types of training [ 18 , 19 ]. Furthermore, there has been insufficient research on the long-term impact of emergency preparedness training and exercises on individuals and their effectiveness in actual events [ 18 , 20 ]. As shown in a review of training for the WHO Ebola emergency response, most published articles focused on competencies for a single profession instead of adopting a multidisciplinary approach [ 21 ]. Building standardized and accredited core competencies for all healthcare workers in Health EDRM is necessary to ensure the delivery of safe and quality care in disasters.

As a first step in the health workforce Health EDRM capacity building effort, the WHO Health Emergencies Programme (WHE) Learning Strategy was developed in 2018 to provide standards and frameworks in training. This strategy is intended for all WHO Health Emergency personnel, WHO partners, and volunteers at national, regional, and international levels [ 22 ]. It introduced the WHE Competency Framework based on the Competency = Attributes + Skills + Knowledge (CASK) model with attributes, skills, and knowledge as key components of competencies. The WHE Training Framework was also established to train WHO staff at different levels through online courses, face-to-face training, and simulation exercises [ 22 ]. However, the applicability of the competency and training frameworks is still being piloted at the time of writing. Further evidence for the essential elements of WHE competencies and their ideal training modalities will be crucial before their wider dissemination.

The WHO Health EDRM Research Network (Health EDRM RN) has previously reported the need to identify knowledge and evidence gaps in the capacity development of the health workforce with respect to Health EDRM [ 23 , 24 ].

The present study aimed to address the evidence gap by identifying the competencies and curricula currently used by disaster risk management-related agencies. It alsoexplored the knowledge gap of the match between these existing competencies and curricula on the one hand, and the WHE core Competency and Training Frameworks and principles of Health EDRM on the other hand. Our research question was therefore: do the current competency models and curricula match the WHE core competency domains and the comprehensive emergency management and risk-based approach of Health EDRM?

Materials and methods

This competency mapping study was part of a research project funded by the WHO Centre for Health Development (WHO Kobe Centre - WKC) under the Health Workforce Development for Health EDRM (Research Area 4 of WKC’s four key research themes). This competency mapping study included a literature review and a cross-sectional survey, the latter of which was included because some competency models and curricula may have been missed in the literature review. The literature review was performed in the English and Japanese languages using a systematic approach to the literature search. Searches of the English language literature published from 1 January 1990 to 11 March 2020 were conducted using MEDLINE (1966), EMBASE (1980), and CINAHL (1980). The Japanese language literature search was performed using the ICHUSHI database and included articles published from 1 January 1990 to 23 October 2020. The English and Japanese search terms can be found in the Supplementary file 1 .

For both languages, we included only papers matching the definition of disaster or humanitarian crisis and those relevant to the competencies and curricula of Health EDRM. We excluded papers based purely on military settings or on training for a single type of clinical procedure or surgery, papers focusing on research methodologies, as well as conference abstracts and papers where full-text versions were not available at the university library systems in Hong Kong or Japan.

Title and abstract screening were performed by a single reviewer for the 7,396 English-language and the 2,690 Japanese-language articles. A total of 359 English-language and 743 Japanese-language articles were selected and underwent full-text screening by two reviewers. If disagreement between the two reviewers about the suitability of an article could not be resolved after discussion, a third reviewer was invited to review the article independently. After this procedure, two more competency models were identified from a subsequent manual search. Finally, seventeen eligible articles in English language and two articles in Japanese language were included (Fig.  1 ).

figure 1

PRISMA flow chart

The cross-sectional study was conducted in October 2021 using an online platform (Survey Monkey). The survey was distributed to over 230 members of the WHO Health EDRM RN and other experts recommended by the WHO collaboration group on health workforce development [ 24 ]. The eligibility criteria included respondents aged 18 or above working in Health EDRM professional development, education, and training programs. The questionnaire (Supplementary file 2 ) comprised of 28 questions, including agency information, the identification of relevant management and technical competencies, curricula, and evidence gaps. The management and technical competencies included were based on the relevant components in the WHO Health EDRM Framework and reviewed by experts in the investigator team. Ethical approval was obtained from The Chinese University of Hong Kong Survey and Behavioural Research Ethics Committee.

The competency models and curricula identified in the literature review and the cross-sectional survey were compared, and all the competency models and curricula were analyzed. Gap analyses compared the competency models and curricula identified in the literature review and the survey against the Competency and Training Frameworks and principles of Health EDRM.

Summary of literature review and cross-sectional survey

Literature review.

A summary table of the competencies and curricula identified in the literature search is provided in Supplementary file 3 . There were 19 articles describing 15 competency models and curricula for health workers and professionals [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. On six occasions, curricula alongside a competency model were listed [ 25 , 26 , 27 , 28 , 29 , 33 , 34 , 35 , 41 , 43 ], on another six occasions, competency models alone were described [ 32 , 36 , 38 , 39 , 40 , 42 ]. In the remaining three, only curricula were described [ 30 , 31 , 37 ].

Cross-sectional survey

Sixty-five responses were received in the online survey. The response rate was estimated to be 25%, based on the number of invitations sent by email. However, the exact response rate is not known as recipients of the invitation email were encouraged to forward the invitation to other relevant potential respondents. The survey did not generate any additional competency model or curriculum. Table 1 shows the summary profile of the responding agencies. Most respondents were from academic institutions (60%), followed by those from national governments (19%).

Twenty-one respondents (32%) reported that their agencies had defined the core competencies for Health EDRM managers and frontline responders while 20 respondents (31%) listed the contents in the curricula provided for the managers and frontline responders.

Competency models from literature review and survey

Among the 12 competency models identified in the literature review, the majority were from the US. Most papers included expert panel reviews while some used the Delphi method for building consensus. Most models were all-hazard in coverage although a few had been developed specifically to counter bioterrorism. Most papers focused on public health workers, but some also targeted public health managers and professionals while others focused on health professionals of various disciplines. Most competency models included knowledge and skills, emergency response systems (including incident management principles), communications, critical thinking, ethical and legal aspects, and managerial and leadership skills. Some competency models provided specific competencies for different tiers of frontline and managerial personnel [ 25 , 26 , 27 , 29 , 36 ].

Table 2 shows the detailed responses from the survey. Management skills were most commonly described, including planning, organizing, applying management processes, establishing effective communication systems, and providing effective leadership (95-100% of respondents). Fewer respondents reported the requirement of management skills for frontline workers, except for effective communication systems (90%).

Managers were expected to master a wide range of technical competencies. However, seven items were less frequently included as required competencies for frontline workers (<60% of respondents): 1. human resource management, 2. managing emergency operations centres, 3. managing monitoring and evaluation systems, 4. surge capacity planning, 5. program management, 6. development of Health EDRM policies, strategies and legislation, and 7. financial resources – planning and managing budgets.

Curricula from the literature review and survey

The majority of the nine identified curricula were from the US. Six articles described a proposed curriculum to address a specific competency model. Structured short training programs like the Core/Basic/Advanced Disaster Life Support and the National Disaster Life Support Decontamination courses [ 37 ] were geared towards delivering specific disaster preparedness and response knowledge and skills in 8-16 hours. More extended curricula, including those by CDC/ Columbia University School of Nursing [ 28 , 30 , 31 , 33 ], were self-paced and included 15 activities to cover various competencies. Some curricula included exercises and simulation-based training [ 37 , 41 , 43 ].

Three of the six curricula described how the candidates were assessed against defined competencies (Supplementary file 3 g). There were no standardized assessment methods: a wide variety of assessments were used, including pre- and post-test scores, self-assessment or trainer-rated performance, exercises (including simulation-based ones) performance results, and observed field-based performance.

Most of the curricula cover the required management and technical competencies across prevention, preparedness, response, and recovery for managers and frontline workers. However, “understanding community capacities, leadership and involvement” and “cultural competencies” were less often covered in curricula (75%). Training delivered by institutions included practical skills training and tabletop/full-scale exercises (85%), followed by blended learning (70%), didactic teaching (65%), online training (65%) and work-based mentorship (55%). Program duration was usually less than one week (40%) or longer than one year (35%). Only 20% were 1-4 weeks and 5% were 1-6 months. Among training modalities that required recertification (55% of the responses), most required recertification every 1-2 years (45%).

Gaps in competencies and curricula

Gaps in competencies and curricula were identified by comparing the above findings with WHE core competencies and Health EDRM principles. WHE core competencies include six areas, namely 1. Moving forward in a changing environment, 2. Applying technical expertise, 3. Communication, 4. Teamwork, 5. Building and promoting partnerships, and 6. Leadership. By comparison, none of the 15 competency models and curricula identified in the literature review and survey included the area of “moving forward in a changing environment,” and only one curriculum covered “teamwork” (Fig.  2 , Supplementary file 3 d). “Leadership” was only included in three of the 15 competency models and curricula. Most competency models encompassed technical competencies on disaster preparedness and response, but fewer included technical competencies on recovery.

figure 2

Percentage of published competency models and curricula covering the WHE core competencies and Health EDRM principles

The Health EDRM principle of a comprehensive emergency management perspective (across prevention, preparedness, readiness, response, and recovery) was only included in two models/curricula. The Health EDRM risk-based approach that emphasizes reducing hazards, exposures, and vulnerabilities was only included in three models or curricula (Fig.  2 ).

The competency models and curricula cited by survey respondents included a higher coverage of the WHE core competencies and the Health EDRM principles than those identified from the literature review (Supplementary 3 e and f). However, gaps were still seen in the risk-based approach and technical competencies in emergency recovery and leadership.

Research priorities for developing Health EDRM competencies (from survey respondents)

Thirty-one survey respondents (48%) provided their views on research priorities for developing Health EDRM competencies in their countries. Effective leadership and planning for Health EDRM were ranked as the most important research priorities for managerial and frontline personnel (Table 3 ).

The literature review in this study identified studies on competency models and curricula for managers and frontline personnel managing the risks of disasters and emergencies. It was observed that most identified competencies were related to technical aspects of disaster preparedness and response, with fewer focused on recovery. In the WHO Health EDRM Framework, the importance of adopting a comprehensive approach to emergency management was highlighted, covering all aspects from prevention to preparedness, readiness, response, and recovery.

This study found a gap here in most of the current competency models and curricula. This study also discovered no standardized method for assessing competencies while various techniques such as pre- and post-test scores, self-assessment, trainer-rated performance, exercise-based performance results, and field-based performance were used instead.

The cross-sectional survey revealed that Health EDRM agencies increasingly recognize the importance of planning, organizing, and applying risk management systems and programs across prevention, preparedness, response, and recovery. It should be highlighted that different levels of planning (strategic vs operational) might be required for managers and frontline personnel. Similarly, the importance of disaster response planning has been reflected in many competency models and curricula [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 35 , 36 , 37 , 39 , 40 , 41 , 42 , 43 ]. Besides, Olu et al. also included risk assessments, capacity assessments, and population needs assessments in their competency model, as well as risk reduction implementation and post-emergency health systems recovery [ 41 ]. Our survey also confirmed that effective planning across prevention, preparedness, response, and recovery remains an essential priority for research, both for managers and frontline personnel.

Leadership competencies and training constitute another crucial area as reflected in the survey results, as well as being a priority for further research. While the competency of effective organizational management, such as emergency or incident management systems, was well recognized across the literature [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], leadership competency goes beyond organizational structures. Olson et al. highlighted leadership and systems thinking in their competency model [ 35 ] while Olu et al. included effective leadership, teamwork, and the management skills required for disaster risk management in their model [ 41 ].

Decision-making and leadership theories provide the framework supporting the need for leadership competencies [ 44 , 45 ]. Systems thinking, such as situational awareness, is also a crucial Health EDRM competency for individuals to establish a mental picture of disaster situations [ 45 ].

In addition, it is essential to consider the appropriateness of specific competencies for specific organizations. Not all organizations require all of the competencies that have been identified. The WHE Learning Strategy published in 2018 specifically separated functional competencies from core competencies [ 22 ]. The WHE core competencies include skills, knowledge, and attributes performance standards across all health emergency workforce, while the functional competencies depend on the specific role of the emergency personnel. The requirement for all WHE personnel to fulfill the behavioral indicators for these core competencies was probably a key driver for this distinction since unlike the universally required Health EDRM core competencies, the functional competencies required will differ between members of the health workforce in different technical roles (depending on the disaster-related or humanitarian work undertaken).

Including attributes, in addition to knowledge and skills, in the WHE core competency framework was a fundamental change. This was highlighted in our study as a lack of coverage of attributes in the models and curricula in the published literature. For example, “moving forward in a changing environment” included indicators such as open-mindedness, a learning mindset, flexibility, and adaptability.

While some of these attributes, such as flexibility and adaptability, have appeared in various current competency models, the emphasis on the inclusion of these new competency standards in staff recruitment, training and assessment, as well as appraisal and performance management in the WHE core competency framework, may help to establish a high-quality workforce for dealing with modern emergencies.

Limitations

There were several limitations to the literature review and the cross-sectional survey. First of all, since the literature review was conducted before the COVID-19 pandemic, so it cannot reflect the changes during and after the pandemic. Second, although a systematic search strategy was employed to identify articles for this literature review, the methodology of independent screening and systematic data retrieval used in systematic reviews was not adopted in this study. Third, the medical databases selected in this study for literature review may have missed studies in development-related fields and those not published in the academic literature. Therefore, the articles included in this review and the subsequent retrieval of information may be subject to bias. Fourth, many curricula identified in this study did not specify their length of training and thus the effect of training duration on the Health EDRM competencies included in the curricula could not be investigated.

The limitations of the cross-sectional survey included the small sample size and the issue of selection bias due to the way participants were recruited. Concerning the competencies and the curricula cited in the survey, differences could arise due to the specific nature and mandate of the agencies involved in disaster management, whether they be local or international. Furthermore, most respondents did not provide the competencies or curricula in writing and thus the information may be subject to respondent bias. In addition, due to the lack of detail on the competencies and curricula provided, we cannot be sure that the competency models and curricula described in the survey do not overlap with those found in the literature review. Regarding the priorities for research, the respondents’ personal background and experience could have affected the responses.

Conclusions

Competency mapping in this literature review and cross-sectional survey identified that Health EDRM managers are expected to master many managerial and technical skills while decision-making and leadership skills were increasingly recognized as essential. Future competency models and curricula should separate the general core and the role-specific functional competencies, as well as covering comprehensive core competency domains such as those highlighted by the WHE. A comprehensive risk management framework, such as the WHO Health EDRM Framework, should be adopted in designing both general core and role-specific functional competencies using a Health EDRM risk-based approach.

Availability of data and materials

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Competency = Attributes + Skills + Knowledge model

Chemical, biological, radiological and nuclear

Centers for Disease Control and Prevention, United States

  • Health emergency and disaster risk management

WHO Health EDRM Research Network

United States

WHO Health Emergencies Programme

World Health Organization

World Health Organization. GLOSSARY of Health Emergency and Disaster Risk Management terminology. https://www.who.int/publications/i/item/9789240003699 . Accessed 27 Oct 2023.

Ritchie H, Rosado P, Roser M. Natural disasters. https://ourworldindata.org/natural-disasters . Accessed March 2023.

Hung KK, Lam EC, Chan EY, Graham CA. Disease pattern and chronic illness in rural China: The Hong Kong Red Cross basic health clinic after 2008 Sichuan earthquake. Emerg Med Australas. 2013;25(3):252–9.

Article   PubMed   Google Scholar  

World Health Organization. WHO Coronavirus (COVID-19) Dashboard. https://covid19.who.int/ . Accessed 7 March 2023.

Chan EY, Kim JH, Lo ES, Huang Z, Hung H, Hung KK, Wong EL, Lee EK, Wong MC, Wong SY. What happened to people with non-communicable diseases during COVID-19: implications of H-EDRM policies. Int J Environ Res Public Health. 2020;17(15):5588.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Hung KK, Walline JH, Graham CA. COVID-19: emergency medicine perspectives from Hong Kong. Eur J Emerg Med. 2020;27(3):163-164.

Hung KK, Walline JH, Chan EY. Health service utilization in Hong Kong during the COVID-19 pandemic–a cross-sectional public survey. Int J Health Policy Manag. 2022;11(4):508–13. https://doi.org/10.34172/ijhpm.2020.183 .

World Health Organization. Global competency and outcomes framework for universal health coverage. World Health Organization; 2022. https://www.who.int/publications/i/item/9789240034662 . Accessed 14 March 2023.

World Health Organization. Health Emergency and Disaster Risk Management framework. Geneva: World Health Organization; 2019. https://www.who.int/hac/techguidance/preparedness/health-emergency-and-disaster-risk-management-framework-eng.pdf . Accessed 7 March 2023.

The United Nations Office for Disaster Risk Reduction. Sendai Framework for Disaster Risk Reduction 2015–2030. Geneva: United Nations; 2015.

Google Scholar  

World Health Organization. Global strategy on human resources for health: workforce 2030. Geneva: World Health Organization; 2016.

Errett NA, Egan S, Garrity S, Rutkow L, Walsh L, Thompson CB, Strauss-Riggs K, Altman B, Schor K, Barnett DJ. Attitudinal determinants of local public health workers’ participation in Hurricane Sandy recovery activities. Health Secur. 2015;13(4):267–73.

Harrison KL, Errett NA, Lainie Rutkow JD, Thompson CB, Anderson MK, Ferrell JL, Freiheit JM, Hudson R, Koch MM, McKee M, Mejia-Echeverry A. An intervention for enhancing public health crisis response willingness among local health department workers: a qualitative programmatic analysis. Am J Disaster Med. 2014;9(2):87–96.

Elaine Daily R, Padjen P. A review of competencies developed for disaster healthcare providers: limitations of current processes and applicability. Prehosp Disaster Med. 2010;25(5):387–95.

Gallardo AR, Djalali A, Foletti M, Ragazzoni L, Della Corte F, Lupescu O, Arculeo C, Von Arnim G, Friedl T, Ashkenazi M, Fisher P. Core competencies in disaster management and humanitarian assistance: a systematic review. Disaster Med Public Health Prepared. 2015;9(4):430–9.

Article   Google Scholar  

Said NB, Chiang VC. The knowledge, skill competencies, and psychological preparedness of nurses for disasters: a systematic review. Int Emerg Nurs. 2020;1(48):100806.

Hansoti B, Kellogg DS, Aberle SJ, Broccoli MC, Feden J, French A, Little CM, Moore B, Sabato J, Sheets T, Weinberg R. Preparing emergency physicians for acute disaster response: a review of current training opportunities in the US. Prehosp Disaster Med. 2016;31(6):643–7.

Kako M, Hammad K, Mitani S, Arbon P. Existing approaches to chemical, biological, radiological, and nuclear (CBRN) education and training for health professionals: findings from an integrative literature review. Prehosp Disaster Med. 2018;33(2):182.

Catlett C, Perl T, Jenckes MW, Robinson KA, Mitchell D, Hage J, Feuerstein CJ, Chuang S, Bass EB. Training of clinicians for public health events relevant to bioterrorism preparedness. Evid Rep/Technol Assess (Summary). 2001;1(51):1–4.

Skryabina E, Reedy G, Amlot R, Jaye P, Riley P. What is the value of health emergency preparedness exercises? A scoping review study. Int J Disaster Risk Reduct. 2017;1(21):274–83.

Mondiale de la Santé O, World Health Organization. A review of the role of training in WHO Ebola emergency response. Weekly Epidemiol Record. 2016;91(14):181-6

World Health Organization. WHE learning strategy: WHO Health Emergencies Programme learning strategy. World Health Organization; 2018. https://www.who.int/publications/m/item/whe-learning-strategy . Accessed 25 Jan 2024.

Kayano R, Chan EY, Murray V, Abrahams J, Barber SL. WHO Thematic Platform for Health Emergency and Disaster Risk Management Research Network (TPRN): Report of the Kobe Expert Meeting. Int J Environ Res Public Health. 2019;16:1232.

Article   PubMed   PubMed Central   Google Scholar  

Hung KK, Mashino S, Chan EY, MacDermot MK, Balsari S, Ciottone GR, Della Corte F, Dell’Aringa MF, Egawa S, Evio BD, Hart A. Health workforce development in Health Emergency and Disaster Risk Management: the need for evidence-based recommendations. Int J Environ Res Public Health. 2021;18(7):3382.

Gebbie K, Merrill J. Public health worker competencies for emergency response. J Public Health Manag Pract. 2002;8(3):73–81.

Gebbie K, Merril J, Hwang I, et al. Identifying individual competency in emerging areas of practice: an applied approach. Qaul Health Res. 2002;12:990–9.

Centers for Disease Control and Prevention. Bioterrorism and emergency readiness: competencies for all public health workers. 2002.

Barnett DJ, Everly GS, Parker CL, Links JM. Applying educational gaming to public health workforce emergency preparedness. Am J Prevent Med. 2005;28:390–5.

Parker CL, Barnett DJ, Fews AL, Blodgett D, Links JM. The road map to preparedness: a competency-based approach to all-hazards emergency readiness training for the public health workforce. Public Health Rep. 2005;120:504–14.

Rottman SJ, Shoaf KI, Dorian A. Development of a training curriculum for public health preparedness. J Public Health Manag Pract. 2005;11(6):S128-31.

Calhoun JG, Rowney R, Eng E, Hoffman Y. Competency mapping and analysis for public health preparedness training initiatives. Public Health Reports. 2005;120(suppl 1):91–9.

Hsu EB, Thomas TL, Bass EB, et al. Healthcare worker competencies for disaster training. BMC Med Educ. 2006;6:19.

Hites LS, Lafreniere AV, Wingate MS, et al. Expanding the public health emergency preparedness competency set to meet specialized local and evolving national needs: a needs assessment and training approach. J Public Health Manag Pract. 2007;13:497–505.

Tachivana T, Tachivana H. A pilot study on the effects of a training program using the case method on acquiring health emergency management competency. Journal of the Showa Medical Society. 2007 Oct 28;67(5):422-34. [In Japanese] 橘とも子, 橘秀昭. ケースメソッドを用いた研修プログラム の 健康危機管理コンピテンシー獲得効果に関するパイロット研究. 昭和医学会雑誌. 2007 Oct 28;67(5):422-34. https://doi.org/10.14930/jsma1939.67.422

Olson D, Hoeppner M, Larson S, Ehrenberg A, Leitheiser AT. Lifelong learning for public health practice education: a model curriculum for bioterrorism and emergency readiness. Public Health Reports. 2008;123(2_suppl):53-64.

Subbarao I, Lyznicki J, Hsu EB, et al. A consensus-based educational framework and competency set for the discipline of disaster medicine and public health preparedness. Disaster Med Public Health Prepared. 2008;2:57–68.

Coule PL, Schwartz RB. The national disaster life support programs: a model for competency-based standardized and locally relevant training. J Public Health Manag Pract. 2009;15(2):S25-30.

Tachivana T, Arata Y, Ōhara T. A study on the acquisition level of health emergency management competency in the community--a qualitative survey by occupation and position required of all public health workers using the Delphi method. Journal of Health and Welfare Statistics. 2011;58(6):27-35. [In Japanese] 橘とも子, 荒田吉彦, 大原智子. 地域における健康危機管理コンピテンシー の 習得レベルに関する研究--デルファイ法を用いたすべて の 公衆衛生従事者に求められる職種別・職位別質的調査. 厚生 の 指標. 厚生労働統計協会 編. 2011;58(6):27-35.

Walsh L, Subbarao I, Gebbie K, et al. Core competencies for disaster medicine and public health. Disaster Med Public Health Prep. 2012;6:44–52.

Schultz CH, Koenig KL, Whiteside M, et al., National Standardized All-Hazard Disaster Core Competencies Task F. Development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Ann Emerg Med. 2012;59: 196-208 e1.

Olu O, Usman A, Kalambay K, Anyangwe S, Voyi K, Orach CG, Azazh A, Mapatano MA, Nsenga N, Manga L, Woldetsadik S. What should the African health workforce know about disasters? Proposed competencies for strengthening public health disaster risk management education in Africa. BMC Med Educ. 2018;18(1):1.

International Federation of Red Cross and Red Crescent Societies. Core competency framework for surge personnel. https://www.ifrc.org/document/core-competency-framework-surge-personnel#:~:text=The%20Core%20Competency%20Framework%20for,Red%20Cross%20Red%20Crescent%20Movement . Accessed 8 Sept 2023.

Ripoll-Gallardo A, Ragazzoni L, Mazzanti E, Meneghetti G, Franc JM, Costa A, Della Corte F. Residents working with Médecins Sans Frontières: training and pilot evaluation. Scand J Trauma Resuscitation Emerg Med. 2020;28(1):1–1.

Arifah AR, Tariq M, Rosliza AM, Juni MH. Leadership theories in disease outbreak management. Int J Public Health Clin Sci. 2018;5(2):1–6.

Glick JA. Decision making by leaders in high consequence disasters: a study of decision making by US federal coordinating officers. The George Washington University; 2013. https://scholarspace.library.gwu.edu/concern/gw_etds/xs55mc19t . Accessed 25 Jan 2024.

Download references

Acknowledgments

This research was supported by the World Health Organization Centre for Health Development (WHO Kobe Centre-WKC: K19008).

Author information

Authors and affiliations.

Accident and Emergency Medicine Academic Unit, The Chinese University of Hong Kong, Trauma & Emergency Centre, Prince of Wales Hospital, Shatin, New Territories, Hong Kong

Kevin K. C. Hung, Makiko K. MacDermot, Theresa S. I. Hui, Suet Yi Chan, Catherine P. Y. Mok, Pak Ho Leung, Emily Y. Y. Chan & Colin A. Graham

Collaborating Centre for Oxford University and Chinese University of Hong Kong for Disaster and Medical Humanitarian Response (CCOUC), JC School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong, China

Kevin K. C. Hung, Chi Shing Wong, Emily Y. Y. Chan & Colin A. Graham

Research Institute of Nursing Care for People and Community, University of Hyogo, Akashi, 673-8588, Japan

Sonoe Mashino

World Health Organization Centre for Health Development, Kobe, 651-0073, Japan

Ryoma Kayano

Monash University Disaster Resilience Initiative, Monash University, Clayton, Australia

Jonathan Abrahams

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, KH MM TH and CG; Methodology, KH MM TH SM RK JA CG; Formal Analysis, KH MM TH SC SM CM PL; Data Curation, KH MM TH; Writing – Original Draft Preparation, KH MM TH; Writing – Review & Editing, all authors; Supervision, EC CG; Project Administration, KH; Funding Acquisition, KH.

Corresponding author

Correspondence to Colin A. Graham .

Ethics declarations

Ethics approval and consent to participate.

Ethical Approval was obtained from the Survey and Behavioural Research Ethics Committee of The Chinese University of Hong Kong. Participants consented to survey participation electronically.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: supplementary file 1..

Search terms in English and Japanese language.

Additional file 2.

Online questionnaire.

Additional file 3.

  Supplementary 3a. Summary table for curricula alongside a competency model identified in English and Japanese language literature review. Supplementary 3b. Summary table for competency models identified in English and Japanese language literature review. Supplementary 3c. Summary table for curricula (without a competency model) identified in English and Japanese language literature review. Supplementary 3d. Gap analysis of published competency model and curricula against the WHO Health Emergency Programme Core Competencies and health EDRM perspectives. Supplementary 3e. Gap analysis of survey competency model against selected WHO Health Emergency Programme Core Competencies and health EDRM perspectives. Supplementary 3f. Gap analysis of survey curricula against selected WHO Health Emergency Programme Core Competencies and health EDRM perspectives. Supplementary 3g. Assessment of competency attainment in published curricula with a competency model. Supplementary 3h. Knowledge and skills listed by survey respondents not included in the list of management and technical competencies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hung, K.K.C., MacDermot, M.K., Hui, T.S.I. et al. Mapping study for health emergency and disaster risk management competencies and curricula: literature review and cross-sectional survey. Global Health 20 , 15 (2024). https://doi.org/10.1186/s12992-023-01010-y

Download citation

Received : 27 June 2023

Accepted : 21 December 2023

Published : 21 February 2024

DOI : https://doi.org/10.1186/s12992-023-01010-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Competency model
  • Health workforce

Globalization and Health

ISSN: 1744-8603

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

in literature review articles authors should

U.S. flag

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

  • formulating the research question(s) and objective(s),
  • searching the extant literature,
  • screening for inclusion,
  • assessing the quality of primary studies,
  • extracting data, and
  • analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

  • Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
  • Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
  • Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
  • Analyzing data using quantitative or qualitative methods.
  • Presenting results in summary of findings tables.
  • Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Table 9.1. Typology of Literature Reviews (adapted from Paré et al., 2015).

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

  • Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
  • Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
  • Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
  • Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
  • A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
  • Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
  • Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
  • Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
  • Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
  • Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
  • Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
  • Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
  • Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
  • Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
  • Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
  • Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
  • Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
  • Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
  • Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
  • Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
  • Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
  • Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
  • Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
  • Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
  • Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
  • King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
  • Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
  • Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
  • Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
  • Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
  • Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
  • Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
  • Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
  • Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
  • Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
  • Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
  • Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
  • Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
  • Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
  • Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
  • Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
  • Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
  • Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
  • Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
  • Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
  • Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
  • Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
  • Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
  • Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
  • Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
  • Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
  • Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
  • Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
  • Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
  • Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
  • Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
  • Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
  • Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
  • Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
  • Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

  • Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
  • PDF version of this title (4.5M)
  • Disable Glossary Links

In this Page

  • Introduction
  • Overview of the Literature Review Process and Steps
  • Types of Review Articles and Brief Illustrations
  • Concluding Remarks

Related information

  • PMC PubMed Central citations
  • PubMed Links to PubMed

Recent Activity

  • Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

statistics

  • Open access
  • Published: 20 February 2024

Evaluation of whether there is residual polyp tissue after hysteroscopic morcellation at Cam and Sakura City Hospital: a retrospective cohort study

  • Mustafa Can Sivas 1 ,
  • Karolin Ohanoglu Cetinel 1 &
  • Hilal Serap Arslan 2  

BMC Women's Health volume  24 , Article number:  133 ( 2024 ) Cite this article

30 Accesses

Metrics details

In polypectomy with mechanical hysteroscopic morcellators, the tissue removal procedure continues until no polyp tissue remains. The decision that the polypoid tissues were removed completely is made based on visual evaluation. In a situation where the polyp tissue was visually completely removed and no doubt that the polyp has been completely removed, short spindle-like tissue fragments on the polyp floor continue in most patients. There are no studies in the literature on whether visual evaluation provides adequate information at the cellular level in many patients in whom polypoid tissues have been determined to be completely removed. The aim of the present study was to analyze the pathological results of the curettage procedure, which was applied following the completion of polyp removal with operative hysteroscopy, and to evaluate whether there was residual polyp tissue in the short spindle-like tissue fragments that the mechanical hysteroscopic morcellator could not remove. The secondary aims of this study were to compare conventional loop resection hysteroscopy with hysteroscopic morcellation for the removal of endometrial polyps in terms of hemoglobin/hematocrit changes, polypectomy time and the amount of medium deficit.

A total of 70 patients with a single pedunculate polypoid image of 1.5-2 cm, which was primarily visualized by office hysteroscopy, were included in the study. Patients who had undergone hysteroscopic polypectomy were divided into two groups according to the surgical device used: the morcellator group ( n  = 35, Group M) and the resectoscope group ( n  = 35, Group R). The histopathological results of hysteroscopic specimens and curettage materials of patients who had undergone curettage at the end of operative hysteroscopy were evaluated. In addition, the postoperative 24th hour Hb/HCT decrease amounts in percentage, the polypectomy time which was measured from the start of morcellation, and deficit differences were compared between groups.

In total, 7 patients in the morcellator group had residual polyp tissue detected in the full curettage material. The blood loss was lower in the morcellator group than in the resectoscope group (M, R; (-0.07 ± 0.08), (-0,11 ± 0.06), ( p  < 0.05), respectively). The deficit value of the morcellator group were higher (M, R; (500 ml), (300 ml), ( p  < 0.05), respectively). The polypectomy time was shorter in the morcellator group (M, R; mean (2.30 min), (4.6 min), ( p  < 0.05)).

Conclusions

Even if the lesion is completely visibly removed during hysteroscopic morcellation, extra caution should be taken regarding the possibility of residual tissue. There is a need for new studies investigating the presence of residual polyp tissue.

Peer Review reports

Hysteroscopy is the gold standard for diagnostic and therapeutic tools used in the evaluation and treatment of conditions affecting the uterine cavity, such as abnormal uterine bleeding and infertility [ 1 , 2 , 3 ]. Conventional resectoscope hysteroscopy is the most common procedure performed to remove endometrial polyps using bipolar or monopolar electrical energy. Recent studies have shown that mechanical hysteroscopic morcellators that simultaneously cut and remove the targeted endometrial pathology are safe and effective alternatives to conventional hysteroscopy [ 4 , 5 , 6 ]. Advances in hysteroscopy instruments and ancillary equipment used in visualization have enabled especially mechanical hysteroscopic morcellators to be used in an office setting without the need for operating room conditions or general anesthesia [ 7 , 8 ].

In operative hysteroscopy, the tissue removal procedure continues until no polyp tissue remains. The decision that polypoid tissues were removed completely is made by visual evaluation [ 7 , 9 ]. To ensure that the root of the polyp is completely removed, some surgeons proceed slightly deeper into the myometrium [ 10 ]. In mechanical hysteroscopic morcellator, in a situation where the polyp tissue was visually completely removed and no doubt that the polyp has been completely removed, short spindle-like tissue fragments on the polyp floor continue in most patients, no matter how long the morcellation process was applied. In the literature, there is no study reporting whether short spindle-like tissue fragments belong to the endometrial tissue or the polyp.

In patients in whom polypoid tissues are determined to be completely removed, does visual evaluation provide sufficient information at the cellular level? Could the insignificantly small spindle structures that cannot be removed with a morcellator during morcellation, which are seen and perceived as the endometrial wall’s own tissue and considered unrelated to the polyp structure, be residual polyp tissue? We hypothesized that short spindle-like tissue fragments that the hysteroscopic morcellator cannot remove contain residual polyp tissue.

Therefore, the aim of the study was to analyze the pathological results of the curettage procedure, which was applied following the completion of polyp removal with operative hysteroscopy, and to evaluate whether there was residual polyp tissue in the short spindle-like tissue fragments that the mechanical hysteroscopic morcellator could not remove. Additionally, the aim of the study was to compare loop resection hysteroscopy with hysteroscopic morcellation in terms of hematocrit (HCT) / hemoglobin (Hb) changes, the amount of deficit and the duration of polyp removal.

The study was designed and completed as a retrospective cohort study. This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the Basaksehir Cam and Sakura City Hospital local ethical committee (date: 28.09.2022, number: 2022.09.316/protocol no: 316).

Inclusion and exclusion criteria

Patients who underwent hysteroscopic polypectomy between August 2020 and August 2022 were analyzed. Patients between the ages of 20 and 45 years were included in the study. Patient records were analyzed, and patients with a single pedunculate polyp 1.5-2 cm in size that was diagnosed during office hysteroscopy, and which was compatible with the operative hysteroscopy image, were included in the study. Patients in the menopausal or perimenopausal period, which could affect the study results due to atrophy or irregularity in the endometrial wall, or infertile patients with polyps were excluded from the study. All patients were in the proliferative phase of the menstrual cycle.

In light of the information obtained from the surgery notes, the patients who fully provided the following surgical steps in writing in the file records were included in the study. The inclusion and exclusion criteria were applied to 560 patients. Only 70 patients met all criteria. Statistical analysis was carried out for 70 patients, and the study was completed.

Design of the study and surgical steps for the two groups

Patients who had undergone hysteroscopic polypectomy and subsequently underwent full curettage were divided into two groups, the morcellator group (Group M) and the resectoscope group (Group R), according to the surgical device used. Pathology results for both the hysteroscopy and curettage materials of all patients were obtained from the system records. Pathology results were analyzed to determine whether the hysteroscopic material was a polyp and whether there was residual polyp tissue in the curettage report. The deficit values stated in the surgical notes of the patients included in the study were compared between the two groups. The polypectomy time was calculated from the beginning to the end of the morcellation period. The blood Hb and HCT levels of all patients before and 24 h after surgery were analyzed from the system records, and whether there was a difference in the amount of blood loss between the two techniques was determined. The percentage decreases in Hb and HCT values were taken as a basis for determining the amount of bleeding.

In the morcellator (M) group, a hysteroscopy procedure (TruClear Elite Hysteroscope Mini, Medtronic Corp., Minneapolis, Minnesota, ABD) was started after cervical dilation of the Hegar bougie no.6. The morcellator system had a 6 mm diameter rigid hysteroscope and a 2.9 mm probe tip designed for polyp removal. Cavity distension was achieved by using a hysteroscopic pump (Hysterolux™ Fluid Management System, Medtronic Corp., Minneapolis, MN, USA), which provides constant pressure with 0.9% NaCl (saline) solution. The intrauterine cavity pressure was set to 100 mm/Hg. All morcellated tissue fragments were simultaneously suctioned and deposited in the filter system by means of the vacuum feature of the hysteroscope system. The vacuum pulling force was set to 150 mm/Hg.

In the resectoscope (R) group, an 8.5 mm diameter resectoscope (Rigid hysteroscope, Olympus Europa SE & Co., Wendenstraße, Hamburg, Germany) with a 24 Fr bipolar loop (bipolar resection electrode, Olympus) was introduced after cervical dilatation until the 9th bougie in the lithotomy position. Cavity distension was achieved by using a hysteroscopic pump (Hysteroflow-2, Olympus Europa SE & Co., Wendenstraße, Hamburg, Germany), which provides constant pressure with 0.9% NaCl (saline) solution.

The deficit volume was calculated by taking the difference between the amount of fluid used by the hysteroscopic pump while providing cavity distension and the fluid collected by the vacuum system or resectoscope drainage route [ 11 ]. After the operative hysteroscopy surgery was completed, all patients underwent routine full curettage to exclude possible malignancies or additional pathologies. No: 5 Karman cannula and a single-lock suction injector were used for the full curettage procedure. Polypoid tissue material and curettage material were sent for pathological examination in different containers as required by the routine practice of the hospital. The results of the hysteroscopy, which was performed by two surgeons (M.C.S., K.O.C.), and the pathology results of these operations that were reported by the same pathologist were collected from the records.

Within the framework of the routine surgical steps performed in our hospital, operations were performed under spinal anesthesia and in the standard lithotomy position. Full curettage was performed at the end after operative hysteroscopy operations as a routine practice. Additionally, after each operation, hysteroscopic polypectomy and endometrial curettage materials were carried in two separate containers, and the samples were subjected to pathological analysis separately. Polypectomy and endometrial curettage materials were evaluated by the same pathologist. Surgical materials were fixed in 10% formalin. Paraffin-embedded blocks and 5 μm sections were taken. Slides were stained with hematoxylin and eosin. Preparations were examined under a light microscope. The patient records were consistent with the routine surgical procedures (surgical steps and material handling/evaluation methods).

Statistical analysis

When determining the sample size, calculations were performed in accordance with the protocol with gpower3.1 software ( https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower ), for a power value of 0.80, a margin of error of 0.05 and an effect size of 0.6. The Shapiro‒Wilk test was used to test whether the variables were normally distributed. Variables with a normal distribution are presented as the mean ± standard deviation, and an independent sample t test was used for comparisons between two independent groups. Variables that were not normally distributed are given median (minimum-maximum) values, and the Mann‒Whitney U test was used for comparisons between two independent groups. Categorical variables are presented as frequencies (n) and percentages (%), and Pearson’s chi-square test and Fisher’s exact test were used for comparisons. The statistical analysis was performed with the IBM SPSS Statistics 22.0 program (IBM Corp., Armonk, NY, USA). A p value of 0.05 was taken as the threshold for statistical significance.

A total of 70 patients were included in the study, 35 in the morcellator group and 35 in the resectoscope group. All of the hysteroscopy pathology results exhibited polyps. In addition to polypoid tissue, fibroid tissue was also reported in 2 patients in Group R and in 4 patients in Group M. According to the endometrial sampling results, in Group R, focal hyperplasia with atypia was observed along with residual polyp tissue in one patient, and focal hyperplasia with atypia without residual tissue was reported in one patient.

Evaluation of descriptive statistics

No statistically significant differences were found in terms of age, gravidity, parity or type of delivery between the two groups ( p  > 0.05) (Tables  1 and 2 ). Moreover, there was no statistically significant difference in the hysteroscopy (H/S) pathology results between the two groups ( p  > 0.05) (Table  2 ).

Evaluation of the presence of residual tissue

In the morcellator technique, residual polyp tissues were detected in the endometrial samplings of 7 out of 35 patients. In the resectoscope technique, residual polyp tissues were detected in 11 of 35 patients. No statistically significant difference was found between the two groups ( p  > 0.05) (Table  2 ).

Comparison of hematocrit/hemoglobin value changes, amount of deficit and polyp removal time between groups

In the morcellator system, the preoperative and postoperative Hb levels decreased from a median of 11.98 (8.7–14.6) g/dl to 11.18 (7.7–14.1) g/dl by 0.80. On the other hand, in the bipolar resectoscope system, Hb levels decreased from 11.68 (9.1–15.4) g/dl to 10.51 (7.9–12.8) g/dl by 1.17. A statistically significant difference was found in the comparison of the percentage decrease in preop-postop Hb values between the two groups (Group M (-0.07 ± 0.08) and Group R (-0.11 ± 0.06) ( p  < 0.05)) (Table  1 ). It was determined that there was less blood loss in patients who underwent surgery with the morcellator system.

In the morcellator system, the preoperative and postoperative HCT values decreased from a median of 37.41 (27.7–44.2) to 34.58 (26.8–44) by 2.83. Whereas in the bipolar resectoscope system, the median decreased from 35.85 (28.2–45.2) to 32.43 (25.9–38.7) by 3.42. No statistically significant difference was found in terms of the change in HCT percentage between the two groups (Group M (-0.08 ± 0.091), Group R (-0.10 ± 0.06) ( p  > 0.05)) (Table  1 ).

The median value of the deficit was 500 ml (100–2500) for the morcellation technique and 300 ml (100–2000) for the bipolar resectoscope method. The deficit values associated with the morcellation technique were higher than those associated with the resectoscope technique which was statistically significant ( p  < 0.05) (Table  1 ).

A statistically significant difference was found in the comparison of polyp removal time between the two techniques (Group M: 2.30 ± 0.2 min; Group R: 4.6 ± 1.5 min; p  < 0.05).

Comparison of all parameters according to the presence or absence of residual tissue

When patients with residual tissue and those without residue were compared in terms of age, gravidity, parity, type of delivery, H/S pathology, HCT percentage change, Hb percentage change and deficit values, ​​there were no statistically significant differences ( p  > 0.05) (Tables  3 and 4 ).

In this study, it was demonstrated that residual polyp tissue fragments could remain in morcellator technique. Additionally, there was less blood loss in the morcellator system. Polyp removal time was shorter in the morcellator system, and the amount of deficit was detected more in the morcellator technique.

During resection with bipolar energy, many pieces of extracted tissue of various sizes are scattered in the cavity. Although visual control is achieved, polypoid tissue fragments which are released in the resectoscope technique may remain in the cavity. For this reason, false-positive residual polyps may have been detected in the curettage materials of Group R. In this respect, based on the literature review and analysis that we conducted during the study process, we think that a different sampling method is needed to more objectively evaluate the presence of residual tissue in the resectoscope group. In our study, we observed that the resectoscope technique was at least as successful as the morcellator technique in terms of the absence of residue remaining, and we found no significant difference between the two techniques. We have presented our data comparing the morcellator technique and the resectoscope technique in the Results section to provide insight for the reader. However, we find it more appropriate and prefer not to provide further comments due to the reasons we mentioned above. We think that there may be a significant difference in the resectoscope group as a result of evaluating the presence of residue in the resectoscope group with a different, more appropriate method.

On the other hand, the main point that our study investigated was whether there was any residual tissue when it was visually determined that the polypoid structure was completely removed in the morcellator technique. In the morcellator technique, there is a blade system at the morcellator probe tip that rotates back and forth. When the blade moves back, the window at the morcellator tip opens. The polyp tissue enters with the negative pressure effect of the vacuum system, and the window closes simultaneously as the blade moves forward and cuts the polyp. The polyp tissue piece that is released into a closed environment inside the morcellator tip is removed by a vacuum system. In addition, with the negative pressure effect provided by the vacuum, possible blood, debris or free tissue pieces in the uterine cavity are removed from the cavity as they are pulled through the window at the morcellator tip. As a result, the removed polypoid structures are simultaneously aspirated [ 5 ]. False positivity in the curettage material results of the morcellator group is therefore not expected. In these respects, our study results bring a valuable perspective to the literature in terms of drawing attention to the fact that residual tissue may remain in the morcellator technique.

There is no study in the literature providing histopathological data on whether residual tissue remains after polyp removal by operative hysteroscopy. One study is available aiming to investigate the recurrence of the polyp structure. In this study, the patients were followed for 4 years, the percentage of recurrent polyps was evaluated, and the use of the morcellation technique and bipolar resectoscope technique were compared. The recurrence rate was 9.8% in the bipolar system and 2.6% in the morcellator (TruClear) system [ 12 ]. In this study, it was examined whether new polyp tissue developed in the same patients who underwent surgery, within a 4-year period and whether secondary lesions were also considered polyps after surgical removal. In the same study, it was stated that whether the new polyp tissue, defined as recurrence, developed from the same location of the uterine cavity was checked and confirmed using the operating notes. However, even if the polyp originates from the same point in the uterine cavity, it is not possible to determine whether secondary polyp tissue developed from residual tissue that remained after the previous operation. It is always possible that the secondary polyp structure is a new cellular organization at the previous location in the cavity and may have affected the study. In addition, studies with longer follow-up periods are needed to determine whether these rates will change after more than 4 years. In this respect, our study tried to reveal the presence of residual tissue histopathologically following hysteroscopic polyp removal and evaluated the possibility of recurrence from a different perspective.

In terms of the decreases in blood Hb and HCT before and after the operation, we determined that there was less blood loss in the morcellator system. A statistically significant difference was detected in the comparison of the change in Hb percentages. Although there was no statistically significant difference in the percentage change in HCT, a larger percentage decrease was observed in the resectoscope group. We are of the opinion that statistical significance in the percentages of HCT change will be achieved in studies conducted with a larger number of patients.

Polyp removal time was shorter in the morcellator system. Previously published studies have shown that the morcellation technique reduces both the total operative time and the polyp removal time [ 5 , 10 , 13 , 14 , 15 ]. In our study, we obtained similar results on the basis of the polypectomy duration.

A larger deficit was measured in the morcellator technique. There is no consensus in the literature in terms of deficits. The applied intracavitary pressure, the vacuum pressure applied by the morcellator system, the surgeon’s experience and sleight of hand may have affected this parameter [ 5 , 9 , 16 , 17 , 18 ].

There was no perforation or any other complication in either technique. Although the low number of patients may not be sufficient to evaluate the frequency of perforation, perforation during the morcellation technique was found to be less common in the literature [ 9 , 17 ].

This study is the first in the literature to evaluate whether residual polyp tissue remains after hysteroscopic morcellation. Similarly, this is the first study in the literature comparing the two techniques in terms of the degree of decrease in preoperative-postoperative hemoglobin and hematocrit values. This study contributes to the literature by comparing the two techniques in terms of polypectomy duration and deficit volume.

Limitations of the study

The first limitation of this study is that it is necessary to evaluate the presence of residue in the resectoscope group with a different, more appropriate method. Therefore, this study does not have sufficient standardization to compare the success of the two techniques. For this reason, no detailed comparative comments were made between the two techniques. The second limitation is that the presence of residual polyp tissue was investigated in the pathology results of curettage procedure which sampling the entire cavity. Although it was visually confirmed that there was no secondary polyp tissue in the cavity, using a technique that samples only the area where short spindle-like tissue fragments are present will make the study more powerful. Another limitation is the small patient population. More patients are needed to evaluate the frequency of perforation complications and the change in HCT percentage between the two groups.

Most surgeons visually determine that the polyp tissue is completely removed and terminate the operation during operative hysteroscopy. Even if the lesion is completely visibly removed during hysteroscopic morcellation, extra caution should be taken regarding the possibility of residual tissue. There is a need for new studies investigating the presence of residual polyp tissue.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to local ethical and legal requirements but are available from the corresponding author on reasonable request.

Abbreviations

Morcellator

Resectoscope

  • Hysteroscopy

Ma T, Readman E, Hicks L, Porter J, Cameron M, Ellett L, et al. Is outpatient hysteroscopy the new gold standard? Results from an 11 year prospective observational study. Aust N Z J Obstet Gynaecol. 2017;57(1):74–80. https://doi.org/10.1111/ajo.12560 . Epub 2016 Nov 15. PMID: 27861704.

Article   PubMed   Google Scholar  

Berceanu C, Cernea N, Căpitănescu RG, Comănescu AC, Paitici Ş, Rotar IC, et al. Endometrial polyps. Rom J Morphol Embryol. 2022 Apr-Jun;63(2):323–34. PMID: 36374138; PMCID: PMC9804076.

Mak JN, Imran A, Burnet S. Office hysteroscopy: back to the future! Climacteric. 2020;23(4):350–4. https://doi.org/10.1080/13697137.2020.1750589 . Epub 2020 Apr 22. PMID: 32319835.

Article   CAS   PubMed   Google Scholar  

Lee MM, Matsuzono T. Hysteroscopic intrauterine morcellation of submucosal fibroids: preliminary results in Hong Kong and comparisons with conventional hysteroscopic monopolar loop resection. Hong Kong Med J. 2016;22(1):56–61. https://doi.org/10.12809/hkmj154600 . Epub 2016 Jan 8. PMID: 26744122.

Franchini M, Ceci O, Casadio P, Carugno J, Giarrè G, Gubbini G, et al. Mechanical hysteroscopic tissue removal or hysteroscopic morcellator: understanding the past to predict the future. A narrative review. Facts Views Vis Obgyn. 2021;13(3):193–201. Epub 2021 Jun 10. PMID: 34555873; PMCID: PMC8823270.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Noventa M, Ancona E, Quaranta M, Vitagliano A, Cosmi E, D’Antona D, et al. Intrauterine Morcellator devices: the icon of Hysteroscopic Future or Merely a marketing image? A systematic review regarding safety, efficacy, advantages, and contraindications. Reprod Sci. 2015;22(10):1289–96. https://doi.org/10.1177/1933719115578929 . Epub 2015 Apr 14. PMID: 25878200.

Smith PP, Middleton LJ, Connor M, Clark TJ. Hysteroscopic morcellation compared with electrical resection of endometrial polyps: a randomized controlled trial. Obstet Gynecol. 2014;123(4):745 – 51. https://doi.org/10.1097/AOG.0000000000000187 . PMID: 24785600.

Rodríguez-Mias NL, Cubo-Abert M, Gomila-Villalonga L, Gómez-Cabeza J, Poza-Barrasús JL, Gil-Moreno A. Hysteroscopic myomectomy without anesthesia. Obstet Gynecol Sci. 2019;62(3):183–5. https://doi.org/10.5468/ogs.2019.62.3.183 . Epub 2019 Apr 4. PMID: 31139595; PMCID: PMC6520548.

Article   PubMed   PubMed Central   Google Scholar  

Hamerlynck TW, Schoot BC, van Vliet HA, Weyers S. Removal of endometrial polyps: Hysteroscopic Morcellation versus Bipolar Resectoscopy, a Randomized Trial. J Minim Invasive Gynecol. 2015 Nov-Dec;22(7):1237–43. https://doi.org/10.1016/j.jmig.2015.07.006 . Epub 2015 Jul 17. PMID: 26192235.

Ceci O, Franchini M, Cardinale S, Cannone R, Giarrè G, Fascilla FD, et al. Comparison of endometrial polyp recurrence in fertile women after office hysteroscopic endometrial polypectomy using two widely spread techniques. J Obstet Gynaecol Res. 2020;46(10):2084–91. https://doi.org/10.1111/jog.14400 . Epub 2020 Jul 26. PMID: 32715585.

Emanuel MH, Wamsteker K. The Intra Uterine Morcellator: a new hysteroscopic operating technique to remove intrauterine polyps and myomas. J Minim Invasive Gynecol. 2005 Jan-Feb;12(1):62 – 6. https://doi.org/10.1016/j.jmig.2004.12.011 . PMID: 15904601.

AlHilli MM, Nixon KE, Hopkins MR, Weaver AL, Laughlin-Tommaso SK, Famuyide AO. Long-term outcomes after intrauterine morcellation vs hysteroscopic resection of endometrial polyps. J Minim Invasive Gynecol. 2013 Mar-Apr;20(2):215 – 21. https://doi.org/10.1016/j.jmig.2012.10.013 . Epub 2013 Jan 5. PMID: 23295201.

Rovira Pampalona J, Degollada Bastos M, Mancebo Moreno G, Ratia Garcia E, Buron Pust A, Mateu Pruñonosa JC, et al. Outpatient hysteroscopic polypectomy: Bipolar Energy System (Versapoint®) versus mechanical Energy System (TRUCLEAR System®) - preliminary results. Gynecol Obstet Invest. 2015;80(1):3–9. Epub 2015 Mar 20. PMID: 25824810.

Tsuchiya A, Komatsu Y, Matsuyama R, Tsuchiya H, Takemura Y, Nishii O, Parallel Group Comparison Study. Intraoperative and Postoperative Clinical Evaluation of the Hysteroscopic Morcellator System for Endometrial Polypectomy: A Prospective, Randomized, Single-blind,. Gynecol Minim Invasive Ther. 2018 Jan-Mar;7(1):16–21. doi: 10.4103/GMIT.GMIT_6_17. Epub 2018 Feb 16. PMID: 30254929; PMCID: PMC6135153.

van Dongen H, Emanuel MH, Wolterbeek R, Trimbos JB, Jansen FW. Hysteroscopic morcellator for removal of intrauterine polyps and myomas: a randomized controlled pilot study among residents in training. J Minim Invasive Gynecol. 2008 Jul-Aug;15(4):466–71. Epub 2008 Apr 18. PMID: 18588849.

van Wessel S, van Vliet HAAM, Schoot BC, Weyers S, Hamerlynck TWO. Hysteroscopic morcellation versus bipolar resection for removal of type 0 and 1 submucous myomas: a randomized trial. Eur J Obstet Gynecol Reprod Biol. 2021;259:32–7. Epub 2021 Jan 29. PMID: 33571839.

Yin X, Cheng J, Ansari SH, Campo R, Di W, Li W, et al. Hysteroscopic tissue removal systems for the treatment of intrauterine pathology: a systematic review and meta-analysis. Facts Views Vis Obgyn. 2018;10(4):207–13. PMID: 31367293; PMCID: PMC6658200.

CAS   PubMed   Google Scholar  

Shazly SA, Laughlin-Tommaso SK, Breitkopf DM, Hopkins MR, Burnett TL, Green IC, et al. Hysteroscopic Morcellation Versus Resection for the treatment of Uterine Cavitary lesions: a systematic review and Meta-analysis. J Minim Invasive Gynecol. 2016 Sep-Oct;23(6):867–77. Epub 2016 May 7. PMID: 27164165.

Download references

Acknowledgements

Not applicable.

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and affiliations.

Department of Obstetrics and Gynecology, Republic of Türkiye Ministry of Health, Basaksehir Cam and Sakura City Hospital, Basaksehir Neighborhood, G-434 Street, No: 2L, Basaksehir, Istanbul, Türkiye

Mustafa Can Sivas & Karolin Ohanoglu Cetinel

Department of Pathology, Republic of Türkiye Ministry of Health, Basaksehir Cam and Sakura City Hospital, Istanbul, Türkiye

Hilal Serap Arslan

You can also search for this author in PubMed   Google Scholar

Contributions

Study concept and design were performed by MCS. Surgical procedures were performed by KOC and MCS. Pathological examinations were carried out by HSA. Data collection was performed by KOC and MCS. Data analysis was performed by KOC, MCS and HSA. The first draft of the manuscript was written by MCS. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mustafa Can Sivas .

Ethics declarations

Ethics approval and consent to participate.

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the Basaksehir Cam and Sakura City Hospital local ethical committee (date: 28.09.2022, number: 2022.09.316/protocol no: 316). Informed consent was obtained from all the participants. All methods were performed following the relevant guidelines and regulations. Written and signed informed consent was obtained from all participants.

Consent for publication

Competing interests.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sivas, M.C., Ohanoglu Cetinel, K. & Arslan, H.S. Evaluation of whether there is residual polyp tissue after hysteroscopic morcellation at Cam and Sakura City Hospital: a retrospective cohort study. BMC Women's Health 24 , 133 (2024). https://doi.org/10.1186/s12905-024-02978-4

Download citation

Received : 23 October 2023

Accepted : 17 February 2024

Published : 20 February 2024

DOI : https://doi.org/10.1186/s12905-024-02978-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Endometrial sampling
  • Morcellation

BMC Women's Health

ISSN: 1472-6874

in literature review articles authors should

Cart

  • SUGGESTED TOPICS
  • The Magazine
  • Newsletters
  • Managing Yourself
  • Managing Teams
  • Work-life Balance
  • The Big Idea
  • Data & Visuals
  • Reading Lists
  • Case Selections
  • HBR Learning
  • Topic Feeds
  • Account Settings
  • Email Preferences

Bring Human Values to AI

  • Jacob Abernethy,
  • François Candelon,
  • Theodoros Evgeniou,
  • Abhishek Gupta,
  • Yves Lostanlen

in literature review articles authors should

When it launched GPT-4, in March 2023, OpenAI touted its superiority to its already impressive predecessor, saying the new version was better in terms of accuracy, reasoning ability, and test scores—all of which are AI-performance metrics that have been used for some time. However, most striking was OpenAI’s characterization of GPT-4 as “more aligned”—perhaps the first time that an AI product or service has been marketed in terms of its alignment with human values.

In this article a team of five experts offer a framework for thinking through the development challenges of creating AI-enabled products and services that are safe to use and robustly aligned with generally accepted and company-specific values. The challenges fall into five categories, corresponding to the key stages in a typical innovation process from design to development, deployment, and usage monitoring. For each set of challenges, the authors present an overview of the frameworks, practices, and tools that executives can leverage to face those challenges.

Speed and efficiency used to be the priority. Now issues such as safety and privacy matter too.

Idea in Brief

The problem.

Products and services increasingly leverage artificial intelligence to improve efficiency and performance, but the results can be unpredictable, intrusive, offensive, and even dangerous.

The Solution

Companies need to factor AI’s behavior and values into their innovation and development processes to ensure that they bring to market AI-enabled offerings that are safe to use and are aligned with generally accepted and company-specific values.

How to Proceed

This article identifies six key challenges that executives and entrepreneurs will face and describes how to meet them. Companies that move early to acquire the needed capabilities will find them an important source of competitive advantage.

When it launched GPT-4, in March 2023, OpenAI touted its superiority to its already impressive predecessor, saying the new version was better in terms of accuracy, reasoning ability, and test scores—all of which are AI-performance metrics that have been used for some time. However, most striking was OpenAI’s characterization of GPT-4 as “more aligned”—perhaps the first time that an AI product or service has been marketed in terms of its alignment with human values.

  • JA Jacob Abernethy is an associate professor at the Georgia Institute of Technology and a cofounder of the water analytics company BlueConduit.
  • FC François Candelon is a managing director and senior partner at Boston Consulting Group (BCG), and the global director of the BCG Henderson Institute.
  • Theodoros Evgeniou is a professor at INSEAD and a cofounder of the trust and safety company Tremau.
  • AG Abhishek Gupta is the director for responsible AI at Boston Consulting Group, a fellow at the BCG Henderson Institute, and the founder and principal researcher of the Montreal AI Ethics Institute.
  • YL Yves Lostanlen has held executive roles at and advised the CEOs of numerous companies, including AI Redefined and Element AI.

in literature review articles authors should

Partner Center

IMAGES

  1. How to write a literature review: Tips, Format and Significance

    in literature review articles authors should

  2. How to Write a Literature Review in 5 Simple Steps

    in literature review articles authors should

  3. literature review article examples Sample of research literature review

    in literature review articles authors should

  4. 50 Smart Literature Review Templates (APA) ᐅ TemplateLab

    in literature review articles authors should

  5. How to write a literature review in research paper

    in literature review articles authors should

  6. a brief guide to writing a literature review

    in literature review articles authors should

VIDEO

  1. How to Write a Literature Review

  2. Write Your Literature Review FAST

  3. Literature Review 101

  4. Chapter 5 Review of Literature PART 02

  5. Literature Review In ONE Day

  6. Literature review

COMMENTS

  1. Writing a Literature Review

    For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines' purposes and conventions in scholarship.

  2. Ten Simple Rules for Writing a Literature Review

    Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1]. For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2].

  3. How to Write a Literature Review

    Step 1 - Search for relevant literature Step 2 - Evaluate and select sources Step 3 - Identify themes, debates, and gaps Step 4 - Outline your literature review's structure Step 5 - Write your literature review Free lecture slides Other interesting articles Frequently asked questions Introduction Quick Run-through Step 1 & 2 Step 3 Step 4 Step 5

  4. Writing a literature review

    Rather than just being an exhaustive list of all that has been published, a literature review should be an informative, personal but unbiased synopsis of the information, providing a balanced view that includes conflicting findings and inconsistencies, as well as established and current thinking.

  5. Approaching literature review for academic purposes: The Literature

    A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field.

  6. Literature review as a research methodology: An ...

    Of course, there already exist some guidelines for conducting literature reviews that suggest different types of reviews, such as narrative or integrative reviews (e.g., Baumeister & Leary, 1997Wong, Greenhalgh, Westhorp, Buckingham, & Pawson, 2013 ), systematic reviews, and meta-analysis (e.g., Davis, Mengersen, Bennett, & Mazerolle, 2014Libera...

  7. How to write a superb literature review

    The best proposals are timely and clearly explain why readers should pay attention to the proposed topic. It is not enough for a review to be a summary of the latest growth in the literature: the ...

  8. Writing a literature review

    When writing a literature review it is important to start with a brief introduction, followed by the text broken up into subsections and conclude with a summary to bring everything together. A summary table including title, author, publication date and key findings is a useful feature to present in your review (see Table 1 for an example).

  9. Writing a Literature Review

    Preparing a literature review involves: Searching for reliable, accurate and up-to-date material on a topic or subject. Synthesising these key ideas, theories and concepts into a summary of what is known. Discussing and evaluating these ideas, theories and concepts. Preparing the ground for the application of these ideas to new research.

  10. 5. The Literature Review

    A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

  11. What is a literature review?

    A literature review should: Provide a comprehensive and updated review of the literature; Explain why this review has taken place; Articulate a position or hypothesis; Acknowledge and account for conflicting and corroborating points of view From S age Research Methods Purpose of a Literature Review

  12. How to Undertake an Impactful Literature Review: Understanding Review

    The outcome of these review articles should attempt to make a strong theoretical contribution through the synthesis of existing literature; simply describing prior art is not considered a theory, but it does aid in determining theoretical contributions. ... The author uses relevant literature to validate their hypotheses: Carrying out retrieval ...

  13. Reviewing systematic literature reviews: ten key questions ...

    Systematic literature review articles are important for synthesizing knowledge in management and business research. However, to date, we lack clear guidelines how to review such articles. This editorial takes the perspective of the reviewer. It presents ten key questions and criteria that reviewers should ask when reviewing systematic literature reviews.

  14. A Step-by-Step Guide to Writing a Scientific Review Article

    An additional benefit to the authors is that high-quality review articles can often be cited many years after publication (1, 2). The reader of a scientific review article should gain an understanding of the current state of knowledge on the subject, points of controversy, and research questions that have yet to be answered .

  15. Literature Review: The What, Why and How-to Guide

    Systematic review: "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

  16. Literature Review Guide

    From the Online Library. Conducting Your Literature Review by Susanne Hempel. ISBN: 9781433830921. Publication Date: 2019-09-10. Literature Review and Research Design by Dave Harris. ISBN: 9780367250362. Publication Date: 2019-12-13. Writing the Literature Review by Sara Efrat Efron; Ruth Ravid. ISBN: 9781462536900.

  17. Literature reviews as independent studies: guidelines for academic

    A literature review - or a review article - is "a study that analyzes and synthesizes an existing body of literature by identifying, challenging, and advancing the building blocks of a theory through an examination of a body (or several bodies) of prior work (Post et al. 2020, p. 352).Literature reviews as standalone pieces of work may allow researchers to enhance their understanding of ...

  18. Ten Simple Rules for Writing a Literature Review

    Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7].

  19. Eight problems with literature reviews and how to fix them

    The authors also excluded/ignored grey literature (see point 5, below). In a review of tropical forest management impacts 32 and in a review of forest conservation policies 33, searches for ...

  20. The Literature Review: A Foundation for High-Quality Medical Education

    By understanding the goals of a literature review and following a few basic processes, authors can enhance both the quality of their educational research and the likelihood of publication in the Journal of Graduate Medical Education ( JGME) and in other journals. Go to: The Literature Review Defined

  21. Five tips for developing useful literature summary tables for writing

    Literature summary tables are not only meant to provide an overview of basic information (authors, country, purpose and findings) about included articles, but they should also provide detailed information about the theoretical and conceptual frameworks and the methods used in the included article.

  22. Literature review guidelines

    A good review critically analyses how accurately previous authors have reported their findings and whether they have refrained from asserting conclusions not supported by data. Discuss the methodological diversity of studies reported in the literature review and the implications of this diversity for new knowledge or future research.

  23. Mapping study for health emergency and disaster risk management

    The literature review was performed in the English and Japanese languages using a systematic approach to the literature search. Searches of the English language literature published from 1 January 1990 to 11 March 2020 were conducted using MEDLINE (1966), EMBASE (1980), and CINAHL (1980).

  24. Full article: Bibliometric portrait of the theory of community-based

    As the theory gains popularity, many articles use them to bring academic strength. Therefore, the need for a review article arises to understand the spectrum of a theory and its use. The present study is one such theory-based review article where the authors present a bibliometric and semi-structured analysis of the articles relating to the TCBE.

  25. Chapter 9 Methods for Literature Reviews

    Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generat...

  26. Evaluation of whether there is residual polyp tissue after

    In this respect, based on the literature review and analysis that we conducted during the study process, we think that a different sampling method is needed to more objectively evaluate the presence of residual tissue in the resectoscope group. ... The authors have no relevant financial or non-financial interests to disclose. Additional ...

  27. Diagnostics

    Through a comprehensive literature search, 71 articles meeting the inclusion criteria were included in the qualitative analysis, investigating the hardware, software, and operational aspects of these systems. ... Following a comprehensive literature search, two authors (P.S. and D.A.) systematically evaluated the titles and abstracts of ...

  28. Bring Human Values to AI

    Bring Human Values to AI. Speed and efficiency used to be the priority. Now issues such as safety and privacy matter too. Summary. When it launched GPT-4, in March 2023, OpenAI touted its ...