literature review of economics

• Recommended Databases
• Recommended Journals
• Recommended Books
• Reference Resources
• Open and Accessible Materials
• Faculty Resources
• Statistics Resources
• Use EconLit
• Use PAIS Index
• Use Nexis Uni
• Evaluate Articles
• Source Management Tools
• Annotated Bibliography
• Literature Review
• Useful Links

What's on this Page

This page is meant to help you create a literature review for academic projects and publications. Each tab outlines a different aspect of what a literature review is and how to build one. If you need help finding sources for your literature reviews, check out How To pages.

How to Build a Literature Review

• What is a Lit Review?
• Why Write a Lit Review?
• Building a Lit Review
• Prepping for a Lit Review
• Basic Example
• Other Resources/Examples

What is a Literature Review?

A literature review is a comprehensive summary and analysis of previously published research on a particular topic. Literature reviews should give the reader an overview of the important theories and themes that have previously been discussed on the topic, as well as any important researchers who have contributed to the discourse. This review should connect the established conclusions to the hypothesis being presented in the rest of the paper.

What a Literature Review Is Not:

• Annotated Bibliography: An annotated bibliography summarizes and assesses each resource individually and separately. A literature review explores the connections between different articles to illustrate important themes/theories/research trends within a larger research area. 
• Timeline: While a literature review can be organized chronologically, they are not simple timelines of previous events. They should not be a list of any kind. Individual examples or events should be combined to illustrate larger ideas or concepts.
• Argumentative Paper: Literature reviews are not meant to be making an argument. They are explorations of a concept to give the audience an understanding of what has already been written and researched about an idea. As many perspectives as possible should be included in a literature review in order to give the reader as comprehensive understanding of a topic as possible.

Why Write a Literature Review?

After reading the literature review, the reader should have a basic understanding of the topic. A reader should be able to come into your paper without really knowing anything about an idea, and after reading the literature, feel more confident about the important points.

A literature review should also help the reader understand the focus the rest of the paper will take within the larger topic. If the reader knows what has already been studied, they will be better prepared for the novel argument that is about to be made.

A literature review should help the reader understand the important history, themes, events, and ideas about a particular topic. Connections between ideas/themes should also explored. Part of the importance of a literature review is to prove to experts who do read your paper that you are knowledgeable enough to contribute to the academic discussion. You have to have done your homework.

A literature review should also identify the gaps in research to show the reader what hasn't yet been explored. Your thesis should ideally address one of the gaps identified in the research. Scholarly articles are meant to push academic conversations forward with new ideas and arguments. Before knowing where the gaps are in a topic, you need to have read what others have written.

What does a literature review look like?

As mentioned in other tabs, literature reviews should discuss the big ideas that make up a topic. Each literature review should be broken up into different subtopics. Each subtopic should use groups of articles as evidence to support the ideas. There are several different ways of organizing a literature review. It will depend on the patterns one sees in the groups of articles as to which strategy should be used. Here are a few examples of how to organize your review:

Chronological

If there are clear trends that change over time, a chronological approach could be used to organize a literature review. For example, one might argue that in the 1970s, the predominant theories and themes argued something. However, in the 1980s, the theories evolved to something else. Then, in the 1990s, theories evolved further. Each decade is a subtopic, and articles should be used as examples. 

Themes/Theories

There may also be clear distinctions between schools of thought within a topic, a theoretical breakdown may be most appropriate. Each theory could be a subtopic, and articles supporting the theme should be included as evidence for each one. 

If researchers mainly differ in the way they went about conducting research, literature reviews can be organized by methodology. Each type of method could be a subtopic,  and articles using the method should be included as evidence for each one.

Preliminary Steps for Literature Review

• Define your research question
• Compile a list of initial keywords to use for searching based on question
• Search for literature that discusses the topics surrounding your research question
• Assess and organize your literature into logical groups
• Identify gaps in research and conduct secondary searches (if necessary)
• Reassess and reorganize literature again (if necessary)
• Write review

Here is an example of a literature review, taken from the beginning of a research article. You can find other examples within most scholarly research articles. The majority of published scholarship includes a literature review section, and you can use those to become more familiar with these reviews.

Source:  Perceptions of the Police by LGBT Communities

• ISU Writing Assistance The Julia N. Visor Academic Center provides one-on-one writing assistance for any course or need. By focusing on the writing process instead of merely on grammar and editing, we are committed to making you a better writer.
• University of Toronto: The Literature Review Written by Dena Taylor, Health Sciences Writing Centre
• Purdue OWL - Writing a Lit Review Goes over the basic steps
• UW Madison Writing Center - Review of Literature A description of what each piece of a literature review should entail.
• USC Libraries - Literature Reviews Offers detailed guidance on how to develop, organize, and write a college-level research paper in the social and behavioral sciences.
• Creating the literature review: integrating research questions and arguments Blog post with very helpful overview for how to organize and build/integrate arguments in a literature review
• Understanding, Selecting, and Integrating a Theoretical Framework in Dissertation Research: Creating the Blueprint for Your “House” Article focusing on constructing a literature review for a dissertation. Still very relevant for literature reviews in other types of content.

A note that many of these examples will be far longer and in-depth than what's required for your assignment. However, they will give you an idea of the general structure and components of a literature review. Additionally, most scholarly articles will include a literature review section. Looking over the articles you have been assigned in classes will also help you.

• Sample Literature Review (Univ. of Florida) This guide will provide research and writing tips to help students complete a literature review assignment.
• Sociology Literature Review (Univ. of Hawaii) Written in ASA citation style - don't follow this format.
• Sample Lit Review - Univ. of Vermont Includes an example with tips in the footnotes.
• << Previous: Annotated Bibliography
• Next: Useful Links >>
• Last Updated: May 10, 2024 12:20 PM
• URL: https://guides.library.illinoisstate.edu/economics

Additional Links

• Directions and Parking
• Accessibility Services
• Library Spaces
• Staff Directory

• Articles and Working Papers
• Data Citation
• Dictionaries, Handbooks, Encyclopedias
• Economic Indicators
• Tools for the Research Process
• Zotero (Citation Manager) This link opens in a new window
• Developing a Literature Review
• Evaluating your Sources
• Searching the Literature
• Google Scholar

Contact your Commerce Librarian

What is a literature review.

A literature review is a narrative compilation of selected academic sources related to your topic. Lit reviews describe the research you have studied and develop in your reason for the study, as well as provide criticism of past research. The end result should be a narrative showing the inherent need for your research in the field. Grounding your intended research in the current movements of the field will provide you with evidence of trends on where the field is headed. It also offers you the snapshot of the methodologies used in those studies. You can see what questions are being asked and find answers based on differing approaches to the topic.

An ideal literature review serves two purposes in your study. It strengthens your thesis and justifies your research question. By providing a critical summary of foundational and contemporary research on the topic, a literature review can show readers how your research fills important knowledge gaps. Pinpointing the other work in the field can show the unique perspective your study will provide. It can also offer a thoughtful critique of existing work that shows your full understanding of the opportunities and obstacles in your discipline.

Do not confuse it with:

• an  Annotated Bibliography , which lists citations to books, articles and documents, followed by a brief descriptive and evaluative paragraph,
• a Book Review , or short critical discussion about the merits and weaknesses or a specific works,
• or a Business Report , which provides analysis of a situation, either a real one or from a case study, applyng business principles and theories to identify a range of possible solutions to a problem. 

Why a Literature Review?

To demonstrate that you can:

• effectively use research methods to collect and curate information that is useful in answering significant questions;
• foster the ability to make decisions based on rigorous evidence;
• effectively communicate research results in a written form;
• develop the discipline to work with autonomy;
• understand the value, purpose, and methodologies of insightful research.

Purpose of a literature review from Academic Research Foundations: Quantitative by Rolin Moe

• << Previous: Zotero (Citation Manager)
• Next: Developing a Literature Review >>
• Last Updated: Mar 5, 2024 12:01 PM
• URL: https://guides.lib.virginia.edu/economics
• 434-924-3021
• [email protected]
• Ask a Librarian
• UVA Shannon Library P.O. Box 400113 160 McCormick Road Charlottesville, VA 22904

About the Library

• Staff Directory
• Fellowships

Using the Library

• Library Use Policies
• Off-Grounds Access
• ITS Computing Accounts
• Accessibility Services
• Emergency Information
• UVA Privacy Policy
• Tracking Opt-out

Other Sites

• Cavalier Advantage
• Library Staff Site

This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy .

Journal of Economic Literature

Current issue, find articles in this journal.

The pace of economics publishing

Daniel Hamermesh discusses publishing in economics and how to speed it up.

Forthcoming Articles

View articles accepted for publication in JEL .

Editors and contacts at the Journal of Economic Literature .

• Research Guides
• University Libraries

Doing a Literature Review

• Scholarly Sources
• News & Primary Sources
• Policy & Government Info
• Country & political risk
• US economic
• Company Info
• International trade & FDI
• Using US census data
• Industry & Market Information
• Business Risk Data & Indicators
• White Papers & Grey Lit
• Reference Sources
• Health Economics
• Cryptocurrency & Bitcoin
• Context Matters: Doing an Environmental Scan
• Author Profile

I.  What is a Literature Review? The purpose of a review is to analyze critically a segment of a published body of knowledge through summary, classification, and comparison of prior research studies. It can be a simple summary of the sources, but it usually has an organizational pattern, combining both summary and synthesis.

• Review of the Literature (Wisconsin)
• Systematic Literature Review vs Narrative Reviews
• Get Lit: the Literature Review Candace Schaefer in the Texas A&M University Writing Center.

III.  What Major Steps and Basic Elements Literature Reviews Require?

• Overview of the subject, issue or theory under consideration, along with the objectives of literature review
• Perform a literature review, finding materials relevant to the subject being explored
• Division of works under review into categories (e.g. those in support of a particular position, those against, etc)
• Explanation of how each work is similar to and how it varies from the others
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research
• Write a Lit Review (UCSC)

IV.    Which Citation Tool Are You Going to Use to Manage the Literature Sources? Choose your citation tool before conducing your literature reviews.  There are a number of choices, including following software supported by the Libraries and the University:

• RefWorks Available at no cost to Texas A&M affiliates.
• EndNote Available for free through a campus-wide site license.

Cited Reference Searching

Cited references are the sources consulted in writing an article or a book, often referred to within the text of the work. A list of cited references may appear as Bibliographic Notes, Footnotes or Endnotes, References, List of Sources Cited or Consulted. In order for an article to be cited, it needs to have been published for a long enough period of time for another published article, citing it to appear.

These listings can be helpful in a number of ways:

• Finding an article on a relevant topic and accumulating similar helpful resources
• Following a specific idea or theory back to its first appearance in the literature
• Finding articles that build on a specific theory or the most recent article on a topic
• Identifying experts or leaders on a specific topic
• Documenting scholarly reputation and impact for tenure and promotion

The cited reference databases are efficient in pulling together many articles on a topic with their references and in identifying which articles on a topic have been cited most frequently.  They can also help identify the “top” journals in a field by impact factor, which may be useful for assessing them.

• Web of Science This link opens in a new window covers the world’s leading scholarly literature in the sciences, social sciences, arts, and humanities and examines proceedings of international conferences, symposia, seminars, colloquia, workshops, and conventions. It also includes cited references and citation mapping functions.

Searches can be done by:

• Title or Topic
•  Author or Editor – The Author Finder tool includes variations on an author’s name
• Journal or Publication Name
• Grant Name or Funding Agency
• Limited by year, Language, Document Type 

The citation of the article  will be retrieved with its references as well as the number of times cited and by whom.

You can refine your search results by subject area, useful when there is more than one author with the same name, or by document type.  You can see the number of articles in your set contributed by particular authors and institutions and can create a citation report to identify which articles in your results have been cited the most.

You can easily export your results to bibliographic software like EndNote or RefWorks.

Articles can be searched by:

• Abstract word or keyword
• Source or journal
• Author (by name or by affiliation)
• Limit by date or document type

The database allows accounts to be set up and can save search alerts and journals lists.  Scopus also provides journal analytics including data and graphs to illustrate the total citations, articles published, trend line and % not cited over time.  It has the ability to exclude self-citations.

• << Previous: Context Matters: Doing an Environmental Scan
• Last Updated: May 6, 2024 3:56 PM
• URL: https://tamu.libguides.com/economics

• Welcome! Start here
• Prepare for research
• Peer-reviewed articles and working papers
• Books and e-books
• News and periodicals
• Data and statistics

Introduction: Write your research with literature review and citation

What is a literature review, help with writing papers.

At the Library, we offer a collection of guides and resources to help you conduct your literature review, cite your sources (including data), and develop your academic writing skills.

Research, and the literature review in particular, is a cyclical process. There is an art to the sometimes messy, thrilling, and frustrating process of conducting a lit review.

• Read widely but selectively.
• Follow the citation trail -- building on previous research by reviewing bibliographies of articles and books that are close to your interest.
• Synthesize previous research on the topic.
• Focus on ways to have the body of literature tell its own story. Do not add your own interpretations at this point.
• Look for patterns and find ways of tying the pieces together.

Conducting a literature review

• Throw out a wide net and read, read, read. 
• How many sources do you need? What types of sources? Which citation style should you use? What time period should it cover? Is currency important? What do you need to be aware of related to scholarly versus popular materials?
• Explored synonyms and alternative phrases in your searches. You will eventually begin to find the same articles and materials in your searches.

Writing a literature review

• The initial work Identify the organizational structure you want to use: chronologically, thematically, or methodologically
• Start writing: let the literature tell the story, find the best examples, summarize instead of quote, synthesize by rephrasing (but cite!) in the context of your work

Below are a few key resources to get you started:

• Citation by Academic Engagement Last Updated Oct 24, 2023 27278 views this year
• Citing datasets
• Developing a Thesis for a Research Paper This very helpful guide from the University of North Carolina at Chapel Hill Writing Center discusses the art of crafting a thesis.
• Improve Your Research Skills This guide discusses some of the basics of doing college level research, including tips for evaluating sources and a glossary of terms with examples.
• Brown University Writing Center "The Writing Center is a satellite office of the Dean of the College. The Center is staffed by graduate students from a variety of academic disciplines. Staff members are experienced writers and teachers who participate in ongoing training in composition and Writing Center theory and practice...Writing Center conferences generally last an hour. Experienced as well as inexperienced writers are encouraged to come to the Center with their writing concerns. Writing Center Associates are prepared to discuss all stages of the writing process, from finding a topic up through revision and editing strategies. Associates can help writers deal with writer's block, audience awareness, argumentation, organization, grammar, research skills, the conventions of academic writing, English as a Second Language, and issues of clarity and style."
• << Previous: Data and statistics
• Last Updated: Sep 18, 2023 10:05 AM
• URL: https://libguides.brown.edu/economics

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

Up: Home : Doing a Dissertation in Economics > Doing it > Literature Review

• Literature Review

A literature review establishes where your dissertation ‘fits in’ to the existing body of knowledge.

However, many academic papers have very brief literature reviews, and sometimes they are confined to the introduction rather than being a separate section in their own right.

You may want to follow this route – or you may decide to omit the literature review entirely. But many students normally spend some time making a separate section in which they discuss the existing literature.

Why do one?

Because a literature review allows you to demonstrate, in essay format, that you understand and can analyse the existing literature. The literature review allows you to answer the implicit question ‘What is the existing state of knowledge on this topic?’, and answer it in such a way that introduces any other work that you are doing.

The other key reason for doing a literature review is that it forces you to organise your thoughts. This can often make any theoretical or empirical work you do in other sections clearer, as you understand the topic more thoroughly.

How do I do one?

When researching your dissertation, it is not uncommon to read 20-30 journal articles. These will form the basis of discussion in any literature review. As you will probably have to read some articles anyway, the reading burden is not excessive.

Identifying which articles are important is a stumbling block. Asking members of your department or supervisors for key readings can get you started. Remember that your university library will likely have many electronic journals and databases in which you can search for papers. Databases such as Econlit are helpful, although they might miss some important contributions as they depend on how you phrase your search. Once you have a few recent papers you can normally use the bibliography to steer you towards further readings.

Of course, as well as reading the articles, you need to demonstrate that you understand them. The section ‘ Effective Reading ‘ can help – in particular, many students waste time trying to understand overly complex and irrelevant journal articles. Being selective, and understanding what is important, can save huge amounts of time and angst.

The next step is planning what you are going to write. As discussed in the ‘ Essay Writing ‘ section you might want to organise the section thematically.

Themes (sub-sections) might be different theories which try to explain a phenomenon, or they might discuss how the debate has evolved.

Previous: Introduction

Next: Economic Theory and Models

Share this page: Email , Facebook , LinkedIn , Twitter

• Should I Do One?
• Choosing a Title
• A word about title wording
• Searching for inspiration
• The nuts and bolts
• Initial Plans
• Introduction
• Economic Theory and Models
• Evidence and Empirics
• Your Course

Published by The Economics Network at the University of Bristol . All rights reserved. Feedback: [email protected] Supported by the Royal Economic Society and the Scottish Economic Society

• Search Menu
• Advance articles
• Featured articles

Virtual Issues

• Browse content in B - History of Economic Thought, Methodology, and Heterodox Approaches
• Browse content in B4 - Economic Methodology
• B49 - Other
• Browse content in C - Mathematical and Quantitative Methods
• Browse content in C0 - General
• C01 - Econometrics
• Browse content in C1 - Econometric and Statistical Methods and Methodology: General
• C10 - General
• C11 - Bayesian Analysis: General
• C12 - Hypothesis Testing: General
• C13 - Estimation: General
• C14 - Semiparametric and Nonparametric Methods: General
• C15 - Statistical Simulation Methods: General
• Browse content in C2 - Single Equation Models; Single Variables
• C21 - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions
• C22 - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
• C23 - Panel Data Models; Spatio-temporal Models
• C26 - Instrumental Variables (IV) Estimation
• Browse content in C3 - Multiple or Simultaneous Equation Models; Multiple Variables
• C31 - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
• C33 - Panel Data Models; Spatio-temporal Models
• C34 - Truncated and Censored Models; Switching Regression Models
• C35 - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions
• C36 - Instrumental Variables (IV) Estimation
• Browse content in C4 - Econometric and Statistical Methods: Special Topics
• C41 - Duration Analysis; Optimal Timing Strategies
• C44 - Operations Research; Statistical Decision Theory
• Browse content in C5 - Econometric Modeling
• C50 - General
• C51 - Model Construction and Estimation
• C52 - Model Evaluation, Validation, and Selection
• C53 - Forecasting and Prediction Methods; Simulation Methods
• C54 - Quantitative Policy Modeling
• C55 - Large Data Sets: Modeling and Analysis
• C57 - Econometrics of Games and Auctions
• Browse content in C6 - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling
• C61 - Optimization Techniques; Programming Models; Dynamic Analysis
• C62 - Existence and Stability Conditions of Equilibrium
• C63 - Computational Techniques; Simulation Modeling
• C67 - Input-Output Models
• Browse content in C7 - Game Theory and Bargaining Theory
• C70 - General
• C72 - Noncooperative Games
• C73 - Stochastic and Dynamic Games; Evolutionary Games; Repeated Games
• C78 - Bargaining Theory; Matching Theory
• Browse content in C8 - Data Collection and Data Estimation Methodology; Computer Programs
• C83 - Survey Methods; Sampling Methods
• Browse content in C9 - Design of Experiments
• C90 - General
• C91 - Laboratory, Individual Behavior
• C92 - Laboratory, Group Behavior
• C93 - Field Experiments
• Browse content in D - Microeconomics
• Browse content in D0 - General
• D01 - Microeconomic Behavior: Underlying Principles
• D02 - Institutions: Design, Formation, Operations, and Impact
• D03 - Behavioral Microeconomics: Underlying Principles
• Browse content in D1 - Household Behavior and Family Economics
• D11 - Consumer Economics: Theory
• D12 - Consumer Economics: Empirical Analysis
• D13 - Household Production and Intrahousehold Allocation
• D14 - Household Saving; Personal Finance
• D15 - Intertemporal Household Choice: Life Cycle Models and Saving
• D18 - Consumer Protection
• Browse content in D2 - Production and Organizations
• D21 - Firm Behavior: Theory
• D22 - Firm Behavior: Empirical Analysis
• D23 - Organizational Behavior; Transaction Costs; Property Rights
• D24 - Production; Cost; Capital; Capital, Total Factor, and Multifactor Productivity; Capacity
• D25 - Intertemporal Firm Choice: Investment, Capacity, and Financing
• D29 - Other
• Browse content in D3 - Distribution
• D30 - General
• D31 - Personal Income, Wealth, and Their Distributions
• Browse content in D4 - Market Structure, Pricing, and Design
• D40 - General
• D41 - Perfect Competition
• D42 - Monopoly
• D43 - Oligopoly and Other Forms of Market Imperfection
• D44 - Auctions
• D47 - Market Design
• Browse content in D5 - General Equilibrium and Disequilibrium
• D50 - General
• D51 - Exchange and Production Economies
• D52 - Incomplete Markets
• D53 - Financial Markets
• D57 - Input-Output Tables and Analysis
• D58 - Computable and Other Applied General Equilibrium Models
• Browse content in D6 - Welfare Economics
• D60 - General
• D61 - Allocative Efficiency; Cost-Benefit Analysis
• D62 - Externalities
• D63 - Equity, Justice, Inequality, and Other Normative Criteria and Measurement
• D64 - Altruism; Philanthropy
• Browse content in D7 - Analysis of Collective Decision-Making
• D70 - General
• D71 - Social Choice; Clubs; Committees; Associations
• D72 - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
• D73 - Bureaucracy; Administrative Processes in Public Organizations; Corruption
• D74 - Conflict; Conflict Resolution; Alliances; Revolutions
• D78 - Positive Analysis of Policy Formulation and Implementation
• Browse content in D8 - Information, Knowledge, and Uncertainty
• D80 - General
• D81 - Criteria for Decision-Making under Risk and Uncertainty
• D82 - Asymmetric and Private Information; Mechanism Design
• D83 - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
• D84 - Expectations; Speculations
• D85 - Network Formation and Analysis: Theory
• D86 - Economics of Contract: Theory
• Browse content in D9 - Micro-Based Behavioral Economics
• D90 - General
• D91 - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
• D92 - Intertemporal Firm Choice, Investment, Capacity, and Financing
• Browse content in E - Macroeconomics and Monetary Economics
• Browse content in E0 - General
• E03 - Behavioral Macroeconomics
• Browse content in E1 - General Aggregative Models
• E10 - General
• E12 - Keynes; Keynesian; Post-Keynesian
• E13 - Neoclassical
• E17 - Forecasting and Simulation: Models and Applications
• Browse content in E2 - Consumption, Saving, Production, Investment, Labor Markets, and Informal Economy
• E20 - General
• E21 - Consumption; Saving; Wealth
• E22 - Investment; Capital; Intangible Capital; Capacity
• E23 - Production
• E24 - Employment; Unemployment; Wages; Intergenerational Income Distribution; Aggregate Human Capital; Aggregate Labor Productivity
• E25 - Aggregate Factor Income Distribution
• E26 - Informal Economy; Underground Economy
• E27 - Forecasting and Simulation: Models and Applications
• Browse content in E3 - Prices, Business Fluctuations, and Cycles
• E30 - General
• E31 - Price Level; Inflation; Deflation
• E32 - Business Fluctuations; Cycles
• Browse content in E4 - Money and Interest Rates
• E40 - General
• E41 - Demand for Money
• E42 - Monetary Systems; Standards; Regimes; Government and the Monetary System; Payment Systems
• E43 - Interest Rates: Determination, Term Structure, and Effects
• E44 - Financial Markets and the Macroeconomy
• E49 - Other
• Browse content in E5 - Monetary Policy, Central Banking, and the Supply of Money and Credit
• E50 - General
• E51 - Money Supply; Credit; Money Multipliers
• E52 - Monetary Policy
• E58 - Central Banks and Their Policies
• Browse content in E6 - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook
• E60 - General
• E61 - Policy Objectives; Policy Designs and Consistency; Policy Coordination
• E62 - Fiscal Policy
• E65 - Studies of Particular Policy Episodes
• Browse content in E7 - Macro-Based Behavioral Economics
• E70 - General
• E71 - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on the Macro Economy
• Browse content in F - International Economics
• Browse content in F1 - Trade
• F10 - General
• F11 - Neoclassical Models of Trade
• F12 - Models of Trade with Imperfect Competition and Scale Economies; Fragmentation
• F13 - Trade Policy; International Trade Organizations
• F14 - Empirical Studies of Trade
• F15 - Economic Integration
• F16 - Trade and Labor Market Interactions
• F17 - Trade Forecasting and Simulation
• F18 - Trade and Environment
• F19 - Other
• Browse content in F2 - International Factor Movements and International Business
• F20 - General
• F21 - International Investment; Long-Term Capital Movements
• F22 - International Migration
• F23 - Multinational Firms; International Business
• Browse content in F3 - International Finance
• F30 - General
• F31 - Foreign Exchange
• F32 - Current Account Adjustment; Short-Term Capital Movements
• F33 - International Monetary Arrangements and Institutions
• F34 - International Lending and Debt Problems
• F36 - Financial Aspects of Economic Integration
• Browse content in F4 - Macroeconomic Aspects of International Trade and Finance
• F40 - General
• F41 - Open Economy Macroeconomics
• F42 - International Policy Coordination and Transmission
• F43 - Economic Growth of Open Economies
• F44 - International Business Cycles
• Browse content in F5 - International Relations, National Security, and International Political Economy
• F50 - General
• F53 - International Agreements and Observance; International Organizations
• Browse content in F6 - Economic Impacts of Globalization
• F60 - General
• F62 - Macroeconomic Impacts
• F63 - Economic Development
• F64 - Environment
• F65 - Finance
• Browse content in G - Financial Economics
• Browse content in G0 - General
• G01 - Financial Crises
• G02 - Behavioral Finance: Underlying Principles
• Browse content in G1 - General Financial Markets
• G10 - General
• G11 - Portfolio Choice; Investment Decisions
• G12 - Asset Pricing; Trading volume; Bond Interest Rates
• G13 - Contingent Pricing; Futures Pricing
• G14 - Information and Market Efficiency; Event Studies; Insider Trading
• G15 - International Financial Markets
• G18 - Government Policy and Regulation
• Browse content in G2 - Financial Institutions and Services
• G20 - General
• G21 - Banks; Depository Institutions; Micro Finance Institutions; Mortgages
• G22 - Insurance; Insurance Companies; Actuarial Studies
• G23 - Non-bank Financial Institutions; Financial Instruments; Institutional Investors
• G24 - Investment Banking; Venture Capital; Brokerage; Ratings and Ratings Agencies
• G28 - Government Policy and Regulation
• Browse content in G3 - Corporate Finance and Governance
• G30 - General
• G31 - Capital Budgeting; Fixed Investment and Inventory Studies; Capacity
• G32 - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill
• G33 - Bankruptcy; Liquidation
• G34 - Mergers; Acquisitions; Restructuring; Corporate Governance
• G38 - Government Policy and Regulation
• Browse content in G4 - Behavioral Finance
• G40 - General
• G41 - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making in Financial Markets
• Browse content in H - Public Economics
• Browse content in H0 - General
• H00 - General
• Browse content in H1 - Structure and Scope of Government
• H11 - Structure, Scope, and Performance of Government
• Browse content in H2 - Taxation, Subsidies, and Revenue
• H20 - General
• H21 - Efficiency; Optimal Taxation
• H23 - Externalities; Redistributive Effects; Environmental Taxes and Subsidies
• H24 - Personal Income and Other Nonbusiness Taxes and Subsidies; includes inheritance and gift taxes
• H25 - Business Taxes and Subsidies
• H26 - Tax Evasion and Avoidance
• Browse content in H3 - Fiscal Policies and Behavior of Economic Agents
• H30 - General
• H31 - Household
• Browse content in H4 - Publicly Provided Goods
• H41 - Public Goods
• Browse content in H5 - National Government Expenditures and Related Policies
• H50 - General
• H51 - Government Expenditures and Health
• H52 - Government Expenditures and Education
• H53 - Government Expenditures and Welfare Programs
• H55 - Social Security and Public Pensions
• H56 - National Security and War
• Browse content in H6 - National Budget, Deficit, and Debt
• H60 - General
• H63 - Debt; Debt Management; Sovereign Debt
• Browse content in H7 - State and Local Government; Intergovernmental Relations
• H71 - State and Local Taxation, Subsidies, and Revenue
• H75 - State and Local Government: Health; Education; Welfare; Public Pensions
• Browse content in H8 - Miscellaneous Issues
• H81 - Governmental Loans; Loan Guarantees; Credits; Grants; Bailouts
• Browse content in I - Health, Education, and Welfare
• Browse content in I0 - General
• I00 - General
• Browse content in I1 - Health
• I10 - General
• I11 - Analysis of Health Care Markets
• I12 - Health Behavior
• I13 - Health Insurance, Public and Private
• I14 - Health and Inequality
• I15 - Health and Economic Development
• I18 - Government Policy; Regulation; Public Health
• Browse content in I2 - Education and Research Institutions
• I20 - General
• I21 - Analysis of Education
• I22 - Educational Finance; Financial Aid
• I23 - Higher Education; Research Institutions
• I24 - Education and Inequality
• I25 - Education and Economic Development
• I26 - Returns to Education
• I28 - Government Policy
• Browse content in I3 - Welfare, Well-Being, and Poverty
• I30 - General
• I31 - General Welfare
• I32 - Measurement and Analysis of Poverty
• I38 - Government Policy; Provision and Effects of Welfare Programs
• I39 - Other
• Browse content in J - Labor and Demographic Economics
• Browse content in J0 - General
• J00 - General
• J01 - Labor Economics: General
• J08 - Labor Economics Policies
• Browse content in J1 - Demographic Economics
• J10 - General
• J11 - Demographic Trends, Macroeconomic Effects, and Forecasts
• J12 - Marriage; Marital Dissolution; Family Structure; Domestic Abuse
• J13 - Fertility; Family Planning; Child Care; Children; Youth
• J14 - Economics of the Elderly; Economics of the Handicapped; Non-Labor Market Discrimination
• J15 - Economics of Minorities, Races, Indigenous Peoples, and Immigrants; Non-labor Discrimination
• J16 - Economics of Gender; Non-labor Discrimination
• J17 - Value of Life; Forgone Income
• Browse content in J2 - Demand and Supply of Labor
• J20 - General
• J21 - Labor Force and Employment, Size, and Structure
• J22 - Time Allocation and Labor Supply
• J23 - Labor Demand
• J24 - Human Capital; Skills; Occupational Choice; Labor Productivity
• Browse content in J3 - Wages, Compensation, and Labor Costs
• J30 - General
• J31 - Wage Level and Structure; Wage Differentials
• J32 - Nonwage Labor Costs and Benefits; Retirement Plans; Private Pensions
• J33 - Compensation Packages; Payment Methods
• J38 - Public Policy
• Browse content in J4 - Particular Labor Markets
• J41 - Labor Contracts
• J42 - Monopsony; Segmented Labor Markets
• J44 - Professional Labor Markets; Occupational Licensing
• J47 - Coercive Labor Markets
• Browse content in J5 - Labor-Management Relations, Trade Unions, and Collective Bargaining
• J50 - General
• J52 - Dispute Resolution: Strikes, Arbitration, and Mediation; Collective Bargaining
• Browse content in J6 - Mobility, Unemployment, Vacancies, and Immigrant Workers
• J60 - General
• J61 - Geographic Labor Mobility; Immigrant Workers
• J62 - Job, Occupational, and Intergenerational Mobility
• J63 - Turnover; Vacancies; Layoffs
• J64 - Unemployment: Models, Duration, Incidence, and Job Search
• J65 - Unemployment Insurance; Severance Pay; Plant Closings
• J68 - Public Policy
• Browse content in J7 - Labor Discrimination
• J71 - Discrimination
• Browse content in J8 - Labor Standards: National and International
• J81 - Working Conditions
• J82 - Labor Force Composition
• J83 - Workers' Rights
• Browse content in K - Law and Economics
• Browse content in K0 - General
• K00 - General
• Browse content in K1 - Basic Areas of Law
• K14 - Criminal Law
• Browse content in K3 - Other Substantive Areas of Law
• K31 - Labor Law
• K33 - International Law
• K35 - Personal Bankruptcy Law
• Browse content in K4 - Legal Procedure, the Legal System, and Illegal Behavior
• K40 - General
• K41 - Litigation Process
• K42 - Illegal Behavior and the Enforcement of Law
• Browse content in L - Industrial Organization
• Browse content in L0 - General
• L00 - General
• Browse content in L1 - Market Structure, Firm Strategy, and Market Performance
• L10 - General
• L11 - Production, Pricing, and Market Structure; Size Distribution of Firms
• L12 - Monopoly; Monopolization Strategies
• L13 - Oligopoly and Other Imperfect Markets
• L14 - Transactional Relationships; Contracts and Reputation; Networks
• L15 - Information and Product Quality; Standardization and Compatibility
• Browse content in L2 - Firm Objectives, Organization, and Behavior
• L20 - General
• L22 - Firm Organization and Market Structure
• L23 - Organization of Production
• L25 - Firm Performance: Size, Diversification, and Scope
• Browse content in L3 - Nonprofit Organizations and Public Enterprise
• L31 - Nonprofit Institutions; NGOs; Social Entrepreneurship
• Browse content in L4 - Antitrust Issues and Policies
• L41 - Monopolization; Horizontal Anticompetitive Practices
• L42 - Vertical Restraints; Resale Price Maintenance; Quantity Discounts
• L43 - Legal Monopolies and Regulation or Deregulation
• Browse content in L5 - Regulation and Industrial Policy
• L50 - General
• L51 - Economics of Regulation
• Browse content in L6 - Industry Studies: Manufacturing
• L60 - General
• L62 - Automobiles; Other Transportation Equipment; Related Parts and Equipment
• L63 - Microelectronics; Computers; Communications Equipment
• Browse content in L7 - Industry Studies: Primary Products and Construction
• L71 - Mining, Extraction, and Refining: Hydrocarbon Fuels
• Browse content in L8 - Industry Studies: Services
• L81 - Retail and Wholesale Trade; e-Commerce
• L82 - Entertainment; Media
• L86 - Information and Internet Services; Computer Software
• Browse content in L9 - Industry Studies: Transportation and Utilities
• L93 - Air Transportation
• L94 - Electric Utilities
• L96 - Telecommunications
• Browse content in M - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics
• Browse content in M0 - General
• M00 - General
• Browse content in M1 - Business Administration
• M11 - Production Management
• M14 - Corporate Culture; Social Responsibility
• Browse content in M2 - Business Economics
• M21 - Business Economics
• Browse content in M3 - Marketing and Advertising
• M31 - Marketing
• M37 - Advertising
• Browse content in M5 - Personnel Economics
• M50 - General
• M51 - Firm Employment Decisions; Promotions
• M52 - Compensation and Compensation Methods and Their Effects
• M54 - Labor Management
• M55 - Labor Contracting Devices
• Browse content in N - Economic History
• Browse content in N1 - Macroeconomics and Monetary Economics; Industrial Structure; Growth; Fluctuations
• N10 - General, International, or Comparative
• N13 - Europe: Pre-1913
• Browse content in N2 - Financial Markets and Institutions
• N20 - General, International, or Comparative
• Browse content in N3 - Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy
• N31 - U.S.; Canada: Pre-1913
• N32 - U.S.; Canada: 1913-
• N33 - Europe: Pre-1913
• N34 - Europe: 1913-
• Browse content in N4 - Government, War, Law, International Relations, and Regulation
• N42 - U.S.; Canada: 1913-
• N43 - Europe: Pre-1913
• N44 - Europe: 1913-
• N45 - Asia including Middle East
• Browse content in N9 - Regional and Urban History
• N90 - General, International, or Comparative
• N92 - U.S.; Canada: 1913-
• N94 - Europe: 1913-
• Browse content in O - Economic Development, Innovation, Technological Change, and Growth
• Browse content in O1 - Economic Development
• O10 - General
• O11 - Macroeconomic Analyses of Economic Development
• O12 - Microeconomic Analyses of Economic Development
• O13 - Agriculture; Natural Resources; Energy; Environment; Other Primary Products
• O14 - Industrialization; Manufacturing and Service Industries; Choice of Technology
• O15 - Human Resources; Human Development; Income Distribution; Migration
• O16 - Financial Markets; Saving and Capital Investment; Corporate Finance and Governance
• O17 - Formal and Informal Sectors; Shadow Economy; Institutional Arrangements
• O18 - Urban, Rural, Regional, and Transportation Analysis; Housing; Infrastructure
• Browse content in O2 - Development Planning and Policy
• O23 - Fiscal and Monetary Policy in Development
• Browse content in O3 - Innovation; Research and Development; Technological Change; Intellectual Property Rights
• O30 - General
• O31 - Innovation and Invention: Processes and Incentives
• O32 - Management of Technological Innovation and R&D
• O33 - Technological Change: Choices and Consequences; Diffusion Processes
• O34 - Intellectual Property and Intellectual Capital
• O38 - Government Policy
• Browse content in O4 - Economic Growth and Aggregate Productivity
• O40 - General
• O41 - One, Two, and Multisector Growth Models
• O42 - Monetary Growth Models
• O43 - Institutions and Growth
• O44 - Environment and Growth
• O47 - Empirical Studies of Economic Growth; Aggregate Productivity; Cross-Country Output Convergence
• Browse content in O5 - Economywide Country Studies
• O51 - U.S.; Canada
• O55 - Africa
• Browse content in P - Economic Systems
• Browse content in P0 - General
• P00 - General
• Browse content in P2 - Socialist Systems and Transitional Economies
• P26 - Political Economy; Property Rights
• Browse content in Q - Agricultural and Natural Resource Economics; Environmental and Ecological Economics
• Browse content in Q1 - Agriculture
• Q15 - Land Ownership and Tenure; Land Reform; Land Use; Irrigation; Agriculture and Environment
• Q16 - R&D; Agricultural Technology; Biofuels; Agricultural Extension Services
• Browse content in Q3 - Nonrenewable Resources and Conservation
• Q33 - Resource Booms
• Browse content in Q4 - Energy
• Q41 - Demand and Supply; Prices
• Q43 - Energy and the Macroeconomy
• Browse content in Q5 - Environmental Economics
• Q51 - Valuation of Environmental Effects
• Q52 - Pollution Control Adoption Costs; Distributional Effects; Employment Effects
• Q53 - Air Pollution; Water Pollution; Noise; Hazardous Waste; Solid Waste; Recycling
• Q54 - Climate; Natural Disasters; Global Warming
• Q55 - Technological Innovation
• Q56 - Environment and Development; Environment and Trade; Sustainability; Environmental Accounts and Accounting; Environmental Equity; Population Growth
• Q58 - Government Policy
• Browse content in R - Urban, Rural, Regional, Real Estate, and Transportation Economics
• Browse content in R1 - General Regional Economics
• R10 - General
• R11 - Regional Economic Activity: Growth, Development, Environmental Issues, and Changes
• R12 - Size and Spatial Distributions of Regional Economic Activity
• R13 - General Equilibrium and Welfare Economic Analysis of Regional Economies
• R15 - Econometric and Input-Output Models; Other Models
• Browse content in R2 - Household Analysis
• R21 - Housing Demand
• R23 - Regional Migration; Regional Labor Markets; Population; Neighborhood Characteristics
• Browse content in R3 - Real Estate Markets, Spatial Production Analysis, and Firm Location
• R31 - Housing Supply and Markets
• R33 - Nonagricultural and Nonresidential Real Estate Markets
• Browse content in R4 - Transportation Economics
• R41 - Transportation: Demand, Supply, and Congestion; Travel Time; Safety and Accidents; Transportation Noise
• R48 - Government Pricing and Policy
• Browse content in R5 - Regional Government Analysis
• R51 - Finance in Urban and Rural Economies
• Browse content in Z - Other Special Topics
• Browse content in Z1 - Cultural Economics; Economic Sociology; Economic Anthropology
• Z10 - General
• Z12 - Religion
• Z13 - Economic Sociology; Economic Anthropology; Social and Economic Stratification
• Author Guidelines
• Submission Site
• Open Access
• About The Review of Economic Studies
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

About the journal

The Review  is essential reading for economists and has a reputation for publishing path-breaking papers in theoretical and applied economics.

Highly Cited Articles

The Review of Economic Studies  remains a highly read and cited publication by researchers, as demonstrated by our latest Impact Factor. To celebrate this continued success, we have curated a free collection of the most cited papers published in recent years. Use this collection to support your current work, or get up to date with important discussions in the field of economics. 

Read the papers

Why should you submit your paper to The Review of Economic Studies?

Watch this video now to find out more about the journal’s efficient, high-quality reviewing process from authors who have experienced it.

Celebrating Economics Impact Factors 

Oxford University Press publishes a portfolio of high impact economics journals, as reflected by our leading Impact Factors in the field. Browse our range of economics titles, which publish contemporary research on a variety of topics, and find the research that best supports your work.

Explore past  virtual issues  from The Review of Economic Studies. With topics including:

• The Roots and Consequences of Conflicts
• Health and Economics
• 30 years of the REStud Tour
• International Trade
• Family Economics

Latest articles

From oupblog.

It’s cheaper to preserve the Amazon than we might think

"The cattle need ladders to graze here.” That is what my wife’s relatives used to tell her after they moved to the Amazon rainforest. She visited their farm when she was 13, and the planted grass was taller than she was. Grass grows tall there because of the substantial amount of nutrients left on the ground immediately after deforesting...

Read the blog post | Read the article

What America’s history of mass migration can teach us about attitudes to immigrants

International migration is one of the most pervasive social phenomena of our times. According to recent UN estimates, as of 2017, there were almost 260 million migrants around the world...

Read the blog post  |  Read the article

Out with the old?

Innovation is a primary driver of economic growth and of the rise in living standards, and a substantial body of research has been devoted to documenting the welfare benefits from it.

Climate and the inequality of nations

Countries grow richer as one moves away from the equator, and the same is generally true if one looks at differences among regions within countries. However, this was not always the case...

Summary Turnaround Statistics

The average turnaround time for a paper is 53 days. 

Find out more about the journal's review speeds for first round submissions.

Excellence in Refereeing Prize 2022

The Review of Economic Studies  would like to thank all of the exceptional reviewers who have contributed their time and expertise to the journal. The winners of the Excellence in Refereeing award have been announced.

Explore the full list

Why Publish in The Review of Economic Studies ?

Email alerts

Register to receive table of contents email alerts as soon as new issues of The Review of Economic Studies are published online.

JEL Codes Explained

Articles from Oxford Journals economics titles are classified according to the system used by the Journal of Economic Literature (commonly known as 'JEL codes').

Recommend to your library

Fill out our simple online form to recommend The Review of Economic Studies to your library.

Recommend now

Trending articles

Discover the top Review of Economic Studies  articles sorted by Altmetric Score, mentioned online in the past three months.

Explore trending articles

Related Titles

• Recommend to your Library
• Journals Career Network

Affiliations

• Online ISSN 1467-937X
• Print ISSN 0034-6527
• Copyright © 2024 Review of Economic Studies Ltd
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Researching and writing for Economics students

4 literature review and citations/references.

Figure 4.1: Literature reviews and references

Your may have done a literature survey as part of your proposal. This will be incorporated into your dissertation, not left as separate stand-alone. Most economics papers include a literature review section, which may be a separate section, or incorporated into the paper’s introduction. (See organising for a standard format.)

Some disambiguation:

A ‘Literature survey’ paper: Some academic papers are called ‘literature surveys’. These try to summarise and discuss the existing work that has been done on a particular topic, and can be very useful. See, for example, works in The Journal of Economic Perspectives, the Journal of Economic Literature, the “Handbook of [XXX] Economics”

Many student projects and undergraduate dissertations are mainly literature surveys.

4.1 What is the point of a literature survey?

Your literature review should explain:

what has been done already to address your topic and related questions, putting your work in perspective, and

what techniques others have used, what are their strengths and weaknesses, and how might they be relevant tools for your own analysis.

Figure 4.2: Take notes on this as you read, and write them up.

4.2 What previous work is relevant?

Focus on literature that is relevant to your topic only.

But do not focus only on articles about your exact topic ! For example, if your paper is about the relative price of cars in the UK, you might cite papers (i) about the global automobile market, (ii) about the theory and evidence on competition in markets with similar features and (iii) using econometric techniques such as “hedonic regression” to estimate “price premia” in other markets and in other countries.

Consider: If you were Colchester a doctor and wanted to know whether a medicine would be effective for your patients, would you only consider medical studies that ran tests on Colchester residents, or would you consider more general national and international investigations?

4.3 What are “good” economics journal articles?

You should aim to read and cite peer-reviewed articles in reputable economics journals. (Journals in other fields such as Finance, Marketing and Political Science may also be useful.) These papers have a certain credibility as they have been checked by several referees and one or more editors before being published. (In fact, the publication process in Economics is extremely lengthy and difficult.)

Which journals are “reputable”? Economists spend a lot of time thinking about how to rank and compare journals (there are so many papers written about this topic that they someone could start a “Journal of Ranking Economics Journals”. For example, “ REPEC ” has one ranking, and SCIMAGO/SCOPUS has another one. You may want to focus on journals ranked in the top 100 or top 200 of these rankings. If you find it very interesting and relevant paper published somewhere that is ranked below this, is okay to cite it, but you may want to be a bit more skeptical of its findings.

Any journal you find on JSTOR is respectable, and if you look in the back of your textbooks, there will be references to articles in journals, most of which are decent.

You may also find unpublished “working papers”; these may also be useful as references. However, it is more difficult to evaluate the credibility of these, as they have not been through a process of peer review. However, if the author has published well and has a good reputation, it might be more likely that these are worth reading and citing.

Unpublished “working papers”

You may also find unpublished “working papers” or ‘mimeos’; these may also be useful as references. In fact, the publication process in Economics is so slow (six years from first working paper to publication is not uncommon) that not consulting working papers often means not being current.

However, it is more difficult to evaluate the credibility of this ‘grey literature’, as they have not been through a process of peer review. However, if the author has published well and has a good reputation, it might be more likely that these are worth reading and citing. Some working paper series are vetted, such as NBER; in terms of credibility, these might be seen as something in between a working paper and a publication.

Which of the following are “peer-reviewed articles in reputable economics journals”? Which of the following may be appropriate to cite in your literature review and in your final project? 8

Klein, G, J. (2011) “Cartel Destabilization and Leniency Programs – Empirical Evidence.” ZEW - Centre for European Economic Research Discussion Paper No. 10-107

Spencer, B. and Brander, J.A. (1983) “International R&D Rivalry and Industrial Strategy”, Review of Economic Studies Vol. 50, 707-722

Troisi, Jordan D., Andrew N. Christopher, and Pam Marek. “Materialism and money spending disposition as predictors of economic and personality variables.” North American Journal of Psychology 8.3 (2006): 421.

The Economist,. ‘Good, Bad And Ugly’. Web. 11 Apr. 2015. [accessed on…]

Mecaj, Arjola, and María Isabel González Bravo. “CSR Actions and Financial Distress: Do Firms Change Their CSR Behavior When Signals of Financial Distress Are Identified?.” Modern Economy 2014 (2014).

Universities, U. K. “Creating Prosperity: the role of higher education in driving the UK’s creative economy.” London Universities UK (2010).

4.4 How to find and access articles

You should be able to find and access all the relevant articles online. Leafing through bound volumes and photocopying should not be neededs. (Having been a student in the late 90’s and 2000’s, I wish I could get those hours back.)

Figure 4.3: The old way!

Good online tools include Jstor (jstor.org) and Google Scholar (scholar.google.co.uk). Your university should have access to Jstor, and Google is accessible to all (although the linked articles may require special access). You will usually have the ‘most access’ when logged into your university or library computing system.If you cannot access a paper, you may want to consult a reference librarian.

It is also ok, if you cannot access the journal article itself, to use the last working paper version (on Google scholar find this in the tab that says “all X versions”, where X is some number, and look for a PDF). However, authors do not always put up the most polished versions, although they should do to promote open-access. As a very last resort, you can e-mail the author and ask him or her to send you the paper.

When looking for references, try to find ones published in respected refereed economics journals (see above ).

4.5 Good starting points: Survey article, course notes, and textbooks

A “survey article” is a good place to start; this is a paper that is largely a categorization and discussion of previous work on a particular topic. You can often find such papers in journals such as

• the Journal of Economic Perspectives,
• the Journal of Economic Surveys,
• and the Journal of Economic Literature.

These will be useful as a “catalog” of papers to read and considers citing. They are also typically very readable and offer a decent introduction to the issue or the field.

It is also helpful to consult module (course) notes and syllabi from the relevant field. Do not only limit yourself to the ones at your own university; many of universities make their course materials publicly accessible online. These will not only typically contain reading lists with well-respected and useful references, they may also contain slides and other material that will help you better understand your topic and the relevant issues.

However, be careful not to take material from course notes without properly citing it. (Better yet, try to find the original paper that the course notes are referring to.)

Textbooks serve as another extremely useful jumping off point. Look through your own textbooks and other textbooks in the right fields. Textbooks draw from, and cite a range of relevant articles and papers. (You may also want to go back to textbooks when you are finding the articles you are reading too difficult. Textbooks may present a simpler version of the material presented in an article, and explain the concepts better.)

4.6 Backwards and forwards with references

When you find a useful paper, look for its “family.” You may want to go back to earlier, more fundamental references, by looking at the articles that this paper cited. See what is listed as “keywords” (these are usually given at the top of the paper), and “JEL codes”. Check what papers this paper cites, and check what other papers cited this paper. On Google scholar you can follow this with a link “Cited by…” below the listed article. “Related articles” is also a useful link.

4.7 Citations

Keep track of all references and citations

You may find it helpful to use software to help you manage your citations

A storage “database” of citations (e.g., Jabref, Zotero, Endnote, Mendeley); these interface well with Google Scholar and Jstor

An automatic “insert citation” and “insert bibliography” in your word processing software

Use a tool like Endnote to manage and insert the bibliographies, or use a bibliography manager software such as Zotero or Jabref,

Further discussion: Citation management tools

List of works cited

Put your list of references in alphabetical order by author’s last name (surname).

Include all articles and works that you cite in your paper; do not include any that you don’t cite.

Avoiding plagiarism and academic offenses**

Here is a definition of plagiarism

The main point is that you need to cite everything that is not your own work. Furthermore, be clear to distinguish what is your own work and your own language and what is from somewhere/someone else.

Why cite? Not just to give credit to others but to make it clear that the remaining uncited content is your own.

Here are some basic rules:

(Rephrased from University of Essex material, as seen in Department of Economics, EC100 Economics for Business Handbook 2017-18, https://www1.essex.ac.uk/economics/documents/EC100-Booklet_2017.pdf accessed on 20 July 2019, pp. 15-16)

Do not submit anything that is not your own work.

Never copy from friends.

Do not copy your own work or previously submitted work. (Caveat: If you are submitting a draft or a ‘literature review and project plan’ at an earlier stage, this can be incorporated into your final submission.

Don’t copy text directly into your work, unless:

• you put all passages in quotation marks: beginning with ’ and ending with ’, or clearly offset from the main text
• you cite the source of this text.

It is not sufficient merely to add a citation for the source of copied material following the copied material (typically the end of a paragraph). You must include the copied material in quotation marks. … Ignorance … is no defence.’ (ibid, pp. 15 )

(‘Ibid’ means ‘same as the previous citation’.)

Your university may use sophisticated plagiarism-detection software. Markers may also report if the paper looks suspect

Before final submission, they may ask you to go over your draft and sign that you understand the contents and you have demonstrated that the work is your own.

Not being in touch with your supervisor may put you under suspicion.

Your university may give a Viva Voce oral exam if your work is under suspicion. It is a cool-sounding word but probably something you want to avoid.

Your university may store your work in its our database, and can pursue disciplinary action, even after you have graduated.

Penalties may be severe, including failure with no opportunity to retake the module (course). You may even risk your degree!

Comprehension questions; answers in footnotes

True or false: “If you do not directly quote a paper you do not need to cite it” 9

You should read and cite a paper (choose all that are correct)… 10

• If it motivates ‘why your question is interesting’ and how it can be modeled economically
• Only if it asks the same question as your paper
• Only if it is dealing with the same country/industry/etc as you are addressing
• If it has any connection to your topic, question, or related matters
• If it answers a similar question as your paper
• If it uses and discusses techniques that inform those you are using

4.8 How to write about previous authors’ analysis and findings

Use the right terminology.

“Johnson et al. (2000) provide an analytical framework that sheds substantial doubt on that belief. When trying to obtain a correlation between institutional efficiency and wealth per capita, they are left with largely inconclusive results.”

They are not trying to “obtain a correlation”; they are trying to measure the relationship and test hypotheses.

“Findings”: Critically examine sources

Don’t take everything that is in print (or written online) as gospel truth. Be skeptical and carefully evaluate the arguments and evidence presented. Try to really survey what has been written, to consider the range of opinions and the preponderance of the evidence. You also need to be careful to distinguish between “real research” and propaganda or press releases.

The returns to higher education in Atlantis are extremely high. For the majority of Atlanian students a university degree has increased their lifetime income by over 50%, as reported in the “Benefits of Higher Education” report put out by the Association of Atlantian Universities (2016).

But don’t be harsh without explanation:

Smith (2014) found a return to education in Atlantis exceeding 50%. This result is unlikely to be true because the study was not a very good one.

“Findings:” “They Proved”

A theoretical economic model can not really prove anything about the real world; they typically rely on strong simplifying assumptions.

Through their economic model, they prove that as long as elites have incentives to invest in de facto power, through lobbying or corruption for example, they will invest as much as possible in order to gain favourable conditions in the future for their businesses.

In their two period model, which assumes \[details of key assumptions here\] , they find that when an elite Agent has an incentive to invest in de facto power, he invests a strictly positive amount, up to the point where marginal benefit equals marginal cost”

Empirical work does not “prove” anything (nor does it claim to).

It relies on statistical inference under specific assumptions, and an intuitive sense that evidence from one situation is likely to apply to other situations.

“As Smith et al (1999) proved using data from the 1910-1920 Scandanavian stock exchange, equity prices always increase in response to reductions in corporate tax rates.”

“Smith et al (199) estimated a VAR regression for a dynamic CAP model using data from the 1910-1920 Scandanavian stock exchange. They found a strongly statistically significant negative coefficient on corporate tax rates. This suggests that such taxes may have a negative effect on publicly traded securities. However, as their data was from a limited period with several simultaneous changes in policy, and their results are not robust to \[something here\] , further evidence is needed on this question.”

Use the language of classical 11 statistics:

Hypothesis testing, statistical significance, robustness checks, magnitudes of effects, confidence intervals.

Note that generalisation outside the data depends on an intuitive sense that evidence from one situation is likely to apply to other situations.

“Findings”: How do you (or the cited paper) claim to identify a causal relationship?

This policy was explained by Smith and Johnson (2002) in their research on subsidies and redistribution in higher education. Their results showed that people with higher degree have higher salaries and so pay higher taxes. Thus subsidizing higher education leads to a large social gain.

The results the student discusses seem to show an association between higher degrees and higher salaries. The student seems to imply that the education itself led to higher salaries. This has not been shown by the cited paper. Perhaps people who were able to get into higher education would earn higher salaries anyway. There are ways economists used to try to identify a “causal effect” (by the way, this widely used term is redundant as all effects must have a cause), but a mere association between two variables is not enough

As inflation was systematically lower during periods of recession, we see that too low a level of inflation increases unemployment.

Economists have long debated the nature of this “Phillips curve” relationship. There is much work trying to determine whether the association (to the extent it exists) is a causal one. We could not rule out reverse causality, or third factor that might cause changes in both variables.

4.9 …Stating empirical results

Don’t write: “I accept the null hypothesis.”

Do write: “The results fail to reject the null hypothesis, in spite of a large sample size and an estimate with small standard errors” (if this is the case)

Note: The question of what to infer from acceptance/rejection of null hypotheses is a complex difficult one in Classical (as opposed to Bayesian) statistics. This difficulty is in part philosophical: classical hypothesis testing is deductive , while inference is necessarily inductive.

4.10 What to report

You need to read this paper more clearly; it is not clear what they conclude nor what their evidence is.

4.11 Organising your literature review

A common marking comment:

These papers seem to be discussed in random order – you need some structure organising these papers thematically, by finding, by technique, or chronologically perhaps.

How should you organise it? In what order?

Thematically (usually better)

By method, by theoretical framework, by results or assumptions, by field

Chronologically (perhaps within themes)

Exercise: Compare how the literature review section is organized in papers you are reading.

Figure 4.4: Organising a set of references

Q: What sort of structure am I using in the above outline?

It may also be helpful to make a ‘table’ of the relevant literature, as in the figure below. This will help you get a sense of the methods and results, and how the papers relate, and how to assess the evidence. You may end up putting this in the actual paper.

Figure 4.5: Organisational table from Reinstein and Riener, 2012b

4.12 What if you have trouble reading and understanding a paper?

Consult a survey paper, textbook, or lecture notes that discuss this paper and this topic

Try to find an easier related paper

Ask your supervisor for help; if he or she can

Try to understand what you can; do not try to “fake it”

4.13 Some literature survey do’s and don’ts

Do not cite irrelevant literature.

Do not merely list all the papers you could find.

Discuss them, and their relevance to your paper.

What are their strengths and weaknesses? What techniques do they use, and what assumptions do they rely on? How do they relate to each other?

Use correct citation formats.

Try to find original sources (don’t just cite a web link).

Don’t just cut and paste from other sources. And make sure to attribute every source and every quote. Be clear: which part of your paper is your own work and what is cited from others? The penalties for plagiarism can be severe!

• Critically examine the sources, arguments, and methods

4.14 Comprehension questions: literature review

How to discuss empirical results: “Causal” estimation, e.g., with Instrumental Variables

Which is the best way to state it? 12

“As I prove in table 2, more lawyers lead to slower growth (as demonstrated by the regression analysis evidence).”

“Table 2 provides evidence that a high share of lawyers in a city’s population leads to slower growth.”

3.“Table 2 shows that a high share of lawyers in a city’s population is correlated with slower growth.”

Which is better? 13

• “However, when a set of observable determinants of city growth (such as Census Region growth) are accounted for, the estimate of this effect becomes less precise.”
• “In the correct regression I control for all determinants of city growth and find that there is no effect of lawyers on growth”

Stating empirical results: descriptive

“Using the US data from 1850-1950, I find that inflation is lower during periods of recession. This is statistically significant in a t-test [or whatever test] at the 99% level, and the difference is economically meaningful. This is consistent with the theory of …, which predicts that lower inflation increases unemployment. However, other explanations are possible, including reverse causality, and unmeasured covarying lags and trends.”

“I find a significantly lower level of inflation during periods of recession, and the difference is economically meaningful. This relationship is statistically significant and the data is accurately measured. Thus I find that inflation increases unemployment.”

Some tips on writing a good paper– relevant to literature reviews

• Answer the question
• Provide clear structure and signposting
• Demonstrate an ability for critical analysis
• Refer to your sources
• Produce a coherent, clear argument
• Take time to proofread for style and expresssion
• Source “Assignment Writing Skills EBS 3rd year 2012”"

Answer: only b is a ‘peer reviewed article in a reputable economics journal’. All of these might be useful to cite, however. ↩

False. You need to cite any content and ideas that are not your own. ↩

Answers: 1, 5, and 6. Note that 2 and 3 are too narrow criteria, and 4 is too broad. ↩

or Bayesian if you like ↩

The second one; if this is really causal evidence. ↩

The first one. There is no ‘correct regression’. It is also not really correct in classical statistics to ‘find no effect’. ↩

• Grand Valley State University

University Libraries

Search the Library for Articles, Books, and More Find It!

More research tools:

• Subject Guides
• Find Materials
• ECO 495: Senior Economic Project
• Literature Review

ECO 495: Senior Economic Project: Literature Review

• Internet Resources
• Databases and Journals
• Citing Data

Definition of Literature Reviews

A literature review surveys books, scholarly articles, and any other sources relevant to your research topic or thesis statement. It should provide a theoretical summary or critical evaluation of these scholarly works. You will need to analyze, evaluate , and s ynthesize the research that you’ve found on your topic. A literature review should give context to your thesis and, if possible, reveal any gaps in current literature. 

5 Steps for completing your Literature Review

• Look at other literature reviews
• Choose a topic that interests you
• This is important to do otherwise the literature you find will be too massive 
• You can look for books using the Library Search
• ​Preferably in EconLit
• Find sources from the reference papers that relate to your topic
• Search for those items by typing in the titles in the University Libraries “ Find It ” box
• I recommend keeping notes with Evernote because it is Open Source and once you create an account you can access it from any device
• I recommend Zotero , again it is Open Source and accessible from anywhere
• Once you've collected, read, noted, and saved your citations and resources you should begin to see patterns
• ​​Skim your notes to sort out themes (methodologies, data, results, etc.) 
• Does a topic develop over time 
• Do authors agree with each other or disagree on methodology or conclusions
• What strengths or weaknesses did you find in the literature
• Don't forget that you're trying to relate this literature to the story you wish to tell and you may find some of your articles fall out of your scope--make note of that to determine whether to mention them or not--talk to your professor about out of scope titles
• Make an outline or structural form of your review
• Remember your audience when writing
• Avoid too much jargon
• ​Be concise; don't go off on tangents; stay focused on your thesis statement
• Your purpose for writing the review
• Overview of the problem
• What is the scope of your review
• Talk about the amount of literature you found
• Chronological order
• Advancements of theories 
• Questions related to topic
• Summarize your findings
• Expose gaps in knowledge
• Provide a rationale for future research
• References in APA style

What is in a Literature Review?

A literature review may consist of simply a summary of key sources, but i t usually has an organizational pattern and combines both summary and synthesis , often within specific conceptual categories. 

• A summary is a recap of the important information of the source
• A synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem.

What is the purpose of a literature review?

• To demonstrate to your readers what you know about your topic
• To bring your readers up-to-date and fill them in on what has been published on your topic
• To allow you a better understanding of your topic

Lit. Review, How-to videos

• << Previous: Welcome
• Next: Internet Resources >>
• Last Updated: Apr 17, 2024 1:21 PM
• URL: https://libguides.gvsu.edu/ECO495

Login to LibApps

Text Us! (616) 818-0219

Mary Idema Pew Library (616) 331-3500 [email protected]

Steelcase Library (616) 331-6799

Frey Foundation Learning Commons (616) 331-5933

Social Media

Committed to Equality

Federal Depository Library Program

• GVSU is an AA/EO Institution
• Privacy Policy
• Disclosures
• Copyright © 2018 GVSU

An Overview of Key Indicators and Evaluation Tools for Assessing Economic Value of Heritage Towns: A Literature Review

• REVIEW PAPER
• Published: 10 May 2024

Cite this article

• Varsha Vinod   ORCID: orcid.org/0000-0003-1153-2865 1 ,
• Satyaki Sarkar   ORCID: orcid.org/0000-0002-5161-2344 1 &
• Supriyo Roy   ORCID: orcid.org/0000-0003-0600-0696 2  

Explore all metrics

Heritage towns not only have a significant concentration of heritage structures, but are also the centre of age-old local economic activities. It has been widely acknowledged that such historic cores have a unique identity accompanied by a distinct townscape that is worthy of preservation. In the urge to develop, heritage of such towns often takes a backseat allowing space for new development. The management of value and potential of cultural heritage can be significant for community development and economic prominence, thereby boosting the quality of life in this state of constant evolution. With cultural heritage as a catalyst, there are numerous possibilities to accomplish integrated and inclusive development in heritage towns. This study attempts to understand the role of heritage in urban development of heritage towns and identify the prime movers of heritage economy. In order to identify key indicators, evaluation tools and methods used to determine the economic value of heritage towns, systematic literature review method has been used. The review unfolds the emergence of aspects such as social, physical, economic and cultural importance along with the predominant heritage value that contribute towards the heritage economy. Factors such as business environment, infrastructure, funding, quality of environment etc. are found to have notable influence in the heritage ecosystem. Hence, the economic evaluation of heritage town needs to be comprehensive in nature to plan for further heritage-led urban economic development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

K.B. Bhavin Shukla, Development in the context of urban conservation: examining the interrelationship between architectural heritage and intangible culture in historic settlements. CEPT Archives, Ahmedabad, 2014

M. Panasyuk et al., Historical heritage factor in evaluating development prospects of the regional multicultural community. Soc. Behav. Sci. 188 , 193–196 (2015)

Article   Google Scholar  

Y.B. Frédérick Brousseau-Gauthier, A heritage economy: business model innovation for economic wealth, social well-being, and environmental health. Technol. Innov. Manag. Rev. 3 (9), 32–37 (2013)

U. Halim, N. Misha, H. Halimin, N.M. Yusof, Heritage tourism and its impact on the local communities: a case study in Ranau, Sabah. Int. J. Acad. Res. Bus. Soc. Sci 11 (12), 1805–1830 (2021)

Google Scholar  

Guidelines for sustainable cultural tourism in historic towns and cities. EU Sustainable Tourism Guidelines, 2009

S. Marius-Răzvan, L. Ioana-Alexandra, A. Stroe, V. Valentina, Innovative valuing of the cultural heritage assets. Economic implication on local employability, small entrepreneurship development and social inclusion. Procedia Soc. Behav. Sci. 188 , 16–26 (2015)

C. Vegheș, Cultural heritage, sustainable development and inclusive growth: global lessons for the local communities under a marketing approach. Eur. J. Sustain. Dev. 7 (4), 349–360 (2018)

V. Vasile, M.R. Surugiu, I.A. Login, Heritage entrepreneurship and ecotourism—a new vision on ecosystem protection and in-situ specific activities for cultural heritage consumption. Econ. Finance 42 , 139–154 (2016)

I. Aranburu, A. Fierro, Airbnb branding: heritage as a branding element in the sharing economy. Sustainability 11 (74), 1–14 (2018)

S. Muhuri, P.S. Mishra, Value assessment of existing architectural heritage for future generation using criteria importance through inter-criteria correlation and grey relational analysis method: a case of Odisha temple architecture in India. Curr. Sci. (2021)

V.I. Veklenko, M.M. Ali, M.A. Alghafri, Cultural heritage management as a generator of socio-economic development (the case study of Syria). SHS Web Conf. 76 , 01025 (2020)

L.D. Spina, Multidimensional assessment for “Culture-Led” and “Community-Driven” urban regeneration as driver for trigger economic vitality in urban historic centers. Sustainability 11 (24), 7237 (2019)

I. Ianoș, A.L. Cercleux, G.L. Merciu, F.-C. Merciu, Evaluation of the economic values of urban heritage in the central area of Ploiesti municipality. in International Conference of Knowledge based Organisation , 2020

C. Cheong, R.Mason, Donovan R. Rypkema, Measuring economic impacts of historic preservation. Advisory Council on Historic Preservation by PlaceEconomics, 2013

G. Reher, Heritage as action search. Sustainabilty 14 (282), 1–9 (2021)

K.S. Smith, Cultural heritage tourism in Washington, DC: a community-based model for neighborhood economic development. Glob. Urban Dev. 4 (1), 1–13 (2008)

D. Papathanasiou-Zuhrt, The know-how booklet for cultural heritage operators. Ciberespacio, Madrid. 36 (2012)

R. Mason, Assessing values in conservation planning : methodological issues and choices. Research report: The Getty Conservation Institute, Los Angeles, 2002

O. Muzychenko-Kozlovska, M. Gvozd, W. Sroka, N. Shpak, Simulation of the influence of external factors on the level of use of the regional tourism potential: a practical aspect. Adm. Sci. 11 (3), 85 (2021)

D. Babic, Social responsible heritage management—empowering citizens to act as heritage managers. Soc. Behav. Sci. 188 , 27–34 (2015)

L.D. Spina, Adaptive sustainable reuse for cultural heritage: a multiple criteria decision aiding approach supporting urban development processes. Sustainability 12 (4), 1363 (2020)

M. Ottenbacher, T. Mukhambetov, Cluster approach in cultural heritage tourism (case of the Central Asian section of silk road). Cent. Asian J. Soc. Sci. Humanit. 7 (1), 49–70 (2021)

E. Summary, Economic impact of heritage preservation in Texas. University of Texas, Austin and Edward J Blousetein School of Plnning and Public Policy, 2015

Y. Dai, A behavioral cultural-based development analysis of entrepreneurship in China. Sustainability 11 (3), 91 (2021)

A. Zednik, J.E. Araña, D. Throsby, Public preferences for heritage conservation strategies: a choice modelling approach. J. Cult. Econ. 45 (3), 333–358 (2021)

R. Mason, Assessing values in conservation planning : methodological issues and choices. The Getty Convention, 2002

N. Aayog, Improving heritage management in India. Government of India, 2019

E. U. H. 2. research, Social platform for holistic heritage impact assessment, 2020

M. Bosone, L. Fusco Girard, Antonia Gravagnuolo, Methodologies for impact assessment of cultural heritage adaptive reuse. Report for European Union's Horizon 2020 Research, 2021

F. Appendino, Heritage-related indicators for urban sustainable development: a systematic review. HAL Open Sci. 2021

D.F. Weiss-Ibáñez, D. Papathanasiou-Zuhrt, Cognitive processing of information with visitor value in cultural heritage environments. Procedia Econ. Finance 15 , 303–311 (2014)

L. Zhao, J. Huang, A. Law, Y. Li, Research frameworks, methodologies, and assessment methods concerning the adaptive reuse of architectural heritage: a review. Built Herit. 5 , 1–19 (2021)

W. Wang, J. Zhou, Z. Zhang, Lu. Zixian, Z. Gong, J. Zhou, Management effectiveness evaluation of world cultural landscape heritage: a case from China. Herit. Sci. 10 (1), 10–22 (2022)

A. Bond, J. Teller, Ana Pereira Roders, Determining effectiveness in heritage impact assessments. in 33rd Annual Meeting of the International Association for Impact Assessment , 2013

T. Wandschneider, Small rural towns and local economic development: Evidence from two poor states in India, 2004

S.J. Daluddung, Community benchmarks: an analysis of performance measurements in urban planning management. Portland State University, 2005

C. Koboldt, Optimizing the use of cultural heritage. in Economic Perspectives of Cultural Heritage Conference , 1995

B. Sudhakara Reddy, P. Balachandra, Benchmarking Bangalore City for sustainability—an indicator-based approach, 2013

L. O. Edmunds, Maximizing the local economic impact of heritage conservation for cultural tourism. in Asia Tourism Forum 2016 – The 12th Biennial Conference of Hospitality and Tourism Industry in Asia , 2013

A.P. Russo, M. Lavanga, G. Mingardo, J. van der Borg, The impacts of culture on the economic development of cities. European Institute for Comparative Urban Research, 2005

H. S. T.-HS2, Wider economic impact assessment with a S-CGE model. 2022

B.H. Roberts, Tool kit for rapid economic assessment, planning, and development of cities in Asia. Asian Development Bank, 2015

M. Kloos, C. Neugebauer, B. Ashrafi, Heritage impact assessment, beyond an assessment tool: a comparative analysis of urban development impact on visual integrity in four UNESCO world heritage properties. J. Cult. Herit. 47 , 199–207 (2020)

P. Economics, Measuring the economics of preservation: recent findings. Advisory Council on Historic Preservation, 2011

M.E. Alinejad, Z. Razaghi, The role of tourism in economic development. J. Am. Sci. 8 (8), 139–144 (2012)

E. Corporation, The economic impact of historic preservation in Philadelphia. Preservation Alliance for Greater Philadelphia, 2010

C. Grodach, A. Loukaitou‐Sideris, Cultural development strategies and urban revitalization. Int. J. Cult. Policy 13 (4), 349–370 (2007)

A.I. Petrişor, G.L. Merciu, F.-C. Merciu, Economic valuation of cultural heritage using the travel cost method: the historical centre of the municipality of bucharest as a case study. Heritage 4 (3), 2356–2376 (2021)

J. Rollo, Y. Esteban, H. Tong, X. Yin, Y. Xu, Developing a comprehensive assessment model of social value with respect to heritage value for sustainable heritage management. Sustainability 13 (23), 13373 (2021)

I.G.P.C.W. Group, Economic and social impacts of cultural and creative sectors. OECD, 2021

D. Throsby, Investment in urban heritage. Urban Development Series Knowledge Papers, 2012

G. J. R. J. Anushree Deb, Socio-economics of urban heritage in India. Working Paper, 2013

M. Karamidehkordi, M. Karbasioun, A. Papzan, Qualitative analysis of tourism difficulties in Yasechah village: going toward rural development in ChaharMahal VA Bakhtiary, Iran by using grounded theory. J. Am. Sci. 8 (6), 280–286 (2012)

C. Surugiu, M.-R. Surugiua, Heritage tourism entrepreneurship and social media: opportunities and challenges. Soc. Behav. Sci. 188 , 74–81 (2015)

M. Morris, The economic impact of historic resource preservation. California Cultural and Historical Endowment, 2012

I. Rašić Bakarić, S. Slijepčević, D.J. Alibegović, Impact assessment of entrepreneurial zones on local economic outcomes. Econ. Res. 32 (1), 3112–3127 (2019)

V. Delgado, J.L. Arumi, O. Reicher, Use of indicators in strategic environmental assessments of urban-planning instruments: a case study. Sustainability 13 (22), 12639 (2021)

B. Nguza-Mduba, R. Garidzirai, Does tourism contribute to local economic development (LED) in the City of Cape Town municipality? A time series analysis. Afr. J. Hosp. Tour. Leis. 9 (1), 1–13 (2021)

A.P. Roders, B.J.F. Colenbrander, P.C. Guzmán, Measuring links between cultural heritage management and sustainable urban development: an overview of global monitoring tools. Cities 60 , 192–201 (2016)

M. Mazzanti, S. Mourato, Economic valuation of cultural heritage: evidence and prospects. The Getty Convention, 2002

E.a.A.D. Department of Rural, Heritage impact assessment report. Taung Skull Fossil, 2015

P. De Toro, L. Fusco Girard, A. Gravagnuolo, S. Iodice, M. Bosone, Indicators for ex-post evaluation of cultural heritage adaptive reuse impacts in the perspective of the circular economy. Sustainability 13 (9), 4759 (2021)

B. Rodawski, A. Sztando, J. Ładysz, L. Kurowski, Selected methods of estimation of the cultural heritage economic value with the special reference to historical town districts adaptation. in Urban Heritage : Research, Interpretation, Education (2007), pp. 13–18

UNESCO, Historic urban landscape. UNESCO, 2011

A.M. Alghafri, The effect of cultural heritage sites in local economic development—the case study of Syria. IUPR (Russian Journal) 2 (57), 3–9 (2019)

S.A. Murray, B. Korte, C.J. Dey, M. Lenzen, Environmental impact assessment including indirect effects—a case study using input–output analysis. Environ. Impact Assess. Rev. 23 (3), 263–282 (2003)

B.W. Ritchie, F. Papandrea, J. Bennett, A.S. Choi, Economic valuation of cultural heritage sites: a choice modeling approach. Tour. Manag. 31 (2), 213–220 (2010)

T.K. Gumede, Heritage tourism as a strategy for the local economic development in KwaZulu-Natal, South Africa. Afr. J. Hosp. Tour. Leisure 8 (3), 1–15 (2019)

S. Caurla, V. Bellon-Maurel, E. Loiseau, T. Beaussier, Coupling economic models and environmental assessment methods to 4 support regional policies: a critical review. J. Clean. Prod. 216 , 408–421 (2019)

D.D. Rypkema, Heritage conservation and the local economy. Glob. Urban Dev. 4 (1), 1–8 (2008)

B.P.f.B. Lives, Approaches to impact analysis. OECD, 2015

K.P. David Throsby, Assessing the impacts of heritage-led urban rehabilitation the case of George Town, Penang, Malaysia. Research Report—Getty Conservation Institute, 2022

D. Holland, P. Hong, H.Y. Li, C. Altshuler, the world economic forecasting model at the United Nations. Department of Economic and Social Affairs, United Nations, 2016

Download references

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and affiliations.

Department of Architecture and Planning, Birla Institute of Technology, Mesra, Jharkhand, India

Varsha Vinod & Satyaki Sarkar

Department of Management, Birla Institute of Technology, Mesra, Jharkhand, India

Supriyo Roy

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Varsha Vinod .

Ethics declarations

Conflict of interest.

On behalf of all authors, the corresponding author declares that there is no conflict of interest regarding the publication of this manuscript.

Ethical Approval

On behalf of all authors, the corresponding author declares that the accepted principles of ethical at study had been followed, and there are no financial or personal conflicts of interest that might have impacted the research reported in this publication.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Vinod, V., Sarkar, S. & Roy, S. An Overview of Key Indicators and Evaluation Tools for Assessing Economic Value of Heritage Towns: A Literature Review. J. Inst. Eng. India Ser. A (2024). https://doi.org/10.1007/s40030-024-00807-3

Download citation

Received : 19 July 2023

Accepted : 24 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1007/s40030-024-00807-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Heritage towns
• Cultural heritage economy
• Evaluation methods
• Key indicators
• Literature review
• Find a journal
• Publish with us
• Track your research

Artificial Intelligence for Quality of Life Study: A Systematic Literature Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Open access
• Published: 09 May 2024

Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research

• Jingcheng Du 1 ,
• Ekin Soysal 1 , 3 ,
• Dong Wang 2 ,
• Long He 1 ,
• Bin Lin 1 ,
• Jingqi Wang 1 ,
• Frank J. Manion 1 ,
• Yeran Li 2 ,
• Elise Wu 2 &
• Lixia Yao 2  

BMC Medical Research Methodology volume  24 , Article number:  108 ( 2024 ) Cite this article

143 Accesses

3 Altmetric

Metrics details

Systematic literature reviews (SLRs) are critical for life-science research. However, the manual selection and retrieval of relevant publications can be a time-consuming process. This study aims to (1) develop two disease-specific annotated corpora, one for human papillomavirus (HPV) associated diseases and the other for pneumococcal-associated pediatric diseases (PAPD), and (2) optimize machine- and deep-learning models to facilitate automation of the SLR abstract screening.

This study constructed two disease-specific SLR screening corpora for HPV and PAPD, which contained citation metadata and corresponding abstracts. Performance was evaluated using precision, recall, accuracy, and F1-score of multiple combinations of machine- and deep-learning algorithms and features such as keywords and MeSH terms.

Results and conclusions

The HPV corpus contained 1697 entries, with 538 relevant and 1159 irrelevant articles. The PAPD corpus included 2865 entries, with 711 relevant and 2154 irrelevant articles. Adding additional features beyond title and abstract improved the performance (measured in Accuracy) of machine learning models by 3% for HPV corpus and 2% for PAPD corpus. Transformer-based deep learning models that consistently outperformed conventional machine learning algorithms, highlighting the strength of domain-specific pre-trained language models for SLR abstract screening. This study provides a foundation for the development of more intelligent SLR systems.

Peer Review reports

Introduction

Systematic literature reviews (SLRs) are an essential tool in many areas of health sciences, enabling researchers to understand the current knowledge around a topic and identify future research and development directions. In the field of health economics and outcomes research (HEOR), SLRs play a crucial role in synthesizing evidence around unmet medical needs, comparing treatment options, and preparing the design and execution of future real-world evidence studies. SLRs provide a comprehensive and transparent analysis of available evidence, allowing researchers to make informed decisions and improve patient outcomes.

Conducting a SLR involves synthesizing high-quality evidence from biomedical literature in a transparent and reproducible manner, and seeks to include all available evidence on a given research question, and provides some assessment regarding quality of the evidence [ 1 , 2 ]. To conduct an SLR one or more bibliographic databases are queried based on a given research question and a corresponding set of inclusion and exclusion criteria, resulting in the selection of a relevant set of abstracts. The abstracts are reviewed, further refining the set of articles that are used to address the research question. Finally, appropriate data is systematically extracted from the articles and summarized [ 1 , 3 ].

The current approach to conducting a SLR is through manual review, with data collection, and summary done by domain experts against pre-specified eligibility criteria. This is time-consuming, labor-intensive, expensive, and non-scalable given the current more-than linear growth of the biomedical literature [ 4 ]. Michelson and Reuter estimate that each SLR costs approximately $141,194.80 and that on average major pharmaceutical companies conduct 23.36 SLRs, and major academic centers 177.32 SLRs per year, though the cost may vary based on the scope of different reviews [ 4 ]. Clearly automated methods are needed, both from a cost/time savings perspective, and for the ability to effectively scan and identify increasing amounts of literature, thereby allowing the domain experts to spend more time analyzing the data and gleaning the insights.

One major task of SLR project that involves large amounts of manual effort, is the abstract screening task. For this task, selection criteria are developed and the citation metadata and abstract for articles tentatively meeting these criteria are retrieved from one or more bibliographic databases (e.g., PubMed). The abstracts are then examined in more detail to determine if they are relevant to the research question(s) and should be included or excluded from further consideration. Consequently, the task of determining whether articles are relevant or not based on their titles, abstracts and metadata can be treated as a binary classification task, which can be addressed by natural language processing (NLP). NLP involves recognizing entities and relationships expressed in text and leverages machine-learning (ML) and deep-learning (DL) algorithms together with computational semantics to extract information. The past decade has witnessed significant advances in these areas for biomedical literature mining. A comprehensive review on how NLP techniques in particular are being applied for automatic mining and knowledge extraction from biomedical literature can be found in Zhao et al. [ 5 ].

Materials and methods

The aims of this study were to: (1) identify and develop two disease-specific corpora, one for human papillomavirus (HPV) associated diseases and the other for pneumococcal-associated pediatric diseases suitable for training the ML and DL models underlying the necessary NLP functions; (2) investigate and optimize the performance of the ML and DL models using different sets of features (e.g., keywords, Medical Subject Heading (MeSH) terms [ 6 ]) to facilitate automation of the abstract screening tasks necessary to construct a SLR. Note that these screening corpora can be used as training data to build different NLP models. We intend to freely share these two corpora with the entire scientific community so they can serve as benchmark corpora for future NLP model development in this area.

SLR corpora preparation

Two completed disease-specific SLR studies by Merck & Co., Inc., Rahway, NJ, USA were used as the basis to construct corpora for abstract-level screening. The two SLR studies were both relevant to health economics and outcome research, including one for human papillomavirus (HPV) associated diseases (referred to as the HPV corpus), and one for pneumococcal-associated pediatric diseases (which we refer to as the PAPD corpus). Both of the original SLR studies contained literature from PubMed/MEDLINE and EMBASE. Since we intended for the screening corpora to be released to the community, we only kept citations found from PubMed/MEDLINE in the finalized corpora. Because the original SLR studies did not contain the PubMed ID (PMID) for each article, we matched each article’s citation information (if available) against PubMed and then collected meta-data such as authors, journals, keywords, MeSH terms, publication types, etc., using PubMed Entrez Programming Utilities (E-utilities) Application Programming Interface (API). The detailed description of the two corpora can be seen in Table  1 . Both of the resulting corpora are publicly available at [ https://github.com/Merck/NLP-SLR-corpora ].

Machine learning algorithms

Although deep learning algorithms have demonstrated superior performance on many NLP tasks, conventional machine learning algorithms have certain advantages, such as low computation costs and faster training and prediction speed.

We evaluated four traditional ML-based document classification algorithms, XGBoost [ 7 ], Support Vector Machines (SVM) [ 8 ], Logistic regression (LR) [ 9 ], and Random Forest [ 10 ] on the binary inclusion/exclusion classification task for abstract screening. Salient characteristics of these models are as follows:

XGBoost: Short for “eXtreme Gradient Boosting”, XGBoost is a boosting-based ensemble of algorithms that turn weak learners into strong learners by focusing on where the individual models went wrong. In Gradient Boosting, individual weak models train upon the difference between the prediction and the actual results [ 7 ]. We set max_depth at 3, n_estimators at 150 and learning rate at 0.7.

Support vector machine (SVM): SVM is one of the most robust prediction methods based on statistical learning frameworks. It aims to find a hyperplane in an N-dimensional space (where N = the number of features) that distinctly classifies the data points [ 8 ]. We set C at 100, gamma at 0.005 and kernel as radial basis function.

Logistic regression (LR): LR is a classic statistical model that in its basic form uses a logistic function to model a binary dependent variable [ 9 ]. We set C at 5 and penalty as l2.

Random forest (RF): RF is a machine learning technique that utilizes ensemble learning to combine many decision trees classifiers through bagging or bootstrap aggregating [ 10 ]. We set n_estimators at 100 and max_depth at 14.

These four algorithms were trained for both the HPV screening task and the PAPD screening task using the corresponding training corpus.

For each of the four algorithms, we examined performance using (1) only the baseline feature criteria (title and abstract of each article), and (2) with five additional meta-data features (MeSH, Authors, Keywords, Journal, Publication types.) retrieved from each article using the PubMed E-utilities API. Conventionally, title and abstract are the first information a human reviewer would depend on when making a judgment for inclusion or exclusion of an article. Consequently, we used title and abstract as the baseline features to classify whether an abstract should be included at the abstract screening stage. We further evaluated the performance with additional features that can be retrieved by PubMed E-utilities API, including MeSH terms, authors, journal, keywords and publication type. For baseline evaluation, we concatenated the titles and abstracts and extracted the TF-IDF (term frequency-inverse document frequency) vector for the corpus. TF-IDF evaluates how relevant a word is to a document in a collection of documents. For additional features, we extracted TF-IDF vector using each feature respectively and then concatenated the extracted vectors with title and abstract vector. XGBoost was selected for the feature evaluation process, due to its relatively quick computational running time and robust performance.

Deep learning algorithms

Conventional ML methods rely heavily on manually designed features and suffer from the challenges of data sparsity and poor transportability when applied to new use cases. Deep learning (DL) is a set of machine learning algorithms based on deep neural networks that has advanced performance of text classification along with many other NLP tasks. Transformer-based deep learning models, such as BERT (Bidirectional encoder representations from transformers), have achieved state-of-the-art performance in many NLP tasks [ 11 ]. A Transformer is an emerging architecture of deep learning models designed to handle sequential input data such as natural language by adopting the mechanisms of attention to differentially weigh the significance of each part of the input data [ 12 ]. The BERT model and its variants (which use Transformer as a basic unit) leverage the power of transfer learning by first pre-training the models over 100’s of millions of parameters using large volumes of unlabeled textual data. The resulting model is then fine-tuned for a particular downstream NLP application, such as text classification, named entity recognition, relation extraction, etc. The following three BERT models were evaluated against both the HPV and Pediatric pneumococcal corpus using two sets of features (title and abstract versus adding all additional features into the text). For all BERT models, we used Adam optimizer with weight decay. We set learning rate at 1e-5, batch size at 8 and number of epochs at 20.

BERT base: this is the original BERT model released by Google. The BERT base model was pre-trained on textual data in the general domain, i.e., BooksCorpus (800 M words) and English Wikipedia (2500 M words) [ 11 ].

BioBERT base: as the biomedical language is different from general language, the BERT models trained on general textual data may not work well on biomedical NLP tasks. BioBERT was further pre-trained (based on original BERT models) in the large-scale biomedical corpora, including PubMed abstracts (4.5B words) and PubMed Central Full-text articles (13.5B words) [ 13 ].

PubMedBERT: PubMedBERT was pre-trained from scratch using abstracts from PubMed. This model has achieved state-of-the-art performance on several biomedical NLP tasks on Biomedical Language Understanding and Reasoning Benchmark [ 14 ].

Text pre-processing and libraries that were used

We have removed special characters and common English words as a part of text pre-processing. Default tokenizer from scikit-learn was adopted for tokenization. Scikit-learn was also used for TF-IDF feature extraction and machine learning algorithms implementation. Transformers libraries from Hugging Face were used for deep learning algorithms implementation.

Evaluation datasets were constructed from the HPV and Pediatric pneumococcal corpora and were split into training, validation and testing sets with a ratio of 8:1:1 for the two evaluation tasks: (1) ML algorithms performance assessment; and (2) DL algorithms performance assessment. Models were fitted on the training sets, and model hyperparameters were optimized on the validation sets and the performance were evaluated on the testing sets. The following major metrics are expressed by the noted calculations:

Where True positive is an outcome where the model correctly predicts the positive (e.g., “included” in our tasks) class. Similarly, a True negative is an outcome where the model correctly predicts the negative class (e.g., “excluded” in our tasks). False positive is an outcome where the model incorrectly predicts the positive class, and a False negative is an outcome where the model incorrectly predicts the negative class. We have repeated all experiments five times and reported the mean scores with standard deviation.

Table  2 shows the baseline comparison using different feature combinations for the SLR text classification tasks using XGBoost. As noted, adding additional features in addition to title and abstract was effective in further improving the classification accuracy. Specifically, using all available features for the HPV classification increased accuracy by ? ∼  3% and F1 score by ? ∼  3%; using all available features for Pediatric pneumococcal classification increased accuracy by ? ∼  2% and F1 score by ? ∼  4%. As observed, adding additional features provided a stronger boost in precision, which contributed to the overall performance improvement.

The comparison of the article inclusion/exclusion classification task for four machine learning algorithms with all features is shown in Table  3 . XGBoost achieved the highest accuracy and F-1 scores in both tasks. Table  4 shows the comparison between XGBoost and deep learning algorithms on the classification tasks for each disease. Both XGBoost and deep learning models consistently have achieved higher accuracy scores when using all features as input. Among all models, BioBERT has achieved the highest accuracy at 0.88, compared with XGBoost at 0.86. XGBoost has the highest F1 score at 0.8 and the highest recall score at 0.9 for inclusion prediction.

Discussions and conclusions

Abstract screening is a crucial step in conducting a systematic literature review (SLR), as it helps to identify relevant citations and reduces the effort required for full-text screening and data element extraction. However, screening thousands of abstracts can be a time-consuming and burdensome task for scientific reviewers. In this study, we systematically investigated the use of various machine learning and deep learning algorithms, using different sets of features, to automate abstract screening tasks. We evaluated these algorithms using disease-focused SLR corpora, including one for human papillomavirus (HPV) associated diseases and another for pneumococcal-associated pediatric diseases (PADA). The publicly available corpora used in this study can be used by the scientific community for advanced algorithm development and evaluation. Our findings suggest that machine learning and deep learning algorithms can effectively automate abstract screening tasks, saving valuable time and effort in the SLR process.

Although machine learning and deep learning algorithms trained on the two SLR corpora showed some variations in performance, there were also some consistencies. Firstly, adding additional citation features significantly improved the performance of conventional machine learning algorithms, although the improvement was not as strong in transformer-based deep learning models. This may be because transformer models were mostly pre-trained on abstracts, which do not include additional citation information like MeSH terms, keywords, and journal names. Secondly, when using only title and abstract as input, transformer models consistently outperformed conventional machine learning algorithms, highlighting the strength of subject domain-specific pre-trained language models. When all citation features were combined as input, conventional machine learning algorithms showed comparable performance to deep learning models. Given the much lower computation costs and faster training and prediction time, XGBoost or support vector machines with all citation features could be an excellent choice for developing an abstract screening system.

Some limitations remain for this study. Although we’ve evaluated cutting-edge machine learning and deep learning algorithms on two SLR corpora, we did not conduct much task-specific customization to the learning algorithms, including task-specific feature engineering and rule-based post-processing, which could offer additional benefits to the performance. As the focus of this study is to provide generalizable strategies for employing machine learning to abstract screening tasks, we leave the task-specific customization to future improvement. The corpora we evaluated in this study mainly focus on health economics and outcome research, the generalizability of learning algorithms to another domain will benefit from formal examination.

Extensive studies have shown the superiority of transformer-based deep learning models for many NLP tasks [ 11 , 13 , 14 , 15 , 16 ]. Based on our experiments, however, adding features to the pre-trained language models that have not seen these features before may not significantly boost their performance. It would be interesting to find a better way of encoding additional features to these pre-trained language models to maximize their performance. In addition, transfer learning has proven to be an effective technique to improve the performance on a target task by leveraging annotation data from a source task [ 17 , 18 , 19 ]. Thus, for a new SLR abstract screening task, it would be worthwhile to investigate the use of transfer learning by adapting our (publicly available) corpora to the new target task.

When labeled data is available, supervised machine learning algorithms can be very effective and efficient for article screening. However, as there is increasing need for explainability and transparency in NLP-assisted SLR workflow, supervised machine learning algorithms are facing challenges in explaining why certain papers fail to fulfill the criteria. The recent advances in large language models (LLMs), such as ChatGPT [ 20 ] and Gemini [ 21 ], show remarkable performance on NLP tasks and good potentials in explainablity. Although there are some concerns on the bias and hallucinations that LLMs could bring, it would be worthwhile to evaluate further how LLMs could be applied to SLR tasks and understand the performance of using LLMs to take free-text article screening criteria as the input and provide explainanation for article screening decisions.

Data availability

The annotated corpora underlying this article are available at https://github.com/Merck/NLP-SLR-corpora .

Bullers K, Howard AM, Hanson A, et al. It takes longer than you think: librarian time spent on systematic review tasks. J Med Libr Assoc. 2018;106:198–207. https://doi.org/10.5195/jmla.2018.323 .

Article   PubMed   PubMed Central   Google Scholar  

Carver JC, Hassler E, Hernandes E et al. Identifying Barriers to the Systematic Literature Review Process. In: 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement . 2013. 203–12. https://doi.org/10.1109/ESEM.2013.28 .

Lame G. Systematic literature reviews: an introduction. Proc Des Society: Int Conf Eng Des. 2019;1:1633–42. https://doi.org/10.1017/dsi.2019.169 .

Article   Google Scholar  

Michelson M, Reuter K. The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun. 2019;16:100443. https://doi.org/10.1016/j.conctc.2019.100443 .

Recent advances in. biomedical literature mining | Briefings in Bioinformatics | Oxford Academic. https://academic.oup.com/bib/article/22/3/bbaa057/5838460?login=true (accessed 30 May 2022).

Medical Subject Headings - Home Page. https://www.nlm.nih.gov/mesh/meshhome.html (accessed 30 May 2022).

Chen T, Guestrin C, XGBoost:. A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York, NY, USA: Association for Computing Machinery 2016. 785–94. https://doi.org/10.1145/2939672.2939785 .

Noble WS. What is a support vector machine? Nat Biotechnol. 2006;24:1565–7. https://doi.org/10.1038/nbt1206-1565 .

Article   CAS   PubMed   Google Scholar  

Logistic Regression . https://doi.org/10.1007/978-1-4419-1742-3 (accessed 30 May 2022).

Random forest classifier for remote sensing classification. International Journal of Remote Sensing: Vol 26, No 1. https://www.tandfonline.com/doi/abs/10.1080/01431160412331269698 (accessed 30 May 2022).

Devlin J, Chang M-W, Lee K, et al. BERT: pre-training of Deep Bidirectional transformers for Language understanding. arXiv. 2019. https://doi.org/10.48550/arXiv.1810.04805 .

Vaswani A, Shazeer N, Parmar N et al. Attention is All you Need. In: Advances in Neural Information Processing Systems . Curran Associates, Inc. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html (accessed 30 May 2022).

BioBERT. a pre-trained biomedical language representation model for biomedical text mining | Bioinformatics | Oxford Academic. https://academic.oup.com/bioinformatics/article/36/4/1234/5566506 (accessed 3 Jun 2020).

Gu Y, Tinn R, Cheng H, et al. Domain-specific Language Model Pretraining for Biomedical Natural Language Processing. ACM Trans Comput Healthc. 2021;3(2):1–2. https://doi.org/10.1145/3458754 .

Article   CAS   Google Scholar  

Chen Q, Du J, Allot A, et al. LitMC-BERT: transformer-based multi-label classification of biomedical literature with an application on COVID-19 literature curation. arXiv. 2022. https://doi.org/10.48550/arXiv.2204.08649 .

Chen Q, Allot A, Leaman R, et al. Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations. arXiv. 2022. https://doi.org/10.48550/arXiv.2204.09781 .

Kermany DS, Goldbaum M, Cai W, et al. Identifying Medical diagnoses and Treatable diseases by Image-based deep learning. Cell. 2018;172:1122–e11319. https://doi.org/10.1016/j.cell.2018.02.010 .

Howard J, Ruder S. Universal Language Model fine-tuning for text classification. arXiv. 2018. https://doi.org/10.48550/arXiv.1801.06146 .

Do CB, Ng AY. Transfer learning for text classification. In: Advances in Neural Information Processing Systems . MIT Press. 2005. https://proceedings.neurips.cc/paper/2005/hash/bf2fb7d1825a1df3ca308ad0bf48591e-Abstract.html (accessed 30 May 2022).

Achiam J et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).

https:// gemini.google.com/app/a4dcd2e2d7672354 . (accessed 01 Feb 2024).

Download references

Acknowledgements

We thank Dr. Majid Rastegar-Mojarad for conducting some additional experiments during revision.

This research was supported by Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc., Rahway, NJ, USA.

Author information

Authors and affiliations.

Intelligent Medical Objects, Houston, TX, USA

Jingcheng Du, Ekin Soysal, Long He, Bin Lin, Jingqi Wang & Frank J. Manion

Merck & Co., Inc, Rahway, NJ, USA

Dong Wang, Yeran Li, Elise Wu & Lixia Yao

McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA

Ekin Soysal

You can also search for this author in PubMed   Google Scholar

Contributions

Study concept and design: JD and LY Corpus preparation: DW, YL and LY Experiments: JD and ES Draft of the manuscript: JD, DW, FJM and LY Acquisition, analysis, or interpretation of data: JD, ES, DW and LY Critical revision of the manuscript for important intellectual content: JD, ES, DW, LH, BL, JW, FJM, YL, EW, LY Study supervision: LY.

Corresponding author

Correspondence to Lixia Yao .

Ethics declarations

Disclaimers.

The content is the sole responsibility of the authors and does not necessarily represent the official views of Merck & Co., Inc., Rahway, NJ, USA or Intelligent Medical Objects.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Competing interests.

DW is an employee of Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc., Rahway, NJ, USA. EW, YL, and LY were employees of Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc., Rahway, NJ, USA for this work. JD, LH, JW, and FJM are employees of Intelligent Medical Objects. ES was an employee of Intelligent Medical Objects during his contributions, and is currently an employee of EBSCO Information Services. All the other authors declare no competing interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Du, J., Soysal, E., Wang, D. et al. Machine learning models for abstract screening task - A systematic literature review application for health economics and outcome research. BMC Med Res Methodol 24 , 108 (2024). https://doi.org/10.1186/s12874-024-02224-3

Download citation

Received : 19 May 2023

Accepted : 18 April 2024

Published : 09 May 2024

DOI : https://doi.org/10.1186/s12874-024-02224-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Machine learning
• Deep learning
• Text classification
• Article screening
• Systematic literature review

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]