how is a bibliography useful in research

How to Write a Bibliography for a Research Paper

Do not try to “wow” your instructor with a long bibliography when your instructor requests only a works cited page. It is tempting, after doing a lot of work to research a paper, to try to include summaries on each source as you write your paper so that your instructor appreciates how much work you did. That is a trap you want to avoid. MLA style, the one that is most commonly followed in high schools and university writing courses, dictates that you include only the works you actually cited in your paper—not all those that you used.

Academic Writing, Editing, Proofreading, And Problem Solving Services

Get 10% off with 24start discount code, assembling bibliographies and works cited.

• If your assignment calls for a bibliography, list all the sources you consulted in your research.
• If your assignment calls for a works cited or references page, include only the sources you quote, summarize, paraphrase, or mention in your paper.
• If your works cited page includes a source that you did not cite in your paper, delete it.
• All in-text citations that you used at the end of quotations, summaries, and paraphrases to credit others for their ideas,words, and work must be accompanied by a cited reference in the bibliography or works cited. These references must include specific information about the source so that your readers can identify precisely where the information came from.The citation entries on a works cited page typically include the author’s name, the name of the article, the name of the publication, the name of the publisher (for books), where it was published (for books), and when it was published.

The good news is that you do not have to memorize all the many ways the works cited entries should be written. Numerous helpful style guides are available to show you the information that should be included, in what order it should appear, and how to format it. The format often differs according to the style guide you are using. The Modern Language Association (MLA) follows a particular style that is a bit different from APA (American Psychological Association) style, and both are somewhat different from the Chicago Manual of Style (CMS). Always ask your teacher which style you should use.

A bibliography usually appears at the end of a paper on its own separate page. All bibliography entries—books, periodicals, Web sites, and nontext sources such radio broadcasts—are listed together in alphabetical order. Books and articles are alphabetized by the author’s last name.

Most teachers suggest that you follow a standard style for listing different types of sources. If your teacher asks you to use a different form, however, follow his or her instructions. Take pride in your bibliography. It represents some of the most important work you’ve done for your research paper—and using proper form shows that you are a serious and careful researcher.

Bibliography Entry for a Book

A bibliography entry for a book begins with the author’s name, which is written in this order: last name, comma, first name, period. After the author’s name comes the title of the book. If you are handwriting your bibliography, underline each title. If you are working on a computer, put the book title in italicized type. Be sure to capitalize the words in the title correctly, exactly as they are written in the book itself. Following the title is the city where the book was published, followed by a colon, the name of the publisher, a comma, the date published, and a period. Here is an example:

Format : Author’s last name, first name. Book Title. Place of publication: publisher, date of publication.

• A book with one author : Hartz, Paula.  Abortion: A Doctor’s Perspective, a Woman’s Dilemma . New York: Donald I. Fine, Inc., 1992.
• A book with two or more authors : Landis, Jean M. and Rita J. Simon.  Intelligence: Nature or Nurture?  New York: HarperCollins, 1998.

Bibliography Entry for a Periodical

A bibliography entry for a periodical differs slightly in form from a bibliography entry for a book. For a magazine article, start with the author’s last name first, followed by a comma, then the first name and a period. Next, write the title of the article in quotation marks, and include a period (or other closing punctuation) inside the closing quotation mark. The title of the magazine is next, underlined or in italic type, depending on whether you are handwriting or using a computer, followed by a period. The date and year, followed by a colon and the pages on which the article appeared, come last. Here is an example:

Format:  Author’s last name, first name. “Title of the Article.” Magazine. Month and year of publication: page numbers.

• Article in a monthly magazine : Crowley, J.E.,T.E. Levitan and R.P. Quinn.“Seven Deadly Half-Truths About Women.”  Psychology Today  March 1978: 94–106.
• Article in a weekly magazine : Schwartz, Felice N.“Management,Women, and the New Facts of Life.”  Newsweek  20 July 2006: 21–22.
• Signed newspaper article : Ferraro, Susan. “In-law and Order: Finding Relative Calm.”  The Daily News  30 June 1998: 73.
• Unsigned newspaper article : “Beanie Babies May Be a Rotten Nest Egg.”  Chicago Tribune  21 June 2004: 12.

Bibliography Entry for a Web Site

For sources such as Web sites include the information a reader needs to find the source or to know where and when you found it. Always begin with the last name of the author, broadcaster, person you interviewed, and so on. Here is an example of a bibliography for a Web site:

Format : Author.“Document Title.” Publication or Web site title. Date of publication. Date of access.

Example : Dodman, Dr. Nicholas. “Dog-Human Communication.”  Pet Place . 10 November 2006.  23 January 2014 < http://www.petplace.com/dogs/dog-human-communication-2/page1.aspx >

After completing the bibliography you can breathe a huge sigh of relief and pat yourself on the back. You probably plan to turn in your work in printed or handwritten form, but you also may be making an oral presentation. However you plan to present your paper, do your best to show it in its best light. You’ve put a great deal of work and thought into this assignment, so you want your paper to look and sound its best. You’ve completed your research paper!

Back to  How To Write A Research Paper .

ORDER HIGH QUALITY CUSTOM PAPER

• Utility Menu

fa3d988da6f218669ec27d6b6019a0cd

A publication of the harvard college writing program.

Harvard Guide to Using Sources 

• The Honor Code
• Bibliography

If you are using Chicago style footnotes or endnotes, you should include a bibliography at the end of your paper that provides complete citation information for all of the sources you cite in your paper. Bibliography entries are formatted differently from notes. For bibliography entries, you list the sources alphabetically by last name, so you will list the last name of the author or creator first in each entry. You should single-space within a bibliography entry and double-space between them. When an entry goes longer than one line, use a hanging indent of .5 inches for subsequent lines. Here’s a link to a sample bibliography that shows layout and spacing . You can find a sample of note format here .

Complete note vs. shortened note

Here’s an example of a complete note and a shortened version of a note for a book:

1. Karen Ho, Liquidated: An Ethnography of Wall Street (Durham: Duke University Press, 2009), 27-35.

1. Karen Ho, Liquidated , 27-35.

Note vs. Bibliography entry

The bibliography entry that corresponds with each note is very similar to the longer version of the note, except that the author’s last and first name are reversed in the bibliography entry. To see differences between note and bibliography entries for different types of sources, check this section of the Chicago Manual of Style .

For Liquidated , the bibliography entry would look like this:

Ho, Karen, Liquidated: An Ethnography of Wall Street . Durham: Duke University Press, 2009.

Citing a source with two or three authors

If you are citing a source with two or three authors, list their names in your note in the order they appear in the original source. In the bibliography, invert only the name of the first author and use “and” before the last named author.

1. Melissa Borja and Jacob Gibson, “Internationalism with Evangelical Characteristics: The Case of Evangelical Responses to Southeast Asian Refugees,” The Review of Faith & International Affairs 17, no. 3 (2019): 80-81, https://doi.org/10.1080/15570274.2019.1643983 .

Shortened note:

1. Borja and Gibson, “Internationalism with Evangelical Characteristics,” 80-81.

Bibliography:

Borja, Melissa, and Jacob Gibson. “Internationalism with Evangelical Characteristics: The Case of Evangelical Responses to Southeast Asian Refugees.” The Review of Faith & International Affairs 17. no. 3 (2019): 80–93. https://doi.org/10.1080/15570274.2019.1643983 .

Citing a source with more than three authors

If you are citing a source with more than three authors, include all of them in the bibliography, but only include the first one in the note, followed by et al. ( et al. is the shortened form of the Latin et alia , which means “and others”).

1. Justine M. Nagurney, et al., “Risk Factors for Disability After Emergency Department Discharge in Older Adults,” Academic Emergency Medicine 27, no. 12 (2020): 1271.

Short version of note:

1. Justine M. Nagurney, et al., “Risk Factors for Disability,” 1271.

Nagurney, Justine M., Ling Han, Linda Leo‐Summers, Heather G. Allore, Thomas M. Gill, and Ula Hwang. “Risk Factors for Disability After Emergency Department Discharge in Older Adults.” Academic Emergency Medicine 27, no. 12 (2020): 1270–78. https://doi.org/10.1111/acem.14088 .

Citing a book consulted online

If you are citing a book you consulted online, you should include a URL, DOI, or the name of the database where you found the book.

1. Karen Ho, Liquidated: An Ethnography of Wall Street (Durham: Duke University Press, 2009), 27-35, https://doi-org.ezp-prod1.hul.harvard.edu/10.1215/9780822391371 .

Bibliography entry:

Ho, Karen. Liquidated: An Ethnography of Wall Street . Durham: Duke University Press, 2009. https://doi-org.ezp-prod1.hul.harvard.edu/10.1215/9780822391371 .

Citing an e-book consulted outside of a database

If you are citing an e-book that you accessed outside of a database, you should indicate the format. If you read the book in a format without fixed page numbers (like Kindle, for example), you should not include the page numbers that you saw as you read. Instead, include chapter or section numbers, if possible.

1. Karen Ho, Liquidated: An Ethnography of Wall Street (Durham: Duke University Press, 2009), chap. 2, Kindle.

Ho, Karen. Liquidated: An Ethnography of Wall Street . Durham: Duke University Press, 2009. Kindle.

• Citation Management Tools
• In-Text Citations
• Examples of Commonly Cited Sources
• Frequently Asked Questions about Citing Sources in Chicago Format
• Sample Bibliography

PDFs for This Section

• Citing Sources
• Online Library and Citation Tools

Citation Guide

• What is a Citation?
• Citation Generator
• Chicago/Turabian Style
• Paraphrasing and Quoting
• Examples of Plagiarism

What is a Bibliography?

What is an annotated bibliography, introduction to the annotated bibliography.

• Writing Center
• Writer's Reference Center
• Helpful Tutorials
• the authors' names
• the titles of the works
• the names and locations of the companies that published your copies of the sources
• the dates your copies were published
• the page numbers of your sources (if they are part of multi-source volumes)

Ok, so what's an Annotated Bibliography?

An annotated bibliography is the same as a bibliography with one important difference: in an annotated bibliography, the bibliographic information is followed by a brief description of the content, quality, and usefulness of the source. For more, see the section at the bottom of this page.

What are Footnotes?

Footnotes are notes placed at the bottom of a page. They cite references or comment on a designated part of the text above it. For example, say you want to add an interesting comment to a sentence you have written, but the comment is not directly related to the argument of your paragraph. In this case, you could add the symbol for a footnote. Then, at the bottom of the page you could reprint the symbol and insert your comment. Here is an example:

This is an illustration of a footnote. 1 The number “1” at the end of the previous sentence corresponds with the note below. See how it fits in the body of the text? 1 At the bottom of the page you can insert your comments about the sentence preceding the footnote.

When your reader comes across the footnote in the main text of your paper, he or she could look down at your comments right away, or else continue reading the paragraph and read your comments at the end. Because this makes it convenient for your reader, most citation styles require that you use either footnotes or endnotes in your paper. Some, however, allow you to make parenthetical references (author, date) in the body of your work.

Footnotes are not just for interesting comments, however. Sometimes they simply refer to relevant sources -- they let your reader know where certain material came from, or where they can look for other sources on the subject. To decide whether you should cite your sources in footnotes or in the body of your paper, you should ask your instructor or see our section on citation styles.

Where does the little footnote mark go?

Whenever possible, put the footnote at the end of a sentence, immediately following the period or whatever punctuation mark completes that sentence. Skip two spaces after the footnote before you begin the next sentence. If you must include the footnote in the middle of a sentence for the sake of clarity, or because the sentence has more than one footnote (try to avoid this!), try to put it at the end of the most relevant phrase, after a comma or other punctuation mark. Otherwise, put it right at the end of the most relevant word. If the footnote is not at the end of a sentence, skip only one space after it.

What's the difference between Footnotes and Endnotes?

The only real difference is placement -- footnotes appear at the bottom of the relevant page, while endnotes all appear at the end of your document. If you want your reader to read your notes right away, footnotes are more likely to get your reader's attention. Endnotes, on the other hand, are less intrusive and will not interrupt the flow of your paper.

If I cite sources in the Footnotes (or Endnotes), how's that different from a Bibliography?

Sometimes you may be asked to include these -- especially if you have used a parenthetical style of citation. A "works cited" page is a list of all the works from which you have borrowed material. Your reader may find this more convenient than footnotes or endnotes because he or she will not have to wade through all of the comments and other information in order to see the sources from which you drew your material. A "works consulted" page is a complement to a "works cited" page, listing all of the works you used, whether they were useful or not.

Isn't a "works consulted" page the same as a "bibliography," then?

Well, yes. The title is different because "works consulted" pages are meant to complement "works cited" pages, and bibliographies may list other relevant sources in addition to those mentioned in footnotes or endnotes. Choosing to title your bibliography "Works Consulted" or "Selected Bibliography" may help specify the relevance of the sources listed.

This information has been freely provided by plagiarism.org and can be reproduced without the need to obtain any further permission as long as the URL of the original article/information is cited. 

How Do I Cite Sources? (n.d.) Retrieved October 19, 2009, from http://www.plagiarism.org/plag_article_how_do_i_cite_sources.html

The Importance of an Annotated Bibliography

An Annotated Bibliography is a collection of annotated citations. These annotations contain your executive notes on a source. Use the annotated bibliography to help remind you of later of the important parts of an article or book. Putting the effort into making good notes will pay dividends when it comes to writing a paper!

Good Summary

Being an executive summary, the annotated citation should be fairly brief, usually no more than one page, double spaced.

• Focus on summarizing the source in your own words.
• Avoid direct quotations from the source, at least those longer than a few words. However, if you do quote, remember to use quotation marks. You don't want to forget later on what is your own summary and what is a direct quotation!
• If an author uses a particular term or phrase that is important to the article, use that phrase within quotation marks. Remember that whenever you quote, you must explain the meaning and context of the quoted word or text. 

Common Elements of an Annotated Citation

• Summary of an Article or Book's thesis or most important points (Usually two to four sentences)
• Summary of a source's methodological approach. That is, what is the source? How does it go about proving its point(s)? Is it mostly opinion based? If it is a scholarly source, describe the research method (study, etc.) that the author used. (Usually two to five sentences)
• Your own notes and observations on the source beyond the summary. Include your initial analysis here. For example, how will you use this source? Perhaps you would write something like, "I will use this source to support my point about . . . "
• Formatting Annotated Bibliographies This guide from Purdue OWL provides examples of an annotated citation in MLA and APA formats.

• << Previous: Examples of Plagiarism
• Next: ACM Style >>
• Last Updated: May 2, 2024 1:23 PM
• URL: https://libguides.limestone.edu/citation

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

31 Bibliography

Annotated bibliography.

A  bibliography  is an alphabetized list of sources showing the author, date, and publication information for each source.

An  annotation  is like a note; it’s a brief paragraph that explains what the writer learned from the source.

Annotated bibliographies combine bibliographies and brief notes about the sources.

Writers often create annotated bibliographies as a part of a research project, as a means of recording their thoughts and deciding which sources to actually use to support the purpose of their research. Some writers include annotated bibliographies at the end of a research paper as a way of offering their insights about the source’s usability to their readers.

Instructors in college often assign annotated bibliographies as a means of helping students think through their source’s quality and appropriateness to their research question or topic.  (23)

Formatting the Annotated Bibliography

The  citations  (bibliographic information – title, date, author, publisher, etc.) in the annotated bibliography are formatted using the particular style manual (APA, MLA, Chicago, etc.) that your discipline requires.

Annotations  are written in paragraph form, usually 3-7 sentences (or 80-200 words). Depending on your assignment your annotations will generally include the following:

• Summary:  Summarize the information given in the source. Note the intended audience. What are the main arguments? What is the point of this book or article? What topics are covered? If someone asked what this article/book is about, what would you say?
• Evaluate/Assess:  Is this source credible? Who wrote it? What are their credentials? Who is the publisher? Is it a useful source? How does it compare with other sources in your bibliography? Is the information reliable? Is this source biased or objective? What is the goal of this source?
• Reflect/React:  Once you’ve summarized and assessed a source, you need to ask how it fits into your research. State your reaction and any additional questions you have about the information in your source. Was this source helpful to you? How does it help you shape your argument? How can you use this source in your research project? Has it changed how you think about your topic? Compare each source to other sources in your annotated bibliography in terms of its usefulness and thoroughness in helping answer your research question.  (24)

Annotated Bibliography Examples

In the following examples, the bold font indicates the reflection component of the annotation that is sometimes required in an assignment.

APA style 6  th  edition for the journal citation:

Waite, L. J., Goldschneider, F. K., & Witsberger, C. (1986). Nonfamily living and the erosion of traditional family orientations among young adults.  American Sociological Review  , 51, 541-554.

The authors, researchers at the Rand Corporation and Brown University, use data from the National Longitudinal Surveys of Young Women and Young Men to test their hypothesis that nonfamily living by young adults alters their attitudes, values, plans, and expectations, moving them away from their belief in traditional sex roles. They find their hypothesis strongly supported in young females, while the effects were fewer in studies of young males. Increasing the time away from parents before marrying increased individualism, self-sufficiency, and changes in attitudes about families.  In contrast, an earlier study by Williams cited below shows no significant gender differences in sex role attitudes as a result of nonfamily living.  (25)

MLA 8 style for a website citation:

Anderson, L.V. “Can You Libel Someone on Twitter?” Slate.com, The Slate Group, A Graham Holdings. Company, 26 Nov. 2012, http://www.slate.com/articles/technology/explainer/2012/11/libel_on_twitter_you_can_be_sued_for_libel_for_what_you_write_on_facebook.html . Accessed 2 Apr. 2018.

This article provides an overview of defamation law in the United States compared to the United Kingdom, in layman’s terms. It also explains how defamation law applies to social media platforms and individuals who use social media. Libelous comments posted on social media can be subject to lawsuit, depending on the content of the statement, and whether the person is a public or private figure. The article is found on the website, Slate.com, which is a web-based daily magazine that focuses on general interest topics. While the writer’s credentials are unavailable, she does thank Sandra S. Baron, Executive Director of the Media Law Resource Center and Jeff Hermes, director of the Digital Media Law Project for providing information. She also links to the United States laws that she cites.  I would use the article to compare United States law to United Kingdom law and for background information.  (1)

Information creation is a process. Scholars produce information in the forms of peer-reviewed journal articles, books, and conference presentations, to name a few. As a student researcher, you will be expected to create research projects such as essays, reports, visual presentations, and annotated bibliographies. Most scholarly writing makes an argument—whether it is to persuade your readers that your claim is true or to act on it. In order to create a sound argument, you must gather sources that will argue and counter-argue your claims.

When creating an argument, the researcher typically organizes their report or presentation with the claim/thesis at the beginning, which answers their research question. Then they provide reasons and supporting evidence to validate their claim. They acknowledge and respond to counter-arguments by citing sources that disagree with them, and refuting or conceding those counter-claims. Their conclusion restates their thesis and discusses why their research is important to the scholarly conversation, as well as potential areas for further research.

A Roman numeral outline is one way to organize your argument before you begin writing. It helps to identify sources for each section of your outline, so you know if you need further research to support your argument.

An annotated bibliography is one way to present research, and can be used as a cumulative assignment, or a precursor to your actual research paper. A good annotated bibliography will provide a variety of sources that met all your research needs—background, evidence, argument, and method. In other words, you should be able to take your annotated bibliography and write a complete research report based on those sources.  (1)

Introduction to College Research Copyright © by Lumen Learning is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

University Library, University of Illinois at Urbana-Champaign

Bibliography and Historical Research

Introduction.

• National Bibliography
• Personal Bibliography
• Corporate Bibliography
• Subject Bibliography
• Searching the Catalog for Bibliographies
• Browsing the Catalog for Bibliographies
• Other Tools for Finding Bibliographies
• Return to HPNL Website

Ask a Librarian

This guide created by Geoffrey Ross, May 4, 2017.

A bibliography is a list of documents, usually published documents like books and articles. This type of bibliography is more accurately called "enumerative bibliography". An enumerative bibliography will attempt to be as comprehensive as possible, within whatever parameters established by the bibliographer.

Bibliographies will list both secondary and primary sources. They are perhaps most valuable to historians for identifying primary sources. (They are still useful for finding secondary sources, but increasingly historians rely on electronic resources, like article databases, to locate secondary sources.)

Think of a bibliography as a guide to the source base for a specific field of inquiry. A high quality bibliography will help you understand what kinds of sources are available, but also what kinds of sources are not available (either because they were never preserved, or because they were never created in the first place).

Take for example the following bibliography:

• British Autobiographies: An Annotated Bibliography of British Autobiographies Published or Written before 1951 by William Matthews Call Number: 016.920041 M43BR Publication Date: 1955

Like many bibliographies, this one includes an introduction or prefatory essay that gives a bibliographic overview of the topic. If you were hoping to use autobiographies for a paper on medieval history, the following information from the preface would save you from wasting your time in a fruitless search:

The essay explains that autobiography does not become an important historical source until the early modern period:

Finally, the essay informs us that these early modern autobiographies are predominantly religious in nature--a useful piece of information if we were hoping to use them as evidence of, for example, the early modern textile trade:

All bibliographies are organized differently, but the best include indexes that help you pinpoint the most relevant entries.

A smart researcher will also use the index to obtain an overview of the entire source base: the index as a whole presents a broad outline of the available sources--the extent of available sources, as well as the the strengths and weaknesses of the source base. Browsing the subject index, if there is one, is often an excellent method of choosing a research topic because it enables you quickly to rule out topics that cannot be researched due to lack of primary sources.

The index to British Autobiographies , for example, tells me that I can find many autobiographies that document British social clubs (like White's and Boodle's), especially from the 19th century:

Unlike indexes you might be familiar with from non-fiction books, the indexes in bibliographies usually reference specific entries, not page numbers.

A bibliography's index will often help guide you systematically through the available sources, as in this entry which prompts you to look under related index entries for even more sources:

There are four main types of enumerative bibliography used for historical research:

Click here to learn more about bibliography as a discipline .

• Next: National Bibliography >>
• Last Updated: Jan 4, 2023 2:21 PM
• URL: https://guides.library.illinois.edu/bibliography

How to Create a Bibliography

• First Online: 01 October 2023

Cite this chapter

• Rohan Reddy 4 ,
• Samuel Sorkhi 4 ,
• Saager Chawla 4 &
• Mahadevan Raj Rajasekaran 5  

731 Accesses

This chapter describes the fundamental principles and practices of referencing sources in scientific writing and publishing. Understanding plagiarism and improper referencing of the source material is paramount to producing original work that contains an authentic voice. Citing references helps authors to avoid plagiarism, give credit to the original author, and allow potential readers to refer to the legitimate sources and learn more information. Furthermore, quality references serve as an invaluable resource that can enlighten future research in a field. This chapter outlines fundamental aspects of referencing as well as how these sources are formatted as per recommended citation styles. Appropriate referencing is an important tool that can be utilized to develop the credibility of the author and the arguments presented. Additionally, online software can be useful in helping the author organize their sources and promote proper collaboration in scientific writing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

AWELU (2022) The functions of references. Lund University. https://www.awelu.lu.se/referencing/the-functions-of-references/ . Accessed 28 Dec 2022

Masic I (2013) The importance of proper citation of references in biomedical articles. Acta Inform Med 21(3):148–155

Article   PubMed   PubMed Central   Google Scholar  

Neville C (2012) Referencing: principles, practice and problems. RGUHS J Pharm Sci 2(2):1–8

Google Scholar  

Horkoff T (2015) Writing for success 1st Canadian edition: BCcampus. Chapter 7. Sources: choosing the right ones. https://opentextbc.ca/writingforsuccess/chapter/chapter-7-sources-choosing-the-right-ones/ . Accessed 28 Dec 2022

A guide to database and catalog searching: bibliographic elements. Northwestern State University. https://libguides.nsula.edu/howtosearch/bibelements . Updated 5 Aug 2021; Accessed 28 Dec 2022

Research process: step 7: citing and keeping track of sources. University of Rio Grande. https://libguides.rio.edu/c.php?g=620382&p=4320145 . Updated 12 Dec 2022; Accessed 28 Dec 2022

Pears R, Shields G (2022) Cite them right. Bloomsbury Publishing

Harvard system (1999) Bournemouth University. http://ibse.hk/Harvard_System.pdf . Accessed 28 Dec 2022

Gitanjali B (2004) Reference styles and common problems with referencing: Medknow. https://www.jpgmonline.com/documents/author/24/11_Gitanjali_3.pdf . Accessed 28 Dec 2022

Williams K, Spiro J, Swarbrick N (2008) How to reference Harvard referencing for Westminster Institute students. Westminster Institute of Education Oxford Brookes University

Harvard: reference list and bibliography: University of Birmingham. 2022. https://intranet.birmingham.ac.uk/as/libraryservices/library/referencing/icite/harvard/referencelist.aspx . Accessed 28 Dec 2022

Limited SN (2017) Responsible referencing. Nat Methods 14(3):209

Article   Google Scholar  

Hensley MK (2011) Citation management software: features and futures. Ref User Serv Q 50(3):204–208

Ramesh A, Rajasekaran MR (2014) Zotero: open-source bibliography management software. RGUHS J Pharm Sci 4:3–6

Shapland M (2000) Evaluation of reference management software on NT (comparing Papyrus with Procite, Reference Manager, Endnote, Citation, GetARef, Biblioscape, Library Master, Bibliographica, Scribe, Refs). University of Bristol

EndNote 20. Clarivate. 2022. https://endnote.com/product-details . Accessed 28 Dec 2022

Angélil-Carter S (2014) Stolen language? Plagiarism in writing. Routledge

Book   Google Scholar  

Howard RM (1995) Plagiarisms, authorships, and the academic death penalty. Coll Engl 57(7):788–806

Barrett R, Malcolm J (2006) Embedding plagiarism education in the assessment process. Int J Educ Integr 2(1):38–45

Introna L, Hayes N, Blair L, Wood E (2003) Cultural attitudes towards plagiarism. University of Lancaster, Lancaster, pp 1–57

Neville C (2016) The complete guide to referencing and avoiding plagiarism. McGraw-Hill Education (UK), London, p 43

Campion EW, Anderson KR, Drazen JM (2001) Internet-only publication. N Engl J Med 345(5):365

Li X, Crane N (1996) Electronic styles: a handbook for citing electronic information. Information Today, Inc.

APA (2005) Concise rules of APA style. APA, Washington, DC

Snyder PJ, Peterson A (2002) The referencing of internet web sites in medical and scientific publications. Brain Cogn 50(2):335–337

Article   PubMed   Google Scholar  

Barroga EF (2014) Reference accuracy: authors’, reviewers’, editors’, and publishers’ contributions. J Korean Med Sci 29(12):1587–1589

Standardization IOf (1997) Information and documentation—bibliographic references—part 2: electronic documents or parts thereof, ISO 690-2: 1997. ISO

Download references

Conflict of Interest

None declared.

Author information

Authors and affiliations.

Department of Urology, University of California at San Diego, San Diego, CA, USA

Rohan Reddy, Samuel Sorkhi & Saager Chawla

Department of Urology, University of California and VA San Diego Healthcare System, San Diego, CA, USA

Mahadevan Raj Rajasekaran

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mahadevan Raj Rajasekaran .

Editor information

Editors and affiliations.

Retired Senior Expert Pharmacologist at the Office of Cardiology, Hematology, Endocrinology, and Nephrology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA

Gowraganahalli Jagadeesh

Professor & Director, Research Training and Publications, The Office of Research and Development, Periyar Maniammai Institute of Science & Technology (Deemed to be University), Vallam, Tamil Nadu, India

Pitchai Balakumar

Division Cardiology & Nephrology, Office of Cardiology, Hematology, Endocrinology and Nephrology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA

Fortunato Senatore

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Reddy, R., Sorkhi, S., Chawla, S., Rajasekaran, M.R. (2023). How to Create a Bibliography. In: Jagadeesh, G., Balakumar, P., Senatore, F. (eds) The Quintessence of Basic and Clinical Research and Scientific Publishing. Springer, Singapore. https://doi.org/10.1007/978-981-99-1284-1_39

Download citation

DOI : https://doi.org/10.1007/978-981-99-1284-1_39

Published : 01 October 2023

Publisher Name : Springer, Singapore

Print ISBN : 978-981-99-1283-4

Online ISBN : 978-981-99-1284-1

eBook Packages : Biomedical and Life Sciences Biomedical and Life Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

University Library

Start your research.

• Research Process
• Find Background Info
• Find Sources through the Library
• Evaluate Your Info
• Cite Your Sources
• Evaluate, Write & Cite

• is the right thing to do  to give credit to those who had the idea
• shows that you have read and understand  what experts have had to say about your topic
• helps people find the sources  that you used in case they want to read more about the topic
• provides   evidence  for your arguments
• is professional and  standard practice   for students and scholars

What is a Citation?

A citation identifies for the reader the original source for an idea, information, or image that is referred to in a work.

• In the body of a paper, the  in-text citation  acknowledges the source of information used.
• At the end of a paper, the citations are compiled on a  References  or  Works Cited  list. A basic citation includes the author, title, and publication information of the source. 

From:  Lemieux  Library,  University  of Seattle 

Why Should You Cite?

Quoting Are you quoting two or more consecutive words from a source? Then the original source should be cited and the words or phrase placed in quotes. 

Paraphrasing If an idea or information comes from another source,  even if you put it in your own words , you still need to credit the source.  General vs. Unfamiliar Knowledge You do not need to cite material which is accepted common knowledge. If in doubt whether your information is common knowledge or not, cite it. Formats We usually think of books and articles. However, if you use material from web sites, films, music, graphs, tables, etc. you'll also need to cite these as well.

Plagiarism is presenting the words or ideas of someone else as your own without proper acknowledgment of the source. When you work on a research paper and use supporting material from works by others, it's okay to quote people and use their ideas, but you do need to correctly credit them. Even when you summarize or paraphrase information found in books, articles, or Web pages, you must acknowledge the original author.

Citation Style Help

Helpful links:

• MLA ,  Works Cited : A Quick Guide (a template of core elements)
• CSE  (Council of Science Editors)

For additional writing resources specific to styles listed here visit the  Purdue OWL Writing Lab

Citation and Bibliography Resources

• How to Write an Annotated Bibliography
• << Previous: Evaluate Your Info
• Next: Evaluate, Write & Cite >>

Creative Commons Attribution 3.0 License except where otherwise noted.

Land Acknowledgement

The land on which we gather is the unceded territory of the Awaswas-speaking Uypi Tribe. The Amah Mutsun Tribal Band, comprised of the descendants of indigenous people taken to missions Santa Cruz and San Juan Bautista during Spanish colonization of the Central Coast, is today working hard to restore traditional stewardship practices on these lands and heal from historical trauma.

The land acknowledgement used at UC Santa Cruz was developed in partnership with the Amah Mutsun Tribal Band Chairman and the Amah Mutsun Relearning Program at the UCSC Arboretum .

• Try for free

How to Write a Bibliography (MLA, APA Examples)

Learn how to easily write a bibliography by following the format outlined in this article.

This resource will help your students properly cite different resources in the bibliography of a research paper, and how to format those citations, for books, encyclopedias, films, websites, and people.

What is a bibliography?

According to Infoplease.com, A bibliography is a list of the types of sources you used to get information for your report. It is included at the end of your report, on the last page (or last few pages).

What are the types of bibliography styles (MLA, APA, etc.)?

The 3 most common bibliography/citation styles are:

• MLA Style: The Modern Language Association works cited page style
• APA Style: The American Psychological Association style
• Chicago Style: The bibliography style defined by the Chicago Manual of Style

We’ll give examples of how to create bibliography entries in various styles further down in this article. 

What sources do you put in a bibliography?

An annotated bibliography should include a reference list of any sources you use in writing a research paper. Any printed sources from which you use a text citation, including books, websites, newspaper articles, journal articles, academic writing, online sources (such as PDFs), and magazines should be included in a reference list. In some cases, you may need or want to cite conversations or interviews, works of art, visual works such as movies, television shows, or documentaries - these (and many others) can also be included in a reference list.

How to get started writing your bibliography

You will find it easier to prepare your MLA, APA, or Chicago annotated bibliography if you keep track of each book, encyclopedia, journal article, webpage or online source you use as you are reading and taking notes. Start a preliminary, or draft, bibliography by listing on a separate sheet of paper all your sources. Note down the full title, author’s last name, place of publication, web address, publisher, and date of publication for each source.

Haven't started your paper yet and need an outline? These sample essay outlines include a research paper outline from an actual student paper.

How to write a bibliography step-by-step (with examples)

General Format: Author (last name first). Title of the book. Publisher, Date of publication.

MLA Style: Sibley, David Allen. What It’s Like to Be a Bird. From Flying to Nesting, Eating to Singing, What Birds Are Doing, and Why. Alfred A. Knopf, 2020.

APA Style: Sibley, D.A. (2020). What It’s Like to Be a Bird. From Flying to Nesting, Eating to Singing, What Birds Are Doing, and Why . Alfred A. Knopf.

Notes: Use periods, not commas, to separate the data in the entry. Use a hanging indent if the entry is longer than one line. For APA style, do not use the full author’s first name.

Websites or webpages:

  MLA Style: The SB Nation Family of Sites. Pension Plan Puppets: A Toronto Maple Leafs Blog, 2022, www.pensionplanpuppets.com. Accessed 15 Feb. 2022.

APA Style: American Heart Association. (2022, April 11). How to keep your dog’s heart healthy. https://www.heart.org/en/news/2022/04/11/how-to-keep-your-dogs-heart-healthy

Online news article from a newspaper site:

APA Style: Duehren, A. (2022, April 9). Janet Yellen faces challenge to keep pressure on Russia. Wall Street Journal. https://www.wsj.com/articles/janet-yellen-faces-challenge-to-keep-pressure-on-russia-while-addressing-global-consequences-11650366000

Print journal articles:

MLA Style: Booch, Grady. "Patterns in Object-Oriented Design." IEEE Software Engineering, vol. 6, no. 6, 2006, pp. 31-50.

APA Style: Booch, G. (2006). Patterns in object-oriented design. IEEE Software Engineering, 6(6), 31–50.

Note: It is suggested that you include a DOI and a webpage address when referencing either a printed journal article, and electronic journal article, or an journal article that appears in both formats. 

MLA Style: Gamma, Eric, and Peter A. Coad. “Exceptions to the Unified Modeling Language in Python Patterns.” IEEE Software Engineering, vol. 2, no. 6, 8 Mar. 2006, pp. 190-194. O’Reilly Software Engineering Library, https://doi.org/10.1006/se.20061. Accessed 26 May 2009.

APA Style: Masters, H., Barron, J., & Chanda, L. (2017). Motivational interviewing techniques for adolescent populations in substance abuse counseling. NAADAC Notes, 7(8), 7–13. https://www.naadac.com/notes/adolescent-techniques

ML:A Style: @Grady_Booch. “That’s a bold leap over plain old battery power cars.” Twitter, 13 Mar. 2013, 12:06 p.m., https://twitter.com/Grady_Booch/status/1516379006727188483.

APA Style: Westborough Library [@WestboroughLib]. (2022, April 12). Calling all 3rd through 5th grade kids! Join us for the Epic Writing Showdown! Winner receives a prize! Space is limited so register, today. loom.ly/ypaTG9Q [Tweet; thumbnail link to article]. Twitter. https://twitter.com/WestboroughLib/status/1516373550415896588.

Print magazine articles:

General format: Author (last name first), "Article Title." Name of magazine. Volume number, (Date): page numbers.

MLA Style: Stiteler, Sharon. "Tracking Red-Breasted Grosbeak Migration." Minnesota Bird Journal, 7 Sept. 2019, pp. 7-11.

APA Style: Jordan, Jennifer, "Filming at the Top of the World." Museum of Science Magazine. Volume 47, No. 1, (Winter 1998): p. 11.

Print newspaper articles:

General format: Author (last name first), "Article Title." Name of newspaper, city, state of publication. (date): edition if available, section, page number(s).

MLA Style: Adelman, Martin. "Augustus Announces Departure from City Manager Post." New York Times, late ed., 15 February 2020, p. A1

APA Style: Adelman, M. (2020, February 15). Augustus announced departure from city manager post. New York Times, A1.

Encyclopedias:

General Format: Encyclopedia Title, Edition Date. Volume Number, "Article Title," page numbers.

MLA Style: “Gorillas.” The Encyclopedia Brittanica. 15th ed. 2010.

APA Style: Encyclopedia Brittanica, Inc. (1997.) Gorillas. In The Encyclopedia Brittanica (15th ed., pp. 50-51). Encyclopedia Brittanica, Inc.

Personal interviews:

General format: Full name (last name first). Personal Interview. (Occupation.) Date of interview.

MLA Style: Smithfield, Joseph. Personal interview. 19 May 2014.

APA Style: APA does not require a formal citation for a personal interview. Published interviews from other sources should be cited accordingly.

Films and movies:

General format: Title, Director, Distributor, Year.

MLA Style: Fury. Directed by David Ayer, performances by Brad Pitt, Shia LaBeouf, Jon Bernthal, Sony Pictures, 2014.

APA Style: Ayer, D. (Director). (2014). Fury [Film]. Sony Pictures.

Featured High School Resources

Related Resources

About the author

TeacherVision Editorial Staff

The TeacherVision editorial team is comprised of teachers, experts, and content professionals dedicated to bringing you the most accurate and relevant information in the teaching space.

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

Academic integrity and documentation, explore more of umgc.

• Online Guide to Writing

Types of Documentation

Bibliographies and Source Lists

What is a bibliography.

A bibliography is a list of books and other source material that you have used in preparing a research paper. Sometimes these lists will include works that you consulted but did not cite specifically in your assignment. Consult the style guide required for your assignment to determine the specific title of your bibliography page as well as how to cite each source type. Bibliographies are usually placed at the end of your research paper.

What is an annotated bibliography?

A special kind of bibliography, the annotated bibliography, is often used to direct your readers to other books and resources on your topic. An instructor may ask you to prepare an annotated bibliography to help you narrow down a topic for your research assignment. Such bibliographies offer a few lines of information, typically 150-300 words, summarizing the content of the resource after the bibliographic entry.   

Example of Annotated Bibliographic Entry in MLA Style

Waddell, Marie L., Robert M. Esch, and Roberta R. Walker. The Art of Styling         Sentences: 20 Patterns for Success. 3rd ed. New York: Barron’s, 1993.         A comprehensive look at 20 sentence patterns and their variations to         teach students how to write effective sentences by imitating good style.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

What Is a Bibliography?

• Writing Research Papers
• Writing Essays
• English Grammar
• M.Ed., Education Administration, University of Georgia
• B.A., History, Armstrong State University

A bibliography is a list of books, scholarly  articles , speeches, private records, diaries, interviews, laws, letters, websites, and other sources you use when researching a topic and writing a paper. The bibliography appears at the end.

The main purpose of a bibliography entry is to give credit to authors whose work you've consulted in your research. It also makes it easy for a reader to find out more about your topic by delving into the research that you used to write your paper. In the academic world, papers aren't written in a vacuum; academic journals are the way new research on a topic circulates and previous work is built upon.

Bibliography entries must be written in a very specific format, but that format will depend on the particular style of writing you follow. Your teacher or publisher will tell you which style to use, and for most academic papers it will be either MLA , American Psychological Association (APA), Chicago (author-date citations or footnotes/endnotes format), or Turabian style .

The bibliography is sometimes also called the references, works cited, or works consulted page.

Components of a Bibliography Entry

Bibliography entries will compile:

• Authors and/or editors (and translator, if applicable)
• Title of your source (as well as edition, volume, and the book title if your source is a chapter or article in a multi-author book with an editor)
• Publication information (the city, state, name of the publisher, date published, page numbers consulted, and URL or DOI, if applicable)
• Access date, in the case of online sources (check with the style guide at the beginning of your research as to whether you need to track this information)

Order and Formatting

Your entries should be listed in alphabetical order by the last name of the first author. If you are using two publications that are written by the same author, the order and format will depend on the style guide.

In MLA, Chicago, and Turabian style, you should list the duplicate-author entries in alphabetical order according to the title of the work. The author's name is written as normal for his or her first entry, but for the second entry, you will replace the author's name with three long dashes. 

In APA style, you list the duplicate-author entries in chronological order of publication, placing the earliest first. The name of the author is used for all entries.

For works with more than one author, styles vary as to whether you invert the name of any authors after the first. Whether you use title casing or sentence-style casing on titles of sources, and whether you separate elements with commas or periods also varies among different style guides. Consult the guide's manual for more detailed information.

Bibliography entries are usually formatted using a hanging indent. This means that the first line of each citation is not indented, but subsequent lines of each citation are indented. Check with your instructor or publication to see if this format is required, and look up information in your word processor's help program if you do not know how to create a hanging indent with it.

Chicago's Bibliography vs. Reference System

Chicago has two different ways of citing works consulted: using a bibliography or a references page. Use of a bibliography or a references page depends on whether you're using author-date parenthetical citations in the paper or footnotes/endnotes. If you're using parenthetical citations, then you'll follow the references page formatting. If you're using footnotes or endnotes, you'll use a bibliography. The difference in the formatting of entries between the two systems is the location of the date of the cited publication. In a bibliography, it goes at the end of an entry. In a references list in the author-date style, it goes right after the author's name, similar to APA style.

• What Is a Citation?
• What Is a Senior Thesis?
• Bibliography: Definition and Examples
• Formatting Papers in Chicago Style
• Turabian Style Guide With Examples
• MLA Bibliography or Works Cited
• How to Write a Bibliography For a Science Fair Project
• APA In-Text Citations
• MLA Sample Pages
• How to Use Block Quotations in Writing
• Writing a History Book Review
• What Is a Style Guide and Which One Do You Need?
• Definition of Appendix in a Book or Written Work
• MLA Style Parenthetical Citations
• Bibliography, Reference List or Works Cited?
• Tips for Typing an Academic Paper on a Computer

• Grades 6-12
• School Leaders

Free printable Mother's Day questionnaire 💐!

How To Write a Bibliography (Three Styles, Plus Examples)

Give credit where credit is due.

Writing a research paper involves a lot of work. Students need to consult a variety of sources to gather reliable information and ensure their points are well supported. Research papers include a bibliography, which can be a little tricky for students. Learn how to write a bibliography in multiple styles and find basic examples below.

IMPORTANT: Each style guide has its own very specific rules, and they often conflict with one another. Additionally, each type of reference material has many possible formats, depending on a variety of factors. The overviews shown here are meant to guide students in writing basic bibliographies, but this information is by no means complete. Students should always refer directly to the preferred style guide to ensure they’re using the most up-to-date formats and styles.

What is a bibliography?

When you’re researching a paper, you’ll likely consult a wide variety of sources. You may quote some of these directly in your work, summarize some of the points they make, or simply use them to further the knowledge you need to write your paper. Since these ideas are not your own, it’s vital to give credit to the authors who originally wrote them. This list of sources, organized alphabetically, is called a bibliography.

A bibliography should include all the materials you consulted in your research, even if you don’t quote directly from them in your paper. These resources could include (but aren’t limited to):

• Books and e-books
• Periodicals like magazines or newspapers
• Online articles or websites
• Primary source documents like letters or official records

Bibliography vs. References

These two terms are sometimes used interchangeably, but they actually have different meanings. As noted above, a bibliography includes all the materials you used while researching your paper, whether or not you quote from them or refer to them directly in your writing.

A list of references only includes the materials you cite throughout your work. You might use direct quotes or summarize the information for the reader. Either way, you must ensure you give credit to the original author or document. This section can be titled “List of Works Cited” or simply “References.”

Your teacher may specify whether you should include a bibliography or a reference list. If they don’t, consider choosing a bibliography, to show all the works you used in researching your paper. This can help the reader see that your points are well supported, and allow them to do further reading on their own if they’re interested.

Bibliography vs. Citations

Citations refer to direct quotations from a text, woven into your own writing. There are a variety of ways to write citations, including footnotes and endnotes. These are generally shorter than the entries in a reference list or bibliography. Learn more about writing citations here.

What does a bibliography entry include?

Depending on the reference material, bibliography entries include a variety of information intended to help a reader locate the material if they want to refer to it themselves. These entries are listed in alphabetical order, and may include:

• Author/s or creator/s
• Publication date
• Volume and issue numbers
• Publisher and publication city
• Website URL

These entries don’t generally need to include specific page numbers or locations within the work (except for print magazine or journal articles). That type of information is usually only needed in a footnote or endnote citation.

What are the different bibliography styles?

In most cases, writers use one of three major style guides: APA (American Psychological Association), MLA (Modern Language Association), or The Chicago Manual of Style . There are many others as well, but these three are the most common choices for K–12 students.

Many teachers will state their preference for one style guide over another. If they don’t, you can choose your own preferred style. However, you should also use that guide for your entire paper, following their recommendations for punctuation, grammar, and more. This will ensure you are consistent throughout.

Below, you’ll learn how to write a simple bibliography using each of the three major style guides. We’ve included details for books and e-books, periodicals, and electronic sources like websites and videos. If the reference material type you need to include isn’t shown here, refer directly to the style guide you’re using.

APA Style Bibliography and Examples

Source: Verywell Mind

Technically, APA style calls for a list of references instead of a bibliography. If your teacher requires you to use the APA style guide , you can limit your reference list only to items you cite throughout your work.

How To Write a Bibliography (References) Using APA Style

Here are some general notes on writing an APA reference list:

• Title your bibliography section “References” and center the title on the top line of the page.
• Do not center your references; they should be left-aligned. For longer items, subsequent lines should use a hanging indent of 1/2 inch.
• Include all types of resources in the same list.
• Alphabetize your list by author or creator, last name first.
• Do not spell out the author/creator’s first or middle name; only use their initials.
• If there are multiple authors/creators, use an ampersand (&) before the final author/creator.
• Place the date in parentheses.
• Capitalize only the first word of the title and subtitle, unless the word would otherwise be capitalized (proper names, etc.).
• Italicize the titles of books, periodicals, or videos.
• For websites, include the full site information, including the http:// or https:// at the beginning.

Books and E-Books APA Bibliography Examples

For books, APA reference list entries use this format (only include the publisher’s website for e-books).

Last Name, First Initial. Middle Initial. (Publication date). Title with only first word capitalized . Publisher. Publisher’s website

• Wynn, S. (2020). City of London at war 1939–45 . Pen & Sword Military. https://www.pen-and-sword.co.uk/City-of-London-at-War-193945-Paperback/p/17299

Periodical APA Bibliography Examples

For journal or magazine articles, use this format. If you viewed the article online, include the URL at the end of the citation.

Last Name, First Initial. Middle Initial. (Publication date). Title of article. Magazine or Journal Title (Volume number) Issue number, page numbers. URL

• Bell, A. (2009). Landscapes of fear: Wartime London, 1939–1945. Journal of British Studies (48) 1, 153–175. https://www.jstor.org/stable/25482966

Here’s the format for newspapers. For print editions, include the page number/s. For online articles, include the full URL.

Last Name, First Initial. Middle Initial. (Year, Month Date) Title of article. Newspaper title. Page number/s. URL

• Blakemore, E. (2022, November 12) Researchers track down two copies of fossil destroyed by the Nazis.  The Washington Post. https://www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/

Electronic APA Bibliography Examples

For articles with a specific author on a website, use this format.

Last Name, First Initial. Middle Initial. (Year, Month Date). Title . Site name. URL

• Wukovits, J. (2023, January 30). A World War II survivor recalls the London Blitz . British Heritage . https://britishheritage.com/history/world-war-ii-survivor-london-blitz

When an online article doesn’t include a specific author or date, list it like this:

Title . (Year, Month Date). Site name. Retrieved Month Date, Year, from URL

• Growing up in the Second World War . (n.d.). Imperial War Museums. Retrieved May 12, 2023, from https://www.iwm.org.uk/history/growing-up-in-the-second-world-war

When you need to list a YouTube video, use the name of the account that uploaded the video, and format it like this:

Name of Account. (Upload year, month day). Title [Video]. YouTube. URL

• War Stories. (2023, January 15). How did London survive the Blitz during WW2? | Cities at war: London | War stories [Video]. YouTube. https://youtu.be/uwY6JlCvbxc

For more information on writing APA bibliographies, see the APA Style Guide website.

APA Bibliography (Reference List) Example Pages

Source: Simply Psychology

More APA example pages:

• Western Australia Library Services APA References Example Page
• Ancilla College APA References Page Example
• Scribbr APA References Page Example

MLA Style Bibliography Examples

Source: PressBooks

MLA style calls for a Works Cited section, which includes all materials quoted or referred to in your paper. You may also include a Works Consulted section, including other reference sources you reviewed but didn’t directly cite. Together, these constitute a bibliography. If your teacher requests an MLA Style Guide bibliography, ask if you should include Works Consulted as well as Works Cited.

How To Write a Bibliography (Works Cited and Works Consulted) in MLA Style

For both MLA Works Cited and Works Consulted sections, use these general guidelines:

• Start your Works Cited list on a new page. If you include a Works Consulted list, start that on its own new page after the Works Cited section.
• Center the title (Works Cited or Works Consulted) in the middle of the line at the top of the page.
• Align the start of each source to the left margin, and use a hanging indent (1/2 inch) for the following lines of each source.
• Alphabetize your sources using the first word of the citation, usually the author’s last name.
• Include the author’s full name as listed, last name first.
• Capitalize titles using the standard MLA format.
• Leave off the http:// or https:// at the beginning of a URL.

Books and E-Books MLA Bibliography Examples

For books, MLA reference list entries use this format. Add the URL at the end for e-books.

Last Name, First Name Middle Name. Title . Publisher, Date. URL

• Wynn, Stephen. City of London at War 1939–45 . Pen & Sword Military, 2020. www.pen-and-sword.co.uk/City-of-London-at-War-193945-Paperback/p/17299

Periodical MLA Bibliography Examples

Here’s the style format for magazines, journals, and newspapers. For online articles, add the URL at the end of the listing.

For magazines and journals:

Last Name, First Name. “Title: Subtitle.” Name of Journal , volume number, issue number, Date of Publication, First Page Number–Last Page Number.

• Bell, Amy. “Landscapes of Fear: Wartime London, 1939–1945.” Journal of British Studies , vol. 48, no. 1, pp. 153–175. www.jstor.org/stable/25482966

When citing newspapers, include the page number/s for print editions or the URL for online articles.

Last Name, First Name. “Title of article.” Newspaper title. Page number/s. Year, month day. Page number or URL

• Blakemore, Erin. “Researchers Track Down Two Copies of Fossil Destroyed by the Nazis.” The Washington Post. 2022, Nov. 12. www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/

Electronic MLA Bibliography Examples

Last Name, First Name. Year. “Title.” Month Day, Year published. URL

• Wukovits, John. 2023. “A World War II Survivor Recalls the London Blitz.” January 30,   2023. https://britishheritage.com/history/world-war-ii-survivor-london-blitz

Website. n.d. “Title.” Accessed Day Month Year. URL.

• Imperial War Museum. n.d. “Growing Up in the Second World War.” Accessed May 9, 2023. https://www.iwm.org.uk/history/growing-up-in-the-second-world-war.

Here’s how to list YouTube and other online videos.

Creator, if available. “Title of Video.” Website. Uploaded by Username, Day Month Year. URL.

• “How did London survive the Blitz during WW2? | Cities at war: London | War stories.” YouTube . Uploaded by War Stories, 15 Jan. 2023. youtu.be/uwY6JlCvbxc.

For more information on writing MLA style bibliographies, see the MLA Style website.

MLA Bibliography (Works Cited) Example Pages

Source: The Visual Communication Guy

More MLA example pages:

• Writing Commons Sample Works Cited Page
• Scribbr MLA Works Cited Sample Page
• Montana State University MLA Works Cited Page

Chicago Manual of Style Bibliography Examples

The Chicago Manual of Style (sometimes called “Turabian”) actually has two options for citing reference material : Notes and Bibliography and Author-Date. Regardless of which you use, you’ll need a complete detailed list of reference items at the end of your paper. The examples below demonstrate how to write that list.

How To Write a Bibliography Using The Chicago Manual of Style

Source: South Texas College

Here are some general notes on writing a Chicago -style bibliography:

• You may title it “Bibliography” or “References.” Center this title at the top of the page and add two blank lines before the first entry.
• Left-align each entry, with a hanging half-inch indent for subsequent lines of each entry.
• Single-space each entry, with a blank line between entries.
• Include the “http://” or “https://” at the beginning of URLs.

Books and E-Books Chicago Manual of Style Bibliography Examples

For books, Chicago -style reference list entries use this format. (For print books, leave off the information about how the book was accessed.)

Last Name, First Name Middle Name. Title . City of Publication: Publisher, Date. How e-book was accessed.

• Wynn, Stephen. City of London at War 1939–45 . Yorkshire: Pen & Sword Military, 2020. Kindle edition.

Periodical Chicago Manual of Style Bibliography Examples

For journal and magazine articles, use this format.

Last Name, First Name. Year of Publication. “Title: Subtitle.” Name of Journal , Volume Number, issue number, First Page Number–Last Page Number. URL.

• Bell, Amy. 2009. “Landscapes of Fear: Wartime London, 1939–1945.” Journal of British Studies, 48 no. 1, 153–175. https://www.jstor.org/stable/25482966.

When citing newspapers, include the URL for online articles.

Last Name, First Name. Year of Publication. “Title: Subtitle.” Name of Newspaper , Month day, year. URL.

• Blakemore, Erin. 2022. “Researchers Track Down Two Copies of Fossil Destroyed by the Nazis.” The Washington Post , November 12, 2022. https://www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/.

Electronic Chicago Manual of Style Bibliography Examples

Last Name, First Name Middle Name. “Title.” Site Name . Year, Month Day. URL.

• Wukovits, John. “A World War II Survivor Recalls the London Blitz.” British Heritage. 2023, Jan. 30. britishheritage.com/history/world-war-ii-survivor-london-blitz.

“Title.” Site Name . URL. Accessed Day Month Year.

• “Growing Up in the Second World War.” Imperial War Museums . www.iwm.org.uk/history/growing-up-in-the-second-world-war. Accessed May 9, 2023.

Creator or Username. “Title of Video.” Website video, length. Month Day, Year. URL.

• War Stories. “How Did London Survive the Blitz During WW2? | Cities at War: London | War Stories.” YouTube video, 51:25. January 15, 2023. https://youtu.be/uwY6JlCvbxc.

For more information on writing Chicago -style bibliographies, see the Chicago Manual of Style website.

Chicago Manual of Style Bibliography Example Pages

Source: Chicago Manual of Style

More Chicago example pages:

• Scribbr Chicago Style Bibliography Example
• Purdue Online Writing Lab CMOS Bibliography Page
• Bibcitation Sample Chicago Bibliography

Now that you know how to write a bibliography, take a look at the Best Websites for Teaching & Learning Writing .

Plus, get all the latest teaching tips and ideas when you sign up for our free newsletters .

You Might Also Like

This Free Poetry Worksheet Bundle Is Perfect for Your Poetry Unit

Haiku, limerick, ode, and more! Continue Reading

Copyright © 2024. All rights reserved. 5335 Gate Parkway, Jacksonville, FL 32256

How to Write a Research Paper: Annotated Bibliography

• Anatomy of a Research Paper
• Developing a Research Focus
• Background Research Tips
• Searching Tips
• Scholarly Journals vs. Popular Journals
• Thesis Statement
• Annotated Bibliography
• Citing Sources
• Evaluating Sources
• Literature Review
• Academic Integrity
• Scholarship as Conversation
• Understanding Fake News
• Data, Information, Knowledge

What is an Annotated Bibliography?

UMary Writing Center

Check out the resources available from the  Writing Center . 

Write an Annotated Bibliography

What is an annotated bibliography?

It is a list of citations for various books, articles, and other sources on a topic. 

An annotation is a short summary and/or critical evaluation of a source.

Annotated bibliographies answer the question: "What would be the most relevant, most useful, or most up-to-date sources for this topic?"

 Annotated bibliographies can be part of a larger research project, or can be a stand-alone report in itself. 

Annotation versus abstracts 

An abstract is a paragraph at the beginning of the paper that discusses the main point of the original work. They typically do not include evaluation comments. 

Annotations can either be descriptive or evaluative. The annotated bibliography looks like a works cited page but includes an annotation after each source cited. 

Types of Annotations: 

Descriptive Annotations: Focuses on description. Describes the source by answering the following questions. 

Who wrote the document?

What does the document discuss?

When and where was the document written? 

Why was the document produced?

How was it provided to the public?

Evaluative Annotations: Focuses on description and evaluation. Includes a summary and critically assess the work for accuracy, relevance, and quality. 

Evaluative annotations help you learn about your topic, develop a thesis statement, decide if a specific source will be useful for your assignment, and determine if there is enough valid information available to complete your project.

What does the annotation include?

Depending on your assignment and style guide, annotations may include some or all of the following information. 

• Should be no more than 150 words or 4 to 6 sentences long. 
• What is the main focus or purpose of the work?
• Who is the intended audience?
• ​How useful or relevant was the article to your topic?
• Was there any unique features that useful to you?
• What is the background and credibility of the author?
• What are any conclusions or observations that your reached about the article?

Which citation style to use?

There are many styles manuals with specific instructions on how to format your annotated bibliography. This largely depends on what your instructor prefers or your subject discipline. Check out our citation guides for more information. 

Additional Information

Why doesn't APA have an official APA-approved format for annotated bibliographies?

Always consult your instructor about the format of an annotated bibliography for your class assignments. These guides provide you with examples of various styles for annotated bibliographies and they may not be in the format required by your instructor. 

Citation Examples and Annotations

Book Citation with Descriptive Annotation

Liroff, R. A., & G. G. Davis. (1981). Protecting open space: Land use control in the Adirondack Park. Cambridge, MA: Ballinger.

This book describes the implementation of regional planning and land use regulation in the Adirondack Park in upstate New York. The authors provide program evaluations of the Adirondack Park Agency’s regulatory and local planning assistance programs.

Journal Article Citation with Evaluative Annotation

Gottlieb, P. D. (1995). The “golden egg” as a natural resource: Toward a normative theory of growth management. Society and Natural Resources, 8, (5): 49-56.

This article explains the dilemma faced by North American suburbs, which demand both preservation of local amenities (to protect quality of life) and physical development (to expand the tax base). Growth management has been proposed as a policy solution to this dilemma. An analogy is made between this approach and resource economics. The author concludes that the growth management debate raises legitimate issues of sustainability and efficiency.

Examples were taken from http://lib.calpoly.edu/support/how-to/write-an-annotated-bibliography/#samples

Book Citation

Lee, Seok-hoon, Yong-pil Kim, Nigel Hemmington, and Deok-kyun Yun. “Competitive Service Quality Improvement (CSQI): A Case Study in the Fast-Food Industry.” Food Service Technology 4 (2004): 75-84.

In this highly technical paper, three industrial engineering professors in Korea and one services management professor in the UK discuss the mathematical limitations of the popular SERVQUAL scales. Significantly, they also aim to measure service quality in the fast-food industry, a neglected area of study. Unfortunately, the paper’s sophisticated analytical methods make it inaccessible to all but the most expert of researchers.

Battle, Ken. “Child Poverty: The Evolution and Impact of Child Benefits.”  A Question of Commitment: Children's Rights in Canada . Ed. Katherine Covell and R.Brian Howe. Waterloo, ON: Wilfrid Laurier University Press. 2007. 21-44.

             Ken Battle draws on a close study of government documents, as well as his own research as an extensively-published policy analyst, to explain Canadian child benefit programs.  He outlines some fundamental assumptions supporting the belief that all society members should contribute to the upbringing of children.  His comparison of child poverty rates in a number of countries is a useful wake-up to anyone assuming Canadian society is doing a good job of protecting children.  Battle pays particular attention to the National Child Benefit (NCB), arguing that it did not deserve to be criticized by politicians and journalists.  He outlines the NCB’s development, costs, and benefits, and laments that the Conservative government scaled it back in favour of the inferior Universal Child Care Benefit (UCCB).  However, he relies too heavily on his own work; he is the sole or primary author of almost half the sources in his bibliography.  He could make this work stronger by drawing from others' perspectives and analyses.  However, Battle does offer a valuable source for this essay, because the chapter provides a concise overview of government-funded assistance currently available to parents.  This offers context for analyzing the scope and financial reality of child poverty in Canada.

Journal Article Example

  Kerr, Don and Roderic Beaujot. “Child Poverty and Family Structure in Canada, 1981-1997.”  Journal of Comparative Family Studies  34.3 (2003): 321-335.

             Sociology professors Kerr and Beaujot analyze the demographics of impoverished families.  Drawing on data from Canada’s annual Survey of Consumer Finances, the authors consider whether each family had one or two parents, the age of single parents, and the number of children in each household.  They analyze child poverty rates in light of both these demographic factors and larger economic issues.  Kerr and Beaujot use this data to argue that. 

Examples were taken from  http://libguides.enc.edu/writing_basics/ annotatedbib/mla

Check out these resources for more information about Annotated Bibliographies. 

• Purdue Owl- Annotated Bibliographies
• University of North Carolina at Chapel Hill- Annotated Bibliographies
• << Previous: Thesis Statement
• Next: Citing Sources >>
• Last Updated: Apr 4, 2024 5:51 PM
• URL: https://libguide.umary.edu/researchpaper

Bibliography In Research Paper: Know Everything About It

Learn about what is a bibliography in a research paper and how to write a bibliography that makes your research paper undeniably perfect.

Imagine yourself as a research student and you have to submit a thesis for which you collected all the information and wrote your thesis but forgot to add a bibliography to it. Now your professor rejects your thesis for that one mistake. How would you feel? Wouldn’t it be disappointing? 

It indeed is disappointing but do you really think a bibliography is that important? In fact, have you ever wondered what a bibliography is? You must’ve seen it at the end of every book or article that you read but you wouldn’t have paid much attention to it. Read the article to know the importance of a bibliography in a research paper and how to write it according to the guidelines given. 

What is a Bibliography?

A bibliography is a list of sources or materials that you might have referred to while writing an article or a research paper. If the reader wants to explore more about the content apart from what you’ve written, they would refer to the sources that you’ve mentioned and a bibliography can be helpful. 

For example, if you’re reading a fiction novel about the culture of a European country in the 18th- century and you wish to learn more about it, what would you do? Here comes the importance of a bibliography. The author might have mentioned his research works – from where he collected information regarding European culture and he might’ve cited all those previous works in the form of a bibliography at the end of the book. This could help readers like you to explore more about the topic.

Bibliography vs References

Bibliography and references appear synonymous and thus, most of us try to use them interchangeably. But did you know this isn’t right? While both of them appear similar, there is this small difference that actually makes a lot of sense if you get to know it.

If we see the definition of reference, is the citation of all the works/sources that one used “ within the body of the paper”. This is the most important key difference. While references refer to the work that is present within the paper, a bibliography refers to the works which are “ not specifically referred to within the body of the paper” that is, it needn’t be necessary for a bibliography to have the exact sources on its body and you can cite a source even if you just used it to refer something regarding the topic.

Steps To Write A Bibliography In Research Paper

• The initial and foremost step to be followed before writing a bibliography is to make a note of all the books/sources/works you read when you’re doing your background research for a paper.
• Once you have a list of the books or sources, you need to check for the following information in the source :

If it is printed, check out for

• Authors’ name
• Title of publication
• Date of publication
• The publishing company of a book
• Page number(s)

If it is from a website, check out for

• Author and editor name
• Title of the webpage
• The company that posted that webpage
• URL of that webpage
• Before writing out your bibliography, you should know that there are two major guidelines for writing a bibliography (MLA Format and APA Format), and use them according to the need of your paper.
• Once you’re done writing a bibliography, make sure that you cited all the sources that you mentioned in the paper and whether you cited them in the correct format or not.
• If there are any mistakes or necessary changes, make them before you submit your paper.

Writing a bibliography might be a tiresome process as there’s a huge list of sources that need to be mentioned but adding a bibliography gives your research paper a more professional touch and sets your paper apart from the rest of the crowd.

Over 65,000 accurate scientific figures to boost your impact

We know! It’s hard to believe but like most scientific facts, it’s the truth. Mind the Graph has over 65,000 scientific figures that boost the impact of your research paper and helps your readers to connect with your work and understand the concepts better. 

Subscribe to our newsletter

Exclusive high quality content about effective visual communication in science.

Unlock Your Creativity

Create infographics, presentations and other scientifically-accurate designs without hassle — absolutely free for 7 days!

About Sowjanya Pedada

Sowjanya is a passionate writer and an avid reader. She holds MBA in Agribusiness Management and now is working as a content writer. She loves to play with words and hopes to make a difference in the world through her writings. Apart from writing, she is interested in reading fiction novels and doing craftwork. She also loves to travel and explore different cuisines and spend time with her family and friends.

Content tags

Your browser does not support javascript. Some site functionality may not work as expected.

• What is an Annotation?

Why Do an Annotated Bibliography?

• What Should be Included in the Annotation?
• What Format Should I Use for the Citations?
• Evaluating Sources
• Summative Annotations
• Evaluative Annotations
• Examples from the Web
• Additional Resources
• University of Washington Libraries
• Library Guides
• Annotated Bibliographies

Annotated Bibliographies: Why Do an Annotated Bibliography?

Here are some reasons:

• An annotated bibliography provides information about each source you have used One of the reasons behind citing sources and compiling a general bibliography is so that you can prove you have done some valid research to back up your argument and claims. Annotated bibliographies include additional details that provide an overview of the sources themselves.
• Each annotation provides essential details about a source Readers, researchers or instructors reading an annotated bibliography will get a snapshot of the important details that they need to know about each source. As a researcher, you have become an expert on your topic: you have the ability to explain the content of your sources, assess their usefulness, and share this information with others who may be less familiar with them.
• Some types of annotations provide details about what the paper's author has done with or thinks about each source Certain types of annotations (Evaluative Annotations, for example) may provide a brief analysis of the source by the paper's author, including details about what information is most important or not, how it the source fits into the broader scope of the paper and why it may or may not be useful to others.
• << Previous: What is an Annotation?
• Next: What Should be Included in the Annotation? >>
• Last Updated: Oct 3, 2023 12:04 PM
• URL: https://guides.lib.uw.edu/bothell/annotatedbibliographies

Quick Links:

Frequently asked questions

What’s the difference between a bibliography and a reference list.

Though the terms are sometimes used interchangeably, there is a difference in meaning:

• A reference list only includes sources cited in the text – every entry corresponds to an in-text citation .
• A bibliography also includes other sources which were consulted during the research but not cited.

Frequently asked questions: Knowledge Base

Methodology refers to the overarching strategy and rationale of your research. Developing your methodology involves studying the research methods used in your field and the theories or principles that underpin them, in order to choose the approach that best matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. interviews, experiments , surveys , statistical tests ).

In a dissertation or scientific paper, the methodology chapter or methods section comes after the introduction and before the results , discussion and conclusion .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research , you also have to consider the internal and external validity of your experiment.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Harvard referencing uses an author–date system. Sources are cited by the author’s last name and the publication year in brackets. Each Harvard in-text citation corresponds to an entry in the alphabetised reference list at the end of the paper.

Vancouver referencing uses a numerical system. Sources are cited by a number in parentheses or superscript. Each number corresponds to a full reference at the end of the paper.

A Harvard in-text citation should appear in brackets every time you quote, paraphrase, or refer to information from a source.

The citation can appear immediately after the quotation or paraphrase, or at the end of the sentence. If you’re quoting, place the citation outside of the quotation marks but before any other punctuation like a comma or full stop.

In Harvard referencing, up to three author names are included in an in-text citation or reference list entry. When there are four or more authors, include only the first, followed by ‘ et al. ’

A bibliography should always contain every source you cited in your text. Sometimes a bibliography also contains other sources that you used in your research, but did not cite in the text.

MHRA doesn’t specify a rule about this, so check with your supervisor to find out exactly what should be included in your bibliography.

Footnote numbers should appear in superscript (e.g. 11 ). You can use the ‘Insert footnote’ button in Word to do this automatically; it’s in the ‘References’ tab at the top.

Footnotes always appear after the quote or paraphrase they relate to. MHRA generally recommends placing footnote numbers at the end of the sentence, immediately after any closing punctuation, like this. 12

In situations where this might be awkward or misleading, such as a long sentence containing multiple quotations, footnotes can also be placed at the end of a clause mid-sentence, like this; 13 note that they still come after any punctuation.

When a source has two or three authors, name all of them in your MHRA references . When there are four or more, use only the first name, followed by ‘and others’:

Note that in the bibliography, only the author listed first has their name inverted. The names of additional authors and those of translators or editors are written normally.

A citation should appear wherever you use information or ideas from a source, whether by quoting or paraphrasing its content.

In Vancouver style , you have some flexibility about where the citation number appears in the sentence – usually directly after mentioning the author’s name is best, but simply placing it at the end of the sentence is an acceptable alternative, as long as it’s clear what it relates to.

In Vancouver style , when you refer to a source with multiple authors in your text, you should only name the first author followed by ‘et al.’. This applies even when there are only two authors.

In your reference list, include up to six authors. For sources with seven or more authors, list the first six followed by ‘et al.’.

The words ‘ dissertation ’ and ‘thesis’ both refer to a large written research project undertaken to complete a degree, but they are used differently depending on the country:

• In the UK, you write a dissertation at the end of a bachelor’s or master’s degree, and you write a thesis to complete a PhD.
• In the US, it’s the other way around: you may write a thesis at the end of a bachelor’s or master’s degree, and you write a dissertation to complete a PhD.

The main difference is in terms of scale – a dissertation is usually much longer than the other essays you complete during your degree.

Another key difference is that you are given much more independence when working on a dissertation. You choose your own dissertation topic , and you have to conduct the research and write the dissertation yourself (with some assistance from your supervisor).

Dissertation word counts vary widely across different fields, institutions, and levels of education:

• An undergraduate dissertation is typically 8,000–15,000 words
• A master’s dissertation is typically 12,000–50,000 words
• A PhD thesis is typically book-length: 70,000–100,000 words

However, none of these are strict guidelines – your word count may be lower or higher than the numbers stated here. Always check the guidelines provided by your university to determine how long your own dissertation should be.

At the bachelor’s and master’s levels, the dissertation is usually the main focus of your final year. You might work on it (alongside other classes) for the entirety of the final year, or for the last six months. This includes formulating an idea, doing the research, and writing up.

A PhD thesis takes a longer time, as the thesis is the main focus of the degree. A PhD thesis might be being formulated and worked on for the whole four years of the degree program. The writing process alone can take around 18 months.

References should be included in your text whenever you use words, ideas, or information from a source. A source can be anything from a book or journal article to a website or YouTube video.

If you don’t acknowledge your sources, you can get in trouble for plagiarism .

Your university should tell you which referencing style to follow. If you’re unsure, check with a supervisor. Commonly used styles include:

• Harvard referencing , the most commonly used style in UK universities.
• MHRA , used in humanities subjects.
• APA , used in the social sciences.
• Vancouver , used in biomedicine.
• OSCOLA , used in law.

Your university may have its own referencing style guide.

If you are allowed to choose which style to follow, we recommend Harvard referencing, as it is a straightforward and widely used style.

To avoid plagiarism , always include a reference when you use words, ideas or information from a source. This shows that you are not trying to pass the work of others off as your own.

You must also properly quote or paraphrase the source. If you’re not sure whether you’ve done this correctly, you can use the Scribbr Plagiarism Checker to find and correct any mistakes.

In Harvard style , when you quote directly from a source that includes page numbers, your in-text citation must include a page number. For example: (Smith, 2014, p. 33).

You can also include page numbers to point the reader towards a passage that you paraphrased . If you refer to the general ideas or findings of the source as a whole, you don’t need to include a page number.

When you want to use a quote but can’t access the original source, you can cite it indirectly. In the in-text citation , first mention the source you want to refer to, and then the source in which you found it. For example:

It’s advisable to avoid indirect citations wherever possible, because they suggest you don’t have full knowledge of the sources you’re citing. Only use an indirect citation if you can’t reasonably gain access to the original source.

In Harvard style referencing , to distinguish between two sources by the same author that were published in the same year, you add a different letter after the year for each source:

• (Smith, 2019a)
• (Smith, 2019b)

Add ‘a’ to the first one you cite, ‘b’ to the second, and so on. Do the same in your bibliography or reference list .

To create a hanging indent for your bibliography or reference list :

• Highlight all the entries
• Click on the arrow in the bottom-right corner of the ‘Paragraph’ tab in the top menu.
• In the pop-up window, under ‘Special’ in the ‘Indentation’ section, use the drop-down menu to select ‘Hanging’.
• Then close the window with ‘OK’.

It’s important to assess the reliability of information found online. Look for sources from established publications and institutions with expertise (e.g. peer-reviewed journals and government agencies).

The CRAAP test (currency, relevance, authority, accuracy, purpose) can aid you in assessing sources, as can our list of credible sources . You should generally avoid citing websites like Wikipedia that can be edited by anyone – instead, look for the original source of the information in the “References” section.

You can generally omit page numbers in your in-text citations of online sources which don’t have them. But when you quote or paraphrase a specific passage from a particularly long online source, it’s useful to find an alternate location marker.

For text-based sources, you can use paragraph numbers (e.g. ‘para. 4’) or headings (e.g. ‘under “Methodology”’). With video or audio sources, use a timestamp (e.g. ‘10:15’).

In the acknowledgements of your thesis or dissertation, you should first thank those who helped you academically or professionally, such as your supervisor, funders, and other academics.

Then you can include personal thanks to friends, family members, or anyone else who supported you during the process.

Yes, it’s important to thank your supervisor(s) in the acknowledgements section of your thesis or dissertation .

Even if you feel your supervisor did not contribute greatly to the final product, you still should acknowledge them, if only for a very brief thank you. If you do not include your supervisor, it may be seen as a snub.

The acknowledgements are generally included at the very beginning of your thesis or dissertation, directly after the title page and before the abstract .

In a thesis or dissertation, the acknowledgements should usually be no longer than one page. There is no minimum length.

You may acknowledge God in your thesis or dissertation acknowledgements , but be sure to follow academic convention by also thanking the relevant members of academia, as well as family, colleagues, and friends who helped you.

All level 1 and 2 headings should be included in your table of contents . That means the titles of your chapters and the main sections within them.

The contents should also include all appendices and the lists of tables and figures, if applicable, as well as your reference list .

Do not include the acknowledgements or abstract   in the table of contents.

To automatically insert a table of contents in Microsoft Word, follow these steps:

• Apply heading styles throughout the document.
• In the references section in the ribbon, locate the Table of Contents group.
• Click the arrow next to the Table of Contents icon and select Custom Table of Contents.
• Select which levels of headings you would like to include in the table of contents.

Make sure to update your table of contents if you move text or change headings. To update, simply right click and select Update Field.

The table of contents in a thesis or dissertation always goes between your abstract and your introduction.

An abbreviation is a shortened version of an existing word, such as Dr for Doctor. In contrast, an acronym uses the first letter of each word to create a wholly new word, such as UNESCO (an acronym for the United Nations Educational, Scientific and Cultural Organization).

Your dissertation sometimes contains a list of abbreviations .

As a rule of thumb, write the explanation in full the first time you use an acronym or abbreviation. You can then proceed with the shortened version. However, if the abbreviation is very common (like UK or PC), then you can just use the abbreviated version straight away.

Be sure to add each abbreviation in your list of abbreviations !

If you only used a few abbreviations in your thesis or dissertation, you don’t necessarily need to include a list of abbreviations .

If your abbreviations are numerous, or if you think they won’t be known to your audience, it’s never a bad idea to add one. They can also improve readability, minimising confusion about abbreviations unfamiliar to your reader.

A list of abbreviations is a list of all the abbreviations you used in your thesis or dissertation. It should appear at the beginning of your document, immediately after your table of contents . It should always be in alphabetical order.

Fishbone diagrams have a few different names that are used interchangeably, including herringbone diagram, cause-and-effect diagram, and Ishikawa diagram.

These are all ways to refer to the same thing– a problem-solving approach that uses a fish-shaped diagram to model possible root causes of problems and troubleshoot solutions.

Fishbone diagrams (also called herringbone diagrams, cause-and-effect diagrams, and Ishikawa diagrams) are most popular in fields of quality management. They are also commonly used in nursing and healthcare, or as a brainstorming technique for students.

Some synonyms and near synonyms of among include:

• In the company of
• In the middle of
• Surrounded by

Some synonyms and near synonyms of between  include:

• In the space separating
• In the time separating

In spite of   is a preposition used to mean ‘ regardless of ‘, ‘notwithstanding’, or ‘even though’.

It’s always used in a subordinate clause to contrast with the information given in the main clause of a sentence (e.g., ‘Amy continued to watch TV, in spite of the time’).

Despite   is a preposition used to mean ‘ regardless of ‘, ‘notwithstanding’, or ‘even though’.

It’s used in a subordinate clause to contrast with information given in the main clause of a sentence (e.g., ‘Despite the stress, Joe loves his job’).

‘Log in’ is a phrasal verb meaning ‘connect to an electronic device, system, or app’. The preposition ‘to’ is often used directly after the verb; ‘in’ and ‘to’ should be written as two separate words (e.g., ‘ log in to the app to update privacy settings’).

‘Log into’ is sometimes used instead of ‘log in to’, but this is generally considered incorrect (as is ‘login to’).

Some synonyms and near synonyms of ensure include:

• Make certain

Some synonyms and near synonyms of assure  include:

Rest assured is an expression meaning ‘you can be certain’ (e.g., ‘Rest assured, I will find your cat’). ‘Assured’ is the adjectival form of the verb assure , meaning ‘convince’ or ‘persuade’.

Some synonyms and near synonyms for council include:

There are numerous synonyms and near synonyms for the two meanings of counsel :

AI writing tools can be used to perform a variety of tasks.

Generative AI writing tools (like ChatGPT ) generate text based on human inputs and can be used for interactive learning, to provide feedback, or to generate research questions or outlines.

These tools can also be used to paraphrase or summarise text or to identify grammar and punctuation mistakes. Y ou can also use Scribbr’s free paraphrasing tool , summarising tool , and grammar checker , which are designed specifically for these purposes.

Using AI writing tools (like ChatGPT ) to write your essay is usually considered plagiarism and may result in penalisation, unless it is allowed by your university. Text generated by AI tools is based on existing texts and therefore cannot provide unique insights. Furthermore, these outputs sometimes contain factual inaccuracies or grammar mistakes.

However, AI writing tools can be used effectively as a source of feedback and inspiration for your writing (e.g., to generate research questions ). Other AI tools, like grammar checkers, can help identify and eliminate grammar and punctuation mistakes to enhance your writing.

The Scribbr Knowledge Base is a collection of free resources to help you succeed in academic research, writing, and citation. Every week, we publish helpful step-by-step guides, clear examples, simple templates, engaging videos, and more.

The Knowledge Base is for students at all levels. Whether you’re writing your first essay, working on your bachelor’s or master’s dissertation, or getting to grips with your PhD research, we’ve got you covered.

As well as the Knowledge Base, Scribbr provides many other tools and services to support you in academic writing and citation:

• Create your citations and manage your reference list with our free Reference Generators in APA and MLA style.
• Scan your paper for in-text citation errors and inconsistencies with our innovative APA Citation Checker .
• Avoid accidental plagiarism with our reliable Plagiarism Checker .
• Polish your writing and get feedback on structure and clarity with our Proofreading & Editing services .

Yes! We’re happy for educators to use our content, and we’ve even adapted some of our articles into ready-made lecture slides .

You are free to display, distribute, and adapt Scribbr materials in your classes or upload them in private learning environments like Blackboard. We only ask that you credit Scribbr for any content you use.

We’re always striving to improve the Knowledge Base. If you have an idea for a topic we should cover, or you notice a mistake in any of our articles, let us know by emailing [email protected] .

The consequences of plagiarism vary depending on the type of plagiarism and the context in which it occurs. For example, submitting a whole paper by someone else will have the most severe consequences, while accidental citation errors are considered less serious.

If you’re a student, then you might fail the course, be suspended or expelled, or be obligated to attend a workshop on plagiarism. It depends on whether it’s your first offence or you’ve done it before.

As an academic or professional, plagiarising seriously damages your reputation. You might also lose your research funding or your job, and you could even face legal consequences for copyright infringement.

Paraphrasing without crediting the original author is a form of plagiarism , because you’re presenting someone else’s ideas as if they were your own.

However, paraphrasing is not plagiarism if you correctly reference the source . This means including an in-text referencing and a full reference , formatted according to your required citation style (e.g., Harvard , Vancouver ).

As well as referencing your source, make sure that any paraphrased text is completely rewritten in your own words.

Accidental plagiarism is one of the most common examples of plagiarism . Perhaps you forgot to cite a source, or paraphrased something a bit too closely. Maybe you can’t remember where you got an idea from, and aren’t totally sure if it’s original or not.

These all count as plagiarism, even though you didn’t do it on purpose. When in doubt, make sure you’re citing your sources . Also consider running your work through a plagiarism checker tool prior to submission, which work by using advanced database software to scan for matches between your text and existing texts.

Scribbr’s Plagiarism Checker takes less than 10 minutes and can help you turn in your paper with confidence.

The accuracy depends on the plagiarism checker you use. Per our in-depth research , Scribbr is the most accurate plagiarism checker. Many free plagiarism checkers fail to detect all plagiarism or falsely flag text as plagiarism.

Plagiarism checkers work by using advanced database software to scan for matches between your text and existing texts. Their accuracy is determined by two factors: the algorithm (which recognises the plagiarism) and the size of the database (with which your document is compared).

To avoid plagiarism when summarising an article or other source, follow these two rules:

• Write the summary entirely in your own words by   paraphrasing the author’s ideas.
• Reference the source with an in-text citation and a full reference so your reader can easily find the original text.

Plagiarism can be detected by your professor or readers if the tone, formatting, or style of your text is different in different parts of your paper, or if they’re familiar with the plagiarised source.

Many universities also use   plagiarism detection software like Turnitin’s, which compares your text to a large database of other sources, flagging any similarities that come up.

It can be easier than you think to commit plagiarism by accident. Consider using a   plagiarism checker prior to submitting your essay to ensure you haven’t missed any citations.

Some examples of plagiarism include:

• Copying and pasting a Wikipedia article into the body of an assignment
• Quoting a source without including a citation
• Not paraphrasing a source properly (e.g. maintaining wording too close to the original)
• Forgetting to cite the source of an idea

The most surefire way to   avoid plagiarism is to always cite your sources . When in doubt, cite!

Global plagiarism means taking an entire work written by someone else and passing it off as your own. This can include getting someone else to write an essay or assignment for you, or submitting a text you found online as your own work.

Global plagiarism is one of the most serious types of plagiarism because it involves deliberately and directly lying about the authorship of a work. It can have severe consequences for students and professionals alike.

Verbatim plagiarism means copying text from a source and pasting it directly into your own document without giving proper credit.

If the structure and the majority of the words are the same as in the original source, then you are committing verbatim plagiarism. This is the case even if you delete a few words or replace them with synonyms.

If you want to use an author’s exact words, you need to quote the original source by putting the copied text in quotation marks and including an   in-text citation .

Patchwork plagiarism , also called mosaic plagiarism, means copying phrases, passages, or ideas from various existing sources and combining them to create a new text. This includes slightly rephrasing some of the content, while keeping many of the same words and the same structure as the original.

While this type of plagiarism is more insidious than simply copying and pasting directly from a source, plagiarism checkers like Turnitin’s can still easily detect it.

To avoid plagiarism in any form, remember to reference your sources .

Yes, reusing your own work without citation is considered self-plagiarism . This can range from resubmitting an entire assignment to reusing passages or data from something you’ve handed in previously.

Self-plagiarism often has the same consequences as other types of plagiarism . If you want to reuse content you wrote in the past, make sure to check your university’s policy or consult your professor.

If you are reusing content or data you used in a previous assignment, make sure to cite yourself. You can cite yourself the same way you would cite any other source: simply follow the directions for the citation style you are using.

Keep in mind that reusing prior content can be considered self-plagiarism , so make sure you ask your instructor or consult your university’s handbook prior to doing so.

Most institutions have an internal database of previously submitted student assignments. Turnitin can check for self-plagiarism by comparing your paper against this database. If you’ve reused parts of an assignment you already submitted, it will flag any similarities as potential plagiarism.

Online plagiarism checkers don’t have access to your institution’s database, so they can’t detect self-plagiarism of unpublished work. If you’re worried about accidentally self-plagiarising, you can use Scribbr’s Self-Plagiarism Checker to upload your unpublished documents and check them for similarities.

Plagiarism has serious consequences and can be illegal in certain scenarios.

While most of the time plagiarism in an undergraduate setting is not illegal, plagiarism or self-plagiarism in a professional academic setting can lead to legal action, including copyright infringement and fraud. Many scholarly journals do not allow you to submit the same work to more than one journal, and if you do not credit a coauthor, you could be legally defrauding them.

Even if you aren’t breaking the law, plagiarism can seriously impact your academic career. While the exact consequences of plagiarism vary by institution and severity, common consequences include a lower grade, automatically failing a course, academic suspension or probation, and even expulsion.

Self-plagiarism means recycling work that you’ve previously published or submitted as an assignment. It’s considered academic dishonesty to present something as brand new when you’ve already gotten credit and perhaps feedback for it in the past.

If you want to refer to ideas or data from previous work, be sure to cite yourself.

Academic integrity means being honest, ethical, and thorough in your academic work. To maintain academic integrity, you should avoid misleading your readers about any part of your research and refrain from offences like plagiarism and contract cheating, which are examples of academic misconduct.

Academic dishonesty refers to deceitful or misleading behavior in an academic setting. Academic dishonesty can occur intentionally or unintentionally, and it varies in severity.

It can encompass paying for a pre-written essay, cheating on an exam, or committing plagiarism . It can also include helping others cheat, copying a friend’s homework answers, or even pretending to be sick to miss an exam.

Academic dishonesty doesn’t just occur in a classroom setting, but also in research and other academic-adjacent fields.

Consequences of academic dishonesty depend on the severity of the offence and your institution’s policy. They can range from a warning for a first offence to a failing grade in a course to expulsion from your university.

For those in certain fields, such as nursing, engineering, or lab sciences, not learning fundamentals properly can directly impact the health and safety of others. For those working in academia or research, academic dishonesty impacts your professional reputation, leading others to doubt your future work.

Academic dishonesty can be intentional or unintentional, ranging from something as simple as claiming to have read something you didn’t to copying your neighbour’s answers on an exam.

You can commit academic dishonesty with the best of intentions, such as helping a friend cheat on a paper. Severe academic dishonesty can include buying a pre-written essay or the answers to a multiple-choice test, or falsifying a medical emergency to avoid taking a final exam.

Plagiarism means presenting someone else’s work as your own without giving proper credit to the original author. In academic writing, plagiarism involves using words, ideas, or information from a source without including a citation .

Plagiarism can have serious consequences , even when it’s done accidentally. To avoid plagiarism, it’s important to keep track of your sources and cite them correctly.

Common knowledge does not need to be cited. However, you should be extra careful when deciding what counts as common knowledge.

Common knowledge encompasses information that the average educated reader would accept as true without needing the extra validation of a source or citation.

Common knowledge should be widely known, undisputed, and easily verified. When in doubt, always cite your sources.

Most online plagiarism checkers only have access to public databases, whose software doesn’t allow you to compare two documents for plagiarism.

However, in addition to our Plagiarism Checker , Scribbr also offers an Self-Plagiarism Checker . This is an add-on tool that lets you compare your paper with unpublished or private documents. This way you can rest assured that you haven’t unintentionally plagiarised or self-plagiarised .

Compare two sources for plagiarism

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Methodology refers to the overarching strategy and rationale of your research project . It involves studying the methods used in your field and the theories or principles behind them, in order to develop an approach that matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. experiments, surveys , and statistical tests ).

In shorter scientific papers, where the aim is to report the findings of a specific study, you might simply describe what you did in a methods section .

In a longer or more complex research project, such as a thesis or dissertation , you will probably include a methodology section , where you explain your approach to answering the research questions and cite relevant sources to support your choice of methods.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organise your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organisation to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

Operationalisation means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioural avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalise the variables that you want to measure.

Triangulation in research means using multiple datasets, methods, theories and/or investigators to address a research question. It’s a research strategy that can help you enhance the validity and credibility of your findings.

Triangulation is mainly used in qualitative research , but it’s also commonly applied in quantitative research . Mixed methods research always uses triangulation.

These are four of the most common mixed methods designs :

• Convergent parallel: Quantitative and qualitative data are collected at the same time and analysed separately. After both analyses are complete, compare your results to draw overall conclusions. 
• Embedded: Quantitative and qualitative data are collected at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.
• Explanatory sequential: Quantitative data is collected and analysed first, followed by qualitative data. You can use this design if you think your qualitative data will explain and contextualise your quantitative findings.
• Exploratory sequential: Qualitative data is collected and analysed first, followed by quantitative data. You can use this design if you think the quantitative data will confirm or validate your qualitative findings.

An observational study could be a good fit for your research if your research question is based on things you observe. If you have ethical, logistical, or practical concerns that make an experimental design challenging, consider an observational study. Remember that in an observational study, it is critical that there be no interference or manipulation of the research subjects. Since it’s not an experiment, there are no control or treatment groups either.

The key difference between observational studies and experiments is that, done correctly, an observational study will never influence the responses or behaviours of participants. Experimental designs will have a treatment condition applied to at least a portion of participants.

Exploratory research explores the main aspects of a new or barely researched question.

Explanatory research explains the causes and effects of an already widely researched question.

Experimental designs are a set of procedures that you plan in order to examine the relationship between variables that interest you.

To design a successful experiment, first identify:

• A testable hypothesis
• One or more independent variables that you will manipulate
• One or more dependent variables that you will measure

When designing the experiment, first decide:

• How your variable(s) will be manipulated
• How you will control for any potential confounding or lurking variables
• How many subjects you will include
• How you will assign treatments to your subjects

There are four main types of triangulation :

• Data triangulation : Using data from different times, spaces, and people
• Investigator triangulation : Involving multiple researchers in collecting or analysing data
• Theory triangulation : Using varying theoretical perspectives in your research
• Methodological triangulation : Using different methodologies to approach the same topic

Triangulation can help:

• Reduce bias that comes from using a single method, theory, or investigator
• Enhance validity by approaching the same topic with different tools
• Establish credibility by giving you a complete picture of the research problem

But triangulation can also pose problems:

• It’s time-consuming and labour-intensive, often involving an interdisciplinary team.
• Your results may be inconsistent or even contradictory.

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word ‘between’ means that you’re comparing different conditions between groups, while the word ‘within’ means you’re comparing different conditions within the same group.

A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference between this and a true experiment is that the groups are not randomly assigned.

In experimental research, random assignment is a way of placing participants from your sample into different groups using randomisation. With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.

Quasi-experimental design is most useful in situations where it would be unethical or impractical to run a true experiment .

Quasi-experiments have lower internal validity than true experiments, but they often have higher external validity  as they can use real-world interventions instead of artificial laboratory settings.

Within-subjects designs have many potential threats to internal validity , but they are also very statistically powerful .

Advantages:

• Only requires small samples
• Statistically powerful
• Removes the effects of individual differences on the outcomes

Disadvantages:

• Internal validity threats reduce the likelihood of establishing a direct relationship between variables
• Time-related effects, such as growth, can influence the outcomes
• Carryover effects mean that the specific order of different treatments affect the outcomes

Yes. Between-subjects and within-subjects designs can be combined in a single study when you have two or more independent variables (a factorial design). In a mixed factorial design, one variable is altered between subjects and another is altered within subjects.

In a factorial design, multiple independent variables are tested.

If you test two variables, each level of one independent variable is combined with each level of the other independent variable to create different conditions.

While a between-subjects design has fewer threats to internal validity , it also requires more participants for high statistical power than a within-subjects design .

• Prevents carryover effects of learning and fatigue.
• Shorter study duration.
• Needs larger samples for high power.
• Uses more resources to recruit participants, administer sessions, cover costs, etc.
• Individual differences may be an alternative explanation for results.

Samples are used to make inferences about populations . Samples are easier to collect data from because they are practical, cost-effective, convenient, and manageable.

Probability sampling means that every member of the target population has a known chance of being included in the sample.

Probability sampling methods include simple random sampling , systematic sampling , stratified sampling , and cluster sampling .

In non-probability sampling , the sample is selected based on non-random criteria, and not every member of the population has a chance of being included.

Common non-probability sampling methods include convenience sampling , voluntary response sampling, purposive sampling , snowball sampling , and quota sampling .

In multistage sampling , or multistage cluster sampling, you draw a sample from a population using smaller and smaller groups at each stage.

This method is often used to collect data from a large, geographically spread group of people in national surveys, for example. You take advantage of hierarchical groupings (e.g., from county to city to neighbourhood) to create a sample that’s less expensive and time-consuming to collect data from.

Sampling bias occurs when some members of a population are systematically more likely to be selected in a sample than others.

Simple random sampling is a type of probability sampling in which the researcher randomly selects a subset of participants from a population . Each member of the population has an equal chance of being selected. Data are then collected from as large a percentage as possible of this random subset.

The American Community Survey  is an example of simple random sampling . In order to collect detailed data on the population of the US, the Census Bureau officials randomly select 3.5 million households per year and use a variety of methods to convince them to fill out the survey.

If properly implemented, simple random sampling is usually the best sampling method for ensuring both internal and external validity . However, it can sometimes be impractical and expensive to implement, depending on the size of the population to be studied,

If you have a list of every member of the population and the ability to reach whichever members are selected, you can use simple random sampling.

Cluster sampling is more time- and cost-efficient than other probability sampling methods , particularly when it comes to large samples spread across a wide geographical area.

However, it provides less statistical certainty than other methods, such as simple random sampling , because it is difficult to ensure that your clusters properly represent the population as a whole.

There are three types of cluster sampling : single-stage, double-stage and multi-stage clustering. In all three types, you first divide the population into clusters, then randomly select clusters for use in your sample.

• In single-stage sampling , you collect data from every unit within the selected clusters.
• In double-stage sampling , you select a random sample of units from within the clusters.
• In multi-stage sampling , you repeat the procedure of randomly sampling elements from within the clusters until you have reached a manageable sample.

Cluster sampling is a probability sampling method in which you divide a population into clusters, such as districts or schools, and then randomly select some of these clusters as your sample.

The clusters should ideally each be mini-representations of the population as a whole.

In multistage sampling , you can use probability or non-probability sampling methods.

For a probability sample, you have to probability sampling at every stage. You can mix it up by using simple random sampling , systematic sampling , or stratified sampling to select units at different stages, depending on what is applicable and relevant to your study.

Multistage sampling can simplify data collection when you have large, geographically spread samples, and you can obtain a probability sample without a complete sampling frame.

But multistage sampling may not lead to a representative sample, and larger samples are needed for multistage samples to achieve the statistical properties of simple random samples .

In stratified sampling , researchers divide subjects into subgroups called strata based on characteristics that they share (e.g., race, gender, educational attainment).

Once divided, each subgroup is randomly sampled using another probability sampling method .

You should use stratified sampling when your sample can be divided into mutually exclusive and exhaustive subgroups that you believe will take on different mean values for the variable that you’re studying.

Using stratified sampling will allow you to obtain more precise (with lower variance ) statistical estimates of whatever you are trying to measure.

For example, say you want to investigate how income differs based on educational attainment, but you know that this relationship can vary based on race. Using stratified sampling, you can ensure you obtain a large enough sample from each racial group, allowing you to draw more precise conclusions.

Yes, you can create a stratified sample using multiple characteristics, but you must ensure that every participant in your study belongs to one and only one subgroup. In this case, you multiply the numbers of subgroups for each characteristic to get the total number of groups.

For example, if you were stratifying by location with three subgroups (urban, rural, or suburban) and marital status with five subgroups (single, divorced, widowed, married, or partnered), you would have 3 × 5 = 15 subgroups.

There are three key steps in systematic sampling :

• Define and list your population , ensuring that it is not ordered in a cyclical or periodic order.
• Decide on your sample size and calculate your interval, k , by dividing your population by your target sample size.
• Choose every k th member of the population as your sample.

Systematic sampling is a probability sampling method where researchers select members of the population at a regular interval – for example, by selecting every 15th person on a list of the population. If the population is in a random order, this can imitate the benefits of simple random sampling .

Populations are used when a research question requires data from every member of the population. This is usually only feasible when the population is small and easily accessible.

A statistic refers to measures about the sample , while a parameter refers to measures about the population .

A sampling error is the difference between a population parameter and a sample statistic .

There are eight threats to internal validity : history, maturation, instrumentation, testing, selection bias , regression to the mean, social interaction, and attrition .

Internal validity is the extent to which you can be confident that a cause-and-effect relationship established in a study cannot be explained by other factors.

Attrition bias is a threat to internal validity . In experiments, differential rates of attrition between treatment and control groups can skew results.

This bias can affect the relationship between your independent and dependent variables . It can make variables appear to be correlated when they are not, or vice versa.

The external validity of a study is the extent to which you can generalise your findings to different groups of people, situations, and measures.

The two types of external validity are population validity (whether you can generalise to other groups of people) and ecological validity (whether you can generalise to other situations and settings).

There are seven threats to external validity : selection bias , history, experimenter effect, Hawthorne effect , testing effect, aptitude-treatment, and situation effect.

Attrition bias can skew your sample so that your final sample differs significantly from your original sample. Your sample is biased because some groups from your population are underrepresented.

With a biased final sample, you may not be able to generalise your findings to the original population that you sampled from, so your external validity is compromised.

Construct validity is about how well a test measures the concept it was designed to evaluate. It’s one of four types of measurement validity , which includes construct validity, face validity , and criterion validity.

There are two subtypes of construct validity.

• Convergent validity : The extent to which your measure corresponds to measures of related constructs
• Discriminant validity: The extent to which your measure is unrelated or negatively related to measures of distinct constructs

When designing or evaluating a measure, construct validity helps you ensure you’re actually measuring the construct you’re interested in. If you don’t have construct validity, you may inadvertently measure unrelated or distinct constructs and lose precision in your research.

Construct validity is often considered the overarching type of measurement validity ,  because it covers all of the other types. You need to have face validity , content validity, and criterion validity to achieve construct validity.

Statistical analyses are often applied to test validity with data from your measures. You test convergent validity and discriminant validity with correlations to see if results from your test are positively or negatively related to those of other established tests.

You can also use regression analyses to assess whether your measure is actually predictive of outcomes that you expect it to predict theoretically. A regression analysis that supports your expectations strengthens your claim of construct validity .

Face validity is about whether a test appears to measure what it’s supposed to measure. This type of validity is concerned with whether a measure seems relevant and appropriate for what it’s assessing only on the surface.

Face validity is important because it’s a simple first step to measuring the overall validity of a test or technique. It’s a relatively intuitive, quick, and easy way to start checking whether a new measure seems useful at first glance.

Good face validity means that anyone who reviews your measure says that it seems to be measuring what it’s supposed to. With poor face validity, someone reviewing your measure may be left confused about what you’re measuring and why you’re using this method.

It’s often best to ask a variety of people to review your measurements. You can ask experts, such as other researchers, or laypeople, such as potential participants, to judge the face validity of tests.

While experts have a deep understanding of research methods , the people you’re studying can provide you with valuable insights you may have missed otherwise.

There are many different types of inductive reasoning that people use formally or informally.

Here are a few common types:

• Inductive generalisation : You use observations about a sample to come to a conclusion about the population it came from.
• Statistical generalisation: You use specific numbers about samples to make statements about populations.
• Causal reasoning: You make cause-and-effect links between different things.
• Sign reasoning: You make a conclusion about a correlational relationship between different things.
• Analogical reasoning: You make a conclusion about something based on its similarities to something else.

Inductive reasoning is a bottom-up approach, while deductive reasoning is top-down.

Inductive reasoning takes you from the specific to the general, while in deductive reasoning, you make inferences by going from general premises to specific conclusions.

In inductive research , you start by making observations or gathering data. Then, you take a broad scan of your data and search for patterns. Finally, you make general conclusions that you might incorporate into theories.

Inductive reasoning is a method of drawing conclusions by going from the specific to the general. It’s usually contrasted with deductive reasoning, where you proceed from general information to specific conclusions.

Inductive reasoning is also called inductive logic or bottom-up reasoning.

Deductive reasoning is a logical approach where you progress from general ideas to specific conclusions. It’s often contrasted with inductive reasoning , where you start with specific observations and form general conclusions.

Deductive reasoning is also called deductive logic.

Deductive reasoning is commonly used in scientific research, and it’s especially associated with quantitative research .

In research, you might have come across something called the hypothetico-deductive method . It’s the scientific method of testing hypotheses to check whether your predictions are substantiated by real-world data.

A dependent variable is what changes as a result of the independent variable manipulation in experiments . It’s what you’re interested in measuring, and it ‘depends’ on your independent variable.

In statistics, dependent variables are also called:

• Response variables (they respond to a change in another variable)
• Outcome variables (they represent the outcome you want to measure)
• Left-hand-side variables (they appear on the left-hand side of a regression equation)

An independent variable is the variable you manipulate, control, or vary in an experimental study to explore its effects. It’s called ‘independent’ because it’s not influenced by any other variables in the study.

Independent variables are also called:

• Explanatory variables (they explain an event or outcome)
• Predictor variables (they can be used to predict the value of a dependent variable)
• Right-hand-side variables (they appear on the right-hand side of a regression equation)

A correlation is usually tested for two variables at a time, but you can test correlations between three or more variables.

On graphs, the explanatory variable is conventionally placed on the x -axis, while the response variable is placed on the y -axis.

• If you have quantitative variables , use a scatterplot or a line graph.
• If your response variable is categorical, use a scatterplot or a line graph.
• If your explanatory variable is categorical, use a bar graph.

The term ‘ explanatory variable ‘ is sometimes preferred over ‘ independent variable ‘ because, in real-world contexts, independent variables are often influenced by other variables. This means they aren’t totally independent.

Multiple independent variables may also be correlated with each other, so ‘explanatory variables’ is a more appropriate term.

The difference between explanatory and response variables is simple:

• An explanatory variable is the expected cause, and it explains the results.
• A response variable is the expected effect, and it responds to other variables.

There are 4 main types of extraneous variables :

• Demand characteristics : Environmental cues that encourage participants to conform to researchers’ expectations
• Experimenter effects : Unintentional actions by researchers that influence study outcomes
• Situational variables : Eenvironmental variables that alter participants’ behaviours
• Participant variables : Any characteristic or aspect of a participant’s background that could affect study results

An extraneous variable is any variable that you’re not investigating that can potentially affect the dependent variable of your research study.

A confounding variable is a type of extraneous variable that not only affects the dependent variable, but is also related to the independent variable.

‘Controlling for a variable’ means measuring extraneous variables and accounting for them statistically to remove their effects on other variables.

Researchers often model control variable data along with independent and dependent variable data in regression analyses and ANCOVAs . That way, you can isolate the control variable’s effects from the relationship between the variables of interest.

Control variables help you establish a correlational or causal relationship between variables by enhancing internal validity .

If you don’t control relevant extraneous variables , they may influence the outcomes of your study, and you may not be able to demonstrate that your results are really an effect of your independent variable .

A control variable is any variable that’s held constant in a research study. It’s not a variable of interest in the study, but it’s controlled because it could influence the outcomes.

In statistics, ordinal and nominal variables are both considered categorical variables .

Even though ordinal data can sometimes be numerical, not all mathematical operations can be performed on them.

In scientific research, concepts are the abstract ideas or phenomena that are being studied (e.g., educational achievement). Variables are properties or characteristics of the concept (e.g., performance at school), while indicators are ways of measuring or quantifying variables (e.g., yearly grade reports).

The process of turning abstract concepts into measurable variables and indicators is called operationalisation .

There are several methods you can use to decrease the impact of confounding variables on your research: restriction, matching, statistical control, and randomisation.

In restriction , you restrict your sample by only including certain subjects that have the same values of potential confounding variables.

In matching , you match each of the subjects in your treatment group with a counterpart in the comparison group. The matched subjects have the same values on any potential confounding variables, and only differ in the independent variable .

In statistical control , you include potential confounders as variables in your regression .

In randomisation , you randomly assign the treatment (or independent variable) in your study to a sufficiently large number of subjects, which allows you to control for all potential confounding variables.

A confounding variable is closely related to both the independent and dependent variables in a study. An independent variable represents the supposed cause , while the dependent variable is the supposed effect . A confounding variable is a third variable that influences both the independent and dependent variables.

Failing to account for confounding variables can cause you to wrongly estimate the relationship between your independent and dependent variables.

To ensure the internal validity of your research, you must consider the impact of confounding variables. If you fail to account for them, you might over- or underestimate the causal relationship between your independent and dependent variables , or even find a causal relationship where none exists.

Yes, but including more than one of either type requires multiple research questions .

For example, if you are interested in the effect of a diet on health, you can use multiple measures of health: blood sugar, blood pressure, weight, pulse, and many more. Each of these is its own dependent variable with its own research question.

You could also choose to look at the effect of exercise levels as well as diet, or even the additional effect of the two combined. Each of these is a separate independent variable .

To ensure the internal validity of an experiment , you should only change one independent variable at a time.

No. The value of a dependent variable depends on an independent variable, so a variable cannot be both independent and dependent at the same time. It must be either the cause or the effect, not both.

You want to find out how blood sugar levels are affected by drinking diet cola and regular cola, so you conduct an experiment .

• The type of cola – diet or regular – is the independent variable .
• The level of blood sugar that you measure is the dependent variable – it changes depending on the type of cola.

Determining cause and effect is one of the most important parts of scientific research. It’s essential to know which is the cause – the independent variable – and which is the effect – the dependent variable.

Quantitative variables are any variables where the data represent amounts (e.g. height, weight, or age).

Categorical variables are any variables where the data represent groups. This includes rankings (e.g. finishing places in a race), classifications (e.g. brands of cereal), and binary outcomes (e.g. coin flips).

You need to know what type of variables you are working with to choose the right statistical test for your data and interpret your results .

Discrete and continuous variables are two types of quantitative variables :

• Discrete variables represent counts (e.g., the number of objects in a collection).
• Continuous variables represent measurable amounts (e.g., water volume or weight).

You can think of independent and dependent variables in terms of cause and effect: an independent variable is the variable you think is the cause , while a dependent variable is the effect .

In an experiment, you manipulate the independent variable and measure the outcome in the dependent variable. For example, in an experiment about the effect of nutrients on crop growth:

• The  independent variable  is the amount of nutrients added to the crop field.
• The  dependent variable is the biomass of the crops at harvest time.

Defining your variables, and deciding how you will manipulate and measure them, is an important part of experimental design .

Including mediators and moderators in your research helps you go beyond studying a simple relationship between two variables for a fuller picture of the real world. They are important to consider when studying complex correlational or causal relationships.

Mediators are part of the causal pathway of an effect, and they tell you how or why an effect takes place. Moderators usually help you judge the external validity of your study by identifying the limitations of when the relationship between variables holds.

If something is a mediating variable :

• It’s caused by the independent variable
• It influences the dependent variable
• When it’s taken into account, the statistical correlation between the independent and dependent variables is higher than when it isn’t considered

A confounder is a third variable that affects variables of interest and makes them seem related when they are not. In contrast, a mediator is the mechanism of a relationship between two variables: it explains the process by which they are related.

A mediator variable explains the process through which two variables are related, while a moderator variable affects the strength and direction of that relationship.

When conducting research, collecting original data has significant advantages:

• You can tailor data collection to your specific research aims (e.g., understanding the needs of your consumers or user testing your website).
• You can control and standardise the process for high reliability and validity (e.g., choosing appropriate measurements and sampling methods ).

However, there are also some drawbacks: data collection can be time-consuming, labour-intensive, and expensive. In some cases, it’s more efficient to use secondary data that has already been collected by someone else, but the data might be less reliable.

A structured interview is a data collection method that relies on asking questions in a set order to collect data on a topic. They are often quantitative in nature. Structured interviews are best used when:

• You already have a very clear understanding of your topic. Perhaps significant research has already been conducted, or you have done some prior research yourself, but you already possess a baseline for designing strong structured questions.
• You are constrained in terms of time or resources and need to analyse your data quickly and efficiently
• Your research question depends on strong parity between participants, with environmental conditions held constant

More flexible interview options include semi-structured interviews , unstructured interviews , and focus groups .

The interviewer effect is a type of bias that emerges when a characteristic of an interviewer (race, age, gender identity, etc.) influences the responses given by the interviewee.

There is a risk of an interviewer effect in all types of interviews , but it can be mitigated by writing really high-quality interview questions.

A semi-structured interview is a blend of structured and unstructured types of interviews. Semi-structured interviews are best used when:

• You have prior interview experience. Spontaneous questions are deceptively challenging, and it’s easy to accidentally ask a leading question or make a participant uncomfortable.
• Your research question is exploratory in nature. Participant answers can guide future research questions and help you develop a more robust knowledge base for future research.

An unstructured interview is the most flexible type of interview, but it is not always the best fit for your research topic.

Unstructured interviews are best used when:

• You are an experienced interviewer and have a very strong background in your research topic, since it is challenging to ask spontaneous, colloquial questions
• Your research question is exploratory in nature. While you may have developed hypotheses, you are open to discovering new or shifting viewpoints through the interview process.
• You are seeking descriptive data, and are ready to ask questions that will deepen and contextualise your initial thoughts and hypotheses
• Your research depends on forming connections with your participants and making them feel comfortable revealing deeper emotions, lived experiences, or thoughts

The four most common types of interviews are:

• Structured interviews : The questions are predetermined in both topic and order.
• Semi-structured interviews : A few questions are predetermined, but other questions aren’t planned.
• Unstructured interviews : None of the questions are predetermined.
• Focus group interviews : The questions are presented to a group instead of one individual.

A focus group is a research method that brings together a small group of people to answer questions in a moderated setting. The group is chosen due to predefined demographic traits, and the questions are designed to shed light on a topic of interest. It is one of four types of interviews .

Social desirability bias is the tendency for interview participants to give responses that will be viewed favourably by the interviewer or other participants. It occurs in all types of interviews and surveys , but is most common in semi-structured interviews , unstructured interviews , and focus groups .

Social desirability bias can be mitigated by ensuring participants feel at ease and comfortable sharing their views. Make sure to pay attention to your own body language and any physical or verbal cues, such as nodding or widening your eyes.

This type of bias in research can also occur in observations if the participants know they’re being observed. They might alter their behaviour accordingly.

As a rule of thumb, questions related to thoughts, beliefs, and feelings work well in focus groups . Take your time formulating strong questions, paying special attention to phrasing. Be careful to avoid leading questions , which can bias your responses.

Overall, your focus group questions should be:

• Open-ended and flexible
• Impossible to answer with ‘yes’ or ‘no’ (questions that start with ‘why’ or ‘how’ are often best)
• Unambiguous, getting straight to the point while still stimulating discussion
• Unbiased and neutral

The third variable and directionality problems are two main reasons why correlation isn’t causation .

The third variable problem means that a confounding variable affects both variables to make them seem causally related when they are not.

The directionality problem is when two variables correlate and might actually have a causal relationship, but it’s impossible to conclude which variable causes changes in the other.

Controlled experiments establish causality, whereas correlational studies only show associations between variables.

• In an experimental design , you manipulate an independent variable and measure its effect on a dependent variable. Other variables are controlled so they can’t impact the results.
• In a correlational design , you measure variables without manipulating any of them. You can test whether your variables change together, but you can’t be sure that one variable caused a change in another.

In general, correlational research is high in external validity while experimental research is high in internal validity .

A correlation coefficient is a single number that describes the strength and direction of the relationship between your variables.

Different types of correlation coefficients might be appropriate for your data based on their levels of measurement and distributions . The Pearson product-moment correlation coefficient (Pearson’s r ) is commonly used to assess a linear relationship between two quantitative variables.

A correlational research design investigates relationships between two variables (or more) without the researcher controlling or manipulating any of them. It’s a non-experimental type of quantitative research .

A correlation reflects the strength and/or direction of the association between two or more variables.

• A positive correlation means that both variables change in the same direction.
• A negative correlation means that the variables change in opposite directions.
• A zero correlation means there’s no relationship between the variables.

Longitudinal studies can last anywhere from weeks to decades, although they tend to be at least a year long.

The 1970 British Cohort Study , which has collected data on the lives of 17,000 Brits since their births in 1970, is one well-known example of a longitudinal study .

Longitudinal studies are better to establish the correct sequence of events, identify changes over time, and provide insight into cause-and-effect relationships, but they also tend to be more expensive and time-consuming than other types of studies.

Longitudinal studies and cross-sectional studies are two different types of research design . In a cross-sectional study you collect data from a population at a specific point in time; in a longitudinal study you repeatedly collect data from the same sample over an extended period of time.

Cross-sectional studies cannot establish a cause-and-effect relationship or analyse behaviour over a period of time. To investigate cause and effect, you need to do a longitudinal study or an experimental study .

Cross-sectional studies are less expensive and time-consuming than many other types of study. They can provide useful insights into a population’s characteristics and identify correlations for further research.

Sometimes only cross-sectional data are available for analysis; other times your research question may only require a cross-sectional study to answer it.

A hypothesis states your predictions about what your research will find. It is a tentative answer to your research question that has not yet been tested. For some research projects, you might have to write several hypotheses that address different aspects of your research question.

A hypothesis is not just a guess. It should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations, and statistical analysis of data).

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (‘ x affects y because …’).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses. In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Individual Likert-type questions are generally considered ordinal data , because the items have clear rank order, but don’t have an even distribution.

Overall Likert scale scores are sometimes treated as interval data. These scores are considered to have directionality and even spacing between them.

The type of data determines what statistical tests you should use to analyse your data.

A Likert scale is a rating scale that quantitatively assesses opinions, attitudes, or behaviours. It is made up of four or more questions that measure a single attitude or trait when response scores are combined.

To use a Likert scale in a survey , you present participants with Likert-type questions or statements, and a continuum of items, usually with five or seven possible responses, to capture their degree of agreement.

A questionnaire is a data collection tool or instrument, while a survey is an overarching research method that involves collecting and analysing data from people using questionnaires.

A true experiment (aka a controlled experiment) always includes at least one control group that doesn’t receive the experimental treatment.

However, some experiments use a within-subjects design to test treatments without a control group. In these designs, you usually compare one group’s outcomes before and after a treatment (instead of comparing outcomes between different groups).

For strong internal validity , it’s usually best to include a control group if possible. Without a control group, it’s harder to be certain that the outcome was caused by the experimental treatment and not by other variables.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

In a controlled experiment , all extraneous variables are held constant so that they can’t influence the results. Controlled experiments require:

• A control group that receives a standard treatment, a fake treatment, or no treatment
• Random assignment of participants to ensure the groups are equivalent

Depending on your study topic, there are various other methods of controlling variables .

Questionnaires can be self-administered or researcher-administered.

Self-administered questionnaires can be delivered online or in paper-and-pen formats, in person or by post. All questions are standardised so that all respondents receive the same questions with identical wording.

Researcher-administered questionnaires are interviews that take place by phone, in person, or online between researchers and respondents. You can gain deeper insights by clarifying questions for respondents or asking follow-up questions.

You can organise the questions logically, with a clear progression from simple to complex, or randomly between respondents. A logical flow helps respondents process the questionnaire easier and quicker, but it may lead to bias. Randomisation can minimise the bias from order effects.

Closed-ended, or restricted-choice, questions offer respondents a fixed set of choices to select from. These questions are easier to answer quickly.

Open-ended or long-form questions allow respondents to answer in their own words. Because there are no restrictions on their choices, respondents can answer in ways that researchers may not have otherwise considered.

Naturalistic observation is a qualitative research method where you record the behaviours of your research subjects in real-world settings. You avoid interfering or influencing anything in a naturalistic observation.

You can think of naturalistic observation as ‘people watching’ with a purpose.

Naturalistic observation is a valuable tool because of its flexibility, external validity , and suitability for topics that can’t be studied in a lab setting.

The downsides of naturalistic observation include its lack of scientific control , ethical considerations , and potential for bias from observers and subjects.

You can use several tactics to minimise observer bias .

• Use masking (blinding) to hide the purpose of your study from all observers.
• Triangulate your data with different data collection methods or sources.
• Use multiple observers and ensure inter-rater reliability.
• Train your observers to make sure data is consistently recorded between them.
• Standardise your observation procedures to make sure they are structured and clear.

The observer-expectancy effect occurs when researchers influence the results of their own study through interactions with participants.

Researchers’ own beliefs and expectations about the study results may unintentionally influence participants through demand characteristics .

Observer bias occurs when a researcher’s expectations, opinions, or prejudices influence what they perceive or record in a study. It usually affects studies when observers are aware of the research aims or hypotheses. This type of research bias is also called detection bias or ascertainment bias .

Data cleaning is necessary for valid and appropriate analyses. Dirty data contain inconsistencies or errors , but cleaning your data helps you minimise or resolve these.

Without data cleaning, you could end up with a Type I or II error in your conclusion. These types of erroneous conclusions can be practically significant with important consequences, because they lead to misplaced investments or missed opportunities.

Data cleaning involves spotting and resolving potential data inconsistencies or errors to improve your data quality. An error is any value (e.g., recorded weight) that doesn’t reflect the true value (e.g., actual weight) of something that’s being measured.

In this process, you review, analyse, detect, modify, or remove ‘dirty’ data to make your dataset ‘clean’. Data cleaning is also called data cleansing or data scrubbing.

Data cleaning takes place between data collection and data analyses. But you can use some methods even before collecting data.

For clean data, you should start by designing measures that collect valid data. Data validation at the time of data entry or collection helps you minimize the amount of data cleaning you’ll need to do.

After data collection, you can use data standardisation and data transformation to clean your data. You’ll also deal with any missing values, outliers, and duplicate values.

Clean data are valid, accurate, complete, consistent, unique, and uniform. Dirty data include inconsistencies and errors.

Dirty data can come from any part of the research process, including poor research design , inappropriate measurement materials, or flawed data entry.

Random assignment is used in experiments with a between-groups or independent measures design. In this research design, there’s usually a control group and one or more experimental groups. Random assignment helps ensure that the groups are comparable.

In general, you should always use random assignment in this type of experimental design when it is ethically possible and makes sense for your study topic.

Random selection, or random sampling , is a way of selecting members of a population for your study’s sample.

In contrast, random assignment is a way of sorting the sample into control and experimental groups.

Random sampling enhances the external validity or generalisability of your results, while random assignment improves the internal validity of your study.

To implement random assignment , assign a unique number to every member of your study’s sample .

Then, you can use a random number generator or a lottery method to randomly assign each number to a control or experimental group. You can also do so manually, by flipping a coin or rolling a die to randomly assign participants to groups.

Exploratory research is often used when the issue you’re studying is new or when the data collection process is challenging for some reason.

You can use exploratory research if you have a general idea or a specific question that you want to study but there is no preexisting knowledge or paradigm with which to study it.

Exploratory research is a methodology approach that explores research questions that have not previously been studied in depth. It is often used when the issue you’re studying is new, or the data collection process is challenging in some way.

Explanatory research is used to investigate how or why a phenomenon occurs. Therefore, this type of research is often one of the first stages in the research process , serving as a jumping-off point for future research.

Explanatory research is a research method used to investigate how or why something occurs when only a small amount of information is available pertaining to that topic. It can help you increase your understanding of a given topic.

Blinding means hiding who is assigned to the treatment group and who is assigned to the control group in an experiment .

Blinding is important to reduce bias (e.g., observer bias , demand characteristics ) and ensure a study’s internal validity .

If participants know whether they are in a control or treatment group , they may adjust their behaviour in ways that affect the outcome that researchers are trying to measure. If the people administering the treatment are aware of group assignment, they may treat participants differently and thus directly or indirectly influence the final results.

• In a single-blind study , only the participants are blinded.
• In a double-blind study , both participants and experimenters are blinded.
• In a triple-blind study , the assignment is hidden not only from participants and experimenters, but also from the researchers analysing the data.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field.

It acts as a first defence, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

In general, the peer review process follows the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review is a process of evaluating submissions to an academic journal. Utilising rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication.

For this reason, academic journals are often considered among the most credible sources you can use in a research project – provided that the journal itself is trustworthy and well regarded.

Anonymity means you don’t know who the participants are, while confidentiality means you know who they are but remove identifying information from your research report. Both are important ethical considerations .

You can only guarantee anonymity by not collecting any personally identifying information – for example, names, phone numbers, email addresses, IP addresses, physical characteristics, photos, or videos.

You can keep data confidential by using aggregate information in your research report, so that you only refer to groups of participants rather than individuals.

Research misconduct means making up or falsifying data, manipulating data analyses, or misrepresenting results in research reports. It’s a form of academic fraud.

These actions are committed intentionally and can have serious consequences; research misconduct is not a simple mistake or a point of disagreement but a serious ethical failure.

Research ethics matter for scientific integrity, human rights and dignity, and collaboration between science and society. These principles make sure that participation in studies is voluntary, informed, and safe.

Ethical considerations in research are a set of principles that guide your research designs and practices. These principles include voluntary participation, informed consent, anonymity, confidentiality, potential for harm, and results communication.

Scientists and researchers must always adhere to a certain code of conduct when collecting data from others .

These considerations protect the rights of research participants, enhance research validity , and maintain scientific integrity.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

The two main types of social desirability bias are:

• Self-deceptive enhancement (self-deception): The tendency to see oneself in a favorable light without realizing it.
• Impression managemen t (other-deception): The tendency to inflate one’s abilities or achievement in order to make a good impression on other people.

Demand characteristics are aspects of experiments that may give away the research objective to participants. Social desirability bias occurs when participants automatically try to respond in ways that make them seem likeable in a study, even if it means misrepresenting how they truly feel.

Participants may use demand characteristics to infer social norms or experimenter expectancies and act in socially desirable ways, so you should try to control for demand characteristics wherever possible.

Response bias refers to conditions or factors that take place during the process of responding to surveys, affecting the responses. One type of response bias is social desirability bias .

When your population is large in size, geographically dispersed, or difficult to contact, it’s necessary to use a sampling method .

This allows you to gather information from a smaller part of the population, i.e. the sample, and make accurate statements by using statistical analysis. A few sampling methods include simple random sampling , convenience sampling , and snowball sampling .

Stratified and cluster sampling may look similar, but bear in mind that groups created in cluster sampling are heterogeneous , so the individual characteristics in the cluster vary. In contrast, groups created in stratified sampling are homogeneous , as units share characteristics.

Relatedly, in cluster sampling you randomly select entire groups and include all units of each group in your sample. However, in stratified sampling, you select some units of all groups and include them in your sample. In this way, both methods can ensure that your sample is representative of the target population .

A sampling frame is a list of every member in the entire population . It is important that the sampling frame is as complete as possible, so that your sample accurately reflects your population.

Convenience sampling and quota sampling are both non-probability sampling methods. They both use non-random criteria like availability, geographical proximity, or expert knowledge to recruit study participants.

However, in convenience sampling, you continue to sample units or cases until you reach the required sample size.

In quota sampling, you first need to divide your population of interest into subgroups (strata) and estimate their proportions (quota) in the population. Then you can start your data collection , using convenience sampling to recruit participants, until the proportions in each subgroup coincide with the estimated proportions in the population.

Random sampling or probability sampling is based on random selection. This means that each unit has an equal chance (i.e., equal probability) of being included in the sample.

On the other hand, convenience sampling involves stopping people at random, which means that not everyone has an equal chance of being selected depending on the place, time, or day you are collecting your data.

Stratified sampling and quota sampling both involve dividing the population into subgroups and selecting units from each subgroup. The purpose in both cases is to select a representative sample and/or to allow comparisons between subgroups.

The main difference is that in stratified sampling, you draw a random sample from each subgroup ( probability sampling ). In quota sampling you select a predetermined number or proportion of units, in a non-random manner ( non-probability sampling ).

Snowball sampling is best used in the following cases:

• If there is no sampling frame available (e.g., people with a rare disease)
• If the population of interest is hard to access or locate (e.g., people experiencing homelessness)
• If the research focuses on a sensitive topic (e.g., extra-marital affairs)

Snowball sampling relies on the use of referrals. Here, the researcher recruits one or more initial participants, who then recruit the next ones. 

Participants share similar characteristics and/or know each other. Because of this, not every member of the population has an equal chance of being included in the sample, giving rise to sampling bias .

Snowball sampling is a non-probability sampling method , where there is not an equal chance for every member of the population to be included in the sample .

This means that you cannot use inferential statistics and make generalisations – often the goal of quantitative research . As such, a snowball sample is not representative of the target population, and is usually a better fit for qualitative research .

Snowball sampling is a non-probability sampling method . Unlike probability sampling (which involves some form of random selection ), the initial individuals selected to be studied are the ones who recruit new participants.

Because not every member of the target population has an equal chance of being recruited into the sample, selection in snowball sampling is non-random.

Reproducibility and replicability are related terms.

• Reproducing research entails reanalysing the existing data in the same manner.
• Replicating (or repeating ) the research entails reconducting the entire analysis, including the collection of new data . 
• A successful reproduction shows that the data analyses were conducted in a fair and honest manner.
• A successful replication shows that the reliability of the results is high.

The reproducibility and replicability of a study can be ensured by writing a transparent, detailed method section and using clear, unambiguous language.

Convergent validity and discriminant validity are both subtypes of construct validity . Together, they help you evaluate whether a test measures the concept it was designed to measure.

• Convergent validity indicates whether a test that is designed to measure a particular construct correlates with other tests that assess the same or similar construct.
• Discriminant validity indicates whether two tests that should not be highly related to each other are indeed not related

You need to assess both in order to demonstrate construct validity. Neither one alone is sufficient for establishing construct validity.

Construct validity has convergent and discriminant subtypes. They assist determine if a test measures the intended notion.

Content validity shows you how accurately a test or other measurement method taps  into the various aspects of the specific construct you are researching.

In other words, it helps you answer the question: “does the test measure all aspects of the construct I want to measure?” If it does, then the test has high content validity.

The higher the content validity, the more accurate the measurement of the construct.

If the test fails to include parts of the construct, or irrelevant parts are included, the validity of the instrument is threatened, which brings your results into question.

Construct validity refers to how well a test measures the concept (or construct) it was designed to measure. Assessing construct validity is especially important when you’re researching concepts that can’t be quantified and/or are intangible, like introversion. To ensure construct validity your test should be based on known indicators of introversion ( operationalisation ).

On the other hand, content validity assesses how well the test represents all aspects of the construct. If some aspects are missing or irrelevant parts are included, the test has low content validity.

Face validity and content validity are similar in that they both evaluate how suitable the content of a test is. The difference is that face validity is subjective, and assesses content at surface level.

When a test has strong face validity, anyone would agree that the test’s questions appear to measure what they are intended to measure.

For example, looking at a 4th grade math test consisting of problems in which students have to add and multiply, most people would agree that it has strong face validity (i.e., it looks like a math test).

On the other hand, content validity evaluates how well a test represents all the aspects of a topic. Assessing content validity is more systematic and relies on expert evaluation. of each question, analysing whether each one covers the aspects that the test was designed to cover.

A 4th grade math test would have high content validity if it covered all the skills taught in that grade. Experts(in this case, math teachers), would have to evaluate the content validity by comparing the test to the learning objectives.

• Discriminant validity indicates whether two tests that should not be highly related to each other are indeed not related. This type of validity is also called divergent validity .

Criterion validity and construct validity are both types of measurement validity . In other words, they both show you how accurately a method measures something.

While construct validity is the degree to which a test or other measurement method measures what it claims to measure, criterion validity is the degree to which a test can predictively (in the future) or concurrently (in the present) measure something.

Construct validity is often considered the overarching type of measurement validity . You need to have face validity , content validity , and criterion validity in order to achieve construct validity.

Attrition refers to participants leaving a study. It always happens to some extent – for example, in randomised control trials for medical research.

Differential attrition occurs when attrition or dropout rates differ systematically between the intervention and the control group . As a result, the characteristics of the participants who drop out differ from the characteristics of those who stay in the study. Because of this, study results may be biased .

Criterion validity evaluates how well a test measures the outcome it was designed to measure. An outcome can be, for example, the onset of a disease.

Criterion validity consists of two subtypes depending on the time at which the two measures (the criterion and your test) are obtained:

• Concurrent validity is a validation strategy where the the scores of a test and the criterion are obtained at the same time
• Predictive validity is a validation strategy where the criterion variables are measured after the scores of the test

Validity tells you how accurately a method measures what it was designed to measure. There are 4 main types of validity :

• Construct validity : Does the test measure the construct it was designed to measure?
• Face validity : Does the test appear to be suitable for its objectives ?
• Content validity : Does the test cover all relevant parts of the construct it aims to measure.
• Criterion validity : Do the results accurately measure the concrete outcome they are designed to measure?

Convergent validity shows how much a measure of one construct aligns with other measures of the same or related constructs .

On the other hand, concurrent validity is about how a measure matches up to some known criterion or gold standard, which can be another measure.

Although both types of validity are established by calculating the association or correlation between a test score and another variable , they represent distinct validation methods.

The purpose of theory-testing mode is to find evidence in order to disprove, refine, or support a theory. As such, generalisability is not the aim of theory-testing mode.

Due to this, the priority of researchers in theory-testing mode is to eliminate alternative causes for relationships between variables . In other words, they prioritise internal validity over external validity , including ecological validity .

Inclusion and exclusion criteria are typically presented and discussed in the methodology section of your thesis or dissertation .

Inclusion and exclusion criteria are predominantly used in non-probability sampling . In purposive sampling and snowball sampling , restrictions apply as to who can be included in the sample .

Scope of research is determined at the beginning of your research process , prior to the data collection stage. Sometimes called “scope of study,” your scope delineates what will and will not be covered in your project. It helps you focus your work and your time, ensuring that you’ll be able to achieve your goals and outcomes.

Defining a scope can be very useful in any research project, from a research proposal to a thesis or dissertation . A scope is needed for all types of research: quantitative , qualitative , and mixed methods .

To define your scope of research, consider the following:

• Budget constraints or any specifics of grant funding
• Your proposed timeline and duration
• Specifics about your population of study, your proposed sample size , and the research methodology you’ll pursue
• Any inclusion and exclusion criteria
• Any anticipated control , extraneous , or confounding variables that could bias your research if not accounted for properly.

To make quantitative observations , you need to use instruments that are capable of measuring the quantity you want to observe. For example, you might use a ruler to measure the length of an object or a thermometer to measure its temperature.

Quantitative observations involve measuring or counting something and expressing the result in numerical form, while qualitative observations involve describing something in non-numerical terms, such as its appearance, texture, or color.

The Scribbr Reference Generator is developed using the open-source Citation Style Language (CSL) project and Frank Bennett’s citeproc-js . It’s the same technology used by dozens of other popular citation tools, including Mendeley and Zotero.

You can find all the citation styles and locales used in the Scribbr Reference Generator in our publicly accessible repository on Github .

To paraphrase effectively, don’t just take the original sentence and swap out some of the words for synonyms. Instead, try:

• Reformulating the sentence (e.g., change active to passive , or start from a different point)
• Combining information from multiple sentences into one
• Leaving out information from the original that isn’t relevant to your point
• Using synonyms where they don’t distort the meaning

The main point is to ensure you don’t just copy the structure of the original text, but instead reformulate the idea in your own words.

Plagiarism means using someone else’s words or ideas and passing them off as your own. Paraphrasing means putting someone else’s ideas into your own words.

So when does paraphrasing count as plagiarism?

• Paraphrasing is plagiarism if you don’t properly credit the original author.
• Paraphrasing is plagiarism if your text is too close to the original wording (even if you cite the source). If you directly copy a sentence or phrase, you should quote it instead.
• Paraphrasing  is not plagiarism if you put the author’s ideas completely into your own words and properly reference the source .

To present information from other sources in academic writing , it’s best to paraphrase in most cases. This shows that you’ve understood the ideas you’re discussing and incorporates them into your text smoothly.

It’s appropriate to quote when:

• Changing the phrasing would distort the meaning of the original text
• You want to discuss the author’s language choices (e.g., in literary analysis )
• You’re presenting a precise definition
• You’re looking in depth at a specific claim

A quote is an exact copy of someone else’s words, usually enclosed in quotation marks and credited to the original author or speaker.

Every time you quote a source , you must include a correctly formatted in-text citation . This looks slightly different depending on the citation style .

For example, a direct quote in APA is cited like this: ‘This is a quote’ (Streefkerk, 2020, p. 5).

Every in-text citation should also correspond to a full reference at the end of your paper.

In scientific subjects, the information itself is more important than how it was expressed, so quoting should generally be kept to a minimum. In the arts and humanities, however, well-chosen quotes are often essential to a good paper.

In social sciences, it varies. If your research is mainly quantitative , you won’t include many quotes, but if it’s more qualitative , you may need to quote from the data you collected .

As a general guideline, quotes should take up no more than 5–10% of your paper. If in doubt, check with your instructor or supervisor how much quoting is appropriate in your field.

If you’re quoting from a text that paraphrases or summarises other sources and cites them in parentheses , APA  recommends retaining the citations as part of the quote:

• Smith states that ‘the literature on this topic (Jones, 2015; Sill, 2019; Paulson, 2020) shows no clear consensus’ (Smith, 2019, p. 4).

Footnote or endnote numbers that appear within quoted text should be omitted.

If you want to cite an indirect source (one you’ve only seen quoted in another source), either locate the original source or use the phrase ‘as cited in’ in your citation.

A block quote is a long quote formatted as a separate ‘block’ of text. Instead of using quotation marks , you place the quote on a new line, and indent the entire quote to mark it apart from your own words.

APA uses block quotes for quotes that are 40 words or longer.

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Common examples of primary sources include interview transcripts , photographs, novels, paintings, films, historical documents, and official statistics.

Anything you directly analyze or use as first-hand evidence can be a primary source, including qualitative or quantitative data that you collected yourself.

Common examples of secondary sources include academic books, journal articles , reviews, essays , and textbooks.

Anything that summarizes, evaluates or interprets primary sources can be a secondary source. If a source gives you an overview of background information or presents another researcher’s ideas on your topic, it is probably a secondary source.

To determine if a source is primary or secondary, ask yourself:

• Was the source created by someone directly involved in the events you’re studying (primary), or by another researcher (secondary)?
• Does the source provide original information (primary), or does it summarize information from other sources (secondary)?
• Are you directly analyzing the source itself (primary), or only using it for background information (secondary)?

Some types of sources are nearly always primary: works of art and literature, raw statistical data, official documents and records, and personal communications (e.g. letters, interviews ). If you use one of these in your research, it is probably a primary source.

Primary sources are often considered the most credible in terms of providing evidence for your argument, as they give you direct evidence of what you are researching. However, it’s up to you to ensure the information they provide is reliable and accurate.

Always make sure to properly cite your sources to avoid plagiarism .

A fictional movie is usually a primary source. A documentary can be either primary or secondary depending on the context.

If you are directly analysing some aspect of the movie itself – for example, the cinematography, narrative techniques, or social context – the movie is a primary source.

If you use the movie for background information or analysis about your topic – for example, to learn about a historical event or a scientific discovery – the movie is a secondary source.

Whether it’s primary or secondary, always properly cite the movie in the citation style you are using. Learn how to create an MLA movie citation or an APA movie citation .

Articles in newspapers and magazines can be primary or secondary depending on the focus of your research.

In historical studies, old articles are used as primary sources that give direct evidence about the time period. In social and communication studies, articles are used as primary sources to analyse language and social relations (for example, by conducting content analysis or discourse analysis ).

If you are not analysing the article itself, but only using it for background information or facts about your topic, then the article is a secondary source.

In academic writing , there are three main situations where quoting is the best choice:

• To analyse the author’s language (e.g., in a literary analysis essay )
• To give evidence from primary sources
• To accurately present a precise definition or argument

Don’t overuse quotes; your own voice should be dominant. If you just want to provide information from a source, it’s usually better to paraphrase or summarise .

Your list of tables and figures should go directly after your table of contents in your thesis or dissertation.

Lists of figures and tables are often not required, and they aren’t particularly common. They specifically aren’t required for APA Style, though you should be careful to follow their other guidelines for figures and tables .

If you have many figures and tables in your thesis or dissertation, include one may help you stay organised. Your educational institution may require them, so be sure to check their guidelines.

Copyright information can usually be found wherever the table or figure was published. For example, for a diagram in a journal article , look on the journal’s website or the database where you found the article. Images found on sites like Flickr are listed with clear copyright information.

If you find that permission is required to reproduce the material, be sure to contact the author or publisher and ask for it.

A list of figures and tables compiles all of the figures and tables that you used in your thesis or dissertation and displays them with the page number where they can be found.

APA doesn’t require you to include a list of tables or a list of figures . However, it is advisable to do so if your text is long enough to feature a table of contents and it includes a lot of tables and/or figures .

A list of tables and list of figures appear (in that order) after your table of contents, and are presented in a similar way.

A glossary is a collection of words pertaining to a specific topic. In your thesis or dissertation, it’s a list of all terms you used that may not immediately be obvious to your reader. Your glossary only needs to include terms that your reader may not be familiar with, and is intended to enhance their understanding of your work.

Definitional terms often fall into the category of common knowledge , meaning that they don’t necessarily have to be cited. This guidance can apply to your thesis or dissertation glossary as well.

However, if you’d prefer to cite your sources , you can follow guidance for citing dictionary entries in MLA or APA style for your glossary.

A glossary is a collection of words pertaining to a specific topic. In your thesis or dissertation, it’s a list of all terms you used that may not immediately be obvious to your reader. In contrast, an index is a list of the contents of your work organised by page number.

Glossaries are not mandatory, but if you use a lot of technical or field-specific terms, it may improve readability to add one to your thesis or dissertation. Your educational institution may also require them, so be sure to check their specific guidelines.

A glossary is a collection of words pertaining to a specific topic. In your thesis or dissertation, it’s a list of all terms you used that may not immediately be obvious to your reader. In contrast, dictionaries are more general collections of words.

The title page of your thesis or dissertation should include your name, department, institution, degree program, and submission date.

The title page of your thesis or dissertation goes first, before all other content or lists that you may choose to include.

Usually, no title page is needed in an MLA paper . A header is generally included at the top of the first page instead. The exceptions are when:

• Your instructor requires one, or
• Your paper is a group project

In those cases, you should use a title page instead of a header, listing the same information but on a separate page.

When you mention different chapters within your text, it’s considered best to use Roman numerals for most citation styles. However, the most important thing here is to remain consistent whenever using numbers in your dissertation .

A thesis or dissertation outline is one of the most critical first steps in your writing process. It helps you to lay out and organise your ideas and can provide you with a roadmap for deciding what kind of research you’d like to undertake.

Generally, an outline contains information on the different sections included in your thesis or dissertation, such as:

• Your anticipated title
• Your abstract
• Your chapters (sometimes subdivided into further topics like literature review, research methods, avenues for future research, etc.)

While a theoretical framework describes the theoretical underpinnings of your work based on existing research, a conceptual framework allows you to draw your own conclusions, mapping out the variables you may use in your study and the interplay between them.

A literature review and a theoretical framework are not the same thing and cannot be used interchangeably. While a theoretical framework describes the theoretical underpinnings of your work, a literature review critically evaluates existing research relating to your topic. You’ll likely need both in your dissertation .

A theoretical framework can sometimes be integrated into a  literature review chapter , but it can also be included as its own chapter or section in your dissertation . As a rule of thumb, if your research involves dealing with a lot of complex theories, it’s a good idea to include a separate theoretical framework chapter.

An abstract is a concise summary of an academic text (such as a journal article or dissertation ). It serves two main purposes:

• To help potential readers determine the relevance of your paper for their own research.
• To communicate your key findings to those who don’t have time to read the whole paper.

Abstracts are often indexed along with keywords on academic databases, so they make your work more easily findable. Since the abstract is the first thing any reader sees, it’s important that it clearly and accurately summarises the contents of your paper.

The abstract is the very last thing you write. You should only write it after your research is complete, so that you can accurately summarize the entirety of your thesis or paper.

Avoid citing sources in your abstract . There are two reasons for this:

• The abstract should focus on your original research, not on the work of others.
• The abstract should be self-contained and fully understandable without reference to other sources.

There are some circumstances where you might need to mention other sources in an abstract: for example, if your research responds directly to another study or focuses on the work of a single theorist. In general, though, don’t include citations unless absolutely necessary.

The abstract appears on its own page, after the title page and acknowledgements but before the table of contents .

Results are usually written in the past tense , because they are describing the outcome of completed actions.

The results chapter or section simply and objectively reports what you found, without speculating on why you found these results. The discussion interprets the meaning of the results, puts them in context, and explains why they matter.

In qualitative research , results and discussion are sometimes combined. But in quantitative research , it’s considered important to separate the objective results from your interpretation of them.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

The best way to remember the difference between a research plan and a research proposal is that they have fundamentally different audiences. A research plan helps you, the researcher, organize your thoughts. On the other hand, a dissertation proposal or research proposal aims to convince others (e.g., a supervisor, a funding body, or a dissertation committee) that your research topic is relevant and worthy of being conducted.

A noun is a word that represents a person, thing, concept, or place (e.g., ‘John’, ‘house’, ‘affinity’, ‘river’). Most sentences contain at least one noun or pronoun .

Nouns are often, but not always, preceded by an article (‘the’, ‘a’, or ‘an’) and/or another determiner such as an adjective.

There are many ways to categorize nouns into various types, and the same noun can fall into multiple categories or even change types depending on context.

Some of the main types of nouns are:

• Common nouns and proper nouns
• Countable and uncountable nouns
• Concrete and abstract nouns
• Collective nouns
• Possessive nouns
• Attributive nouns
• Appositive nouns
• Generic nouns

Pronouns are words like ‘I’, ‘she’, and ‘they’ that are used in a similar way to nouns . They stand in for a noun that has already been mentioned or refer to yourself and other people.

Pronouns can function just like nouns as the head of a noun phrase and as the subject or object of a verb. However, pronouns change their forms (e.g., from ‘I’ to ‘me’) depending on the grammatical context they’re used in, whereas nouns usually don’t.

Common nouns are words for types of things, people, and places, such as ‘dog’, ‘professor’, and ‘city’. They are not capitalised and are typically used in combination with articles and other determiners.

Proper nouns are words for specific things, people, and places, such as ‘Max’, ‘Dr Prakash’, and ‘London’. They are always capitalised and usually aren’t combined with articles and other determiners.

A proper adjective is an adjective that was derived from a proper noun and is therefore capitalised .

Proper adjectives include words for nationalities, languages, and ethnicities (e.g., ‘Japanese’, ‘Inuit’, ‘French’) and words derived from people’s names (e.g., ‘Bayesian’, ‘Orwellian’).

The names of seasons (e.g., ‘spring’) are treated as common nouns in English and therefore not capitalised . People often assume they are proper nouns, but this is an error.

The names of days and months, however, are capitalised since they’re treated as proper nouns in English (e.g., ‘Wednesday’, ‘January’).

No, as a general rule, academic concepts, disciplines, theories, models, etc. are treated as common nouns , not proper nouns , and therefore not capitalised . For example, ‘five-factor model of personality’ or ‘analytic philosophy’.

However, proper nouns that appear within the name of an academic concept (such as the name of the inventor) are capitalised as usual. For example, ‘Darwin’s theory of evolution’ or ‘ Student’s t table ‘.

Collective nouns are most commonly treated as singular (e.g., ‘the herd is grazing’), but usage differs between US and UK English :

• In US English, it’s standard to treat all collective nouns as singular, even when they are plural in appearance (e.g., ‘The Rolling Stones is …’). Using the plural form is usually seen as incorrect.
• In UK English, collective nouns can be treated as singular or plural depending on context. It’s quite common to use the plural form, especially when the noun looks plural (e.g., ‘The Rolling Stones are …’).

The plural of “crisis” is “crises”. It’s a loanword from Latin and retains its original Latin plural noun form (similar to “analyses” and “bases”). It’s wrong to write “crisises”.

For example, you might write “Several crises destabilized the regime.”

Normally, the plural of “fish” is the same as the singular: “fish”. It’s one of a group of irregular plural nouns in English that are identical to the corresponding singular nouns (e.g., “moose”, “sheep”). For example, you might write “The fish scatter as the shark approaches.”

If you’re referring to several species of fish, though, the regular plural “fishes” is often used instead. For example, “The aquarium contains many different fishes , including trout and carp.”

The correct plural of “octopus” is “octopuses”.

People often write “octopi” instead because they assume that the plural noun is formed in the same way as Latin loanwords such as “fungus/fungi”. But “octopus” actually comes from Greek, where its original plural is “octopodes”. In English, it instead has the regular plural form “octopuses”.

For example, you might write “There are four octopuses in the aquarium.”

The plural of “moose” is the same as the singular: “moose”. It’s one of a group of plural nouns in English that are identical to the corresponding singular nouns. So it’s wrong to write “mooses”.

For example, you might write “There are several moose in the forest.”

Bias in research affects the validity and reliability of your findings, leading to false conclusions and a misinterpretation of the truth. This can have serious implications in areas like medical research where, for example, a new form of treatment may be evaluated.

Observer bias occurs when the researcher’s assumptions, views, or preconceptions influence what they see and record in a study, while actor–observer bias refers to situations where respondents attribute internal factors (e.g., bad character) to justify other’s behaviour and external factors (difficult circumstances) to justify the same behaviour in themselves.

Response bias is a general term used to describe a number of different conditions or factors that cue respondents to provide inaccurate or false answers during surveys or interviews . These factors range from the interviewer’s perceived social position or appearance to the the phrasing of questions in surveys.

Nonresponse bias occurs when the people who complete a survey are different from those who did not, in ways that are relevant to the research topic. Nonresponse can happen either because people are not willing or not able to participate.

In research, demand characteristics are cues that might indicate the aim of a study to participants. These cues can lead to participants changing their behaviors or responses based on what they think the research is about.

Demand characteristics are common problems in psychology experiments and other social science studies because they can bias your research findings.

Demand characteristics are a type of extraneous variable that can affect the outcomes of the study. They can invalidate studies by providing an alternative explanation for the results.

These cues may nudge participants to consciously or unconsciously change their responses, and they pose a threat to both internal and external validity . You can’t be sure that your independent variable manipulation worked, or that your findings can be applied to other people or settings.

You can control demand characteristics by taking a few precautions in your research design and materials.

Use these measures:

• Deception: Hide the purpose of the study from participants
• Between-groups design : Give each participant only one independent variable treatment
• Double-blind design : Conceal the assignment of groups from participants and yourself
• Implicit measures: Use indirect or hidden measurements for your variables

Some attrition is normal and to be expected in research. However, the type of attrition is important because systematic research bias can distort your findings. Attrition bias can lead to inaccurate results because it affects internal and/or external validity .

To avoid attrition bias , applying some of these measures can help you reduce participant dropout (attrition) by making it easy and appealing for participants to stay.

• Provide compensation (e.g., cash or gift cards) for attending every session
• Minimise the number of follow-ups as much as possible
• Make all follow-ups brief, flexible, and convenient for participants
• Send participants routine reminders to schedule follow-ups
• Recruit more participants than you need for your sample (oversample)
• Maintain detailed contact information so you can get in touch with participants even if they move

If you have a small amount of attrition bias , you can use a few statistical methods to try to make up for this research bias .

Multiple imputation involves using simulations to replace the missing data with likely values. Alternatively, you can use sample weighting to make up for the uneven balance of participants in your sample.

Placebos are used in medical research for new medication or therapies, called clinical trials. In these trials some people are given a placebo, while others are given the new medication being tested.

The purpose is to determine how effective the new medication is: if it benefits people beyond a predefined threshold as compared to the placebo, it’s considered effective.

Although there is no definite answer to what causes the placebo effect , researchers propose a number of explanations such as the power of suggestion, doctor-patient interaction, classical conditioning, etc.

Belief bias and confirmation bias are both types of cognitive bias that impact our judgment and decision-making.

Confirmation bias relates to how we perceive and judge evidence. We tend to seek out and prefer information that supports our preexisting beliefs, ignoring any information that contradicts those beliefs.

Belief bias describes the tendency to judge an argument based on how plausible the conclusion seems to us, rather than how much evidence is provided to support it during the course of the argument.

Positivity bias is phenomenon that occurs when a person judges individual members of a group positively, even when they have negative impressions or judgments of the group as a whole. Positivity bias is closely related to optimism bias , or the e xpectation that things will work out well, even if rationality suggests that problems are inevitable in life.

Perception bias is a problem because it prevents us from seeing situations or people objectively. Rather, our expectations, beliefs, or emotions interfere with how we interpret reality. This, in turn, can cause us to misjudge ourselves or others. For example, our prejudices can interfere with whether we perceive people’s faces as friendly or unfriendly.

There are many ways to categorize adjectives into various types. An adjective can fall into one or more of these categories depending on how it is used.

Some of the main types of adjectives are:

• Attributive adjectives
• Predicative adjectives
• Comparative adjectives
• Superlative adjectives
• Coordinate adjectives
• Appositive adjectives
• Compound adjectives
• Participial adjectives
• Proper adjectives
• Denominal adjectives
• Nominal adjectives

Cardinal numbers (e.g., one, two, three) can be placed before a noun to indicate quantity (e.g., one apple). While these are sometimes referred to as ‘numeral adjectives ‘, they are more accurately categorised as determiners or quantifiers.

Proper adjectives are adjectives formed from a proper noun (i.e., the name of a specific person, place, or thing) that are used to indicate origin. Like proper nouns, proper adjectives are always capitalised (e.g., Newtonian, Marxian, African).

The cost of proofreading depends on the type and length of text, the turnaround time, and the level of services required. Most proofreading companies charge per word or page, while freelancers sometimes charge an hourly rate.

For proofreading alone, which involves only basic corrections of typos and formatting mistakes, you might pay as little as £0.01 per word, but in many cases, your text will also require some level of editing , which costs slightly more.

It’s often possible to purchase combined proofreading and editing services and calculate the price in advance based on your requirements.

Then and than are two commonly confused words . In the context of ‘better than’, you use ‘than’ with an ‘a’.

• Julie is better than Jesse.
• I’d rather spend my time with you than with him.
• I understand Eoghan’s point of view better than Claudia’s.

Use to and used to are commonly confused words . In the case of ‘used to do’, the latter (with ‘d’) is correct, since you’re describing an action or state in the past.

• I used to do laundry once a week.
• They used to do each other’s hair.
• We used to do the dishes every day .

There are numerous synonyms and near synonyms for the various meanings of “ favour ”:

There are numerous synonyms and near synonyms for the two meanings of “ favoured ”:

No one (two words) is an indefinite pronoun meaning ‘nobody’. People sometimes mistakenly write ‘noone’, but this is incorrect and should be avoided. ‘No-one’, with a hyphen, is also acceptable in UK English .

Nobody and no one are both indefinite pronouns meaning ‘no person’. They can be used interchangeably (e.g., ‘nobody is home’ means the same as ‘no one is home’).

Some synonyms and near synonyms of  every time include:

• Without exception

‘Everytime’ is sometimes used to mean ‘each time’ or ‘whenever’. However, this is incorrect and should be avoided. The correct phrase is every time   (two words).

Yes, the conjunction because is a compound word , but one with a long history. It originates in Middle English from the preposition “bi” (“by”) and the noun “cause”. Over time, the open compound “bi cause” became the closed compound “because”, which we use today.

Though it’s spelled this way now, the verb “be” is not one of the words that makes up “because”.

Yes, today is a compound word , but a very old one. It wasn’t originally formed from the preposition “to” and the noun “day”; rather, it originates from their Old English equivalents, “tō” and “dæġe”.

In the past, it was sometimes written as a hyphenated compound: “to-day”. But the hyphen is no longer included; it’s always “today” now (“to day” is also wrong).

IEEE citation format is defined by the Institute of Electrical and Electronics Engineers and used in their publications.

It’s also a widely used citation style for students in technical fields like electrical and electronic engineering, computer science, telecommunications, and computer engineering.

An IEEE in-text citation consists of a number in brackets at the relevant point in the text, which points the reader to the right entry in the numbered reference list at the end of the paper. For example, ‘Smith [1] states that …’

A location marker such as a page number is also included within the brackets when needed: ‘Smith [1, p. 13] argues …’

The IEEE reference page consists of a list of references numbered in the order they were cited in the text. The title ‘References’ appears in bold at the top, either left-aligned or centered.

The numbers appear in square brackets on the left-hand side of the page. The reference entries are indented consistently to separate them from the numbers. Entries are single-spaced, with a normal paragraph break between them.

If you cite the same source more than once in your writing, use the same number for all of the IEEE in-text citations for that source, and only include it on the IEEE reference page once. The source is numbered based on the first time you cite it.

For example, the fourth source you cite in your paper is numbered [4]. If you cite it again later, you still cite it as [4]. You can cite different parts of the source each time by adding page numbers [4, p. 15].

A verb is a word that indicates a physical action (e.g., ‘drive’), a mental action (e.g., ‘think’) or a state of being (e.g., ‘exist’). Every sentence contains a verb.

Verbs are almost always used along with a noun or pronoun to describe what the noun or pronoun is doing.

There are many ways to categorize verbs into various types. A verb can fall into one or more of these categories depending on how it is used.

Some of the main types of verbs are:

• Regular verbs
• Irregular verbs
• Transitive verbs
• Intransitive verbs
• Dynamic verbs
• Stative verbs
• Linking verbs
• Auxiliary verbs
• Modal verbs
• Phrasal verbs

Regular verbs are verbs whose simple past and past participle are formed by adding the suffix ‘-ed’ (e.g., ‘walked’).

Irregular verbs are verbs that form their simple past and past participles in some way other than by adding the suffix ‘-ed’ (e.g., ‘sat’).

The indefinite articles a and an are used to refer to a general or unspecified version of a noun (e.g., a house). Which indefinite article you use depends on the pronunciation of the word that follows it.

• A is used for words that begin with a consonant sound (e.g., a bear).
• An is used for words that begin with a vowel sound (e.g., an eagle).

Indefinite articles can only be used with singular countable nouns . Like definite articles, they are a type of determiner .

Editing and proofreading are different steps in the process of revising a text.

Editing comes first, and can involve major changes to content, structure and language. The first stages of editing are often done by authors themselves, while a professional editor makes the final improvements to grammar and style (for example, by improving sentence structure and word choice ).

Proofreading is the final stage of checking a text before it is published or shared. It focuses on correcting minor errors and inconsistencies (for example, in punctuation and capitalization ). Proofreaders often also check for formatting issues, especially in print publishing.

Whether you’re publishing a blog, submitting a research paper , or even just writing an important email, there are a few techniques you can use to make sure it’s error-free:

• Take a break : Set your work aside for at least a few hours so that you can look at it with fresh eyes.
• Proofread a printout : Staring at a screen for too long can cause fatigue – sit down with a pen and paper to check the final version.
• Use digital shortcuts : Take note of any recurring mistakes (for example, misspelling a particular word, switching between US and UK English , or inconsistently capitalizing a term), and use Find and Replace to fix it throughout the document.

If you want to be confident that an important text is error-free, it might be worth choosing a professional proofreading service instead.

There are many different routes to becoming a professional proofreader or editor. The necessary qualifications depend on the field – to be an academic or scientific proofreader, for example, you will need at least a university degree in a relevant subject.

For most proofreading jobs, experience and demonstrated skills are more important than specific qualifications. Often your skills will be tested as part of the application process.

To learn practical proofreading skills, you can choose to take a course with a professional organisation such as the Society for Editors and Proofreaders . Alternatively, you can apply to companies that offer specialised on-the-job training programmes, such as the Scribbr Academy .

Though they’re pronounced the same, there’s a big difference in meaning between its and it’s .

• ‘The cat ate its food’.
• ‘It’s almost Christmas’.

Its and it’s are often confused, but its (without apostrophe) is the possessive form of ‘it’ (e.g., its tail, its argument, its wing). You use ‘its’ instead of ‘his’ and ‘her’ for neuter, inanimate nouns.

Then and than are two commonly confused words with different meanings and grammatical roles.

• Then (pronounced with a short ‘e’ sound) refers to time. It’s often an adverb , but it can also be used as a noun meaning ‘that time’ and as an adjective referring to a previous status.
• Than (pronounced with a short ‘a’ sound) is used for comparisons. Grammatically, it usually functions as a conjunction , but sometimes it’s a preposition .

Use to and used to are commonly confused words . In the case of ‘used to be’, the latter (with ‘d’) is correct, since you’re describing an action or state in the past.

• I used to be the new coworker.
• There used to be 4 cookies left.
• We used to walk to school every day .

A grammar checker is a tool designed to automatically check your text for spelling errors, grammatical issues, punctuation mistakes , and problems with sentence structure . You can check out our analysis of the best free grammar checkers to learn more.

A paraphrasing tool edits your text more actively, changing things whether they were grammatically incorrect or not. It can paraphrase your sentences to make them more concise and readable or for other purposes. You can check out our analysis of the best free paraphrasing tools to learn more.

Some tools available online combine both functions. Others, such as QuillBot , have separate grammar checker and paraphrasing tools. Be aware of what exactly the tool you’re using does to avoid introducing unwanted changes.

Good grammar is the key to expressing yourself clearly and fluently, especially in professional communication and academic writing . Word processors, browsers, and email programs typically have built-in grammar checkers, but they’re quite limited in the kinds of problems they can fix.

If you want to go beyond detecting basic spelling errors, there are many online grammar checkers with more advanced functionality. They can often detect issues with punctuation , word choice, and sentence structure that more basic tools would miss.

Not all of these tools are reliable, though. You can check out our research into the best free grammar checkers to explore the options.

Our research indicates that the best free grammar checker available online is the QuillBot grammar checker .

We tested 10 of the most popular checkers with the same sample text (containing 20 grammatical errors) and found that QuillBot easily outperformed the competition, scoring 18 out of 20, a drastic improvement over the second-place score of 13 out of 20.

It even appeared to outperform the premium versions of other grammar checkers, despite being entirely free.

A teacher’s aide is a person who assists in teaching classes but is not a qualified teacher. Aide is a noun meaning ‘assistant’, so it will always refer to a person.

‘Teacher’s aid’ is incorrect.

A visual aid is an instructional device (e.g., a photo, a chart) that appeals to vision to help you understand written or spoken information. Aid is often placed after an attributive noun or adjective (like ‘visual’) that describes the type of help provided.

‘Visual aide’ is incorrect.

A job aid is an instructional tool (e.g., a checklist, a cheat sheet) that helps you work efficiently. Aid is a noun meaning ‘assistance’. It’s often placed after an adjective or attributive noun (like ‘job’) that describes the specific type of help provided.

‘Job aide’ is incorrect.

There are numerous synonyms for the various meanings of truly :

Yours truly is a phrase used at the end of a formal letter or email. It can also be used (typically in a humorous way) as a pronoun to refer to oneself (e.g., ‘The dinner was cooked by yours truly ‘). The latter usage should be avoided in formal writing.

It’s formed by combining the second-person possessive pronoun ‘yours’ with the adverb ‘ truly ‘.

A pathetic fallacy can be a short phrase or a whole sentence and is often used in novels and poetry. Pathetic fallacies serve multiple purposes, such as:

• Conveying the emotional state of the characters or the narrator
• Creating an atmosphere or set the mood of a scene
• Foreshadowing events to come
• Giving texture and vividness to a piece of writing
• Communicating emotion to the reader in a subtle way, by describing the external world.
• Bringing inanimate objects to life so that they seem more relatable.

AMA citation format is a citation style designed by the American Medical Association. It’s frequently used in the field of medicine.

You may be told to use AMA style for your student papers. You will also have to follow this style if you’re submitting a paper to a journal published by the AMA.

An AMA in-text citation consists of the number of the relevant reference on your AMA reference page , written in superscript 1 at the point in the text where the source is used.

It may also include the page number or range of the relevant material in the source (e.g., the part you quoted 2(p46) ). Multiple sources can be cited at one point, presented as a range or list (with no spaces 3,5–9 ).

An AMA reference usually includes the author’s last name and initials, the title of the source, information about the publisher or the publication it’s contained in, and the publication date. The specific details included, and the formatting, depend on the source type.

References in AMA style are presented in numerical order (numbered by the order in which they were first cited in the text) on your reference page. A source that’s cited repeatedly in the text still only appears once on the reference page.

An AMA in-text citation just consists of the number of the relevant entry on your AMA reference page , written in superscript at the point in the text where the source is referred to.

You don’t need to mention the author of the source in your sentence, but you can do so if you want. It’s not an official part of the citation, but it can be useful as part of a signal phrase introducing the source.

On your AMA reference page , author names are written with the last name first, followed by the initial(s) of their first name and middle name if mentioned.

There’s a space between the last name and the initials, but no space or punctuation between the initials themselves. The names of multiple authors are separated by commas , and the whole list ends in a period, e.g., ‘Andreessen F, Smith PW, Gonzalez E’.

The names of up to six authors should be listed for each source on your AMA reference page , separated by commas . For a source with seven or more authors, you should list the first three followed by ‘ et al’ : ‘Isidore, Gilbert, Gunvor, et al’.

In the text, mentioning author names is optional (as they aren’t an official part of AMA in-text citations ). If you do mention them, though, you should use the first author’s name followed by ‘et al’ when there are three or more : ‘Isidore et al argue that …’

Note that according to AMA’s rather minimalistic punctuation guidelines, there’s no period after ‘et al’ unless it appears at the end of a sentence. This is different from most other styles, where there is normally a period.

Yes, you should normally include an access date in an AMA website citation (or when citing any source with a URL). This is because webpages can change their content over time, so it’s useful for the reader to know when you accessed the page.

When a publication or update date is provided on the page, you should include it in addition to the access date. The access date appears second in this case, e.g., ‘Published June 19, 2021. Accessed August 29, 2022.’

Don’t include an access date when citing a source with a DOI (such as in an AMA journal article citation ).

Some variables have fixed levels. For example, gender and ethnicity are always nominal level data because they cannot be ranked.

However, for other variables, you can choose the level of measurement . For example, income is a variable that can be recorded on an ordinal or a ratio scale:

• At an ordinal level , you could create 5 income groupings and code the incomes that fall within them from 1–5.
• At a ratio level , you would record exact numbers for income.

If you have a choice, the ratio level is always preferable because you can analyse data in more ways. The higher the level of measurement, the more precise your data is.

The level at which you measure a variable determines how you can analyse your data.

Depending on the level of measurement , you can perform different descriptive statistics to get an overall summary of your data and inferential statistics to see if your results support or refute your hypothesis .

Levels of measurement tell you how precisely variables are recorded. There are 4 levels of measurement, which can be ranked from low to high:

• Nominal : the data can only be categorised.
• Ordinal : the data can be categorised and ranked.
• Interval : the data can be categorised and ranked, and evenly spaced.
• Ratio : the data can be categorised, ranked, evenly spaced and has a natural zero.

Statistical analysis is the main method for analyzing quantitative research data . It uses probabilities and models to test predictions about a population from sample data.

The null hypothesis is often abbreviated as H 0 . When the null hypothesis is written using mathematical symbols, it always includes an equality symbol (usually =, but sometimes ≥ or ≤).

The alternative hypothesis is often abbreviated as H a or H 1 . When the alternative hypothesis is written using mathematical symbols, it always includes an inequality symbol (usually ≠, but sometimes < or >).

As the degrees of freedom increase, Student’s t distribution becomes less leptokurtic , meaning that the probability of extreme values decreases. The distribution becomes more and more similar to a standard normal distribution .

When there are only one or two degrees of freedom , the chi-square distribution is shaped like a backwards ‘J’. When there are three or more degrees of freedom, the distribution is shaped like a right-skewed hump. As the degrees of freedom increase, the hump becomes less right-skewed and the peak of the hump moves to the right. The distribution becomes more and more similar to a normal distribution .

‘Looking forward in hearing from you’ is an incorrect version of the phrase looking forward to hearing from you . The phrasal verb ‘looking forward to’ always needs the preposition ‘to’, not ‘in’.

• I am looking forward in hearing from you.
• I am looking forward to hearing from you.

Some synonyms and near synonyms for the expression looking forward to hearing from you include:

• Eagerly awaiting your response
• Hoping to hear from you soon
• It would be great to hear back from you
• Thanks in advance for your reply

People sometimes mistakenly write ‘looking forward to hear from you’, but this is incorrect. The correct phrase is looking forward to hearing from you .

The phrasal verb ‘look forward to’ is always followed by a direct object, the thing you’re looking forward to. As the direct object has to be a noun phrase , it should be the gerund ‘hearing’, not the verb ‘hear’.

• I’m looking forward to hear from you soon.
• I’m looking forward to hearing from you soon.

Traditionally, the sign-off Yours sincerely is used in an email message or letter when you are writing to someone you have interacted with before, not a complete stranger.

Yours faithfully is used instead when you are writing to someone you have had no previous correspondence with, especially if you greeted them as ‘ Dear Sir or Madam ’.

Just checking in   is a standard phrase used to start an email (or other message) that’s intended to ask someone for a response or follow-up action in a friendly, informal way. However, it’s a cliché opening that can come across as passive-aggressive, so we recommend avoiding it in favor of a more direct opening like “We previously discussed …”

In a more personal context, you might encounter “just checking in” as part of a longer phrase such as “I’m just checking in to see how you’re doing”. In this case, it’s not asking the other person to do anything but rather asking about their well-being (emotional or physical) in a friendly way.

“Earliest convenience” is part of the phrase at your earliest convenience , meaning “as soon as you can”. 

It’s typically used to end an email in a formal context by asking the recipient to do something when it’s convenient for them to do so.

ASAP is an abbreviation of the phrase “as soon as possible”. 

It’s typically used to indicate a sense of urgency in highly informal contexts (e.g., “Let me know ASAP if you need me to drive you to the airport”).

“ASAP” should be avoided in more formal correspondence. Instead, use an alternative like at your earliest convenience .

Some synonyms and near synonyms of the verb   compose   (meaning “to make up”) are:

People increasingly use “comprise” as a synonym of “compose.” However, this is normally still seen as a mistake, and we recommend avoiding it in your academic writing . “Comprise” traditionally means “to be made up of,” not “to make up.”

Some synonyms and near synonyms of the verb comprise are:

• Be composed of
• Be made up of

People increasingly use “comprise” interchangeably with “compose,” meaning that they consider words like “compose,” “constitute,” and “form” to be synonymous with “comprise.” However, this is still normally regarded as an error, and we advise against using these words interchangeably in academic writing .

A fallacy is a mistaken belief, particularly one based on unsound arguments or one that lacks the evidence to support it. Common types of fallacy that may compromise the quality of your research are:

• Correlation/causation fallacy: Claiming that two events that occur together have a cause-and-effect relationship even though this can’t be proven
• Ecological fallacy : Making inferences about the nature of individuals based on aggregate data for the group
• The sunk cost fallacy : Following through on a project or decision because we have already invested time, effort, or money into it, even if the current costs outweigh the benefits
• The base-rate fallacy : Ignoring base-rate or statistically significant information, such as sample size or the relative frequency of an event, in favor of  less relevant information e.g., pertaining to a single case, or a small number of cases
• The planning fallacy : Underestimating the time needed to complete a future task, even when we know that similar tasks in the past have taken longer than planned

The planning fallacy refers to people’s tendency to underestimate the resources needed to complete a future task, despite knowing that previous tasks have also taken longer than planned.

For example, people generally tend to underestimate the cost and time needed for construction projects. The planning fallacy occurs due to people’s tendency to overestimate the chances that positive events, such as a shortened timeline, will happen to them. This phenomenon is called optimism bias or positivity bias.

Although both red herring fallacy and straw man fallacy are logical fallacies or reasoning errors, they denote different attempts to “win” an argument. More specifically:

• A red herring fallacy refers to an attempt to change the subject and divert attention from the original issue. In other words, a seemingly solid but ultimately irrelevant argument is introduced into the discussion, either on purpose or by mistake.
• A straw man argument involves the deliberate distortion of another person’s argument. By oversimplifying or exaggerating it, the other party creates an easy-to-refute argument and then attacks it.

The red herring fallacy is a problem because it is flawed reasoning. It is a distraction device that causes people to become sidetracked from the main issue and draw wrong conclusions.

Although a red herring may have some kernel of truth, it is used as a distraction to keep our eyes on a different matter. As a result, it can cause us to accept and spread misleading information.

The sunk cost fallacy and escalation of commitment (or commitment bias ) are two closely related terms. However, there is a slight difference between them:

• Escalation of commitment (aka commitment bias ) is the tendency to be consistent with what we have already done or said we will do in the past, especially if we did so in public. In other words, it is an attempt to save face and appear consistent.
• Sunk cost fallacy is the tendency to stick with a decision or a plan even when it’s failing. Because we have already invested valuable time, money, or energy, quitting feels like these resources were wasted.

In other words, escalating commitment is a manifestation of the sunk cost fallacy: an irrational escalation of commitment frequently occurs when people refuse to accept that the resources they’ve already invested cannot be recovered. Instead, they insist on more spending to justify the initial investment (and the incurred losses).

When you are faced with a straw man argument , the best way to respond is to draw attention to the fallacy and ask your discussion partner to show how your original statement and their distorted version are the same. Since these are different, your partner will either have to admit that their argument is invalid or try to justify it by using more flawed reasoning, which you can then attack.

The straw man argument is a problem because it occurs when we fail to take an opposing point of view seriously. Instead, we intentionally misrepresent our opponent’s ideas and avoid genuinely engaging with them. Due to this, resorting to straw man fallacy lowers the standard of constructive debate.

A straw man argument is a distorted (and weaker) version of another person’s argument that can easily be refuted (e.g., when a teacher proposes that the class spend more time on math exercises, a parent complains that the teacher doesn’t care about reading and writing).

This is a straw man argument because it misrepresents the teacher’s position, which didn’t mention anything about cutting down on reading and writing. The straw man argument is also known as the straw man fallacy .

A slippery slope argument is not always a fallacy.

• When someone claims adopting a certain policy or taking a certain action will automatically lead to a series of other policies or actions also being taken, this is a slippery slope argument.
• If they don’t show a causal connection between the advocated policy and the consequent policies, then they commit a slippery slope fallacy .

There are a number of ways you can deal with slippery slope arguments especially when you suspect these are fallacious:

• Slippery slope arguments take advantage of the gray area between an initial action or decision and the possible next steps that might lead to the undesirable outcome. You can point out these missing steps and ask your partner to indicate what evidence exists to support the claimed relationship between two or more events.
• Ask yourself if each link in the chain of events or action is valid. Every proposition has to be true for the overall argument to work, so even if one link is irrational or not supported by evidence, then the argument collapses.
• Sometimes people commit a slippery slope fallacy unintentionally. In these instances, use an example that demonstrates the problem with slippery slope arguments in general (e.g., by using statements to reach a conclusion that is not necessarily relevant to the initial statement). By attacking the concept of slippery slope arguments you can show that they are often fallacious.

People sometimes confuse cognitive bias and logical fallacies because they both relate to flawed thinking. However, they are not the same:

• Cognitive bias is the tendency to make decisions or take action in an illogical way because of our values, memory, socialization, and other personal attributes. In other words, it refers to a fixed pattern of thinking rooted in the way our brain works.
• Logical fallacies relate to how we make claims and construct our arguments in the moment. They are statements that sound convincing at first but can be disproven through logical reasoning.

In other words, cognitive bias refers to an ongoing predisposition, while logical fallacy refers to mistakes of reasoning that occur in the moment.

An appeal to ignorance (ignorance here meaning lack of evidence) is a type of informal logical fallacy .

It asserts that something must be true because it hasn’t been proven false—or that something must be false because it has not yet been proven true.

For example, “unicorns exist because there is no evidence that they don’t.” The appeal to ignorance is also called the burden of proof fallacy .

An ad hominem (Latin for “to the person”) is a type of informal logical fallacy . Instead of arguing against a person’s position, an ad hominem argument attacks the person’s character or actions in an effort to discredit them.

This rhetorical strategy is fallacious because a person’s character, motive, education, or other personal trait is logically irrelevant to whether their argument is true or false.

Name-calling is common in ad hominem fallacy (e.g., “environmental activists are ineffective because they’re all lazy tree-huggers”).

Ad hominem is a persuasive technique where someone tries to undermine the opponent’s argument by personally attacking them.

In this way, one can redirect the discussion away from the main topic and to the opponent’s personality without engaging with their viewpoint. When the opponent’s personality is irrelevant to the discussion, we call it an ad hominem fallacy .

Ad hominem tu quoque (‘you too”) is an attempt to rebut a claim by attacking its proponent on the grounds that they uphold a double standard or that they don’t practice what they preach. For example, someone is telling you that you should drive slowly otherwise you’ll get a speeding ticket one of these days, and you reply “but you used to get them all the time!”

Argumentum ad hominem means “argument to the person” in Latin and it is commonly referred to as ad hominem argument or personal attack. Ad hominem arguments are used in debates to refute an argument by attacking the character of the person making it, instead of the logic or premise of the argument itself.

The opposite of the hasty generalization fallacy is called slothful induction fallacy or appeal to coincidence .

It is the tendency to deny a conclusion even though there is sufficient evidence that supports it. Slothful induction occurs due to our natural tendency to dismiss events or facts that do not align with our personal biases and expectations. For example, a researcher may try to explain away unexpected results by claiming it is just a coincidence.

To avoid a hasty generalization fallacy we need to ensure that the conclusions drawn are well-supported by the appropriate evidence. More specifically:

• In statistics , if we want to draw inferences about an entire population, we need to make sure that the sample is random and representative of the population . We can achieve that by using a probability sampling method , like simple random sampling or stratified sampling .
• In academic writing , use precise language and measured phases. Try to avoid making absolute claims, cite specific instances and examples without applying the findings to a larger group.
• As readers, we need to ask ourselves “does the writer demonstrate sufficient knowledge of the situation or phenomenon that would allow them to make a generalization?”

The hasty generalization fallacy and the anecdotal evidence fallacy are similar in that they both result in conclusions drawn from insufficient evidence. However, there is a difference between the two:

• The hasty generalization fallacy involves genuinely considering an example or case (i.e., the evidence comes first and then an incorrect conclusion is drawn from this).
• The anecdotal evidence fallacy (also known as “cherry-picking” ) is knowing in advance what conclusion we want to support, and then selecting the story (or a few stories) that support it. By overemphasizing anecdotal evidence that fits well with the point we are trying to make, we overlook evidence that would undermine our argument.

Although many sources use circular reasoning fallacy and begging the question interchangeably, others point out that there is a subtle difference between the two:

• Begging the question fallacy occurs when you assume that an argument is true in order to justify a conclusion. If something begs the question, what you are actually asking is, “Is the premise of that argument actually true?” For example, the statement “Snakes make great pets. That’s why we should get a snake” begs the question “are snakes really great pets?”
• Circular reasoning fallacy on the other hand, occurs when the evidence used to support a claim is just a repetition of the claim itself.  For example, “People have free will because they can choose what to do.”

In other words, we could say begging the question is a form of circular reasoning.

Circular reasoning fallacy uses circular reasoning to support an argument. More specifically, the evidence used to support a claim is just a repetition of the claim itself. For example: “The President of the United States is a good leader (claim), because they are the leader of this country (supporting evidence)”.

An example of a non sequitur is the following statement:

“Giving up nuclear weapons weakened the United States’ military. Giving up nuclear weapons also weakened China. For this reason, it is wrong to try to outlaw firearms in the United States today.”

Clearly there is a step missing in this line of reasoning and the conclusion does not follow from the premise, resulting in a non sequitur fallacy .

The difference between the post hoc fallacy and the non sequitur fallacy is that post hoc fallacy infers a causal connection between two events where none exists, whereas the non sequitur fallacy infers a conclusion that lacks a logical connection to the premise.

In other words, a post hoc fallacy occurs when there is a lack of a cause-and-effect relationship, while a non sequitur fallacy occurs when there is a lack of logical connection.

An example of post hoc fallacy is the following line of reasoning:

“Yesterday I had ice cream, and today I have a terrible stomachache. I’m sure the ice cream caused this.”

Although it is possible that the ice cream had something to do with the stomachache, there is no proof to justify the conclusion other than the order of events. Therefore, this line of reasoning is fallacious.

Post hoc fallacy and hasty generalisation fallacy are similar in that they both involve jumping to conclusions. However, there is a difference between the two:

• Post hoc fallacy is assuming a cause and effect relationship between two events, simply because one happened after the other.
• Hasty generalisation fallacy is drawing a general conclusion from a small sample or little evidence.

In other words, post hoc fallacy involves a leap to a causal claim; hasty generalisation fallacy involves a leap to a general proposition.

The fallacy of composition is similar to and can be confused with the hasty generalization fallacy . However, there is a difference between the two:

• The fallacy of composition involves drawing an inference about the characteristics of a whole or group based on the characteristics of its individual members.
• The hasty generalization fallacy involves drawing an inference about a population or class of things on the basis of few atypical instances or a small sample of that population or thing.

In other words, the fallacy of composition is using an unwarranted assumption that we can infer something about a whole based on the characteristics of its parts, while the hasty generalization fallacy is using insufficient evidence to draw a conclusion.

The opposite of the fallacy of composition is the fallacy of division . In the fallacy of division, the assumption is that a characteristic which applies to a whole or a group must necessarily apply to the parts or individual members. For example, “Australians travel a lot. Gary is Australian, so he must travel a lot.”

Base rate fallacy can be avoided by following these steps:

• Avoid making an important decision in haste. When we are under pressure, we are more likely to resort to cognitive shortcuts like the availability heuristic and the representativeness heuristic . Due to this, we are more likely to factor in only current and vivid information, and ignore the actual probability of something happening (i.e., base rate).
• Take a long-term view on the decision or question at hand. Look for relevant statistical data, which can reveal long-term trends and give you the full picture.
• Talk to experts like professionals. They are more aware of probabilities related to specific decisions.

Suppose there is a population consisting of 90% psychologists and 10% engineers. Given that you know someone enjoyed physics at school, you may conclude that they are an engineer rather than a psychologist, even though you know that this person comes from a population consisting of far more psychologists than engineers.

When we ignore the rate of occurrence of some trait in a population (the base-rate information) we commit base rate fallacy .

Cost-benefit fallacy is a common error that occurs when allocating sources in project management. It is the fallacy of assuming that cost-benefit estimates are more or less accurate, when in fact they are highly inaccurate and biased. This means that cost-benefit analyses can be useful, but only after the cost-benefit fallacy has been acknowledged and corrected for. Cost-benefit fallacy is a type of base rate fallacy .

In advertising, the fallacy of equivocation is often used to create a pun. For example, a billboard company might advertise their billboards using a line like: “Looking for a sign? This is it!” The word sign has a literal meaning as billboard and a figurative one as a sign from God, the universe, etc.

Equivocation is a fallacy because it is a form of argumentation that is both misleading and logically unsound. When the meaning of a word or phrase shifts in the course of an argument, it causes confusion and also implies that the conclusion (which may be true) does not follow from the premise.

The fallacy of equivocation is an informal logical fallacy, meaning that the error lies in the content of the argument instead of the structure.

Fallacies of relevance are a group of fallacies that occur in arguments when the premises are logically irrelevant to the conclusion. Although at first there seems to be a connection between the premise and the conclusion, in reality fallacies of relevance use unrelated forms of appeal.

For example, the genetic fallacy makes an appeal to the source or origin of the claim in an attempt to assert or refute something.

The ad hominem fallacy and the genetic fallacy are closely related in that they are both fallacies of relevance. In other words, they both involve arguments that use evidence or examples that are not logically related to the argument at hand. However, there is a difference between the two:

• In the ad hominem fallacy , the goal is to discredit the argument by discrediting the person currently making the argument.
• In the genetic fallacy , the goal is to discredit the argument by discrediting the history or origin (i.e., genesis) of an argument.

False dilemma fallacy is also known as false dichotomy, false binary, and “either-or” fallacy. It is the fallacy of presenting only two choices, outcomes, or sides to an argument as the only possibilities, when more are available.

The false dilemma fallacy works in two ways:

• By presenting only two options as if these were the only ones available
• By presenting two options as mutually exclusive (i.e., only one option can be selected or can be true at a time)

In both cases, by using the false dilemma fallacy, one conceals alternative choices and doesn’t allow others to consider the full range of options. This is usually achieved through an“either-or” construction and polarised, divisive language (“you are either a friend or an enemy”).

The best way to avoid a false dilemma fallacy is to pause and reflect on two points:

• Are the options presented truly the only ones available ? It could be that another option has been deliberately omitted.
• Are the options mentioned mutually exclusive ? Perhaps all of the available options can be selected (or be true) at the same time, which shows that they aren’t mutually exclusive. Proving this is called “escaping between the horns of the dilemma.”

Begging the question fallacy is an argument in which you assume what you are trying to prove. In other words, your position and the justification of that position are the same, only slightly rephrased.

For example: “All freshmen should attend college orientation, because all college students should go to such an orientation.”

The complex question fallacy and begging the question fallacy are similar in that they are both based on assumptions. However, there is a difference between them:

• A complex question fallacy occurs when someone asks a question that presupposes the answer to another question that has not been established or accepted by the other person. For example, asking someone “Have you stopped cheating on tests?”, unless it has previously been established that the person is indeed cheating on tests, is a fallacy.
• Begging the question fallacy occurs when we assume the very thing as a premise that we’re trying to prove in our conclusion. In other words, the conclusion is used to support the premises, and the premises prove the validity of the conclusion. For example: “God exists because the Bible says so, and the Bible is true because it is the word of God.”

In other words, begging the question is about drawing a conclusion based on an assumption, while a complex question involves asking a question that presupposes the answer to a prior question.

“ No true Scotsman ” arguments aren’t always fallacious. When there is a generally accepted definition of who or what constitutes a group, it’s reasonable to use statements in the form of “no true Scotsman”.

For example, the statement that “no true pacifist would volunteer for military service” is not fallacious, since a pacifist is, by definition, someone who opposes war or violence as a means of settling disputes.

No true Scotsman arguments are fallacious because instead of logically refuting the counterexample, they simply assert that it doesn’t count. In other words, the counterexample is rejected for psychological, but not logical, reasons.

The appeal to purity or no true Scotsman fallacy is an attempt to defend a generalisation about a group from a counterexample by shifting the definition of the group in the middle of the argument. In this way, one can exclude the counterexample as not being “true”, “genuine”, or “pure” enough to be considered as part of the group in question.

To identify an appeal to authority fallacy , you can ask yourself the following questions:

• Is the authority cited really a qualified expert in this particular area under discussion? For example, someone who has formal education or years of experience can be an expert.
• Do experts disagree on this particular subject? If that is the case, then for almost any claim supported by one expert there will be a counterclaim that is supported by another expert. If there is no consensus, an appeal to authority is fallacious.
• Is the authority in question biased? If you suspect that an expert’s prejudice and bias could have influenced their views, then the expert is not reliable and an argument citing this expert will be fallacious.To identify an appeal to authority fallacy, you ask yourself whether the authority cited is a qualified expert in the particular area under discussion.

Appeal to authority is a fallacy when those who use it do not provide any justification to support their argument. Instead they cite someone famous who agrees with their viewpoint, but is not qualified to make reliable claims on the subject.

Appeal to authority fallacy is often convincing because of the effect authority figures have on us. When someone cites a famous person, a well-known scientist, a politician, etc. people tend to be distracted and often fail to critically examine whether the authority figure is indeed an expert in the area under discussion.

The ad populum fallacy is common in politics. One example is the following viewpoint: “The majority of our countrymen think we should have military operations overseas; therefore, it’s the right thing to do.”

This line of reasoning is fallacious, because popular acceptance of a belief or position does not amount to a justification of that belief. In other words, following the prevailing opinion without examining the underlying reasons is irrational.

The ad populum fallacy plays on our innate desire to fit in (known as “bandwagon effect”). If many people believe something, our common sense tells us that it must be true and we tend to accept it. However, in logic, the popularity of a proposition cannot serve as evidence of its truthfulness.

Ad populum (or appeal to popularity) fallacy and appeal to authority fallacy are similar in that they both conflate the validity of a belief with its popular acceptance among a specific group. However there is a key difference between the two:

• An ad populum fallacy tries to persuade others by claiming that something is true or right because a lot of people think so.
• An appeal to authority fallacy tries to persuade by claiming a group of experts believe something is true or right, therefore it must be so.

To identify a false cause fallacy , you need to carefully analyse the argument:

• When someone claims that one event directly causes another, ask if there is sufficient evidence to establish a cause-and-effect relationship. 
• Ask if the claim is based merely on the chronological order or co-occurrence of the two events. 
• Consider alternative possible explanations (are there other factors at play that could influence the outcome?).

By carefully analysing the reasoning, considering alternative explanations, and examining the evidence provided, you can identify a false cause fallacy and discern whether a causal claim is valid or flawed.

False cause fallacy examples include: 

• Believing that wearing your lucky jersey will help your team win 
• Thinking that everytime you wash your car, it rains
• Claiming that playing video games causes violent behavior 

In each of these examples, we falsely assume that one event causes another without any proof.

The planning fallacy and procrastination are not the same thing. Although they both relate to time and task management, they describe different challenges:

• The planning fallacy describes our inability to correctly estimate how long a future task will take, mainly due to optimism bias and a strong focus on the best-case scenario.
• Procrastination refers to postponing a task, usually by focusing on less urgent or more enjoyable activities. This is due to psychological reasons, like fear of failure.

In other words, the planning fallacy refers to inaccurate predictions about the time we need to finish a task, while procrastination is a deliberate delay due to psychological factors.

A real-life example of the planning fallacy is the construction of the Sydney Opera House in Australia. When construction began in the late 1950s, it was initially estimated that it would be completed in four years at a cost of around $7 million.

Because the government wanted the construction to start before political opposition would stop it and while public opinion was still favorable, a number of design issues had not been carefully studied in advance. Due to this, several problems appeared immediately after the project commenced.

The construction process eventually stretched over 14 years, with the Opera House being completed in 1973 at a cost of over $100 million, significantly exceeding the initial estimates.

An example of appeal to pity fallacy is the following appeal by a student to their professor:

“Professor, please consider raising my grade. I had a terrible semester: my car broke down, my laptop got stolen, and my cat got sick.”

While these circumstances may be unfortunate, they are not directly related to the student’s academic performance.

While both the appeal to pity fallacy and   red herring fallacy can serve as a distraction from the original discussion topic, they are distinct fallacies. More specifically:

• Appeal to pity fallacy attempts to evoke feelings of sympathy, pity, or guilt in an audience, so that they accept the speaker’s conclusion as truthful.
• Red herring fallacy attempts to introduce an irrelevant piece of information that diverts the audience’s attention to a different topic.

Both fallacies can be used as a tool of deception. However, they operate differently and serve distinct purposes in arguments.

Argumentum ad misericordiam (Latin for “argument from pity or misery”) is another name for appeal to pity fallacy . It occurs when someone evokes sympathy or guilt in an attempt to gain support for their claim, without providing any logical reasons to support the claim itself. Appeal to pity is a deceptive tactic of argumentation, playing on people’s emotions to sway their opinion.

Yes, it’s quite common to start a sentence with a preposition, and there’s no reason not to do so.

For example, the sentence “ To many, she was a hero” is perfectly grammatical. It could also be rephrased as “She was a hero to  many”, but there’s no particular reason to do so. Both versions are fine.

Some people argue that you shouldn’t end a sentence with a preposition , but that “rule” can also be ignored, since it’s not supported by serious language authorities.

Yes, it’s fine to end a sentence with a preposition . The “rule” against doing so is overwhelmingly rejected by modern style guides and language authorities and is based on the rules of Latin grammar, not English.

Trying to avoid ending a sentence with a preposition often results in very unnatural phrasings. For example, turning “He knows what he’s talking about ” into “He knows about what he’s talking” or “He knows that about which he’s talking” is definitely not an improvement.

No, ChatGPT is not a credible source of factual information and can’t be cited for this purpose in academic writing . While it tries to provide accurate answers, it often gets things wrong because its responses are based on patterns, not facts and data.

Specifically, the CRAAP test for evaluating sources includes five criteria: currency , relevance , authority , accuracy , and purpose . ChatGPT fails to meet at least three of them:

• Currency: The dataset that ChatGPT was trained on only extends to 2021, making it slightly outdated.
• Authority: It’s just a language model and is not considered a trustworthy source of factual information.
• Accuracy: It bases its responses on patterns rather than evidence and is unable to cite its sources .

So you shouldn’t cite ChatGPT as a trustworthy source for a factual claim. You might still cite ChatGPT for other reasons – for example, if you’re writing a paper about AI language models, ChatGPT responses are a relevant primary source .

ChatGPT is an AI language model that was trained on a large body of text from a variety of sources (e.g., Wikipedia, books, news articles, scientific journals). The dataset only went up to 2021, meaning that it lacks information on more recent events.

It’s also important to understand that ChatGPT doesn’t access a database of facts to answer your questions. Instead, its responses are based on patterns that it saw in the training data.

So ChatGPT is not always trustworthy . It can usually answer general knowledge questions accurately, but it can easily give misleading answers on more specialist topics.

Another consequence of this way of generating responses is that ChatGPT usually can’t cite its sources accurately. It doesn’t really know what source it’s basing any specific claim on. It’s best to check any information you get from it against a credible source .

No, it is not possible to cite your sources with ChatGPT . You can ask it to create citations, but it isn’t designed for this task and tends to make up sources that don’t exist or present information in the wrong format. ChatGPT also cannot add citations to direct quotes in your text.

Instead, use a tool designed for this purpose, like the Scribbr Citation Generator .

But you can use ChatGPT for assignments in other ways, to provide inspiration, feedback, and general writing advice.

GPT  stands for “generative pre-trained transformer”, which is a type of large language model: a neural network trained on a very large amount of text to produce convincing, human-like language outputs. The Chat part of the name just means “chat”: ChatGPT is a chatbot that you interact with by typing in text.

The technology behind ChatGPT is GPT-3.5 (in the free version) or GPT-4 (in the premium version). These are the names for the specific versions of the GPT model. GPT-4 is currently the most advanced model that OpenAI has created. It’s also the model used in Bing’s chatbot feature.

ChatGPT was created by OpenAI, an AI research company. It started as a nonprofit company in 2015 but became for-profit in 2019. Its CEO is Sam Altman, who also co-founded the company. OpenAI released ChatGPT as a free “research preview” in November 2022. Currently, it’s still available for free, although a more advanced premium version is available if you pay for it.

OpenAI is also known for developing DALL-E, an AI image generator that runs on similar technology to ChatGPT.

ChatGPT is owned by OpenAI, the company that developed and released it. OpenAI is a company dedicated to AI research. It started as a nonprofit company in 2015 but transitioned to for-profit in 2019. Its current CEO is Sam Altman, who also co-founded the company.

In terms of who owns the content generated by ChatGPT, OpenAI states that it will not claim copyright on this content , and the terms of use state that “you can use Content for any purpose, including commercial purposes such as sale or publication”. This means that you effectively own any content you generate with ChatGPT and can use it for your own purposes.

Be cautious about how you use ChatGPT content in an academic context. University policies on AI writing are still developing, so even if you “own” the content, you’re often not allowed to submit it as your own work according to your university or to publish it in a journal.

ChatGPT is a chatbot based on a large language model (LLM). These models are trained on huge datasets consisting of hundreds of billions of words of text, based on which the model learns to effectively predict natural responses to the prompts you enter.

ChatGPT was also refined through a process called reinforcement learning from human feedback (RLHF), which involves “rewarding” the model for providing useful answers and discouraging inappropriate answers – encouraging it to make fewer mistakes.

Essentially, ChatGPT’s answers are based on predicting the most likely responses to your inputs based on its training data, with a reward system on top of this to incentivise it to give you the most helpful answers possible. It’s a bit like an incredibly advanced version of predictive text. This is also one of ChatGPT’s limitations : because its answers are based on probabilities, they’re not always trustworthy .

OpenAI may store ChatGPT conversations for the purposes of future training. Additionally, these conversations may be monitored by human AI trainers.

Users can choose not to have their chat history saved. Unsaved chats are not used to train future models and are permanently deleted from ChatGPT’s system after 30 days.

The official ChatGPT app is currently only available on iOS devices. If you don’t have an iOS device, only use the official OpenAI website to access the tool. This helps to eliminate the potential risk of downloading fraudulent or malicious software.

ChatGPT conversations are generally used to train future models and to resolve issues/bugs. These chats may be monitored by human AI trainers.

However, users can opt out of having their conversations used for training. In these instances, chats are monitored only for potential abuse.

Yes, using ChatGPT as a conversation partner is a great way to practice a language in an interactive way.

Try using a prompt like this one:

“Please be my Spanish conversation partner. Only speak to me in Spanish. Keep your answers short (maximum 50 words). Ask me questions. Let’s start the conversation with the following topic: [conversation topic].”

Yes, there are a variety of ways to use ChatGPT for language learning , including treating it as a conversation partner, asking it for translations, and using it to generate a curriculum or practice exercises.

AI detectors aim to identify the presence of AI-generated text (e.g., from ChatGPT ) in a piece of writing, but they can’t do so with complete accuracy. In our comparison of the best AI detectors , we found that the 10 tools we tested had an average accuracy of 60%. The best free tool had 68% accuracy, the best premium tool 84%.

Because of how AI detectors work , they can never guarantee 100% accuracy, and there is always at least a small risk of false positives (human text being marked as AI-generated). Therefore, these tools should not be relied upon to provide absolute proof that a text is or isn’t AI-generated. Rather, they can provide a good indication in combination with other evidence.

Tools called AI detectors are designed to label text as AI-generated or human. AI detectors work by looking for specific characteristics in the text, such as a low level of randomness in word choice and sentence length. These characteristics are typical of AI writing, allowing the detector to make a good guess at when text is AI-generated.

But these tools can’t guarantee 100% accuracy. Check out our comparison of the best AI detectors to learn more.

You can also manually watch for clues that a text is AI-generated – for example, a very different style from the writer’s usual voice or a generic, overly polite tone.

Our research into the best summary generators (aka summarisers or summarising tools) found that the best summariser available in 2023 is the one offered by QuillBot.

While many summarisers just pick out some sentences from the text, QuillBot generates original summaries that are creative, clear, accurate, and concise. It can summarise texts of up to 1,200 words for free, or up to 6,000 with a premium subscription.

Try the QuillBot summarizer for free

Deep learning requires a large dataset (e.g., images or text) to learn from. The more diverse and representative the data, the better the model will learn to recognise objects or make predictions. Only when the training data is sufficiently varied can the model make accurate predictions or recognise objects from new data.

Deep learning models can be biased in their predictions if the training data consist of biased information. For example, if a deep learning model used for screening job applicants has been trained with a dataset consisting primarily of white male applicants, it will consistently favour this specific population over others.

A good ChatGPT prompt (i.e., one that will get you the kinds of responses you want):

• Gives the tool a role to explain what type of answer you expect from it
• Is precisely formulated and gives enough context
• Is free from bias
• Has been tested and improved by experimenting with the tool

ChatGPT prompts are the textual inputs (e.g., questions, instructions) that you enter into ChatGPT to get responses.

ChatGPT predicts an appropriate response to the prompt you entered. In general, a more specific and carefully worded prompt will get you better responses.

Yes, ChatGPT is currently available for free. You have to sign up for a free account to use the tool, and you should be aware that your data may be collected to train future versions of the model.

To sign up and use the tool for free, go to this page and click “Sign up”. You can do so with your email or with a Google account.

A premium version of the tool called ChatGPT Plus is available as a monthly subscription. It currently costs £16 and gets you access to features like GPT-4 (a more advanced version of the language model). But it’s optional: you can use the tool completely free if you’re not interested in the extra features.

You can access ChatGPT by signing up for a free account:

• Follow this link to the ChatGPT website.
• Click on “Sign up” and fill in the necessary details (or use your Google account). It’s free to sign up and use the tool.
• Type a prompt into the chat box to get started!

A ChatGPT app is also available for iOS, and an Android app is planned for the future. The app works similarly to the website, and you log in with the same account for both.

According to OpenAI’s terms of use, users have the right to reproduce text generated by ChatGPT during conversations.

However, publishing ChatGPT outputs may have legal implications , such as copyright infringement.

Users should be aware of such issues and use ChatGPT outputs as a source of inspiration instead.

According to OpenAI’s terms of use, users have the right to use outputs from their own ChatGPT conversations for any purpose (including commercial publication).

However, users should be aware of the potential legal implications of publishing ChatGPT outputs. ChatGPT responses are not always unique: different users may receive the same response.

Furthermore, ChatGPT outputs may contain copyrighted material. Users may be liable if they reproduce such material.

ChatGPT can sometimes reproduce biases from its training data , since it draws on the text it has “seen” to create plausible responses to your prompts.

For example, users have shown that it sometimes makes sexist assumptions such as that a doctor mentioned in a prompt must be a man rather than a woman. Some have also pointed out political bias in terms of which political figures the tool is willing to write positively or negatively about and which requests it refuses.

The tool is unlikely to be consistently biased toward a particular perspective or against a particular group. Rather, its responses are based on its training data and on the way you phrase your ChatGPT prompts . It’s sensitive to phrasing, so asking it the same question in different ways will result in quite different answers.

Information extraction  refers to the process of starting from unstructured sources (e.g., text documents written in ordinary English) and automatically extracting structured information (i.e., data in a clearly defined format that’s easily understood by computers). It’s an important concept in natural language processing (NLP) .

For example, you might think of using news articles full of celebrity gossip to automatically create a database of the relationships between the celebrities mentioned (e.g., married, dating, divorced, feuding). You would end up with data in a structured format, something like MarriageBetween(celebrity 1 ,celebrity 2 ,date) .

The challenge involves developing systems that can “understand” the text well enough to extract this kind of data from it.

Knowledge representation and reasoning (KRR) is the study of how to represent information about the world in a form that can be used by a computer system to solve and reason about complex problems. It is an important field of artificial intelligence (AI) research.

An example of a KRR application is a semantic network, a way of grouping words or concepts by how closely related they are and formally defining the relationships between them so that a machine can “understand” language in something like the way people do.

A related concept is information extraction , concerned with how to get structured information from unstructured sources.

Yes, you can use ChatGPT to summarise text . This can help you understand complex information more easily, summarise the central argument of your own paper, or clarify your research question.

You can also use Scribbr’s free text summariser , which is designed specifically for this purpose.

Yes, you can use ChatGPT to paraphrase text to help you express your ideas more clearly, explore different ways of phrasing your arguments, and avoid repetition.

However, it’s not specifically designed for this purpose. We recommend using a specialised tool like Scribbr’s free paraphrasing tool , which will provide a smoother user experience.

Yes, you use ChatGPT to help write your college essay by having it generate feedback on certain aspects of your work (consistency of tone, clarity of structure, etc.).

However, ChatGPT is not able to adequately judge qualities like vulnerability and authenticity. For this reason, it’s important to also ask for feedback from people who have experience with college essays and who know you well. Alternatively, you can get advice using Scribbr’s essay editing service .

No, having ChatGPT write your college essay can negatively impact your application in numerous ways. ChatGPT outputs are unoriginal and lack personal insight.

Furthermore, Passing off AI-generated text as your own work is considered academically dishonest . AI detectors may be used to detect this offense, and it’s highly unlikely that any university will accept you if you are caught submitting an AI-generated admission essay.

However, you can use ChatGPT to help write your college essay during the preparation and revision stages (e.g., for brainstorming ideas and generating feedback).

ChatGPT and other AI writing tools can have unethical uses. These include:

• Reproducing biases and false information
• Using ChatGPT to cheat in academic contexts
• Violating the privacy of others by inputting personal information

However, when used correctly, AI writing tools can be helpful resources for improving your academic writing and research skills. Some ways to use ChatGPT ethically include:

• Following your institution’s guidelines
• Critically evaluating outputs
• Being transparent about how you used the tool

Ask our team

Want to contact us directly? No problem. We are always here for you.

• Chat with us
• Email [email protected]
• Call +44 (0)20 3917 4242
• WhatsApp +31 20 261 6040

Our support team is here to help you daily via chat, WhatsApp, email, or phone between 9:00 a.m. to 11:00 p.m. CET.

Our APA experts default to APA 7 for editing and formatting. For the Citation Editing Service you are able to choose between APA 6 and 7.

Yes, if your document is longer than 20,000 words, you will get a sample of approximately 2,000 words. This sample edit gives you a first impression of the editor’s editing style and a chance to ask questions and give feedback.

How does the sample edit work?

You will receive the sample edit within 24 hours after placing your order. You then have 24 hours to let us know if you’re happy with the sample or if there’s something you would like the editor to do differently.

Read more about how the sample edit works

Yes, you can upload your document in sections.

We try our best to ensure that the same editor checks all the different sections of your document. When you upload a new file, our system recognizes you as a returning customer, and we immediately contact the editor who helped you before.

However, we cannot guarantee that the same editor will be available. Your chances are higher if

• You send us your text as soon as possible and
• You can be flexible about the deadline.

Please note that the shorter your deadline is, the lower the chance that your previous editor is not available.

If your previous editor isn’t available, then we will inform you immediately and look for another qualified editor. Fear not! Every Scribbr editor follows the  Scribbr Improvement Model  and will deliver high-quality work.

Yes, our editors also work during the weekends and holidays.

Because we have many editors available, we can check your document 24 hours per day and 7 days per week, all year round.

If you choose a 72 hour deadline and upload your document on a Thursday evening, you’ll have your thesis back by Sunday evening!

Yes! Our editors are all native speakers, and they have lots of experience editing texts written by ESL students. They will make sure your grammar is perfect and point out any sentences that are difficult to understand. They’ll also notice your most common mistakes, and give you personal feedback to improve your writing in English.

Every Scribbr order comes with our award-winning Proofreading & Editing service , which combines two important stages of the revision process.

For a more comprehensive edit, you can add a Structure Check or Clarity Check to your order. With these building blocks, you can customize the kind of feedback you receive.

You might be familiar with a different set of editing terms. To help you understand what you can expect at Scribbr, we created this table:

View an example

When you place an order, you can specify your field of study and we’ll match you with an editor who has familiarity with this area.

However, our editors are language specialists, not academic experts in your field. Your editor’s job is not to comment on the content of your dissertation, but to improve your language and help you express your ideas as clearly and fluently as possible.

This means that your editor will understand your text well enough to give feedback on its clarity, logic and structure, but not on the accuracy or originality of its content.

Good academic writing should be understandable to a non-expert reader, and we believe that academic editing is a discipline in itself. The research, ideas and arguments are all yours – we’re here to make sure they shine!

After your document has been edited, you will receive an email with a link to download the document.

The editor has made changes to your document using ‘Track Changes’ in Word. This means that you only have to accept or ignore the changes that are made in the text one by one.

It is also possible to accept all changes at once. However, we strongly advise you not to do so for the following reasons:

• You can learn a lot by looking at the mistakes you made.
• The editors don’t only change the text – they also place comments when sentences or sometimes even entire paragraphs are unclear. You should read through these comments and take into account your editor’s tips and suggestions.
• With a final read-through, you can make sure you’re 100% happy with your text before you submit!

You choose the turnaround time when ordering. We can return your dissertation within 24 hours , 3 days or 1 week . These timescales include weekends and holidays. As soon as you’ve paid, the deadline is set, and we guarantee to meet it! We’ll notify you by text and email when your editor has completed the job.

Very large orders might not be possible to complete in 24 hours. On average, our editors can complete around 13,000 words in a day while maintaining our high quality standards. If your order is longer than this and urgent, contact us to discuss possibilities.

Always leave yourself enough time to check through the document and accept the changes before your submission deadline.

Scribbr is specialised in editing study related documents. We check:

• Graduation projects
• Dissertations
• Admissions essays
• College essays
• Application essays
• Personal statements
• Process reports
• Reflections
• Internship reports
• Academic papers
• Research proposals
• Prospectuses

Calculate the costs

The fastest turnaround time is 24 hours.

You can upload your document at any time and choose between three deadlines:

At Scribbr, we promise to make every customer 100% happy with the service we offer. Our philosophy: Your complaint is always justified – no denial, no doubts.

Our customer support team is here to find the solution that helps you the most, whether that’s a free new edit or a refund for the service.

Yes, in the order process you can indicate your preference for American, British, or Australian English .

If you don’t choose one, your editor will follow the style of English you currently use. If your editor has any questions about this, we will contact you.

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Guidance on...

Guidance on terminology, application, and reporting of citation searching: the TARCiS statement

• Related content
• Peer review
• Julian Hirt , research fellow and lecturer 1 2 3 ,
• Thomas Nordhausen , research fellow 4 ,
• Thomas Fuerst , medical information specialist 5 ,
• Hannah Ewald , medical information specialist 5 ,
• Christian Appenzeller-Herzog , medical information specialist 5
• on behalf of the TARCiS study group
• 1 Pragmatic Evidence Lab, Research Centre for Clinical Neuroimmunology and Neuroscience Basel, University Hospital Basel and University of Basel, Basel, Switzerland
• 2 Department of Health, Eastern Switzerland University of Applied Sciences, St Gallen, Switzerland
• 3 Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
• 4 Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany
• 5 University Medical Library, University of Basel, 4051 Basel, Switzerland
• Correspondence to: C Appenzeller-Herzog christian.appenzeller{at}unibas.ch
• Accepted 19 March 2024

Evidence syntheses adhering to systematic literature searching techniques are a cornerstone of evidence based healthcare. Beyond term based searching in electronic databases, citation searching is a prevalent search technique to identify relevant sources of evidence. However, for decades, citation searching methodology and terminology has not been standardised. An evidence guided, four round Delphi consensus study was conducted with 27 international methodological experts in order to develop the Terminology, Application, and Reporting of Citation Searching (TARCiS) statement. TARCiS comprises 10 specific recommendations, each with a rationale and explanation on when and how to conduct and report citation searching in the context of systematic literature searches. The statement also presents four research priorities, and it is hoped that systematic review teams are encouraged to incorporate TARCiS into standardised workflows.

Synthesising scientific evidence by looking at the citation relationships of a scientific record (ie, citation searching) was the underlying objective when the Science Citation Index, the antecedent of Web of Science, was introduced in 1963. 1 Although the availability of electronic citation indexes has increased, evidence syntheses in systematic reviews do not primarily rely on citation searching for literature retrieval but rather on search methods based on text and keywords. 2 When used in systematic review workflows, citation searching traditionally constitutes a supplementary search technique that builds on an initial set of references from the primary database search (seed references). 3

Citation searching is an umbrella term that entails various methods of citation based literature retrieval ( fig 1 ). Checking references cited by seed references, also known as backward citation searching, is the most prevalent and a mandatory step when conducting Cochrane reviews. 4 In forward citation searching, systematic reviewers can also assess the eligibility of articles that cite the seed references. Backward and forward citation searching are known as direct citation searching ( fig 1 ). They can be supplemented by indirect retrieval methods—namely, by co-citing citation searching (retrieving articles that share cited references with a seed reference) and co-cited citation searching (retrieving articles that share citing references with a seed reference).

Overview of citation searching methods. Direct (dark blue boxes) and indirect (light blue boxes) citation relationships of references are shown, relative to a seed reference; arrows denote the direction of citation (ie, source A citing target B); horizontal axis denotes time (ie, the chronology in which references were published relative to the seed reference). Visual examples of cited references (accessible via backward citation searching), citing references (accessible via forward citation searching), co-citing references (accessible via co-citing citation searching), and co-cited references (accessible via co-cited citation searching) are shown. Note that the total number of the co-citing and co-cited references of a seed reference far exceeds the number shown in the light blue boxes

• Download figure
• Open in new tab
• Download powerpoint

Citation searching can contribute substantially to evidence retrieval and can show similar or even superior effectiveness and efficiency compared with text and keyword based searches. An audit of the different search methods used in a systematic review of complex evidence, for instance, revealed that 44% of all included studies were identified by backward citation searching, and 7% by forward citation searching. In comparison, initial text and keyword searches accounted for only 25% of included studies. 5 For the scoping review that collected methodological studies as a foundation for the present work, these figures were 28% and 12% for backward and forward citation searching, respectively, compared with 52% for extensive primary database searching. 6

The conduct of systematic reviews is prominently guided by standard recommendations such as those in the Cochrane handbook, 4 whereas their reporting is standardised by the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. 7 In contrast and despite its application by systematic reviewers for decades, standardised methodology and terminology for citation searching is not available. Of the three aspects on when to do citation searching, how to conduct citation searching, and how to report citation searching, limited guidance exists only for the third aspect in the PRISMA extension for reporting literature searches (PRISMA-S). 8 Unsurprisingly, methodological studies show considerable heterogeneity in terms of citation searching terminology and recommended best practices. 6 Even in a sample of Cochrane reviews, 13% did not use backward citation searching despite this being a mandatory step. 9 The lack of standardisation not only impairs the transparency, reproducibility, and comparability of systematic reviews, but might also reduce article recall that could affect pooled effect estimates, guidance, and clinical decision making. On the other hand, uninformed use of citation searching in contexts where it is less useful might cause undue workloads.

We systematically collected evidence on the use, benefit, and reporting of citation searching 6 and put it through a four round, online Delphi study. Together with the Terminology, Application, and Reporting of Citation Searching (TARCiS) study group, an international panel of methodological experts, we aimed to develop consensus for recommendations on when and how to conduct citation searching, and on how to report it, including a consensus set of citation searching terms. Furthermore, we framed research priorities for future methodological development of citation searching in the context of systematic literature searches.

Summary points

The TARCiS (Terminology, Application, and Reporting of Citation Searching) statement provides guidance in which contexts citation searching is likely to be beneficial for systematic reviewers

TARCiS comprises 10 specific recommendations on when and how to conduct citation searching and how to report it in the context of systematic literature searches, and also frames four research priorities

The statement will contribute to a unified terminology, systematic application, and transparent reporting of citation searching and support those who are conducting or assessing citation searching methods

To develop the TARCiS statement, a stepwise approach comprising a scoping review of the methodological literature (step 1; reported in detail in a separate publication 6 ) and a Delphi study (step 2; reported in this publication) was chosen. The methods were prespecified in two study protocols. 10 11 The complete process is shown in figure 2 .

Flow diagram of the development process of the TARCiS (Terminology, Application, and Reporting of Citation Searching) statement. Actions and outcomes of the development phases of the TARCiS statement are shown. Appendix 1 shows more detailed reporting of consensus scores

Step 1: Scoping review

We conducted a scoping review on the terminology that describes citation searching, the methods and tools used for citation searching, and its benefit. We considered methodological studies of any design that aimed to assess the role of citation searching, compared multiple citation searching methods, or compared technical uses of citation searching within health related topics. We searched five bibliographic databases, conducted backward and forward citation searches of eligible studies and pertinent reviews, and consulted librarians and information specialists for further eligible studies. The results were summarised by descriptive statistics and narratively. The detailed methods of the scoping review have been published elsewhere. 6 10

Step 2: Delphi study

To develop consensus on recommendations and research priorities as tentatively derived from the results of step 1, 6 we performed a multistage online Delphi study. Delphi refers to a structured process where collective knowledge from an expert panel is synthesised using a series of questionnaires, each one questionnaire adapted on the basis of the responses to a previous version. 12 13 14 We recruited an international panel of individuals experienced in conducting or reporting citation searching methods. For this, we invited authors of methodological studies, as identified in step 1, 6 and methodological experts from international systematic review organisations or from our professional networks by email to participate in the Delphi study.

The Delphi study comprised four prespecified rounds. 10 11 The first round was pretested by four non-study related academic affiliates. Each round covered four to five thematic parts (appendix 2; table 1 ). Briefly, part A dealt with the terminology framework to describe citation searching methods in eight domains (for details, refer to table 4 in Hirt et al 6 ). Part B contained pre-formulated recommendations on conduct and reporting of citation searching. Each recommendation was supported by a rationale and explanation text that were also subjected to collective consensus finding. Part C covered research priorities that were also derived from the scoping review. 6 Part D contained a free text field to collect general comments from the panellists. Part E was designed to collect sociodemographic information and was limited to Delphi round 1.

Data collection through four rounds of Delphi study to develop consensus on recommendations and research priorities of the TARCiS statement

• View inline

Non-participating panellists were recorded as non-participators for a given round. Panellists who missed all rounds were recorded as non-responders. Recommendations and research priorities that had not yet reached the prespecified consensus of at least 75% were refined for the subsequent Delphi round. These refinements were based on the panellists’ comments. In rare cases, when additional valid suggestions from panellists for reformulation of rationale or explanation texts were submitted, recommendations that already reached the agreement threshold were also adapted and forwarded to the next Delphi round. For more methodological details on the Delphi study, see table 1 and the published protocols. 10 11

Deviations from the Delphi study protocol

For round 3 of the Delphi, we had originally planned to formulate one recommendation for each of the eight terminology domains ( table 1 , see also description to part A above). Depending on the votes, however, this approach might have led to the selection of inconsistent terms (eg, backward citation searching v forward citation tracking). Hence, we decided to use the terms that received the most votes in Delphi round 2 to formulate four term sets, which were consistent across all eight domains. Secondly, instead of using SosciSurvey 15 as a survey tool, 8 we switched to the Unipark/Enterprise Feedback Suite survey, 16 which provided enhanced design and functional features. Thirdly, in addition to personalised emails (person based approach), we originally intended to recruit panellists using professional mailing lists and central requests to systematic review organisations (organisation based approach). 8 However, because we had already recruited sufficient panellists using the person based approach (including individuals who were affiliated with various systematic review organisations), we waived the organisation based approach.

We identified 47 methodological studies that assessed the use, benefit, and reporting of citation searching. In 45 studies (96%), the use of citation searching showed an added value. Thirty two studies (68%) analysed the impact of citation searching in one or more previous systematic reviews. Application, terminology, and reporting of citation searching were heterogeneous. Details on the results of the scoping review can be found elsewhere. 6

Recruitment and characteristics of panellists

Of 35 experts identified and contacted, 30 declared an interest in participating and were invited to Delphi round 1. Three (10%) of the 30 panellists were non-responders. Table 2 summarises the personal and professional characteristics of the 27 participating panellists.

Characteristics of 27 panellists* participating in the Delphi study to develop consensus on recommendations and research priorities of the TARCiS statement

TARCiS statement: final recommendations, rationale and explanations, and research priorities

Items for data collection through the four Delphi rounds in parts A-E are summarised in table 1 . The Delphi study started with 41 terms describing different aspects of citation searching, eight draft recommendations with rationale texts on the conduct and reporting of citation searching, and one research priority (appendix 1). After Delphi round 4, the finalised TARCiS statement comprised 10 recommendations with rationale and explanation texts and four research priorities that reached consensus scores between 83% and 100%. Figure 2 and appendix 1 show details on content and consensus scores in rounds 1-4. An overview of all 14 TARCiS items omitting rationale and explanation texts is presented in box 1 . A terminology and reporting item checklist based on TARCiS recommendations 1 and 10 is available in appendix 3 and on the TARCiS website. 17

TARCiS statement

Recommendations on terminology, conduct, and reporting of citation searching.

The following terminology should be used to describe search methods that exploit citation relationships:

“Citation searching” as an umbrella term.

“Backward citation searching” to describe the sub-method retrieving and screening cited references.

“Reference list checking” to describe the sub-method retrieving and screening cited references by manually reviewing reference lists.

“Forward citation searching” to describe the sub-method retrieving and screening citing references.

“Co-cited citation searching” to describe the sub-method retrieving and screening co-cited references.

“Co-citing citation searching” to describe the sub-method retrieving and screening co-citing references.

“Iterative citation searching” to describe one or more repetition(s) of a search method that exploits citation relationships.

“Seed references” to describe relevant articles that are known beforehand and used as a starting point for any citation search.

For systematic search topics that are difficult to search for, backward and forward citation searching should be seriously considered as supplementary search techniques.

For systematic search topics that are easier to search for and addressed by a highly sensitive search, backward and forward citation searching are not explicitly recommended as supplementary search techniques. Reference list checking of included records can be used to confirm the sensitivity of the search strategy.

Backward and forward citation searching as supplementary search techniques should be based on all included records of the primary search (ie, all records that meet the inclusion criteria of the review after full text screening of the primary search results). Occasionally, it can be justified to deviate from this recommendation and either use further pertinent records as additional seed references or only a defined sample of the included records.

Backward citation searching should ideally be conducted by screening the titles and abstracts of the seed references as provided by a citation index. Screening titles as provided when checking reference lists of the seed references can still be performed.

Using the combined coverage of two citation indexes for citation searching to achieve more extensive coverage should be considered if access is available. This combination is especially meaningful if seed references cannot be found in one index and reference lists were not checked.

Before screening, the results of supplementary backward and forward citation searching should be deduplicated.

If citation searching finds additional eligible records, another iteration of citation searching should be considered using these records as new seed references.

Standalone citation searching should not be used for literature searches that aim at completeness of recall.

Reporting of citation searching should clearly state:

the seed references (along with a justification should the seed references differ from the set of included records from the results of the primary database search),

the directionality of searching (backward, forward, co-cited, co-citing),

the date(s) of searching (which might differ between rounds of iterative citation searching) (not applicable for reference list checking),

the number of citation searching iterations (and possibly the reason for stopping if the last iteration still retrieved additional eligible records),

all citation indexes searched (eg, Lens.org, Google Scholar, Scopus, citation indexes in Web of Science) and, if applicable, the tools that were used to access them (eg, Publish or Perish, citationchaser),

if applicable, information about the deduplication process (eg, manual/automated, the software or tool used),

the method of screening (ie, state whether the records were screened in the same way as the primary search results or, if not, describe the alternative method used), and

the number of citation searching results in the right column box of the PRISMA 2020 flow diagram for new or updated systematic reviews that included searches of databases, registers, and other sources .

Research priorities

The effectiveness, applicability, and conduct of indirect citation searching methods as supplementary search methods in systematic reviewing require further research (including retrieval of additional unique references, their relevance for the review and prioritisation of results).

Further research is needed to assess the value of citation searching. Potential research topics could be:

influence of citation searching on results and conclusions of systematic evidence syntheses,

topics or at least determinants of topics where citation searching likely/not likely has additional value, or

economic evaluation of citation searching to assess the cost and time of conducting citation searching in relation to its benefit.

Further research is needed to assess the best way to perform citation searching. Potential research topics could be:

optimal selection of seed references,

optimal use of indexes and tools and their combination to conduct citation searching,

methods and tools for deduplication of citation searching results,

subjective influences on citation searching (eg, experience of researcher, prevention of mistakes), or

reproducibility of citation searching.

Further research is needed to reproduce existing studies: Any recommendations in this Delphi that are based on only 1-2 studies require reproduction of these studies in the form of larger, prospectively planned studies that grade the evidence for each recommendation and propose additional research where the grade of evidence is weak.

The TARCiS checklist for terminology and reporting of citation searching is available for download. 17

PRISMA=Preferred Reporting Items for Systematic reviews and Meta-Analyses; TARCiS=Terminology, Application, and Reporting of Citation Searching.

Recommendation 1

Rationale and explanation supporting recommendation 1.

As compiled in a recent scoping review, 6 the reporting of citation searching methods is frequently unclear and far from being standardised. For example, “citation searching,” “snowballing,” or “co-citation searching” are sometimes used as methodological umbrella terms but also to denote a specific method such as backward or forward citation searching. 6 For clarity, standardised vocabulary is needed. The set of terms brought forward in this recommendation is consistent in itself as well as with the terminology used in PRISMA-S and PRISMA 2020 guidelines 8 18 and hence well suited for uniform reporting of citation searching.

Recommendation 2

Rationale and explanation supporting recommendation 2:.

Evidence indicates that the ability of citation searching as a supplementary search technique to find additional unique records in a systematic literature search varies between reviews. 6 Searches for particular study designs (qualitative, mixed method, observational, prognostic, or diagnostic test studies) or health science topics such as non-pharmacological, non-clinical, public health, policy making, service delivery, or alternative medicine have been linked with effective supplementary citation searching. 19 20 21 22 The underlying reasons include poor transferability to text based searching owing to poor conceptual clarity, inconsistent terminology, or vocabulary overlaps with unrelated topics. 5 The ability of citation searching to find any publication type including unpublished or grey literature or literature that is not indexed in major databases (eg, concerning a developing country) might also be relevant. 23 However, a clear categorisation of topics that are difficult to search for is currently not possible and it remains for the review authors themselves to judge whether their review topic is likely to fall into this category.

For people conducting the search who have difficulty assessing whether the topic is difficult or easier to search for, we recommend that they opt for citation searching or consult an experienced information specialist. 24 If the search strategy does not exhaustively capture the topic, backward and forward citation searching might compensate for some of the potential loss of information.

Recommendation 3

Rationale and explanation supporting recommendation 3.

Evidence indicates that the ability of citation searching as a supplementary search technique to find additional unique references in a systematic literature search varies between reviews. 6 Searches for clearly defined clinical interventions as part of PICO (participant, intervention, comparison, outcome) questions have been linked with less effective supplementary citation searching, especially when the search strategies are sensitive and conducted in several databases. However, a clear categorisation of topics that are easier to search for is currently not possible, and it remains for the review authors themselves to judge whether their review topic is likely to fall into this category.

By checking reference lists within the full texts of seed references, review authors can test the sensitivity of their primary search strategy (ie, electronic database search). 25 If no additional relevant, unique studies are found, the primary search might have been sensitive enough. If additional relevant, unique studies are found, these could indicate that the primary search was not sensitive enough.

For individuals conducting the search who have difficulty assessing whether the topic is difficult or easier to search for, we recommend that they opt for citation searching or consult an experienced information specialist. 24 If, for whatever reason, the search strategy does not exhaustively capture the topic, backward and forward citation searching could compensate for some of the potential loss of information.

Recommendation 4

Rationale and explanation supporting recommendation 4.

The more seed references used, the better the chance that citation searching finds additional relevant unique records. While using only a sample of the included records as seed references might be enough, there is currently no evidence that could help decide how many seeds are needed or how to decide which might perform better. Hence, we recommend using all the records that meet the inclusion criteria of the review after full text screening of the primary database search results.

However, review authors could deviate from this recommendation if they deal with a very small or large number of included records. A very small number of included records might not yield additional relevant records or only have limited value. In this case, review authors could use further records as seed references for citation searching (eg, systematic reviews on the topic that were flagged during the screening phase). 26 A very large number of included records could lead to too many records to screen. In this case, review authors might use a selected sample of included records as seed references for citation searching. In the event of such deviation, authors should describe their rationale and sampling method (eg, random sample).

Recommendation 5

Rationale and explanation supporting recommendation 5.

Citation searching workflows encompass two consecutive steps: retrieval of records and screening of retrieved records for eligibility. When using an electronic citation index for citation searching, retrieval and screening are usually separated. While forward citation searching requires a citation index, backward citation searching can also be performed by manually checking the reference lists of the seed references. Reference list checking is sometimes part of an established workflow, for example, during the eligibility assessment of the full text record or during data extraction. 25 Merging these two steps allows researchers to know the context in which a reference was used and that all references can be screened. However, reference list checking has three disadvantages:

The retrieval and screening phases are no longer separated, which makes reporting of the methods or results difficult and unclear

Citations from reference list checking cannot be deduplicated against each other or against the primary search results, which could add an unnecessarily high workload (see recommendation 7)

Eligibility assessments are restricted to the titles (instead of titles and abstracts) which could lead to relevant records being overlooked due to uninformative titles mentioned in vague contexts.

In recent years, online citation searching options via citation indexes or free to access citation searching tools have become more readily available leading to faster and easier procedures. 27 28 29 30 More and even better tools to facilitate this workflow are expected in the future. Combining citation searching via citation indexes with automated deduplication (free online tools available) 31 32 33 makes this recommendation feasible. A caveat is that a search in one citation index will in most cases fail to retrieve all the cited references. 34 35 Thus, references to some documents (such as websites, registry entries, or grey literature) that are less likely to be indexed in databases might only be retrievable by checking reference lists or only in some citation indexes. 3

Recommendation 6

Rationale and explanation supporting recommendation 6.

A single citation index or citation analysis tool might not cover all seed references and is likely to not find all the citing and cited literature. Citation indexes do not offer 100% coverage because some references are currently not indexed in one or several citation index(es) 36 and because of data quality problems. 37 Evidence indicates that when using more than one citation index for citation searching, the results of the different indexes can complement each other. 38 39 40 Thus, retrieval of backward and forward citation searching results from more than one citation index or citation analysis tool (eg, Lens.org via citationchaser, Scopus, citation indexes in Web of Science) followed by deduplication (see recommendation 7) can increase the sensitivity of citation searching. It is similar to the complementary effect of using multiple electronic databases for the primary database search, which is the preferred method in systematic search workflows. 4 In recent years, online citation searching options have increased and many open access tools make rapid electronic citation searching universally accessible. 27 28 29 30

Recommendation 7

Rationale and explanation supporting recommendation 7.

The concept of citation searching as a supplementary search method relies on the notion that reference lists and cited-by lists of eligible references are topically related to these references. 6 This topical relation implies a considerable degree of overlap within these lists leading to several duplicates. Furthermore, the overlap likely also extends to the results of the primary database search that was performed on the same topic. Based on these considerations and on the fact that the results of the primary database search have already been screened for eligibility, the screening load of citation searching results can be substantially cut by removing those references that have already been screened for eligibility (deduplication against the primary database search) and those references that appear as duplicates during citation searching. 34 Depending on the method of deduplication, this procedure can be done in one go.

While deduplication can be conducted manually, standard bibliographic management software and specialised tools currently provide automated deduplication solutions, allowing for easier and faster processing. 34 41 42 If citation searching leads to only very few results, omission of the deduplication step can be considered to save time and administrative effort.

Recommendation 8

Rationale and explanation supporting recommendation 8.

Citation searching methods can be conducted over one or more iterations, a process that we refer to as iterative citation searching. 43 The first iteration is based on the original seed references (see recommendation 4). If eligibility screening of the results of this first iteration leads to the inclusion of further eligible records, these records serve as new seed references for the second iteration, and so forth. Evidence indicates that conducting iterative citation searching can contribute to the identification of more eligible records. 6 43 44 45

Iterations beyond the first round of citation searching require additional time and effort and could interrupt the ongoing review process, so the decision in favour of or against further iterations should be guided by an informal cost-benefit assessment. Relevant factors to be assessed include the review topic (difficult or easier to search for), sensitivity of the primary search, aim for completeness of the literature search, and the estimated potential benefit of the iteration(s) (eg, based on the number or percentage of included records found with the previous citation searching iteration).

Review authors should report the number of iterations and possibly the reason for stopping if the last iteration still retrieved additional eligible records. Furthermore, stating “citation searching was done on all included records” can lead to confusion. Most authors might mean all records were included after full text screening of the primary search results. But strictly speaking, “all included records” also includes those records retrieved via citation searching. The second interpretation implies that iterative citation searching is required until the last iteration leads to no further identification of eligible records.

As outlined in the rationale of recommendation 7, results of citation searching iterations can be deduplicated against all previously retrieved records to reduce the screening load.

Recommendation 9

Rationale and explanation supporting recommendation 9.

We refer to standalone citation searching when any form of citation searching is used as the primary search method without extensive prior database searching. 6 This is contrary to citation searching as a supplementary search method to a primary database search. Seed references for standalone citation searching could, for example, be records from researchers’ personal collections or retrieved from less sensitive literature searches. Standalone citation searching can be based on a broad set of seed references. It can comprise backward and forward citation searching as well as indirect methods that collect co-citing and co-cited references.

When study authors have replicated published systematic reviews with standalone citation searching, they have mostly missed literature that was included in the systematic review. 27 46 47 48 Since search methods for systematic reviews and scoping reviews should aim at completeness of recall, standalone citation searching is not a suitable method for these types of literature review.

Recommendation 10

Rationale and explanation supporting recommendation 10.

Relevant guidance for researchers conducting citation searching in systematic literature searching can be found in item 5 of PRISMA-S. 8 Accordingly, required reporting items are the directionality of citation searching (examination of cited or citing references), methods and resources used for citation searching (bibliographies in full text articles or citation indexes), and the seed references that citation searching was performed on. 8 Additional information for the reporting of citation searching can be found in PRISMA-S items 1 (database name), 13 (dates of searches), and 16 (deduplication). 8 Although PRISMA-S can be seen as the minimum reporting standard for citation searching as a supplementary search technique, other important elements that emerged from our scoping review 6 need to be reported to achieve full transparency or reproducibility. These elements are listed in recommendation 10 as a supplement to PRISMA-S to comprehensively guide the reporting of supplementary citation searching in systematic literature searching.

Concerning reporting of citation searching results in the PRISMA 2020 flow diagram, 49 two variants are possible: reporting only those records that are additional to the primary search results after deduplication, or reporting all retrieved records followed by insertion of an additional box where the number of deduplicated records is reported.

Researchers should be aware that the detail of the citation searching methods do not have to be reported in the main methods of a study. Detailed search information can be provided in an appendix or an online public data repository.

Examples of good reporting

“As supplementary search methods, we performed . . . direct forward and backward CT [citation searching] of included studies and pertinent review articles that were flagged during the screening of search results (on February 10, 2021). For forward CT, we used Scopus, Web of Science [core collection as provided by the University of Basel; Editions = SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC], and Google Scholar. For backward CT, we used Scopus and, if seed references were not indexed in Scopus, we manually extracted the seed references’ reference list. We iteratively repeated forward and backward CT on newly identified eligible references until no further eligible references or pertinent reviews could be identified (three iterations; the last iteration on May 5, 2021).” 6

“To supplement the database searches, we performed a forward (citing) and backwards (cited) citation analysis on 2 August 2022 using SpiderCite ( https://sr-accelerator.com/#/spidercite ).” 50

“Reference lists of any included studies and retrieved relevant SRs [systematic reviews] published in the last five years were checked for any eligible studies that might have been missed by the database searches.” 51

Research priority 1

Rationale and explanation supporting research priority 1.

Indirect citation searching involves the collection and screening for eligibility of records that share references in their bibliography or citations with one of the seed references (ie, co-citing or co-cited references). 10 Indirect citation searching typically retrieves a large volume of records to be screened. 46 48 Therefore, prioritisation algorithms for the screening of records and cut-off thresholds that might discriminate between potentially relevant and non-relevant records have been proposed with the aim to reduce the workload of eligibility screening. 27 47 The methodological studies that have pioneered indirect citation searching methods for health related topics have so far exclusively focused on standalone citation searching. 6 It is currently unclear whether the added workload and resources for searching and screening warrant indirect citation searching methods as supplementary search techniques in systematic reviews of any type (qualitative or quantitative studies, difficult or easier topics to search for).

Research priority 2

Research priority 3, research priority 4, tarcis recommendations and research priorities.

In keeping with our study aims, the TARCiS recommendations cover three aspects of citation searching in the context of systematic literature searches. They offer guidance regarding when to conduct, how to conduct, and how to report citation searching. The strength of each recommendation reflects the panellists’ assessment of the strength of evidence to support them.

In systematic evidence syntheses, citation searching techniques can be used to fill gaps in the results of primary database searches, but their application is not universally indicated. TARCiS recommendations 2 and 3 provide critical assistance in cost-benefit considerations (ie, whether a systematic search is likely to benefit from the use of citation searching). Systematic searchers of defined pharmaceutical interventions, for instance, might take from this guidance that they can skip citation searching because their primary database search might already allow for high sensitivity at reasonable specificity and expedite other supplementary search techniques, such as clinical trial registry searching. 52 Accordingly, TARCiS does not recommend the use of citation searching in easier-to-search-for topics, but—as formulated in research priority 2—more research is needed to more reliably discriminate between topics that are easier to search for and those that are difficult to search for.

TARCiS recommendations 4-8 comprise guidance for technical aspects of citation searching. This guidance includes the selection of seed references, use of electronic citation indexes, deduplication, and iterative citation searching. While composing these recommendations, the TARCiS study group has considered that individual workflows must be framed in line with institutional licenses for subscription only databases and software. For illustration, one such workflow that is based on the licenses as provided by the University of Basel was deposited as an online video. 53

Concerning guidance for reporting of citation searching, we developed a consensus terminology set for citation searching methods (TARCiS recommendation 1) as well as a recommendation for preferred reporting items for citation searching (TARCiS recommendation 10), along with a downloadable checklist. 17 TARCiS recommendation 10 increases the reporting standards provided by PRISMA-S 8 by dealing with the reporting of citation searching iterations, software tools that facilitate citation searching via a citation index, and the method of eligibility screening. Furthermore, TARCiS recommendation 10 standardises the reporting of citation searching results in the PRISMA 2020 flow diagram. We suggest that systematic reviewers, methodologists, journal reviewers, and editors use the TARCiS statement terminology and reporting checklist 17 (appendix 3) as an additional checklist until future work by the PRISMA-S study group produces an updated reporting guideline that renders the TARCiS checklist obsolete.

Dissemination

TARCiS is intended to be used by researchers, systematic reviewers, information specialists, librarians, editors, peer reviewers, and others who are conducting citation searching or assessing citation searching methods. To enhance dissemination among these stakeholders, we aim to provide additional open access publications in scientific and non-scientific journals relevant in the field of information retrieval and evidence syntheses.

We have launched a TARCiS website ( https://tarcis.unibas.ch/ ) and plan to disseminate the TARCiS terminology and reporting checklist 17 on various platforms, including EQUATOR. We aim to make the TARCiS statement available via the Library of Guidance for Health Scientists (LIGHTS), a living database for methods guidance 54 ; the Systematic Review Toolbox, an online catalogue of tools for evidence syntheses 55 ; and ResearchGate, a social scientific network to share and discuss publications.

We will also share the TARCiS terminology and reporting checklist 17 with editors of journals relevant in the field of information retrieval and evidence syntheses to request for inclusion in their instructions for authors and raise awareness of this topic. We hope that this effort will guide authors and peer reviewers to use TARCiS and assist their conduct, reporting, and evaluation of citation searching. We will also share the TARCiS statement with primary teaching stakeholders in evidence syntheses and systematic literature searching (eg, York Health Economics Consortium, RefHunter, Cochrane, Joanna Briggs Institute, and the Campbell Collaboration) and suggest its inclusion in future editions of their handbooks. We will present and discuss the TARCiS statement on international conferences and share our publications and presentations via relevant mailing lists and newsletters, X (formerly Twitter), and LinkedIn.

Limitations

A limitation of the TARCiS statement is that, despite the expectation and intent to recruit panellists from all parts of the world, their locations were limited to Australia, Europe, and North America. In addition, only a few panellists were recruited from countries where English was not the dominant language. Furthermore, both the evidence collected in our scoping review and the participating panellists are primarily involved with health related research. These considerations might reduce the generalisability of our recommendations and research priorities in other countries, languages, and research areas.

Conclusions

TARCiS comprises 10 specific recommendations on when and how to conduct citation searching and how to report it in the context of systematic literature searches. Furthermore, TARCiS frames four research priorities. It will contribute to a unified terminology, systematic application, and transparent reporting of citation searching and support researchers, systematic reviewers, information specialists, librarians, editors, peer reviewers, and others who are conducting or assessing citation searching methods. In addition, TARCiS might inform future methodological research on the topic. We encourage systematic review teams to incorporate TARCiS into their standardised workflows.

Ethics statements

Ethical approval.

This study is based on published information and uses surveys of topical experts and therefore did not fall under the regulations of the Swiss Human Research Act, and we did not need to apply for ethical approval according to Swiss law. Data protection and privacy issues for the survey are outlined in the main text.

Data availability statement

The survey sheets and questionnaires used for this study are included in the supplementary content. Data generated and analysed during this study (except for sociodemographic information) are available on the Open Science Framework ( https://osf.io/y7kh3 ).

Acknowledgments

We would like to dedicate this work to Cecile Janssens, who died soon after agreeing to join our Delphi panel. We thank Jill Hayden (Dalhousie University) and Claire Duddy (UK) for participating in our Delphi panel; and Christian Buhtz (Martin Luther University Halle-Wittenberg), Jasmin Eppel-Meichlinger (Karl Landsteiner University of Health Sciences), Tania Rivero (University of Berne), and Monika Wechsler (University of Basel) for participating in the pretest of the Delphi survey.

TARCiS study group: Alison Avenell (University of Aberdeen, UK), Alison Bethel (University of Exeter, UK), Andrew Booth (University of Sheffield, UK; and University of Limerick, Ireland), Christopher Carroll (University of Sheffield, UK), Justin Clark (Bond University, Australia), Julie Glanville (Glanville.info, UK ), Su Golder (University of York, UK), Elke Hausner (Institute for Quality and Efficiency in Health Care, Germany), Tanya Horsley (Royal College of Physicians and Surgeons of Canada, Canada), David Kaunelis (Canadian Agency for Drugs and Technologies in Health, Canada), Shona Kirtley (University of Oxford, UK), Irma Klerings (Donau University, Austria), Jonathan Koffel (USA), Paul Levay (National Institute for Health and Care Excellence, UK), Kathrine McCain (Drexel University, USA), Maria-Inti Metzendorf (Heinrich-Heine University Duesseldorf, Germany), David Moher (University of Ottawa, Canada), Linda Murphy (University of California at Irvine, USA), Melissa Rethlefsen (University of New Mexico, USA), Amy Riegelman (University of Minnesota, USA), Morwenna Rogers (University of Exeter, UK), Margaret J Sampson (Children’s Hospital of Eastern Ontario, Canada), Jodi Schneider (University of Illinois at Urbana-Champaign, USA), Terena Solomons (Curtin University, Australia), Alison Weightman (Cardiff University, UK)

Contributors: All authors made substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; drafted the article or revised it critically for important intellectual content; and approved the final version to be published. JH, TN, TF, HE, and CA-H had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. JH, TN, TF, HE, and CA-H contributed to the study concept and methodology; acquisition, analysis, interpretation, validation, and visualisation of data; and critical revision of the manuscript for important intellectual content. JH, HE, and CA-H conducted the statistical analysis. JH and CA-H drafted the manuscript; provided administrative, technical, and material support; and supervised the study. The TARCiS study group authors are the Delphi panellists who were involved in Delphi rounds 1-4; they received the final manuscript draft for critical revision, important intellectual input, and approval for publication. CA-H is the guarantor for the study. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: The authors did not receive a specific grant for this study.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no specific support for the submitted work. CA-H received payments to his institution for a citation searching workshop by the University of Applied Sciences Northwestern Switzerland. JH received consulting fees from Medical University Brandenburg and payments for lecturing from the University of Applied Sciences Northwestern Switzerland, Catholic University of Applied Sciences, and Netzwerk Fachbibliotheken Gesundheit. From the TARCiS study group: JS received support from Alfred P Sloan Foundation; was funded by the US National Institutes of Health, National Science Foundation, US Office of Research Integrity, United States Institute of Museum and Library Services, and University of Illinois Urbana-Champaign; received book royalties from Morgan and Claypool; received consulting fees or honorariums from the European Commission, Jump ARCHES, NSF, and the Medical Library Association; received travel support by UIUC; contributes to the CREC (Communication of Retractions, Removals, and Expressions of Concern) Working Group; has non-financial associations with Crossref, COPE (Committee on Publication Ethics), the International Association of Scientific, Technical and Medical Publishers, the National Information Standards Organisation, and the Center for Scientific Integrity (parent organisation of Retraction Watch); and declares the National Information Standards Organisation as a subawardee on her Alfred P Sloan Foundation grant G-2022-19409. JG received payments for lecturing by York Health Economics Consortium. MJS received consulting fees at the Canadian Agency for Drugs and Technologies in Health and National Academy of Medicine (formerly Institute of Medicine) and for lecturing and support for attending a meeting at Institute for Quality and Efficiency in Health Care; and has a leadership role as secretary of the Ottawa Valley Health Library Association. AW received payments to her institution for a citation analysis workshop run via York Health Economics Consortium. SK declares non-financial interests as a member of the UK EQUATOR Centre and a coauthor of the PRISMA-S reporting guideline and was funded by Cancer Research UK (grant C49297/A27294); the current work was unrelated to this funding. PL is an employee of the National Institute for Health and Care Excellence. MR received payments by the Medical Library Association and declares non-financial interests as a member of the PhD programme affiliated with BMJ Publishing Group. ABo is a co-convenor of the Cochrane Qualitative and Implementation Methods Group and has authored methodological guidance on literature searching. All the other authors have no competing interests to disclose.

Provenance and peer review: Not commissioned; externally peer reviewed.

• Varley-Campbell J ,
• Britten N ,
• ↵ Higgins J, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022): Cochrane; 2022. www.training.cochrane.org/handbook .
• Greenhalgh T ,
• Nordhausen T ,
• Appenzeller-Herzog C ,
• Bossuyt PM ,
• Rethlefsen ML ,
• Kirtley S ,
• Waffenschmidt S ,
• PRISMA-S Group
• Briscoe S ,
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. Internal DELPHI Protocol. 2023. https://osf.io/4nh25 .
• Radbruch L ,
• Brearley SG
• Pawlowski SD
• Schulz KF ,
• ↵ Leiner DJ. SoSci Survey (Version 3.1.06) [Computer software]. 2019. https://www.soscisurvey.de .
• ↵ Gmb HTX. Unipark: 2023. https://www.unipark.com/en .
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. TARCiS terminology and reporting item checklist. 2023. https://bit.ly/tarcispdf .
• McKenzie JE ,
• Preston L ,
• Carroll C ,
• Gardois P ,
• Paisley S ,
• Kaltenthaler E
• Lasda Bergman EM
• Haddaway NR ,
• Collins AM ,
• Coughlin D ,
• Livingston EH
• Horsley T ,
• Dingwall O ,
• Westphal A ,
• Kriston L ,
• Hölzel LP ,
• von Wolff A
• Janssens ACJW ,
• Brockman JE ,
• Grainger MJ ,
• Smalheiser NR ,
• Schneider J ,
• Torvik VI ,
• Fragnito DP ,
• Pallath A ,
• Ouzzani M ,
• Hammady H ,
• Fedorowicz Z ,
• Elmagarmid A
• ↵ Zotero Version 6. 2023. https://www.zotero.org:443 .
• Glasziou P ,
• Del Mar C ,
• Bannach-Brown A ,
• Stehlik P ,
• ↵ ClarivateAnalytics. Cited Reference Search. 2023. https://webofscience.help.clarivate.com/en-us/Content/cited-reference-search.htm .
• Falagas ME ,
• Pitsouni EI ,
• Malietzis GA ,
• Franceschini F ,
• Maisano D ,
• Mastrogiacomo L
• Ainsworth N ,
• Rodriguez-Lopez R
• Borissov N ,
• Lines RLJ ,
• Gucciardi DF ,
• ↵ Janssens AC. Updating systematic reviews and meta-analyses, the easy way: 2021. https://cecilejanssens.medium.com/updating-systematic-reviews-and-meta-analyses-the-easy-way-cbb2e23b48b9 .
• Kleijnen J ,
• Knipschild P
• Janssens AC ,
• Nightingale R ,
• Fautrel B ,
• Patterson J ,
• Hunter KE ,
• Webster AC ,
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. Citation searching in multiple citation indexes. 2023. https://osf.io/jaeu5 .
• Schönenberger CM ,
• ↵ Marshall C, Sutton A, O’Keefe H, et al. The Systematic Review Toolbox: 2022. http://www.systematicreviewtools.com/ .

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

• Reference Manager
• Simple TEXT file

People also looked at

Review article, modernizing and harmonizing regulatory data requirements for genetically modified crops—perspectives from a workshop.

• 1 Corteva™ Agriscience, Indianapolis, IN, United States
• 2 CropLife International, Arlington, VA, United States
• 3 BASF Corporation, Research Triangle Park, NC, United States
• 4 Corteva™ Agriscience, Johnston, IA, United States
• 5 Syngenta Seeds LLC, Research Triangle Park, NC, United States
• 6 Bayer Crop Science, Chesterfield, MO, United States

Genetically modified (GM) crops that have been engineered to express transgenes have been in commercial use since 1995 and are annually grown on 200 million hectares globally. These crops have provided documented benefits to food security, rural economies, and the environment, with no substantiated case of food, feed, or environmental harm attributable to cultivation or consumption. Despite this extensive history of advantages and safety, the level of regulatory scrutiny has continually increased, placing undue burdens on regulators, developers, and society, while reinforcing consumer distrust of the technology. CropLife International held a workshop at the 16th International Society of Biosafety Research (ISBR) Symposium to examine the scientific basis for modernizing global regulatory frameworks for GM crops. Participants represented a spectrum of global stakeholders, including academic researchers, GM crop developers, regulatory consultants, and regulators. Concurrently examining the considerations of food and feed safety, along with environmental safety, for GM crops, the workshop presented recommendations for a core set of data that should always be considered, and supplementary (i.e., conditional) data that would be warranted only on a case-by-case basis to address specific plausible hypotheses of harm. Then, using a case-study involving a hypothetical GM maize event expressing two familiar traits (insect protection and herbicide tolerance), participants were asked to consider these recommendations and discuss if any additional data might be warranted to support a science-based risk assessment or for regulatory decision-making. The discussions during the workshop highlighted that the set of data to address the food, feed, and environmental safety of the hypothetical GM maize, in relation to a conventional comparator, could be modernized compared to current global regulatory requirements. If these scientific approaches to modernize data packages for GM crop regulation were adopted globally, GM crops could be commercialized in a more timely manner, thereby enabling development of more diverse GM traits to benefit growers, consumers, and the environment.

1 Introduction

Genetically modified (GM) crops that have been engineered to express transgenes have been commercially cultivated since 1995 and are annually grown on 200 million hectares globally. These crops have delivered important societal benefits, such as increased crop yields, resilience to adverse growing conditions, reduced tillage leading to improved soil health, reduction in the need for crop protection inputs, preservation of natural resources, and improved rural economies ( Klümper and Qaim, 2014 ; Dively et al., 2018 ; Zilberman et al., 2018 ; Smyth, 2020 ; Ala-Kokko et al., 2021 ; Macall et al., 2021 ; Peshin et al., 2021 ; Brookes, 2022a ; Brookes, 2022b ; Brookes, 2022c ). These benefits have led to rapid adoption of GM technology for agricultural production, including 80% of global cotton and 73% of global soybean. One-third of global maize production includes GM traits for herbicide tolerance, insect protection, or both ( AgbioInvestor, 2023 ). GM traits have been introduced in other row crops such as oilseed rape, sugar beet, and alfalfa and, at a smaller scale, in specialty crops such as apples, eggplant, squash and potatoes ( ISAAA, 2020 ). Hundreds of studies have been conducted to assess the safety of GM crops, and there have been no substantiated cases of resulting harm to people or livestock that consume GM crops or to the environment in which they are grown ( European Commission, 2010 ; Snell et al., 2012 ; Van Eenennaam and Young, 2014 ; NASEM, 2016 ).

Despite this track record of safety and benefits, regulatory data requirements for approval and commercialization of GM crops have continued to grow globally. GM technology is primarily limited to major global crops, like maize and soybean, and to major input traits, such as insect protection and herbicide tolerance. While there are many efforts underway to use GM technology for other traits and to improve minor crops, especially for small holders in the developing world ( David, 2009 ; Shelton, 2021 ; Woodruff, 2024 ), securing the regulatory approvals to enable cultivation and avoid potential trade disruptions can present often insurmountable challenges to commercialization. Only a few large multinational developers can afford the US$115 million cost and also persist for the 16 years that it currently takes, on average, to bring a new trait to the global market. More than one-third of those costs, and more than one-half of that time, are taken by the regulatory process ( AgbioInvestor, 2022 ). These extensive and complex regulatory systems also mean that governments must invest significant resources in developing and maintaining regulatory bodies staffed with sufficient people and expertise, creating a burden on taxpayers and society. Countries that cannot afford such an investment are missing out on the benefits of GM crops.

CropLife International and its member companies that develop GM crops (BASF, Bayer Crop Science, Corteva™ Agriscience, and Syngenta) have proposed a modernized regulatory framework and streamlining of data requirements for GM crops that is based on scientific rationale and builds on the 25 years of experience with the technology, and the history of its safe use ( Mathesius et al., 2020 ; Anderson et al., 2021 ; Bachman et al., 2021 ; Brune et al., 2021 ; Goodwin et al., 2021 ; McClain et al., 2021 ; Roper et al., 2021 ; Waters et al., 2021 ). The development of the proposed framework was motivated and guided by considering four key questions. 1) Are today’s regulations for GM crop approvals risk-proportionate? 2) Do today’s data requirements act as an unnecessary barrier to beneficial innovation? 3) How can knowledge and experience accumulated over the last 25 years inform modernization of regulations? 4) Can data requirements be streamlined and harmonized across countries and authorities? These questions were used to guide the determination of the types of data that are necessary to ensure GM crops are developed and deployed without increased risks for food and feed safety or the environment compared to conventional crops. Under this framework, core data, which are important for the problem formulation step of the risk assessment of the GM crop, were identified. The core data are used for problem formulation to identify plausible cause-and-effect hypotheses of harm from the GM crop. Depending upon the outcome of the problem formulation for a specific crop by trait combination, additional supplementary (i.e., conditional) studies may be needed, on a case-by-case basis, to analyze any plausible risk identified. Figure 1A outlines proposed core and supplemental studies for a Food and Feed Safety Assessment; Figure 1B outlines proposed core and supplemental studies for an Environmental Risk Assessment. CropLife International took an approach that is consistent with principles of risk assessment such that the proposed data requirements can fully inform decision-making by a regulatory agency, without the extraneous data present in many current regulatory submissions that does not meaningfully contribute to the risk assessment of the GM crop.

Figure 1 . (A) proposes a set of data recommended for a science-based food and feed safety assessment for a typical GM crop and considers as core studies: basic molecular characterization, protein characterization and expression, and protein safety (i.e., history of safe use of the protein and source organism and bioinformatics to identify potential toxins and allergens). The outcomes of these core data are used to inform the problem formulation step and decide, on a case-by-case basis which, if any, supplementary studies are needed to make a conclusion on safety ( Brune et al., 2021 ; Waters et al., 2021 ). (A) is adapted from Brune et al., 2021 and Waters et al., 2021 . (B) proposes a set of data recommended for a science-based environmental risk assessment for a typical GM crop and considers as data: understanding the receiving environment and the basic biology of the unmodified plant; assessing the agronomic similarity of the GM crop to its conventional counterparts (i.e., agronomic comparative assessment); and understanding the intended trait of the GM plant and assessment of how the intended trait may lead to environmental harm. The core data should be used first to inform the problem formulation. If a conclusion cannot be made about the pathway to harm using the core data, additional case-by-case hypothesis-driven supplementary studies should be considered ( Anderson et al., 2021 ).

To further examine whether CropLife International’s proposed modernized data requirements are sufficient for food and feed safety assessments and for environmental risk assessments, a workshop was held at the 16th International Society of Biosafety Research (ISBR) Symposium (St. Louis, USA) in 2023. Using a case study of a hypothetical GM maize event containing two familiar transgenic traits (herbicide resistance and insect protection). The workshop participants were charged with considering whether the proposed data in the case study are scientifically both necessary and sufficient to determine the food, feed and environmental safety of the hypothetical GM crop.: CropLife International member representatives that served as moderators during the workshop authored this publication to report the outcomes and summarize the discussions that took place among the participants. The participants varied in their backgrounds and prior experience with risk assessment and included individuals from regulatory agencies, technology developers, consultant groups, and academia. A wide range of geographical areas were represented.

2 Case study description

For the case study, a hypothetical GM maize event was presented to the workshop participants for evaluation. The hypothetical event was intentionally simple for this exercise (i.e., a familiar crop with traits that are similar to many transgenic events that have already been reviewed and approved by regulatory agencies globally, with several in commercial production for many years), which enabled the participants to analyze in greater depth the need for data that is routinely submitted but may not contribute to the safety assessment. More specifically, a maize ( Zea mays ) event containing a single insertion encoding for two proteins from a single T-DNA introduced using standard disarmed Agrobacterium tumefaciens-based transformation was described. The two hypothetical traits provide protection against lepidopteran pests and tolerance to treatment with glyphosate herbicide, using a hypothetical Cry1 protein from Bacillus thuringiensis ( Bt ) and a hypothetical EPSPS protein variant isolated from maize, respectively. The workshop participants were asked to separately consider a food and feed safety assessment or an environmental risk assessment for this same hypothetical GM maize event. Additional distinctions between the presentation of the case study for the different assessments are outlined below.

2.1 Food and feed safety assessment

For the Food and Feed Safety Assessment, the results from hypothetical evaluations of core data on the characterization and safety assessment of the event were provided (summarized in Table 1 ). Throughout this paper, the term ‘data’ refers to both the results of experiments or studies as well as information gathered from literature reviews, consensus documents and other similar sources. As described in Waters et al. (2021) , the core data for a food and feed safety assessment are: 1) molecular characterization, 2) protein characterization, and 3) protein safety (allergenicity and toxicity). The results of the molecular characterization demonstrated that there was an insertion of a single T-DNA sequence into the maize genomic DNA without any vector backbone sequences. There were no changes in the intended protein coding sequence and constitutive expression of both proteins were driven by familiar promoter elements (35S from cauliflower mosaic virus and ubiquitin promoter from Zea mays , respectively). Finally, the inserted DNA and the traits were indicated as being stable over three generations. The protein characterization data given to participants indicated that the molecular weight and amino acid sequence were as expected for both proteins. The function of the hypothetical Cry1 protein was established as having activity limited to target lepidopteran pest species, with no activity against other insect orders. Field tolerance to glyphosate from the hypothetical EPSPS protein variant was also as expected. The protein safety data indicated that both proteins are similar to proteins that have a history of safe use for food and feed; neither EPSPS proteins nor Cry proteins have any known toxicity or allergenicity concerns. Bioinformatics analysis comparing the amino acid sequences of both hypothetical proteins to a protein database also demonstrated that neither protein is related to any protein of toxicological concern nor related to any allergens in the qualified allergen database.

Table 1 . Summary of food and feed safety assessment core data of the hypothetical GM maize.

A familiar crop with familiar traits and minimal genetic disruptions was used for the workshop to promote discussion of what data is really needed to establish the food and feed safety of a GM crop event. It was also noted to workshop participants that extensive protein expression data in the plant was not obtained, nor was detailed proximate or nutrient composition data included. Further, while it was established that bioinformatics confirmed no homology to known allergens or toxins, no exposure assessments, no animal feeding studies, or other more direct assessments of potential for harm from the hypothetical event were included. As presented, the case study stated that considering 1) the assessment from the core data, 2) the familiarity of the crop and traits, and 3) the lack of direct interaction with other metabolic pathways of the plant, there was no hypothesis of food and/or feed safety risks for the new GM maize crop, and therefore additional supplementary data are not warranted to establish food and feed safety, in accordance with the approach established in Brune et al. (2021) , McClain et al. (2021) and Roper et al. (2021) .

2.2 Environmental risk assessment

For Environmental Risk Assessment (ERA), the intention of the case study was to model how problem formulation and core data should be leveraged to inform ERA of a GM crop for cultivation safety. Problem formulation is a process used in the ERA to develop plausible pathways to harm resulting from cultivation of the GM crop. Problem formulation first considers core data, then considers other data on a case-by-case basis if it is deemed necessary to inform the risk assessment. For ERA, core data includes information related to the receiving environment, description of basic biology of the unmodified plant, assessment of the agronomic similarity of the GM crop to its conventional counterparts, and characterization of the intended traits of the GM crop (summarized in Table 2 ). For the purpose of the case study, the protection goal was broadly stated as protection of biodiversity, specifically protection of beneficial or charismatic species. For the purposes of the workshop, the core characteristics of the event as described for the food and feed assessment were considered the same (e.g., molecular features), with additional information focused on agronomic and environmental aspects provided to guide the ERA discussion.

Table 2 . Summary of environmental risk assessment core data.

The participants were presented with the following set of core data (summarized in Table 2 ) and were asked to consider if a plausible pathway to harm could be developed related to weediness, invasiveness, gene flow to wild relatives or hazard to non-target organisms: 1) assessment of the receiving environment indicating no wild relatives of maize present in the cultivation country and no changes to the standard agronomic practices relative to non-modified maize; 2) assessment of the basic biology of maize, using consensus documents, demonstrating non-modified maize has no weediness characteristics and requires human intervention for propagation and survival; 3) multilocation field trial data demonstrating hypothetical maize was agronomically similar to non-modified maize; and 4) assessment of the intended phenotype (i.e., insect protection and herbicide tolerant traits are not intended to increase fitness or survival in the environment).

Based on the core data assessed, the case study proposed that there are no plausible hypotheses for how cultivation of the hypothetical maize could result in environmental harm related to weediness, invasiveness, and gene flow to wild relatives. Thus, additional data will not further contribute to meaningful assessment of environmental safety. However, the case study proposed that a plausible pathway to harm to non-target organisms could be developed based on the intended insect protection phenotype. The hypothetical Cry1 protein was presented as providing protection against specific lepidopteran insect pests (European corn borer, Asian corn borer, Southwestern corn borer, corn earworm, and fall armyworm).

The mode of action of Cry proteins in GM crops is well-documented ( Bravo et al., 2007 ; OECD, 2007 ). In this case study, additional supplemental protein expression data and non-target organism hazard data were provided to the participants, and they were asked to consider if additional plausible pathways to harm could be developed. The set of supplemental data (summarized in Table 3 ) was as follows: 1) multilocation field trial data measuring the concentration of the hypothetical Cry1 protein in several plant tissues to inform exposure assessment; 2) an exposure assessment for different non-target organisms to consider the likelihood and magnitude of exposure to the hypothetical Cry1 protein; and 3) results of non-target organism Tier I hazard studies for several surrogate species representing different taxonomic orders (e.g., ladybird beetle, a soil dwelling organism, and a non-target predator) conducted with the Cry1 protein in the diet.

Table 3 . Summary of environmental risk assessment supplementary data.

The multilocation field trial data showed that the Cry1 protein was only detectable (above the limit of detection) in the leaf and whole plant, with the highest concentration found in R1 leaf. The protein was below the limit of detection of the analytical assay in pollen and root. Based on the tissue expression, the exposure assessment concluded that since there is no expression of the Cry1 protein in pollen, there would be no route of exposure to non-target pollen feeding organisms (e.g., honeybee). Finally, the Tier I hazard studies indicated that no hazard was observed at concentrations that exceeded >10x the expected environmental concentration.

Usually, the assessment of adverse effects in non-target organisms follows a tiered approach that starts with laboratory studies at levels that exceed worst-case exposure conditions ( Romeis et al., 2011 ). Tier I laboratory studies with non-target organisms are typically conducted using at least 10X the worst-case expected environmental concentration. In this case, the results of the hypothetical Tier I dietary studies indicated no hazard (i.e., adverse effects) at concentrations that exceeded 10x the worst-case expected environmental concentration, and thus a conclusion that evidence is sufficient without conducting additional hazard testing was indicated. Based on data from the exposure assessment and non-target hazard assessment studies, the case study proposed that there were no plausible pathways to harm to non-target organisms due to lack of exposure and/or lack of risk because there were no adverse effects at concentrations that exceeded 10X the worst-case expected environmental concentration. Participants were asked to consider whether they agreed with the conclusions proposed by the case study based on core data and additional supplementary data related to protein expression, non-target organism exposure, and non-target organism hazard.

Additional information such as molecular data to confirm that the insert is an intact single copy, stable across generations, and that there is no insertion of DNA from the plasmid backbone were not provided in the ERA case study. These additional data for product characterization have historically been submitted to regulators as part of cultivation applications, but they are not directly relevant to ERA ( Anderson et al., 2021 ).

3 Learnings from breakout group discussions

After participants attended the introductory presentation session of the workshop, they were distributed into smaller discussion groups of approximately 10 people, with CropLife International member representatives serving as moderators. Each participant had the opportunity to choose either the Food and Feed Safety Assessment or the Environmental Risk Assessment, depending on their respective areas of interest.

The goal of the smaller group discussion sessions was to allow participants to go into deeper conversations about the proposed modernized paradigm for a risk assessment of a GM crop. Discussions were aided by a distribution of a printed booklet that included a description of the hypothetical GM maize event and the data collected, and that outlined the key concepts of using the core data for a Food and Feed Safety Assessment and Environmental Risk Assessment. Moderators provided some time for the participants to review the information and then introduced the case study by giving a brief overview of the information provided in each data section of the case study. Participants were encouraged to provide feedback and to bring up questions and/or comments about topics/elements of the case study that they considered not sufficiently covered by the data provided. They were also asked to complete a worksheet allowing for comments on the specific steps of the assessment process.

Discussions during this small group session were productive and highly informative. Overall, the participants were engaged, willing to discuss, and mostly supportive of the general assessment framework of primarily using core data and only using further assessments on a case-by-case basis.

A summary of key points from the breakout group discussions is shared below. This section is not intended to be a complete summary of the discussion, rather the authors have captured points of interest with an emphasis on points that are worth considering for future workshops and discussions on this topic.

3.1 Food and feed safety assessment

In the small group session, participants were asked to consider 1) the assessment from the core studies (see Table 1 ), 2) the familiarity of the crop and traits, and 3) the lack of direct interaction with other metabolic pathways of the plant, and then decide whether there was a hypothesis of food and/or feed safety risks for the new GM maize crop. Because of these considerations, the position for the case-study was that, for the hypothetical event, additional supplemental studies are not warranted to establish food and feed safety, and the participants discussed whether they agreed with this position.

Below are some key feedback and questions captured during the workshop regarding the proposed approach for the assessment of Food and Feed Safety of the hypothetical GM maize event.

3.1.1 Molecular characterization (transformation method, transformation construct, DNA insert characterization)

Overall, the participants agreed that the proposed molecular characterization core data is aligned with what is currently provided and that the information was sufficient to inform a food and feed safety assessment. One potential exception to the core data package that was discussed is data demonstrating that the insert is stable over at least three generations. The participants suggested that this study could be considered as supplemental, and not necessarily required as part of the core data package, if the insert is demonstrated to be inserted into the chromosome and is not interrupting endogenous genes or regulatory elements, and there is no other reason to expect that the insert might be unstable (e.g., insertion site near a transposon). There was some discussion that three generations of data may not be considered enough by all regulatory agencies and that additional generations could be required for polyploid crop species. Additionally, participants raised questions about Agrobacterium transformation not being targeted and discussed providing data on whether any internal genes were modified. It was also noted by workshop participants that the use of Next-Generation Sequencing (NGS) to characterize the insert is not yet accepted by all regulatory agencies, but also there was recognition of the utility of NGS to provide a more comprehensive characterization of the insert and the insertion site compared to traditional methods (e.g., Southern blots).

3.1.2 Protein characterization (molecular weight, protein sequence confirmation, protein function)

Participants agreed that the protein characterization information was sufficient to inform the food and feed safety assessment, with some discussions around whether a registrant would always be able to provide what is required, as some proteins may be more challenging to characterize (e.g., difficulties in isolating the proteins in an active form, generating specific antibodies, or generating SDS-PAGE and Western blot data). A question was also raised on maize codon optimization and if the protein would still be considered the same as the native version. Future workshops can reinforce that maize codon optimization of the GM trait gene does not alter the trait protein sequence. Thus, it should not change the safety profile of the protein if there is no change to the amino acid sequence. Discussion also occurred regarding familiarity with promoters and the relationship to expression levels. The participants discussed if there might be a need to better understand the protein expression levels for unfamiliar promoters and also if increased expression levels might raise a concern of potentially increased allergenicity risk.

3.1.3 Protein safety/toxicology/allergenicity (background, source, history of safe use, bioinformatics)

Participants agreed that the EPSPS protein information for safety was sufficient to inform the food and feed risk assessments, but questions were raised about Cry proteins around digestibility and heat stability. There was also discussion regarding how similar a protein would need to be to a known protein to be considered familiar. Additionally, concerns were raised in the small group discussion on the limited protein expression data provided in the case study as it related to an exposure assessment. In response, the moderators noted that an exposure assessment is not necessary, because no hazard was identified from the proteins. However, when a hazard is identified, then protein expression levels are needed to enable assessment of potential exposure ( Brune et al., 2021 ).

3.1.4 Additional information needed to determine event safety

It was stated by one participant that if there was a disruption of a native gene, then composition data could be requested. Discussion also occurred regarding the concept of History of Safe Use (HOSU), and the amount of data, time and similarity (e.g., consideration of minor protein sequence differences) needed to establish something as having sufficient familiarity to be considered safe without additional data. One participant suggested that protein sequence data would be needed to demonstrate a HOSU and could be useful in determining the activity of the protein.

3.1.5 General feedback for food and feed safety assessment

Although participants generally agreed that the case study with a familiar crop and familiar traits is a good starting point for the discussions, several suggestions were made for further discussions to also provide a case study on an unfamiliar event or protein, to lay out how each study informs the safety assessment, to provide more on the problem formulation process, and to provide more graphics and to use examples. Discussion also occurred around the challenges of communicating and making changes to the currently provided data in regulatory applications. On this topic, proposals from participants included suggestions to emphasize more the end goal of getting needed products on the market sooner with less regulatory burden for all stakeholders and to publish more data prior to submission of the application in the scientific literature, and to be ready to provide additional data upon request.

3.2 Environmental risk assessment

After introducing the case study, the CropLife International moderator described a list (provided with the case study) of the specific potential pathways to harm that are relevant to the cultivation of the hypothetical maize event. Additionally, an explanation for how the core data can be used to sufficiently assess environmental risk was provided. For plausible pathways to harm that may not be sufficiently addressed by the core data (i.e., potential harm to non-target organisms), another list of potential pathways to harm that are specific to non-target organism (NTO) exposure was also presented.

Below are some key feedback and questions captured during the workshop regarding the proposed approach for the Environmental Risk Assessment of the hypothetical GM maize event.

3.2.1 Weediness potential

There was an overall consensus among the workshop groups that weediness can be adequately assessed using only core data. Participants agreed that there is not a plausible pathway to harm in the case study since maize is highly domesticated and volunteers will not survive without human intervention and management. One group discussed questions around the potential for dormancy, which may be a weediness trait, and whether it can be assessed in the core data (multilocation field trial; Table 2 ). It was concluded within the groups that the similarity in agronomic characteristics between the GM maize event and the non-GM maize in the case study core data is sufficient to show that there is a highly unlikely risk of weediness potential. This follows the principle of placing risk in the context of current practice (i.e., that the modified maize will have no greater risk than that of cultivation of the non-modified maize) ( Raybould and MacDonald, 2018 ). However, one workshop group had unresolved discussions on whether a difference in agronomic performance between different geographical regions may result in differences in the risk assessment and what specific agronomic elements are the most relevant to consider. Some participants in this group proposed scenarios in which the agronomic data generated in field trials performed outside of the cultivation country may not sufficiently represent the agronomic outcomes of field trials performed within the cultivation country.

3.2.2 Gene flow potential to wild relatives

There was general consensus that there is no environmental safety concern of gene flow in the case study based on the core data because there were no wild relatives present in the hypothetical cultivating environment. There was some interest from participants in further exploring how the risk assessment and data requirements will change if the cultivation environment did contain wild relatives. Also, there was some discussion on the threshold of relatedness between the GM maize and a wild relative species that constitutes a safety concern in terms of gene flow. Ultimately, there was additional consensus that product registrants should demonstrate that there are no wild relative species that are reproductively compatible with GM maize (regardless of species relatedness) to position that there is no gene flow concern. Alternatively, if there are wild relative species in the area of cultivation an assessment of the likelihood and consequences of trait introgression into the wild relative population may be warranted based on a problem formulation approach ( Anderson et al., 2021 ). Participants generally stressed the importance of citing published literature (e.g., accepted consensus references on crop-specific biology) as part of the core data to support the environmental risk assessment. Although it was acknowledged that gene flow will not likely occur between GM maize and wild relatives in the case study example, there was some discussion around whether gene flow may occur between the GM maize and adjacent local non-GM maize varieties and negatively impact crop integrity and biodiversity. The case study focused on assessing plausible pathways to harm related to gene flow between GM maize and sexually compatible weedy relatives. Future workshops can address concerns that were raised about coexistence of GM and non-GM cropping systems. Such a workshop may have to distinguish between environmental risks and market or socio-political concerns. For example, countries that have landrace populations for which the genetic make-up per se is a protection goal may have societal concerns about coexistence (for example, there could be changes the genetic identity of the landrace).

3.2.3 Plausible pathways to harm for non-target organisms (NTO)

All groups aligned that the only plausible pathway to harm from the case study that could not be sufficiently addressed with core data alone was the potential for harm to NTOs from potential exposure to the hypothetical Cry1 protein ( Table 2 ). Participants discussed the plausible pathways to harm that are specific to NTOs. There was general agreement that no additional data was needed to assess the potential for the EPSPS protein conferring the herbicide tolerance trait to cause harm to NTOs. However, participants acknowledged that public perception of herbicide tolerance traits could influence regulatory decisions and may need to be considered when determining the registrability of a GM crop. Such perceptions are not reflective of an actual risk, and the additional data generated do not inform the science-based risk assessment. For other pathways to harm, there was consensus that if there was either no hazard or no detectable exposure, then there is low risk to NTOs. For example, honeybees that may directly consume maize pollen and NTO lepidopterans that may indirectly consume maize pollen that drifts onto their host plants should have low risk in the ERA case study since the GM maize event has expression less than the limit of detection (LOD) of the insecticidal protein in pollen tissue ( Table 2 ). It was generally accepted by workshop participants that if expression of the insecticidal protein is <LOD in tissues that might be consumed by an NTO, further toxicity testing to determine hazard is not warranted.

Participants were also mostly aligned that aquatic environments generally experience minimal exposure to GM crop tissue and so additional toxicity testing is not needed for aquatic NTO species in most situations. However, some participants expressed uncertainty on whether this may be an issue if GM crops are cultivated very close to aquatic environments, which may affect exposure levels to NTO aquatic species. For NTO species where there is a plausible pathway to harm, all groups agreed that further data (exposure assessment or NTO Tier I laboratory testing) might be needed. Some discussions among participants regarding appropriate surrogate species to use for NTO testing and to what extent test species need to match those found in the cultivation regions were not resolved in the workshop. There was some additional discussion around the large body of scientific literature describing the surrogate species concept for testing Cry proteins and other types of plant incorporated protectants (e.g., Romeis, et al., 2011 ; Romeis et al., 2013 ; Bachman et al., 2021 ). While the terms “focal species” and “indicator species” were not discussed directly as part of the workshop, understanding protection goals and selecting appropriate surrogate species or indicator species to inform the science-based assessment of risk is an important consideration ( Rose, 2007 ; Roberts et al., 2020 ). Despite the lack of consensus on species selection, there was clear alignment among participants that NTO species representatives should only be tested if there is a valid hypothesis that there is a plausible pathway to harm for that specific organism type. For this reason, NTO studies should only be conducted when hypothesis-driven ( Figure 1B ).

3.2.4 General feedback and future considerations for ERA

Although participants agreed that a generic ERA case study is a good starting place, participants indicated that future workshops using a modified case study tailored for specific geographical regions will be even more helpful. As different countries have different sets of questions and concerns from local regulatory agencies, using more country-specific scenarios and less familiar pest-control traits in a case study may be more directly relevant in that region.

Related to gene flow, there was not a consensus about potential for harm in small team discussions. Future workshops would benefit from guided discussion to help develop problem formulation for gene flow. For example, it could be established as a baseline that for gene flow to occur naturally in the environment, and when assessing the potential for harm from gene flow between GM maize and local maize varieties, it should be compared to potential for harm from gene flow of non-GM maize and local maize varieties ( OECD, 2023 ). Furthermore, future workshops can reinforce that if gene flow to local maize varieties is a relevant concern for a specific cultivation country, then there is a large body of literature to leverage to assess if additional data is needed to inform the risk assessment (See OECD, 2023 Annex B for recent review) such a workshop would need to distinguish between the true environmental impact and concerns related to trade or economic issues.

Also, there were productive discussions on the topic of data transportability. Participants generally accepted the concept of transportability for lab study data. However, due to a lack of time for discussion, some unresolved questions remained regarding the transportability of field study data. Future workshops will benefit from guided discussion to help explain the principle of data transportability. An underlying principle of data transportability is that if no biologically relevant differences between a GM crop and its conventional counterparts are observed in one country or region, data from these studies can be used to inform the risk assessment in another country, regardless of agroclimatic zone ( Bachman et al., 2021 ). Following the recommendations for modernizing global regulatory frameworks for GM crops, additional agronomic data should only be collected in the local environment if there are plausible pathways for harm that cannot be fully informed by the core data.

Furthermore, there was some interest from participants in discussing how the proposed risk assessment paradigm might apply to combined GM products (i.e., breeding stacks), yield and stress traits (e.g., drought resistance), and streamlining of import registrations.

One topic that generated discussion across groups was the value of product characterization data in an environmental risk assessment. In the proposed modernized regulatory framework ( Anderson et al., 2021 ), underlying characterization data for the GM event are not regarded as core to the regulatory assessments (such as molecular data to confirm that the insert is an intact single copy, stable across generations, and that there is no plasmid backbone DNA). Although these data do not directly inform the ERA ( Anderson et al., 2021 ), it was discussed that an understanding of the characteristics of the GM product provides foundational information that enables the regulatory assessments to focus on the intended introduced trait during the problem formulation stage. Therefore, it was proposed to consider including, as part of the modernized ERA framework, a set of foundational information and data from the characterization of the GM event that confirms that (1) the intended gene sequence was inserted and functions as intended, as well as the number of such insertions; (2) the plants produce the intended newly expressed protein (NEP); (3) the intended phenotype is achieved.

4 Key considerations and takeaways from the workshop

The case study for the workshop considered a single event, albeit one that contained genetic material encoding for two proteins leading to two distinct traits (herbicide tolerance and insect protection). However, the majority of commercialized products contain multiple GM events that are combined through conventional breeding (also known as stacked trait products). The typical regulatory process first assesses all single events, before applying regulatory processes, if any, to the stacked trait products. In this sense, the case study used for the workshop reflected a realistic scenario in which regulators assess a single event regardless of whether the event will be commercialized as a single event or as a stacked trait product.

Regulatory processes for stacked trait products vary globally, with many countries recognizing the long, safe history of conventional breeding and not requiring additional assessment once all the single events are approved. It is the position of CropLife International that additional safety assessment of a stacked trait product produced by conventional breeding should not be required unless there is a plausible and testable hypothesis for interaction of the traits ( Goodwin et al., 2021 ). This case study did not address stacked trait products however, further iterations could include consideration of stacked trait products and how to evaluate possible interaction of traits.

The workshop was convened to explore the proposed modernized data requirements for regulatory assessments of GM crops ( Anderson et al., 2021 ; Waters et al., 2021 ). The participants were charged with considering whether currently implemented regulations for GM crops are risk-proportionate or whether they create an unwarranted barrier to the introduction of new traits. The organizers presented a position that knowledge and experience from 25 years of research and development could inform regulatory modernization and that streamlined data requirements could advance harmonization across countries and authorities.

Overall, considering the case study discussed, the participants at the workshop found the proposed modernized data requirements generally to be necessary and sufficient for decision making to support the safe commercial introduction of a new GM crop. There was a clear consensus that some of the current data requirements are no longer routinely warranted for familiar traits such as that discussed in the case study, given the track record of GM crops not presenting unexpected or unintended effects on food or feed safety or environmental risk relative to their conventional counterparts. Participants appreciated the benefit of harmonized hypothesis-based risk assessments to enable future deployment of GM crops that can address emerging agricultural challenges associated with increasing demand for affordable healthy food and changing agricultural environments. The points discussed in this publication will be used to further clarify recommendations for supplementary case-by-case data and guide the development of future, more targeted workshops and related discussions. In particular, applying the proposed framework to traits and crops with which there is less familiarity and established HOSU than those used in the case study may be associated with greater uncertainty in the foundational information of the GM event. Additional case studies involving less familiar traits and different crops should be used to further test the robustness of the modernized regulatory framework.

The workshop focused on what data was scientifically necessary and sufficient to make a conclusion on the food, feed and environmental safety of the GM crop. However, several participants noted that certain data not included in the case study was either required in their jurisdiction or routinely submitted by applicants. While it was beyond the scope of this workshop, future targeted workshops or symposia could address the extent to which regulatory authorities have the flexibility to decide, on a case-by-case basis, what data is necessary to make a conclusion on safety. In some jurisdictions the recommendations of the modernization project could be implemented by applicants by including a scientific rationale in their submission for why a specific study is not necessary. In other cases, changes to laws, regulations, or written guidance would be needed to implement these recommendations.

The case study for the first workshop, as described in this publication, was a valuable tool to foster discussion about science-based data requirements for the assessment of GM crops. If these scientific approaches to modernize data packages for GM crop regulation were adopted globally, delays to the commercialization of GM crops could be reduced, thereby allowing farmers access to new GM traits that will benefit not just growers, but consumers and the environment as well. For more information on the case study used in the workshop, or if there is interest in hosting a similar workshop, please contact the corresponding author.

Author contributions

NS: Conceptualization, Writing–original draft, Writing–review and editing. AS: Conceptualization, Writing–original draft, Writing–review and editing. JS: Writing–original draft, Writing–review and editing. JA: Writing–original draft, Writing–review and editing. MH: Writing–original draft, Writing–review and editing. DM: Writing–original draft, Writing–review and editing. CM: Writing–original draft, Writing–review and editing. MS: Writing–original draft, Writing–review and editing. SS: Writing–original draft, Writing–review and editing. EU-W: Writing–original draft, Writing–review and editing.

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. CropLife International supported the open access publication of this work. The funder was not involved in the writing of this article, or the decision to submit it for publication.

Conflict of interest

Authors NS, JA, and CM were employed by Corteva™ Agriscience. Author AS was employed by CropLife International. Authors JS and MS were employed by BASF Corporation. Authors MH and SS were employed by Syngenta Seeds LLC. Authors DM and E-UW were employed by Bayer Crop Science. BASF Corporation, Bayer Crop Science, Corteva™ Agriscience, and Syngenta Seeds LLC are commercial developers of GM crops.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

AgbioInvestor (2022). Time and cost to develop a new GM trait. Available at: https://agbioinvestor.com/wp-content/uploads/2022/05/AgbioInvestor-Time-and-Cost-to-Develop-a-New-GM-Trait.pdf (Accessed November 18, 2023).

Google Scholar

AgbioInvestor (2023). Global GM crop area review. Available at: https://gm.agbioinvestor.com/downloads (Accessed November 29, 2023).

Ala-Kokko, K., Lanier Nalley, L., Shew, A. M., Tack, J. B., Chaminuka, P., Matlock, M. D., et al. (2021). Economic and ecosystem impacts of GM maize in South Africa. Glob. Food Secur. 29, 100544. doi:10.1016/j.gfs.2021.100544

CrossRef Full Text | Google Scholar

Anderson, J., Bachman, P. M., Burns, A., Chakravarthy, S., Goodwin, L., Privalle, L., et al. (2021). Streamlining data requirements for the environmental risk assessment of genetically modified (GM) crops for cultivation approvals. J. Regul. Sci. 9 (1), 26–37. doi:10.21423/jrs-v09i1anderson

Bachman, P. M., Anderson, J., Burns, A., Chakravarthy, S., Goodwin, L., Privalle, L., et al. (2021). Data transportability for studies performed to support an environmental risk assessment for genetically modified (GM) crops. J. Regul. Sci. 9 (1), 38–44. doi:10.21423/jrs-v09i1bachman

Bravo, A., Gill, S. S., and Soberón, M. (2007). Mode of action of Bacillus thuringiensis Cry and Cyt toxins and their potential for insect control. Toxicon 49 (4), 423–435. doi:10.1016/j.toxicon.2006.11.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Brookes, G. (2022a). Farm income and production impacts from the use of genetically modified (GM) crop technology 1996-2020. Gm. Crops Food 13 (1), 171–195. doi:10.1080/21645698.2022.2105626

Brookes, G. (2022b). Genetically modified (GM) crop use 1996–2020: environmental impacts associated with pesticide use change. Gm. Crops Food 13 (1), 262–289. doi:10.1080/21645698.2022.2118497

Brookes, G. (2022c). Genetically modified (GM) crop use 1996–2020: impacts on carbon emissions. Gm. Crops Food 13 (1), 242–261. doi:10.1080/21645698.2022.2118495

Brune, P., Chakravarthy, S., Graser, G., Mathesius, C. A., McClain, S., Petrick, J. S., et al. (2021). Core and supplementary studies to assess the safety of genetically modified (GM) plants used for food and feed. J. Regul. Sci. 9 (1), 45–60. doi:10.21423/jrs-v09i1brune

Clark, B. W., Phillips, T. A., and Coats, J. R. (2005). Environmental fate and effects of Bacillus thuringiensis (bt) proteins from transgenic crops: a review. J. Agric. Food Chem. 53 (12), 4643–4653. doi:10.1021/jf040442k

David, M. A. (2009). GAIN report: Nigeria agricultural biotechnology annual report. Available at: https://apps.fas.usda.gov/newgainapi/api/Report/DownloadReportByFileName?fileName=AGRICULTURAL%20BIOTECHNOLOGY%20ANNUAL_Lagos_Nigeria_8-3-2009 (Accessed February 12, 2024).

Dively, G. P., Venugopal, P. D., Bean, D., Whalen, J., Holmstrom, K., Kuhar, T. P., et al. (2018). Regional pest suppression associated with widespread Bt maize adoption benefits vegetable growers. Proc. Natl. Acad. Sci. 115 (13), 3320–3325. doi:10.1073/pnas.1720692115

European Commission, Directorate-General for Research and Innovation (2010). A decade of EU-funded GMO research (2001-2010) . Brussels: European Commission . doi:10.2777/97784

Goodwin, L., Hunst, P., Burzio, L. A., Rowe, L., Money, S., and Chakravarthy, S. (2021). Stacked trait products are as safe as non-genetically modified (GM) products developed by conventional breeding practices. J. Regul. Sci. 9 (1), 22–25. doi:10.21423/jrs-v09i1goodwin

Icoz, I., and Stotzky, G. (2008). Fate and effects of insect-resistant Bt crops in soil ecosystems. Soil Biol. Biochem. 40 (3), 559–586. doi:10.1016/j.soilbio.2007.11.002

International Service for the Acquisition of Agri-biotech Applications (ISAAA) (2020). Global status of commercialized biotech/GM crops in 2019 (brief 55). Available at: https://www.isaaa.org/resources/publications/briefs/55/executivesummary/default.asp .

International Service for the Acquisition of Agri-biotech Applications (ISAAA) (2023). ISAAA’s GM approval database. Available at: https://www.isaaa.org/gmapprovaldatabase (Accessed September 14, 2023).

Klümper, W., and Qaim, M. (2014). A meta-analysis of the impacts of genetically modified crops. PLoS ONE 9 (11), e111629. doi:10.1371/journal.pone.0111629

Macall, D. M., Trabanino, C. R., Soto, A. H., and Smyth, S. J. (2020). Genetically modified maize impacts in Honduras: production and social issues. Transgenic Res. 29 (5), 575–586. doi:10.1007/s11248-020-00221-y

Mathesius, C. A., Sauve-Ciencewicki, A., Anderson, J., Cleveland, C., Fleming, C., Frierdich, G. E., et al. (2020). Recommendations for assessing human dietary exposure to newly expressed proteins in genetically modified crops. J. Regul. Sci. 8, 1–12. doi:10.21423/JRS-V08MATHESIUS

McClain, S., Herman, R. A., Islamovic, E., Ranjan, R., Silvanovich, A., Song, P., et al. (2021). Allergy risk assessment for newly expressed proteins (NEPs) in genetically modified (GM) plants. J. Regul. Sci. 9 (1), 67–75. doi:10.21423/jrs-v09i1mcclain

National Academies of Sciences, Engineering, and Medicine (2016). Genetically engineered crops: experiences and prospects . Washington, DC, USA: The National Academies Press US . doi:10.17226/23395

OECD (2003). Consensus document on the biology of Zea mays subsp. mays (maize) . ENV/JM/MONO(2007)14. Paris, France: Organisation for Economic Cooperative Development . Available at: https://one.oecd.org/document/env/jm/mono(2003)11/en/pdf .

OECD (2007). Consensus document on safety information on transgenic plants expressing Bacillus thuringiensis - derived insect control protein . ENV/JM/MONO(2003)11. Paris, France: Organisation for Economic Cooperative Development . Available at: https://one.oecd.org/document/env/jm/mono(2007)14/en/pdf .

OECD (2023). Consensus document on environmental considerations for the release of transgenic plants, harmonisation of regulatory oversight in biotechnology . ENV/CBC/MONO(2023)30. Paris, France: Organisation for Economic Cooperative Development . doi:10.1787/62ed0e04-en

Peshin, R., Hansra, B. S., Singh, K., Nanda, R., Sharma, R., Yangsdon, S., et al. (2021). Long-term impact of Bt cotton: an empirical evidence from North India. J. Clean. Prod. 312, 127575. doi:10.1016/j.jclepro.2021.127575

Raybould, A., and Macdonald, P. (2018). Policy-led comparative environmental risk assessment of genetically modified crops: testing for increased risk rather than profiling phenotypes leads to predictable and transparent decision-making. Front. Bioeng. Biotechnol. 6, 43. doi:10.3389/fbioe.2018.00043

Raybould, A., Stacey, D., Vlachos, D., Graser, G., Li, X., and Joseph, R. (2007). Non-target organism risk assessment of MIR604 maize expressing mCry3A for control of corn rootworm. J. Appl. Entomology 131 (6), 391–399. doi:10.1111/j.1439-0418.2007.01200.x

Roberts, A., Boeckman, C. J., Mühl, M., Romeis, J., Teem, J. L., Valicente, F. H., et al. (2020). Sublethal endpoints in non-target organism testing for insect-active GE crops. Front. Bioeng. Biotechnol. 8, 556. doi:10.3389/fbioe.2020.00556

Romeis, J., Hellmich, R., Candolfi, M., Carstens, K., De Schrijver, A., Gatehouse, A., et al. (2011). Recommendations for the design of laboratory studies on non-target arthropods for risk assessment of genetically engineered plants. Transgenic Res. 20 (1), 1–22. doi:10.1007/s11248-010-9446-x

Romeis, J., McLean, M. A., and Shelton, A. M. (2013). When bad science makes good headlines: bt maize and regulatory bans. Nat. Biotech. 31 (5), 386–387. doi:10.1038/nbt.2578

Roper, J., Lipscomb, E. A., Petrick, J. S., Ranjan, R., Sauve-Ciencewicki, A., and Goodwin, L. (2021). Toxicological assessment of newly expressed proteins (NEPs) in genetically modified (GM) plants. J. Regul. Sci. 9 (1), 61–66. doi:10.21423/jrs-v09i1roper

Rose, R. I. (2007). White paper on tier-based testing for the effects of proteinaceous insecticidal plant-incorporated protectants on NonTarget arthropods for regulatory risk assessments. Available at: https://www.epa.gov/sites/default/files/2015-09/documents/tier-based-testing.pdf (Accessed April 2, 2024).

Shelton, A. M. (2021). Bt eggplant: a personal account of using biotechnology to improve the lives of resource-poor farmers. Am. Entomologist 67 (3), 52–59. doi:10.1093/ae/tmab036

Smyth, S. J. (2020). The human health benefits from GM crops. Plant Biotechnol. J. 18 (4), 887–888. doi:10.1111/pbi.13261

Snell, C., Bernheim, A., Bergé, J.-B., Kuntz, M., Pascal, G., Paris, A., et al. (2012). Assessment of the health impact of GM plant diets in long-term and multigenerational animal feeding trials: a literature review. Food Chem. Toxicol. 50 (3), 1134–1148. doi:10.1016/j.fct.2011.11.048

Stotzky, G. (2005). Persistence and biological activity in soil of the insecticidal proteins from Bacillus thuringiensis, especially from transgenic plants. Plant Soil 266 (1), 77–89. doi:10.1007/s11104-005-5945-6

U.S. Environmental Protection Agency (US EPA) (2010). Bacillus thurigiensis Cry3Bb1 corn biopesticides registration action document. Available at: https://www3.epa.gov/pesticides/chem_search/reg_actions/pip/cry3bb1-brad.pdf .

Van Eenennaam, A. L., and Young, A. E. (2014). Prevalence and impacts of genetically engineered feedstuffs on livestock populations. J. Animal Sci. 92 (11), 4255–4278. doi:10.2527/jas.2014-8124

van Frankenhuyzen, K. (2009). Insecticidal activity of Bacillus thuringiensis crystal proteins. J. Invertebr. Pathol. 101 (1), 1–16. doi:10.1016/j.jip.2009.02.009

Waters, S., Ramos, A., Henderson Culler, A., Hunst, P., Zeph, L., Gast, R., et al. (2021). Recommendations for science-based safety assessment of genetically modified (GM) plants for food and feed uses. J. Regul. Sci. 9 (1), 16–21. doi:10.21423/JRS-V09I1WATERS

Woodruff, S. (2024). Gardeners can now grow a genetically modified purple tomato made with snapdragon DNA. Available at: https://www.npr.org/sections/health-shots/2024/02/06/1228868005/purple-tomato-gmo-gardeners .

Zilberman, D., Holland, T. G., and Trilnick, I. (2018). Agricultural GMOs—what we know and where scientists disagree. Sustainability 10 (5), 1514. doi:10.3390/su10051514

Keywords: genetically modified (GM), regulation, food and feed, safety assessment, environmental risk assessment (ERA), problem formulation, cultivation, data requirements

Citation: Storer NP, Simmons AR, Sottosanto J, Anderson JA, Huang MH, Mahadeo D, Mathesius CA, Sanches da Rocha M, Song S and Urbanczyk-Wochniak E (2024) Modernizing and harmonizing regulatory data requirements for genetically modified crops—perspectives from a workshop. Front. Bioeng. Biotechnol. 12:1394704. doi: 10.3389/fbioe.2024.1394704

Received: 02 March 2024; Accepted: 12 April 2024; Published: 10 May 2024.

Reviewed by:

Copyright © 2024 Storer, Simmons, Sottosanto, Anderson, Huang, Mahadeo, Mathesius, Sanches da Rocha, Song and Urbanczyk-Wochniak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Abigail R. Simmons, [email protected]

This article is part of the Research Topic

Advancing Science in Support of Sustainable Bio-Innovation: 16th ISBR Symposium

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: novel view synthesis with neural radiance fields for industrial robot applications.

Abstract: Neural Radiance Fields (NeRFs) have become a rapidly growing research field with the potential to revolutionize typical photogrammetric workflows, such as those used for 3D scene reconstruction. As input, NeRFs require multi-view images with corresponding camera poses as well as the interior orientation. In the typical NeRF workflow, the camera poses and the interior orientation are estimated in advance with Structure from Motion (SfM). But the quality of the resulting novel views, which depends on different parameters such as the number and distribution of available images, as well as the accuracy of the related camera poses and interior orientation, is difficult to predict. In addition, SfM is a time-consuming pre-processing step, and its quality strongly depends on the image content. Furthermore, the undefined scaling factor of SfM hinders subsequent steps in which metric information is required. In this paper, we evaluate the potential of NeRFs for industrial robot applications. We propose an alternative to SfM pre-processing: we capture the input images with a calibrated camera that is attached to the end effector of an industrial robot and determine accurate camera poses with metric scale based on the robot kinematics. We then investigate the quality of the novel views by comparing them to ground truth, and by computing an internal quality measure based on ensemble methods. For evaluation purposes, we acquire multiple datasets that pose challenges for reconstruction typical of industrial applications, like reflective objects, poor texture, and fine structures. We show that the robot-based pose determination reaches similar accuracy as SfM in non-demanding cases, while having clear advantages in more challenging scenarios. Finally, we present first results of applying the ensemble method to estimate the quality of the synthetic novel view in the absence of a ground truth.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .