results and future research directions

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 7. The Results
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The results section is where you report the findings of your study based upon the methodology [or methodologies] you applied to gather information. The results section should state the findings of the research arranged in a logical sequence without bias or interpretation. A section describing results should be particularly detailed if your paper includes data generated from your own research.

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070.

Importance of a Good Results Section

When formulating the results section, it's important to remember that the results of a study do not prove anything . Findings can only confirm or reject the hypothesis underpinning your study. However, the act of articulating the results helps you to understand the problem from within, to break it into pieces, and to view the research problem from various perspectives.

The page length of this section is set by the amount and types of data to be reported . Be concise. Use non-textual elements appropriately, such as figures and tables, to present findings more effectively. In deciding what data to describe in your results section, you must clearly distinguish information that would normally be included in a research paper from any raw data or other content that could be included as an appendix. In general, raw data that has not been summarized should not be included in the main text of your paper unless requested to do so by your professor.

Avoid providing data that is not critical to answering the research question . The background information you described in the introduction section should provide the reader with any additional context or explanation needed to understand the results. A good strategy is to always re-read the background section of your paper after you have written up your results to ensure that the reader has enough context to understand the results [and, later, how you interpreted the results in the discussion section of your paper that follows].

Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Brett, Paul. "A Genre Analysis of the Results Section of Sociology Articles." English for Specific Speakers 13 (1994): 47-59; Go to English for Specific Purposes on ScienceDirect;Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008; Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit; "Reporting Findings." In Making Sense of Social Research Malcolm Williams, editor. (London;: SAGE Publications, 2003) pp. 188-207.

Structure and Writing Style

I.  Organization and Approach

For most research papers in the social and behavioral sciences, there are two possible ways of organizing the results . Both approaches are appropriate in how you report your findings, but use only one approach.

• Present a synopsis of the results followed by an explanation of key findings . This approach can be used to highlight important findings. For example, you may have noticed an unusual correlation between two variables during the analysis of your findings. It is appropriate to highlight this finding in the results section. However, speculating as to why this correlation exists and offering a hypothesis about what may be happening belongs in the discussion section of your paper.
• Present a result and then explain it, before presenting the next result then explaining it, and so on, then end with an overall synopsis . This is the preferred approach if you have multiple results of equal significance. It is more common in longer papers because it helps the reader to better understand each finding. In this model, it is helpful to provide a brief conclusion that ties each of the findings together and provides a narrative bridge to the discussion section of the your paper.

NOTE:   Just as the literature review should be arranged under conceptual categories rather than systematically describing each source, you should also organize your findings under key themes related to addressing the research problem. This can be done under either format noted above [i.e., a thorough explanation of the key results or a sequential, thematic description and explanation of each finding].

II.  Content

In general, the content of your results section should include the following:

• Introductory context for understanding the results by restating the research problem underpinning your study . This is useful in re-orientating the reader's focus back to the research problem after having read a review of the literature and your explanation of the methods used for gathering and analyzing information.
• Inclusion of non-textual elements, such as, figures, charts, photos, maps, tables, etc. to further illustrate key findings, if appropriate . Rather than relying entirely on descriptive text, consider how your findings can be presented visually. This is a helpful way of condensing a lot of data into one place that can then be referred to in the text. Consider referring to appendices if there is a lot of non-textual elements.
• A systematic description of your results, highlighting for the reader observations that are most relevant to the topic under investigation . Not all results that emerge from the methodology used to gather information may be related to answering the " So What? " question. Do not confuse observations with interpretations; observations in this context refers to highlighting important findings you discovered through a process of reviewing prior literature and gathering data.
• The page length of your results section is guided by the amount and types of data to be reported . However, focus on findings that are important and related to addressing the research problem. It is not uncommon to have unanticipated results that are not relevant to answering the research question. This is not to say that you don't acknowledge tangential findings and, in fact, can be referred to as areas for further research in the conclusion of your paper. However, spending time in the results section describing tangential findings clutters your overall results section and distracts the reader.
• A short paragraph that concludes the results section by synthesizing the key findings of the study . Highlight the most important findings you want readers to remember as they transition into the discussion section. This is particularly important if, for example, there are many results to report, the findings are complicated or unanticipated, or they are impactful or actionable in some way [i.e., able to be pursued in a feasible way applied to practice].

NOTE:   Always use the past tense when referring to your study's findings. Reference to findings should always be described as having already happened because the method used to gather the information has been completed.

III.  Problems to Avoid

When writing the results section, avoid doing the following :

• Discussing or interpreting your results . Save this for the discussion section of your paper, although where appropriate, you should compare or contrast specific results to those found in other studies [e.g., "Similar to the work of Smith [1990], one of the findings of this study is the strong correlation between motivation and academic achievement...."].
• Reporting background information or attempting to explain your findings. This should have been done in your introduction section, but don't panic! Often the results of a study point to the need for additional background information or to explain the topic further, so don't think you did something wrong. Writing up research is rarely a linear process. Always revise your introduction as needed.
• Ignoring negative results . A negative result generally refers to a finding that does not support the underlying assumptions of your study. Do not ignore them. Document these findings and then state in your discussion section why you believe a negative result emerged from your study. Note that negative results, and how you handle them, can give you an opportunity to write a more engaging discussion section, therefore, don't be hesitant to highlight them.
• Including raw data or intermediate calculations . Ask your professor if you need to include any raw data generated by your study, such as transcripts from interviews or data files. If raw data is to be included, place it in an appendix or set of appendices that are referred to in the text.
• Be as factual and concise as possible in reporting your findings . Do not use phrases that are vague or non-specific, such as, "appeared to be greater than other variables..." or "demonstrates promising trends that...." Subjective modifiers should be explained in the discussion section of the paper [i.e., why did one variable appear greater? Or, how does the finding demonstrate a promising trend?].
• Presenting the same data or repeating the same information more than once . If you want to highlight a particular finding, it is appropriate to do so in the results section. However, you should emphasize its significance in relation to addressing the research problem in the discussion section. Do not repeat it in your results section because you can do that in the conclusion of your paper.
• Confusing figures with tables . Be sure to properly label any non-textual elements in your paper. Don't call a chart an illustration or a figure a table. If you are not sure, go here .

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070; Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008;  Caprette, David R. Writing Research Papers. Experimental Biosciences Resources. Rice University; Hancock, Dawson R. and Bob Algozzine. Doing Case Study Research: A Practical Guide for Beginning Researchers . 2nd ed. New York: Teachers College Press, 2011; Introduction to Nursing Research: Reporting Research Findings. Nursing Research: Open Access Nursing Research and Review Articles. (January 4, 2012); Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit ; Ng, K. H. and W. C. Peh. "Writing the Results." Singapore Medical Journal 49 (2008): 967-968; Reporting Research Findings. Wilder Research, in partnership with the Minnesota Department of Human Services. (February 2009); Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Schafer, Mickey S. Writing the Results. Thesis Writing in the Sciences. Course Syllabus. University of Florida.

Writing Tip

Why Don't I Just Combine the Results Section with the Discussion Section?

It's not unusual to find articles in scholarly social science journals where the author(s) have combined a description of the findings with a discussion about their significance and implications. You could do this. However, if you are inexperienced writing research papers, consider creating two distinct sections for each section in your paper as a way to better organize your thoughts and, by extension, your paper. Think of the results section as the place where you report what your study found; think of the discussion section as the place where you interpret the information and answer the "So What?" question. As you become more skilled writing research papers, you can consider melding the results of your study with a discussion of its implications.

Driscoll, Dana Lynn and Aleksandra Kasztalska. Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

• << Previous: Insiderness
• Next: Using Non-Textual Elements >>
• Last Updated: May 25, 2024 4:09 PM
• URL: https://libguides.usc.edu/writingguide

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 16 October 2023

Forecasting the future of artificial intelligence with machine learning-based link prediction in an exponentially growing knowledge network

• Mario Krenn   ORCID: orcid.org/0000-0003-1620-9207 1 ,
• Lorenzo Buffoni 2 ,
• Bruno Coutinho 2 ,
• Sagi Eppel 3 ,
• Jacob Gates Foster 4 ,
• Andrew Gritsevskiy   ORCID: orcid.org/0000-0001-8138-8796 3 , 5 , 6 ,
• Harlin Lee   ORCID: orcid.org/0000-0001-6128-9942 4 ,
• Yichao Lu   ORCID: orcid.org/0009-0001-2005-1724 7 ,
• João P. Moutinho 2 ,
• Nima Sanjabi   ORCID: orcid.org/0009-0000-6342-5231 8 ,
• Rishi Sonthalia   ORCID: orcid.org/0000-0002-0928-392X 4 ,
• Ngoc Mai Tran 9 ,
• Francisco Valente   ORCID: orcid.org/0000-0001-6964-9391 10 ,
• Yangxinyu Xie   ORCID: orcid.org/0000-0002-1532-6746 11 ,
• Rose Yu 12 &
• Michael Kopp 6  

Nature Machine Intelligence volume  5 ,  pages 1326–1335 ( 2023 ) Cite this article

26k Accesses

6 Citations

1059 Altmetric

Metrics details

• Complex networks
• Computer science
• Research data

A tool that could suggest new personalized research directions and ideas by taking insights from the scientific literature could profoundly accelerate the progress of science. A field that might benefit from such an approach is artificial intelligence (AI) research, where the number of scientific publications has been growing exponentially over recent years, making it challenging for human researchers to keep track of the progress. Here we use AI techniques to predict the future research directions of AI itself. We introduce a graph-based benchmark based on real-world data—the Science4Cast benchmark, which aims to predict the future state of an evolving semantic network of AI. For that, we use more than 143,000 research papers and build up a knowledge network with more than 64,000 concept nodes. We then present ten diverse methods to tackle this task, ranging from pure statistical to pure learning methods. Surprisingly, the most powerful methods use a carefully curated set of network features, rather than an end-to-end AI approach. These results indicate a great potential that can be unleashed for purely ML approaches without human knowledge. Ultimately, better predictions of new future research directions will be a crucial component of more advanced research suggestion tools.

Similar content being viewed by others

Learning on knowledge graph dynamics provides an early warning of impactful research

TrendyGenes, a computational pipeline for the detection of literature trends in academia and drug discovery

Accelerating science with human-aware artificial intelligence

The corpus of scientific literature grows at an ever-increasing speed. Specifically, in the field of artificial intelligence (AI) and machine learning (ML), the number of papers every month is growing exponentially with a doubling rate of roughly 23 months (Fig. 1 ). Simultaneously, the AI community is embracing diverse ideas from many disciplines such as mathematics, statistics and physics, making it challenging to organize different ideas and uncover new scientific connections. We envision a computer program that can automatically read, comprehend and act on AI literature. It can predict and suggest meaningful research ideas that transcend individual knowledge and cross-domain boundaries. If successful, it could greatly improve the productivity of AI researchers, open up new avenues of research and help drive progress in the field.

The doubling rate of papers per month is roughly 23 months, which might lead to problems for publishing in these fields, at some point. The categories are cs.AI, cs.LG, cs.NE and stat.ML.

In this work, we address the ambitious vision of developing a data-driven approach to predict future research directions 1 . As new research ideas often emerge from connecting seemingly unrelated concepts 2 , 3 , 4 , we model the evolution of AI literature as a temporal network. We construct an evolving semantic network that encapsulates the content and development of AI research since 1994, with approximately 64,000 nodes (representing individual concepts) and 18 million edges (connecting jointly investigated concepts).

We use the semantic network as an input to ten diverse statistical and ML methods to predict the future evolution of the semantic network with high accuracy. That is, we can predict which combinations of concepts AI researchers will investigate in the future. Being able to predict what scientists will work on is a first crucial step for suggesting new topics that might have a high impact.

Several methods were contributions to the Science4Cast competition hosted by the 2021 IEEE International Conference on Big Data (IEEE BigData 2021). Broadly, we can divide the methods into two classes: methods that use hand-crafted network-theoretical features and those that automatically learn features. We found that models using carefully hand-crafted features outperform methods that attempt to learn features autonomously. This (somewhat surprising) finding indicates a great potential for improvements of models free of human priors.

Our paper introduces a real-world graph benchmark for AI, presents ten methods for solving it, and discusses how this task contributes to the larger goal of AI-driven research suggestions in AI and other disciplines. All methods are available at GitHub 5 .

Semantic networks

The goal here is to extract knowledge from the scientific literature that can subsequently be processed by computer algorithms. At first glance, a natural first step would be to use large language model (such as GPT3 6 , Gopher 7 , MegaTron 8 or PaLM 9 ) on each article to extract concepts and their relations automatically. However, these methods still struggle in reasoning capabilities 10 , 11 ; thus, it is not yet directly clear how these models can be used for identifying and suggesting new ideas and concept combinations.

Rzhetsky et al. 12 pioneered an alternative approach, creating semantic networks in biochemistry from co-occurring concepts in scientific papers. There, nodes represent scientific concepts, specifically biomolecules, and are linked when a paper mentions both in its title or abstract. This evolving network captures the field’s history and, using supercomputer simulations, provides insights into scientists’ collective behaviour and suggests more efficient research strategies 13 . Although creating semantic networks from concept co-occurrences extracts only a small amount of knowledge from each paper, it captures non-trivial and actionable content when applied to large datasets 2 , 4 , 13 , 14 , 15 . PaperRobot extends this approach by predicting new links from large medical knowledge graphs and formulating new ideas in human language as paper drafts 16 .

This approach was applied and extended to quantum physics 17 by building a semantic network of over 6,000 concepts. There, the authors (including one of us) formulated the prediction of new research trends and connections as an ML task, with the goal of identifying concept pairs not yet jointly discussed in the literature but likely to be investigated in the future. This prediction task was one component for personalized suggestions of new research ideas.

Link prediction in semantic networks

We formulate the prediction of future research topics as a link-prediction task in an exponentially growing semantic network in the AI field. The goal is to predict which unconnected nodes, representing scientific concepts not yet jointly researched, will be connected in the future.

Link prediction is a common problem in computer science, addressed with classical metrics and features, as well as ML techniques. Network theory-based methods include local motif-based approaches 18 , 19 , 20 , 21 , 22 , linear optimization 23 , global perturbations 24 and stochastic block models 25 . ML works optimized a combination of predictors 26 , with further discussion in a recent review 27 .

In ref. 17 , 17 hand-crafted features were used for this task. In the Science4Cast competition, the goal was to find more precise methods for link-prediction tasks in semantic networks (a semantic network of AI that is ten times larger than the one in ref. 17 ).

Potential for idea generation in science

The long-term goal of predictions and suggestions in semantic networks is to provide new ideas to individual researchers. In a way, we hope to build a creative artificial muse in science 28 . We can bias or constrain the model to give topic suggestions that are related to the research interest of individual scientists, or a pair of scientists to suggest topics for collaborations in an interdisciplinary setting.

Generation and analysis of the dataset

Dataset construction.

We create a dynamic semantic network using papers published on arXiv from 1992 to 2020 in the categories cs.AI, cs.LG, cs.NE and stat.ML. The 64,719 nodes represent AI concepts extracted from 143,000 paper titles and abstracts using Rapid Automatic Keyword Extraction (RAKE) and normalized via natural language processing (NLP) techniques and custom methods 29 . Although high-quality taxonomies such as the Computer Science Ontology (CSO) exist 30 , 31 , we choose not to use them for two reasons: the rapid growth of AI and ML may result in new concepts not yet in the CSO, and not all scientific domains have high-quality taxonomies like CSO. Our goal is to build a scalable approach applicable to any domain of science. However, future research could investigate merging these approaches (see ‘Extensions and future work’).

Concepts form the nodes of the semantic network, and edges are drawn when concepts co-appear in a paper title or abstract. Edges have time stamps based on the paper’s publication date, and multiple time-stamped edges between concepts are common. The network is edge-weighted, and the weight of an edge stands for the number of papers that connect two concepts. In total, this creates a time-evolving semantic network, depicted in Fig. 2 .

Utilizing 143,000 AI and ML papers on arXiv from 1992 to 2020, we create a list of concepts using RAKE and other NLP tools, which form nodes in a semantic network. Edges connect concepts that co-occur in titles or abstracts, resulting in an evolving network that expands as more concepts are jointly investigated. The task involves predicting which unconnected nodes (concepts not yet studied together) will connect within a few years. We present ten diverse statistical and ML methods to address this challenge.

Network-theoretical analysis

The published semantic network has 64,719 nodes and 17,892,352 unique undirected edges, with a mean node degree of 553. Many hub nodes greatly exceed this mean degree, as shown in Fig. 3 , For example, the highest node degrees are 466,319 (neural network), 198,050 (deep learning), 195,345 (machine learning), 169,555 (convolutional neural network), 159,403 (real world), 150,227 (experimental result), 127,642 (deep neural network) and 115,334 (large scale). We fit a power-law curve to the degree distribution p ( k ) using ref. 32 and obtained p ( k )  ∝   k −2.28 for degree k  ≥ 1,672. However, real complex network degree distributions often follow power laws with exponential cut-offs 33 . Recent work 34 has indicated that lognormal distributions fit most real-world networks better than power laws. Likelihood ratio tests from ref. 32 suggest truncated power law ( P  = 0.0031), lognormal ( P  = 0.0045) and lognormal positive ( P  = 0.015) fit better than power law, while exponential ( P  = 3 × 10 −10 ) and stretched exponential ( P  = 6 × 10 −5 ) are worse. We couldn’t conclusively determine the best fit with P  ≤ 0.1.

Nodes with the highest (466,319) and lowest (2) non-zero degrees are neural network and video compression technique, respectively. The most frequent non-zero degree is 64 (which occures 313 times). The plot, in log scale, omits 1,247 nodes with zero degrees.

We observe changes in network connectivity over time. Although degree distributions remained heavy-tailed, the ordering of nodes within the tail changed due to popularity trends. The most connected nodes and the years they became so include decision tree (1994), machine learning (1996), logic program (2000), neural network (2005), experimental result (2011), machine learning (2013, for a second time) and neural network (2015).

Connected component analysis in Fig. 4 reveals that the network grew more connected over time, with the largest group expanding and the number of connected components decreasing. Mid-sized connected components’ trajectories may expose trends, like image processing. A connected component with four nodes appeared in 1999 (brightness change, planar curve, local feature, differential invariant), and three more joined in 2000 (similarity transformation, template matching, invariant representation). In 2006, a paper discussing support vector machine and local feature merged this mid-sized group with the largest connected component.

Primary (left, blue) vertical axis: number of connected components with more than one node. Secondary (right, orange) vertical axis: number of nodes in the largest connected component. For example, the network in 2019 comprises of one large connected component with 63,472 nodes and 1,247 isolated nodes, that is, nodes with no edges. However, the 2001 network has 19 connected components with size greater than one, the largest of which has 2,733 nodes.

The semantic network reveals increasing centralization over time, with a smaller percentage of nodes (concepts) contributing to a larger fraction of edges (concept combinations). Figure 5 shows that the fraction of edges for high-degree nodes rises, while it decreases for low-degree nodes. The decreasing average clustering coefficient over time supports this trend, suggesting nodes are more likely to connect to high-degree central nodes. This could be due to the AI community’s focus on a few dominating methods or more consistent terminology use.

This cumulative histogram illustrates the fraction of nodes (concepts) corresponding to the fraction of edges (connections) for given years (1999, 2003, 2007, 2011, 2015 and 2019). The graph was generated by adding edges and nodes dated before each year. Nodes are sorted by increasing degrees. The y value at x  = 80 represents the fraction of edges contributed by all nodes in and below the 80th percentile of degrees.

Problem formulation

At the big picture, we aim to make predictions in an exponentially growing semantic network. The specific task involves predicting which two nodes v 1 and v 2 with degrees d ( v 1/ 2 ) ≥  c lacking an edge in the year (2021 −  δ ) will have w edges in 2021. We use δ  = 1, 3, 5, c  = 0, 5, 25 and w  = 1, 3, where c is a minimal degree. Note that c  = 0 is an intriguing special case where the nodes may not have an associated edge in the initial year, requiring the model to predict which nodes will connect to entirely new edges. The task w  = 3 goes beyond simple link prediction and seeks to identify uninvestigated concept pairs that will appear together in at least three papers. An interesting alternative task could be predicting the fastest-growing links, denoted as ‘trend’ prediction.

In this task, we provide a list of 10 million unconnected node pairs (each node having a degree ≥ c ) for the year (2021 −  δ ), with the goal of sorting this list by descending probability that they will have at least w edges in 2021.

For evaluation, we employ the receiver operating characteristic (ROC) curve 35 , which plots the true-positive rate against the false-positive rate at various threshold settings. We use the area under the curve (AUC) of the ROC curve as our evaluation metric. The advantage of AUC over mean square error is its independence from the data distribution. Specifically, in our case, where the two classes have a highly asymmetric distribution (with only about 1–3% of newly connected edges) and the distribution changes over time, AUC offers meaningful interpretation. Perfect predictions yield AUC = 1, whereas random predictions result in AUC = 0.5. AUC represents the percentage that a random true element is ranked higher than a random false one. For other metrics, see ref. 36 .

To tackle this task, models can use the complete information of the semantic network from the year (2021 −  δ ) in any way possible. In our case, all presented models generate a dataset for learning to make predictions from (2021 − 2 δ ) to (2021 −  δ ). Once the models successfully complete this task, they are applied to the test dataset to make predictions from (2021 −  δ ) to 2021. All reported AUCs are based on the test dataset. Note that solving the test dataset is especially challenging due to the δ -year shift, causing systematic changes such as the number of papers and density of the semantic network.

AI-based solutions

We demonstrate various methods to predict new links in a semantic network, ranging from pure statistical approaches and neural networks with hand-crafted features (NF) to ML models without NF. The results are shown in Fig. 6 , with the highest AUC scores achieved by methods using NF as ML model inputs. Pure network features without ML are competitive, while pure ML methods have yet to outperform those with NF. Predicting links generated at least three times can achieve a quasi-deterministic AUC > 99.5%, suggesting an interesting target for computational sociology and science of science research. We have performed numerous tests to exclude data leakage in the benchmark dataset, overfitting or data duplication both in the set of articles and the set of concepts. We rank methods based on their performance, with model M1 as the best performing and model M8 as the least effective (for the prediction of a new edge with δ  = 3, c  = 0). Models M4 and M7 are subdivided into M4A, M4B, M7A and M7B, differing in their focus on feature or embedding selection (more details in Methods ).

Here we show the AUC values for different models that use machine learning techniques (ML), hand-crafted network features (NF) or a combination thereof. The left plot shows results for the prediction of a single new link (that is, w  = 1) and the right plot shows the results for the prediction of new triple links w  = 3. The task is to predict δ  = [1, 3, 5] years into the future, with cut-off values c  = [0, 5, 25]. We sort the models by the the results for the task ( w  = 1,  δ  = 3,  c  = 0), which was the task in the Science4Cast competition. Data points that are not shown have a AUC below 0.6 or are not computed due to computational costs. All AUC values reported are computed on a validation dataset δ years ahead of the training dataset that the models have never seen. Note that the prediction of new triple edges can be performed nearly deterministic. It will be interesting to understand the origin of this quasi-deterministic pattern in AI research, for example, by connecting it to the research interests of scientists 88 .

Model M1: NF + ML. This approach combines tree-based gradient boosting with graph neural networks, using extensive feature engineering to capture node centralities, proximity and temporal evolution 37 . The Light Gradient Boosting Machine (LightGBM) model 38 is employed with heavy regularization to combat overfitting due to the scarcity of positive examples, while a time-aware graph neural network learns dynamic node representations.

Model M2: NF + ML. This method utilizes node and edge features (as well as their first and second derivatives) to predict link formation probabilities 39 . Node features capture popularity, and edge features measure similarity. A multilayer perceptron with rectified linear unit (ReLU) activation is used for learning. Cold start issues are addressed with feature imputation.

Model M3: NF + ML. This method captures hand-crafted node features over multiple time snapshots and employs a long short-term memory (LSTM) to learn time dependencies 40 . The features were selected to be highly informative while having a low computational cost. The final configuration uses degree centrality, degree of neighbours and common neighbours as features. The LSTM outperforms fully connected neural networks.

Model M4: pure NF. Two purely statistical methods, preferential attachment 41 and common neighbours 27 , are used 42 . Preferential attachment is based on node degrees, while common neighbours relies on the number of shared neighbours. Both methods are computationally inexpensive and perform competitively with some learning-based models.

Model M5: NF + ML. Here, ten groups of first-order graph features are extracted to obtain neighbourhood and similarity properties, with principal component analysis 43 applied for dimensionality reduction 44 . A random forest classifier is trained on the balanced dataset to predict new links.

Model M6: NF + ML. The baseline solution uses 15 hand-crafted features as input to a four-layer neural network, predicting the probability of link formation between node pairs 17 .

Model M7: end-to-end ML (auto node embedding). The baseline solution is modified to use node2vec 45 and ProNE embeddings 46 instead of hand-crafted features. The embeddings are input to a neural network with two hidden layers for link prediction.

Model M8: end-to-end ML (transformers). This method learns features in an unsupervised manner using transformers 47 . Node2vec embeddings 45 , 48 are generated for various snapshots of the adjacency matrix, and a transformer model 49 is pre-trained as a feature extractor. A two-layer ReLU network is used for classification.

Extensions and future work

Developing an AI that suggests research topics to scientists is a complex task, and our link-prediction approach in temporal networks is just the beginning. We highlight key extensions and future work directly related to the ultimate goal of AI for AI.

High-quality predictions without feature engineering. Interestingly, the most effective methods utilized carefully crafted features on a graph with extracted concepts as nodes and edges representing their joint publication history. Investigating whether end-to-end deep learning can solve tasks without feature engineering will be a valuable next step.

Fully automated concept extraction. Current concept lists, generated by RAKE’s statistical text analysis, demand time-consuming code development to address irrelevant term extraction (for example, verbs, adjectives). A fully automated NLP technique that accurately extracts meaningful concepts without manual code intervention would greatly enhance the process.

Leveraging ontology taxonomies. Alongside fully automated concept extraction, utilizing established taxonomies such as the CSO 30 , 31 , Wikipedia-extracted concepts, book indices 17 or PhySH key phrases is crucial. Although not comprehensive for all domains, these curated datasets often contain hierarchical and relational concept information, greatly improving prediction tasks.

Incorporating relation extraction. Future work could explore relation extraction techniques for constructing more accurate, sparser semantic networks. By discerning and classifying meaningful concept relationships in abstracts 50 , 51 , a refined AI literature representation is attainable. Using NLP tools for entity recognition, relationship identification and classification, this approach may enhance prediction performance and novel research direction identification.

Generation of new concepts. Our work predicts links between known concepts, but generating new concepts using AI remains a challenge. This unsupervised task, as explored in refs. 52 , 53 , involves detecting concept clusters with dynamics that signal new concept formation. Incorporating emerging concepts into the current framework for suggesting research topics is an intriguing future direction.

Semantic information beyond concept pairs. Currently, abstracts and titles are compressed into concept pairs, but more comprehensive information extraction could yield meaningful predictions. Exploring complex data structures such as hypergraphs 54 may be computationally demanding, but clever tricks could reduce complexity, as shown in ref. 55 . Investigating sociological factors or drawing inspiration from material science approaches 56 may also improve prediction tasks. A recent dataset for the study of the science of science also includes more complex data structures than the ones used in our paper, including data from social networks such as Twitter 57 .

Predictions of scientific success. While predicting new links between concepts is valuable, assessing their potential impact is essential for high-quality suggestions. Introducing a metric of success, like estimated citation numbers or citation growth rate, can help gauge the importance of these connections. Adapting citation prediction techniques from the science of science 58 , 59 , 60 , 61 to semantic networks offers a promising research direction.

Anomaly detections. Predicting likely connections may not align with finding surprising research directions. One method for identifying surprising suggestions involves constraining cosine similarity between vertices 62 , which measures shared neighbours and can be associated with semantic (dis)similarity. Another approach is detecting anomalies in semantic networks, which are potential links with extreme properties 63 , 64 . While scientists often focus on familiar topics 3 , 4 , greater impact results from unexpected combinations of distant domains 12 , encouraging the search for surprising associations.

End-to-end formulation. Our method breaks down the goal of extracting knowledge from scientific literature into subtasks, contrasting with end-to-end deep learning that tackles problems directly without subproblems 65 , 66 . End-to-end approaches have shown great success in various domains 67 , 68 , 69 . Investigating whether such an end-to-end solution can achieve similar success in our context would be intriguing.

Our method represents a crucial step towards developing a tool that can assist scientists in uncovering novel avenues for exploration. We are confident that our outlined ideas and extensions pave the way for achieving practical, personalized, interdisciplinary AI-based suggestions for new impactful discoveries. We firmly believe that such a tool holds the potential to become a influential catalyst, transforming the way scientists approach research questions and collaborate in their respective fields.

Details on concept set generation and application

In this section, we provide details on the generation of our list of 64,719 concepts. For more information, the code is accessible on GitHub . The entire approach is designed for immediate scalability to other domains.

Initially, we utilized approximately 143,000 arXiv papers from the categories cs.AI, cs.LG, cs.NE and stat.ML spanning 1992 to 2020. The omission of earlier data has a negligible effect on our research question, as we show below. We then iterated over each individual article, employing RAKE (with an extended stopword list) to suggest concept candidates, which were subsequently stored.

Following the iteration, we retained concepts composed of at least two words (for example, neural network) appearing in six or more articles, as well as concepts comprising a minimum of three words (for example, recurrent neural network) appearing in three or more articles. This initial filter substantially reduced noise generated by RAKE, resulting in a list of 104,948 concepts.

Lastly, we developed an automated filtering tool to further enhance the quality of the concept list. This tool identified common, domain-independent errors made by RAKE, which primarily included phrases that were not concepts (for example, dataset provided or discuss open challenge). We compiled a list of 543 words not part of meaningful concepts, including verbs, ordinal numbers, conjunctions and adverbials. Ultimately, this process produced our final list of 64,719 concepts employed in our study. No further semantic concept/entity linking is applied.

By this construction, the test sets with c  = 0 could lead to very rare contamination of the dataset. That is because each concept will have at least one edge in the final dataset. The effects, however, are negligible.

The distribution of concepts in the articles can be seen in Extended Data Fig. 1 . As an example, we show the extraction of concepts from five randomly chosen papers:

Memristor hardware-friendly reinforcement learning 70 : ‘actor critic algorithm’, ‘neuromorphic hardware implementation’, ‘hardware neural network’, ‘neuromorphic hardware system’, ‘neural network’, ‘large number’, ‘reinforcement learning’, ‘case study’, ‘pre training’, ‘training procedure’, ‘complex task’, ‘high performance’, ‘classical problem’, ‘hardware implementation’, ‘synaptic weight’, ‘energy efficient’, ‘neuromorphic hardware’, ‘control theory’, ‘weight update’, ‘training technique’, ‘actor critic’, ‘nervous system’, ‘inverted pendulum’, ‘explicit supervision’, ‘hardware friendly’, ‘neuromorphic architecture’, ‘hardware system’.

Automated deep learning analysis of angiography video sequences for coronary artery disease 71 : ‘deep learning approach’, ‘coronary artery disease’, ‘deep learning analysis’, ‘traditional image processing’, ‘deep learning’, ‘image processing’, ‘f1 score’, ‘video sequence’, ‘error rate’, ‘automated analysis’, ‘coronary artery’, ‘vessel segmentation’, ‘key frame’, ‘visual assessment’, ‘analysis method’, ‘analysis pipeline’, ‘coronary angiography’, ‘geometrical analysis’.

Demographic influences on contemporary art with unsupervised style embeddings 72 : ‘classification task’, ‘social network’, ‘data source’, ‘visual content’, ‘graph network’, ‘demographic information’, ‘social connection’, ‘visual style’, ‘historical dataset’, ‘novel information’

The utility of general domain transfer learning for medical language tasks 73 : ‘natural language processing’, ‘long short term memory’, ‘logistic regression model’, ‘transfer learning technique’, ‘short term memory’, ‘average f1 score’, ‘class classification model’, ‘domain transfer learning’, ‘weighted average f1 score’, ‘medical natural language processing’, ‘natural language process’, ‘transfer learning’, ‘f1 score’, ’natural language’, ’deep model’, ’logistic regression’, ’model performance’, ’classification model’, ’text classification’, ’regression model’, ’nlp task’, ‘short term’, ‘medical domain’, ‘weighted average’, ‘class classification’, ‘bert model’, ‘language processing’, ‘biomedical domain’, ‘domain transfer’, ‘nlp model’, ‘main model’, ‘general domain’, ‘domain model’, ‘medical text’.

Fast neural architecture construction using envelopenets 74 : ‘neural network architecture’, ‘neural architecture search’, ‘deep network architecture’, ‘image classification problem’, ‘neural architecture search method’, ‘neural network’, ‘reinforcement learning’, ‘deep network’, ‘image classification’, ‘objective function’, ‘network architecture’, ‘classification problem’, ‘evolutionary algorithm’, ‘neural architecture’, ‘base network’, ‘architecture search’, ‘training epoch’, ‘search method’, ‘image class’, ‘full training’, ‘automated search’, ‘generated network’, ‘constructed network’, ‘gpu day’.

Time gap between the generation of edges

We use articles from arXiv, which only goes back to the year 1992. However, of course, the field of AI exists at least since the 1960s 75 . Thus, this raises the question whether the omission of the first 30–40 years of research has a crucial impact in the prediction task we formulate, specifically, whether edges that we consider as new might not be so new after all. Thus, in Extended Data Fig. 2 , we compute the time between the formation of edges between the same concepts, taking into account all or just the first edge. We see that the vast majority of edges are formed within short time periods, thus the effect of omission of early publication has a negligible effect for our question. Of course, different questions might be crucially impacted by the early data; thus, a careful choice of the data source is crucial 61 .

Positive examples in the test dataset

Table 1 shows the number of positive cases within the 10 million examples in the 18 test datasets that are used for evaluation.

Publication rates in quantum physics

Another field of research that gained a lot of attention in the recent years is quantum physics. This field is also a strong adopter of arXiv. Thus, we analyse in the same way as for AI in Fig. 1 . We find in Extended Data Fig. 3 no obvious exponential increase in papers per month. A detailed analysis of other domains is beyond the current scope. It will be interesting to investigate the growth rates in different scientific disciplines in more detail, especially given that exponential increase has been observed in several aspects of the science of science 3 , 76 .

Details on models M1–M8

What follows are more detailed explanations of the models presented in the main text. All codes are available at GitHub. The feature importance of the best model M1 is shown here, those of other models are analysed in the respective workshop contributions (cited in the subsections).

Details on M1

The best-performing solution is based on a blend of a tree-based gradient boosting approach and a graph neural network approach 37 . Extensive feature engineering was conducted to capture the centralities of the nodes, the proximity between node pairs and their evolution over time. The centrality of a node is captured by the number of neighbours and the PageRank score 77 , while the proximity between a node pair is derived using the Jaccard index. We refer the reader to ref. 37 for the list of all features and their feature importance.

The tree-based gradient boosting approach uses LightGBM 38 and applies heavy regularization to combat overfitting due to the scarcity of positive samples. The graph neural network approach employs a time-aware graph neural network to learn node representations on dynamic semantic networks. The feature importance of model M1, averaged over 18 datasets, is shown in Table 2 . It shows that the temporal features do contribute largely to the model performance, but the model remains strong even when they are removed. An example of the evolution of the training (from 2016 to 2019) and test set (2019 to 2021) for δ  = 3, c  = 25, ω  = 1 is shown in Extended Data Fig. 4 .

Details on M2

The second method assumes that the probability that nodes u and v form an edge in the future is a function of the node features f ( u ), f ( v ) and some edge feature h ( u ,  v ). We chose node features f that capture popularity at the current time t 0 (such as degree, clustering coefficient 78 , 79 and PageRank 77 ). We also use these features’ first and second time derivatives to capture the evolution of the node’s popularity over time. After variable selection during training, we chose h to consist of the HOP-rec score (high-order proximity for implicit recommendation) 80 , 81 and a variation of the Dice similarity score 82 as a measure of similarity between nodes. In summary, we use 31 node features for each node, and two edge features, which gives 31 × 2 + 2 = 64 features in total. These features are then fed into a small multilayer perceptron (5 layers, each with 13 neurons) with ReLU activation.

Cold start is the problem that some nodes in the test set do not appear in the training set. Our strategy for a cold start is imputation. We say a node v is seen if it appeared in the training data, and unseen otherwise; similarly, we say that a node is born at time t if t is the first time stamp where an edge linking this node has appeared. The idea is that an unseen node is simply a node born in the future, so its features should look like a recently born node in the training set. If a node is unseen, then we impute its features as the average of the features of the nodes born recently. We found that with imputation during training, the test AUC scores across all models consistently increased by about 0.02. For a complete description of this method, we refer the reader to ref. 39 .

Details on M3

This approach, detailed in ref. 40 , uses hand-crafted node features that have been captured in multiple time snapshots (for example, every year) and then uses an LSTM to benefit from learning the time dependencies of these features. The final configuration uses two main types of feature: node features including degree and degree of neighbours, and edge features including common neighbours. In addition, to balance the training data, the same number of positive and negative instances have been randomly sampled and combined.

One of the goals was to identify features that are very informative with a very low computational cost. We found that the degree centrality of the nodes is the most important feature, and the degree centrality of the neighbouring nodes and the degree of mutual neighbours gave us the best trade-off. As all of the extracted features’ distributions are highly skewed to the right, meaning most of the features take near zero values, using a power transform such as Yeo–Johnson 83 helps to make the distributions more Gaussian, which boosts the learning. Finally, for the link-prediction task, we saw that LSTMs perform better than fully connected neural networks.

Details on M4

The following two methods are based on a purely statistical analysis of the test data and are explained in detail in ref. 42 .

Preferential attachment. In the network analysis, we concluded that the growth of this dataset tends to maintain a heavy-tailed degree distribution, often associated with scale-free networks. As mentioned before the γ value of the degree distribution is very close to 2, suggesting that preferential attachment 41 is probably the main organizational principle of the network. As such, we implemented a simple prediction model following this procedure. Preferential attachment scores in link prediction are often quantified as

with k i , j the degree of nodes i and j . However, this assumes the scoring of links between nodes that are already connected to the network, that is k i , j  > 0, which is not the case for all the links we must score in the dataset. As a result, we define our preferential attachment model as

Using this simple model with no free parameters we could score new links and compare them with the other models. Immediately we note that preferential attachment outperforms some learning-based models, even if it never manages to reach the top AUC, but it is extremely simple and with negligible computational cost.

Common neighbours. We explore another network-based approach to score the links. Indeed, while the preferential attachment model we derived performed well, it uses no information about the distance between i and j , which is a popular feature used in link-prediction methods 27 . As such, we decided to test a method known as common neighbours 18 . We define Γ ( i ) as the set of neighbors of node i and Γ ( i ) ∩  Γ ( j ) as the set of common neighbours between nodes i and j . We can easily score the nodes with

the intuition being that nodes that share a larger number of neighbours are more likely to be connected than distant nodes that do not share any.

Evaluating this score for each pair ( i ,  j ) on the dataset of unconnected pairs, which can be computed as the second power of the adjacency matrix, A 2 , we obtained an AUC that is sometimes higher than preferential attachment and sometimes lower than it but is still consistently quite close with the best learning-based models.

Details on M5

This method is based on ref. 44 . First, ten groups of first-order graph features are extracted to get some neighbourhood and similarity properties from each pair of nodes: degree centrality of nodes, pair’s total number of neighbours, common neighbours index, Jaccard coefficient, Simpson coefficient, geometric coefficient, cosine coefficient, Adamic–Adar index, resource allocation index and preferential attachment index. They are obtained for three consecutive years to capture the temporal dynamics of the semantic network, leading to a total of 33 features. Second, principal component analysis 43 is applied to reduce the correlation between features, speed up the learning process and improve generalization, which results in a final set of seven latent variables. Lastly, a random forest classifier is trained (using a balanced dataset) to estimate the likelihood of new links between the AI concepts.

In this paper, a modification was performed in relation to the original formulation of the method 44 : two of the original features, average neighbour degree and clustering coefficient, were infeasible to extract for some of the tasks covered in this paper, as their computation can be heavy for such a very large network, and they were discarded. Due to some computational memory issues, it was not possible to run the model for some of the tasks covered in this study, and so those results are missing.

Details on M6

The baseline solution for the Science4Cast competition was closely related to the model presented in ref. 17 . It uses 15 hand-crafted features of a pair of nodes v 1 and v 2 . (Degrees of v 1 and v 2 in the current year and previous two years are six properties. The number of shared neighbours in total of v 1 and v 2 in the current year and previous two years are six properties. The number of shared neighbours between v 1 and v 2 in the current year and the previous two years are three properties). These 15 features are the input of a neural network with four layers (15, 100, 10 and 1 neurons), intending to predict whether the nodes v 1 and v 2 will have w edges in the future. After the training, the model computes the probability for all 10 million evaluation examples. This list is sorted and the AUC is computed.

Details on M7

The solution M7 was not part of the Science4Cast competition and therefore not described in the corresponding proceedings, thus we want to add more details.

The most immediate way one can apply ML to this problem is by automating the detection of features. Quite simply, the baseline solution M6 is modified such that instead of 15 hand-crafted features, the neural network is instead trained on features extracted from a graph embedding. We use two different embedding approaches. The first method is employs node2vec (M7A) 45 , for which we use the implementations provided in the nodevectors Python package 84 . The second one uses the ProNE embedding (M7B) 46 , which is based on sparse matrix factorizations modulated by the higher-order Cheeger inequality 85 .

The embeddings generate a 32-dimensional representation for each node, resulting in edge representations in [0, 1] 64 . These features are input into a neural network with two hidden layers of size 1,000 and 30. Like M6, the model computes the probability for evaluation examples to determine the ROC. We compare ProNE to node2vec, a common graph embedding method using a biased random walk procedure with return and in–out parameters, which greatly affect network encoding. Initial experiments used default values for a 64-dimensional encoding before inputting into the neural network. The higher variance in node2vec predictions is probably due to its sensitivity to hyperparameters. While ProNE is better suited for general multi-dataset link prediction, node2vec’s sensitivity may help identify crucial network features for predicting temporal evolution.

Details on M8

This model, which is detailed in ref. 47 , does not use any hand-crafted features but learns them in a completely unsupervised manner. To do so, we extract various snapshots of the adjacency matrix through time, capturing graphs in the form of A t for t  = 1994, …, 2019. We then embed each of these graphs into 128-dimensional Euclidean space via node2vec 45 , 48 . For each node u in the semantic graph, we extract different 128-dimensional vector embeddings n u ( A 1994 ), …,  n u ( A 2019 ).

Transformers have performed extremely well in NLP tasks 49 ; thus, we apply them to learn the dynamics of the embedding vectors. We pre-train a transformer to help classify node pairs. For the transformer, the encoder and decoder had 6 layers each; we used 128 as the embedding dimension, 2,048 as the feed-forward dimension and 8-headed attention. This transformer acts as our feature extractor. Once we pre-train our transformer, we add a two-layer ReLU network with hidden dimension 128 as a classifier on top.

Data availability

All 18 datasets tested in this paper are available via Zenodo at https://doi.org/10.5281/zenodo.7882892 ref. 86 .

Code availability

All of the models and codes described above can be found via GitHub at https://github.com/artificial-scientist-lab/FutureOfAIviaAI ref. 5 and a permanent Zenodo record at https://zenodo.org/record/8329701 ref. 87 .

Clauset, A., Larremore, D. B. & Sinatra, R. Data-driven predictions in the science of science. Science 355 , 477–480 (2017).

Article   Google Scholar  

Evans, J. A. & Foster, J. G. Metaknowledge. Science 331 , 721–725 (2011).

Article   MathSciNet   MATH   Google Scholar  

Fortunato, S. et al. Science of science. Science 359 , eaao0185 (2018).

Wang, D. & Barabási, A.-L. The Science of Science (Cambridge Univ. Press, 2021).

Krenn, M. et al. FutureOfAIviaAI. GitHub https://github.com/artificial-scientist-lab/FutureOfAIviaAI (2023).

Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33 , 1877–1901 (2020).

Google Scholar  

Rae, J. W. et al. Scaling language models: methods, analysis & insights from training gopher. Preprint at https://arxiv.org/abs/2112.11446 (2021).

Smith, S. et al. Using DeepSpeed and Megatron to train Megatron-Turing NLG 530B, a large-scale generative language model. Preprint at https://arxiv.org/abs/2201.11990 (2022).

Chowdhery, A. et al. Palm: scaling language modeling with pathways. Preprint at https://arxiv.org/abs/2204.02311 (2022).

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y. & Iwasawa, Y. Large language models are zero-shot reasoners. Preprint at https://arxiv.org/abs/2205.11916 (2022).

Zhang, H., Li, L. H., Meng, T., Chang, K.-W. & Broeck, G. V. d. On the paradox of learning to reason from data. Preprint at https://arxiv.org/abs/2205.11502 (2022).

Rzhetsky, A., Foster, J. G., Foster, I. T. & Evans, J. A. Choosing experiments to accelerate collective discovery. Proc. Natl Acad. Sci. USA 112 , 14569–14574 (2015).

Foster, J. G., Rzhetsky, A. & Evans, J. A. Tradition and innovation in scientists’ research strategies. Am. Sociol. Rev. 80 , 875–908 (2015).

Van Eck, N. J. & Waltman, L. Text mining and visualization using vosviewer. Preprint at https://arxiv.org/abs/1109.2058 (2011).

Van Eck, N. J. & Waltman, L. in Measuring Scholarly Impact: Methods and Practice (eds Ding, Y. et al.) 285–320 (Springer, 2014).

Wang, Q. et al. Paperrobot: Incremental draft generation of scientific ideas. Preprint at https://arxiv.org/abs/1905.07870 (2019).

Krenn, M. & Zeilinger, A. Predicting research trends with semantic and neural networks with an application in quantum physics. Proc. Natl Acad. Sci. USA 117 , 1910–1916 (2020).

Liben-Nowell, D. & Kleinberg, J. The link-prediction problem for social networks. J. Am. Soc. Inf. Sci. Technol. 58 , 1019–1031 (2007).

Albert, I. & Albert, R. Conserved network motifs allow protein–protein interaction prediction. Bioinformatics 20 , 3346–3352 (2004).

Zhou, T., Lü, L. & Zhang, Y.-C. Predicting missing links via local information. Eur. Phys. J. B 71 , 623–630 (2009).

Article   MATH   Google Scholar  

Kovács, I. A. et al. Network-based prediction of protein interactions. Nat. Commun. 10 , 1240 (2019).

Muscoloni, A., Abdelhamid, I. & Cannistraci, C. V. Local-community network automata modelling based on length-three-paths for prediction of complex network structures in protein interactomes, food webs and more. Preprint at bioRxiv https://doi.org/10.1101/346916 (2018).

Pech, R., Hao, D., Lee, Y.-L., Yuan, Y. & Zhou, T. Link prediction via linear optimization. Physica A 528 , 121319 (2019).

Lü, L., Pan, L., Zhou, T., Zhang, Y.-C. & Stanley, H. E. Toward link predictability of complex networks. Proc. Natl Acad. Sci. USA 112 , 2325–2330 (2015).

Guimerà, R. & Sales-Pardo, M. Missing and spurious interactions and the reconstruction of complex networks. Proc. Natl Acad. Sci. USA 106 , 22073–22078 (2009).

Ghasemian, A., Hosseinmardi, H., Galstyan, A., Airoldi, E. M. & Clauset, A. Stacking models for nearly optimal link prediction in complex networks. Proc. Natl Acad. Sci. USA 117 , 23393–23400 (2020).

Zhou, T. Progresses and challenges in link prediction. iScience 24 , 103217 (2021).

Krenn, M. et al. On scientific understanding with artificial intelligence. Nat. Rev. Phys. 4 , 761–769 (2022).

Rose, S., Engel, D., Cramer, N. & Cowley, W. in Text Mining: Applications and Theory (eds Berry, M. W. & Kogan, J.) Ch. 1 (Wiley, 2010).

Salatino, A. A., Thanapalasingam, T., Mannocci, A., Osborne, F. & Motta, E. The computer science ontology: a large-scale taxonomy of research areas. In Proc. Semantic Web–ISWC 2018: 17th International Semantic Web Conference Part II Vol. 17, 187–205 (Springer, 2018).

Salatino, A. A., Osborne, F., Thanapalasingam, T. & Motta, E. The CSO classifier: ontology-driven detection of research topics in scholarly articles. In Proc. Digital Libraries for Open Knowledge: 23rd International Conference on Theory and Practice of Digital Libraries Vol. 23, 296–311 (Springer, 2019).

Alstott, J., Bullmore, E. & Plenz, D. powerlaw: a Python package for analysis of heavy-tailed distributions. PLoS ONE 9 , e85777 (2014).

Fenner, T., Levene, M. & Loizou, G. A model for collaboration networks giving rise to a power-law distribution with an exponential cutoff. Soc. Netw. 29 , 70–80 (2007).

Broido, A. D. & Clauset, A. Scale-free networks are rare. Nat. Commun. 10 , 1017 (2019).

Fawcett, T. ROC graphs: notes and practical considerations for researchers. Pattern Recognit. Lett. 31 , 1–38 (2004).

Sun, Y., Wong, A. K. & Kamel, M. S. Classification of imbalanced data: a review. Int. J. Pattern Recognit. Artif. Intell. 23 , 687–719 (2009).

Lu, Y. Predicting research trends in artificial intelligence with gradient boosting decision trees and time-aware graph neural networks. In 2021 IEEE International Conference on Big Data (Big Data) 5809–5814 (IEEE, 2021).

Ke, G. et al. LightGBM: a highly efficient gradient boosting decision tree. In Proc. 31st International Conference on Neural Information Processing Systems 3149–3157 (Curran Associates Inc., 2017).

Tran, N. M. & Xie, Y. Improving random walk rankings with feature selection and imputation Science4Cast competition, team Hash Brown. In 2021 IEEE International Conference on Big Data (Big Data) 5824–5827 (IEEE, 2021).

Sanjabi, N. Efficiently predicting scientific trends using node centrality measures of a science semantic network. In 2021 IEEE International Conference on Big Data (Big Data) 5820–5823 (IEEE, 2021).

Barabási, A.-L. Network science. Phil. Trans. R. Soci. A 371 , 20120375 (2013).

Moutinho, J. P., Coutinho, B. & Buffoni, L. Network-based link prediction of scientific concepts—a Science4Cast competition entry. In 2021 IEEE International Conference on Big Data (Big Data) 5815–5819 (IEEE, 2021).

Jolliffe, I. T. & Cadima, J. Principal component analysis: a review and recent developments. Phil. Trans. R. Soc. A 374 , 20150202 (2016).

Valente, F. Link prediction of artificial intelligence concepts using low computational power. In 2021 IEEE International Conference on Big Data (Big Data) 5828–5832 (2021).

Grover, A. & Leskovec, J. node2vec: scalable feature learning for networks. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 855–864 (ACM, 2016).

Zhang, J., Dong, Y., Wang, Y., Tang, J. & Ding, M. ProNE: fast and scalable network representation learning. In Proc. Twenty-Eighth International Joint Conference on Artificial Intelligence 4278–4284 (International Joint Conferences on Artificial Intelligence Organization, 2019).

Lee, H., Sonthalia, R. & Foster, J. G. Dynamic embedding-based methods for link prediction in machine learning semantic network. In 2021 IEEE International Conference on Big Data (Big Data) 5801–5808 (IEEE, 2021).

Liu, R. & Krishnan, A. PecanPy: a fast, efficient and parallelized python implementation of node2vec. Bioinformatics 37 , 3377–3379 (2021).

Vaswani, A. et al. Attention is all you need. In Proc. 31st International Conference on Neural Information Processing Systems 6000–6010 (Curran Associates Inc., 2017).

Zelenko, D., Aone, C. & Richardella, A. Kernel methods for relation extraction. J. Mach. Learn. Res. 3 , 1083–1106 (2003).

MathSciNet   MATH   Google Scholar  

Bach, N. & Badaskar, S. A review of relation extraction. Literature Review for Language and Statistics II 2 , 1–15 (2007).

Salatino, A. A., Osborne, F. & Motta, E. How are topics born? Understanding the research dynamics preceding the emergence of new areas. PeerJ Comput. Sc. 3 , e119 (2017).

Salatino, A. A., Osborne, F. & Motta, E. AUGUR: forecasting the emergence of new research topics. In Proc. 18th ACM/IEEE on Joint Conference on Digital Libraries 303–312 (IEEE, 2018).

Battiston, F. et al. The physics of higher-order interactions in complex systems. Nat. Phys. 17 , 1093–1098 (2021).

Coutinho, B. C., Wu, A.-K., Zhou, H.-J. & Liu, Y.-Y. Covering problems and core percolations on hypergraphs. Phys. Rev. Lett. 124 , 248301 (2020).

Article   MathSciNet   Google Scholar  

Olivetti, E. A. et al. Data-driven materials research enabled by natural language processing and information extraction. Appl. Phys. Rev. 7 , 041317 (2020).

Lin, Z., Yin, Y., Liu, L. & Wang, D. SciSciNet: a large-scale open data lake for the science of science research. Sci. Data 10 , 315 (2023).

Azoulay, P. et al. Toward a more scientific science. Science 361 , 1194–1197 (2018).

Liu, H., Kou, H., Yan, C. & Qi, L. Link prediction in paper citation network to construct paper correlation graph. EURASIP J. Wirel. Commun. Netw. 2019 , 1–12 (2019).

Reisz, N. et al. Loss of sustainability in scientific work. New J. Phys. 24 , 053041 (2022).

Frank, M. R., Wang, D., Cebrian, M. & Rahwan, I. The evolution of citation graphs in artificial intelligence research. Nat. Mach. Intell. 1 , 79–85 (2019).

Newman, M. Networks (Oxford Univ. Press, 2018).

Kwon, D. et al. A survey of deep learning-based network anomaly detection. Cluster Comput. 22 , 949–961 (2019).

Pang, G., Shen, C., Cao, L. & Hengel, A. V. D. Deep learning for anomaly detection: a review. ACM Comput. Surv. 54 , 1–38 (2021).

Collobert, R. et al. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 , 2493–2537 (2011).

MATH   Google Scholar  

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015).

Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM 60 , 84–90 (2017).

Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518 , 529–533 (2015).

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529 , 484–489 (2016).

Wu, N., Vincent, A., Strukov, D. & Xie, Y. Memristor hardware-friendly reinforcement learning. Preprint at https://arxiv.org/abs/2001.06930 (2020).

Zhou, C. et al. Automated deep learning analysis of angiography video sequences for coronary artery disease. Preprint at https://arxiv.org/abs/2101.12505 (2021).

Huckle, N., Garcia, N. & Nakashima, Y. Demographic influences on contemporary art with unsupervised style embeddings. In Proc. Computer Vision–ECCV 2020 Workshops Part II Vol. 16, 126–142 (Springer, 2020).

Ranti, D. et al. The utility of general domain transfer learning for medical language tasks. Preprint at https://arxiv.org/abs/2002.06670 (2020).

Kamath, P., Singh, A. & Dutta, D. Fast neural architecture construction using envelopenets. Preprint at https://arxiv.org/abs/1803.06744 (2018).

Minsky, M. Steps toward artificial intelligence. Proc. IRE 49 , 8–30 (1961).

Bornmann, L., Haunschild, R. & Mutz, R. Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanit. Soc. Sci. Commun. 8 , 224 (2021).

Brin, S. & Page, L. The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. 30 , 107–117 (1998).

Holland, P. W. & Leinhardt, S. Transitivity in structural models of small groups. Comp. Group Studies 2 , 107–124 (1971).

Watts, D. J. & Strogatz, S. H. Collective dynamics of ‘small-world’ networks. Nature 393 , 440–442 (1998).

Yang, J.-H., Chen, C.-M., Wang, C.-J. & Tsai, M.-F. HOP-rec: high-order proximity for implicit recommendation. In Proc. 12th ACM Conference on Recommender Systems 140–144 (2018).

Lin, B.-Y. OGB_collab_project. GitHub https://github.com/brucenccu/OGB_collab_project (2021).

Sorensen, T. A. A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on danish commons. Biol. Skar. 5 , 1–34 (1948).

Yeo, I.-K. & Johnson, R. A. A new family of power transformations to improve normality or symmetry. Biometrika 87 , 954–959 (2000).

Ranger, M. nodevectors. GitHub https://github.com/VHRanger/nodevectors (2021).

Bandeira, A. S., Singer, A. & Spielman, D. A. A Cheeger inequality for the graph connection Laplacian. SIAM J. Matrix Anal. Appl. 34 , 1611–1630 (2013).

Krenn, M. et al. Predicting the future of AI with AI. Zenodo https://doi.org/10.5281/zenodo.7882892 (2023).

Krenn, M. et al. FutureOfAIviaAI code. Zenodo https://zenodo.org/record/8329701 (2023).

Jia, T., Wang, D. & Szymanski, B. K. Quantifying patterns of research-interest evolution. Nat. Hum. Behav. 1 , 0078 (2017).

Download references

Acknowledgements

We thank IARAI Vienna and IEEE for supporting and hosting the IEEE BigData Competition Science4Cast. We are specifically grateful to D. Kreil, M. Neun, C. Eichenberger, M. Spanring, H. Martin, D. Geschke, D. Springer, P. Herruzo, M. McCutchan, A. Mihai, T. Furdui, G. Fratica, M. Vázquez, A. Gruca, J. Brandstetter and S. Hochreiter for helping to set up and successfully execute the competition and the corresponding workshop. We thank X. Gu for creating Fig. 2 , and M. Aghajohari and M. Sadegh Akhondzadeh for helpful comments on the paper. The work of H.L., R.S. and J.G.F. was supported by grant TWCF0333 from the Templeton World Charity Foundation. H.L. is additionally supported by NSF grant DMS-1952339. J.P.M. acknowledges the support of FCT (Portugal) through scholarship SFRH/BD/144151/2019. B.C. thanks the support from FCT/MCTES through national funds and when applicable co-funded EU funds under the project UIDB/50008/2020, and FCT through the project CEECINST/00117/2018/CP1495/CT0001. N.M.T. and Y.X. are supported by NSF grant DMS-2113468, the NSF IFML 2019844 award to the University of Texas at Austin, and the Good Systems Research Initiative, part of University of Texas at Austin Bridging Barriers.

Open access funding provided by Max Planck Society.

Author information

Authors and affiliations.

Max Planck Institute for the Science of Light (MPL), Erlangen, Germany

Mario Krenn

Instituto de Telecomunicações, Lisbon, Portugal

Lorenzo Buffoni, Bruno Coutinho & João P. Moutinho

University of Toronto, Toronto, Ontario, Canada

Sagi Eppel & Andrew Gritsevskiy

University of California Los Angeles, Los Angeles, CA, USA

Jacob Gates Foster, Harlin Lee & Rishi Sonthalia

Cavendish Laboratories, Cavendish, VT, USA

Andrew Gritsevskiy

Institute of Advanced Research in Artificial Intelligence (IARAI), Vienna, Austria

Andrew Gritsevskiy & Michael Kopp

Alpha 8 AI, Toronto, Ontario, Canada

Independent Researcher, Barcelona, Spain

Nima Sanjabi

University of Texas at Austin, Austin, TX, USA

Ngoc Mai Tran

Independent Researcher, Leiria, Portugal

Francisco Valente

University of Pennsylvania, Philadelphia, PA, USA

Yangxinyu Xie

University of California, San Diego, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

M. Krenn and R.Y. initiated the research. M. Krenn and M. Kopp organized the Science4Cast competition. M. Krenn generated the datasets and initial codes. S.E. and H.L. analysed the network-theoretical properties of the semantic network. M. Krenn, L.B., B.C., J.G.F., A.G, H.L., Y.L, J.P.M, N.S., R.S., N.M.T, F.V., Y.X and M. Kopp provided codes for the ten models. M. Krenn wrote the paper with input from all co-authors.

Corresponding author

Correspondence to Mario Krenn .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Machine Intelligence thanks Alexander Belikov, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Mirko Pieropan, in collaboration with the Nature Machine Intelligence team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1.

Number of concepts per article.

Extended Data Fig. 2

Time Gap between the generation of edges. Here, left shows the time it takes to create a new edge between two vertices and right shows the time between the first and the second edge.

Extended Data Fig. 3

Publications in Quantum Physics.

Extended Data Fig. 4

Evolution of the AUC during training for Model M1.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Krenn, M., Buffoni, L., Coutinho, B. et al. Forecasting the future of artificial intelligence with machine learning-based link prediction in an exponentially growing knowledge network. Nat Mach Intell 5 , 1326–1335 (2023). https://doi.org/10.1038/s42256-023-00735-0

Download citation

Received : 21 January 2023

Accepted : 11 September 2023

Published : 16 October 2023

Issue Date : November 2023

DOI : https://doi.org/10.1038/s42256-023-00735-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Cookies & Privacy
• GETTING STARTED
• Introduction
• FUNDAMENTALS
• Acknowledgements
• Research questions & hypotheses
• Concepts, constructs & variables
• Research limitations
• Getting started
• Sampling Strategy
• Research Quality
• Research Ethics
• Data Analysis

FUTURE RESEARCH

Types of future research suggestion.

The Future Research section of your dissertation is often combined with the Research Limitations section of your final, Conclusions chapter. This is because your future research suggestions generally arise out of the research limitations you have identified in your own dissertation. In this article, we discuss six types of future research suggestion. These include: (1) building on a particular finding in your research; (2) addressing a flaw in your research; examining (or testing) a theory (framework or model) either (3) for the first time or (4) in a new context, location and/or culture; (5) re-evaluating and (6) expanding a theory (framework or model). The goal of the article is to help you think about the potential types of future research suggestion that you may want to include in your dissertation.

Before we discuss each of these types of future research suggestion, we should explain why we use the word examining and then put or testing in brackets. This is simply because the word examining may be considered more appropriate when students use a qualitative research design; whereas the word testing fits better with dissertations drawing on a quantitative research design. We also put the words framework or model in brackets after the word theory . We do this because a theory , framework and model are not the same things. In the sections that follow, we discuss six types of future research suggestion.

Addressing research limitations in your dissertation

Building on a particular finding or aspect of your research, examining a conceptual framework (or testing a theoretical model) for the first time, examining a conceptual framework (or testing a theoretical model) in a new context, location and/or culture.

• Expanding a conceptual framework (or testing a theoretical model)

Re-evaluating a conceptual framework (or theoretical model)

In the Research Limitations section of your Conclusions chapter, you will have inevitably detailed the potential flaws (i.e., research limitations) of your dissertation. These may include:

An inability to answer your research questions

Theoretical and conceptual problems

Limitations of your research strategy

Problems of research quality

Identifying what these research limitations were and proposing future research suggestions that address them is arguably the easiest and quickest ways to complete the Future Research section of your Conclusions chapter.

Often, the findings from your dissertation research will highlight a number of new avenues that could be explored in future studies. These can be grouped into two categories:

Your dissertation will inevitably lead to findings that you did not anticipate from the start. These are useful when making future research suggestions because they can lead to entirely new avenues to explore in future studies. If this was the case, it is worth (a) briefly describing what these unanticipated findings were and (b) suggesting a research strategy that could be used to explore such findings in future.

Sometimes, dissertations manage to address all aspects of the research questions that were set. However, this is seldom the case. Typically, there will be aspects of your research questions that could not be answered. This is not necessarily a flaw in your research strategy, but may simply reflect that fact that the findings did not provide all the answers you hoped for. If this was the case, it is worth (a) briefly describing what aspects of your research questions were not answered and (b) suggesting a research strategy that could be used to explore such aspects in future.

You may want to recommend that future research examines the conceptual framework (or tests the theoretical model) that you developed. This is based on the assumption that the primary goal of your dissertation was to set out a conceptual framework (or build a theoretical model). It is also based on the assumption that whilst such a conceptual framework (or theoretical model) was presented, your dissertation did not attempt to examine (or test) it in the field . The focus of your dissertations was most likely a review of the literature rather than something that involved you conducting primary research.

Whilst it is quite rare for dissertations at the undergraduate and master's level to be primarily theoretical in nature like this, it is not unknown. If this was the case, you should think about how the conceptual framework (or theoretical model) that you have presented could be best examined (or tested) in the field . In understanding the how , you should think about two factors in particular:

What is the context, location and/or culture that would best lend itself to my conceptual framework (or theoretical model) if it were to be examined (or tested) in the field?

What research strategy is most appropriate to examine my conceptual framework (or test my theoretical model)?

If the future research suggestion that you want to make is based on examining your conceptual framework (or testing your theoretical model) in the field , you need to suggest the best scenario for doing so.

More often than not, you will not only have set out a conceptual framework (or theoretical model), as described in the previous section, but you will also have examined (or tested) it in the field . When you do this, focus is typically placed on a specific context, location and/or culture.

If this is the case, the obvious future research suggestion that you could propose would be to examine your conceptual framework (or test the theoretical model) in a new context, location and/or culture. For example, perhaps you focused on consumers (rather than businesses), or Canada (rather than the United Kingdom), or a more individualistic culture like the United States (rather than a more collectivist culture like China).

When you propose a new context, location and/or culture as your future research suggestion, make sure you justify the choice that you make. For example, there may be little value in future studies looking at different cultures if culture is not an important component underlying your conceptual framework (or theoretical model). If you are not sure whether a new context, location or culture is more appropriate, or what new context, location or culture you should select, a review the literature will often help clarify where you focus should be.

Expanding a conceptual framework (or theoretical model)

Assuming that you have set out a conceptual framework (or theoretical model) and examined (or tested) it in the field , another series of future research suggestions comes out of expanding that conceptual framework (or theoretical model).

We talk about a series of future research suggestions because there are so many ways that you can expand on your conceptual framework (or theoretical model). For example, you can do this by:

Examining constructs (or variables) that were included in your conceptual framework (or theoretical model) but were not focused.

Looking at a particular relationship aspect of your conceptual framework (or theoretical model) further.

Adding new constructs (or variables) to the conceptual framework (or theoretical model) you set out (if justified by the literature).

It would be possible to include one or a number of these as future research suggestions. Again, make sure that any suggestions you make have are justified , either by your findings or the literature.

With the dissertation process at the undergraduate and master's level lasting between 3 and 9 months, a lot a can happen in between. For example, a specific event (e.g., 9/11, the economic crisis) or some new theory or evidence that undermines (or questions) the literature (theory) and assumptions underpinning your conceptual framework (or theoretical model). Clearly, there is little you can do about this. However, if this happens, reflecting on it and re-evaluating your conceptual framework (or theoretical model), as well as your findings, is an obvious source of future research suggestions.

• Privacy Policy

Home » Implications in Research – Types, Examples and Writing Guide

Implications in Research – Types, Examples and Writing Guide

Table of Contents

Implications in Research

Implications in research refer to the potential consequences, applications, or outcomes of the findings and conclusions of a research study. These can include both theoretical and practical implications that extend beyond the immediate scope of the study and may impact various stakeholders, such as policymakers, practitioners, researchers , or the general public.

Structure of Implications

The format of implications in research typically follows the structure below:

• Restate the main findings: Begin by restating the main findings of the study in a brief summary .
• Link to the research question/hypothesis : Clearly articulate how the findings are related to the research question /hypothesis.
• Discuss the practical implications: Discuss the practical implications of the findings, including their potential impact on the field or industry.
• Discuss the theoretical implications : Discuss the theoretical implications of the findings, including their potential impact on existing theories or the development of new ones.
• Identify limitations: Identify the limitations of the study and how they may affect the generalizability of the findings.
• Suggest directions for future research: Suggest areas for future research that could build on the current study’s findings and address any limitations.

Types of Implications in Research

Types of Implications in Research are as follows:

Theoretical Implications

These are the implications that a study has for advancing theoretical understanding in a particular field. For example, a study that finds a new relationship between two variables can have implications for the development of theories and models in that field.

Practical Implications

These are the implications that a study has for solving practical problems or improving real-world outcomes. For example, a study that finds a new treatment for a disease can have implications for improving the health of patients.

Methodological Implications

These are the implications that a study has for advancing research methods and techniques. For example, a study that introduces a new method for data analysis can have implications for how future research in that field is conducted.

Ethical Implications

These are the implications that a study has for ethical considerations in research. For example, a study that involves human participants must consider the ethical implications of the research on the participants and take steps to protect their rights and welfare.

Policy Implications

These are the implications that a study has for informing policy decisions. For example, a study that examines the effectiveness of a particular policy can have implications for policymakers who are considering whether to implement or change that policy.

Societal Implications

These are the implications that a study has for society as a whole. For example, a study that examines the impact of a social issue such as poverty or inequality can have implications for how society addresses that issue.

Forms of Implications In Research

Forms of Implications are as follows:

Positive Implications

These refer to the positive outcomes or benefits that may result from a study’s findings. For example, a study that finds a new treatment for a disease can have positive implications for patients, healthcare providers, and the wider society.

Negative Implications

These refer to the negative outcomes or risks that may result from a study’s findings. For example, a study that finds a harmful side effect of a medication can have negative implications for patients, healthcare providers, and the wider society.

Direct Implications

These refer to the immediate consequences of a study’s findings. For example, a study that finds a new method for reducing greenhouse gas emissions can have direct implications for policymakers and businesses.

Indirect Implications

These refer to the broader or long-term consequences of a study’s findings. For example, a study that finds a link between childhood trauma and mental health issues can have indirect implications for social welfare policies, education, and public health.

Importance of Implications in Research

The following are some of the reasons why implications are important in research:

• To inform policy and practice: Research implications can inform policy and practice decisions by providing evidence-based recommendations for actions that can be taken to address the issues identified in the research. This can lead to more effective policies and practices that are grounded in empirical evidence.
• To guide future research: Implications can also guide future research by identifying areas that need further investigation, highlighting gaps in current knowledge, and suggesting new directions for research.
• To increase the impact of research : By communicating the practical and theoretical implications of their research, researchers can increase the impact of their work by demonstrating its relevance and importance to a wider audience.
• To enhance the credibility of research : Implications can help to enhance the credibility of research by demonstrating that the findings have practical and theoretical significance and are not just abstract or academic exercises.
• To foster collaboration and engagement : Implications can also foster collaboration and engagement between researchers, practitioners, policymakers, and other stakeholders by providing a common language and understanding of the practical and theoretical implications of the research.

Example of Implications in Research

Here are some examples of implications in research:

• Medical research: A study on the efficacy of a new drug for a specific disease can have significant implications for medical practitioners, patients, and pharmaceutical companies. If the drug is found to be effective, it can be used to treat patients with the disease, improve their health outcomes, and generate revenue for the pharmaceutical company.
• Educational research: A study on the impact of technology on student learning can have implications for educators and policymakers. If the study finds that technology improves student learning outcomes, educators can incorporate technology into their teaching methods, and policymakers can allocate more resources to technology in schools.
• Social work research: A study on the effectiveness of a new intervention program for individuals with mental health issues can have implications for social workers, mental health professionals, and policymakers. If the program is found to be effective, social workers and mental health professionals can incorporate it into their practice, and policymakers can allocate more resources to the program.
• Environmental research: A study on the impact of climate change on a particular ecosystem can have implications for environmentalists, policymakers, and industries. If the study finds that the ecosystem is at risk, environmentalists can advocate for policy changes to protect the ecosystem, policymakers can allocate resources to mitigate the impact of climate change, and industries can adjust their practices to reduce their carbon footprint.
• Economic research: A study on the impact of minimum wage on employment can have implications for policymakers and businesses. If the study finds that increasing the minimum wage does not lead to job losses, policymakers can implement policies to increase the minimum wage, and businesses can adjust their payroll practices.

How to Write Implications in Research

Writing implications in research involves discussing the potential outcomes or consequences of your findings and the practical applications of your study’s results. Here are some steps to follow when writing implications in research:

• Summarize your key findings: Before discussing the implications of your research, briefly summarize your key findings. This will provide context for your implications and help readers understand how your research relates to your conclusions.
• Identify the implications: Identify the potential implications of your research based on your key findings. Consider how your results might be applied in the real world, what further research might be necessary, and what other areas of study could be impacted by your research.
• Connect implications to research question: Make sure that your implications are directly related to your research question or hypotheses. This will help to ensure that your implications are relevant and meaningful.
• Consider limitations : Acknowledge any limitations or weaknesses of your research, and discuss how these might impact the implications of your research. This will help to provide a more balanced view of your findings.
• Discuss practical applications : Discuss the practical applications of your research and how your findings could be used in real-world situations. This might include recommendations for policy or practice changes, or suggestions for future research.
• Be clear and concise : When writing implications in research, be clear and concise. Use simple language and avoid jargon or technical terms that might be confusing to readers.
• Provide a strong conclusion: Provide a strong conclusion that summarizes your key implications and leaves readers with a clear understanding of the significance of your research.

Purpose of Implications in Research

The purposes of implications in research include:

• Informing practice: The implications of research can provide guidance for practitioners, policymakers, and other stakeholders about how to apply research findings in practical settings.
• Generating new research questions: Implications can also inspire new research questions that build upon the findings of the original study.
• Identifying gaps in knowledge: Implications can help to identify areas where more research is needed to fully understand a phenomenon.
• Promoting scientific literacy: Implications can also help to promote scientific literacy by communicating research findings in accessible and relevant ways.
• Facilitating decision-making : The implications of research can assist decision-makers in making informed decisions based on scientific evidence.
• Contributing to theory development : Implications can also contribute to the development of theories by expanding upon or challenging existing theories.

When to Write Implications in Research

Here are some specific situations of when to write implications in research:

• Research proposal : When writing a research proposal, it is important to include a section on the potential implications of the research. This section should discuss the potential impact of the research on the field and its potential applications.
• Literature review : The literature review is an important section of the research paper where the researcher summarizes existing knowledge on the topic. This is also a good place to discuss the potential implications of the research. The researcher can identify gaps in the literature and suggest areas for further research.
• Conclusion or discussion section : The conclusion or discussion section is where the researcher summarizes the findings of the study and interprets their meaning. This is a good place to discuss the implications of the research and its potential impact on the field.

Advantages of Implications in Research

Implications are an important part of research that can provide a range of advantages. Here are some of the key advantages of implications in research:

• Practical applications: Implications can help researchers to identify practical applications of their research findings, which can be useful for practitioners and policymakers who are interested in applying the research in real-world contexts.
• Improved decision-making: Implications can also help decision-makers to make more informed decisions based on the research findings. By clearly identifying the implications of the research, decision-makers can understand the potential outcomes of their decisions and make better choices.
• Future research directions : Implications can also guide future research directions by highlighting areas that require further investigation or by suggesting new research questions. This can help to build on existing knowledge and fill gaps in the current understanding of a topic.
• Increased relevance: By highlighting the implications of their research, researchers can increase the relevance of their work to real-world problems and challenges. This can help to increase the impact of their research and make it more meaningful to stakeholders.
• Enhanced communication : Implications can also help researchers to communicate their findings more effectively to a wider audience. By highlighting the practical applications and potential benefits of their research, researchers can engage with stakeholders and communicate the value of their work more clearly.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

SYSTEMATIC REVIEW article

Knowledge hiding: current research status and future research directions.

• 1 Business School, Huaqiao University, Quanzhou, China
• 2 Department of Management, Kedge Business School, Talence, France
• 3 Business School, Beijing Normal University, Beijing, China

This article provides a review of scientific articles addressing the topic of knowledge hiding in organizations. Based on a descriptive analysis, bibliometric analysis, and content analysis of a sample of 81 articles published in the academic journals in the Web of Science from 2012 to 2020, we identify the main areas and current dynamics of knowledge hiding research. Our results show that the central research themes of knowledge hiding include five clusters: concept and dimensions, antecedents, consequences, theories, and influence mechanisms. Based on our findings, we suggest future research should further develop the concept and dimensions of knowledge hiding; probe deeper into the consequences of knowledge hiding; explore multilateral, cross-level, and collective knowledge hiding; employ innovative theoretical perspectives and research methods to study knowledge hiding; and address how cultural and other contextual factors may shape the knowledge hiding behavior.

Introduction

Knowledge management plays a crucial role in each organization, which can affect the firms' and employees' performance. However, due to the practice of “knowledge hiding,” it is often challenging to achieve satisfactory results in knowledge management ( Connelly and Kelloway, 2003 ). Previous research has pointed out that employees are not willing to share knowledge, due to reasons such as protection and control of knowledge ownership, expertise dominance, and defensive awareness ( Huo et al., 2016 ). About 50% of employees have the intention to withhold, mislead, or conceal knowledge that has been requested by another person ( Peng, 2013 ). This behavior of deliberately not providing the required knowledge to colleagues when requested is called “knowledge hiding” ( Connelly et al., 2012 ), which has become an independent concept that is different from the opposite side of knowledge sharing ( Zhao et al., 2019 ).

Obviously, knowledge hiding is very likely to reduce the efficiency of knowledge exchange among members, hinder the generation of new ideas/thoughts, or even destroy trust ( Connelly et al., 2012 ), increasing the risk of knowledge loss and inhibiting the creativity of individuals and teams ( Cerne et al., 2014 ; Bogilović et al., 2017 ). Along this vein, it makes sense to solve the dilemma of insufficient knowledge sharing through the elimination of knowledge hiding, facilitating knowledge conversion within organizations. As a result, based on a descriptive analysis, bibliometric analysis, and content analysis, we conduct an in-depth analysis of knowledge hiding publications in international Science Citation Index (SCI) and Social Science Citation Index (SSCI) journals. We aim to address these research questions:

1. What is the current publication trend in knowledge hiding?

2. Which themes involving knowledge hiding have been studied by scholars?

3. What are the areas involving knowledge hiding that seem to require future research?

Previous authors have conducted reviews on knowledge hiding (e.g., Xiao and Cooke, 2019 ; Anand et al., 2020 ; de Garcia et al., 2020 ), which are valuable. However, the review of Xiao and Cooke (2019) is based on 52 articles and all of which are written in English or Chinese, and published over the period 1997–2017. Similarly, the review of Anand et al. (2020) is drawing on 52 studies. In their work, de Garcia et al. (2020) have reviewed a total of 57 articles that are published up to April 2018, and their study focuses on distinguishing knowledge hiding and knowledge hoarding from knowledge collection and donation perspectives. Our review differs from these previous works in terms of volume, timeframe, method and the analysis. First, we have combined bibliometric analysis, content analysis and descriptive analysis in this review, which allows for incorporating rich data with less interpretative or subjectivity biases. In contrast to previous reviews, we further overview the concepts and dimensions, antecedents, consequences, theoretical foundations, and influence mechanisms of knowledge hiding. In the meantime, we have included bigger volume of articles in this review. In so doing, we are able to complement the previous reviews, offering a more objective account of evolution of this research topic.

Methodology

Our study has followed the systematic review process ( Pickering and Byrne, 2014 ). Within this process, we employ the principles of Tranfield et al. (2003) , which include (1) setting the scope, (2) conducting the search and data extraction, (3) selecting the studies and analyzing the data, and (4) extracting data and reporting the findings. To ensure the data validity and reliability, we limited our databases by searching the sample of English-written articles from the Web of Science over the period between 1995 and 2020. Further, the main reason for using SCI and SSCI databases is that web of science is “generally considered credible among the scientific community, and [are] commonly used by researchers from a wide range of fields ( de Garcia et al., 2020 , p. 4). Several reviews have used these databases (e.g., Bernatović et al., 2021 ; Vlačić et al., 2021 ).

Retrieval conditions were “Title = knowledge hiding” or “Title = knowledge withholding,” and the time span was “All years (1950–2020).” The database was “Web of Science Core Collection” and the search basis was “Web of Science Category = Unrestricted Category.” In total, we obtained a sample of 233 articles. Subsequent analysis of these 233 articles' abstracts was conducted. In order to ensure data accuracy, we carefully selected studies that fit the definition given by Connelly et al. (2012) and excluded those that belonged to disciplines such as information management. This yielded 81 articles related to knowledge hiding. For these 81 articles, we undertook the reading of full texts, using Excel to record the key findings, theoretical lens, and methodologies. Building upon the content extraction, the authors classified the core clusters in five main themes according to their characteristics: concept and dimensions, antecedents, consequences, theoretical frameworks, and influence mechanism. Figure 1 shows the flow diagram of analysis.

Figure 1 . Flow diagram.

Analysis and Findings

Publication by year.

The analysis of the number of publications per year on knowledge hiding in international journals (see Figure 2 ) shows that scholars started to systematically study knowledge hiding as an organizational behavior in the 2010s. A growing number of studies have addressed knowledge hiding but it dates back only to 2012, when knowledge hiding was first proposed as an independent concept in the work of Connelly et al. (2012) . Knowledge hiding research has gone through two periods: the initial stage (from 2012 to 2018) and the fast development stage (from 2019 to 2020). During the initial stage, publications on knowledge hiding in mainstream international journals were rare, and there were only between one and five articles published per year. Since 2019, there has been a sharp increase in knowledge hiding publications; the number of publications has jumped to more than 30 articles per year (see Figure 2 ).

Figure 2 . Annual distribution of articles on knowledge hiding.

Journal Distribution of Knowledge Hiding Research

From 2012 to 2020, research on knowledge hiding has been published in 43 SCI/SSCI journals (see Table 1 ), with 40 articles (49.38%) published in Journal Citation Reports (JCR) Q1 journals, 19 articles (23.46%) published in JCR Q2 journals, 8 articles (9.88%) published in JCR Q3 journals, and 11 articles (13.58%) published in JCR Q4 journals; 15 articles (18.52%) published in the Chartered Association of Business Schools (ABS3) journals, 10 articles (12.35%) published in ABS4 journals, one article (1.23%) published in Financial Times (FT50) journals; and one article (1.23%) published each in UT Dallas top 100 business school research rankings (UTD24) and ABS4 * journals. The top 10 journals that published most of the knowledge hiding articles are Journal of Knowledge Management, Journal of Organizational Behavior, Management Decision, International Journal of Hospitality Management, European Journal of Work and Organizational Psychology, Knowledge Management Research and Practice, International Journal of Information Management, Asian Business and Management, Leadership and Organization Development Journal , and Journal of Managerial Psychology . The majority of knowledge hiding research has been published in JCR Q1/Q2 journals, and a considerable proportion has been published in ABS3/4 journals.

Table 1 . Top publishing journals on knowledge hiding.

Publishing Activity by Authors, Authors' Institutions, and Locations

Knowledge hiding has attracted considerable attention from researchers and practitioners. As shown in Table 2 , Matej Cerne published the most articles (eight) on knowledge hiding followed by Škerlavaj and Connelly, with seven and six articles respectively. The most active institutions in the research field of knowledge hiding were University of Ljubljana (eight publications), followed by BI Norwegian Business School, McMaster University and Tongji University, each with seven publications. Table 3 lists the locations of authors' institutions, with the top four being China, Pakistan, Canada and United Arab Emirates.

Table 2 . Top publishing authors and institutions on knowledge hiding.

Table 3 . Publishing activity by authors' institution location.

Publishing Activity by Data Sources

Our analysis shows that previous data on knowledge hiding have tended to be collected in one single location, such as China, Pakistan, United Arab Emirates, Saudi Arabia, United States, and so on (see Table 4 ). Eight publications used data that were collected from multi-countries and regions (e.g., North America, Germany and Austria, Europe, Slovenia, Croatia, Serbia, Bosnia and Herzegovina, Montenegro and Macedonia). The top three locations from which researchers have collected knowledge hiding data were China (29 publications), Pakistan (13 publications) and United Arab Emirates (5 publications).

Table 4 . Locations from which researchers have collected knowledge hiding data.

Highly Cited Publications

Citations can show the research focus of scholars and reveal their main theoretical lens. Highly cited articles are often regarded as important references in the field. Table 5 presents the top 15 highly cited publications on knowledge hiding.

Table 5 . Top 15 articles on knowledge hiding by the number of citations.

Further, through a co-citation analysis, co-authorship analysis, keyword and co-occurrence analysis, and content analysis, we find that most research on knowledge hiding focuses on the concept and dimensions of the topic. For instance, as one of the highly cited publications, it is important to acknowledge that Connelly et al. (2012) take the lead in defining the concept of knowledge hiding and propose evasive hiding, playing dumb, and rationalized hiding as three dimensions of knowledge hiding. Based on the work of Connelly et al. (2012) ; Zhao et al. (2016) further examine the interpersonal antecedents of the three dimensions of knowledge hiding. Hernaus et al. (2019) distinguish the three dimensions of knowledge hiding and address how individual competitiveness may lead to knowledge hiding. Connelly and Zweig (2015) point out that the three dimensions of knowledge hiding are not equally and always harmful, where under certain circumstances, some knowledge hiding can be beneficial. Among the highly cited publications, scholars also focus on the antecedents of knowledge hiding, paying particular attention to workplace stressors, psychological ownership, and territoriality of knowledge. For example, Zhao et al. (2016) ; Škerlavaj et al. (2018) , and Khalid et al. (2018) have examined the influence mechanisms of workplace stressors, such as workplace ostracism, abusive supervision, and interpersonal injustice, on knowledge hiding. Peng (2013) ; Huo et al. (2016) , and Singh (2019) emphasize the predictive effect of psychological ownership and territoriality of knowledge on knowledge hiding. Serenko and Bontis (2016) ; Hernaus et al. (2019) , and Malik et al. (2019) also investigate the antecedents of knowledge hiding with different focuses (e.g., intra-organizational knowledge hiding, the individual-level and job-related factors within academia, organizational politics). These studies represent the two most important research directions of knowledge hiding.

Following, among the highly cited publications, we find that individual and team creativity, interpersonal relationships, and retaliation show the key consequences of knowledge hiding. The main contributions in the field include the work of Cerne et al. (2014) , who point out that “when employees hide knowledge, they trigger a reciprocal distrust loop in which coworkers are unwilling to share knowledge with them” (p. 172). In recent years, Connelly and Zweig (2015) , and Serenko and Bontis (2016) also prove that knowledge hiding can lead to retaliation. Cerne et al. (2017) and Malik et al. (2019) examine the destructive effect of knowledge hiding on individual creativity. Bogilović et al. (2017 ) and Fong et al. (2018) analyze the impacts of individual-level knowledge hiding on team-level creativity. These studies represent the mainstream consequences of knowledge hiding.

Additionally, we identify that the research focus on knowledge hiding has moved from the individual level to a multilevel influence mechanism. For example, Huo et al. (2016) ; Cerne et al. (2017) ; Fong et al. (2018) , and Hernaus et al. (2019) explore the moderating effect of team-level task interdependence on the relationship between individual-level variables and knowledge hiding. In addition, team-level cultural factors (e.g., mastery climate, workplace ethics) and organizational justice are variables that scholars have examined when exploring the multilevel influence mechanism of knowledge hiding ( Huo et al., 2016 ; Cerne et al., 2017 ; Khalid et al., 2018 ).

Major Research Clusters and Topics

Using CiteSpace4.0 software, we conducted the descriptive analysis, bibliometric analysis, and content analysis of the 81 knowledge hiding articles that are published in the international journals from 2012 to 2020. In order to clearly demonstrate the current status of knowledge hiding research, we structure our findings into the following five clusters (see Figure 3 ).

Figure 3 . Research framework of knowledge hiding. Source: extended and developed from Connelly et al. (2012) and Xiao and Cooke (2019) .

Concept and Dimensions

The bibliometric analysis suggests that keywords related to the concept of knowledge hiding include knowledge sharing, knowledge withholding, and knowledge management process. Based on these keywords and the results of our content analysis, we extract “concept and dimensions” as the first cluster that reflects the research interests in knowledge hiding.

The concept of knowledge hiding was first defined as the act of deliberately not providing knowledge or providing knowledge that is not what the seeker needs when facing a colleague's request ( Connelly et al., 2012 ). These were the first authors to discuss the linkages and differences between knowledge hiding and related concepts, such as knowledge sharing/non-sharing ( Anand et al., 2020 ), knowledge withholding ( Webster et al., 2008 ), knowledge hoarding ( Xiao and Cooke, 2019 ; de Garcia et al., 2020 ), counterproductive/deviant behavior ( Connelly and Zweig, 2015 ; Serenko and Bontis, 2016 ), workplace deception ( Connelly et al., 2012 ), and incivility ( Zhao et al., 2016 ). Later, scholars further proposed concepts such as knowledge sharing hostility ( Stenius et al., 2016 ), disengagement from knowledge sharing ( Zhao et al., 2016 ), knowledge contribution loafing ( Fang, 2017 ), and knowledge manipulation ( Bogilović et al., 2017 ). In recent years, scholars have tried to differentiate knowledge hiding from other related concepts (e.g., employee silence and knowledge protection) ( Bari et al., 2020 ).

In order to distinguish these different concepts, we compare relevant concepts through questioning whether knowledge seeking exists, the degree of knowledge sharing, and the intentionality of the behavior (see Figure 4 ). In general, scholars have widely accepted the definition of knowledge hiding given by Connelly et al. (2012) . The mainstream view believes that knowledge hiding is an important aspect of knowledge withholding, and it is not the opposite of knowledge sharing ( Connelly et al., 2012 ; Serenko and Bontis, 2016 ; Zhao et al., 2016 ). Consequently, one cannot simply equate knowledge hiding with non-sharing or a lack of knowledge sharing. In addition to subjective intention, the reasons that individuals do not share knowledge with others can be related to a lack of relevant knowledge or the inability to share the knowledge. It is worth pointing out that there are different opinions in boundaries between knowledge hiding and concepts such as knowledge non-sharing, counterproductive knowledge behavior, and knowledge protection. Hence, there still exists some confusion and cross-use of related concepts in the knowledge hiding research. In addition, the existing literature has seldom defined knowledge hiding from the indigenous/cross-cultural perspective.

Figure 4 . Comparison between knowledge hiding and related concepts. Source: extended and developed from Connelly et al. (2012) and de Garcia et al. (2020) .

Connelly et al. (2012) have developed three dimensions of knowledge hiding and an employee self-evaluation scale with 12 items, with each dimension measuring four items. Among them, evasive hiding means that the hider provides invalid knowledge or pretends to agree to help, but lacks follow-up action. An example item is “I agreed to help him/her but never really intended to.” Playing dumb refers to pretending to be ignorant of the relevant knowledge or not understanding the knowledge seeker's question, with a sample item “I pretended I did not know what he/she was talking about.” Rationalized hiding means that the hider explains the reasons for not providing required knowledge, such as the necessity to keep it confidential or offering that knowledge sharing is not allowed by the superiors. An example item is “I explained that the information is confidential and available only to people on a particular project.” Most scholars believe that rationalized hiding is different in nature from evasive hiding and playing dumb, because rationalized hiding does not involve deception, but the evasive hiding and playing dumb do have a high degree of deception.

The scale of Connelly et al. (2012) has been proved to have high reliability and validity in a series of empirical studies. In general, scholars use this scale and its original items directly, making some contextual adaptation of expressions only according to the particular research needs. There are other knowledge hiding scales, such as Peng's ( Peng, 2013 ) three-item counterproductive knowledge behavior scale and knowledge withholding behavior scales developed by Lin and Huang (2010) ; Tsay et al. (2014) , and Serenko and Bontis (2016) . Anand et al. (2020) have advocated that knowledge hiding is composed of unintentional hiding (driven by contingent situation), motivational hiding (driven by performance and competition), controlled hiding (driven by psychological ownership), victimized hiding (driven by hostility and abuse), and favored hiding (driven by identity and norms). Jha and Varkkey (2018) identify the four strategies adopted by supervisors to hide knowledge from subordinates, namely, playing innocent, misleading, rationalized hiding, and counter-questioning.

Antecedents

The antecedents of knowledge hiding include the Big Five personality traits, abusive supervision, negative workplace gossip, and career insecurity. Combined with the research framework of knowledge hiding (see Figure 3 ), the second cluster as antecedents is popular among scholars. Inspired by the work of Connelly et al. (2012) and Xiao and Cooke (2019) , we review knowledge hiding antecedents from four aspects: knowledge characteristics, individual factors, team and interpersonal factors, and organizational factors.

Knowledge characteristic is one of the first antecedents popular among scholars. Due to the complex nature of knowledge, Connelly et al. (2012) point out that such complexity affects the willingness of individuals to provide help when facing colleagues' knowledge requests. Simply, it often requires more time and energy to generate complex knowledge that knowledge owners tend to keep the knowledge for themselves. Hernaus et al. (2019) argue that people are more likely to hide tacit knowledge rather than explicit knowledge. In addition, the task relevance and the value of knowledge have a positive relation with knowledge hiding ( Connelly et al., 2012 ; Huo et al., 2016 ).

Individual factors mainly include personality traits and psychological factors such as emotion and cognition. In terms of personality traits, scholars focus mainly on the influence of the Big Five personality traits, in particular neuroticism. For example, Pan and Zhang (2018) reveal that employees with high conscientiousness and low neuroticism are less likely to hide knowledge, while people with high neuroticism are more likely to hide knowledge ( Anaza and Nowlin, 2017 ). Pan et al. (2018) verify the effects of a “dark triad of personality” (Machiavellianism, narcissism, and psychopathy) on different dimensions of knowledge hiding. Fang (2017) and Aljawarneh and Atan (2018) examine the relationship between anxiety and knowledge hiding and the relationship between cynicism and knowledge hiding.

When it comes to the cognitive perception, prior research has focused mainly on the individual's self-efficacy, territoriality and psychological ownership, psychological safety, psychological contract breach, perceived pressure or job insecurity, perceived workplace status, and career prospects. Tsay et al. (2014) ; Jha and Varkkey (2018) , and Hernaus et al. (2019) argue that individuals' confidence in their knowledge and perception of their competitiveness affect their willingness to share knowledge. Peng (2013) ; Huo et al. (2016) ; Kang (2016) ; Singh (2019) ; Khalid et al. (2020) , and Zhai et al. (2020) believe individuals' perceived exclusivity of knowledge, knowledge power, and knowledge privacy are the primary factors that determine how much knowledge they are willing to share with colleagues. He et al. (2020) ; Lin et al. (2020) , and Wu (2020) explore the formation mechanism of knowledge hiding from the perspectives of psychological safety and perceived threats. Pradhan et al. (2019) ; Ghani et al. (2020a) , and Jahanzeb et al. (2020a) emphasize the negative impacts of employee psychological contract breaches on knowledge sharing in the organizations. Jha and Varkkey (2018) ; Škerlavaj et al. (2018) , and Feng and Wang (2019) examine the impacts of workplace stressors, such as time pressure and job insecurity, on knowledge hiding.

Prior studies have also investigated knowledge hiding from employee and supervisor perspectives. In their work, Butt (2019) and Butt and Ahmad (2019) show that concerns about career prospects are important individual-level reasons for supervisors to hide knowledge from subordinates. Liu et al. (2020) find that perceived workplace status affects knowledge hiding through two opposing mechanisms: perception of knowledge sharing obligation and perception of being envied. The goal orientation has also attracted some scholars' attention in recent years when studying knowledge hiding behavior. Research by Zhu et al. (2019) shows that performance-driven goal orientation has a positive relationship with employees' knowledge hiding behaviors, which allows employees to achieve the competitive goal of surpassing colleagues. Nadeem et al. (2021) argue that shared goals are negatively related to knowledge hiding. Moh'd et al. (2021) analyze the relationship between achievement goal orientation (e.g., learning goals, performance display/performance-avoidance goal orientation) and knowledge hiding. Some scholars highlight that individual motivational factors (such as expected results/rewards and perceived knowledge sharing costs) affect knowledge hiding ( Lin and Huang, 2010 ; Shen et al., 2019 ). Although emotion and cognition have been regarded as the two core elements that drive individual behavior (e.g., Lee and Allen, 2002 ), studies on how emotional/affective factors influence knowledge hiding are still underdeveloped. We believe only Zhao and Xia (2019) have studied the negative emotional state of nursing staff as the antecedent of their knowledge hiding behavior.

Team-level and interpersonal factors reflect leadership, interpersonal relationships, and their respective interactions. When considering leadership, scholars pay the most attention to abusive leadership, followed by ethical leadership. Khalid et al. (2018) point out that knowledge hiding is not necessarily an employee's intention to directly harm other organization members, but a negative reaction of employees to abusive supervision. Further, as indicated by displaced aggression theory, when employees encounter abusive leaders, they are more likely to retaliate by targeting innocent victims, namely, their colleagues but not the leaders. Based on the reactance theory, Feng and Wang (2019) point out that when employees experience frustration resulting from the abuse of their supervisors, they will take revenge in a direct or indirect way so that they can maintain a sense of freedom. However, because of their supervisors' supreme power and status in organizations, employees usually do not directly retaliate against supervisors so as not to cause stronger hostility and reciprocal retaliation. Ethical leadership can also influence employees' behavior intentionally or unintentionally through the role model effect. Abdullah et al. (2019) ; Anser et al. (2020) , and Men et al. (2020) argue a significant but negative correlation between ethical leadership and subordinates' knowledge hiding behavior. Interestingly, the study by Xia et al. (2019) describes an inverted U–shaped curve relationship between knowledge leadership and employee knowledge hiding. Through a multilevel model, Lin et al. (2020) find that individual-focused empowering leadership can improve the supervisor-subordinate relationship and therefore inhibit knowledge hiding, whereas differentiated empowering leadership can cause group relational conflict and then lead to knowledge hiding. Based on social exchange theories, Abdillah et al. (2020) argue that altruistic leaders' humility, patience, understanding, sympathy, and compassion will be perceived by employees as uniquely socio-emotional resources, which can enhance the positive emotion of employees, improve the quality of the exchange between supervisors and subordinates (obtaining the trust and respect of the subordinates), and encourage employees be willing to make extra efforts for the organization and eliminate selfish behaviors that harm the interests of the organization, thus effectively preventing employee knowledge hiding behaviors.

From the perspective of interpersonal abuse, prior research shows that employees who encounter interpersonal unfair treatment are less willing to share their personal knowledge assets with others ( Abubakar et al., 2019 ), whereas fair interpersonal interaction is significantly negatively correlated with the three dimensions of knowledge hiding ( Ghani et al., 2020b ). Among these, the factor of passive-aggressiveness in the workplace attracts more attention from scholars. Aljawarneh and Atan (2018) find that incivility in the workplace can drive employees to feel cynical and thus hide knowledge as a countermeasure. Zhao et al. (2016) and Riaz et al. (2019) point out that, as a typical workplace passive-aggressiveness, workplace ostracism would significantly increase employees' deceptive knowledge hiding (e.g., evasive hiding and playing dumb). Similarly, research by Yao et al. (2020a , b ) shows that negative interpersonal experiences, such as workplace bullying and negative workplace gossip, accelerate the exhaustion of employee resources, such as emotions, time, energy, and organizational identity, leading them to hide knowledge. Anand et al. (2020) also find that hostility and abusive colleagues/supervisors drive employees to hide knowledge.

Concerning the impacts of interpersonal relationship on knowledge hiding, current research has focused on exploring the effects of supervisor-subordinate relationships. Scholars first divide supervisor-subordinate relationships into formal work-related relationships (contractual relationship, Leader-Member Exchange) and informal non-work-related relationships (Chinese personal guanxi relationships, Supervisor-Subordinate Guanxi) ( He et al., 2020 ), or into economic LMX and social LMX ( Babič et al., 2019 ), and then explore their impacts on employees' knowledge hiding behaviors. Previous research reveals that LMX negatively affects evasive hiding and playing dumb ( Zhao et al., 2019 ). However, this reciprocal social exchange is more likely to reduce the level of knowledge hiding within the team, especially when the relationship between individuals and their supervisors has social LMX characteristics ( Cerne et al., 2014 ). Furthermore, upward LMX social comparison leads to envy among team members, so it is a potential interpersonal antecedent of knowledge hiding among colleagues ( Weng et al., 2020 ). It is worth noting that team prosocial motivation and social LMX (but not economic LMX) have an interaction effect on knowledge hiding ( Babič et al., 2019 ). Lin and Huang (2010) ; Butt (2019) ; Butt and Ahmad (2019) ; Semerci (2019) examine the influences of interpersonal factors such as trust, reciprocity, relationship recognition, lack of interpersonal relationship, relationship conflict, and interpersonal competition. Interestingly, Lin and Huang (2010) point out that emotional bonds such as trust and reciprocity among team members can make individuals give up hiding too much knowledge to avoid retaliation from others. In addition, task conflicts and relationship conflicts have additive effects on knowledge hiding ( Semerci, 2019 ).

At the organizational level, scholars have explored the roles of organizational culture, knowledge management policies and systems, organizational politics, organizational justice, organizational recognition, and a competitive performance environment on employees' conduct of knowledge hiding. First, the knowledge sharing culture has been proved to be closely related to the extent to which the knowledge hiding behavior can be accepted and adopted by the members of the organization ( Connelly et al., 2012 ). For example, Anaza and Nowlin (2017) point out that the lack of incentives for knowledge sharing and the lack of supervisor feedback on subordinates' knowledge sharing will lead employees to hide knowledge. Jha and Varkkey (2018) highlight that a lack of organizational recognition of knowledge sharing and workload increase due to knowledge sharing increase employee knowledge hiding.

Social norms, organization policies, and management systems have also been found to have a profound impact on employees' tendency to hide knowledge. For instance, Butt and Ahmad (2019) argue that knowledge hiding is deeply embedded in many local companies and is regarded as a common code of conduct in the United Arab Emirates. Serenko and Bontis (2016) find that organizational knowledge management systems and policies have a significant direct impact on employee knowledge hiding, whereas injustice prompts employees to spontaneously engage in knowledge hiding behavior. Malik et al. (2019) propose that perceived organizational politics positively predict knowledge hiding. Abubakar et al. (2019) find that distributional, procedural, and interactional injustice increase the level of knowledge hiding among employees. Research by Jahanzeb et al. (2020b) confirms that employees who encounter organizational unfairness consider knowledge hiding as a means to rationalize the cognitive separation between oneself and the organization in order to maintain one's dignity. Finally, some scholars have examined the impact of a competitive working environment. For example, Anaza and Nowlin (2017) explain how internal competition can lead to knowledge hiding. Similar findings can be found in the work of Anand et al. (2020) , who argue that organizational internal performance and competitive factors drive employees to hide knowledge.

Consequences

Based on the highly cited publications and the keyword analysis, we find that consequences, performance, behavior , and employee/team creativity are some keywords that reflect the outcome of knowledge hiding. Therefore, we use the term consequences to summarize the third cluster concerning the knowledge hiding research.

Current research focuses mainly on the individual- and team-level consequences of knowledge hiding. A small number of studies examine the individual-level consequences of knowledge hiding between supervisors and subordinates. In terms of individual-level results, the existing research has examined the effects of knowledge hiding on individual job performance, psychological status and attitude, workplace behavior, and supervisor-subordinate/coworker relationships. For instance, most studies have found that knowledge hiding among colleagues and between supervisors and subordinates can reduce task performance, organizational citizenship behavior (OCB), and creativity ( Connelly et al., 2012 ; Cerne et al., 2014 ; Arain et al., 2019 , 2020a , b ; Jahanzeb et al., 2019 ; Malik et al., 2019 ; Singh, 2019 ; Zhu et al., 2019 ).

However, there are some mixing findings. For example, Wang et al. (2019) argue that perceived colleague knowledge hiding does not reduce the performance of salespersons. Instead, it encourages them to work harder to improve their sales performance. Burmeister et al. (2019) find that knowledge hiding (playing dumb, in contrast to evasive hiding and rationalized hiding) has opposite effects on OCB, and knowledge hiders experience different emotions. Khoreva and Wechtler (2020) point out that evasive hiding is negatively related to in-role performance, and playing dumb is positively related to it. In addition, both evasive hiding and rationalized hiding will hinder innovation performance. Regarding psychological status and attitudes, research suggests that knowledge hiding increases employees' moral disengagement ( Arain et al., 2020a ) and decreases their psychological safety, well-being, job satisfaction, and sense of thriving ( Jiang et al., 2019 ; Offergelt et al., 2019 ; Khoreva and Wechtler, 2020 ). Furthermore, knowledge hiding can trigger knowledge seekers' deviant behaviors, turnover intention, upward silence, and non-engagement in knowledge sharing ( Connelly and Zweig, 2015 ; Offergelt et al., 2019 ; Singh, 2019 ; Arain et al., 2020a ).

Concerning interpersonal relationships, studies reveal that knowledge hiding among colleagues or between supervisors and subordinates can damage workplace relationships, which can even lead to a trust crisis ( Connelly et al., 2012 ; Cerne et al., 2014 ; Arain et al., 2020b ). In particular, Connelly et al. (2012) , Cerne et al. (2014) , and Connelly and Zweig (2015) highlight that knowledge hiding can result in a vicious circle of rejecting knowledge sharing. Studies also find that knowledge hiding has significant negative effects on team performance ( Zhang and Min, 2019 ), team creativity ( Fong et al., 2018 ; Bari et al., 2019 ), team viability ( Wang et al., 2019 ), team learning, and absorptive capability ( Fong et al., 2018 ; Zhang and Min, 2019 ).

In summary, scholars have made advancements on the impacts of knowledge hiding on the individual level, but research on its impacts on team and organizational levels is still at a nascent stage. Few scholars have recently analyzed the “boomerang effect” or “negative reinforcement cycle” of knowledge hiding—the impact of knowledge hiding on the hiders' psychological status, job performance, and creativity (e.g., Cerne et al., 2014 ; Jiang et al., 2019 )—and its double-edged sword effect ( Wang et al., 2019 ), which has opened up a new avenue for research.

Theoretical Perspectives

The fourth cluster concentrates on theories that are popular among scholars that they use to conduct knowledge hiding research. The theories applied in the field of knowledge hiding are mainly from two domains—managerial theory and psychological theory—and include theories such as “exchange” (represented by social exchange theory), “resources” [represented by Conservation of Resources (COR) Theory], “learning” (represented by social learning theory), “cognition” (represented by social cognitive theory), “ownership” (represented by psychological ownership theory), “goal orientation” (represented by achievement goal theory), “personality traits,” “job characteristics,” social identity theory, displaced aggression theory, and justice theory (see Table 6 ). Although scholars have introduced other theories to study knowledge hiding, the effectiveness of this theoretical development needs to be enhanced. For example, how to theorize individual emotions has not yet been made systematic and thus needs to be further explored in future research. Furthermore, we find that theories that are mostly used to examine the motivation/antecedents of knowledge hiding or the direct/indirect (mediating) influence of antecedent variables on knowledge hiding are less used to illustrate the consequences of knowledge hiding and the boundary conditions.

Table 6 . Theoretical perspectives used in knowledge hiding research.

Influence Mechanisms

There are findings on the mediating roles of antecedent variables that affect knowledge hiding. Emotional and cognitive factors (e.g., leadership, workplace stressors, interpersonal relationships, personality traits, and psychological ownership) can induce knowledge hiding. In terms of leadership, Abdullah et al. (2019) point out that ethical leadership inhibits employees' knowledge hiding by enhancing their relational social capital. Anser et al. (2020) find that the ethical behavior of ethical leaders can enhance the perception of “meaningful work” for service industries employees, thereby reducing the possibility of engaging in knowledge hiding behaviors. Khalid et al. (2018) find that perception of interpersonal justice mediates the relationship between abusive supervision and knowledge hiding. Feng and Wang (2019) believe that abusive supervision indirectly affects knowledge hiding through job insecurity. Pradhan et al. (2019) show that psychological contract breaching and the attacks toward supervisors play a partial mediating role in the process in which abusive supervision affects knowledge hiding. Ghani et al. (2020a) further point out that abusive supervision can easily lead to psychological contract breach, thus leading employees to attack their colleagues and deliberately hide knowledge from them. In addition, Lin et al. (2020) find that individual-focused empowering leadership enhances the psychological safety of subordinates, thereby reducing their knowledge hiding, whereas differentiated empowering leadership causes group relational conflicts, thereby increasing subordinate knowledge hiding. Abdillah et al. (2020) study the dual mediating mechanisms of altruistic leadership, which inhibits and prevents employees from knowledge hiding, pointing out that the positive emotions induced by altruistic leadership and LMX have important effects.

Regarding workplace stressors and interpersonal relationships, Aljawarneh and Atan (2018) find that cynicism mediates the relationship between tolerance of workplace incivility and knowledge hiding. Riaz et al. (2019) find that workplace ostracism has a significant impact on evasive hiding and playing dumb, and that work strain plays a mediating role. Yao et al. (2020a , b ) have shown that relational identification and interpersonal trust play a chain-mediating role in the relationship between negative workplace gossip and knowledge hiding. At the same time, emotional exhaustion and organizational identification play a chain-mediating role in the relationship between workplace bullying and knowledge hiding. Jahanzeb et al. (2020b) believe that the experience of injustice causes employees to be psychologically separated from the organization and thus employees will show more knowledge hiding behaviors. Zhao et al. (2019) demonstrate that organizational identification mediates the negative impact of LMX on evasive hiding and playing dumb. Weng et al. (2020) point out that employees' upward LMX social comparison with their colleagues leads to envy of and knowledge hiding toward their colleagues. He et al. (2020) discover that psychological safety fully mediates the influence of LMX on knowledge hiding and partially mediates the influence of supervisor-subordinate guanxi on knowledge hiding.

Another aspect is shown through personality traits. Wang et al. (2014) find that perceived social identity mediates the relationship between the Big Five personality traits and knowledge hiding. Pan et al. (2018) examine the positive relationship between the “dark triad of personality” (Machiavellianism, narcissism, and psychopathy) and knowledge hiding, as well as the mediating effect of transactional psychological contracts on this relationship. Zhao and Xia (2019) point out that the negative affect states of nurses staff can “activate” their moral disengagement mechanism, allowing them to redefine their knowledge hiding behaviors as reasonable and acceptable, and thus exacerbating their knowledge hiding tendency. The final aspect is psychological ownership. Research by Peng (2013) and Huo et al. (2016) show that employees' psychological ownership of knowledge enhances their territorial awareness, which in turn causes them to hide knowledge from colleagues. Liu et al. (2020) confirm that the influence of workplace status on employee knowledge hiding is carried out through two opposite mechanisms: perceived knowledge sharing responsibility and envy. The former negatively mediates the relationship between the two, and the latter positively mediates it.

Some scholars have also studied the mediating effect of knowledge hiding. For instance, scholars examine the process through which knowledge hiding impairs individual or team creativity and innovation performance. Cerne et al. (2014) find that the knowledge hiding makes hiders reduce their own creativity, and colleague distrust plays a mediating role. Arain et al. (2019) show that supervisor knowledge hiding can reduce subordinates' self-efficacy and thus reduce their innovation. Khoreva and Wechtler (2020) point out that playing dumb and rationalized hiding can indirectly influence employee innovation performance through the mediating effect of well-being. Fong et al. (2018) confirm that a decrease in absorptive capacity is the key mediator in the relationship between knowledge hiding and team creativity. Zhang and Min (2019) state that team learning partially mediates the relationship between knowledge hiding and project team performance.

Moreover, researchers have studied the process through which knowledge hiding affects employees' subsequent interpersonal behaviors. For instance, Burmeister et al. (2019) find that guilt and shame play opposite mediating roles in the relationship between individual knowledge hiding and its subsequent interpersonal-oriented OCB. Arain et al. (2020b) point out that supervisor knowledge hiding negatively influences subordinates' OCB toward their supervisors, and subordinate distrust in their supervisors plays a mediating role. Supervisor knowledge hiding can also activate employee moral disengagement, prompting them to reduce OCB toward their supervisors and increase silence behaviors ( Arain et al., 2020a ). Jiang et al. (2019) suggest that knowledge hiding makes the hiders feel the insecurity of self-expression and interpersonal risk, thereby reducing their psychological safety and endangering their ability to thrive at work. Despite these advancements, it is necessary to develop a robust framework that integrates multipath models based on different innovative theoretical perspectives.

Regarding the moderating role of contextual factors on knowledge hiding, the existing research mainly explores the contingency influence of individual differences, job characteristics, team characteristics, and team/organizational climate. In terms of individual differences, some scholars find that organizational psychological ownership can effectively reduce the knowledge hiding resulting from territoriality ( Peng, 2013 ). Furthermore, psychological ownership significantly moderates the inverted U-shaped relationship between knowledge leadership and knowledge hiding. This curved relationship is more obvious among employees with high psychological ownership ( Xia et al., 2019 ). High psychological ownership can also minimize the impact of abusive supervision on knowledge hiding ( Ghani et al., 2020a ). Other scholars explore the boundary effect of positive traits, such as individualism/collectivist values ( Semerci, 2019 ), positive affectivity ( Jahanzeb et al., 2020a ), benevolence or tolerance ( Jahanzeb et al., 2020b ), prosocial motivations ( Škerlavaj et al., 2018 ), harmonious work enthusiasm ( Anser et al., 2020 ), professional commitment ( Malik et al., 2019 ), trust-related affect/cognition ( Nadeem et al., 2021 ), social skills ( Wang et al., 2019 ), and cultural intelligence ( Bogilović et al., 2017 ). In addition to these studies, scholars examine the impacts of negative traits on knowledge hiding, such as negative reciprocity ( Zhao et al., 2016 ; Jahanzeb et al., 2019 ), instrumental thinking ( Abdullah et al., 2019 ), hostile attribution bias ( Khalid et al., 2020 ), moral disengagement ( Zhao et al., 2016 ), and cynicism ( Jiang et al., 2019 ).

In relation to job characteristics, task interdependence has attracted a lot of attention. Huo et al. (2016) point out that task interdependence can reduce the territorial awareness and knowledge hiding caused by psychological ownership. Hernaus et al. (2019) find that task interdependence can help reduce the probability of employees' evasive knowledge hiding due to maintaining their competitiveness. Fong et al. (2018) show that task interdependence moderates the relationship between knowledge hiding and team absorptive capacity. Weng et al. (2020) suggest that the interdependence of cooperative and competitive goals has opposite moderating effects on the relationship between upward LMX social comparison and knowledge hiding. In addition, Pan and Zhang (2018) also analyze the influence of work autonomy on the intensity of the relationship between neuroticism and knowledge hiding.

Regarding the team/organizational climate, research shows that in an environment that values information exchange and cooperation, the negative influence of knowledge hiding will be greatly weakened. Accordingly, Cerne et al. (2014) study the boundary effect of the team achievement-motivation climate (e.g., performance climate and mastery climate) on the relationship between knowledge hiding and the decrease in the hider's creativity. They discover that the negative effect of knowledge hiding on the hider's creativity is reduced in a mastery climate. Furthermore, Cerne et al. (2017) find the moderating effects of mastery climate, task interdependence, and autonomy on the relationship between knowledge hiding and innovative work behavior. Bari et al. (2019) obtain similar findings which point out that a perceived mastery climate reduces the negative impact of evasive hiding and playing dumb on team creativity. Feng and Wang (2019) find that the interaction between abusive supervision and a mastery climate is negatively related to knowledge hiding, and the interaction between abusive supervision and a performance climate is positively related to knowledge hiding. On the one hand, when the organization pays more attention to individual performance feedback, performance-prove goal orientation can positively predict knowledge hiding. On the other hand, when the organization pays more attention to group performance feedback, performance-prove goal orientation is negatively correlated with knowledge hiding ( Zhu et al., 2019 ). Compared to individual rewards, team-based rewards are more likely to reduce the distrust caused by knowledge hiding, promoting the team to work hard to achieve a common goal, forming a relatively stable team structure, and improving team viability ( Wang et al., 2019 ). Yao et al. (2020a , b ) reveal the buffering effect of a forgiveness climate on the relationship between negative workplace gossip/workplace bullying and knowledge hiding. Khalid et al. (2018) clarify the role of Islamic work ethics in moderating the relationship between abusive supervision and knowledge hiding. Among these findings, the existing research on the moderating effects still focuses more on the first stage of the antecedents–knowledge hiding–consequences linkage, but there is a lack of systematic development of the moderation mechanism in the second stage.

Future Research Directions

Based on a descriptive analysis, bibliometric analysis, and content analysis, we find that research on knowledge hiding focuses mainly on five clusters. Despite the ongoing progress, several research gaps are worth further addressing.

(1) Comprehensive studies on the concept and dimensions of knowledge hiding are needed to provide a robust conceptual framework. Although the definition and three-dimensional view of knowledge hiding by Connelly et al. (2012) are widely adopted by many scholars, more research is needed to carry out in-depth comparative analysis to clarify the connections and differences between knowledge hiding and similar concepts (e.g., knowledge non-sharing, knowledge sharing hostility, knowledge contribution loafing, counterproductive knowledge behavior, knowledge hoarding, knowledge protection, employee silence, etc.). Further, more studies should continue exploring the dimensions of knowledge hiding. There is a lack of focus on knowledge hiders' psychological motivation and respective knowledge hiding strategies. For example, research on proactive, reactive, and passive knowledge hiding could enrich the field research. In addition, more studies should further explore the unique reasons and consequences of a rationalized hiding behavior. There is a need to verify the ethical aspect of rationalized hiding, when knowledge hiding is used to protect confidential information or the interests of third parties ( Zhao et al., 2019 ).

(2) Future studies need to further explore the consequences of knowledge hiding. Based on a systematic review (see Figure 3 ), we find that previous studies have focused mainly on the antecedents of knowledge hiding. Although some studies have addressed the impacts of knowledge characteristics, individual factors, team-level and interpersonal factors, and organizational-level factors on knowledge hiding, more work is needed to provide comprehensive studies on the generating mechanisms and the respective coping strategies of knowledge hiding. Prior studies have shown that knowledge hiding has impacts on individual-level outcomes (e.g., individual creativity, in-role and extra-role performance, and coworker relationships) and team-level outcomes (e.g., team creativity). However, there is a lack of research on organizational-level outcomes. Moreover, prior studies focus mainly on the impacts of knowledge hiding on the knowledge seekers and the whole team, but seldom has the research discussed the potential effects of knowledge hiding on the knowledge hiders themselves. Therefore, future research should devote more attention to the negative effects of knowledge hiding on the knowledge hiders, the team, and the organization, and also explore the consequences of different dimensions of knowledge hiding. For example, more studies could address the research gap as to whether knowledge hiding may stimulate self-reflection and prompt moral and psychological compensation for the knowledge hiders. To enrich the multilevel mediating and moderating variables, future studies could explore the boundary conditions of knowledge hiding and their respective knowledge management strategies. In short, it is necessary to increase research on the consequences of knowledge hiding to enrich the antecedents–knowledge hiding–consequences research path.

(3) More studies on multilateral, cross-level, and collective knowledge hiding are needed, and it is appropriate to introduce new paradigms for knowledge hiding research. Existing research on knowledge hiding highlights mainly two parties: the hider (A) and the seeker (B) (i.e., B seeks knowledge from A, while A hides knowledge from B). Most studies address knowledge hiding among colleagues at the horizontal level. In recent years, some scholars have started to show interest in knowledge hiding at the vertical level, that is, the top-down knowledge hiding of superiors from subordinates. However, the research on the antecedents and the generating mechanisms of knowledge hiding at the vertical level is still in the stage of exploration. There is a lack of research on bottom-up knowledge hiding (of the subordinates from their superiors). Therefore, it is necessary to study knowledge hiding adopted by people from different hierarchies (e.g., bottom, mid, and high levels) in the organizations, comparing the differences between top-down and bottom-up knowledge hiding, so as to identify regular patterns of cross-level knowledge flow within the organizations. Future research could also examine whether the knowledge hiding of top managers could trigger a trickle-down effect, referring to the fact that the behaviors of the top leaders will affect employees in the formal vertical power chain, given that knowledge hiding can be a multi-participant phenomenon. Therefore, future research could examine the contagious effects of knowledge hiding (e.g., B seeks knowledge from A, but A hides knowledge from B; B then feels lost and hides knowledge from other colleagues), diffusion effects (e.g., B seeks knowledge from A while A hides knowledge from B; A asks C to hide knowledge from B as well), bystander effects (e.g., B seeks knowledge from A, while A hides knowledge from B; C witnesses A's knowledge hiding and is influenced by it, so C also hides knowledge from B and other colleagues), and collective knowledge hiding.

(4) Future scholars should innovate theoretical perspectives and integrate multidisciplinary theories into knowledge hiding research. At present, knowledge hiding research is based mainly on theories such as social exchange, social cognition, social capital, social learning, conservation of resources, territoriality, and psychological ownership. To enrich the field research, it is necessary to diversify the theories. For example, future studies could explore the influence of social exchange relations (e.g., relative LMX) on knowledge hiding, comparing the influence of social LMX and economic LMX on employee willingness to hide knowledge. Future scholars could also conduct multi-interdisciplinary research studies. The research on how an individual's previous workplace behavior affects his or her subsequent workplace behavior has attracted great interest from scholars and mainstream journals in organizational behavior in recent years. Given that knowledge hiding is a typical morality-related behavior, future research could introduce novel and original theoretical viewpoints. For example, a moral balance model and a moral cleansing effect in disciplines such as moral psychology and cognitive psychology, can be used to explore how an individual's previous knowledge hiding behavior influences subsequent behavior in the workplace. Furthermore, knowledge hiding is considered as an emotion-driven behavior. Therefore, scholars could consider employing Lazarus's cognitive–motivational–relational (CMR) theory of emotion ( Lazarus, 1991 ) to better understand the psychological process behind knowledge hiding. Moreover, there is a lack of research on the relationship between individual affect/emotion and knowledge hiding. Therefore, scholars could employ theories, such as affective events theory and self-conscious moral emotion theory, to analyze the subsequent behavior of the hiders and seekers who are driven by affect/emotion.

(5) Research designs need more diversification. Most of the prior studies focus on the individuals, and few research studies focus on both individual and team effects. Knowledge hiding is a complex organizational behavior that concerns individual, team/interpersonal, and organizational levels. Therefore, future research could introduce data tracking technologies, such as big data analysis, to study and compare the dynamic and static (long-term and short-term) effects of multilevel knowledge hiding. Moreover, it is necessary to diversify research methods in the field. Most existing research uses one-wave or multistage surveys, employee self-evaluation, and empirical tests, with few studies using case studies and interviews. These research methods may suffer from a lack of reliability of data sources. Future research could integrate multiple methodologies (e.g., combining case studies, experimental research, surveys, and objective data mining) to verify data, which could improve the internal and external validity of the research and enhance the robustness of conclusions. In particular, it is necessary to focus on the combination of experimental and empirical research, making full use of the strengths of each method to validate the research. Researchers could carry out preliminary tests on relevant hypotheses through experimental research and then supplement them with surveys for secondary verification.

(6) Future research should integrate more cultural, sectoral, and organizational factors to enrich the findings. As discussed in the findings, most of the knowledge hiding data were collected in China and Pakistan. It is necessary to develop the diversity of knowledge hiding data in terms of country of origin. In addition, there is a lack of cross-country academic collaboration. Collaborating across borders could help to generate new ideas and allow for collecting data from different sources. Meanwhile, it would be very interesting to promote cross-country studies to identify the different definitions, perceptions, implementations, and patterns of knowledge hiding, whilst paying more attention to the relationship between cultural dimensions and knowledge hiding. Apart from cross-cultural and cross-country variables, future research could also investigate industry characteristics (such as knowledge-intensive and non-knowledge-intensive industries and masculine and feminine industries), team standards/norms (such as team moral norms), and firm size (small medium enterprises vs. multinational companies) so as to identify the boundary conditions of individual knowledge hiding behavior. Through conducting sector-specific and cross-sector comparison for knowledge hiding, we would be able to adjust knowledge management methods.

Conclusions

This article provides a systematic review of knowledge hiding. It contributes to the identification of publication patterns on knowledge hiding between 2012 and 2020. Further, we have highlighted the most influential studies, mapped the research gaps, and provided the potential research directions in the field.

This study is not without limitations. We use SCI and SSCI web of science as the databases. Using this literature search method excludes book chapters, reports, unpublished dissertations, with/without peer reviewed conference proceedings, newsletters, government documents, and working papers. Consequently, this review may not have captured the full range of scholarly literature on knowledge hiding. In the future, to reduce the publication bias ( Kepes et al., 2012 ), it would be interesting to include other databases to search literatures, for instance, the work published in the Emerging Sources Citation Index (ESCI) journals can be considered. Second, the research on knowledge hiding is emerging, and some scholars may argue that it is not yet mature enough to review the research field. In our opinion, it is only with such a complete literature review that a clear picture of knowledge hiding research can be developed so that scholars can better define research problems, innovate the research theories and methods, and enrich the field research with a robust framework.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

PH, CJ, ZX, and CS designed and supervised the study. PH collected the data. PH and ZX analyzed the data. PH, CJ, and CS wrote the manuscript. All authors contributed equally to this manuscript, reviewed, and approved this manuscript for publication.

Funding was provided by Huaqiao University's Academic Project Supported by the Fundamental Research Funds for the Central Universities (20SKGC-QT02) and the National Natural Science Foundation of China (72172048).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abdillah, M. R., Wu, W., and Anita, R. (2020). Can altruistic leadership prevent knowledge-hiding behaviour? Testing dual mediation mechanisms. Knowl. Manag. Res. Pract . doi: 10.1080/14778238.2020.1776171

CrossRef Full Text | Google Scholar

Abdullah, M. I., Dechun, H., Ali, M., and Usman, M. (2019). Ethical leadership and knowledge hiding: a moderated mediation model of relational social capital, and instrumental thinking. Front. Psychol. 10:2403. doi: 10.3389/fpsyg.2019.02403

PubMed Abstract | CrossRef Full Text | Google Scholar

Abubakar, A. M., Behravesh, E., Rezapouraghdam, H., and Yildiz, S. B. (2019). Applying artificial intelligence technique to predict knowledge hiding behavior. J. Inf. Manag. 49, 45–57. doi: 10.1016/j.ijinfomgt.2019.02.006

Aljawarneh, N. M. S., and Atan, T. (2018). Linking tolerance to workplace incivility, service innovative, knowledge hiding, and job search behavior: the mediating role of employee cynicism. Negot. Confl. Manag. Res. 11, 298–320. doi: 10.1111/ncmr.12136

Anand, A., Centobelli, P., and Cerchione, R. (2020). Why should I share knowledge with others? A review-based framework on events leading to knowledge hiding. J. Organ. Chang. Manag . 33, 379–399. doi: 10.1108/JOCM-06-2019-0174

Anaza, N. A., and Nowlin, E. L. (2017). What's mine is mine: a study of salesperson knowledge withholding & hoarding behavior. Ind. Mark. Manag. 64, 14–24. doi: 10.1016/j.indmarman.2017.03.007

Anser, M. K., Ali, M., Usman, M., Rana, M. L. T., and Yousaf, Z. (2020). Ethical leadership and knowledge hiding: an intervening and interactional analysis. Serv. Ind. J. 41, 307–329. doi: 10.1080/02642069.2020.1739657

Arain, G. A., Bhatti, Z. A., Ashraf, N., and Fang, Y. H. (2020b). Top-down knowledge hiding in organizations: an empirical study of the consequences of supervisor knowledge hiding among local and foreign workers in the Middle East. J. Bus. Ethics 164, 611–625. doi: 10.1007/s10551-018-4056-2

Arain, G. A., Bhatti, Z. A., Hameed, I., and Fang, Y. H. (2019). Top-down knowledge hiding and innovative work behavior (IWB): a three-way moderated-mediation analysis of self-efficacy and local/foreign status. J. Knowl. Manag . 24, 127–149. doi: 10.1108/JKM-11-2018-0687

Arain, G. A., Hameed, I., Umrani, W. A., Khan, A. K., and Sheikh, A. Z. (2020a). Consequences of supervisor knowledge hiding in organizations: a multilevel mediation analysis. Appl. Psychol . 70, 1242–1266. doi: 10.1111/apps.12274

Babič, K., Cerne, M., Connelly, C. E., Dysvik, A., and Škerlavaj, M. (2019). Are we in this together? Knowledge hiding in teams, collective prosocial motivation and leader-member exchange. J. Knowl. Manag . 23, 1502–1522. doi: 10.1108/JKM-12-2018-0734

Bari, M. W., Abrar, M., Shaheen, S., Bashir, M., and Fanchen, M. (2019). Knowledge hiding behaviors and team creativity: the contingent role of perceived mastery motivational climate. Sage Open 9. doi: 10.1177/2158244019876297

Bari, M. W., Ghaffar, M., and Ahmad, B. (2020). Knowledge-hiding behaviors and employees' silence: mediating role of psychological contract breach. J. Knowl. Manag . 24, 2171–2194. doi: 10.1108/JKM-02-2020-0149

Bernatović, I., Gomezel, A. S., and Cerne, M. (2021). Mapping the knowledge-hiding field and its future prospects: a bibliometric co-citation, co-word, and coupling analysis. Knowl. Manag. Res. Pract. doi: 10.1080/14778238.2021.1945963

Bogilović, S., Cerne, M., and Škerlavaj, M. (2017). Hiding behind a mask? Cultural intelligence, knowledge hiding, and individual and team creativity. Eur. J. Work Organ. Psychol. 26, 710–723. doi: 10.1080/1359432X.2017.1337747

Burmeister, A., Fasbender, U., and Gerpott, F. H. (2019). Consequences of knowledge hiding: the differential compensatory effects of guilt and shame. J. Occup. Organ. Psychol. 92, 281–304. doi: 10.1111/joop.12249

Butt, A. S. (2019). Determinants of top-down knowledge hiding in firms: an individual-level perspective. Asian Bus. Manag. 20, 259–279. doi: 10.1057/s41291-019-00091-1

Butt, A. S., and Ahmad, A. B. (2019). Are there any antecedents of top-down knowledge hiding in firms? Evidence from the United Arab Emirates. J. Knowl. Manag. 23, 1605–1627. doi: 10.1108/JKM-04-2019-0204

Cerne, M., Hernaus, T., Dysvik, A., and Škerlavaj, M. (2017). The role of multilevel synergistic interplay among team mastery climate, knowledge hiding, and job characteristics in stimulating innovative work behavior. Hum. Resour. Manag. J. 27, 281–299. doi: 10.1111/1748-8583.12132

Cerne, M., Nerstad, C. G. L., Dysvik, A., and Škerlavaj, M. (2014). What goes around comes around: knowledge hiding, perceived motivational climate, and creativity. Acad. Manag. Ann. 57, 172–192. doi: 10.5465/amj.2012.0122

CrossRef Full Text

Connelly, C. E., and Kelloway, E. K. (2003). Predictors of employees' perceptions of knowledge sharing cultures. Leadersh. Organ. Dev. J. 24, 294–301. doi: 10.1108/01437730310485815

Connelly, C. E., and Zweig, D. (2015). How perpetrators and targets construe knowledge hiding in organizations. Eur. J. Work Organ. Psychol. 24, 479–489. doi: 10.1080/1359432X.2014.931325

Connelly, C. E., Zweig, D., Webster, J., and Trougakos, J. P. (2012). Knowledge hiding in organizations. J. Organ. Behav. 33, 64–88. doi: 10.1002/job.737

de Garcia, P. S., Oliveira, M., and Brohman, K. (2020). Knowledge sharing, hiding and hoarding: how are they related? Knowl. Manag. Res. Pract . doi: 10.1080/14778238.2020.1774434

Fang, Y. H. (2017). Coping with fear and guilt using mobile social networking applications: knowledge hiding, loafing, and sharing. Telemat. Inform. 34, 779–797. doi: 10.1016/j.tele.2017.03.002

Feng, J., and Wang, C. (2019). Does abusive supervision always promote employees to hide knowledge? From both reactance and COR perspectives. J. Knowl. Manag. 23, 1455–1474. doi: 10.1108/JKM-12-2018-0737

Fong, P. S. W., Men, C., Luo, J., and Jia, R. (2018). Knowledge hiding and team creativity: the contingent role of task interdependence. Manag. Decis. 56, 329–343. doi: 10.1108/MD-11-2016-0778

Gagné, M., Tian, A. W., Soo, C., Zhang, B., Ho, K. S. B., and Hosszu, K. (2019). Different motivations for knowledge sharing and hiding: the role of motivating work design. J. Organ. Behav. 40, 783–799. doi: 10.1002/job.2364

Ghani, U., Teo, T., Li, Y., Usman, M., Islam, Z. U., Gul, H., et al. (2020a). Tit for tat: abusive supervision and knowledge hiding-the role of psychological contract breach and psychological ownership. Int. J. Environ. Res. Public Health 17:1240. doi: 10.3390/ijerph17041240

Ghani, U., Zhai, X. S., Spector, J. M., Chen, N. S., Lin, L., Ding, D. H., et al. (2020b). Knowledge hiding in higher education: role of interactional justice and professional commitment. High. Educ. 79, 325–344. doi: 10.1007/s10734-019-00412-5

He, P., Sun, R., Zhao, H., Zheng, L., and Shen, C. (2020). Linking work-related and non-work-related supervisor–subordinate relationships to knowledge hiding: a psychological safety lens. Asian Bus. Manag. doi: 10.1057/s41291-020-00137-9

Hernaus, T., Cerne, M., Connelly, C., Vokic, N. P., and Škerlavaj, M. (2019). Evasive knowledge hiding in academia: when competitive individuals are asked to collaborate. J. Knowl. Manag. 23, 597–618. doi: 10.1108/JKM-11-2017-0531

Huo, W., Cai, Z., Luo, J., Men, C., and Jia, R. (2016). Antecedents and intervention mechanisms: a multi-level study of R & D team's knowledge hiding behavior. J. Knowl. Manag. 20, 880–897. doi: 10.1108/JKM-11-2015-0451

Jahanzeb, S., De Clercq, D., and Fatima, T. (2020a). Bridging the breach: using positive affectivity to overcome knowledge hiding after contract breaches. J. Psychol. 154, 249–272. doi: 10.1080/00223980.2019.1705235

Jahanzeb, S., De Clercq, D., and Fatima, T. (2020b). Organizational injustice and knowledge hiding: the roles of organizational dis-identification and benevolence. Manag. Decis. 59, 446–462. doi: 10.1108/MD-05-2019-0581

Jahanzeb, S., Fatima, T., Bouckenooghe, D., and Bashir, F. (2019). The knowledge hiding link: a moderated mediation model of how abusive supervision affects employee creativity. Eur. J. Work Organ. Psychol. 28, 810–819. doi: 10.1080/1359432X.2019.1659245

Jha, J. K., and Varkkey, B. (2018). Are you a cistern or a channel? Exploring factors triggering knowledge-hiding behavior at the workplace: evidence from the Indian R&D professionals. J. Knowl. Manag . 22, 824–849. doi: 10.1108/JKM-02-2017-0048

Jiang, Z., Hu, X., Wang, Z., and Jiang, X. (2019). Knowledge hiding as a barrier to thriving: the mediating role of psychological safety and moderating role of organizational cynicism. J. Organ. Behav. 40, 800–818. doi: 10.1002/job.2358

Kang, S. W. (2016). Knowledge withholding: psychological hindrance to the innovation diffusion within an organisation. Knowl. Manag. Res. Pract. 14, 144–149. doi: 10.1057/kmrp.2014.24

Kepes, S., Banks, G. C., McDaniel, M., and Whetzel, D. L. (2012). Publication bias in the organizational sciences. Organ. Res. Methods 15, 624–662. doi: 10.1177/1094428112452760

Khalid, M., Bashir, S., Khan, K. A., and Abbas, N. (2018). When and how abusive supervision leads to knowledge hiding behaviors: an Islamic work ethics perspective. Leadersh. Organ. Dev. J. 39, 794–806. doi: 10.1108/LODJ-05-2017-0140

Khalid, M., Gulzar, A., and Khan, K. A. (2020). When and how the psychologically entitled employees hide more knowledge? Int. J. Hosp. Manag. 89:102413. doi: 10.1016/j.ijhm.2019.102413

Khoreva, V., and Wechtler, H. (2020). Exploring the consequences of knowledge hiding: an agency theory perspective. J. Manag. Psychol. 35, 71–84. doi: 10.1108/JMP-11-2018-0514

Lazarus, R. S. (1991). Progress on a cognitive-motivational-relational theory of emotion. Am. Psychol. 46, 819–834. doi: 10.1037/0003-066X.46.8.819

Lee, K., and Allen, N. J. (2002). Organizational citizenship behavior and workplace deviance: the role of affect and cognitions. J. Appl. Psychol . 87, 131–142. doi: 10.1037/0021-9010.87.1.131

Lin, M., Zhang, X., Ng, B. C. S., and Zhong, L. (2020). To empower or not to empower? Multilevel effects of empowering leadership on knowledge hiding. Int. J. Hosp. Manag. 89:102540. doi: 10.1016/j.ijhm.2020.102540

Lin, T. C., and Huang, C. C. (2010). Withholding effort in knowledge contribution: the role of social exchange and social cognitive on project teams. Inf. Manag. 47, 188–196. doi: 10.1016/j.im.2010.02.001

Liu, Y., Zhu, J. N. Y., and Lam, L. W. (2020). Obligations and feeling envied: a study of workplace status and knowledge hiding. J. Manag. Psychol. 35, 347–359. doi: 10.1108/JMP-05-2019-0276

Malik, O. F., Shahzad, A., Raziq, M. M., Khan, M. M., Yusaf, S., and Khan, A. (2019). Perceptions of organizational politics, knowledge hiding, and employee creativity: the moderating role of professional commitment. Pers. Individ. Differ. 142, 232–237. doi: 10.1016/j.paid.2018.05.005

Men, C., Fong, P. S. W., Huo, W., Zhong, J., Jia, R., and Luo, J. (2020). Ethical leadership and knowledge hiding: a moderated mediation model of psychological safety and mastery climate. J. Bus. Ethics 166, 461–472. doi: 10.1007/s10551-018-4027-7

Moh'd, S. S., Cerne, M., and Zhang, P. (2021). An exploratory configurational analysis of knowledge hiding antecedents in project teams. Proj. Manag. J. 52, 31–44. doi: 10.1177/8756972820939768

Nadeem, M. A., Liu, Z., Ghani, U., Younis, A., and Xu, Y. (2021). Impact of shared goals on knowledge hiding behavior: the moderating role of trust. Manag. Decis . 59, 1312–1332. doi: 10.1108/MD-09-2019-1197

Offergelt, F., Spörrle, M., Moser, K., and Shaw, J. D. (2019). Leader-signaled knowledge hiding: effects on employees' job attitudes and empowerment. J. Organ. Behav. 40, 819–833. doi: 10.1002/job.2343

Pan, W., and Zhang, Q. (2018). Withholding knowledge in teams: an interactionist perspective of personality, justice, and autonomy. Soc. Behav. Pers. 46, 2009–2024. doi: 10.2224/sbp.7390

Pan, W., Zhang, Q., Teo, T. S. H., and Lim, V. K. G. (2018). The dark triad and knowledge hiding. J. Inf. Manag. 42, 36–48. doi: 10.1016/j.ijinfomgt.2018.05.008

Peng, H. (2013). Why and when do people hide knowledge? J. Knowl. Manag. 17, 398–415. doi: 10.1108/JKM-12-2012-0380

Pickering, C., and Byrne, J. (2014). The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early-career researchers. High. Educ. Res. Dev. 33, 534–548. doi: 10.1080/07294360.2013.841651

Pradhan, S., Srivastava, A., and Mishra, D. K. (2019). Abusive supervision and knowledge hiding: the mediating role of psychological contract violation and supervisor directed aggression. J. Knowl. Manag. 24, 216–234. doi: 10.1108/JKM-05-2019-0248

Riaz, S., Xu, Y., and Hussain, S. (2019). Workplace ostracism and knowledge hiding: the mediating role of job tension. Sustainability 11:5547. doi: 10.3390/su11205547

Semerci, A. B. (2019). Examination of knowledge hiding with conflict, competition and personal values. Int. J. Confl. Manag. 30, 111–131. doi: 10.1108/IJCMA-03-2018-0044

Serenko, A., and Bontis, N. (2016). Understanding counterproductive knowledge behavior: antecedents and consequences of intra-organizational knowledge hiding. J. Knowl. Manag. 20, 1199–1224. doi: 10.1108/JKM-05-2016-0203

Shen, X., Li, Y., Sun, Y., Chen, J., and Wang, F. (2019). Knowledge withholding in online knowledge spaces: social deviance behavior and secondary control perspective. J. Assoc. Inf. Sci. Technol. 70, 385–401. doi: 10.1002/asi.24192

Singh, S. K. (2019). Territoriality, task performance, and workplace deviance: empirical evidence on role of knowledge hiding. J. Bus. Res. 97, 10–19. doi: 10.1016/j.jbusres.2018.12.034

Škerlavaj, M., Connelly, C. E., Cerne, E., and Dysvik, A. (2018). Tell me if you can: time pressure, prosocial motivation, perspective taking, and knowledge hiding. J. Knowl. Manag. 22, 1489–1509. doi: 10.1108/JKM-05-2017-0179

Stenius, M., Hankonen, N., Ravaja, N., and Haukkala, A. (2016). Why share expertise? A closer look at the quality of motivation to share or withhold knowledge. J. Knowl. Manag. 20, 181–198. doi: 10.1108/JKM-03-2015-0124

Tranfield, D., Denyer, D., and Palminder, S. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br. J. Manag . 14, 207–222. doi: 10.1111/1467-8551.00375

Tsay, C. H. H., Lin, T. C., Yoon, J., and Huang, C. C. (2014). Knowledge withholding intentions in teams: the roles of normative conformity, affective bonding, rational choice and social cognition. Decis. Support Syst. 67, 53–65. doi: 10.1016/j.dss.2014.08.003

Vlačić, B., Corbo, L., Costa e Silva, S., and Dabi,ć, M. (2021). The evolving role of artificial intelligence in marketing: a review and research agenda. J. Bus. Res. 128, 187–203. doi: 10.1016/j.jbusres.2021.01.055

Wang, Y., Han, M., Xiang, D., and Hampson, D. P. (2019). The double-edged effects of perceived knowledge hiding: empirical evidence from the sales context. J. Knowl. Manag. 23, 279–296. doi: 10.1108/JKM-04-2018-0245

Wang, Y. S., Lin, H. H., Li, C. R., and Lin, S. J. (2014). What drives students' knowledge-withholding intention in management education? An empirical study in Taiwan. Acad. Manag. Learn. Educ. 13, 547–568. doi: 10.5465/amle.2013.0066

Webster, J., Brown, G., Zweig, D., Connelly, C. E., Brodt, S., and Sitkin, S. (2008). Beyond knowledge sharing: withholding knowledge at work. Res. Pers. Hum. Resour. Manag. 27, 1–37. doi: 10.1016/S0742-7301(08)27001-5

Weng, Q., Latif, K., Khan, A. K., Tariq, H., Butt, H. P., Obaid, A., et al. (2020). Loaded with knowledge, yet green with envy: leader–member exchange comparison and coworkers-directed knowledge hiding behavior. J. Knowl. Manag. 24, 1653–1680. doi: 10.1108/JKM-10-2019-0534

Wu, D. (2020). Empirical study of knowledge withholding in cyberspace: integrating protection motivation theory and theory of reasoned behavior. Comput. Hum. Behav. 105:106229. doi: 10.1016/j.chb.2019.106229

Xia, Q., Yan, S., Zhang, Y., and Chen, B. (2019). The curvilinear relationship between knowledge leadership and knowledge hiding: the moderating role of psychological ownership. Leadersh. Organ. Dev. J. 40, 669–683. doi: 10.1108/LODJ-10-2018-0362

Xiao, M., and Cooke, F. L. (2019). Why and when knowledge hiding in the workplace is harmful: a review of the literature and directions for future research in the Chinese context. Asia Pac. J. Hum. Resour. 57, 470–502. doi: 10.1111/1744-7941.12198

Yao, Z., Luo, J., and Zhang, X. (2020a). Gossip is a fearful thing: the impact of negative workplace gossip on knowledge hiding. J. Knowl. Manag. 24, 1755–1775. doi: 10.1108/JKM-04-2020-0264

Yao, Z., Zhang, X., Luo, J., and Huang, H. (2020b). Offense is the best defense: the impact of workplace bullying on knowledge hiding. J. Knowl. Manag. 24, 675–695. doi: 10.1108/JKM-12-2019-0755

Zhai, X., Wang, M., and Ghani, U. (2020). The SOR (Stimulus-Organism-Response) paradigm in online learning: an empirical study of students' knowledge hiding perceptions. Interact. Learn. Environ. 28, 586–601. doi: 10.1080/10494820.2019.1696841

Zhang, Z., and Min, M. (2019). The negative consequences of knowledge hiding in NPD project teams: the roles of project work attributes. Int. J. Constr. Proj. Manag. 37, 225–238. doi: 10.1016/j.ijproman.2019.01.006

Zhao, H., Liu, W., Li, J., and Yu, X. (2019). Leader–member exchange, organizational identification, and knowledge hiding: the moderating role of relative leader–member exchange. J. Organ. Behav. 40, 834–848. doi: 10.1002/job.2359

Zhao, H., and Xia, Q. (2019). Nurses' negative affective states, moral disengagement, and knowledge hiding: the moderating role of ethical leadership. J. Nurs. Manag. 27, 357–370. doi: 10.1111/jonm.12675

Zhao, H., Xia, Q., He, P., Sheard, G., and Wan, P. (2016). Workplace ostracism and knowledge hiding in service organizations. Int. J. Hosp. Manag. 59, 84–94. doi: 10.1016/j.ijhm.2016.09.009

Zhu, Y., Chen, T., Wang, M., Jin, Y., and Wang, Y. (2019). Rivals or allies: how performance-prove goal orientation influences knowledge hiding. J. Organ. Behav. 40, 849–868. doi: 10.1002/job.2372

Keywords: knowledge hiding, systematic literature review, future research directions, content analysis, bibliometric analysis, descriptive analysis

Citation: He P, Jiang C, Xu Z and Shen C (2021) Knowledge Hiding: Current Research Status and Future Research Directions. Front. Psychol. 12:748237. doi: 10.3389/fpsyg.2021.748237

Received: 27 July 2021; Accepted: 05 October 2021; Published: 29 October 2021.

Reviewed by:

Copyright © 2021 He, Jiang, Xu and Shen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Zhixing Xu, xuzhixing@bnu.edu.cn ; Chuangang Shen, psychshen@hqu.edu.cn

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Pharmacokinetics and Drug Interactions in the Elderly and Special Issues in Elderly African-American Populations: Workshop Summary (1997)

Chapter: 3 conclusions and future research directions, conclusions and future research directions.

The ongoing study of pharmacokinetics, pharmacodynamics, and drug interactions in elderly persons is critical for the development of safe and effective therapies and for the prevention of drug toxicities and adverse drug reactions. Aging is associated with an increase in chronic illness and anatomical and physiological changes that affect drug distribution, metabolism, and excretion. Thus, as the number of older Americans increases, it can be expected that polypharmacy in this population will have significant health, social, and economic consequences. Additionally, research should focus on alleviating the disease burden in elderly minority populations.

Box 3.1 summarizes the committee's conclusions regarding future directions for research in this field. The remainder of this chapter provides a more detailed discussion of the committee's conclusions.

RESEARCH NEEDS AND OPPORTUNITIES

Expanding the scientific knowledge base.

Although progress has been made in understanding the aging process, there is still a paucity of data at the intracellular, organ, system, and population levels. The impact of aging on cells and organ systems has commonly been studied in isolation; however, a more integrated approach is needed that will examine the effects of aging on the body. Pharmacokinetic and pharmacodynamic models need to be developed that encompass the entire range of changes occurring at multiple levels throughout the body.

The following list highlights specific areas of research that would add to the body of knowledge and clarify our understanding of the aging process especially with regard to improving pharmacotherapy. This list is by no means comprehensive, as numerous research avenues could yield important information on the impact of pharmacotherapy and drug interactions in the elderly. Areas for future research include the impact of aging, gender, genetics, and ethnicity on physiology and metabolic processes. Specifically,

age-related changes in cellular transport mechanisms and extrahepatic metabolism and transport including the activity of different enzyme isoforms;

biomarkers of drug exposure;

mechanisms that cause variable responses to medications in aging racial and ethnic populations;

age-related hormonal changes affecting drug metabolism or drug sensitivity;

the impact of nutrition on the aging process;

mechanisms underlying diseases prevalent in the elderly (e.g., hypertension, diabetes, osteoporosis, and Parkinson's and Alzheimer's disease);

in vitro models for multiple drug regimens and multiple drug interactions that may be predictive of and correlated with in vivo research;

models for drug interaction related to altered reflex activity and changing homeostatic mechanisms;

the potential beneficial and adverse health effects of nutraceuticals; and

social and psychological aspects of medication use in the elderly (e.g., access to medications, adherence to prescription regimens), with a special emphasis on minority populations.

Addressing Issues in Minority Populations

Many diseases are disproportionately prevalent in elderly African-American and other minority populations. The causes and implications of this excess burden need to be more completely understood and addressed. For example, hypertension and the impact of antihypertensive medications in elderly African Americans have not been fully studied even though the morbidity and mortality is higher in this population than in other segments of the aging population.

Research is needed on multiple levels (molecular, cellular, system, population) to clarify the effect of race and ethnicity on disease prevalence and on variations in the effectiveness of pharmacotherapeutic and other treatment interventions. Such research would be a valuable tool in increasing our understanding of the physiology of aging for all populations and may have implications for pharmacotherapies aimed at various elderly groups. Research on diseases and health conditions that primarily affect minority elderly populations needs to be a priority to alleviate the disease burden experienced by these populations.

Recruitment of Elderly Patients into Clinical Trials

In 1989, the FDA published a guideline for the inclusion of elderly persons in clinical trials (FDA, 1989). However, a number of characteristics of the elderly population may present barriers to conducting clinical trials that are representative of this population. Studies need to include the oldest segment of the population (see Chapter 2 ). In addition, subgroups of the elderly population should be stratified based primarily on their functional status and disease burden and less on their chronological age.

Recruiting minorities for inclusion in studies should be a priority, although it is important to recognize the trends toward multiracial backgrounds and the complexities associated with categorizing race or ethnicity. The workshop speakers presented many innovative ideas about increasing the recruitment of minority populations. The committee supports a number of approaches, including providing transportation, involving the minority community, providing extensive patient education, and decentralizing clinical trials (i.e.., going to patients' homes or to community centers to provide and assess treatment). In addition, collaborative efforts and consortia need to be strengthened between historically minority and other academic institutions. These partnerships will be vital to recruiting minority investigators and to attracting and sustaining minority students in research programs. Further, patient recruitment efforts can draw on the populations available to both institutions.

Obtaining informed consent in elderly populations involves complex issues that need to be addressed including the extent of dementia or cognitive impairment in some elderly patients and their vulnerability to coercion. Informed consent forms have evolved into highly technical legal documents, and a reevaluation of how to best meet their original purpose is needed. Other ethical issues that need to be addressed include studies on vulnerable populations (e.g., nursing home residents) and the confidentiality of patient information.

Research Methodologies and Tools

Trials of acute drug use are well funded; however, there are few long-term studies that examine chronic effects and drug interactions. Inasmuch as elderly persons are living longer and may take the same medications for many years, increased postmarketing surveillance is needed to examine the effects of long-term use of drugs. Incentives to strengthen postmarketing surveillance should be considered. Some of these drugs (e.g., hormone replacement therapy, antidepressants, and lipid-lowering medications) may be used as preventive measures (e.g., treating high cholesterol levels in the absence of cardiovascular disease or prescribing hormone replacement therapy to prevent hip fractures); however, their long-term health effects are not fully known. Further, the pharmacodynamics of many of these medications are only beginning to be investigated.

Research Methodologies

Studying the impact of pharmacotherapy on the elderly population is often difficult from a methodological standpoint. Cross-sectional studies are problematic because confounding variables abound among the elderly, and it is difficult to distinguish the effects of aging from those of disease. Randomized clini-

cal trials often recruit study subjects who represent the younger segments of the elderly population, and who have fewer comorbid conditions, use fewer medications, and may be more compliant in terms of following prescription medication regimens. In addition, many studies use small numbers of patients, frequently with homogenous geographic and ethnic backgrounds. Longitudinal studies are needed that involve large numbers of patients who reflect the diversity of “real world” populations. Furthermore, studies of elderly populations should include observational studies, case-control studies, and cohort studies to take advantage of the realm of methodological approaches that are available. Studies of optimal pharmacotherapies need to consider their cost-effectiveness and delivery. Outcome measures must also be reexamined, and quality-of-life outcomes need to be considered. One workshop speaker recalled the adage that, “adding life to years is at least as important as adding years to life.”

Databases Available for Research

There is a notable lack of adequate databases to research the prevalence and health impact of adverse drug reactions in elderly populations. Prescription information on elderly persons is not currently linked to diagnostic information or health outcomes data. For example, state Medicaid databases are used to reimburse pharmacies for prescriptions, therefore, the data on medication utilization are quite complete and accurate. However, diagnostic information for outpatient care is often incomplete or unavailable, and is not linked to pharmacy utilization databases.

Current changes underway in the health care delivery system may provide opportunities for new databases to be developed, although there are concerns that these changes may instead result in the loss of publicly available data. Trends of interest include the purchase of pharmacy benefit companies by pharmaceutical manufacturers and the increased use of managed care through proprietary health plans paid for by Medicaid and Medicare. Increased use of managed care to provide health care for the elderly offers opportunities for databases to be implemented that would link health outcomes (particularly adverse drug reactions) and prescription information, while paying close attention to patient confidentiality issues. However, these changes may instead be implemented to restructure datasets and require new levels of approval for data use or publishing. It is crucial that the larger issues involving potential censoring or loss of publicly available data on prescription drug use and health outcomes be addressed. The increased privatization of health care services for the elderly may lead to barriers to accessing datasets due to proprietary and competitive interests.

An area of interest to the committee is exploring the feasibility of developing a cooperative national data resource that would expand the researcher's

ability to examine population-based data rather than utilizing a piecemeal approach to data collection. This data resource could include information on diagnosis, medications prescribed, clinical interventions, health outcomes, and other relevant data. Of utmost importance would be maintaining patient confidentiality. This resource could be utilized as a repository to which individual researchers could submit peer-reviewed and approved research questions. Examining the feasibility of developing such a data resource would require the input of patients, health care providers, researchers, ethicists, and other interested persons and groups.

Dissemination of Information

Drug-related information can be complex, and information overload is a common phenomenon among health professionals and patients. Because information regarding drug use and interactions changes rapidly, information systems should be available to health care professionals that can provide up-to-date information that is unbiased, case specific, interactive, and readily accessible. In addition, it is important to develop diverse information dissemination strategies to effectively meet the needs of the heterogeneous elderly population. The information provided should be presented in a manner that can be understood by patients who cannot necessarily process complex information, yet need to make informed decisions and understand their options for treatment. Private- and public-sector initiatives are required to address this critical challenge. Health sciences centers that focus on training medical, nursing, and pharmacy students should be used to conduct independent, unbiased research and continuing education programs for practicing physicians, pharmacists, nurses, and the interested public (Woosley, 1994).

Capacity Building: Researchers and Clinicians

One of the major factors limiting the expansion of research in the area of geriatrics, and particularly geriatric pharmacology and clinical therapeutics, is the small number of health professionals entering this field. This is an area in which, as demographers can attest, the patient base is expanding and will continue to grow. Quality geriatric care depends on the development of multidisciplinary teams (including nursing, physical and occupational therapy, social work) to assess the concomitant problems and implement multiple interventions. However, reimbursements do not adequately cover the time required to handle the complexity of geriatric care. A recent report by the Alliance for Aging Research (1996) found that the United States has less than one fourth the number

of academic physician-scientists needed in geriatrics to teach and conduct research.

There is a pressing need to develop innovative approaches for recruiting and retaining researchers and clinicians. Programs are needed at many points along the career path, beginning early in the college and postbaccalaureate years to kindle an interest in the field of geriatrics and continuing throughout the professional years to retain the best investigators and clinicians available. Recruitment of minority investigators should be included within broader programs supporting young and mid-career investigators. The options available for approaching this issue include

1- to 2-year postbaccalaureate programs to provide research and clinical experience to young people considering a career in this field;

opportunities for medical, nursing, pharmacy, and other health professional students to have additional exposure to geriatric treatment and research during their education;

collaborative efforts between minority academic institutions and academic health sciences centers to encourage minority students to pursue a research program in this field;

loan-forgiveness programs to assist young researchers with high debt loads from health professional or graduate schools;

new sources of fellowships (e.g., through FDA, pharmaceutical companies, or insurance companies);

increased commitment to funding from fellowship training to first awards to independent grant support;

merit awards at the midcareer level and specialized sabbaticals to retrain midcareer professionals; and

retraining in research methodologies during sabbaticals for midcareer level geriatricians.

Currently there is only a limited understanding of the impact of aging on pharmacokinetics, pharmacodynamics, and drug interactions. Research is needed at the molecular, cellular, organ, system, and population levels for safer and more effective medications to be developed, delivered, and utilized by elderly persons. In addition, attention must be given to understanding and alleviating the disproportionate disease burden in elderly African-American and other minority populations.

The committee's major conclusions are summarized in Box 3.1 at the beginning of this chapter. The committee discussed and reflected only on the workshop presentations and acknowledges that there are numerous research

opportunities in geriatric pharmacology that need to be explored. Research in geriatric pharmacology and clinical therapeutics will require a commitment to fund studies that can further elucidate the relationship between pharmacokinetics and adverse drug interactions in the elderly and the complex individual variability of the aging process. Increasing the knowledge base will enable more effective therapeutic interventions and improved quality of life for the growing population of elderly persons.

Alliance for Aging Research. 1996. Will You Still Treat Me When I'm 65? The National Shortage of Geriatricians . Washington, DC: Alliance for Aging Research.

FDA (Food and Drug Administration). 1989. Guideline for the Study of Drugs Likely to Be Used in the Elderly. Rockville, MD: FDA Center for Drug Evaluation and Research.

Woosley RL. 1994. Centers for education and research in therapeutics. Clinical Pharmacology and Therapeutics 56(6 Part 1):693–697.

Reports in the popular press about the increasing longevity of Americans and the aging of the baby boom generation are constant reminders that the American population is becoming older. Consequently, an issue of growing medical, health policy, and social concern is the appropriate and rational use of medications by the elderly.

Although becoming older does not necessarily correlate with increasing illness, aging is associated with anatomical and physiological changes that affect how medications are metabolized by the body. Furthermore, aging is often related to an increased frequency of chronic illness (often combined with multiple health problems) and an increased use of medications. Thus, a better understanding of the absorption, distribution, metabolism, and excretion of drugs; of the physiologic responses to those medications; as well as of the interactions among multiple medications is crucial for improving the health of older people.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Donaldson MS, Mohr JJ; Institute of Medicine (US). Exploring Innovation and Quality Improvement in Health Care Micro-Systems: A Cross-Case Analysis. Washington (DC): National Academies Press (US); 2001.

Exploring Innovation and Quality Improvement in Health Care Micro-Systems: A Cross-Case Analysis.

• Hardcopy Version at National Academies Press

CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH AND POLICY

• Limitations of This Research

There are limitations to all sampling strategies and to qualitative research, in particular. The strength of this method was that the sample selection used input from a pool of reognized experts in the organization, delivery, and improvement of health care. Even with a pool of recognized experts, it is reasonable to expect that some high performing micro-systems were overlooked. It was also possible that less than high performing micro-systems were included. In fact, a concern was how to ensure that the micro-systems included in the study were high performing or successful micro-systems, and probes were included in the interview to assess what evidence micro-systems might offer to validate statements about their level of performance. We did not, however, seek validation from documents or other written materials. Although the intent of the sampling strategy was to study high performing micro-systems, a very small number of apparently negative cases were useful for comparison. More importantly, as expected, each site had some areas of very strong performance and other areas that were undistinguished, and they formed a natural cross-case comparison group. Although the sites were selected because of expert opinion, the database is limited by being self report. It is possible that the leaders of the micro-systems had an interest in making their micro-system appear to be better than it is, and we did not have any independent verification of their assertions. For this reason, we did not make any judgments about the validity of respondents' assertions and have limited the analysis to descriptive summaries and themes based on the respondents' own words.

TABLE 18 Micro-System Examples of Investment in Improvement

View in own window

TABLE 19 Micro-System Examples of Alignment of Role and Training

A second limitation of this study was that the interviews were not tape-recorded to provide a raw data “gold standard” for later reference. For this reason, we went to considerable effort to ensure the quality of note taking as described in the methods section, and we obtained respondents' consent to follow-up with them to clarify notes. Follow-up was necessary in only a few instances. The notes were voluminous and rich in detail.

A third limitation is that for most of the interviews, one respondent represented each of the forty-three micro-systems. A more comprehensive assessment would include interviews with at least one person from each of the key roles within the micro-system, including patients. Such tradeoffs in qualitative analysis between breadth and depth are inevitable, 31 but given that this was an exploratory study, we decided to include as many micro-systems as possible with follow-up in later studies.

Research currently underway will expand on this work by taking a more comprehensive look at individual micro-systems and the outcomes of care provided to determine if high performing micro-systems achieve superior results for patients.

• Directions for Further Research

This research has been exploratory in that it is the first systematic look at health care micro-systems. The power of the research is that it gave a voice to individual micro-systems and provided a way to explore them while creating constructs that may be generalizable to other micro-systems. It has begun the work of defining and characterizing health care micro-systems. The greater value of this analysis will be to go beyond the findings of this research to develop tools to help existing micro-systems improve and to replicate and extend the achievements of these micro-systems.

The basic concept of health care micro-systems—small, organized groups of providers and staff caring for a defined population of patients—is not new. The key components of micro-systems (patients, populations, providers, activities, and information technology) exist in every health care setting. However, current methods for organizing and delivering health care, preparing future health professionals, conducting health services research, and formulating policy have made it difficult to recognize the interdependence and function of the micro-system.

Further analysis of the database would likely yield additional themes. All can be the basis of hypothesis testing for continued work. For example, further work might establish criteria of effectiveness and test whether the features identified as the eight themes are predictive of effectiveness. More refined or additional questions might clarify aspects of the general themes that are critical. More intensive data gathering, for example, of multiple members of the micro-system, including patients could validate results and expand our understanding of these micro-systems.

Two questions were central as we undertook this study: (1) would the term micro-system be meaningful to clinicians in the field? (2) Would they participate and give us detailed enough information to draw inferences? The answers to both questions were clearly: Yes.

Overall, we discovered that the idea of a micro-system was very readily understood by all we interviewed. They had no difficulty in identifying and describing their own micro-systems and, when appropriate because they directed several (such as several intensive care units), differentiating among them in terms of their characteristics.

The study was assisted in its work by an extremely able and distinguished steering group and Subcommittee whose reputations in the field unquestionably enabled us to secure the participation of nearly all who were invited despite our requesting an hour and a half of a busy clinician's time. Many of those interviewed willingly went on for a longer than the allotted 90 minutes and sent us additional materials. Some who were interrupted by urgent clinical business rescheduled time to complete the interviews.

Although this was a selected—not a randomly sampled—group, and there was clearly great enthusiasm and of innovative work going on at the grass-roots level. Many of those interviewed expressed clear ideas about how they were reorganizing practices, their principles for doing so, and their commitment to an ongoing process. Respondents described their early limited successes or outright failures. We heard what had and had not been successful as they tried to disseminate their practices throughout their organizations. We believe there is much that could profitably learned and shared beyond the individual sites that has not been yet been pulled together by a unifying conceptual framework or effective mechanism for deploying what is being learned.

We were struck by two findings in particular: First, the importance of leadership at the macro-system as well as clinical level; and second, the general lack of information infrastructure in these practices. Micro-system leaders repeatedly stressed the importance of executive and governance-level support. This support was singled out repeatedly as a sine qua non to their ability to succeed. It was also apparent that although some steps have been taken to incorporate the explosion of information technologies that are being deployed for managing patient information, free-standing practices as well as much of clinical practice within hospitals have only begun to integrate data systems, use them for real-time clinical practice, or as information tools for improving the quality of care for a patient population. The potential is enormous, but as yet, almost untapped. They appear to be at a threshold of incorporating information technologies into daily practice. The potential created by the development of knowledge servers, decision support tools, consumer informatics 32 continuous electronic patient-clinician communication, and computer-based electronic health records puts most of these micro-systems almost at “time zero” for what will likely be dramatic changes in the integration of information for real-time patient care and a strong baseline for future comparison.

As research on micro-systems moves forward, it will be important to transfer what has been learned from research on teams and organizations to new research that will be conducted on micro-systems. For example, research that will be helpful includes information about the different stages of development and maturity of the organization, creating the organizational environment to support teams, socializing new members (clinicians and staff) to the team, environments that support micro-systems, the characteristics of effective leadership, and how micro-systems can build linkages that result in well-coordinated care within and across organizational boundaries.

• IOM Quality of Care Study

This study was intended to provide more than a database for research, however. It was undertaken to provide an evidence base for the IOM Committee on the Quality of Health Care in America in formulating its conclusions and recommendations. Because that committee was charged with the formulation of recommendations about changes that can lead to threshold improvement in the quality of care in this country, its members believed that it was extremely important to draw not only on their expertise and the literature but also on the best evidence it could find of excellent performance and to do so in a systematic way as exemplified by this study. As that study was not limited by type of health care, the goals of such a project necessitated drawing from a wide range of sites serving a variety of patient populations. It also suggests a sample size that for qualitative analytic methods was quite broad but not unwieldy. The number of sites interviewed—43—served these purposes well. We had several of each “kind” of micro-system (e.g., primary care, critical care) but they varied in location, composition, and in their own approaches to organizing and delivering care, thus providing a very rich database of observation. That report, which is expected to be published in early 2001, will use the responses and analysis described in this technical report to underpin its recommendations about how health care micro-systems, macro-systems, and other organizational forms that have not yet emerged, can improve their performance.

• Cite this Page Donaldson MS, Mohr JJ; Institute of Medicine (US). Exploring Innovation and Quality Improvement in Health Care Micro-Systems: A Cross-Case Analysis. Washington (DC): National Academies Press (US); 2001. CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH AND POLICY.
• PDF version of this title (465K)

In this Page

Recent activity.

• CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH AND POLICY - Exploring Innovatio... CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH AND POLICY - Exploring Innovation and Quality Improvement in Health Care Micro-Systems

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Building The ‘Bridge’ Between Research and Practice

• Posted May 20, 2024
• By Ryan Nagelhout

The way Doug Mosher tells the story, he didn't really come to the Harvard Graduate School of Education. HGSE came to him.

Mosher, Ph.D.’24, was working as a first-grade teacher at an underperforming elementary school in Nashville when a consultant came to introduce what he describes as “an awesome vocabulary intervention.”

The consultant, Claire White, Ed.M.’99, Ed.D.’05, was an Ed School alum whose goal was to help third- and fourth-grade students improve their language skills and reading vocabulary by discussing “controversial topics that are engaging,” says Mosher. White had worked with HGSE Professor Catherine Snow on the project and was now applying it in the field.

At first, his colleagues were reluctant to try something new, but Mosher was intrigued, and worked with White to modify the word generation lessons for his younger students. It was a “chance” brush with putting academic research into practice that changed the trajectory of his entire life.

“I feel so lucky to have been in that position,” says Mosher, a Ph.D. marshal for the HGSE class of 2024. “It just seemed fun, and I was at a point where I was looking for some new ideas to try in the classroom and this just seemed awesome.”

Mosher dove into the project for the next three years, helping White track student performance, collect data, and build lesson plans that he used in his own classroom. The program saw positive results, and soon the vocabulary intervention was implemented in other classrooms in the school. Mosher said he learned a lot, first and foremost that he really enjoyed doing academic research. And so when White told Mosher he could earn his doctorate doing this kind of work at HGSE – and maybe even get paid to do it – he was intrigued.

“I was just so excited about research. Having questions and designing things and then testing them out,” says Mosher. “I thought I was going to be a teacher forever. But I was starting to burn out. I was working really long hours. It’s a lot of pressure at an underperforming school to turn it around, and a lot of excitement. But at the same time, I was thinking I have to go back to school eventually.”

Teaching wasn’t exactly Mosher’s first love. A professional saxophonist, Mosher started substitute teaching when he moved to Nashville in the early 2000s. He learned to love the classroom, though, finding that same rush of energy and excitement he’d also experienced performing on stage.

Mosher applied to HGSE, particularly interested in the vocabulary research being done by Professor James Kim at the READS Lab, where he now conducts his own research. The three-part dissertation he defended this spring is a capstone of sorts, what Mosher describes as a shifting of his purpose in life.

“It’s been fun to see my true passion shift more toward research and working with schools and districts,” says Mosher. “Music will always be a part of my life, but I feel like this is my purpose now.”

That shift has changed how he views teachers, too. The learning environment at HGSE, he explains, is a big departure from the stereotypical music teacher myth that a “cold” and “suffering” teacher gets the most out of their students. Mosher called the faculty “a warm safety blanket” that created a welcoming learning environment over the last six years.

“It’s kind of what we try to do in intervention research,” says Mosher. “Create lessons that are engaging, build interest, build knowledge, make connections. That’s what all the faculty do.”

With Kim and the READS Lab, Mosher has worked on projects to improve reading comprehension in elementary school students using its Model of Reading Engagement (MORE) program. The project recently received a grant from the U.S. Department of Education to scale that model for use in new school districts. Mosher, always looking for chances to connect back with the classroom, describes the work as building “the bridge over the gap” that often exists between research and practice.

“Doug's exceptional research program shows how small improvements in the quality of teachers’ talk can have a big impact on students’ ability to read challenging science and social texts with greater understanding and engagement,” says Kim.

The work has certainly been noticed by the members of his cohort as well. Mosher calls his nomination to be a Ph.D. marshal “out of the blue.” He recalls the initial anxiety of joining a group of talented educators with experience working in so many impressive fields before arriving at HGSE. To be recognized by them, he says, reflects the support he’s felt from the community.

“I’m just very honored and touched that they voted me as a marshal,” says Mosher. “The cohort I’m in is full of really awesome, interesting, passionate people who are really dedicated to their areas of study. I was very surprised, but touched and honored.”

Mosher noted the difficulties his cohort experienced over the last six years, including a pandemic that disrupted research and entire ways of life. While some classmates moved away for good, Mosher doesn’t see himself leaving anytime soon.

“It feels like home,” says Mosher, whose father grew up in New England and has seen more family move to the area in recent years as well. “It's a really exciting thing to live in a place where I’ve always wanted to be. I finally ended up here and I don’t really want to leave.”

Mosher’s former school in Nashville, by the way, is now thriving. And here in Cambridge, so is he.

The latest research, perspectives, and highlights from the Harvard Graduate School of Education

Related Articles

Improving the Teacher Workforce

With her research work, doctoral marshal Mary Laski, Ph.D.'24, is trying to make teaching in K–12 schools more sustainable and attractive

Paxton, Maheshwari-Kanoria to Receive 2024 Alumni Council Awards

Alums will be honored for their educational contributions at HGSE Convocation

Exploring the Effects of Desegregation

Limitations and Future Research Directions

• First Online: 30 August 2022

Cite this chapter

• Christina Niklaus 2  

73 Accesses

In the following, important short-term research challenges which became evident during the elaboration of this work are summarized, opening directions for further research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Author information

Authors and affiliations.

Rehetobel, Switzerland

Christina Niklaus

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature

About this chapter

Niklaus, C. (2022). Limitations and Future Research Directions. In: From Complex Sentences to a Formal Semantic Representation using Syntactic Text Simplification and Open Information Extraction. Springer Vieweg, Wiesbaden. https://doi.org/10.1007/978-3-658-38697-9_18

Download citation

DOI : https://doi.org/10.1007/978-3-658-38697-9_18

Published : 30 August 2022

Publisher Name : Springer Vieweg, Wiesbaden

Print ISBN : 978-3-658-38696-2

Online ISBN : 978-3-658-38697-9

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

wn0sdwk000IM1

• Emboldened and disciplined exploration – results from our The Future of Exploration 2024 Survey

Emboldened and disciplined exploration: results from our Future of Exploration 2024 Survey

Explore the findings of this year’s survey

1 minute read

• LinkedIn LinkedIn Share on LinkedIn
• Facebook Facebook Share on Facebook
• Twitter Share on Twitter
• Email Share by email

Julie Wilson

Research Director, Global Exploration

Latest articles by Julie

Subsurface predictions: what to watch in exploration and discovered resources in 2024, global exploration: 4 things to look for in 2024, can colombia’s gas sustain it through the energy transition, where next for oil & gas exploration, climbing the oil and gas ladder: four lessons from women in exploration leadership.

David Parkinson

Vice President, Upstream Consulting

David leads our exploration strategy team and is also responsible for exploration consulting.

Latest articles by David

Three critical factors to developing africa’s oil and gas resources, can carbon offset crude move from idea to reality, what is the future of oil and gas exploration.

Blair Stryke

Research Analyst, Subsurface

Latest articles by Blair

The Future of Exploration 2024 Survey takes the pulse of the exploration sector and helps you better understand its key issues and challenges. In a recent webinar, we compared this year's responses to last year’s results, allowing you to follow trends and shifts in attitudes.

Stream the webinar to watch our discussion of:

• How internal and external pressures on the exploration sector are shifting – and why.
• Who are the contenders for Wood Mackenzie’s coveted exploration awards?
• Why exploration is both emboldened and disciplined.

Our survey reveals the key challenges the industry is facing, which explorers stand out from the pack and the role of exploration in today’s energy company.

Fill in the form at the top of the page to stream the webinar on demand and access a deep dive into key survey findings. 

Related content, future of exploration survey results 2024: explorers emboldened and disciplined.

Our Survey respondents from across the globe reported that conventional exploration is a higher priority for resource capture than last year. These leading explorers report a significantly diminished impact of the energy transition on their exploration business against the backdrop of ongoing energy security concerns. The results show an intriguing dichotomy between rising internal support and falling social licence to operate. Capital discipline continues as explorers are asked to deliver more with less. We examine the top challenges explorers face, noting some interesting changes from last year. Majors continue to dominate the rankings of most-admired explorers and operate many of the most exciting discoveries of 2023 and highly-anticipated wells of 2024. Explorers are most excited about opportunities along the Atlantic Margin, while concern grows about policy shifts.

What's included

This report contains:

Future Of Exploration Survey Results 2024.pdf

PDF 1.35 MB

You can pay by card or invoice

Already have a subscription? Sign In

A two-decade decline in exploration is driving the need for carbon neutral investment in Australia’s upstream sector

Rebooting asia’s energy transition, hitting the brakes: how the energy transition could decelerate in the us, green steel: challenging the status quo .