latest research papers on e learning

Systematic Literature Review of E-Learning Capabilities to Enhance Organizational Learning

• Open access
• Published: 01 February 2021
• Volume 24 , pages 619–635, ( 2022 )

Cite this article

You have full access to this open access article

• Michail N. Giannakos 1 ,
• Patrick Mikalef 1 &
• Ilias O. Pappas   ORCID: orcid.org/0000-0001-7528-3488 1 , 2  

21k Accesses

32 Citations

Explore all metrics

E-learning systems are receiving ever increasing attention in academia, business and public administration. Major crises, like the pandemic, highlight the tremendous importance of the appropriate development of e-learning systems and its adoption and processes in organizations. Managers and employees who need efficient forms of training and learning flow within organizations do not have to gather in one place at the same time or to travel far away to attend courses. Contemporary affordances of e-learning systems allow users to perform different jobs or tasks for training courses according to their own scheduling, as well as to collaborate and share knowledge and experiences that result in rich learning flows within organizations. The purpose of this article is to provide a systematic review of empirical studies at the intersection of e-learning and organizational learning in order to summarize the current findings and guide future research. Forty-seven peer-reviewed articles were collected from a systematic literature search and analyzed based on a categorization of their main elements. This survey identifies five major directions of the research on the confluence of e-learning and organizational learning during the last decade. Future research should leverage big data produced from the platforms and investigate how the incorporation of advanced learning technologies (e.g., learning analytics, personalized learning) can help increase organizational value.

Similar content being viewed by others

Technology-Enhanced Organizational Learning: A Systematic Literature Review

Knowledge Transfer Through E-learning: Case of Tunisian Post

Organizational e-Learning Systems’ Success in Industry

Avoid common mistakes on your manuscript.

1 Introduction

E-learning covers the integration of information and communication technology (ICT) in environments with the main goal of fostering learning (Rosenberg and Foshay 2002 ). The term “e-learning” is often used as an umbrella term to portray several modes of digital learning environments (e.g., online, virtual learning environments, social learning technologies). Digitalization seems to challenge numerous business models in organizations and raises important questions about the meaning and practice of learning and development (Dignen and Burmeister 2020 ). Among other things, the digitalization of resources and processes enables flexible ways to foster learning across an organization’s different sections and personnel.

Learning has long been associated with formal or informal education and training. However organizational learning is much more than that. It can be defined as “a learning process within organizations that involves the interaction of individual and collective (group, organizational, and inter-organizational) levels of analysis and leads to achieving organizations’ goals” (Popova-Nowak and Cseh 2015 ) with a focus on the flow of knowledge across the different organizational levels (Oh 2019 ). Flow of knowledge or learning flow is the way in which new knowledge flows from the individual to the organizational level (i.e., feed forward) and vice versa (i.e., feedback) (Crossan et al. 1999 ; March 1991 ). Learning flow and the respective processes constitute the cornerstone of an organization’s learning activities (e.g., from physical training meetings to digital learning resources), they are directly connected to the psycho-social experiences of an organization’s members, and they eventually lead to organizational change (Crossan et al. 2011 ). The overall organizational learning is extremely important in an organization because it is associated with the process of creating value from an organizations’ intangible assets. Moreover, it combines notions from several different domains, such as organizational behavior, human resource management, artificial intelligence, and information technology (El Kadiri et al. 2016 ).

A growing body of literature lies at the intersection of e-learning and organizational learning. However, there is limited work on the qualities of e-learning and the potential of its qualities to enhance organizational learning (Popova-Nowak and Cseh 2015 ). Blockages and disruptions in the internal flow of knowledge is a major reason why organizational change initiatives often fail to produce their intended results (Dee and Leisyte 2017 ). In recent years, several models of organizational learning have been published (Berends and Lammers 2010 ; Oh 2019 ). However, detailed empirical studies indicate that learning does not always proceed smoothly in organizations; rather, the learning meets interruptions and breakdowns (Engeström et al. 2007 ).

Discontinuities and disruptions are common phenomena in organizational learning (Berends and Lammers 2010 ), and they stem from various causes. For example, organizational members’ low self-esteem, unsupportive technology and instructors (Garavan et al. 2019 ), and even crises like the Covid-19 pandemic can result in demotivated learners and overall unwanted consequences for their learning (Broadbent 2017 ). In a recent conceptual article, Popova-Nowak and Cseh ( 2015 ) emphasized that there is a limited use of multidisciplinary perspectives to investigate and explain the processes and importance of utilizing the available capabilities and resources and of creating contexts where learning is “attractive to individual agents so that they can be more engaged in exploring ways in which they can contribute through their learning to the ongoing renewal of organizational routines and practices” (Antonacopoulou and Chiva 2007 , p. 289).

Despite the importance of e-learning, the lack of systematic reviews in this area significantly hinders research on the highly promising value of e-learning capabilities for efficiently supporting organizational learning. This gap leaves practitioners and researchers in uncharted territories when faced with the task of implementing e-learning designs or deciding on their digital learning strategies to enhance the learning flow of their organizations. Hence, in order to derive meaningful theoretical and practical implications, as well as to identify important areas for future research, it is critical to understand how the core capabilities pertinent to e-learning possess the capacity to enhance organizational learning.

In this paper, we define e-learning enhanced organizational learning (eOL) as the utilization of digital technologies to enhance the process of improving actions through better knowledge and understanding in an organization. In recent years, a significant body of research has focused on the intersection of e-learning and organizational learning (e.g., Khandakar and Pangil 2019 ; Lin et al. 2019 ; Menolli et al. 2020 ; Turi et al. 2019 ; Xiang et al. 2020 ). However, there is a lack of systematic work that summarizes and conceptualizes the results in order to support organizations that want to move from being information-based enterprises to being knowledge-based ones (El Kadiri et al. 2016 ). In particular, recent technological advances have led to an increase in research that leverages e-learning capacities to support organizational learning, from virtual reality (VR) environments (Costello and McNaughton 2018 ; Muller Queiroz et al. 2018 ) to mobile computing applications (Renner et al. 2020 ) to adaptive learning and learning analytics (Zhang et al. 2019 ). These studies support different skills, consider different industries and organizations, and utilize various capacities while focusing on various learning objectives (Garavan et al. 2019 ). Our literature review aims to tease apart these particularities and to investigate how these elements have been utilized over the past decade in eOL research. Therefore, in this review we aim to answer the following research questions (RQs):

RQ1: What is the status of research at the intersection of e-learning and organizational learning, seen through the lens of areas of implementation (e.g., industries, public sector), technologies used, and methodologies (e.g., types of data and data analysis techniques employed)?

RQ2: How can e-learning be leveraged to enhance the process of improving actions through better knowledge and understanding in an organization?

Our motivation for this work is based on the emerging developments in the area of learning technologies that have created momentum for their adoption by organizations. This paper provides a review of research on e-learning capabilities to enhance organizational learning with the purpose of summarizing the findings and guiding future studies. This study can provide a springboard for other scholars and practitioners, especially in the area of knowledge-based enterprises, to examine e-learning approaches by taking into consideration the prior and ongoing research efforts. Therefore, in this paper we present a systematic literature review (SLR) (Kitchenham and Charters 2007 ) on the confluence of e-learning and organizational learning that uncovers initial findings on the value of e-learning to support organizational learning while also delineating several promising research streams.

The rest of this paper is organized as follows. In the next section, we present the related background work. The third section describes the methodology used for the literature review and how the studies were selected and analyzed. The fourth section presents the research findings derived from the data analysis based on the specific areas of focus. In the fifth section, we discuss the findings, the implications for practice and research, and the limitations of the selected methodological approach. In the final section, we summarize the conclusions from the study and make suggestions for future work.

2 Background and Related Work

2.1 e-learning systems.

E-learning systems provide solutions that deliver knowledge and information, facilitate learning, and increase performance by developing appropriate knowledge flow inside organizations (Menolli et al. 2020 ). Putting into practice and appropriately managing technological solutions, processes, and resources are necessary for the efficient utilization of e-learning in an organization (Alharthi et al. 2019 ). Examples of e-learning systems that have been widely adopted by various organizations are Canvas, Blackboard, and Moodle. Such systems provide innovative services for students, employees, managers, instructors, institutions, and other actors to support and enhance the learning processes and facilitate efficient knowledge flow (Garavan et al. 2019 ). Functionalities, such as creating modules to organize mini course information and learning materials or communication channels such as chat, forums, and video exchange, allow instructors and managers to develop appropriate training and knowledge exchange (Wang et al. 2011 ). Nowadays, the utilization of various e-learning capabilities is a commodity for supporting organizational and workplace learning. Such learning refers to training or knowledge development (also known in the literature as learning and development, HR development, and corporate training: Smith and Sadler-Smith 2006 ; Garavan et al. 2019 ) that takes place in the context of work.

Previous studies have focused on evaluating e-learning systems that utilize various models and frameworks. In particular, the development of maturity models, such as the e-learning capability maturity model (eLCMM), addresses technology-oriented concerns (Hammad et al. 2017 ) by overcoming the limitations of the domain-specific models (e.g., game-based learning: Serrano et al.  2012 ) or more generic lenses such as the e-learning maturity model (Marshall 2006 ). The aforementioned models are very relevant since they focus on assessing the organizational capabilities for sustainably developing, deploying, and maintaining e-learning. In particular, the eLCMM focuses on assessing the maturity of adopting e-learning systems and adds a feedback building block for improving learners’ experiences (Hammad et al. 2017 ). Our proposed literature review builds on the previously discussed models, lenses, and empirical studies, and it provides a review of research on e-learning capabilities with the aim of enhancing organizational learning in order to complement the findings of the established models and guide future studies.

E-learning systems can be categorized into different types, depending on their functionalities and affordances. One very popular e-learning type is the learning management system (LMS), which includes a virtual classroom and collaboration capabilities and allows the instructor to design and orchestrate a course or a module. An LMS can be either proprietary (e.g., Blackboard) or open source (e.g., Moodle). These two types differ in their features, costs, and the services they provide; for example, proprietary systems prioritize assessment tools for instructors, whereas open-source systems focus more on community development and engagement tools (Alharthi et al. 2019 ). In addition to LMS, e-learning systems can be categorized based on who controls the pace of learning; for example, an institutional learning environment (ILE) is provided by the organization and is usually used for instructor-led courses, while a personal learning environment (PLE) is proposed by the organization and is managed personally (i.e., learner-led courses). Many e-learning systems use a hybrid version of ILE and PLE that allows organizations to have either instructor-led or self-paced courses.

Besides the controlled e-learning systems, organizations have been using environments such as social media (Qi and Chau 2016 ), massive open online courses (MOOCs) (Weinhardt and Sitzmann 2018 ) and other web-based environments (Wang et al. 2011 ) to reinforce their organizational learning potential. These systems have been utilized through different types of technology (e.g., desktop applications, mobile) that leverage the various capabilities offered (e.g., social learning, VR, collaborative systems, smart and intelligent support) to reinforce the learning and knowledge flow potential of the organization. Although there is a growing body of research on e-learning systems for organizational learning due to the increasingly significant role of skills and expertise development in organizations, the role and alignment of the capabilities of the various e-learning systems with the expected competency development remains underexplored.

2.2 Organizational Learning

There is a large body of research on the utilization of technologies to improve the process and outcome dimensions of organizational learning (Crossan et al. 1999 ). Most studies have focused on the learning process and on the added value that new technologies can offer by replacing some of the face-to-face processes with virtual processes or by offering new, technology-mediated phases to the process (Menolli et al. 2020 ; Lau 2015 ) highlighted how VR capabilities can enhance organizational learning, describing the new challenges and frameworks needed in order to effectively utilize this potential. In the same vein, Zhang et al. ( 2017 ) described how VR influences reflective thinking and considered its indirect value to overall learning effectiveness. In general, contemporary research has investigated how novel technologies and approaches have been utilized to enhance organizational learning, and it has highlighted both the promises and the limitations of the use of different technologies within organizations.

In many organizations, alignment with the established infrastructure and routines, and adoption by employees are core elements for effective organizational learning (Wang et al. 2011 ). Strict policies, low digital competence, and operational challenges are some of the elements that hinder e-learning adoption by organizations (Garavan et al. 2019 ; Wang 2018 ) demonstrated the importance of organizational, managerial, and job support for utilizing individual and social learning in order to increase the adoption of organizational learning. Other studies have focused on the importance of communication through different social channels to develop understanding of new technology, to overcome the challenges employees face when engaging with new technology, and, thereby, to support organizational learning (Menolli et al. 2020 ). By considering the related work in the area of organizational learning, we identified a gap in aligning an organization’s learning needs with the capabilities offered by the various technologies. Thus, systematic work is needed to review e-learning capabilities and how these capabilities can efficiently support organizational learning.

2.3 E-learning Systems to Enhance Organizational Learning

When considering the interplay between e-learning systems and organizational learning, we observed that a major challenge for today’s organizations is to switch from being information-based enterprises to become knowledge-based enterprises (El Kadiri et al. 2016 ). Unidirectional learning flows, such as formal and informal training, are important but not sufficient to cover the needs that enterprises face (Manuti et al. 2015 ). To maintain enterprises’ competitiveness, enterprise staff have to operate in highly intense information and knowledge-oriented environments. Traditional learning approaches fail to substantiate learning flow on the basis of daily evidence and experience. Thus, novel, ubiquitous, and flexible learning mechanisms are needed, placing humans (e.g., employees, managers, civil servants) at the center of the information and learning flow and bridging traditional learning with experiential, social, and smart learning.

Organizations consider lack of skills and competences as being the major knowledge-related factors hampering innovation (El Kadiri et al. 2016 ). Thus, solutions need to be implemented that support informal, day-to-day, and work training (e.g., social learning, collaborative learning, VR/AR solutions) in order to develop individual staff competences and to upgrade the competence affordances at the organizational level. E-learning-enhanced organizational learning has been delivered primarily in the form of web-based learning (El Kadiri et al. 2016 ). More recently, the TEL tools portfolio has rapidly expanded to make more efficient joint use of novel learning concepts, methodologies, and technological enablers to achieve more direct, effective, and lasting learning impacts. Virtual learning environments, mobile-learning solutions, and AR/VR technologies and head-mounted displays have been employed so that trainees are empowered to follow their own training pace, learning topics, and assessment tests that fit their needs (Costello and McNaughton 2018 ; Mueller et al. 2011 ; Muller Queiroz et al. 2018 ). The expanding use of social networking tools has also brought attention to the contribution of social and collaborative learning (Hester et al. 2016 ; Wei and Ram 2016 ).

Contemporary learning systems supporting adaptive, personalized, and collaborative learning expand the tools available in eOL and contribute to the adoption, efficiency, and general prospects of the introduction of TEL in organizations (Cheng et al. 2011 ). In recent years, eOL has emphasized how enterprises share knowledge internally and externally, with particular attention being paid to systems that leverage collaborative learning and social learning functionalities (Qi and Chau 2016 ; Wang  2011 ). This is the essence of computer-supported collaborative learning (CSCL). The CSCL literature has developed a framework that combines individual learning, organizational learning, and collaborative learning, facilitated by establishing adequate learning flows and emerges effective learning in an enterprise learning (Goggins et al. 2013 ), in Fig.  1 .

Representation of the combination of enterprise learning and knowledge flows. (adapted from Goggins et al. 2013 )

Establishing efficient knowledge and learning flows is a primary target for future data-driven enterprises (El Kadiri et al. 2016 ). Given the involved knowledge, the human resources, and the skills required by enterprises, there is a clear need for continuous, flexible, and efficient learning. This can be met by contemporary learning systems and practices that provide high adoption, smooth usage, high satisfaction, and close alignment with the current practices of an enterprise. Because the required competences of an enterprise evolve, the development of competence models needs to be agile and to leverage state-of-the art technologies that align with the organization’s processes and models. Therefore, in this paper we provide a review of the eOL research in order to summarize the findings, identify the various capabilities of eOL, and guide the development of organizational learning in future enterprises as well as in future studies.

3 Methodology

To answer our research questions, we conducted an SLR, which is a means of evaluating and interpreting all available research relevant to a particular research question, topic area, or phenomenon of interest. A SLR has the capacity to present a fair evaluation of a research topic by using a trustworthy, rigorous, and auditable methodology (Kitchenham and Charters 2007 ). The guidelines used (Kitchenham and Charters 2007 ) were derived from three existing guides adopted by medical researchers. Therefore, we adopted SLR guidelines that follow transparent and widely accepted procedures (especially in the area of software engineering and information systems, as well as in e-learning), minimize potential bias (researchers), and support reproducibility (Kitchenham and Charters 2007 ). Besides the minimization of bias and support for reproducibility, an SLR allows us to provide information about the impact of some phenomenon across a wide range of settings, contexts, and empirical methods. Another important advantage is that, if the selected studies give consistent results, SLRs can provide evidence that the phenomenon is robust and transferable (Kitchenham and Charters 2007 ).

3.1 Article Collection

Several procedures were followed to ensure a high-quality review of the literature of eOL. A comprehensive search of peer-reviewed articles was conducted in February 2019 (short papers, posters, dissertations, and reports were excluded), based on a relatively inclusive range of key terms: “organizational learning” & “elearning”, “organizational learning” & “e-learning”, “organisational learning” & “elearning”, and “organisational learning” & “e-learning”. Publications were selected from 2010 onwards, because we identified significant advances since 2010 (e.g., MOOCs, learning analytics, personalized learning) in the area of learning technologies. A wide variety of databases were searched, including SpringerLink, Wiley, ACM Digital Library, IEEE Xplore, Science Direct, SAGE, ERIC, AIS eLibrary, and Taylor & Francis. The selected databases were aligned with the SLR guidelines (Kitchenham and Charters 2007 ) and covered the major venues in IS and educational technology (e.g., a basket of eight IS journals, the top 20 journals in the Google Scholar IS subdiscipline, and the top 20 journals in the Google Scholar Educational Technology subdiscipline). The search process uncovered 2,347 peer-reviewed articles.

3.2 Inclusion and Exclusion Criteria

The selection phase determines the overall validity of the literature review, and thus it is important to define specific inclusion and exclusion criteria. As Dybå and Dingsøyr ( 2008 ) specified, the quality criteria should cover three main issues – namely, rigor, credibility, and relevance – that need to be considered when evaluating the quality of the selected studies. We applied eight quality criteria informed by the proposed Critical Appraisal Skills Programme (CASP) and related works (Dybå and Dingsøyr 2008 ). Table 1 presents these criteria.

Therefore, studies were eligible for inclusion if they were focused on eOL. The aforementioned criteria were applied in stages 2 and 3 of the selection process (see Fig.  2 ), when we assessed the papers based on their titles and abstracts, and read the full papers. From March 2020, we performed an additional search (stage 4) following the same process for papers published after the initial search period (i.e., 2010–February 2019). The additional search returned seven papers. Figure 2 summarizes the stages of the selection process.

Stages of the selection process

3.3 Analysis

Each collected study was analyzed based on the following elements: study design (e.g., experiment, case study), area (e.g., IT, healthcare), technology (e.g., wiki, social media), population (e.g., managers, employees), sample size, unit of analysis (individual, firm), data collections (e.g., surveys, interviews), research method, data analysis, and the main research objective of the study. It is important to highlight that the articles were coded based on the reported information, that different authors reported information at different levels of granularity (e.g., an online system vs. the name of the system), and that in some cases the information was missing from the paper. Overall, we endeavored to code the articles as accurately and completely as possible.

The coding process was iterative with regular consensus meetings between the two researchers involved. The primary coder prepared the initial coding for a number of articles and both coders reviewed and agreed on the coding in order to reach the final codes presented in the Appendix . Disagreements between the coders and inexplicit aspects of the reviewed papers were discussed and resolved in regular consensus meetings. Although this process did not provide reliability indices (e.g., Cohen’s kappa), it did provide certain reliability in terms of consistency of the coding and what Krippendorff ( 2018 ) stated as the reliability of “the degree to which members of a designated community concur on the readings, interpretations, responses to, or uses of given texts or data”, which is considered acceptable research practice (McDonald et al. 2019 ).

In this section, we present the detailed results of the analysis of the 47 papers. Analysis of the studies was performed using non-statistical methods that considered the variables reported in the Appendix . This section is followed by an analysis and discussion of the categories.

4.1 Sample Size and Population Involved

The categories related to the sample of the articles and included the number of participants in each study (size), their position (e.g., managers, employees), and the area/topic covered by the study. The majority of the studies involved employees (29), with few studies involving managers (6), civil servants (2), learning specialists (2), clients, and researchers. Regarding the sample size, approximately half of the studies (20) were conducted with fewer than 100 participants; some (12) can be considered large-scale studies (more than 300 participants); and only a few (9) can be considered small scale (fewer than 20 participants). In relation to the area/topic of the study, most studies (11) were conducted in the context of the IT industry, but there was also good coverage of other important areas (i.e., healthcare, telecommunications, business, public sector). Interestingly, several studies either did not define the area or were implemented in a generic context (sector-agnostic studies, n = 10), and some studies were implemented in a multi-sector context (e.g., participants from different sections or companies, n = 4).

4.2 Research Methods

When assessing the status of research for an area, one of the most important aspects is the methodology used. By “method” in the Appendix , we refer to the distinction between quantitative, qualitative, and mixed methods research. In addition to the method, in our categorization protocol we also included “study design” to refer to the distinction between survey studies (i.e., those that gathered data by asking a group of participants), experiments (i.e., those that created situations to record beneficial data), and case studies (i.e., those that closely studied a group of individuals).

Based on this categorization, the Appendix shows that the majority of the papers were quantitative (34) and qualitative (7), with few studies (6) utilizing mixed methods. Regarding the study design, most of the studies were survey studies (26), 13 were case studies, and fewer were experiments (8). For most studies, the individual participant (40) was the unit of analysis, with few studies having the firm as the unit of analysis, and only one study using the training session as a unit of analysis. Regarding the measures used in the studies, most utilized surveys (39), with 11 using interviews, and only a few studies using field notes from focus groups (2) and log files from the systems (2). Only eight studies involved researchers using different measures to triangulate or extend their findings. Most articles used structural equation modeling (SEM) (17) to analyze their data, with 13 studies employing descriptive statistics, seven using content analysis, nine using regression analysis or analyses of variances/covariance, and one study using social network analysis (SNA).

4.3 Technologies

Concerning the technology used, most of the studies (17) did not study a specific system, referring instead in their investigation to a generic e-learning or technological solution. Several studies (9) named web-based learning environments, without describing the functionalities of the identified system. Other studies focused on online learning environments (4), collaborative learning systems (3), social learning systems (3), smart learning systems (2), podcasting (2), with the rest of the studies using a specific system (e.g., a wiki, mobile learning, e-portfolios, Second Life, web application).

4.4 Research Objectives

The research objectives of the studies could be separated into six main categories. The first category focuses on the intention of the employees to use the technology (9); the second focuses on the performance of the employees (8); the third focuses on the value/outcome for the organization (4); the fourth focuses on the actual usage of the system (7); the fifth focuses on employees’ satisfaction (4); and the sixth focuses on the ability of the proposed system to foster learning (9). In addition to these six categories, we also identified studies that focused on potential barriers for eOL in organizations (Stoffregen et al. 2016 ), the various benefits associated with the successful implementation of eOL (Liu et al. 2012 ), the feasibility of eOL (Kim et al. 2014 ; Mueller et al. 2011 ), and the alignment of the proposed innovation with the other processes and systems in the organization (Costello and McNaughton 2018 ).

4.5 E-learning Capabilities in Various Organizations and for Various Objectives

The technology used has an inherent role for both the organization and the expected eOL objective. E-learning systems are categorized based on their functionalities and affordances. Based on the information reported in the selected papers, we ranked them based on the different technologies and functionalities (e.g., collaborative, online, smart). To do so, we focused on the main elements described in the selected paper; for instance, a paper that described the system as wiki-based or indicated that the system was Second Life was ranked as such, rather than being added to collaborative systems or social learning respectively. We did this because we wanted to capture all the available information since it gave us additional insights (e.g., Second Life is both a social and a VR system).

To investigate the connection between the various technologies used to enhance organizational learning and their application in the various organizations, we utilized the coding (see Appendix ) and mapped the various e-learning technologies (or their affordances) with the research industries to which they applied (Fig.  3 ). There was occasionally a lack of detailed information about the capabilities of the e-learning systems applied (e.g., generic, or a web application, or an online system), which limited the insights. Figure 3 provides a useful mapping of the confluence of e-learning technologies and their application in the various industries.

Association of the different e-learning technologies with the industries to which they are applied in the various studies. Note: The size of the circles depicts the frequency of studies, with the smallest circle representing one study and the largest representing six studies. The mapping is extracted from the data in the Appendix , which outlines the papers that belong in each of the circles

To investigate the connection between the various technologies used to enhance organizational learning and their intended objectives, we utilized the coding of the articles (see Appendix ) and mapped the various e-learning technologies (or their affordances) with the intended objectives, as reported in the various studies (Fig.  4 ). The results in Fig.  4 show the objectives that are central in eOL research (e.g., performance, fostering learning, adoption, and usage) as well as those objectives on which few studies have focused (e.g., alignment, feasibility, behavioral change). In addition, the results also indicate the limited utilization of the various e-learning capabilities (e.g., social, collaborative, smart) to achieve objectives connected with those capabilities (e.g., social learning and behavioral change, collaborative learning, and barriers).

Association of the different e-learning technologies with the objectives investigated in the various studies. Note: The size of the circles depicts the frequency of studies, with the smallest circle representing one study and the largest representing five studies. The mapping is extracted from the data in the Appendix , which outlines the papers that belong in each of the circles

5 5. Discussion

After reviewing the 47 identified articles in the area of eOL, we can observe that all the works acknowledge the importance of the affordances offered by different e-learning technologies (e.g., remote collaboration, anytime anywhere), the importance of the relationship between eOL and employees’ satisfaction and performance, and the benefits associated with organizational value and outcome. Most of the studies agree that eOL provides employees, managers, and even clients with opportunities to learn in a more differentiated manner, compared to formal and face-to-face learning. However, how the organization adopts and puts into practice these capabilities to leverage them and achieve its goals are complex and challenging procedures that seem to be underexplored.

Several studies (Lee et al. 2015a ; Muller Queiroz et al. 2018 ; Tsai et al. 2010 ) focused on the positive effect of perceived managerial support, perceived usefulness, perceived ease of use, and other technology acceptance model (TAM) constructs of the e-learning system in supporting all three levels of learning (i.e., individual, collaborative, and organizational). Another interesting dimension highlighted by many studies (Choi and Ko 2012 ; Khalili et al. 2012 ; Yanson and Johnson 2016 ) is the role of socialization in the adoption and usage of the e-learning systems that offer these capabilities. Building connections and creating a shared learning space in the e-learning system is challenging but also critical for the learners (Yanson and Johnson 2016 ). This is consistent with the expectancy-theoretical explanation of how social context impacts on employees’ motivation to participate in learning (Lee et al. 2015a ; Muller Queiroz et al. 2018 ).

The organizational learning literature suggests that e-learning may be more appropriate for the acquisition of certain types of knowledge than others (e.g., procedural vs. declarative, or hard-skills vs. soft-skills); however, there is no empirical evidence for this (Yanson and Johnson 2016 ). To advance eOL research, there is a need for a significant move to address complex, strategic skills by including learning and development professionals (Garavan et al. 2019 ) and by developing strategic relationships. Another important element is to utilize e-learning technology that addresses and integrates organizational, individual, and social perspectives in eOL (Wang  2011 ). This is also identified in our literature review since we found only limited specialized e-learning systems in domain areas that have traditionally benefited from such technology. For instance, although there were studies that utilized VR environments (Costello and McNaughton 2018 ; Muller Queiroz et al. 2018 ) and video-based learning systems (Wei et al. 2013 ; Wei and Ram 2016 ), there was limited focus in contemporary eOL research on how specific affordances of the various environments that are used in organizations (e.g., Carnetsoft, Outotec HSC, and Simscale for simulations of working environments; or Raptivity, YouTube, and FStoppers to gain specific skills and how-to knowledge) can benefit the intended goals or be integrated with the unique qualities of the organization (e.g., IT, healthcare).

For the design and the development of the eOL approach, the organization needs to consider the alignment of individual learning needs, organizational objectives, and the necessary resources (Wang  2011 ). To achieve this, it is advisable for organizations to define the expected objectives, catalogue the individual needs, and select technologies that have the capacity to support and enrich learners with self-directed and socially constructed learning practices in the organization (Wang  2011 ). This needs to be done by taking into consideration that on-demand eOL is gradually replacing the classic static eOL curricula and processes (Dignen and Burmeister 2020 ).

Another important dimension of eOL research is the lenses used to approach effectiveness. The selected papers approached effectiveness with various objectives, such as fostering learning, usage of the e-learning system, employees’ performance, and the added organizational value (see Appendix ). To measure these indices, various metrics (quantitative, qualitative, and mixed) have been applied. The qualitative dimensions emphasize employees’ satisfaction and system usage (e.g., Menolli et al. 2020 ; Turi et al. 2019 ), as well as managers’ perceived gained value and benefits (e.g., Lee et al. 2015b ; Xiang et al. 2020 ) and firms’ perceived effective utilization of eOL resources (López-Nicolás and Meroño-Cerdán 2011 ). The quantitative dimensions focus on usage, feasibility, and experience at different levels within an organization, based on interviews, focus groups, and observations (Costello and McNaughton 2018 ; Michalski 2014 ; Stoffregen et al. 2016 ). However, it is not always clear the how eOL effectiveness has been measured, nor the extent to which eOL is well aligned with and is strategically impactful on delivering the strategic agenda of the organization (Garavan et al. 2019 ).

Research on digital technologies is developing rapidly, and big data and business analytics have the potential to pave the way for organizations’ digital transformation and sustainable development (Mikalef et al. 2018 ; Pappas et al. 2018 ); however, our review finds surprisingly limited use of big data and analytics in eOL. Despite contemporary e-learning systems adopting data-driven mechanisms, as well as advances in learning analytics (Siemens and Long 2011 ), the results of our analysis indicate that learner-generated data in the context of eOL are used in only a few studies to extract very limited insights with respect to the effectiveness of eOL and the intended objectives of the respective study (Hung et al. 2015 ; Renner et al. 2020 ; Rober and Cooper 2011 ). Therefore, eOL research needs to focus on data-driven qualities that will allow future researchers to gain deeper insights into which capabilities need to be developed to monitor the effectiveness of the various practices and technologies, their alignment with other functions of the organization, and how eOL can be a strategic and impactful vehicle for materializing the strategic agenda of the organization.

5.1 Status of eOL Research

The current review suggests that, while the efficient implementation of eOL entails certain challenges, there is also a great potential for improving employees’ performance as well as overall organizational outcome and value. There are also opportunities for improving organizations’ learning flow, which might not be feasible with formal learning and training. In order to construct the main research dimensions of eOL research and to look more deeply at the research objectives of the studies (the information we coded as objectives in the Appendix ), we performed a content analysis and grouped the research objectives. This enabled us to summarize the contemporary research on eOL according to five major categories, each of which is describes further below. As the research objectives of the published work shows, the research on eOL conducted during the last decade has particularly focused on the following five directions.

Investigating the capabilities of different technologies in different organizations.

Research has particularly focused on how easy the technology is to use, on how useful it is, or on how well aligned/integrated it is with other systems and processes within the organization. In addition, studies have used different learning technologies (e.g., smart, social, personalized) to enhance organizational learning in different contexts and according to different needs. However, most works have focused on affordances such as remote training and the development of static courses or modules to share information with learners. Although a few studies have utilized contemporary e-learning systems (see Appendix ), even in these studies there is a lack of alignment between the capabilities of those systems (e.g., open online course, adaptive support, social and collaborative learning) and the objectives and strategy of the organization (e.g., organizational value, fostering learning).

Enriching the learning flow and learning potential in different levels within an organization.

The reviewed work has emphasized how different factors contribute to different levels of organizational learning, and it has focused on practices that address individual, collaborative, and organizational learning within the structure of the organization. In particular, most of the reviewed studies recognize that organizational learning occurs at multiple levels: individual, team (or group), and organization. In other words, although each of the studies carried out an investigation within a given level (except for Garavan et al. 2019 ), there is a recognition and discussion of the different levels. Therefore, the results align with the 4I framework of organizational learning that recognizes how learning across the different levels is linked by social and psychological processes: intuiting, interpreting, integrating, and institutionalizing (the 4Is) (Crossan et al. 1999 ). However, most of the studies focused on the institutionalizing-intuiting link (i.e., top-down feedback); moreover, no studies focused on contemporary learning technologies and processes that strengthen the learning flow (e.g., self-regulated learning).

Identifying critical aspects for effective eOL.

There is a considerable amount of predominantly qualitative studies that focus on potential barriers to eOL implementation as well as on the risks and requirements associated with the feasibility and successful implementation of eOL. In the same vein, research has emphasized the importance of alignment of eOL (both in processes and in technologies) within the organization. These critical aspects for effective eOL are sometimes the main objectives of the studies (see Appendix ). However, most of the elements relating to the effectiveness of eOL were measured with questionnaires and interviews with employees and managers, and very little work was conducted on how to leverage the digital technologies employed in eOL, big data, and analytics in order to monitor the effectiveness of eOL.

Implementing employee-centric eOL.

In most of the studies, the main objective was to increase employees’ adoption, satisfaction, and usage of the e-learning system. In addition, several studies focused on the e-learning system’s ability to improve employees’ performance, increase the knowledge flow in the organization, and foster learning. Most of the approaches were employee-centric, with a small amount of studies focusing on managers and the firm in general. However, employees were seen as static entities within the organization, with limited work investigating how eOL-based training exposes employees to new knowledge, broadens their skills repertoire, and has tremendous potential for fostering innovation (Lin and Sanders 2017 ).

Achieving goals associated with the value creation of the organization.

A considerable number of studies utilized the firm (rather than the individual employee) as the unit of analysis. Such studies focused on how the implementation of eOL can increase employee performance, organizational value, and customer value. Although this is extremely helpful in furthering knowledge about eOL technologies and practices, a more granular investigation of the different e-learning systems and processes to address the various goals and strategies of the organization would enable researchers to extract practical insights on the design and implementation of eOL.

5.2 Research Agenda

By conducting an SLR and documenting the eOL research of the last decade, we have identified promising themes of research that have the potential to further eOL research and practice. To do so, we define a research agenda consisting of five thematic areas of research, as depicted in the research framework in Fig.  5 , and we provide some suggestions on how researchers could approach these challenges. In this visualization of the framework, on the left side we present the organizations as they were identified from our review (i.e., area/topic category in the Appendix ) and the multiple levels where organizational learning occurs (Costello and McNaughton 2018 ). On the right side, we summarize the objectives as they were identified from our review (i.e., the objectives category in the Appendix ). In the middle, we depict the orchestration that was conducted and how potential future research on eOL can improve the orchestration of the various elements and accelerate the achievement of the intended objectives. In particular, our proposed research agenda includes five research themes discussed in the following subsections.

E-learning capabilities to enhance organizational research agenda

5.2.1 Theme 1: Couple E-learning Capabilities With the Intended Goals

The majority of the eOL studies either investigated a generic e-learning system using the umbrella term “e-learning” or did not provide enough details about the functionalities of the system (in most cases, it was simply defined as an online or web system). This indicates the very limited focus of the eOL research on the various capabilities of e-learning systems. In other words, the literature has been very detailed on the organizational value and employees’ acceptance of the technology, but less detailed on the capabilities of this technology that needs to be put into place to achieve the intended goals and strategic agenda. However, the capabilities of the e-learning systems and their use are not one-size-fits-all, and the intended goals (to obtain certain skills and competences) and employees’ needs and backgrounds play a determining role in the selection of the e-learning system (Al-Fraihat et al. 2020 ).

Only in a very few studies (Mueller et al. 2011 ; Renner et al. 2020 ) were the capabilities of the e-learning solutions (e.g., mobile learning, VR) utilized, and the results were found to significantly contribute to the intended goals. The intended knowledge can be procedural, declarative, general competence (e.g., presentation, communication, or leadership skills) or else, and its particularities and the pedagogical needs of the intended knowledge (e.g., a need for summative/formative feedback or for social learning support) should guide the selection of the e-learning system and the respective capabilities. Therefore, future research needs to investigate how the various capabilities offered by contemporary learning systems (e.g., assessment mechanisms, social learning, collaborative learning, personalized learning) can be utilized to adequately reinforce the intended goals (e.g., to train personnel to use a new tool, to improve presentation skills).

5.2.2 Theme 2: Embrace the Particularities of the Various Industries

Organizational learning entails sharing knowledge and enabling opportunities for growth at the individual, group, team, and organizational levels. Contemporary e-learning systems provide the medium to substantiate the necessary knowledge flow within organizations and to support employees’ overall learning. From the selected studies, we can infer that eOL research is either conducted in an industry-agnostic context (either generic or it was not properly reported) or there is a focus on the IT industry (see Appendix ). However, when looking at the few studies that provide results from different industries (Garavan et al. 2019 ; Lee et al. 2014 ), companies indicate that there are different practices, processes, and expectations, and that employees have different needs and perceptions with regards to e-learning systems and eOL in general. Such particularities influence the perceived dimensions of a learning organization. Some industries noted that eOL promoted the development of their learning organizations, whereas others reported that eOL did not seem to contribute to their development as a learning organization (Yoo and Huang 2016 ). Therefore, it is important that the implementation of organizational learning embraces the particularities of the various industries and future research needs to identify how the industry-specific characteristics can inform the design and development of organizational learning in promoting an organization’s goals and agenda.

5.2.3 Theme 3: Utilize E-learning Capabilities to Implement Employee-centric Approaches

For efficient organizational learning to be implemented, the processes and technologies need to recognize that learning is linked by social and psychological processes (Crossan et al. 1999 ). This allows employees to develop learning in various forms (e.g., social, emotional, personalized) and to develop elements such as self-awareness, self-control, and interpersonal skills that are vital for the organization. Looking at the contemporary eOL research, we notice that the exploration of e-learning capabilities to nurture the aforementioned elements and support employee-centric approaches is very limited (e.g., personalized technologies, adaptive assessment). Therefore, future research needs to collect data to understand how e-learning capabilities can be utilized in relation to employees’ needs and perceptions in order to provide solutions (e.g., collaborative, social, adaptive) that are employee-centric and focused on development, and that have the potential to move away from standard one-size-fits-all e-learning solutions to personalized and customized systems and processes.

5.2.4 Theme 4: Employ Analytics-enabled eOL

There is a lot of emphasis on measuring, via various qualitative and quantitative metrics, the effectiveness of eOL implemented at different levels in organizations. However, most of these metrics come from surveys and interviews that capture employees’ and managers’ perceptions of various aspects of eOL (e.g., fostering of learning, organizational value, employees’ performance), and very few studies utilize analytics (Hung et al. 2015 ; Renner et al. 2020 ; Rober and Cooper 2011 ). Given how digital technologies, big data, and business analytics pave the way towards organizations’ digital transformation and sustainable development (Mikalef et al. 2018 ; Pappas et al. 2018 ), and considering the learning analytics affordances of contemporary e-learning systems (Siemens and Long 2011 ), future work needs to investigate how learner/employee-generated data can be employed to inform practice and devise more accurate and temporal effectiveness metrics when measuring the importance and impact of eOL.

5.2.5 Theme 5: Orchestrate the Employees’ Needs, Resources, and Objectives in eOL Implementation

While considerable effort has been directed towards the various building blocks of eOL implementation, such as resources (intangible, tangible, and human skills) and employees’ needs (e.g., vision, growth, skills development), little is known so far about the processes and structures necessary for orchestrating those elements in order to achieve an organization’s intended goals and to materialize its overall agenda. In other words, eOL research has been very detailed on some of the elements that constitute efficient eOL, but less so on the interplay of those elements and how they need to be put into place. Prior literature on strategic resource planning has shown that competence in orchestrating such elements is a prerequisite to successfully increasing business value (Wang et al. 2012 ). Therefore, future research should not only investigate each of these elements in silos, but also consider their interplay, since it is likely that organizations with similar resources will exert highly varied levels in each of these elements (e.g., analytics-enabled, e-learning capabilities) to successfully materialize their goals (e.g., increase value, improve the competence base of their employees, modernize their organization).

5.3 Implications

Several implications for eOL have been revealed in this literature review. First, most studies agree that employees’ or trainees’ experience is extremely important for the successful implementation of eOL. Thus, keeping them in the design and implementation cycle of eOL will increase eOL adoption and satisfaction as well as reduce the risks and barriers. Another important implication addressed by some studies relates to the capabilities of the e-learning technologies, with easy-to-use, useful, and social technologies resulting in more efficient eOL (e.g., higher adoption and performance). Thus, it is important for organizations to incorporate these functionalities in the platform and reinforce them with appropriate content and support. This should not only benefit learning outcomes, but also provide the networking opportunities for employees to broaden their personal networks, which are often lost when companies move from face-to-face formal training to e-learning-enabled organizational learning.

5.4 Limitations

This review has some limitations. First, we had to make some methodological decisions (e.g., selection of databases, the search query) that might lead to certain biases in the results. However, tried to avoid such biases by considering all the major databases and following the steps indicated by Kitchenham and Charters ( 2007 ). Second, the selection of empirical studies and coding of the papers might pose another possible bias. However, the focus was clearly on the empirical evidence, the terminology employed (“e-learning”) is an umbrella term that covers the majority of the work in the area, and the coding of papers was checked by two researchers. Third, some elements of the papers were not described accurately, leading to some missing information in the coding of the papers. However, the amount of missing information was very small and could not affect the results significantly. Finally, we acknowledge that the selected methodology (Kitchenham and Charters 2007 ) includes potential biases (e.g., false negatives and false positives), and that different, equally valid methods (e.g., Okoli and Schabram 2010 ) might have been used and have resulted in slightly different outcomes. Nevertheless, despite the limitations of the selected methodology, it is a well-accepted and widely used literature review method in both software engineering and information systems (Boell and Cecez-Kecmanovic 2014 ), providing certain assurance of the results.

6 Conclusions and Future Work

We have presented an SLR of 47 contributions in the field of eOL over the last decade. With respect to RQ1, we analyzed the papers from different perspectives, such as research methodology, technology, industries, employees, and intended outcomes in terms of organizational value, employees’ performance, usage, and behavioral change. The detailed landscape is depicted in the Appendix and Figs.  3 and 4 ; with the results indicating the limited utilization of the various e-learning capabilities (e.g., social, collaborative) to achieve objectives connected with those capabilities (e.g., social learning and behavioral change, collaborative learning and overcoming barriers).

With respect to RQ2, we categorized the main findings of the selected papers into five areas that reflect the status of eOL research, and we have discussed the challenges and opportunities emerging from the current review. In addition, we have synthesized the extracted challenges and opportunities and proposed a research agenda consisting of five elements that provide suggestions on how researchers could approach these challenges and exploit the opportunities. Such an agenda will strengthen how e-learning can be leveraged to enhance the process of improving actions through better knowledge and understanding in an organization.

A number of suggestions for further research have emerged from reviewing prior and ongoing work on eOL. One recommendation for future researchers is to clearly describe the eOL approach by providing detailed information about the technologies and materials used, as well as the organizations. This will allow meta-analyses to be conducted and it will also identify the potential effects of a firm’s size or area on the performance and other aspects relating to organizational value. Future work should also focus on collecting and triangulating different types of data from different sources (e.g., systems’ logs). The reviewed studies were conducted mainly by using survey data, and they made limited use of data coming from the platforms; thus, the interpretations and triangulation between the different types of collected data were limited.

Al-Fraihat, D., Joy, M., & Sinclair, J. (2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102 , 67–86.

Article   Google Scholar  

Alharthi, A. D., Spichkova, M., & Hamilton, M. (2019). Sustainability requirements for eLearning systems: A systematic literature review and analysis. Requirements Engineering, 24 (4), 523–543.

Alsabawy, A. Y., Cater-Steel, A., & Soar, J. (2013). IT infrastructure services as a requirement for e-learning system success. Computers & Education, 69 , 431–451.

Antonacopoulou, E., & Chiva, R. (2007). The social complexity of organizational learning: The dynamics of learning and organizing. Management Learning, 38 , 277–295.

Berends, H., & Lammers, I. (2010). Explaining discontinuity in organizational learning: A process analysis. Organization Studies, 31 (8), 1045–1068.

Boell, S. K., & Cecez-Kecmanovic, D. (2014). A hermeneutic approach for conducting literature reviews and literature searches. Communications of the Association for Information Systems, 34 (1), 12.

Google Scholar  

Bologa, R., & Lupu, A. R. (2014). Organizational learning networks that can increase the productivity of IT consulting companies. A case study for ERP consultants. Expert Systems with Applications, 41 (1), 126–136.

Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and academic performance. The Internet and Higher Education, 33 , 24–32.

Cheng, B., Wang, M., Moormann, J., Olaniran, B. A., & Chen, N. S. (2012). The effects of organizational learning environment factors on e-learning acceptance. Computers & Education, 58 (3), 885–899.

Cheng, B., Wang, M., Yang, S. J., & Peng, J. (2011). Acceptance of competency-based workplace e-learning systems: Effects of individual and peer learning support. Computers & Education, 57 (1), 1317–1333.

Choi, S., & Ko, I. (2012). Leveraging electronic collaboration to promote interorganizational learning. International Journal of Information Management, 32 (6), 550–559.

Costello, J. T., & McNaughton, R. B. (2018). Integrating a dynamic capabilities framework into workplace e-learning process evaluations. Knowledge and Process Management, 25 (2), 108–125.

Crossan, M. M., Lane, H. W., & White, R. E. (1999). An organizational learning framework: From intuition to institution. Academy of Management Review, 24 , 522–537.

Crossan, M. M., Maurer, C. C., & White, R. E. (2011). Reflections on the 2009 AMR decade award: Do we have a theory of organizational learning? Academy of Management Review, 36 (3), 446–460.

Dee, J., & Leisyte, L. (2017). Knowledge sharing and organizational change in higher education. The Learning Organization, 24 (5), 355–365. https://doi.org/10.1108/TLO-04-2017-0034

Dignen, B., & Burmeister, T. (2020). Learning and development in the organizations of the future. Three pillars of organization and leadership in disruptive times (pp. 207–232). Cham: Springer.

Chapter   Google Scholar  

Dybå, T., & Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50 (9–10), 833–859.

El Kadiri, S., Grabot, B., Thoben, K. D., Hribernik, K., Emmanouilidis, C., Von Cieminski, G., & Kiritsis, D. (2016). Current trends on ICT technologies for enterprise information systems. Computers in Industry, 79 , 14–33.

Engeström, Y., Kerosuo, H., & Kajamaa, A. (2007). Beyond discontinuity: Expansive organizational learning remembered. Management Learning, 38 (3), 319–336.

Gal, E., & Nachmias, R. (2011). Online learning and performance support in organizational environments using performance support platforms. Performance Improvement, 50 (8), 25–32.

Garavan, T. N., Heneghan, S., O’Brien, F., Gubbins, C., Lai, Y., Carbery, R., & Grant, K. (2019). L&D professionals in organisations: much ambition, unfilled promise. European Journal of Training and Development, 44 (1), 1–86.

Goggins, S. P., Jahnke, I., & Wulf, V. (2013). Computer-supported collaborative learning at the workplace . New York: Springer.

Book   Google Scholar  

Hammad, R., Odeh, M., & Khan, Z. (2017). ELCMM: An e-learning capability maturity model. In Proceedings of the 15th International Conference (e-Society 2017) (pp. 169–178).

Hester, A. J., Hutchins, H. M., & Burke-Smalley, L. A. (2016). Web 2.0 and transfer: Trainers’ use of technology to support employees’ learning transfer on the job. Performance Improvement Quarterly, 29 (3), 231–255.

Hung, Y. H., Lin, C. F., & Chang, R. I. (2015). Developing a dynamic inference expert system to support individual learning at work. British Journal of Educational Technology, 46 (6), 1378–1391.

Iris, R., & Vikas, A. (2011). E-Learning technologies: A key to dynamic capabilities. Computers in Human Behavior, 27 (5), 1868–1874.

Jia, H., Wang, M., Ran, W., Yang, S. J., Liao, J., & Chiu, D. K. (2011). Design of a performance-oriented workplace e-learning system using ontology. Expert Systems with Applications, 38 (4), 3372–3382.

Joo, Y. J., Lim, K. Y., & Park, S. Y. (2011). Investigating the structural relationships among organisational support, learning flow, learners’ satisfaction and learning transfer in corporate e-learning. British Journal of Educational Technology, 42 (6), 973–984.

Kaschig, A., Maier, R., Sandow, A., Lazoi, M., Barnes, S. A., Bimrose, J., … Schmidt, A. (2010). Knowledge maturing activities and practices fostering organisational learning: results of an empirical study. In European Conference on Technology Enhanced Learning (pp. 151–166). Berlin: Springer.

Khalili, A., Auer, S., Tarasowa, D., & Ermilov, I. (2012). SlideWiki: Elicitation and sharing of corporate knowledge using presentations. International Conference on Knowledge Engineering and Knowledge Management (pp. 302–316). Berlin: Springer.

Khandakar, M. S. A., & Pangil, F. (2019). Relationship between human resource management practices and informal workplace learning. Journal of Workplace Learning, 31 (8), 551–576.

Kim, M. K., Kim, S. M., & Bilir, M. K. (2014). Investigation of the dimensions of workplace learning environments (WLEs): Development of the WLE measure. Performance Improvement Quarterly, 27 (2), 35–57.

Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01, 2007 . https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=35909B1B280E2032BF116BDC9DCB71EA? .

Krippendorff, K. (2018). Content analysis: an introduction to its methodology. Thousand Oaks: Sage Publications.

Lai, H. J. (2017). Examining civil servants’ decisions to use Web 2.0 tools for learning, based on the decomposed theory of planned behavior. Interactive Learning Environments, 25 (3), 295–305.

Lau, K. (2015). Organizational learning goes virtual? A study of employees’ learning achievement in stereoscopic 3D virtual reality. The Learning Organization, 22 (5), 289–303.

Lee, J., Choi, M., & Lee, H. (2015a). Factors affecting smart learning adoption in workplaces: Comparing large enterprises and SMEs. Information Technology and Management, 16 (4), 291–302.

Lee, J., Kim, D. W., & Zo, H. (2015b). Conjoint analysis on preferences of HRD managers and employees for effective implementation of m-learning: The case of South Korea. Telematics and Informatics, 32 (4), 940–948.

Lee, J., Zo, H., & Lee, H. (2014). Smart learning adoption in employees and HRD managers. British Journal of Educational Technology, 45 (6), 1082–1096.

Lin, C. H., & Sanders, K. (2017). HRM and innovation: A multi-level organizational learning perspective. Human Resource Management Journal, 27 (2), 300–317.

Lin, C. Y., Huang, C. K., & Zhang, H. (2019). Enhancing employee job satisfaction via e-learning: The mediating role of an organizational learning culture. International Journal of Human–Computer Interaction, 35 (7), 584–595.

Liu, Y. C., Huang, Y. A., & Lin, C. (2012). Organizational factors’ effects on the success of e-learning systems and organizational benefits: An empirical study in Taiwan. The International Review of Research in Open and Distributed Learning, 13 (4), 130–151.

López-Nicolás, C., & Meroño-Cerdán, ÁL. (2011). Strategic knowledge management, innovation and performance. International Journal of Information Management, 31 (6), 502–509.

Manuti, A., Pastore, S., Scardigno, A. F., Giancaspro, M. L., & Morciano, D. (2015). Formal and informal learning in the workplace: A research review. International Journal of Training and Development, 19 (1), 1–17.

March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2 (1), 71–87.

Marshall, S. (2006). New Zealand Tertiary Institution E-learning Capability: Informing and Guiding eLearning Architectural Change and Development. Report to the ministry of education . NZ: Victoria University of Wellington.

McDonald, N., Schoenebeck, S., & Forte, A. (2019). Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. In Proceedings of the ACM on Human–Computer Interaction, 3(CSCW) (pp. 1–23).

Menolli, A., Tirone, H., Reinehr, S., & Malucelli, A. (2020). Identifying organisational learning needs: An approach to the semi-automatic creation of course structures for software companies. Behaviour & Information Technology, 39 (11), 1140–1155.

Michalski, M. P. (2014). Symbolic meanings and e-learning in the workplace: The case of an intranet-based training tool. Management Learning, 45 (2), 145–166.

Mikalef, P., Pappas, I. O., Krogstie, J., & Giannakos, M. (2018). Big data analytics capabilities: A systematic literature review and research agenda. Information Systems and e-Business Management, 16 (3), 547–578.

Mitić, S., Nikolić, M., Jankov, J., Vukonjanski, J., & Terek, E. (2017). The impact of information technologies on communication satisfaction and organizational learning in companies in Serbia. Computers in Human Behavior, 76 , 87–101.

Mueller, J., Hutter, K., Fueller, J., & Matzler, K. (2011). Virtual worlds as knowledge management platform—A practice-perspective. Information Systems Journal, 21 (6), 479–501.

Muller Queiroz, A. C., Nascimento, M., Tori, A., Alejandro, R. Brashear, Veloso, T., de Melo, V., de Souza Meirelles, F., & da Silva Leme, M. I. (2018). Immersive virtual environments in corporate education and training. In AMCIS. https://aisel.aisnet.org/amcis2018/Education/Presentations/12/ .

Navimipour, N. J., & Zareie, B. (2015). A model for assessing the impact of e-learning systems on employees’ satisfaction. Computers in Human Behavior, 53 , 475–485.

Oh, S. Y. (2019). Effects of organizational learning on performance: The moderating roles of trust in leaders and organizational justice. Journal of Knowledge Management, 23, 313–331.

Okoli, C., & Schabram, K. (2010). A guide to conducting a systematic literature review of information systems research. Sprouts: Working Papers on Information Systems, 10 (26), 1–46.

Pappas, I. O., Mikalef, P., Giannakos, M. N., Krogstie, J., & Lekakos, G. (2018). Big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies. Information Systems and e-Business Management, 16, 479–491.

Popova-Nowak, I. V., & Cseh, M. (2015). The meaning of organizational learning: A meta-paradigm perspective. Human Resource Development Review, 14 (3), 299–331.

Qi, C., & Chau, P. Y. (2016). An empirical study of the effect of enterprise social media usage on organizational learning. In Pacific Asia Conference on Information Systems (PACIS'16). Proceedings , Paper 330. http://aisel.aisnet.org/pacis2016/330 .

Renner, B., Wesiak, G., Pammer-Schindler, V., Prilla, M., Müller, L., Morosini, D., … Cress, U. (2020). Computer-supported reflective learning: How apps can foster reflection at work. Behaviour & Information Technology, 39 (2), 167–187.

Rober, M. B., & Cooper, L. P. (2011, January). Capturing knowledge via an” Intrapedia”: A case study. In 2011 44th Hawaii International Conference on System Sciences (pp. 1–10). New York: IEEE.

Rosenberg, M. J., & Foshay, R. (2002). E-learning: Strategies for delivering knowledge in the digital age. Performance Improvement, 41 (5), 50–51.

Serrano, Á., Marchiori, E. J., del Blanco, Á., Torrente, J., & Fernández-Manjón, B. (2012). A framework to improve evaluation in educational games. The IEEE Global Engineering Education Conference (pp. 1–8). Marrakesh, Morocco.

Siadaty, M., Jovanović, J., Gašević, D., Jeremić, Z., & Holocher-Ertl, T. (2010). Leveraging semantic technologies for harmonization of individual and organizational learning. In European Conference on Technology Enhanced Learning (pp. 340–356). Berlin: Springer.

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46 (5), 30.

Škerlavaj, M., Dimovski, V., Mrvar, A., & Pahor, M. (2010). Intra-organizational learning networks within knowledge-intensive learning environments. Interactive Learning Environments, 18 (1), 39–63.

Smith, P. J., & Sadler-Smith, E. (2006). Learning in organizations: Complexities and diversities . London: Routledge.

Stoffregen, J. D., Pawlowski, J. M., Ras, E., Tobias, E., Šćepanović, S., Fitzpatrick, D., … Friedrich, H. (2016). Barriers to open e-learning in public administrations: A comparative case study of the European countries Luxembourg, Germany, Montenegro and Ireland. Technological Forecasting and Social Change, 111 , 198–208.

Subramaniam, R., & Nakkeeran, S. (2019). Impact of corporate e-learning systems in enhancing the team performance in virtual software teams. In Smart Technologies and Innovation for a Sustainable Future (pp. 195–204). Berlin: Springer.

Tsai, C. H., Zhu, D. S., Ho, B. C. T., & Wu, D. D. (2010). The effect of reducing risk and improving personal motivation on the adoption of knowledge repository system. Technological Forecasting and Social Change, 77 (6), 840–856.

Turi, J. A., Sorooshian, S., & Javed, Y. (2019). Impact of the cognitive learning factors on sustainable organizational development. Heliyon, 5 (9), e02398.

Wang, M. (2011). Integrating organizational, social, and individual perspectives in Web 2.0-based workplace e-learning. Information Systems Frontiers, 13 (2), 191–205.

Wang, M. (2018). Effects of individual and social learning support on employees’ acceptance of performance-oriented e-learning. In E-Learning in the Workplace (pp. 141–159). Springer. https://doi.org/10.1007/978-3-319-64532-2_13 .

Wang, M., Ran, W., Liao, J., & Yang, S. J. (2010). A performance-oriented approach to e-learning in the workplace. Journal of Educational Technology & Society, 13 (4), 167–179.

Wang, M., Vogel, D., & Ran, W. (2011). Creating a performance-oriented e-learning environment: A design science approach. Information & Management, 48 (7), 260–269.

Wang, N., Liang, H., Zhong, W., Xue, Y., & Xiao, J. (2012). Resource structuring or capability building? An empirical study of the business value of information technology. Journal of Management Information Systems, 29 (2), 325–367.

Wang, S., & Wang, H. (2012). Organizational schemata of e-portfolios for fostering higher-order thinking. Information Systems Frontiers, 14 (2), 395–407.

Wei, K., & Ram, J. (2016). Perceived usefulness of podcasting in organizational learning: The role of information characteristics. Computers in Human Behavior, 64 , 859–870.

Wei, K., Sun, H., & Li, H. (2013). On the driving forces of diffusion of podcasting in organizational settings: A case study and propositions. In PACIS 2013. Proceedings , 217. http://aisel.aisnet.org/pacis2013/217 .

Weinhardt, J. M., & Sitzmann, T. (2018). Revolutionizing training and education? Three questions regarding massive open online courses (MOOCs). Human Resource Management Review, 29 (2), 218–225.

Xiang, Q., Zhang, J., & Liu, H. (2020). Organisational improvisation as a path to new opportunity identification for incumbent firms: An organisational learning view. Innovation, 22 (4), 422–446. https://doi.org/10.1080/14479338.2020.1713001 .

Yanson, R., & Johnson, R. D. (2016). An empirical examination of e-learning design: The role of trainee socialization and complexity in short term training. Computers & Education, 101 , 43–54.

Yoo, S. J., & Huang, W. D. (2016). Can e-learning system enhance learning culture in the workplace? A comparison among companies in South Korea. British Journal of Educational Technology, 47 (4), 575–591.

Zhang, X., Jiang, S., Ordóñez de Pablos, P., Lytras, M. D., & Sun, Y. (2017). How virtual reality affects perceived learning effectiveness: A task–technology fit perspective. Behaviour & Information Technology, 36 (5), 548–556.

Zhang, X., Meng, Y., de Pablos, P. O., & Sun, Y. (2019). Learning analytics in collaborative learning supported by Slack: From the perspective of engagement. Computers in Human Behavior, 92 , 625–633.

Download references

Open Access funding provided by NTNU Norwegian University of Science and Technology (incl St. Olavs Hospital - Trondheim University Hospital).

Author information

Authors and affiliations.

Norwegian University of Science and Technology, Trondheim, Norway

Michail N. Giannakos, Patrick Mikalef & Ilias O. Pappas

University of Agder, Kristiansand, Norway

Ilias O. Pappas

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ilias O. Pappas .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Giannakos, M.N., Mikalef, P. & Pappas, I.O. Systematic Literature Review of E-Learning Capabilities to Enhance Organizational Learning. Inf Syst Front 24 , 619–635 (2022). https://doi.org/10.1007/s10796-020-10097-2

Download citation

Accepted : 09 December 2020

Published : 01 February 2021

Issue Date : April 2022

DOI : https://doi.org/10.1007/s10796-020-10097-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Organizational learning
• Literature review
• Learning environments
• Find a journal
• Publish with us
• Track your research

ORIGINAL RESEARCH article

E-learning research trends in higher education in light of covid-19: a bibliometric analysis.

• 1 University of Bisha, Bisha, Saudi Arabia
• 2 University of Oum El Bouaghi, Oum El Bouaghi, Algeria
• 3 Binghamton University, Binghamton, NY, United States
• 4 University of Rochester, Rochester, NY, United States

This paper provides a broad bibliometric overview of the important conceptual advances that have been published during COVID-19 within “e-learning in higher education.” E-learning as a concept has been widely used in the academic and professional communities and has been approved as an educational approach during COVID-19. This article starts with a literature review of e-learning. Diverse subjects have appeared on the topic of e-learning, which is indicative of the dynamic and multidisciplinary nature of the field. These include analyses of the most influential authors, of models and networks for bibliometric analysis, and progress towards the current research within the most critical areas. A bibliometric review analyzes data of 602 studies published (2020–2021) in the Web of Science (WoS) database to fully understand this field. The data were examined using VOSviewer, CiteSpace, and KnowledgeMatrix Plus to extract networks and bibliometric indicators about keywords, authors, organizations, and countries. The study concluded with several results within higher education. Many converging words or sub-fields of e-learning in higher education included distance learning, distance learning, interactive learning, online learning, virtual learning, computer-based learning, digital learning, and blended learning (hybrid learning). This research is mainly focused on pedagogical techniques, particularly e-learning and collaborative learning, but these are not the only trends developing in this area. The sub-fields of artificial intelligence, machine learning, and deep learning constitute new research directions for e-learning in light of COVID-19 and are suggestive of new approaches for further analysis.

Introduction

The idea of e-learning was originated in the 1990s to explain learning thoroughly through technical advances. When instructional architecture and technologies have advanced, more attention has been paid to studying with the pedagogy. University education, further education, and e-learning have also recently adopted prominent roles in e-learning, too. It is now possible to provide e-learning for off-the-formal training through the internet. It also increased the need for personalization and advanced social people’s tools ( Siemens, 2005 ). In addition, it is often referred to as being able to read. It will help mix much learning more conveniently, but it has to be done, given the success of “traditional” e-learning pages. When the educational and technological assets join, this will be something more than a personal matter.

The COVID-19 pandemic has forced the closure of many activities, especially educational activities. To limit the spread of the pandemic, universities, institutes, and academic schools had to switch to e-learning using the available educational platforms. Social distancing is critical, and the COVID-19 pandemic has brought an end to face-to-face education, negatively impacting educational activities ( Maatuk et al., 2021 ). This closure has stimulated the growth of distance education activities as an alternative to face-to-face education in their various forms. Accordingly, many universities have shared the best ways to deliver course materials remotely, engage students, and conduct assessments.

The concept of e-learning, although widely known has not yet been fully explored ( Nicholson, 2007 ). Many countries designed and deployed distance education systems during the COVID-19 pandemic to ensure that higher education could continue without interruption ( Tesar, 2020 ). Several opportunities and challenges related to e-learning, higher education, and COVID-19 arose as a result of this, prompting a flurry of research into the area. When looking at the scientific studies published during the COVID-19 pandemic, it shows clearly that many international journals have published a large number of academic articles about e-learning in higher education during COVID-19 ( Karakose and Demirkol, 2021 ). Furthermore, a vast amount of bibliometric research has been carried out in this field. However, there is very little research focused entirely on the relationship between e-learning, higher education, and COVID-19, using scientometric or bibliometric analysis ( Furstenau et al., 2021 ).

This paper will discuss bibliometric indicators for e-learning in higher education during COVID-19 studies and proceed with a network analysis to define the most important sub-areas in this topic. To define the trends of e-learning in higher education during COVID-19, the following questions are proposed:

Q1: What are the most important sub-fields of e-learning in higher education in light of COVID-19?

Q2: Who are the most influential authors on the subject of e-learning in higher education in light of COVID-19?

Q3: What countries and research institutions are the most referenced for research on the subject of e-learning in higher education in light of COVID-19?

Q4: What are the research gaps and recent trends in the subject of e-learning in higher education in light of COVID-19?

An analysis was conducted to provide a broad and long-term perspective on the vocabulary of learning publications. It helps to recognize emerging problems within the multifaceted and increasing study fields of the world of e-learning. Newly published studies can improve knowledge and bridge the knowledge gap through findings regarding e-learning trends; this applies particularly to higher education due to the importance of knowing the latest information about distance learning and its methods. For this reason, the research is valuable for analyzing the volume of publications that have been made on the subject matter and to solidify the knowledge base on what has been studied by different expert researchers in education. So this will create new progress and new proposals to improve education in the event of a future pandemic.

In recent years, there has been an increasing interest in research within areas related to e-learning: online learning, blended learning, technology acceptance model, smart learning, interactive learning environments, intelligent tutoring systems, digital learning were reported ( Oprea, 2014 ; Castro-Schez et al., 2020 ; de Moura et al., 2020 ; Kao, 2020 ; Nylund and Lanz, 2020 ; Pal and Vanijja, 2020 ; Patricia, 2020 ; Şerban and Ioan, 2020 ).

A substantial quantity of literature has been written and published on the bibliometric analysis of e-learning. These studies mainly aim to identify the most critical areas (keywords) of e-learning. Networks such as that conducted by Chiang et al. (2010) showed that the significant research areas in e-learning are as follows: Education and Educational Research, Information Science and Library Science, and Computer.

Science/Multidisciplinary Applications

Cheng et al. (2014) analyzed data from 324 articles published between 2000 and 2012 in academic journals and conference proceedings from 2000 to 2012 to determine the vital research areas (the results identify six research themes in the field e-learning). Tibaná-Herrera et al. (2018a) used VOSViewer to conduct a bibliometric analysis of SCOPUS and SCImago Journal & Country Rank to establish the “e-learning” thematic category of scientific publications, thereby contributing to the discipline’s consolidation, accessibility, and development by researchers.

Bai et al. (2020) have also pursued similar work in analyzing 7,214 articles published in 10 journals on the subject of e-learning from 1999 to 2018; this study offers valuable hints on the future direction of how e-learning may evolve. Fatima and Abu (2019) examined 9,826 records from the Web of Science (WoS) database between 1989 to 2018 to identify significant contributions to the area of e-learning. The findings of this study show that the United States and the United Kingdom have contributed more than half of the research in e-Learning. According to a recent survey by Mashroofa et al. (2020) , the University of London is the most prolific institution globally. According to the WoS database, the institution has published 131 studies on e-learning; the bibliometric analysis of 6,934 results revealed that the publications received 59,784 citations.

Hung (2012) employed text mining and bibliometrics to examine 689 refereed journal articles and proceedings, comparing them to these research results. These works are divided into two domains, each of which has four groups. The study’s findings now offer evidence that e-learning methods vary across top countries and early adopter countries.

There have been multiple previous attempts to do a systematic review of e-learning publications ( Lahti et al., 2014 ; Zare et al., 2016 ; Garcia et al., 2018 ; Rodrigues et al., 2019 ; Araka et al., 2020 ; Valverde-Berrocoso et al., 2020 ), these studies mainly aimed to identify research areas, the most used and most important methods, and tools in e-learning.

Many studies have examined the results of e-learning publications through meta-analysis ( ŠUmak et al., 2011 ; Lahti et al., 2014 ; Mothibi, 2015 ; Cabero-Almenara et al., 2016 ; Yuwono and Sujono, 2018 ).

The study’s contribution is that no controlled studies have compared differences in networks, models, and software outputs to define the most critical research areas in e-learning and the most influenced authors, organizations, and countries.

The study makes an important contribution to the analysis of current models and networks of e-learning in higher education during the COVID-19 pandemic, aiming to define the most critical research areas in e-learning and the most influenced authors, organizations, and countries. In addition, it looks at the framework of e-learning and its future research trends in light of COVID-19. This has been done through numerous investigations ( Tibaná-Herrera et al., 2018a , c ; Hilmi and Mustapha, 2020 ; López-Belmonte et al., 2021 ).

Materials and Methods

Bibliometric data.

We retrieved published research via a topic search of the Science e-learning in higher education during the COVID-19 pandemic using the WoS database on August 12, 2021. The following search terms were used: topic = (“e-learning” “COVID-19” “higher education”), in title-abs-key from 2020 to 2021, and were 602 studies (475 articles, 80 articles; early access, 25 proceedings paper, 22 reviews) distributed over 2 years, as shown in Figure 1 .

Figure 1. Publications per year (KnowledgeMatrix Plus outputs).

The following selection criteria were used to choose the studies. First, for the title, we looked at the following: the studies that looked at the topic of e-learning in higher education during COVID-19. Second, for the abstract, we looked at the following: the studies that addressed the problem of e-learning in COVID-19. Third, for the keyword, we looked at the following: the studies that included e-learning, higher education, universities, and COVID-19. Fourth, the subject areas were limited to a selection of works that dealt with this subject in the following disciplines: business management and accounting, educational sciences, social sciences, and psychology.

The bibliometric study data represents the overall research on “E-learning in higher education in light of the COVID-19” in the WoS database. These data covered the last 2 years (2020 and 2021) in which the use of e-learning was expected due to the closure and quarantine procedures.

The reasons for choosing this database over others, particularly Scopus and ScienceDirect, are due to several considerations; due to WoS data, the field of scientometrics has advanced significantly. WoS is more than simply a database of academic papers. Many information objectives are supported by this selected, organized, and balanced database, including full citation links and improved metadata ( Birkle et al., 2020 ). WoS databases include high-quality research covering Science Citation Index Expanded (SCI-Expanded), Social Sciences Citation Index (SSCI), Arts & Humanities Citation Index (A&HCI), Emerging Sources Citation Index (ESCI) ( Falagas et al., 2008 ).

Figure 1 illustrates how interest in e-learning research has increased in recent years, particularly between 2020 and 2021. Among the 602 studies with 4,280 citations, 230 in 2020 (1,400 citations), and 372 in 2021 (2,880 citations), the importance of higher education institutions, including universities, in this modern teaching and learning approach and their significance in the educational process during COVID-19 is evident. They are different from the periods approved in the previous studies ( Chiang et al., 2010 ; Cheng et al., 2014 ; Bai et al., 2020 ; Fatimah et al., 2021 ). Therefore, this field of research (e-learning) has been renewed, and researchers should pay more attention to it to provide effective methods and approaches in light of the continuing epidemic.

Methods and Tools

According to the methods and approaches of bibliometric analysis (see: Zupic and Čater, 2015 , p. 04). the study relied on the co-occurrence indicator (co-word) to find out the main keywords on which previous studies focused as well as the co-authorship, publications, and citations indicators to find prominent authors, organizations, and countries in the topic of e-learning in higher education in COVID-19.

Following the methodology of preparing the bibliometric study in management and organization, which was explained by Zupic and Čater (2015) , the bibliometric analysis was carried out by completing the following steps: research design, study questions, and analysis approach selection (co-occurrence, publication, citation, and co-authorship); bibliometric data compilation, selection, and filtration, analysis (choosing the appropriate bibliometric software, clean the data, and generate networks); visualization, and interpretation.

The bibliometric analysis was performed to design networks of e-learning and define the most frequent keywords and the most cited authors, organizations, and countries to explain new and current trends within this topic. This is achieved depending on different software: CiteSpace converts research domain concepts into mapping functions between research frontiers and intellectual bases and is effective for information visualization ( Chen, 2016 ); VOSviewer is used to design the networks and is a powerful function for co-occurrence analysis and citation analysis ( Van Eck and Waltman, 2013 ). KnowledgeMatrix Plus is a powerful tool for analyzing frequency and statistics ( Chen and Song, 2017 ). This software was not used in previous studies ( Chiang et al., 2010 ; Cheng et al., 2014 ; Bai et al., 2020 ; Fatimah et al., 2021 ).

Results and Discussion

Keywords frequency.

Figures 2A,B and Supplementary Table 1 present the most frequent keywords that have been repeated more than five, which amounted to 131.

Figure 2. (A) Network of keywords (VOSviewer outputs). (B) Network of keywords (CiteSpace outputs).

Figure 2A shows nine sub-areas (clusters) for research in e-learning within higher education during the era of COVID-19. First, the red cluster shows searches related to the following: higher education, students, motivation, attitudes, systems, technology acceptance model, and user acceptance. Second, the green cluster shows searches related to the pandemic, blended learning, online learning, hybrid learning, flipped classrooms, virtual learning, and distance education. Third, the navy-blue cluster shows searches related to higher education online, online teaching, online assessment, formative assessment. Fourth, the yellow cluster relates to stress, health, care, quarantine, mental health, anxiety, college students, adults, children. Fifth, the violet cluster shows searches related to surgery, surgical education, skills, strategies, student satisfaction, and simulation. Sixth, the light blue cluster shows searches related to e-learning, performance, quality, remote learning, digital learning, assessment, evaluation. Seventh, the orange cluster shows searches related to education, Covid-19, coronavirus, sars-cov-2, distance learning, medical education. Eighth, the brown cluster included: computer-based learning, self-instruction/distance learning, internet/web-based education, curriculum, knowledge, science, and technology. Finally, the pink clusters showed searches related to artificial intelligence, machine learning, and deep learning. The researcher can also take these subfields as topics for research in e-learning, especially the last cluster, which formed a recent research trend for many scholars ( Bhardwaj et al., 2021 ; Kashive et al., 2021 ; Rasheed and Wahid, 2021 ).

Figure 2B shows that the research on this topic requires focusing on several issues. These are the most frequently mentioned keywords in Supplementary Table 1 , including COVID-19 crisis, technology acceptance model (TAM), distance education, stress, ICT, special education needs, mental health, student satisfaction, surgical teaching, self-efficacy, technology adoption, using the machine, and e-learning. At the same time, many studies used different terms to express the same meaning, such as interactive learning, online learning, and Distance learning. This is similar to what was found in previous studies on e-learning ( Chiang et al., 2010 ; Cheng et al., 2014 ; Bai et al., 2020 ; Fatimah et al., 2021 ).

Reference Authors

Figures 3A–C show the network of the most referenced authors on the topic of “E-learning in higher education in COVID-19” based on co-authorship:

Figure 3. (A) Network of authors (VOSviewer outputs). (B) Publications and citations per author (KnowledgeMatrix Plus outputs). (C) Network of cited authors in COVID-19 (CiteSpace outputs).

Figure 3A shows that there is a research partnership between eight authors. The co-authorship is the affiliation and the country: Fernando Augusto Bozza, Rosana Souza Rodrigues, Walter Araujo Zin, Alan Guimaraes and Gabriel Madeira Werberich, Federal University of Rio de Janeiro, Brazil. Joana Sofia F. Pinto, Willian Reboucas Schmitt and Manuela Franca, Complexo Hosp Univ Porto, Radiol Dept, Porto, Portugal. As for the rest, they have separate and individual publications. Figures 3A–C present the top authors based on publications and citations.

Figure 3B shows that the first author on this topic on “E-learning in higher education in COVID-19” is Antonio José Moreno-Guerrero, Univ Granada, Dept Didact & Sch Org, Spain. Among this research, we find “Impact of Educational Stage in the Application of Flipped Learning: A Contrasting Analysis with Traditional Teaching” ( Pozo Sánchez et al., 2019 ). We also find research on e-learning in mathematics teaching: an educational experience in adult high school ( Moreno-Guerrero et al., 2020 ) as well as research on the following: the effectiveness of innovating educational practices with flipped learning and remote sensing in earth and environmental sciences ( López Núñez et al., 2020 ); machine learning and big data and their impact on literature; a bibliometric review with scientific mapping in WoS; and a flipped learning approach as an educational innovation in water literacy ( López Belmonte et al., 2020 ; López Núñez et al., 2020 ). Moreno-Guerrero talked about e-learning and did not discuss the COVID-19 ( Moreno-Guerrero et al., 2020 ); otherwise, Lüftenegger discussed e-learning and COVID-19 ( Holzer et al., 2021 ; Korlat et al., 2021 ; Pelikan et al., 2021 ).

Figure 3C shows that the most important authors searched in COVID-19 and touched on e-learning are Maram Meccawy, Isabel Chiyon, and Anand Nayyar among others.

Reference Organizations

Figures 4A–C displays the most referenced organizations on the topic of “E-learning in higher education in COVID-19” based on publications, citations, and co-authorship.

Figure 4. (A) Network of organizations (VOSviewer outputs). (B) Network of organizations (CiteSpace outputs). (C) Citations per publications by the organization (KnowledgeMatrix Plus outputs).

Figures 4A–C demonstrate that the leading research organization for publications, citations, and co-authorship on this topic is the University of Toronto with 16 publications and 207 citations, followed by the University of King Abdulaziz with 15 publications and 57 citations the Jordan University of Science and Technology with 11 publications and 115 citations, then the University of Vienna with 10 publications and 30 citations, then the University of Sharjah with 10 publications and 20 citations, then the University of Granada with 9 publications and 79 citations, then the University of Porto with 9 publications and 14 citations, then Monterrey Institute of Technology and Graduate Studies with 9 publications and 2 citations, then the University of Jordan with 8 publications and 46 citations, and finally, the University of Colorado with 8 publications and 16 citations. That is due to several reasons, including the interest of these organizations in publishing in the WoS database. Then their interest in publishing in the subject of the study. We thus find it among the top 500 universities. 1

Reference Countries

Figures 5A–C display the most referenced countries on the topic of “E-learning in higher education in COVID-19” based on publications, citations, and co-authorship.

Figure 5. (A) Network of countries (VOSviewer outputs). (B) Network of countries (CiteSpace outputs). (C) Citations per publications by country (KnowledgeMatrix Plus outputs).

Figures 5A–C illustrate that the top countries for publications, citations, and co-authorship in this topic are as follows: the United States with 344 publications and 1,167 citations, the United Kingdom with 132 publications and 530 citations, China with 117 publications and 592 citations, Spain with 104 publications and 321 citations, Italy with 98 publications and 175 citations, Brazil with 74 publications and 224 citations, Canada with 67 publications and 368 citations, India with 64 publications and 139 citations, Saudi Arabia with 60 publications and 216 citations, and Germany with 59 publications and 133 citations. These show extensive collaboration, especially between the United States and the United Kingdom with 11 collaborations, between the United States and Canada with 10 collaborations, and between the United States and China with 9 collaborations; other countries show an average of 3–5 collaborations.

The results of the bibliometric analysis showed that there are nine sub-fields of research within a topic: motivation and students’ attitudes to e-learning systems in higher education (technology acceptance model), comparison between blended learning and virtual learning, online assessment versus formative assessment of students in higher education, stress, anxiety, and mental health of college students in COVID-19, surgical education strategies to develop students’ skills, quality and performance of higher education strategies of e-learning in COVID-19, challenges of medical education and distance learning during COVID-19, and changing higher education curricula using technology.

Finally, using artificial intelligence, machine learning, and deep learning to transform the e-learning Industry, this final sub-field formed a recent research trend for many scholars ( Bhardwaj et al., 2021 ; Kashive et al., 2021 ; Rasheed and Wahid, 2021 ).

The bibliometric study shows that the first author in e-learning is Antonio José Moreno-Guerrero, Univ Granada, Dept Didact & Sch Org, and Spain. His writings ( Pozo Sánchez et al., 2019 ; López Núñez et al., 2020 ; Moreno-Guerrero et al., 2020 ) are considered a useful reference in e-learning and blended learning. Therefore, Marko Lüftenegger is one of the most influential author in the topic of “E-learning in higher education in COVID-19” ( Holzer et al., 2021 ; Korlat et al., 2021 ; Pelikan et al., 2021 )

The results of the bibliometric analysis showed that the top research organizations in this domain are as follows: the University of Toronto, the University of King Abdulaziz, Jordan University of Science and Technology, the University of Vienna, the University of Sharjah, the University of Granada, the University of Porto, Monterrey Institute of Technology and Graduate Studies, the University of Jordan, and the University of Colorado. The results also illustrate that the top countries are: United States, United Kingdom, China, Spain, Italy, Brazil, Canada, India, Saudi Arabia, Germany, due to several reasons, including the interest of these organizations and countries in publishing in the Web of Science database and their interest in publishing in the subject of the study.

Our research overlaps with that of López-Belmonte et al. (2021) , who tried to investigate the development of e-learning in higher education in the academic literature listed on the WoS. The same analysis, as well as bibliometric analysis, was carried out. The findings revealed no set path for research because of the research on e-learning in higher education, recent creation, and a scarcity of relevant research. According to the results of the bibliometric analysis, the study was aimed at determining acceptance and implementation of the educational curriculum in the teaching and learning processes.

This paper discusses the use of a bibliometric approach to track e-learning trends in higher education during the COVID-19 pandemic through the WoS database. From a methodological perspective, our proposed approach can visually represent the temporal links of the most cited articles internally in various streams and provide a comprehensive overview of the evolution of topics in the WoS database. Also, direct citation network analysis enables researchers to test articles important in e-learning and get a comprehensive overview of the issues published.

The study provided an insight into the world’s e-learning research in terms of mapping research publications. A scientific study was conducted using 602 e-learning documents from 2020 to 2021, and these were obtained through the WoS database. Over the years, the analysis identified trends in contributions in this area and headline sources for most researchers and leading institutions. The study is convergent with many previous studies in this area, including Chiang et al. (2010) , Hung (2012) , Cheng et al. (2014) , Tibaná-Herrera et al. (2018b) , Fatima and Abu (2019) , Bai et al. (2020) , and Mashroofa et al. (2020) . However, our study relies on many software to compare various theoretical models and networks of e-learning.

Based on the analysis data’s inference, growth trends in research publishing in e-learning of different forms have increased in recent years, especially so for the last 2 years (230 in 2020 and 386 in 2021). The significant findings of the bibliometric analysis are as follows: there are nine sub-fields of study in the subject of “E-learning in higher education in COVID-19,” and the prominent authors in this area are as follows: Antonio José Moreno-Guerrero and Marko Lüftenegger; the University of Toronto Canada is the most frequently cited organization in this domain; the United States is the leading country in terms of publications and citations; and the sub-field of artificial intelligence, machine learning, and deep learning to transform the eLearning Industry has emerged as a recent research trend for many scholars.

The study examined a very important topic, which is one of the current topics, “e-learning in higher education during COVID-19,” using bibliometric analysis of 602 studies published in Web of Science databases from 2020 to 2021. We found that the study sample should be larger; it needs further studies and a longer time, especially when we analyze citation, and research on this topic will thus continue in future years. Also, there are many tools and methods used in the bibliometric analysis that were not used in our study, including what has been mentioned ( Tibaná-Herrera et al., 2018b ; Gul et al., 2020 ; López-Belmonte et al., 2021 ; Rashid et al., 2021 ).

The findings of this study will assist interested academics and educational policymakers ( Brika et al., 2021 ) in the field of e-learning in understanding the current state of e-learning and identifying the different research trends in light of COVID-19. Additionally, it will serve as the beginning point for new research during the COVID-19 crisis, which will examine various problems and trends.

The findings of this research may help evaluate e-learning institutions’ quality and promote future educational trends. The findings may be utilized by e-learning institutions to evaluate quality as strategic dimensions and policy makers’ vision.

Data Availability Statement

The original contributions presented in the study are included in the article/ Supplementary Material , further inquiries can be directed to the corresponding author.

Author Contributions

All authors contributed to the design and implementation of the research, performed the revision, verified the analytical methods, supervised the findings of this work, discussed the results, and contributed to the final manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors extend their appreciation to the Deputyship for Research & Innovation, Ministry of Education in Saudi Arabia, to fund this research work through the project number (UB-56-1442).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.762819/full#supplementary-material

• ^ https://www.topuniversities.com/university-rankings/world-university-rankings/2021 , https://www.timeshighereducation.com/world-university-rankings/2021/world-ranking#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/stats

Araka, E., Maina, E., Gitonga, R., and Oboko, R. (2020). Research trends in measurement and intervention tools for self-regulated learning for e-learning environments—systematic review (2008–2018). Res. Pract. Technol. Enhanc. Learn. 15, 1–21.

Google Scholar

Bai, Y., Li, H., and Liu, Y. (2020). Visualizing research trends and research theme evolution in the E-learning field: 1999–2018. Scientometrics 126, 1389–1414. doi: 10.1007/s11192-020-03760-7

CrossRef Full Text | Google Scholar

Bhardwaj, P., Gupta, P. K., Panwar, H., Siddiqui, M. K., Morales-Menendez, R., and Bhaik, A. (2021). Application of deep learning on student engagement in e-learning environments. Comput. Electr. Eng. 93:107277. doi: 10.1016/j.compeleceng.2021.107277

Birkle, C., Pendlebury, D. A., Schnell, J., and Adams, J. (2020). Web of Science as a data source for research on scientific and scholarly activity. Quant. Sci. Stud. 1, 363–376. doi: 10.1162/qss_a_00018

Brika, S. K. M., Algamdi, A., Chergui, K., Musa, A. A., and Zouaghi, R. (2021). Quality of higher education: a bibliometric review study. Front. Educ. 6:666087. doi: 10.3389/feduc.2021.666087

Cabero-Almenara, J., Marín-Díaz, V., and Sampedro-Requena, B. E. (2016). Meta-analysis of research in e-learning published in Spanish journals. Int. J. Educ. Technol. High. Educ. 13, 25.

Castro-Schez, J. J., Glez-Morcillo, C., Albusac, J., and Vallejo, D. (2020). An intelligent tutoring system for supporting active learning: a case study on predictive parsing learning. Inf. Sci. 544, 446–468. doi: 10.1016/j.ins.2020.08.079

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, C. (2016). CiteSpace: A Practical Guide for Mapping Scientific Literature. Hauppauge, NY: Nova Publishers.

Chen, C., and Song, M. (2017). “Science mapping tools and applications,” in Representing Scientific Knowledge (Cham: Springer), 57–137.

Cheng, B., Wang, M., Mørch, A. I., Chen, N. S., and Spector, J. M. (2014). Research on e-learning in the workplace 2000–2012: a bibliometric analysis of the literature. Educ. Res. Rev. 11, 56–72.

Chiang, J. K., Kuo, C. W., and Yang, Y. H. (2010). “A bibliometric study of e-learning literature on SSCI database,” in Proceedings of the International Conference on Technologies for E-Learning and Digital Entertainment , (Berlin: Springer), 145–155.

de Moura, V. F., de Souza, C. A., and Viana, A. B. N. (2020). The use of Massive Open Online Courses (MOOCs) in blended learning courses and the functional value perceived by students. Comput. Educ. 161, 104077. doi: 10.1016/j.compedu.2020.104077

Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., and Pappas, G. (2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J. 22, 338–342. doi: 10.1096/fj.07-9492lsf

Fatima, N., and Abu, K. S. (2019). E-learning research papers in web of science: a bibliometric analysis. Libr. Philos. Pract. 1–14.

Fatimah, F. A. T. I. M. A. H., Rajiani, S., and Abbas, E. (2021). Cultural and individual characteristics in adopting computer-supported collaborative learning during covid-19 outbreak: Willingness or obligatory to accept technology? Manag. Sci. Lett. 11, 373–378. doi: 10.5267/j.msl.2020.9.032

Furstenau, L. B., Rabaioli, B., Sott, M. K., Cossul, D., Bender, M. S., Farina, E. M. J. D. M., et al. (2021). A bibliometric network analysis of coronavirus during the first eight months of COVID-19 in 2020. Int. J. Environ. Res. Public Health 18:952. doi: 10.3390/ijerph18030952

Garcia, R., Falkner, K., and Vivian, R. (2018). Systematic literature review: self-regulated learning strategies using e-learning tools for Computer Science. Comput. Educ. 123, 150–163. doi: 10.1016/j.compedu.2018.05.006

Gul, S., Ur Rehman, S., Ashiq, M., and Khattak, A. (2020). Mapping the scientific literature on COVID-19 and mental health. Psychiatr. Danub. 32:463. doi: 10.24869/psyd.2020.463

Hilmi, M. F., and Mustapha, Y. (2020). “E-Learning research in the middle east: a bibliometric analysis,” in Proceedings of the 2020 Sixth International Conference on e-Learning (econf) , (Piscataway, NJ: IEEE), 243–248. doi: 10.1109/econf51404.2020.9385513

Holzer, J., Lüftenegger, M., Korlat, S., Pelikan, E., Salmela-Aro, K., Spiel, C., et al. (2021). Higher education in times of COVID-19: university students’ basic need satisfaction, self-regulated learning, and well-being. AERA Open 7, 1–13. doi: 10.1177/23328584211003164

Hung, J. L. (2012). Trends of e-learning research from 2000 to 2008: use of text mining and bibliometrics. Br. J. Educ. Technol. 43, 5–16.

Kao, C. W. (2020). The effect of a digital game-based learning task on the acquisition of the English Article System. System 95, 102373.

Karakose, T., and Demirkol, M. (2021). Exploring the emerging COVID-19 research trends and current status in the field of education: a bibliometric analysis and knowledge mapping. Educ. Process Int. J. 10, 7–27. doi: 10.22521/edupij.2021.102.1

Kashive, N., Powale, L., and Kashive, K. (2021). Understanding user perception toward artificial intelligence (AI) enabled e-learning. Int. J. Inf. Learn. Technol. 38, 1–19. doi: 10.1108/IJILT-05-2020-0090

Korlat, S., Kollmayer, M., Holzer, J., Lüftenegger, M., Pelikan, E., Schober, B., et al. (2021). Gender differences in digital learning during COVID-19: competence beliefs, intrinsic value, learning engagement, and perceived teacher support. Front. Psychol. 12:63776. doi: 10.3389/fpsyg.2021.637776

Lahti, M., Hätönen, H., and Välimäki, M. (2014). Impact of e-learning on nurses’ and student nurses’ knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int. J. Nurs. Stud. 51, 136–149. doi: 10.1016/j.ijnurstu.2012.12.017

López Belmonte, J., Segura-Robles, A., Moreno-Guerrero, A.-J., and Parra-González, M. E. (2020). Machine learning and big data in impact literature. a bibliometric review with scientific mapping in web of science. Symmetry 12:495.

López Núñez, J. A., López, J., Moreno-Guerrero, A. J., and Pozo, S. (2020). Effectiveness of innovative educational practices with flipped learning and remote sensing in earth and environmental sciences—A case study. Remote Sens. 12:897. doi: 10.3390/rs12050897

López-Belmonte, J., Segura-Robles, A., Moreno-Guerrero, A.-J., and Parra-González, M. E. (2021). Projection of e-learning in higher education: a study of its scientific production in the web of science. Eur. J. Invest. Health Psychol. Educ. 11, 20–32. doi: 10.3390/ejihpe11010003

Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H., and Alharbi, H. (2021). The COVID-19 pandemic and E-learning: challenges and opportunities from the perspective of students and instructors. J. Comput. High. Educ. 1–18. doi: 10.1007/s12528-021-09274-2

Mashroofa, M. M., Jusoh, M., and Chinna, K. (2020). Bibliometric Analysis on Global e-Learning Literature in the Web of Science Database: With Special Reference to Sri Lankan Context. Lincoln, NE: University Libraries of the University of Nebraska–Lincoln.

Moreno-Guerrero, A.-J., Aznar-Díaz, I., Cáceres-Reche, P., and Alonso-García, S. (2020). E-Learning in the teaching of mathematics: an educational experience in adult high school. Mathematics 8:840. doi: 10.3390/math8050840

Mothibi, G. (2015). A meta-analysis of the relationship between E-Learning and students’ academic achievement in higher education. J. Educ. Pract. 6, 6–9.

Nicholson, P. (2007). “A history of e-learning: echoes of the pioneers,” in Com-puters and Education: e-Learning, From Theory to Practice , ed. B. Fernández Manjón (Dordrecht: Springer), 1–11. doi: 10.1007/978-1-4020-4914-9_1

Nylund, H., and Lanz, M. (2020). Interactive learning activities for the education of factory-level order-to-delivery processes. Procedia Manuf. 45, 504–509. doi: 10.1016/j.promfg.2020.04.065

Oprea, C. L. (2014). The Internet-a tool for interactive learning. Procedia Soc. Behav. Sci. 142, 786–792. doi: 10.1016/j.sbspro.2014.07.617

Pal, D., and Vanijja, V. (2020). Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India. Child. Youth Serv. Rev. 119:105535. doi: 10.1016/j.childyouth.2020.105535

Patricia, A. (2020). College students’ use and acceptance of emergency online learning due to Covid-19. Int. J. Educ. Res. Open 1:100011. doi: 10.1016/j.ijedro.2020.100011

Pelikan, E. R., Lüftenegger, M., Holzer, J., Korlat, S., Spiel, C., and Schober, B. (2021). Learning during COVID-19: the role of self-regulated learning, motivation, and procrastination for perceived competence. Z. Erziehungswiss. 24, 393–418. doi: 10.1007/s11618-021-01002-x

Pozo Sánchez, S., López Belmonte, J., Moreno Guerrero, A. J., and López Núñez, J. A. (2019). Impact of educational stage in the application of flipped learning: a contrasting analysis with traditional teaching. Sustainability 11:5968. doi: 10.3390/su11215968

Rasheed, F., and Wahid, A. (2021). Learning style detection in E-learning systems using machine learning techniques. Expert Syst. Appl. 174:114774. doi: 10.1016/j.eswa.2021.114774

Rashid, S., Rehman, S. U., Ashiq, M., and Khattak, A. (2021). A scientometric analysis of forty-three years of research in social support in education (1977–2020). Educ. Sci. 11:149. doi: 10.3390/educsci11040149

Rodrigues, H., Almeida, F., Figueiredo, V., and Lopes, S. L. (2019). Tracking e-learning through published papers: a systematic review. Comput. Educ. 136, 87–98.

Şerban, C., and Ioan, L. (2020). QLearn: towards a framework for smart learning environments. Procedia Comput. Sci. 176, 2812–2821. doi: 10.1016/j.procs.2020.09.273

Siemens, G. (2005). Connectivism: a learning theory for the digital age. Int. J. Instr. Technol. Distance Learn. 2, 3–10.

ŠUmak, B., HeričKo, M., and PušNik, M. (2011). A meta-analysis of e-learning technology acceptance: the role of user types and e-learning technology types. Comput. Hum. Behav. 27, 2067–2077. doi: 10.1016/j.chb.2011.08.005

Tesar, M. (2020). Towards a post-Covid-19 “New Normality?”: physical and social distancing, the move to online and higher education. Policy Future Educ. 18, 556–559.

Tibaná-Herrera, G., Fernández-Bajón, M. T., and De Moya-Anegón, F. (2018a). Categorization of E-learning as an emerging discipline in the world publication system: a bibliometric study in SCOPUS. Int. J. Educ. Technol. High. Educ. 15, 1–23.

Tibaná-Herrera, G., Fernández-Bajón, M. T., and de Moya-Anegón, F. (2018c). Global analysis of the E-learning scientific domain: a declining category? Scientometrics 114, 675–685. doi: 10.1007/s11192-017-2592-7

Tibaná-Herrera, G., Fernández-Bajón, M. T., and de Moya Anegón, F. (2018b). “Output, collaboration and impact of e-learning research: bibliometric analysis and visualizations at the country and institutional level (Scopus 2003-2016). Prof. Inf. 27, 1082–1096. doi: 10.3145/epi.2018.sep.12

Valverde-Berrocoso, J., Garrido-Arroyo, M. D. C., Burgos-Videla, C., and Morales-Cevallos, M. B. (2020). Trends in educational research about e-learning: a systematic literature Review (2009–2018). Sustainability 12:5153.

Van Eck, N. J., and Waltman, L. (2013). VOSviewer Manual. Leiden: Univeristeit Leiden. 1–53.

Yuwono, K. T., and Sujono, H. D. (2018). The effectiveness of E-Learning: a meta-analysis. J. Phys. Conf. Ser. 1140:012024.

Zare, M., Pahl, C., Rahnama, H., Nilashi, M., Mardani, A., Ibrahim, O., et al. (2016). Multi-criteria decision-making approach in E-learning: a systematic review and classification. Appl. Soft Comput. 45, 108–128.

Zupic, I., and Čater, T. (2015). Bibliometric methods in management and organization. Organ. Res. Methods 18, 429–472.

Keywords : e-learning, higher education, COVID-19, bibliometric analysis, Web of Science (WoS) database

Citation: Brika SKM, Chergui K, Algamdi A, Musa AA and Zouaghi R (2022) E-Learning Research Trends in Higher Education in Light of COVID-19: A Bibliometric Analysis. Front. Psychol. 12:762819. doi: 10.3389/fpsyg.2021.762819

Received: 22 August 2021; Accepted: 31 December 2021; Published: 03 March 2022.

Reviewed by:

Copyright © 2022 Brika, Chergui, Algamdi, Musa and Zouaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Said Khalfa Mokhtar Brika, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Research article
• Open access
• Published: 01 October 2021

Adaptive e-learning environment based on learning styles and its impact on development students' engagement

• Hassan A. El-Sabagh   ORCID: orcid.org/0000-0001-5463-5982 1 , 2  

International Journal of Educational Technology in Higher Education volume  18 , Article number:  53 ( 2021 ) Cite this article

67k Accesses

68 Citations

27 Altmetric

Metrics details

Adaptive e-learning is viewed as stimulation to support learning and improve student engagement, so designing appropriate adaptive e-learning environments contributes to personalizing instruction to reinforce learning outcomes. The purpose of this paper is to design an adaptive e-learning environment based on students' learning styles and study the impact of the adaptive e-learning environment on students’ engagement. This research attempts as well to outline and compare the proposed adaptive e-learning environment with a conventional e-learning approach. The paper is based on mixed research methods that were used to study the impact as follows: Development method is used in designing the adaptive e-learning environment, a quasi-experimental research design for conducting the research experiment. The student engagement scale is used to measure the following affective and behavioral factors of engagement (skills, participation/interaction, performance, emotional). The results revealed that the experimental group is statistically significantly higher than those in the control group. These experimental results imply the potential of an adaptive e-learning environment to engage students towards learning. Several practical recommendations forward from this paper: how to design a base for adaptive e-learning based on the learning styles and their implementation; how to increase the impact of adaptive e-learning in education; how to raise cost efficiency of education. The proposed adaptive e-learning approach and the results can help e-learning institutes in designing and developing more customized and adaptive e-learning environments to reinforce student engagement.

Introduction

In recent years, educational technology has advanced at a rapid rate. Once learning experiences are customized, e-learning content becomes richer and more diverse (El-Sabagh & Hamed, 2020 ; Yang et al., 2013 ). E-learning produces constructive learning outcomes, as it allows students to actively participate in learning at anytime and anyplace (Chen et al., 2010 ; Lee et al., 2019 ). Recently, adaptive e-learning has become an approach that is widely implemented by higher education institutions. The adaptive e-learning environment (ALE) is an emerging research field that deals with the development approach to fulfill students' learning styles by adapting the learning environment within the learning management system "LMS" to change the concept of delivering e-content. Adaptive e-learning is a learning process in which the content is taught or adapted based on the responses of the students' learning styles or preferences. (Normadhi et al., 2019 ; Oxman & Wong, 2014 ). By offering customized content, adaptive e-learning environments improve the quality of online learning. The customized environment should be adaptable based on the needs and learning styles of each student in the same course. (Franzoni & Assar, 2009 ; Kolekar et al., 2017 ). Adaptive e-learning changes the level of instruction dynamically based on student learning styles and personalizes instruction to enhance or accelerate a student's success. Directing instruction to each student's strengths and content needs can minimize course dropout rates, increase student outcomes and the speed at which they are accomplished. The personalized learning approach focuses on providing an effective, customized, and efficient path of learning so that every student can participate in the learning process (Hussein & Al-Chalabi, 2020 ). Learning styles, on the other hand, represent an important issue in learning in the twenty-first century, with students expected to participate actively in developing self-understanding as well as their environment engagement. (Klasnja-Milicevic et al., 2011 ; Nuankaew et al., 2019 ; Truong, 2016 ).

In current conventional e-learning environments, instruction has traditionally followed a “one style fits all” approach, which means that all students are exposed to the same learning procedures. This type of learning does not take into account the different learning styles and preferences of students. Currently, the development of e-learning systems has accommodated and supported personalized learning, in which instruction is fitted to a students’ individual needs and learning styles (Beldagli & Adiguzel, 2010 ; Benhamdi et al., 2017 ; Pashler et al., 2008 ). Some personalized approaches let students choose content that matches their personality (Hussein & Al-Chalabi, 2020 ). The delivery of course materials is an important issue of personalized learning. Moreover, designing a well-designed, effective, adaptive e-learning system represents a challenge due to complication of adapting to the different needs of learners (Alshammari, 2016 ). Regardless of using e-learning claims that shifting to adaptive e-learning environments to be able to reinforce students' engagement. However, a learning environment cannot be considered adaptive if it is not flexible enough to accommodate students' learning styles. (Ennouamani & Mahani, 2017 ).

On the other hand, while student engagement has become a central issue in learning, it is also an indicator of educational quality and whether active learning occurs in classes. (Lee et al., 2019 ; Nkomo et al., 2021 ; Robinson & Hullinger, 2008 ). Veiga et al. ( 2014 ) suggest that there is a need for further research in engagement because assessing students’ engagement is a predictor of learning and academic progress. It is important to clarify the distinction between causal factors such as learning environment and outcome factors such as achievement. Accordingly, student engagement is an important research topic because it affects a student's final grade, and course dropout rate (Staikopoulos et al., 2015 ).

The Umm Al-Qura University strategic plan through common first-year deanship has focused on best practices that increase students' higher-order skills. These skills include communication skills, problem-solving skills, research skills, and creative thinking skills. Although the UQU action plan involves improving these skills through common first-year academic programs, the student's learning skills need to be encouraged and engaged more (Umm Al-Qura University Agency, 2020 ). As a result of the author's experience, The conventional methods of instruction in the "learning skills" course were observed, in which the content is presented to all students in one style that is dependent on understanding the content regardless of the diversity of their learning styles.

According to some studies (Alshammari & Qtaish, 2019 ; Lee & Kim, 2012 ; Shih et al., 2008 ; Verdú, et al., 2008 ; Yalcinalp & Avc, 2019 ), there is little attention paid to the needs and preferences of individual learners, and as a result, all learners are treated in the same way. More research into the impact of educational technologies on developing skills and performance among different learners is recommended. This “one-style-fits-all” approach implies that all learners are expected to use the same learning style as prescribed by the e-learning environment. Subsequently, a review of the literature revealed that an adaptive e-learning environment can affect learning outcomes to fill the identified gap. In conclusion: Adaptive e-learning environments rely on the learner's preferences and learning style as a reference that supports to create adaptation.

To confirm the above: the author conducted an exploratory study via an open interview that included some questions with a sample of 50 students in the learning skills department of common first-year. Questions asked about the difficulties they face when learning a "learning skills" course, what is the preferred way of course content. Students (88%) agreed that the way students are presented does not differ according to their differences and that they suffer from a lack of personal learning that is compatible with their style of work. Students (82%) agreed that they lack adaptive educational content that helps them to be engaged in the learning process. Accordingly, the author handled the research problem.

This research supplements to the existing body of knowledge on the subject. It is considered significant because it improves understanding challenges involved in designing the adaptive environments based on learning styles parameter. Subsequently, this paper is structured as follows: The next section presents the related work cited in the literature, followed by research methodology, then data collection, results, discussion, and finally, some conclusions and future trends are discussed.

Theoretical framework

This section briefly provides a thorough review of the literature about the adaptive E-learning environments based on learning styles.

Adaptive e-learning environments based on learning styles

The adaptive e-learning employment in higher education has been slower to evolve, and challenges that led to the slow implementation still exist. The learning management system offers the same tools to all learners, although individual learners need different details based on learning style and preferences. (Beldagli & Adiguzel, 2010 ; Kolekar et al., 2017 ). The interactive e-learning environment requisite evaluating the learner's desired learning style, before the course delivery, such as an online quiz or during the course delivery, such as tracking student reactions (DeCapua & Marshall, 2015 ).

In e-learning environments, adaptation is constructed on a series of well-designed processes to fit the instructional materials. The adaptive e-learning framework attempt to match instructional content to the learners' needs and styles. According to Qazdar et al. ( 2015 ), adaptive e-learning (AEL) environments rely on constructing a model of each learner's needs, preferences, and styles. It is well recognized that such adaptive behavior can increase learners' development and performance, thus enriching learning experience quality. (Shi et al., 2013 ). The following features of adaptive e-learning environments can be identified through diversity, interactivity, adaptability, feedback, performance, and predictability. Although adaptive framework taxonomy and characteristics related to various elements, adaptive learning includes at least three elements: a model of the structure of the content to be learned with detailed learning outcomes (a content model). The student's expertise based on success, as well as a method of interpreting student strengths (a learner model), and a method of matching the instructional materials and how it is delivered in a customized way (an instructional model) (Ali et al., 2019 ). The number of adaptive e-learning studies has increased over the last few years. Adaptive e-learning is likely to increase at an accelerating pace at all levels of instruction (Hussein & Al-Chalabi, 2020 ; Oxman & Wong, 2014 ).

Many studies assured the power of adaptive e-learning in delivering e-content for learners in a way that fitting their needs, and learning styles, which helps improve the process of students' acquisition of knowledge, experiences and develop their higher thinking skills (Ali et al., 2019 ; Behaz & Djoudi, 2012 ; Chun-Hui et al., 2017 ; Daines et al., 2016 ; Dominic et al., 2015 ; Mahnane et al., 2013 ; Vassileva, 2012 ). Student characteristics of learning style are recognized as an important issue and a vital influence in learning and are frequently used as a foundation to generate personalized learning experiences (Alshammari & Qtaish, 2019 ; El-Sabagh & Hamed, 2020 ; Hussein & Al-Chalabi, 2020 ; Klasnja-Milicevic et al., 2011 ; Normadhi et al., 2019 ; Ozyurt & Ozyurt, 2015 ).

The learning style is a parameter of designing adaptive e-learning environments. Individuals differ in their learning styles when interacting with the content presented to them, as many studies emphasized the relationship between e-learning and learning styles to be motivated in learning situations, consequently improving the learning outcomes (Ali et al., 2019 ; Alshammari, 2016 ; Alzain et al., 2018a , b ; Liang, 2012 ; Mahnane et al., 2013 ; Nainie et al., 2010 ; Velázquez & Assar, 2009 ). The word "learning style" refers to the process by which the learner organizes, processes, represents, and combines this information and stores it in his cognitive source, then retrieves the information and experiences in the style that reflects his technique of communicating them. (Fleming & Baume, 2006 ; Jaleel & Thomas, 2019 ; Jonassen & Grabowski, 2012 ; Klasnja-Milicevic et al., 2011 ; Nuankaew et al., 2019 ; Pashler et al., 2008 ; Willingham et al., 2105 ; Zhang, 2017 ). The concept of learning style is founded based on the fact that students vary in their styles of receiving knowledge and thought, to help them recognizing and combining information in their mind, as well as acquire experiences and skills. (Naqeeb, 2011 ). The extensive scholarly literature on learning styles is distributed with few strong experimental findings (Truong, 2016 ), and a few findings on the effect of adapting instruction to learning style. There are many models of learning styles (Aldosarim et al., 2018 ; Alzain et al., 2018a , 2018b ; Cletus & Eneluwe, 2020 ; Franzoni & Assar, 2009 ; Willingham et al., 2015 ), including the VARK model, which is one of the most well-known models used to classify learning styles. The VARK questionnaire offers better thought about information processing preferences (Johnson, 2009 ). Fleming and Baume ( 2006 ) developed the VARK model, which consists of four students' preferred learning types. The letter "V" represents for visual and means the visual style, while the letter "A" represents for auditory and means the auditory style, and the letter "R/W" represents "write/read", means the reading/writing style, and the letter "K" represents the word "Kinesthetic" and means the practical style. Moreover, VARK distinguishes the visual category further into graphical and textual or visual and read/write learners (Murphy et al., 2004 ; Leung, et al., 2014 ; Willingham et al., 2015 ). The four categories of The VARK Learning Style Inventory are shown in the Fig. 1 below.

VARK learning styles

According to the VARK model, learners are classified into four groups representing basic learning styles based on their responses which have 16 questions, there are four potential responses to each question, where each answer agrees to one of the extremes of the dimension (Hussain, 2017 ; Silva, 2020 ; Zhang, 2017 ) to support instructors who use it to create effective courses for students. Visual learners prefer to take instructional materials and send assignments using tools such as maps, graphs, images, and other symbols, according to Fleming and Baume ( 2006 ). Learners who can read–write prefer to use written textual learning materials, they use glossaries, handouts, textbooks, and lecture notes. Aural learners, on the other hand, prefer to learn through spoken materials, dialogue, lectures, and discussions. Direct practice and learning by doing are preferred by kinesthetic learners (Becker et al., 2007 ; Fleming & Baume, 2006 ; Willingham et al., 2015 ). As a result, this research work aims to provide a comprehensive discussion about how these individual parameters can be applied in adaptive e-learning environment practices. Dominic et al., ( 2015 ) presented a framework for an adaptive educational system that personalized learning content based on student learning styles (Felder-Silverman learning model) and other factors such as learners' learning subject competency level. This framework allowed students to follow their adaptive learning content paths based on filling in "ils" questionnaire. Additionally, providing a customized framework that can automatically respond to students' learning styles and suggest online activities with complete personalization. Similarly, El Bachari et al. ( 2011 ) attempted to determine a student's unique learning style and then adapt instruction to that individual interests. Adaptive e-learning focused on learner experience and learning style has a higher degree of perceived usability than a non-adaptive e-learning system, according to Alshammari et al. ( 2015 ). This can also improve learners' satisfaction, engagement, and motivation, thus improving their learning.

According to the findings of (Akbulut & Cardak, 2012 ; Alshammari & Qtaish, 2019 ; Alzain et al., 2018a , b ; Shi et al., 2013 ; Truong, 2016 ), adaptation based on a combination of learning style, and information level yields significantly better learning gains. Researchers have recently initiated to focus on how to personalize e-learning experiences using personal characteristics such as the student's preferred learning style. Personal learning challenges are addressed by adaptive learning programs, which provide learners with courses that are fit to their specific needs, such as their learning styles.

• Student engagement

Previous research has emphasized that student participation is a key factor in overcoming academic problems such as poor academic performance, isolation, and high dropout rates (Fredricks et al., 2004 ). Student participation is vital to student learning, especially in an online environment where students may feel isolated and disconnected (Dixson, 2015 ). Student engagement is the degree to which students consciously engage with a course's materials, other students, and the instructor. Student engagement is significant for keeping students engaged in the course and, as a result, in their learning (Barkley & Major, 2020 ; Lee et al., 2019 ; Rogers-Stacy, et al, 2017 ). Extensive research was conducted to investigate the degree of student engagement in web-based learning systems and traditional education systems. For instance, using a variety of methods and input features to test the relationship between student data and student participation (Hussain et al., 2018 ). Guo et al. ( 2014 ) checked the participation of students when they watched videos. The input characteristics of the study were based on the time they watched it and how often students respond to the assessment.

Atherton et al. ( 2017 ) found a correlation between the use of course materials and student performance; course content is more expected to lead to better grades. Pardo et al., ( 2016 ) found that interactive students with interactive learning activities have a significant impact on student test scores. The course results are positively correlated with student participation according to previous research. For example, Atherton et al. ( 2017 ) explained that students accessed learning materials online and passed exams regularly to obtain higher test scores. Other studies have shown that students with higher levels of participation in questionnaires and course performance tend to perform well (Mutahi et al., 2017 ).

Skills, emotion, participation, and performance, according to Dixson ( 2015 ), were factors in online learning engagement. Skills are a type of learning that includes things like practicing on a daily foundation, paying attention while listening and reading, and taking notes. Emotion refers to how the learner feels about learning, such as how much you want to learn. Participation refers to how the learner act in a class, such as chat, discussion, or conversation. Performance is a result, such as a good grade or a good test score. In general, engagement indicated that students spend time, energy learning materials, and skills to interact constructively with others in the classroom, and at least participate in emotional learning in one way or another (that is, be motivated by an idea, willing to learn and interact). Student engagement is produced through personal attitudes, thoughts, behaviors, and communication with others. Thoughts, effort, and feelings to a certain level when studying. Therefore, the student engagement scale attempts to measure what students are doing (thinking actively), how they relate to their learning, and how they relate to content, faculty members, and other learners including the following factors as shown in Fig.  2 . (skills, participation/interaction, performance, and emotions). Hence, previous research has moved beyond comparing online and face-to-face classes to investigating ways to improve online learning (Dixson, 2015 ; Gaytan & McEwen, 2007 ; Lévy & Wakabayashi, 2008 ; Mutahi et al., 2017 ). Learning effort, involvement in activities, interaction, and learning satisfaction, according to reviews of previous research on student engagement, are significant measures of student engagement in learning environments (Dixson, 2015 ; Evans et al., 2017 ; Lee et al., 2019 ; Mutahi et al., 2017 ; Rogers-Stacy et al., 2017 ). These results point to several features of e-learning environments that can be used as measures of student participation. Successful and engaged online learners learn actively, have the psychological inspiration to learn, make good use of prior experience, and make successful use of online technology. Furthermore, they have excellent communication abilities and are adept at both cooperative and self-directed learning (Dixson, 2015 ; Hong, 2009 ; Nkomo et al., 2021 ).

Engagement factors

Overview of designing the adaptive e-learning environment

The paper follows the (ADDIE) Instructional Design Model: analysis, design, develop, implement, and evaluate to answer the first research question. The adaptive learning environment offers an interactive decentralized media environment that takes into account individual differences among students. Moreover, the environment can spread the culture of self-learning, attract students, and increase their engagement in learning.

Any learning environment that is intended to accomplish a specific goal should be consistent to increase students' motivation to learn. so that they have content that is personalized to their specific requirements, rather than one-size-fits-all content. As a result, a set of instructional design standards for designing an adaptive e-learning framework based on learning styles was developed according to the following diagram (Fig. 3 ).

The ID (model) of the adaptive e-learning environment

According to the previous figure, The analysis phase included identifying the course materials and learning tools (syllabus and course plan modules) used for the study. The learning objectives were included in the high-level learning objectives (C4-C6: analysis, synthesis, evaluation).

The design phase included writing SMART objectives, the learning materials were written within the modules plan. To support adaptive learning, four content paths were identified, choosing learning models, processes, and evaluation. Course structure and navigation were planned. The adaptive structural design identified the relationships between the different components, such as introduction units, learning materials, quizzes. Determining the four path materials. The course instructional materials were identified according to the following Figure 4 .

Adaptive e-course design

The development phase included: preparing and selecting the media for the e-course according to each content path in an adaptive e-learning environment. During this process, the author accomplished the storyboard and the media to be included on each page of the storyboard. A category was developed for the instructional media for each path (Fig. 5 )

Roles and deployment diagram of the adaptive e-learning environment

The author developed a learning styles questionnaire via a mobile App. as follows: https://play.google.com/store/apps/details?id=com.pointability.vark . Then, the students accessed the adaptive e-course modules based on their learning styles.

The Implementation phase involved the following: The professional validation of the course instructional materials. Expert validation is used to evaluate the consistency of course materials (syllabi and modules). The validation was performed including the following: student learning activities, learning implementation capability, and student reactions to modules. The learner's behaviors, errors, navigation, and learning process are continuously geared toward improving the learner's modules based on the data the learner gathered about him.

The Evaluation phase included five e-learning specialists who reviewed the adaptive e-learning. After that, the framework was revised based on expert recommendations and feedback. Content assessment, media evaluation in three forms, instructional design, interface design, and usage design included in the evaluation. Adaptive learners checked the proposed framework. It was divided into two sections. Pilot testing where the proposed environment was tested by ten learners who represented the sample in the first phase. Each learner's behavior was observed, questions were answered, and learning control, media access, and time spent learning were all verified.

Research methodology

Research purpose and questions.

This research aims to investigate the impact of designing an adaptive e-learning environment on the development of students' engagement. The research conceptual framework is illustrated in Fig.  6 . Therefore, the articulated research questions are as follows: the main research question is "What is the impact of an adaptive e-learning environment based on (VARK) learning styles on developing students' engagement? Accordingly, there are two sub research questions a) "What is the instructional design of the adaptive e-learning environment?" b) "What is the impact of an adaptive e-learning based on (VARK) learning styles on development students' engagement (skills, participation, performance, emotional) in comparison with conventional e-learning?".

The conceptual framework (model) of the research questions

Research hypotheses

The research aims to verify the validity of the following hypothesis:

There is no statistically significant difference between the students' mean scores of the experimental group that exposed to the adaptive e-learning environment and the scores of the control group that was exposed to the conventional e-learning environment in pre-application of students' engagement scale.

There is a statistically significant difference at the level of (0.05) between the students' mean scores of the experimental group (adaptive e-learning) and the scores of the control group (conventional e-learning) in post-application of students' engagement factors in favor of the experimental group.

Research design

This research was a quasi-experimental research with the pretest-posttest. Research variables were independent and dependent as shown in the following Fig. 7 .

Research "Experimental" design

Both groups were informed with the learning activities tracks, the experimental group was instructed to use the adaptive learning environment to accomplish the learning goals; on the other hand, the control group was exposed to the conventional e-learning environment without the adaptive e-learning parameters.

Research participants

The sample consisted of students studying the "learning skills" course in the common first-year deanship aged between (17–18) years represented the population of the study. All participants were chosen in the academic year 2109–2020 at the first term which was taught by the same instructors. The research sample included two classes (118 students), selected randomly from the learning skills department. First-group was randomly assigned as the control group (N = 58, 31 males and 27 females), the other was assigned as experimental group (N = 60, 36 males and 24 females) was assigned to the other class. The following Table 1 shows the distribution of students' sample "Demographics data".

The instructional materials were not presented to the students before. The control group was expected to attend the conventional e-learning class, where they were provided with the learning environment without adaptive e-learning parameter based on the learning styles that introduced the "learning skills" course. The experimental group was exposed to the use of adaptive e-learning based on learning styles to learn the same course instructional materials within e-course. Moreover, all the student participants were required to read the guidelines to indicate their readiness to participate in the research experiment with permission.

Research instruments

In this research, the measuring tools included the VARK questionnaire and the students' engagement scale including the following factors (skills, participation/interaction, performance, emotional). To begin, the pre-post scale was designed to assess the level of student engagement related to the "learning skills" course before and after participating in the experiment.

VARK questionnaire

Questionnaires are a common method for collecting data in education research (McMillan & Schumacher, 2006 ). The VARK questionnaire had been organized electronically and distributed to the student through the developed mobile app and registered on the UQU system. The questionnaire consisted of 16 items within the scale as MCQ classified into four main factors (kinesthetic, auditory, visual, and R/W).

Reliability and Validity of The VARK questionnaire

For reliability analysis, Cronbach’s alpha is used for evaluating research internal consistency. Internal consistency was calculated through the calculation of correlation of each item with the factor to which it fits and correlation among other factors. The value of 0.70 and above are normally recognized as high-reliability values (Hinton et al., 2014 ). The Cronbach's Alpha correlation coefficient for the VARK questionnaire was 0.83, indicating that the questionnaire was accurate and suitable for further research.

Students' engagement scale

The engagement scale was developed after a review of the literature on the topic of student engagement. The Dixson scale was used to measure student engagement. The scale consisted of 4 major factors as follows (skills, participation/interaction, performance, emotional). The author adapted the original "Dixson scale" according to the following steps. The Dixson scale consisted of 48 statements was translated and accommodated into Arabic by the author. After consulting with experts, the instrument items were reduced to 27 items after adaptation according to the university learning environment. The scale is rated on a 5-point scale.

The final version of the engagement scale comprised 4 factors as follows: The skills engagement included (ten items) to determine keeping up with, reading instructional materials, and exerting effort. Participation/interaction engagement involved (five items) to measure having fun, as well as regularly engaging in group discussion. The performance engagement included (five items) to measure test performance and receiving a successful score. The emotional engagement involved (seven items) to decide whether or not the course was interesting. Students can access to respond engagement scale from the following link: http://bit.ly/2PXGvvD . Consequently, the objective of the scale is to measure the possession of common first-year students of the basic engagement factors before and after instruction with adaptive e-learning compared to conventional e-learning.

Reliability and validity of the engagement scale

The alpha coefficient of the scale factors scores was presented. All four subscales have a strong degree of internal accuracy (0.80–0.87), indicating strong reliability. The overall reliability of the instruments used in this study was calculated using Alfa-alpha, Cronbach's with an alpha value of 0.81 meaning that the instruments were accurate. The instruments used in this research demonstrated strong validity and reliability, allowing for an accurate assessment of students' engagement in learning. The scale was applied to a pilot sample of 20 students, not including the experimental sample. The instrument, on the other hand, had a correlation coefficient of (0.74–0.82), indicating a degree of validity that enables the instrument's use. Table 2 shows the correlation coefficient and Cronbach's alpha based on the interaction scale.

On the other hand, to verify the content validity; the scale was to specialists to take their views on the clarity of the linguistic formulation and its suitability to measure students' engagement, and to suggest what they deem appropriate in terms of modifications.

Research procedures

To calculate the homogeneity and group equivalence between both groups, the validity of the first hypothesis was examined which stated "There is no statistically significant difference between the students' mean scores of the experimental group that exposed to the adaptive e-learning environment and the scores of the control group that was exposed to the conventional e-learning environment in pre-application of students' engagement scale", the author applied the engagement scale to both groups beforehand, and the scores of the pre-application were examined to verify the equivalence of the two groups (experimental and control) in terms of students' engagement.

The t-test of independent samples was calculated for the engagement scale to confirm the homogeneity of the two classes before the experiment. The t-values were not significant at the level of significance = 0.05, meaning that the two groups were homogeneous in terms of students' engagement scale before the experiment.

Since there was no significant difference in the mean scores of both groups ( p  > 0.05), the findings presented in Table 3 showed that there was no significant difference between both experimental and control groups in engagement as a whole, and each student engagement factor separately. The findings showed that the two classes were similar before start of research experiment.

Learner content path in adaptive e-learning environment

The previous well-designed processes are the foundation for adaptation in e-learning environments. There are identified entries for accommodating materials, including classification depending on learning style.: kinesthetic, auditory, visual, and R/W. The present study covered the 1st semester during the 2019/2020 academic year. The course was divided into modules that concentrated on various topics; eleven of the modules included the adaptive learning exercise. The exercises and quizzes were assigned to specific textbook modules. To reduce irrelevant variation, all objects of the course covered the same content, had equal learning results, and were taught by the same instructor.

The experimental group—in which students were asked to bring smartphones—was taught, where the how-to adaptive learning application for adaptive learning was downloaded, and a special account was created for each student, followed by access to the channel designed by the through the application, and the students were provided with instructions and training on how entering application with the appropriate default element of the developed learning objects, while the control group used the variety of instructional materials in the same course for the students.

In this adaptive e-course, students in the experimental group are presented with a questionnaire asked to answer that questions via a developed mobile App. They are provided with four choices. Students are allowed to answer the questions. The correct answer is shown in the students' responses to the results, but the learning module is marked as incomplete. If a student chooses to respond to a question, the correct answer is found immediately, regardless of the student's reaction.

Figure  8 illustrates a visual example from learning styles identification through responding VARK Questionnaire. The learning process experienced by the students in this adaptive Learning environment is as shown in Fig.  4 . Students opened the adaptive course link by tapping the following app " https://play.google.com/store/apps/details?id=com.pointability.vark ," which displayed the appropriate positioning of both the learning skills course and the current status of students. It directed students to the learning skills that they are interested in learning more. Once students reached a specific situation in the e-learning environment, they could access relevant digital instructional materials. Students were then able to progress through the various styles offered by the proposed method, giving them greater flexibility in their learning pace.

Visual example from "learning of the learning styles" identification and adaptive e-learning course process

The "flowchart" diagram below illustrates the learner's path in an adaptive e-learning environment, depending on the (VARK) learning styles (visual, auditory, kinesthetic, reading/writing) (Fig. 9 ).

Student learning path

According to the previous design model of the adaptive framework, the students responded "Learning Styles" questionnaire. Based on each student's results, the orientation of students will direct to each of "Visual", "Aural", "Read-Write", and "Kinesthetic". The student took at the beginning the engagement scale online according to their own pace. When ready, they responded "engagement scale".

Based on the results, the system produced an individualized learning plan to fill in the gap based on the VARK questionnaire's first results. The learner model represents important learner characteristics such as personal information, knowledge level, and learning preferences. Pre and post measurements were performed for both experimental and control groups. The experimental group was exposed only to treatment (using the adaptive learning environment).

To address the second question, which states: “What is the impact "effect" of adaptive e-learning based on (VARK) learning styles on development students' engagement (skills, participation/interaction, performance, emotional) in comparison with conventional e-learning?

The validity of the second hypothesis of the research hypothesis was tested, which states " There is a statistically significant difference at the level of (0.05) between the students' mean scores of the experimental group (adaptive e-learning) and the scores of the control group (conventional e-learning) in post-application of students' engagement factors in favor of the experimental group". To test the hypothesis, the arithmetic means, standard deviations, and "T"-test values were calculated for the results of the two research groups in the application of engagement scale factors".

Table 4 . indicates that students in the experimental group had significantly higher mean of engagement post-test (engagement factors items) scores than students in the control group ( p  < 0.05).

The experimental research was performed to evaluate the impact of the proposed adaptive e-learning. Independent sample t-tests were used to measure the previous behavioral engagement of the two groups related to topic of this research. Subsequently, the findings stated that the experimental group students had higher learning achievement than those who were taught using the conventional e-learning approach.

To verify the effect size of the independent variable in terms of the dependent variable, Cohen (d) was used to investigate that adaptive learning can significantly students' engagement. According to Cohen ( 1992 ), ES of 0.20 is small, 0.50 is medium, and 0.80 is high. In the post-test of the student engagement scale, however, the effect size between students' scores in the experimental and control groups was calculated using (d and r) using means and standard deviations. Cohen's d = 0.826, and Effect-size r = 0.401, according to the findings. The ES of 0.824 means that the treated group's mean is in the 79th percentile of the control group (Large effect). Effect sizes can also be described as the average percentile rank of the average treated learner compared to the average untreated learner in general. The mean of the treated group is at the 50th percentile of the untreated group, indicating an ES of 0.0. The mean of the treated group is at the 79th percentile of the untreated group, with an ES of 0.8. The results showed that the dependent variable was strongly influenced in the four behavioral engagement factors: skills: performance, participation/interaction, and emotional, based on the fact that effect size is a significant factor in determining the research's strength.

Discussions and limitations

This section discusses the impact of an adaptive e-learning environment on student engagement development. This paper aimed to design an adaptive e-learning environment based on learning style parameters. The findings revealed that factors correlated to student engagement in e-learning: skills, participation/interaction, performance, and emotional. The engagement factors are significant because they affect learning outcomes (Nkomo et al., 2021 ). Every factor's items correlate to cognitive process-related activities. The participation/interaction factor, for example, referred to, interactions with the content, peers, and instructors. As a result, student engagement in e-learning can be predicted by interactions with content, peers, and instructors. The results are in line with previous research, which found that customized learning materials are important for increasing students' engagement. Adaptive e-learning based on learning styles sets a strong emphasis on behavioral engagement, in which students manage their learning while actively participating in online classes to adapt instruction according to each learning style. This leads to improved learning outcomes (Al-Chalabi & Hussein, 2020 ; Chun-Hui et al., 2017 ; Hussein & Al-Chalabi, 2020 ; Pashler et al., 2008 ). The experimental findings of this research showed that students who learned through adaptive eLearning based on learning styles learned more; as learning styles are reflected in this research as one of the generally assumed concerns as a reference for adapting e-content path. Students in the experimental group reported that the adaptive eLearning environment was very interesting and able to attract their attention. Those students also indicated that the adaptive eLearning environment was particularly useful because it provided opportunities for them to recall the learning content, thus enhancing their overall learning impression. This may explain why students in the experimental group performed well in class and showed more enthusiasm than students in the control group. This research compared an adaptive e-learning environment to a conventional e-learning approach toward engagement in a learning skills course through instructional content delivery and assessment. It can also be noticed that the experimental group had higher participation than the control group, indicating that BB activities were better adapted to the students' learning styles. Previous studies have agreed on the effectiveness of adaptive learning; it provides students with quality opportunity that is adapted to their learning styles, and preferences (Alshammari, 2016 ; Hussein & Al-Chalabi, 2020 ; Roy & Roy, 2011 ; Surjono, 2014 ). However, it should be noted that this study is restricted to one aspect of content adaptation and its factors, which is learning materials adapting based on learning styles. Other considerations include content-dependent adaptation. These findings are consistent with other studies, such as (Alshammari & Qtaish, 2019 ; Chun-Hui et al., 2017 ), which have revealed the effectiveness of the adaptive e-learning environment. This research differs from others in that it reflects on the Umm Al-Qura University as a case study, VARK Learning styles selection, engagement factors, and the closed learning management framework (BB).

The findings of the study revealed that adaptive content has a positive impact on adaptive individuals' achievement and student engagement, based on their learning styles (kinesthetic; auditory; visual; read/write). Several factors have contributed to this: The design of adaptive e-content for learning skills depended on introducing an ideal learning environment for learners, and providing support for learning adaptation according to the learning style, encouraging them to learn directly, achieving knowledge building, and be enjoyable in the learning process. Ali et al. ( 2019 ) confirmed that, indicating that education is adapted according to each individual's learning style, needs, and characteristics. Adaptive e-content design that allows different learners to think about knowledge by presenting information and skills in a logical sequence based on the adaptive e-learning framework, taking into account its capabilities as well as the diversity of its sources across the web, and these are consistent with the findings of (Alshammari & Qtaish, 2019 ).

Accordingly, the previous results are due to the following: good design of the adaptive e-learning environment in light of the learning style and educational preferences according to its instructional design (ID) standards, and the provision of adaptive content that suits the learners' needs, characteristics, and learning style, in addition to the diversity of course content elements (texts, static images, animations, and video), variety of tests and activities, diversity of methods of reinforcement, return and support from the instructor and peers according to the learning style, as well as it allows ease of use, contains multiple and varied learning sources, and allows referring to the same point when leaving the environment.

Several studies have shown that using adaptive eLearning technologies allows students to improve their learning knowledge and further enhance their engagement in issues such as "skills, performance, interaction, and emotional" (Ali et al., 2019 ; Graf & Kinshuk, 2007 ; Murray & Pérez, 2015 ); nevertheless, Murray and Pérez ( 2015 ) revealed that adaptive learning environments have a limited impact on learning outcome.

The restricted empirical findings on the efficacy of adapting teaching to learning style are mixed. (Chun-Hui et al., 2017 ) demonstrated that adaptive eLearning technologies can be beneficial to students' learning and development. According to these findings, adaptive eLearning can be considered a valuable method for learning because it can attract students' attention and promote their participation in educational activities. (Ali et al., 2019 ); however, only a few recent studies have focused on how adaptive eLearning based on learning styles fits in diverse cultural programs. (Benhamdi et al., 2017 ; Pashler et al., 2008 ).

The experimental results revealed that the proposed environment significantly increased students' learning achievements as compared to the conventional e-learning classroom (without adaptive technology). This means that the proposed environment's adaptation could increase students' engagement in the learning process. There is also evidence that an adaptive environment positively impacts other aspects of quality such as student engagement (Murray & Pérez, 2015 ).

Conclusions and implications

Although this field of research has stimulated many interests in recent years, there are still some unanswered questions. Some research gaps are established and filled in this study by developing an active adaptive e-learning environment that has been shown to increase student engagement. This study aimed to design an adaptive e-learning environment for performing interactive learning activities in a learning skills course. The main findings of this study revealed a significant difference in learning outcomes as well as positive results for adaptive e-learning students, indicating that it may be a helpful learning method for higher education. It also contributed to the current adaptive e-learning literature. The findings revealed that adaptive e-learning based on learning styles could help students stay engaged. Consequently, adaptive e-learning based on learning styles increased student engagement significantly. According to research, each student's learning style is unique, and they prefer to use different types of instructional materials and activities. Furthermore, students' preferences have an impact on the effectiveness of learning. As a result, the most effective learning environment should adjust its output to the needs of the students. The development of high-quality instructional materials and activities that are adapted to students' learning styles will help them participate and be more motivated. In conclusion, learning styles are a good starting point for creating instructional materials based on learning theories.

This study's results have important educational implications for future studies on the effect of adaptive e-learning on student interaction. First, the findings may provide data to support the development and improvement of adaptive environments used in blended learning. Second, the results emphasize the need for more quasi-experimental and descriptive research to better understand the benefits and challenges of incorporating adaptive e-learning in higher education institutions. Third, the results of this study indicate that using an adaptive model in an adaptive e-learning environment will encourage, motivate, engage, and activate students' active learning, as well as facilitate their knowledge construction, rather than simply taking in information passively. Fourth, new research is needed to design effective environments in which adaptive learning can be used in higher education institutions to increase academic performance and motivation in the learning process. Finally, the study shows that adaptive e-learning allows students to learn individually, which improves their learning and knowledge of course content, such as increasing their knowledge of learning skills course topics beyond what they can learn in a conventional e-learning classroom.

Contribution to research

The study is intended to provide empirical evidence of adaptive e-learning on student engagement factors. This research, on the other hand, has practical implications for higher education stakeholders, as it is intended to provide university faculty members with learning approaches that will improve student engagement. It is also expected to offer faculty a framework for designing personalized learning environments based on learning styles in various learning situations and designing more adaptive e-learning environments.

Research implication

Students with their preferred learning styles are more likely to enjoy learning if they are provided with a variety of instructional materials such as references, interactive media, videos, podcasts, storytelling, simulation, animation, problem-solving, games, and accessible educational tools in an e-learning environment. Also, different learning strategies can be accommodated. Other researchers would be able to conduct future studies on the use of the "adaptive e-learning" approach throughout the instructional process, at different phases of learning, and in various e-courses as a result of the current study. Meanwhile, the proposed environment's positive impact on student engagement gained considerable interest for future educational applications. Further research on learning styles in different university colleges could contribute to a foundation for designing adaptive e-courses based on students' learning styles and directing more future research on learning styles.

Implications for practice or policy:

Adaptive e-learning focused on learning styles would help students become more engaged.

Proving the efficacy of an adaptive e-learning environment via comparison with conventional e-learning .

Availability of data and materials

The author confirms that the data supporting the findings of this study are based on the research tools which were prepared and explained by the author and available on the links stated in the research instruments sub-section. The data analysis that supports the findings of this study is available on request from the corresponding author.

Akbulut, Y., & Cardak, C. (2012). Adaptive educational hypermedia accommodating learning styles: A content analysis of publications from 2000 to 2011. Computers & Education . https://doi.org/10.1016/j.compedu.2011.10.008 .

Article   Google Scholar  

Al-Chalabi, H., & Hussein, A. (2020). Analysis & implementation of personalization parameters in the development of computer-based adaptive learning environment. SAR Journal Science and Research., 3 (1), 3–9. https://doi.org/10.18421//SAR31-01 .

Aldosari, M., Aljabaa, A., Al-Sehaibany, F., & Albarakati, S. (2018). Learning style preferences of dental students at a single institution in Riyadh Saudi Arabia, evaluated using the VARK questionnaire . Advances in Medical Education and Practice. https://doi.org/10.2147/AMEP.S157686 .

Ali, N., Eassa, F., & Hamed, E. (2019). Personalized Learning Style for Adaptive E-Learning System, International Journal of Advanced Trends in Computer Science and Engineering . 223-230. Retrieved June 26, 2020 from http://www.warse.org/IJATCSE/static/pdf/file/ijatcse4181.12019.pdf .

Alshammari, M., & Qtaish, A. (2019). Effective adaptive e-learning systems according to learning style and knowledge level. JITE Research, 18 , 529–547. https://doi.org/10.28945/4459 .

Alshammari, M. (2016). Adaptation based on learning style and knowledge level in e-learning systems, Ph.D. thesis , University of Birmingham.  Retrieved April 18, 2019 from http://etheses.bham.ac.uk//id/eprint/6702/ .

Alshammari, M., Anane, R., & Hendley, R. (2015). Design and Usability Evaluation of Adaptive E-learning Systems based on Learner Knowledge and Learning Style. Human-Computer Interaction Conference- INTERACT , Vol. (9297), (pp. 157–186). https://doi.org/10.1007/978-3-319-22668-2_45 .

Alzain, A., Clack, S., Jwaid, A., & Ireson, G. (2018a). Adaptive education based on learning styles: Are learning style instruments precise enough. International Journal of Emerging Technologies in Learning (iJET), 13 (9), 41–52. https://doi.org/10.3991/ijet.v13i09.8554 .

Alzain, A., Clark, S., Ireson, G., & Jwaid, A. (2018b). Learning personalization based on learning style instruments. Advances in Science Technology and Engineering Systems Journal . https://doi.org/10.25046/aj030315 .

Atherton, M., Shah, M., Vazquez, J., Griffiths, Z., Jackson, B., & Burgess, C. (2017). Using learning analytics to assess student engagement and academic outcomes in open access enabling programs”. Journal of Open, Distance and e-Learning, 32 (2), 119–136.

Barkley, E., & Major, C. (2020). Student engagement techniques: A handbook for college faculty . Jossey-Bass . 10:047028191X.

Google Scholar  

Becker, K., Kehoe, J., & Tennent, B. (2007). Impact of personalized learning styles on online delivery and assessment. Campus-Wide Information Systems . https://doi.org/10.1108/10650740710742718 .

Behaz, A., & Djoudi, M. (2012). Adaptation of learning resources based on the MBTI theory of psychological types. IJCSI International Journal of Computer Science, 9 (2), 135–141.

Beldagli, B., & Adiguzel, T. (2010). Illustrating an ideal adaptive e-learning: A conceptual framework. Procedia - Social and Behavioral Sciences, 2 , 5755–5761. https://doi.org/10.1016/j.sbspro.2010.03.939 .

Benhamdi, S., Babouri, A., & Chiky, R. (2017). Personalized recommender system for e-Learning environment. Education and Information Technologies, 22 , 1455–1477. https://doi.org/10.1007/s10639-016-9504-y .

Chen, P., Lambert, A., & Guidry, K. (2010). Engaging online learners: The impact of Web-based learning technology on college student engagement. Computers & Education, 54 , 1222–1232.

Chun-Hui, Wu., Chen, Y.-S., & Chen, T. C. (2017). An adaptive e-learning system for enhancing learning performance: based on dynamic scaffolding theory. Eurasia Journal of Mathematics, Science and Technology Education. https://doi.org/10.12973/ejmste/81061 .

Cletus, D., & Eneluwe, D. (2020). The impact of learning style on student performance: mediate by personality. International Journal of Education, Learning and Training. https://doi.org/10.24924/ijelt/2019.11/v4.iss2/22.47Desmond .

Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science., 1 (3), 98–101. https://doi.org/10.1111/1467-8721.ep10768783 .

Daines, J., Troka, T. and Santiago, J. (2016). Improving performance in trigonometry and pre-calculus by incorporating adaptive learning technology into blended models on campus. https://doi.org/10.18260/p.25624 .

DeCapua, A. & Marshall, H. (2015). Implementing a Mutually Adaptive Learning Paradigm in a Community-Based Adult ESL Literacy Class. In M. Santos & A. Whiteside (Eds.). Low Educated Second Language and Literacy Acquisition. Proceedings of the Ninth Symposium (pps. 151-171). Retrieved Nov. 14, 2020 from https://www.researchgate.net/publication/301355138_Implementing_a_Mutually_Adaptive_Learning_Paradigm_in_a_Community-Based_Adult_ESL_Literacy_Class .

Dixson, M. (2015). Measuring student engagement in the online course: The online student engagement scale (OSE). Online Learning . https://doi.org/10.24059/olj.v19i4.561 .

Dominic, M., Xavier, B., & Francis, S. (2015). A Framework to Formulate Adaptivity for Adaptive e-Learning System Using User Response Theory. International Journal of Modern Education and Computer Science, 7 , 23. https://doi.org/10.5815/ijmecs.2015.01.04 .

El Bachari, E., Abdelwahed, E., & M., El. . (2011). E-Learning personalization based on Dynamic learners’ preference. International Journal of Computer Science and Information Technology., 3 , 200–216. https://doi.org/10.5121/ijcsit.2011.3314 .

El-Sabagh, H. A., & Hamed, E. (2020). The Relationship between Learning-Styles and Learning Motivation of Students at Umm Al-Qura University. Egyptian Association for Educational Computer Journal . https://doi.org/10.21608/EAEC.2020.25868.1015 ISSN-Online: 2682-2601.

Ennouamani, S., & Mahani, Z. (2017). An overview of adaptive e-learning systems. Eighth International ConfeRence on Intelligent Computing and Information Systems (ICICIS) . https://doi.org/10.1109/INTELCIS.2017.8260060 .

Evans, S., Steele, J., Robertson, S., & Dyer, D. (2017). Personalizing post titles in the online classroom: A best practice? Journal of Educators Online, 14 (2), 46–54.

Fleming, N., & Baume, D. (2006). Learning styles again: VARKing up the Right Tree! Educational Developments, 7 , 4–7.

Franzoni, A., & Assar, S. (2009). Student learning style adaptation method based on teaching strategies and electronic media. Journal of Educational Technology & Society , 12(4), 15–29. Retrieved March 21, 2020, from http://www.jstor.org/stable/jeductechsoci.12.4.15 .

Fredricks, J., Blumenfeld, P., & Paris, A. (2004). School Engagement: Potential of the Concept . State of the Evidence: Review of Educational Research. https://doi.org/10.3102/00346543074001059 .

Book   Google Scholar  

Gaytan, J., & McEwen, M. (2007). Effective Online Instructional and Assessment Strategies. American Journal of Distance Education, 21 (3), 117–132. https://doi.org/10.1080/08923640701341653 .

Graf, S. & Kinshuk. K. (2007). Providing Adaptive Courses in Learning Management Systems with respect to Learning Styles. Proceeding of the World Conference on eLearning in Corporate. Government. Healthcare. and Higher Education (2576–2583). Association for the Advancement of Computing in Education (AACE). Retrieved January 18, 2020 from  https://www.learntechlib.org/primary/p/26739/ . ISBN 978-1-880094-63-1.

Guo, P., Kim, V., & Rubin, R. (2014). How video production affects student engagement: an empirical study of MOOC videos. Proceedings of First ACM Conference on Learning @ Scale Confernce . March 2014, (pp. 41-50). https://doi.org/10.1145/2556325.2566239 .

Hinton, P. R., Brownlow, C., McMurray, I., & Cozens, B. (2014). SPSS Explained (2nd ed., pp. 339–354). Routledge Taylor & Francis Group.

Hong, S. (2009). Developing competency model of learners in distance universities. Journal of Educational Technology., 25 , 157–186.

Hussain, I. (2017). Pedagogical implications of VARK model of learning. Journal of Literature, Languages and Linguistics, 38 , 33–37.

Hussain, M., Zhu, W., Zhang, W., & Abidi, S. (2018). Student engagement predictions in an e-learning system and their impact on student course assessment scores. Computational Intelligence, and Neuroscience. https://doi.org/10.1155/2018/6347186 .

Hussein, A., & Al-Chalabi, H. (2020). Pedagogical Agents in an Adaptive E-learning System. SAR Journal of Science and Research., 3 , 24–30. https://doi.org/10.18421/SAR31-04 .

Jaleel, S., & Thomas, A. (2019). Learning styles theories and implications for teaching learning . Horizon Research Publishing. 978-1-943484-25-6.

Johnson, M. (2009). Evaluation of Learning Style for First-Year Medical Students. Int J Schol Teach Learn . https://doi.org/10.20429/ijsotl.2009.030120 .

Jonassen, D. H., & Grabowski, B. L. (2012). Handbook of individual differences, learning, and instruction. Routledge . https://doi.org/10.1016/0022-4405(95)00013-C .

Klasnja-Milicevic, A., Vesin, B., Ivanovic, M., & Budimac, Z. (2011). E-Learning personalization based on hybrid recommendation strategy and learning style identification. Computers & Education, 56 (3), 885–899. https://doi.org/10.1016/j.compedu.2010.11.001 .

Kolekar, S. V., Pai, R. M., & Manohara Pai, M. M. (2017). Prediction of learner’s profile based on learning styles in adaptive e-learning system. International Journal of Emerging Technologies in Learning, 12 (6), 31–51. https://doi.org/10.3991/ijet.v12i06.6579 .

Lee, J., & Kim, D. (2012). Adaptive learning system applied bruner’ EIS theory. International Conference on Future Computer Supported Education, IERI Procedia, 2 , 794–801. https://doi.org/10.1016/j.ieri.2012.06.173 .

Lee, J., Song, H.-D., & Hong, A. (2019). Exploring factors, and indicators for measuring students’ sustainable engagement in e-learning. Sustainability, 11 , 985. https://doi.org/10.3390/su11040985 .

Leung, A., McGregor, M., Sabiston, D., & Vriliotis, S. (2014). VARK learning styles and student performance in principles of Micro-vs. Macro-Economics. Journal of Economics and Economic Education Research, 15 (3), 113.

Lévy, P. & Wakabayashi, N. (2008). User's appreciation of engagement in service design: The case of food service design. Proceedings of International Service Innovation Design Conference 2008 - ISIDC08 . Busan, Korea. Retrieved October 28, 2019 from https://www.researchgate.net/publication/230584075 .

Liang, J. S. (2012). The effects of learning styles and perceptions on application of interactive learning guides for web-based. Proceedings of Australasian Association for Engineering Education Conference AAEE . Melbourne, Australia. Retrieved October 22, 2019 from https://aaee.net.au/wpcontent/uploads/2018/10/AAEE2012-Liang.-Learning_styles_and_perceptions_effects_on_interactive_learning_guide_application.pdf .

Mahnane, L., Laskri, M. T., & Trigano, P. (2013). A model of adaptive e-learning hypermedia system based on thinking and learning styles. International Journal of Multimedia and Ubiquitous Engineering, 8 (3), 339–350.

Markey, M. K. & Schmit, K, J. (2008). Relationship between learning style Preference and instructional technology usage. Proceedings of American Society for Engineering Education Annual Conference & Expodition . Pittsburgh, Pennsylvania. Retrieved March 15, 2020 from https://peer.asee.org/3173 .

McMillan, J., & Schumacher, S. (2006). Research in education: Evidence-based inquiry . Pearson.

Murphy, R., Gray, S., Straja, S., & Bogert, M. (2004). Student learning preferences and teaching implications: Educational methodologies. Journal of Dental Education, 68 (8), 859–866.

Murray, M., & Pérez, J. (2015). Informing and performing: A study comparing adaptive learning to traditional learning. Informing Science. The International Journal of an Emerging Transdiscipline , 18, 111–125. Retrieved Febrauary 4, 2021 from http://www.inform.nu/Articles/Vol18/ISJv18p111-125Murray1572.pdf .

Mutahi, J., Kinai, A. , Bore, N. , Diriye, A. and Weldemariam, K. (2017). Studying engagement and performance with learning technology in an African classroom, Proceedings of Seventh International Learning Analytics & Knowledge Conference , (pp. 148–152), Canada: Vancouver.

Nainie, Z., Siraj, S., Abuzaiad, R. A., & Shagholi, R. (2010). Hypothesized learners’ technology preferences based on learning styles dimensions. The Turkish Online Journal of Educational Technology, 9 (4), 83–93.

Naqeeb, H. (2011). Learning Styles as Perceived by Learners of English as a Foreign Language in the English Language Center of The Arab American University—Jenin. Palestine. an Najah Journal of Research, 25 , 2232.

Nkomo, L. M., Daniel, B. K., & Butson, R. J. (2021). Synthesis of student engagement with digital technologies: a systematic review of the literature. International Journal of Educational Technology in Higher Education . https://doi.org/10.1186/s41239-021-00270-1 .

Normadhi, N. B., Shuib, L., Nasir, H. N. M., Bimba, A., Idris, N., & Balakrishnan, V. (2019). Identification of personal traits in adaptive learning environment: Systematic literature review. Computers & Education, 130 , 168–190. https://doi.org/10.1016/j.compedu.2018.11.005 .

Nuankaew, P., Nuankaew, W., Phanniphong, K., Imwut, S., & Bussaman, S. (2019). Students model in different learning styles of academic achievement at the University of Phayao, Thailand. International Journal of Emerging Technologies in Learning (iJET)., 14 , 133. https://doi.org/10.3991/ijet.v14i12.10352 .

Oxman, S. & Wong, W. (2014). White Paper: Adaptive Learning Systems. DV X Innovations DeVry Education Group. Retrieved December 14, 2020 from shorturl.at/hnsS8 .

Ozyurt, Ö., & Ozyurt, H. (2015). Learning style-based individualized adaptive e-learning environments: Content analysis of the articles published from 2005 to 2014. Computers in Human Behavior, 52 , 349–358. https://doi.org/10.1016/j.chb.2015.06.020 .

Pardo, A., Han, F., & Ellis, R. (2016). Exploring the relation between self-regulation, online activities, and academic performance: a case study. Proceedings of Sixth International Conference on Learning Analytics & Knowledge , (pp. 422-429). https://doi.org/10.1145/2883851.2883883 .

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: concepts and evidence. Psychology Faculty Publications., 9 (3), 105–119. https://doi.org/10.1111/j.1539-6053.2009.01038.x .

Qazdar, A., Cherkaoui, C., Er-Raha, B., & Mammass, D. (2015). AeLF: Mixing adaptive learning system with learning management system. International Journal of Computer Applications., 119 , 1–8. https://doi.org/10.5120/21140-4171 .

Robinson, C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84 , 101–109.

Rogers-Stacy, C., Weister, T., & Lauer, S. (2017). Nonverbal immediacy behaviors and online student engagement: Bringing past instructional research into the present virtual classroom. Communication Education, 66 (1), 37–53.

Roy, S., & Roy, D. (2011). Adaptive e-learning system: a review. International Journal of Computer Trends and Technology (IJCTT), 1 (1), 78–81. ISSN:2231-2803.

Shi, L., Cristea, A., Foss, J., Qudah, D., & Qaffas, A. (2013). A social personalized adaptive e-learning environment: a case study in topolor. IADIS International Journal on WWW/Internet., 11 , 13–34.

Shih, M., Feng, J., & Tsai, C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51 (2), 955–967. https://doi.org/10.1016/j.compedu.2007.10.004 .

Silva, A. (2020). Towards a Fuzzy Questionnaire of Felder and Solomon for determining learning styles without dichotomic in the answers. Journal of Learning Styles, 13 (15), 146–166.

Staikopoulos, A., Keeffe, I., Yousuf, B. et al., (2015). Enhancing student engagement through personalized motivations. Proceedings of IEEE 15th International Conference on Advanced Learning Technologies , (pp. 340–344), Taiwan: Hualien. https://doi.org/10.1109/ICALT.2015.116 .

Surjono, H. D. (2014). The evaluation of Moodle-based adaptive e-learning system. International Journal of Information and Education Technology, 4 (1), 89–92. https://doi.org/10.7763/IJIET.2014.V4.375 .

Truong, H. (2016). Integrating learning styles and adaptive e-learning system: current developments, problems, and opportunities. Computers in Human Behavior, 55 (2016), 1185–1193. https://doi.org/10.1016/j.chb.2015.02.014 .

Umm Al-Qura University Agency for Educational Affairs (2020). Common first-year Deanship, at Umm Al-Qura University. Retrieved February 3, 2020 from https://uqu.edu.sa/en/pre-edu/70021 .

Vassileva, D. (2012). Adaptive e-learning content design and delivery based on learning style and knowledge level. Serdica Journal of Computing, 6 , 207–252.

Veiga, F., Robu, V., Appleton, J., Festas, I & Galvao, D. (2014). Students' engagement in school: Analysis according to self-concept and grade level. Proceedings of EDULEARN14 Conference 7th-9th July 2014 (pp. 7476-7484). Barcelona, Spain. Available Online at: http://hdl.handle.net/10451/12044 .

Velázquez, A., & Assar, S. (2009). Student learning styles adaptation method based on teaching strategies and electronic media. Educational Technology & SocieTy., 12 , 15–29.

Verdú, E., Regueras, L., & De Castro, J. (2008). An analysis of the research on adaptive Learning: The next generation of e-learning. WSEAS Transactions on Information Science and Applications, 6 (5), 859–868.

Willingham, D., Hughes, E., & Dobolyi, D. (2015). The scientific status of learning styles theories. Teaching of Psychology., 42 (3), 266–271. https://doi.org/10.1177/0098628315589505 .

Yalcinalp & Avcı. (2019). Creativity and emerging digital educational technologies: A systematic review. The Turkish Online Journal of Educational Technology, 18 (3), 25–45.

Yang, J., Huang, R., & Li, Y. (2013). Optimizing classroom environment to support technology enhanced learning. In A. Holzinger & G. Pasi (Eds.), Human-computer interaction and knowledge discovery in complex (pp. 275–284). Berlin: Springer.

Zhang, H. (2017). Accommodating different learning styles in the teaching of economics: with emphasis on fleming and mills¡¯s sensory-based learning style typology. Applied Economics and Finance, 4 (1), 72–78.

Download references

Acknowledgements

The author would like to thank the Deanship of Scientific Research at Umm Al-Qura University for the continuous support. This work was supported financially by the Deanship of Scientific Research at Umm Al-Qura University to Dr.: Hassan Abd El-Aziz El-Sabagh. (Grant Code: 18-EDU-1-01-0001).

Author information

Hassan A. El-Sabagh is an assistant professor in the E-Learning Deanship and head of the Instructional Programs Department, Umm Al-Qura University, Saudi Arabia, where he has worked since 2012. He has extensive experience in the field of e-learning and educational technologies, having served primarily at the Educational Technology Department of the Faculty of Specific Education, Mansoura University, Egypt since 1997. In 2011, he earned a Ph.D. in Educational Technology from Dresden University of Technology, Germany. He has over 14 papers published in international journals/conference proceedings, as well as serving as a peer reviewer in several international journals. His current research interests include eLearning Environments Design, Online Learning; LMS-based Interactive Tools, Augmented Reality, Design Personalized & Adaptive Learning Environments, and Digital Education, Quality & Online Courses Design, and Security issues of eLearning Environments. (E-mail: [email protected]; [email protected]).

Authors and Affiliations

E-Learning Deanship, Umm Al-Qura University, Mecca, Saudi Arabia

Hassan A. El-Sabagh

Faculty of Specific Education, Mansoura University, Mansoura, Egypt

You can also search for this author in PubMed   Google Scholar

Contributions

The author read and approved the final manuscript.

Corresponding author

Correspondence to Hassan A. El-Sabagh .

Ethics declarations

Competing interests.

The author declares that there is no conflict of interest

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

El-Sabagh, H.A. Adaptive e-learning environment based on learning styles and its impact on development students' engagement. Int J Educ Technol High Educ 18 , 53 (2021). https://doi.org/10.1186/s41239-021-00289-4

Download citation

Received : 24 May 2021

Accepted : 19 July 2021

Published : 01 October 2021

DOI : https://doi.org/10.1186/s41239-021-00289-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Adaptive e-Learning
• Learning style
• Learning impact

• Open access
• Published: 25 January 2022

A systematic review on trends in using Moodle for teaching and learning

• Sithara H. P. W. Gamage   ORCID: orcid.org/0000-0001-9209-9113 1 ,
• Jennifer R. Ayres   ORCID: orcid.org/0000-0002-4538-6512 1 &
• Monica B. Behrend   ORCID: orcid.org/0000-0002-7324-487X 2  

International Journal of STEM Education volume  9 , Article number:  9 ( 2022 ) Cite this article

33k Accesses

82 Citations

6 Altmetric

Metrics details

The Moodle Learning Management System (LMS) is widely used in online teaching and learning, especially in STEM education. However, educational research on using Moodle is scattered throughout the literature. Therefore, this review aims to summarise this research to assist three sets of stakeholders—educators, researchers, and software developers. It identifies: (a) how and where Moodle has been adopted; (b) what the concerns, trends, and gaps are to lead future research and software development; and (c) innovative and effective methods for improving online teaching and learning.

The review used the 4-step PRISMA-P process to identify 155 suitable journal articles from 104 journals in 55 countries published from January 2015 to June 2021. The database search was conducted with Scopus and Web of Science. Insights into the educational use of Moodle were determined through bibliometric analysis with Vosviewer outputs and thematic analysis.

This review shows that Moodle is mainly used within University STEM disciplines and effectively improves student performance, satisfaction, and engagement. Moodle is increasingly being used as a platform for adaptive and collaborative learning and used to improve online assessments. The use of Moodle is developing rapidly to address academic integrity, ethics, and security issues to enhance speed and navigation, and incorporate artificial intelligence.

More qualitative research is required on the use of Moodle, particularly investigating educators’ perspectives. Further research is also needed on the use of Moodle in non-STEM and non-tertiary disciplines. Further studies need to incorporate educational theories when designing courses using the Moodle platform.

Introduction

Various learning management systems (LMSs) are available to develop, manage, and distribute digital resources for face-to-face and online teaching. An LMS provides interaction between traditional teaching techniques and digital learning resources, and simultaneously offers students personalised e-learning opportunities (Aljawarneh, 2020 ). E-learning is an area that has seen considerable growth, particularly since 2020 with the onset of the COVID-19 pandemic, which has limited face-to-face teaching possibilities for many educational institutions globally (Dias et al., 2020 ; Raza et al., 2021 ). Educational institutions have had to adapt to restrictions imposed on physical interaction, which have precluded most conventional forms of education, assessment, research, and scientific discourse (Byrnes et al., 2021 ).

The role of LMSs has gained prominence within the context of STEM (Science, Technology, Engineering, and Mathematics) programs and courses over the last decade through improved access to broadband internet and advancements in online teaching and learning technologies. Many educational institutions have effectively used LMSs and continue to research the effectiveness of using various types of LMSs. Recent studies focussing on STEM education suggest that various LMSs and associated tools increase student engagement, motivation, collaboration (Araya & Collanqui, 2021 ; Campbell et al., 2020 ; Hwang, 2020 ; Jones et al., 2021 ), performance, retention, and critical thinking (Alkholy et al., 2015 ; Ardianti et al., 2020 ; Bernacki et al., 2020 ; Cadaret & Yates, 2018 ; Hempel et al., 2020 ; Oguguo et al., 2021 ). In addition, LMSs allow STEM educators to track learning outcomes, predict achievements (for early detection of students at risk), and then use the identified information to adapt and modify teaching practices (Dominguez et al., 2016 ; Hempel et al., 2020 ; Price et al., 2021 ; Sergis et al., 2017 ; Zakaria et al., 2019 ; Zheng et al., 2019 ). The future of STEM education can continue to be improved with innovative LMSs and technology-enhanced learning materials (Zhao et al., 2018 ), such as online laboratories (Henke et al., 2021 ), online tutorials (Rissanen & Costello, 2021 ) and virtual reality applications (Christopoulos et al., 2020 ). A recent systematic review on research trends in STEM education (Li et al., 2020 ) indicates that ‘learning environments’ which include an LMS is one key area that will continue to evolve.

Currently, 561 LMSs are available worldwide for academic/educational purposes, according to Capterra ( 2021 ) an international software review and selection platform. The learning platforms that were most widely used and researched during 2015–2020 include Edmodo, Moodle, MOOC, and Google Classroom (Setiadi et al., 2021 ). Research on comparisons of various LMSs is rare but some comparisons between LMSs such as Moodle, Sakai, SumTotal, Blackboard, Canvas, and ATutor are available in the literature (Shkoukani, 2019 ; Xin et al., 2021 ). According to a recent systematic review on tendencies in the use of LMSs (Altinpulluk & Kesim, 2021 ), Moodle is the most popular and preferred open-source LMS. Moodle has a high rate of acceptance in the community and in many institutions and has a wide variety of active courses, available in many languages (Al-Ajlan & Zedan, 2008 ; Sergis et al., 2017 ). A recent study that determined the effect of LMSs on students’ performance in educational measurement and evaluation recommends that LMSs such as Moodle should be learnt and used by lecturers (Oguguo et al., 2021 ).

Currently, the world's leading open-source LMS, Moodle (Moodle Project, 2020a ), is used by various disciplines within academia, including STEM education. A keyword search of “Moodle” in publications, categorised by discipline area from 2015 to 2021, indicated that more than 60% of publications containing the keyword “Moodle” are in the STEM area. Moodle is a cloud-based LMS and among the top 20 best LMSs based on user experiences in 2018 (Henrick, 2018 ). The number of Moodle users continues to increase from 78 million in 2015 (Singh, 2015 ) to over 294 million in 2021(Moodle Project, 2021a)—an increase of over 250%. Although Moodle is becoming increasingly popular, to date, no review has provided information on the use of Moodle across a vast number of disciplines in different educational institutions at different levels of education. This review aims to comprehensively analyse the literature on the adaptation of Moodle as an educational tool over the past 6 years to provide information for three sets of stakeholders—educators, researchers, and software developers. The review addresses two main research questions:

Where is Moodle used, adapted, and researched?

How is Moodle used in teaching and learning?

This systematic review focuses on recent research (January 2015–June 2021) in using Moodle within academic institutions. The review took a multidisciplinary approach to encompass all subjects and levels within academia. To align with the first research question, Where is Moodle used, adapted, and researched?, a bibliometric analysis was performed to identify the dissemination of the literature and summarise the bibliometrics of the publications. Then, a thematic analysis was performed to address the second research question, How is Moodle used in teaching and learning? .

Bibliometric analysis

Prisma-p process.

This study adopted a strict systematic review protocol that followed the 4-step PRISMA-P process (Moher et al., 2015 ). This process has the following steps: (1) Identification of the relevant literature pertaining to this study, (2) Screening using the criteria determined by the authors, (3) Classification of the screened articles in a methodical manner using codes and themes predetermined by the authors, and (4) Determining the articles for inclusion in this review.

Identification

Scopus and Web of Science (WOS) were used to perform the literature search due to their comprehensive journal coverage, ease of keyword searching, accessibility within academia, and popularity within multiple disciplines (Colares et al., 2020 ; Souza et al., 2019 ). The term “Moodle” found articles with a wide range of Moodle topics when used in the search databases, while an initial search of Moodle review articles suggested several keywords, such as “Moodle quiz” and “e-learning”. The Scopus search was limited to the selected years with the option of only “Article” or “Review” chosen along with using the title, abstract, and keywords to identify “Moodle” articles. The WOS search was run with “Moodle” and selected all topics in the search parameters. Both database searches were last run on 30 June 2021.

In this phase, literature identified from both database searches was screened to exclude articles that were: (1) published before 2015, (2) written in any language other than English, (3) published but had not been through the peer review process (e.g., conference papers, book chapters, letters), and (4) was not relevant to this review. An individual article's relevance was determined by examining the title, abstract, results, and methods. Any articles that did not fulfil these screening criteria were excluded from this study.

Classification

The articles identified and screened were multidisciplinary; therefore, these articles were then classified. Initially, the classification process allocated codes to the journal articles related to the article's research discipline (see Table 1 for codes)—for example, STEM disciplines encompass subject matter of science, technology, engineering, and maths. If more than one discipline was covered in the article, the multidisciplinary (MD) code was used. The articles were then classified into specific subject matter and education levels, including undergraduate, postgraduate, and multi-level. These codes were based on categories of the International Standards Classification for Education (ISCED, 2012 ). The not-determined (ND) code was used, if needed, for discipline and education level.

The articles selected for review were limited to Jan 2015–June 2021 and included the word “Moodle” in either the title, abstract, or keywords. The 4-step process applied for selecting the articles included in this review is shown in Fig.  1 .

Four-step process for this systematic review

Bibliographic analysis

The Vosviewer software, Version 1.6.15, was applied for bibliometric analysis using the Scopus and WOS database search results. Vosviewer is freely available software that automates term identification and constructs bibliometric maps based on network data (Colares et al., 2020 ; de Souza et al., 2019 ). The combined downloaded results from Scopus and WOS were used to create a CSV file. The CSV file was updated after the 4-step systematic review protocol process and articles irrelevant to this study were removed from the file. The CSV file was then loaded into Vosviewer to create a co-occurrence map of bibliographic data. The software enables the user to build co-occurrence maps in various areas, such as keywords, journal citation counts, and publication title (van Eck & Waltman, 2020 ). Bibliometric analysis was conducted on each article, including the year of publication, keywords, journal publication citation count, and the country of publication.

Thematic analysis (TA)

Following the classification of the included journal articles, further insights and trends within the articles were established by thematic analysis. This process was consistent with Braun and Clarke ( 2006 ) thematic analysis (TA) method which identifies and analyses patterns of meanings (themes) in qualitative data. This method can be applied within a range of theoretical frameworks and can be used to analyse almost all forms of qualitative data, both small and large data sets, to address different types of research questions (Clarke & Braun, 2014 ). The TA used in this review involves the generation of codes and themes. The codes capture features of each paper which have potential relevance to the research questions. The themes were constructed from the coding to capture broader patterns.

To generate the trends identified in the literature, the six-phase Braun and Clarke ( 2006 ) method was used as follows:

Familiarisation with the data : The selected articles were read to become familiar with the topics covered by each article, noting any common concepts covered by each study.

Coding : Codes were generated for important features relevant to teaching and learning covered by each article (Research Question 2). This coding is not simply a method of data reduction; it is an analytic process.

Searching for themes : A theme is a coherent and meaningful pattern in the reviewed articles which is relevant to the research questions. The themes were not necessarily in the articles but were constructed. This review constructed eight themes of interest relevant to teaching and learning (Research Question 2).

Reviewing themes : This step involved reflecting on the themes to tell a story, defining the nature of the theme, and identifying relationships between the themes and different sub-themes within the themes.

Defining and naming themes : This step involved specifying the ‘essence’ of each theme and constructing an informative name for each theme.

Writing up : Writing-up involves creating a coherent and persuasive story about the reviewed papers which includes analysis of current and future research.

The themes, sub-themes, and definitions of each theme are shown in Table 2 .

Results and discussion

The initial database searches identified 538 Moodle-related articles. The literature was then screened for the period Jan 2015–June 2021, journal or review articles only, and articles published in English. This screening reduced the identified articles to 285, 167 from Scopus, and 118 from WOS. These initially screened articles were downloaded from the relevant databases and checked for duplicates. After screening for duplicates, the abstracts from the 211 remaining articles were reviewed, resulting in the elimination of a further 24 articles. The full text of the remaining 187 articles was read, eliminating another 32 articles as they were not directly related to this study. Thus, a total of 155 journal articles were used in this systematic review.

Journals and citations

Moodle is prevalent in various disciplines, as revealed by 104 journals relevant to this study. Journal titles that published two or more articles are shown in Fig.  2 . The journal with the most published Moodle-related articles was International Journal of Emerging Technologies in Learning (10 articles), followed by Computer Application in Engineering Education (8 articles), and then Journal of e-Learning & Knowledge Society , and the Journal of Technology and Science Education (5 articles per journal).

Journal titles that have published more than 2 articles used in this study

Scopus was used for the citation count unless the article was only available in WOS; then, the WOS citation count per article was used. The 155 journal articles reviewed in this study have a combined citation count of 608 with the most cited (71 times) being a review article comparing 17 blended courses using Moodle LMS (Conijn et al., 2017 ). Total citation counts of the articles by published year were 95 in 2015, 92 in 2016, 270 in 2017, 83 in 2018, 50 in 2019, and 21 in 2020. Of the top 10 cited articles (listed in Table 3 ), five articles were published in 2017, accounting for 198 citations of the total 270 for that year, with the remaining 72 citations across 24 papers. Of the top 10 authors, four are attributed to the top-cited paper (Conijn et al., 2017 ). All the top 10 cited authors have articles in the top 10 cited list (Table 4 ).

Author-affiliated countries

Vosviewer has the facility to produce a density map of co-occurrences in countries (van Eck & Waltman, 2020 ). Figure  3 shows the density map of countries publishing more than two articles. Fifty-five countries contributed research to the 155 articles, with 37 countries publishing more than two papers. The higher the count of publications, the brighter the yellow, with Spain contributing 17 articles, the United States of America (USA) 14, Australia 12, the Russian Federation 10, Malaysia 8, Italy 7, and Portugal 5 articles. The software positions the countries with a similar number of articles published close to each other. Therefore, Vosviewer provides the reader with an instantaneous pictorial result of countries publishing Moodle articles.

Density map showing countries contributing to more than 2 publications

Popular keywords

The keywords from the 155 articles were analysed in Vosviewer. In total, 926 keywords were used, of which 154 were used three times or more. Table 5 shows the top 10 keywords. The most occurring keyword was Moodle (61), followed by e-learning (31), teaching (26), and education (25), and learning management system (25).

Along with the ability to extract the top keywords used within the articles, Vosviewer produced cluster graphics of keywords. Figure  4 a shows the cluster graphic of keywords of more than three uses or a higher density with a larger marker on the graphic; hence, the most significant markers are Moodle, e-Learning, and Education. The map also has the feature to zoom in and out, showing more keywords and highlighting the most occurring keywords. Figure  4 b shows the option in Vosviewer to see the links that connect the keywords within the articles (in this instance, Education was highlighted). The keywords associated with Education in the 155 articles (with more than 3 occurrences) can be seen with linked keywords, such as Moodle, Student, and e-Learning. All keywords can be highlighted individually for associations to be seen.

Vosviewer cluster graphic of keyword results: a  Keywords with more than 3 uses, b  The links highlighted when the word ‘education’ is highlighted

Discipline and education level of studies

Research into Moodle assessments is being published in many different subject areas, such as science, technology, engineering, and maths (STEM), health sciences (HS), and veterinary medicine (VM). Figure  5 shows the number of publications per full year (2015–2020) and the articles' discipline.

No of publications by discipline 2015–2020

L—languages, A—arts, VM, veterinary medicine, TD—teaching degree, STEM—science, technology, engineering and maths, ND—not determined, MD—multi-discipline, HS—health sciences, CS—computer science, BS—business studies.

The number of total publications was lowest in 2015 and 2016 with 18 and 19 publications, respectively. This number increased each year after that: 2017 (n = 24), 2018 (n = 26), 2019 (n = 32), and 2020 (n = 36). The two main disciplines throughout this publication period were STEM and HS. The STEM discipline contained various subjects, with most being engineering (civil) and science (i.e., physics and chemistry). HS subjects published include nursing, medical practice, and dentistry. Some articles that did not fit into a particular discipline (ND) covered various subjects, such as security issues identified within e-learning or articles that deal with databases (Chaparro-Peláez et al., 2019 ; Mudiyanselage & Pan, 2020 ).

Of the 155 articles, 116 articles evaluated Moodle within a university setting, with 112 at undergraduate (UG) level, nine postgraduate (PG), and seven articles examined at both UG and PG courses. School-age students (S) were the focus in six studies, teaching staff (T) in four articles, and S and T in two articles. A total of 31 articles did not determine (ND) the level of education for the study or were not focused on individuals but rather systems (Chafiq et al., 2018 ; Conejo et al., 2016 ).

The trends demonstrated in the research articles are categorised into eight main themes (see Table 2 ). Theme 1 compares various Moodle features explained in the study. Themes 2 to 4 highlight the trends in pedagogy, which include curriculum development, learning, and assessment processes in e-learning. Theme 5 analyses ethical aspects of e-learning, and Theme 6 highlights trends in new software development aiming to improve e-learning, particularly Moodle. Themes 7 and 8 provide an overview of research approaches, methods, and common student success indicators. Figure  6 shows the number of papers that discuss each of the eight themes, although several papers discuss multiple themes. Figure  7 shows the percentage of papers related to each theme and sub-theme.

Frequency of articles describing each theme

Percentage of research papers related to each theme and sub-theme

Theme 1: Moodle features

Of the reviewed articles discussing Moodle LMS, 23% discuss Moodle ‘Activities’. An activity, a general name for a group of Moodle features, is usually something that a student will engage in and that interacts with other students or the teacher. The activities identified included: Moodle quizzes, forums, workshops, lessons, wikis, and surveys. Of these, Moodle quizzes and workshops were the most prevalent, with 16 and eight articles, respectively (see Fig.  8 ). Some activities, such as videos, virtual tours, e-portfolios, are external tools easily embedded into the Moodle system.

Number of articles reporting research on each Moodle activity

None of the articles discussed Moodle activities, such as Choice, Database, Feedback, Glossary, H5P activity, or SCORM (for course content). One study (Sánchez et al., 2015) recommends Moodle's “Survey” tool for anonymous surveys, yet if this tool is used along with Moodle’s “Group” option, the users can determine who responds to the survey. Therefore, the “Feedback” activity is a better anonymous survey tool than the “Survey” activity.

Except for Shkoukani ( 2019 ), who analysed features for the 20 most popular LMSs in 2018, few studies compare Moodle with other LMSs. Only 2% of papers analysed in this study have compared Moodle with other LMSs, and they only compared Moodle with Blackboard or Canvas (Aljawarneh, 2020 ; Shdaifat & Obeidallah, 2019 ). Further analysis between LMSs focusing on features, integration, cost, and security are pivotal for e-learning success.

Theme 2: curriculum development

In 53% of the reviewed articles, LMS Moodle was used for curriculum development, including implementing learning modules and assessments for blended and online courses. While about half of the articles (45%) explain how this can be used at the course level (e.g., Awofeso et al., 2016 ; Brateanu et al., 2019 ; Chootongchai & Songkram, 2018 ), 4% of the articles explain how this can be used for framework design (multiple courses to achieve program objectives) (e.g., Kouis et al., 2020 ; Saleh & Salama, 2018 ; Smolyaninova & Bezyzvestnykh, 2019 ).

Educators bear responsibility for ensuring optimal tools are utilised for the most effective computerised assessment that enables students and teachers to address or avoid assessment-related problems (Marczak et al., 2016 ). However, only 4% of papers analyse the teachers’ perspectives of using Moodle (Babo & Suhonen, 2018 ; Badia et al., 2019 ; García-Martín & García-Sánchez, 2020 ; Jackson, 2017 ; Marczak et al., 2016 ; Valero & Cárdenas, 2017 ). Badia et al. ( 2019 ) conducted a study using 132 teachers across 43 schools indicated further research should be conducted on: Why do only certain Moodle activities positively impact learning outcomes? What can technological designers and teachers do to improve the level of learning outcomes achieved through the use of Moodle activities?

Of the 155 articles reviewed, only eight used educational theoretical frameworks for their research and development (see Table 6 ). According to the studies shown in Table  6 , online assessments can be theorised using Classical Test Theory (CTT) and Item Response Theory (IRT). Online content development, particularly adaptive content, can be theorised using Computer Adaptive Testing (CAT), the Technology Acceptance Model (TAM), Merrill's problem-centric framework, and Self-determination theory. The DeLone and McLean Information Systems (IS) theories can be used to measure the effectiveness of advanced online materials and for the implementation of e-learning systems.

Theme 3: learning focus

Adaptive, collaborative, or problem-based content developments were discussed in 20% of the articles, with only 4% considering learning styles and critical thinking.

Adaptive learning

LMSs provide large data databases and fast access to a systematic analysis of information. Therefore, designing adaptive or self-learning modules and automatic assessments which adapt to the learner’s preferences has become much easier. Of the articles reviewed, 8% either demonstrate or improve automated content. The areas addressed within these articles were randomly generated tests, questions with multiple possible answers, automated marking systems and rubrics, provision of positive and motivational automatic summative and formative feedback, auto-adaptive content for learners with diverse backgrounds, interactive content, self-assessed quiz and multimedia books for instructional design (Azevedo et al., 2019 ; Brateanu et al., 2019 ; Gutiérrez et al., 2016 ; Ljubimova et al., 2015 ; Paiva et al., 2015 ).

Further research has investigated integrating instructional design theories, psychological elements, and learning theories into adaptive learning (Abuhassna & Yahaya, 2018 ; Conejo et al., 2016 ; Saleh & Salama, 2018 ). Tlili et al. ( 2019 ) conducted a study that aimed to model the learners’ personalities using a learning analytics approach called intelligent Moodle (iMoodle), with results compared to the traditional method of modelling learners' personalities using questionnaires (Tlili et al., 2019 ). A further study investigated automatic detection of learning styles by analysing student learning behaviour by constructing a mathematical model (Xiao & Rahman, 2017 ). Further research has been suggested in the areas of exploring the extent to which automatic feedback encourages positive motivational beliefs and self‐esteem among students (Gaona, et al., 2018 ), improving real-time adaptation learning modules, intelligent non-human tutoring, and using educational data mining techniques to investigate and predict students' attitude to learning.

Collaborative learning

Collaborative learning was discussed by 12% of the reviewed articles. Of these, a number focused on Moodle's peer assessment tool “workshop” and demonstrated how to use “workshop” to allow students to mark their fellow students’ work and reduce the marking load for teaching staff (ArchMiller et al., 2017 ; Slee & Jacobs, 2017 ; Strang, 2015 ). Peer review and feedback were generally accepted as helping to develop students’ meta-cognitive skills relating to critical reflection (Wilson et al., 2015 ). However, qualitative studies show that students and staff have divided opinions regarding the “workshop” tool for peer assessment (Divjak & Maretić, 2017 ; Dolezal et al., 2018 ; Wilson et al., 2015 ). While students agree with a limited number of peer assessments, staff experience an increase or no decrease in their marking workload (Wilson et al., 2015 ). However, peer assessments using “workshop” are still time-consuming for both the teacher and students and could lose their charm if they are overused (Dolezal et al., 2018 ). In studies that have used peer assessments to allow students to grade their peers, some students reported the peer assessment method as “unfair and “unprofessional” (Divjak & Maretić, 2017 ; Dolezal et al., 2018 ; Wilson et al., 2015 ). The “workshop” tool in Moodle does not have a built-in measure for peer assessment validity. One study which addressed the concern of students’ validity contributing to marking assignments reported that the grades were consistent with what faculty expected based on t tests and reliability estimates (Strang, 2015 ).

The Moodle activity “Forum” can be used to improve problem-based learning via group projects (Awofeso et al., 2016 ). “Forums” allowed students to maintain much more direct contact when they were not in the class and made it easier for students to meet and work on their projects even though they were in different places (Marti et al., 2015 ; Phungsuk et al., 2017 ). A further study reported that online learning systems positively influenced students' thinking and innovation skills (Chootongchai & Songkram, 2018 ).

Learning styles

Of the identified articles, 3% investigated learning styles—namely, Active vs Reflective, Sensitive vs Intuitive, Visual vs Verbal, Sequential vs Global—when implementing e-course content (Kouis et al., 2020 ; Ljubimova et al., 2015 ; Xiao & Rahman, 2017 ). These studies have shown that students' independent work can be guided through interactive technology, and these teaching methods would eliminate students' passivity in the classroom and enhance their cognitive activity. While some studies have proposed automatic detection of learning styles by analysing student's learning behaviour through mathematical models (Xiao & Rahman, 2017 ), other studies have developed simpler matrix systems that would allow the teacher to carry out a manual selection of tools for Moodle Learning after considering student's learning styles (Ljubimova et al., 2015 ; Meza-Fernández & Sepúlveda-Sariego, 2017 ; Xiao & Rahman, 2017 ). However, identifying students' learning styles to maintain assessment quality needs further investigation (Meza-Fernández & Sepúlveda-Sariego, 2017 ).

Theme 4: assessment

A third (33%) of the reviewed papers focused on assessment including summative and formative assessment, online exams, marking, and feedback (Adesemowo et al., 2016 ; Albano & Dello Iacono, 2019 ; Basol & Balgalmis, 2016 ; George-Williams et al., 2019 ). Moodle can create large data pools of various questions, including multiple-choice, open answer, generative questions, and complex tasks (Conejo et al., 2016 ). Nevertheless, most papers focused on summative assessment based on Moodle quizzes investigating both teachers’ and students’ opinions when implementing multiple-choice questions (Babo & Suhonen, 2018 ; Cakiroglu et al., 2017 ; Dimic et al., 2018 ; McKenzie & Roodenburg, 2017 ; Shdaifat & Obeidallah, 2019 ). According to a 5-year study, the ‘luck’ factor associated with multiple-choice questions is fair (Babo et al., 2020 ). Studies that have investigated the students' point of view indicate that the students agree that Moodle is easy to use and complements teaching, although most students still prefer classical assessment techniques (Cakiroglu et al., 2017 ; McVey, 2016 ; Popovic et al., 2018 ). However, one study found no direct relationships between students' preferences and academic performance (Cakiroglu et al., 2017 ).

Some studies which focused on the assessment process investigated the usefulness of the online environment for instructors to organise assessments, the usefulness of giving responsibilities to students during assessment (mainly via peer assessments), and using Moodle statistics and analytics to evaluate and improve the quality assessment process (Cakiroglu et al., 2017 ; Gamage et al., 2019 ; Hussain & Jaeger, 2018 ).

Marking and feedback

Four percent of reviewed articles focused on improving and streamlining the marking and feedback processes for both students and teachers. These studies indicate that online marking systems associated with Moodle lower the long-term costs, increase the speed of providing feedback, provide greater flexibility with respect to location and timing and reduce the space required to manage the assessment process (García López & García Mazarío, 2016 ; Koneru, 2017 ; Villa et al., 2018 ). A study with 57 academics conducted at Monash University, Australia, highlighted Moodle's reliability, and improved impartiality of the assessment process (George-Williams et al., 2019 ). The study concluded that this impartiality is generally achieved through the removal of personal, academic judgment, which results in more reliable, consistent marking practices.

Theme 5: ethics

The reviewed articles investigated two strands of ethics: (1) ethics relating to users' data security and privacy, and (2) academic integrity. While 4% of all reviewed articles highlighted security and privacy concerns, 6% of the articles discussed academic integrity issues caused by the increased use of LMSs for assessment purposes. Although personal data protection has legal compliance, such as the policies in the European Union and the Privacy Act 1988 in Australia, several articles discussed the privacy concerns of cloud-based services. The use of cloud-based services has resulted in teaching materials being stolen, and instructors' or administrators' credentials being compromised (Daniels & Iwago, 2017 ; Kiennert et al., 2019 ; Mudiyanselage & Pan, 2020 ).

Two re-occurring academic integrity issues associated with online assessments were highlighted: students plagiarising and students using third parties to complete assignments (Amoako & Osunmakinde, 2020 ; Guillén-Gámez & García-Magariño, 2015 ). Although instances of these two integrity problems occur in traditional teaching and learning methods, face-to-face invigilated exam environments can help minimise the effect of these issues. One alternative to invigilated exams is online quizzes which have become popular due to their ability to automate marking. However, cheating cannot be controlled unless it is held in an invigilated room. Several studies attempted to address this issue by introducing new software and analytical tools to detect academic misconduct. These tools include: limiting IP range for the users during online exams (Adesemowo et al., 2016 ); using timestamps and data processing techniques to identify unauthorised users (Genci, 2014 ); using facial verification software (Guillén-Gámez & García-Magariño, 2015 ) and using plagiarism detection software (Adesemowo et al., 2016 ; Genci, 2014 ; Guillén-Gámez & García-Magariño, 2015 ; Kaya & Özel, 2015 ).

Theme 6: technical developments.

Application of moodle analytics.

Online LMSs make it more manageable to gather and analyse students’ data. Ten percent of the articles reviewed discussed the in-built statistical tools such as the facility index and discrimination index along with the databases available in LMSs for the use of educational and research purposes (Fenu et al., 2017 ; Gamage et al., 2019 ; Monllaó Olivé et al., 2020 ). The articles used data mining and statistical tools to measure and analyse student engagement, student satisfaction, and online courses' performance. Analysing the tools available would be beneficial for monitoring student retention rates (Monllaó Olivé et al., 2020 ), identifying underachieving students (Saqr et al., 2017 ), predicting students' trends and attitudes, and accreditation purposes (El Tantawi et al., 2015 ; Saleh & Salama, 2018 ; Strang, 2016 ). Data and analytics tools may also be used to automate personality assessments and create intelligent (adaptive) learning platforms (Tlili et al., 2019 ).

Software development and adaptation

This review found that 24% of the articles discussed or evaluated software development and adaptations, including the use of existing software to improve the learning experience within Moodle. Software applications that can be integrated into Moodle include:

Apple's Siri and Google's GRScloud-based speech recognition for language learning (Daniels & Iwago, 2017 ).

OpenIRS-UCM (García López & García Mazarío, 2016 ), Kahoot, Poll-Everywhere and Zappar (Hsiung, 2018 ) which are tools for interactive polling.

The ever-increasing number of new software/Add-Ins available for Moodle is indicative of the interest of software developers and researchers to improve the useability of Moodle for online teaching and education. Course developers utilise plug-ins to assist with automatic essay marking, randomising questions, and identifying ineffective questions (Koneru, 2017 ; Schweighofer et al., 2019 ; Villa et al., 2018 ). Table 7 lists several software applications that can be integrated into LMSs and, in particular, Moodle.

To date (June 2021), Moodle has 1753 available plug-ins that can add new functions that improve administration, assessment, collaboration, communication, content and the interface (Moodle Project, 2020b ). The Moodle statistics for 2019 show that the most popular plug-ins (based on the number of downloads) were communication and content plug-ins, such as Moove, BigBlueBN, Adaptable, H5P, and Eguru (Moodle Project, 2020b ). The articles in this review covering Jan 2015–June 2021 show that most reported advancements in new software developments for Moodle relate to improving assessment processes. The development advancements include improving the security of the assessment processes (Adesemowo et al., 2016 ; Kaya & Özel, 2015 ), improving the mechanisms to generate quiz questions, and improving feedback and response time (Conejo et al., 2016 ; Kruger et al., 2015 ). Security improvements include, but are not limited to, improving user data verification (Amoako & Osunmakinde, 2020 ), facial recognition (Guillén-Gámez & García-Magariño, 2015 ), limiting IP range (Adesemowo et al., 2016 ), and scanning students IDs (Ross, 2017 ). Daniels and Iwago ( 2017 ) also reported on integrating Google speech recognition for speech assessments. Improving students cognitive, innovative, and collaborative learning skills were a key area of development in some reported studies (Chootongchai & Songkram, 2018 ; Finogeev et al., 2020 ; García López & García Mazarío, 2016 ), along with the improvement of user interface evaluation (Fenu et al., 2017 ).

Artificial intelligence tools are an increasing area of research which investigates intellectual mechanisms for managing personalised learning. Gray et al. ( 2018 ) reported on the software developments that aid students in their report writing and allow arguments, justification, and conclusions to be formed without any human input. Software development also encompasses the ability to direct students to relevant content and assessments after automatic analysis of the students' behaviour (Finogeev et al., 2020 ) and can also evaluate summaries written by students using information available on websites and online repositories (Ramírez-Noriega et al., 2018 ). As software advancements to assist students with their assignments are increasing, so is plagiarism. Plagiarism detection systems are successfully integrated into Moodle with plug-ins, such as Urkund, Turnitin, Plagiarisma, and SafeAssign which can detect textual plagiarism. Source code detection software for programming courses are under development (Kaya & Özel, 2015 ).

Despite advances in software and technology for e-learning and online LMSs, numerous fundamental gaps/drawbacks still exist, with the majority on technical issues (Adesemowo et al., 2016 ; Marczak et al., 2016 ; Rachman‐Elbaum, et al, 2017 ), such as server/browser response times, lag time in resolving technical issues, lack of equipment available to students and the possible high cost associated with the initial development of programs (Chang Chan et al., 2019 ; El Tantawi et al., 2015 ; Marczak et al., 2016 ; Zamalia & Porter, 2016 ).

Theme 7: research approach and methods

The research approaches used are categorised into quantitative analysis, qualitative analysis, mixed methods, technical and other. Of the 155 articles reviewed, 67 papers used a quantitative (QN) research approach which aimed to quantify a phenomenon relevant to online teaching and learning (see Fig.  9 ). Forty-eight papers used a qualitative (QL) research approach which involved descriptive data collection, student, teacher, or other stakeholder thoughts and experiences; 28 papers used mixed methods—both qualitative and quantitative approaches; 37 papers discussed technical (T) components of LMS and included new software development and framework design; and, 37 papers were categorised as “other”, namely, research that did not fall into the above three categories, e.g., applications of existing LMSs and tools, reviewing/comparing existing LMSs or tools.

Venn diagram for QN (quantitative), QL (qualitative), and T (technical) types of research

Qualitative research studies in this review evaluated mainly the students’ perspective: their preferences, perceptions, satisfaction, and attitudes towards online learning, including the online tools being utilised (Botelho et al., 2020 ; Cakiroglu et al., 2017 ; García-Martín & García-Sánchez, 2020 ; Tsai & Tang, 2017 ). Only two research studies focused exclusively on teacher opinions, perceptions, and experiences in e-assessment, Moodle activities, and their learning impacts (Babo & Suhonen, 2018 ; Badia et al., 2019 ). Four articles reported on both student and teacher perspectives and discussed attitudes towards summative and formative assessments and flexibility in e-learning (Jackson, 2017 ; Kamenez et al., 2018 ; Marczak et al., 2016 ; Valero & Cárdenas, 2017 ). Jackson ( 2017 ) reported that Moodle is a technology that enables creativity among teachers and recommended that management incorporate training programs of LMSs for both teachers and students into their strategic plans.

Theme 8: student success factors

The qualitative, quantitative, and mixed methods research have common indexes used as student success indicators, namely, student performance, engagement, and satisfaction indicators (as described in Table 2 ). Figure  7 h shows the articles that discussed student success factors with 14% using student performance, 16% student engagement, and 8% student satisfaction. Student performance and engagement are mainly found in quantitative research, whereas student satisfaction indicators are found in qualitative research. Qualitative research measuring student satisfaction are fewer than quantitative research analysing student performance and engagement.

This comprehensive systematic review on Moodle use for online teaching and learning covers a wide range of educational institutions. The review identifies methods used and developments over the last 6 years published in 155 journal articles across 104 journals over 55 countries and 10 disciplines. The findings have been summarised bibliographically and thematically where appropriate, providing vital information to educators, researchers, and software developers. The critical limitation of this review is that only Scopus and Web of Science databases were used for the search, and papers that are not covered by either database are not included in this analysis.

The bibliographic analysis identified Moodle as a well-established and advanced learning platform for multiple disciplines and particularly used in STEM education. Most of the literature (75%) focus on university settings, with the majority (96%) on undergraduate studies. The bibliographic analysis shows the increasing trend in Moodle educational research and provides information about the top journals, leading authors, keywords, and high citations. The thematic analysis finds that Moodle is a powerful tool used to support learning in various ways. Both educators and students benefit from using the Moodle LMS, although currently at varying degrees. The most prevalent tools being used are Moodle “quizzes” and “workshops”, and external tools that can be easily embedded into the Moodle system are videos, virtual tours, and e-portfolios. Moodle enables the creativity of individual teachers to develop course-specific materials for students. In addition, Moodle saves time due to randomly generated tests, questions with multiple possible answers, automated marking systems and rubrics, and positive and motivational automatic summative and formative feedback. There is strong evidence that Moodle increases student engagement, performance, and satisfaction while enhancing flexibility in their learning environments. Areas showing a rapid growth in research are adaptive content and assessment development, improvements in data security, and user verification. Regardless of recent advancements in online teaching and learning, some studies report numerous fundamental gaps and drawbacks.

The gaps identified in this review are significant for future research. Some gaps include comparing Moodle with other LMSs and elaborating on the many e-learning tools and associated plug-ins available in the market but not analysed in educational research. Future research could focus on aspects pivotal for e-learning success: features, integration, cost, and security. Further research is needed to outline Moodle e-learning experiences in primary and secondary education settings, with qualitative studies needed, particularly focusing on teachers’ perspectives in a tertiary education setting. As only 5% of the studies have considered educational theories, future research needs to strengthen the theoretical underpinning of studies. Existing educational theories could successfully theorise the efficiency of content developments and the effectiveness of online study materials and assignments. Data gathering tools and statistical tools embedded into LMSs along with theoretical frameworks could lead to insightful research. As only 10% of articles discussed ethical aspects, more publications are needed to analyse ethical issues associated with e-learning, particularly focusing on the increasing number of artificial intelligence tools. More research on these aspects will help educators to utilise LMSs for successful online or blended course developments. As this review is based only on published articles, more applications of Moodle might be occurring, particularly in developing countries. Therefore, an area of future study could be a study examining statistics of Moodle usage rather than published papers.

Availability of data and materials

The data sets used and analysed during the current study are available from the corresponding author on request.

Abuhassna, H., & Yahaya, N. (2018). Students’ Utilization of distance learning through an interventional online module based on Moore Transactional Distance theory. Eurasia Journal of Mathematics, Science and Technology Education, 14 (7), 3043–3052. https://doi.org/10.29333/ejmste/91606

Article   Google Scholar  

Adesemowo, A. K., Johannes, H., Goldstone, S., & Terblanche, K. (2016). The experience of introducing secure e-assessment in a South African university first-year foundational ICT networking course. Africa Education Review, 13 (1), 67–86. https://doi.org/10.1080/18146627.2016.1186922

Al-Ajlan, A., & Zedan, H. (2008). Why Moodle?. Paper presented at the 12th IEEE International workshop on future trends of distributed computing systems. 2008. doi: https://doi.org/10.1109/ftdcs13956.2008 .

Albano, G., & Dello Iacono, U. (2019). GeoGebra in e-learning environments: A possible integration in mathematics and beyond. Journal of Ambient Intelligence and HumanizedCcomputing, 10 (11), 4331–4343. https://doi.org/10.1007/s12652-018-1111-x

Aljawarneh, S. A. (2020). Reviewing and exploring innovative ubiquitous learning tools in higher education. Journal of Computing in Higher Education, 32 (1), 57–73. https://doi.org/10.1007/s12528-019-09207-0

Alkholy, S., Gendron, F., Dahms, T., & Ferreira, M. P. (2015). Assessing student perceptions of indigenous science co-educators, interest in STEM, and identity as a scientist: A pilot study. Ubiquitous Learning, 7 (3–4), 41–51.

Altinpulluk, H., & Kesim, M. (2021). A systematic review of the tendencies in the use of learning management systems. The Turkish Online Journal of Distance Education, 22 (3), 40–54. https://doi.org/10.17718/tojde.961812

Amoako, P. Y. O., & Osunmakinde, I. O. (2020). Emerging bimodal biometrics authentication for non-venue-based assessments in open distance e-learning (OdeL) environments. International Journal of Technology Enhanced Learning, 12 (2), 218–244. https://doi.org/10.1504/IJTEL.2020.106287

Araya, R., & Collanqui, P. (2021). Are cross-border classes feasible for students to collaborate in the analysis of energy efficiency strategies for socioeconomic development while keeping CO 2 concentration controlled? Sustainability (basel, Switzerland), 13 (3), 1–20. https://doi.org/10.3390/su13031584

ArchMiller, A., Fieberg, J., Walker, J. D., & Holm, N. (2017). Group peer assessment for summative evaluation in a graduate-level statistics course for ecologists. Assessment and Evaluation in Higher Education, 42 (8), 1208–1220. https://doi.org/10.1080/02602938.2016.1243219

Ardianti, S., Sulisworo, D., Pramudya, Y., & Raharjo, W. (2020). The impact of the use of STEM education approach on the blended learning to improve student’s critical thinking skills. Universal Journal of Educational Research, 8 (3B), 24–32. https://doi.org/10.13189/ujer.2020.081503

Awofeso, N., Hassan, M., & Hamidi, S. (2016). Individual and collaborative technology-mediated learning using question & answer online discussion forums: Perceptions of public health learners in Dubai UAE. Open Learning, 31 (1), 54–63. https://doi.org/10.1080/02680513.2015.1120662

Azevedo, J. M., Oliveira, E. P., & Beites, P. D. (2019). Using learning analytics to evaluate the quality of multiple-choice questions: A perspective with Classical Test Theory and Item Response Theory. The International Journal of Information and Learning Technology, 36 (4), 322–341. https://doi.org/10.1108/IJILT-02-2019-0023

Babo, R., Babo, L. V., Suhonen, J. T., & Tukiainen, M. (2020). E-assessment with multiple-choice questions: A 5-year study of students’ opinions and experience. Journal of Information Technology Education: Innovations in Practice, 19 , 1–29. https://doi.org/10.28945/4491

Babo, R., & Suhonen, J. (2018). E-assessment with multiple choice questions: A qualitative study of teachers’ opinions and experience regarding the new assessment strategy. International Journal of Learning Technology, 13 (3), 220–248. https://doi.org/10.1504/IJLT.2018.095964

Badia, A., Martín, D., & Gómez, M. (2019). Teachers’ perceptions of the use of Moodle activities and their learning impact in secondary education. Technology, Knowledge and Learning, 24 (3), 483–499. https://doi.org/10.1007/s10758-018-9354-3

Basol, G., & Balgalmis, E. (2016). A multivariate investigation of gender differences in the number of online tests received-checking for perceived self-regulation. Computers in Human Behavior, 58 , 388–397. https://doi.org/10.1016/j.chb.2016.01.010

Bernacki, M. L., Vosicka, L., & Utz, J. C. (2020). Can a brief, digital skill training intervention help undergraduates “learn to learn” and improve their STEM achievement? Journal of Educational Psychology, 112 (4), 765–781. https://doi.org/10.1037/edu0000405

Botelho, M., Gao, X., & Bhuyan, S. Y. (2020). Mixed-methods analysis of videoed expert-student dialogue supporting clinical competence assessments. European Journal of Dental Education, 24 (3), 398–406. https://doi.org/10.1111/eje.12515

Brateanu, A., Strang, T. M., Garber, A., Mani, S., Spencer, A., Spevak, B., Thomascik, J., Mehta, N., & Colbert, C. Y. (2019). Using an adaptive, self-directed web-based learning module to enhance residents’ medical knowledge prior to a new clinical rotation. Medical Science Educator, 29 (3), 779–786. https://doi.org/10.1007/s40670-019-00772-8

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Byrnes, K. G., Kiely, P. A., Dunne, C. P., McDermott, K. W., & Coffey, J. C. (2021). Communication, collaboration and contagion: “Virtualisation” of anatomy during COVID-19. Clinical Anatomy, 34 (1), 82–89. https://doi.org/10.1002/ca.23649

Cadaret, C. N., & Yates, D. T. (2018). Retrieval practice in the form of online homework improved information retention more when spaced 5 days rather than 1 day after class in two physiology courses. Advances in Physiology Education, 42 (2), 305–310. https://doi.org/10.1152/advan.00104.2017

Cakiroglu, U., Erdogdu, F., Kokoc, M., & Atabay, M. (2017). Student’s preference in online assessment process: Influence on academic performance. The Turkish Online Journal of Distance Education, 18 (1), 132–132. https://doi.org/10.17718/tojde.285721

Campbell, L. O., Heller, S., & Pulse, L. (2020). Student-created video: An active learning approach in online environments. Interactive Learning Environments . https://doi.org/10.1080/10494820.2020.1711777

Capterra (2021). LMS software. https://www.capterra.com/learning-management-system-software/?feature=%5B38347%5D&sortOrder=sponsored . Accessed 15 Oct 2021.

Chafiq, N., Talbi, M., & Ghazouani, M. (2018). Design and implementation of a risk management tool: A case study of the Moodle platform. International Journal of Advanced Computer Science and Applications, 9 (8), 458–461.

Chang Chan, A.Y.-C., Custer, E. J. F. M., van Leeuwen, M. S., Bleys, R. L. A. W., & ten Cate, O. (2019). Correction to: Does an additional online anatomy course improve performance of medical students on gross anatomy examinations? Medical Science Educator, 29 (3), 891–891. https://doi.org/10.1007/s40670-019-00758-6

Chaparro-Peláez, J., Iglesias-Pradas, S., Rodríguez-Sedano, F. J., & Acquila-Natale, E. (2019). Extraction, processing and visualization of peer assessment data in Moodle. Applied Sciences, 10 (1), 163. https://doi.org/10.3390/app10010163

Chemsi, G., Sadiq, M., Radid, M., & Talbi, M. (2020). Study of the self-determined motivation among students in the context of online pedagogical activities. International Journal of Emerging Technologies in Learning, 15 (5), 17–29. https://doi.org/10.3991/ijet.v15i05.11392

Chootongchai, S., & Songkram, N. (2018). Design and development of SECI and Moodle online learning s to enhance thinking and innovation skills for higher education learners. International Journal of Emerging Technologies in Learning, 13 (3), 154–172. https://doi.org/10.3991/ijet.v13i03.7991

Christopoulos, A., Pellas, N., & Laakso, M.-J. (2020). A learning analytics theoretical framework for STEM education virtual reality applications. Education Sciences, 10 (11), 317. https://doi.org/10.3390/educsci10110317

Clarke, V., & Braun, V. (2014). Thematic analysis. In T. Teo (Ed.), Encyclopedia of critical psychology (pp. 1947–1952). Springer.

Chapter   Google Scholar  

Colares, G. S., Dell’Osbel, N., Wiesel, P. G., Oliveira, G. A., Lemos, P. H. Z., da Silva, F. P., Lutterbeck, C. A., Kist, L. T., & Machado, Ê. L. (2020). Floating treatment wetlands: A review and bibliometric analysis. The Science of the Total Environment, 714 , 136776–136776. https://doi.org/10.1016/j.scitotenv.2020.136776

Conejo, R., Guzmán, E., & Trella, M. (2016). The SIETTE automatic assessment environment. International Journal of Artificial Intelligence in Education, 26 (1), 270–292. https://doi.org/10.1007/s40593-015-0078-4

Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting student performance from LMS data: A comparison of 17 blended courses using Moodle LMS. IEEE Transactions on Learning Technologies, 10 (1), 17–29. https://doi.org/10.1109/TLT.2016.2616312

de Souza, M. P., Hoeltz, M., Brittes Benitez, L., Machado, Ê. L., & de Souza Schneider, R. C. (2019). Microalgae and clean technologies: A review. Clean: Soil, Air, Water, 47 (11), 1800380. doi: https://doi.org/10.1002/clen.201800380 .

Daniels, P., & Iwago, K. (2017). The suitability of cloudbased speech recognition engines for language learning. JALT CALL Journal, 13 (3), 229–239.

Dias, S. B., Hadjileontiadou, S. J., Diniz, J., & Hadjileontiadis, L. J. (2020). Deep LMS: A deep learning predictive model for supporting online learning in the Covid-19 era. Scientific Reports, 10 (1), 19888–19888. https://doi.org/10.1038/s41598-020-76740-9

Dimic, G., Predic, B., Rancic, D., Petrovic, V., Macek, N., & Spalevic, P. (2018). Association analysis of Moodle e-tests in blended learning educational environment. Computer Applications in Engineering Education, 26 (3), 417–430. https://doi.org/10.1002/cae.21894

Divjak, B., & Maretić, M. (2017). Learning analytics for peer-assessment: (Dis)advantages, reliability and implementation. Journal of Information and Organizational Sciences, 41 (1), 21–34. https://doi.org/10.31341/jios.41.1.2

Dolezal, D., Posekany, A., Roschger, C., Koppensteiner, G., Motschnig, R., & Pucher, R. (2018). Person-centered learning using peer review method: An evaluation and a concept for student-centered classrooms. International Journal of Engineering Pedagogy, 8 (1), 127–147. https://doi.org/10.3991/ijep.v8i1.8099

Dominguez, M., Bernacki, M. L., & Uesbeck, P. M. (2016). Predicting STEM achievement with learning management system data: Prediction modeling and a test of an early warning system. Paper presented at the EDM.

El Tantawi, M. M. A., Abdelsalam, M. M., Mourady, A. M., & Elrifae, I. M. B. (2015). e-Assessment in a limited-resources dental school using an open-source learning management system. Journal of Dental Education, 79 (5), 571–583. https://doi.org/10.1002/j.0022-0337.2015.79.5.tb05917.x

Fenu, G., Marras, M., & Meles, M. (2017). A learning analytics tool for usability assessment in Moodle environments. Journal of e-Learning and Knowledge Society, 13 (3), 23–34. https://doi.org/10.20368/1971-8829/1388

Finogeev, A., Gamidullaeva, L., Bershadsky, A., Fionova, L., Deev, M., & Finogeev, A. (2020). Convergent approach to synthesis of the information learning environment for higher education. Education and Information Technologies, 25 (1), 11–30. https://doi.org/10.1007/s10639-019-09903-5

Gamage, S. H. P. W., Ayres, J. R., Behrend, M. B., & Smith, E. J. (2019). Optimising Moodle quizzes for online assessments. International Journal of STEM Education, 6 (1), 1–14. https://doi.org/10.1186/s40594-019-0181-4

Gaona, J., Reguant, M., Valdivia, I., Vásquez, M., & Sancho-Vinuesa, T. (2018). Feedback by automatic assessment systems used in mathematics homework in the engineering field. Computer Applications in Engineering Education, 26 (4), 994–1007. https://doi.org/10.1002/cae.21950

García López, A., & García Mazarío, F. (2016). The use of technology in a model of formative assessment. Journal of Technology and Science Education, 6 (2), 91–103. https://doi.org/10.3926/jotse.190

García-Martín, J., & García-Sánchez, J.-N. (2020). The effectiveness of four instructional approaches used in a MOOC promoting personal skills for success in life. Revista De Psicodidáctica (english Ed.), 25 (1), 36–44. https://doi.org/10.1016/j.psicoe.2019.08.001

Genci, J. (2014). About one way to discover formative a cheating . 312 , 83–90. Cham: Switzerland: Springer International Publishing.

George-Williams, S., Carroll, M.-R., Ziebell, A., Thompson, C., & Overton, T. (2019). Curtailing marking variation and enhancing feedback in large scale undergraduate chemistry courses through reducing academic judgement: A case study. Assessment and Evaluation in Higher Education, 44 (6), 881–893. https://doi.org/10.1080/02602938.2018.1545897

Gray, W. G., Lado, M. J., Zhang, Z., Iskander, M. F., Garcia-Gorrostieta, J. M., Lopez-Lopez, A., & Gonzalez-Lopez, S. (2018). Automatic argument assessment of final project reports of computer engineering students. Computer Applications in Engineering Education, 26 (5), 1217–1226. https://doi.org/10.1002/cae.21996

Guillén-Gámez, F. D., & García-Magariño, I. (2015). Use of facial authentication in E-learning: A study of how it affects students in different Spanish-speaking areas. International Journal of Technology Enhanced Learning, 7 (3), 264–280. https://doi.org/10.1504/IJTEL.2015.072818

Guillen-Gamez, F. D., Garcia-Magarino, I., Bravo, J., & Plaza, I. (2015). Exploring the influence of facial verification software on student academic performance in online learning environments. International Journal of Engineering Education, 31 (6A), 1622–1628.

Google Scholar  

Gutiérrez, I., Álvarez, V., Puerto Paule, M., Pérez-Pérez, J. R., & de Freitas, S. (2016). Adaptation in e-learning content specifications with dynamic sharable objects. Systems (basel), 4 (2), 24. https://doi.org/10.3390/systems4020024

Hempel, B., Kiehlbaugh, K., & Blowers, P. (2020). Scalable and practical teaching practices faculty can deploy to increase retention: A faculty cookbook for increasing student success. Education for Chemical Engineers, 33 , 45–65. https://doi.org/10.1016/j.ece.2020.07.004

Henke, K., Nau, J., Bock, R. N., & Wuttke, H.-D. (2021). A hybrid online laboratory for basic STEM education. In Uskov V.L., Howlett R.J., & J. L.C. (Eds.), Smart Education and e-Learning 2021 , 240 , 29–39, New York, N. Y., Springer.

Henrick, G. (2018). Moodle 2 interactive tool guide gets an interactive treatment. Moodle News. https://www.moodlenews.com/2015/moodle-2-interactive-tool-guide-gets-an-interactive-treatment/ . Accessed 26 Feb 2019.

Hsiung, W. Y. (2018). The use of e-resources and innovative technology in transforming traditional teaching in chemistry and its impact on learning chemistry. International Journal of Interactive Mobile Technologies, 12 (7), 86–96. https://doi.org/10.3991/ijim.v12i7.9666

Hussain, Y. A., & Jaeger, M. (2018). LMS-supported PBL assessment in an undergraduate engineering program: Case study. Computer Applications in Engineering Education, 26 (5), 1915–1929. https://doi.org/10.1002/cae.22037

Hwang, C. S. (2020). Using continuous student feedback to course-correct during COVID-19 for a monmajors chemistry course. Journal of Chemical Education, 97 (9), 3400–3405. https://doi.org/10.1021/acs.jchemed.0c00808

ISCED, (2012). International Standard Classification of Education (ISCED) 2011, https://doi.org/10.15220/978-92-9189-123-8-en . Accessed 22 Jan 2021

Jackson, E. A. (2017). Impact of MOODLE platform on the pedagogy of students and staff: Cross-curricular comparison. Education and Information Technologies, 22 (1), 177–193. https://doi.org/10.1007/s10639-015-9438-9

Jones, D., Lotz, N., & Holden, G. (2021). A longitudinal study of virtual design studio (VDS) use in STEM distance design education. International Journal of Technology and Design Education, 31 (4), 839–865. https://doi.org/10.1007/s10798-020-09576-z

Kamenez, N. V., Vaganova, O. I., Smirnova, Z. V., Bulayeva, M. N., Kuznetsova, E., & Maseleno, A. (2018). Experience of the use of electronic training in the educational process of the Russian higher educational institution. International Journal of Engineering and Technology (UAE), 7 (4), 4085–4089.

Kaya, M., & Özel, S. A. (2015). Integrating an online compiler and a plagiarism detection tool into the Moodle distance education system for easy assessment of programming assignments. Computer Applications in Engineering Education, 23 (3), 363–373. https://doi.org/10.1002/cae.21606

Kiennert, C., De Vos, N., Knockaert, M., & Garcia-Alfaro, J. (2019). The influence of conception paradigms on data protection in e-learning platforms: A case study. IEEE Access, 7 , 64110–64119. https://doi.org/10.1109/ACCESS.2019.2915275

Koneru, I. (2017). Exploring moodle functionality for managing Open Distance Learning e-assessments. The Turkish Online Journal of Distance Education, 18 (4), 129–141. https://doi.org/10.17718/tojde.340402

Kouis, D., Kyprianos, K., Ermidou, P., Kaimakis, P., & Koulouris, A. (2020). A framework for assessing LMSs e-courses content type compatibility with learning styles dimensions. Journal of e-Learning and Knowledge Society, 16 (2), 73–86. https://doi.org/10.20368/1971-8829/1135204

Kruger, D., Inman, S., Ding, Z., Kang, Y., Kuna, P., Liu, Y., Lu, X., Oro, S., & Wang, Y. (2015). Improving teacher effectiveness: Designing better assessment tools in learning management systems. Future Internet, 7 (4), 484–499. https://doi.org/10.3390/fi7040484

Li, Y., Wang, K., Xiao, Y., & Froyd, J. E. (2020). Research and trends in STEM education: A systematic review of journal publications. International Journal of STEM Education, 7 (1), 1–16. https://doi.org/10.1186/s40594-020-00207-6

Ljubimova, E. M., Galimullina, E. Z., & Ibatullin, R. R. (2015). The development of university students’ self-sufficiency based on interactive technologies by their immersion in the professional. International Education Studies, 8 (4), 192. https://doi.org/10.5539/ies.v8n4p192

Marczak, M., Krajka, J., & Malec, W. (2016). Web-based assessment and language teachers-from Moodle to WebClass. International Journal of Continuing Engineering Education and Life Long Learning, 26 (1), 44–59. https://doi.org/10.1504/IJCEELL.2016.075048

Marjanovic, U., Delić, M., & Lalic, B. (2016). Developing a model to assess the success of e-learning systems: Evidence from a manufacturing company in transitional economy. Information Systems and e-Business Management, 14 (2), 253–272. https://doi.org/10.1007/s10257-015-0282-7

Marti, E., Gurguí, A., Gil, D., Hernández-Sabaté, A., Rocarias, J., & Poveda, F. (2015). PBL On Line: A proposal for the organization, part-time monitoring and assessment of PBL group activities. Journal of Technology and Science Education, 5 (2), 87–96. https://doi.org/10.3926/jotse.145

Matazi, I., Messoussi, R., Bellmallem, S.-E., Oumaira, I., Bennane, A., & Touahni, R. (2018). Development of intelligent multi-agents system for collaborative e-learning support. Bulletin of Electrical Engineering Informatics, 7 (2), 294–305. https://doi.org/10.11591/eei.v7i2.860

McKenzie, W., & Roodenburg, J. (2017). Using PeerWise to develop a contributing student pedagogy for postgraduate psychology. Australasian Journal of Educational Technology, 33 (1), 32–47. https://doi.org/10.14742/ajet.3169

McVey, M. (2016). Preservice teachers’ perception of assessment strategies in online teaching. Journal of Digital Learning in Teacher Education, 32 (4), 119–127. https://doi.org/10.1080/21532974.2016.1205460

Meza-Fernández, S., & Sepúlveda-Sariego, A. (2017). Representational model on Moodle’s activity: Learning styles and navigation strategies. International Journal of Educational Technology in Higher Education, 14 (1), 1–9. https://doi.org/10.1186/s41239-017-0052-3

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 (1), 1–9. https://doi.org/10.1186/2046-4053-4-1

Monllaó Olivé, D., Huynh, D. Q., Reynolds, M., Dougiamas, M., & Wiese, D. (2020). A supervised learning framework: Using assessment to identify students at risk of dropping out of a MOOC. Journal of Computing in Higher Education, 32 (1), 9–26. https://doi.org/10.1007/s12528-019-09230-1

Moodle Project, 2020b. Moodle Plug-ins. https://moodle.org/plugins/?q =. Accessed 12 Jan 2021.

Moodle Project, (2020a). Moodle statistics. https://stats.moodle.org/ . Accessed 20 Oct 2020.

Mudiyanselage, A. K., & Pan, L. (2020). Security test MOODLE: A penetration testing case study. International Journal of Computers & Applications, 42 (4), 372–382. https://doi.org/10.1080/1206212X.2017.1396413

Neitola, M. T. T. (2019). Circuit theory e-assessment realized in an open-source learning environment. International Journal of Engineering Pedagogy, 9 (1), 4–18. https://doi.org/10.3991/ijep.v9i1.9072

Nunes, F. B., Herpich, F., Voss, G. B., Lima, J. V. D., & Medina, R. D. (2015). An adaptive environment based on Moodle with treating of quality of context. International Journal of Knowledge Learning, 10 (2), 198–221. https://doi.org/10.1504/IJKL.2015.071618

Oguguo, B. C. E., Nannim, F. A., Agah, J. J., Ugwuanyi, C. S., Ene, C. U., & Nzeadibe, A. C. (2021). Effect of learning management system on student’s performance in educational measurement and evaluation. Education and Information Technologies, 26 (2), 1471–1483. https://doi.org/10.1007/s10639-020-10318-w

Paiva, R. C., Ferreira, M. S., Mendes, A. G., & Eusébio, A. M. J. (2015). Interactive and multimedia contents associated with a system for computer-aided assessment. Journal of Educational Computing Research, 52 (2), 224–256. https://doi.org/10.1177/0735633115571305

Park, Y., & Jo, I.-H. (2017). Using log variables in a learning management system to evaluate learning activity using the lens of activity theory. Assessment and Evaluation in Higher Education, 42 (4), 531–547. https://doi.org/10.1080/02602938.2016.1158236

Phungsuk, R., Viriyavejakul, C., & Ratanaolarn, T. (2017). Development of a problem-based learning model via a virtual learning environment. Kasetsart jJurnal of Social Sciences, 38 (3), 297–306. https://doi.org/10.1016/j.kjss.2017.01.001

Popovic, N., Popovic, T., Dragovic, I. R., & Cmiljanic, O. (2018). A Moodle-based blended learning solution for physiology education in Montenegro: A case study. Advances in Physiology Education, 42 (1), 111–117. https://doi.org/10.1152/ADVAN.00155.2017

Price, E., Lau, A. C., Goldberg, F., Turpen, C., Smith, P. S., Dancy, M., & Robinson, S. (2021). Analyzing a faculty online learning community as a mechanism for supporting faculty implementation of a guided-inquiry curriculum. International Journal of STEM Education, 8 (1), 17–17. https://doi.org/10.1186/s40594-020-00268-7

Rachman-Elbaum, S., Stark, A. H., Kachal, J., Johnson, T., & Porat-Katz, B. S. (2017). Online training introduces a novel approach to the Dietetic Care Process documentation. Nutrition & Dietetics, 74 (4), 365–371. https://doi.org/10.1111/1747-0080.12331

Ramírez-Noriega, A., Juárez-Ramírez, R., Jiménez, S., Inzunza, S., & Martínez-Ramírez, Y. (2018). Ashur: Evaluation of the relation summary-content without human reference using rouge. Computing and Informatics, 37 (2), 509–532. https://doi.org/10.4149/cai_2018_2_509

Raza, S. A., Qazi, W., Khan, K. A., & Salam, J. (2021). Social isolation and acceptance of the Learning Management System (LMS) in the time of COVID-19 pandemic: An expansion of the UTAUT model. Journal of Educational Computing Research, 59 (2), 183–208. https://doi.org/10.1177/0735633120960421

Rissanen, A., & Costello, J. M. (2021). The effectiveness of interactive online tutorials in first-year large biology course. Journal of Applied Research in Higher Education . https://doi.org/10.1108/JARHE-09-2020-0312

Ross, R. (2017). MoodleNFC: Integrating smart student ID cards with Moodle for laboratory assessment. Australasian Journal of Engineering Education., 22 (2), 73–80. https://doi.org/10.1080/22054952.2017.1414557

Saleh, M., & Salama, R. M. (2018). Recommendations for building adaptive cognition-based e-learning. International Journal of Advanced Computer Science and Applications, 9 (8), 385–393.

Sancho-Vinuesa, T., Masià, R., Fuertes-Alpiste, M., & Molas-Castells, N. (2018). Exploring the effectiveness of continuous activity with automatic feedback in online calculus. Computer Applications in Engineering Education, 26 (1), 62–74. https://doi.org/10.1002/cae.21861

Saqr, M., Fors, U., & Tedre, M. (2017). How learning analytics can early predict under-achieving students in a blended medical education course. Medical Teacher, 39 (7), 757–767. https://doi.org/10.1080/0142159X.2017.1309376

Schweighofer, J., Taraghi, B., & Ebner, M. (2019). Development of a quiz: Implementation of a (self-) assessment tool and its integration in Moodle. International Journal of Emerging Technologies in Learning, 14 (23), 141–151. https://doi.org/10.3991/ijet.v14i23.11484

Sergis, S., Vlachopoulos, P., Sampson, D. G., & Pelliccione, L. (2017). Implementing teaching model templates for supporting flipped classroom-enhanced STEM education in Moodle. In A. Marcus-Quinn & T. Hourigan (Eds.), Handbook on Digital Learning for K-12 Schools (pp. 191–215). Springer International Publishing.

Setiadi, P. M., Alia, D., Sumardi, S., Respati, R., & Nur, L. (2021). Synchronous or asynchronous? Various online learning platforms studied in Indonesia 2015–2020. In Journal of Physics. Conference Series,1987 , Bristol: IOP Publishing.

Shdaifat, A. M., & Obeidallah, R. (2019). Quiz tool within Moodle and Blackboard mobile applications. International Journal of Interactive Mobile Technologies, 13 (8), 32–42. https://doi.org/10.3991/ijim.v13i08.10552

Shkoukani, M. (2019). Explore the major characteristics of learning management systems and their impact on e-learning success. International Journal of Advanced Computer Science and Applications, 10 (1), 296–301.

Singh, J., 2015. Moodle Statistics – Moodle now has more than 78 million users all over the world #MoodleWorld #Moodle. https://www.lmspulse.com/2015/moodle-statistics-moodle-now-has-more-than-78-million-users-all-over-the-world-moodleworld-moodle/ . Accessed 10 Oct 2020.

Slee, N. J. D., & Jacobs, M. H. (2017). Trialling the use of Google Apps together with online marking to enhance collaborative learning and provide effective feedback [version 2 peer review: 2 approved with reservations]. F1000 research, 4 , 177. doi: https://doi.org/10.12688/f1000research.6520.2 .

Smolyaninova, O., & Bezyzvestnykh, E. (2019). Implementing teachers’ training technologies at a federal university: E-portfolio, digital laboratory, PROLog Module System. International Journal of Online and Biomedical Engineering, 15 (4), 69–87. https://doi.org/10.3991/ijoe.v15i04.9288

Strang, K. D. (2015). Effectiveness of peer assessment in a professionalism course using an online workshop. Journal of Information Technology Education: Innovations in Practice, 14 (1), 1–16.

Strang, K. D. (2016). Predicting student satisfaction and outcomes in online courses using learning activity indicators. Journal of Interactive Learning Research, 27 (2), 125–152. https://doi.org/10.4018/IJWLTT.2017010103

Tlili, A., Denden, M., Essalmi, F., Jemni, M., Chang, M., Kinshuk, K., & Chen, N.-S. (2019). Automatic modeling learner’s personality using learning analytics approach in an intelligent Moodle learning platform. Interactive Learning Environments . https://doi.org/10.1080/10494820.2019.1636084

Tsai, M.-H., & Tang, Y.-C. (2017). Learning attitudes and problem-solving attitudes for blended problem-based learning. Library Hi Tech, 35 (4), 615–628. https://doi.org/10.1108/LHT-06-2017-0102

van Eck, N.J., Waltman, L., 2020. VOSviewer manual. http://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.1.pdf . Accessed 14 July 2020.

Valero, G., & Cárdenas, P. (2017). Formative and summative assessment in veterinary pathology and other courses at a Mexican veterinary college. Journal of Veterinary Medical Education, 44 (2), 331–337. https://doi.org/10.3138/jvme.1015-169R

Villa, V., Motyl, B., Paderno, D., & Baronio, G. (2018). TDEG based framework and tools for innovation in teaching technical drawing: The example of LaMoo project. Computer Applications in Engineering Education, 26 (5), 1293–1305. https://doi.org/10.1002/cae.22022

Wang, F. H. (2019). On the relationships between behaviors and achievement in technology-mediated flipped classrooms: A two-phase online behavioral PLS-SEM model. Computers and Education, 142 , 103653. https://doi.org/10.1016/j.compedu.2019.103653

Wilson, M. J., Diao, M. M., & Huang, L. (2015). “I’m not here to learn how to mark someone else’s stuff”: An investigation of an online peer-to-peer review workshop tool. Assessment and Evaluation in Higher Education, 40 (1), 15–32. https://doi.org/10.1080/02602938.2014.881980

Xiao, L. L., & Rahman, S. S. B. A. (2017). Predicting learning styles based on students’ learning behaviour using correlation analysis. Current Science (bangalore), 113 (11), 2090–2096. https://doi.org/10.18520/cs/v113/i11/2090-2096

Xin, N. S., Shibghatullah, A. S., Subaramaniam, K. A. P., & Wahab, M. H. A. (2021). A systematic review for online learning management system. Journal of Physics. Conference Series, 1874 (1), 12030. https://doi.org/10.1088/1742-6596/1874/1/012030

Zakaria, N. A., Saharudin, M. S., Yusof, R., & Abidin, Z. Z. (2019). Code pocket: Development of interactive online learning of STEM’s subject. International Journal of Recent Technology and Engineering, 8 (2), 5537–5542. https://doi.org/10.35940/ijrte.B3297.078219

Zamalia, M., & Porter, A. L. (2016). Students’ perceived understanding and competency in probability concepts in an e-learning environment: An Australian experience. Pertanika Journal of Social Science and Humanities, 24 , 73–82.

Zhao, D., Chis, A., Muntean, G., & Muntean, C. (2018). A large-scale pilot study on game-based learning and blended learning methodologies in undergraduate programming courses. Paper presented at the Proc. Int. Conf. Educ. New Learn. Technol.(EDULEARN).

Zheng, J., Xing, W., & Zhu, G. (2019). Examining sequential patterns of self- and socially shared regulation of STEM learning in a CSCL environment. Computers and Education, 136 , 34–48. https://doi.org/10.1016/j.compedu.2019.03.005

Download references

Acknowledgements

This study is funded by NBERC Teaching & Learning (T&L) Seed Funding – University of South Australia, 2020.

Author information

Authors and affiliations.

STEM, University of South Australia, University Boulevard, Mawson Lakes, Adelaide, South Australia, 5095, Australia

Sithara H. P. W. Gamage & Jennifer R. Ayres

Research & Innovations Services, University of South Australia, University Boulevard, Mawson Lakes, Adelaide, South Australia, 5095, Australia

Monica B. Behrend

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the writing of the paper. SHPWG planned the review and conducted the thematic analysis. JRA conducted the bibliometric analysis. MBB analyse the overall the content and provided pedagogical data. The names are in order of the amount of contribution given. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sithara H. P. W. Gamage .

Ethics declarations

Ethics approval and consent to participate.

N/A. This is a meta data analysis based on published literature.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Gamage, S.H.P.W., Ayres, J.R. & Behrend, M.B. A systematic review on trends in using Moodle for teaching and learning. IJ STEM Ed 9 , 9 (2022). https://doi.org/10.1186/s40594-021-00323-x

Download citation

Received : 11 July 2021

Accepted : 25 December 2021

Published : 25 January 2022

DOI : https://doi.org/10.1186/s40594-021-00323-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Learning management systems
• Thematic analysis

To read this content please select one of the options below:

Please note you do not have access to teaching notes, a systematic review of e-learning systems adoption before and during the covid-19.

Global Knowledge, Memory and Communication

ISSN : 2514-9342

Article publication date: 15 August 2022

Issue publication date: 13 February 2024

This systematic review aims to assess the studies collected by identifying factors influencing the acceptance of e-learning systems before and during the current propagation of the COVID-19 pandemic.

Design/methodology/approach

This study undertook a literature review on the in-depth revision of studies published before 2021. The reviewed research papers meet the inclusion and exclusion criteria. A total of 97 out of 214 articles met the inclusion criteria and were subsequently used in this review.

The findings revealed that the survey questionnaire is the most common data collection instrument used regardless of the research objectives. 2019 was a remarkable year because of the emergence of the COVID-19 pandemic.

Research limitations/implications

This systematic review relied on specific databases (ScienceDirect, Emerald, IEEE and Google Scholar) to search for the articles included in this paper. However, these databases may not comprehensively represent all papers published on e-learning using the technology acceptance model (TAM).

Practical implications

This paper suggests a guide for managers and scholars in educational institutions and acts as a roadmap for practitioners and academics in the educational field and policymakers. This research spotlights the significant factors influencing the acceptance and adoption of e-learning.

Originality/value

This research assessed articles that examined the TAM in e-learning and classified them according to their methodology, country of dissemination, context and distribution within the year of publication. This paper contributes to the body of knowledge in a way that will benefit stakeholders in an educational setting.

• Technology acceptance model (TAM)
• Online learning
• Higher education

Acknowledgements

The research leading to these results has received funding from The Research Council (TRC) of the Sultanate of Oman under the Block Funding Program with agreement no. TRC/BFP/ASU/01/2018.

Abdelfattah, F. , Al Mashaikhya, N.Y. , Dahleez, K.A. and El Saleh, A. (2024), "A systematic review of e-learning systems adoption before and during the COVID-19", Global Knowledge, Memory and Communication , Vol. 73 No. 3, pp. 292-311. https://doi.org/10.1108/GKMC-02-2022-0033

Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

E-learning: technologies, application and challenges

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Shaping the Future of Online Learning

Published may 22, 2024.

If you’ve been enrolled in any educational course or postsecondary educational program since 2020, chances are you’ve witnessed the rise in online learning firsthand .

The COVID-19 global pandemic shuttered storefronts, theaters, and classrooms alike, causing major disruptions in how goods and services were delivered. As consumers adopted Instacart for their grocery needs and streamed new blockbuster movies from the comfort of their living rooms, students needed an innovative way to bring their classes home. A year into the pandemic over 60% of all undergraduate students were enrolled in at least one online course , with 28% exclusively enrolled in online courses, according to the National Center for Education Statistics.

There are other reasons for the widespread adoption, including accessibility. Rural and international students who may be far removed from traditional educational institutions can now attend Harvard classes anywhere there’s an internet connection. Or, consider working adults seeking to progress or switch careers. Life doesn’t stop for a class, and attending one in-person can be prohibitive. While still challenging, logging into a virtual classroom is far more manageable. Online education is for everyone.

Technological and pedagogical developments have helped online learning progress beyond the days of discussion boards and essay uploads. Now, students can enjoy a multimedia educational experience that is rooted in the latest research, all while participating in the community of their “virtual campus”.

If you’re one of the millions of learners who have experienced online education, you might be interested to learn where it’s going next. At Harvard Online, the question, “what is the future of online learning?” guides an ongoing conversation that drives us everyday.

In this blog, we sat down with Catherine Breen , Managing Director of Harvard Online. With more than two decades of senior executive leadership at Harvard University and oversight of Harvard Online, Breen has an invaluable perspective on the future of online learning, and the exciting role Harvard Online is playing in bringing the future into the present. 

Catherine Breen, Managing Director of Harvard Online, in a team meeting.

Harvard Online (HO): How has the online learning landscape evolved in recent years? 

Catherine Breen (CB): At the beginning of the COVID-19 lockdown, there was a massive escalation in demand for online learning. Demand began to recede slowly as the months wore on and by late 2022, it started to level out. But we observed two big changes: Internally, the demand for Harvard Online content was still almost three times higher than pre-pandemic. Externally, in reaction to the demand surge, there was significant and rapid growth of new online course offerings and companies that purveyed varying types of digital products.    

HO: What is shaping the future of online learning today? 

CB: Because of the rapid and massive shift to online that occurred around the globe in the spring of 2020, the landscape changed permanently. There are many things shaping the future but here are just a few that I can see from my perspective:

• Increased adoption of online learning across all ages and levels of education: Everyone expanded their online course catalogs; new companies and offerings sprung up everywhere.
• Greater tech investment across organizations and industries: Organizations are investing more time, money, and effort into technology infrastructure, tools, and platforms to support online learning and participants in these courses.
• New pedagogical methods to bridge the gap between traditional and novel learning methods: Instructors have adapted their teaching methods for online, hybrid, and blended environments.
• Enhanced accessibility to quality education and learning experiences: Efforts have been made to improve access for students of all types, abilities, geographies, and backgrounds so that everyone can participate effectively.    

HO: What are the remaining challenges that online learning faces? 

CB: While these changes have improved the online learning experience, challenges remain, including addressing the digital divide, maximizing student engagement, and refining the quality of online courses.

The pandemic accelerated the adoption of online learning and its impact will likely continue to shape higher education for many years to come.  

HO: How does online learning contribute to Harvard's mission of promoting accessibility and inclusion in education, especially for learners who may not have traditional access to higher education?

CB: Online learning levels the playing field for learners in many ways.

Most students think that a Harvard-quality education is out of reach, for a variety of reasons. With online courses, however, learners from around the country and the world can take courses with Harvard instructors at their own pace at a more affordable price point.

Our online courses also typically incorporate a range of multimedia elements, allowing students with different learning styles to flourish. We also ensure that our online learning experiences are accessible to all learners, including those with disabilities. This commitment to inclusivity aligns with the broader goals of promoting equitable access to education.

Lastly, our online courses often include discussion forums and virtual communities where learners can connect and collaborate. This allows for interactions among students from diverse backgrounds and experiences, fostering a sense of belonging and inclusion.  

It’s clear that online learning has a lot to offer everyone, and it’s only getting better. In our next blog in this series, we’ll hear more from Cathy on how institutions can implement online learning modalities effectively. 

If you missed the first blog in this series detailing the future of online learning, you can check out the first blog here . To learn more about Harvard Online, explore our fully online course catalog here .

Related Blog Posts

Uniref brings harvard courses on web programming to syrian refugees.

Harvard Online is proud to provide access to education and experiences that help communities thrive.

Harvard Online in Your Workplace: Elevate Your Team's Professional Development

At Harvard Online we understand the value of an educated and skilled workforce.

A Decade of Innovation: Online Learning at Harvard

We are always asking, “What does the future look like for teaching and learning?”

Subscribe to the PwC Newsletter

Join the community, trending research, llava-uhd: an lmm perceiving any aspect ratio and high-resolution images.

To address the challenges, we present LLaVA-UHD, a large multimodal model that can efficiently perceive images in any aspect ratio and high resolution.

LightAutoML: AutoML Solution for a Large Financial Services Ecosystem

We present an AutoML system called LightAutoML developed for a large European financial services company and its ecosystem satisfying the set of idiosyncratic requirements that this ecosystem has for AutoML solutions.

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

Low-rank adaptation is a popular parameter-efficient fine-tuning method for large language models.

Diffusion for World Modeling: Visual Details Matter in Atari

Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a Model Of eNvironment Dreams), a reinforcement learning agent trained in a diffusion world model.

How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

Compared to both open-source and proprietary models, InternVL 1. 5 shows competitive performance, achieving state-of-the-art results in 8 of 18 benchmarks.

Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

For fine-grained language understanding, we train a Multimodal Large Language Model to refine the captions of the images.

Retrieval-Augmented Generation for AI-Generated Content: A Survey

pku-dair/rag-survey • 29 Feb 2024

We first classify RAG foundations according to how the retriever augments the generator, distilling the fundamental abstractions of the augmentation methodologies for various retrievers and generators.

Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection

idea-research/grounding-dino-1.5-api • 16 May 2024

Empirical results demonstrate the effectiveness of Grounding DINO 1. 5, with the Grounding DINO 1. 5 Pro model attaining a 54. 3 AP on the COCO detection benchmark and a 55. 7 AP on the LVIS-minival zero-shot transfer benchmark, setting new records for open-set object detection.

A decoder-only foundation model for time-series forecasting

Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset.

EasySpider: A No-Code Visual System for Crawling the Web

NaiboWang/EasySpider • ACM The Web Conference 2023

As such, web-crawling is an essential tool for both computational and non-computational scientists to conduct research.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals

Machine learning articles from across Nature Portfolio

Machine learning is the ability of a machine to improve its performance based on previous results. Machine learning methods enable computers to learn without being explicitly programmed and have multiple applications, for example, in the improvement of data mining algorithms.

AI networks reveal how flies find a mate

Artificial neural networks that model the visual system of a male fruit fly can accurately predict the insect’s behaviour in response to seeing a potential mate — paving the way for the building of more complex models of brain circuits.

• Pavan Ramdya

Predicting tumour origin with cytology-based deep learning: hype or hope?

The majority of patients with cancers of unknown primary have unfavourable outcomes when they receive empirical chemotherapy. The shift towards using precision medicine-based treatment strategies involves two options: tissue-agnostic or site-specific approaches. Here, we reflect on how cytology-based deep learning tools can be leveraged in these approaches.

• Nicholas Pavlidis

A multidimensional dataset for structure-based machine learning

MISATO, a dataset for structure-based drug discovery combines quantum mechanics property data and molecular dynamics simulations on ~20,000 protein–ligand structures, substantially extends the amount of data available to the community and holds potential for advancing work in drug discovery.

• Matthew Holcomb
• Stefano Forli

Latest Research and Reviews

Reagent-free detection of Plasmodium falciparum malaria infections in field-collected mosquitoes using mid-infrared spectroscopy and machine learning

• Emmanuel P. Mwanga
• Prisca A. Kweyamba
• Fredros O. Okumu

Intelligent prediction of Alzheimer’s disease via improved multifeature squeeze-and-excitation-dilated residual network

• Zengbei Yuan

An effective ensemble learning approach for classification of glioma grades based on novel MRI features

• Mohammed Falih Hassan
• Ahmed Naser Al-Zurfi
• Khandakar Ahmed

Deep-learning segmentation to select liver parenchyma for categorizing hepatic steatosis on multinational chest CT

• Zhongyi Zhang
• Xiangchun Liu

Development and validation of a reliable DNA copy-number-based machine learning algorithm ( CopyClust ) for breast cancer integrative cluster classification

• Cameron C. Young
• Katherine Eason
• Oscar M. Rueda

COVID-19 mortality prediction in Hungarian ICU settings implementing random forest algorithm

• Ágoston Hamar
• Daryan Mohammed
• Katalin Gombos

News and Comment

Audio long read: How does ChatGPT ‘think’? Psychology and neuroscience crack open AI large language models

To understand the 'brains' of LLMs, researchers are attempting to reverse-engineering artificial intelligence systems.

• Matthew Hutson
• Benjamin Thompson

Superstar porous materials get salty thanks to computer simulations

Model predicts the structure of previously elusive compounds with practical applications.

• Ariana Remmel

AlphaFold3 — why did Nature publish it without its code?

Criticism of our decision to publish AlphaFold3 raises important questions. We welcome readers’ views.

China’s ChatGPT: why China is building its own AI chatbots

ChatGLM is one of hundreds of AI language models being developed for the Chinese language. It comes close to ChatGPT on many measures, say its creators.

• Celeste Biever

Generalization—a key challenge for responsible AI in patient-facing clinical applications

Generalization – the ability of AI systems to apply and/or extrapolate their knowledge to new data which might differ from the original training data – is a major challenge for the effective and responsible implementation of human-centric AI applications. Current debate in bioethics proposes selective prediction as a solution. Here we explore data-based reasons for generalization challenges and look at how selective predictions might be implemented technically, focusing on clinical AI applications in real-world healthcare settings.

• Nabeel Seedat
• Mihaela van der Schaar

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

deep learning Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

Synergic Deep Learning for Smart Health Diagnosis of COVID-19 for Connected Living and Smart Cities

COVID-19 pandemic has led to a significant loss of global deaths, economical status, and so on. To prevent and control COVID-19, a range of smart, complex, spatially heterogeneous, control solutions, and strategies have been conducted. Earlier classification of 2019 novel coronavirus disease (COVID-19) is needed to cure and control the disease. It results in a requirement of secondary diagnosis models, since no precise automated toolkits exist. The latest finding attained using radiological imaging techniques highlighted that the images hold noticeable details regarding the COVID-19 virus. The application of recent artificial intelligence (AI) and deep learning (DL) approaches integrated to radiological images finds useful to accurately detect the disease. This article introduces a new synergic deep learning (SDL)-based smart health diagnosis of COVID-19 using Chest X-Ray Images. The SDL makes use of dual deep convolutional neural networks (DCNNs) and involves a mutual learning process from one another. Particularly, the representation of images learned by both DCNNs is provided as the input of a synergic network, which has a fully connected structure and predicts whether the pair of input images come under the identical class. Besides, the proposed SDL model involves a fuzzy bilateral filtering (FBF) model to pre-process the input image. The integration of FBL and SDL resulted in the effective classification of COVID-19. To investigate the classifier outcome of the SDL model, a detailed set of simulations takes place and ensures the effective performance of the FBF-SDL model over the compared methods.

A deep learning approach for remote heart rate estimation

Weakly supervised spatial deep learning for earth image segmentation based on imperfect polyline labels.

In recent years, deep learning has achieved tremendous success in image segmentation for computer vision applications. The performance of these models heavily relies on the availability of large-scale high-quality training labels (e.g., PASCAL VOC 2012). Unfortunately, such large-scale high-quality training data are often unavailable in many real-world spatial or spatiotemporal problems in earth science and remote sensing (e.g., mapping the nationwide river streams for water resource management). Although extensive efforts have been made to reduce the reliance on labeled data (e.g., semi-supervised or unsupervised learning, few-shot learning), the complex nature of geographic data such as spatial heterogeneity still requires sufficient training labels when transferring a pre-trained model from one region to another. On the other hand, it is often much easier to collect lower-quality training labels with imperfect alignment with earth imagery pixels (e.g., through interpreting coarse imagery by non-expert volunteers). However, directly training a deep neural network on imperfect labels with geometric annotation errors could significantly impact model performance. Existing research that overcomes imperfect training labels either focuses on errors in label class semantics or characterizes label location errors at the pixel level. These methods do not fully incorporate the geometric properties of label location errors in the vector representation. To fill the gap, this article proposes a weakly supervised learning framework to simultaneously update deep learning model parameters and infer hidden true vector label locations. Specifically, we model label location errors in the vector representation to partially reserve geometric properties (e.g., spatial contiguity within line segments). Evaluations on real-world datasets in the National Hydrography Dataset (NHD) refinement application illustrate that the proposed framework outperforms baseline methods in classification accuracy.

Prediction of Failure Categories in Plastic Extrusion Process with Deep Learning

Hyperparameters tuning of faster r-cnn deep learning transfer for persistent object detection in radar images, a comparative study of automated legal text classification using random forests and deep learning, a semi-supervised deep learning approach for vessel trajectory classification based on ais data, an improved approach towards more robust deep learning models for chemical kinetics, power system transient security assessment based on deep learning considering partial observability, a multi-attention collaborative deep learning approach for blood pressure prediction.

We develop a deep learning model based on Long Short-term Memory (LSTM) to predict blood pressure based on a unique data set collected from physical examination centers capturing comprehensive multi-year physical examination and lab results. In the Multi-attention Collaborative Deep Learning model (MAC-LSTM) we developed for this type of data, we incorporate three types of attention to generate more explainable and accurate results. In addition, we leverage information from similar users to enhance the predictive power of the model due to the challenges with short examination history. Our model significantly reduces predictive errors compared to several state-of-the-art baseline models. Experimental results not only demonstrate our model’s superiority but also provide us with new insights about factors influencing blood pressure. Our data is collected in a natural setting instead of a setting designed specifically to study blood pressure, and the physical examination items used to predict blood pressure are common items included in regular physical examinations for all the users. Therefore, our blood pressure prediction results can be easily used in an alert system for patients and doctors to plan prevention or intervention. The same approach can be used to predict other health-related indexes such as BMI.

Export Citation Format

Share document.