mobile games literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PeerJ Comput Sci

Mobile-assisted and gamification-based language learning: a systematic literature review

Kashif ishaq.

1 Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Malaysia

Nor Azan Mat Zin

Fadhilah rosdi, muhammad jehanghir.

2 Institute of Education and Research, University of the Punjab, Lahore, Punjab, Pakistan

Samia Ishaq

3 School Education Department, Sheikhupura, Pakistan

4 Department of Computer Science, University of Management & Technology, Lahore, Lahore, Pakistan

Associated Data

The following information was supplied regarding data availability:

This is a Systematic Literature Review and does not depend on raw data or code.

Learning a new language is a challenging task. In many countries, students are encouraged to learn an international language at school level. In particular, English is the most widely used international language and is being taught at the school level in many countries. The ubiquity and accessibility of smartphones combined with the recent developments in mobile application and gamification in teaching and training have paved the way for experimenting with language learning using mobile phones. This article presents a systematic literature review of the published research work in mobile-assisted language learning. To this end, more than 60 relevant primary studies which have been published in well-reputed venues have been selected for further analysis. The detailed analysis reveals that researchers developed many different simple and gamified mobile applications for learning languages based on various theories, frameworks, and advanced tools. Furthermore, the study also analyses how different applications have been evaluated and tested at different educational levels using different experimental settings while incorporating a variety of evaluation measures. Lastly, a taxonomy has been proposed for the research work in mobile-assisted language learning, which is followed by promising future research challenges in this domain.

Introduction

Mobile technology developments are quickly expanding the field of learning in non-formal education areas by rendering universal and instance-oriented access to privileged digital resources ( Cheon et al., 2012 ). Mobile learning (m-learning) technology requires mobile devices to improve learning and academic performance by having the opportunity to learn remotely at all times in compliance with students' comforts. There have been many advantages to m-learning, including cost reductions, ubiquitous communication, research assistance, and location-based services. The goal of m-learning is to put the educational sector and associations at the center of academic progress to satisfy the users' demand for flexibility and ubiquity ( Ishaq et al., 2020d ). Mobile devices are mostly used in developed countries, particularly for language learning purposes. Specifically, Mobile-Assisted Language Learning (MALL) implies mobile phones in the learning and teaching of languages. The mobile phone allows pupils to learn quickly to develop their language comprehension skills. Besides, a significant improvement in pedagogical methods was brought about by integrating smartphone apps and games with the curriculum, thus enabling the students to learn freely in time, space and motivation on an individual basis ( Ishaq et al., 2019 ).

A significant trend in mobile learning apps development involves gamification concepts that incorporates play and fun elements to inspire and attract the learner, generally referred to as serious games. A serious game’s key objective is to accomplish a learning objective in a fun mode, whereby the locus of control is with the learner ( Sandberg, Maris & Geus, 2011 ). Currently, schooling is not limited to a single life stage and not exclusively in traditional education institutions. Children should not only study at school but also informally outside of school. Their casual reactions outside of the classroom provide an almost as valuable learning experience as the classrooms’ organized learning environment. The integration of multimedia learning content enables learners to access appropriate information within and outside the school ( Sandberg, Maris & Geus, 2011 ).

The previous review studies in the MALL domain mostly concentrated on technology-based learning and handheld devices, while less on the research frameworks, content, learning, and teaching resources, as shown in Table 1 . This table compares current measures based on five essential viewpoints: targeted digital repositories, teaching and learning methods, quality assessment evaluation, research framework, and learning material. Only quality articles were reviewed published in quality journals (except workshops and seminars) and performed quality assessments by discussing research frameworks, content, and teaching and learning tools.

This article was structured as follows: “Literature Review” presented a review of relevant literature. “Research Methodology” presented the methodology adopted to perform this study, along with questions and objectives, whereas “Assessment and Discussion of Research Questions” identified and summarized answers to specific study questions. “Discussion and Future Directions” presented a blend of the discussed research by defining its taxonomy, while “Conclusion” concluded this article.

Literature review

Most of the surveys and systematically reviewed on MALL do not cover publication channels (Books or Scientific Journals), quality assessment, frameworks/model used, mobile and gamified applications used for teaching and learning, and comparison of these applications. Also, the focus was more on higher education students than on primary education. A more recent systematic review on the usage of mobile devices for language learning evaluated limited studies and from restricted repositories ( Mahdi, 2018 ) ( Cho et al., 2018 ). The author reviewed 20 studies for mobile devices’ effect on students’ achievement in which student’s vocabulary learning results using handheld devices were compared to those using conventional learning ( Cho et al., 2018 ).

In another study, the effects of embedded portable devices in learning and teaching were examined by reviewing 110 experimental and quasi-experimental journal papers published in 1993–2013. For the usage of mobile devices in school, a moderate mean efficiency was 0.523. Sung, Chang & Liu (2016) analyzed the impact size of moderators and the benefits and drawbacks of mobile learning, then synthesized based on the descriptive analysis from individual experiments at various levels of moderator variables. Another SLR of researches from 2007 to the present was on mobile education in K–12 in which ( Liu et al., 2014 ) reviewed a maximum of 63 articles from 15 journals, mostly exploratory and concentrated on the educational facilities associated with smartphone usage in learning. Furthermore, patterns and critical problems are also discussed for future research. Burston (2013) summarized 345 MALL research studies from 1994 to 2012, in a short overview of 80 words, to encourage researchers by presenting their historical background. The analysis included the home country, first or second or foreign language, the technologies employed for mobile apps, targeted research areas, type of students, demographic, study time, and the outcome summary.

Finally, ( Hwang & Tsai, 2011 ) targeted influential journals from 2001 to 2010 to investigate mobile and ubiquitous learning researches related to enhanced learning technology. The publications included Educational Technology & Society, Innovations in Education and Teaching International, Journal of Computer Assisted Learning, Computers and Education, Educational Technology Research & Development, and British Education Technology Journal. The researcher presented information for many journals, a selected search sample (primary school, secondary school, tertiary education, instructors and employees), study fields (language and arts, engineering, science, math, social sciences, and more), and countries involved. However, none focused on quality assessment, framework/models, grade or education level, adopted content, approaches, statistical analyses, and comparisons between MALL and gamified apps. In our review, both areas were thoroughly discussed and differentiated from the above studies then systematically chose methods and coded them with standard naming, according to strict guidelines.

This Systematic Literature Review (SLR) discusses MALL—learners in-depth, mobile, and game-based languages, and involves the five perspectives as in Table 1 . Based on the structural analysis criteria, 67 research articles have been finalized and assessed in quantitative and qualitative terms for further analysis. The SLR’s significance presents the new classification criteria, MALL research trends, developed/adopted research models, learning and teaching methods, learning content, comparisons of mobile and game-based apps, research methodologies, and approaches used to evaluate the studies. This SLR may allow instructors to build a standardized MALL environment with learning and teaching apps, learning materials, frameworks, and relevant methodologies.

The study of Sung, Chang & Liu (2016) chose experimental and quasi-experimental journal articles from the period 1993–2013 from ERIC and SSCI repositories (only eight journals were selected) to know the use of mobile experimental studies of teaching and learning achievement of students through these devices. The study Hwang & Tsai (2011) published in 2011, in which journal articles were selected from SSCI repository of the 2001–2010 period to know the status of mobile and ubiquitous learning, as well as research sample group along with learning domains related to technology, were adopted in selected articles. In the study of Liu et al. (2014) , the selected articles were taken from 2007 to 2012 for teaching and learning of K-12 education. The study aimed to know the effectiveness and trends of mobile devices in K-12 education, and the participants were younger than 18 years.

The study of Burston (2013) selected the studies from the period of 1994 to 2012 in the area of MALL and annotation in which only 80 words summary was provided consisted of country, native language, mobile technology used, learning area, type of learners, and numbers, and an overview of results. The study Mahdi (2018) examined the effect of vocabulary learning using mobile devices by selecting 16 studies from ERIC, IEEE Xplore, IGI, Sage, ScienceDirect and Springer. The study Cho et al. (2018) finalized 20 studies to see the effect of mobile devices in language learning on students’ achievement, and ERIC, EBSCOhost, JSTOR and ProQuest repositories were filtered for the literature search. This systematic literature review aims to explore the Web of Science core collection for high-quality literature search to know the research trends of Mobile and Gamification based language learning. Moreover, a quality assessment of the selected articles was conducted along with the discussion of research frameworks/models and teaching and learning tools at all education levels.

Research methodology

This survey implemented recommendations for systematic reviews given in information engineering analysis by ( Brereton et al., 2007 ). Based on these criteria, a search method was defined to eliminate possible study biases after finalizing research queries. Within this procedure, three critical phases of our analysis approach were to prepare, conduct, and review the study, as shown in Fig. 1 and discussed in the following sections.

Review plan

An appropriate search strategy was created for all related studies. As shown in Figs. 1 and ​ and2, 2 , the analysis methods indicate search procedures for the associated articles, describing the classification system and mapping of items. This study follows an organized approach:

• Research objectives
• Specifying research questions (RQs)
• Organizing searches of databases
• Studies selection
• Screening relevant studies
• Data extraction
• Results synthesizing
• Finalizing the review report

(i) RQ1 attempted to report our objective to develop an articles’ library related to the MALL pupils and make the dataset accessible to other scholars. Furthermore, significant work was identified that provided direction to investigate students’ issues in learning English. The answer to RQ1 discovered trends, geographical areas, and publication channels in the articles.

(ii) To identify the theories and frameworks/conceptual models used for MALL research and relate them with one another and to different application areas for MALL. The solution of RQ2 provided the answer to this objective.

(iii) RQ3 attempted to identify different target application areas of MALL from teaching and learning perspectives. Furthermore, other modes of exposition were identified for these MALL applications.

(iv) How the researchers accommodated different MALL applications content focusing on reading, writing, speaking, and listening perspectives. For this purpose, RQ4 attempted to achieve this objective.

(v) RQ5 attempted to outline the standard process, tools, and instruments used to evaluate MALL applications. Furthermore, MALL applications’ evaluation measures were identified concerning different perspectives, including teaching and learning, and technical perspectives.

(vi) RQ6 attempted to perform a comparative analysis to evaluate the effectiveness of simple mobile application-based language learning with gamified mobile applications for language learning.

The research objectives and motivations of this research are transformed into relevant research questions (RQ), as shown in Table 2 .

Review conduct

There were four steps in the review process formulated. In the first step, examination was made from web of science ( WoS, 2020 ) with SCI-Expended, SCIE, ESCI and A&HCI indices consisting of high-impact research papers, for relevant primary studies. In the second step, the collection of studies was filtered based on predefined inclusion/exclusion criteria. We also established quality assessment standards to boost further the consistency in the third step of our analysis. Backward snowballing was then carried out in the final fourth step, to retrieve relevant candidate articles.

Automated search in web of science (WoS core collection)

A systematic investigation was made to filter irrelevant research and obtain adequate information. Our source was a curated database, Web of Science Core library that included over 21,100 peer-reviewed journals, top-class academic journals distributed worldwide (including Open Access journals), covering more than 250 disciplines ( WoS, 2020 ). WoS is a tool that helps users collect, interpret, and share information from databases promptly ( Princeton University, 2020 ). To conduct an SLR in an organized and timely manner, the researcher used this platform to retrieve the research articles by incorporating ‘AND’ and ‘OR’ Boolean operators with keywords to develop a search string. Figure 3 presented an overview of the search result obtained from the Web of Science.

Table 3 lists the final search string incorporated ‘AND’ and ‘OR’ Boolean operators with keywords, used to explore WoS Core Collection. Only titles were searched from the database, and a filter of indices and period were applied to restrict the search query for the study.

Selection based on inclusion/exclusion criteria

• 1. Inclusion criteria

The paper comprised in the review must be in MALL, mobile learning, and m-learning that must target the research questions. Paper published in the journals or conferences also from 2010 to 2020 was included in the review. Papers discussing MALL at school, college, and university level, focusing on learning, teaching, learning, and teaching (both), games (Example: Serious Game, Mobile games, learning application) were also included in the review.

• 2. Exclusion criteria

The articles were excluded not written in English and did not discuss or focused MALL, mobile learning or m-learning in schools, colleges, and universities to teach and learn English. A selection process of relevant articles for inclusion/exclusion criteria in detail was shown in Fig. 4 .

Selection based on quality assessment

The collection of appropriate studies based on quality assessment (QA) was considered the critical step for carrying out any review. As the fundamental studies differed in nature, the essential assessment tool ( Fernandez, Insfran & Abrahão, 2011 ) and ( Ouhbi et al., 2015 ) used to conduct QA were also supplemented in our analysis by quantitative, qualitative, and mixed approaches. The accuracy of the selected records was determined using a QA questionnaire. The first author conducted the QA, using the following parameters for each study:

(a) If the analysis led to MALL, mobile language learning, m-learning, the result was indeed (1), otherwise (0).

(b) When there were simple answers for the MALL, mobile learning, m-learning of English, the analysis provided the following scores: ‘Yes (2),’ ‘Limited (1)’ and ‘No (0).’

(c) If the studies provided an empirical result, then award (1) else score (0).

(d) Studies analyzed concerning graded rankings of countries and journals ( SCImago Journal & Country Rank (SJR), 2018 ) and conferences in computer science ( CORE Conference Portal, 2018 ). Table 4 indicated potential findings for publications from known and reliable sources.

After combining the scores to the above questions, a final score (between 0 and 8) was determined for each study. Only papers included with four or more ratings in the final results.

Selection based on snowballing

Following the standard assessment, backward snowballing was performed, employing a reference list from any completed analysis to retrieve papers ( Mehmood et al., 2020 ) and chose only those significant articles that met inclusion/exclusion requirements. After reading the introduction and then other portions of the document, the article's inclusion/exclusion was determined.

Review report

This section provided an overview of the selected studies.

Overview of intermediate selection process outcome

MALL was a very active topic, and our analysis approach had to extract relevant research empirically and systematically from the Web of Science core collection. The next step of our systematic analysis was compiling records that form the foundation for this analysis. Approximately 57,000 papers were examined from the archive by providing the keywords for 2010– 2020. After creating a knowledge base from the digital library (Web of Science), the author reviewed the title, abstract, and accompanying complete document for each search result, as needed. During this process, irrelevant papers or papers of less than four pages were eliminated. During the inspection process, selected documents in the fields of MALL, mobile learning, and m-learning were read extensively to assess their significance & contribution and then created a comprehensive knowledge foundation of papers based on their findings to accomplish this research's core objective.

Overview of selected studies

Table 5 presented significant results of the primary search, filtering, and reviewed processes that included Web of Science indices. At the filtering/inspection stage, this amount decreased to 63 articles by the automatic search.

Assessment and discussion of research questions

In this section, finalized 67 primary research studies based on our research questions were scrutinized.

RQ1: What are the high-quality publication channels for MALL research, and which geographical areas have been targeting MALL research over the years?

The analysis of MALL with the integration of game elements in learning tools, methods, content, and the theoretical perspective choice was a crucial challenge for scholars to use in education. Identifying fine publication sites and scientometric analysis based on meta-information in MALL domain was required for the purpose. This section consisted of perceptive knowledge of research publication sources, types, years, grade level distribution, geographical distribution, publication channel-wise distribution of selected studies to analyze MALL research.

The studies finalized from the Web of Science (core collection) were presented yearly, as shown in Table 6 and Fig. 5 . Twelve was the maximum number of publications selected from the year 2019, out of the total 67, indicated more interest in developing MALL with integrating games in teaching and learning. However, less interest in MALL with game research integration was observed in 2010–2016 and 2018–2020, resulting in less improvement in teaching and learning in relevance to students and market needs.

Table 7 and Fig. 6 present the geographically distributed studies. The majority of publications, or 39 out of 67, were from different Asian countries, whereas European countries published 15 reviews. North American countries published eight studies, while Africa and Ocean-continent have three and two studies published, respectively.

The data presented in Table 8 showing that the maximum number of articles were from highly recognized journals indexed in the Web of Science. Only one item was from a good ranked conference. Computer & Education journal was on the top of the list from which seven papers were selected and next, the Educational Technology & Society journal with four articles.

According to conditions defined in “Research Methodology” B.3, each finalized study’s QA score was granted, as shown in Table 4 , offering the QA score ranging from 4–8, with less than four discarded scores. MALL researcher might find this QA supportive to choose related studies while addressing its usage and challenges. Articles published in Q1 journals mostly scored maximum while scoring four from less recognized journals but relevant to the subject matter. A total of 26 out of 67 scored maximum (i.e., eight (8), indicating fulfillment of all QA criteria, whereas nine (9) studies scored four out of 67, which is the lowest in QA).

Table 9 presented the overall classification output and QA of finalized studies, and Table 10 showed the overall quality assessment score. Studies were classified based on five factors: the empirical type/method, research type, and method. Categories of research types were; Evaluation framework, Evaluation research, Solution proposal, and Review. The taxonomy presented in section V was constructed on these defined research types. Studies that analytically validated their results from the statistical analysis, experimentations, surveys, or case studies to increase their quality standards awarded a score. In category (c) of quality assessment criteria, only 8 out of 67 reviews did not present empirical results, thus awarded zero scores. Only five studies scored zero for category (d) of quality assessment criteria; the rest of them scored higher, indicating competent sources. Four (4) recorded the lowest score awarded for a study.

RQ2. What are the widely used theories, models, and frameworks proposed or adopted for MALL research?

This section provided the framework/conceptual model based on the situation (proposed/adopted by the researcher) after an essential discussion of the theories, frameworks, and models.

Theory of planned behavior: The Theory of planned behavior (TPB) noted that behavioral intentions motivated human behavior. Behavioral intentions rely on three determinants; an individual's mood, cultural norms, and perceived influence ( Cheon et al., 2012 ).

Sociocultural theory: Sociocultural theory recognized human evolution as socially influenced by cooperation with more professional members of society, through which children learned their cultural norms, ideologies, and problem-solving techniques ( Kearney et al., 2012 ) ( Mcleod, 2020 ).

Experiential Learning Theory: The concept of experiential learning was implemented in a learning game. Players performed exercises to seek information within a gameplay environment that made the learning process enjoyable, engaging, and interesting ( Tsai et al., 2016 ).

Dual-Coding theory: The dual-coding theory suggested that the verbal and imagery systems (mental images and representations) could be co-activated as rich and substantive referential relations connected dual-coded objects ( Teng, 2018 ).

Theory of cognitive style: Two types of learning, field independence and field dependence, were based on critical and interpersonal personality characteristics. Field-independent students were independent-minded, and the social environment could not readily affect their learning style. Field-dependent students tended to learn in visual settings or scenarios ( Ou-Yang & Wu, 2016 ).

Attribution theory: Bouchaib ( Bouchaib, Ahmadou & Abdelkader, 2018 ) used Weiner attribution theory (1992) as a framework that provided failure or success reasons or explanations for people in education. Moreover, it was also a social cognitive theory of motivation.

Instructional theory: The instructional theory sought to explain how people can learn, evolve, and established environments that promote learning opportunities and strengthen teaching ( Fisser, Voogt & Bom, 2012 ) ( Top Hat, 2019 ).

Frameworks & models

‘Here and now’ mobile learning framework: Three characteristics (Engaging, Authentic, and Informal) Framework identified the effect of mobile learning on the learning environment ( Martin & Ertzberger, 2013 ).

Mobile English learning outcome (MELO) framework: This research developed the Mobile English Learning Objective (MELO) to examine the correlation between learners’ understanding of mobile technologies using playfulness, resistance to change, and self-management constructs of the Framework ( Huang et al., 2011 ).

M-COPE: It is a study that came up with the M-COPE framework to support teachers in building mobile learning experiences using the ADDIE model, which allowed teachers to understand five main aspects of mobile learning: Mobile affordance, Ethics, Pedagogy, and Outcomes ( Dennen & Hao, 2014 ).

Mobile English learning continuance intention (MELCI): The study came up with TAM components ( Davis, 1989 ) adopted in mobile English learning. Continuance intention framework to examine the influence of mobile learning satisfaction with self-management of learning ( Huang, 2014 ).

Pedagogical Framework: Task-based language teaching (TBLT) was a pedagogical framework concerning language instruction strategy that focused on curricular design units and an instructional cycle aspect. It focused on the interaction method, which implied that the negotiation of importance led to a detailed and internationally changed input ( Chen & Lin, 2018 ).

The affordance framework: The affordance framework, which comprised pedagogical, social, and technological components, examined the ICT tools affordance for language learning ( Ramadoss & Wang, 2012 ).

Technology Acceptance Model (TAM): The TAM demonstrated many knowledge predictors by many researchers, focusing on the rational action theory. TAM has five components: perceived ease of usage, perceived effectiveness, the intention of behavior, intention to use, and discrete use. The model played an essential role in forecasting the use of various factors ( Chang et al., 2013 ).

Extended Technology Acceptance Model: The study examined mobile applications' impact on students' usage intention, perceived ease of use (PEOU), and usefulness. The research outcome was an extended technology acceptance model ( Bohm & Constantine, 2016 ).

ARCS Model: A motivational model ARCS (Attention, Relevance, Confidence, and Satisfaction) that motivated learners and procedural assistance in compliance to demand used in this research. This model has three features: (1) emphasized motivation and emotional stimulation, (2) integrated with other theories and design, (3) enhanced the instruction effects and learning process ( Wu, 2018 ).

A Theoretical Model: Discussion of two frameworks: (1) Framework for the rational analysis of mobile education (FRAME). (2) TAM and TAM 2, the author proposed a new theoretical model that focused on the technology enhancement, non-formal learning setting, and learner-centered to develop a better understanding for EFL learners using mobile English learning resources (MELR) for Chinese postgraduate students ( Zhang & Pérez-Paredes, 2019 ).

In this section, the Theory, Framework, and Model used in the studies were described and summarized in Table 11 .

A conceptual model based on the Theory of Planned Behavior (TPB) explained how young adults’ acceptance and attitude affect their intention to embrace mobile devices in their classroom practice ( Cheon et al., 2012 ). The Framework in ( Kearney et al., 2012 ) criticized pedagogy, allowed a comparison of mobile approaches and pedagogical methods, and their importance to the sociocultural nature of learning in mobile learning environments validated by m-learning researchers’ discussion among designers of the Framework, implementation in a project, and pedagogical experts. Similarly, three characteristics (Engaging, Authentic, and Informal) Framework were created to see mobile learning's effect on the learning environment by measuring students’ achievement and attitude ( Martin & Ertzberger, 2013 ). A hypothesized m-learning model was proposed based on the Technology Acceptance Model (TAM) components—Perceived Usefulness (Near-term/Long-term usefulness), Perceived Ease of use, Personal Innovativeness, and Behavioral Intention. This model was assessed based on data collection from 230 participants using a survey questionnaire ( Liu, Li & Carlsson, 2010 ). A paradigm for the Mobile English Learning Objective (MELO), in which learners' understanding of mobile technologies may be directly correlated with three objects: Playfulness, Resistance to change, and Self-management that were adapted after discussing extensive literature ( Huang et al., 2011 ).

The M-COPE framework supported teachers to build mobile learning experiences using the ADDIE model. This Framework allowed teachers to understand five main aspects of mobile learning: mobile affordance, Ethics, Pedagogy, and Outcomes. It was validated by five experts ( Dennen & Hao, 2014 ). The computational model concept and their proposed connections resulting from Davis (1989) TAM were introduced in which perceived ease of use, perceived usefulness, and perceived contextual variables were measured. This model was validated through SmartPLS 2.0 ( Bohm & Constantine, 2016 ). A motivational model, ARCS (Attention, Relevance, Confidence, and Satisfaction) that considered learners' motivation and procedural assistance in compliance to demand, was used to measure effectiveness and learning motivation. This model has three features: (1) emphasize motivation and emotion stimulation, (2) integrated with other theories and design, and (3) enhances the instruction effects and learning process ( Wu, 2018 ). A mobile English learning continuance intention framework was used. TAM ( Davis, 1989 ) was adopted to see the influence of mobile learning English satisfaction with self-management of learning to measure Perceived Usefulness, Perceived Playfulness, and Resistance to change. The internal consistency and reliability were measured through the PLS-SEM application ( Huang, 2014 ).

The study adopted the TAM by incorporating perceived convenience and curiosity factor for mobile learning. To examine the validity of the measurement model, Confirmatory Factor Analysis (CFA) was performed with SmartPLS ( Chang et al., 2013 ). In contrast, experiential learning theory was used because players seek knowledge by doing activities in the game environment. Moreover, the learning process was exciting, challenging, and relevant, along with substantial experience provided to the players where learner motivation and effectiveness were measured ( Tsai et al., 2016 ). In a dual-coding theory, the working memory model and multimedia principle used for pedagogy in this study were validated by doing an extensive literature review ( Teng, 2018 ). In this research, self-directed learning, Garrison’s comprehensive theoretical approach of (1997), was used to measure self-directed learning (motivation, self-management, and self-monitoring) ( Botero, Questier & Zhu, 2018 ) whereas ‘English Fun Dubbing’ application was designed for practicing English orally, and evaluation of the application's effectiveness (convenience, flexibility, user-friendliness, rich material, language context) was carried out by Zhang (2016) . An application was developed to observe the multimedia glosses' effect on second language listening comprehension and vocabulary learning in a mobile learning environment. Its effectiveness was measured by the author ( Çakmak & Erçetin, 2017 ).

Business language testing service (BULATS) adopted a framework based on Teaching Method, Learning Satisfaction, Learning Effectiveness, and qualitative analysis to measure satisfaction and attitude ( Shih, 2017 ). ‘MyEVA Mobile’ is a mobile-based application developed to explore the learning attitude, learning achievements, learning styles, and university students' strategies to improve the students’ vocabulary that worked any time and without the internet. The framework components are ‘Smartphone client,’ ‘Wireless device,’ and ‘Log Analyzer Server.’ Learning behavior, Perpetual learning styles, and Knowledge proficiency were measured in this study ( Ou-Yang & Wu, 2016 ). In another study ( Chen & Lin, 2018 ), task-based language teaching (TBLT) was used as a pedagogical framework to measure technology-mediated TBLT. 'Weiner attribution theory (1992) was used as a framework that provided failure or success reasons or explanations for people in education ( Bouchaib, Ahmadou & Abdelkader, 2018 ). Moreover, WhatsApp and Google were used to enhance the reading ability to measure learner autonomy ( Hazaea & Alzubi, 2018 ) and retain business vocabulary, whereby an application ‘Excel@EnglishPolyU’ was developed and tested with undergraduate students ( Kohnke, Zhang & Zou, 2019 ). To design and implement a mobile-based English vocabulary application for distance learning students in South Africa, ‘VocUp’ was developed, and its evaluation measures usability, scalability, reliability, and flexibility ( Makoe & Shandu, 2018 ).

A theoretical framework for ‘Word Score’ serious game was developed based on Bom (2011) by exploring the literature to measure this study’s motivation ( Fisser, Voogt & Bom, 2012 ). In contrast, an architectural model was proposed in the study. The Pedagogical model, and learner model, were used by pilot testing ( Bourekkache & Kazar, 2020 ), whereas a mobile game for tenses’ SOS Table’ was developed. The evaluation measured its effectiveness, user's motivation, acceptance, and attitude within the Framework of mobile learning tools ( Önal, Çevik & Şenol, 2019 ). Moreover, by discussing two frameworks: (1) Framework for a rational analysis of mobile education (FRAME) and (2) TAM and TAM 2, the author proposed a new theoretical model that focuses on the technology enhancement, non-formal learning setting, and learner-centered to develop a better understanding for EFL learners on mobile English learning resources (MELR) in English language learning for Chinese postgraduate students ( Zhang & Pérez-Paredes, 2019 ). Furthermore, it was also a social cognitive theory of motivation. Simultaneously, the affordance framework consisted of pedagogical, social, and technological components, used to examine the ICT tools affordance for language learning ( Ramadoss & Wang, 2012 ). TAM was used in the study to measure perceived usefulness and ease of use ( Ishaq et al., 2020b ).

Summarize the Table 11 , researchers developed applications and games for language learning in which majority of these developed for English Language Learning and vocabulary. Similarly, less application and games were developed for pedagogy ( Kearney et al., 2012 ) ( Dennen & Hao, 2014 ) reading ( Hsu, Hwang & Chang, 2013 ) ( Lin, 2014 ), pronunciation ( Liakin, Cardoso & Liakina, 2014 ) idioms ( Wong, 2013 ) ( Amer, 2014 ) tenses ( Önal, Çevik & Şenol, 2019 ) grammar ( Chu, Wang & Wang, 2019 ; Ramadoss & Wang, 2012 ) spellings ( Tshering et al., 2018 ) achievement, attitude, usefulness, usability and effectiveness ( Martin & Ertzberger, 2013 ) ( Liu, Li & Carlsson, 2010 ) ( Ishaq et al., 2020a ; Ishaq et al., 2020b ; Ishaq et al., 2019 ; Ishaq et al., 2020d ). From the selected studies, various models/frameworks were used as a base for developing an application in which TAM ( Hsu, Hwang & Chang, 2013 ) ( Lin, 2014 ) ( Chang et al., 2013 ) ( Ishaq et al., 2020b ; Ishaq et al., 2019 ) mostly used. Many applications did not use any model or Framework as the base for the purpose.

RQ3. What are different application domains for the MALL application, and in which various forms are these applications exposed for the end-users?

Several tools have been designed and developed by the researchers in MALL to support the students and teachers in learning the English language. A more in-depth analysis presenting that most of these tools and applications are student-centric and focus on learning ( Ishaq et al., 2020b ). While some of them support teachers in teaching and students in learning, thus can be categorized as a teaching and learning tool ( Ishaq et al., 2020b ). On the other hand, very few tools focus only on teaching ( Ramadoss & Wang, 2012 ). Lastly, another emerging trend has been observed where these tools are augmented with gamification to make them more exciting and useful for the stakeholders ( Chu, Wang & Wang, 2019 ). Table 12 presents these tools in different categories above.

Learning aspect

A mobile adaptive language learning system in which recommended material for reading and reading annotation facilities was provided to pupils to improve learning outcomes ( Hsu, Hwang & Chang, 2013 ). A Mobile learning tool and five-step vocabulary learning (FSVL) strategy were developed in a situational learning environment to assess their effects on English as a foreign language (EFL) performance and learning motivation. According to the mobile learning tool, a global positioning system (GPS) was used to develop and provide learning material ( Huang et al., 2016 ). Similarly, a mobile learning application ‘Raz-Kids’ for an extensive reading program was designed by ‘Learning A to Z.’ The tool aimed to create the concentration of the students to enhance their essential reading ability. Furthermore, it provides e-books of alphabets for students and teachers’ class management tools ( Lin, 2014 ).

The study conducted experimental research using ASR, Non-ASR and Control Group. The ASR group Nuance Dragon Dictation application was designed and installed on their mobile phone to finish weekly pronunciation activities with immediately written visuals (Pictures) feedback provided by the application without any human interaction. Five native French speakers pronounced all words and phrases to test the application ( Liakin, Cardoso & Liakina, 2014 ). In a study by Chang et al. (2013) , ‘Mebook,’ a multimedia eBook system that can be played in the format of MP3 was developed. It integrated text, images, voice, and pictures that provided directions for listening, speaking, writing, and reading.

Furthermore, it also included speed adjustment of audio and switching of language from English to Chinese and Chinese to English. Duolingo, a free mobile assisted language learning tool (for vocabulary acquisition) is available in the web and mobile (Android and iOS) versions to examine out of class engagement informally. Translation exercises of sentences and words are the delivery method of this application. Moreover, instructors can see logs and progress to keep an eye on the students ( Gafni, Achituv & Rahmani, 2017 ). A mobile-based gaming approach for English grammar and vocabulary learning ‘Save the princess with Teddy’ was developed and students’ learning achievement with gameplay at different levels was analyzed. It was hoped to enhance students’ learning motivation with the game and assistance to learn the English grammar concept ( Chu, Wang & Wang, 2019 ). In the context of academic English, a mobile data-driven learning (DDL) experiment was discussed by designing and developing an application’ AKWIC (academic key words in context)’ for voluntary. This application provided support for academic writing (vocabulary learning), and data were obtained from logs generated by the application ( Quan, 2016 ).

An SMS text message application, ‘Fetion’ explores the effectiveness of vocabulary learning using mobile technology. Specifically, SMS from mobile phones enhanced English vocabulary than outdated printed material ( Zhang, Song & Burston, 2011 ). In contrast, a location-based vocabulary learning activities application for Thai and German users was proposed in a study with personalized learning motivation. Moreover, the aim was to enhance user acceptance by providing relevant learning content to evaluate an online questionnaire ( Bohm & Constantine, 2016 ). A free mobile assisted language learning tool (vocabulary acquisition) ‘Duolingo,’ available for web and mobile (Android and iOS) versions examined out of class engagement informally. Translation exercises of sentences and words are the content of this application, and instructors can see logs and progress to track students’ progress ( Botero, Questier & Zhu, 2018 ). ‘Mobile assisted listening application’ was developed to examine L2 multimedia glosses’ effects in a mobile learning environment, for vocabulary and listening comprehension, validated through a pilot study ( Çakmak & Erçetin, 2017 ).

An offline mobile assisted language learning application system, ‘My English Vocabulary Assistant mobile edition’ (MYEVA Mobile) was developed to engage students in mixed-modality vocabulary learning to improve their vocabulary. The users learned targeted words via four vocabulary learning strategies: flashcard, imagery, word card, and Chinese assonance. A log analyzer server was installed, and log files were used to analyze the results ( Ou-Yang & Wu, 2016 ). Furthermore, a mobile application like WhatsApp and Internet search engine (Google) were used by students to access materials for reading and interaction with peers and instructors outside the class. This application was used to improve learning skills in EFL reading ( Hazaea & Alzubi, 2018 ).

A crowdsourced mobile-based ‘LingoBee,’ an open-source and freely available application, explored creativity and mobility in language learning. It was the part of the European countries’ project “SIMOLA”, designed to help collaborative language learning using the idea from social networks and crowdsourcing. Users created content for language learning, stored it on their device, and shared it with other users via a cloud-based repository. The application was evaluated through user studies in European countries ( Petersen, Procter-Legg & Cacchione, 2013 ). In contrast, an application ‘English Practice’ was used to analyze the usage behavior of 53,825 users from 12 countries, and the evaluation was conducted through log files generated by the Google Firebase analytics tool ( Pham, Nguyen & Chen, 2017 ).

Teaching and learning

An online assessment tool, ‘Grammar Grabber,’ was discussed in this study to evaluate grammar at the school level. Using this tool, users do practice online and get constant feedback to know their performance. The authors investigated this online assessment tool to perceive pedagogical, technological, and social affordances using multiple-choice questions and allowing re-attempt for wrongly answered questions. The log files of the application were used as the dataset ( Ramadoss & Wang, 2012 ). An application, ‘Literacy and Numeracy Drive’ to teach and learn languages, was discussed at the school level. This application provided some practice and then some assessment exercises for the English language. This application aimed to teach singular/plural, use of has/have/had/is/am/are/ was/were, and reading comprehension ( Ishaq et al., 2020b ). A serious online game that was designed to extend the students’ vocabulary in primary school was discussed in this study. This game was part of the national project ‘Educational Time Extension (ETE).’ The students spend more time than expected class timing to learn and improve their vocabulary. Applications for teaching and learning at the college level was not adopted/developed for English language learning in the selected studies.

LINE App, an application that was used to investigate the teaching of English for specific purpose (ESP) effects on Business Language Testing Service (BULATS) ( Shih, 2017 ). Whereas ‘Mobagogy’ a professional learning community of academicians to investigate the use of mobile technologies in learning and teaching while ‘The bird in the Hand’ initiative by the UK to examine the experience of trainee and newly qualified teachers by providing them smartphones to use in their offices and teaching schools to enhance professional practices of pedagogy. Australian University developed the application Mobagogy’, and The Bird in the Hand was sponsored by the UK ( Kearney et al., 2012 ) to evaluate the benefits of mobile applications and an application 'English Fun Dubbing' designed for English’s oral practice its users. Learning material in this application was: animations, songs, movies, short videos, and textbooks developed by the Chinese Sci-Tech Company ( Zhang, 2016 ). To enhance teaching and learning of English vocabulary, a mobile-based application ‘VocUp’ was designed and implemented to evaluate external parties ( Makoe & Shandu, 2018 ). A mobile application, ‘WhatsApp’ was used in this study to examine the efficiency of teaching 40 phrases.

Furthermore, the study also aimed to enhance vocabulary acquisition ( Shahbaz & Khan, 2017 ). An application ‘Anglictina (English) TODAY’ was developed by a Ph.D. student of Computer Science with a language teacher’s help. This application helped pupils for the preparation of final exams by learning from anywhere at any time. To discuss students’ perception of using a mobile application, this was developed based on their needs to make EFL vocabulary teaching and learning useful ( Klimova & Polakova, 2020 ).

Gamification based learning and teaching

A serious game ‘MEL application’ developed in which 25 animals from five continents (Asia, South-America, Africa, Oceania, and North-America) were involved developed. The game was played during the visit to the zoo and at home in which two modes were available in which one was totally related to the zoo situation. One was independent of GPS that allowed children to access animals in different continents ( Sandberg, Maris & Geus, 2011 ). To investigate a mobile English learning application that would be supplement and support for learning English at school was adopted for the study by enhancing game features. The application’s learning material on animals from the zoo resulted in the students’ outperformance using the mobile application for a fortnight than users for a fixed period ( Sandberg, Maris & Hoogendoorn, 2014 ). A game-based system, ‘Happy English Learning System,’ in which learning material was integrated to experiment, was installed on mobile devices to see the learner's motivation for achievement. The activities were divided into levels of difficulties: Easy, Medium, and Advanced, according to the ability of users ( Tsai et al., 2016 ).

Game-based applications Excel@ EnglishPolyU, Alphabet vs. Aliens, and Books vs. Brains@PolyU, business vocabulary after completing level challenges may acquire by the learners ( Kohnke, Zhang & Zou, 2019 ). Similarly, an application, ‘PHONE Words,’ was developed by Alice English Education Studio for vocabulary learning, designed with game functions (MEVLA-GF) and without game functions (MEVLA-NGF) to investigate the perceptions and measure the learning performance of learners. This application also provides an assessment with a gamified competition mechanism ( Chen, Liu & Huang, 2019 ). To give the students’ opportunity, ‘SOS Table,’ a mobile-based game application, was designed and developed to repeat tenses, high-frequency words, and sentence string. Time challenge was provided to create negative, positive, or correct sentences with words and pronunciation after completing the task ( Önal, Çevik & Şenol, 2019 ). The serious game was not adopted/developed at the college level for English language learning in the selected studies.

In the study ( Holden & Sykes, 2011 ), an augmented reality-based language learning game developed for Spanish in Southwestern US to explore foreign and second language learning’s benefits and complexities. Similarly, a mobile-based English Vocabulary system for practice was designed and developed not to replace traditional classroom teaching methods. The aim was to provide students with assistance to review, proficiency, and practice in and after the class. The game system was designed based on the ARCS model, which provided assistance in vocabulary learning and listening and worked as a testing tool ( Wu, 2018 ). ‘Idiomobile’ a mobile-based game that was made available for specific handsets based on the knowledge of using idiomatic expressions in a critical situation ( Amer, 2014 ).

RQ 4. What is the specific content adopted for teaching and learning in MALL research?

Teaching and learning through MALL by applying pedagogical skills or tools after adopting appropriate content are summarized and discussed in this section. The current section is divided into four sub-levels, i.e., Learning, Teaching and Learning, and Serious Game based presented in Table 13 .

The content usage for learning aspect in MALL about reading perspective discussed for the undergraduate students of public university enrolled in “Computing and Information Technology (CIT)” adopted as curriculum ( Cheon et al., 2012 ). Whilst personalized recommended learning material with reading annotation services were categorized into three difficulty levels: elementary, intermediated, and intermediated high level ( Hsu, Hwang & Chang, 2013 ). In the study ( Bohm & Constantine, 2016 ), discussed that multimedia learning material in the written and audio-visual form provided using the app according to the user’s relevant current position. Words and pictures were adequate for learners to learn vocabulary by solving word pair quizzes. The content adopted in ( Chang et al., 2013 ) was broad and varied, including current issues related to education, economy, environment, social, politics, technology, etc. For English learning, the topic areas covered were speaking, listening, reading, and vocabulary. To teach most popular idioms, content relevant to idioms and collocations was collected for application from books and websites ( Amer, 2014 ).

A cooperative creation of multimedia content by adding it to new vocabulary is then stored on the devices and uploaded on the cloud by the users that can view or brows LingoBee repository by sorting it chronologically or alphabetically ( Petersen, Procter-Legg & Cacchione, 2013 ). The study ( Tsai et al., 2016 ) discussed the curriculum for English subject consisted of the topics: phonetic symbols, alphabet, grammar, phrases, vocabulary, and sentence patterns used by pupils to review it and also can share with the class by using social media (i.e., Facebook) whereas the course content in application 'Duolingo' focused variety of subject-specific vocabulary acquisition by learning nearly 2,000 words. The application's delivery techniques were translating from source to users’ target language, choosing right translated phrases, paring of words from languages, flashcard practicing, and translation of unknown words ( Botero, Questier & Zhu, 2018 ).

Teaching is a method of discussing and interfering with people’s desires, perspectives, and emotions to learn specific things ( P. byinfed.org, 2020 ). The reading perspective in teaching is discussed by Ou-Yang & Wu (2016) in which the content used for vocabulary teaching, suitable for 'Test of English for International Communication (TOEIC) that is a standardized test used in Taiwan. Fifty words were taken for vocabulary was chosen from TOEIC, including different difficulty levels.

Current studies investigated the content usage according to speaking, listening, reading, and writing in teaching and learning through MALL. An application designed by the Chinese Sci-Tech company to assist in English practice orally provided rich learning material of animation movies, short videos, songs, and textbooks with English speakers speaking for different age group students ( Zhang, 2016 ). From the reading perspective, ‘Word Capsules (short vocabulary test)’ was developed in which words (content) were selected from 10,000 words, who gripped numerous phrases in English have a vast vocabulary and able to survive with challenges of studying at a higher level in English ( Makoe & Shandu, 2018 ). In contrast, for all the viewpoints, practicing and retaining new phrases and words, ten lessons of vocabulary and ten lessons of phrases were adopted as content that the pupils have to translate to English from native language ( Klimova & Polakova, 2020 ).

The content used in game-based applications for reading perspective investigated from selected studies in which the content was pictures and videos to learn about the zoo animals ( Sandberg, Maris & Geus, 2011 ). Learning Spanish Augmented reality game-based language learning tool was also investigated in which classroom curriculum was adopted for the game for three to four-week timeframe ( Holden & Sykes, 2011 ). For vocabulary practice through a game-based system, a presentation model combined learning content with difficulty levels (easy, medium, and advance) ( Wu, 2018 ). Still, for vocabulary learning using MEVLA-GF, the perception of interviewees identified gamified design, interface design, and content design that satisfy the learners' need for different purposes and at level ( Chen, Liu & Huang, 2019 ). In the study of Chu, Wang & Wang (2019) , learning material consisted of vocabulary, sentence examples, grammar examples, and grammar concept mapping for English games in elementary school. The purpose was to make a strong foundation of English grammar and enhance its knowledge with the understandability of relationships between concepts.

RQ 5. How and in what different perspectives the MALL applications were evaluated; and what were the evaluation measures and tools used for their evaluation?

RQ5 was posed to investigate the evaluation methodologies and tools used by the MALL studies are presented in Tables 14 and ​ and15. 15 . Table 14 shows that 33 out of 67 studies adopted or developed a questionnaire to collect data from the selected population. In contrast, only six studies used interviews and observations for their studies. The mixed-method approach in which questionnaires, interviews, and observations were used as a tool for collecting data by 18 studies while only five studies used discussion and other methods for their data collection.

The evaluation in this section was teaching and learning and technical perspective. Researchers used various methods to collect data, and statistical tests were applied to analyze the data to produce relevant results. The statistical tests for analysis applied through various tools were, i.e., SPSS, PLS, etc. The methodology and tools used by the selected studies were presented in the following section.

Measures for teaching and learning

In the teaching and learning section, terminologies were described from the selected studies also presented the instruments used in these studies were shown in Table 15 .

Pedagogy: a study of educational techniques, including the purposes of instruction and the approaches to accomplish them ( Peel, 2017 ).

Motivation: originates from the term ‘motive’ that means needs, wants, wishes, or drives in people. It is the method of inspiring individuals to take steps to meet the targets ( MSG Management Study Guide, 2020 ).

Perception: To organize, define, and interpret sensory input to reflect and recognize the input or situation presented ( Perception, 2020 ).

Curiosity: originated from the flow concept, which means people prefer to communicate in the state of flow with their surroundings. Curiosity is preserved when people consider the world as fun or fascinating ( Chang et al., 2013 ).

Continuance Intention: The level to which people intend to keep using smartphone English learning ( Chang et al., 2013 ).

Attitude: An optimistic, pessimistic, or mixed assessment of an entity communicated at a certain level of anxiety. This represents an advantageous or disadvantageous assessment of a person, position, object, or event ( iEduNote.com, 2020 ).

Achievement: Somebody succeeded in achieving something, particularly after much effort ( Collins English Dictionary, 2020 ).

Self-Directed Learning: A learning approach that encourages learners to take control of their learning method (diagnosis requirements for learning, assessment of learning priorities, choice of learning methods, and the measurement of academic achievement and outcomes) ( IGI Global, 2020 ).

Behavior Pattern: A repeated way for a person or group to behave against a particular object or condition ( Dictionary.com, 2020 ).

Learning Behavior: Learning Behavior stresses the crucial relationship between child and youth learning and their social experience and behavior ( Northampton Centre for Learning Behaviour, 2020 ).

Learner Autonomy: The autonomy of learning is when pupils take care of their learning, both in respect of what they learn and, in the direction, they study it ( Oxford University Press ELT, 2013 ).

Affordance: An affordance is an object quality or a condition that makes it possible for the person to take action ( Definitions for Affordance, 2020 ).

Technical evaluation measures

It is the Human-Computer Interaction (HCI) category to measure the applications' UI/UX ( Ishaq et al., 2019 ). The following were the terminologies discussed in selected studies also, approaches used in these studies were shown in Tables 14 and ​ and15 15 .

Usefulness: The perceived usefulness (PU) is also one of the separate structures in the TAM. It is the extent to which a person believes that a specific process may improve the efficiency of his/her job ( World Leaders in Research-Based User Experience, 2020 ) ( Ishaq et al., 2020c ).

Ease of Use: The primary use of computer programs in the TAM is a significant determinant of the target. A discreet individual describes a simple usage as evident in implementing a procedure and directly impacts the perceived usefulness ( Ishaq et al., 2020b ).

Effectiveness: Effectiveness is a participant's ability to execute a task in a given context. In general, efficacy is assessed by determining whether participants can carry out such tasks ( Harrison, Flood & Duce, 2013 ).

Perceived convenience: The extent to which users perceive mobile English learning to be comfortable in terms of time, location, and the method to complete a task ( Chang et al., 2013 ).

Usability: Usability is a common factor that determines how convenient it is to use interface design. The term "usability" also applies to the approaches used during the design process to increase ease of use ( World Leaders in Research-Based User Experience, 2020 ).

Efficiency: Efficiency is the user's ability to deliver their role quickly and productively, representing the user's value during its use. Quality can be calculated in various ways, for example, the time taken to complete or the number of keystrokes necessary to finish a task ( Collins English Dictionary, 2020 ).

Scalability: Scalability is characteristic of an entity, structure, model, or process, defining its ability, under increased or expanding workload or scope to handle and compete well ( Hayes, 2020 ).

Reliability: Reliability means the possibility that a product, device, or service can perform its intended function properly for a given period or work without interruption within a fixed environment ( ASQ, 2020 ).

Flexibility: Flexibility is a characteristic that explains how a person can tolerate changes in situations and think in novel, imaginative ways about issues and tasks ( Flexibility, 2020 ).

Evaluation methodologies

The methodology is the basic techniques or methods used to identify, collect, retrieve, and interpret information on the topic ( Paul, 2000 ). A quantitative study using a quasi-experiment (Pre-test and Post-test) was conducted by ( Hsu, Hwang & Chang, 2013 ) ( Wu, 2018 ) ( Chang et al., 2013 ) ( Kalogirou, Beauchamp & Whyte, 2017 ) ( Fisser, Voogt & Bom, 2012 ) to collect data through questionnaires whereas, a quantitative study using questionnaire only was conducted by ( Zhang, 2016 ) ( Kohnke, Zhang & Zou, 2019 ) ( Bourekkache & Kazar, 2020 ). A mixed-method approach was used ( Amer, 2014 ) ( Botero, Questier & Zhu, 2018 ) ( Ou-Yang & Wu, 2016 ; Quan, 2016 ) ( Önal, Çevik & Şenol, 2019 ; Zhang & Pérez-Paredes, 2019 ) ( Ishaq et al., 2020b ) to conduct interviews and questionnaire for the collection of information, whereas a qualitative approach in which interviews were conducted by ( Hazaea & Alzubi, 2018 ) ( Makoe & Shandu, 2018 ).

Statistical analysis

Statistical analysis is data compilation and evaluation, allowing patterns and developments to be discovered ( Rouse, 2020 ). The following were the statistical analysis techniques used by the selected studies:

Mean and Standard Deviation: Mean which is the average of data set (adding all the numbers then divided by its total point) ( Wei, 2020 ) was calculated by Liu, Li & Carlsson (2010) , Lin (2014) , Amer (2014) , Petersen, Procter-Legg & Cacchione (2013) , Çakmak & Erçetin (2017) , Ou-Yang & Wu (2016) , Quan (2016) and Fisser, Voogt & Bom (2012) whereas Standard Deviation (SD) measures a dataset's dispersion relative to its mean and is calculated as the square root of the variance ( Hargrave, 2020 ). It was calculated by Wu (2018) , Botero, Questier & Zhu (2018) , Fisser, Voogt & Bom (2012) and Ishaq et al. (2020b) .

Analysis of Variance: Analysis of variance (ANOVA) is a statistical method that evaluates a nominal level variable with two or more categories in a scale level dependent variable ( Statistics Solutions, 2013a ) was calculated by Liu, Li & Carlsson (2010) , Hsu, Hwang & Chang (2013) , Lin (2014) , Çakmak & Erçetin (2017) and Kalogirou, Beauchamp & Whyte (2017) .

T-test: The independent t-test is a method that contrasts two sets of a variable usually distributed on a mean value of a constant (e.g., interval or ratio) ( Statistics Solutions, 2013b ) was calculated by Wu (2018) , Petersen, Procter-Legg & Cacchione (2013) and Önal, Çevik & Şenol (2019) .

Analysis of covariance: Analysis of Covariance (ANCOVA) is the inclusion of a continuous variable in addition to the variables of interest (i.e., the dependent and independent variable) as means for control ( Statistics Solutions, 2013c ) calculated in Hsu, Hwang & Chang (2013) and Ou-Yang & Wu (2016) .

Multivariate Analysis of Variance: Multivariate Analysis of Variance (MANOVA) is similar to ANOVA, except that instead of one metric dependent variable, having two or more dependent variables and is concerned with examining the differences between groups ( Statistics Solutions, 2013d ) calculated by Lin (2014) and Çakmak & Erçetin (2017) .

Linear Regression: Linear regression is an analysis that assesses whether one or more predictor variables explain the dependent (criterion) variable ( Statistics Solutions, 2013e ) calculated by Petersen, Procter-Legg & Cacchione (2013) .

Frequencies: A frequency distribution is a graphical or tabular representation that indicates the number of observations over a given interval ( Young, 2020 ) calculated by Zhang (2016) , Kohnke, Zhang & Zou (2019) , Bourekkache & Kazar (2020) and Zhang & Pérez-Paredes (2019) .

It is showing by Tables 14 and ​ and15 15 that questionnaire tool was used to evaluate motivation ( Sandberg, Maris & Geus, 2011 ), perception ( Hsu, Hwang & Chang, 2013 ), effectiveness ( Huang et al., 2016 ), usefulness ( Lin, 2014 ), convenience ( Wu, 2018 ), curiosity ( Chang et al., 2013 ), achievement ( Petersen, Procter-Legg & Cacchione, 2013 ), attitude ( Tsai et al., 2016 ) ( Gafni, Achituv & Rahmani, 2017 ), behavior ( Ou-Yang & Wu, 2016 ), and perception ( Klimova & Polakova, 2020 ), whereas interviews were conducted to measure the learners’ autonomy ( Hazaea & Alzubi, 2018 ), usability, scalability, reliability, and flexibility ( Makoe & Shandu, 2018 ). Few studies used questionnaire and interview (both) to see the attitude and motivation ( Amer, 2014 ), effectiveness ( Quan, 2016 ) ( Önal, Çevik & Şenol, 2019 ), perception ( Chen, Liu & Huang, 2019 ), affordance ( Ramadoss & Wang, 2012 ), usefulness, and ease of use ( Ishaq et al., 2020b ). In contrast, only ( Shih, 2017 ) study used a questionnaire with class observation to measure students' satisfaction and attitude.

Tools used for analysis

Table 16 presented that 39 of the papers used statistical package for social sciences (SPSS) software which was of quantitative in nature, used to evaluate their research. Partial Least Squares (PLS) was also used by two (02) studies ( Huang, 2014 ) ( Chang et al., 2013 ).

Nvivo is a tool that was qualitatively used by Wang, Zou & Xing (2014) and Quan (2016) . After conducting interviews of their respondents, while only two (02) studies ( Klimova & Polakova, 2020 ) ( Ramadoss & Wang, 2012 ) evaluated their results manually to present in the articles. Mplus is a statistical modeling program that allows researchers to analyze the data used by Cheon et al. (2012) , whereas ‘Wenjuan Wang’ an online tool to analyze data used in China by Zhang (2016) . ‘Facets,’ ‘Google Firebase,’ ‘Notes taking,’ online tool, and ‘Google Sheet’ used by Uematsu (2012) , Pham, Nguyen & Chen (2017) , Hazaea & Alzubi (2018) , Kohnke, Zhang & Zou (2019) and Ng et al. (2020) , respectively. In this literature review, eleven (11) studies did not use any online or desktop application to analyze the data.

RQ6: compare the usage of simple mobile applications with gamified applications (Serious Game) for language learning?

Figure 7 shows the trend of language learning applications and games developed during selected studies of 2010 to 2020. It can be observed that in the years 2010 and 2011, few applications related to mobile-based and gamified language learning applications have been proposed as fewer people were familiar with this area. From the year 2013 to 2014, an increasing number of gamified application gamified applications were proposed for the studies. Similarly, there are increments from 2017 to onwards for mobile-based gamified applications to learn how educational institutes were observed in the finalized studies.

Mobile and gamified language learning applications from the selected studies concerning level, purpose, evaluation method, and results are presented in Table 17 . The applications are divided into two categories: (1) Mobile application, (2) Gamified applications (serious games). Mobile applications are proposed for teaching ( Kearney et al., 2012 ) ( Zhang, 2016 ), reading ( Hsu, Hwang & Chang, 2013 ) ( Lin, 2014 ) ( Hazaea & Alzubi, 2018 ), vocabulary learning ( Zhang, Song & Burston, 2011 ) ( Huang et al., 2016 ) ( Bohm & Constantine, 2016 ) ( Petersen, Procter-Legg & Cacchione, 2013 ) ( Botero, Questier & Zhu, 2018 ) ( Çakmak & Erçetin, 2017 ) ( Ou-Yang & Wu, 2016 ; Quan, 2016 ) ( Makoe & Shandu, 2018 ) ( Klimova & Polakova, 2020 ), pronunciation ( Liakin, Cardoso & Liakina, 2014 ), writing, reading, listening and speaking ( Chang et al., 2013 ), listening comprehension ( Çakmak & Erçetin, 2017 ), grammar ( Chu, Wang & Wang, 2019 ) and reading comprehension ( Ishaq et al., 2020b ). The results presented by studies for the applications of pedagogy, reading, pronunciation, writing, listening, speaking, grammar outperformed the control group. It may be concluded that mobile applications for vocabulary learning have a positive impact on language learning in which pupils from the experimental group outperformed the control group except ( Klimova & Polakova, 2020 ) ( Ishaq et al., 2020b ) where the students and teachers mentioned they are not satisfied with the design, content, assessment methods adopted in the applications besides pronunciation support and teachers' notifications were also missing.

Gamified applications are proposed for vocabulary learning ( Sandberg, Maris & Geus, 2011 ) ( Sandberg, Maris & Hoogendoorn, 2014 ) ( Wu, 2018 ) ( Chen, Liu & Huang, 2019 ) ( Fisser, Voogt & Bom, 2012 ), Spanish language learning ( Holden & Sykes, 2011 ), Idioms ( Amer, 2014 ), Vocabulary, Grammar, and Sentence Structure ( Tsai et al., 2016 ), and Tenses, Words, Sentences ( Önal, Çevik & Şenol, 2019 ). The results presented by studies for gamified applications of vocabulary learning, Idioms, grammar, sentence structure, tenses, and words outperformed the control groups with much engagement, interest, and positive feedback. Although mobile applications are effective, gamified applications are more effective with enhanced interest and engagement, resulting in positive learning outcomes ( Ishaq et al., 2020a ; Ishaq et al., 2020b ; Ishaq et al., 2019 ; Ishaq et al., 2020d ). Furthermore, it may be concluded that gamified applications are trending for all the subjects at all levels of education to get significant learning performance ( Ishaq et al., 2020d ) ( Ishaq et al., 2020c ; Dichev & Dicheva, 2017 ) ( Zin & Yue, 2013 ; Zin, Jaafar & Yue, 2009 ). Furthermore, ( Ishaq et al., 2020b ) recommended gamified application in the public sector school to effectively learn the English language (specifically reading comprehension at primary level) after addressing all the stakeholders' issues.

Discussion and future directions

This section summarizes and discusses the results related to the systematic literature review.

Taxonomic hierarchy

In this comprehensive literature review, the aim was to investigate current MALL information and its application with 63 selected papers. To achieve this objective, a taxonomic hierarchy of finalized studies shown in Fig. 8 was established and examined trends and perspectives on adapted/developed frameworks, content, and teaching and learning tools. However, these dimensions were further broken down into several sub-levels that illustrated each area's scope with its role in enhancing the efficiency of language learners.

Future directions

It was suggested that MALL specific theories or models might be developed because the models used from fields other than MALL was to be adopted or adapted as all components of the models used in the studies could not be applied to measure the constructs. The researchers used varying theories or models to develop a theoretical framework of the respective studies, and this practice was too time-consuming, and the models were minimum valid. The researcher confronted the same issue while measuring usability, usefulness, effectiveness, ease of use, user-friendliness, and user behavior. To ensure the effectiveness of learning content used in mobile and gamified applications, it was suggested that duly approved content material, i.e., written or pictorial by the concerned authorities, was observed from the literature. Whereas specified and authentic learning content was missing, and desired learning outcomes were not achieved. More work is needed concerning teachers' professional development and exploration of their competencies in the field of MALL. The teacher-student relationship was inevitable regarding the teaching-learning process because it was the only source of incorporating various teaching methods and techniques to make learning effective ( Omer, Farooq & Abid, 2020 ). In this regard, to compete for the world, teachers must link up with technological devices and gamified applications to engage their students in learning and entertainment to achieve a high learning outcome ( Isa et al., 2019 ).

The applications used for learning did not fulfill the requirements related to usability, usefulness, effectiveness, ease of use, user-friendliness, and user behavior, so it is proposed to develop gamified applications (serious game) having all features supporting the above constructs. From the literature review, a gamified application for English language learning at the primary level was lacking. Students of primary class levels loved to play more than indulging in books. A conventional environment made the students passive in the class and lessened their motivation level.

During the literature review, research work regarding development of gamified application for reading comprehension was lacking. No such conceptual model gamification has been developed which may be used for language learning with reference to cultural context. Assessment of grammar or vocabulary in MALL applications were found during literature review but were not as such advanced enough that might perform personalized assessment of students’ learning outcomes. Same was found in case of reward system and entertainment elements.

Therefore, gamified learning applications may be designed for primary grade students to make language learning more enjoyable, entertaining, attractive, and engaging them to achieve high learning outcomes effectively.

This systematic literature review aimed to understand research patterns in MALL to learn the English language through mobile or gamified applications, approaches, and frameworks/models developed, or adopted. A comprehensive analysis of literature was undertaken to ensure a detailed discussion of the problems and their remedies. It was searched with as many known terminologies associated with MALL and then analyzed the results accordingly. The search was ended in August 2020, which would not have comprised studies that were carried out after the date. The Web of Science core collection was analyzed, and 63 out of 57,364 publications were selected.

The findings shown that nearly every selected article was published in a recognized journal, whereas only single research at a conference. The two primary forms of study adopted in these studies were “Solution Proposal” and “evaluation research”. The majority of the chosen researches were evidence-based and could lead to the full advantages of MALL to teachers and students. MALL's most frequent key aspects were language learning strategies and evaluation of students’ results. In contrast, MALL specific frameworks and theories, approved content were less addressed aspect of MALL.

The deficiencies in SLR related explicitly were research technique, incorrect data collection, or misclassification. However, with separate keywords from the Web of Science core collection repository, the research approach minimized the possibility of selection error. External concerns were addressed by implementing specific inclusion/exclusion guidelines, and two independent experts were requested to evaluate all extractions.

For future research on MALL, more attention might be paid to primary or secondary school students and teachers, approved curricula design for English subject, and tools’ design, particularly serious games. Further evaluation research might be conducted to analyze existing MALL content.

Acknowledgments

This study is a part of my doctoral thesis work at the Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Malaysia. I would like to express my deepest gratitude to my advisors, Prof. Dr. Nor Azan Mat Zin, Dr. Fadhilah Rosdi, and Prof. Dr. Adnan Abid. I also wish to express my sincere thanks to all the co-authors who have contributed to this work.

Funding Statement

This work was supported by the Universiti Kebangsaan Malaysia under the Grand Challenge Fund (Grant number DCP-2017-007/2). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

Adnan Abid is an Academic Editor for PeerJ Computer Science.

Kashif Ishaq conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Nor Azan Mat Zin conceived and designed the experiments, analyzed the data, authored or reviewed drafts of the paper, and approved the final draft.

Fadhilah Rosdi conceived and designed the experiments, analyzed the data, authored or reviewed drafts of the paper, and approved the final draft.

Muhammad Jehanghir performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, and approved the final draft.

Samia Ishaq performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, and approved the final draft.

Adnan Abid conceived and designed the experiments, analyzed the data, performed the computation work, authored or reviewed drafts of the paper, and approved the final draft.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Mobile Gaming Patterns and Their Impact on Learning Outcomes: A Literature Review

2012, Lecture Notes in Computer Science

Related Papers

International Journal of Technology Enhanced Learning

Roland Klemke

Dragan Cisic

Journal of Computer Assisted Learning

This paper reports a study that attempts to explore how using mobile technologies in direct physical interaction with space and with other players can be combined with principles of engagement and self-motivation to create a powerful and engaging learning experience. We developed a mobile gaming experience designed to encourage the development of children's conceptual understanding of animal behaviour. Ten children (five boys and five girls) aged between 11 and 12 years played and explored the game. The findings from this study offer interesting insights into the extent to which mobile gaming might be employed as a tool for supporting learning. It also highlights a number of major challenges that this format raises for the organisation of learning within schools and the design of such resources.

Peter Mozelius

Educational Technology Research and Development

Filippos D Giannakas , Andreas Papasalouros

With the increasing popularity of smartphones and tablets, Game-Based Learning (GBL) is undergoing a rapid shift to mobile platforms. This transformation is driven by mobility, wireless interfaces, and built-in sensors that these smart devices offer in order to enable blended and context-sensitive mobile learning (m-Learning) activities. Thus, m-Learning is becoming more independent and ubiquitous (u-Learning). In order to identify and analyze the main trends and the future challenging issues involved in designing mGBL learning strategies, as well as to bring to the foreground important issues pertaining to mobile and context-aware ubiquitous GBL, the work at hand conducts a comprehensive survey of this particular area. Specifically, it introduces and applies a six-dimensional framework consisted of Spatio-temporal, Collaboration/Social, Session, Personalization, Data security & privacy, and Pedagogy, with the aim of scrutinizing the contributions in the field of mGBL published from 2004 to 2016. It was found that the transition to mGBL presents several difficulties, and therefore cannot be conceived as a simple and quick modification of existing GBL solutions. In this respect, this work is anticipated to foster the development of well-designed solutions that are intensive not only in their technological aspect, but in pedagogical qualities as well. December 2017 - Educational Technology Research and Development, Springer DOI10.1007/s11423-017-9552-z

ev.fe.uni-lj.si

Matevz Pogacnik

Achraf Touati , Youngkyun Baek

This study investigated students' perceptions of competence and enjoyment of a mobile game within the context of mobile game-based learning. The proposed model showed that perceived competence and game attitude were the main pre-dictors of enjoyment, while no direct relationship was found between perceived competence and gaming achievement. The model simultaneously considered other factors such as prior game experience and intensity of use, and final analysis revealed that these two variables were directly related. Another important finding was the strong impact of prior game experience on perceived competence. Results are interpreted with reference to implications for possible means of improving learning outcomes when using mobile learning games in the academic context.

Advancing Mobile Learning in Contemporary Educational Spaces

Kristin Villanueva

The evolution of mobile games and the subsequent evolution of designing mobile gaming experiences to support learning is captured in this chapter. The authors will address the affordances and limitations of mobile gaming as well as the learning theories and frameworks that lay the foundation for learning through mobile games. Current perceptions of mobile gaming in the classroom and the implementation of mobile games in schools are discussed. In addition, this chapter explores mobile games for social change, their prevalence, and their role in facilitating learning. The chapter also discusses differing perceptions on the issue of mobile games in and out of the classroom as well as the potential for marketability.

RELATED PAPERS

Srini Kalyanaraman

Michel Bitbol

Journal Of Administration and Educational Management (ALIGNMENT)

Dyna Sibarani

Les Temps modernes

Sylvie Patron

Akkadica 144

Roberto Dan , Annarita S . Bonfanti

Juan Jacobo Giraldo Botero

Jurnal Teknik ITS

harno pratomo

Journal of Physics B: Atomic, Molecular and Optical Physics

Journal of the American College of Cardiology

Luis Alfonso Diaz Martinez

International Journal of Research and Development in Pharmacy & Life Sciences

Anita Anita

原版定做美国佛罗里达州立大学毕业证 fsu本科文凭托业成绩单原版一模一样

Ghaid Al-Rabadi

Sylvaine Laulom

Proceedings of the 35th …

Christoph Koch

Diego Suárez

Journal of Physics: Conference Series

yaser hafez

IEEE Access

arXiv (Cornell University)

Arnold Knopfmacher

British Journal of Nutrition

Ei Sakaguchi

Monish Bhatia

Landscape 3: una sintesi di elementi diacronici Uomo e ambiente nel mondo antico: un equilibrio possibile?

Giacomo Antonelli

Carlota Miranda Urbano

Journal of Tropical Medicine

Zenebe Melaku

Nature Reviews Earth & Environment

Dorothy Merritts

Behavioural Brain Research

Eliseo Iglesias-soler

Basic research in cardiology

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Gamification Mobile Applications: A Literature Review of Empirical Studies

• Conference paper
• First Online: 22 March 2023
• Cite this conference paper

• Liping Yang   ORCID: orcid.org/0000-0001-6998-3584 12 &
• Matthias Gottlieb   ORCID: orcid.org/0000-0001-5983-6556 12  

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 633))

Included in the following conference series:

• International Conference on Interactive Collaborative Learning

Gamification is an increasingly important topic and appears to be an emerging trend in many areas. However, research on gamified mobile applications is still fragmented, and a systematic overview of gamification in mobile apps is missing. Thus, we conducted a systematic literature review of 43 empirical studies to explore the state of the art in integrating gamification approaches into mobile apps. The literature review covers gamification design elements, psychological outcomes, behavioral outcomes, research areas, and the effect of gamification. The results show that rewards are the most commonly used design element of gamification in mobile apps. Gamification has a potential impact on psychology and behavior and provides positive benefits. The review’s findings extend the knowledge base by providing insight for further studies and the design of gamified systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Alsalman, D.M., Ali, Z.B., Alnosaier, Z., Alotaibi, N., Alanzi, T.M.: Caregiver’s opinions on the design of the screens of a future gamified mobile application for self-management of type 1 diabetes in children in Saudi Arabia. Int. J. Telemedicine Appl. 2021 (2021)

Google Scholar  

Sandy, A.T., Ulfa, S., Wedi, A.: Use of gamification in indonesian for non-native speakers (BIPA). In: Journal of Physics: Conference Series 1908 (2021)

Behl, A., Pereira, V.: What’s behind a scratch card? Designing a mobile application using gamification to study customer loyalty: An experimental approach. Australas. J. Inf. Syst. 25 , 1–24 (2021)

Dymora, P., Niemiec, K.: Gamification as a supportive tool for school children with dyslexia. Informatics 6 (4), 48 (2019)

Article   Google Scholar  

Earle, A.M., LaBrie, J.W., Boyle, S.C., Smith, D.: In pursuit of a self-sustaining college alcohol intervention: Deploying gamified PNF in the real world. Addict. Behav. 80 , 71–81 (2018)

Edwards, E.A., et al.: Creating a theoretically grounded, gamified health app: lessons from developing the cigbreak smoking cessation mobile phone game. JMIR Serious Games 6 (4), e10252 (2018)

Featherstone, M., Habgood, J.: UniCraft: exploring the impact of asynchronous multiplayer game elements in gamification. Int. J. Hum. Comput. Stud. 127 , 150–168 (2019)

Gamston, C.E., Hollingsworth, J.C., Fox, B.I., Rogers, S., O’Barr, M.E., Lloyd, K.B.: Evaluation of the impact of enhanced virtual forms and gamification on intervention identification in a pharmacist-led ambulatory care clinic. Exploratory Res. Clin. Soc. Pharm. 4 , 100068 (2021)

Giunti, G., Mylonopoulou, V., Romero, O.R.: More stamina, a gamified mHealth solution for persons with multiple sclerosis: Research through design. JMIR mHealth uHealth 6 (3) (2018)

Hidayat, W.N., Fitranti, A., Firdaus, A.F., Kartikasari, C.D.I., Sutikno, T.A.: Gamification based mobile application as learning media innovation for basic programming lessons. In: IOP Conference Series: Materials Science and Engineering, vol. 732. IOP Publishing, Bristol (2020)

Hightow-Weidman, L., et al.: A gamified smartphone app to support engagement in care and medication adherence for HIV-positive young men who have sex with men (AllyQuest): development and pilot study. JMIR Public Health Surveill. 4 (4), e8923 (2018)

Hoffmann, A., Faust-Christmann, C.A., Zolynski, G., Bleser, G.: Toward gamified pain management apps: mobile application rating scale⇓based quality assessment of pain-mentor⇔s first prototype through an expert study. JMIR Formative Res. 4 (5), e13170 (2020)

Huang, L.S., Lau, N.: Enhancing the smart tourism experience for people with visual impairments by gamified application approach through needs analysis in Hong Kong. Sustainability 12 (15), 27 (2020)

Ibrahim, E.N.M., Jamali, N., Suhaimi, A.I.H.: Exploring gamification design elements for mental health support. Int. J. Adv. Technol. Eng. Explor. 8 (74), 114–125 (2021)

Jamaludin, N.F., Wook, T.S.M.T., Noor, S.F.M., Qamar, F.: Gamification design elements to enhance adolescent motivation in diagnosing depression. Int. J. Interact. Mobile Technol. 15 (10), 154–172 (2021)

Kamada, M., et al.: Large-scale fandom-based gamification intervention to increase physical activity: a quasi-experimental study. Med. Sci. Sports Exerc. 54 (1), 181–188 (2022)

Kamilakis, M., Chorianopoulos, K.: Mobile mapmaking: a field study of gamification and cartographic editing. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) INTERACT 2019. LNCS, vol. 11747, pp. 427–435. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29384-0_26

Chapter   Google Scholar  

LeGrand, S., et al.: Epic allies, a gamified mobile phone app to improve engagement in care, antiretroviral uptake, and adherence among young men who have sex with men and young transgender women who have sex with men: Protocol for a randomized controlled trial. JMIR Res. Protoc. 7 (4), e8811 (2018)

Leitao, R., Maguire, M., Turner, S., Arenas, F., Guimaraes, L.: Ocean literacy gamified: a systematic evaluation of the effect of game elements on students’ learning experience. Environ. Educ. Res. 28 (2), 276–294 (2022)

Lidia, A.C., Julio, R.T., Petra, D.S.P., Rafael, P.J.: How to encourage recycling behaviour? the case of wasteapp: a gamified mobile application. Sustainability 10 (5), 20 (2018)

Luger-Bazinger, C. and Hornung-Prähauser, V.: Innovation for sustainable cities: the effects of nudging and gamification methods on urban mobility and sustainability behaviour. GI_Forum 9 (2), 251–258 (2021)

Mahmud, S.N.D., Husnin, H., Soh, T.M.T.: Teaching presence in online gamified education for sustainability learning. Sustainability (Switzerland) 12 (9), 3801 (2020)

Marcial, D.E., dela Peña, L., Montemayor, J., Dy, J.: The design of a gamified responsible use of social media. In: Frontiers in Education, vol. 6 (2021)

Mat, R.C., Kazunori, M., Rahman, A.A.: The development of mobile Japanese halal gamification (MJHG). Int. J. Interact. Mobile Technol. 14 (17), 113–129 (2020)

Mora-Gonzalez, J., Pérez-López, I.J., Delgado-Fernández, M.: The “$in TIME” gamification project: using a mobile app to improve cardiorespiratory fitness levels of college students. Games Health J. 9 (1), 37–44 (2020)

Morais Pereira Simões, M.D.S. et al.: Use of a smartphone app combined with gamification to increase the level of physical activity of adults and older adults: protocol of a sequential multiple assignment randomized trial. Trials 20 (1), 1–8 (2019)

Navarro-Alamán, J., Lacuesta, R., Garcia-Magariño, I., Gallardo, J.: A methodology for the design and development of gamified mobile apps for monitoring cancer survivors. J. Biomed. Inform. 106 , 103439 (2020)

Omotosho, A., Tyoden, T., Ayegba, P., Ayoola, J.: A gamified approach to improving student’s participation in farm practice - a case study of landmark university. Int. J. Interact. Mobile Technol. 13 (5), 94–109 (2019)

Pernencar, C., et al.: Planning a health promotion program: mobile app gamification as a tool to engage adolescents. Procedia Comput. Sci. 138 , 113–118 (2018)

Prakasa, F.B.P., Suyoto, S., Emanuel, A.W.R.: Designing mobile application gamification for tourism village in Indonesia. In: IOP Conference Series: Materials Science and Engineering, vol. 1098. IOP Publishing, Bristol (2021)

Pramana, G., Parmanto, B., Lomas, J., Lindhiem, O., Kendall, P.C., Silk, J.: Using mobile health gamification to facilitate cognitive behavioral therapy skills practice in child anxiety treatment: open clinical trial. JMIR Serious Games 20 (5), e8902 (2018)

Prandi, C., et al.: Gamifying cultural experiences across the urban environment. Multimedia Tools Appl. 78 (3), 3341–3364 (2018). https://doi.org/10.1007/s11042-018-6513-4

Prasetyo, N.A., Suyoto, S.: Design mobile app for increase the visitor museum using Gamification method. TELKOMNIKA (Telecommun. Comput. Electron. Control) 16 (6), 2791–2798 (2018)

Rajani, N.B., Mastellos, N., Filippidis, F.T.: Impact of gamification on the self-efficacy and motivation to quit of smokers: observational study of two gamified smoking cessation mobile apps. JMIR Serious Games 9 (2), e27290 (2021)

Rantala, A., et al.: A gamified mobile health intervention for children in day surgery care: Protocol for a randomized controlled trial. Nurs. Open 9 (2), 1465–1476 (2022)

Ricci, A., Gosetto, L., Blondon, K., Ehrler, F.: The new smart-meds: redesign of a gamified app to improve medication adherence using a mixed methods design. Stud. Health Technol. Inf. 275 , 182–186 (2020)

Rodrigues, M., Monteiro, V., Novais, P., and Analide, C. Getting residents closer to public institutions through gamification. In: International Symposium on Ambient Intelligence, vol. 806, pp. 33–39. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-030-01746-0_4

Suryanto, P., Emanuel, A.W.R.: Pranowo: design of dayak kanayatn language learning mobile applications using gamification. Int. J. Eng. Pedagogy 10 (4), 54–68 (2020)

Tamtama, G.I.W., Suryanto, P.: Suyoto: design of English vocabulary mobile apps using gamification: an Indonesian case study for kindergarten. Int. J. Eng. Pedagogy 10 (1), 105–162 (2020)

Tark, R., Metelitsa, M., Akkermann, K., Saks, K., Mikkel, S., Haljas, K.: Usability, acceptability, feasibility, and effectiveness of a gamified mobile health intervention (triumf) for pediatric patients: qualitative study. JMIR Serious Games 7 (3), e13776 (2019)

Widarti, E., Emanuel, A.W.R.: Mobile application design for heritage tourism uses gamification approach in Indonesia. Int. J. Eng. Pedagogy 10 (5), 89–102 (2020)

Xu, L.Q., et al.: Mobile health-based gamification intervention to increase physical activity participation among patients with coronary heart disease: study protocol of a randomised controlled trial. BMJ Open 12 (1), 10 (2022)

Zolfaghari, M., Shirmohammadi, M., Shahhosseini, H., Mokhtaran, M., Mohebbi, S.Z.: Development and evaluation of a gamified smart phone mobile health application for oral health promotion in early childhood: a randomized controlled trial. BMC Oral Health 21 (1), 1–9 (2021)

Chapman, J.R., Rich, P.J.: Does educational gamification improve students’ motivation? If so, which game elements work best? J. Educ. Bus. 93 (7), 315–322 (2018)

Hamari, J.: Gamification. The Blackwell encyclopedia of sociology, pp.1–3 (2007)

Goggin, G.: Ubiquitous apps: politics of openness in global mobile cultures. Digit. Creativity 22 (3), 148–159 (2011)

Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining gamification. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9–15. ACM, New York (2011)

Sailer, M., Hense, J.U., Mayr, S.K., Mandl, H.: How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. Comput. Hum. Behav. 69 , 371–380 (2017)

Huotari, K., Hamari, J.: Defining gamification: a service marketing perspective. In: Proceeding of the 16th International Academic MindTrek Conference, pp. 17–22. ACM, New York (2012)

Brockmann, T., Zarnekow, R., Robra-Bissantz, S., Lattemann, C., Stieglitz, S.: Gamification: Using Game Elements in Serious Contexts, pp. 19–29 (2017)

Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work?--a literature review of empirical studies on gamification. In: 2014 47th Hawaii International Conference on System Sciences, pp. 3025–3034. IEEE, New York (2014)

Rachels, J.R., Rockinson-Szapkiw, A.J.: The effects of a mobile gamification app on elementary students’ Spanish achievement and self-efficacy. Comput. Assist. Lang. Learn. 31 (1–2), 72–89 (2018)

Hamari, J., Koivisto, J., Pakkanen, T.: Do persuasive technologies persuade?-a review of empirical studies. In: Spagnolli, A., Chittaro, L., Gamberini, L. (eds.) International conference on persuasive technology, vol. 8462, pp. 118–136. Springer, Heidelberg (2014)

Hervás, R., Ruiz-Carrasco, D., Mondéjar, T., Bravo, J.: Gamification mechanics for behavioral change: a systematic review and proposed taxonomy. In: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare, pp. 395–404 (2017)

Brocke, J.V. et al.: Reconstructing the giant: on the importance of rigour in documenting the literature search process (2009)

Download references

Acknowledgement

The intention on which this report is based was funded by the German Federal Ministry of Education and Research under the funding code 534800. The responsibility for the content of this publication lies with the authors.

Author information

Authors and affiliations.

IT Service Center, Technical University of Munich, Arcisstr. 21, 80333, Munich, Germany

Liping Yang & Matthias Gottlieb

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Liping Yang .

Editor information

Editors and affiliations.

CTI Global, Frankfurt, Germany

Michael E. Auer

Federal Ministry of Education, Science and Research, Vienna, Austria

Wolfgang Pachatz

Tallinn University of Technology, Tallinn, Estonia

Tiia Rüütmann

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Yang, L., Gottlieb, M. (2023). Gamification Mobile Applications: A Literature Review of Empirical Studies. In: Auer, M.E., Pachatz, W., Rüütmann, T. (eds) Learning in the Age of Digital and Green Transition. ICL 2022. Lecture Notes in Networks and Systems, vol 633. Springer, Cham. https://doi.org/10.1007/978-3-031-26876-2_88

Download citation

DOI : https://doi.org/10.1007/978-3-031-26876-2_88

Published : 22 March 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-26875-5

Online ISBN : 978-3-031-26876-2

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Review article
• Open access
• Published: 10 July 2017

The effect of games and simulations on higher education: a systematic literature review

• Dimitrios Vlachopoulos 1 &
• Agoritsa Makri 2  

International Journal of Educational Technology in Higher Education volume  14 , Article number:  22 ( 2017 ) Cite this article

405k Accesses

297 Citations

41 Altmetric

Metrics details

The focus of higher education institutions is the preparation of future professionals. To achieve this aim, innovative teaching methods are often deployed, including games and simulations, which form the subject of this paper. As the field of digital games and simulations is ever maturing, this paper attempts to systematically review the literature relevant to games and simulation pedagogy in higher education. Two researchers collaborate to apply a qualitative method, coding and synthesizing the results using multiple criteria. The main objective is to study the impact of games and simulations with regard to achieving specific learning objectives. On balance, results indicate that games and/or simulations have a positive impact on learning goals. The researchers identify three learning outcomes when integrating games into the learning process: cognitive, behavioural, and affective. As a final step, the authors consolidate evidence for the benefit of academics and practitioners in higher education interested in the efficient use of games and simulations for pedagogical purposes. Such evidence also provides potential options and pathways for future research.

Introduction

As rapidly evolving technological applications, games and simulations are already widely integrated in the traditional educational process. They are deployed extensively in the field of education, with an existing body of work examining the relation between games and education (Yang, Chen, & Jeng, 2010 ; Chiang, Lin, Cheng, & Liu, 2011 ). In recent years, digital or web-based games have increasingly supported learning. In the context of online education, this research area attracts a significant amount of interest from the scientific and educational community, for example tutors, students and game designers. With the growing expansion of technology, instructors and those who create educational policy are interested in introducing innovative technological tools, such as video games, virtual worlds, and Massive Multi-Player Online Games (MMPOGs) (Buckless, 2014 ; Gómez, 2014 ).

Games and simulations show mixed effects across a number of sectors, such as student performance, engagement, and learning motivation. However, as these studies focus only on certain disciplines, there remains a gap in the literature concerning a clear framework of use across academic programmes. As a result, the issue of efficiently integrating games and simulations in the educational process is often up to the instructor’s discretion. Accordingly, the aim of this paper is to develop a framework to allow educators across disciplines to better understand the advantages and draw backs of games and simulations specific to their pedagogical goals.

Rationale of the study

The researchers set out to examine the effectiveness of games and simulations in the learning experience, and immediately encounter the first challenge, which relates to a lack of clear empirical evidence on the issue (Farrington, 2011 ). The scientific field is too extensive and requires further investigation. Furthermore, there is currently no formal policy framework or guidelines recommended by governments or educational institutions on the adoption of games and simulations in education. This is the case for many countries in Europe, the US, and Australia, where it is the responsibility of the instructor or institution to incorporate games into the curriculum.

The main motivation for the current review lies in the fact that games are already, to a certain degree, integrated into educational systems to achieve a variety of learning outcomes (Connolly, 2012 ), yet a comprehensive policy is still lacking. In this paper, the first step was an attempt to conceptualize the terms “game” and “simulations”. Although the two terms are neither wholly synonymous, or completely differentiated, in the main body of this review, the focus will be on lumping them together and perceiving them as points across a multidimensional continuum (Aldrich, 2009 ; Renken, 2016 ), since these educational technologies are consolidated under the umbrella of an interactive virtual environment in digital education.

A primary aim is to identify studies concentrating on the use of games and simulations for learning purposes, and to analyse the results by comparing them to prior studies’ findings. Two research questions guide the review analysis: a) How can the best practices/methods for designing and incorporating games and simulations in student learning be identified? b) How can games/simulations enhance Higher Education?

The major difference between the current review and the previous reviews in the field is the conceptualization of the terms “games and simulations”, which acts as an umbrella for further typologies. In other words, the researchers include more genres of games and simulations in their systematic review, compared to the other literature reviews. In addition, the researchers’ intention is to focus on the impacts of games and simulations on learning outcomes. The researchers don’t focus only on the cognitive outcomes, which is the most obvious and common topic among other researchers but, simultaneously, they analyze behavioural and affective effects as well. Furthermore, most of the previous reviews focus on the impacts of games and simulations on the learning process of certain subjects (e.g. Science, Business, Nursing, etc.), whereas this study expands research in a wide spectrum of academic disciplines and subjects. Overall, the current study offers a systematic review that opens new areas for further discussion, highlighting that collaborative learning, teamwork and students’ engagement also play a significant role for a successful learning process.

Conceptualising games and simulations

In recent years, the interest in examining game use in higher education has increased. This includes educational games (Çankaya & Karamete, 2009 ), digital game-based learning (DGBL) (Yang, 2012 ), and applied games (van Roessel & van Mastrigt-Ide, 2011 ). In addition, scholars, sometimes, include interactive exercises (Mueller, 2003 ), video games (Biddiss & Irwin, 2010 ), or even expand to next generation video games (Bausch, 2008 ), in the category of games. With respect to web-based games, the technological platforms that implement digital game code include computers and consoles (Salen & Zimmerman, 2004 ). They can run on a web browser on mobile phones and other mobile gaming devices (Willoughby, 2008 ) (e.g., tablets).

Despite the abundance of game types, there is a lack of clear, shared definitions and terminology among scholars and educators, which has led to “terminological ambiguity” (Klabbers, 2009 ). Nevertheless, the need for shared terminology remains when discussing the different forms of games and simulations in higher education. Although academics and game developers may use varying taxonomy to categorise games, the majority broadly agree on the following seven genres (Gros, 2007 ):

Action games: response-based video games.

Adventure games: the player solves problems to progress through levels within a virtual world.

Fighting games: these involve fighting with computer-controlled characters or those controlled by other players.

Role-playing games: players assume the roles of fictional characters.

Simulations: games modelled after natural or man-made systems or phenomena, in which players have to achieve pre-specified goals.

Sports games: these are based on different kinds of sports.

Strategy games: these recreate historical scenes or fictional scenarios, in which players must devise an appropriate strategy to achieve the goal.

In recent years, several well-designed empirical studies investigating the effects of serious games on learning outcomes have been published. Sawyer refers to serious games as those games produced by the video game industry that have a substantial connection to the acquisition of knowledge (Sawyer, 2002 ). Zyda ( 2005 ) expands Sawyer’s definition, adding that serious games are games whose primary purpose is not entertainment, enjoyment or fun. Serious games, educational gaming, as well as virtual worlds developed for educational purposes reveal the potential of these technologies to engage and motivate beyond leisure activities (Anderson et al., 2009 ). At the same time, there is extensive literature exploring the potential learning benefits offered by game-based learning (GBL), which can be defined as the use of game-based technology to deliver, support, and enhance teaching, learning, assessment, and evaluation (Connolly, 2007 ).

• Simulations

Simulations create a scenario-based environment, where students interact to apply previous knowledge and practical skills to real-world problems, also allowing teachers to reach their own goals, as well (Andreu-Andrés & García-Casas, 2011 ; García-Carbonell & Watts, 2012 ; Angelini, 2015 ). During scenario-based training, the player acquires important skills, such as interpersonal communication, teamwork, leadership, decision-making, task prioritising and stress management (Flanagan, 2004 ). The practical scenario may be carried out individually or within a team (Robertson et al., 2009 ), leading to collaboration and knowledge sharing.

With the explosion of Web 2.0 technology, increased opportunities to engage with technological applications in a collaborative and participatory way have emerged, promoting information access, shared ideas, knowledge exchange, and content production (McLoughlin & Lee, 2008 ). Digital simulations, which engage students in the interactive, authentic, and self-driven acquisition of knowledge, are being adopted in higher education. Connolly and Stansfield ( 2006 ) define game-based e-learning as a digital approach which delivers, supports, and enhances teaching, learning, assessment, and evaluation. Game-based e-learning is differentiated from GBL, which tends to cover both computer and non-computer games.

Delivery platforms are an essential aspect for game designers when creating and distributing games and simulations (e.g. computer, video, online, mobile, 3D, etc.). Designers must pay attention to characteristics such as the technical challenges, modules and techniques associated with the game design, the players involved in gaming, and the teaching modes (e.g. single, multi-player, collaborative, synchronous, etc.). This study examines the diverse curricular areas and learning objectives each game intends to access. The above-mentioned game classification is presented below (Fig. 1 ).

Classification of games and simulations

The main difference between games and simulations is the following: games are tools which are artificial and pedagogical; they include conflict, rules, and predetermined goals, whereas simulations are dynamic tools, representing reality, claiming fidelity, accuracy, and validity (Sauve, 2007 ).

Previously conducted reviews/meta-analyses on games and simulations in higher education

To establish a context , the researchers, initially, examined the relevant literature on the effectiveness of all types of games and simulations in learning outcomes. Many papers are analysed and summarised as follows, providing useful guidance for this study.

Through their systematic review, Tsekleves et al. ( 2014 ) provide insight into the barriers and benefits of using serious games in education. (Regarding benefits, the authors catalogue: achievement and rewards, interactivity and feedback, motivation and competition, playfulness and problem-based learning, collaborative learning, progression and repetition, as well as realism and immersion. Finally, they propose some guidelines to help stakeholders better implement serious games in education. Similarly, Bellotti, ( 2013 ) suggest useful guidelines for the performance assessment of serious games. Following user performance assessments, they offer an overview on the effectiveness of serious games in relation to learning outcomes. Results reveal the effectiveness of serious games in motivating and achieving learning goals, the importance of providing appropriate user feedback, while emphasizing that new types of games are best deployed through proper instructor guidance. Moreover, they stress aspects they consider important, such as performance assessment with a view to fostering adaptivity, as well as personalisation, and meeting needs on an individual basis (e.g. learning styles, information provision rates, feedback, etc.).

The instructor’s role is also outlined by Lameras et al. ( 2016 ) who provide conceptual and empirical evidence on the manner in which learning attributes and game mechanics should be designed and incorporated by faculty, specifically with a view to fully integrate these into lesson plans and the learning process as a whole. Games allow practitioners to quickly come to grips with the way in which learning activities, outcomes, feedback and roles may vary, as well as to enhance the in-game learning experience. Similarly, the systematic review of 64 articles by de Smale, ( 2015 ) concludes that there is a positive or neutral relationship between the use of simulations and games and learning achievement. The researchers arrive at three recurring conditions for the successful use of simulations and games: the specificity of the game, its integration in the course, and the role of a guiding instructor, which are all conditions in line with Bellotti et al. ( 2013 )‘s results.

Young et al. ( 2012 ) choose 39 articles that meet the inclusion criteria related to video games and academic achievement, concentrating on the use of traditional games versus video games for educational purposes. The studies are categorised by subject, namely History, Mathematics, Physical Education, Science, and Languages. Results indicate that there exists limited evidence of the benefits of including education games in the traditional classroom environments, a finding which is contrary to the aforementioned studies. Smetana and Bell ( 2012 ) examine computer simulations to support instruction and learning in Science. In their comparative study between computer games and traditional games, they conclude that computer games can be as effective, if not more so, than traditional games in promoting knowledge, developing procedural skills and facilitating conceptual change. To integrate them properly as supplementary elements (Rajan, 2013 ), games require the adoption of high-quality support structures, student participation, as well the promotion of cognitive and metacognitive skills. This finding contradicts the study carried out by Girard, ( 2013 ). This study treats video games as serious games but considers their effectiveness as a controversial issue, finding that only few games result in improved learning, while others have no positive effect on knowledge and skills acquisition, when compared to more traditional methods of teaching.

In contrast, in their meta-analysis, Clark et al. ( 2015 ) systematically review articles to study the detailed effects of digital games on learning outcomes, concluding that games are important in supporting productive learning and highlighting the significant role of gaming design beyond its medium. Prior to this review, but running along the same lines, Backlund and Hendrix ( 2013 ), in their meta-analysis reported positive outcomes in learning when using serious games in the educational process. Wouters, ( 2013 ) performing meta-analytic techniques, used comparisons as well, to investigate whether serious games are more effective and more motivating than conventional instructional methods. They found higher effectiveness in terms of learning and retention, but less motivation compared to traditional instructional methods. Indeed, serious games tend to be more effective if regarded as a supplement to other instructional methods, and involve students in groups and multiple training sessions.

These findings are compatible with those in the survey conducted by Rutten, ( 2012 ), which focuses on implementing games as laboratory activities, concluding that simulations have gained a prominent position in classrooms by enhancing the teacher’s repertoire, either as a supplement to traditional teaching methods or as a partial replacement of the curriculum. Nevertheless, they stress that the acquisition of laboratory skills cannot be wholly conducted via simulations. However, in areas where simulations have been widely accepted as a training tool, simulations can play a significant role in making lab activities more effective when offered as pre-lab training. Fu, ( 2016 ), through a systematic literature review, identify the multi-dimensional positive impact of serious games in business education, with the most frequent outcomes being knowledge acquisition and content understanding. The study also confirms that GBL and serious games can influence player engagement, perpetual and cognitive skills and social or soft skills. The affective and motivational outcomes are examined in entertainment games, games for learning and serious games, which reflects the trend of using gaming elements as both a medium of entertainment as well as a mode of learning. Ritzhaupt, ( 2014 ) produce meta-analysis based on 73 articles, demonstrating that achievement measures (e.g., standardised test scores) are the most commonly investigated, while the second most frequent is affective measures (e.g., usability or attitudes towards technology) followed by behavioural measures (e.g., task behaviour).

Merchant, ( 2014 ), via a meta-analysis, compare the effectiveness of games, simulations and virtual worlds in improving learning outcomes. Findings indicate that playing games individually enhance student performance more than playing collaboratively. Nonetheless, the researchers claim that there is no statistically significant difference between the effects of individual and cooperative instructional modules regarding simulations. Student learning outcomes deteriorate after repeated measures, since after spending a certain amount of time playing games, the learning outcome gains start to diminish. On the contrary, Shin, ( 2015 ), through meta-analysis, aim to identify the effects of patient simulation in nursing education. They find significant post-intervention improvements in various domains for participants who receive simulation education compared to the control groups, thus leading to the conclusion that simulations are more effective than traditional learning methods, enhancing the player’s psychomotor, affective, and cognitive skills. In their work, simulations provide students with authentic clinical situations, allowing them to practice and experience in realistic and safe environments.

Connolly et al. ( 2012 ) develop a multi-dimensional approach to categorising games and offer a review of 129 papers on computer games and serious games, explicitly targeting cognitive, behavioural, affective and motivational impacts, as well as engagement. The most frequent outcomes are knowledge acquisition and content understanding, as well as affective and motivational outcomes. Gegenfurtner, ( 2014 ) in their meta-analysis of the cognitive domain, examine how design elements in simulation-based settings affect self-efficacy and transfer of learning. They conclude that gathering feedback post-training, as opposed to during the process, results in higher estimates of self-efficacy and transfer of learning.

Researchers also look at games and simulations from a theoretical perspective. Li and Tsai ( 2013 ), examine the theoretical background and models employed in the study of games and simulations. They focus principally on the theories of cognitivism, constructivism, enactivism, and the socio-cultural perspective. Results indicate that although cognitivism and constructivism are the major theoretical foundations employed by game-based science learning researchers, enactivism and the socio-cultural perspective are the emerging theoretical paradigms drawing increasing attention in this field. This literature review indicates an increasing recognition of the effectiveness of digital games in promoting scientific knowledge and concept learning, while giving lesser importance to facilitation of problem-solving skills, exploring outcomes from the viewpoint of scientific processes, affect, engagement and socio-contextual learning. This view is echoed by other researchers, such as Warren, ( 2016 ), who systematically review and demonstrate the effectiveness of simulation games on satisfaction, knowledge, attitudes, skills and learning outcomes within nurse practitioner programmes. After comparing online simulation-based learning with traditional lectures, they find an increase in student knowledge and confidence when using simulation games. Peterson ( 2010 ) also performs a meta-analysis, examining the use of computerised games and simulations in language education from a psycholinguistic and socio-cultural viewpoint. Results show valuable opportunities for effective language learning, confirming that games are beneficial in helping students learn another language.

Sitzmann ( 2011 ), using interactive cognitive complexity theory, offers a comparative review on the instructional effectiveness of computer simulations. To perform the review, she examines three affective outcomes (motivation, effort, and self-efficacy), one behavioural (effort), two cognitive (declarative knowledge and retention), and two skill-based learning outcomes (procedural knowledge and transfer). She concludes that, post-training, simulation-trained learners demonstrate higher self-efficacy and procedural knowledge. Furthermore, she highlights the significance of using specific methods to improve simulation learning, namely, integration of game use within an instructional programme, high level of learner activity, no gaming time limit, and adopting the simulation game as a supplement to other methods, which is inconsistent with Wouters et al.’s survey ( 2013 ). Hsu et al. ( 2012 ) provide a cross-analysed content analysis agreeing with the previous researchers that topics such as “Motivation, Perceptions and Attitudes” are of utmost importance.

In a recent review of business literature, Carenys and Moya ( 2016 ) discuss the impact of digital game-based learning (DGBL) on students. They examine DGBL both from a theoretical point of view and on a practical basis through three stages: a) the evaluation of digital games in the preparatory stage, b) specifying which research has been deemed appropriate for DGBL deployment, and c) the learning outcomes (cognitive, behavioural, affective, and multi-dimensional) that can be attained through digital games. This study moved current research forward in understanding the effectiveness of digital games and advanced the use of digital games in the classroom.

A variety of meta-analyses and systematic reviews have examined the implementation of games and simulations in the learning process, either as a main course element or as a supplement to conventional lectures, illustrating the ever increasing interest of researchers in this promising field.

Synthesis of previous reviews/meta-analyses

After studying the previous reviews, it is evident that the most commonly referred games in past reviews are digital and computerized games (Sitzmann, 2011 ; Young et al., 2012 ; Smetana & Bell, 2012 ; Girard et al., 2013 ; Merchant et al., 2014 ; Clark et al., 2015 ; Carenys & Moya, 2016 ; Warren et al., 2016 ). The technological revolution and the invasion of Internet in Higher Education urge students to build digital and collaborative skills for the twenty-first century through gaming. Also, the emergence of a participatory culture in education spurs researchers to get involved with digital games and simulations. Other games mentioned are serious games and their impact on the learning process (Connolly et al., 2012 ; Bellotti et al., 2013 ; Backlund & Hendrix, 2013 ; Wouters et al., 2013 ; Tsekleves et al., 2014 ; Fu et al., 2016 ). The researchers refer to serious games since they are basically considered as games with educational orientation and not with just entertaining ones.

Another important element we have identified is whether games should be fully or partially integrated into the learning process. Most of the researchers agree that games should be treated mainly as supplementary elements (Sitzmann, 2011 ) since full integration requires high-quality mechanisms, students’ engagement, and instructors’ support. In other cases, the integration of games in the curriculum could either function as a supplement to existing teaching techniques or as a partial substitute for traditional teaching methods (Rutten et al., 2012 ). Moreover, games could even be fully integrated for achieving better learning outcomes (Lameras et al., 2016 ) because games add diversity in educational teaching modules. Nevertheless, the integration of games depends on instructors’ contribution and the way they design and incorporate games in their teaching. This means that instructors should be equipped with knowledge and experience, and be aware of providing guidance to students as regards the proper way of playing games.

The beneficial contribution of game-based learning is broadly identified by the majority of previous reviewers, especially regarding cognitive outcomes. Results indicate that games can be as effective as traditional learning modes, revealing their effectiveness in promoting knowledge acquisition (Smetana & Bell, 2012 ; Backlund & Hendrix, 2013 ; Clark et al., 2015 ; Warren et al., 2016 ), as well as content understanding and concept learning (Connoly et al., 2012 ; Li & Tsai, 2013 ; Fu et al., 2016 ). Additionally, students achieve their learning goals through playfulness and problem-based learning (Tsekleves et al., 2014 ), thus leading to self-efficacy and transfer of learning (Gegenfurtner et al., 2014 ).

Another substantial impact emerged is the effectiveness of games not only in the cognitive domain but also in the affective and behavioural domains (Ritzhaupt et al., 2014 ; Shin et al., 2015 ; Tsekleves et al., 2014 ; Fu et al., 2016 ; Carenys & Moya, 2016 ). The affective domain is thoroughly discussed by the reviewers. In particular, games influence students’ motivation, engagement, and satisfaction of the game-based learning. Regarding behavioural outcomes, few reviews have been conducted, showing that games offer a plethora of opportunities for collaborative learning, enhance interactivity and feedback among players, and develop social and soft skills as well. Some other studies contradict these findings, in a way that they do not reveal positive effects of games (Young et al., 2012 ; Girard et al., 2013 ; Merchant et al., 2014 ), or reveal a rather neutral effect (de Smale et al., 2015 ). In these reviews, games and simulations appear to have some or no positive effects on knowledge and skills acquisition when comparing with traditional instructional methods.

Research method

Research selection.

The authors developed a pre-defined review protocol to answer the research questions, specifically aimed at minimising researcher bias. The literature review was carried out between July and October 2016 and followed the design stages described below.

The reviewed papers are identified through keywords in referenced electronic databases, such as Google Scholar, Web of Science, ERIC, PsycInfo, PsycArticles Fulltext Search, InterDok, ProQuest, Scopus, BEI, and SearchPlus. The keywords for learning outcomes are a combination of the term games or simulations paired with the term higher education , employing the Boolean operator “AND”. Additional keywords for learning outcomes are learning objectives, learning goals, learning objectives and effects . Keywords for platform and delivery methods include computer-based, web-based, digital, virtual, online, and technology. Keywords for games and simulations are educational games, business simulations, role-playing simulations, game-based learning, video games, and serious games . Moreover, the Boolean operator “OR” is employed to combine all these keywords. The study sets the broadest range of keywords, so as not to limit the scope of related articles.

Furthermore, the researchers conducted a comprehensive database search in bibliographic indices for the data selection. The search is related to a variety of scientific fields of study, including Education, Psychology, Information Technology, Management, and other scientific areas (e.g., Engineering, STEM, Health, etc).

Assessment and extraction

The dataset consists of journal articles referring to games, simulations or learning in their title and/or abstract. The researchers piloted and evaluated their selection criteria based on prior studies. The study selection process was conducted in two separate phases: a) the researchers, working independently, initially, and, subsequently, together, screened the titles and abstracts for inclusion criteria, and b) in the event of disagreement or insufficient information, they carried out a thorough consideration of the body of the articles (i.e. methodology and results), again independently, first, and, then, together, resulting in consensus. Then, whether to include the text or not was discussed, based on pre-determined criteria. The inclusion criteria used are as follows:

Only empirical articles across a variety of study designs may be included, so as to achieve rich data.

The participants should be over the age of 18 (e.g., students in higher education, college students, instructors, etc.)

Articles that provide an evaluation of student learning outcomes (via the use of games for pedagogical purposes) may also be included.

The resources should, mainly, consist of journal articles and conference papers, which, due to the peer review process, ensure a high quality of material to examine. Existing meta-analysis and systematic literature reviews should be included as well, in order to cross-validate the review findings.

The articles should be available in either English or French.

The articles should adhere to the objective of the study and the definition of the terms games and simulations as pedagogical applications.

Studies containing samples from higher education institutions should also be included. Conversely, research on the effects of games in primary or secondary education should be discarded.

The review should include games and simulations used in traditional, as well as in online environments.

Only peer-reviewed journal articles published between 2010 and 2016 should be included, as the intention is to include the most current research.

Several exclusion criteria, listed below, were also applied in this study:

Non-empirical studies or studies which solely describe the design of a learning environment.

Participants who are younger than 18 years old.

Non-GBL tools and entertainment games.

Book chapters -not only are books difficult to search for on databases, they are, also, hard to access as full texts. Additionally, books are not always subject to the same peer review process as scientific articles. Dissertations, theses, editorials, book reviews and reports are also excluded for similar reasons.

Articles that cannot be accessed as full texts are excluded.

Articles that do not match the research objectives.

Research focused on types of education other than higher education.

As mentioned above, articles published before 2010.

The following figure illustrates the inclusion and exclusion criteria (Fig. 2 ):

The inclusion and exclusion criteria

Application of these criteria resulted in an initial dataset, yielding 8859 studies, published between 2010 and 2016. The selected papers are derived from 67 academic journals representing a variety of disciplines. Most papers come from the scientific journal “Computers & Education”, while “British Journal of Educational Technology”, and “Simulation & Gaming” were the next two journals appearing with the most frequency. In the final stage, several meetings were organized between researchers to discuss the findings, and to decide on presentation.

The results show a steady increase in published papers discussing games from 2012 onwards. After systematically reviewing their abstracts, a final shortlist of 628 potential full text articles emerged. Two hundred and seventeen out of the 628 were excluded, primarily, due to undesirable focus (e.g. theoretical scenarios for using simulations in education). For each of the remaining 411 studies, the researchers identified and recorded some basic themes, for example, types of learning outcomes, effect or impact of game and simulation methods on learning goals, participants and settings, research questions, research methodology and results. Of these, 123 papers, which are found to contribute data, are selected for the review, whereas the remaining 288 articles are excluded, due to the fact that they are written in a language that the researchers do not understand, or because they are focused on a field other than higher education. The majority of these articles are published in scientific journals or conference proceedings, whereas 25 studies are either meta-analyses and/or systematic reviews. An outline of the entire review is depicted in the following figure (Fig. 3 ):

Research review methodological scheme

Data analysis and synthesis

The identified articles are analysed using a qualitative content analysis technique, which leads to a coding scheme, including a main category, three sub-categories and several associated topics related to the main categories. The researchers unanimously agree upon the coding that emerged from the analysis of the reviewed papers. To ensure inter-rater reliability (p) with respect to the quality of article coding procedures, a small random sample ( n  = 20) of the selected articles is coded in duplicate. The calculated reliability exceeds 93%, which is a high quality of agreement across coding categories. Furthermore, a review of mixed-methodology studies provides high-quality evidence, due to a combination of quantitative and qualitative elements in terms of methodological triangulation.

The researchers examined the studies from varying viewpoints. Firstly, they analysed the data set characteristics, such as the continent on which the studies are conducted, the subject discipline, the methodological research design, the types of games and simulations identified, and the time period in which the majority of the studies took place. The emphasis is on the analysis, measures, and design of the quantitative methodology (experimental, quasi-experimental, pre-test, post-test surveys, etc.), as well as the qualitative methods used in the reviewed surveys.

To sum up, the review studies are selected through a systematic process with pre-arranged criteria. There is no intended bias applied to the selected studies, and although the majority of studies come from Europe, this is simply the result of the systematic selection process.

Data set characteristics

When analysing the data, the researchers came across some interesting characteristics. Other than the meta-analytic studies and review research, the locations of the remaining surveys are as follows: 33% conducted in Europe, 22% in Asia, and 18% in the USA, whereas 24% of the articles do not directly mention a location (Fig. 4 ). Most of the articles come from the USA, the UK, and the Netherlands.

Continets where studies are conducted

With respect to genre, there is a diverse representation of games and simulations. The most prominent game genre identified in the relevant literature seems to be simulation games in general, that is to say, virtual/online games or simulations, computer-based learning, role-playing games, serious games, and business simulation games. This representation is illustrated below (Fig. 5 ):

Representation of the game genres

With respect to the busiest publication period, the majority of studies that meet the inclusion criteria were published between 2013 and 2016, as shown in the following bar chart (Fig. 6 ). This finding demonstrates a notable trend amongst researchers discussing the topic of games and simulations in recent years, due to increased awareness of the use of technological games in higher education.

Years of published articles

The data also represents a wide range of subject areas. Some cover multiple areas, for example Engineering, Management, Science, Law, Social Sciences and Humanities (Tao et al., 2015 ), or even just two areas, such as Biology and Computer Sciences (Yang & Chang, 2013 ), while others refer to only one academic discipline. The subject areas are sorted into larger categories, with the most common area being Business Management and Marketing. The results are shown in the figure below (Fig. 7 ):

Subject disciplene

The reviewed articles include data from 99 samples and 20,406 participants, which is a considerably large grouping. The population tested in the literature review ranges from 5 participants in small qualitative studies (Ke et al., 2015 ) to 5071 participants in extensive quantitative quasi-experimental research (Lu et al., 2014 ). Most of the participants are young undergraduate, graduate or post-graduate students, and faculty members. The studies consistently indicate a good gender balance in participants. In some studies, there is both student and faculty participation (Kapralos et al., 2011 ; Felicia, 2011 ; Hess & Gunter, 2013 ; Hämäläinen & Oksanen, 2014 ; Beuk, 2015 ; Crocco, 2016 ), whereas in others, only instructors are chosen as participants (Tanner, 2012 ; Badea, 2015 ; Franciosi, 2016 ). On the whole, most studies use students as participants.

Procedures and research methodologies

Most studies use either an experimental or a quasi-experimental design employing a pre-test and/or a post-test evaluation, with four using only a pre-test questionnaire, and six using only post-test evaluations. The effects of games and simulations on learning outcomes are measured through calculating the difference between pre-test and post-test scores of the experimental or quasi-experimental design. More specifically, the researchers compare the increases in scores between control and experimental groups to evaluate the effectiveness of using the tested games and simulations. The studies include longitudinal surveys (e.g. Hainey, 2011 ) conducted for a specified number of years, whereas others are comparative studies (e.g., Boeker, 2013 ; Poikela, 2015 ).

Researchers use quantitative methods in the majority of studies (68.6%), while13.1% use qualitative methodology. Some studies follow a mixed research methodology (nearly 18.2%), providing pragmatic perceptions and methodological triangulation of the results. The measures utilized in quantitative studies include knowledge questionnaires, as well as academic, evaluation, and cognitive tests, while in qualitative studies the methods used include interviews, case studies, observations and focus groups.

The studies portray a variety of time periods spent playing games and simulations: some of the participants interact with games over a single session, while others are involved in the gaming process for several weeks or even months (e.g., Yang & Chang, 2013 ; Woo, 2014 ). The studies include multi-player games (e.g., Silvia, 2012 ; Yin, 2013 ), as well as single-player games.

Learning outcomes of games and simulations

In the present review, keeping in mind the aforementioned research questions (p.3), the researchers break down their findings in relation to the learning outcomes of games and simulations into three categories, namely cognitive, behavioural, and affective outcomes. A map of the emerging concepts, which will be further discussed, is illustrated below (Fig. 8 ):

Learning outcomes of Games/Simulations

Cognitive outcomes

Many reviewed studies discuss the impact of GBL activities in learner knowledge acquisition and conceptual understanding (Hainey et al., 2011 ; Connolly et al., 2012 ; Fu et al., 2016 ; Geithner & Menzel, 2016 ). There has been an impact evaluation across subject disciplines, such as Computer Science (Strycker, 2016 ), Engineering (Chaves et al., 2015 ), Physics (Adams, 2016 ), Medicine (Dankbaar, 2016 ), Nursing (Sarabia-Cobo, 2016 ), Management (Geithner & Menzel, 2016 ), Political Sciences (Jones & Bursens, 2015 ), Education (Ke, 2015 ), Languages (Franciosi, 2016 ), and Social Sciences (Cózar-Gutiérrez & Sáez-López, 2016 ).

Knowledge acquisition

Cognitive outcomes refer “to the knowledge structures relevant to perceiving games as artefacts for linking knowledge-oriented activities with cognitive outcomes” (Lameras et al., 2016 , p. 10). Tasks framed as games and simulations are deployed to develop a diverse range of cognitive skills, such as deep learning (Vos & Brennan, 2010 ; Young et al., 2012 ; Erhel & Jamet, 2013 ; Crocco et al., 2016 ), critical thinking and scientific reasoning (Beckem & Watkins, 2012 ; Halpern et al., 2012 ; Ahmad, 2013 ), action-directed learning (Lu et al., 2014 ), transformative learning (Kleinheskel, 2014 ), decision-making (Tiwari, 2014 ), knowledge acquisition and content understanding (Terzidou, 2012 ; Elias, 2014 ; Fu et al., 2016 ), spatial abilities (Adams et al., 2016 ), and problem solving (Liu, 2011 ; Lancaster, 2014 ).

The effect of games and simulations on learning remains a controversial issue amongst researchers in the field, as it will be further confirmed in this article. Some reviewed studies indicate improved learning, while others show no positive effect on knowledge and skill acquisition compared to traditional learning methods. The value of simulations can be examined from the perspective of content change as discussed in Kovalic and Kuo’s study ( 2012 ). Simulations are directly linked to the course content and students are given the opportunity to apply and better understand theoretical concepts. Additionally, simulations provide an environment in which students can experiment with different strategies, adopt different roles, and take charge of their own decisions by assuming responsibility. The latter issue is discussed at length by Liu et al. ( 2011 ), who find that, when solving problems, students are more likely to learn via playing a game than via a traditional learning experience.

Serious gaming, especially given the context of enthusiastic students, has proved to be an effective training method in domains such as medical education, for example, in clinical decision-making and patient interaction (de Wit-Zuurendonk & Oei, 2011 ). Similarly, Kleinheskel ( 2014 ) illustrates the importance of designing self-reflective simulating activities for nursing students, and aligning such design with cognitive outcomes. When students self-reflect on simulated clinical experiences, they add to their existing knowledge, and apply new knowledge to transformative learning. Poikela et al. ( 2015 ), in a simulated nursing procedure, compare a computer-based simulation with a lecture to examine the meaningful learning students may achieve via the two teaching methods. They conclude that students who participate in the computer simulation are more likely to report meaningful learning outcomes than those taking the lecture, due to the strong presence of reflection-based activities and metacognitive themes. Similar results are present in Chen, ( 2015 ), survey in which both solitary players and collaborative groups achieve equally positive learning outcomes in a game. Students significantly improve judging by their pre- and post-test assessments, which indicates that the gaming experience affects their overall performance, and, most likely, promotes conceptual understanding. Moreover, collaborative GBL allows students to re-construct and co-construct knowledge, thus encouraging problem-solving through peer discussion.

Challenging games enhance participant performance (Wang & Chen, 2010 ; Gold, 2016 ). This finding is supported by von Wangenheim, ( 2012 ), who analyse the cognitive dimension of an educational game focusing on memory, understanding and conceptual application. The validity of micro-simulation games is identified by participants in Lukosch, ( 2016 ), research who evaluate a specific microgame as an excellent instrument for enhancing situated and experiential learning by transferring knowledge to an actual situation at the workplace. The results comply with those of Riemer and Schrader ( 2015 ), where the application of comprehension and transfer of knowledge are best achieved using simulations.

Furthermore, the impact of game-based learning on learning performance has been observed by numerous researchers across diverse subjects, as reported above (Zacharia & Olympiou, 2011 ; Rutten et al., 2012 ; Beckem & Watkins, 2012 ; Boeker et al., 2013 ; Shin et al., 2015 ; Hou, 2015 ; Chen et al., 2015 ; Tao et al., 2015 ). For instance, Divjak and Tomić ( 2011 ) provide evidence that computer games impact mathematical learning, revealing the positive effect of games on student learning outcomes. Reviews by Young et al. ( 2012 ) confirm the effectiveness of using videogames on History, Languages, and Physical Education. The analysis of four experimental virtual conditions in pre- and post-test assessments reveal that virtual experimentation promotes conceptual understanding in Physics students (Zacharia & Olympiou, 2011 ). A 3D visualisation and simulation laboratory activity on protein structure is more effective than traditional instruction modules, as described in White, ( 2010 ), research resulting in students preferring to work with visualized simulations.

Simulation games also positively affect clinical practice situations. “The Ward”, a simulation game in Stanley and Latimer’s ( 2011 ) research proves to be an enjoyable and valuable learning tool in addressing clinical skill practice, nursing practice knowledge, critical thinking and decision-making. Vos and Brennan ( 2010 ) highlight the effectiveness of marketing simulation games, where students perceive simulations as an enjoyable learning approach, contributing to decision-making, as well as other valuable knowledge and skills, a finding consistent with Tiwari et al. ( 2014 ) survey. Swanson et al. ( 2011 ) created a rubric to measure the effectiveness of teaching strategies in nursing education. The experimental post-test assessment survey aims to evaluate the effects of three teaching strategies on the outcome of performance and retention of intervention activities, student satisfaction, self-confidence and practical educational preferences. Results reveal significantly higher retention scores compared to the first assessment, indicating that high scores in the improved rubric are related to the interactivity of the simulation scenario.

Nevertheless, it should not be taken for granted that students consistently prefer virtual learning settings to more traditional face-to-face environments (Hummel et al., 2011 ). Serious games concerning cognitive perceptions show varying results. For example, simulations are shown to support the comprehension and application of knowledge, albeit less effectively than quizzes and adventures (Riemer & Schrader, 2015 ). In Fu et al. ( 2016 ) review, despite GBL providing a motivating and enjoyable experience, there is a lack of strong evidence to show that games lead to effective learning outcomes. In some cases, there is inconsistency in student views regarding the integration of online games as a positive learning method (Bolliger, 2015 ). Similar views are supported by some researchers, who acknowledge students’ and educators’ hesitation towards virtual simulations and serious games, but they insist on the inclusion of games into course material, and on instructors’ familiarization with their use (Kapralos et al., 2011 ).

Perceptual skills

Other studies confirm the power of games and simulations in developing cognition abilities, especially in the instances of virtual simulations enhancing complex cognitive skills (Helle et al., 2011 ; Siewiorek, 2013 ), such as self-assessment (Arias Aranda, 2010 ), or higher-order thinking (Crocco et al., 2016 ). These are meta-cognitive skills, regarded as essential elements of in-depth learning. The incorporation of game mechanisms into simulations is widely recognised by researchers as beneficial, especially regarding laboratory tasks, where simulation scenarios urge students towards problem-solving and, reflection, thus achieving metacognitive outcomes (Hou & Li, 2014 ; Hou, 2015 ). Kikot, ( 2013 ) concur with the above researchers, stating that students perceive simulation-based learning (SBL) environments positively when asked to achieve dynamic learning outcomes, including thinking, interpreting, and associative skills.

Silvia ( 2012 ) also references cognitive and metacognitive outcomes derived from a multi-role simulation. The simulation helps students apply the concepts they learn in class by connecting the theoretical issues with real-world situations, thus developing their analytical skills, and through comparing different viewpoints, which leads to enhanced critical thinking. Students use the interactive nature of simulations to develop arguments, make judgements and evaluate situations. More importantly, simulations encourage students to develop self-awareness. Similarly, Cela-Ranilla, ( 2014 ) conducted a study in which students display a tendency to perform better in analytical work, such as monitoring, planning and assessment rather than in action-based work. Wouters et al. ( 2013 ), on the other hand, find serious games to be more effective in terms of learning and retention.

Learners can also actively participate in a web-based simulation to facilitate immersion and reflection, leading to deeper understanding of the content (Helle et al., 2011 ). A simulation framework can facilitate learning in terms of flow experience and learning strategies. Indeed, in a study conducted by Li, Cheng, and Liu ( 2013 ), the framework helps students lacking background knowledge to balance challenge and skill perceptions, while for students with average to advanced levels of knowledge, it facilitates the learning experience by either reducing the challenge perception or promoting the skill perception. Along the same lines, Pasin and Giroux ( 2011 ), analyse the mistakes students make in simulations using an empirical prototype. Results show that, although simple decision-making skills are easily acquired through conventional teaching methods, simulation games are useful tools for mastering managerial skills, such as complex and dynamic decision-making. Lin and Tu ( 2012 ) also confirm that simulations enable students to train themselves in decision-making.

Instructors’ engagement

Students are challenged to develop interpersonal, analytical and creative skills, discouraging absenteeism, feelings of boredom and reluctance, leading to academic achievement. However, simulations not only exhibit positive effects in the learning experience of the student, but, also, do so for instructors, as well, in the context of teaching experience. For academics, simulations raise the level of performance, encouraging students to be more alert and attentive during class activities (Navidad, 2013 ), and thus to achieve better learning outcomes. In this vein, instructors are urged to design simulations to be as challenging as possible to stimulate student interest in interacting with the simulation as well as with their peers. Felicia ( 2011 ) denotes that instructors agree with students in acknowledging the educational benefits of video games, such as an understanding of difficult concepts, improvement of spatial awareness and analytical skills, critical thinking, and problem-solving strategies. To enable them to do so, instructors emphasize the importance of clearly expressed learning goals to guide students when using simulations in an online instructional technology course (Kovalik & Kuo, 2012 ).

Even setting aside the potential learning benefits derived from participation in GBL, a stronger connection between games and curricula remains to be forged, as well as the application of more dynamic academic challenges, so as to better adapt to the knowledge of diverse learners (Pløhn, 2013 ). Following such reasoning, as indicated in the literature, faculty plays a key role in achieving learning goals via the use of games and simulations. The instructor role correlates with the demand for abstract learning concepts. In their meta-analysis, Wouters and Van Oostendorp ( 2013 ) show how instructors, acting in a facilitating and supporting role, can foster learning, particularly in selecting and discussing new information and where higher order skills are involved in the learning outcomes. Similarly, instructors can monitor student behaviour and evaluate not only the capabilities, but also the attitudes of tomorrow’s higher education managers during the decision-making process. Rutten et al. ( 2012 ) focus in their literature review on the level of instructional support in GBL, and suggest that a pedagogical framework for the application of computer simulations in education requires a corresponding integration of the educator’s role.

Behavioural outcomes

Behavioural objectives for higher education students refer to the enhancement of teamwork and improvement in relational abilities (Ranchhod, 2014 ), as well as stronger organisational skills, adaptability and the ability to resolve conflicts (Vos & Brennan, 2010 ).

Social skills/teamwork

Simulation games are often seen as powerful tools in promoting teamwork and team dynamics (Stanley & Latimer, 2011 ; Tiwari et al., 2014 ; Lin, 2016 ; Wang, 2016 ), collaboration (Hanning, 2012 ), social and emotional skills (Ahmad et al., 2013 ), and other soft skills, including project management, self-reflection, and leadership skills (Siewiorek, 2012 ; Wang et al., 2016 ), which are acquired through a reality-based scenarios with action-oriented activities (Geithner & Menzel, 2016 ).

In a Spanish management course, simulations enabled students to build pivotal capacities, such as management abilities and team working to enable the success of future managers (Arias Aranda et al., 2010 ). A computer simulation at a university in Taiwan led to comparatively higher learning gains against traditional teaching through collaborative laboratory activities (Shieh, 2010 ), by facilitating students to carry out more active learning and improving their conceptual understanding. Simulation scenarios provide improved social and communication skills, which lead to the enhancement of student knowledge (Sarabia-Cobo et al., 2016 ).

Additionally, collaboration is considered an essential element in the learning process (Elias, 2014 ). The findings of Hummel et al. ( 2011 ) reveal that serious online games improve the quality of learning when it comes to problem-based situations in the workplace by using active collaboration. For this reason, faculty members are urged to create learning environments to support active participation by students in the educational process. Moreover, according to the constructivist approach, the instructor’s role is a significant factor in empowering groups to construct knowledge in a collaborative manner (Hämäläinen & Oksanen, 2014 ). The instructors engage higher education students in the process of formulating hypotheses, interpreting context, providing explanations, and describing observations, by designing and implementing a collaborative and interactive GBL environment. In Yin et al.’s study ( 2013 ), students react positively to participatory simulations, due to the belief that the system helps them advance their conceptual understanding effectively through scaffolding, discussion, and reflection. Participants in Cózar-Gutiérrez and Sáez-López’s study ( 2016 ), while stating that video games are non-essential tools in an educational context, nevertheless, value GBL as an immersive environment that facilitates increased activity and student engagement.

Teamwork, however, seems to be a controversial issue in Costa, ( 2014 ) which evaluates improvement of knowledge sharing. Some learners consider teamwork as a means to facilitate decision making in a game, while others express dissatisfaction due to their peers, be it the latter’s reluctance to take on responsibility or poor negotiation capabilities. Research by Bolliger et al. ( 2015 ) similarly indicates that some learners remain hesitant, as they feel the use of games may actually decrease opportunities for communication with peers and instructors. Merchant et al. ( 2014 ) conclude that student performance is enhanced when playing individually rather than in a group.

Interaction and feedback

In GBL methods, meaningful feedback is a key factor in students achieving the objectives, as well as in being encouraged to reflect on misunderstandings and to transfer learning to new educational contexts (Swanson et al., 2011 ). In the current study, the scope is to investigate learner-learner interaction and social feedback through game mechanics. Higher education students evaluate games and simulations focusing on behavioural change and improvement of interactive abilities. The computer game DELIVER! for example, is evaluated very positively by students due to its focus on active student participation and overall positive impact on social interaction (von Wangenheim et al., 2012 ). Simulations provide visual feedback, encouraging active exploration of the student’s own understanding, enabling a move beyond knowing-in action and beginning to reflect-on and in-action during training, resulting in the contextual application of prior knowledge (Söderström, 2014 ). Real-time feedback in simulation games enables students to clearly define the objectives and expectations in the interactive environment, leading to a reduction in anxiety and uncertainty, thus encouraging better performance (Nkhoma et al., 2014 ).

The literature extensively documents the interaction between behavioural outcomes, learning performance and communication especially in Online Distance Learning (ODL). Indeed, regular feedback on student performance during DGBL facilitates deep learning (Erhel & Jamet, 2013 ). A survey conducted by Chen, ( 2010 ) shows that online games can be social and interactive technologies, helping students form friendships with their peers and providing multiple types of interaction.

Ke et al. ( 2015 ) stress the importance of player interaction, indicating that the inherent interaction between players and their gaming-situated learning environment supplies structured challenges and feedback. Huang, ( 2010 ) share the same view, confirming that, due to the necessity of receiving feedback from peers and the game itself, increased interaction opportunities arise in game-play, adding that interaction is a decisive factor in the construction of knowledge (Seng & Yatim, 2014 ). In a survey conducted by Denholm et al. ( 2012 ), students report improved team working through the use of serious games. They attribute this to receiving feedback, and stressing that even conflict is often considered valuable as it brings diverse views to the fore.

To conclude, the main body of literature explores the impact of games and simulations on learning outcomes on the behavioural level, especially when students are involved in interactive and participatory simulation tasks. The majority of studies reveal a positive effect on behavioural outcomes, concluding that students benefit from appropriate feedback, and reflection through game-based communication activities.

Affective outcomes

Many studies highlight the affective outcomes of using games and simulations in the learning process. The majority of them includes student engagement (Auman, 2011 ; Hainey et al., 2011 ; Lin & Tu, 2012 ; Kikot et al., 2013 ; Lu et al., 2014 ; Ke et al., 2015 ), motivation (Liu et al., 2011 ; Liao & Wang, 2011 ; Costa et al., 2014 ; Lukosch et al., 2016 ), and satisfaction (Cvetić et al., 2013 ; Dzeng, 2014 ; Lancaster, 2014 ; Sarabia-Cobo et al., 2016 ).

Motivation and engagement

Engagement and motivation are major factors in enhancing higher education learning objectives (Connolly et al., 2012 ; Erhel & Jamet, 2013 ; Ke et al., 2015 ; Nadolny & Halabi, 2015 ). Motivation is considered a central factor in the majority of reviewed studies (Felicia, 2011 ; Ljungkvist & Mozelius, 2012 ; von Wangenheim et al., 2012 ; Bellotti et al., 2013 ; Hannig et al., 2013 ; Ahmad et al., 2013 ; Pløhn, 2013 ; Li et al., 2013 ; Denholm et al., 2012 ; Dzeng et al., 2014 ; Lancaster, 2014 ; Ariffin et al., 2014 ; Bolliger et al., 2015 ; Cózar-Gutiérrez, & Sáez-López, 2016 ; Dankbaar et al., 2016 ; Fu et al., 2016 ). Some results suggest the effectiveness of GBL in motivating and achieving learning goals can be found at the lower levels of Bloom’s taxonomy (e.g. Connolly et al., 2012 ). In the context of digital SBL environments, other motivational dimensions are highlighted, such as self-efficacy (Sitzmann, 2011 ), in conjunction with the transfer of learning (Gegenfurtner et al., 2014 ).

Motivation is a combination of elements such as attention, relevance, confidence, and satisfaction, which can increase germane cognitive loads. Chang, ( 2010 ) examine the effects of motivation in an instructional simulation game, called SIMPLE. According to the post-game evaluation, student motivation comes from peer learning and user cooperation. Moreover, when instructors teach strategy, this enhances student motivation and engagement, encouraging acceptance of the game, and leading to stronger interest in course-directed learning. Thus, teachers should create a flexible learning environment, giving due consideration to peer interaction, learning motivation, pedagogical support and encouragement to help students develop their autonomy and retain an interest in learning.

Another important element contributing to affective outcomes is challenge. Hainey et al. ( 2011 ) find the presence of a challenge to be the top ranked motivation for online game players, while recognition is the lowest ranked motivation regardless of gender or amount of players in the game. Gamers in a multiplayer environment tend to report competition, cooperation, recognition, fantasy and curiosity when playing games, while online players experience challenge, cooperation, recognition and control. By contrast, fanatical computer game players experience disappointment and a lack of challenge, as they tend to value the technical aspect over the challenges presented by game play. In Hess and Gunter’s survey ( 2013 ), students in a game-based course are motivated, because of the positive social interaction they experience while playing the game; this intrinsic motivation is positively correlated to student performance. Computer games can thus be seen as a learning tool motivating players to acquire many competences. Connolly et al. ( 2012 ) share the same view, seeing the role of challenge as a predictive factor with respect to game engagement and achievement. Similarly, in Ke et al.’s study ( 2015 ), the game-play actions include optimal challenge expectation for the user. These results can also be seen in Badea ( 2015 ), who concludes that the majority of participants in her study acknowledge the highly motivating quality of games, which are complemented by the relaxed class atmosphere when games are used.

However, despite the benefits reaped from the implementation of games and simulations concerning affective outcomes, some researchers underline that motivation is not always related to GBL, emphasizing cases where students who use games in solitary or collaborative environments experience no significant difference in terms of learning motivation (Chen et al., 2015 ). There are indeed cases where serious games are no more motivating than conventional instructional methods (Wouters et al., 2013 ). In Cela-Ranilla et al.’s survey ( 2014 ), despite the suitability of the 3D simulation environment, students do not feel highly motivated or particularly engaged, mostly because they prefer analysis to actions in the particular learning process.

Faculty role

The benefits of a pedagogical shift from a teacher-focused and lecture-based classroom to a student-centred, active-learning environment through the adoption of simulation-based strategies to achieve engagement are relevant to both students and instructors (Auman, 2011 ). There is a progression in student emotion from uncertainty and nervousness to satisfaction and excitement within the gaming experience. Auman ( 2011 ), as an instructor, provides a positive description: she is drawn in by student enthusiasm, her interest in the material is reinvigorated, she feels empowered in her teaching, and ready to guide her class. In this context, it’s easy to see how instructors ought to play a significant role in motivating and engaging students to achieve learning goals. De Porres and Livingston ( 2016 ) concur with Auman ( 2011 ), as their study also indicates increased levels of excitement in doctoral students studying Computer Science, when evaluated in a post-test intervention.

Faculty acting as motivators are key in engaging students in the learning process, working to ensure focus on pre-existing knowledge, as well as to transfer knowledge to game settings (Lameras et al., 2016 ), to reward students for their effort, and support them by providing continuous guidance and pathways for further consideration. The quality of the teacher/facilitator has a strong influence on the learning satisfaction of the students. Also, instructors should facilitate and engage students via in-game discussion forums to help overcome misconceptions, and to lead the game-based learning. The way instructors interact, facilitate and motivate students to construct GBL experiences depends on the design stage, particularly on the way games are incorporated into the curriculum in a traditional course (Wouters et al., 2013 ). This is because motivation exhibits a significant correlation with cognitive and skill performance (Woo, 2014 ). In research conducted by Franciosi ( 2016 ), despite faculty acknowledging the beneficial impact of games on student motivation, they nevertheless, remain doubtful about the effectiveness of games in learning outcomes, thus resulting in neutral attitudes. Interestingly, although instructors perceive simulations as engaging learning technologies, they do not however consider them superior to traditional teaching methods (Tanner et al., 2012 ).

Another aspect, less frequently discussed in the relevant literature, is students’ performing self-assessments with regard to effective learning, as seen in Jones and Bursens study ( 2015 ). This ability is supported by constructivism, since simulations are developed in an active learning environment, where faculty act more as facilitators rather than as instructors and students are provided with feedback to carry out their self-assessments.

Attitudes and satisfaction

A vital element in achieving learning goals is the relationship between motivational processing and the outcome processing (satisfaction), especially in an online instructional game, as seen in the experiment carried out by Huang et al. ( 2010 ). There seems to be a significant relation between these two variables, which suggests that designers of DGBL need to consider extrinsic rewards to achieve motivational development and satisfaction. Learning satisfaction is strongly correlated with student motivation and attitude towards GBL before the game, with actual enjoyment and effort during the game, as well as with the quality of the teacher/facilitator (Mayer, 2013 ). Specifically, students with a higher level of inner motivation and positive attitude towards GBL are more likely to have higher learning expectations, and to experience more satisfaction in their GBL participation.

In general, most studies report that students develop a positive attitude toward the pedagogical adoption of games and simulations in education (Divjak & Tomić, 2011 ; Bekebrede, 2011 ; Ibrahim et al., 2011 ; Beckem & Watkins, 2012 ; Tanner et al., 2012 ; von Wangenheim et al., 2012 ; Halpern et al., 2012 ; Terzidou et al., 2012 ; Hanning et al., 2013 ; Giovanello, 2013 ; Cvetić et al., 2013 ; Kovalik & Kuo, 2012 ; Li & Tsai, 2013 ; Hainey et al., 2011 ; Boeker et al., 2013 ; Nkhoma et al., 2014 ; Costa et al., 2014 ; Chaves et al., 2015 ; Riemer & Schrader, 2015 ; Angelini, 2016 ; Geithner & Menzel, 2016 ). The participants in Dudzinski et al. ( 2013 ) respond positively towards a serious web-based game, describing the experience as interesting, stimulating and helpful, as well as a valuable addition to their pharmacy curriculum. Other students perceive simulation games as fun, but not particularly useful as an instructional method compared to lectures, and about equally useful as case discussions (Beuk, 2015 ). In another study, the majority of students show a positive attitude towards games, positing that they make subjects more fun and provide more opportunities for learning (Ibrahim et al., 2011 ). This finding is consistent with Bekebrede et al. ( 2011 ) on the perceptions of Dutch students belonging to the “net generation”, who have been raised with technology-based games. Data reveals student preference towards active, collaborative and technology-rich learning via digital games that bring added value to the educational process.

For students, satisfaction is a deciding factor in their decision to continue using such learning methods (Liao & Wang, 2011 ; Liao, 2015 ). Terzidou et al. ( 2012 ) discuss affective outcomes, especially the way interviewees feel before and after their participation in the game. Prior to participating, the interviewees report feelings of entertainment, fascination, and satisfaction before their participation in the game, which increase after use, indicating that participants find the use of 3D virtual game appealing.

Chen et al. ( 2010 ) reveal that the majority of students show negative feelings about online gaming. Shieh et al.’s ( 2010 ) mixed methodology research reveals that experimental groups show positive attitudes toward an innovative learning environment and outperform the control groups (in conventional classes). Some studies depict either neutral effects (Rajan et al., 2013 ; Beuk, 2015 ; Bolliger et al., 2015 ; Dankbaar et al., 2016 ; Strycker, 2016 ) or negative attitudes towards game use in the learning experience (Jiménez-Munguía & Luna-Reyes, 2012 ). Students experience more anxiety and boredom during conventional courses, which acts as an impediment to acquiring substantial problem-solving skills. The educational benefits of GBL are particularly apparent in subjects over which students report greater anxiety, where it can be proven that increased enjoyment levels correlate positively with improvements in deep learning and higher-order thinking (Crocco et al., 2016 ). Liarokapis, ( 2010 ) show Computer Science students evaluating a serious online game, and finding it a valuable pedagogical tool, which is both useful and entertaining.

Genre/familiarity issues

Students achieving high scores respond more positively to online games compared to low achieving students. Regarding genre perceptions, male students express more enthusiasm towards digital gaming than female students, or at least spend more time playing computer games compared to girls (Hainey et al., 2011 ). This may be due to the fact that boys tend to be more familiar with computers and web-based technologies. Girls may choose to avoid digital game-based learning methods, due to their negative preconceptions about gaming, preventing them from harnessing the positive aspects of online gaming (Chen et al., 2010 ). These studies indicate a difference in perception based on gender when engaging in DGBL environments. However, research by Riemer and Schrader ( 2015 ) concluded that female students reported a more positive attitude and perception of affective quality compared to the male students. Also, high assessment scores in web-based games depend on the professional experience of the players. Unexpectedly, in Dzeng et al.’s experimental survey ( 2014 ), despite the high test scores achieved in both web-based and paper-based games, students without work experience achieve the highest post-test scores, probably because they are more familiar with using technological tools. The experiments in Erhel and Jamet’s study ( 2013 ) indicate that serious games promote learning and motivation, provided they include features that prompt learners to actively process the educational content.

To sum up, games and simulations lead to improved affective outcomes for university students such as attitudes, motivation, emotional involvement, self-efficacy and satisfaction. A growing body of literature supports the positive attitude shown by students towards games and simulations, as they consider them essential instructional tools that provide motivation and engagement in an active learning environment.

Research interest in the incorporation of games and simulations in higher education is constantly developing (Girard et al., 2013 ). The pedagogical shift, from lecture-centred to student-centred environments and the increasing use of games as innovative learning technologies, calls for a transformation in higher education. In this respect, games and simulations are expected to play a significant role in the learning process. In the present study, the focus is on the positive effects of games and simulations on university students’ learning outcomes. The reviewed papers are diverse in terms of research objectives, theoretical background, methodological avenues adopted, game genres, scientific domain or delivery platform, and various perspectives concerning cognitive, behavioural and affective outcomes employed. Many articles ( n  = 123) are identified, providing either empirical results or offering meta-analytic evidence.

There seems to be a lack of shared definitions or taxonomy necessary for a common classification, which, therefore, results in terminological ambiguity (Klabbers, 2009 ). The majority of GBL researchers compare the effectiveness of implementing web-based learning games to conventional instructional options (Shin et al., 2015 ).

Mapping the results, empirical evidence is identified with respect to cognitive learning outcomes including knowledge acquisition, conceptual application, content understanding and action-directed learning. Games and simulations are educational interventions, which create a supportive environment in which students may acquire knowledge across subjects and disciplines. Students have the opportunity to better understand theoretical concepts, provided that games are used as a supplement in traditional lecture-based courses. Additionally, simulations are often perceived as enjoyable learning tools, which require active and collaborative participation and contribute to the improvement of critical thinking and reasoning, higher-order- and metacognitive thinking. Simulations provide students the opportunity to observe the outcomes of their actions, and take responsibility for decision-making via problem-solving competencies, thus leading to a more active, transformative and experiential reception of knowledge.

Another important finding is that simulations have positive effects on both students and instructors. Positive outcomes exist when instructors set achievable learning goals, interact with students promoting knowledge, support, facilitate, and motivate them to construct new game-based knowledge (Kovalik & Kuo, 2012 ; Lameras et al., 2016 ). Instructors are encouraged to design games and simulations in order to make students fully aware of game activities, providing all the while continuous instructional guidance. These results generally confirm the findings from prior systematic reviews and meta-analyses. However, findings diverge slightly in Young et al.’s survey ( 2012 ), who claim that there is limited or no evidence about the effective implementation of games in the lecture-based curriculum.

This review also covers behavioural outcomes, mainly the development of social, emotional, and collaborative skills, helping students to foster strong relationships with peers, empowering them to collaborate and work in groups more efficiently, become organised, adapt to new tasks, and resolve emerging conflicts. Furthermore, reality-based scenarios and action-oriented game activities promote fruitful interactions and meaningful feedback, which leads to collaborative construction of knowledge. Overall, digital games and simulations urge students to interact not only with the game, but with their instructors and co-players as well. These results have been extensively covered in the literature review, with the majority of researchers agreeing with the current study’s results, confirming the positive effects of games and simulations on the behavioural level of learning outcomes (Bellotti et al., 2013 ; Tsekleves et al., 2014 ; Fu et al., 2016 ; Carenys & Moya, 2016 ).

However, although most reviews acknowledge the positive effects of games in behavioural outcomes, some reviewed studies contradict these positive findings, claiming that teamwork is a controversial issue when it comes to the improvement of knowledge sharing. The use of games seems to decrease opportunities for peer interaction and communication with instructors (Bolliger et al., 2015 ), whereas playing individually is sometimes considered better than working in a team (Merchant et al., 2014 ). Also, in some cases, games and simulations through collaborative activities distract students and hinder learning (Dankbaar et al., 2016 ).

The current review makes a significant contribution by investigating the affective outcomes when incorporating games and simulations in the curriculum, especially motivational and engagement outcomes, emotional development, satisfaction, attitude, emotion, self-assessment, and self-efficacy. Results show that games and simulations motivate, engage and promote effective learning goals by providing opportunities for learners to actively experience, practice, interact, and reflect in a collaborative, game-based, and learner-centred setting. The measures evaluating student attitudes reveal an increasingly positive trend towards games and simulations, especially in post-interventions (Bekebrede et al., 2011 ; Giovanello et al., 2013 ; Costa et al., 2014 ; Angelini, 2016 ; Geithner & Menzel, 2016 ).

To this end, there has been a purposeful highlighting of the instructor’s role as facilitator and motivator in this literature review. Through in-game activities and extended discussion, instructors promote student interaction and help them overcome the lack of understanding of content curriculum and achieve better learning outcomes. The literature also stresses the role of emotional development, which facilitates improvement of learning outcomes. Specifically, there seems to be a progression in student emotion, from negative feelings including uncertainty, anxiety, nervousness, and disappointment during pre-intervention, to positive feelings of satisfaction, confidence, excitement, enjoyment, effort, fascination, and enthusiasm during in-game and post-game interventions (Huang et al., 2010 ; Hummel et al., 2011 ; Liao & Wang, 2011 ; Terzidou et al., 2012 ; Woo, 2014 ; Liao et al., 2015 ).

Most of the pre-existing evidence is compatible with the findings of this systematic review (Sitzmann, 2011 ; Connolly et al., 2012 ; Wouters et al., 2013 ; Ritzhaupt et al., 2014 ; Gegenfurtner et al., 2014 ; Shin et al., 2015 ; Lameras et al., 2016 ; Carenys & Moya, 2016 ; Fu et al., 2016 ; Warren et al., 2016 ). Nevertheless, one study indicates that the overall positive perception of students depends on the different forms of games (Riemer & Schader, 2015 ), namely, simulations promote a less positive effect compared to quizzes and adventures. Some other studies diverge further in their findings, indicating either neutral (Rajan et al., 2013 ; Strycker, 2016 ; Franciosi, 2016 ) or negative student attitudes towards the use of games (Chen et al., 2010 ; Jiménez-Munguía & Luna-Reyes, 2012 ). Also, there are limited results on the effect of games on student self-efficacy, with one study demonstrating moderate post-training self-efficacy (Sitzmann, 2011 ).

Comparing the findings of the current study with the findings of previous systematic reviews and meta-analyses leads to an interesting discussion. The results of the present review illustrate that the majority of the revised articles focus on different genres of games and simulations. The mostly represented genres are virtual/online games and simulations since they can enhance learning in certain disciplines, such as Computer Studies. This finding is in agreement with most of the previous reviews (e.g. Clark et al., 2015 ; Carenys & Moya, 2016 ; Warren et al., 2016 ). Also, simulation games are found to be popular in this review, due to the fact that they are implemented in authentic learning environments, namely in Health Sciences and Biology. Also, in this study, a great representation of role - playing games and business simulation games are obviously resulted from the previous articles, due to the fact that they are implemented in specific academic disciplines, such as Business Management and Marketing. Nevertheless, in this review, serious games are not represented as much as in other reviews (e.g.Tsekleves et al., 2014 ; Fu et al., 2016 ).

Additionally, this study concentrates on the positive effects of games and simulations on learning outcomes, a finding that is compatible with previous reviews (e.g. Bellotti et al., 2013 ; Lameras et al., 2016 ; Clark et al., 2015 . This review confirms that games and simulations contribute to cognitive learning outcomes, including knowledge acquisition, conceptual application, content understanding, and action-directed learning. Other previous reviewers echoed these findings (Smetana & Bell, 2012 ; Shin et al., 2015 ; Wouters et al., 2013 ; Fu et al., 2016 ) emphasizing the important role of games in knowledge acquisition and content understanding. It has been illustrated that university students benefit from the incorporation of games into the learning process, if used as a supplement in traditional lectures, a finding that complies with other reviews (Sitzmann, 2011 ; Wouters et al., 2013 ). However, simulations’ implementation is influenced by instructors’ guidance and motivation, as these factors encourage faculty to design simulations to achieve learning outcomes.

This review also sheds light on behavioural outcomes of using games in instructional design. The emphasis is on the positive effects, namely the development of social and soft skills, emotional skills, the empowerment of collaboration with peers, and the promotion of interaction and feedback, findings that are in line with past reviews (Shin et al., 2015 ; Carenys & Moya, 2016 ). Despite the positive behavioural effects of utilizing games, some reviews find collaboration and teamwork as a hindrance for learning. The application of games seems to decrease peer interaction and communication with faculty, whereas in Merchant et al.’s review ( 2014 ), playing individually is more preferable than playing collaboratively. The current review concludes by highlighting the affective outcomes, and the emphasis is given on motivational and engaging factors that lead to emotional development, satisfaction, self-efficacy and self-assessment, findings that are also documented in other reviews (Sitzmann, 2011 ; Hsu et al., 2012 ; Tsekleves et al., 2014 ).

To conclude, this review discusses the multitude of surveys on the cognitive, behavioural, and affective outcomes related to the use of playing games and simulations in higher education. The multi-dimensional analysis of the empirical data provides a framework for understanding the major outcomes of GBL. Despite the significant benefits in learning outcomes highlighted in this paper, the high cost of designing games and simulations is still a significant challenge. To overcome this cost barrier, governments, researchers, instructors, and game designers should collaborate to find affordable solutions, for enabling the development of games and simulations. Since this review does not concern itself with advanced aspects of learning, the focus should next turn to a metacognitive-oriented survey, which will study the promotion of metacognitive skills in students, such as self-regulation, self-reflection, self-awareness, evaluation, planning, building on the ideas of others, debating, and so forth.

Future research

Considering the above discussion points, and the importance of games and simulations as derived from the relevant literature, some suggested avenues for future research are as follows:

Researchers should focus on applying the relevant theoretical frameworks, such as cognitivism, constructivism, and socio-cultural perspectives to cognitive, behavioural and affective outcomes, respectively.

More research should be conducted investigating gender issues with respect to the effectiveness of games on developmental aspects of behaviour, such as scaffolding and immersion, to counteract the present gap in the existing literature.

Comparative surveys should be included with a design focused on different target groups (adult students, or K-12 students in laboratory conditions).

Evaluation models via a mixed-method design are encouraged, especially to investigate how game designers could tailor game designs to applying different learning preferences and styles.

University instructors should take a more active role in the alignment of games with the curriculum ensuring that games and simulations are implemented in a blended learning module (face-to-face, online material, etc.), or even acting as games masters, scaffolding virtual experiences to university learners.

Faculty should design games with a view to multiplayer cooperation to achieve effectiveness in learning outcomes. Students should also be involved as co-designers, recommending innovative ideas and radical approaches in an effort to meet their own needs. An innovative approach is the adoption of metagames (Young et al., 2012 ), which consist of additional learning resources (blogs, wikis, etc.) encouraging collaboration between players.

This study makes a significant contribution to research, since no other literature review or meta-analysis has been conducted so far investigating educational and web-based games and simulations with such an extensive subject and discipline coverage in higher education. Today’s demand for student-centred teaching methods to develop highly qualified learners, capable of learning in an active and collaborative environment, calls for the deployment of game-based activities and simulations that will enable them to face the challenges of the dawning era.

Adams, D. M., Pilegard, C., & Mayer, R. E. (2016). Evaluating the cognitive consequences of playing portal for a short duration. Journal of Educational Computing Research, 54 (2), 173–195.

Article   Google Scholar  

Ahmad, S. M. S., Fauzi, N. F. M., Hashim, A. A., & Zainon, W. M. N. W. (2013). A study on the effectiveness of computer games in teaching and learning. International Journal of Advanced Studies in Computers, Science and Engineering, 2 (1), 1.

Google Scholar  

Aldrich, C. (2009). Virtual worlds, simulations, and games for education: A unifying view. Innovate: Journal of Online Education, 5 (5), 1.

MathSciNet   Google Scholar  

Anderson, E. F., McLoughlin, L., Liarokapis, F., Peters, C., Petridis, P., & Freitas, S. D. (2009). Serious games in cultural heritage. In M. Ashley & F. Liarokapis (Eds.), VAST 2009: 10th International Symposium on Virtual Reality, Archaeology and Cultural Heritage 22-25 September 2009 (pp. 29–48). St. Julians, Malta: Eurographics Association.

Andreu Andrés, M. A., & García Casas, M. (2011). Perceptions of gaming as experiential learning by engineering students. International Journal of Engineering Education, 27 (4), 795–804.

Angelini, M. L. (2016). Integration of the pedagogical models “simulation” and “flipped classroom” in teacher instruction. SAGE Open, 6 (1). doi: 10.1177/2158244016636430 .

Angelini, M. L., García-Carbonell, A., & Martínez-Alzamora, N. (2015). Estudio cuantitativo discreto sobre la simulación telemática en el aprendizaje del ingles. [Quantitative study about telematic simulation in learning]. RIE Revista Iberoamericana de Educación, 69 (2), 51–68.

Arias Aranda, D., Haro Domiguez, C., & Romerosa Martinez, M. M. (2010). An innovative approach to the learning process in management: The use of simulators in higher education. REVISTA DE EDUCACION, 353 , 333–334.

Ariffin, M., Oxley, A., & Sulaiman, S. (2014). Evaluating game-based learning effectiveness in higher education. Procedia - Social and Behavioral Sciences, 123 , 20–27 http://dx.doi.org/10.1016/j.sbspro.2014.01.1393 . Accessed 11 Sept 2016.

Auman, C. (2011). Using simulation games to increase student and instructor engagement. College Teaching, 59 (4), 154–161.

Backlund, P., & Hendrix, M. (2013). Educational games-are they worth the effort? A literature survey of the effectiveness of serious games. In Games and virtual worlds for serious applications (VS-GAMES), 2013 5th international conference on (pp. 1-8). IEEE.

Badea, M. (2015). English classes and effectiveness of games in higher education. Journal Plus Education, 13 (2), 79–85.

Bausch, L., Beran, J., Cahanes, S., & Krug, L. (2008). Physiological responses while playing Nintendo Wii sports. Journal of Undergraduate Kinesiology Research, 3 , 19–25.

Beckem, J. M., & Watkins, M. (2012). Bringing life to learning: Immersive experiential learning simulations for online and blended courses. Journal of Asynchronous Learning Networks, 16 (5), 61–70.

Bekebrede, G., Warmelink, H. J. G., & Mayer, I. S. (2011). Reviewing the need for gaming in education to accommodate the net generation. Computers & Education, 57 (2), 1521–1529.

Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction, 2013 , 1.

Beuk, F. (2015). Sales simulation games student and instructor perceptions. Journal of Marketing Education , 1-13 (2015). Doi: 0273475315604686.

Biddiss, E., & Irwin, J. (2010). Active video games to promote physical activity in children and youth. Archives of Pediatrics and Adolescent Medicine, 164 , 664–672.

Boeker M, Andel., P, Vach, W., & Frankenschmidt, A. (2013). Game-based E-learning is more effective than a conventional instructional method: A randomized controlled trial with third-year medical students. PloS One 8(12): e82328. doi: 10.1371/journal.pone.0082328 .

Bolliger, D. U., Mills, D., White, J., & Kohyama, M. (2015). Japanese students’ perceptions of digital game use for English-language learning in higher education. Journal of Educational Computing Research, 53 (3), 384–408 0735633115600806.

Buckless, F. A., Krawczyk, K., & Showalter, D. S. (2014). Using virtual worlds to simulate real-world audit procedures. Issues in Accounting Education, 29 (3), 389–417.

Çankaya, S., & Karamete, A. (2009). The effects of educational computer games on students’ attitudes towards mathematics course and educational computer games. Procedia-Social and Behavioral Sciences, 1 (1), 145–149.

Carenys, J., & Moya, S. (2016). Digital game-based learning in accounting and business education. Accounting Education , 25 (6), 598-651.

Cela-Ranilla, J. M., Esteve-Mon, F. M., Esteve-González, V., & Gisbert-Cervera, M. (2014). Developing self-management and teamwork using digital games in 3D simulations. Australasian Journal of Educational Technology, 30 (6), 634–651.

Chang, Y. C., Peng, H. Y., & Chao, H. C. (2010). Examining the effects of learning motivation and of course design in an instructional simulation game. Interactive Learning Environments, 18 (4), 319–339.

Chaves, R. O., von Wangenheim, C. G., Furtado, J. C. C., Oliveira, S. R. B., Santos, A., & Favero, E. L. (2015). Experimental evaluation of a serious game for teaching software process modeling. IEEE Transactions on Education, 58 (4), 289–296.

Chen, L., Chen, T.-L., & Liu, H.-K. J. (2010). Perception of young adults on online games: Implications for higher education. TOJET: The Turkish Online Journal of Educational Technology, 9 (3), 76–84.

Chen, C. H., Wang, K. C., & Lin, Y. H. (2015). The Comparison of solitary and collaborative modes of game-based learning on Students' science learning and motivation. Educational Technology & Society, 18 (2), 237–248.

Chiang, Y. T., Lin, S. S. J., Cheng, C. Y., & Liu, E. Z. F. (2011). Exploring online game players’ flow experiences and positive affect. The Turkish Online Journal of Educational Technology, 10 (1), 106–114.

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2015). Digital games, design, and learning a systematic review and meta-analysis. Review of educational research, 86 (1), 79–122 0034654315582065.

Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of the empirical evidence on computer games and serious games. Computers & Education, 59 (2), 661–686.

Connolly, T., & Stansfield, M. (2006). Using games-based eLearning technologies in overcoming difficulties in teaching information systems. Journal of Information Technology Education, 5 (1), 459–476.

Connolly, T. M., Stansfield, M., & Hainey, T. (2007). An application of games-based learning within software engineering. British Journal of Educational Technology, 38 (3), 416–428.

Costa, G. J. M. D., Kikot, T., Fernandes, S., & Águas, P. (2014). Why use-centered game-based learning in higher education? The case of cesim simbrand. Journal of Spatial and Organizational Dynamics , 2 (3), 229-241.

Cózar-Gutiérrez, R., & Sáez-López, J.M. (2016). Game-based learning and gamification in initial teacher training in the social sciences: An experiment with MinecraftEdu. International Journal of Educational Technology in Higher Education , 13 (1), 1 (2016). doi: 10.1186/s41239-016-0003-4 .

Crocco, F., Offenholley, K., & Hernandez, C. (2016). A proof-of-concept study of game-based learning in higher education. Simulation & Gaming, 47 (4), 403-422 (2016). Doi: 1046878116632484.

Cvetić, B., Vasiljević, D., & Danilović, M. (2013). DRP game: New tool to enhance teaching and learning in logistics and supply chain management. In 1st Logistics International Conference (pp. 299-303).

Dankbaar, M. E. W., Alsma, J., Jansen, E. E. H., van Merrienboer, J. J. G., van Saase, J. L. C. M., & Schuit, S. C. E. (2016). An experimental study on the effects of a simulation game on students’ clinical cognitive skills and motivation. Advances in Health Sciences Education, 21 (3), 505–521.

DePorres, D., & Livingston, R. E. (2016). Launching new doctoral students: Embracing the Hero’s journey. Developments in Business Simulation and Experiential Learning, 43 (1), 121–128.

de Wit-Zuurendonk, L. D., & Oei, S. G. (2011). Serious gaming in women’s health care. BJOG: An International Journal of Obstetrics & Gynaecology, 118 (s3), 17–21.

Denholm, J., Protopsaltis, A., & de Freitas, S. (2012). The value of team-based mixed-reality (TBMR) games in higher education. In European Conference on Games Based Learning (p. 571). Academic Conferences international limited.

de Smale, S., Overmans, T., Jeuring, J., & van de Grint, L. (2015). The effect of simulations and games on learning objectives in tertiary education: A systematic review. In Games and Learning Alliance 4 th International Conference , GALA 2015 Rome.

Divjak, B., & Tomić, D. (2011). The impact of game-based learning on the achievement of learning goals and motivation for learning mathematics-literature review. Journal of Information and Organizational Sciences, 35 (1), 15–30.

Dudzinski, M., Greenhill, D., Kayyali, R., Nabhani, S., Philip, N., Caton, H., Ishtiaq, S., & Gatsinzi, F. (2013). The design and evaluation of a multiplayer serious game for pharmacy students. In European Conference on Games Based Learning (p. 140). Academic Conferences international limited.

Dzeng, R. J., Lin, K. Y., & Wang, P. R. (2014). Building a construction procurement negotiation training game model: Learning experiences and outcomes. British Journal of Educational Technology, 45 (6), 1115–1135.

Elias, A. (2014). Simulating the European Union: Reflections on module design. International Studies Perspectives, 15 (4), 407–422.

Erhel, S., & Jamet, E. (2013). Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness. Computers & Education, 67 , 156–167.

Farrington, J. (2011). From the research: Myths worth dispelling: Seriously, the game is up. Performance Improvement Quarterly, 24 , 105–110.

Felicia, P. (2011). Assessing how game-based learning is perceived in Irish education. In Proceedings of the 7th European Conference on Management Leadership and Governance: ECGBL 2011 (p. 168). Academic Conferences limited.

Flanagan, B., Nestel, D., & Joseph, M. (2004). Making patient safety the focus: Crisis resource management in the undergraduate curriculum. Medical Education, 38 (1), 56–66.

Franciosi, S. J. (2016). Acceptability of RPG simulators for foreign language training in Japanese higher education. Simulation & Gaming, 47 (1), 31–50 (2015). Doi: 1046878115608621.

Fu, K., Hainey, T., & Baxter, G. (2016). A systematic literature review to identify empirical evidence on the use of computer games in business education and training. In 10th European Conference on Games Based Learning: ECGBL 2016 (p. 232).

García Carbonell, A., & Watts Hooge, F. I. (2012). Investigación empírica del aprendizaje con simulación telemática. Revista Iberoamericana de Educación (versión digital)., 59 (3), 1–11.

Gegenfurtner, A., Quesada-Pallarès, C., & Knogler, M. (2014). Digital simulation-based training: A meta-analysis. British Journal of Educational Technology, 45 (6), 1097–1114.

Geithner, S., & Menzel, D. (2016). Effectiveness of learning through experience and reflection in a Project Management simulation. Simulation & Gaming, 47 (2), 228–256 (2016). Doi: 1046878115624312.

Giovanello, S. P., Kirk, J. A., & Kromer, M. K. (2013). Student perceptions of a role-playing simulation in an introductory international relations course. Journal of Political Science Education, 9 (2), 197–208.

Girard, C., Ecalle, J., & Magnan, A. (2013). Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, 29 (3), 207–219.

Gold, S. (2016). Design and effectiveness of a self-study pedagogical approach to using a simulation game in the classroom. Developments in Business Simulation and Experiential Learning, 43 (1).

Gómez, M. (2014). Ciencias Sociales y gamificación, ¿una pareja con futuro? In J. Pagès (Ed.), Una mirada al pasado y un proyecto de futuro: investigación e innovación en didáctica de las ciencias sociales (pp. 257–262). Barcelona: Universitat Autónoma de Barcelona; Asociación Universitaria de Profesores de Didáctica de las Ciencias Sociales.

Gros, B. (2007). Digital games in education: The Design of Games Based Learning Environments. Journal of Research on Technology in Education, 40 (1), 23–39.

Hainey, T., Connolly, T. M., Stansfield, M., & Boyle, E. A. (2011). Evaluation of a game to teach requirements collection and analysis in software engineering at tertiary education level. Computers & Education, 56 (1), 21–35.

Halpern, D. F., Millis, K., Graesser, A. C., Butler, H., Forsyth, C., & Cai, Z. (2012). Operation ARA: A computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills and Creativity, 7 (2), 93–100.

Hämäläinen, R., & Oksanen, K. (2014). Collaborative 3D learning games for future learning: Teachers’ instructional practices to enhance shared knowledge construction among students. Technology, Pedagogy and Education, 23 (1), 81–101.

Hannig, A., Kuth, N., Özman, M., Jonas, S., & Spreckelsen, C. (2012). eMedOffice: A web-based collaborative serious game for teaching optimal design of a medical practice. BMC Medical Education, 12 (1), 1.

Hannig, A., Lemos, M., Spreckelsen, C., Ohnesorge-Radtke, U., & Rafai, N. (2013). Skills-o-mat: Computer supported interactive motion-and game-based training in mixing alginate in dental education. Journal of Educational Computing Research, 48 (3), 315–343.

Helle, L., Nivala, M., Kronqvist, P., Gegenfurtner, A., Björk, P., & Säljö, R. (2011). Traditional microscopy instruction versus process-oriented virtual microscopy instruction: A naturalistic experiment with control group. Diagnostic Pathology, 6 (1), 1.

Hess, T., & Gunter, G. (2013). Serious game-based and nongame-based online courses: Learning experiences and outcomes. British Journal of Educational Technology, 44 (3), 372–385.

Hou, H. T. (2015). Integrating cluster and sequential analysis to explore learners’ flow and behavioral patterns in a simulation game with situated-learning context for science courses: A video-based process exploration. Computers in Human Behavior, 48 , 424–435.

Hou, H. T., & Li, M. C. (2014). Evaluating multiple aspects of a digital educational problem-solving-based adventure game. Computers in Human Behavior, 30 , 29–38.

Hsu, Y.-C., Ho, H. N. J., Tsai, C.-C., Hwang, G.-J., Chu, H.-C., Wang, C.-Y., & Chen, N.-S. (2012). Research trends in technology-based learning from 2000 to 2009: A content analysis of publications in selected journals. Educational Technology & Society, 15 (2), 354–370.

Huang, W. H., Huang, W. Y., & Tschopp, J. (2010). Sustaining iterative game playing processes in DGBL: The relationship between motivational processing and outcome processing. Computers & Education, 55 (2), 789–797.

Hummel, H. G., Van Houcke, J., Nadolski, R. J., Van der Hiele, T., Kurvers, H., & Löhr, A. (2011). Scripted collaboration in serious gaming for complex learning: Effects of multiple perspectives when acquiring water management skills. British Journal of Educational Technology. doi: 10.1111/j.1467-8535.2010.01122.x .

Ibrahim, R., Wahab, S., Yusoff, R. C. M., Khalil, K., Desaru, I. N. S. T. E. D. T., & Jaafar, A. (2011). Student perceptions of educational games in higher education: An empirical study. Issues in Information Systems, 12 (1), 120–133.

Jiménez-Munguía, M. C., & Luna-Reyes, L. F. (2012). Development of analytical abilities and collaborative learning assessment in undergraduate students through simulation games. In Electrical Communications and Computers (CONIELECOMP), 2012 22nd International Conference on (pp. 12–16). IEEE.

Jones, R., & Bursens, P. (2015). The effects of active learning environments: How simulations trigger affective learning. European Political Science, 14 (3), 254–265.

Kapralos, B., Hogan, M., Pribetic, A. I., & Dubrowski, A. (2011). Virtual simulations and serious games in a laptop-based university: Gauging faculty and student perceptions. Interactive Technology and Smart Education, 8 (2), 106–120.

Ke, F., Xie, K., & Xie, Y. (2015). Game-based learning engagement: A theory-and data-driven exploration. British Journal of Educational Technology. doi: 10.1111/bjet.12314 .

Kikot, T., Costa, G., Magalhães, R., & Fernandes, S. (2013). Simulation games as tools for integrative dynamic learning: The case of the management course at the University of Algarve. Procedia Technology, 9 , 11–21.

Klabbers, J. H. G. (2009). Terminological ambiguity game and simulation. Simulation & Gaming, 40 (4), 446–463.

Kleinheksel, A. J. (2014). Transformative learning through virtual patient simulations: Predicting critical student reflections. Clinical Simulation in Nursing, 10 (6), e301–e308.

Kovalik, C. L., & Kuo, C. L. (2012). Innovation Diffusion: Learner benefits and instructor insights with the Diffusion Simulation Game. Simulation & Gaming. 43 (6), 803-824.

Lameras, P., Arnab, S., Dunwell, I., Stewart, C., Clarke, S., & Petridis, P. (2016). Essential features of serious games design in higher education: Linking learning attributes to game mechanics. British Journal of Educational Technology. doi: 10.1111/bjet.12467 .

Lancaster, R. J. (2014). Serious game simulation as a teaching strategy in pharmacology. Clinical Simulation in Nursing, 10 (3), e129–e137.

Li, Z. Z., Cheng, Y. B., & Liu, C. C. (2013). A constructionism framework for designing game-like learning systems: Its effect on different learners. British Journal of Educational Technology, 44 (2), 208–224.

Li, M. C., & Tsai, C. C. (2013). Game-based learning in science education: A review of relevant research. Journal of Science Education and Technology, 22 (6), 877–898.

Liarokapis, F., Anderson, E. F., Oikonomou, A. (2010). Serious games for use in a higher education environment. In Proceedings of the Emerging Games Platforms, Technologies and Applications Workshop (EGPTA’10), 15th Int‟l Computer Games Conference: AI, Interactive Multimedia, Virtual Worlds and Serious Games Louisville, Kentucky, USA (pp. 28-31).

Liao, Y. W., Huang, Y. M., & Wang, Y. S. (2015). Factors affecting students’ continued usage intention toward business simulation games an empirical study. Journal of Educational Computing Research, 53 (2), 260–283.

Liao, Y.-W., & Wang, Y.-S. (2011). Investigating the factors affecting students’ continuance intention to use business simulation games in the context of digital learning. In International Conference on Innovation, Management and Service (pp. 119-124).

Lin, Y. L. (2016). Differences among different DGBLs learners. International Journal of Business and Management, 11 (1), 181–188.

Lin, Y. L., & Tu, Y. Z. (2012). The values of college students in business simulation game: A means-end chain approach. Computers & Education, 58 (4), 1160–1170.

Liu, C. C., Cheng, Y. B., & Huang, C. W. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57 (3), 1907–1918.

Ljungkvist, P., & Mozelius, P. (2012). Educational games for self learning in introductory programming courses-a straightforward design approach with progression mechanisms. In Proceedings Of The 6th European Conference On Games Based Learning, ECGBL (pp. 285-293).

Lu, J., Hallinger, P., & Showanasai, P. (2014). Simulation-based learning in management education: A longitudinal quasi-experimental evaluation of instructional effectiveness. Journal of Management Development, 33 (3), 218–244.

Lukosch, H., Kurapati, S., Groen, D., & Verbraeck, A. (2016). Microgames for situated learning a case study in interdependent planning. Simulation & Gaming, 47 (3), 346–367 (2016). Doi: 1046878116635468.

Mayer, I., Warmelink, H., & Bekebrede, G. (2013). Learning in a game-based virtual environment: A comparative evaluation in higher education. European Journal of Engineering Education, 38 (1), 85–106.

McLoughlin, C., & Lee, M. J. W. (2008). The three P’s of pedagogy for the networked society: Personalization, participation, and productivity. International Journal of Teaching and Learning in Higher Education, 20 (1), 10–27.

Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70 , 29–40.

Mueller, F., Agamanolis, S., & Picard, R. (2003). Exertion interfaces: Sports over a distance for social bonding and fun. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 561–568). ACM.

Nadolny, L., & Halabi, A. (2015). Student participation and achievement in a large lecture course with game-based learning. Simulation & Gaming, 47 (1), 51–72 (2015). Doi: 1046878115620388.

Navidad, F. C. (2013). Students' devised classroom games-simulations: An innovative tool on mathematics achievement and motivation in nursing students. International Proceedings of Economics Development and Research, 60 , 14–18. doi: 10.7763/IPEDR .

Nkhoma, M., Calbeto, J., Sriratanaviriyakul, N., Muang, T., Ha Tran, Q., & Kim Cao, T. (2014). Towards an understanding of real-time continuous feedback from simulation games. Interactive Technology and Smart Education, 11 (1), 45–62.

Pasin, F., & Giroux, H. (2011). The impact of a simulation game on operations management education. Computers & Education, 57 (1), 1240–1254.

Peterson, M. (2010). Computerized games and simulations in computer-assisted language learning: A meta-anlysis of research. Simulation & Gaming, 41 (1), 72–93.

Pløhn, T. (2013). Nuclear mayhem-a pervasive game designed to support learning. In European Conference on Games Based Learning (p. 475). Academic Conferences international limited.

Poikela, P., Ruokamo, H., & Teräs, M. (2015). Comparison of meaningful learning characteristics in simulated nursing practice after traditional versus computer-based simulation method: A qualitative videography study. Nurse Education Today, 35 (2), 373–382.

Rajan, P., Raju, P., & Sankar, C.S. (2013). Serious Games to Improve Student Learning in Engineering Classes . Paper presented at 120 th ASEE Annual Conference & Exposition , 23-26 June, 2013, Atlanta, Georgia. https://peer.asee.org/22448 Accessed 2 Oct 2016.

Ranchhod, A., Gurău, C., Loukis, E., & Trivedi, R. (2014). Evaluating the educational effectiveness of simulation games: A value generation model. Information Sciences, 264 , 75–90.

Renken, M., Peffer, M., Otrel-Cass, K., Girault, I., & Chiocarriello, A. (2016). Computer simulations on a multidimensional continuum: A definition and examples. In Simulations as Scaffolds in Science Education (pp. 5-14). London: Springer. 

Riemer, V., & Schrader, C. (2015). Learning with quizzes, simulations, and adventures: Students' attitudes, perceptions and intentions to learn with different types of serious games. Computers & Education, 88 , 160–168.

Ritzhaupt, A., Poling, N., Frey, C., & Johnson, M. (2014). A synthesis on digital games in education: What the research literature says from 2000 to 2010. Journal of Interactive Learning Research, 25 (2), 261–280.

Robertson, B., Schumacher, L., Gosman, G., Kanfer, R., Kelley, M., & DeVita, M. (2009). Simulation-based crisis team training for multidisciplinary obstetric providers. Simululation in Healthcare, 4 (2), 77–83. doi: 10.1097/SIH.0b013e3181917cd .

Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58 (1), 136–153.

Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals . Cambridge, MA, USA: MIT Press.

Sarabia-Cobo, C. M., Alconero-Camarero, A. R., Lavín-Alconero, L., & Ibáñez-Rementería, I. (2016). Assessment of a learning intervention in palliative care based on clinical simulations for nursing students. Nurse Education Today, 45 , 219–224.

Sauve, L., Renaud, L., Kaufman, D., & Marquis, J. S. (2007). Distinguishing between games and simulation: A systematic review. Education Technology & Society, 10 (3), 247–256.

Sawyer, B. (2002). Serious games: Improving public policy through game-based learning and simulation . USA: Woodrow Wilson International Center for Scholars.

Seng, W. Y., & Yatim, M. H. M. (2014). Computer game as learning and teaching tool for object oriented programming in higher education institution. Procedia-Social and Behavioral Sciences, 123 , 215–224.

Shieh, R. S., Chang, W., & Tang, J. (2010). The impact of implementing technology-enabled active learning (TEAL) in university physics in Taiwan. Asia-Pacific Education Researcher (De La Salle University Manila), 19 (3).

Shin, S., Park, J. H., & Kim, J. H. (2015). Effectiveness of patient simulation in nursing education: Meta-analysis. Nurse Education Today, 35 (1), 176–182.

Siewiorek, A., Saarinen, E., Lainema, T., & Lehtinen, E. (2012). Learning leadership skills in a simulated business environment. Computers & Education, 58 , 121–135.

Siewiorek, A., Gegenfurtner, A., Lainema, T., Saarinen, E., & Lehtinen, E. (2013). The effects of computer-simulation game training on participants’ opinions on leadership styles. British Journal of Educational Technology, 44 (6), 1012–1035.

Silvia, C. (2012). The impact of simulations on higher-level learning. Journal of Public Affairs Education, 18 (2), 397–422.

Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64 (2), 489–528.

Smetana, L. K., & Bell, R. L. (2012). Computer simulations to support science instruction and learning: A critical review of the literature. International Journal of Science Education, 34 (9), 1337–1370.

Söderström, T., Häll, L., Nilsson, T., & Ahlqvist, J. (2014). Computer simulation training in health care education fuelling reflection-in-action? Simulation & Gaming, 45 (6), 805–828.

Stanley, D., & Latimer, K. (2011). ‘the Ward’: A simulation game for nursing students. Nurse Education in Practice, 11 (1), 20–25.

Strycker, J. (2016). Utilizing a simulation within an online school technology leadership course. Online Learning Journal, 20 (1), 130–144.

Swanson, E. A., Nicholson, A. C., Boese, T. A., Cram, E., Stineman, A. M., & Tew, K. (2011). Comparison of selected teaching strategies incorporating simulation and student outcomes. Clinical Simulation in Nursing, 7 (3), e81–e90.

Tanner, J. R., Stewart, G., Totaro, M. W., & Hargrave, M. (2012). Business simulation games: Effective teaching tools or window dressing? American Journal of Business Education (Online), 5 (2), 115.

Tao, Y. H., Yeh, C. C. R., & Hung, K. C. (2015). Validating the learning cycle models of business simulation games via student perceived gains in skills and knowledge. Educational Technology & Society, 18 (1), 77–90.

Terzidou, T., Tsiatsos, T., Dae, A., Samaras, O., & Chasanidou, A. (2012). Utilizing virtual worlds for game based learning: Grafica, a 3D educational game in second life. In 2012 IEEE 12th International Conference on Advanced Learning Technologies (pp. 624-628). IEEE.

Tiwari, S. R., Nafees, L., & Krishnan, O. (2014). Simulation as a pedagogical tool: Measurement of impact on perceived effective learning. The International Journal of Management Education, 12 (3), 260–270.

Tsekleves, E., Cosmas, J., & Aggoun, A. (2014). Benefits, barriers and guideline recommendations for the implementation of serious games in education for stakeholders and policymakers. British Journal of Educational Technology, 47 (1), 164–183. doi: 10.1111/bjet.12223 .

van Roessel, L., & van Mastrigt-Ide J. (2011). Collaboration and team composition in applied game creation processes. DiGRA '11, proceedings of the 2011 DiGRA international conference, think design play, 1-14.

von Wangenheim, C. G., Savi, R., & Borgatto, A. F. (2012). DELIVER!–an educational game for teaching earned value management in computing courses. Information and Software Technology, 54 (3), 286–298.

Vos, L., & Brennan, R. (2010). Marketing simulation games: Student and lecturer perspectives. Marketing Intelligence & Planning, 28 (7), 882–897.

Wang, C., Huang, C. C., Lin, S. J., & Chen, J. W. (2016). Using multimedia tools and high-fidelity simulations to improve medical students' resuscitation performance: An observational study. BMJ Open, 6 (9), e012195.

Wang, L. C., & Chen, M. P. (2010). The effects of game strategy and preference-matching on flow experience and programming performance in game-based learning. Innovations in Education and Teaching International, 47 (1), 39–52.

Warren, J. N., Luctkar-Flude, M., Godfrey, C., & Lukewich, J. (2016). A systematic review of the effectiveness of simulation-based education on satisfaction and learning outcomes in nurse practitioner programs. Nurse Education Today, 46 , 99–108.

White, B., Kahriman, A., Luberice, L., & Idleh, F. (2010). Evaluation of software for introducing protein structure. Biochemistry and Molecular Biology Education, 38 (5), 284–289.

Willoughby, T. (2008). A short-term longitudinal study of internet and computer game use by adolescent boys and girls: Prevalence, frequency of use, and psychosocial predictors. Developmental Psychology, 44 (1), 195–204.

Woo, J. C. (2014). Digital game-based learning supports student motivation, cognitive success, and performance outcomes. Educational Technology & Society, 17 (3), 291–307.

Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60 (1), 412–425.

Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105 (2), 249–265.

Yang, J. C., Chen, C. H., & Jeng, M. C. (2010). Integrating video-capture virtual reality technology into a physically interactive learning environment for English learning. Computers & Education, 55 (3), 1346–1356.

Yang, J. C., Chien, K. H., & Liu, T. C. (2012). A digital game-based learning system energy for energy education: An energy conservation pet. The Turkish Online Journal of Educational Technology, 11 (2), 27–37.

Yang, Y. C., & Chang, C. L. (2013). Empowering students through digital game authorship: Enhancing concentration, critical thinking, and academic achievement. Computers & Education, 68 , 334–344.

Yin, C., Song, Y., Tabata, Y., Ogata, H., & Hwang, G. J. (2013). Developing and implementing a framework of participatory simulation for mobile learning using scaffolding. Educational Technology & Society, 16 (2), 137–150.

Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M., & Yukhymenko, M. (2012). Our princess is in another castle a review of trends in serious gaming for education. Review of Educational Research, 82 (1), 61–89.

Zacharia, Z. C., & Olympiou, G. (2011). Physical versus virtual manipulative experimentation in physics learning. Learning and Instruction, 21 (3), 317–331.

Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38 (9), 25–32.

Download references

Acknowledgements

The research was sponsored by Laureate International Universities, through the “David Wilson Award for Excellence in Teaching and Learning”, won by Dr. Dimitrios Vlachopoulos (2015-2017).

Authors' contributions

DV conceived the study. AM conducted the literature review and prepared the summaries and critical reflection on the corresponding literature. DV participated in the design of the study and analysis. AM participated in the preparation of the article's structure, graphs, and reference list. Both authors read and approved the final manuscript.

Competing interest

The authors declared that they have no competing interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and affiliations.

Laureate Online Education / European University Cyprus, Amsterdam, Netherlands

Dimitrios Vlachopoulos

European University Cyprus, Nicosia, Cyprus

Agoritsa Makri

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Dimitrios Vlachopoulos .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Vlachopoulos, D., Makri, A. The effect of games and simulations on higher education: a systematic literature review. Int J Educ Technol High Educ 14 , 22 (2017). https://doi.org/10.1186/s41239-017-0062-1

Download citation

Received : 28 November 2016

Accepted : 30 May 2017

Published : 10 July 2017

DOI : https://doi.org/10.1186/s41239-017-0062-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Game-based learning
• Digital games
• Pedagogical use
• Higher education
• Learning outcomes
• Cognitive goals
• Behavioural goals
• Affective goals