1. Introduction

Virtually recreating real-world situations allows learners to develop procedural knowledge and skills by performing complex tasks without any of the real risks typically associated with bad decisions [4]. Learning objectives are all achieved while having fun. The ludic aspect is, in fact, what separates serious games from other learning environments that may also make use of information technology. Industry predictions attest to the popularity of game-based learning. According to a recent Global Game-Based Learning Market report, serious games will probably reach $17 billion by the year 2023, surpassing 37% in terms of growth rate. This is being driven by improvements and innovation in virtual and augmented reality and artificial intelligence [5].

Nevertheless, the literature on serious games also acknowledges limits and difficulties. Primarily, good games are not easy to design. Games developed by the entertainment industry, set up exclusively for amusement, often fail. Arguably, designing successful serious games is even more difficult. Serious games must not only be entertaining; they must also help students achieve learning goals. Presented by this dual challenge, many serious game projects continue to fail [6].

When a learning environment is well-designed, it generates a more positive impact on the player’s attitude and learning experience [7]. However, relatively few studies have tried to identify the antecedents of serious game success in the eyes of learners. Although their popularity continues to steadily increase, few studies have empirically examined the factors that affect learners’ satisfaction with serious games. The purpose of our research was to identify factors affecting learner satisfaction in a serious gaming environment and to assess the relative importance of these factors.

We collected data collection from 110 university business students enrolled in courses using serious games to simulate real life managerial situations. We used SmartPLS3 software to conduct statistical analysis. Results from our analysis show that among the twelve antecedents of satisfaction identified through our literature review, sense of control, fidelity, effectiveness, and entertainment influence directly students’ satisfaction with serious games. However, further analyses revealed the presence of multicollinearity among the independent variables of the model making some factors appear statistically unimportant. Treatment of our initial model was attempted to remedy the situation by considering the existence of mediating variables and theoretically justified superordinate constructs depicted as second-order factors. Our results show that while some factors affect learners’ satisfaction with serious games directly, others do so indirectly through mediating factors or as part of higher-order factors and, as such, should not be neglected when designing successful games.

2. Literature Review

2.1.  Benefits of Serious Games

Serious games are increasingly used in academia [8]; [9].  Several reasons explain the growing interest in serious games. First, an increasing number of educational institutions are transitioning from traditional pedagogical and didactic to more learner-centered approaches. The latter emphasizes the more active and participative role of learners [10] and serious games can create authentic (ecologically valid) and immersive environments [11]. Their interactivity helps learners in better understanding complex tasks, knowledge acquisition, and retention. Serious games also promote better concentration more than passive learning environments [12].  This represents a major transition from a traditional listening/observing only learning model to one founded on live experiences and practice. Compared to traditional classrooms where knowledge transmission is typically passive, serious games are more effective because of the interactivity they allow [13]. Moreover, the playfulness of games promotes greater interest in learning activities, engagement, focus, and enthusiasm.  Serious games provide several benefits to learners [14]. Well-designed games are fun, enjoyable and increase student motivation. Linked to motivation is the concept of satisfaction. Studies have evidenced the importance of this concept in education [15], and yet, few studies have examined the antecedents of learners’ satisfaction while interacting with serious games [16]; [17].

2.2. Satisfaction with Serious Games

In business, the concept of customer satisfaction is a vital component of a company’s success. Research shows that employee satisfaction enhances productivity and organizational performance [18]. Customer satisfaction is positively linked to perceived service quality. Satisfaction also impacts positively on customer loyalty [19]. In the area of education, studies have demonstrated a positive relationship between learner satisfaction and both motivation and achievement [20]. Gameplay can generate strong, positive feelings in learners. These, in turn, motivate learners to continue playing (i.e., continue using the technology) and learning. Learners’ satisfaction with serious games is more pronounced when positive emotions are high [21].

Satisfaction is one of the most researched concepts in the area of Information Systems (IS) [22]. When compared to other measures, user satisfaction is among the strongest of indicators employed by researchers to measure IS success [23]. According to [24], user satisfaction is “the affective attitude towards a specific computer application by someone who interacts with the application directly”. Through a literature review, our paper first seeks to identify antecedents of serious game satisfaction from the perspective of learners. We then propose and empirically test a model to investigate the degree to which these antecedents relate to satisfaction.

2.3. Antecedents of Satisfaction with Serious Games

Twelve main factors were identified from an intensive review of the literature [25] as potential predictors of satisfaction, defined below :

Ease of Use is borrowed from the Technology Acceptance Model [26], it is defined as the degree to which a particular technology is perceived to be intuitive and simple to use.

Reliability is the degree to which the serious game worked smoothly and without problems. In Marketing, reliability has been identified as a crucial dimension of perceived service quality (i.e., the literature on SERVQUAL), while in Information Systems (IS) /Information Technology (IT) research, it refers to whether a system functions properly and can be defined as the degree to which a user possesses inherent confidence in the system [27].

Aesthetics reflects a subjective experience that is evoked by players during the process of interaction with the game interface [28]; [29]; [30], namely using sound, sight, and motion [31].

Fidelity/Realism describes the psychological and physical similarities that exist between a particular game and the environment it seeks to represent [32], the realism of a simulation [33].

Instructor assistance and support. Several studies show that the teacher plays an essential role in the success of IT integration in classrooms [34]. We define instructor assistance as the degree to which learners perceive that the instructor provided encouragement, help, and information on the use of the serious game. Instructors provide assistance at two levels, theoretical and technical [35].

Perceived control is defined as the degree to which learners feel capable of influencing and bringing about change to the world of play [36]. This can be achieved via multiple customization options such as, the pace of the game, the choice of the type of feedback the learner believes would be useful to progress more successfully through the game and even the capability of manipulating the appearance of one’s playable character [37]. Research shows that when the learner has more control over their learning, they become more involved in the game and even try more complex strategies [38] and a resulting increase in positive attitudes [39].

Progressive challenge is the degree to which the game challenges the learner with increasingly difficult problems and increasingly complex challenges. Successful game progression is built on previously acquired knowledge, abilities, and skills developed and learned while playing [40]. Successful learning ensues when players progressively adapt to and succeed in performing tasks that are increasingly challenging and complex.

Effectiveness is defined as the capacity/power of the serious game to improve the knowledge of learners [41]. Whereas conventional games seek to boost the playfulness aspect of the game, serious games seek to create learning environments that support improvements in skills training (e.g., professionals). These skills are other than those associated purely with play. Indeed, research shows that serious games can be very effective in improving professional and academic skills [42].

Status/Performance tracking and feedback is the degree to which the simulation’s progress information and feedback is perceived as constructive and helpful. Assessment and feedback are pillars of teaching effectiveness [43]. They help learners achieve the game’s objectives [44]. Performance feedback must be specific and presented at the right time [45]. Feedback includes information on the state of the player, instructor help/assistance during play, tips and clues [41]. It can take the form of visual, auditory, tactile prompts [46].

Entertainment is the degree to which the learner finds the game play pleasurable. Research defines pleasure as an attitude or an emotion that can satisfy intrinsic needs and produce a positive reaction [47]. The entertainment value derived from gameplay is largely determined by the quality of design of the “educational ludo” environment [48]. Entertainment value impacts positively on learners’ satisfaction.

Immersion is the ability of a game to create an immersive experience in the learner [49], the feeling of “being there” [50]. It occurs when the learner temporarily accepts the reality of a game and focuses on this reality through his involvement in the game. It reflects the degree of player engagement in the game environment [49].

Goal clarity refers to the need to clearly define what objectives must be achieved during game play. It is a fundamental principle of instructional design [51]; [52]. Studies (e.g., [53]) demonstrate that serious game environments can fail to promote concentration or rouse curiosity in learners mainly because their goals are ill-defined, ambiguous, or misunderstood. Clarity of objectives is crucial in effective game design because it serves to increase commitment and concentration [54].

3. Hypotheses

In this section, we present the hypothesized relationship between each factor and the learner’s satisfaction with the serious game (see Table 1). The questionnaire items for each measure/factor are presented in the Appendix. For the sake of parsimony, only questionnaire items that successfully passed the card-sorting exercise (item validation) are presented.

Table 1: Hypotheses of Original Model

H1	A positive and significant relationship exists between ease of use and learners’ satisfaction
H2	A positive and significant relationship exists between reliability and learners’ satisfaction
H3	A positive and significant relationship exists between aesthetics and learners’ satisfaction
H4	A positive and significant relationship exists between fidelity and learners’ satisfaction
H5	A positive and significant relationship exists between instructor assistance and support and learners’ satisfaction
H6	A positive and significant relationship exists between feeling of control and learners’ satisfaction
H7	A positive and significant relationship exists between progressive challenge and learners’ satisfaction
H8	A positive and significant relationship exists between and learners’ satisfaction
H9	A positive and significant relationship exists between performance/status tracking and feedback and learners’ satisfaction
H10	A positive and significant relationship exists between entertainment and learners’ satisfaction
H11	A positive and significant relationship exists between immersion and learners’ satisfaction
H12	A positive and significant relationship exists between goal clarity and learners’ satisfaction

4.      Methodology

PLS-SEM (Partial Least Square – Structural Equation Modeling) is a multivariate statistical modelling technique increasingly being used by researchers in IS/IT, education and in other areas because of several advantages. It is often favoured when analyzing models using smaller sample sizes or data with non-normal distributions. Specifically, we used the statistical software package called SmartPLS.

4.1.  Item Creation and Refinement

We generated an initial set of items and then refined and validated them before including them in a questionnaire. Steps were as follows.

• Creation or adaptation of items based on the definition of concepts [55].
• Validation of the questionnaire items using the card-sorting method [56].

The card sorting method seeks to verify, whether the items initially chosen by the researcher to be included in the questionnaire relate accurately to the concepts that (s)he is trying to measure. For this, four (4) judges were recruited: two professors from the IS/IT area and two students enrolled in a master’s program in Information Systems/Information Technology. The validation process required each judge to sort the initial questionnaire items by properly combining each one with the actual construct that item was intended to measure. When at least 3 out of 4 judges (or 75% of judges) would successfully associate a specific item to its expected construct, the item in question was deemed to accurately reflect the concept it was intended to measure and was therefore included in the questionnaire.

4.2.  Data Collection and Sample Size

After validating the initial set of items using the pretest and the sort-by-card method, the items were included in an online questionnaire created using the Google Forms application. Three (3) lecturers from the business school of our university were contacted to help us recruit students. These lecturers were chosen because, in their classes, they used a serious gaming platform that simulated the typical challenges and problems encountered by managers of small- and medium-sized businesses. Over a period of approximately one (1) week, we visited the classrooms of these lecturers to explain to students the purpose of our study and invite them to participate. We printed information sheets containing our contact details and the URL for the online questionnaire. During our visits to classrooms, these sheets were provided to students. All questionnaire items were of the Likert type ranging from 1 (Strongly disagree) to 5 (Strongly agree).

Data collection was done over a period of three (3) weeks. Two emails were sent to the lecturers to remind their students to participate in the study. Participation was completely voluntary. All in all, there were 345 students enrolled in all the classes we had visited. A total response of n = 110 was received across all students enrolled in the courses, a response rate of about 33%.

Demographic information was as follows: As for age, 23- to 27-year-old students represented the largest group of participants (45% of respondents). As for gender, there was just about the same number of women as there were of men: 55 and 49 respectively while six (6) respondents preferred not to specify their gender or chose “other”. In terms of education, 37% of participants hadn’t finished their undergraduate studies. The remaining students specified that they had, at least, obtained a bachelor (or equivalent degree).

4.3.  Original Model

The Partial Least Square – Structural Equation Modeling (PLS-SEM) approach allows the modelling and testing of complex cause-and-effect relationships. Two of the strengths of PLS-SEM is that the technique allows the modeling of latent variables and works well with smaller samples [57]. We used SmartPLS version 3.3.0.

Measurement Model – In SmartPLS, the measurement model was solved using the factor weighing scheme. Evaluating the measurement model includes an analysis of (1) reliability, (2) convergent validity, and (3) divergent validity [58]. We used Cronbach’s alpha to analyze reliability and refine our scales. The cut-off value was 0.7 as recommended by [59]. Convergent validity was achieved by retaining items with loadings above or equal to 0.7. We also tentatively retained items with loadings between 0.4 and 0.7. These were later scrutinized further by examining their composite reliability (CR) and average variance extracted values (AVE). If CR was greater or equal to 0.7 and AVE was greater or equal to 0.50, the item was considered acceptable and was retained in the model [60]. All items with loadings that did not attain a minimum threshold of 0.4 were removed, one by one, starting with the item with the smallest loading. After each item removal, the model was recalculated and re-evaluated. This process continued until only acceptable items remained. The final loadings of indicators from the measurement model are presented in the Appendix.

Next, we turned towards establishing discriminant validity. For this, the Heterotrait-Monotrait Ratios (HTMT) were examined. A value of 0.85 or less for HTMT indicate good discriminant validity [61]. The two highest HTMT values in our model were between effectiveness and entertainment and between entertainment and immersion, with values of 0.724 and 0.703, respectively. Acceptable Cronbach alpha, CR, and AVE values presented in Table 2 all point to a good measurement model.

Table 2 – Construct Validity and Reliability

Construct	Cronbach alpha	Composite Reliability	Average Variance Extracted
Assistance and Support	0.922	0.944		0.809
Fidelity	0.922	0.945		0.810
Effectiveness	0.902	0.928		0.720
Aesthetics	0.874	0.911		0.641
Entertainment	0.859	0.914		0.780
Goal Clarity	0.839	0.886		0.611
Immersion	0.834	0.881		0.599
Satisfaction	0.832	0.900		0.749
Reliability	0.829	0.883		0.654
Progressive Challenge	0.804	0.872		0.633
Performance and Feedback	0.775	0.869		0.689
Ease of Use	0.767	0.852		0.660
Control	0.700	0.869		0.768

Structural Model – The structural model was generated using a path weighing scheme. A bootstrap of 5000 subsamples was used to generate the p-values. Path coefficients and t-values using SmartPLS3 are presented in Table 3 below which also shows whether the associated hypothesis is supported or not.

Out of all the paths specified in our original model, only four (4) were significant (at the 0.05 level). The highest path value is between entertainment and satisfaction (0.258), followed by effectiveness and satisfaction (0.205), control and satisfaction (0.187), and fidelity and satisfaction (0.164), in decreasing order. As such, only hypotheses H4, H6, H8, and H10 were supported. The paths not showing up as significant were perplexing. For instance, in our model, ease of use has no significant effect on learners’ satisfaction. This result contradicts numerous studies which evidence the importance of perceived ease of use on users’ technology acceptance. We suspected the presence of multicollinearity in our model.

Table 3: Path Coefficients and p-values (Original Model)

Supported Hypotheses	Path Coeff.	p-value
H4 (Fidelity à Satisfaction)	0.164	0.049*
H6 (Control à Satisfaction)	0.187	0.049*
H8 (Effectiveness à Satisfaction)	0.205	0.038*
H10 (Entertainment à Satisfaction)	0.258	0.014*
Note: * = p < 0.05
Unsupported Hypotheses	Path Coef.	p-value
H1 (Ease of Use à Satisfaction)	0.006	0.937
H2 (Reliability à Satisfaction)	0.088	0.250
H3 (Aesthetics à Satisfaction)	-0.021	0.788
H5 (Assistance and Support à Satisfaction)	0.035	0.656
H7 (Progressive Challenge à Satisfaction)	0.098	0.250
H9 (Performance and Feedback à Satisfaction)	-0.027	0.702
H11 (Immersion à Satisfaction)	0.096	0.267
H12 (Goal Clarity à Satisfaction)	0.082	0.334

4.4.  Multicollinearity, Higher-Order Constructs, and Mediation

Statistical models that contain several independent variables that are highly correlated to one another and not just with the dependent variable can cause a serious problem called multicollinearity. It makes accurate interpretation of statistical results difficult because it increases the standard errors of the coefficients. By overinflating the standard errors, multicollinearity may cause coefficients of some independent variables to appear to be non-significant. In other words, multicollinearity makes it look like the relationships between some independent variables and the dependent variables are statistically insignificant.

To test for multicollinearity, we first ran a SmartPLS model which contained only two (2) constructs: ease of use and satisfaction. As expected, and supported by literature, the path was now highly significant (path=0.408; p=0.000) which suggested the presence of multicollinearity [62] and/or mediating variables [63] in the original model (i.e., misspecified paths).

According to [62], the presence of multicollinearity among the independent factors of a model may indicate that higher-order structures are present and should be considered. Respecifying a model with these higher-order structures can effectively reduce the effects of multicollinearity but should be theoretically argued and empirically tested. In other words, higher-order constructs should not be extracted through a purely data-driven approach. They should be grounded in theoretical logic extracted from the existing literature.

In a model, a lack of statistically significant paths when these are supported by theory (or strong conjectural evidence), can suggest the presence of mediating variables that are not being considered or are incorrectly specified in that model. When mediation occurs (figure 1), an independent variable (X) has little or no statistically discernable direct effect on a dependent variable (Y). Instead, X acts on Y via an intervening variable (M). As such, the independent variable (X) affects the mediating variable (M) which, in turn, affects the dependent variable (Y). In other words, an independent variable (X) influences the dependent variable (Y), but this influence is not direct. Instead, variable X influences variable Y through some other variable (M). Mediation can be complete or partial.

Figure 1: Mediation

4.5.  Alternate Model

Following established guidelines to assess your measurement model and testing that discriminant validity falls within acceptable boundaries are very important steps in Structural Equation Modelling (SEM) but it does not guarantee that your structural model will yield significant paths [61]. Indeed, many of our initial hypotheses were not supported in our original model. Given that our analyses suggest the presence of higher-order constructs and mediation, we reworked our original model by respecifying alternative paths, paired with theoretical justification. These modifications to our original model are currently specified as propositions given that, in the coming year, we intend to explore in more detail existing literature to better substantiate these proposed alternate paths and the addition of higher-order structures.

Higher-Order Construct 1 à Platform Transparency

During our initial analyses, a significant correlation was noted between reliability and ease of use (0.434). Also, among all constructs considered in our study, these two were the only factors that were “technological” in nature.

In a now-famous article entitled “The Computer for the 21^st Century” published in Scientific American in 1991 (pp 66-75), the author Mark Weiser writes that “(t)he most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it”. The article argues that the best technologies are made or eventually become transparent to users. Technology should never be cumbersome to use. Learners’ attention should be drawn to the contents of the communication rather than to the equipment used to communicate it, and players/learners should not be concerned by technological issues. Arguably, difficult to use, unintuitive interfaces, and unreliable IS/IT platforms riddled with recurrent bugs and glitches, are likely to take away from the learner’s overall serious game experience and diminish one’s enjoyment of it, and thus, should reflect negatively on the user’s satisfaction with the serious game.

To represent this in our model, we re-specified reliability and ease of use as dimensions of a second-order construct which we named “Platform transparency”. We define this second-order construct as the perception that the game’s underlying technological platform does not distract from the user’s overall gaming experience and learning objectives because of inadequate design and technical problems.

When technological platforms supporting serious gaming are inherently unstable, unintuitive, and/or difficult to use, it is likely to impede on the overall learning experience, reducing gaming enjoyment and negatively impacting one’s sense of control over the entire experience. Previous empirical studies in IS support our assertions. There is a negative relationship between ease of use and one’s perceptions of control and enjoyment while using information technologies (e.g., [64]).

• P1: The more the game’s technological platform is perceived to be transparent during gaming (easy to use and reliable), the greater the learner’s entertainment while playing the serious game.
• P2: The more the game’s technological platform is perceived to be transparent during gaming (easy to use and reliable), the greater the learner’s sense of control while playing the serious game.

Higher-Order Construct 2 à Guidance

In the original model, our analysis showed non-significant relationships between satisfaction and three (3) independent variables, namely goal clarity, performance and feedback, and assistance and support. Rather than modeling three (3) direct paths to satisfaction, we considered the possibility that mediation was present. It could be argued that, rather than impacting on satisfaction directly, these three (3) antecedents may be factors influencing learners’ perceived effectiveness of the serious game, which would then impact satisfaction. In other words, perceived effectiveness could be a moderating variable. However, the path coefficients from assistance and support (path=0.132; p=0.102), from performance and feedback (path=0.016; p=0.843), and from goal clarity (path=0.130; p=0.112) to perceived effectiveness of the serious game were all non-significant.

Next, we reviewed the definitions of these three constructs collectively while re-examined their individual items to see whether an over-arching concept could connect them together. This led us to re-specify all three (3) concepts as dimensions of a common, second-order construct. We labeled this higher-order structure as “Perceived guidance”. We define this as the degree to which the user perceives that the serious gaming experience provides a clear understanding of goals and sufficient guidance and orientation on how to achieve them by adequately informing learners of their progress and by providing information about what they are doing right/wrong. We propose that the more effective serious games are those characterized by clear, achievable goals where learners’ progress is guided by regular, useful feedback reinforcing successful actions and results all while encouraging users to take corrective measures or try alternate strategies following unsuccessful ones. As such, we reason that guidance will impact satisfaction, but indirectly. The relationship will likely be mediated by perceived effectiveness. The following proposition is then stipulated.

• P3: The greater the perceived guidance received during the serious gaming experience, the more the learner will perceive the serious game as effective.

Mediation

Based on our literature review, our original model hypothesized a significant path between the serious game’s progressive challenge and learner’s satisfaction. Statistically, this path was not significant. We then reasoned that the learner’s perceived effectiveness of the serious game could be mediating this relationship, making it look like progressive challenge has no effect on learners’ satisfaction. Arguably, games designed to be progressively challenging are also likely to be perceived as being more effective by learners given that such games would allow learners to progress in their knowledge and skill acquisition gradually. We postulate the following proposition:

• P4: The greater the perceived progressive challenge of the serious game, the more the learner will perceive the game as being effective for learning.

In our original model, the path between immersion and satisfaction was statistically non-significant. This path was dropped and, instead, we specified a path between the concepts of immersion and entertainment. Studies have recognized that games’ ability to create immersive experiences is among the most pleasurable aspects of video gameplay [65]. This brings us to the following proposition:

• P5: The greater the perceived immersion during serious gameplay, the greater the entertainment.

A path was also drawn linking aesthetics to immersion. Aesthetically superior games help in generating an immersive experience while playing suggesting that both concepts are linked. This relationship is expressed in the following proposition:

• P6: The better the aesthetics of a serious game, the greater the immersive experience perceived by the learner.

Research has also identified a link between the entertainment value of games and their perceived effectiveness [66]. Well-designed, fun games can put learners in a better mood which then positively impacts their perceived effectiveness of the serious game in supporting learning [67]. As such, we propose that a positive relationship exists between the entertainment value of a serious game and the perceived effectiveness of the game as a learning platform.

• P7: The greater the perceived entertainment value of a serious game, the greater the perceived effectiveness of that serious game.

In our original model, the path between fidelity and satisfaction was found to be positive and significant. However, the literature that pertains to the realism of games also suggests that there may be mediating effects between these two factors. It is likely that positive relationships exist between fidelity (how closely the game simulates reality) and the training effectiveness of the serious game and immersion as well [68]. This brings us to the following propositions:

• P8: The greater the perceived fidelity of the serious game, the more the learner will perceive it as effective.
• P9: The greater the perceived fidelity of the serious game, the more the learner will perceive it as immersive.

With these propositions, we specified a new, alternative model which included the higher-order constructs described previously and added paths for mediation (Figure 2). We ran it using bootstrapping with 5000 subsamples.

Figure 2: Structural Model (alternate model)

Tables 4 (hypotheses) and 5 (propositions) present the results of the structural model. In this respecified/alternate model, all paths are significant. This alternate model explains R² = 65.7% of the variance in our data on satisfaction.

Table 4: Hypotheses Path Coefficients and p-values (Alternate Model)

Hypotheses	Path Coeff.	p-value
H4 (Fidelity à Satisfaction)	0.194	0.009**
H6 (Control à Satisfaction)	0.255	0.001**
H8 (Effectiveness à Satisfaction)	0.273	0.002**
H10 (Entertainment à Satisfaction)	0.322	0.000***
Note: ** = p < 0.01; *** = p < 0.001

Table 5: Proposition Path Coefficients and p-values (Alternate Model)

Propositions	Path Coeff.	p-value
P1 (Platform Transparency à Entertainment)	0.310	0.000***
P2 (Platform Transparency à Control)	0.284	0.017*
P3 (Guidance à Effectiveness)	0.220	0.001**
P4 (Progressive Challenge à Effectiveness)	0.252	0.002**
P5 (Immersion à Entertainment)	0.612	0.000***
P6 (Aesthetics à Immersion)	0.219	0.011*
P7 (Entertainment à Effectiveness)	0.360	0.000***
P8 (Fidelity à Effectiveness)	0.197	0.017*
P9 (Fidelity à Immersion)	0.401	0.000***
Note: * = p < 0.05; ** = p < 0.01; *** = p < 0.001

5.      Conclusion

Through a literature review on serious games, we identified several factors which seemingly impact learners’ satisfaction with serious games. Using these factors, we tested an original model using PLS-SEM. This original model specified paths going directly from each factor directly into satisfaction suggesting direct effects.

However, several paths from our original model were statistically non-significant. We respecified an alternate model supported by theoretical foundations and previous research. It included both mediation and second-order factors which helped provide a richer, more complete picture of how factors identified through our literature review relate with one another to positively impact learners’ satisfaction with serious games. Every factor considered in this study was found to have a positive effect on learners’ satisfaction, although not necessarily a direct effect. Several relationships were mediated through other factors while some factors are best combined into higher-order structures. Results suggest that none should be neglected when designing serious games.

Overall, the ludic component plays a central role in serious games. Entertainment value derived from playing serious games has a strong, positive impact on learner satisfaction. Although they are primarily learning tools, people perceive them as games, and as such, expect them to be designed to be fun to play. The importance of entertainment value is followed by whether the learner perceives the serious game as effective, the learners’ sense of control during play, and last, by the fidelity/realism of the game. These findings suggest that to design successful serious games that satisfy learners, it is important to create a ludic environment that is fun, simulates reality well, all while making learners feel like they are in control of this environment. Perceived effectiveness of the serious game holds an important role in generating satisfaction in learners. Perceived effectiveness is a direct antecedent to satisfaction and acts as a mediating variable for several of the other variables identified in our literature review. Although other variables gained mediator status in our alternate model, none mediated a greater number of variables than did effectiveness. Our results also show that some variables do not impact satisfaction directly. Their effects on satisfaction are completely mediated through other factors. This includes aesthetics, immersion, and progressive challenge.

An important limitation of this study is the sample size. Although PLS-SEM handles small sample sizes well, small sample sizes raise questions about the generalizability of results. Another limitation of this study is that it is cross-sectional. This means that results do not take into account how time influences the relationship among variables. A factor can be more influential on satisfaction from the onset of play. Others may become more important in explaining satisfaction later on, as the learner amasses more experience. For instance, the importance of providing increasingly challenging and more varied learning situations may become more and more important in explaining whether players continue to derive satisfaction from a serious game as their experiences and skill set grow. As such, the temporal stability of factors as antecedents of satisfaction should be investigated in future studies. Moreover, our surveys use self-reporting. These can cause respondents to inflate their opinions and responses making it hard to capture the true relationships among constructs. Interviews could be conducted in the future to validate our results. Since the respondents are all business students a generalization can be challenged for other disciplines. Finally, research is further needed to identify additional factors that may impact satisfaction with serious games and to explore the nature of the relationships that may exist between these factors.

Conflict of Interest

The authors declare no conflict of interest.

1. J. Alvarez, D. Djaouti, O. Ramouroux, Apprendre avec les serious games? Editions Canopé, Poitiers, France, 2016.
2. B. Cowan, B. Kapralos, “An overview of serious game engines and frameworks,” Intelligent Systems Reference Library, 119, 15-38, 2017, doi:10.1007/978-3-319-49879-9_2.
3. P. Cohard, “L’apprentissage dans les serious games : proposition d’une typologie,” @GRH, 16(3), 11-40, 2015, doi : 10.3917/grh.153.0011.
4. M.S. Calabora, A. Moraa, S. Moyab, “The future of serious games in accounting education: a delphi study,” Journal of Accounting Education, 46, 43-52, 2019, doi:/10.1016/j.jaccedu.2018.12.004.
5. R. Valentine, “Metaari: game-based learning market will reach $17 billion by 2023,” Gamesindustry.biz, 8th August 2018, https://www.gamesindustry.biz/articles/2018-08-08-metaari-game-based-learning-market-will-reach-usd17-billion-by-2023.
6. A. Usoltsev, “Serious games: 5 gamification pitfalls to avoid,” August 8 2018, https://zenodo.org/record/2847262
7. A. Messaoud, “Conception d’un jeu sérieux actif pour l’apprentissage intergénérationnel du français,” Québec, Canada: Université de Laval, 2016.
8. S. De Freitas, “Are games effective learning tools? a review of educational games,” Educational Technology & Society, 21(2), 74–84, 2017, http://www.jstor.org/stable/26388380.
9. C.J. Stettina, T. Offerman, I. Sidhu, B. De Mooij, “Gaming for agility: using serious games to enable agile project & portfolio management capabilities in practice,” Engineering, Technology and Innovation (ICE) & IEEE International Technology Management Conference, Stuttgart, 2018, doi.org/10.1109/ice.2018.8436384
10. R. Rodela, A. Ligtenberg, R. Bosma, “Conceptualizing serious games as a learning-based intervention in the context of natural resources and environmental governance,” Water, 11(245), 1-15, 2019, doi.org/10.3390/w11020245.
11. J. Lumsden, A. Skinner, A.T. Woods, N.S. Lawrence, M. Munafò, “The effects of gamelike features and test location on cognitive test performance and participant enjoyment,” PeerJ., 4, 2-19, 2016, doi.org/10.7717/peerj.2184.
12. I. Bououd, R. Fafi. “Using serious games to correct french dictations: proposal for a new unity3d/nooj connector,” Association for Computational Linguistics, 49–52, Septembre 4 2017, doi.org/10.18653/v1/W17-3808.
13. A. Duguet, S. Morlaix, “Le numérique à l’université : facteur explicatif des méthodes pédagogiques?,” Revue Internationale de Pédagogie de l’Enseignement Supérieur, 34(3), 1-18, 2018, doi.org/10.4000/ripes.1682.
14. J. Stewart, L. Bleumers, J, Van Looy, I. Mariën, A. All, D. Schurmans, K. Willaert, F. De Grove, A. Jacobs, G.J. Misuraca, L. Stewart, V.-L.J. Bleumers, M. Ilse, A. All, D. Schurman, K. Willaert, “The potential of digital games for empowerment and social inclusion of groups at risk of social and economic exclusion: evidence and opportunity for policy,” Institute for Prospective Technological Studies – European Commission, II, 172, 2013, doi.org/10.2791/88148.
15. W.S. Chen, A.Y. Yao, “An empirical evaluation of critical factors influencing learner satisfaction in blended learning: a pilot study,” Universal Journal of Educational Research, 4(7), 1667-1671, 2016, doi:10.13189/ujer.2016.040719.
16. K.C.M. Bul, I.H.A. Franken, S. Van der Oord, P.M. Kato, M. Danckaerts, L.J. Vreeke, A.Willems, H.J.J. van Oers, R. van den Heuvel, R. van Slagmaat, and A. Maras. “Development and user satisfaction of plan-it commander a serious game for children with ADHD,” Games for Health Journal, 4(6), 502-512, 2015, doi.org/10.1089/g4h.2015.0021.
17. M.A. Khenissi, F. Essalmi, M. Jemni, “Comparison between serious games and learning version of existing games,” Procedia – Social and Behavioral Sciences, 191, 487-494, 2015, doi.org/10.1016/j.sbspro.2015.04.380.
18. B. Budie, R. Appel-Meulenbroek, A. Kemperman, M. Weijs-Perree, “Employee satisfaction with the physical work environment: the importance of a need based approach,” International Journal of Strategic Property Management, 23(1), 36-49, 2019, doi.org/10.3846/ijspm.2019.6372.
19. A. Akhmedova, A.Manresa, D. Escobar Rivera, A. Bikfalvi, “Service quality in the sharing economy: a review and research agenda,” International Journal of Consumer Studies, 45(4), 889-910, 2021, doi.org/10.1111/ijcs.12680.
20. N.M. Martirosyan, D.P. Saxon, R. Wanjohi, “Student satisfaction and academic performance in armenian higher education,” American International Journal of Contemporary Research, 4(2), 1-5, 2014, http://www.aijcrnet.com/journals/Vol_4_No_2_February_2014/1.pdf.
21. E. Alsadoon, “The impact of social presence on learners’ satisfaction in mobile learning,” TOJET: The Turkish Online Journal of Educational Technology, 17(1), 226-233, 2018, https://files.eric.ed.gov/fulltext/EJ1165749.pdf.
22. R. Vaezi, A. Mills, W.W. Chin, H. Zafar, “User satisfaction research in information systems: historical roots and approaches,” Communications of the Association of Information Systems, 38, 501-532, 2016, doi.org/10.17705/1CAIS.03827
23. W.H. DeLone, E.R. McLean, “Information systems success: the quest for the dependent variable,” Information Systems Research, 3(1), 60-95, 1992, doi.org/10.1287/isre.3.1.60.
24. W.J. Doll, G. Torkzadeh, “The measurement of end-user computing satisfaction,” MIS Quarterly, 12(2), 259-274, 1988, doi.org/10.2307/248851.
25. G. Petri, C. Gresse von Wangenheim, “How to evaluate educational games: a systematic literature review, Journal of Universal Computer Science, 22(7), 992-1021, 2016, doi.org/10.1016/j.compedu.2017.01.004.
26. F.D. Davis, “Perceived usefulness, perceived ease of use and user acceptance of information technology,” MIS Quarterly, 13(3), 319-340, 1989, doi.org/10.2307/249008.
27. Y.H. Al-Mamary, A. Shamsuddin, A.H. Nor Aziati, “Key factors enhancing acceptance of management information systems in yemeni companies,” Journal of Business and Management Research, 5, 108-111, 2014, https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1054.2814&rep=rep1&type=pdf.
28. S. Dhar, V. Ordonez, T.L. Berg, “High level describable attributes for predicting aesthetics and interestingness,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, 1657–1664, 2011, doi:10.1109/CVPR.2011.5995467.
29. M. Pajusalu, The evaluation of user interface aesthetics, Master Thesis, Rennes/Tallinn: Tallinn University Institute of Informatics, 2012, http://www.cs.tlu.ee/teemad/get_file.php?id=202.
30. J.A. Abubakar, A.S. Bahrin, M.K. Ahmad, A.N. Zulkifli, “Conceptual model of game aesthetics for perceived learning in narrative games,” International Journal on Advanced Science Engineering Information Technology, 7(3), 993-999, 2016, doi: 0.1109/IUSER.2016.7857944.
31. Z. Herbert, Sight, Sound, Motion: Applied Media Aesthetics (6th ed.), Wadsworth Publishing Co. Inc., 2010.
32. C. Crawford, The Art of Computer Design, Berkeley, CA: Osborne/McGraw-Hill, 1984.
33. J. Pellegrino, A. Scott, “The Transition from Simulation to Game-Based Learning,” Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Dec. 6-9, 2004, https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.5486&rep=rep1&type=pdf
34. K. McKnight, K. O’Malley, R. Ruzic, M.K. Horsley, J.J. Franey, K. Bassett, “Teaching in a digital age: how educators use technology to improve student learning,” Journal of Research on Technology in Education, 48(3), 194-211, 2016, doi.org/10.1080/15391523.2016.1175856.
35. D. Djaouti, Serious Game Design: Considération Théoriques et Techniques sur la Création de Jeux Vidéo à Vocation Utilitaire, PhD Thesis, Toulouse University, 2011.
36. M. Busch et al., “Personalization in Serious and Persuasive Games and Gamified Interactions,” in Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY ’15) ACM, 811-816, 2015, https://uwspace.uwaterloo.ca/bitstream/handle/10012/12804/p811-busch.pdf?sequence=3
37. N. Whitton, Learning and Teaching with Computer Games in Higher Education, Games-Based Learning Advancements for Multi-sensory Human Computer Interfaces: Techniques and Effective Practices, Chapter II, 18-33, 2009, doi.org/10.4018/978-1-60566-360-9.ch002
38. D.I. Cordova, M.R. Lepper, “Intrinsic motivation and the process of learning: beneficial effects of contextualization, personalization and choice,” Journal of Educational Psychology, 88, 715-730, 1996, doi.org/10.1037/0022-0663.88.4.715.
39. J.J. Vogel, D.S. Vogel, J. Cannon-Bowers, C.A. Bowers, K. Muse, M. Wright, “Computer gaming and interactive simulations for learning: a meta-analysis,” Journal of Educational Computing Research, 34(3), 229–243, 2006, doi.org/10.2190/FLHV-K4WA-WPVQ-H0YM.
40. N. Shin, L.M. Sutherland, C.A. Norris, E. Soloway, “Effects of game technology on elementary student learning in mathematics,” British Journal of Educational Technology, 43(4), 540-560, 2011, doi.org/10.1111/j.1467-8535.2011.01197.x.
41. B. Ingadottir, K. Blondal, D. Thue, S. Zoega, I. Thylen, T. Jaarsma, “Development, usability, and efficacy of a serious game to help patients learn about pain management after surgery: an evaluation study,” JMIR Serious Games, 5(2), 1-15, 2017, doi: 10.2196/games.6894.
42. S. Tobias, J.D. Fletcher, Computer Games and Instruction, Charlotte: IAP Information Age Publishing, 2011.
43. D. Drummond, A. Hadchouel, A. Tesnière, “Serious games for health: three steps forwards,” Advances in Simulation, 2(3), 1-8, 2017, doi.org/10.1186/s41077-017-0036-3.
44. J. Loisier, Étude sur l’Apport des Jeux Sérieux pour la Formation à Distance au Canada Francophone, Réseau d’Enseignement Francophone à Distance (REFAD), 2015, https://www.refad.ca/documents/Etude_Jeux_serieux_en_FAD.pdf.
45. M.D. Kickmeier-Rust, D. Albert, “Micro-adaptivity: protecting immersion in didactically adaptive digital educational games,” Journal of Computer Assisted Learning, 26(2), 95-105, 2010, doi.org/10.1111/j.1365-2729.2009.00332.x.
46. N. Iten, D. Petko, “Learning with serious games: is fun playing the game a predictor of learning success?,” British Journal of Educational Technology, 47(1), 151–163, 2016, doi.org/10.1111/bjet.12226.
47. R. Tamborini, N.D. Bowman, A. Eden, M. Grizzard, A. Organ, “Defining media enjoyment as the satisfaction of intrinsic needs,” Journal of Communication, 60, 758–777, 2010, doi.org/10.1111/j.1460-2466.2010.01513.x.
48. I. Marfisi-Schottman, S. George, F. Tarpin-Bernard, P. Prevot, “Comment Évaluer la Qualité d’un Learning Game Pendant sa Conception?,” in Technologies de l’Information et de la Communication pour l’Enseignement, TICE2012, Lyon: France, 80-90, 2012, https://hal.archives-ouvertes.fr/hal-00963678/document.
49. G. Calleja, “Revising Immersion: A Conceptual Model for The Analysis of Digital Game Involvement,” in Proceedings of the 2007 Digital Games Research Association (DiGRA) International Conference, 82-90, 2007, http://www.digra.org/wp-content/uploads/digital-library/07312.10496.pdf.
50. A. McMahan, “Immersion, Engagement and Presence: A New Method for Analyzing 3-D Video Games,” in M. J. Wolf, & B. Perron (Eds), The Video Game, Theory Reader, New York: Routledge: Taylor & Francis Group, 67-87, 2003.
51. M. Ebrahimzadeh, S. Alavi, “Digital video games: e-learning enjoyment as a predictor of vocabulary learning,” Electronic Journal of Foreign Language Teaching, 14(2), 145–158, 2017, http://e-flt.nus.edu.sg/v14n22017/ebrahimzadeh.pdf.
52. J.G. Peery, C. Pasalar, “Designing the learning experiences in serious games: the overt and the subtle-the virtual clinic learning environment,” Informatics, 5(30), 1-14, 2018, doi.org/10.3390/informatics5030030.
53. M. Ma, A. Oikonomou, L.C. Jain, Serious Games and Edutainment Applications (ed. London Limited). New York: Springer-Verlag, 2011.
54. Y. Wang, R. Pramod, C. Sankar, P.K. Raju, “Relationships between Goal Clarity, Concentration and Learning Effectiveness When Playing Serious Games,” in Proceeding of Twentieth Americas Conference on Information Systems, 2014 https://www.researchgate.net/publication/263426149_Relationships_between_Goal_Clarity_Concentration_and_Learning_Effectiveness_when_Playing_Serious_Games
55. G.A. Churchill, “A paradigm for developing better measures of marketing constructs,” Journal of Marketing Research, 16, 64-73, 1979, doi.org/10.1177/002224377901600110.
56. G.C. Moore, I. Benbasat, “Development of an instrument to measure the perceptions of adopting an information technology innovation,” Information Systems Research, 2(3), 192-222, 1991, doi.org/10.1287/isre.2.3.192.
57. F. Ali, S.M. Rasoolimanesh, M. Sarstedt, C.M. Ringle, K. Ryu, “An Assessment of the use of partial least squares structural equation modeling (pls-sem) in hospitality research,” International Journal of Contemporary Hospitality Management, 30(1), 514-538, 2018, doi.org/10.1108/IJCHM-10-2016-0568.
58. W. Esposito-Vinzi, V., Henseler, J., & Wang, H. (2009). How to Write Up and Report PLS Analyses. Handbook of Partial Least Squares, pp 655-690
59. J.C. Nunnally, I.H. Bernstein, Psychometric Theory (ed. 3rd). New York: McGraw-Hill, 1994.
60. J.F. Hair, C.M. Ringle, M. Sarstedt, “PLS-SEM: indeed a silver bullet,” Journal of Marketing Theory and Practice, 19(2), 139-152, 2011, doi.org/10.2753/MTP1069-6679190202.
61. J. Henseler, C.M. Ringle, M. Sarstedt, “A new criterion for assessing discriminant validity in variance-based structural equation modeling,” Journal of the Academy of Marketing Science, 43(1), 115-135, 2015, doi.org/10.1007/s11747-014-0403-8.
62. C. Schwarz, A. Schwarz, W.C. Black, “Examining the impact of multicollinearity in discovering higher-order factor models,” Communications of the Association for Information Systems, 34(62), 1191-1208, 2014, doi:10.17705/1CAIS.03463.
63. R. Agler, P. De Boeck, “On the interpretation and use of mediation: multiple perspectives on mediation analysis,” Frontiers in Psychology, 15, 1-11, 2017, doi:10.3389/fpsyg.2017.01984.
64. V. Venkatesh, “Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model,” Information Systems Research, 11(4), 342–365, 2000, doi.org/10.1287/isre.11.4.342.11872.
65. L. Michailidis, E. Balaguer-Ballester, X. He, “Flow and Immersion in Video Games: The Aftermath of a Conceptual Challenge, Frontiers in Psychology, 05 September 2018, doi: 10.3389/fpsyg.2018.01682.
66. N. Nazry, M. Nazrina, D.M. Romano, “Mood and learning in navigation-based serious games,” Computers in Human Behavior, 73, 596–604, 2017, doi.org/10.1016/j.chb.2017.03.040.
67. Y. Zhonggen, “A meta-analysis of use of serious games in education over a decade,” International Journal of Computer Games Technology, 1, 1-8, 2019, doi.org/10.1155/2019/4797032.
68. D. Rojas, B. Kapralos, S. Cristancho, K. Collins, A. Hogue, C. Conati, A. Dubrowski, “Developing effective serious games: the effect of background sound on visual fidelity perception with varying texture resolution,” Studies in Health Technology and Informatics, 173, 386-392, 2012, doi:10.3233/978-1-61499-022-2-386.

(Click Here)