1. Introduction

Gamification is defined by [1] as the application of game elements in a non-game context. Its main objective is to involve students in their learning process and not only internalisation of what is exposed [2]. There are several educational software programmes based on gamification, which aim to increase enthusiasm and attention in the classroom to achieve learning outcomes. The key aim is to develop skills or to feel pleasure, all of which are intellectual characteristics of motivation in humans [3], being generally considered as an effective way to create constructivist learning environment [4-5]. However, certain aspects need to be clarified as the effectiveness in its use under online distance learning.

Since March of 2020, as a way to control the spread of the COVID-19 virus, citizens all over the world were ordered to stay at home by their governments. Therefore, all educational institutions, including universities, had to adapt their teaching methodologies to online formats in record time. So that it was necessary to implement new modes of teaching and learning compatibles with virtual classes. In this paper, an experience of using Kahoots during online teaching period due to COVID-19 pandemic is described.

The experience took place with second year students of the double degree between “Food Science and Technology” at Extremadura University (UEx) and “Food Quality and Safety” at Chongquin University (UEC) in the subject named “Bases and Technology of Plant Production”. Since the 2020/21 academic year, and due to the COVID-19 pandemy, all the formation was online and the real-time quiz Kahoot was used to consolidate knowledge. Under these special conditions, the first part of this paper analyse the degree of Kahoot acceptance and motivation using the existing survey on the website. Secondly, in order to study whether the use of Kahoots helps in the consolidation of knowledge, not only in theoretical questions by means of multiple-choice tests, but also in the recognition of different plant species by means of test with photos, certain questions belonging to the Kahoots used in class, were repeated in a partial exam and in the final one.

2. Literature Review

Recently, the use of digital devices with internet connection has become popular in many aspects of the life, such as social activities, business, and in the educational system [6-8]. In fact, there are several Educational Software’s used in a classroom environment for teaching and learning using gamification.

In line with [9] gamification is “using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems”. Games have some distinctive features which play a key role in gamification, between them [10]:

• users are all participants
• challenges/tasks that users perform and progress towards defined objectives
• points that are accumulated because of executing tasks
• levels which users pass depending on the points
• badges which serve as rewards for completing actions
• ranking of users according to their achievements

The objectives to be achieved by the implementation of games in learning can be summarised as being engaging, fun and exciting for players, thus stimulating motivation for learning, offering the opportunity for participants to explore their curiosity through new challenges [11]. Therefore, it combats one of the main frustrations of teachers, at all levels of education, namely demotivation. A problem that, as many studies have pointed out, cannot be solved by conventional methods [12-14] and gamification has been suggested as an effective tool to improve students´ motivation and engagement [15]. Current game-based learning should focus on the application of extrinsic game-like elements, such as levels, challenges, point systems and rankings, to create a user experience that engages students to complete learning tasks [16].

In recent years, tools for gamifying learning in the classroom have proliferated allowing educators new tools in giving lessons and many of them take the form of applications for mobile devices [17-19]. Kahoot is one example of a playful, game-based student response system which has received wide acceptance with more than 70 million active users across the world monthly [20] reaching that, in 2019, over 2.5 billion people from more than 200 countries have played Kahoot [21]. It is based on three basic values: play, learn and socialise, being simplicity of creating the game experience and its easy access through mobile devices are two key points.

Kahoot is a very versatile tool that allows multiple options: true-false, simple or multiple answers, allows to include images, videos, or to configure different variables, such as the available time to answer the questions [21-22]. The questions are projected in the classroom and the students, through their smartphones, answers them, adjusting to the marked time. Provides immediate feedback on the degree to which students have acquired knowledge and identifies content that has not been understood. After each question, the respective winner and points accumulated is shown to provide a final ranking, as if it were a competition. Finally, after each question, the teacher can explain each answer, which is where the pedagogical potential lies [23]. The game ranks players based on speed and accuracy, and at the end of the quiz, the names of the top five players are displayed on the leaderboard. In this way, students can compete to see who gets the most questions right in the shortest time, being this its main handicap, according to [17, 24]. The fact that higher scores are obtained for quicker answers can create anxiety and stress for many students, who risk too much to get more points, to the detriment of spending enough time thinking about the correct answer. Results can be downloaded by educators allow them to highlight problematic questions and identify students who may be struggling [21-23].

Gamification in general and Kahoot in particular, has been widely used in teaching, generally at pre-university levels and in face-to-face teaching. However, analysing the most recent reviews on the use of gamification in teaching [25-26] highlight that, of the 93 and 24 studies analysed, both at pre- and university levels, the positive properties can be summarised as an increase in motivation, in classroom dynamics and amenity, students´ and teachers´ attitudes, positive effect on learning performance and decreasing of student anxiety [26-28]. On the contrary, the main problems encountered included bad internet connection, the impossibility to change the answer after submission and the time pressure to give answers, which reduces students’ reflection and some students’ guessing without thinking because of the fear of losing [17, 21, 23-24]. With regards to the advantages and limitations related to game-based learning in a foreign language, the same advantages and disadvantages were found in a recent review [29].

Finally, the response of most universities worldwide to the crisis caused by the Coronavirus disease (COVID-19) was the transition to online environments. In these cases, the use of motivational and energising tools in the classroom becomes even more necessary. However, their effect on online teaching needs to be studied and clarified. To clear up these issues, with particular emphasis on the students’ opinion and their effectiveness on the consolidation of knowledge, this study has been carried out.

3. Methodology

3.1.  Double degree between Extremadura and Chongquin Universities

      Since June 2019, the University of Extremadura in Spain (UEx) and Chongquin University in China (UEC) established the double degree between “Food Science and Technology” at UEx and “Food Quality and Safety” at UEC.

      The initial planning of the double degree had a duration of four years, in which students will receive the following teaching:

• First year. UEC students will receive an intensive Spanish course.
• Second and third year. UEC students will be taught eight subjects, two per semester and year, taught in Spanish by UEX professors.
• Fourth year. UEC students will travel to UEx to be taught two subjects and to develop their Final Projects.

However, due to the outbreak of Covid-19, the governments of many countries, including China and Spain, forced their citizens to stay at home. This led to major changes not only in the lives of citizens, but also in the education system. The main consequence was the impossibility of teaching the classes in person, as the UEx teachers could not travel to the UEC. However, the UEC maintained face-to-face attendance during 2020/2021 course. Therefore, the classes were online, using the Zoom platform, and while the UEx teachers were in Spain, the UEC students and teachers were together in a classroom in China. In order to reinforce the comprehension of the lessons, and so that the language was not an impediment to the development of them, the Chinese teacher of the subject, as well as a translator Chinese-Spanish participated actively in all the sessions. Moreover, all presentations were provided to the students translated with the enough advance and the subtitles available in Power Point 365 were used.

3.2.  Procedure

      In the specific case of this study, the subject taught was “Bases and Technology of Plant Production”, taught in the second semester of the second year. This subject is divided into 14 lessons taught in 34 sessions of 1.5 hours sessions. The number of students enrolled was 73, divided into two groups that received exactly the same contents. The evaluation of the contents´ subject was divided into two parts:

1. Evaluation of the theoretical contents by means of a multiple-choice test
2. Evaluation of the differentiation of the main crops studied (extensive, horticultural and fruit crops) by means of their recognition by means of different photographs

In order to review the theoretical content and assess the students’ level of understanding, a Kahoot quiz with 15 questions was given at the end of each lesson. To ensure that the time available to answer each question was sufficient, and taking into account the language handicap, it was extended to 1 minute. In the middle of the course, after seven topics and, therefore, seven Kahoots, a partial exam was held with 50 multiple-choice questions with four possible answers, but only one correct. To determine the knowledge memorisation or if significant learning was achieved, two aleatory questions were chosen to be repeated from each of the Kahoots gamed after each lesson, i.e. a total of 14 questions. Also, at the end of the course, there was a final exam consisting of two parts, a 50-question test, in which the same questions of the partial were repeated and other two questions aleatory chosen from the last seven lessons´ Kahoots made by the students throughout the course, thus a total of 28 questions, 14 from the partial and 14 of the last seven Kahoots (Figure 1).

The time elapsed between the midterm and the final exam, i.e. 17 weeks, will also allow us to know whether the knowledge acquired is consolidated over time.

Figure 1. Phases in the study.

For the evaluation of the recognition of the different plant species studied, along the last month of the course, four Kahoots were carried out with 11 questions asking which species could be seen in the photo (Figure 2). In this case, the final exam was a test with 15 questions, in which eight photos were repeated, two from each of the Kahoots carried out in class (Figure 1).

Figure 2. Example of identification question on the Kahoot website.

3.3.  Opinion on the use of Kahoot as a learning tool

      After the seventh questionnaire, students were asked to fill in the feedback available in the website itself to determine the students’ perception of the use of the tool based on the student response system with different scales (Table 1).

Table 1. Opinion questions and their scales.

Question	Scale
1. How fun was it?	1         1 (no fun) to 5 (very fun)
2. Did the players learn anything from this Kahoot?	No-Yes
3. Could players recommend this Kahoot?	No-Yes
4. How did the players feel about this Kahoot?	Negative – Neutral – Positive

3.4.  Data analysis

The obtained data after the evaluation of the questionnaire items was quantitative and based on a descriptive analysis in SPSS 23.

4.       Result and Discussion

4.1.  Degree of theory knowledge consolidation.

Figure 3 shows how Kahoot improved students´ memorisation of knowledge in the theoretical questions, as they obtained better results when the questions were repeated, not only in the midterm, with much less time elapsed, but also in the final exam. Specifically, the average score obtained in the 14 chosen questions from Kahoots 1 to 7 increased in about 31.4% in the midterm, and 29.6% in the final exam. This is a very good result because the time elapsed from the first to seven Kahoots to the repetition of those questions was between 32 to 17 weeks. Regarding the repeated questions from Kahoots 8 to 14, the increases were about 26.0%.

Figure 3. Average of the obtained scores in the 14 chosen questions corresponding to Kahoots 1 to 7 and 8 to 14, in their repetition in the midterm and in the final exam.

The creator of Kahoot stated in his study at the Norwegian University of Science and Technology (NTNU) that the use of Kahoot increased learning by 22% over students who used other game dynamics [30]. A previous study [24] ], also found lightly lower increases in a similar study, but in presential learning, and [31-32] found a good degree on knowledge consolidation, in contrast to [33-34] who did not find any significant effect on the knowledge consolidation, considering that repetition and frequency of use are directly related to its efficiency.

4.2.   Degree of identification knowledge consolidation.

The first rounds of each Kahoot had an average of 6.2 correct answers from 8 possible, a high percentage, indicating that the contents are easily interpreted and memorised. The increase between the questions in the first Kahoot and their repetition in the final exam was of 17.3%, quite lower in comparison with the theory questions. But, considering the rate achieved is out of 8 points, the results are good, being even better in their repetition. In this case, two facts are worth noting: firstly, the time that has passed is much shorter, from 2 to 8 weeks; and secondly, the fact that it is a question of recognising photos, with the photographic memory being much more actively involved.

Figure 4. Average of the scores obtained in the 8 chosen questions corresponding to Kahoots 10 to 14 and in their repetition in the final exam.

So, our finding would be according with that found in the review of [25], which included 36 studies investigating the learning outcome of only using Kahoot. Of them, 70% of the studies showed that Kahoot significantly improves the final grade or test results compared to other teaching approaches.

4.3.  Opinion on the use of Kahoot as a learning tool

Table 2 confirms that the students´acceptation was really good, with percentages of the positive answers for all the items higher than 90%.

For the question 1 relative to the fun of playing the game, only 3 of the surveyed students showed a neutral or negative opinion regarding the fun of the game. As is showed in question 2, more than 97% of students feel they have learned something and more than 95% would recommend using Kahoot as a review tool (question 3). Only when they were asked about how it makes them feel (question 4), the percentage of students with either an indifferent (9.59%) or negative (2.74%) opinion was substantially higher, although a large majority (87.67%) had a positive perception (Table 2).

Table 2: Questionnaire data analysis (N=73).

	1 (%) No Negative	2 (%)	3 (%) Neutral	4 (%)	5 (%) Yes Positive	Average (1 to 5)	Median
1. Fun	0	1.37	2.74	23.29	72.60	4.74	5
2. Learn	2.74	–	–	–	97.26	–	Yes
3. Recom.	4.11	–	–	–	95.89	–	Yes
4. Feel	2.74	–	9.59	–	89.67	–	Positive

The results found in this study are notable for the fact that the teaching was online and not face-to-face. Numerous studies [27-28, 31-32, 35] agree in conclude that Kahoot is an effective instrument for carrying out presential activities in the classroom in a fun way, breaking the monotony and improving student participation, fostering a positive relationship between the group of students. In fact, authors such as [33] indicate a higher rate of class attendance. Most students perceive Kahoot as a game and not as an assessment system [36]. All these high levels of acceptance such as those found by [24], with very similar values in classroom teaching of this subject, with more than 95% of students with very positive evaluations of the use of Kahoot, or [31] who obtained lower values, with 87% acceptance. Rejection is often related to the inability to change the answer after submission, stressful time pressure to give answers, lack of time to answer, fear of losing, and difficulty in catching up if an incorrect answer has been given. [25, 33].

Moreover, the fact that the teaching is in another language does not seem to influence the students’ opinion. However, the reinforcement of the Chinese-Spanish translator probably reduced the complication of understanding another language. Finally, in terms of perception in language classes, [37] found a greater willingness to actively participate in Arabic classes, with [38] demonstrating that students learned vocabulary more easily using Kahoot.

5.       Conclusion

Although some of the limitations of this study are that it was developed in a single semester, that the number of students is quite limited and that, all students belong to the same group instead of dividing them into two groups, one control and one using Kahoot, more than 90% of students found it fun, that helps to learn and it’s recommended as an engagement and students´ motivation tool. However, a small proportion felt indifferent or unpleasant when using this tool. This is where future research should investigate further to see what factors lead to lower acceptance, with particular emphasis on the stress caused by limited response time, the impossibility of changing the response, and how to remedy them.

The results demonstrate its effectiveness in consolidating knowledge. The number of answers that students remembered between questions asked in class using Kahoot and then repeated between 17 and 34 weeks was between 26 and 31%, regardless of the time elapsed between the two. Despite of the obtained results are consistent with previous research; further research must be done in order to corroborate the effectiveness of these questionnaires. This would require more experiments with more students and of longer duration, and more attention to the anxiety caused by the short time to answer.

Conflict of Interest

The author declares no conflict of interest.

Acknowledgment

Author gratefully thank the students of “Bases and Technology of Plant Production” at Chongquin University, for their participation.

1. S. Deterding, M. Sicart, L. Nacke, K. O’Hara, D. Dixon, Gamification: Using game design elements in non-gaming contexts, 2011, doi:10.1145/1979742.1979575.
2. R. Al-azawi, F. Al-faliti, M. Al-blushi, “Educational gamification vs. game-based learning: Comparative study,” in International Journal of Innovation, Management and Technology, 7(4), 132–136, 2016, doi: 10.18178/ijimt.2016.7.4.659.
3. R.E. White, “The power of play: a research summary on play and learning,” Smart Play, 15–25, 2013.
4. M.J. Poblaciones, T. Garcia-White, C. Marin, “Students´ Perception of Real-Time Quiz Kahoot! As a Review Tool in Higher Education,” International Journal of Engineering Pedagogy, 11(4), 165-175, 2021. doi:10.3991/ijep.v11i4.21359
5. S. Papadakis, “The use of computer games in classroom environment,” International Journal of Teaching, 9(1), 1-25, 2018. doi:10.1504/IJTCS.2018.10011113.
6. B. Bature, “The role of information and communication technology as a tool for effective teaching and learning of mathematics,” Journal of Applied of Computational Mathematics, 5(6), 1–3, 2016, doi:10.4172/2168-9679.1000333
7. A. Colbert, N. Yee, G. George, “The digital workforce and the workplace of the future,” Academy of Management Journal, 59(3), 731–739, 2016. doi: 10.5465/amj.2016.4003.
8. K. Bedi, “Tablet PC and smartphone uses in education (TabletTours),” In 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 940-945, 2014, doi: 10.1109/MIPRO.2014.6859703.
9. K.M. Kapp, “The gamification of learning and instruction: Game-based methods and strategies for training and education,” Pfeiffer, San Francisco, CA, 2012.
10.  G. Kiryakova, N. Angelova, L. Yordanova, “Gamification in Education,” 9th International Balkan Education and Science Conference, 2014.
11. F. Rozi, Y. Rosmansyah, B. Dabarsyah, “A Systematic Literature Review on Adaptive Gamification: Components, Methods, and Frameworks,” in 2019 International Conference on Electrical Engineering and Informatics (ICEEI), 187–190, 2019, doi:10.1109/ICEEI47359.2019.8988857.
12. G. Barata, S. Gama, J. Jorge, D. Gonçalves, “Gamification for smarter learning: Tales from the trenches,” Smart Learning Environments, 2(1), 1–23, 2015, doi:10.1186/s40561-015-0017-8
13. K.R. Bell, “Online 3.0-The rise of the gamer educator the potential role of gamification in online education,” Dissertation available from ProQuest. AAI3635727. https://repository.upenn.edu/dissertations/AAI3635727, 2014.
14. P. Buckley, E. Doyle, “Gamification and student motivation,” Interactive Learning Environments, 24(6), 1162–1175, 2016, doi: 10.1080/10494820.2014.964263.
15. D.O. Göksün, G. Gúrsoy, “Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz,” Computers &Education, 135, 15-19, 2019, doi:10.1016/j.compedu.2019.02.015
16. F.L. Khaleel, N.S. Ashaari, T.S.M.T. Wook, A. Ismail, “Gamification Elements for Learning Applications,” International Journal on Advanced Science, Engineering and Information Technology, 6(6), 868, 2016. doi:10.18517/ijaseit.6.6.1379.
17. M. Esteves, A. Pereira, N. Veiga, R. Vasco, A. Veiga, “The use of new learning technologies in Higher education classroom: case of study,” International Journal of Engineering Pedagogy, 8, 115-127, 2018, doi:10.3991/ijep.v8i2.8146
18. T. Hailikari, T. Tuononen, A. Parpala, “Students’ experiences of the factors affecting their study progress: differences in study profiles,” Journal of Further and Higher Education, 42(1), 1–12, 2018. doi: 10.1080/0309877X.2016.1188898
19. J. Hamari, J. Koivisto, H. Sarsa, “Does Gamification Work? – A Literature Review of Empirical Studies on Gamification,” Proceedings of the 47th Hawaii International Conference System Science, 2014, doi: 10.1109/HICSS.2014.377.
20. E. Harrell, “Kahoot! Reached 70 million unique users on its platform. Kahoot!,” [Accessed 7 November 2021], 2019, Available at: https://kahoot.com/blog/2018/01/18/70-million-unique-users-kahoot/
21. I. Vick, “Training professionals from three countries share their Kahoot!’,” 19, 2019, https://kahoot.com/blog/2019/09/10/top-training-tips-kahoot-around-world/: Kahoot!.
22. Kahoot! (2015). KAHOOT! is one of the worlds faster growing learning brands. [Accessed 14 November 2021], 2015, available at http://support.kahoot.com/hc/en-us/categories/115000091647-About-Kahoot.
23. A. Alawadhi, E.A.S. Abu-Ayyash, “Students’ perceptions of Kahoot! An exploratory mixed-method study in EFL undergraduate classrooms in the UAE,” Education and Information Technologies, 1-30, 2021, doi:10.1007/s10639-020-10425-8.
24. M.J. Poblaciones, T. Garcia-White, C. Marin, “Students´ Perception of Real-Time Quiz Kahoot! As a Review Tool in Higher Education: A Case of Study,” International Journal of Engineering and Pedagogy, 11(4), 2021, doi: 10.3991/ijep.v11i4.21359.
25. A. Wang, R. Thair, “The effect of using Kahoot! For learning. A literature reviews,” Computers & Education, 149, 103818, 2020, doi:10.1016/j.compedu.2020.103818.
26. M. Kalogiannakis, S. Papadakis, A.I. Zourmpakis, A.I. “Gamification in Science Education,” A systematic review of the Literature, 11, 22. 2021, doi:10.3390/educsci110.10022.
27. D. López Carrillo, A. Calonge García, T. Rodríguez Laguna, G. Ros Magán, A.J. Lebron Moreno, “Using Gamification in a teaching innovation Project at the University of Alcalá,” A New approach. Electronic Journal e-Learn, 17, 93-106, 2021, doi:10.34190/jel.17.2.03.
28. D.R. Asa, C. Gunn, “Improving problems solving skills in introductory physics using Kahoot!,” Physics Education, 53, 053001, 2018, doi:10.1088/1361-6552/aacade.
29. B. Klimova, J. Kacetl, “Computer game-based foreign language learning: Its benefits and limitations”. In International Conference on Technology in Education, 26–34, 2018, Springer.
30. A. Inge, “Results from using various quiz-approaches in class,” Kahoot Journal. [Accessed 14 November 2021], 2013, available at: https://goo.gl/Wqu8D1
31. R. Cutri, L. Marim, J. Cordeiro, H. Gil, C. Guerald, “Kahoot, A New and Cheap Way to Get Classroom-Response Instead of Using Clickers,” ASEE Annual Conference & Exposition Proceedings, 2016, doi:10.18260/p.25512.
32. R. Raju, S. Bhat, S. Bath, R. D´Souza, A.B. Singh, “Effective usage of gamification techniques to boost student´s engagement,” Journal of Engineering Transformations, 34, 713-717, 2021, doi:10.16920/jeet/2021/v34i0/157171.
33. A. Alawadhi, E.A.S. Abu-Ayyash “Students’ perceptions of Kahoot! An exploratory mixed-method study in EFL undergraduate classrooms in the UAE,” Education and Information Technologies, 1-30. 2021, doi:10.1007/s10639-020-10425-8.
34. L. Rodríguez-Fernández, “Smartphones and learning: use of Kahoot in the university classroom,” Mediterranean Journal of Communication, 8(1), 181-190, 2017. doi: 10.14198/MEDCOM2017.8.1.13
35. G. Del Cerro, “Aprender jugando, resolviendo: diseñando experiencias positivas de aprendizaje,” XII Jornadas Internacionales de Innovación Universitaria Educar para transformar: Aprendizaje experiencial, 2015, Available in: https://goo.gl/x6Z70t.
36. A. Fernández, J. Olmos, J. Alegre, “Valor pedagógico del repositorio común de conocimientos para cursos de dirección de empresas,” Revista d´innovació Educativa, 16, 2016, doi: 10.7203/attic.16.8044
37. P. Kaur, R. Naderajan, “Kahoot! In the English language classroom,” South East Asia Journal of Contemporary Business, Economics and Law., 20(6), 49–54, 2019.
38. M. Sahrir, G. Yusri, “Online vocabulary games for teaching and learning Arabic,” Journal of Language Studies, 12(3), 961–977, 2012.

(Click Here)