Effective Learning of Tax Regulations using Different Chatbot Techniques

Article history: Received: 30 August, 2020 Accepted: 01 November, 2020 Online: 20 November, 2020 Teaching tax-related regulations have always been a challenge due to the inclusion of external variables that hinder the learning process, such as the high complexity of tax systems and legislation variability. Universities have sought different alternatives to support the teaching process both outside and inside the classroom, ranging from recreational activities to active learning. This article will show the experience resulting from using a chatbot to support learning in accounting students for the teaching of tax regulations related to the Chilean tax system, comparing two types of tools, on the one hand, a free conversation chatbot using natural language processing versus a rule-based chatbot driven by a decision tree. The experimentation process was carried out with 50 higher education students, divided into an experimental group and a control group, in two different courses. The results obtained demonstrated in both cases greater effectiveness of the use of the chatbot in learning the tax matter, both in the free conversation chatbot where the experimental group obtained a 15.7% improvement versus the control group that obtained a 1.05% improvement, as in the chatbot that applied decision tree where the experimental group obtained a 32% improvement versus the control group with 5.2%. Considering the complexity of the content in tax matters and the little innovation in the existing teaching subjects in the area and that the students improve learning using both chatbot tools compared to other learning techniques, is considered a relevant contribution to teaching innovation.


Introduction
This article is the continuation of a lengthy investigation developed in courses on the subject of taxation [1], which seeks to generate support for learning the Chilean tax system through the use of a chatbot with an automated conversational system technique; these technological systems aim to establish new mechanisms that have a direct impact on student learning, especially on issues such as the tax regime of a country, an area characterized by the complexity in the application of tax regulations.
Significant studies have taken place about the behavior and learning of students [2], [3], where motivation has been one of the key factors due to its direct impact on the perception and predisposition that students have towards the teaching activities delivered by teachers [4]. In this sense, the lack of motivation is one of the most critical causes when studying the failure and dropout of educational systems [5]. Socioeconomic and health factors also influence learning processes [6], [7], as well as the design problems of the different learning resources created by teachers [8], [9]. However, what research mostly agrees with is that this is a cross-cutting problem in society [3], [10], [11].
One technique that has become a common factor is the use of technology and software to support the teaching process and enhance the students learning [12], [13]. The integration of technologies in the classroom is a constant challenge that teachers and educational communities must face, but it is necessary based on the global demands on students' development of skills [14]. Based on this, there are numerous innovations, but the most outstanding and most attractive to students are those related to digital credentials, virtual assistants, and blockchain [15], [16].
This work puts into application in educational innovation, the use of a chatbot for learning theory, and application of Chilean tax regulations. The learning processes of tax regulations are always a great challenge for education and tax entities [17], [18]. In the case ASTESJ ISSN: 2415-6698 of Chile, the last ten years have seen critical structural changes in the tax system related to income tax, the reforms of which have been analyzed and studied [19], both due to the complexity that the tax system has acquired well as the particular characteristics defined for corporate taxes, which have direct effects on the economies of the countries [20]. Because of the above, different investigations have revealed the importance of teaching correctly and innovatively the subject of taxes [21]. With this, it is that innovative teaching mechanisms must be sought to produce effective learning in students.
The research objective is to compare the effectiveness in the learning of tax regulations in students of accounting careers through the use of two chatbot techniques, that is, one, through an automated conversational system and, two, based on a tree decision. This article presents the result of applying chatbots linked to the study of the tax system regulations, which cannot be exempt from applying, integrating, and using information technologies in their learning process [22].
Considering the complexity of the tax systems and the constant legislative variability produce an effect of obsolescence in the subjects, affecting the students' rejection towards these subjects and, consequently, very high rates of failure and dropout [23]. To address this problem, there were two chatbots with knowledge of tax matters. The first with a free communication with the students as applied in [1] and the second using a decision tree. The study was carried out in an N=50, dividing the students into an experimental and control group, in the accounting career in a Chilean higher education institution. The results obtained are promising, where, in the case of the free communication chatbot there is a difference of 14.65% in favor of the experimental group and in the case of the decision tree chatbot there was a difference of 26.8% in the results in favor of the experimental group.
For this article, we will consider a chatbot as a software tool that allows users to have a conversation between a human being and an artificial entity [24]. One of these technology precursors is Joseph Weizenbaum, with Eliza [25] applied to psychoanalysis with promising results [26]. As the winners of the Loebner [27] contest point out, chatbots have evolved from simple pattern matching systems to increasingly complicated patterns of computer interaction and reasoning, being applied in education [28], entertainment [29], health [30], among others. The rise of chatbots has occurred in customer service, such as Hennes & Mauritz AB, which seeks to replicate a human conversation to facilitate information collection by a customer [31]. The purpose of chatbots is to automate the process of communication and support for people [32].
At an educational level, there have been various applications to chatbots with mixed results, but most of them agree that an improvement in learning and student satisfaction is evident [33], [34], generating greater interest in the subjects test [35]. For this reason, three main factors stand out to determine their participation, the performance expectation, the effort expectation, and the habit [36], which indicates that the commitment is linked to the benefits versus the effort to which the students are subjected by supplementing your learning with a chatbot system. This is why it is critical to innovate in non-traditional areas and that the effect has a direct application in business, accounting, and auditing matters.
The research questions posed are in line with determining the effectiveness of the two tools applied and their impact on effective learning: • Does the use of the two chatbot techniques positively contribute to learning tax regulations?
• Is learning through a chatbot more effective than through the use of gamified activities and e-learning videos?
Based on the above, we established as a research hypothesis: Incorporating a chatbot, independent of the applied technique, for the learning process of tax regulations in higher education provides better results due to the students' academic performance.
Due to the above, a null hypothesis is also established: The inclusion of technological tools, particularly chatbots, does not improve effective learning on tax matters compared to traditional methodologies or e-learning.
Concerning the structure of this paper, the theoretical framework that will deliver the definitions and general aspects that must be taken into account is presented below; later, other experiences applied in the matter will be seen, and then proceed with the experimental design and case study for the two techniques to be compared. This article ends with the results obtained and their corresponding discussion.

Related Works
Australian work by [46] shows that a significant number of accounting schools resist the adoption of new technologies in their training processes. The reasons are the lack of interest in implementing technologies with an extra workload, the little support, and the lack of resources, but the most relevant is the lack of educators' time to learn and develop skills and competencies to adopt new technologies in education systems efficiently. The work recommends considering a new model that reflects the current innovative technological nature to provide education, anywhere, anytime, and for anyone.
As far as tax learning is concerned, the work of [47] mentions how tax education faces significant challenges in the 21st century and delivers results to change current educational paradigms, prompting a more effective teaching practice based on conceptual knowledge over technical ability. In [48] it is explained how the subject of taxes is elementary to complement other subjects within accounting programs, and it seeks to find effective teaching methods using computer software, which, although it presents good results, its use is not every day. In [49] the work to support the understanding of tax systems and the effect on the systemic change that arises when different organizations are willing to learn from each other is presented. For this, real case studies are used in the companies Oxfam and Unilever. In Spain, there is the work of [50], which analyzes part of the challenges that exist for teaching tax law, considering students who are studying in a highly multidisciplinary degree; and in [51] whose economic concept is the distribution of the tax burden between buyers and sellers, it is suggested that collaboration between teachers is more effective than the lesson study model. The work of [52] demonstrates how tax learning is necessary even in non-accounting programs, relating the interest of professionals in learning tax matters and the needs of companies.
As mentioned in [53], chatbots as learning tools are still in their infancy. The study carried out in [54] suggests that although initially arousing considerable interest in behavior, some tasks may not be interesting enough to impact subsequent interest in the broader domain of the study; for this purpose, in [55] a four-phase model of lasting interest development. In [56], it is stated that digital tools can provoke feelings of isolation and detachment caused by the lack of interaction with teachers, which is approached through the design of a hybrid-chatbot. Finally, it must be guaranteed that a chatbot's novelty does not define the end-user experience, although it is undoubtedly part of the initial interactions with these types of tools [54].

Solution Design
The solution was thought of in accounting students from a Chilean university, where the learning approach is related to tax regulations. In this context, it is essential to point out the Chilean tax system's difficulty due to its complexity, variability, and a very extensive law [57].
The proposed solution's objective was the implementation of two chatbots with pedagogical resources, which, using an automated conversational system and decision trees, could be applied to the context of tax regulations at the university level.
To respond to the proposed objective, two chatbots were designed, one through an automated conversational system and the other based on a decision tree, both focused on knowledge of tax regulations under the Chilean legal context. Also, the different tests and activities were designed to demonstrate the interaction and verify the effectiveness of the tools in the learning process.
The chatbot architecture is fundamental to explain its operation, and, as is well known, there are several architectures [37], [38]. In general, there is a standard operation that can be divided into three stages, as seen in Figure 1, where the user interacts with the voice manager through a device, where the analysis and generation of the query are carried out under the work context, considering that the bot must generate a response without considering the sense of intention. Finally, the generation of messages includes the planning of words, sentences, and coherence to deliver. With these aspects defined, we can establish that the user writes a question in his application, which must be answered by the chatbot and with greater complexity and volume of knowledge, then a more significant number of interactions between a bot and user [40], [41].
Decision trees are highly complex data structures sharing binary trees [42], [43]. This type of tree is characterized by storing its different nodes in a hierarchical and orderly manner under a defined criterion based on the problem addressed. Compared to linked lists, queues, or stacks, in this case, the data is not stored linearly [44].
In this case, the constraints to be applied to the decision tree have characteristics of an N-ary tree with single essential control used in binary search trees. The walkthroughs and handling of the decision tree structure are handed over to the natural language processing engine. The algorithm used by the classification and interpretation engine conforms to a conditional inference tree; consider that the Recast.ai engine handles the classification of intentions automatically, so no changes were made to it in this research. Figure 1: General chatbot architecture [39] In this case, we start with a single node and then branch out into possible results based on the intent analysis of the natural language processing engine [45]. The decision node will represent the intention in which the user's interaction will be classified, giving rise to the existence of a terminal node that will show the result of the intention. The first node will correspond to the concept of tax and the last to specific answers.

Pedagogical context
The learning context of this research's tax regulations focuses on the career of auditor accountant (public accountant) of a Chilean university. Thus, teaching and learning the rules that regulate tax regulations is a constant challenge, especially to achieve effective learning. The different indicators of the courses approval do not generate significant delays in the students' curricular progress. The professional profile for graduating the auditor accountant career (public accountant) considers the development of specific competencies that are directly related to the work performed: • Generic fundamental training competencies: proactivity and responsibility; carries out its work with professional ethics.
• Specific disciplinary competencies: understand the conceptual framework of the current internal fiscal tax legislation.
• Specific professional competencies: apply the conceptual framework of the current internal fiscal tax legislation; apply the general, and special regulations of the tax legislation of the taxes levied on business income.

Learning Objectives
The Taxation courses of the auditor accountant career (public accountant) have a content structure that contributes to the learning results; in this way, the contents associated with this experiment are: (1) Understanding of the examination regulations, their practical application in the taxpayer's life cycle; (2) Understanding of the corporate tax structure.
For the technique through an automated conversational system, the learning outcomes associated with the experiment and the course are [1]: • OL1: Recognizes basic concepts related to corporation tax to apply them to the company.
• OL2: Identify basic concepts of the different corporate taxes to compose tax returns.
• OL3: Applies basic concepts of the tax object to calculate taxes.
For the decision tree-based technique, the learning outcomes associated with the experiment and the course are: • OL4: Defines the types of tax examination to identify the procedure applied in each of them.
• OL5: It includes the indirect control rules for their correct application in the taxpayers' life cycle.
• OL6: It includes direct inspection rules to identify the rights that assist taxpayers.

Methodology
For methodological purposes, we worked with the Taxation 1 and Taxation 3 courses of the auditor-accountant career, where the main subject of these courses is the regulations on audit and corporate taxes, respectively. The experimentation process for both chatbots is considered the same methodological steps are shown in the BPMN diagram in Figure 2. First, a class of theoretical contents is held in both courses with all students without differentiating or separating individuals. This class is face-to-face and in a classroom with a projection. After that, a pre-test is applied to all students based on the contents taught in the face-to-face class of the Tax 1 and Tax 3 courses. Both courses are then divided into two groups, which we will call the experimental group and the control group. In the case of the Taxation 1 course, the use of videos through the Moodle platform was applied to the control group, and in the case of the experimental group, the use of the chatbot based on the decision tree technique was applied. In the case of the Taxation 3 course, the control group was given playful activities in the classroom led by a teacher, and in the case of the experimental group, the use of the chatbot with the technique was applied through an automated conversational system. A post-test is applied to both courses to measure the difference generated in the learning process. In this way, the results obtained in the tests by each of the groups of both courses will be compared, and the effectiveness of using the chatbot with the two techniques will be shown. Finally, it should be noted that, in the Taxation course, 1 of the of 16 students, eight remained in the experimental group and 8 in the control group. In the case of the Taxation course, 3 of 34 students, 18 remained in the experimental group, and 16 in the control group.
As mentioned above, two tests were applied to each course, one that was developed before dividing the group and another afterward to measure effective learning. An example of interaction in the evaluations is presented in Figure 3, using Google Forms.

Mechanics
In the Tax 1 course, the students belonging to the control group saw explanatory videos with the contents. For the experimental group, a guide to questions to be solved was delivered with the bot's support. The chatbot used was called Tribuchat, and it was linked to a Telegram chat, with which the students who made use of their mobile devices interacted under the guidance of the chatbot. Figure 4 shows an example of interaction with the bot.
In the case of the Tax 3 course, the students belonging to the control group did playful activities that included the themes addressed in the planning. The teacher-guided these activities. For the experimental group, a guide to questions to be solved was delivered with the bot's support. The chatbot was called Tribubot, and it was linked to a Facebook Messenger chat, with which the students, using their mobile devices, freely interacted with the chatbot. Figure 5 shows an example of interaction with the bot.
It is important to note that feeding both bots was carried out, in which the teachers in charge of the subjects generated the expert knowledge base on the subject and thinking about the interaction with the students, which was loaded into the tool SAP Recast.ai. Finally, interaction actions were included to measure a formative nature's learning through questions asked by the same bot to the students.

Results
The display of results obtained will be classified in the general results, later categorizing by gender, and finally according to performance. Table 1 shows the results obtained in the tests applied to the Taxation 1 and 3 courses, and the averages of correct, incorrect, omitted answers, and the percentage of correct answers that were. In Table 2, we can see the characterization and results in the test of the groups of Taxation 1 and 3 courses based on composition according to gender.  Table 3 shows groups' behavior according to the average of the Tax 1 and 3 course; for this, those below the average, equal and above, are grouped in their categories of the experimental and control groups. In addition, to statistically validate the results, some tests were applied. Because the new experiment points directly to tax course 1, then as the results were analyzed, we begin with Table 4, where the samples' normality is appreciated through a Shapiro-Wilks Test. As can be seen, for the sample of the experimental group in the post-test, normality was rejected with a high level of significance, so that we will use non-parametric tests for the comparison of means.  Table 5 shows the result of applying the Wilcoxon test for paired data. In this case, the difference in means for the control group between each test is not statistically significant; on the contrary, in the experimental group, it is observed that the mean of the post-test is higher with a high level of statistical significance (0.0001526 < <0.05). In this first investigation, the data concludes that the research hypothesis is accepted, giving a difference between groups of p=9,722e-06. Specifically, the experimental group results were statistically much higher than those shown by the control group; therefore, the chatbot was more effective than the other method used.
Regarding the comparison of the two tools, only the sample data of each experimental group were taken. The research hypothesis is addressed: "The students' performance is different given the applied chatbot". To do this, first, a difference in proportion test was applied, where this test analyzes whether the percentage or proportion of a sample is statistically significance concerning another. Table 6 represents the data of the experimental group students who obtained a performance higher than the global average (control + experimental) in each post-test and the total sample of students of the experimental group.

Students
Tax 1 course Tax 3 course  Above average  7  15  Totals  8  18 Furthermore, with a p-value = 1, it can be concluded that this proportion is the same in both groups, and we cannot find a significance difference.
Finally, the result changes since the hypothesis focus only on a one-sided inequality and the test; therefore, a significance mean difference can be accepted and affirmed that the first chatbot students had a better performance (p = 0.04418). Also, it is noticeable that course 1 was much higher than those of course 2, but there was also a larger sample in that case (a little more than twice as many students), so that although the inequality between chatbots could be maintained. With the above, we have ruled out the null hypothesis.

Discussions
Due to what is stated in Table 1, it is evident that the control group of the Tax 1 course presents an advantage of 12.56% over the experimental group in the pretest, which was reversed in the posttest, leaving a 10.97% in favor of the experimental group; which leads us to deduce in the first instance that if the performance of the students in the experimental group is compared with the control group, the chatbot tool produced an evident improvement in learning to the detriment of the control group that took explanatory videos. This justifies the first research question about contributing positively to learning the contents related to the Chilean tax system, which is characterized mainly by the complexity of its normative structure.
The above also allows supporting the second research question on the effectiveness in the use of the chatbot. Indeed, the differences between incorrect answers and errors made are evident, where the control group presents discrete performance compared to the experimental group.
In the case of Table 1 related to the Tax 3 course, it is evidenced that the control group also presented an advantage of 4.94% over the experimental group in the pretest, reversing the situation in the posttest by 27, 82% in favor of the experimental group. The evidence is also significance when comparing the wrong questions that did not decrease.
If the behavior by gender in Table 2 is studied, it is essential to note that in the Tax 1 course, the men belonging to the control group did not show improvement (0%) versus the men in the experimental group (27.8%). In women's case, the improvement in those belonging to the control group is only 2.3% versus 32.3% in the experimental group. In the case of the Tax 3 course, the result in Table 2 is not so different compared to the Tax 1 course. Indeed, it can be found that men who make use of the chatbot show more than a 60% improvement in its results; on the other hand, those who do not use the chatbot improve by 5.7%; in the case of women, the difference is also greater, because those women who use bot the percentage of improvement is greater than 58% but those who do not use it improve 13.4%.
If the behavior is analyzed according to the means in the Tax 1 course, as shown in Table 3, the control group goes from having 75% in the average to 37.5%, leaving 37.5% above the average versus the initial 12.5%. In the case of the experimental group of 62.5% who were in the mean, only 25% remained, and those who were above the mean grew from 12.5% to 75% of the group, which together with finally remaining 0 % below the mean, then the effect of the tool applied in the experimental group is greater than that of the control group since many students being below the mean or within it rose to be at or above it. The above is a critical case to highlight and take into account for later statistical analysis. Finally, as shown in Table 3 about the Taxation 3 course, initially, 5.56% of the students who use chatbot were below the average; after using the chatbot, that number decreases to 0%, which shows a positive impact of the tool. It is not minor to identify that the control group also improves the results after applying the gamified activities.

Conclusions
It is already known that the teaching of tax matters is a constant challenge that expert teachers in the matter must face daily, especially considering the complexity of the contents and the repeated modifications of the legislation in the last ten years.
Throughout this paper, we have been able to see different experiences that try to improve effective learning, comparing two chatbot techniques, that is, through an automated conversational system and another based on a decision tree, showing that independent of the technique of the chatbot used has better results than guided learning through videos or gamification techniques. The results showed that, in all cases, students who use a chatbot, regardless of the technique used, perform better than other tools, justifying their effectiveness according to what was raised in the research questions. The results show that the performance of the students of the experimental group produced an evident improvement in the learning of the contents related to the Chilean tax system, which is characterized by its complexity, derived mainly from the withdrawn legislative reforms in the last ten years.
As future work, the perspective should be taken to analyze and quantify the impact of the use of this type of tool and the cognitive load that it generates in students. Besides, it would be essential to study the change suffered by the levels of frustration of students compared to the different techniques that exist in novel and impact studies, directly related to emotionality and the application of algorithms. In this sense, an artificial emotion can be implemented as a synthetic abstraction derived from the observation of human emotions, which allows improving the effectiveness of the decision made [58]. It is also relevant to test other techniques and tools such as ROL-type games [59], gamification techniques [60], or the application of multilingual verbal communication with a focus on human-robot interaction [61].
Finally, it should be noted that through this research, it is shown that a chatbot improves the results in learning subjects related to tax regulations and, therefore, it is necessary to continue with studies that cover other aspects of effective teaching and learning.