Evaluation of Information Competencies in the School Setting in Santiago de Chile

Article history: Received: 07 June, 2021 Accepted: 19 July, 2021 Online: 16 August, 2021 This study evaluated the competencies related to digital information use through technological tools aiming to acquire applicable knowledge by searching and retrieving information. Methodologically, a quasi-experimental design without a control group was applied to a sample of primary education students from Chile (n=266). First, a diagnosis of the digital-informational skills is performed, and, later, the results of a course in a blended learning context -b-learning(treatment) are shown. The results show significant differences between the participant groups, confirming the learning in information competencies and distinguishing an initial level from a posterior intermediate level.


Introduction
In the present context of Information and Communications Technology (ICT), the development of basic skills, and contexts of data overstocking, they have gradually become the three greatest fields for process of teaching-learning in different schooling levels. Additionally, expectations and objectives for information use in a widely intertwined world are set. This way, key skills in the ICT context are search, use, evaluation, and information processing [1,2].
Thus, it is understood that the integration of new data and communication systems, mainly from the set of technologies such as the Internet, corresponds to a social reality that is challenging the global educational system [3,4] and, particularly, the educational system in Chile. The concern of the educational administration is well reflected in the successive efforts to inquire about and propose improvements in material coverage (facilities, hardware and software material coverage, connectivity at a national level), in training and innovation of teachers or the organization of teachers at a management or curriculum development level. However, the scientific knowledge on acquisition and development of digital and information competencies requires the application of systemic methodologies for collection, analysis, and validation of the information that allows drawing applicable conclusions, with a capacity of generalization to advance in this new curricular content.
The quality of education in Chile, in its different levels, is the reason why it is so stressed on considering including key competencies in the foundations of the mandatory educational curriculum and, a variable that considerably impacts in this improvement is the training level performed at a teacher level and the school education stage [5,6]. The evaluation and training are included in the implementation of ICT innovative actions at educational centers.
The objective of this article is to evaluate digital and information competencies along with the implementation and evaluation of an ICT innovative project leading to the training in searching and processing of information in students at the school educational system. Specifically, a diagnosis of the digitalinformational competencies level in school students was performed to later propose an experimental design able to verify the effectiveness of a training program of blended learning for the development of digital-informational education, competencies in the permanent training of school students. Finally, a variation level in the evaluated dimensions is set -search, evaluation, processing, and information communication -at the group of analyzed students.
The rest of the article is organized as follows: firstly, this document provides information on digital-informational skills in a formal educational setting and the current situation of ICT. Then, the applied methodology, and the sample characteristics are detailed. Hereafter, the pre-test and post-test results are summarized. Lastly, discussion and conclusions are shown..

Digital-informational skills and the school setting
The fact of using ICT as learning objects, vehicles of information literacy and tools of permanent training of students from the school system is going to provide an added value to the research field, showing themselves not only as methodological elements but also as findings and achievements. The ICT use in the development of school education is often presented as a positive impact on the development of digital skills that belong to the 21st century and skills in general [7,8]. Thus, digital literacy can be understood as the development of a wide range of skills derived from the use of applied technology, allowing students to research, create contents and communicate digitally, and therefore, to constantly participate in the development of a society with a strong digital component [7][8][9].
Despite the classic denomination of "digital natives" [10], where it is established that these digital skills are in current students by default, several studies have determined that their knowledge is in medium and low levels [11][12][13][14][15]. Thus, considering this information, it is necessary to develop these skills aiming to prevent differences in the use of digital technologies. In the particular case of Chile, several studies have shown that students are capable of solving tasks related to the use of the information as consumers, as well as organizing and managing digital information. On the other hand, very few students can have success in skills related to the use of information as producers and creators of contents [9,16,17].
Skills related to information management through digital tools -particularly free access resources found on the Internet-with the objective of acquiring valid and verified knowledge through the search and collection of information, along with the interpretation of text information from the reading of new data, reflection, and its evaluation, will be understood as information literacy, also known as ALFIN, by its Spanish initials [18][19][20].
Beyond the existing controversy on the inportance of terminology used in the very definition of concepts such as "Processing of Information" and "Digital Skill" [21][22][23], the fact is that "Informational Skill" can be placed within the scope of action of this key competency. At the center of the generic skills the information literacy is found, which has become a new paradigm at the ICT scene, which is understood as the cognitive-affective fabric that allows to people not only recognizing their informational needs but also acting, understanding, finding, evaluating, and using information of the most diverse nature and sources.

Digital learning in a blending learning setting
For the development of digital and information literacy, the adoption of online modality is increasingly massive, in particular when the courses are integrated into academic study plans, or in contexts where this type of education is required [24]. Besides, the experience of adopting online or b-learning modalities has been strengthened in extreme situations like confinement derived from COVID-19 disease, where teachers understand the context of online learning. However, during the implementation, a variety of problems have arised, like facilities availability, Internet access, the need for new forms of planning, implementation, and evaluation of learning, and collaboration with the parents [25][26][27]. Notwithstanding the above, digital technologies represent an educational resource for a better adaptation to different types of students and their different personal situations. Elements such as focusing on diversity and encouraging the development of key skills are taken into consideration, while they offer multiple possibilities for collaboration between teachers and partners in new communicative scenes of online character.
Thus, the challenge of providing an interactive learning experience for students in big classmate groups and the concern on the quality of teaching in this type of milieu have been key catalysts for the reconsideration of educational approaches in the different educational levels [28].
Blended learning (b-learning), also known as hybrid learning, constitutes an evolving field of research within the wider dimension of electronic learning (e-learning), and it related to instruction practices combining traditional presential approaches with online learning or mediated by technology with the Internet as the main tool used [24,29]. From the beginning of the 21st century, a significant number of studies in this field, with their diverse implications in educational fields have been developed. However, there is still limited evidence that b-learning improves the results of learning in students [24,30].

General context
The interest of organizations such as the European Union, the OECD, and UNESCO for this type of research is relevant [31][32][33], since the circle of evaluation-formation and educational innovation constitutes one of the strategic axes in the education field for the development, not only of key areas in all the productive fabric but also in critical thought in a globalized world. This way, the formation of citizens regarding skills focused on demands of work environment in the 21st century should constitute a fundamental concern for current governments.
Currently, a great impact in the study of information competencies in research has been reached [34][35][36][37][38]. However, efforts on the progress of evaluation tools of the proficiency level in digital and information skills are not wider enough. Most of the research carried out have established and implemented measure scales of self-perception of their own digital and information skill, of their elaboration, and in a setting of the general evaluation of digital skills [39][40][41][42][43][44][45][46][47]. It is important to point out that, while most of the research previously analyzed and cited keep a precise and concrete application, only a few of them present an integration of the evaluation and the use of digital and information competencies within the educational curriculum [47][48][49]. Although there is a specific increase of these digital skills in teachers from diverse areas, particularly teachers who teach one course, the development focused on the digital and information competencies in Chile has not been produced, and rather, they have been settled from the general implementation of digital skills [9,[50][51][52][53][54].
Additionally, a significant part of the research carried out in the formal education field, from national as well as international contexts, and especially in primary school and university educational levels, establish training programs that develop themselves from specific curricular or disciplinary aspects [39,42,[55][56][57][58][59][60]. Nevertheless, due to the multidimensional structure that digital and information skills raise, the experiences within these training programs tend to aim at the specific dimension of technology, collection, and processing of data, rather than a complex, global and holistic view of them.

Objectives and hypothesis
Analyze the level of ICT competencies of the school education body to, secondly, present an experimental design able to check the efficacy of a teaching program for the increase of information competencies in the permanent formation of students from school education. Finally, the variation level of the measured dimensions -search, evaluation, processing, and communication of the information-is established in the group of students that were analyzed.

Research design
This is an information analysis study based on investigation data in students of the last stage of primary school [61], where a quasi-experimental test type design without a control group was used [62,63]. In this study, an educational intervention with ICT and information competencies was performed to support innovation and integration models in the teaching and learning process..

Participants
The study sample is students of both sexes who belong to the last year of primary education in Chile, in the Province of Maipo area. The sample were stratified according to the school levels they belonged to, along with their source schools.
The sample was composed of 266 students and, according to the socio-demographic features, distribution can be recapitulated as: • By variable sex of the sample was 50.4% of female students and 49,6% of male students. • By age of students was established between 12 to 17 years, with an average of 13.89 years. • The distribution based on education level was made according to sex: 63.53% of students belonged to the 8º grade of primary school, where 50.88% were women and 49.11% were men. 36.46% of students belonged to the 7º grade of primary school, where 50.51% were men and 49.48% were women. The sample derivates from the population of primary school students, whose ages range between 12 and 17 years old, belonging to the Province of Maipo in the Santiago Metropolitan Region (Figure 1), equivalent to N=106,745 [64], with a level of significance of α = 0.05 and a maximum of homogeneity p = q = 0.5 for the sample of 266 participants, where the sampling error obtained is equal to 8.2%

Data collection instrument (test)
The dependent variable has been defined as the level of acquired competency of the students on digital and information skills, measured before (pre-test) and after (post-test) the implementation of the educational intervention (treatment).
The instrument was applied in the pre-test stages as a diagnosis of the competencies, and the post-test was applied for the evaluation of the information competencies reached by the participants students, measuring the level of performance obtained after the treatment. The test is composed of 29 elements of dichotomous nature, from 37 questions of single and multiple selection. 7 questions in the information search dimension, 10 questions in an evaluation dimension, 5 questions in the information processing dimension and other 9 questions on communication and dissemination of the information are presented (table 1).

Treatment
The independent or treatment variable involved the teaching intervention applied to the participant during 30 training hours in a blended context [65][66][67], and adapted from the proposal described in [68]. This way, the teaching intervention applied in the training of sample corresponds to an adaptation of the instrument for the development of information competencies of secondary education [65,68] which has been tested and validated in that area.

Data analysis
Concerning the data analysis, after the exploratory initial analysis of the distributions of the variables and the evenness of the variances and covariances structures, parametric techniques, ANOVA with repeated measures are applied. Intra-subjects factors (pre-test/post-test) and inter-subjects factors (type of school) are incorporated. After the study with repeated measures, other techniques complementing the results are applied, such as the ttest. Additionally, all scores have been calculated so that each of the items is valued with a maximum score of 1 and a minimum score of 0. Besides, the scores of dimensions such as the average score of the set of items of the n dimension multiplied for 10 have been calculated. Thus, all dimensions are ranged from 0 to 10 points, which facilitates interpretation and contrast. Finally, the total score is calculated as the sum of the scores in the 4 dimensions. To ensure the internal consistency of the test results, the Cronbach's alpha test was performed, whose value was equal to 0.731, considered as adequate or acceptable [69,70]. All these statistical processes have been carried out in the SPSS 25 and R 4.0 software.

Procedure
The data in tests, in all cases, was collected mainly through online questionnaires. Responsible teachers were contacted, and the students participated by performing both pre-test and post-test online.

General and descriptive results
Firstly, it is possible to determine values that approximate an average of 5 points (table 2) for intra-subjects characteristics (sex, grade and school). In parallel, there may be an important dispersion of values, which increases in post-test results.
It is observed that at the sample's level, the average reaches higher scores in Search, Processing, and Communication dimensions, while lower scores are registered in the Evaluation dimension (table 3). At a general level, the average score is greater in the student samples in the post-test. The dispersion of values increases in the post-test, so the teaching e-learning process has created higher levels of inequity between the measured students.   Moreover, it is observed how the distribution of pretest and post-test is similar in both groups, and their progress is similarly reached, while in the Evaluation dimension there is a decline or no progress at all for both cases. It does not seem to be interaction according to the school; therefore, it is not considered a covariable. By obtaining correlations between dimensions, it is possible to establish that all values are significant and positive (figure 2).
By comparing between groups and the results obtained in the post-test, it is possible to perceive differences between the course and the school groups. However, based on the sex variable, values in the scores in different dimensions do not show great differences (figure 3). It is also possible to establish that in total scores, from all the variables analyzed in the contrasts, the sex variable is not significant in an inter-subject level nor an intra-subject interaction (table 4).

ANOVA results of repeated measures
A hypothesis in the use of the ANOVA test with repeated measures is the matrix homogeneity of the covariances of the dependent variables [71,72]. This hypothesis is determined through the Box test, with the following results (table 5): Thus, the significance level showed that in this test a value of 0.087 was obtained, which exceeds 0.05 and, therefore, with a probability grade of 95% that the hypothesis on covariances matrix observed of the dependent variables are equal between groups.
Descriptive statistics and the results of the ANOVA test of repeated measures applied to data (table 6) showed that there is a significant interaction depending on the Grade, so it was decided to keep this variable, regardless that it is not significant at an individual level. The training action was significantly more effective in the eighth-grade group course than in the seventh-grade course. While the levels are similar in the pre-test scores -slightly higher in the eighth-grade group -, the training action had better results in the eighth-grade students. Nevertheless, the School variable does not have interaction in pre-post test scores (Figure 4).  In table 7 it is possible to see the effects caused by the characteristics of the subjects as a transversal measure. Thus, it is possible to establish that the School variable is significant, but that is not the case for the Grade variable. This implies that at a general level, and calculating the average score of each subject in the pretest and post-test, and also considering the criteria of the average score variable, there are significant differences for the School variable, but not for the Grade variable. Although all students in different schools and courses reached substantial progress in their digital competencies, there are also constant differences between the schools participating in the research.

Discussion
The students participating in the study, despite belonging to two different schools and two different teaching levels, in which there are different methodologies and contents within the elementary school, behave as a heterogeneous group in the information competencies area, with statistically significant differences between groups compared in the totals as much as in the analyzed dimensions (tables 1-7 and figures 1-2), which is complemented with the average age of the test responding students along with the sex variable, factors that do not have great incidence in the results due to the developed b-learning educational intervention.
Regarding the diverse analyzed variables, students behave in a heterogeneous manner, despite all the participants increasing their scores in the post-test measures. Within the established dimensions, the differences existing in the evaluation and the processing of the digital information areas are highlighted, a situation that has been stated in other investigations [16]. This way, particular uses in the application of technologies skills are presented. Regarding values on indicators of digital competencies and forms in which they show, participant students keep medium to low digital competencies indexes, which coincide with what has been carried out in others similar research [4,49,73].

Conclusions
Within the complexity of the Information Society, which is constantly mediated by the impact of ICT, it demands to the educational processes the incorporation of key skills, where the digital and information skills are prominent, related to the treatment of information in a virtual setting and the competency of digital processing. From this aspect, the training of the students from the first cycles of teaching gains importance. Thus, this study evaluated the efficacy of an educational intervention in a blearning setting for the training in information competencies at an elementary educational level (seventh and eighth grades), obtaining progress considered significant but not enough for the ICT context where we live. Empirical values obtained in the different dimensions confirm an appropriation in the digital competencies learning, where it is possible to differentiate an initial level from a posterior level after the applying of the educational intervention, and also between other variables as grade from school variables, establishing different realities in each educational context.
In the conclusions, the importance of the evaluation and training in digital and information skills is considered, addressing the fundamental dimensions that compose them, and establishing the efficacy of the implemented educational intervention. On the other hand, the research contributes to the study area according to the scale of reference -Santiago de Chile -and the training context -elementary students. Even though in Chile there are efforts to know the use of technologies in secondary teaching students [9], in the case of elementary students there are no studies that establish the characteristics and the levels concerning the information competencies in a digital setting.
Finally, after an analysis of the research contributions, weaknesses are established, which are focused on the design and development of the evaluation instrument and the experimental level of the applied design.