Semantic modeling of portfolio assessment in e-learning environment

In previous work, it has been proposed a conceptual framework that consist of an ontology network named AOnet which is a semantic tool conceptualizing different types of assessments. Continuing that work, this paper presents a proposal to implement portfolios assessment in e-learning environments. The proposal consists of a semantic model that describes key components and relations of this domain to set the bases to develop a tool to generate, manage and perform portfolios assessment.


Introduction
One of the most important task in e-learning is to assess student, because constant assessment avoid students lost interest in their studies. The use of different evaluation methods make students feel that connection is established between them and instructors, and their learning efforts are properly assessed [1] (Sun, Tsai, Finger, Chaen & Yeh, 2008). As Bolivar says [2], assessment is a systematic procedure used by teachers to determine the level of student knowledge in a particular discipline, before, during and at the end of an academic subject. Note that assessment should not just be about grading but about assisting the process of learning itself. Accordingly, assessment should be developed and refined in order to assist student to learn effectively and efficiently [3]. From a pedagogical point of view, e-learning demands greater customization and efficiency to leading learners. Then, it requires identify how and when to assess and considering which outfit to be used. Even though portfolios are used in e-learning as tool to keep track of student progress, when its main objective is to assess, it is known as portfolio assessment [4]. In previous work it has been presented a conceptual framework consist of an ontology network, called AONet, which is a semantic tool that implements different kind of assessment used in e-learning [5]. Following with this activity, this work presents an e-portfolio assessment www.astesj.com 150 specification. As Lorenzo & Ittelson says [6], an e-portfolio is a collection of digital artifacts like demonstrations, comments, resources and results that characterize a person, group or institution. E-portfolios are widely used to assess students with the advantage of including different type of assessment like essay, conceptual map, objective test that allow teacher evaluate different knowledge level.
In e-learning context, there is a lot of work that address technological aspect but a few work about pedagogical one. In this way, this work contributes to address pedagogical issues because it proposes a conceptual model of portfolio based on ontology. This technology allows coding expert knowledge through logical rules.
This work is organized as follows. Next section explains the background knowledge: the AONet ontology network and the use of e-portfolio in education. Next a literature revision is presented. Then the Portfolio ontology is introduced and a case study is developed. Finally conclusions and future works are presented.

Background
This work is part of a mayor task done by author in the elearning area. In this section, it is presented the ontology network called AOnet [5]. An ontology network is a set of ontologies related together via a variety of different relationships such as mapping, modularization, version, and dependency. The elements of this set are called networked ontologies [7].

Extending the AOnet ontology network
AONet was developed as the conceptual description of a tool for semi-automatic e-assessment development in the e-learning context. Then, AONet conceptualizes in a modular way, all aspect of assessment domain. In first stage, it has considered three different domains: course topic, educational resources, and assessment. Figure 1 shows AOnet, its domain and ontologies in each of these, which are related one to another through meta relationships. In this case, an assessment is an educational resource and as such, it has metadata that describe it. In order to develop an assessment, different instruments can be used. An assessment evaluates topics or subjects that must be learned, which are conceptualizing in Course Domain ontology.
A new stage of this network is presented. In this stage, a new domain was added to the AONet: the Agent domain that represents the stakeholders involved in an educational process since the stakeholders could play different roles that in some cases is necesary to identify. In [8] it is presented a discussion about stakeholders involved in e-learning process considering highe education. In addition, a new ontology in the EducationalResources domain was added: the Portfolio ontology that represents portfolio educational resource. Figure 2 shows the new version of the AONet ontology network.

ePortfolios in learning
The electronic portfolio (e-portfolio), in education, is a collection of a students' work that can advance learning by providing a way for them to organize, archive, and display work. The uses of e-portfolios are most common in the courses with departments of education. Most preservice teachers are asked to compile an e-portfolio to demonstrate competencies needed to gain teaching certification or licensure. Student e-portfolios are increasingly being used in other disciplines such as communications, math, business, nursing, engineering and architecture. In education e-portfolios have five major functions [9]: (1) Document skills and learning; (2) Record and track development within a program; (3) Plan educational programs; (4) Evaluate and monitor performance; (5) Evaluate a course.
The primary aim of an e-portfolio may be to collect evidence for summative assessment, to demonstrate achievement, record progress and set targetsas in records of achievement and individual learning plans (ILPs)or to nurture a continuing process of personal development and reflective learning, more commonly experienced in higher and continuing education contexts, but now also occurring in further education and schools [10]. E-Portfolios have become a popular alternative to paper-based portfolios because they provide the opportunity to review, communicate and give feedback in an asynchronous manner. Therefore, during the last years, a great amount of e-portfolios system have been developed [11]. These systems use different technologies that students and teachers could use to create multiple e-portfolios from the same repository or set of repositories. Most of the e-portfolio systems are integrated to Learning Management Systems (LMS), giving lot of functionalities.
When e-portfolios is part of an assessment framework, there are advantages and disadvantages for practitioners. On the one hand, digital portfolios offer more efficient working practices, enabling marking and verification to take place incrementally. On the other hand, the diversity of evidence contained in portfolios can make them harder and more time consuming to assess [12]. So, in this context, it is necessary a tool to facilitate the assessment.

Literature review
EPorfolios are widely used in the field of academic institutions as a valuable tool for continuous learning, not only for careers offered in distance learning but also for traditional careers. In

Agent
Agent Specification rincludedIn usesSymbolsOf recent years, there have been a lot of e-portfolios systems as shown in [11]. Using different technologies, they offer learners and teachers the ability to create their own portfolios that customize their career advancement. Many of these tools are integrated into Learning Management Systems (known by its acronym LMS), offering different features. Some allow the institution to have control over these systems in order to maintain the image, others fail to provide social networking functionality needed in this age. The difficulties in implementing these technologies persist due mainly to technical and teaching skills that teachers are expected to have.
That is, the ePortfolio that was originally static web pages, have evolved demanding the availability of commercial and open source tools in the form of web applications for management of ePortfolios. These tools are developed on formats that do not reflect open standards, and do not facilitate the exchange of the information contained among other limitations. This makes difficult to share the artifacts contained in an ePortfolio between different tools or applications and, even more, sharing an ePortfolio of a student between different educational platforms or LMSs in the context of different educational institutions.
Current limitations on the use of portfolios in the learning process could be grouped as follows: First, the use of ePortfolio to estimate learning should be accepted by teachers. To do this, a tool to support the generation and management of portfolios whose content is valid and reliable, from a pedagogical perspective is needed [13]. This means it must be defined a mechanism to validate if the portfolio covers all learning objectives of a course and meets certain pedagogical principles and allows assessment as proposed by [2]. Secondly, systems must support e-learning portfolios with different types of devices with different levels of complexity and have to adapt to different stages of learning. Thus, the e-learning systems should allow teachers to manage ePortfolios containing diagnostic, summative and formative assessments, while must be capable to manage self-assessments, co-evaluation and hetero-assessments. Third, ePortfolio management should be solid against the collaborative work to resolve differences arising from the administration of multicultural information and the advent of complex environments semantically incompatible.
In order to optimize the benefits of using ePortfolios the integration of the components that comprise it and determine the level of learning should be favored. As part of its components, which are called artifacts, teacher can find work performed by learners, evaluations and results or achievements. These artifacts represent means to facilitate educational continuity between programs within an educational institution. and became into evidence that can be shared and integrated between institutions and organizations throughout their academic performance. To do this requires specific technologies to present a solution for interoperability.
The drawbacks associated with interoperability occur when trying to share and reuse heterogeneous information resources [14]. With the use of technological solutions for the interoperability of information, it is not required to have prior agreement between the institutions managing education on LMS platforms for sharing and reuse of components or devices covered in ePortfolio.
As part of technologies that provide solutions for interoperability and the need for integration of heterogeneous and diverse information, the use of Semantic Web technologies in e-learning in recent years is evident. The Semantic technologies can be exploited as a platform for the implementation of a system of elearning as it provides all the tools that this type of education requires [15]: ontology based conceptualizations of learning materials, components standardization to share information and educational courses in composition with proactive delivery of learning materials through LMS [16].
The use of semantic technologies for the management of ePortfolios is also evident. In [17] the authors use an ontology to formalize ePortfolios, representing taxonomically different types of portfolios in learning. Although defined ontology allows the description of the portfolios by properties, work does not contemplate the use of standards for it. The work does not address the management of ePortfolios and does not include the learning assessment process.
In their paper, Taibi et al [18] use ontologies to model ePortfolios and social relations in informal learning environments. To do this, they extend the FOAF ontology, modeling interactions and collaborative work based on social networks. While the authors mention standards, they are only based on IMS (IMS Learning Model Portfolio) and do not address in depth the aspect of assessment ePortfolios.
In [19] authors propose a model of ePortfolio based on an ontology but is aimed at promoting self-regulated learning (SRL). In SRL students are aware of their knowledge, skills and, therefore, they consider learning as a controllable process and assume greater responsibility in the outcome of this process. In this work, authors do not consider rules or restrictions that allow specific recommendations of pedagogical nature.

Portfolio assessment ontology
In order to conceptualize the portfolio domain, it has been defined the Portfolio ontology, which is shown in Figure 3. The main concept is Portfolio, which has two subtypes: LearnerPortfolio and TeacherPortfolio, representing portfolios owning by students and teachers respectively.
The LearnerPortfolio concept has the PortfolioAssessment subtype. Also in figure 3, it can perceive that Portfolio is composed by artifact, a concept representing whatever learning object, for instance Assessment is a subtype of artifact. As is declared in [20,21] a portfolio contains artifacts created by student and teacher such as problems solutions, reflections, descriptions, teacher and classmate feedback, among others. In the ontology, the Portfolio concept is related to the concept Artifact through hasEntry relationship. An artifact is an educational resource, a situation expressed through the inheritance relationship between the Artifact and EducationalResource. The latter concept is shown in green color in order to highlight it is belonging to the EducationalResourceSpecification ontology, which models all educational resources used in the teaching-learning process such as books, notes chair, presentations, videos, assessment among others. This ontology is not shown because it is beyond the scope of this work. The elements that conform a portfolio have different mandatory levels, represented by the MandatoryLevel concept, which is subclassified into Core and Optional. An artifact has an Optional level associated can or cannot be selected to integrate the student's portfolio, but one that has the Core level associated must be included in the students' portfolio.
A PortfolioAssessment must have organization and content [21]. As regards organization, it can be prescribed by teacher or designed by student also [21,22]. The hasOrganization relationship relates Portfolio with Organization concept. Then, there are two relations that relate Organization with the responsible agent for setting it: designedBy and prescribedBy. Note that in figure, the agents Educator and Learner appear in different colors because are belonging to another ontology in the network.
Organizing a portfolio primarily establishes the relationships that must exist between the elements belonging to portfolio. For example, there will be temporal relationships to identify in which order elements should be accessed. For this type of relationship, it is proposed to use the temporal relationship defined by Allen [23]: Before, after, during, meets, overlaps, start and finish.
As regards content, it inherits the structure from Portfolio, which has been described previously. In this case, a PorfolioAssessment limits its content to Assessment only. This restriction is set in the logic rule shown in equation (1).
The AssessmentFeature concept labels both, time when Assessment is added to the student portfolio and time when it is resolved or accessed by student.
A LearnerPortfolio belongs to a student and is supervised by teacher. This is expressed in the ontology through ownedBy relationship, that relates it with Learner, and supervisedBy relationship that relates it with the Educator concept. In the case of TeacherPortfolio occurs only ownedBy relationship relating Educator with TeacherPorfolio concepts.
The PortfolioAssessment allows teacher, responsible for the teaching-learning process, to determine whether the learning objectives are achieved [24]. Through LearningGoal concept, the teacher sets goals that students must achieve. Each of these goals will be associated with elements of the student portfolio.


Moreover, PortfolioAssessment has rubric, which describes criteria or standards associated with learning objective. In the Portfolio ontology, the hasRubric relationship relates PortfolioAssessment and Rubric concepts.
As regards the level of knowledge that is expected to reach the student, revised Bloom's taxonomy was used [25][26].
The KnowlegeLevel concept conceptualizes the levels of knowledge, which will be associated with the goals that must be achieved.
The ontology that describes assessment are shown in Figure 4. According with the actors involve in the assessment it has identified three types of assessments [27]: (a) Self Assessment is an assessment taking and solved by an student, in this case, the student play the role of teacher and student. He/she evaluates his/her own progress and suggest his/her own qualifications, (b) PeerAssessment is the assessment perform among peers, it allows students to reflect critically and possibly suggest scores on learning of their peers, (c) HeteroAssessment allows teacher assess a group of students.
The hasEvaluator and hasAssessed relations restrict the agents involved in each type of assessment. According with that, hasEvaluator and hasAssessed relations link SelfAssessment with Learner classes. In the same way, the hasEvaluator relation relates HeteroAssessment with Educator and hasAssessed relates HeteroAssessment with Learner classes. Also, PeerAssessmen class has four relations instead of two. One pair of hasEvaluator and hasAssessed relations relates PeerAssessment class with Educator classes meaning that the assessment takes effect between peer educators. The other pair of relation relate PeerAssessment with Learner meaning that the assessment takes effect between peer learners. Table 1 shows the logical rules to restrict the type of assessment according with agent involved.
Because a portfolio assessment is used for assessing student, it is expected it has at least same selfAssessments associated. In the model presented, this logic restriction is established by equation (2).


The link of a PortfolioAssessment with the Assessment class also allows of including in a portfolio different moments in the evaluation process [28][29][30]: (a) formative assessment (Formative concept), when you want to find out if the learning objectives are being achieved and what needs to be done to improve student performance. It performs throughout the entire course and its objective is to help students in their learning process and point out shortcomings and mistakes. (b) Summative assessments (Summative concept) is designed with the aim to measure and judge learning in order to certify, assign ratings or determining promotions. In general, this type of assessment is taken when the course is ending. It is a way for teachers, to validate if the goals set at the beginning were met.

|=(x Assessment(x)  y (hasAgent(x,y)Author(y)))
(c) Diagnostic assessments (Diagnostic concept), take effect early in the teaching process with the aim to determine the level of knowledge of students before starting the learning process. It is a way for teacher, to adjust and adapt the course content according to the results [2]. These concepts are linked to the concept Assessment through hasMoment relationship.
Summarizing, AOnet defines a set of tools that can be used in the definition of assessments and thus achieve different levels of knowledge. This work has been presented in [5].

Case study
For a better understanding of the ontology network, a case study is presented. The case study shows instances that correspond to real cases of subjects from Ingeniería en Sistemas de Información study of Universidad Tecnológica Nacional, Facultad Regional Santa Fe.
In order to continue with the implementation of the AOnet considering Portfolios assessment, new relationships linking concepts from different ontologies of the network were incorporated. Assessment concept is an EducationalResource specialization concept.

AOnet Meta relation implementation
The Assessment concept is related to AssessmentMetadata concept through isDescribedBy relationship, stating that an assessment is described by the corresponding metadata. In practice the ontology also implements the inverse relationship named describes, which states that metadata modeled in AssessmentMetadata concept describes assessments belonging to Assessment class. Figure 6 shows some AONet instances used to exemplify the implementation of portfolios for assessment corresponding to the course mentioned above. AIPortfolioAssessment is a portfolio of assessments (instantiated in Assessment class) belonging to the student María Gómez (instance mariaGomez of Learner class) and is supervised by the teacher responsible for the subject Milagros Gutiérrez (instance milagrosGutierrez from Educator concept), situation expressed through relationships ownedBy and supervisedBy).

AIPortfolioAssessment
contains assessments of intelligentAgentUnit and machineLearningUnit. This situation is expressed through the relationship hasEntry that links instances of AIPortfolioAssessment with instances of Assessment concept.
AIPortfolioAssessment has an organization (represented by aIPortfolioOrganization instance and hasOrganization relationship) that uses the before instance of TemporalRelationship concept to indicate that in the portfolio assessments mentioned, assessments corresponding to machineLearningUnit must be resolved before assessments of intelligentAgentUnit.
AIPortfolioAssessment allows educators to set learning objectives, for example to understand the concept of Intelligent Agent. This situation is expressed by the relation estimateAccomplishment that links iAPortfolioAssessment (instance of PortfolioAssessment) with understandIntelligentAgentConcept i(nstance of LearningGoal concept).
The case study presented validates the ontology network from the point of view of their coverage, ie, as an appropriate instrument to express the necessary concepts and relations between them to define portfolios. This ontology is the conceptual framework to build a tool with the necessary functionalities for portfolio management.
One of the advantages that immediately arise when considering ontologies based tools is the representation of knowledge. A fundamental difference between generating knowledge and generating data, is that knowledge can be used by agents to learn, infer new knowledge and communicate with other agents.
Thus, this tool can guide learners in the order that is necessary to incorporate the concepts of a particular material, or noting teachers on the lack of assessments for a given unit or in a particular moment. The tool can also provide means of search and retrieval of learning materials in an efficient way. In short, the use of ontologies to describe a domain can treat the information from a semantic point of view resulting in intelligent tools for knowledge.

Conclusions
The pedagogical objectives of e-portfolios are various: they allow students to describe their learning path, increase awareness of their strengths and weaknesses, take responsibility and increase their autonomy and have a unified way of presenting their competences. However, to ensure the success of a learning activity, teachers must consider different variables that make the e-portfolio assessment harder and more time consuming.
The importance of a system based on formal rules that assist the assessment of an e-portfolio is undeniable. In this work an ontology that formalize portfolios for the learning process assessment in an e-learning context was presented. This ontology is a part of the AONet ontology network that model different areas of interest to be considered in the assessment domain assisted by technologies. The advantages in the use of this ontology network is the domain modular organization that facilitate the collaborative work.
As a continuation of previous work, this contribution can go a step further by incorporating the concept of portfolio as a valuable tool to assess learning considering key aspects in the execution of these techniques, such as intervening actors, levels of knowledge to evaluate, instruments to use, goals to achieve, and so on.
Through logical rules restrictions on modeling these concepts were defined. Rules are also used to represent pedagogical aspects that although not restrict the model, can be used as a guide for teachers who use these tools to assess their students and improve teaching process in general. These pedagogical rules are part of the proposed formalizing the target domain. That is, not only seeks to present a new ontology but is intended with this work contribute to building tools that support teachers in their daily work by providing recommendations based on expert knowledge and improving, in this way, quality teaching process in its entirety.
The use of the ontologies and semantic web tools may provide advanced features for evaluating competency and performance, sharing resources, and seeking help. As a future work we will conduct experiments and analysis methods to continue to evaluate and improve the AONet Ontology Network and to incorporate this new ontology in a tool that is being developed for the semiautomatic generation of assessments as presented in [5].