A Study on Development of Evaluation Metrics for Learners in Physical Computing

A R T I C L E I N F O A B S T R A C T Article history: Received: 16 June, 2019 Accepted: 11 August, 2019 Online: 03 September, 2019 Physical computing is important for ICT (information and communication technology) Education and other informatics education such as software education since physical computing can provide learning-by-doing education for students. It is also a strong tool to increase students’ programming ability using various type of physical computing tools like a robot. In physical computing, it is necessary to have evaluation metrics for learners in order to test students’ progresses and environmental safety, etc. However, in the literature, there is little work for evaluation metrics for learners. In this paper, for the proposed evaluation metrics for learners developed in the previous research, the usefulness and validity are discussed and justified with statistical analysis. The final 23 evaluation metric will be very useful in actual physical computing education.


Introduction
This paper is an extension of work originally presented in ICTC2018 [1].
Physical computing refers to downloading data from the real world to a digital device, processing it in the form of software, and then outputting the results to a monitor, LED or various devices. To put it more simply, computers and the real world are talking through data. The main content of the physical computing was developed by artists dealing with New Media Art. This is because it was necessary to understand computers in order to create works in the field of media art using images or light.
Physical computing is the foundation of the Internet of Things field. The basic nature of the Internet of Things is that it recognizes its surroundings through sensors and shares or processes the necessary information over the network, so it can be said to be an important technology that forms the Internet of Things.
On the other hand, for students who are new to programming, physical computing can be a good way to learn how computers and programs work. For example, students can learn about computers by assembling blocks directly, controlling robots by assembling and controlling them, or using sensors and outputs that can be easily detached to measure the quality and temperature of air which makes it easier for them to understand the principles of machine and programming.
Simplified coding programs and computer devices help students learn about physical computing. Instead of advanced programming languages such as C, students can learn programming simply by using block-coding software called Scratch. And instead of dealing with complex computer parts, students can understand the principles of computers by assembling microcomputer boards such as Arduino and Raspberry Pies, as well as various sensors and outputs to complete their own machine.
With this characteristic that the virtual world and the real world can exchange information, it is expected that the software industry will play an important role in the future thanks to the fact that it is the basis of the Internet of Things field that recognizes the surrounding environment and processes the necessary information, and that it can be approached with a simple and active image in terms of programming education.
In physical computing, evaluation metrics for learners are very important. Those metrics are very useful in evaluating learners' progresses, teaching-learning models, class materials, etc. However, in the literature, there is little research work on evaluation of learners in physical computing. The purpose of this paper is to develop evaluation metrics of learners in physical computing and justify them based on rigorous statistical analysis.
The rest of this paper is organized as follows. In Chapter 2, related works for physical computing are introduced. In Chapter3, evaluation metrics of learners are proposed and discussed. In Chapter 4, statistical analysis is presented for those evaluation ASTESJ ISSN: 2415-6698 metrics. Finally, in Chapter 5, conclusions and further research issues are presented.

Evaluation standards of Physical Computing
In this section, literature review works for evaluation standards for robot computing and physical computing are introduce.
In [2], achievement criteria of robot computing are proposed as below. In their work, criteria of robot computing is classified into 5 categories, understanding of robot, robot experience, robot motor control and sensor experience, control of sensor and creation of works, project in daily life, respectively. (2) Robot type and composition I can understand the kinds of robots used in real life and the parts and applications that make up them.
(3) Robot operating principles Robots understand how they react through external input or recognition and know how they work. [ Step 2] Robot Experience (4) Robotic rules and safe use I can understand the basic rules for building robots and how to use them safely.

(5) Understanding robot behavior
It is able to know necessary procedures for robot operation such as connecting devices and turning them on with variety of robots.
(6) Fabricate simple motion robot Robotic behavior can be created in the order presented through simple block programming. It can also be created by modifying robot movements by applying various themes.   Understanding the motor, learning about the operating procedure, principle of rotation, and component control required for member operation can be understood and simple robot rotation operation can be created and understood.
(9) Various robot-driven work production Based on the principle of motion and understanding of the robot's driving procedure, various robots can be used to make a drive. I can also learn how to control the precise movement of a robot.
(10) Utilization of robot drive works I can explain to my friends how the robot drives, how the robot is programmed in order, and its functions. It develops social relationship cooperation and communication skills through sharing of works and expression activities (recognition of goal achievement, sharing of error-solving experiences, etc.).
(11) Creation of simple sensor robot works I can create a robot or device that can control behavior combined with simple sensors. It can make robots or devices using one of several sensors, including light sensors, ultrasonic sensors and sound sensors.

(15) Rule Design and Robot Fabrication
Understanding the conditions for compliance with the rules, understanding the algorithms that can perform them, and designing the parts and devices needed to drive can be designed to operate through programming that applies algorithms to create robots or devices and perform them. I can think about topics that can solve life problems and design algorithms and creative works that can define necessary parts and functions and control them by individuals or teams.

(17) Making Robot Works in Life
A creative work can be created through programming with algorithms applied to create robots or devices designed by individuals or teams and to perform defined functions according to topics that can solve life problems.

(18) Sharing and Expression of Robot Works
To solve problems found in life, creative works created by individuals or teams can be shared with friends, and better improvements can be found and expressed. In addition, better algorithms can be found, supplemented and shared to perform functions.

Educational Effects of Physical Computing
In [3], they presented EPL and physical computing curriculum using Entry and sensor boards that can be easily learned and used by elementary school students. The results of this study showed that both the comparative groups applying the problem-oriented EPL curriculum and the groups applying the additional problemoriented EPL curriculum and the physical computing curriculum have significant effects on the improvement of students' computational thinking skills. Specifically, the problem-oriented EPL curriculum and the physical computing curriculum using the sensor board contributed to increasing learner's problem-solving skills, and showed that they are more effective in terms of improving satisfaction and computational thinking-related problem-solving skills when the physical computing education is combined with the programming training alone In [4], high school students in the industrial field were trained in programming Android applications that control Arduino using app computers and analyzed the changes in students' creative and fusion thinking abilities. To this end, they designed a teaching course based on a creative problem-solving model and an integrated thinking model. Studies have resulted in significant improvements in diffuse thinking and motivational factors among the subcomponents of creative problem solving. In addition, according to the results of a survey by students on integrated thinking, many students answered that they can devise IoT systems that can be applied to their daily lives based on the knowledge learned through this class. Thus, the education of physical computing using app computers and Aduinos could be seen to have a positive effect on students' ability to think creatively and diversely.
On the other hand, other research works showed benefits of physical computing in ICT education [5][6][7].

Design Principles
In order to development of evaluation standards of learners in physical computing, the following principles are adopted [1].
First, student-oriented activity is emphasized. Second, processoriented activity is emphasized. Third, cooperative activity is encouraged rather than individual study in physical computing. Fourth, attitudinal and emotional experience or achievement are included.
In overall, evaluation standards are developed based on educational philosophy called constructivism [8]. Constructivism has the following major characteristics. First, it is assumed that there is no objective knowledge in the real-world. It means that every can make his or her own knowledge depending on his or her way of thinking or view. Second, it encourages social interaction. In other words, cooperative works are encouraged for study. Third, it also emphasizes problem-solving ability for authentic problems.
Authentic problems mean that problems can happen in our daily life.

Evaluation Metrics
The following evaluation standards of physical computing are proposed in [1].

Division Standards
Study Guidance -Do students know study objective in advance or in the beginning of the activity? -Do students know any prerequisite or background information necessary for the activity?
-Do students know the necessary equipment or teaching aids for the activity?

Design of Analysis
Verification of the reliability of the tool to measure the importance of evaluation metrics for learners in physical computing showed that Cronbach α was 0.90 and was reliable, as shown in Table 2.

Item Number
Cronbach's α 23 0.90 The data collected in this study were analyzed using the Statistical Package for the Social Science (SPSS) WIN 25.0 program. The analyzer method calculated Cronbach α to verify the reliability of the measuring instrument. In addition, the average and standard deviation were obtained in order to find out the importance of the evaluation standards of learners in the physical computing, and the correlation was conducted to understand the relationship between the sub-areas of the evaluation metrics of learners in the physical computing.

-Descriptive Statistics
The average and standard deviation of questions asked about the importance of evaluation metrics of learners in physical computing are as shown in Table 3. Where AVG and SD stand for average and standard deviation, respectively.

Sub-area of Descriptive Statistics
The results of the average and standard deviation of the subarea of the importance of the evaluation metrics of learners in physical computing showed that the average of the importance of the evaluation metrics of learners in physical computing was 3.65, as shown in Table 4.
Among the sub-categories of the importance of learners' evaluation metrics for physical computing, the average was 4.29, followed by "safety" 4.03, "attitude" 3.91, "time" 3.83, "cooperation" 3.77, "theory" 3.61, "output" 3.54 and "student guidance" 3.14 and "report" 2.95 were the lowest. Where AVG and SD stand for average and standard deviation, respectively.
As shown above in Table 4, among the sub-areas of the learners' evaluation metrics in physical computing, the importance of 'emotion' is highest and the importance of 'report' is lower than that of other areas.

Correlation Analysis
The results of the correlation between the sub-area of evaluation metrics of learners in physical computing and the entire evaluation metrics are shown in Table 5.
Where * p<.05, *** p<.001 As shown above in Table 5 showed a statistically significant static correlation with evaluation metrics of learners in physical computing. Therefore, the higher the importance of student guidance, cooperation, safety, output, report, attitude, time, emotion, and theory, the higher the importance of evaluation metrics of learners in physical computing.

Conclusions and Further Research Issues
Physical computing is very important in current ICT education and software education since it can provide learningby-doing activity for learners. In other words, learners can enjoy the physical computing activity and have interests and motivation in physical computing class. Physical computing becomes very popular as ICT education is extended in the future.
In physical computing, evaluating learners is very essential in checking learners' study progress. Also, evaluating learners is also important in developing teaching-learning models in classrooms, study contents, etc. In the literature, there is little work on evaluation standards or metrics of learners. In the previous research work, new evaluation standards are developed and proposed. Through statistical analysis, overall evaluation standards are justified to use in physical computing.
In this research, based on previous research work, the purpose of the work is to justify the results one more time. Using descriptive statistics and correlation analysis, overall 23 evaluation metrics of learners are verified.
The immediate research issues are as follows. First of all, it is necessary to develop more detailed evaluation metrics depending on physical computing tools such as robots, toy, etc. Second, it is also necessary to develop evaluation standards or metrics for study materials. It is not known yet how to develop study materials in physical computing. That is, we need the general design principles and detailed guidelines.