1. Introduction

This paper is an extension of work originally presented in ICTC2018[1].

Physical computing refers to downloading data from the real world to a digital device, processing it in the form of software, and then outputting the results to a monitor, LED or various devices. To put it more simply, computers and the real world are talking through data. The main content of the physical computing was developed by artists dealing with New Media Art. This is because it was necessary to understand computers in order to create works in the field of media art using images or light.

Physical computing is the foundation of the Internet of Things field. The basic nature of the Internet of Things is that it recognizes its surroundings through sensors and shares or processes the necessary information over the network, so it can be said to be an important technology that forms the Internet of Things.

On the other hand, for students who are new to programming, physical computing can be a good way to learn how computers and programs work. For example, students can learn about computers by assembling blocks directly, controlling robots by assembling and controlling them, or using sensors and outputs that can be easily detached to measure the quality and temperature of air which makes it easier for them to understand the principles of machine and programming.

Simplified coding programs and computer devices help students learn about physical computing. Instead of advanced programming languages such as C, students can learn programming simply by using block-coding software called Scratch. And instead of dealing with complex computer parts, students can understand the principles of computers by assembling microcomputer boards such as Arduino and Raspberry Pies, as well as various sensors and outputs to complete their own machine.

 With this characteristic that the virtual world and the real world can exchange information, it is expected that the software industry will play an important role in the future thanks to the fact that it is the basis of the Internet of Things field that recognizes the surrounding environment and processes the necessary information, and that it can be approached with a simple and active image in terms of programming education.

In physical computing, evaluation metrics for learners are very important. Those metrics are very useful in evaluating learners’ progresses, teaching-learning models, class materials, etc. However, in the literature, there is little research work on evaluation of learners in physical computing. The purpose of this paper is to develop evaluation metrics of learners in physical computing and justify them based on rigorous statistical analysis.

The rest of this paper is organized as follows. In Chapter 2, related works for physical computing are introduced. In Chapter3, evaluation metrics of learners are proposed and discussed. In Chapter 4, statistical analysis is presented for those evaluation metrics. Finally, in Chapter 5, conclusions and further research issues are presented.

2. Related Works

2.1. Evaluation standards of Physical Computing

In this section, literature review works for evaluation standards for robot computing and physical computing are introduce.

In [2], achievement criteria of robot computing are proposed as below. In their work, criteria of robot computing is classified into 5 categories, understanding of robot, robot experience, robot motor control and sensor experience, control of sensor and creation of works, project in daily life, respectively.

[Step 1] Understanding Robot

• Definition of robots

I can understand and explain the basic elements of robot definition.

• Robot type and composition

I can understand the kinds of robots used in real life and the parts and applications that make up them.

• Robot operating principles

Robots understand how they react through external input or recognition and know how they work.

[Step 2] Robot Experience

• Robotic rules and safe use

I can understand the basic rules for building robots and how to use them safely.

• Understanding robot behavior

It is able to know necessary procedures for robot operation such as connecting devices and turning them on with variety of robots.

• Fabricate simple motion robot

Robotic behavior can be created in the order presented through simple block programming. It can also be created by modifying robot movements by applying various themes.

• Brief description of operation robot

Simple block programming can create robot behavior and explain the contents of the procedure.

[Step 3] Robot Motor Control and Sensor Experience

• Understanding Rotational Motion

Understanding the motor, learning about the operating procedure, principle of rotation, and component control required for member operation can be understood and simple robot rotation operation can be created and understood.

• Various robot-driven work production

Based on the principle of motion and understanding of the robot’s driving procedure, various robots can be used to make a drive. I can also learn how to control the precise movement of a robot.

• Utilization of robot drive works

I can explain to my friends how the robot drives, how the robot is programmed in order, and its functions. It develops social relationship cooperation and communication skills through sharing of works and expression activities (recognition of goal achievement, sharing of error-solving experiences, etc.).

• Creation of simple sensor robot works

I can create a robot or device that can control behavior combined with simple sensors. It can make robots or devices using one of several sensors, including light sensors, ultrasonic sensors and sound sensors.

• Brief description of sensor robot work

I can identify the types of sensors used in my life, understand the characteristics of sensors, and describe robots or devices with sensors. I can express to my friends the design, function and programming procedures of a robot or device-driven work combined with one sensor. It develops social relationship cooperation and communication skills through sharing of works and expression activities (recognition of goal achievement, sharing of error-solving experiences, etc.).

[Step 4] Control of Sensors and Creation of Works

• Produce various sensor robot works

I can create and program controllable robot or device works by combining various sensors that can solve problems found in my life. It can also create works and explain them logically according to the programming sequence according to the purpose and purpose of production.

• Description of the various sensor robot works

I can describe the function, use and programming procedures and methods of robot or device work combined with various sensors. It also develops social relationship cooperation and communication skills through works sharing and expression activities (recognition of goal achievement, sharing of error-solving experiences, etc.).

• Rule Design and Robot Fabrication

Understanding the conditions for compliance with the rules, understanding the algorithms that can perform them, and designing the parts and devices needed to drive can be designed to operate through programming that applies algorithms to create robots or devices and perform them.

[Step 5] Project in Daily Life

• Designing Robot Works in Life

I can think about topics that can solve life problems and design algorithms and creative works that can define necessary parts and functions and control them by individuals or teams.

• Making Robot Works in Life

A creative work can be created through programming with algorithms applied to create robots or devices designed by individuals or teams and to perform defined functions according to topics that can solve life problems.

• Sharing and Expression of Robot Works

To solve problems found in life, creative works created by individuals or teams can be shared with friends, and better improvements can be found and expressed. In addition, better algorithms can be found, supplemented and shared to perform functions.

2.2. Educational Effects of Physical Computing

In [3], they presented EPL and physical computing curriculum using Entry and sensor boards that can be easily learned and used by elementary school students. The results of this study showed that both the comparative groups applying the problem-oriented EPL curriculum and the groups applying the additional problem-oriented EPL curriculum and the physical computing curriculum have significant effects on the improvement of students’ computational thinking skills. Specifically, the problem-oriented EPL curriculum and the physical computing curriculum using the sensor board contributed to increasing learner’s problem-solving skills, and showed that they are more effective in terms of improving satisfaction and computational thinking-related problem-solving skills when the physical computing education is combined with the programming training alone

In [4], high school students in the industrial field were trained in programming Android applications that control Arduino using app computers and analyzed the changes in students’ creative and fusion thinking abilities. To this end, they designed a teaching course based on a creative problem-solving model and an integrated thinking model. Studies have resulted in significant improvements in diffuse thinking and motivational factors among the subcomponents of creative problem solving. In addition, according to the results of a survey by students on integrated thinking, many students answered that they can devise IoT systems that can be applied to their daily lives based on the knowledge learned through this class. Thus, the education of physical computing using app computers and Aduinos could be seen to have a positive effect on students’ ability to think creatively and diversely.

On the other hand, other research works showed benefits of physical computing in ICT education [5-7].

3. Development of Evaluation Metrics for Learners

3.1. Design Principles

In order to development of evaluation standards of learners in physical computing, the following principles are adopted [1].

First, student-oriented activity is emphasized. Second, process-oriented activity is emphasized. Third, cooperative activity is encouraged rather than individual study in physical computing. Fourth, attitudinal and emotional experience or achievement are included.

In overall, evaluation standards are developed based on educational philosophy called constructivism [8]. Constructivism has the following major characteristics. First, it is assumed that there is no objective knowledge in the real-world. It means that every can make his or her own knowledge depending on his or her way of thinking or view. Second, it encourages social interaction. In other words, cooperative works are encouraged for study. Third, it also emphasizes problem-solving ability for authentic problems. Authentic problems mean that problems can happen in our daily life.

3.2. Evaluation Metrics

The following evaluation standards of physical computing are proposed in [1].

Table 1. Evaluation Standards of Learners

Division	Standards
Study Guidance	-Do students know study objective in advance or in the beginning of the activity? -Do students know any prerequisite or background information necessary for the activity? -Do students know the necessary equipment or teaching aids for the activity? -Do students know study materials for the physical computing activity?
Cooperation	-Does each student have a proper role in physical computing activity? -Is any leader in the group? -Are ways of communication good enough during activity?
Safety	-Do students know safety instruction for the activity? -Do students break or ruin any parts in the activity?
Output	-Do students finish the activity and produce output? -Is output good enough as expected or incomplete? -Are students supposed to have a change to expression their review or impression of the activity?
Report	-Do students write a memo or report during the activity? -Do students produce a final report after end of the activity?
Attitude	-Are students polite in the process of activity? -Does any student cause a brawl and stop the activity?
Time	-Do students spend right time for the whole class? -Do students spend too much time in specific activity of the class?
Emotion	-Are students satisfied with the activity? -Are students interested in physical computing activity? -Do students have some confidence on the subject after activity?
Theory	-Do students know the principle or theory of the physical computing activity? -Can students link the theory with physical computing activity?

4. Statistical Analysis

4.1. Design of Analysis

Verification of the reliability of the tool to measure the importance of evaluation metrics for learners in physical computing showed that Cronbach α was 0.90 and was reliable, as shown in Table 2.

Table 2. Reliability of Measurement Tool

Item Number	Cronbach’s α
23	0.90

The data collected in this study were analyzed using the Statistical Package for the Social Science (SPSS) WIN 25.0 program. The analyzer method calculated Cronbach α to verify the reliability of the measuring instrument. In addition, the average and standard deviation were obtained in order to find out the importance of the evaluation standards of learners in the physical computing, and the correlation was conducted to understand the relationship between the sub-areas of the evaluation metrics of learners in the physical computing.

4.2. Analysis Results

-Descriptive Statistics

The average and standard deviation of questions asked about the importance of evaluation metrics of learners in physical computing are as shown in Table 3.

Table 3. Results of Descriptive Statistics

Evaluation Standard	AVG.	SD
1. Do students know study objective in advance or in the beginning of the activity?	3.66	1.10
2. Do students know any prerequisite or background information necessary for the activity?	3.16	1.14
3. Do students know the necessary equipment or teaching aids for the activity?	2.91	1.17
4. Do students know study materials for the physical computing activity?	2.84	1.11
5. Does each student have a proper role in physical computing activity?	3,88	1.04
6. Is any leader in the group?	3.22	1.21
7. Are ways of communication good enough during activity?	4.22	1.01
8. Do students know safety instruction for the activity?	4.56	0.72
9. Do students break or ruin any parts in the activity?	3.50	1.30
10. Do students finish the activity and produce output?	3,41	0.95
11. Is output good enough as expected or incomplete?	3.16	0.92
12. Are students supposed to have a change to expression their review or impression of the activity?	4.06	0.91
13. Do students write a memo or report during the activity?	2.97	1.23
14. Do students produce a final report after end of the activity?	2.94	1.19
15. Are student polite in the process of activity?	4.13	0.91
16. Does any student cause a brawl and stop the activity?	3.69	0.90
17. Do students spend right time for the whole class?	4.34	0.70
18. Do students spend too much time in specific activity of the class?	3.31	1.09
19. Are students satisfied with the activity?	4.28	0.68
20. Are students interested in physical computing activity?	4.41	0.67
21. Do students have some confidence on the subject after activity?	4.19	0.93
22. Do students know the principle or theory of the physical computing activity?	3.56	1.11
23. Can students link the theory with physical computing activity?	3.66	1.04

Where AVG and SD stand for average and standard deviation, respectively.

4.3.    Sub-area of Descriptive Statistics

The results of the average and standard deviation of the sub-area of the importance of the evaluation metrics of learners in physical computing showed that the average of the importance of the evaluation metrics of learners in physical computing was 3.65, as shown in Table 4.

Among the sub-categories of the importance of learners’ evaluation metrics for physical computing, the average was 4.29, followed by “safety” 4.03, “attitude” 3.91, “time” 3.83, “cooperation” 3.77, “theory” 3.61, “output” 3.54 and “student guidance” 3.14 and “report” 2.95 were the lowest.

Table 4. Results of Descriptive Statistics of Sub-area

Evaluation Standard Area	AVG.	SD
Student Guidance	3.14	0.97
Cooperation	3.77	0.92
Safety	4.03	0.86
Output	3.54	0.67
Report	2.95	1.17
Attitude	3.91	0.78
Time	3.83	0.68
Emotion	4.29	0.58
Theory	3.61	1.02
Overall Evaluation Metrics	3.65	0.56

Where AVG and SD stand for average and standard deviation, respectively.

As shown above in Table 4, among the sub-areas of the learners’ evaluation metrics in physical computing, the importance of ‘emotion’ is highest and the importance of ‘report’ is lower than that of other areas.

4.4. Correlation Analysis

The results of the correlation between the sub-area of evaluation metrics of learners in physical computing and the entire evaluation metrics are shown in Table 5.

Where * p<.05, *** p<.001

As shown above in Table 5, student guide(r=.661, p<.001), cooperation(r=.802, p<.001), safety(r=.716, p<.001), output(r=.419, p<.05), report (r=.764, p<.001), attitude(r=.696, p<.001, time(r=.614, p<.001), emotion(r=.616, p<.001), and theory (r=.606, p<.001) showed a statistically significant static correlation with evaluation metrics of learners in physical computing. Therefore, the higher the importance of student guidance, cooperation, safety, output, report, attitude, time, emotion, and theory, the higher the importance of evaluation metrics of learners in physical computing.

Table 5. Results of Correlation Analysis

Area	Correlation Value
Student Guidance	0.661***(0.000)
Cooperation	0.802***(0.000)
Safety	0.716***(0.000)
Output	0.419*(0.017)
Report	0.764***(0.000)
Attitude	0.696***(0.000)
Time	0.614***(0.000)
Emotion	0.616***(0.000)
Theory	0.606***(0.000)

5. Conclusions and Further Research Issues

Physical computing is very important in current ICT education and software education since it can provide learning-by-doing activity for learners. In other words, learners can enjoy the physical computing activity and have interests and motivation in physical computing class. Physical computing becomes very popular as ICT education is extended in the future.

In physical computing, evaluating learners is very essential in checking learners’ study progress. Also, evaluating learners is also important in developing teaching-learning models in classrooms, study contents, etc. In the literature, there is little work on evaluation standards or metrics of learners. In the previous research work, new evaluation standards are developed and proposed. Through statistical analysis, overall evaluation standards are justified to use in physical computing.

In this research, based on previous research work, the purpose of the work is to justify the results one more time. Using descriptive statistics and correlation analysis, overall 23 evaluation metrics of learners are verified.

The immediate research issues are as follows. First of all, it is necessary to develop more detailed evaluation metrics depending on physical computing tools such as robots, toy, etc. Second, it is also necessary to develop evaluation standards or metrics for study materials. It is not known yet how to develop study materials in physical computing. That is, we need the general design principles and detailed guidelines.

Conflict of Interest

The authors declare no conflict of interest.

1. W. C. Jun, “Development of Evaluation Standards of Learners for Physical Computing”, in 2018 International Conference on Information and Communication Technology Convergence, Jeju Island, Korea, 2018. http://10.1109/ICTC.2018.8539466
2.  C. Kim, “A Study on the Achievement Criteria of Robot Computing Curriculum for Elementary School” Journal of the Korean Society of Information Education, 21(1), 97-104, 2017. http://dx.doi.org/10.14352/jkaie.2017.21.1.97
3.  J. H. Kim, D. H. Kim, “Development of Physical Computing Curriculum in Elementary Schools for Computational Thinking” Journal of the Korean Society of Information Education, 20(1), 69-82, 2016. http://dx.doi.org/10.14352/jkaie.2016.20.1.69
4.  S. Y. Choi, S. M. Kim, “Effects of Physical Computing Education Using App Inventor and Arduino on Industrial High School Students’ Creative and Integrative Thinking” Journal of the Korean Association of Computer Education, 19(6), 45-54, 2016.
5.  J. K. Shim, H. C. Kim, W. G. Lee, “A Study on Gender Differences in Programming Attitude and Achievements on Physical Computing Education in Informatics Curriculum Revised 2015” Journal of the Korean Association of Computer Education, 19(4), 1-9, 2016.
6.  I. H. Yoo, “The Effects on Flow at Using Robots of Introductory Programming Course” Journal of the Korean Society of Information Education, 17(3), 329-337, 2013. http://dx.doi.org/10.14352/jkaie.2013.17.3.329
7.  J. B. Song, S. H. Baik, T. W. Lee, “The Effect of Robot Programming Learning Considered Gender Differences on Female Middle School Student’s Flow Level and Problem Solving Ability” Journal of the Korean Association of Computer Education, 12(1), 44-55, 2009..
8. Constructivism, https://www.learning-theories.com/ constructivism.htm

(Click Here)

(Click Here)