1. Introduction

At present, the learner-centered teaching method combined with the active learning is accepted widely that mainly focuses on learners, and can increase the effectiveness of learning and teaching system.  One famous active learning method is the flipped classroom learning focused on the learners as the center and this system changes the traditional learning system like the lecture-based teaching from listening lecture contents  in the classroom and doing homework or exercises after in-class time to the learners can study by themselves from the knowledge resource like out-of-school learning, such as teaching videos, online learning materials. On the other hand, the learners will practice in-class time to study, and the instructors will provide assignments, problem questions or case studies for learners to have a brainstorm and exchange ideas for  finding those answers under the guidance of the instructors in the classroom.  In addition, the learners and the instructors can join learning activities in the classroom to improve their knowledge, abilities, and skills.

In the previous academic year, a traditional object-oriented analysis and design (OOAD) course was the lecture-based learning which instructors will lecture the lessons in the classroom. After that, the class spends the rest of time for the questions of the case study to practice object-oriented analysis and design skill.  Therefore, there is too less time to discuss and share opinions among learners and instructors in the classroom for implementation.  Moreover, learners cannot make the decision or conclude the different answers for problems or questions about study contents, when learners do group work outside the class time.  A few reasons are that some learners cannot apply the course content to real world problems, or they are not able to compromise their different ideas generated from the understanding content.  Moreover, they cannot achieve the best solutions for the view of object-oriented analysis and design.

For these limitations of traditional OOAD course, the instructors should try the flipped classroom learning in this course to analyze results of using this new learning system for OOAD course.  In addition, the workshop activity outside the classroom were included to practice the object-oriented analysis and design skill. The learners’ behavior of preparing for class and evaluation scores was analyzed to find the results of using the flipped classroom learning with a workshop activity in OOAD course.  Learners’ opinion was also investigated for concluding that the flipped classroom learning system can encourage learners to learn and practice analysis and design skill more or not.

This research paper is an extension of work originally presented in conference [1]. Proposed research details will be described in the following sections.  The flipped classroom learning system and research about it are explained in Section 2.  The research methodology is demonstrated in Section 3.  Next, the research results with some discussions and conclusions are described in Section 4 and Section 5, respectively. Then, the final section is the suggestions.

2. The Flipped Classroom Learning System

The flipped classroom [2-3] is a learning combination between online studying and teaching in the classroom.  This learning system is different from the traditional lecture-based learning that the learners do some practices (e.g. class discussions, presentations or assignments) in class, after they listened to lectures before in-class time outside the classroom. So, the learners can study the courses’ subjects by learning the online content lecture videos or other online media resources outside the classroom, and then they will do in-class activities (e.g. having discussions, finishing exercises) in the classroom to more understand the subject.  It is also helpful for learners to develop their knowledge and skills to compromise on the interests the individual learners.

Moreover, online social community has currently be contained a lot of knowledge and useful information for learning.  This leads to instructors and learners to have valuable resources to teach and study.  Instructors have more powers to generate learning experiences for learners.  Learners will also understand the subject concept deeply from these experiences, thus the knowledge is not restricted to that has been transferred from the instructors [4].  Furthermore, there is some research about the development of Hybrid learning systems like integrated learning and teaching in the classroom [5].  Online tools can assist learners to do assignments, for examples, finding the answers or the guidance for problem solving in class.  In addition, discussions on topics related class lessons or case studies can be run as online groups and learners can search information from other resources for generating knowledge besides course materials.

Therefore, the flipped classroom [6-9] is new learning and teaching system that can improve the academic exam scores of learners and encourages learners to exchange ideas and discuss and about different course contents.  In addition, the flipped classroom learning is flexible enough to integrate with other methods for achieving more effectiveness of learning and adapt to the class topics as much as possible.  Nevertheless, there is the limitation of individual learners’ time that they must spend to study the preparation before in-class time attending.  For example, the research [10] applied the flipped classroom learning in computer programming course that learners had to study the definition and syntax of programming languages before class and get their hands-on practices for deeper learning taking place in the classrooms.  It was found that the flipped classroom allows for more interaction during the face-to-face meetings, and for both instructors and learns to gain experience from the deeper learning in the classroom.  However, the learners need to take time for preparation before class and practicing in class regularly and learn on their own.

The effectiveness of the flipped classroom results from the article [11] was discovered that this learning system can include more content than the traditional lecture because of design and problem-solving scores for the engineering course.  Learners’ scores in the flipped classroom are higher than learners’ scores in the lecture-based class only, when the types of questions are separated into problem solving or no), although learners in both groups were similar in overall scores with insignificant differences.  Moreover, this paper also analyzed the student perception which expressed that most of learners like to study in the flipped classroom learning system, including using teaching online videos, while over half of the learners in the lecture class want to practice more. In the same way of the research [10], the learners must take more time to learn by themselves, though doing case studies and projects for in-class time flipped classroom learning can help the learners to understand the content deeply.

Other research [12-14] found that the flipped classroom learning has the positive impacts on the educational system and has higher performance than the traditional learning in different issues.  The paper [12] adopt the flipped classroom learning to the first-year vocational college students in Taiwan. Although there is no significant effect of the flipped classroom on the certification examination of learners, the face survey interviews of learners can be concluded that using a flipped classroom to enhance student engagement can promote active learning activities both inside and outside of the classroom.  In addition, the finding guided that group activities are the main goal of the in-class portion of the flipped classroom, but activities of individual learner can be helpful for persons who need more individual reflective time to learn.  The study [13] compared the effectiveness of flipped classroom learning with that of traditional lecture-based learning for selected topics in the college level technology course.  The results of a learner survey revealed that the learners awesomely favored use of the flipped classroom teaching style because this system supports collaboration and hands-on activities learning for solving problem during the class time.  Additionally, the flipped classroom approach made a statistically significant difference to the self-efficacy in a technology integration classroom and made learners’ confidence to apply knowledge more.

Another research [14] used surveying and interviewing to identify the performance of the flipped classroom learning.  The results discovered that the flipped classroom can engage the learners to study more in class, and can lead to more perseverance for learning.  Learners of the flipped classroom also were successful for better in productivity and the more responsibility of the learner because of various learning activities, e.g. watching videos, doing exercises and homework.  Consequently, there are interesting remark points that the flipped classroom learning, focusing on learner-centered can make the learners to be satisfied with learning and teaching style.  Unfortunately, the learners, who cannot be adapted to the flipped classroom learning system, will be faced with trouble and their grades may be decreased as well.

The concept and practical cases of related research [2-14] has given all benefits of the flipped classroom learning.  Meanwhile, a major limitation is that the learners must need time to study on their own outside the classroom and preparing before class.  Additionally, collecting suggestions of the learners in OOAD course in the previous academic year explored the learners thought that they should achieve the learning objectives of the course and can be applied to real world cases, when they have time to train the critical thinking skill for analyzing and design more besides learning object-oriented design principles, including trying analysis and design thinking on practical issue problems.  These are the reasons why the objectives of this proposed research extending the research [1] is to analyze the results of using the flipped classroom learning system in OOAD course.  The research [1] studied on the learners’ satisfaction of this learning system and discovered the advantages and limitations of the flipped classroom learning classroom.  This proposed research analyzed and reported the learners’ behavior with learner opinions in detail to confirm the improvement of the learning intention and critical thinking skills of learners. Moreover, the effectiveness of the flipped classroom learning was evaluated by a comparison of learners’ total scores and term project scores in two different academic years, which OOAD course in the current academic year has applied the flipped classroom learning, while in the previous academic year used the traditional lecture-based learning. Furthermore, learners’ opinions about a workshop activity and the suitability of the flipped classroom learning for OOAD courses were surveyed and made a conclusion.

3. Research Methodology

3.1.  Research questions

The researcher tried out the flipped classroom learning system to analyze the learners’ behavior and opinions to find out whether this system encourage learning intention and critical thinking skill or not, including  suitability for OOAD course. Moreover, learners’ scores of the flipped classroom learning were compared to those of the traditional lecture-based learning to evaluate the effectiveness of this learning system.  Furthermore, problems or obstacles in both instructor and learner views were reported for the challenges of using the flipped classroom in learning and teaching management system.

3.2.  Participants and Research Instruments

There was collecting data from 20 students enrolled in object-oriented analysis and design course which were three-year and four-year undergraduate students in Chulalongkorn University.  The accumulated grade point average (GPAX) of learners is displayed in Table 1.  The learners’ behavior, both in-class time and out-of-class time were collected for observation and questionnaire survey. Consequently, the class attendance scores, in-class assignment scores, class preparing scores, individual academic exam scores and the term project scores of the student groups were focused to analyze the effectiveness of the flipped classroom learning system.  Additionally, the answers of a questionnaire about the flipped classroom learning were summarized to gain information on learners’ attitudes and to conclude the results of the flipped classroom learning system.

Table 1.  The Accumulated grade point average of learners

Accumulated grade point average (GPAX)	Learner (persons)	Ratio of all learners
< 2.00	0	0%
2.00 – 2.50	3	15%
2.50 – 3.00	10	50%
3.00 – 3.50	4	20%
> 3.50	3	15%

3.3.  Course Activities

The course activities of the flipped classroom learning system in this proposed research are explained in Figure 1. There were data collecting of the learners’ behavior, both in-class time in the classroom by observations and out-of-class time outside the classroom by a survey. The examples of course activities were watching lecture videos, doing exercises, and answering the questions about the subjects.  In the classroom, there were group activities about case studies or interesting topics  related to subject contents, such as discussions and presentations.  Moreover, the progress of term project continued to be processed and reviewed by another group learners and the instructors in the classroom (in-class time).  Furthermore, the learners gave some presentations both pair and group work for subject contents or term project works in a few weeks.

In addition, a workshop activity about analysis and design in object-oriented view was added into this course in the current academic year.  The workshop issue is a real-world problem about environmental problem-solving arranged outside the classroom.  All learner groups had to gather requirements, analyze and design the system to solve environmental issues.

3.4.  Course Materials

The teaching materials are Thai lecture videos and English documents, e.g. PowerPoint slides created by the instructors.  In addition, some online multimedia, contents, or documents which related to the subject lessons can included in the course materials, such as YouTube videos, online documents on websites, pdf file, and other online knowledge resources.  Table 2 shows the number of lecture videos and their length in each week.

3.5.  Course Environments

Course Ville [12], which developed by Faculty of Engineering, Chulalongkorn University, is the Learning Management System (LMS) for this OOAD course.

Figure 1.  Course activities with the flipped classroom learning system.

Table 2.  The features of lecture videos.

Week	Number of episodes	Length per episode (minutes)	Total length (minutes)
1	–	–	–
2	1	5.49	5.49
3	2	23.46 26.09	49.55
4	1	22.15	22.15
5	2	7.18 9.14	16.32
6	1	17.29	17.29
7	Midterm Exam
8	–	–	–
9	2	14.16 13.09	27.25
10	1	10.15	10.15
11	1	11.46	11.46
12	–	–	–
13	1	12.05	12.05
14	–	–	–
15-16	Final Exam

All course data on Course Ville are be stored in the cloud.  Additionally, some activities can connect to Facebook to engage learners for acknowledging learning activities.  For examples, Course Ville are logged by Facebook account, and links courses to the Facebook Group.  General LMS services are also provided by CourseVille as follows:

• Multimedia course materials (e.g. lecture videos, video files, PowerPoint slides, pdf file documents, YouTube link) can be uploaded by instructors and can be downloaded or accessed by learners.
• News and announcements about the course can be announced to learners by instructors and can be gotten by learners.
• All teaching materials and tasks published by the instructors, can be acknowledged to the learners via the system which link to Facebook.
• Instructors and learners can raise questions and post topics to exchange their own ideas or opinions though the system.
• Instructors can assign tasks or exercises, while learners can do or answer online by the system. In addition, all learner submitted tasks can be scored online with the system by instructors
• Instructors and learners can check and record the class attendance of learners.
• Instructors can use some automatic classroom tools on the system, e.g. student randomizer, countdown timer, and Instant Q (lets instructors to create simple questions/polls to which students can respond suddenly).

4. Research Results

The proposed research experimented on the instructor and learners of OOAD course, and collected data from 20 students enrolled in this course.  The results of using the flipped classroom learning in OOAD course are separated into five parts.

• The results of analysis and study on learners’ behavior when using the flipped classroom learning were the analysis of learners’ preparation before class, learners’ class attendance, including the assignment and task submission.
• The results of effectiveness evaluation of using the flipped classroom learning were focused on academic exam scores by knowledge assessment, the term project scores which were compared to the behavior of learners, i.e. class attending, lecture video viewing and PowerPoint slide learning before class, and finishing all tasks through the LMS.
• The results of analysis and study on the learners’ opinions about the flipped classroom learning and the workshop activity were summarized from the questionnaire about the flipped classroom learning with a workshop activity.
• The report of problems or obstacles in the flipped classroom learning and teaching management
• The summary of the suitability of the flipped classroom learning in OOAD course.

4.1.  The Results of Analysis and Study on Learners’ Behavior

In this research, interesting learners’ behavior is separated into 3 main activities: preparing before class (watching lecture video and studying class documents), attending class, and submitting the assignments in class and after class).  These data were collected from observations and behavior survey questions.

The learners’ behavior of watching lecture videos and studying class documents before class are shown in Table 3.  According to learners’ behavior survey results, while most of the learners (75% of all learners) watched almost of lecture videos (80%-100% of all videos) before class, nobody studied class documents before class more than 80% of all documents.  Additionally, 45% and 35% of all learners studied the documents about 20%– 40% and 40%– 60%, respectively.  When the number of learners who watched lecture videos were compared to those who studied class documents, Figure 2 expressed that most of the learners from all 20 students watched the video 80-100% of all videos, but study class documents about 20-60% of all documents.  The x-axis of bar graph represents the percentage of the number of watched lecture videos to the number of total lecture videos compared to the percentage of the number of class documents to the number of total class documents, and the y-axis denotes the number of students.  Consequently, there were only 20% of all learners who usually study documents during video viewing.  Therefore, most of the learners give precedence to watching the video more than study text documents before classes.  In addition, the results concluded that most learners did not realize the importance of studying the document during the video viewing.

Table 3.  Behavior of watching lecture videos and studying documents
before class.

Behavior of watching lecture videos and studying documents before class			Learner (persons)	Ratio of all learners
% of the number of watched lecture videos to the number of total lecture videos	0% – 20%	–		0%
	20% – 40%	–		0%
	40% – 60%	2		10%
	60% – 80%	3		15%
	80% – 100%	15		75%
% of the number of class documents to the number of total class documents	0% – 20%	1		5%
	20% – 40%	9		45%
	40% – 60%	7		35%
	60% – 80%	3		15%
	80% – 100%	–		0%
% of the number of watched lecture videos with studying the document to the number of total lecture videos	0% – 20%	3		15%
	20% – 40%	2		10%
	40% – 60%	6		30%
	60% – 80%	5		25%
	80% – 100%	4		20%

Figure 2.  The number of learners watching lecture videos compare to those who studied class documents.

Moreover, the behavior analysis of learners’ video watching as Table 4 found that approximately 50% of learners focused to view and listen intently to the whole part of the video. Additionally, 30% of learners view and listen through the end without serious study.  Referring to the lecture videos during each week on Table 2, Table 5 also demonstrates the number of learners who did not watch lecture videos, and watched lecture videos through the end, including viewing only some parts of the lecture videos.  Unfortunately, there were few learners not informing viewing types during some weeks, so this information is “No record”.

Table 4.  The characteristics of watching lecture videos before class.

The characteristics of watching lecture videos	Learner (persons)	Ratio of all learners
only listen through the end	1	5%
only listen intently some part of the video	–	0%
view and listen through the end	6	30%
view and listen intently some part of the video	2	10%
view and listen intently the whole part of the video	11	55%

Table 5.  The video viewing types of learners for each week.

Week	Not view	View the end	View not end	No record	Total (persons)
1	–	–	–	–	–
2	3	6	7	4	20
3	4	5	8	3	20
4	–	12	6	2	20
5	1	18	–	1	20
6	1	15	4	–	20
7	Midterm Exam
8	–	–	–	–	–
9	3	8	6	3	20
10	–	17	1	2	20
11	2	14	1	3	20
12	–	–	–	–	–
13	–	18	–	2	20
14	–	–	–	–	–
15-16	Final Exam

According to the features of the lecture videos on Table 1 and learner’s viewing behavior on Table 5, during the week that had only a single video episode, learners would watch through the end more than the week that had more video episodes.  In addition, learners would obviously watch less through the end during the week with the length of all videos is about or more than 20 minutes.  Then, if there is more than one video episode during the week, more learners would watch through the end of the shorter length of the episode.  In the same way, if the length of a video episode is about or less than 10 minutes, learners would watch through the end of every episode more.

Furthermore, the results from the questionnaire surveyed learners’ time duration preparing for class found that the preparation time by learners 55% of all learners around about 12-24 hours before class as shown in Table 6, and nobody also do prepare more than 1 day.  Another result discovered that the popular time to prepare for the class was during the late evening to midnight as displayed in Table 7.  It was not surprising that most learners preparing themselves in the night before the morning classes.

Table 6.  The preparation time before class.

Time duration	Learner (persons)	Ratio of all learners
before class less than 3 hours	4	20%
before class 3 – 12 hours	4	20%
before class 12 – 24 hours	11	55%
before class 1 day	1	5%
before class more than 1-2 days	–	0%

Table 7.  The time of preparing for class.

Time	Learner (persons)	Ratio of all learners
0.00 – 4.00 a.m.	2	10%
4.00 – 8.00 a.m.	2	10%
8.00 – 12.00 a.m.	–	0%
0.00 – 4.00 p.m.	–	0%
4.00 – 8.00 p.m.	5	25%
8.00 – 12.00 p.m.	11	55%

Table 8.  The class attendance of individual learners.

Learners no.	Not attend	In time	Late	No record	Total (weeks)
1	–	12	–	–	12
2	2	9	1	–	12
3	–	12	–	–	12
4	4	7	1	–	12
5	–	11	–	1	12
6	1	5	6	–	12
7	–	5	5	2	12
8	1	–	11	–	12
9	1	11	–	–	12
10	1	6	5	–	12
11	–	9	3	–	12
12	–	12	–	–	12
13	–	12	–	–	12
14	4	6	1	1	12
15	2	8	2	–	12
16	–	12	–	–	12
17	–	11	1	–	12
18	7	2	2	1	12
19	–	6	4	2	12
20	4	3	5	–	12
Total	11.25%	65%	19.58%	2.92%	100%

Table 9.  The time of coming in class.

Time	Learner (persons)	Ratio of all learners
8.00 – 8.30 a.m.	4	20%
8.30 – 9.00 a.m.	8	40%
9.00 – 9.30 a.m.	6	30%
9.30 – 10.00 a.m.	1	5%
10.00 – 10.30 a.m.	–	0%
10.30 – 11.00 a.m.	1	5%

To observe learners’ behavior in class, attendances in the classroom of individual learners were checked for 12 weeks displayed in Table 8.  The OOAD course class time was 3 hours per week and started at 8.00 a.m. on Monday.  The learners who came in the classroom before 9.00 a.m. were on time, otherwise they were late.  The results found that 65% of all attendance checking are in time because most learners attended classes before 9.00 a.m., whether they missed the class.  However, there was one learner always late for class, and another learner usually missed the class.

In addition, the learners’ common time coming in class is illustrated in Table 9.  Based on the time attendance, there are 60% of all learners attending classes before 9 a.m. and participating the class activities in time.  Although there were a group of late participants (30% of all learners), they did not late more than a half an hour.  Therefore, most learners realize that it is important to attend class and practice themselves in the flipped classroom learning.

By the way, all assignment submissions of learners recorded on CourseVille were composed of exercises or topic discussions in class and individual assignments after class.  Table 10 presented the number of learners who submitted the assignments, and the rest of them who did not submit the assignments.  The results show that almost all the learners submitted assigned works every piece, except few learners.

Table 10.  Individual class attendance.

Assignment no.	Submission	Not submission	Total (persons)
1	17	3	20
2	13	7	20
3	13	7	20
4	19	1	20
5	20	–	20
6	20	–	20
7	17	3	20
8	16	4	20
9	20	–	20
10	16	4	20
Total	85.5%	14.5%	100%

In conclusion, all learners’ behavior expressed that most learners gave corporation for managing the flipped classroom learning.  They took their time to watch lecture videos for preparing before class and attended class regularly.  Although few learners missed the class or did not study before class, they did their best when joined the group activities in class. In addition, learners’ preparation for class by themselves, attending class to do in-class time activities, and doing all individual assignments, were critical thinking skill training.  Therefore, the flipped classroom learning can stimulate learners’ intention and encourage critical thinking skill of learners.

4.2.  The Results of Effectiveness Evaluation of Using the Flipped Classroom Learning

The results of effectiveness evaluation of using the flipped classroom learning are separated into 2 main parts. The first part is the comparison among evaluation scores of course exams and activities that are the midterm exam, the final exam, the term project, the assignments, the class attendance, and the video viewing before class.  The second part is the comparison between evaluation scores of learners in the current academic year with the flipped classroom learning, and in the previous academic year with traditional lecture-based learning.

There were comparisons between all scores of individual learners which consisted of the assessment scores of academic knowledge of each learner (the total scores of the midterm and the final academic exams), and the scores of each course activity in the flipped classroom learning system that were watching lecture video  in preparation before in-class time, the class attendance for joining group activities, and individual assignment or task submission for in-class time).  All scores of individual learners were inconsistent and difficult to find their relations shown in Figure 3.  The x-axis denotes the individual student number (1 – 20) and the y-axis represents the scores of learners composed of the midterm and final exam scores (out of 45 points), the class attendance scores (out of 12 points), the individual assignment scores (out of 10 points), and the lecture video viewing (out of 9 points).

Figure 3.  The exam scores of individual learners compare to each course activity.

Meanwhile, it was found that almost learners’ exam scores were consistent with the summation scores of all course activities in the flipped classroom learning, when all course activity scores, which were watching lecture videos, attending class, and submitting assignments, were combined into the total scores of course activities.

The comparison result has been revealed in Figure 4, which the x-axis and y-axis mean the same parameters of Figure 3.  The two-line graphs in Figure 4 demonstrated that learners, who did not prepare before class by watching lecture videos, did not join in-class learning activities by attending the class, and did not submit their works, would have low academic exam scores.  And there were only a few learners who did all course activities, but still got low academic exam scores.

Figure 4.  The academic exam scores of individual learners compared to the total scores of three course activites.

Moreover, the correlation analysis of academic exam scores with total scores of three course activities was performed to confirm the relationship between learners’ academic exam scores and the total scores of learners’ course activities.  The analysis result displays that the academic exam scores are significantly related to the summation of all three course activities as in Figure 5.  Therefore, the result has been concluded that the learner academic knowledge is not based on only one ability or one skill trained by only one activity but generated from the integrity of all learning and training activities.  Knowledge improvement of individual learners requires studying and practicing all activities that are preparation before studying in-class time, paying attention to the discussion or group activities of in-class time, and training on course exercises or assignments.

Figure 5.  The correlation between the exam scores of individual learners and the total scores of three course activities.

Furthermore, the flipped classroom learning increases time in class for discussing related topics, practicing on analysis and design skills, and doing the term project in the classroom under the guidance and management of the instructors.  Therefore, the comparison results as in Figure 6 expresses that the term project scores of individual learners (out of 30 points) were related to their class attendance scores, except a few learners having the good term project score without attending class regularly.  One cause of these scores is the term project was a group work, so the other members of the group could operate and review the work of absent members.  The x-axis and y-axis of Figure 6 were the same meaning as Figure 3 and Figure 4.  The correlation analysis of the term project and the class attendance was also displayed in Figure 7.

Figure 6.  The term project score of individual learners compare to the class attendance

Figure 7.  The correlation between the term project scores of individual learners and the class attendance

To evaluate the performance of the flipped classroom learning, the total scores of grading and the term project scores of learners in the current academic year were compared to those of learners in the previous academic year by the independent t-test.  These scores are also used to measure critical thinking skills of learners in this research.  Figure 8 shows the term project scores and the total scores of the current academic year with the flipped classroom learning (the table at the top of the figure) and the previous academic year with the traditional learning (the table at the bottom of the figure).

Figure 8.  the term project scores and the total scores of both academic years.

The average total scores of both academic years and the independent t-test results are demonstrated in Figure 9.  The result found that the average total scores of learners in the current academic year were higher than those of learners in the previous academic year.

Figure 9.  The t-test results of the total scores of students in current academic year and the previous academic year

According to Figure 9, the p-value of Levene’s test was greater than 0.05 (Sig. 0.612 > a), that meant the variances of two learner groups (students in the current and the previous academic years) are equal.  Additionally, the p-value of t-test was less than 0.05 (Sig. 0.066/2  < a), then there was a mean difference between total scores of learners in the current academic year and the previous academic year.  Therefore, the average total scores of learners in the flipped classroom learning were significantly higher than those in the traditional learning.

Another result was that the average term project scores of learners in the current academic year were higher than those of learners in the previous academic year.  The average term project scores of both academic years and the independent t-test results are explained in Figure 10.  Like the total scores, the variances of term project scores were equal across the two learner groups because the p-value of Levene’s test was greater than 0.05 (Sig. 0.001 > a).  Then, the t-test result displayed that the p-value of t-test is less than 0.05 (Sig. 0.021/2  < a), thus there was a mean difference between term project scores of students in the current academic year and the previous academic year. That meant the average term project scores of learners in the flipped classroom learning were a little bit significantly higher than those in the traditional lecture-based learning.

Figure 10.  The t-test results of the term project scores of students in current academic year and the previous academic year

Although there was not much difference in the average term project scores, the effectiveness of the flipped classroom learning has been higher than the traditional learning.  The reason is that the number of students in the current academic year were more than those in the previous academic years and all learner groups did the term project with the good quality.  Therefore, the result can be concluded that the flipped classroom learning helped to encourage the critical thinking skill for the learners.

4.3.  The Results of Analysis and Study on the Learners’ Opinions

The learners’ opinions were collected by the questionnaire about the features of the lecture video and all course activities in the flipped classroom learning, including a workshop activity.  All interesting course activities are watching lecture videos and studying lesson documents before class (out-of-class time), working in groups and discussing together on case studies and related topics about lessons, answering instructors’ questions

following to discuss on learners’ answers (in-class time), and doing term project with the discussion (in-class time).  In addition to the flipped classroom activities,  the learner attitudes about this learning system suitability for the OOAD course were summarized from the survey answers.

The opinions about the features of the teaching videos were that 60% of learners (12 students) commented the lecture video is able to have more than one episode per week, while the rest of them (40 % of learners or 8 students) supposed that it should have one episode per week only.  When analyzing the length of the video, 85% of learners (17 students) thought that it should be no longer than 20 minutes or less than 10 minutes as detailed in Table 11.  In addition, a comparison between the lecture video features and the number of learners who watch through the end is demonstrated in Table 12.

According to Table 12, the lecture video features were analyzed by using the number of learners who watching the video through the end.  The opinion of the learners discovered that most of the learners would watch through intentionally, if the length of one lecture video is about 10 minutes. Meanwhile, learners would noticeably watch less through the end, if the length of lecture videos is more than 20 minutes.  The analysis results of the number of episodes and the length of the video per one episode are also consistent with the number of learners who watching the video until the end of each week.

Table 11.  The learner opinion about the appropriate length for one episode of lecture video.

The length for one episode of video	Learner (persons)	Ratio of all learners
less than 10 min.	8	40%
10 – 20 min.	9	45%
20 – 30 min.	2	10%
more than 30 min.	1	5%

Table 12.  Comparison between the lecture video features and the number of learners who watch through the end.

Week	Number of episodes	Total length of lecture videos (minutes)	Number of learners viewing through the end (persons)
1	–	–	–
2	1	5.49	6
3	2	49.55	5
4	1	22.15	12
5	2	16.32	18
6	1	17.29	15
7	Midterm Exam
8	–	–	–
9	2	27.25	8
10	1	10.15	17
11	1	11.46	14
12	–	–	–
13	1	12.05	18
14	–	–	–
15-16	Final Exam

To ensure the performance of the flipped classroom learning, there were seven survey questions about the impacts of the learning course activities in the flipped classroom learning for assessing learners’ opinions.

The effectiveness evaluation results are the average attitude scores from all learners’ opinions that is classified in five levels:

1 = A disagree        2 = B quite disagree    3 = C partly agree
4 = D quite agree    5 = E agree

Figure 11 shows the number of learners who commented on each survey question and the conclusions of the learners’ opinions that agree or disagree on each topic are displayed as follows:

• (1)Watching lecture videos before in-class time helps learners to understand the subjects more easily. Mean score = 4.25
• (2)Studying lesson documents before in-class time helps learners to understand the subjects more easily. Mean score = 3.90
• (3)Case study and group discussion in class make learners to understand the subjects deeply. Mean score = 4.00
• (4) In-class working group and discussing together make learners to understand the subjects deeply. Mean score = 3.85

Figure 11.  The learner opinions about the flipped classroom learning activities

• (5) Doing term project in class study time with discussion together helps learners to develop more quality of the term project. Mean score = 4.25
• (6) Answering questions with discussion on the learners’ answers help the learner to improve their knowledge. Mean score = 4.40
• (7) The flipped classroom learning helps to develop critical thinking, analysis and design skills more.  Mean score = 4.10

According to the bar graphs in Figure 11, most of the learners agree that study preparation before in-class time (out-of-class time) and in-class time activities in the flipped classroom learning make learners to understand the lesson contents easier and deeper, develop a better quality of term projects, and improve their own knowledge. In conclusion, most of the learners are agreed and quit agree that the flipped classroom learning helps to develop critical thinking skill, including analysis and designing skills better than the traditional lecture-based learning.

By the way, the opinion assessment of the learners about the object-oriented analysis and design workshop was evaluated by the questionnaire on five levels like effectiveness assessment.  The average scores of learners’ opinion for each topic are shown as follows:

• (1) You have learned how to analyze with critical thinking the overview of preserving the underwater environment. Mean score = 3.84
• (2) You have learned to identify the problem of coral damage. Mean score = 4.11
• (3) You understand the causes of problem in coral damage. Mean score = 4.26
• (4) You have gotten experiences in interviewing to gather requirements for problem solving about coral damage. Mean score = 3.84
• (5) You have gotten experiences in the analysis and design for the real-world problem. Mean score = 4.16
• (6) You have practiced on real world problem solving. Mean score = 4.05
• (7) You understand the subject contents more by joining this workshop activity. Mean score =3.95
• (8) You have trained on group working to create good quality works. Mean score =4.16

Referring to the average scores of each survey question of the workshop activity, most of the learners agreed that learners have learned how to analyze with critical thinking and identify the problem, including to find out and understand the causes of the problem.  Moreover, the learners had gotten experiences in gathering requirements, analysis and design, and solving the real-world case study.  Furthermore, most learners can understand the subjects deeply by doing workshop activities and practice themselves in working groups for creating good quality works.

4.4.  The Report of Problems or Obstacles

There were two different question types of data collection for problems and obstacles of the flipped classroom learning.  The first question with answer choices were asked for the most important problem or obstacle in the individual learners’ attitude.  And the second question with the short answers was independent writing for the question “What are your difficulties or obstacles in the flipped classroom learning?”.

According to the selected answers by learners, 55% of the learners (11 students) thought that the most important problem or obstacle in the flipped classroom learning system is habit and self-responsibility of individual learners.  Meanwhile, some learners (4 students) are not able to understand the content in the lecture video or the PowerPoint slide that are the most important problem for them in this learning system.  However, there are two students told that their most important problem is no time to study the preparation before class, and only one student who thought that unfriendly LMS, or no time to attend class, or conflicts on discussions or group works is the most important problem.

For the independent writing answers, there were five students left empty and three students wrote “no problem”. The comments of the rest students were grouped as follows:

• Sometimes there is no time for study preparation before in-class time, so it caused the less understanding and not learn much.
• The instructors upload some lecture videos before class study time less than one week, if we are busy, we cannot watch them in advance.
• Cannot understand all the content, and need to study more or ask more in class study time.
• Cannot get the answers to any
• Need time to adjust to this learning system because of the unfamiliar learning system.
• Viewing the lecture video is very troublesome because of the too small screen.
• Get bored with watching the lecture videos through the end because some lecture videos are too long.
• Need the instructors to upload and publish all lecture videos at the beginning course period because we can totally learn when we have a spare time.
• Problems with the internet connection lead to other problems.  For example, sometimes cannot display the lecture videos, cannot access to the LMS, and miss the class announcements

Other problems or obstacles of the flipped classroom learning in the instructor view are displayed as follows:

• The learners lack time to watch the video or study the lesson document before in-class time, and if there are no scores for self-study preparing activities, the learners may not concentrate on viewing the lecture video or learning class documents in advance.
• There is insufficient time for the instructors to prepare learning materials, such as lecture videos and PowerPoint slides for the whole subject contents of the course in advance, so sometimes they has been published before classes less than one week.
• There is the variant of the lesson contents so some lessons in some weeks cannot be understood by viewing the lecture videos with a length of shorter than 20 minutes.

4.5.  The Summary of Suitability of the Flipped Classroom Learning

The assessment results of the survey answers from the questionnaire were analyzed and the mean score of the appropriateness of the flipped classroom learning system was calculated to summarize the suitability of the flipped classroom learning for this course.  The rating given by the learners on five levels as below:

1 = A inappropriate                      2 = B quite inappropriate
3 = C moderately appropriate     4 = D quite appropriate
5 = E appropriate

The mean score for the appropriateness of the flipped classroom learning was 4.40.  The result means almost of the learners believed that the flipped classroom learning system has appropriated for this course, as detailed in Figure 12.

Figure 12.  The number of learners who answered the suitability of the flipped classroom learning.

According to Figure 12, there were 50% of learners (10 students) and 40% of learners (8 students) thought that the flipped classroom learning has been appropriate and has been quite appropriate to the OOAD course. Therefore, these results show that the flipped classroom learning is suitable for the OOAD course with the aggregation of 90 percent of learners.

Moreover, the reasons which support their attitudes are explained as listed below:

• There can be the diversity in the thinking process in this course to define the functionality of the system and design the system. Therefore, explaining, dialoguing, questioning and answering among learners, including between learners and instructors can help in learning as well. The discussion between learners and instructors helps them to have coincident ideas.
• The content of this course is quite many subjects. The flipped classroom learning helps learners to understand the lesson before class, have time to exchange ideas and work in groups more in class.
• This course focuses on the practical so learners have to think and practice by themselves before they consult or discuss with others. In addition, learners can learn, think, ask the question and find the answer by themselves to use their own ability for learning.
• This course is not only focused on academic content, but also emphasize on practical application. It should have activities in the classroom instead of teaching the content which learners can read from documents or find information from outside resources.  It also needs to discuss and practice to understand the contents because the subjects  cannot be understood by only reading.
• The flipped classroom learning system is very beneficial. The learners are enabled to practice critical thinking and self-study skills.  Watching the video before class makes an easier preparing and more intention. Discussion motivates the learners to be active, not sleepy in the classroom.
• If learners cannot attend classes or some contents may be difficult, they are able to study some subjects of the video viewing outside classes which may make more understanding. The flipped classroom learning system helps to save the time of teaching in lecture class duration, so the interesting subjects can be learned more.

Furthermore, the advantages of the flipped classroom learning that the learners gave opinions on their own side are listed below:

• Learners can study the lessons before class that make easier understanding when they come to learn in class. If they don’t understand the lessons, or have any questions, they can ask the instructor in the classroom.  Beside previous advantages, they pay more attention in class because they make understanding the contents before class.
• Learners have selves’ critical thinking before class, so they can remember and understand the contents more, don’t have to read much for exams.
• Learners have practiced critical thinking and done practical works under the care and guidance of the instructor.
• Learners can understand the lessons more and work better on the project because the traditional teaching system spent the whole period for lecturing, no chance to ask questions about the project, and no advisor to check whether their works are right or not.
• Learners can have the lecture video to watch after class and prepare for tests that make reviewing the lessons easily. Additionally, in the case of missing the class, learners still study from watching the video and gain some knowledge.
• Learners can discuss and exchange ideas in the class without wasting the teaching time.
• A non-stressful environment for learning in class.

In conclusion, although there are problems or obstacles in the flipped classroom learning because of limitation out-of-class time for learners preparing a lesson before class and for instructors providing teaching materials, including specifying in-class time activities, there are more benefits to use this new learning system in the OOAD course with many supported reasons by learners’ opinions.  Therefore, the flipped classroom learning has been suitable for the OOAD course in this research results.

5. Conclusions

Using of the flipped classroom learning in the OOAD course was studied to analyze the results of using this learning system.  This research focused on the behavior of learners, learners’ scores, and learner opinions to find out the appropriateness of the flipped classroom learning in this course.  The results of learners’ behavior analysis were found that most learners are interested in the flipped classroom learning activities and enjoy cooperating with the instructors.  In addition, the performance evaluation of the flipped classroom learning for the OOAD course can be concluded that this learning system has been better than the traditional learning system because the average total scores of grading and the average term project scores of learners in the current academic year with the flipped classroom learning were significantly higher than those in the previous academic year with the traditional learning.  Therefore, using the flipped classroom learning can increase the learning outcome of the learners and can be useful for learners in the learning intention and critical thinking skill development.

The summary of learners’ opinions on the flipped classroom learning with a workshop activity was uncovered that the majority of the learners concurred that the flipped classroom learning activities composed by viewing the lecture video and studying the text documents before in-class time, helps them to understand the class contents effortlessly.  Moreover, group discussions of case studies, and doing a term project under the guidance of the instructors in class study time can help understand class contents well and create the better term project.  Furthermore, the solutions to the assignments or the answers of exercises which are explained supporting reasons in the classroom make the learners develop their own knowledge as well.

In conclusion, most of the learners agreed that the flipped classroom learning system can help improve critical thinking skill and object-oriented analysis and design skill. The research results  also pointed out that the flipped classroom learning system helps the learners to pay more attention because of preparation before class.  Meanwhile, the critical thinking skills were cultivated by discussions, group activities in-class time, which can ask questions and get advice from the instructor immediately when in trouble, so it encourages learners to practice and apply the knowledge better. In addition, there were 90% of learners thought the flipped classroom learning is suitable for this course.  Therefore, it can be concluded that the flipped classroom learning is suitable for the OOAD course.

6. Suggestions

In spite of the fact that the flipped classroom learning is appropriate to the OOAD course, this learning system requires readiness in both of the instructors and the learners.  They should prepare for adjusting themselves to the active learning style with the useful learning management system. Instructors should be able to generate suitable learning materials with valuable knowledge, and to transfer that knowledge to learners.  Moreover, learners must have abilities in self-study learning for learning and understanding course content.  In addition, the learner need to concentrate on study preparation before in-class time and doing learning group in-class activities to get their knowledge and practice their skills.  Furthermore, both instructors and learners should be able to create a friendly atmosphere in the classroom.

In fact, according to problems and obstacles of the flipped classroom learning in views of both instructors and learners, this learning system cannot completely be flipped the classroom style.  If instructors have more time to create learning materials with the coordination between different courses in the same field, such as software development, this may increase the achievement of the flipped classroom learning because learners can arrange appropriate times for self-study preparation and cooperating group activities, while the instructor can share available learning resources in order to save time for searching and generating a few duplicate subjects in different courses.

Beside available time of learners and instructors, materials in the course which are prepared or provided by instructors cannot guarantee that all learners are able to study and understand by themselves.  The reasons are that the content of each lesson has the different difficulty, and there still are differences among self-learning skill, intelligence, ability and perseverance of each learner.  Additionally, the expertise of individual instructors to transfer knowledge in each lesson is different.  If there are quality and standard materials, may cause the flipped classroom learning system achievement even better.

Acknowledgment

This classroom action research got funded by Learning Innovation Center, Chulalongkorn University.

1. P. Pugsee, “Effects of using flipped classroom learning in object-oriented analysis and design course” in 10th International Conference on Ubi-media Computing and Workshops (Ubi-Media), Pattaya, Thailand, 2017. https://doi.org/10.1109/UMEDIA.2017.8074130
2.  C. Brinton, M. Chiang, “Social Learning Networks: A brief survey” in 48th Annual Conference on Information Science and Systems (CISS), Princeton, NJ, USA, 2014. https://doi.org/10.1109/CISS.2014. 6814139
3.  A. Wallace, “Social learning platforms and the flipped classroom” Int. J. Inf. Educ. Technol., 4(4), 293-296, 2014. https://doi.org/10.7763/ IJIET.2014.V4.416
4. T. Bristol, “Flipping the classroom” Teach. Learn. Nurs., 9(1), 43-46, 2014. https://doi.org/10.1016/j.teln.2013.11.002
5. Q. Dang, D.D. Gajski, “Bringing in-class online – A hybrid solution” in 4th Interdisciplinary Engineering Design Education Conference (IEDEC), Santa Clara, CA, USA, 2014. https://doi.org/10.1109/ IEDEC.2014.6784674
6.  A. Amresh, A.R. Carberry, J. Femiani, “Evaluating the effectiveness of flipped classrooms for teaching CS1” in IEEE Frontiers in Education Conference (FIE), Oklahoma City, OK, USA, 2013. https://doi.org/ 10.1109/FIE.2013.6684923
7.  J.M.M. Ferreira, “Flipped classrooms: From concept to reality using Google Apps” in 11th International Conference on Remote Engineering and Virtual Instrumentation (REV), Porto, Portugal, 2014. https://doi.org/ 10.1109/REV.2014.6784256
8.  M. L. Carrió-Pastor, H. Skorczynska, “Collaborative learning and communication technologies in teaching business english” Procedia Soc. Behav. Sci., 178, 32-37, 2015. https://doi.org/10.1016/j.sbspro. 2015.03.142
9.  K. T. Wissman, K. A. Rawson, “Why does collaborative retrieval improve memory? Enhanced relational and item-specific processing” J. Mem. Lang, 84, 75-87, 2015. https://doi.org/10.1016/j.jml. 2015.05.003
10.  C. P. Rosiene, J. A. Rosiene, “Flipping a programming course: The good, the bad, and the ugly” in IEEE Frontiers in Education Conference (FIE), El Paso, TX, USA, 2015. https://doi.org/10.1109/FIE.2015. 7344151
11.  G.S. Mason, T.R. Shuman, K.E. Cook, “Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course” IEEE Trans. Educ. 56(4), 430-435, 2013. https://doi.org/10.1109/TE.2013.2249066
12.  Y. Chen, L. Chen, “Effects of the flipped classroom model on student performance for vocational college students” in International Conference on Educational Innovation through Technology (EITT), Tainan, Taiwan, 2016. https://doi.org/10.1109/EITT.2016.30
13.  M. Khan, M. Ibrahim, “Flipped classroom in technology courses – impact on personal efficacy and perception based on learning style preferences” in IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, USA, 2017. https://doi.org/10.1109/ISECon.2017. 7910229
14.  A. Amiri, H. Ahrari; Z. A. Saffar; V. Akre, “The effects of classroom flip on the student learning experience: An investigative study in UAE classrooms” in International Conference on Current Trends in Information Technology (CTIT), Dubai, United Arab Emirates, 2013. https://doi.org/10.1109/CTIT.2013.6749480
15. CourseVille. “The Social Learning Management System of Chulalongkorn University”. https://www.mycourseville.com/

(Click Here)

(Click Here)