1. Introduction

This paper is an extension of work [1] originally presented at the 2017 International Conference on Platform Technology and Service (PlatCon).

The main goal of learning a target language is to facilitate students’ ability to communicate well. In countries that use English as a second language, the goal is to find effective English language learning in a limited period of time. While Korea is known as a country with a high level of education, many students in Korea who get high scores in reading and grammar lack the ability for authentic communication. Therefore, what could be a solution to solve this problem? Integrated English learning may be a solution.

 Integrated English learning is an effort to learn English in an English class mixed with other subjects. Learning English at the same time as learning a subject saves time and brings meaningful language learning to learners.

 Content-based integrated English education (CBIEE) refers to a method of teaching a foreign language whose main theme is extracted from a regular curriculum or curriculum area. Studies about CBIEE were conducted a few times [2, 3, 4] but analyzation of what part was effective for learning language was not conducted. Thus, it is needed to collect and analyze studies about CBIEE and propose directions to investigate via research on CBIEE that have been combined and sorted, and where the effects of CBIEE are found through meta-analysis.Studies collected and analyzed had some weaknesses. For example, it was hard to identify whether the studies were integrated by the contents or the themes, which is a collection of common points of contents; as such, it can be said that theme-based integration is content-based integration. Furthermore, some of the studies analyzed were difficult to classify as theme-based integration. Therefore, in order to complement weaknesses, the title was changed to “A meta-analysis of content and language integrated learning in English.”

2.       Background

2.1.  Content-based Integrated English education

The biggest positive effect of content learning through language instruction or language is that they can learn concepts and language in a natural environment; thus, learners can acquire language at the same time as general knowledge. For example, if they learn about other country’s environment in class, they will be able to learn about the environment and its related language naturally while communicating through it. The learners who have this meaningful communication are instinctively stimulated to acquire a high level of language. It is, therefore, highly effective for learners’ language learning to learn language and learn content knowledge naturally while participating in meaningful, purposeful life activities. The language learned through content has merits not only in terms of words, but also in integrating with higher dimensional thinking because learners’ thoughts are reflected in communication [5].

2.2.  Meta-analysis

Meta-analysis is an analytical method that synthesizes various articles and derives them as a single result. It draws a standard on papers of similar themes and derives the overall result. In short, it is a way to reanalyze the results so far to derive general results. In order to achieve the main goal of this paper, CBIEE related papers needed to be effectively analyzed and integrated through meta-analysis.

There are five steps for meta-study before analyzing statistics: 1) Problem specification, 2) Study identification, retrieval and eligibility criteria, 3) Study features coding book, 4) Coding, and 5) Analysis and interpretation [6].

3.       Method

3.1.  Subject of study

1) Primary target classification for meta-analysis

In this research studies related to CBIEE domestic and overseas papers about foreign language education were collected. Key words or phrases for searching studies were content-based, integrated learning, or theme-based. Among the research, the ones that matched the condition were studies that published between the years 2000 to 2015, with 15 were domestic papers, 7 were from overseas papers, and 14 were domestic master’s theses and doctoral dissertations. These 36 research papers were where meta-analysis was gleaned.

The number of studies for finding effect sizes in meta-analysis was 51 and the number of effect size for detail factors was 110 each, excluding cases that overlapped. Among these studies, studies that were directly related to searching the effect size for improving English skills and affective factors were chosen. The number of studies that suits the condition was 35 and the number of effect size is 110, excluding the cases which overlapped.

Table 1. Number of cases and number of effect sizes on detail case

Domain	Factor (Sub-domains)	Number of case	Detail case	Number of effect size
Cognitive	English skills	26	Listening	18
			Speaking	15
			Reading	14
			Writing	10
			Vocabulary	2
Affective	affective factors	25	Confidence	16
			Interest	22
			Attitude	13
Total (excluding overlapped)		35		110

2) Secondary target for meta-analysis

After classifying primary targets for meta-analysis, a Q-value test for homogeneity was taken and a random-effects model was chosen for meta-analysis. To raise the reliability and conduct a more valid meta-analysis, studies that were conducted only for elementary school students were only included, with studies that had different variables or a small number of cases were removed. Also, the studies that showed relatively large effect sizes without persuasive reasoning were dismissed. In the end, 13 research formats from domestic papers, 5 from overseas papers, 12 from master’s theses for a total of 30 studies with 107 effect sizes were chosen as secondary targets for meta-analysis. Most of the research tried to find the effect of improving language skills and affective factors through CBIEE and the results show that it is both effective in improving language skills and affective factors.

3.2.  Research design

1) Setting up variables of analysis

To conduct a meta-analysis related to CBIEE results, variables of analysis were explored and divided into two categories of variables: moderator variable and dependent variable. Moderator variables are integrated subjects, method of learning, and grade.

Table 2. Variables and coding value Conflict of Interest

Variable	Coding value
1. Integrated subject	(1) Knowledge-based subject (2) Skill-based subject (3) Both
2. Learning method	(1) Lecture (2) Group work (3) Storytelling (4) Game (5) Singing
3. Grade	(1) 4th (2) 5th (3) 6th
4. Dependent variable	(1) Language skills (2) Affective factors

Dependent variables are divided into language skills and affective factors. A meta-analysis to investigate the effects of the dependent variables on the detailed variables was conducted. The effects of language skills and affective factors were measured from the results of post-tests of experimental groups and control groups.

The term confidence, which is a detail factor in affective factors, included anxiety in learning foreign language and confidence in learning foreign language as the same factor, and the term attitude included attitude in foreign language lessons and attitude for learning a foreign language as the same factor. Similarly, interest included interest in foreign language and Interest in subject.

Table 3. Detail variable in dependent variable

Dependent variable	Detail variable
1. Language skills	(1) Listening (2) Speaking (3) Reading (4) Writing
2. Affective factors	(1) Confidence (2) Attitude (3) Interest

2) Instrument for meta-analysis

With the criteria of variables and detail variables mentioned earlier, coding is produced for meta-analysis in Excel. The coding of each quantitative data and variable is inserted and based on this CMA program is used to obtain the result from meta-analysis.

3) Data analysis and interpretation

Research for the meta-analysis of CBIEE was searched and collected for this study. From 2000 to 2015 a total of 30 research papers form annals and graduate-level theses were chosen. By the criteria, moderator and dependent variables are organized, coded, and then analysis performed.

The number of students participating in each study and the experiment method were different. In order to compensate for the errors in each of the different methods weight (w) was utilized. For each effect size, an average effect size for moderator was calculated. Thus, Hedge’s g was used to fix the standard difference in means (Cohen’s d). After the analysis, the results with some implications were interpreted.

4.       Analysis

4.1.  Coding for variables and adjusting weights

Based on the coding criteria, the collected studies were divided into certain categories, such as grade, learning method, and the classification of integrated subject. Thirty studies were represented as experimental and control groups, and individual effect sizes were presented. Each effect size was measured for detailed factors of language skill and affective factors.

Table 4. Coding for dependent variables and effect sizes

Study	k	Experimental  group			Controlled Group			ES
		N	m	sd	N	m	sd
K** (,2012	Listening	20	89.90	9.59	20	88.80	9.12	0.12
	Speaking	20	76.80	7.92	20	75.65	7.93	0.14
	Reading	20	49.70	6.81	20	48.40	6.31	0.19
	Writing	20	40.30	9.63	20	38.50	8.08	0.20
	Interest	20	78.70	9.14	20	77.65	8.37	0.12
	Confidence	20	59.60	5.50	20	57.80	5.39	0.32
L** (2015	Writing	11	29.55	7.54	11	19.18	10.30	1.11
K** (2015)	Listening	32	80.63	11.90	32	76.56	14.28	0.31
	Speaking	32	7.84	2.33	32	7.21	2.23	0.27
	Interest	32	3.41	0.24	32	3.25	0.11	0.84
	Confidence	32	3.28	0.09	32	3.17	0.05	1.55
K** (2014)	Reading	27	67.41	11.04	25	56.20	15.91	0.81
	Writing	27	20.56	5.60	25	17.00	6.92	0.56
	Interest	27	3.85	0.74	25	3.80	0.71	0.07
	Attitude	27	4.19	0.88	25	3.70	0.80	0.57
J** (2003)	Listening	39	3.46	0.68	39	3.43	0.86	0.04
	Speaking	39	3.77	0.96	39	3.38	1.21	0.35
	Interest	39	3.740	1.07	39	3.34	0.58	0.41
	Attitude	39	3.72	0.62	39	3.22	0.47	0.88
C** (2013)	Speaking	27	37.00	3.49	27	35.81	3.45	0.34
	Interest	27	4.49	0.24	27	4.38	0.18	0.47

(N=number of students, ES = effect size)

To compensate any error that was made from the different number of students and quality of study, weight (w) was added to offset error. As such, each effect size was measured again with weight. The effect sizes of each study with adjusted weight are shown in Table 5.

Table 5. Effect sizes with weight (w)

Study	k	Experimental Group	Controlled Group	ES	w
		N	N
K** (2012)	Listening	20	20	0.12	0.80
	Speaking	20	20	0.14	0.80
	Reading	20	20	0.19	0.80
	Writing	20	20	0.20	0.80
	Interest	20	20	0.12	1.12
	Confidence	20	20	0.32	1.10
L** (2015)	Writing	11	11	1.11	0.39
L** (2015)	Listening	32	32	0.31	1.25
	Speaking	32	32	0.27	1.62
	Interest	32	32	0.84	1.35
	Confidence	32	32	1.55	1.44
K** (2014)	Reading	27	25	0.81	0.95
	Writing	27	25	0.56	1.00
	Interest	27	25	0.07	1.38
	Attitude	27	25	0.57	2.09
C** (2003)	Listening	39	39	0.04	1.54
	Speaking	39	39	0.35	1.51
	Interest	39	39	0.41	1.95
	Attitude	39	39	0.88	1.45
C** (2013)	Speaking	27	27	0.34	1.06
	Interest	27	27	0.47	1.53

 

4.2.  Comparing effect of factors in CBIEE

• Overall CBIEE effect on English skills versus affective factors

Overall average CBIEE effect sizes on language skills and affective factors are 0.68 and 0.48, respectively where the difference is statistically significant. CBIEE has larger effect size on language skills than affective factors which means CBIEE has more effects on improving listening, speaking, reading, and writing than improving students’ interests, confidence, and attitude in learning a foreign language.

Table 6. Overall CBIEE effects on language skills versus affective factors

	n	k	Q	p	ES	Z	95 percent CIs
							Lower	Upper
Cog	24	57	187.67	0.05	0.68	24.33	0.62	0.73
Aff	24	50	227.67	0.05	0.48	14.56	0.41	0.54

( Cog = Cognitive skills, Aff = Affective factors, n = number of studies, k = number of detail case, Q = Q-value, p = p-value, Z = Z-value )

• CBIEE effects on different grades

Table 7. CBIEE effects on different grades

G	Var	n	k	Q	p	ES	Z	95 percent CIs
								Lower	Upper
4th	Cog	8	26	124.43	0.05	0.80	22.59	0.73	0.80
	Aff	9	20	87.40	0.05	0.40	8.10	0.31	0.50
5th	Cog	6	16	11.90	0.05	0.38	22.59	0.26	0.51
	Aff	5	10	9.22	0.05	0.53	6.57	0.37	0.68
6th	Cog	6	12	7.96	0.05	0.67	9.27	0.53	0.81
	Aff	7	15	121.46	0.05	0.52	8.90	0.40	0.63

(G = Grades)

Students in 4th grade had an average effect size on language skills (0.80) which were larger than that on affective factors (0.40), but the difference was not statistically significant. Conversely, 5th and 6th grade students (0.53, 0.52) had larger effect size on affective factors compared to 4th grade learners (0.40). As the grade progressed, the gap between minimum and maximum effect sizes grew, which means the effect of CBIEE matters on the choice of content and a teacher’s method of teaching.

• CBIEE effects on language skills

Table 8. CBIEE effects on language skills

Skills	n	k	Q	p	ES	Z	95 percent CIs
							Lower	Upper
Listening	18	18	63.73	0.05	0.66	12.86	0.56	0.75
Speaking	15	15	55.75	0.05	0.65	11.82	0.54	0.75
Reading	14	14	43.50	0.05	0.70	12.23	0.59	0.81
Writing	10	10	20.03	0.05	0.76	11.91	0.64	0.89

There was no difference in effect sizes among language skills. CBIEE’s effect on students’ reading and writing skills was minutely larger than that of listening and speaking skills. Since most activities take place within similar content or themes, they focus on output activities such as reading and writing.

This results show that effect sizes in improving written language is slightly larger than spoken language, although the difference was not statistically significant.

4) CBIEE effects on affective factors

Table 9. CBIEE effects on affective factors

	n	k	Q	p	ES	Z	95 percent CIs
							Lower	Upper
Confidence	16	16	91.33	0.05	0.44	7.84	0.33	0.55
Attitude	12	12	60.84	0.05	0.54	8.06	0.41	0.67
Interest	22	22	74.21	0.05	0.47	9.32	0.37	0.57

There was no difference in effect sizes among affective factors. Improving attitude was the largest effect (0.54) where CBIEE helped to give a positive influence on attitude by removing cognitive burden while learning a foreign language.

By analyzing the statistics, CBIEE proved to have positive effects on affective factor development; especially, CBIEE helped students to raise their concentration better by eliminating cognitive pressure. When integrating different subjects, teachers should know that integrating can raise students’ affective factors when students face difficult tasks.

5) Knowledge-based subjects vs skill-based subjects

Table 10. Knowledge-based versus skill-based integration

Subjects	n	k	Q	p	ES	Z	95 percent CIs
							Lower	Upper
Skill-based	13	45	184.79	0.000	0.67	10.44	0.55	0.80
Knowledge-based	9	30	112.31	0.000	0.52	6.01	0.35	0.68
All  subjects	8	38	99.94	0.000	0.41	6.32	0.29	0.54

Integrating with knowledge-based subjects (0.67) shows larger effect than that of skill-based subjects (0.52), but the difference was not statistically significant. However, integrating with knowledge-based subjects (0.67) is more effective than integrating all subjects (0.41), which the difference is statistically significant. Integrating with only knowledge-based subjects (i.e., mathematics, science, and social studies) is the most effective way for content integration.

6) Teaching method in CBIEE

Table 11. Effect sizes by different teaching methods in CBIEE

Method	n	k	Q	p	ES	Z	95 percent CIs
							Lower	Upper
Lecture	11	45	292.96	0.000	0.48	8.22	0.38	0.78
Group work	5	15	3.37	0.001	0.49	3.37	0.20	0.78
Game	4	16	47.42	0.000	0.69	5.86	0.46	0.92
Singing	2	4	0.99	0.002	0.34	3.10	0.13	0.56
Storytelling	8	30	39.76	0.000	0.42	7.91	0.31	0.52

Using games (0.69) shows the largest effect compared to the rest of the learning methods, although the difference was not statistically significant. Students can enjoy their learning while playing games and they are able to release the stress of learning foreign language. It is, however, challenging to see the effect of using singing in CBIEE, because there are only few cases.

7) Overall results of meta-analysis

CBIEE shows larger effects on improving language skills than improving affective factors. We think that most of the experiments are focused on English courses, not regular courses, and these results seem to have resulted. There are no effect size differences among different methods. Integrating specifically with knowledge-based subjects is more effective than integrating with all subjects in improving language skills, which means in order to develop the English language skills, it is important to emphasize the importance of the mixture of subjects rather than the way of teaching.

Figure 1 Overall/Meta-analytic effects of CBIEE across sub-domains.

5. Conclusions

It was explored in this study the effect of CBIEE by meta-analysis. Through the meta-analysis, it was found that CBIEE is more effective in improving skills than affective factors and it depicts different effect size across sub-domains.

Compared to a previous meta-study of integrated language learning [7], different results were found in this study. After integrated learning, two studies have different results on the improved factors, such as language skills versus affective factors, and spoken language versus written language. Two studies also have opposite effects on integrating knowledge-based subjects versus skill-based subjects.

Due to this it means that integrating with other courses can have different effects depending on whether the teacher focuses on content or language when the language is taught. This meta-analysis displays not only the findings of the effects of CBIEE itself, but also it may be gleaned that different kind of integration with language, for example, task-based or experience-based integration could give different effects to language learners.

In future studies, it will be necessary for the researchers and teachers to pay much attention and effort to examine the effects of applying various types of integrated foreign language classes and to find appropriate education methods for each country’s educational environment.

1. Jong-Keol Kim & Jeong-ryeol Kim, “A Meta-analysis of Theme-based Integrated English Education Effects on English Skills,” in proceedings of the 2017 International Conference on Platform Technology and Service (PlatCon), Busan, Korea, Feb. 2017.
2. Jeong-ryeol Kim & Yeo-rim Ko, “Effects of content-based language teaching in elementary English classes,” Journal of the Korea English Education Society, vol. 7, no. 2, pp.131-152, 2008.
3. Tae-Hyeok Moon, A study of syllabus design for integrated English education in elementary school (Unpublished doctoral dissertation), Korea National University of Education, Chungbuk., 2010.
4. Jun-Eon Park, “Applicability of immersion program in Korean English education,” English Teaching, vol. 53, no. 4, pp. 247-26, 1998.
5. Mimi Met, “Learning language through content: Learning through language,” Foreign Language Annals, vol. 24 no.4, pp.281-295, 1991.
6. Seong-Sam Oh: Theory and practice of meta-analysis. Seoul: Keonguk University Press, 2002.
7. Tae-Hwan Kwon & Jeong-ryeol Kim, “Meta-analysis of the effects of content-integrated English education in knowledge-based subjects and skill-based subjects in elementary school,” Journal of Learner-Centered Curriculum and Instruction, vol. 12, no. 2, pp. 29-49, 2012.

(Click to view)

(Click to view)