THE EFFECTIVENESS OF ENGLISH-SUBTITLED VIDEOS IN ENHANCING SPEAKING, LISTENING AND VOCABULARY SKILLS: A META-ANALYSIS

Languages


INTRODUCTION
In education, learning English is a crucial issue (Ebrahimi et al., 2018).Effective communication in both a native language and a global language like English is essential at the international level (Liando et al., 2018).Speaking is one of the four language skills that include using oral communication to convey ideas, feelings and information (Sari & Apriani, 2020).Kinasih & Olivia (2022) state that public speaking is a language skill that is valuable to develop because it is crucial to communication.Speaking is an interactive process of constructing meaning that involves producing, receiving and processing information (Khadidja & Manar, 2018).
In addition to speaking skills, listening skills are important in language learning.According to Alabsi (2020), listening comprehension is not only about hearing what is being said but also involves understanding and making sense of the spoken language.It is impossible to overstate how crucial listening abilities are for learning a foreign language (Fernando et al., 2023).Furthermore, the success of L2 acquisition depends on listening to L2 input (Alzamil, 2022).However, speaking and listening skills require mastery of vocabulary (Ratnaningsih & Gumiandari, 2022).Vocabulary is also important for people to communicate and express their ideas through language (Alharthi, 2020).To learn English effectively, students must understand that it covers macro skills, where two of them are speaking and listening, which is impossible to learn without vocabulary since it significantly impacts the skills mentioned above (Rosales, 2019).
This research concerns an English-subtitled video on learning speaking, listening and vocabulary.Many EFL learners use English videos as a way to access real and interesting language input.They can enhance their speaking, listening, and vocabulary skills by listening to different voices, situations, and words.However, the use of English videos for language learning is influenced by various factors, such as the kind and quality of the videos, the level and interest of the learners, and the presence and use of subtitles.The use of subtitle videos in language learning has several benefits, including helping students learn how to pronounce new words , making it easier for them to follow the storyline of the video (Hwang et al., 2019), improving their word recognition abilities (Altun & Hussein, 2022), reinforcing their understanding of English expressions that are context-bound, aiding in the acquisition of new vocabulary and improving their concentration in the lines that follow (Fikri et al., 2021).
The English subtitles in videos, such as movies, can help students expand their vocabulary and improve their speaking abilities by identifying the words being said by the actors (Megawati & Nuroh, 2018;Miranda & Wahyudin, 2023;Rahmawanti et al., 2021).Additionally, the combination of aural, visual, and textual mediums makes watching Englishsubtitled video one of the richest ways to present authentic input (Napikul et al., 2018).Moreover Okar & Shahidy (2019) added that the most effective language-learning method is watching content with subtitles.As a part of the learning process, video subtitles can allow students to understand concepts more deeply and improve their language comprehension while also honing their listening skills (Timilsina, 2022).
In addition, learning media for speaking, listening and vocabulary based on Englishsubtitled videos have also been done at Senior High School level (Abidin & Ngadiman, 2021;Matondang & Ashari, 2019;Rawanita, 2018;Sembel et al., 2023;Topkaraoglu, 2018;Wong et al., 2020;Wu & Yang, 2022).Wu & Yang, (2022) have also conducted research at Senior High School level which discusses the effectiveness of textually enhanced captions on Chinese high school EFL learners' incidental vocabulary learning with a total of 133 students and obtained an F score of 1,244.Finally, the use of English-subtitled video-based speaking, listening and vocabulary learning has also been widely conducted at college level (Ahmed, 2022;Andi & Darvishi, 2019;Ashcroft et al., 2018;Dewi, 2023;Fitriyeni, 2020;Jao et al., 2022;Kim, 2020;Ridha et al., 2022;Tran, 2022).Fitriyeni (2020) has conducted research at college level, which discusses improving students' speaking skills using video with a total of 100 students and obtained the result that there is a significant impact of the use of the video in teaching speaking with a t-count of 14,766.
Recent studies have also explored the impact of different types of English videos on EFL learners' language skills.For example, some studies have compared the effects of using TED talks, YouTube videos, and news clips on learners' speaking fluency, accuracy, and complexity .Other studies have investigated the influence of using documentaries, movies, and cartoons on learners' listening comprehension, strategy use, and motivation .Moreover, some studies have examined the benefits of using academic lectures, podcasts, and songs on learners' vocabulary acquisition, retention, and production.These studies have provided valuable insights into the advantages and challenges of using English videos for EFL learners.However, they have also revealed some gaps and limitations in the current state of knowledge in the field.One of the major gaps is the lack of research on the relative impact of using English subtitles on different language skills.Most of the previous studies have focused on comparing different types of subtitles, such as native, foreign, or no subtitles, or different genres of videos, such as documentaries, movies, or cartoons, but not on examining how subtitles affect speaking, listening, and vocabulary skills separately and comparatively.
Therefore, this study aims to fill this gap by investigating the effect of using Englishsubtitled videos on EFL learners' speaking, listening, and vocabulary skills.The novelty of this study lies in its comprehensive and comparative approach to examining the role of subtitles in language learning.Unlike previous studies that have only looked at one or two language skills or one type of subtitles, this study will measure and compare the impact of subtitles on three language skills: speaking, listening, and vocabulary.Furthermore, this study will also analyze the effect of subtitles across different years and sample sizes, to explore the possible trends and variations in the results.This study is expected to contribute to the field of EFL learning and teaching by providing empirical evidence and practical implications for using English-subtitled videos as a language learning resource.The findings of this study could help researchers further explore the role of subtitles in language acquisition and development.

RESEARCH METHOD
This research uses a meta-analysis method.Meta-analysis is a study that researchers do to gather, summarize, review, and analyze data from several prior research findings (Tusaadia et al., 2022).This study is to determine the effectiveness of English-subtitled videos as a learning tool for enhancing speaking, listening, and vocabulary skills.Therefore, the goal is to find out which skill among speaking, listening and vocabulary is most affected by learning with English-subtitled video.Moreover, this study also investigates the year and the number of participants that had the highest impact from using English-subtitled video.To achieve this goal, it is necessary to collect data from the inclusion, selection and exclusion criteria that are compiled in the coding sheet in Microsoft Excel.
The inclusion, selection and exclusion criteria gathered in a Microsoft Excel coding sheet must match articles searched across multiple databases to support statistical analysis of meta-analyses that continue to calculate Effect Size (ES) and Standard Error (SE) values.The studies eligible for a meta-analysis are chosen based on two criteria: inclusion criteria (eligibility criteria) refer to research characteristics relating to population issues based on education level, skill and year, related to "English-subtitled Video-based Speaking, Listening and Vocabulary".Regarding the eligibility criteria, which speak to the characteristics of publications, the year (studies published between 2018-2023) and the types of publications (articles, journals and thesis) are all considered.In contrast, the exclusion criteria are used to find articles that can be used for statistical analysis of articles obtained based on inclusion criteria in meta-analyses.Exclusion criteria include research data as follows: percentage of each error indicator, Effect Size value (ES), Standard Error (SE) and number of sample (N) (W.Sari et al., 2023).
The research procedure employed in this study encompasses several sequential steps aimed at systematically aggregating and analyzing relevant scholarly articles.Initially, an exhaustive search for articles meeting the specified criteria is conducted across prominent databases, including Google Scholar, the Directory of Open Access Journals (DOAJ), and Scopus.The selection criteria are meticulously applied to curate a comprehensive pool of articles pertinent to the research objectives.Following the article selection process, the subsequent phase involves the systematic coding and tabulation of pertinent information extracted from the selected articles.This includes essential details such as the year of publication, author's name, academic level, class, skill under consideration, N-value, F-count, t-count, and r-count.Microsoft Excel serves as the designated platform for this data organization, facilitating a structured and systematic approach to information management.A pivotal transformation of F and t values into r values follows, accompanied by the computation of Effect Size (ES) and Standard Error (SE) values.This step is crucial for harmonizing the diverse statistical measures and ensuring a unified framework for subsequent analyses.The utilization of JASP software for simulations and data analysis constitutes the subsequent phase, leveraging computational tools for robust statistical insights and simulations.The final step involves a comprehensive analysis of the findings derived from the assembled articles, synthesizing the data into a coherent narrative that culminates in the metaanalysis's research findings.The analytical process involves a meticulous examination of the synthesized data, drawing connections, identifying patterns, and deriving meaningful conclusions that contribute to the broader understanding of the research topic.This systematic research procedure, outlined in its entirety, reflects a methodologically rigorous and transparent approach to conducting a meta-analysis, ensuring the reliability and validity of the study's outcomes.Specifically, when the effect size value falls within the range of ≤ 0.15, it is designated as constituting a negligible effect.A subsequent band, ranging from 0.15 to 0.40, is characterized as a small effect, while the category of moderate effect encompasses effect size values ranging from 0.40 to 0.75.Moving further along the spectrum, effect size values between 0.75 and 1.10 are deemed indicative of a high level of influence.Lastly, the very high effect category encapsulates effect size values falling within the range of 1.10 to 1.45.This systematic delineation of influence categories, grounded in the nuanced gradations of effect size values, provides a structured framework for researchers and practitioners to interpret and communicate the magnitude of effects observed in empirical studies.By aligning the effect size with distinct influence categories, researchers can convey the substantive significance of their findings, facilitating a more nuanced and standardized interpretation of the observed effects within the context of their study.This methodological approach, encapsulated in Table 1, underscores the importance of precision and consistency in the characterization of effect size magnitudes and their associated influence levels.

RESULT AND DISCUSSION Data Selection Results
The results of the data set search yielded 142 data.According to the inclusion and exclusion criteria, 87 data and data that did not fit the inclusion and exclusion criteria were 55.The data collected in this study are the value of the Fisher test (F), student test (t), correlation test (r), and the amount of research data (N).Further data processing or analysis can also be carried out during the learning method under certain conditions.From the data collected, there are F, t and r values.These values must be converted into ES and SE values.The conversion results can be seen in Figure 1 and Figure 2 below.Figure 3, which has 87 relevant data points, is separated into 15 data for negligible effect category, where each data consists of 5 data each for speaking, listening and vocabulary.Furthermore, the data for the small effect category consists of 32 data and each of them is 10 data for speaking, then 9 data for listening and finally 13 data for vocabulary.For the moderate effect category there are a total of 18 data with data for speaking as much as 7, for listening is 3, and the last, 8 data for vocabulary.For the moderate effect category, there are only 6 data consisting of 3 data each for listening and vocabulary.Furthermore, for the high influence category, there are 3 data for speaking, 4 data for listening and finally there is only 1 data for vocabulary, so there are a total of 8 data for this category.Finally, there is a high influence effect category with a total of 8 data.Each of these data, data for speaking with a high influence effect category is 3 data, while for listening there are 2 data, and finally, for vocabulary there are 3 data with a high influence effect category.Subsequently, the authors employed a publication bias and a hypothesis test on the gathered data.The coefficient table displays the z and p values in a meta-analysis conducted using JASP software, as seen during the conclusion.The following is the hypothesis: Hypothesis 1: Using English-subtitled effectively enhances students' speaking, listening and vocabulary learning.
Hypothesis 2: There is no publication bias from the data used in the research.

Hypothesis Test
A heterogeneity test was conducted in the first stage to see whether the data used fixed or random effects.The results are as follows in Table 2. From the JASP results obtained through speaking, listening and vocabulary learning media utilizing English-subtitled videos, it can be seen that the data is heterogeneous with a Q=1450.407and a value of p<0.001.Furthermore, consider the estimation of Englishsubtitled video in learning speaking, listening and vocabulary according to Table 3.The following are the estimation results of Englis-subtitled video in speaking, listening and vocabulary learning according to Table 3.In Table 3 about the coefficients, the estimate of the intercept is 0.59, which means that the average effect size for speaking, listening and vocabulary based on English-subtitled video is 0.59.The standard error of the estimate is 0.06, which measures the variability of the estimate.The z-score is 9.883, which is the ratio of the estimate to the standard error.The pvalue is < .001,this means that the null hypothesis that the effect size is equal to 0 can be rejected at the 5% significance level, and the alternative hypothesis that the effect size is greater than 0 can be accepted.This indicates that speaking, listening and vocabulary based on English-subtitled videos has a positive and significant impact on the learning outcomes for students, accounting for 59% of the variance.The remaining 41% of the variance can be attributed to other factors, such as individual differences, motivation, prior knowledge, etc.To The Effectiveness of English ……….. Languages and Language Teaching, January 2024. Vol.12, No.1 | 63 assess the possibility of publication bias, a rank correlation test and a regression test were conducted, and the results are shown in Table 4.There was also a test for publication bias.This test is run to determine whether the data gathered can be taken as a representative of the general population.The rank correlation and regression test results can be used to evaluate this test.The output shown in Table 4 below is derived from JASP results.In Table 4 for rank correlation and regression, Kendall's value of the English-subtitled video learning media can be seen as 0.262, indicating a significant correlation coefficient between effect size and variance.The z-value shows the magnitude of the regression coefficient is 2.604, and the p-value of 0.009 is greater than the value of 0.001, indicating that the second hypothesis is accepted.In other words, there is no publication bias identified.Lastly, the Rosenthal variable shows how many studies with an average effect size equal to 0 should be added to the research sample so that the research results are free from publication bias.In addition, this can also be proven by looking at the funnel plot in Figure 4 below.The findings of the meta-analysis examining the impact of utilizing English-subtitled videos as a learning medium for speaking, listening, and vocabulary skills are visually presented in Figure 4. Notably, the graphical representation of the publication in Figure 4 demonstrates an absence of evidence indicative of publication bias within the analyzed studies.Publication bias is a phenomenon characterized by a tendency for studies with positive or statistically significant results to be more readily published, potentially distorting the overall estimate of effect size.The funnel plot depicted in Figure 4 serves as a diagnostic tool, juxtaposing effect sizes of individual studies against their standard errors, inversely proportional to sample sizes.In the absence of publication bias, a symmetrical inverted funnel shape is anticipated, wherein larger studies cluster proximate to the mean effect size, and smaller studies scatter symmetrically on both sides.The openness of the funnel plot, denoted by the absence of missing studies represented by open circles, serves as a visual indicator of the meta-analysis's resilience against publication bias.This visual examination, reinforced by statistical analyses, fortifies the credibility of the meta-analytic results.Furthermore, the researcher employed a moderator variable test to gauge the influence of several factors, including skills, publication years, and the number of participants.These moderator analyses, documented in the table below, furnish a comprehensive understanding of the nuanced impact of these variables on the meta-analytic outcomes.The incorporation of such moderator analyses not only refines the interpretation of findings but also enhances the study's methodological rigor by accounting for potential confounding factors.

The Influence of English-Subtitled Video based on Skills
Moderator variable analysis is required to ascertain how much an English-subtitled video influences speaking, listening, and vocabulary skills.Table 5 indicates that the effect of using Englis-subtitled videos as a medium for learning speaking, listening and vocabulary is relatively moderate.On speaking ability with an estimate of 0.643 (moderate effect), while on listening ability of 0.586 (moderate effect) and finally on vocabulary ability of 0.549 (moderate effect).This result shows that Englishsubtitled video-based learning media has the same effect on speaking, listening and vocabulary skills.The use of English-subtitled videos has a moderate effect on improving the three skills of speaking, listening and vocabulary.This means that students who watch English-subtitled videos can enhance their language proficiency in these areas, but not significantly.The Q-test values indicate that there is a high heterogeneity among the studies included in the meta-analysis, which suggests that there are other factors that may influence the outcomes, such as the quality of the videos, the duration of the intervention, the level of the learners, and the assessment methods.The RE model values show the confidence intervals of the effect sizes, which are relatively wide, indicating a low precision of the estimates.The researchers will next present funnel plots for each skill, ranging from speaking, listening and vocabulary, to show that the results shown in Table 5 are free from publication bias.Therefore, funnel plots for speaking, listening and vocabulary will be displayed in Figure 5, 6 and 7 below.Based on Figure 5, 6 and 7, from the funnel plots, we can see that no publication bias is identified because all circles are closed.Next, from the forest plot image of speaking, a summary effect value is 64%, which is the highest influence compared to listening, with an effect value of 59% and vocabulary, with an effect value of 55%.Subsequently, as displayed Table 5, the effect size of speaking is 0.64, with a minimum value of 0.41 and maximum value of 0.87.Next, the effect size value of listening is 0.59, with a minimum value of 0.38 and a maximum of 0.79.Finally, the effect size value of vocabulary is 0.55, with a minimum value of 0.37 and maximum of 0.73.Overall, it can be said that the application of Englishsubtitled videos has the highest impact on speaking compared to listening and vocabulary.This data is supported by research conducted (Wulandari, 2019) with an effect size of 1.27, (Hadi, 2019) with an effect size of 1.90, (Jao et al., 2022) with an effect size of 1.47, (Salem, 2019) with an effect size of 1.33, (Bedaiwy, 2022) with an effect size of 2.02, and finally, research conducted by (Alabsi, 2020) with an effect size of 1.19.

The Influence of Englis-Subtitled Video on Speaking, Listening and Vocabulary based on Moderator's Variable
At this point, the author analyzes the data to ascertain how the English-subtitled video influences learning outcomes when viewed based on number of participants and publication The Effectiveness of English ……….. Languages and Language Teaching, January 2024. Vol.12, No.1 | 66 years.The JASP output is shown in the following table according to number of participants and publication years.Based on Table 6, in publication years, starting from 2018-2023, English-subtitled videos as learning media for speaking, listening and vocabulary have a significant impact in 2022 with an estimate of 0.771 with a minimum value of 0.43 and maximum value of 1.11.While in 2019, 2020, 2021, and 2023 have the same impact which is in the moderate category, with an estimate in 2019 of 0.568, in 2020 of 0.441, in 2021 of 0.734 and finally in 2023 with an estimate of 0.531.Furthermore, for 2018, with a small effect with an estimate of 0.387.Finally, in the category of the number of participants, English-subtitled videos are very influential, with the number of participants less than 40 people with an estimate of 0.886 and, a minimum value of 0.62 and a maximum value of 1.11.For the number of participants ranging from 41-80, the effect size is 0.53, which is in the moderate category with a minimum value of 0.36 and the maximum value of 0.71.Furthermore, for the number of participants, more than 80 are included in the small effect with an effect size value of 0.38, a minimum value of 0.24, and the maximum value of 0.52.Based on the above statements, it can be concluded that the use of English-subtitled video is more effectively used with the number of participants less than 40.This data is proved by the research conducted (Ridha et al., 2022) with an effect size of 1.985, which considered in high category, and resulted that Englishsubtitled video make studying more interesting, and (Khadawardi, 2022) with an effect size of 1,838, also considered in high category, which results included that watching Englishsubtitled video had an effective learning result and a positive impact on students' vocabulary.

CONCLUSION
This study aims to measure and compare the impact of subtitles on three language skills: speaking, listening, and vocabulary.In light of the findings from this study, Englishsubtitled videos prove to be a moderately effective medium for enhancing speaking, listening, and vocabulary skills, accounting for 59% of the observed impact.However, a closer examination of individual skills, as outlined in Table 6, highlights that the use of Englishsubtitled videos has the most substantial positive effect on speaking proficiency, with an estimated effect size of 0.643.This effect is notably higher than the impact on listening and vocabulary skills.When assessing the influence of English-subtitled videos over different publication years, it becomes evident that the year 2022 emerges as particularly influential, falling into the high impact category with an estimated effect size of 0.771.Conversely, the lowest influence is observed in studies published in 2018, with an effect size estimate of 0.387.Furthermore, when considering the number of participants, it is apparent that studies with fewer than 40 participants exhibit the most significant impact, with an estimated effect size of 0.886.This suggests that English-subtitled videos are especially effective in improving speaking skills when employed with smaller sample sizes.
The study's findings hold valuable implications for educators and curriculum developers seeking to enhance language learning outcomes.The demonstrated effectiveness of English-subtitled videos, particularly in improving speaking skills, highlights the potential for integrating such multimedia resources into language instruction.As the impact is most pronounced with smaller groups, educators can consider tailoring their approaches to capitalize on this medium, creating more engaging and interactive learning experiences.Moreover, the study underscores the need for up-to-date instructional materials, with the year 2022 showcasing significant impact, emphasizing the importance of staying current with evolving language teaching methodologies.These insights can guide educators and curriculum developers in making informed decisions about the implementation of multimedia resources in language education.

Figure 1 .
Figure 1.Results of ES Values

Figure 3 .
Figure 3. Level of Influence Category

Figure
Figure 4. Funnel Plot

The
Figure 5. Funnel Plot for Speaking Figure 6.Funnel Plot for Listening

Table 1
serves as a determinant for categorizing the level of influence based on Effect Size (ES).The application of this categorization system involves ascertaining the magnitude of the effect size value and subsequently attributing it to one of several predefined influence categories.According to the specifications outlined in Table1, a nuanced classification is established to demarcate the extent of influence associated with varying effect size values.

Table 5
The Influence of English-Subtitled Video based on Skills.

Table 6
The Influence of English-Subtitled Video Based on Publication Years and Participants