Email this article 
https://orcid.org/0000-0003-4395-5221
Think-Aloud Protocol as a Method for Exploring and Understanding the Argumentation Skills of Pre-Service Science Teachers
Abstract
Argumentation is a core component of critical thinking, one of the essential 21st-century skills. However, exploring this skill remains a challenge, often due to limitations in the available assessment methods. This study aims to offer a solution by utilizing the think-aloud protocol to explore and better understand the argumentation skills of Pre-Service Science Teachers (PSTs). A total of 30 PSTs from a state university in Indonesia participated in the research. Data collection and analysis through the think-aloud protocol were conducted in several stages: procedural briefing, implementation, coding and video analysis, and data interpretation. The findings revealed that the think-aloud protocol was effective in providing deeper insights into argumentation skills and yielded more detailed and accurate data across five key components: claims, evidence, reasoning, counterarguments, and rebuttals. On average, the PSTs demonstrated beginner-level argumentation skills, with the strongest performance in the claim component and weaker results in supporting aspects. Specifically, 5 participants were categorized as proficient, 9 as advanced, 6 as intermediate, and 10 as beginners. These results highlight the need for more effective learning strategies that provide PSTs with greater opportunities to practice constructing and critiquing arguments.
Keywords
References
Akbayrak, K. and Namdar, B. (2019), “An Argumentation Activity for Third-Grade Students: Objects in the Plates”, Science Activities, Vol. 56 No. 1, pp. 1–10, doi: 10.1080/00368121.2019.1600464.
Amini Farsani, M., Abdollahzadeh, E. and Beikmohammadi, M. (2019), “Self-regulated Learning, Metacognitive Awareness, and Argumentative writing”, Writing and Pedagogy, Vol. 11 No. 2, pp. 195–222, doi: 10.1558/wap.37051.
Anwar, N.P. and Ali, M.A. (2020), “The effect of socio-scientific issue (SSI) based discussion: A student-centred approach to the teaching of argumentation”, Scholarship of Teaching and Learning in the South, Vol. 4 No. 2, pp. 35–62, doi: 10.36615/sotls.v4i2.76.
Bai, B. (2018), “Understanding primary school students’ use of self-regulated writing strategies through think-aloud protocols”, System, Vol. 78, pp. 15–26, doi: 10.1016/j.system.2018.07.003.
Baumtrog, M.D. (2021), “Designing Critical Questions for Argumentation Schemes”, Argumentation, Vol. 35 No. 4, pp. 629–643, doi: 10.1007/s10503-021-09549-z.
Capkinoglu, E., Yilmaz, S. and Leblebicioglu, G. (2020), “Quality of Argumentation by Seventh‐Graders in Local Socioscientific Issues”, Journal of Research in Science Teaching, Vol. 57 No. 6, pp. 827–855, doi: 10.1002/tea.21609.
Chamot, A. (2001), “The Role of Learning Strategies in Second Language Acquisition”, in Breen, M.P. (Ed.), Learner Contribution to Language Learning: New Directions in Research, Harlow: Longman, Harlow, pp. 27–54.
Chen, X., Zhao, H., Jin, H. and Li, Y. (2024), “Exploring College Students’ Depth and Processing Patterns of Critical Thinking Skills and Their Perception in Argument Map (AM)-Supported Online Group Debate Activities”, Thinking Skills and Creativity, Vol. 51, p. 101467, doi: 10.1016/j.tsc.2024.101467.
Correnti, R., Matsumura, L.C., Wang, E.L., Litman, D. and Zhang, H. (2022), “Building a validity argument for an automated writing evaluation system (eRevise) as a formative assessment”, Computers and Education Open, Vol. 3, p. 100084, doi: 10.1016/j.caeo.2022.100084.
Cowan, J. (2019), “The potential of cognitive think-aloud protocols for educational action-research”, Active Learning in Higher Education, Vol. 20 No. 3, doi: 10.1177/1469787417735614.
Creswell, J.W. (2009), Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (Third Edition), USA: United States of America, United States of America.
Du, H. and List, A. (2021), “Evidence Use in Argument Writing Based on Multiple Texts”, Reading Research Quarterly, Vol. 56 No. 4, pp. 715–735, doi: 10.1002/rrq.366.
Erduran, S., Simon, S. and Osborne, J. (2004), “TAPping Into Argumentation: Developments in the Application of Toulmin’s Argument Pattern for Studying Science Discourse”, Science Education, Vol. 88 No. 6, pp. 915–933, doi: 10.1002/sce.20012.
Ericsson, K.A. and Simon, H.A. (1993), Protocol Analysis: Verbal Reports as Data, Cambridge, MA: MIT Press, Cambridge.
Fakhriyah, F., Rusilowati, A., Nugroho, S.E. and Saptono, S. (2021), “Mengembangkan Kemampuan Argumentasi Ilmiah Calon Guru Sekolah Dasar sebagai Bentuk Penguatan Keterampilan Abad 21”, Prosiding Seminar Nasional Pascasarjana Pascasarjana Universitas Negeri Semarang, Vol. 4 No. 1, pp. 190–194.
Fan, C.-Y. and Chen, G.-D. (2021), “A Scaffolding Tool to Assist Learners in Argumentative Writing”, Computer Assisted Language Learning, Vol. 34 No. 1–2, pp. 159–183, doi: 10.1080/09588221.2019.1660685.
Fang, N. and Tajvidi, M. (2018), “The effects of computer simulation and animation (CSA) on students’ cognitive processes: A comparative case study in an undergraduate engineering course”, Journal of Computer Assisted Learning, Vol. 34 No. 1, doi: 10.1111/jcal.12215.
Fauziah, A.N.M., Wasis, Hendratmoko, A.F., Mahdiannur, M.A., Ermawan, M.Z.F., Suwandi, E. and Ratri, S.Y. (2024), “The Relationship Between Critical Thinking and Scientific Argumentation in Science Learning”, Jurnal Pendidikan IPA Indonesia, Vol. 13 No. 2, pp. 239–254, doi: 10.15294/jpii.v13i2.2585.
Guo, K., Zhong, Y., Li, D. and Chu, S.K.W. (2023), “Effects of Chatbot-Assisted in-Class Debates on Students’ Argumentation Skills and Task Motivation”, Computers & Education, Vol. 203, p. 104862, doi: 10.1016/j.compedu.2023.104862.
Heirweg, S., De Smul, M., Devos, G. and Van Keer, H. (2019), “Profiling upper primary school students’ self-regulated learning through self-report questionnaires and think-aloud protocol analysis”, Learning and Individual Differences, Vol. 70, pp. 155–168, doi: 10.1016/j.lindif.2019.02.001.
Hendratmoko, A.F., Madlazim, M., Widodo, W. and Astutik, S. (2024), “Debate Based on Inquiry Learning (DBOIL): An Innovative Learning Model to Improve Students’ Skills in Scientific Argumentation”, TEM Journal, Vol. 13 No. 4, pp. 3295–3303, doi: 10.18421/TEM134-64.
Hendratmoko, A.F., Madlazim, M., Widodo, W. and Sanjaya, I.G.M. (2023), “The Impact of Inquiry-Based Online Learning with Virtual Laboratories on Students’ Scientific Argumentation Skills”, Turkish Online Journal of Distance Education, Vol. 24 No. 4, pp. 1–20, doi: 10.17718/tojde.1129263.
Hillocks, G. (1986), Research on Written Composition: New Directions for Teaching, Urbana, IL: ERIC Clearinghouse on Reading and Communication Skills, Urbana.
Ho, H.-Y., Chang, T.-L., Lee, T.-N., Chou, C.-C., Hsiao, S.-H., Chen, Y.-H. and Lu, Y.-L. (2019), “Above-And Below-Average Students Think Differently: Their Scientific Argumentation Patterns”, Thinking Skills and Creativity, Vol. 34, p. 100607, doi: 10.1016/j.tsc.2019.100607.
Hollihan, T.A. and Baaske, K.T. (2022), Arguments and Arguing: The Products and Process of Human Decision Making, Long Grove (illinois): Waveland Press, Long Grove (illinois).
Jordano, M.L. and Touron, D.R. (2018), “How often are thoughts metacognitive? Findings from research on self-regulated learning, think-aloud protocols, and mind-wandering”, Psychonomic Bulletin & Review, Vol. 25 No. 4, pp. 1269–1286, doi: 10.3758/s13423-018-1490-1.
Kim, M. and Roth, W.-M. (2018), “Dialogical Argumentation in Elementary Science Classrooms”, Cultural Studies of Science Education, Vol. 13 No. 4, pp. 1061–1085, doi: 10.1007/s11422-017-9846-9.
Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 363-374.
Latif, M.M.M.A. (2019), “Using think-aloud protocols and interviews in investigating writers’ composing processes: combining concurrent and retrospective data”, International Journal of Research & Method in Education, Vol. 42 No. 2, pp. 111–123, doi: 10.1080/1743727X.2018.1439003.
Lin, Y.-R. (2023), “An idiom-driven learning strategy to improve low achievers’ science comprehension, motivation, and argumentation.”, Computers & Education, Vol. 195, p. 104710, doi: 10.1016/j.compedu.2022.104710.
McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3), 276-282.
Mikeska, J.N. and Howell, H. (2020), “Simulations as Practice‐Based Spaces to Support Elementary Teachers in Learning How to Facilitate Argumentation‐Focused Science Discussions”, Journal of Research in Science Teaching, Vol. 57 No. 9, pp. 1356–1399, doi: 10.1002/tea.21659.
Noroozi, O., Dehghanzadeh, H. and Talaee, E. (2020), “A Systematic Review on The Impacts of Game-Based Learning on Argumentation Skills”, Entertainment Computing, Vol. 35, p. 100369, doi: 10.1016/j.entcom.2020.100369.
Noviyanti, N.I., Rosyadah Mukti, W., Dahlia Yuliskurniawati, I., Mahanal, S. and Zubaidah, S. (2019), “Students’ Scientific Argumentation Skills Based on Differences in Academic Ability”, Journal of Physics: Conference Series, Vol. 1241 No. 1, p. 012034, doi: 10.1088/1742-6596/1241/1/012034.
Osborne, J.F., Borko, H., Fishman, E., Gomez Zaccarelli, F., Berson, E., Busch, K.C., Reigh, E., et al. (2019), “Impacts of a Practice-Based Professional Development Program on Elementary Teachers’ Facilitation of and Student Engagement With Scientific Argumentation”, American Educational Research Journal, Vol. 56 No. 4, pp. 1067–1112, doi: 10.3102/0002831218812059.
Özdemir, S. (2018), “The Effect of Argumentative Text Pattern Teaching on Success of Constituting Argumentative Text Elements”, World Journal of Education, Vol. 8 No. 5, p. 112, doi: 10.5430/wje.v8n5p112.
Öztürk, A. and Doğanay, A. (2019), “Development of Argumentation Skills through Socioscientific Issues in Science Course: A Collaborative Action Research”, Turkish Online Journal of Qualitative Inquiry, pp. 52–89, doi: 10.17569/tojqi.453426.
Park, B., Korbach, A. and Brünken, R. (2020), “Does thinking-aloud affect learning, visual information processing and cognitive load when learning with seductive details as expected from self-regulation perspective?”, Computers in Human Behavior, Vol. 111, p. 106411, doi: 10.1016/j.chb.2020.106411.
Pimvichai, J. and Buaraphan, K. (2019), “A Case Study of Helping In-Service Science Teacher to Teach with the Science-Technology-Society Approach and its Influence on Students’ Scientific Argumentation”, International Journal of Education and Practice, Vol. 7 No. 4, pp. 391–403, doi: 10.18488/journal.61.2019.74.391.403.
Prochazka, F., Weber, P. and Schweiger, W. (2018), “Effects of Civility and Reasoning in User Comments on Perceived Journalistic Quality”, Journalism Studies, Vol. 19 No. 1, pp. 62–78, doi: 10.1080/1461670X.2016.1161497.
Rogiers, A., Merchie, E. and Van Keer, H. (2020), “What they say is what they do? Comparing task-specific self-reports, think-aloud protocols, and study traces for measuring secondary school students’ text-learning strategies”, European Journal of Psychology of Education, Vol. 35 No. 2, pp. 315–332, doi: 10.1007/s10212-019-00429-5.
Ruwe, T. and Mayweg-Paus, E. (2023), “‘Your argumentation is good’, says the AI vs humans – The role of feedback providers and personalised language for feedback effectiveness”, Computers and Education: Artificial Intelligence, Vol. 5, p. 100189, doi: 10.1016/j.caeai.2023.100189.
Saldıray, A. and Doğanay, A. (2024), “An action research to develop critical thinking skills in the context of citizenship education in higher education”, Thinking Skills and Creativity, Vol. 53, p. 101584, doi: 10.1016/j.tsc.2024.101584.
Toulmin, S.E. (2003), The Uses of Argument, Updated Ed., Cambridge: Cambridge University Press.
Wardani, A.D., Yuliati, L. and Taufiq, A. (2018), “Kualitas Argumentasi Ilmiah Siswa pada Materi Hukum Newton”, Jurnal Pendidikan: Teori, Penelitian, Dan Pengembangan, Vol. 3 No. 10, pp. 1364–1372, doi: http://dx.doi.org/10.17977/jptpp.v3i10.11734.
Wolcott, M.D. and Lobczowski, N.G. (2021), “Using cognitive interviews and think-aloud protocols to understand thought processes”, Currents in Pharmacy Teaching and Learning, Vol. 13 No. 2, pp. 181–188, doi: 10.1016/j.cptl.2020.09.005.
Xiao, S. and Kuhn, D. (2024), “Inquiry and argumentation skill development work in conjunction”, Cognitive Development, Vol. 71, p. 101464, doi: 10.1016/j.cogdev.2024.101464.
Yilmaz-Na, E. and Sönmez, E. (2023), “Having Qualified Arguments: Promoting Pre-Service Teachers’ Critical Thinking Through Deliberate Computer-Assisted Argument Mapping Practices”, Thinking Skills and Creativity, Vol. 47, p. 101216, doi: 10.1016/j.tsc.2022.101216.
DOI: https://doi.org/10.33394/j-ps.v13i2.15214
Refbacks
- There are currently no refbacks.
Copyright (c) 2025 Ahmad Fauzi Hendratmoko, Sapti Puspitarini, Mohammad Budiyanto, Nanang Rahman, Kiki Septaria, Siti Marpuah
This work is licensed under a Creative Commons Attribution 4.0 International License.
J-PS (Prisma Sains: Jurnal Pengkajian Ilmu dan Pembelajaran Matematika dan IPA IKIP Mataram) p-ISSN (print) 2338-4530, e-ISSN (online) 2540-7899 is licensed under a Creative Commons Attribution 4.0 International License.