Development of Critical Thinking Skills Instruments : Cases for Essay Tests

Nurkhairo Hidayati, Suryanti Suryanti, Restesa Rahmayumita, Selvira Aisya

Abstract


This research aims to develop a rubric used to assess students’ critical thinking. This rubric refers to aspects of Ennis' critical thinking. The research method begins with determining the scope of the rubric so that an initial version of the rubric can be prepared. An initial version of the rubric received peer review. Next, the rubric was tested to assess critical thinking tests with essay-type questions. Data analysis was carried out using Cohen's kappa, while aspects of the critical thinking rubric were analyzed descriptively. The research results show that each aspect of critical thinking has been equipped with descriptors that can be used by raters. The level of reliability based on the kappa value of agreement is quite good, which means that different raters can use the rubric consistently. The value of agreement in the basic clarification aspect is 0.33 (fair), bases for a decision (0.571 in the moderate category), inference (0.455, moderate category), advanced clarification (0.250, fair category, supposition and integration (0.250, fair category), and strategies and tactics (0.182, poor category). Then, it can be concluded that the critical thinking rubric developed can be used to assess critical thinking abilities. As a recommendation, of course, this rubric cannot fulfill all types of critical thinking questions tested. It is also necessary to develop rubrics for question types other than essays to accommodate the wider need for critical thinking rubrics.

Keywords


Critical Thinking; Rubric; Assessment.

Full Text:

PDF

References


Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2014). Strategies for Teaching Students to Think Critically: A Meta-Analysis. Review of Educational Research, 1–40. https://doi.org/10.3102/0034654314551063

Anderson, T., & Soden, R. (2001). Peer interaction and the learning of critical thinking skills. Psychology Learning & Teaching, 1(1), 37–40.

Bezanilla, M. J., Fernández-Nogueira, D., Poblete, M., & Galindo-Domínguez, H. (2019). Methodologies for teaching-learning critical thinking in higher education: The teacher’s view. Thinking Skills and Creativity, 33(100584), 1–10. https://doi.org/10.1016/j.tsc.2019.100584

Biggs, J. (2014). Constructive alignment in university teaching. HERDSA Review of Higher Education, 1(1), 5–22.

Brookhart, S. M., & Chen, F. (2015). The quality and effectiveness of descriptive rubrics. Educational Review, 67(3), 343–368. https://doi.org/10.1080/00131911.2014.929565

Butler, H. A. (2012). Halpern Critical Thinking Assessment Predicts Real-World Outcomes of Critical Thinking. Applied Cognitive Psychology, 26(5), 721–729. https://doi.org/10.1002/acp.2851

Button, C. M., Snook, B., & Grant, M. J. (2020). Inter-Rater Agreement, Data Reliability, and The Crisis of Con fi dence in Psychological Research. Quant Methods Psychol, 16(5), 467–471.

Cooke, N. A. (2017). Posttruth, Truthiness, and Alternative Facts: Information Behavior and Critical Information Consumption for a New Age. The Library Quarterly, 87(3), 211–221. https://doi.org/10.1086/692298

Cooper, K. M., & Brownell, S. (2020). Student Anxiety and Fear of Negative Evaluation in Active Learning Science Classrooms Chapter 56 Student anxiety and fear of negative evaluation in active learning science classrooms. February. https://doi.org/10.1007/978-3-030-33600-4

Din, M. (2020). Evaluating university students’ critical thinking ability as re fl ected in their critical reading skill : A study at bachelor level in Pakistan. Thinking Skills and Creativity, 35(100627), 1–11. https://doi.org/10.1016/j.tsc.2020.100627

Ennis, R. H. (2011). The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions. 1–8. http://faculty.education.illinois.edurhennis/documents. TheNatureofCriticalThinking_51711_000.pdf

Facione, P. A. (2000). The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill. Informal Logic, 20(1), 61–84.

Golter, P., Brown, G., & Wie, B. Van. (2016). Preliminary Validation of a Critical Thinking Rubric for Fluid Mechanics and Heat Transfer. Journal of STEM Education, 17(4), 19–28.

Gray, K., & Koncz, A. (2018). The key attributes employers seek on students’ resumes. http://www.naceweb.org/about-us/press/2017/the- key-attributes-employers-seek-on-students-resumes/

Halpern, D. F. (2010). Halpern Critical Thinking Assessment. SCHUHFRIED. http://www.schuhfried.com/vienna-test-system-vts/all-tests-from-a-z/test/hcta-halpern-critical-thinking-assessment-1/

Hart, C., Da, C., Souza, D. D., Kimpton, A., & Ljbusic, J. (2021). Exploring higher education students’ critical thinking skills through content analysis. Thinking Skills and Creativity, 41(100877), 1–9. https://doi.org/10.1016/j.tsc.2021.100877

Hattie, J., & Gan, M. (2011). Instruction based on feedback. Handbook of research on learning and instruction.

Kholid, M. N., Hamida, P. S., Pradana, L. N., & Maharani, S. (2020). Students’ Critical Thinking Depends On Their Cognitive Style. International Journal of Scientific & Technology Research, 9(01), 1045–1049.

Khonamri, F., & Karimabadi, M. (2015). Collaborative Strategic Reading and Critical Reading Ability of Intermediate Iranian Learners. Theory and Practice in Language Studies, 5(7), 1375–1382. http://dx.doi.org/10.17507/tpls.0507.09

Kong, L.-N., Qin, B., Zhou, Y., Mou, S., & Gao, H.-M. (2014). The effectiveness of problem-based learning on development of nursing students’ critical thinking: A systematic review and meta-analysis. International Journal of Nursing Studies, 51, 458–469. https://doi.org/10.1016/j.ijnurstu.2013.06.009

Ku, K. Y. L., Kong, Q., Song, Y., Deng, L., Kang, Y., & Hu, A. (2019). What predicts adolescents’ critical thinking about real-life news? The roles of social media news consumption and news media literacy. Thinking Skills and Creativity, 33(100570), 1–12. https://doi.org/10.1016/j.tsc.2019.05.004

Kuhn, D. (2019). Critical Thinking as Discourse. Human Development, 62(3), 146–164. https://doi.org/10.1159/000500171

Lai, E. R. (2011). Critical Thinking: A Literature Review. Pearson’s Research Reports, 6(1), 1–49. https://doi.org/10.1046/j.1537-2995.1995.35395184278.x

Lin, S. (2013). Science and non-science undergraduate students’ critical thinking and argumentation performance in reading a science news report. International Journal of Science and Mathematics Education, 12(5), 1023–1046. https://doi.org/10.1007/s10763-013-9451-7

Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing Critical Thinking in Higher Education: Current State and Directions for Next-Generation Assessment. ETS Research Report Series, 1, 1–23.

Pearl, A. O., Rayner, G. M., Larson, I., & Orlando, L. (2018). Thinking about critical thinking: An industry perspective. Industry and Higher Education, 1–11. https://doi.org/10.1177/0950422218796099

Pui, P., Yuen, B., & Goh, H. (2020). Using a Criterion-referenced Rubric to Enhance Student Learning: a case study in a critical thinking and writing module. Higher Education Research & Development. https://doi.org/10.1080/07294360.2020.1795811

Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020). Rubrics to assess critical thinking and information processing in undergraduate STEM courses. International Journal of STEM Education Https://Doi.Org/10.1186/S40594-020-00208-5, 7(9), 1–15. https://doi.org/10.1186/s40594-020-00208-5

Saxton, E., Belanger, S., & Becker, W. (2012). The Critical Thinking Analytic Rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17, 251–270.

Shaw, A., Liu, O. L., Gu, L., Kardonova, E., Chirikov, I., Li, G., Hu, S., Yu, N., Ma, L., Guo, F., Su, Q., Shi, J., & Loyalka, P. (2019). Thinking critically about critical thinking: validating the Russian HEIghten ® critical thinking assessment. Studies in Higher Education, 2–16. https://doi.org/10.1080/03075079.2019.1672640

Smit, R., & Birri, T. (2014). Assuring the quality of standards-oriented classroom assessment with rubrics for complex competencies. Studies in Educational Evaluation, 1–9. https://doi.org/10.1016/j.stueduc.2014.02.002

Stein, B., & Haynes, A. (2011). Engaging Faculty in the Assessment and Improvement of Students’ Critical Thinking Using the Critical Thinking Assessment Test By Barry Stein and Ada Haynes. Change: The Magazine of Higher Learning, 43(2), 44–49.

Tong, F., Tang, S., Irby, B. J., Lara-alecio, R., & Guerrero, C. (2020). The determination of appropriate coe ffi cient indices for inter-rater reliability : Using classroom observation instruments as fi delity measures in large-scale randomized research. International Journal of Educational Research, 99(101514), 1–10. https://doi.org/10.1016/j.ijer.2019.101514

Undergraduate Professional Education in Chemistry ACS Guidelines and Evaluation Procedures for Bachelor’s Degree Programs. (2015). American Chemical Society Committee on Professional Training.

VALUE Rubric Development Project. (2019). Association of American Colleges and Universities. https://www.aacu.org/value/rubrics

Vision for science and mathematics education. (2014). The Royal Society Science Policy Centre.

Wale, B. D., & Bishaw, K. S. (2020). Effects of using inquiry-based learning on EFL students’ critical thinking skills. Asian-Pacific Journal of Second and Foreign Language Education, 5(9), 1–14.




DOI: https://doi.org/10.33394/jk.v10i1.10052

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 The Author(s)

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Jurnal Kependidikan : Jurnal Hasil Penelitian dan Kajian Kepustakaan di Bidang Pendidikan, Pengajaran, dan Pembelajaran

E-ISSN: 2442-7667

Published by LPPM Universitas Pendidikan Mandalika

Email: [email protected] 

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.