The Analysis of Chemistry Test Item Difficulty on XI MIPA Year-End Assessment at SMAN 8 Pontianak

This study aims to categorize the level of difficulty of the questions and describe the cognitive level of the KKO indicators of the 2021/2022 year-end assessment (PAT) of chemistry class XI MIPA at SMAN 8 Pontianak. This research is quantitative descriptive. Data collection techniques were interviews and documentation. Documentation was obtained from chemistry teachers through google forms containing answers and PAT questions. The research subjects amounted to 150 students of class XI MIPA who had taken the End of Year Assessment. The results of the research obtained are difficult categories totaling 4 questions (13.3%), moderate 10 questions (33.3%) and easy 16 questions (53.4%). The results of the analysis obtained in order to improve the distribution of the difficulty level of questions in the future.


INTRODUCTION
Education is everything that schools do so that students have perfect cognitive abilities, mental readiness, and advanced awareness that are useful when they enter society in the future.The focus of educational activities is teacher-centered in maintaining a central position.Therefore, teaching in schools requires various learning modalities that are structured in such a way as to successfully develop learners' cognitive, emotional and psychomotor abilities.In addition, various disciplinary methods are used to ensure teaching is managed (Soyomukti, 2016).Chemistry is a scientific study of matter and its properties, both the changes experienced by matter, and the energy that accompanies these changes (Hidayanti, 2021).The subject that must be studied in high school, especially Mathematics and Science (MIPA) majors, is Chemistry.Students have difficulty understanding chemistry, it's just that teachers are not fully aware of these problems.
Chemistry subjects contain abstract materials, concepts, reactions and there are mathematical operations.This results in most students disliking chemistry so that it has an impact on unwillingness because they have instilled chemistry subjects that are difficult.Chemistry lessons are useful to equip students to have the ability to think logically, analytically, systematically, critically, and creatively, therefore it should be mastered (Rosa, 2015).End of Year Assessment (PAT) is one of the activities to measure the achievement of learner competencies carried out at the end of even semester.The PAT test is prepared based on some of the material and indicators in it that are made by the teacher and then loaded into multiple choice questions.The purpose of measurement is to convey the individual differences of students seen through their learning outcomes.So, it is important to consider the level of difficulty when developing questions.The level of difficulty of a question is the chance of answering a question correctly at a certain level of ability.A good question item that is not too difficult and not too easy ranges from 0.30 -0.70 (Arikunto, 2013).Measuring good learning outcomes is obtained from effective questions, namely by balancing the level of difficulty of the questions (Bagiyono, 2017).
Based on the results of interviews with chemistry teachers of class XI MIPA, it is known that the questions made are estimated to be difficult categories only a few.A total of 150 students participated in the PAT in chemistry subjects, including getting scores below 70.There are even some who get scores of 20 and 40 out of a score of 100.Judging from the PAT results in 2021/2022, there are still many students who do not reach 50% completeness (KKM).In addition, the PAT questions that will be tested on students have never been tested due to lack of time.Most of the time is spent looking for learning methods, preparing materials and learning media because at that time learning began to be carried out face-to-face which was originally online due to Covid-19.Chemistry teachers have also never analyzed in detail the level of difficulty of PAT questions from all chemistry materials.While during the daily test, the teacher has provided repairs or remedials.However, the reality in the field after the PAT was carried out, the scores obtained were still below the KKM so that the achievement of learning outcomes was not optimal.The quality of the test instrument is unknown, which means that it was not analyzed first (Rasmuin et al., 2021).
Previous research conducted by Hamid et al., (2018) regarding the analysis of Final Semester Test items obtained the level of difficulty of the questions, 8 questions with easy criteria, 22 medium questions and 20 difficult questions.The division of the difficulty level of the questions obtained is not standardized, namely 30% easy, 50% moderate, and 20% difficult.The next research conducted by Prabayanti et al., (2018) obtained the level of difficulty of the X Science class increase test questions classified as good, difficult and easy questions are few compared to moderate questions.Previous research has differences, namely the research subject, the object of research and in analyzing the level of difficulty used and the analysis used is not based on indicators and materials.Furthermore, the results of research conducted by Fuadi, (2021) obtained that the grade XI chemistry PAS questions were not distributed to all cognitive domains, namely the C1, C2, C3 and C4 domains.The difference that researchers do from previous research is that after knowing the cognitive level of the indicator, it will describe the cognitive domain which is said to be difficult seen from the index and also the cognitive level appears a lot from the indicator.
Analysis of the level of difficulty is done not only to find out the level of difficulty but to be an improvement solution for the future and improve the questions with the objectives to be achieved.The evaluation tests tested are related to the achievement of learning objectives in the curriculum (Simamora et al., 2021).Analysis of the level of difficulty has been studied by Ruhil et al., (2019) in Indonesian language subjects with the results of 65% of questions with moderate criteria.
Research conducted by Hamid et al., (2018) aims to determine the quality of chemistry items in class X SMAN 2 Ambon but has not reached the cognitive level.The level of difficulty of the question is closely related to the cognitive level.If the level of thinking of students is still LOTS, students are unable to keep up with technological developments that are increasingly stretched, modern, globalized and difficult to adjust to the future era.One of the efforts to renew the quality of education is a change in the assessment instrument (question making).In connection with these problems, this study aims to categorize and describe cognitive levels that can be used as an evaluation of questions and improve the achievement of indicators made, so it is important to conduct research on the analysis of the level of difficulty of even semester chemistry year-end assessment questions for the 2021/2022 academic year, class XI MIPA at SMA Negeri 8 Pontianak.

METHOD
This research is descriptive quantitative, which describes the content of a variable in research, not intended to test certain hypotheses (Marlina, 2020).The research subjects amounted to 150 students in class XI MIPA.Data collection techniques are interviews and documentation.This documentation is in the form of question indicators, PAT questions, and students' answers.The calculation results were then analyzed and categorized as difficult questions based on the index value obtained.The question difficulty index is calculated using the following formula: P =   While: P = difficulty index B = number of learners who answered correctly JS = the number of students who answered the question Furthermore, they are grouped according to the following criteria for the difficulty level of the questions: Table 1.Criteria for Problem Difficulty Level Index Criteria 0,00-0,30 Hard 0,31-0,70 Medium 0,71-1,00 Easy (Arikunto, 2013)

Results of Problem Difficulty Level Analysis
PAT questions in Chemistry Class XI MIPA Even Semester 2021/2022 at SMAN 8 Pontianak contain 5 materials, namely acid base, buffer solution, salt hydrolysis, solubility and solubility products, and colloids which can be seen in table 2.  The data obtained from the analysis of the difficulty level of each PAT question is divided into three criteria, namely difficult, moderate, and easy questions.Problems classified as difficult amounted to 4 questions (13.3%), moderate 10 questions (33.3%) and easy 16 questions (53.4%) which can be seen in Table 3.After analyzing the data, the average level of difficulty of the buffer solution material is stated to have the highest difficulty index value but is still categorized as moderate, followed by salt hydrolysis.

Problem Difficulty Level
The results of the analysis of the level of difficulty of the questions carried out are expected to find various information for improvement and revamping in order to measure what is to be measured (Fitrianawati, 2017).If the question is answered correctly by all students, it cannot be called a good question, and vice versa.Questions are said to be good if the questions are in the medium category (Arikunto, 2013).Overall in terms of difficulty level, even semester chemistry PAT questions at SMA 8 Pontianak in 2021/2022 vary, namely questions in the easy, medium, and difficult categories.
The difficult category questions were 4 questions (13.3%), 10 questions (33.3%) and 16 questions (53.4%).This shows that the distribution of the difficulty level of the questions is uneven so it needs to be improved.These results can provide input to assess the ability of students and improve the initial learning process that has been made.A good question is a medium level of difficulty with a range of 0.30 -0.70.According to Sudjana, (2017) the division of difficulty levels can use a ratio of 3-4-3 or 3-5-2.
Moderate questions are 40% or 50% and difficult questions are 30% or 20%.So, the difficulty level of PAT questions should be 9 questions with easy criteria, 12 questions with moderate criteria, and 9 difficult questions or in other comparisons 9 questions with easy criteria, 15 questions with moderate criteria, and 6 questions with difficult criteria.Analysis of the average level of difficulty for each material obtained buffer solution has the lowest difficulty index of 0.381 but is still categorized as moderate.Buffer solution involves many concepts, calculations that need to be understood.In addition, the concept of salt hydrolysis resembles the concept of buffer solution (Parastuti & Ibnu, 2016).This makes students mistaken in applying the formula (Qadri et al., 2019).
Based on the analysis in table 2, problem solving from experimental data is directly on the problem and can convert information data first.The question of buffer solution considered difficult by students is question number 8 of buffer solution material with an indicator of analyzing the properties / characteristics of buffer solutions based on experimental data.Most students answered wrong with a difficulty index of 0.09.This happens because they do not understand the concept of the characteristics of buffer solutions.Where if an excess weak acid/base reacts with a strong base/acid or a weak acid reacts with a weak base, it will be neutral, it should be in the concept that it will form a buffer solution.They must understand again and convert the data from the problem.
If the concept is not understood then it is hampered in solving the problem.This needs to be considered by the teacher because it is related to solving other problems.In addition, the amount of material learned about the pH and pOH formulas by students in PAT causes errors in the use of buffer solution formulas and the lack of mastery of the differences between acids, bases, and salts.Furthermore, question number 11 with the indicator of determining the pOH of buffer solution from a mixture of a weak base with its salt with an index of 0.28 (28%) is categorized as difficult.Learners are required to be able to solve calculation problems in buffer solution material.Supported by the results of interviews with several students who answered incorrectly, that they were confused about interpreting what the question asked.The use of pH and pOH formulas is still forgotten and the ionization reaction is not mastered because they have not memorized the name of the compound.Buffer solution material is difficult for high school students because the concept is difficult.
Almost all concepts require a fairly high understanding because the understanding of students is much lower based on field facts (Hariani et al., 2016).Supported by research by Genes et al., (2021) stated that the average percentage of 89.3% had difficulty calculating pOH and pH of buffer solutions.This difficulty is due to the low ability of students to determine the pH and pOH formulas, and the students' work on the problem is not thorough.Research conducted by Risnawati & Parham, (2016) that the percentage of success is only 54.72% categorized as low on the indicator of calculating the pH or pOH of buffer solutions.The low number of students solving this problem is due to the lack of practice problems given by the teacher regarding similar problems to students.Based on these results, for the next learning process the teacher should emphasize the understanding, properties of buffer solutions, or be able to relate to examples of solutions that include acids and bases and increase practice problems, especially questions that can distinguish buffer solutions from salt hydrolysis so Hydrogen: Jurnal Kependidikan Kimia, October 2023, 11(5) |682 that students easily understand and get used to solving problems.
Furthermore, the difficulty index of salt hydrolysis material is 0.558 with moderate criteria.Salt hydrolysis material requires a fairly high level of understanding of chemical concepts.Based on table 2, problem solving can use the data presented and some need to be converted first so that the answer is found in multiple choice.The problem that uses direct data without having to convert first is Problem number 12 with the indicator of analyzing a mixture of acids and bases that will undergo hydrolysis.This question only analyzes compounds that can undergo hydrolysis and then chooses answers from the multiple choices available.
Understanding concepts and calculations that are not too difficult in analyzing questions is answered easily by students.However, difficulties in calculations in determining pH because based on interviews the application of the formula is not correct and mathematical operations are still lacking.
The problem of salt hydrolysis material with difficult criteria requires high concept understanding and converting data from the problem so that the stages are more complex than easy criteria questions.Problem number 16 has an indicator of determining the pH of a mixture of a weak base with a strong acid with an index of 0.16 (16%) which is categorized as difficult.Students are required to be able to solve calculation problems on salt hydrolysis material.Based on the results of the interview, the use of the pOH formula is still not understood and the ionization reaction is still lacking.Supported by further research conducted by Abidin, (2019) during the initial observation, the mastery of students was still relatively low because they had not mastered the salt hydrolysis material well.The results of the study obtained several errors made by students, namely writing the reaction equation (ionization reaction) incorrectly so that it has an impact when using the formula, and the use of symbols (roots).In this question, 26.25% misconceptions were found, which is possible that students do not understand the types of ions that can be hydrolyzed (Priyasmika & Sholichah, 2022).This can be overcome by the teacher by giving many questions about calculations about pH and pOH and strengthening the memorization of acid-base compounds.
Ksp material has an average difficulty index of 0.703 which is categorized as easy.Based on the analysis, students can answer well as evidenced by the average difficulty index.Based on the results of the interview at number 18, students have used the appropriate formula, but the calculation method used at the end (root concept) is wrong because the calculation method is different from the practice problem taught.This can occur because the initial concepts of roots and multiplication are still poorly mastered, especially their math skills (Ulfah et al., 2016).It is predicted that the items with difficult criteria have not been taught or the learning has not been completed and the form of the question asked is not suitable to be measured in the material.The level of difficulty can also be based on errors in the question, for example, wrong commands, inappropriate answer options, different answer keys, reading that is difficult to understand (Yonelia & Haryati, 2015).
Solutions that can be applied by teachers in improving the quality of question items are balancing the percentage distribution or the ideal difficulty index, changing questions, and making new questions according to the indicators to be achieved.According to Arikunto, (2013) the follow-up of the three categories is that easy category questions can be retained, re-examined or discarded.Moderate category questions are used and stored in the question bank so that they can be reappeared according to the goals the teacher wants to achieve.There are three possibilities for difficult category questions, namely maintaining or re-issuing them for strict tests, re-examining them by exploring them in depth.

PAT Problem Difficulty Level Based on Cognitive Level
Cognitive level is a domain that covers brain activities that are often involved, namely thinking.Based on table 3, the results of the analysis seen based on KKO that the cognitive level C4 (Analyzing) average difficulty level of 0.55 is categorized as moderate.Cognitive level C4 (HOTS) which consists of five questions, namely numbers 3, 8, 9, 12, and 22.The analysis carried out by students is simple analysis and complex analysis.Complex analysis requires a successive stage of completion so that thinking is quite high.The problem makes learners connect more concepts that are understood and then interpret them into the problem as in number 8. In this problem, it can be analyzed directly from the experimental data without further conversion.
Simple analysis does not make students have difficulty in answering questions.This can be seen in table 2, cognitive level analyzing shown number 3 and 12 can be answered easily.Analysis thinking on the question does not really use the complex analysis stage.High-level skills will lead learners to optimize their brain work.The results of Dewi et al.'s research (2021) where HOTS items developed on ion equilibrium material in solution with a percentage of 85.9% obtained high test scores on high-level thinking skills in problem solving and decision making.This means that the questions presented are understood by students and are not as complex as questions with difficult criteria.If the ability of students is low, it will be difficult to solve the problem because it needs a good understanding to solve the problem.
C3 (Applying) cognitive level obtained an average of 0.66 with moderate criteria.Based on the analysis, students have difficulty at the cognitive level, namely determining the calculation of pH and pOH.In applying the concept, it is directly applied and converts the data first to solve the problem.In applying the concept of buffer solution material and salt hydolysis, students have difficulty in applying concepts that have an impact on the use of formulas because they involve all concepts and then rearrange information from the problem.The application of the formula is found in numbers 11, 16, and 18.In MOTS questions can manage extra thinking to solve the question asked because this question can reverse a material even though the same result (Himmah, 2019).Problems that are often trained will make students understand the flow of solving these problems, for example, given practice problems or problems that are almost the same.
The distribution of cognitive levels seen from the KKO indicators of PAT questions in chemistry class XI MIPA there are C1, C2, C3 and C4 there are no questions level C5 and C6.In accordance with the opinion of Yonelia & Haryati, (2015) it should be to measure the level of thinking of students, the questions used start at a low level to a high level of thinking, so that the evaluation of learning outcomes can measure the ability of students at each level of thinking and be more optimal.This means that for this high school level the cognitive level has been fulfilled, namely C3 and C4.The results of the analysis of cognitive levels seen from the KKO indicators of each question of chemistry subject PAT class XI MIPA at SMAN 8 Pontianak are more MOTS, namely 70% for the purposes of semester exams, which is quite good because it can stimulate students to solve problems.

CONCLUSION
Based on the results of the analysis of the level of difficulty of the Year-End Assessment (PAT) questions for chemistry class XI MIPA in 2021/2022, it is concluded that out of 30 items, there are 4 questions (13.3%) classified as difficult, 10 questions (33.3%) classified as medium and 16 questions (53.4%) classified as easy.Overall, in terms of difficulty level, the Hydrogen: Jurnal Kependidikan Kimia, October 2023, 11(5) |684 dominant question is easy and has not fulfilled the proportion of the level of difficulty of the question.
change of universal indicators in acid/base solutions C4 Easy 4 Determine the concentration of OH-in a weak base solution C3 of strong acid/strong base solution based on titration result data C3 Medium Determine which salt is acidic when dissolved in water C3 Easy Buffer Solution 8 Analyze the nature / characteristics of buffer solutions based on experimental data C4 Hard 9 Predicting mixtures of acids and bases that can function as buffer solutions C4 Medium Determine the pH of buffer solution from a mixture of weak acid and strong base C3 Medium Determine the pOH of a buffer solution from a mixture of a weak base with its salt C3 Hard Salt Hydrolisis Analyzing acid and base mixtures that will undergo hydrolysis C4 Easy Determine the pH of solutions of weak acids with their salts C3 Medium Determining the pH of salt after dilution C3 Easy Determine the pH of a mixture of strong base and weak acid C3 Medium Determine the pH of a mixture of a weak base with a strong acid C3 Hard Equilibrium Constant (KSp) Determine the solubility of a poorly soluble salt C3 Hard Ranking soluble compounds based on Ksp value C3 Medium Explain the effect of similar ions on solubility C2 Easy Determine the pH of a saturated solution based on the Ksp value C3 Easy Predicting the pH of the solution in salt solution by adding NaOH C4 Medium Explain the relationship between Ksp and Qc C2 Easy Determine the pH of strong base and weak acid solutions C3 Easy Colloid Classify types of colloids based on dispersed and dispersing phases C2 Easy Apply the properties of colloids in peat water treatment process C3 Medium Apply colloidal properties to laundry cleaning soap C3

Table 2 :
Distribution of PAT Question Indicators for Even Semester Chemistry Subjects in 2021/2022 at SMAN 8 Pontianak.

Table 3 .
Difficulty Level of PAT Questions for Each Material in the Even Semester Chemistry Subject in 2021/2022

Table 4 .
Average Level of Difficulty of PAT Questions Based on Cognitive Level of Chemistry Subjects Even Semester 2021/2022