Five-Level Conception Diagnostic Test on Electromagnetic Induction Concepts: Feasibility Test and Overview of Students' Concept Understanding Level

During the Physics learning process, students often have difficulty in understanding the concepts, such as Electromagnetic Induction concepts. For example, students assumed that a loop in a magnetic field B whose coil plane area shrinks towards the page produces a constant induced electromotive force. However, the truth is when the surface area of the loop is reduced, the induced electromotive force produced gets smaller. This contradiction is called a misconception. This misconception should be detected immediately using a diagnostic test of conception, especially the five-level diagnostic test. This paper is intended to report the results of the writing of a five-level conception diagnostic test for Electromagnetic Induction concepts, to test its’ feasibility (validity and reliability) and to get an overview of the several students’ understanding level as well as to identify the most dominant internal factors that cause students' misconceptions on the concepts. A total of 11 questions on the diagnostic test is validated and reliable therefore it can be used to test students' understanding level. The results of the internal validity test are 92% (very valid) and the external content validity in terms of true fake and wrong fake values are 3.94% and 5.45%, the construct external validity (rxy=0.591) > rtable=0.3061 and the reliability [ r11=0.608 (very reliable)]. Among 14 students who were tested for their level of understanding 44.16% of them experienced a lack of knowledge.


INTRODUCTION
Physics is one of the branches of natural sciences, which consists of various concepts, studying natural phenomena, objects in nature, and interactions between its' objects (Lailiyah & Ermawati, 2020). During Physics learning, students are more required to understand the concepts than recitation. Therefore, effective and efficient learning strategies are needed to quickly understand the concepts of Physics (Pertiwi & Setyarsih, 2015). But in reality, students are often unable understand the concepts taught by the teacher properly. Such a situation has the potential to cause misunderstanding of concepts in students.
Based on the results of the pre-research activities that was carried out when teaching Electromagnetic Induction concepts at the Islamic state of senior high schools in Lamongan, it was found that there is a potential students' misconception. For example, when the students asked to describe the value of the induced electromotive force produced by a loop in a magnetic field ⃗ ⃗ whose coil plane area shrinks towards the page in Figure 1, the student replied that the size of the loop is the same and the number of magnetic force lines entering will be proportional to the surface area of the loop (A) so that the induced electromotive force produced is constant. Meanwhile, according to Giancoli (2014) in his book entitled "Physics: Principles with Applications", it is mentioned that current can be induced by changing the coil area (A), although the magnetic field ⃗ ⃗ does not change ( Figure 2). In Figure 2, the coil area in area A is reduced by pulling the side of the coil so that the rate of change of flux decreases when we go from (a) to (b) and the value of the induced electromotive force becomes smaller. Likewise, according to Young & Freedman (2013) in his book entitled "University Physics with Modern Physics" that when a loop in the magnetic field ⃗ ⃗ is constant then we attract or press both sides of the coil, the rate of change of the flux decreases and the value of the induced electromotive force becomes smaller so that the current moves counterclockwise. The difference between a concept understood by students and the correct concept is called a misconception (Firdaus et al., 2021;Jannah & Ermawati, 2019;Sadhu et al., 2017).
Misconceptions can occur because the students' initial knowledge is different from the correct concepts; such initial knowledge is called preconception (Utari & Ermawati, 2018). In addition, misconceptions can also occur because of the student's perspective on a concept, the teacher and the learning resources used (Firdaus et al., 2021). The misconceptions experienced by students must be detected immediately from the beginning so as not to drag on and interfere with students' understanding on the following concepts (Sadhu et al., 2017). One effective way to detect such misconceptions is using a conception diagnostic test (Salsabila & Ermawati, 2020;Gurel et al., 2015). The conception diagnostic test was widely reported to date as a four-level diagnostic test consisting of the first level in the form of question with several multiple choices. The second level is the level of students' confidence in answering question at the first level. The third level is the choices of students' reason for answering question at the first level. The fourth level is the level of students' confidence in choosing the correct reason for the third level question. Previously, several four-level conception diagnostic tests were reported by other authors, such as by Firdaus et al., (2021) for Inheritance concepts; Kiray & Simsek (2021) for Density concepts; Tumanggor et al., (2020) for Simple Harmonic Motion concepts; Kurniawati & Ermawati (2020) Yang & Lin (2015) for Number Sense Concepts. Table 1 provides an example of a five-level conception diagnostic question consisting of first to the fifth levels that was written by the Authors of this work. The four-level conception diagnostic test referred to above are the questions in Table 1 starting from the first to the fourth levels. Table 1. An example of a five-level conception diagnostic test question consisting of first to the fifth levels that was written by the Authors of this work.

Level
Question and Answer choices 1 st Level 1. A loop in a magnetic field B whose coil area shrinks towards the page ( ) as shown in Figure 1. a. The surface area of the loop is getting narrower, so the rate of flux change will decrease and produce a smaller induced electromotive force* b. The surface area of the loop is getting narrower, so the rate of flux change will increase and produce a greater induced electromotive force (Humanistic thinking) c. The surface area of the loop does not affect the electromotive force generated (False intuition) d. The induced electromotive force only occurs due to changes in the magnetic field (preconception) e. The surface area of the loop is proportional to the number of lines of magnetic force entering so that the rate of flux change will remain and produce a constant induced electromotive force (Associative thinking) f. The surface area of the loop is getting narrower, so the flux will increase and induce smaller electric forces (Incomplete reasoning) 4 th Level Level of confidence in choosing the reason for the answer: a. Believe* b. Do Not Believe 5 th Level What conclusions can you draw about the value of the induced electromotive force produced when the loop in the shrinking magnetic field ⃗ ⃗ leads to the field?
The surface area of the loop is getting narrower so that the rate of change in the magnetic flux decreases and produces a smaller induced electromotive force* Information: * stated the answer key Table 1 shows the questions at the first to the fourth levels are in the form of multiple choices, where the tested students only get the opportunity to choose one answer that is considered correct. According to the previous authors, the multiple-choice questions can already be used to detect the level of understanding of the tested students. However, relying on students' answers from the first to the fourth levels alone cannot confidence the examiner about whether the student has really understood the concept being asked or not? Therefore, it is necessary to add one question level which is considered able to provide the required confidence level. In this case, the question is an open question at the fifth level in Table 1. The open question can be in the form of requests for students to draw conclusion about the concepts asked in the first level question, to make concept maps, question to draw (drawing question), or other question according to the needs of each question (Putri & Ermawati, 2021;Ramadhani & Ermawati, 2021).
The way to assess students' works results using the four-level conception diagnostic test is given in Table 2, especially in the first to the fourth levels columns. When the combination of answers given by a student are precise (P), believe (B), precise (P) and believe (B), it can be concluded that the student already understood the concept (RU), and so on including student answers that are considered true fake (TF) and wrong fake (WF). What is meant by a true fake answer (TF) is when the combination of students' answers on the first to the fourth levels are precise (P), believe (B), imprecise (IP) and believe (B). It means that the student answered the question precisely and he or she believes that the answer is precise. In addition, he or she choose the imprecise reason and unfortunately he or she believes that the chosen reason is precise. Meanwhile, what is meant by a wrong fake answer (WF) is when the combination of students' answers on the first to the fourth levels are imprecise (IP), believe (B), precise (P) and believe (B). It means that the student is imprecise in answering the question but he or she believes that the answer is precise. Further, he or she chose the precise reason and he or she believes that the reason is precise.
While the assessment that used in the five-level conception diagnostic test is by adding one assessment at the fifth level column in Table 2. When the combination of answers from the first to the fourth levels given by a student are precise and he or she believes that the answer is precise, the reason chosen are also precise and he or she also believes it, then the student can still be possibly considered really understood the concept (RU) or partial understood (PU) or lack of knowledge (LC) or misconceptions (MC) or does not understood the concept (NU) or undefined (UD). There is an unanswered question either in level 1, 2, 3, 4 or 5 UD Note Table 2 shows that students are categorized as really understood the concept (RU) when the combination of their answers to the questions at the first to the fourth levels are precise (P), believe (B), precise (P) and believe (B), the fifth level answer is also given precisely according to the scientific understood (SU). However, when student's answers to the questions at the first to the fourth levels are precise (P), believe (B), precise (P) and believe (B), but the answer to the fifth level question is still only partially precise (PP), then the student is categorized as partial understood the concept (PU). Furthermore, when the combination of answers to the questions at the first to the fourth levels given by a student are precise (P), believe (B), precise (P) and believe (B), but the answer to the fifth level question are misconceptions (M) and wrong (W), then the student is categorized as lack of knowledge (LC). Meanwhile, when student's answers to the questions at the first to the fifth levels are precise (P), believe (B), precise (P) and believe (B), but the answer to the fifth level question is not answered (NA), then the student is categorized as undefined (UD) and so on. The explanation and answer values for the fifth level open-ended question is shown in Table 3. When student's answer to the question at the fifth level is precise and in accordance with the Physics concepts, then the student is declared scientific understood (SU) which is given a value of 100%. Furthermore, when the answer to the question at the fifth level given by a student is still only partially precise, then the student is declared partially precise (PP) which is given a value of 70-90%, and so on. Until now, the five-level conception diagnostic test was adopted by some authors to identify the students' level of understanding, such as by Ramadhani & Ermawati (2021)  al. (2018) for Changes of Matter concepts. However, the five-level conception diagnostic test on Electromagnetic Induction concepts has never been reported.
Based on the explanation above, this paper is intended to report the results of the writing of a five-level conception diagnostic test on Electromagnetic Induction concepts. Furthermore, the written test is tested for feasibility (validity and reliability) and tested applied to a number of students to identify their level of understanding on Electromagnetic Induction concepts.

METHOD Research Type
Considering that the conception diagnostic test is not standardized yet, it means that someone who wants to test their students' level of understanding must write it first then test its' feasibility before using a diagnostic conception test. Therefore, "Research and Development (R&D)" is required in writing a conception diagnostic test. What "Research" means is that the Authors examined the concepts of Electromagnetic Induction (especially on the sub-concepts: Definition of Inductive Electromotive Force, Induction Electromotive Force Due to Changes in the Area of the Coil, Induced Electromotive Force Due to Changes in Orientation Angle θ, Faraday's Law and Lenz's Law) at two books references, i.e., Physics: Principles with Applications, 7 th Edition (Giancoli, 2014) and Physics for Scientists and Engineers, 6 th Edition (Serway & Jewett, 2004). Next stage, the Authors analyzed and summarized the students' potential misconceptions. Table 4 summarizes some of the students' potential misconceptions on Electromagnetic Induction concepts from the results of the analysis. In the "development" stage, the Authors wrote the Draft 1 (consisting of 11 questions of the five-level conception diagnostic test). Table 4. Some students' potential misconceptions that obtained from the results of the analysis on Electromagnetic Induction concepts.

Correct Concept Potential misconceptions
Definition of Induction Electromotiv e Force (EMF) The induced electromotive force (EMF) is the potential difference between the ends of the coil (coil) that will produce an induced electric current.  Here the area of the coil in area A is reduced by pulling the side of the coil so that the change in speed decreases when we go from (a) to (b) and the value of the electromotive force produced is getting smaller. The short induced current acts in the direction shown to try to maintain the original flux (ФB=BA) by producing its own magnetic field in the page. That is as the area of the coil in area A decreases, the current acts to increase B in the initial (inward) direction (Source: Giancoli, 2014).
Students assumed that the size of the loop is the same and the number of magnetic force lines entering will be proportional to the surface area of the loop (A) so that the induced electromotive force produced is constant.

Subject
The subject in this work at the First trial was conducted on 30 students of class XII MIPA 3 at one of Gresik's senior high schools to obtain the results of the external validity and reliability. Meanwhile, the Second trial was conducted on 14 students from class XII MIPA 4 at one of Gresik's senior high schools to detect the students' level of understanding on Electromagnetic Induction concepts.

Instrument, Procedure and Data Analysis
The first draft (consisting of 11 questions of the five-level conception diagnostic test) was written by the Authors then tested for its internal validity by two appointed Lecturers from the Physics Department of UNESA, both in terms of content, construct or language with several indicators. The indicators that used to assess the internal validity in terms of content are (a) whether the questions are in accordance with Electromagnetic Induction concepts or not, (b) whether the questions are in accordance with the question indicators or not, (c) whether the explanation of the questions in accordance with the order of the concepts or not and (d) whether limits of question, answer and explanation of expected reasons are declared clearly or not. Furthermore, the indicators that used to assess the internal validity in terms of constructs are (a) whether the diagnostic test instructions were expressed clearly or not, (b) whether between the criteria of the questions with the Basic Competencies and Bloom's Taxonomy are appropriate or not, (c) whether each question in diagnostic test can determine students' understanding or not, (d) whether the choice of reasons provided able to determine the causes of misconceptions from within the students or not, (e) whether the distractors in the choice of reasons were rational and homogeneous with the first level answer or not and (f) whether the pictures, graphs, tables and the like are in accordance with the provided problems or not. Meanwhile, the indicators that used to assess the internal validity in terms of language are (a) whether the question sentences utilized Indonesian properly and correctly or not, (b) whether the question sentences did not cause double meaning or not and (c) whether the question of each test are stated clearly and easily understood or not.
Each indicator that used to assess the internal validity is given a value ranging from 1 to 4 according to the assessment rubric as shown in Table 5. The way to calculate the percentage (%) of the internal validity is using Equation (3) then interpreted based on Table  6. The results of the internal validity test were Draft 2. The Draft 2 was conducted first test (First Trial) with the respondents are 30 students of class XII MIPA 3 at one of Gresik's senior high schools to obtain the results of the external validity and reliability. The external validity test consists of empirical content and construct validity (questions in accordance with basic competencies). The empirical validity of the content was determined based on the calculation of the % true fake (TF) and wrong fake (WF) answers using Equation (4) and (5) (Kirbulut & Geban, 2014). In theory, the empirical validity of the content will be fulfilled if each %TF and %WF is less than 10% (<10%) (Zahra & Suprapto, 2019). The explanation of the true fake (TF) and the wrong fake (WF) answers was explained in the section "Introduction". where: ΣTF = number of TF, ΣWF = number of WF, Σquestions = number of questions (consisting of 11 questions), and Σstudents = number of students (In this work amounted to 30 students).
The empirical external validity test of the construct was determined based on the calculation of the Pearson Product Moment (rxy) correlation using Equation (6) (Arikunto, 2013) with the standards in Table 7. The reliability of the questions was calculated by the Alpha Cronbach coefficient (r11) as in Equation (7) (Arikunto, 2016) with standards based on Table 8.
where: rxy = the correlation between variables x and y, x = the total value of each question, y = the total value of each student. where: 11 = the reliability coefficient, ks = number of questions, ∑ 2 = the total variance value of each question and 2 = number of grain variances. The outcomes of the external validity and the reliability test resulted in the Final Draft of a five-level conception diagnostic test (consisting of 11 questions) which was feasible to be used for the second trial. The second trial was conducted on 14 students from class XII MIPA 4 at one of Gresik's senior high schools. Furthermore, the results of the second trial that were analyzed based on Table 2 so that was obtained an overview for each tested students' level of understanding on Electromagnetic Induction concepts.
The analysis was carried out to get an overview of students' level of understanding using the five-level conception diagnostic test are (a) calculating the percentage of the internal validity using Equation (3) with standards are very small if a value between 0 and 20, and so on as shown in Table 6  validity of the content was explained in the sub-section "Instrument and Procedure". Meanwhile, the empirical external validity of the construct was classified as very small if the Pearson product moment correlation coefficient (rxy) is calculated using Equation (6) = 0.000 -0.199, and so on for the other range of values in Table 7 (Sugiyono, 2015). Next step is calculating the reliability test with standards that are very small if the calculation (r 11 ) yields Equation (7) = -1.000-0.199, and so on for the remaining value ranges in Table 8 (Sugiyono,  2015). Furthermore, a five-level conception diagnostic test (consisting of 11 questions) which was validated and reliable therefore it can be used to test students' level of understanding. The students' level of understanding was determined based on a combination of students' answer on test results according to Table 2 and was explained in the section "Introduction".

RESULTS AND DISCUSSION Writing of the five-level conception diagnostic test
As already mentioned, 11 questions of the five-level conception diagnostic test were successfully written by the Authors and stated feasible. One of the 11 questions is given in Table 1.

Validity and reliability test
Summary of the internal validity tests, the external validity and the reliability results, consecutively are given in Tables 9-12. Note: The Indicators at 1(a)-1(d), 2(a)-2(f), 3(a)-3(c) are described in the method of the internal validity assessment section. Assessment rubrics can be seen in Table 5. The internal validity standard can be seen in Table 6, V1 = Lecturer Validator 1, and V2 = Lecturer validator 2.
Based on the data in Table 9, it can be seen that the average percentage of the internal validity of Draft 1 is 92% so it is classified as very valid in terms of content, construct and language. Draft 1 became Draft 2, which was further tested for the external validity and the reliability.  6  1  0  7  2  1  8  2  3  9  3  0  10  0  8  11  2  3  Total  13 18 Percentage (%) 3.94 5.45 Note: TF = True Fake is calculated using Equation (4) and WF = Wrong Fake is calculated using Equation (5)  Table 10 states that the values %TF is 3.94% and %WF is 5.45%, where both values are less than 10% (<10%). Thus, the Draft 2 is categorized as valid in terms of empirical content. Table 11 states that the external validity test results in terms of empirical construct, Draft 2 (consisting of 11 questions) is also categorized as valid because of the value of rxy > rtable. The rtable value is determined based on the number of respondents that were 30 students (df = 28) with a significance level of 0.05 for the one-way test so that rtable = 0.3061 (Sugiyono, 2015). Based on the data in Table 12, it was found that the value of r11 > rtable so that Draft 2 was declared reliable (Arikunto, 2016). Furthermore, Draft 2 becomes the Final Draft which is ready to be tested in the second trial, considering that the results of the validity test are valid and reliable.

Trial 2
Table 13 provides an overview of the 14 tested students' level of understanding and their answers to 11 questions consisting 6 sub-concepts. For cells marked in red boxes were examples of students' answers in Table 14.    Based on the data in Table 13, it can be seen that the most dominant student number 1's level of understanding is Really Understood (RU) and Lack of Knowledge (LC) with the same percentage of 36.36%. The student experienced a Really Understood (RU) in subconcepts B, D and F. While in sub-concepts B, D and F, the student is still having difficulties. Furthermore, the most dominant student number 2's level of understanding is Really Understood (RU) of 63.64% in sub-concepts B, C and 7. Meanwhile, students Number 3 and 8 experienced a Lack of Knowledge (LC) with the same percentage of 54 ,55% in subconcepts A, B, D and F.
The most dominant student number 4's level of understanding is Really Understood (RU) with a percentage of 36.36% in sub-concepts B, C and D. However, in other subconcepts (A, E and F) this student experienced a Partial Understood (PU), Lack of Knowledge (LC), Not Understood (NU) and Misconception (MC) that occur due to Humanistic Thinking (HT). Furthermore, students Numbers 6, 7 and 9 also dominantly experienced a Really Understood (RU) with a successive percentage of 45.45%; 54.55% and 54.55%. These students experienced a Really Understood (RU) in sub-concepts D, E and F for the 6 th student, sub-concepts C, D and E for the 7 th student and sub-concepts B, C, D, and F for the 9 th student.
Meanwhile, the most dominant student number 5's level of understanding is a Lack of Knowledge (LC) with a percentage of 45.45% in sub-concepts A, B, D and E. However, in sub-concepts C and F, this student experienced a Really Understood (RU), No Understood (NU) or Misconceptions (MC) that occur due to Preconception (Pr) and Humanistic Thinking (HT).
The next students who were dominantly experienced a Lack of Knowledge (LC) are students Numbers 10, 11 and 12 with a percentage of 45.45% each; 63.64% and 45.45%. The three students experienced a Lack of Knowledge (LC) in different sub-concepts, i.e., subconcepts A, B, D and F for the 10 th student,, sub-concepts B, D, E for the 11 th student and sub-concepts A, B, C and D for the 12 th student. Furthermore, students Number 13 and 14 also dominant experienced a Lack of Knowledge (LC) with a percentage of 72.73% and 63.64%, respectively. The 13 th student experienced a Lack of Knowledge (LC) in subconcepts A, B, D and F. However, the 14 th student experienced a Lack of Knowledge (LC) in all sub-concepts (A, B. C, D, E and F ).
Among 14 students who were tested using the 11 questions, it can be conclude that most of the levels of understanding were categorized as Lack of Knowledge (LC) with a percentage of 44.16%. Meanwhile, the level of understanding of Really Understood (RU), Partial Understood (PU), Misconceptions (MC), No Understood (NU) and Undefined (UD) are 33.77%; 9.09%; 7.14%; 4.55% and 1.30%, respectively.  Figure 3 provides examples of several tested students' levels of understanding based on the combination of answers that given at question Number 2. The question Number 2 is one of the sub-concepts of the Electromotive Force (EMF) induced by changes in the magnetic field, where students were asked to determine the direction of the deviation needle on the galvanometer when a magnetic rod is moved towards one end of the coil as shown in Figure  4. It can be seen that the students' answer at the fifth level, students who experienced a Lack of Knowledge (LC) answered that when the magnetic rod was moved towards one end of the coil, there was an increase in the number of magnetic force lines around the coil. Thus, the rotation direction of the needle on the galvanometer deviates to the left. Meanwhile, the answer of students at the fifth level who experienced a Misconception (MC) due to Associative Thinking (AT) was the number of magnetic force lines surrounded the coil is constant. Thus, the rotation direction of the needle on the galvanometer moves to the left. The answer for students at the fifth level who experienced a Partial Understood (PU) was that when a magnetic rod was moved towards one end of the coil, the number of magnetic force lines increased so that there was an electric current.
Meanwhile, according to the concepts of Physics, "In Figure 5, a current is induced when a magnet is moved towards one end of the coil ( ⃗ ⃗ increases momentarily through the coil) and the direction of the needle on the galvanometer deviates to the right (Giancoli, 2014). Relevant research in identifying students' understanding of Electromagnetic Induction concepts was also conducted by Yustiandi & Saepuzaman (2017), namely "Identification of Difficulties in Learning the Concepts of Electromagnetic Induction in High School" by using a diagnostic test in the form of multiple-choice of 10 questions and interviews. The results showed that students had difficulty in determining the direction of the induced current in the coil (only 6.7% of students answered correctly), determining the direction of the induced current in the loop (only 10% of students answered correctly) and difficulties in understanding the Electromotive Force (only 13.3 % of students answered correctly).
A similar study which also identified students' understanding of Electromagnetic Induction concepts was conducted by Wulandari (2017), namely "Analysis of Mastery of Electromagnetic Induction Concepts in Class XII High School Students". The research method used is by using a diagnostic test in the form of a description of 8 questions. The research shows that 35.33% of students are classified as lacking in mastering the concepts of Self Inductance, 36.16% of students lacking in understanding the concepts of applying the concepts of Electromagnetic Induction in technology and 47.34% of students lacking in understanding the concepts of induced EMF and Faraday's law.

CONCLUSION
The results of the writing of a five-level conception diagnostic test on Electromagnetic Induction concepts of 11 questions has been validated and reliable showed that the diagnostic test of conception is feasible for use. In addition, the diagnostic test of conception was successfully identified the students' level of understanding on Electromagnetic Induction concepts. Among 14 students who were tested for their level of understanding, 44.16% of them experienced a Lack of Knowledge.