Home⇛Journal of Interdisciplinary Perspectives⇛vol. 2 no. 11 (2024)

Ensuring Validity and Reliability in Algebra Midterm Assessment: A Comprehensive Approach to Test Development and Analysis

Matthew E. Cañeda | Arl Joshua F. Gamaya | Manuelin C. Baring

Discipline: Education

 

Abstract:

First-year mathematics education students often face challenges with College and Advanced Algebra assessments. This study aimed to develop and validate a 100-item multiple-choice midterm test for College and Advanced Algebra, ensuring the test's validity and reliability. The test was designed following a structured process that included creating a Table of Specifications (TOS) based on the approved syllabus. To ensure content accuracy and relevance, the test was reviewed by three subject matter experts and evaluated for clarity by 15 students. Pilot testing was conducted with 82 fourth-year Bachelor of Secondary Education Mathematics (BSEd Mathematics) students. The pilot test results underwent detailed item analysis, focusing on metrics such as the difficulty index, discrimination index, and overall reliability using the Kuder-Richardson Formula 20 (KR-20). Of the 100 items, 22 were retained, 48 were revised, and 30 were discarded. The test achieved a reliability coefficient of 0.876, indicating strong internal consistency. The findings suggest that the validated test questionnaire is a dependable tool for accurately assessing students’ knowledge in College and Advanced Algebra, providing valuable feedback for educators and students.



References:

  1. Aguhayon, H., Tingson, R., & Pentang, J. (2023). Addressing students learning gaps in mathematics through differentiated instruction. International Journal of Educational Management and Development Studies, 4(1), 69–87. https://doi.org/10.53378/352967
  2. Anderson, L., & Krathwohl, D.A. (2001). Taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.
  3. Andreev, I. (2024). Bloom’s Taxonomy. Retrieved from https://tinyurl.com/42pmmk9b
  4. Biddix, J.P. (2018). Research methods and applications for student affairs. John Wiley & Sons.
  5. Bilyakovska, O. (2022). Test as an effective means of assessing the quality of students’ knowledge. Academic Notes Series Pedagogical Science, 1(204), 16–20. https://doi.org/10.36550/2415-7988-2022-1-204-16-20
  6. Bobbitt, Z. (2022). Kuder-Richardson Formula 20 (Definition & example). Retrieved from  https://www.statology.-org/kuder-richardson-20/
  7. Cañeda, M.E., Amar, R.P., & Lucin, E.L. (2024a). Development of test questionnaire on selected topics in calculus 1 (final term). International Journal of Research and Scientific Innovation, 9(8), 244-255. https://doi.org/10.51244/IJRSI.2024.1108020
  8. Cañeda, M.E., Logroño, J.J.F., & Culibra, C.D. (2024b). Test questionnaire development on selected topics in calculus 1. Ignatian International Journal for Multidisciplinary Research, 2(8), 1363–1376. https://doi.org/-10.5281/zenodo.13371155
  9. Chigonga, B. (2020). Formative Assessment in Mathematics Education in the Twenty-First Century. IntechOpen.
  10. DeVellis, R.F. (2003). Scale development: theory and applications, applied social research methods. Sage Publications.
  11. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.org/10.1016/j.tsc.2013.12.004
  12. Ferrando, P.J., Lorenzo-Seva, U., & Bargalló-Escrivà, M. T. (2023). Gulliksen's pool: A quick tool for preliminary detection of problematic items in item factor analysis. PloS one, 18(8), e0290611. https://doi.org/10.1371/journal.pone.0290611
  13. Gilbert, G.E., & Prion, S. (2016). Making sense of methods and measurement: Lawshe’s content validity index. Clinical Simulation in Nursing, 12(12), 530–531. https://doi.org/10.1016/-j.ecns.2016.08.002
  14. Irwing, P., & Hughes, D.J. (2018). Test development. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 3–47). Wiley Blackwell.
  15. Jain, S., Dubey, S., & Jain, S. (2016). Designing and validation of questionnaire. International Dental & Medical Journal of Advanced Research, 2(1), 1–3. https://doi.org/10.15713/ins.idmjar.39
  16. Jhangiani, R.S., Chiang, I.A.,  Cuttler, C., & Leighton, D.C. (2019). Research methods in psychology – 2nd Canadian edition. KPU.
  17. Kara, F., & Celikler, D. (2015). Development of achievement test: Validity and reliability study for achievement test on matter changing. Journal of Education and Practice, 6(24), 21–26. https://eric.ed.gov/?id=EJ1078816
  18. Kılıç, A. & Uysal, I. (2022). To what extent are item discrimination values realistic? A new index for two-dimensional structures. International Journal of Assessment Tools in Education, 9, 728-740. https://doi.org/10.21449/ijate.1098757
  19. Kilic, S. (2016). Cronbach’s alpha reliability coefficient. Journal of Mood Disorders, 6(1), 47. https://doi.org/10.5455/jmood.20160307122823
  20. Kline, P. (2000). Handbook of psychological testing, second edition. Routledge.
  21. Kunwar, R. (2018). Development and standardization process of mathematics achievement test for the students of grade x. International Journal of Current Research, 10(11), 75451–75455. https://doi.org/10.24941/ijcr.33168.11.2018
  22. Lazarus, S.S., Johnstone, C.J., Liu, K.K., Thurlow, M.L., Hinkle, A.R., & Burden, K. (2022). An updated state guide to universally designed assessments (NCEO Report 431). Retrieved from https://tinyurl.com/y7tf3ty9
  23. Longe, I.O., & Maharaj, A. (2023). Investigating students’ understanding of complex number and its relation to algebraic group using and APOS theory. Journal of Medives : Journal of Mathematics Education IKIP Veteran Semarang, 7(1), 117. https://doi.org/10.31331/medivesveteran.v7i1.2332  
  24. Mamolo, L.A. (2021). Development of an achievement test to measure students' competency in general mathematics. Anatolian Journal of Education, 6(1), 79-90. https://doi.org/10.29333/aje.2021.616a
  25. Metzgar, M. (2023). Revised Bloom’s taxonomy in a principles of Economics textbook. Acta Educationis Generalis, 13(3), 15–28. https://doi.org/10.2478/atd-2023-0019.  
  26. Ocampo, R., & Usita, N. P. (2015). Development of Lubeg (Syzygiumlineatum (Roxb.) Merr.& Perry) processed products. Asia Pacific Journal of Multidisciplinary Research, 3(4), 118-123. https://tinyurl.com/yxb9ccxn
  27. Oducado, R. M. (2020). Survey instrument validation rating scale. Retrieved from https://doi.org/10.2139/ssrn.3789575
  28. Padua, R.N., & Santos, R.G. (1997). Educational evaluation and measurement: Theory, practice, and application. KATHA Publishing: QC.
  29. Patel, N., & Desai, S. (2020). Abc of face validity for questionnaire. International Journal of Pharmaceutical Sciences Review and Research, 65(1), 164–168. https://doi.org/10.47583/ijpsrr.2020.v65i01.025
  30. Quaigrain, K., Arhin, A. K., & King Fai Hui, S. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1). https://doi.org/10.1080/2331186X.2017.1301013
  31. Rezigalla, A. A., Ibrahim, E. K., & ElHussein, A. B. (2019). Item Analysis: The impact of distractor efficiency on the discrimination power of multiple choice items. Retrieved from https://doi.org/10.21203/rs.2.15899/v1
  32. Stephens, A., Blanton, M., Knuth, E., Isler, I., & Gardiner, A. M. (2015). Just say yes to early algebra! Teaching Children Mathematics, 22(2), 92–101. https://doi.org/10.5951/teacchilmath.22.2.0092
  33. Sullivan G. M. (2011). A primer on the validity of assessment instruments. Journal of graduate medical education, 3(2), 119–120. https://doi.org/10.4300/JGME-D-11-00075.1            
  34. Syahfitri, J., Firman, H., Redjeki, S., & Srivati, S. (2019). Development and validation of critical thinking disposition test in Biology. International Journal of Instruction, 12(4), 381-392. https://doi.org/10.29333-/iji.2019.12425a
  35. Taherdoost, H. (2016). Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. International Journal of Academic Research in Management, 5(3), 28-36. http://dx.doi.org/10.2139/ssrn.3205040
  36. Tejeda, K., & Gallardo, G. (2017). Performance assessment on high school advanced algebra. International Electronic Journal of Mathematics Education, 12(3), 777-798. https://doi.org/10.29333/iejme/648
  37. Wilson, L.O. (2016). Anderson and Krathwohl: Bloom’s taxonomy revised. Retrieved from https://tinyurl.com/4s9vhnee
  38. Yaddanapudi, S., & Yaddanapudi, L.N. (2019). How to design questionnaires. Indian Journal of Anaesthesia, 63(5), 335–337. https://doi.org/10.4103/ija.IJA_274_19 

Tools


Copyright © 2024  KITE Digital Educational Solutions |  Exclusively distributed by CE-Logic