Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course
Abstract
:1. Introduction
2. Materials and Methods
2.1. Design and Instrumentation
2.2. Sample
2.3. Analysis
3. Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Peile, E. Knowing and knowing about. BMJ 2006, 332, 645. [Google Scholar] [CrossRef] [PubMed]
- McCoubrie, P. Improving the fairness of multiple-choice questions: A literature review. Med. Teach. 2004, 26, 709–712. [Google Scholar] [CrossRef] [PubMed]
- Van Der Vleuten, C.P. The assessment of professional competence: Developments, research and practical implications. Adv. Health Sci. Educ. Theory Pract. 1996, 1, 41–67. [Google Scholar] [CrossRef] [PubMed]
- Dickinson, J.R. How many options do multiple-choice questions really have? Dev. Bus. Simul. Exp. Learn. 2013, 40, 171–175. [Google Scholar]
- Cizek, G.J.; Robinson, L.K.; O’Day, D.M. Non-functioning options: A closer look. Educ. Psychol. Meas. 1998, 58, 605–611. [Google Scholar] [CrossRef]
- DiBattista, D.; Kurzawa, L. Examination of the quality of multiple-choice items on classroom tests. Can. J. Scholarsh. Teach. Learn. 2011, 2, 4. [Google Scholar] [CrossRef]
- Haladyna, T.M.; Downing, S.M. A taxonomy of multiple-choice item-writing rules. Appl. Meas. Educ. 1989, 2, 37–50. [Google Scholar] [CrossRef]
- Sadeghi, K.; Masoumi, G.A. Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences. J. Engl. Lang. Teach. Learn. 2017, 19, 123–143. [Google Scholar]
- Shizuka, T.; Takeuchi, O.; Yashima, T.; Yoshizawa, K. A comparison of three- and four-option English tests for university entrance selection purposes in Japan. Lang. Test. 2006, 23, 35–57. [Google Scholar] [CrossRef] [Green Version]
- Tarrant, M.; Ware, J. A comparison of the psychometric properties of three- and four-option multiple-choice questions in nursing assessments. Nurse Educ. Today 2010, 30, 539–543. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Vyas, R.; Supe, A. Multiple choice questions: A literature review on the optimal number of options. Natl. Med. J. India 2008, 21, 130–133. [Google Scholar] [PubMed]
- Landrum, R.E.; Cashin, J.R.; Theis, K.S. More evidence in favor of three-option multiple choice tests. Educ. Psychol. Meas. 1993, 53, 771–778. [Google Scholar] [CrossRef]
- Rodriguez, M.C. Three options are optimal for multiple choice items: A meta analysis of 80 years of research. Educ. Meas. Issues Pract. 2005, 24, 3–13. [Google Scholar] [CrossRef]
- Royal, K.D. Robust (and ethical) education research designs. J. Vet. Med. Educ. 2018, 45, 11–15. [Google Scholar] [CrossRef] [PubMed]
- Deepak, K.K.; Al-Umran, K.U.; AI-Sheikh, M.H.; Dkoli, B.V.; Al-Rubaish, A. Psychometrics of multiple choice questions with non-functioning distracters: Implications to medical education. Indian J. Physiol. Pharmacol. 2015, 59, 428–435. [Google Scholar] [PubMed]
- Dehnad, A.; Nasser, H.; Hosseini, A.F. A comparison between three-and four-option multiple choice questions. Procedia Soc. Behav. Sci. 2014, 98, 398–403. [Google Scholar] [CrossRef]
- Vegada, B.; Shukla, A.; Khilnani, A.; Charan, J.; Desai, C. Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study. Indian J. Pharmacol. 2016, 48, 571–575. [Google Scholar] [CrossRef] [PubMed]
- Redmond, S.P.; Hartigan-Rogers, J.A.; Cobbett, S. High time for a change: Psychometric analysis of multiple-choice questions in nursing. Int. J. Nurs. Educ. Scholarsh. 2012, 9. [Google Scholar] [CrossRef] [PubMed]
- Tarrant, M.; Ware, J.; Mohammed, A.M. An assessment of functioning and non-functioning distractors in multiple-choice questions: A descriptive analysis. BMC Med. Educ. 2009, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tarrant, M.; Ware, J. A framework for improving the quality of multiple-choice assessments. Nurse Educ. 2012, 37, 98–104. [Google Scholar] [CrossRef] [PubMed]
- Brown, W. Some experimental results in the correlation of mental abilities. Br. J. Psychol. 1910, 3, 296–322. [Google Scholar]
- Spearman, C. Correlation calculated from faulty data. Br. J. Psychol. 1910, 3, 271–295. [Google Scholar] [CrossRef]
- Royal, K.D.; Stockdale, M.R. The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations. J. Adv. Med. Educ. Prof. 2017, 5, 84–89. [Google Scholar] [PubMed]
- Messick, S. Validity of Psychological Assessment: Validation of Inferences from Persons’ Responses and Performances as Scientific Inquiry into Score Meaning. Am. Psychol. 1995, 50, 741–749. [Google Scholar] [CrossRef]
Item | Form A | Form B | p-Value Δ (Absolute) | DI Δ (Absolute) | ||||
---|---|---|---|---|---|---|---|---|
# Options | p-Value | DI | # Options | p-Value | DI | |||
1 | 4 | 1.00 | 1.00 | 3 | 0.96 | 1.02 | 0.04 | 0.02 |
2 | 4 | 0.94 | 1.05 | 3 | 0.94 | 0.94 | 0.00 | 0.11 |
3 | 4 | 0.71 | 1.21 | 3 | 0.74 | 1.17 | 0.04 | 0.04 |
4 | 4 | 0.92 | 1.01 | 3 | 0.90 | 0.92 | 0.02 | 0.09 |
5 | 4 | 0.98 | 1.05 | 3 | 1.00 | 1.00 | 0.02 | 0.05 |
6 | 4 | 0.93 | 0.96 | 3 | 0.94 | 1.01 | 0.01 | 0.05 |
7 | 4 | 0.90 | 0.97 | 3 | 0.95 | 0.98 | 0.04 | 0.01 |
8 | 4 | 0.79 | 1.00 | 3 | 0.88 | 0.97 | 0.09 | 0.03 |
9 | 4 | 0.75 | 0.97 | 3 | 0.87 | 1.04 | 0.12 | 0.07 |
10 | 4 | 0.91 | 1.02 | 3 | 0.90 | 1.08 | 0.01 | 0.06 |
11 | 4 | 0.96 | 1.03 | 3 | 0.98 | 1.00 | 0.02 | 0.03 |
12 | 4 | 0.53 | 1.19 | 3 | 0.42 | 0.60 | 0.11 | 0.59 |
13 | 4 | 0.46 | 1.40 | 3 | 0.79 | 1.08 | 0.33 | 0.32 |
14 | 4 | 0.92 | 0.93 | 3 | 1.00 | 1.00 | 0.08 | 0.07 |
15 | 4 | 0.97 | 0.92 | 3 | 0.99 | 0.99 | 0.02 | 0.07 |
16 | 4 | 0.60 | 1.18 | 3 | 0.82 | 1.02 | 0.23 | 0.16 |
17 | 3 | 0.87 | 0.98 | 4 | 0.90 | 0.96 | 0.03 | 0.02 |
18 | 3 | 0.91 | 1.05 | 4 | 0.91 | 1.00 | 0.00 | 0.05 |
19 | 3 | 0.89 | 0.92 | 4 | 0.92 | 0.98 | 0.03 | 0.06 |
20 | 3 | 0.93 | 1.07 | 4 | 0.86 | 1.14 | 0.07 | 0.07 |
21 | 3 | 0.99 | 1.02 | 4 | 0.95 | 0.88 | 0.04 | 0.14 |
22 | 3 | 0.87 | 0.97 | 4 | 0.86 | 1.04 | 0.01 | 0.07 |
23 | 3 | 0.69 | 1.26 | 4 | 0.67 | 1.35 | 0.02 | 0.09 |
24 | 3 | 0.94 | 0.95 | 4 | 0.97 | 1.02 | 0.03 | 0.07 |
25 | 3 | 0.92 | 0.93 | 4 | 0.96 | 0.99 | 0.03 | 0.06 |
26 | 3 | 0.99 | 1.07 | 4 | 0.98 | 0.99 | 0.01 | 0.08 |
27 | 3 | 0.75 | 1.10 | 4 | 0.75 | 1.17 | 0.00 | 0.07 |
28 | 3 | 0.47 | 0.75 | 4 | 0.57 | 0.89 | 0.10 | 0.14 |
29 | 3 | 0.95 | 1.00 | 4 | 0.90 | 1.07 | 0.05 | 0.07 |
30 | 3 | 0.56 | 1.43 | 4 | 0.55 | 0.98 | 0.01 | 0.45 |
Item | Form A | Form B | p-Value Δ (Absolute) | DI Δ (Absolute) | ||||
---|---|---|---|---|---|---|---|---|
# Options | p-Value | DI | # Options | p-Value | DI | |||
1 | 4 | 0.96 | 1.11 | 3 | 0.96 | 1.06 | 0.00 | 0.05 |
2 | 4 | 0.77 | 0.85 | 3 | 0.73 | 0.97 | 0.04 | 0.12 |
3 | 4 | 0.94 | 0.97 | 3 | 0.96 | 0.86 | 0.01 | 0.11 |
4 | 4 | 0.99 | 0.96 | 3 | 0.99 | 1.09 | 0.00 | 0.13 |
5 | 4 | 0.99 | 1.07 | 3 | 0.96 | 1.10 | 0.03 | 0.03 |
6 | 4 | 0.91 | 1.06 | 3 | 0.98 | 1.00 | 0.07 | 0.06 |
7 | 4 | 0.87 | 1.14 | 3 | 0.86 | 1.01 | 0.00 | 0.13 |
8 | 4 | 0.66 | 0.56 | 3 | 0.60 | 0.49 | 0.06 | 0.07 |
9 | 4 | 0.74 | 0.77 | 3 | 0.81 | 0.96 | 0.06 | 0.19 |
10 | 4 | 0.97 | 1.00 | 3 | 0.99 | 1.03 | 0.01 | 0.03 |
11 | 4 | 0.98 | 0.97 | 3 | 0.98 | 1.00 | 0.00 | 0.03 |
12 | 4 | 0.80 | 1.33 | 3 | 0.74 | 0.90 | 0.06 | 0.43 |
13 | 4 | 0.94 | 1.00 | 3 | 0.99 | 1.00 | 0.05 | 0.00 |
14 | 4 | 0.97 | 1.10 | 3 | 0.96 | 1.01 | 0.01 | 0.09 |
15 | 4 | 0.97 | 0.98 | 3 | 0.98 | 1.05 | 0.01 | 0.07 |
16 | 4 | 0.91 | 1.04 | 3 | 0.97 | 0.96 | 0.06 | 0.08 |
17 | 4 | 0.85 | 0.99 | 3 | 0.95 | 1.08 | 0.10 | 0.09 |
18 | 4 | 0.77 | 0.86 | 3 | 0.91 | 0.95 | 0.13 | 0.09 |
19 | 3 | 0.99 | 1.00 | 4 | 0.98 | 1.04 | 0.01 | 0.04 |
20 | 3 | 1.00 | 1.00 | 4 | 0.95 | 1.08 | 0.05 | 0.08 |
21 | 3 | 0.74 | 0.96 | 4 | 0.74 | 0.87 | 0.01 | 0.09 |
22 | 3 | 1.00 | 1.00 | 4 | 0.96 | 1.10 | 0.04 | 0.10 |
23 | 3 | 0.82 | 0.91 | 4 | 0.82 | 1.05 | 0.00 | 0.14 |
24 | 3 | 0.99 | 1.01 | 4 | 0.97 | 0.99 | 0.02 | 0.02 |
25 | 3 | 0.64 | 1.50 | 4 | 0.58 | 1.09 | 0.05 | 0.41 |
26 | 3 | 0.84 | 1.11 | 4 | 0.89 | 0.96 | 0.05 | 0.15 |
27 | 3 | 0.88 | 0.81 | 4 | 0.91 | 0.94 | 0.03 | 0.13 |
28 | 3 | 0.99 | 0.93 | 4 | 0.97 | 1.08 | 0.02 | 0.15 |
29 | 3 | 0.92 | 1.10 | 4 | 0.89 | 1.23 | 0.03 | 0.13 |
30 | 3 | 1.00 | 1.00 | 4 | 1.00 | 1.00 | 0.00 | 0.00 |
31 | 3 | 1.00 | 1.00 | 4 | 0.97 | 0.94 | 0.03 | 0.06 |
32 | 3 | 0.81 | 0.97 | 4 | 0.82 | 0.77 | 0.01 | 0.20 |
33 | 3 | 0.95 | 1.00 | 4 | 0.99 | 1.02 | 0.04 | 0.02 |
34 | 3 | 0.99 | 0.98 | 4 | 1.00 | 1.00 | 0.01 | 0.02 |
35 | 3 | 0.67 | 0.53 | 4 | 0.64 | 0.68 | 0.02 | 0.15 |
36 | 3 | 0.95 | 0.98 | 4 | 0.99 | 0.99 | 0.04 | 0.01 |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Royal, K.; Dorman, D. Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course. Vet. Sci. 2018, 5, 55. https://doi.org/10.3390/vetsci5020055
Royal K, Dorman D. Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course. Veterinary Sciences. 2018; 5(2):55. https://doi.org/10.3390/vetsci5020055
Chicago/Turabian StyleRoyal, Kenneth, and David Dorman. 2018. "Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course" Veterinary Sciences 5, no. 2: 55. https://doi.org/10.3390/vetsci5020055