Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model
Abstract
:1. Introduction
2. Research Design
2.1. Data and Sample
2.2. The Process for Measuring Course Learning Outcome (CLO) Using Rasch Model
- is the probability of turn of the event upon the interaction between the relevant person and assessment item;
- = Euler’s number, (i.e., 2.71828)
- = The ability of person
- = the difficulty of assessment item
2.3. Empirical Model
3. Empirical Results and Analysis
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Bai, X.; Xu, Y.; Ikem, F. Rubric and performance-based assessment. Issues Inf. Syst. 2013, 14, 1–11. [Google Scholar]
- Bloom, B.S. Taxonomy of educational objectives. Cognitive domain. N. Y. McKay 1956, 1, 20–24. [Google Scholar]
- Podolsky, A.; Kini, T.; Darling-Hammond, L. Does teaching experience increase teacher effectiveness? A review of US research. J. Prof. Cap. Community 2019, 4, 286–308. [Google Scholar] [CrossRef]
- Van de Grift, W.; Helms-Lorenz, M.; Maulana, R. Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement. Stud. Educ. Eval. 2014, 43, 150–159. [Google Scholar] [CrossRef]
- Van der Lans, R.M.; Van de Grift, W.J.C.M.; Van Veen, K.; Fokkens Bruinsma, M. Once is not enough: Establishing reliability criteria for feedback and evaluation decisions based on classroom observations. Stud. Educ. Eval. 2016, 50, 88–95. [Google Scholar] [CrossRef]
- Bradley, K.D.; Cunningham, J.; Haines, R.T.; Harris, W.E., Jr.; Mueller, C.E.; Royal, K.D.; Shannon, O.S.; Gilbert, S.; Weber, J. Constructing and Evaluating Measures: Applications of the Rasch Measurement Model; University of Kentucky: Lexington, KY, USA, 2010. [Google Scholar]
- Farhan, M.; Aslam, M.; Jabbar, S.; Khalid, S. Multimedia based qualitative assessment methodology in eLearning: Student teacher engagement analysis. Multimed. Tools Appl. 2018, 77, 4909–4923. [Google Scholar] [CrossRef]
- Rasch, G. Probabilistic Models for Some Intelligence and Achievement Tests; MESA Press: Chicago, IL, USA, 1960. [Google Scholar]
- Rasch, G. On general laws and the meaning of measurement in psychology. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 20–30 June 1960; University of California Press: Berkeley, CA, USA, 1961; Volume 4, pp. 321–333. [Google Scholar]
- Wright, B.D.; Mok, M. Understanding Rasch measurement: Rasch models overview. J. Appl. Meas. 2000, 1, 83–106. [Google Scholar] [PubMed]
- Bond, T.G.; Fox, C.M. Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 3rd ed.; Routledge: Oxford, UK, 2015. [Google Scholar]
- Kalinowski, K.; Krenczyk, D.; Paprocka, I.; Kempa, W.; Grabowik, C. Multi-criteria evaluation methods in the production scheduling. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Orlando, FL, USA, 2016; Volume 145, p. 022019. [Google Scholar]
- Bonk, W.J.; Ockey, G.J. A many-facet Rasch analysis of the second language group oral discussion task. Lang. Test. 2003, 20, 89–110. [Google Scholar] [CrossRef]
- Semerci, Ç. The relationships between achievement focused motivation and critical thinking. Afr. J. Bus. Manag. 2011, 5, 61–79. [Google Scholar]
- Chang, M.L.; Engelhard, G., Jr. Examining the Teachers’ Sense of Efficacy Scale at the item level with Rasch measurement model. J. Psychoeduc. Assess. 2016, 34, 177–191. [Google Scholar] [CrossRef] [Green Version]
- Cetin, B.; Ilhan, M. An analysis of rater severity and leniency in open-ended mathematic questions rated through standard rubrics and rubrics based on the SOLO taxonomy. Egit. Bilim 2017, 42. [Google Scholar] [CrossRef] [Green Version]
- Kaya-Uyanik, G.; Gur-Erdogan, D.; Canan-Gungoren, O. International Journal of Educational Methodology. IJEM 2019, 5, 407. [Google Scholar]
- Chae, J.; Cho, Y.; Lee, M.; Lee, S.; Choi, M.; Park, S. Design and implementation of a system for creating multimedia linked data and its applications in education. Multimed. Tools Appl. 2016, 75, 13121–13134. [Google Scholar] [CrossRef]
- Rozeha, A.R.; Azami, Z.; Mohd Saidfudin, M. Application of Rasch Measurement in Evaluation of Learning Outcomes: A case study in electrical engineering. In Proceedings of the Regional Conference on Engineering Mathematics, Mechanics, Manufacturing & Architecture (EM3ARC), Kuala Lumpur, Malaysia, 27–28 November 2007. [Google Scholar]
- Azrilah, A.A.; Azlinah, M.; Azami, Z.; Sohaimi, Z.; Hamzah, A.G.; Saidfudin, M. Evaluation of Information Professionals Competency Face Validity Test Using Rasch Model. 5th WSEAS. In Proceedings of the IASME International Conference on Engineering Education (EE’08), Stevens Point, WI, USA, 27–31 July 2008; pp. 22–24. [Google Scholar]
- Abdullah, S.; Rahmat, R.A.A.O.; Zaharim, A.; Muhamad, N.; Deros, B.M.; Kofli, N.T.; Tahir, M.M.; Muchtar, A.; Azhari, C.H.; Azhari, C.H.; et al. Implementing continual review of programme educational objectives and outcomes for OBE Curriculum based on stakeholders’ input. Eur. J. Sci. Res. 2009, 29, 89–99. [Google Scholar]
- Talib, A.M.; Alomary, F.O.; Alwadi, H.F. Assessment of Student Performance for Course Examination Using Rasch Measurement Model: A Case Study of Information Technology Fundamentals Course. Educ. Res. Int. 2018, 1–8. [Google Scholar] [CrossRef] [Green Version]
- Van de Grift, W.J.; Houtveen, T.A.; van den Hurk, H.T.; Terpstra, O. Measuring teaching skills in elementary education using the Rasch model. Sch. Eff. Sch. Improv. 2019, 30, 455–486. [Google Scholar] [CrossRef] [Green Version]
(a) | ||
Course Learning Outcomes (CLO) for Communications and Networks Engineering Course (CME322)—Network Design and Analysis | ||
Course Learning Outcome | Bloom taxonomy | |
CLO1 | Describe network technologies such as Ethernet, Virtual local area networks, wireless local area networks, mobility management principles, and mobile Internet Protocol. | Knowledge |
CLO2 | Describe routing principles and illustrate routing algorithms such as link-state and distance-vector. | Knowledge |
CLO3 | Explain different type of delay, loss, and throughput, and recognise different type of network switching mechanisms such as packet- and circuit-switching. | Skills |
CLO4 | Explain transport layer connection/connectionless services, Transport Control Protocol (TCP) reliable data transfer, TCP flow-control and TCP congestion-control mechanisms. | Skills |
CLO5 | Demonstrate and apply error detection and correction schemes, channel access mechanisms and, data centre design and operation. | Competence |
(b) | ||
Course Learning Outcomes (CLO) for Accounting Course ACC102—Introduction to Managerial Accounting | ||
Course Learning Outcome | Bloom taxonomy | |
CLO1 | Describe the basic management accounting concepts and techniques. | Knowledge |
CLO2 | Determine the cost of a manufactured product using job order and process costing systems. | Knowledge |
CLO3 | Explain the purposes of budgeting and prepare the master budget components and relate the budget to planning and control. | Skills |
CLO4 | Apply break-even techniques in CVP analysis. | Skills |
CLO5 | Apply and justify relevant techniques to aid internal users in decision making. | Competence |
CLO6 | Demonstrate oral and written communication skills in evaluating different approaches to management accounting. | Competence |
(a) | ||||||
Percentage Distribution according to Course Learning Outcomes (CLO) for Communications and Networks Engineering Course (CME322)—Network Design and Analysis | ||||||
Evaluation | Quiz (10%) | Mid-term 1 (20%) | Mid-term 2 (20%) | Assignment (10%) | Final Exam (40%) | Total (100%) |
CLO1 | 0.35 | 0.80 | 0.00 | 0.00 | 0.125 | 0.245 |
CLO2 | 0.35 | 0.20 | 0.00 | 0.00 | 0.175 | 0.145 |
CLO3 | 0.30 | 0.00 | 0.55 | 0.00 | 0.20 | 0.22 |
CLO4 | 0.00 | 0.00 | 0.45 | 0.00 | 0.25 | 0.19 |
CLO5 | 0.00 | 0.00 | 0.00 | 1.00 | 0.25 | 0.20 |
Check | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
(b) | ||||||
Percentage Distribution according to Course Learning Outcomes (CLO) for Accounting Course ACC102—Introduction to Managerial Accounting | ||||||
Evaluation | Quiz (10%) | Mid-term 1 (20%) | Mid-term 2 (20%) | Assignment (10%) | Final Exam (40%) | Total (100%) |
CLO1 | 0.20 | 0.00 | 0.00 | 0.00 | 0.10 | 0.06 |
CLO2 | 0.50 | 0.35 | 0.00 | 0.00 | 0.20 | 0.20 |
CLO3 | 0.15 | 0.65 | 0.00 | 0.00 | 0.20 | 0.225 |
CLO4 | 0.15 | 0.00 | 1.00 | 0.00 | 0.25 | 0.315 |
CLO5 | 0.00 | 0.00 | 0.00 | 0.00 | 0.25 | 0.10 |
CLO6 | 0.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.10 |
Check | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
(a) | |||||||
Marks Distribution according to Course Learning Outcomes (CLO) for Communications and Networks Engineering Course (CME322). | |||||||
Student (S) | CLO1 | CLO2 | CLO3 | CLO4 | CLO4 | CLO5 | |
S1 | 67 | 83 | 54 | 56 | 86 | 67 | |
S2 | 80 | 80 | 73 | 93 | 72 | 80 | |
S3 | 92 | 79 | 85 | 91 | 79 | 92 | |
S4 | 75 | 87 | 87 | 82 | 75 | 75 | |
S5 | 75 | 85 | 84 | 77 | 90 | 75 | |
S6 | 96 | 95 | 79 | 78 | 54 | 96 | |
S7 | 71 | 77 | 79 | 91 | 90 | 71 | |
S8 | 84 | 96 | 82 | 83 | 93 | 84 | |
S9 | 78 | 89 | 82 | 70 | 85 | 78 | |
S10 | 90 | 85 | 80 | 75 | 65 | 90 | |
S11 | 77 | 73 | 75 | 88 | 86 | 77 | |
(b) | |||||||
Marks Distribution according to Course Learning Outcomes (CLO) for Accounting Course (ACC102). | |||||||
Student (S) | CLO1 | CLO2 | CLO3 | CLO4 | CLO4 | CLO5 | CLO6 |
S1 | 52 | 59 | 72 | 78 | 72 | 59 | 52 |
S2 | 56 | 63 | 77 | 84 | 77 | 63 | 56 |
S3 | 57 | 64 | 78 | 85 | 78 | 64 | 57 |
S4 | 52 | 59 | 72 | 78 | 72 | 59 | 52 |
S5 | 49 | 55 | 67 | 73 | 67 | 55 | 49 |
S6 | 61 | 68 | 84 | 91 | 84 | 68 | 61 |
S7 | 54 | 60 | 74 | 80 | 74 | 60 | 54 |
S8 | 36 | 41 | 50 | 54 | 50 | 41 | 36 |
S9 | 74 | 83 | 92 | 92 | 92 | 83 | 74 |
S10 | 50 | 56 | 68 | 74 | 68 | 56 | 50 |
S11 | 53 | 59 | 73 | 79 | 73 | 59 | 53 |
S12 | 64 | 72 | 88 | 96 | 88 | 72 | 64 |
S13 | 76 | 86 | 95 | 95 | 95 | 86 | 76 |
S14 | 72 | 81 | 99 | 90 | 99 | 81 | 72 |
S15 | 64 | 72 | 88 | 96 | 88 | 72 | 64 |
S16 | 61 | 68 | 84 | 91 | 84 | 68 | 61 |
S17 | 60 | 68 | 83 | 90 | 83 | 68 | 60 |
S18 | 58 | 65 | 79 | 86 | 79 | 65 | 58 |
S19 | 68 | 77 | 94 | 85 | 94 | 77 | 68 |
S20 | 72 | 81 | 90 | 90 | 90 | 81 | 72 |
(a) | |||||
Logit Value for Each Student for Communications and Networks Engineering Course (CME322). | |||||
Entry Number | Total Score | Total Count | Measure | Model S. E. | Student Identification |
8 | 25 | 5 | 3.72 | 1.89 | S8 |
2 | 23 | 5 | 1.55 | 0.79 | S2 |
3 | 23 | 5 | 1.55 | 0.79 | S3 |
4 | 23 | 5 | 1.55 | 0.79 | S4 |
5 | 23 | 5 | 1.55 | 0.79 | S5 |
9 | 23 | 5 | 1.55 | 0.79 | S9 |
7 | 22 | 5 | 1.04 | 0.64 | S7 |
10 | 22 | 5 | 1.04 | 0.64 | S10 |
11 | 22 | 5 | 1.04 | 0.64 | S11 |
6 | 20 | 5 | 0.41 | 0.5 | S6 |
1 | 17 | 5 | −0.23 | 0.44 | S1 |
Mean | 0.79 | ||||
Standard Deviation | 0.37 | ||||
(b) | |||||
Logit Value for Each Student for Accounting Course (ACC102). | |||||
Entry Number | Total Score | Total Count | Measure | Model S. E. | Student Identification |
9 | 29 | 6 | 51.81 | 5.93 | S9 |
13 | 29 | 6 | 51.81 | 5.93 | S13 |
14 | 29 | 6 | 51.81 | 5.93 | S14 |
20 | 29 | 6 | 51.81 | 5.93 | S20 |
12 | 26 | 6 | 35.83 | 5.92 | S12 |
15 | 26 | 6 | 35.83 | 5.92 | S15 |
19 | 26 | 6 | 35.83 | 5.92 | S19 |
6 | 24 | 6 | 27.32 | 3.09 | S6 |
16 | 24 | 6 | 27.32 | 3.09 | S16 |
17 | 24 | 6 | 27.32 | 3.09 | S17 |
2 | 21 | 6 | 18.81 | 3.7 | S2 |
3 | 21 | 6 | 18.81 | 3.7 | S3 |
7 | 21 | 6 | 18.81 | 3.7 | S7 |
18 | 21 | 6 | 18.81 | 3.7 | S18 |
1 | 18 | 6 | 10.47 | 2.88 | S1 |
4 | 18 | 6 | 10.47 | 2.88 | S4 |
11 | 18 | 6 | 10.47 | 2.88 | S11 |
10 | 16 | 6 | 4.43 | 2 | S10 |
5 | 15 | 6 | 0.27 | 2.45 | S5 |
8 | 8 | 6 | −22.03 | 4.02 | S8 |
Mean | 4.13 | ||||
Standard Deviation | 1.39 |
(a) | |||||
Logit Value for each Course Learning Outcome (CLO) for Communications and Networks Engineering Course (CME322). | |||||
Entry Number | Total Score | Total Count | Measure | Model S. E. | CLO |
5 | 47 | 11 | 0.35 | 0.41 | CLO5 |
1 | 48 | 11 | 0.18 | 0.43 | CLO1 |
3 | 48 | 11 | 0.18 | 0.43 | CLO3 |
4 | 48 | 11 | 0.18 | 0.43 | CLO4 |
2 | 52 | 11 | −0.88 | 0.64 | CLO2 |
Mean | 48.6 | 0.47 | |||
Standard Deviation | 1.7 | 0.08 | |||
(b) | |||||
Logit Value for each Course Learning Outcomes (CLO) for Accounting Course (ACC102). | |||||
Entry Number | Total Score | Total Count | Measure | Model S. E. | CLO |
1 | 51 | 20 | 21.09 | 1.55 | CLO1 |
2 | 64 | 20 | 11.94 | 2 | CLO2 |
6 | 64 | 20 | 11.94 | 2 | CLO6 |
3 | 86 | 20 | −11.96 | 1.99 | CLO3 |
5 | 86 | 20 | −11.96 | 1.99 | CLO5 |
4 | 92 | 20 | −21.04 | 2.1 | CLO4 |
Mean | 73.83 | 1.94 | |||
Standard Deviation | 14.90 | 0.18 |
(a) | |||||||
Probability of Each Student to Achieve Each Course Learning Outcomes (CLO) for Communications and Networks Engineering Course (CME322). | |||||||
Probability of Success | CLO5 | CLO1 | CLO3 | CLO4 | CLO2 | ||
S8 | 0.815 | 0.812 | 0.812 | 0.812 | 0.777 | ||
S2 | 0.594 | 0.589 | 0.589 | 0.589 | 0.537 | ||
S3 | 0.594 | 0.589 | 0.589 | 0.589 | 0.537 | ||
S4 | 0.594 | 0.589 | 0.589 | 0.589 | 0.537 | ||
S5 | 0.594 | 0.589 | 0.589 | 0.589 | 0.537 | ||
S9 | 0.594 | 0.589 | 0.589 | 0.589 | 0.537 | ||
S7 | 0.557 | 0.552 | 0.552 | 0.552 | 0.500 | ||
S10 | 0.557 | 0.552 | 0.552 | 0.552 | 0.500 | ||
S11 | 0.557 | 0.552 | 0.552 | 0.552 | 0.500 | ||
S6 | 0.522 | 0.517 | 0.517 | 0.517 | 0.465 | ||
S1 | 0.507 | 0.502 | 0.502 | 0.502 | 0.450 | ||
(b) | |||||||
Probability of Each Student to Achieve Each Course Learning Outcomes (CLO) for Accounting Course (ACC102). | |||||||
Probability of Success | CLO1 | CLO2 | CLO6 | CLO3 | CLO5 | CLO4 | |
S9 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S13 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S14 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S20 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S12 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S15 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S19 | 0.99 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | |
S6 | 0.82 | 0.75 | 0.75 | 0.75 | 0.75 | 0.73 | |
S16 | 0.82 | 0.75 | 0.75 | 0.75 | 0.75 | 0.73 | |
S17 | 0.82 | 0.75 | 0.75 | 0.75 | 0.75 | 0.73 | |
S2 | 0.90 | 0.85 | 0.85 | 0.85 | 0.85 | 0.83 | |
S3 | 0.90 | 0.85 | 0.85 | 0.85 | 0.85 | 0.83 | |
S7 | 0.90 | 0.85 | 0.85 | 0.85 | 0.85 | 0.83 | |
S18 | 0.90 | 0.85 | 0.85 | 0.85 | 0.85 | 0.83 | |
S1 | 0.79 | 0.71 | 0.71 | 0.71 | 0.71 | 0.69 | |
S4 | 0.79 | 0.71 | 0.71 | 0.71 | 0.71 | 0.69 | |
S11 | 0.79 | 0.71 | 0.71 | 0.71 | 0.71 | 0.69 | |
S10 | 0.61 | 0.50 | 0.50 | 0.50 | 0.50 | 0.48 | |
S5 | 0.71 | 0.61 | 0.61 | 0.61 | 0.61 | 0.59 | |
S8 | 0.92 | 0.88 | 0.88 | 0.88 | 0.88 | 0.87 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nasralla, M.M.; Al-Shattarat, B.; Almakhles, D.J.; Abdelhadi, A.; Abowardah, E.S. Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model. Electronics 2021, 10, 727. https://doi.org/10.3390/electronics10060727
Nasralla MM, Al-Shattarat B, Almakhles DJ, Abdelhadi A, Abowardah ES. Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model. Electronics. 2021; 10(6):727. https://doi.org/10.3390/electronics10060727
Chicago/Turabian StyleNasralla, Moustafa M., Basiem Al-Shattarat, Dhafer J. Almakhles, Abdelhakim Abdelhadi, and Eman S. Abowardah. 2021. "Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model" Electronics 10, no. 6: 727. https://doi.org/10.3390/electronics10060727
APA StyleNasralla, M. M., Al-Shattarat, B., Almakhles, D. J., Abdelhadi, A., & Abowardah, E. S. (2021). Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model. Electronics, 10(6), 727. https://doi.org/10.3390/electronics10060727