Interrater Reliability of the Occupational Therapy Anticipatory Awareness Test: A Performance-Based Cognitive Assessment

Background: Functional cognitive impairments caused by acquired brain injury (ABI) negatively impact an individual’s daily functioning. Impaired self-awareness can pose obstacles to task execution and participation. Traditional assessment methods for self-awareness lack a functional approach, highlighting the need for performance-based assessments such as the Occupational Therapy Anticipatory Awareness Test (OTAAT). The purpose of this study was to establish the interrater reliability of the OTAAT in adults with ABI. Methods: This was a two-phase study. Phase One involved the recruitment of three participants with ABI, who then underwent OTAAT administration within a setting of their choosing. OTAAT performances were recorded. In Phase Two, ten occupational therapists assessed the Phase One participants’ performances using the OTAAT. Phase Two’s raters’ outcomes were obtained by research personnel. These data were then collected and analyzed by research staff to measure interrater reliability using the Intraclass Correlation Coefficient (ICC) via IBM Statistical Package for the Social Sciences (SPSS) software. A two-way random effect, absolute agreement, multiple-rater/measurement ICC was adopted for this study. Results: Data analysis demonstrated strong interrater reliability for the OTAAT, demonstrating its consistency to measure self-awareness in individuals with ABI across different raters. Specifically, the ICC values indicated strong agreement among raters in their assessment of participants’ self-awareness performance using the OTAAT. Conclusion: The OTAAT has strong interrater reliability and holds promise as a valuable addition to neurorehabilitation practice. This study demonstrates the reliability of the OTAAT as a tool for assessing self-awareness in the ABI population.