Next Article in Journal
Tetrahedral Mobile Robots: A Systematic Literature Review
Previous Article in Journal
Naked-Eye Detection of Water Contaminants Enabled by Engineered Plasmonic Resonance in Hybrid Detector Systems
Previous Article in Special Issue
Augmented Reality in Cardiovascular Education (HoloHeart): Assessment of Students’ and Lecturers’ Needs and Expectations at Heidelberg University Medical School
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 15
Issue 18
10.3390/app15189976
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
This is an early access version, the complete PDF, HTML, and XML versions will be available soon.
Article

Expanding the Team: Integrating Generative Artificial Intelligence into the Assessment Development Process

by
Toni A. May
1,*,
Kathleen Provinzano
1,
Kristin L. K. Koskey
2,
Connor J. Sondergeld
3,
Gregory E. Stone
4,
James N. Archer
5 and
Naorah Rimkunas
1
1
College of Community and Public Affairs, SUNY-Binghamton University, Binghamton, NY 13902, USA
2
RAE Group, Medina, OH 44256, USA
3
MetriKs Amerique, Oswego, NY 13126, USA
4
Judith Herb College of Arts, Social Sciences and Education, University of Toledo, Toledo, OH 43606, USA
5
IBM Corp., Armonk, NY 10504, USA
*
Author to whom correspondence should be addressed.
Appl. Sci. 2025, 15(18), 9976; https://doi.org/10.3390/app15189976
Submission received: 4 August 2025 / Revised: 6 September 2025 / Accepted: 7 September 2025 / Published: 11 September 2025
(This article belongs to the Special Issue Emerging Trends in Artificial Intelligence and Computer Science for E-Learning)
Versions Notes

Abstract

Effective assessment development requires collaboration between multidisciplinary team members, and the process is often time-intensive. This study illustrates a framework for integrating generative artificial intelligence (GenAI) as a collaborator in assessment design, rather than a fully automated tool. The context was the development of a 12-item multiple-choice test for social work interns in a school-based training program, guided by design-based research (DBR) principles. Using ChatGPT to generate draft items, psychometricians refined outputs through structured prompts and then convened a panel of five subject matter experts to evaluate content validity. Results showed that while most AI-assisted items were relevant, 75% required modification, with revisions focused on response option clarity, alignment with learning objectives, and item stems. These findings provide initial evidence that GenAI can serve as a productive collaborator in assessment development when embedded in a human-in-the-loop process, while underscoring the need for continued expert oversight and further validation research.
Keywords: assessment development; AI-assisted item writing; content validity; humans-in-the-loop; design-based research (DBR) assessment development; AI-assisted item writing; content validity; humans-in-the-loop; design-based research (DBR)

Share and Cite

MDPI and ACS Style

May, T.A.; Provinzano, K.; Koskey, K.L.K.; Sondergeld, C.J.; Stone, G.E.; Archer, J.N.; Rimkunas, N. Expanding the Team: Integrating Generative Artificial Intelligence into the Assessment Development Process. Appl. Sci. 2025, 15, 9976. https://doi.org/10.3390/app15189976

AMA Style

May TA, Provinzano K, Koskey KLK, Sondergeld CJ, Stone GE, Archer JN, Rimkunas N. Expanding the Team: Integrating Generative Artificial Intelligence into the Assessment Development Process. Applied Sciences. 2025; 15(18):9976. https://doi.org/10.3390/app15189976

Chicago/Turabian Style

May, Toni A., Kathleen Provinzano, Kristin L. K. Koskey, Connor J. Sondergeld, Gregory E. Stone, James N. Archer, and Naorah Rimkunas. 2025. "Expanding the Team: Integrating Generative Artificial Intelligence into the Assessment Development Process" Applied Sciences 15, no. 18: 9976. https://doi.org/10.3390/app15189976

APA Style

May, T. A., Provinzano, K., Koskey, K. L. K., Sondergeld, C. J., Stone, G. E., Archer, J. N., & Rimkunas, N. (2025). Expanding the Team: Integrating Generative Artificial Intelligence into the Assessment Development Process. Applied Sciences, 15(18), 9976. https://doi.org/10.3390/app15189976

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop