Next Article in Journal
Evaluation and Management of Recurrent Atrial Flutter in Neonates
Previous Article in Journal
CGM-Based Glycemic Metrics Support Estimating Nutritional Risk After Total Pancreatectomy: An Exploratory Retrospective Study
Previous Article in Special Issue
Comparison of the Kynurenine/Tryptophan Ratio with the Beck Suicide Intent Scale in Patients Admitted to the Emergency Department Due to Suicide Attempt
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 14
Issue 19
10.3390/jcm14197125
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Content Validity Assessment of a Newly Developed Emergency Medical Dispatch and Triage Protocol in Thailand

by
Thongpitak Huabbangyang
1,
Duangpon Thepmanee
1,
Phudit Buaprasert
2,
Pit Chansomboon
3,
Jiraporn Sri-on
4 and
Rapeeporn Rojsaengroeng
4,*
1
Department of Disaster and Emergency Medical Operation, Faculty of Science and Health Technology, Navamindradhiraj University, Bangkok 10300, Thailand
2
Department of Emergency Medicine, Emory University School of Medicine, Atlanta, GA 30322, USA
3
Department of Emergency Medicine, Udonthani Hospital, Udonthani 41000, Thailand
4
Department of Emergency Medicine, Faculty of Medicine, Vajira Hospital, Navamindradhiraj University, Bangkok 10300, Thailand
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2025, 14(19), 7125; https://doi.org/10.3390/jcm14197125 (registering DOI)
Submission received: 21 August 2025 / Revised: 7 September 2025 / Accepted: 13 September 2025 / Published: 9 October 2025
(This article belongs to the Special Issue Advancements in Emergency Medicine Practices and Protocols)
Versions Notes

Abstract

Background/Objectives: Accurate telephone triage of emergency medical cases plays a crucial role in improving outcomes for critically ill patients. Effective triage enables emergency medical dispatchers to provide appropriate pre-arrival instructions and to deploy operational units according to the patient’s severity level. This study aimed to develop and assess the content validity of the Emergency Medical Triage Protocol and Criteria-Based Dispatch Code (EMTP-CBDC) for Thailand. The objective was to ensure the tool’s content accuracy and applicability in prioritizing emergency responses in line with medical urgency, considering global changes and universal standards. Methods: A cross-sectional descriptive study was conducted from 15–30 April 2024. The content validity of the newly developed EMTP-CBDC, comprising 30 symptom groups, was evaluated by five emergency physician experts with at least 1 year of experience in emergency medical oversight. The assessment focused on four aspects: relevance, clarity, simplicity, and ambiguity. The Content Validity Index (CVI) was calculated at both the item level (I-CVI) and the scale level using the average index (S-CVI/Ave). To adjust for chance agreement, the probability of chance agreement (Pc) and the modified kappa coefficient (k*) were calculated for each item. Results: The content validation revealed I-CVI values ranging from 0.80 to 1.00 across all items. The S-CVI/Ave scores were 0.97 for relevance, 0.93 for clarity, 0.98 for simplicity, and 0.94 for ambiguity. These values surpassed the accepted thresholds for content validity. Conclusions: The EMTP-CBDC developed for Thailand demonstrated good content validity across relevance, clarity, simplicity, and ambiguity. Further studies are needed to establish its reliability and field performance before routine implementation.

1. Introduction

The enhancement of efficient and effective emergency medical services (EMSs) requires a systematic approach aligned with the core principles of emergency medicine, as represented by the Star of Life [1,2]. One of the essential components of EMS is the emergency medical dispatcher (EMD), who conducts telephone triage using standardized emergency medical triage protocols. Several high-income countries have developed and implemented comprehensive triage systems to support accurate dispatch and pre-arrival instructions (PAIs). For example, the Netherlands uses the Dutch Field Triage Protocol (National Protocol of Ambulance Services), developed in accordance with the American College of Surgeons Committee on Trauma (ACS-COT) guidelines [3]. Similarly, the United States [4,5], the United Kingdom [6,7], and Australia [8,9] employ the Medical Priority Dispatch System (MPDS), which includes triage codes for 33 symptom groups.
A systematic review evaluating the effectiveness of MPDS in EMS systems found that its implementation was associated with improved patient outcomes, particularly through reduced response times [10]. In Norway, the Norwegian Index for Medical Dispatch is used to evaluate and triage patients [11,12], while Japan utilizes protocols issued by the Fire and Disaster Management Agency [13,14]. In Thailand, the original criteria-based dispatch (CBD) codes covered 26 symptom groups [15]. These systems provide EMDs with the ability to offer PAIs—basic, evidence-based medical instructions to callers before emergency responders arrive on the scene [4,5,6,7,8,9,11,12,13,14,15]. A previous population-based retrospective cohort study demonstrated that a combination of EMD-reported information and MPDS codes effectively predicted the severity of traffic injuries, resulting in significantly more accurate ambulance dispatch [16].
In Thailand, the Emergency Medical Act B.E. 2551 governs the structure and operation of emergency services, including oversight by the Emergency Medical Committee and the National Institute for Emergency Medicine (NIEMS) [17]. Under Section 28(1) of this Act, hospitals and EMS units are required to triage and respond to emergency cases based on medical urgency using established emergency medical triage protocols and CBD codes. These tools aim to ensure that patients receive timely and appropriate care based on urgency, in line with national policy and international standards.
Since 2009, Thailand has piloted a CBD system comprising 25 symptom groups, which was later integrated into the Information Technology for Emergency Medical System Version 4.0 [18]. In 2025, the EMS database was upgraded to the Intelligence Digital Emergency Medical Services Platform, allowing EMDs to deliver symptom-specific pre-arrival assistant instruction PAIs [19]. Since 2011, NIEMS has issued CBD codes for each symptom group, categorizing patients by urgency levels—red, yellow, green, and white—to facilitate appropriate resource deployment. However, updates to the CBD system have not kept pace with advances in medical knowledge, evolving emergency conditions, or technological innovations such as automated PAI systems.
Thailand’s emergency operational units are categorized into three tiers—basic, advanced, and specialized [20]. During the COVID-19 pandemic, beginning in 2020, the government designated COVID-19 as an endemic disease [21]. NIEMS subsequently updated the CBD system, adding COVID-19 as the 26th symptom group in March 2022. Despite this addition, challenges in the application of the CBD system persisted, and user feedback indicated that the protocol did not fully meet current needs. To address these gaps, NIEMS appointed a panel of experts to revise the emergency medical triage protocol and CBD system, building upon the foundation established in 2013 and updated in 2021. These efforts aim to align the system with contemporary clinical and technological standards while ensuring usability for frontline personnel.
The present study was conducted to improve and validate the content accuracy of Thailand’s newly revised Emergency Medical Triage Protocol and Criteria-Based Dispatch Code (EMTP-CBDC). The objective was to ensure that the revised system provides accurate, up-to-date guidance for EMDs, supports effective triage based on urgency, and aligns with international best practices.

2. Materials and Methods

This cross-sectional descriptive study was conducted from 15 to 30 April 2024. Data collection and reporting followed the Standards for the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [22]. Data were collected from a panel of five emergency physicians selected using a purposive sampling method based on their qualifications and expertise in emergency medicine.
Participants were required to hold a diploma in emergency medicine, have experience in prehospital emergency care, be at least 18 year of age, and have a minimum of 1 year of experience in emergency medical oversight. Physicians who were unwilling or declined to complete the evaluation form were excluded from the study.
The newly developed and improved EMTP-CBDC of Thailand, comprising 30 symptom groups created by the NIEMS, along with a Content Validity Index (CVI) evaluation form, was sent to the experts via email. The evaluation form was administered through Google Forms and assessed four aspects: relevance, clarity, simplicity, and ambiguity, each rated on a 4-point scale: relevance: 1 = not relevant, 2 = needs revision, 3 = relevant with minor revision, 4 = very relevant; clarity: 1 = not clear, 2 = needs revision, 3 = clear with minor revision, 4 = very clear; simplicity: 1 = not simple, 2 = needs revision, 3 = simple with minor revision, 4 = very simple; and ambiguity: 1 = doubtful, 2 = needs revision, 3 = no doubt but minor revision needed, 4 = meaning is clear [23].
After evaluation, the item-level CVI (I-CVI) was calculated for each aspect. Items with a mean I-CVI of ≥ 0.80 were considered valid. Items scoring below this threshold were revised by the authors and re-evaluated by the same panel until they met the validity criterion [24]. In addition to numeric ratings, all experts were invited to provide qualitative comments for each item. Items that received an I-CVI < 1.00 or were flagged by experts as unclear, complex, or ambiguous were revised by the research team according to the feedback received. The revised items were then resubmitted to the same panel of experts for re-evaluation until the predetermined threshold of I-CVI ≥ 0.80 was achieved. This iterative process ensured that high CVI values reflected consensus after revision rather than initial agreement alone.
The equations used to calculate the item-level content validity index (I-CVI), the scale-level index (S-CVI/Ave), the probability of chance agreement (Pc), and the modified kappa coefficient (k*), as well as the bootstrap procedure for estimating the 95% confidence interval of S-CVI/Ave, are presented in Appendix A. Data were recorded in Microsoft Excel for further analysis.
Variable definition: The EMTP-CBDC refers to a tool developed by NIEMS for classifying emergency symptoms and dispatching appropriate EMSs. It includes a structured introduction, chief complaint and injury categories, and telephone guidance for critical, life-threatening conditions.
As the primary objective was to validate the EMTP-CBDC using CVI analysis, a sample size of five experts was deemed appropriate based on previous literature recommending a minimum of five experts for such validation studies [23]. Thus, five participants were purposively selected. Content accuracy was assessed using CVI analysis, including both item-level CVI (I-CVI) and the scale-level average index (S-CVI/Ave). To adjust for chance agreement, the probability of chance agreement (Pc) and the modified kappa coefficient (k*) were calculated for each item. The interpretation of k* followed established thresholds (≥0.74 = excellent, 0.60–0.73 = good, 0.40–0.59 = fair, <0.40 = poor) [25]. In addition, 95% confidence intervals (CIs) for S-CVI/Ave were estimated using nonparametric bootstrap resampling with 10,000 iterations. Statistical analysis was performed using IBM SPSS Statistics for Windows, Version 28.0 (IBM Corp., Armonk, NY), and custom scripts for bootstrap calculations.

3. Results

3.1. Relevance

The evaluation of content accuracy in terms of relevance for the EMTP-CBDC in Thailand showed that the I-CVI values ranged from 0.80 to 1.00 across all 30 items. Items with I-CVI = 1.00 corresponded to unanimous expert agreement (A = 5), yielding Pc = 0.0313 and k* = 1.00 (excellent). Items with I-CVI = 0.80 (A = 4) had Pc = 0.1563 and k* = 0.763, also interpreted as excellent. Overall, the S-CVI/Ave was 0.97 (95% CI: 0.95–0.99), indicating excellent content validity for relevance (Table 1; Table A1).

3.2. Clarity

The evaluation of content accuracy in terms of clarity for the EMTP-CBDC in Thailand showed that the I-CVI values also ranged from 0.80 to 1.00. For unanimous items, k* = 1.00 (excellent), and for items with four of five experts agreeing, k* = 0.763 (excellent). The S-CVI/Ave was 0.93 (95% CI: 0.90–0.97). While slightly lower than the other domains, this result still demonstrated good to excellent clarity (Table 2; Table A2).

3.3. Simplicity

The evaluation of content accuracy in terms of simplicity for the EMTP-CBDC in Thailand showed that the I-CVI values were between 0.80 and 1.00, with corresponding Pc and k* values consistent with the patterns above (Pc = 0.0313, k* = 1.00 for unanimous agreement; Pc = 0.1563, k* = 0.763 for four-out-of-five agreement). The S-CVI/Ave was 0.98 (95% CI: 0.95–1.00), confirming excellent overall simplicity of the items (Table 3; Table A3).

3.4. Ambiguity

The evaluation of content accuracy in terms of ambiguity for the EMTP-CBDC in Thailand showed that the I-CVI values again ranged from 0.80 to 1.00, with Pc and k* identical to the values reported above depending on whether four or five experts agreed. The S-CVI/Ave was 0.94 (95% CI: 0.91–0.97), indicating that the items were sufficiently unambiguous and meeting the predetermined acceptance criteria (Table 4; Table A4).

4. Discussion

This study found that the newly improved EMTP-CBDC, Thailand, developed by the NIEM, demonstrated acceptable content validity across all evaluated aspects—relevance, clarity, simplicity, and unambiguity—according to expert assessments. The evaluation encompassed 30 symptom groups and aligned with the development guidelines of the emergency medical system based on the Star of Life principles, which emphasize systematic dispatch and triage beginning with the emergency call via 1669 [1,2]. The findings support the applicability of this updated tool in the Thai context, particularly during a time when the EMSs face evolving challenges. For example, in response to emerging diseases such as COVID-19, the EMS in Thailand added a 26th symptom group, CBC, to its existing classification system in 2022 [22]. This underscores the need to modernize dispatch tools to accommodate dynamic emergency scenarios. When compared with international triage systems—such as the MPDS used in the United States [4,5], England [6,7], and Australia [8,9]—the EMTP-CBDC shows promise in its alignment with global best practices, including the use of symptom group codes and basic telephone instructions by trained personnel (e.g., emergency physicians, paramedics, emergency nurse practitioners, or EMDs).
A previous study involving 43 EMS systems in China reported that incorporating MPDS led to improved diagnostic concordance for acute coronary syndrome between dispatch and on-scene treatment and reduced the time from emergency call to EMS arrival from 20.0 to 16.0 min (p < 0.001) [24,26]. Similarly, research on Osaka’s EMS system demonstrated that accurate telephone triage using clearly defined symptom groups reduced unnecessary ambulance usage (from 7.4% to 3.7%) and facilitated more efficient EMS resource allocation through concise PAIs [27].
These findings are consistent with international evidence indicating that accurate, content-validated tools, such as MPDS and the Norwegian Index, help reduce EMS response times and improve dispatch accuracy according to patient severity levels [10,11,12,16]. Although Thailand’s current CBD tool has been in use for over 14 years, it has limitations in terms of modernity and adaptability. This study represents a crucial step toward developing more efficient, standardized tools for national EMS operations and offers a foundation for future improvements.
This study had several important limitations. First, the sample size consisted of only five experts, which may not fully represent the diversity of users at various levels of emergency operations. Additionally, all participants were emergency physicians, which could introduce sample bias and limit the perspectives obtained from other EMS professionals, such as emergency medical dispatchers (EMDs), emergency nurse practitioners, paramedics, and call-takers. The decision to restrict the panel to emergency physicians was made by the development committee of the EMTP-CBDC, appointed by the National Institute for Emergency Medicine (NIEMS) of Thailand, which served as the policy-setting authority and the funding agency for this research. The committee recommended that the initial validation stage be conducted with emergency physicians due to their central role in setting national prehospital standards.
Second, this study focused solely on content validity and did not employ a Delphi consensus process, reliability testing, or field validation. The absence of Delphi rounds and field testing limited the opportunity to refine the tool through iterative consensus and to evaluate its performance in real-world dispatch environments.
Third, the entire validation process was conducted over only two weeks, as determined by the development committee under NIEMS. This limited timeframe was due to administrative and logistical constraints, and although experts were able to complete their evaluations electronically and multiple rounds of revision were undertaken, the short duration may have restricted the thoroughness of the process.
Therefore, future research should incorporate a broader range of stakeholders, extend the validation period, and use methods such as Delphi consensus rounds and field testing to strengthen the tool’s psychometric properties, usability, and generalizability.

5. Conclusions

The newly developed EMTP-CBDC for Thailand demonstrated good content validity in terms of relevance, clarity, simplicity, and completeness, based on expert evaluation. While these findings support the tool’s potential suitability for implementation in national dispatch centers and alignment with the needs of a modern EMS system, additional validation is required. In particular, reliability measures (inter-rater and test–retest) and field performance indicators, including dispatch accuracy, adherence to PAIs, and patient outcomes, remain to be established in future studies. To ensure optimal performance in practice, the tool should be tested under operational conditions, undergo continuous review, and be accompanied by structured training programs for call takers and EMDs to promote correct and consistent application.

Author Contributions

Conceptualization, T.H., D.T., P.B., P.C., J.S.-o. and R.R.; methodology, T.H.; software, T.H.; validation, T.H.; formal analysis, T.H.; investigation, T.H. and R.R.; resources, T.H.; data curation, T.H.; writing—original draft preparation, T.H.; writing—review and editing, T.H.; visualization, T.H.; supervision, T.H. and R.R.; project administration, T.H.; funding acquisition, T.H. All authors have read and agreed to the published version of the manuscript.

Funding

The study was funded by Navamindradhiraj University, which played no role in the study design, data collection, or data analysis or in writing the manuscript.

Institutional Review Board Statement

The ethical consideration of the study protocol was granted by at the Institutional Review Board of the Faculty of Medicine, Vajira Hospital, Navamindradhiraj University (COA No. 071/2567; 1 April 2024). The committee reviewed the study protocol, methodology, and potential ethical implications to ensure participant protection, privacy, and compliance with ethical guidelines.

Informed Consent Statement

Written informed consent was obtained from all participants prior to their enrollment in the study, including consent for publication. As all participants were board-certified emergency physicians serving as expert evaluators, the consent process was designed to formally document their voluntary participation, agreement to review the study materials, and permission to use their anonymized evaluations for research purposes. All data were treated with strict confidentiality to ensure participant privacy. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Data Availability Statement

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Acknowledgments

The authors wish to express their sincere gratitude to the National Institute for Emergency Medicine for funding the development of the EMTP-CBDC, and to the Navamindradhiraj University Research Fund for supporting this research.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

I-CVIItem-Level Content Validity Index
S-CVIScale-Level Content Validity Index
CVIContent Validity Index
EMTP-CBDCEmergency Medical Triage Protocol and Criteria-Based Dispatch Code
EMSsemergency medical services
PAIspre-arrival instructions
ACS-COTAmerican College of Surgeons Committee on Trauma
MPDSMedical Priority Dispatch System
CBDcriteria-based dispatch
EMDsemergency medical dispatchers
NIEMSNational Institute for Emergency Medicine
Pcprobability of chance agreement
k*modified kappa
95%CI95% confidence interval

Appendix A

Item-level CVI (I-CVI): I-CVI_j = A_j/N
where A_j is the number of experts rating item j as relevant, and N = 5.
Scale-level CVI: S-CVI = Σ 1(I-CVI_j = 1)/k
S-CVI/Ave = (1/k) Σ I-CVI_j
where k is the total number of items.
Probability of chance agreement (Pc): Pc = C(N, A) × (0.5^N)
where C(N, A) is the binomial coefficient.
Modified kappa (k*): k* = (I-CVI − Pc)/(1 − Pc)
95% CI for S-CVI/Ave (Bootstrap method): S-CVI/Ave = (1/k) Σ I-CVI_j
Bootstrap procedure:
(1)
Sample k items with replacement from {I-CVI_1,…,I-CVI_k}.
(2)
Compute S-CVI/Ave for each resample.
(3)
Repeat B = 10,000 times.
(4)
Take 2.5th and 97.5th percentiles of bootstrap distribution as the 95% CI.

Appendix B

Table A1. Relevance.
Table A1. Relevance.
ItemA (out of 5)I-CVIPck*Interpretation
1. Abdominal pain51.000.031251.000Excellent
2. Anaphylaxis/Allergic reaction51.000.031251.000Excellent
3. Animal bite51.000.031251.000Excellent
4. Back pain (non-trauma)40.800.156250.763Excellent
5. Bleeding disorder (non-trauma)40.800.156250.763Excellent
6. Breathing difficulty51.000.031251.000Excellent
7. Cardiac arrest/Death51.000.031251.000Excellent
8. Chest discomfort51.000.031251.000Excellent
9. Choking51.000.031251.000Excellent
10. Convulsion51.000.031251.000Excellent
11. Chemical hazard51.000.031251.000Excellent
12. Diabetes mellitus51.000.031251.000Excellent
13. Eye problems51.000.031251.000Excellent
14. Fever51.000.031251.000Excellent
15. Headache40.800.156250.763Excellent
16. Overdose/Poison51.000.031251.000Excellent
17. Psychiatry/Behavior change/Suicide51.000.031251.000Excellent
18. Pregnancy/Childbirth51.000.031251.000Excellent
19. Pediatric emergency51.000.031251.000Excellent
20. Sick person/Unspecify40.800.156250.763Excellent
21. Stroke51.000.031251.000Excellent
22. Unconsciousness/Fainting51.000.031251.000Excellent
23. Assault/Sexual assault51.000.031251.000Excellent
24. Burn/Blast51.000.031251.000Excellent
25. Drowning/Diving/Water-related injury51.000.031251.000Excellent
26. Electrocution51.000.031251.000Excellent
27. Falls51.000.031251.000Excellent
28. Heat/Cold exposure 51.000.031251.000Excellent
29. Stab/Gunshot/Penetrating trauma51.000.031251.000Excellent
30. Traffic incidents/Vehicle in water51.000.031251.000Excellent
Abbreviations: A = number of experts rating relevant; Pc = probability of chance agreement; k* = modified kappa; I-CVI = Item-Level Content Validity Index.
Table A2. Clarity.
Table A2. Clarity.
ItemA (out of 5)I-CVIPck*Interpretation
1. Abdominal pain51.000.031251.000Excellent
2. Anaphylaxis/Allergic reaction51.000.031251.000Excellent
3. Animal bite40.800.156250.763Excellent
4. Back pain (non-trauma)40.800.156250.763Excellent
5. Bleeding disorder (non-trauma)40.800.156250.763Excellent
6. Breathing difficulty51.000.031251.000Excellent
7. Cardiac arrest/Death51.000.031251.000Excellent
8. Chest discomfort40.800.156250.763Excellent
9. Choking51.000.031251.000Excellent
10. Convulsion51.000.031251.000Excellent
11. Chemical hazard51.000.031251.000Excellent
12. Diabetes mellitus51.000.031251.000Excellent
13. Eye problems51.000.031251.000Excellent
14. Fever51.000.031251.000Excellent
15. Headache40.800.156250.763Excellent
16. Overdose/Poison51.000.031251.000Excellent
17. Psychiatry/Behavior change/Suicide40.800.156250.763Excellent
18. Pregnancy/Childbirth40.800.156250.763Excellent
19. Pediatric emergency51.000.031251.000Excellent
20. Sick person/Unspecify40.800.156250.763Excellent
21. Stroke51.000.031251.000Excellent
22. Unconsciousness/Fainting40.800.156250.763Excellent
23. Assault/Sexual assault51.000.031251.000Excellent
24. Burn/Blast51.000.031251.000Excellent
25. Drowning/Diving/Water-related injury51.000.031251.000Excellent
26. Electrocution51.000.031251.000Excellent
27. Falls51.000.031251.000Excellent
28. Heat/Cold exposure 51.000.031251.000Excellent
29. Stab/Gunshot/Penetrating trauma51.000.031251.000Excellent
30. Traffic incidents/Vehicle in water40.800.156250.763Excellent
Abbreviations: A = number of experts rating relevant; Pc = probability of chance agreement; k* = modified kappa; I-CVI = Item-Level Content Validity Index.
Table A3. Simplicity.
Table A3. Simplicity.
ItemA (out of 5)I-CVIPck*Interpretation
1. Abdominal pain51.000.031251.000Excellent
2. Anaphylaxis/Allergic reaction51.000.031251.000Excellent
3. Animal bite40.800.156250.763Excellent
4. Back pain (non-trauma)51.000.031251.000Excellent
5. Bleeding disorder (non-trauma)51.000.031251.000Excellent
6. Breathing difficulty51.000.031251.000Excellent
7. Cardiac arrest/Death51.000.031251.000Excellent
8. Chest discomfort51.000.031251.000Excellent
9. Choking51.000.031251.000Excellent
10. Convulsion51.000.031251.000Excellent
11. Chemical hazard51.000.031251.000Excellent
12. Diabetes mellitus51.000.031251.000Excellent
13. Eye problems51.000.031251.000Excellent
14. Fever51.000.031251.000Excellent
15. Headache51.000.031251.000Excellent
16. Overdose/Poison51.000.031251.000Excellent
17. Psychiatry/Behavior change/Suicide51.000.031251.000Excellent
18. Pregnancy/Childbirth51.000.031251.000Excellent
19. Pediatric emergency40.800.156250.763Excellent
20. Sick person/Unspecify51.000.031251.000Excellent
21. Stroke51.000.031251.000Excellent
22. Unconsciousness/Fainting40.800.156250.763Excellent
23. Assault/Sexual assault51.000.031251.000Excellent
24. Burn/Blast51.000.031251.000Excellent
25. Drowning/Diving/Water-related injury51.000.031251.000Excellent
26. Electrocution51.000.031251.000Excellent
27. Falls51.000.031251.000Excellent
28. Heat/Cold exposure 51.000.031251.000Excellent
29. Stab/Gunshot/Penetrating trauma51.000.031251.000Excellent
30. Traffic incidents/Vehicle in water51.000.031251.000Excellent
Abbreviations: A = number of experts rating relevant; Pc = probability of chance agreement; k* = modified kappa; I-CVI = Item-Level Content Validity Index.
Table A4. Ambiguity.
Table A4. Ambiguity.
ItemA (out of 5)I-CVIPck*Interpretation
1. Abdominal pain51.000.031251.000Excellent
2. Anaphylaxis/Allergic reaction51.000.031251.000Excellent
3. Animal bite40.800.156250.763Excellent
4. Back pain (non-trauma)51.000.031251.000Excellent
5. Bleeding disorder (non-trauma)40.800.156250.763Excellent
6. Breathing difficulty51.000.031251.000Excellent
7. Cardiac arrest/Death51.000.031251.000Excellent
8. Chest discomfort40.800.156250.763Excellent
9. Choking51.000.031251.000Excellent
10. Convulsion51.000.031251.000Excellent
11. Chemical hazard40.800.156250.763Excellent
12. Diabetes mellitus51.000.031251.000Excellent
13. Eye problems51.000.031251.000Excellent
14. Fever51.000.031251.000Excellent
15. Headache51.000.031251.000Excellent
16. Overdose/Poison51.000.031251.000Excellent
17. Psychiatry/Behavior change/Suicide51.000.031251.000Excellent
18. Pregnancy/Childbirth40.800.156250.763Excellent
19. Pediatric emergency51.000.031251.000Excellent
20. Sick person/Unspecify51.000.031251.000Excellent
21. Stroke51.000.031251.000Excellent
22. Unconsciousness/Fainting40.800.156250.763Excellent
23. Assault/Sexual assault51.000.031251.000Excellent
24. Burn/Blast40.800.156250.763Excellent
25. Drowning/Diving/Water-related injury51.000.031251.000Excellent
26. Electrocution51.000.031251.000Excellent
27. Falls40.800.156250.763Excellent
28. Heat/Cold exposure 51.000.031251.000Excellent
29. Stab/Gunshot/Penetrating trauma51.000.031251.000Excellent
30. Traffic incidents/Vehicle in water40.800.156250.763Excellent
Abbreviations: A = number of experts rating relevant; Pc = probability of chance agreement; k* = modified kappa; I-CVI = Item-Level Content Validity Index.

References

  1. Lincoln, E.W.; Reed-Schrader, E.; Jarvis, J.L. EMS Quality Improvement Programs. In StatPearls; StatPearls Publishing LLC: Treasure Island, FL, USA, 2024. [Google Scholar]
  2. Haugland, H.; Uleberg, O.; Klepstad, P.; Krüger, A.; Rehn, M. Quality measurement in physician-staffed emergency medical services: A systematic literature review. Int. J. Qual. Health Care 2019, 31, 2–10. [Google Scholar] [CrossRef]
  3. Voskens, F.J.; van Rein, E.A.J.; van der Sluijs, R.; Houwert, R.M.; Lichtveld, R.A.; Verleisdonk, E.J.; Segers, M.; van Olden, G.; Dijkgraaf, M.; Leenen, L.P.H.; et al. Accuracy of prehospital triage in selecting severely injured trauma patients. JAMA Surg. 2018, 153, 322–327. [Google Scholar] [CrossRef]
  4. Olola, C.; Scott, G.; Gardett, I.; Clawson, J.J.; Broadbent, M. Comparison of emergency medical dispatcher stroke identification and paramedic on-scene stroke assessment. Ann. Emerg. Dispatch Response 2017, 5, 6–10. [Google Scholar]
  5. Clawson, J.J.; Scott, G.; Gardett, I.; Youngquist, S.; Taillac, P.; Fivaz, C.; Olola, C. Predictive ability of an emergency medical dispatch stroke diagnostic tool in identifying hospital-confirmed strokes. J. Stroke Cerebrovasc. Dis. 2016, 25, 2031–2042. [Google Scholar] [CrossRef] [PubMed]
  6. Oman, G.; Bury, G. Use of telephone CPR advice in Ireland: Uptake by callers and delays in the assessment process. Resuscitation 2016, 102, 6–10. [Google Scholar] [CrossRef] [PubMed]
  7. Tanner, R.; Masterson, S.; Jensen, M.; Wright, P.; Hennelly, D.; O’Reilly, M.; Murphy, A.W.; Bury, G.; O’Donnell, C.; Deasy, C.; et al. Out-of-hospital cardiac arrests in the older population in Ireland. Emerg. Med. J. 2017, 34, 659–664. [Google Scholar] [CrossRef]
  8. Dicker, B.; Davey, P.; Smith, T.; Beck, B. Incidence and outcomes of out-of-hospital cardiac arrest: A New Zealand perspective. Emerg. Med. Australas. 2018, 30, 662–671. [Google Scholar] [CrossRef] [PubMed]
  9. Nehme, Z.; Andrew, E.; Smith, K. Factors influencing the timeliness of emergency medical service response to time critical emergencies. Prehosp. Emerg. Care 2016, 20, 783–791. [Google Scholar] [CrossRef] [PubMed]
  10. Baabdullah, M.; Faden, H.; Alsubhi, R.; Almalki, A.; Masri, B.; Alharbi, A. The efficiency of the medical priority dispatch system in improving patient outcomes. Saudi J. Emerg. Med. 2020, 1, 110–120. [Google Scholar] [CrossRef]
  11. Ellensen, E.N.; Wisborg, T.; Hunskaar, S.; Zakariassen, E. Dispatch guideline adherence and response interval: A study of emergency medical calls in Norway. BMC Emerg. Med. 2016, 16, 40. [Google Scholar] [CrossRef]
  12. Hoikka, M.; Länkimäki, S.; Silfvast, T.; Ala-Kokko, T.I. Medical priority dispatch codes: Comparison with National Early Warning Score. Scand. J. Trauma Resusc. Emerg. Med. 2016, 24, 142. [Google Scholar] [CrossRef]
  13. Fire and Disaster Management Agency (FDMA), Ministry of Internal Affairs and Communications. Emergency Medical System in Japan; FDMA: Tokyo, Japan, 2020; Available online: https://www.fdma.go.jp/en/post1.html (accessed on 10 February 2024).
  14. Sutham, K.; Khuwuthyakorn, P.; Thinnukool, O. Thailand medical mobile application for patients triage based on criteria-based dispatch protocol. BMC Med. Inform. Decis. Mak. 2020, 20, 66. [Google Scholar] [CrossRef]
  15. Huabbangyang, T.; Sangketchon, C.; Ittiphisit, S.; Uoun, K.; Saumok, C. Predictive factors of outcome in cases of out-of-hospital cardiac arrest due to traffic accident injuries in Thailand: A national database study. Arch. Acad. Emerg. Med. 2022, 10, e64. [Google Scholar]
  16. Ceklic, E.; Ball, S.; Finn, J.; Brown, E.; Brink, D.; Bailey, P.; Whiteside, A.; Brits, R.; Tohira, H. Ambulance dispatch prioritisation for traffic crashes using machine learning: A natural language approach. Int. J. Med. Inform. 2022, 168, 104886. [Google Scholar] [CrossRef]
  17. National Institute for Emergency Medicine. Emergency Medical Act B.E. 2551. Available online: https://www.niems.go.th/1/upload/migrate/file/256004211126059708_buUaNTZDwOtjh84G.pdf (accessed on 10 February 2024).
  18. Information Technology for Emergency Medical System. ITEMS Information. Available online: https://ws.niems.go.th/items_front/CallCenter.aspx (accessed on 10 February 2024).
  19. National Institute for Emergency Medicine. iDEMS. Available online: https://accounts-idems.niems.go.th/realms/EMS-ID/protocol/openid-connect/auth?response_type=code&client_id=portal-management-web&state=S29VYkZmVn5CemZpRGRzV2plMjM1T2Z5WFhhOEVFUnI2dFp1THc4X1Bab2tx&redirect_uri=https%3A%2F%2Femsportal-idems.niems.go.th%2Fpermission-select&scope=openid%20profile%20email&nonce=S29VYkZmVn5CemZpRGRzV2plMjM1T2Z5WFhhOEVFUnI2dFp1THc4X1Bab2tx (accessed on 10 February 2024).
  20. National Institute for Emergency Medicine. Emergency Medical Service Level. Available online: https://www.niems.go.th/1/UploadAttachFile/2023/EBook/417565_20230810140741.pdf (accessed on 10 February 2024).
  21. Huabbangyang, T.; Trakulsrichai, S.; Yuksen, C.; Sricharoen, P. The impact of the coronavirus disease 2019 (COVID-19) pandemic on the use of emergency medical services system in Bangkok, Thailand. Open Access Emerg. Med. 2022, 14, 429–440. [Google Scholar] [CrossRef] [PubMed]
  22. von Elm, E.; Altman, D.G.; Egger, M.; Pocock, S.J.; Gøtzsche, P.C.; Vandenbroucke, J.P. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet 2007, 370, 1453–1457. [Google Scholar] [CrossRef]
  23. Yaghmaie, F. Content validity and its estimation. J. Med. Edu. 2003, 3, e105015. [Google Scholar]
  24. Davis, L.L. Instrument review: Getting the most from a panel of experts. Appl. Nurs. Res. 1992, 5, 194–197. [Google Scholar] [CrossRef]
  25. Polit, D.F.; Beck, C.T. The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Res. Nurs. Health 2006, 29, 489–497. [Google Scholar] [CrossRef] [PubMed]
  26. Dong, X.; Ding, F.; Zhou, S.; Ma, J.; Li, N.; Maimaitiming, M.; Xu, Y.; Guo, Z.; Jia, S.; Li, C.; et al. Optimizing an emergency medical dispatch system to improve prehospital diagnosis and treatment of acute coronary syndrome: Nationwide retrospective study in China. J. Med. Internet Res. 2022, 24, e36929. [Google Scholar] [CrossRef]
  27. Katayama, Y.; Kitamura, T.; Nakao, S.; Himura, H.; Deguchi, R.; Tai, S.; Tsujino, J.; Mizobata, Y.; Shimazu, T.; Nakagawa, Y. Telephone triage for emergency patients reduces unnecessary ambulance use: A propensity score analysis with population-based data in Osaka City, Japan. Front. Public Health 2022, 10, 896506. [Google Scholar] [CrossRef] [PubMed]
Table 1. Relevance.
Table 1. Relevance.
ItemRater1Rater2Rater3Rater4Rater5I-CVI
1. Abdominal pain344331.00
2. Anaphylaxis/Allergic reaction344431.00
3. Animal bite344341.00
4. Back pain (non-trauma)344420.80
5. Bleeding disorder (non-trauma)444420.80
6. Breathing difficulty444431.00
7. Cardiac arrest/Death444441.00
8. Chest discomfort444431.00
9. Choking444441.00
10. Convulsion344441.00
11. Chemical hazard444431.00
12. Diabetes mellitus444431.00
13. Eye problems444431.00
14. Fever444431.00
15. Headache244430.80
16. Overdose/Poison444441.00
17. Psychiatry/Behavior change/Suicide444441.00
18. Pregnancy/Childbirth344431.00
19. Pediatric emergency344441.00
20. Sick person/Unspecify344420.80
21. Stroke444441.00
22. Unconsciousness/Fainting344441.00
23. Assault/Sexual assault344441.00
24. Burn/Blast444431.00
25. Drowning/Diving/Water-related injury444431.00
26. Electrocution444431.00
27. Falls344431.00
28. Heat/Cold exposure 344431.00
29. Stab/Gunshot/Penetrating trauma444441.00
30. Traffic incidents/Vehicle in water333441.00
S-CVI/Ave 0.97
Bootstrap 95% CI 0.95–0.99
Table 2. Clarity.
Table 2. Clarity.
ItemRater1Rater2Rater3Rater4Rater5I-CVI
1. Abdominal pain344331.00
2. Anaphylaxis/Allergic reaction344331.00
3. Animal bite244430.80
4. Back pain (non-trauma)244430.80
5. Bleeding disorder (non-trauma)444320.80
6. Breathing difficulty444331.00
7. Cardiac arrest/Death344441.00
8. Chest discomfort344230.80
9. Choking433431.00
10. Convulsion344441.00
11. Chemical hazard433441.00
12. Diabetes mellitus444441.00
13. Eye problems333331.00
14. Fever444331.00
15. Headache244430.80
16. Overdose/Poison444431.00
17. Psychiatry/Behavior change/Suicide444230.80
18. Pregnancy/Childbirth244330.80
19. Pediatric emergency344441.00
20. Sick person/Unspecify333420.80
21. Stroke444441.00
22. Unconsciousness/Fainting344230.80
23. Assault/Sexual assault344441.00
24. Burn/Blast344431.00
25. Drowning/Diving/Water-related injury433331.00
26. Electrocution444431.00
27. Falls344431.00
28. Heat/Cold exposure 344431.00
29. Stab/Gunshot/Penetrating trauma444341.00
30. Traffic incidents/Vehicle in water244430.80
S-CVI/Ave 0.93
Bootstrap 95% CI 0.90–0.97
Table 3. Simplicity.
Table 3. Simplicity.
ItemRater1Rater2Rater3Rater4Rater5I-CVI
1. Abdominal pain333431.00
2. Anaphylaxis/Allergic reaction333431.00
3. Animal bite233430.80
4. Back pain (non-trauma)444431.00
5. Bleeding disorder (non-trauma)433431.00
6. Breathing difficulty433431.00
7. Cardiac arrest/Death444431.00
8. Chest discomfort444431.00
9. Choking444431.00
10. Convulsion344431.00
11. Chemical hazard444431.00
12. Diabetes mellitus444431.00
13. Eye problems433431.00
14. Fever444431.00
15. Headache333431.00
16. Overdose/Poison444441.00
17. Psychiatry/Behavior change/Suicide433431.00
18. Pregnancy/Childbirth344431.00
19. Pediatric emergency244440.80
20. Sick person/Unspecify333431.00
21. Stroke444441.00
22. Unconsciousness/Fainting244440.80
23. Assault/Sexual assault344441.00
24. Burn/Blast444431.00
25. Drowning/Diving/Water-related injury444431.00
26. Electrocution444431.00
27. Falls344431.00
28. Heat/Cold exposure 344431.00
29. Stab/Gunshot/Penetrating trauma444441.00
30. Traffic incidents/Vehicle in water344431.00
S-CVI/Ave 0.98
Bootstrap 95% CI 0.95–1.00
Table 4. Ambiguity.
Table 4. Ambiguity.
ItemRater1Rater2Rater3Rater4Rater5I-CVI
1. Abdominal pain344431.00
2. Anaphylaxis/Allergic reaction344331.00
3. Animal bite244430.80
4. Back pain (non-trauma)344431.00
5. Bleeding disorder (non-trauma)444420.80
6. Breathing difficulty444431.00
7. Cardiac arrest/Death444431.00
8. Chest discomfort444320.80
9. Choking444331.00
10. Convulsion344431.00
11. Chemical hazard444420.80
12. Diabetes mellitus444331.00
13. Eye problems333431.00
14. Fever444431.00
15. Headache344431.00
16. Overdose/Poison444431.00
17. Psychiatry/Behavior change/Suicide433331.00
18. Pregnancy/Childbirth244330.80
19. Pediatric emergency344431.00
20. Sick person/Unspecify344431.00
21. Stroke444431.00
22. Unconsciousness/Fainting244430.80
23. Assault/Sexual assault344441.00
24. Burn/Blast444420.80
25. Drowning/Diving/Water-related injury444331.00
26. Electrocution444431.00
27. Falls344420.80
28. Heat/Cold exposure 344431.00
29. Stab/Gunshot/Penetrating trauma444441.00
30. Traffic incidents/Vehicle in water244430.80
S-CVI/Ave 0.94
Bootstrap 95% CI 0.91–0.97
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Huabbangyang, T.; Thepmanee, D.; Buaprasert, P.; Chansomboon, P.; Sri-on, J.; Rojsaengroeng, R. Content Validity Assessment of a Newly Developed Emergency Medical Dispatch and Triage Protocol in Thailand. J. Clin. Med. 2025, 14, 7125. https://doi.org/10.3390/jcm14197125

AMA Style

Huabbangyang T, Thepmanee D, Buaprasert P, Chansomboon P, Sri-on J, Rojsaengroeng R. Content Validity Assessment of a Newly Developed Emergency Medical Dispatch and Triage Protocol in Thailand. Journal of Clinical Medicine. 2025; 14(19):7125. https://doi.org/10.3390/jcm14197125

Chicago/Turabian Style

Huabbangyang, Thongpitak, Duangpon Thepmanee, Phudit Buaprasert, Pit Chansomboon, Jiraporn Sri-on, and Rapeeporn Rojsaengroeng. 2025. "Content Validity Assessment of a Newly Developed Emergency Medical Dispatch and Triage Protocol in Thailand" Journal of Clinical Medicine 14, no. 19: 7125. https://doi.org/10.3390/jcm14197125

APA Style

Huabbangyang, T., Thepmanee, D., Buaprasert, P., Chansomboon, P., Sri-on, J., & Rojsaengroeng, R. (2025). Content Validity Assessment of a Newly Developed Emergency Medical Dispatch and Triage Protocol in Thailand. Journal of Clinical Medicine, 14(19), 7125. https://doi.org/10.3390/jcm14197125

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Article metric data becomes available approximately 24 hours after publication online.
Back to TopTop