Highlights
What are the main findings?
- •
- CAREBot, a rule-based chatbot developed for ED-based suicide case management, was rated favorably for usability by patients and social workers, although patients rated its emotional support lower.
- •
- Social workers perceived CAREBot as helpful for supporting structured initial assessments and organizing patient information.
What are the implications of the main findings?
- •
- CAREBot can strengthen standardized assessment and follow-up in ED-based suicide case management.
- •
- Future research should investigate whether integrating LLMs with rule-based safety mechanisms can improve conversational responsiveness without compromising patient safety.
Abstract
Background/Objectives: Emergency departments (EDs) are critical for suicide prevention, yet high workloads often hinder consistent case management. This pilot study assessed the clinical feasibility, usability, and acceptability of CAREBot, a rule-based chatbot designed to support ED-based case management for patients at risk of suicide, among patients and social workers. Methods: We employed an explanatory sequential mixed-methods design. Ten patients and three social workers used CAREBot and completed questionnaires assessing usability, perceived usefulness, and satisfaction. Patients’ psychological status was repeatedly assessed using the PHQ-9, GAD-7, and SBQ-R. Semi-structured interviews with social workers explored integration challenges and workflow impacts. Results: Patients rated ease of use (3.9/5) and information clarity (4.4/5) favorably, whereas emotional comfort was rated lower (2.5/5). Over 4 weeks, 191 URL access events were recorded among the 10 patients (mean, 19.1 per patient). Individual PHQ-9, GAD-7, and SBQ-R score trajectories varied across participants, with both increases and decreases observed. Social workers rated CAREBot favorably for quickly understanding patients’ risk level or condition (4.33/5) and ease of following the scenario flow (5.0/5). Qualitative findings suggested that CAREBot could support initial patient assessments through structured digital screening, although social workers raised concerns regarding risk-scoring accuracy and the management of multiple high-risk patients. Conclusions: CAREBot appears to be feasible and acceptable as an adjunct to ED-based suicide case management, although these preliminary findings require confirmation in larger controlled studies. A rule-based architecture was deliberately selected to provide predictable and controlled interactions in a high-risk environment. Future research should evaluate whether integrating large language models (LLMs) with rule-based safety mechanisms can improve conversational responsiveness and personalization while maintaining patient safety.
1. Introduction
Emergency departments (EDs) serve as a critical point of contact for suicide prevention [1,2]. Timely assessment and appropriate follow-up of patients at risk of suicide are important from both clinical and public health perspectives [3]. Suicide remains a major global public health concern, accounting for an estimated 700,000 deaths annually worldwide [4]. Patients who present to the ED following a suicide attempt are at particularly high risk of subsequent suicidal behavior [3]. Because the ED often represents a key point of contact with the health care system for these patients, timely assessment, intervention, and continuity of care following discharge are essential components of suicide prevention [5].
Despite the importance of continuity of care, the management of patients at risk of suicide in emergency settings presents substantial practical challenges. Social workers involved in suicide prevention and case management perform multiple tasks, including initial assessment, post-discharge follow-up, linkage to community resources, and counseling through telephone, text messaging, and face-to-face encounters. Workforce shortages, high workloads, increasing patient volume, and case complexity can make it difficult to maintain standardized assessment and continuous follow-up [6,7]. The risk of repeated self-harm and suicide following an emergency department presentation further underscores the importance of continued follow-up and risk monitoring [8]. Furthermore, the Safety Planning Intervention with follow-up has been associated with reduced suicidal behavior and improved outpatient treatment engagement among patients discharged from the emergency department [9]. These challenges also highlight different but complementary needs among patients and health care professionals. Patients may benefit from accessible and continuous support beyond individual clinical encounters, whereas health care professionals require patient information to be presented in a structured, interpretable, and clinically relevant manner that is compatible with existing workflows. These complementary needs highlight the importance of integrating structured patient assessments and clinically relevant information into existing suicide prevention and follow-up workflows.
Digital mental health interventions, including chatbot-based approaches, have emerged as potential tools for addressing these challenges [10,11,12]. Previous studies have demonstrated the potential of conversational agents to provide accessible mental health support, including cognitive behavioral therapy-based interventions and AI-based conversational agents [11,12,13,14]. However, the capabilities and limitations of these systems vary according to their underlying architectures. A systematic review by Hua et al. [15] distinguished rule-based, machine learning-based, and large language model (LLM)-based mental health chatbots. Rule-based systems offer predefined interaction pathways that facilitate standardized assessments and consistent information delivery but have limited flexibility in responding to unexpected inputs or providing personalized emotional support. In contrast, LLM-based systems enable more adaptive and context-sensitive interactions, but present challenges related to response reliability, clinical safety, and validation. In a simulation-based evaluation of chatbot responses to suicidal crises, Pichowicz et al. [16] identified limitations in crisis-response appropriateness, further highlighting the need for careful evaluation in high-risk contexts. These findings do not establish the superiority of rule-based systems but support selecting system architectures according to their intended clinical functions.
Beyond conversational interventions, digital tools have also been developed to support suicide risk assessment and clinical decision-making. A systematic review by Seyedsalehi et al. [17] identified substantial gaps in external validation of suicide and self-harm prediction models, despite promising performance in some models. Nevertheless, real-world implementations exist, such as the REACH VET program evaluated by McCarthy et al. [18], which integrated algorithm-based risk identification with clinician-led assessment and care coordination. These studies demonstrate the potential for digital tools to support clinical workflows while emphasizing the importance of translating risk information into actionable care processes. However, evidence remains relatively limited regarding patient-facing tools that integrate repeated structured psychological assessments, predefined risk notifications, and longitudinal monitoring into social workers’ routine case management following ED discharge.
To address this gap, we developed CAREBot, a rule-based chatbot designed to support ED-based case management for patients at risk of suicide. Rather than providing autonomous psychotherapy or predicting suicide risk, CAREBot complements existing case management by providing structured psychological assessments, predefined risk notifications, basic emotional support and resource information, and longitudinal patient monitoring. Its rule-based architecture was selected to provide consistent assessment pathways and controlled information delivery, while recognizing that predefined rules cannot identify all clinically concerning situations and that clinical decisions must remain under professional supervision. Conceptually, CAREBot’s clinical support functions can be interpreted through the Five Rights of Clinical Decision Support, which emphasize delivering appropriate information to the appropriate professional, in a suitable format, through an appropriate channel, and at a relevant point in the clinical workflow [19]. In addition, the Technology Acceptance Model (TAM), particularly perceived usefulness and perceived ease of use, provides a theoretical perspective for interpreting patient and social worker evaluations [20]. These frameworks were used as interpretive perspectives rather than formally tested models. This pilot study aimed to evaluate CAREBot’s usability, acceptability, and clinical feasibility from the perspectives of patients at risk of suicide and social workers involved in ED-based case management. Using a mixed-methods approach, we also examined its perceived utility for patient monitoring, risk-related information management, and workflow integration to identify areas for future refinement.
2. Materials and Methods
2.1. Study Design
This study employed an explanatory sequential mixed-methods design to evaluate the clinical utility and workflow integration of the CAREBot intervention among ED patients at risk of suicide [21]. This design involves first conducting quantitative analyses, followed by qualitative inquiry to provide in-depth explanation and interpretation of the quantitative findings.
The study consisted of two phases. In the first phase, structured surveys were administered to patients and social workers after CAREBot use to quantitatively assess key indicators, including usability, perceived usefulness, and satisfaction. In the second phase, semi-structured in-depth interviews were conducted with social workers to gain deeper insights into key findings and issues emerging from the survey results. The qualitative interviews were designed to complement and expand upon the patterns, limitations, and contextual interpretations identified in the quantitative results.
Qualitative data comprised researcher-documented feedback statements from the three participating social workers and were analyzed using an iterative coding approach informed by thematic analysis principles [22]; detailed procedures are described in Section 2.11. Qualitative research conduct and reporting were guided by the COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines [23].
Specifically, the quantitative findings from Phase 1 informed the refinement of the semi-structured interview questions for Phase 2, ensuring that the qualitative inquiry focused on explaining the underlying reasons for the observed survey scores (Connecting). The integration of the two datasets was formally conducted during the interpretation phase using a joint display to synthesize the findings. This study also adhered to the GRAMMS (Good Reporting of A Mixed Methods Study) guidelines to ensure transparency and rigor in mixed-methods reporting [24]. Specifically, we clearly described the rationale for study design, the integration of quantitative and qualitative data, and the linkages between research phases. This mixed-methods approach provides a methodological framework for a multilayered examination of CAREBot’s feasibility and clinical relevance within the complex emergency department setting.
2.2. Study Setting and Case Management Workflow
This study was conducted at the Samsung Lifeline Center of a tertiary referral hospital in the Republic of Korea with approximately 1980 inpatient beds and about 60,000 emergency department (ED) visits per year. The Samsung Lifeline Center provides case management services for individuals who present to the emergency department after a suicide attempt or who are at risk of suicide. According to the center’s operational records, 73 individuals received case management services between 10 September and 31 December 2022 (90 cases), 340 patients in 2023 (1317 cases), and 272 patients in 2024 (1548 cases).
In the existing case management workflow, social workers conduct an initial assessment of eligible patients and provide subsequent case management after discharge. Follow-up is conducted through telephone calls, text messaging, and face-to-face encounters, during which social workers monitor patients’ psychological status and suicide-related risk, provide counseling, and facilitate linkage to appropriate hospital or community-based resources. These activities require repeated assessment, documentation, and review of patient information across multiple contacts. CAREBot was implemented within this existing workflow to support repeated psychological assessment, longitudinal monitoring, and identification of predefined risk signals, while clinical assessment and intervention decisions remained under the responsibility of the social workers.
2.3. Participants
Participants included adult patients who presented to the emergency department (ED) and were registered for case management at the Samsung Lifeline Center, and who voluntarily consented to participate in the study. Inclusion criteria were as follows: (1) adults aged 19 years or older; (2) individuals with a history of a suicide attempt or identified suicide-related risk factors; and (3) individuals capable of accessing the chatbot via a smartphone or other mobile device. Exclusion criteria included patients deemed clinically inappropriate for participation due to unstable mental status, those unable to communicate, and individuals who withdrew consent during the study. The total number of patients assessed for eligibility and the number of eligible patients who declined participation were not systematically recorded. Consequently, the recruitment rate could not be calculated.
In addition, all three social workers from the Samsung Lifeline Center participated in the study, and a census approach was used to observe and document the workflow and intervention processes related to the chatbot-based intervention.
The sample size was determined pragmatically by the constraints of this pilot implementation. The study was conducted alongside routine case management by the center’s three social workers, within a limited project implementation period. In addition, disruptions to emergency department services during the national medical workforce crisis restricted the admission of patients presenting after suicide attempts, limiting the pool of potential participants. Accordingly, this study aimed to obtain preliminary insights into the feasibility of integrating CAREBot into routine clinical workflows rather than to establish clinical effectiveness or statistical generalizability.
2.4. Ethical Approval
This study was conducted in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB No. 2025-05-164-003; approval date: 22 August 2025). Patient recruitment and data collection commenced only after IRB approval was obtained. Written informed consent was obtained from all patients and social workers prior to enrollment.
2.5. The CAREBot Intervention
CAREBot (Crisis Assessment & Response Expert Bot) is a rule-based chatbot digital intervention developed specifically for emergency department (ED) patients at risk of suicide. The system incorporates automated, scale-based assessments for depression, anxiety, and suicidal ideation—with risk levels automatically derived from the resulting scores—and provides mind safety-related content, emergency contact information, FAQs, and high-risk alert notifications.
2.6. Development Process
CAREBot was developed through an iterative, expert-informed process involving both technical developers and clinical stakeholders. Prior to prototype development, the development team first reviewed the existing case management workflow used by social workers at the Samsung Lifeline Center to identify key functional requirements. This workflow review identified three principal requirements: standardized administration of psychological assessments, structured communication of predefined risk signals, and accessible presentation of patient-level information for follow-up case management. These requirements informed the selection of a rule-based architecture, allowing assessment pathways and risk notification criteria to be predefined while retaining clinical assessment and intervention decisions under the responsibility of social workers. Based on this review, an initial prototype was developed and subsequently reviewed by a multidisciplinary advisory group comprising three social workers, one psychiatrist, and one emergency medicine physician, who met biweekly to evaluate the clinical appropriateness of assessment content, scenario flow, and system functionality (Figure 1). The three social workers serving on the advisory group subsequently participated in the study evaluation, resulting in an overlap between the development advisory and evaluation roles. This iterative review process led to several substantive revisions. For example, the initial prototype displayed raw numerical scores from the PHQ-9, GAD-7, and SBQ-R directly to patients; based on clinical feedback that raw scores could be misinterpreted or distressing without contextual guidance, this was revised so that patients instead received a risk-level indication accompanied by guidance messages on recommended next steps, rather than the numerical score itself.
Figure 1.
Development and iterative refinement process of CAREBot.
2.7. System Development and Structure
In this study, we designed a scenario-based, button-driven chatbot system built on rule-based logic to support case management processes for emergency department (ED) patients at risk of suicide. The system adopts a closed, rule-based structure rather than an open-ended conversational AI, in which users progress through predefined pathways by selecting options step by step. Beyond the assessment modules, frequently asked questions and mind-care (psychoeducational) resources were also organized into predefined pathways that directed users to corresponding pre-authored content. This design choice was intended to improve the predictability and controllability of interactions in a sensitive mental health setting.
The chatbot was developed on the AvoMD platform and incorporates the following key functions required for case management. First, the chatbot implements a psychological assessment function that administers standardized scales, including the PHQ-9, SBQ-R, and GAD-7 [25,26,27]. In addition, an assessment module aligned with SPEDIS, the official case management platform used in Korea’s national Emergency Department-Based Post-Suicide Attempt Case Management Program [28], was included as an optional reference tool to support social workers’ routine case management documentation (Figure 2). Unlike the three standardized psychological scales, SPEDIS was not designated as a longitudinal outcome measure, and regular completion was not required. Scores are automatically calculated based on responses to each item. Second, the chatbot includes a risk detection and notification function that alerts the assigned social worker when responses indicate elevated risk, described in detail in the Risk Detection and Alert System section below. Third, the chatbot provides emotional support content, guidance on hospital and community resources, and frequently asked questions (FAQs), allowing patients to access relevant information at any time. This structure was designed to support standardized repeated assessments, identification of predefined risk signals, and consistent record-keeping in the high-risk emergency department setting.
Figure 2.
User interface of the CAREBot system for assessment and resource guidance in the emergency department. (A) Welcome & Consent. The introductory screen presents information about CAREBot, its intended role in providing counseling support, suicide risk assessment, and information on available resources, and obtains the user’s agreement before proceeding. (B) Main Menu. The main menu provides access to four core functions: suicide risk assessment, counseling and resource connection, healing and well-being content, and frequently asked questions. (C) Suicide Risk Assessment. The assessment screen allows users to select standardized psychological assessments, including the PHQ-9, GAD-7, SBQ-R, and an overall assessment aligned with SPEDIS. (D) Counseling & Resource Connection. This screen provides information on available counseling and support resources, including the Samsung Lifeline Center, 24/7 support services, community support services, SNS counseling, and nationwide suicide prevention services. (E) Healing & Well-Being Content. This screen provides supportive resources intended to promote emotional well-being, including a calming breathing exercise, calming music, a stretching video, and guidance on actions that may help during a crisis. (F) Frequently Asked Questions. The FAQ screen provides predefined information on commonly raised questions regarding counseling, treatment-related financial support, counseling fees, family counseling, and other available services.
The system architecture consisted of three integrated components: a patient-facing chatbot interface built on the AvoMD platform, a data transmission layer, and a research database, kept separate from the AvoMD platform itself. Patients accessed the chatbot via a personalized URL and completed the assigned assessments; responses were then transmitted and processed as described in the Data Storage and Integration and Risk Detection and Alert System sections below (Figure 3).
Figure 3.
CAREBot workflow with risk detection and direct social worker intervention.
2.7.1. User Identification and Access
Participants were assigned an anonymized user identification number (user ID) at enrollment and accessed the chatbot via a dedicated URL containing their respective ID (e.g., P001, P002). The chatbot automatically recognized the user_id parameter in the URL, allowing response data to be stored separately for each user.
This approach enabled participant-level data separation without requiring patients to enter directly identifying personal information into the chatbot. Participants could access the system instantly on their mobile devices via a dedicated link or QR code without the need for separate installation.
2.7.2. Data Storage and Integration
Chatbot response data were transmitted via an API and stored in a secure research database with restricted access. Data fields were structured to include the response timestamp, item-level selections, total score, and user_id, with data recorded simultaneously in scale-specific tables and in a merged table for comprehensive response management. This structure enabled social workers to quickly review patient-specific response trends and identify high-risk responses.
2.7.3. Risk Detection and Alert System
When response results exceeded predefined risk thresholds, an automated trigger was activated to send email alerts to the assigned social worker. This function was designed to notify social workers of predefined risk signals and support subsequent clinical follow-up. The alert emails included the patient’s ID, the name of the assessment scale, and the assessment timestamp.
The predefined cutoff values for automated risk alerts were PHQ-9 ≥ 15, GAD-7 ≥ 10, and SBQ-R ≥ 8, based on the total scores of each psychological scale. These thresholds were based on established cutoff scores reported in the validation literature for the PHQ-9, GAD-7, and SBQ-R [25,26,27] and were reviewed and confirmed by the consulting psychiatrist prior to implementation rather than newly derived for this study. Alerts were triggered based on total scale scores; item-level alerting was not implemented in the current version. Consequently, the system could not independently identify clinically concerning responses to individual items, such as PHQ-9 item 9, when the total score remained below the alert threshold.
Although CAREBot was accessible to patients 24 h a day, clinical monitoring and responses to automated alerts were limited to the Samsung Lifeline Center’s operating hours (08:00–17:00). Alert emails generated outside these hours were automatically delivered to the assigned social worker and reviewed during the next working period. Each risk alert was sent as a single automated email notification, without a secondary notification mechanism or automated repeat alerts for unacknowledged messages. No dedicated after-hours personnel or real-time emergency response service was provided through CAREBot. Accordingly, CAREBot was intended to supplement routine clinical case management rather than serve as a standalone suicide risk detection or continuous crisis monitoring system.
2.7.4. Visualization and Monitoring
Conditional formatting and time-series charts were used to visualize changes in each patient’s psychological scale scores over time. This allowed social workers to intuitively monitor changes in high-risk patients’ status and make informed decisions regarding the timing of case management interventions.
2.8. Outcome Measures
The primary outcomes of this study were usability, acceptability, and clinical feasibility of CAREBot as a digital intervention for patients at risk of suicide in the emergency department. Outcomes were assessed from both patient and social worker perspectives, reflecting differences in their roles and evaluation objectives. The evaluation was conceptually informed by perceived usefulness and ease of use, as described in TAM, and by information delivery and workflow considerations emphasized in the Five Rights of Clinical Decision Support. Patient-reported usability and satisfaction, along with social worker-reported usefulness, integration and workflow fit, and trust and safety concerns, were examined descriptively. Because the questionnaires were developed for this pilot study rather than derived from validated TAM instruments, these domains were not treated as formal measures of theoretical constructs, and no framework-based hypothesis testing was performed.
2.9. Patient-Reported Outcomes
From the patient perspective, outcomes were quantitatively assessed using standardized psychological scales administered via CAREBot. Participants completed the Patient Health Questionnaire-9 (PHQ-9), Suicidal Behaviors Questionnaire-Revised (SBQ-R), and Generalized Anxiety Disorder-7 (GAD-7), with both total scores and item-level responses recorded. Total scores for all three psychological scales were automatically calculated by the CAREBot system based on participants’ selected responses, in accordance with the standard scoring guidelines for each instrument.
During the approximately four-week study period, participants could complete psychological assessments through CAREBot at any time. Consequently, assessment dates and frequencies varied across participants. For longitudinal descriptive analysis, three assessment time points were retrospectively defined using records obtained within each participant’s documented study period. The pre-assessment was defined as the first valid assessment, the mid-assessment as the assessment closest to the midpoint between the first assessment and documented study completion, and the post-assessment as the last valid assessment on or before study completion. Importantly, the post-assessment represented the final available psychological assessment conducted during each participant’s study participation period, prior to or on the date of study completion, rather than an assessment performed after case management closure. These time points were defined for descriptive analysis and did not represent prospectively scheduled assessments.
After completion of the study participation period, patient satisfaction with CAREBot was assessed using an online questionnaire developed by the research team. The questionnaire comprised 8 items rated on a 5-point Likert scale and assessed patients’ perceptions of ease of use, information delivery, emotional support, and overall satisfaction. Open-ended questions were also included to collect feedback on areas for improvement and other experiences with CAREBot. Given the pilot nature of the study and the clinical characteristics of the participants, no in-depth qualitative interviews were conducted with patients.
Chatbot access events were recorded in the research database through URL redirection logs. To calculate access frequency, log entries were sorted chronologically for each participant, and an access was counted only when at least five minutes had elapsed since the immediately preceding log entry. These logs captured URL redirection events, including instances in which participants accessed the chatbot link without necessarily completing an assessment or engaging in a substantive interaction. Therefore, the recorded access frequency should not be interpreted as the number of distinct chatbot conversations or completed assessment sessions. Participants’ self-reported chatbot usage frequency was separately collected through the post-intervention questionnaire.
2.10. Social Worker-Reported Outcomes
From the social worker perspective, outcomes were assessed using both quantitative surveys and qualitative interviews to examine perceptions of CAREBot in relation to routine case management tasks. After completion of the implementation period, a structured survey was administered to assess six domains: Usability, Usefulness, Scenario Appropriateness, Trust & Safety, Integration & Workflow Fit, and Satisfaction. Items were rated on a 5-point Likert scale, with higher scores indicating more favorable perceptions of CAREBot.
In addition, semi-structured interviews were conducted with all participating social workers to explore their experiences using CAREBot in routine case management, including perceived benefits, limitations, safety concerns, and suggestions for improvement.
For the qualitative analysis, we used feedback statements documented by a researcher during the semi-structured interviews. The analysis was limited to these researcher-documented statements rather than complete verbatim interview transcripts. The statements reflected social workers’ experiences with CAREBot, including perceived usefulness, limitations or concerns, and improvement requests.
2.11. Data Analysis
Data were analyzed using R (version 4.3.1). Continuous variables were summarized as means and standard deviations, and categorical variables as frequencies and percentages. Given the pilot nature and small size of this study, changes in scores across the pre-, mid-, and post-assessment time points were explored through visualization of individual patient score trajectories rather than formal statistical significance testing. Accordingly, individual psychological scale scores were presented as time-series graphs. The primary objective was feasibility rather than efficacy testing; therefore, no group-level statistical comparisons were performed.
No a priori quantitative thresholds were established to determine the feasibility or acceptability of CAREBot. Therefore, feasibility was explored descriptively based on participant retention, chatbot usage, and patient- and social worker-reported evaluations, rather than assessed against predefined success criteria.
Qualitative analysis was conducted using researcher-documented feedback statements collected during semi-structured interviews with the three participating social workers. A total of 27 feedback statements were initially identified. Two coders independently reviewed and categorized the statements according to their functional focus and evaluative content (perceived usefulness, limitations or concerns, and improvement requests). Differences in coding were discussed and resolved through consensus. One statement concerning participation in the research rather than CAREBot itself and one duplicate statement were excluded, leaving 25 statements for the final analysis.
The coded statements were organized into five overarching themes based on their shared content and meaning. Three statements concerning interface design, password protection, and mobile accessibility were reported separately as improvement needs rather than classified as a sixth theme. The number of statements and contributing social workers were summarized for each theme. To integrate the quantitative and qualitative findings, relevant social worker survey items were linked to corresponding themes and representative interview quotations in a joint display. A separate summary of the qualitative themes, coding categories, and improvement needs is provided in the Results section.
3. Results
3.1. Participant Characteristics
A total of 13 patients initially accessed the CAREBot system during the study period. Three patients (P006, P009, and P011) were excluded from the final analysis because they did not maintain continuous weekly CAREBot use throughout the four-week study period, resulting in a retention rate of 76.9% (10/13). Discontinuation of CAREBot research participation did not imply discontinuation of routine clinical case management, which was conducted independently of the study. However, safety outcomes following study discontinuation were not systematically collected and could not be evaluated. The mean age of participants was 27.7 years (SD = 9.5), with those aged 20–29 years comprising 70% of the sample. Female participants accounted for 90% of the sample, and male participants for 10%. Overall, 70% of patients had prior experience using services provided by the Samsung Lifeline Center. Regarding CAREBot usage frequency, 80% of participants used the system once per week or less, and 20% used it 2–3 times per week (Table 1).
Table 1.
General characteristics of participants (N = 10).
Additionally, three social workers participated in the study (Table 2). The mean age of the social workers was 29.0 years (SD = 3.2), and all were female. Their lengths of employment at the center were 1 year and 3 months, 1 year and 10 months, and 4 years and 8 months, respectively. Overall, 67% reported using CAREBot at least once per day.
Table 2.
General characteristics of social workers (N = 3).
3.2. Patient-Reported Outcomes
3.2.1. CAREBot Usage Patterns
A total of 191 URL access events were identified among the 10 participants, corresponding to a mean of 19.1 accesses per participant. These log-derived access counts represent URL access events rather than confirmed chatbot interactions. In contrast, 80% of participants reported using CAREBot once per week or less in the post-intervention questionnaire (Table 1). Participant-level URL access counts are presented in Supplementary Table S1.
3.2.2. Patient Satisfaction with CAREBot
Patient satisfaction results showed generally positive evaluations regarding CAREBot’s ease of use and information delivery (Table 3). Notably, the item “The content provided by the chatbot was easy to understand” (Item 2) had a mean score of 4.4 (SD = 0.97), while “The chatbot’s way of presenting information was clear and effective” (Item 3) had a mean score of 4.0 (SD = 1.15), indicating high ratings for information clarity and comprehensibility.
Table 3.
Participants’ satisfaction with CAREBot (N = 10).
In contrast, the item “I felt emotionally comforted through the chatbot” (Item 4) had a lower mean score of 2.5 (SD = 0.97), and “I felt that the chatbot was helpful during crisis situations” (Item 5) averaged 2.8 (SD = 1.14), indicating moderate ratings. These findings indicate that patients rated CAREBot less favorably for perceived emotional comfort and support during crisis situations than for information clarity and comprehensibility.
Five of the ten patients provided responses to the optional open-ended questions. Positive feedback highlighted the convenience of monitoring psychological status, the ability to reflect on emotional difficulties through structured assessments, and reassurance associated with risk alerts and linkage to social workers. However, patients also identified limitations in emotional support, difficulties accessing the chatbot through web links or QR codes, and a need for more interactive features and regular-use reminders. One respondent expressed concern that awareness of risk notifications could discourage use and reported difficulties with alert delivery. These findings identified opportunities to improve accessibility, engagement, and risk notification functions.
3.2.3. Changes in Psychological Assessment Scores
Individual psychological assessment trajectories showed heterogeneous patterns across participants (Table 4; Figure 4). Based on the retrospectively selected pre- and post-assessment scores, PHQ-9 scores decreased in six participants and increased in four participants. GAD-7 scores decreased in six participants and increased in four participants. SBQ-R scores decreased in four participants, remained unchanged in four, and increased in two participants. Notably, participant P010 exhibited persistently elevated psychological assessment scores, with PHQ-9 scores increasing from 23 at the pre-assessment to 25 at both the mid- and post-assessments, and GAD-7 scores increasing from 15 to 21. Similarly, participant P001 showed an increase in PHQ-9 scores from 19 at the pre-assessment to 23 at the post-assessment, despite a temporary decrease at the mid-assessment. These findings indicate considerable variability in individual psychological assessment trajectories and should not be interpreted as evidence of the chatbot’s therapeutic effectiveness.
Table 4.
Individual psychological assessment scores at retrospectively selected time points (N = 10).
Figure 4.
Individual trajectories of psychological assessment scores across the retrospectively selected pre-, mid-, and post-assessment time points.
3.2.4. Post-Discharge Follow-Up Outcomes
Post-intervention follow-up was conducted for 10 patients whose case management had been completed, assessing outpatient utilization, emergency department (ED) revisits, and mortality after discharge. Most patients continued attending psychiatric outpatient clinics to maintain psychological stability, and some were transferred or linked to community mental health services. One ED revisit occurred due to exacerbation of anxiety symptoms and was subsequently managed through outpatient care. Although no deaths were reported during the observation period, elevated psychological assessment scores were observed in some participants, including P010, whose post-assessment PHQ-9 score remained in the severe range (25 points).
During the study period, 82 automated email alerts were recorded among the 10 participants included in the final analysis. During operating hours, social workers promptly contacted patients by telephone to assess their condition, while alerts generated outside operating hours were prioritized for review and intervention on the next working day. Among the 82 automated email alerts, alerts involving different patients occurred within a five-minute interval on one occasion. These alerts were generated during separate telephone-based case management sessions conducted by different social workers, rather than requiring a single social worker to manage multiple patients simultaneously. Staff workload and response times associated with overlapping alerts were not systematically measured.
3.3. Social Worker-Reported Outcomes
Quantitative Survey Results
A structured satisfaction survey was administered to social workers after completion of the chatbot-based intervention to evaluate their perceptions of CAREBot (Table 5). All three social workers participated, and most items received mean scores of 4.0 or higher, indicating generally high ratings.
Table 5.
Individual social worker ratings of CAREBot (N = 3).
In the usability domain, the items “It was easy to follow the overall scenario flow of the chatbot” (Item 2) and “There was no difficulty in finding information or accessing required functions” (Item 3) both achieved a mean score of 5.0, suggesting that CAREBot is an intuitive tool even in the emergency department setting.
In the usefulness domain, CAREBot received positive evaluations for helping social workers quickly assess patients’ risk levels and clinical status, and for serving as a useful reference in intervention planning. However, the item regarding more systematic information management compared to manual documentation (Item 10) received a relatively lower mean score, suggesting room for improvement in integration with existing documentation workflows.
In the scenario appropriateness domain, alignment with clinical guidelines received high ratings, whereas adequacy of response-based risk judgment and consistency with actual counseling workflows were rated at moderate levels. This suggests that CAREBot’s core structure and logic generally align with the clinical context, but further refinement of the scenarios is needed to more precisely reflect emergency department social workers’ real-world decision-making processes.
In the trust and safety domain, reliability of results and appropriateness of high-risk classification were rated favorably (4.33 and 4.00), whereas perceived positive impact on patient safety management received a moderate score (3.67). In the integration and workflow fit and satisfaction domains, both willingness to recommend CAREBot to peer social workers and intention for continued future use received mean scores of 4.0 or higher, indicating positive perceptions regarding its potential adoption in clinical practice.
Overall, the social worker satisfaction results indicate that CAREBot has positive potential in terms of usability, basic usefulness, and field applicability in high-intensity environments such as the emergency department. At the same time, areas including documentation integration and refinement of risk judgment processes were identified as targets for improvement in future service development and follow-up studies.
3.4. Qualitative Interview Findings
Semi-structured interviews with three social workers yielded 27 recorded statements. After excluding one statement concerning research participation rather than CAREBot and one duplicate statement, 25 distinct statements were included in the qualitative analysis. These were organized into five themes (22 statements) and three additional improvement needs. Table 6 summarizes the number of statements and contributing social workers for each theme and the distribution of perceived usefulness, limitations or concerns, and improvement requests. Additional improvement needs not classified into the themes are also presented. Selected qualitative findings were integrated with social worker survey results in the joint display (Table 7).
Table 6.
Summary of qualitative findings and improvement needs by theme.
Table 7.
Joint display integrating quantitative and qualitative findings.
3.4.1. Theme 1. Early Risk Signal Identification and Counseling Preparation
Four statements from two social workers described the perceived usefulness of reviewing risk-related information and completed assessments before patient encounters. Participants considered these features helpful for preparing counseling sessions and avoiding repeated basic questions. These perceptions were consistent with favorable survey ratings for quickly understanding patient risk level (Item 6: mean 4.33, SD 1.15) and understanding the status of new or returning patients (Item 9: mean 4.00, SD 1.00; Table 7).
- •
- “Before meeting the patient, I can review the risk signals in advance, which helps me organize how to start the counseling.” (SW1)
- •
- “Since the basic assessment is already done, it’s helpful that I don’t have to ask everything again from the beginning.” (SW2)
3.4.2. Theme 2. Concerns Regarding Risk Assessment Accuracy and Result Management
Five statements from all three social workers concerned risk assessment accuracy and result management (four limitations or concerns and one improvement request). Participants noted that scale-based scores might underestimate or overestimate suicide-related risk and suggested incorporating more specific questions to aid interpretation. These concerns provide context for the survey ratings for logical risk-level determination (Item 13: mean 3.67, SD 1.15), appropriateness of high-risk classification (Item 16: mean 4.00, SD 1.00), and reflection of key clinical information (Item 14: mean 4.33, SD 1.15; Table 7). Separately, a social worker reported one or two instances in which assessment results were incorrectly linked to patient IDs during simultaneous assessments. These incidents occurred when two social workers conducted separate telephone-based case management sessions with their respective patients at overlapping times, during which both patients completed psychological assessments through CAREBot. This issue raised concerns regarding the reliability of patient-specific data management.
- •
- “Even when there is a risk of suicide, the score does not always come out very high.” (SW3)
- •
- “On the other hand, I’m also concerned that it might appear more dangerous than it actually is when the score is too high.” (SW1)
- •
- “Rather than relying only on simple scale-based assessments, including more specific questions would help with interpreting risk levels.” (SW2)
3.4.3. Theme 3. Perceived Workflow Support and Need for Intervention Prioritization
Five statements from all three social workers addressed workflow support and intervention prioritization (one perceived usefulness statement, one limitation or concern, and three improvement requests). One social worker anticipated that the functionality could reduce workload and improve the efficiency of responding to patients; this was an expected benefit rather than an objectively measured reduction in workload. The corresponding survey item on reduced workload or improved efficiency received a mean score of 4.33 (SD 1.15; Item 8, Table 7). Separately, participants described difficulties determining whom to approach first when multiple patients showed risk signals and requested automated intervention-priority recommendations. The workflow-introduction item received a mean score of 4.00 (SD 1.00; Item 18, Table 7).
- •
- “I think this function could reduce social workers’ workload and improve the efficiency of responding to patients.” (SW3)
- •
- “When multiple patients show risk signals at the same time, I’m always unsure about who to see first.” (SW2)
- •
- “Rather than just showing risk levels, it would be much more helpful if it could recommend, ‘See this patient first.’” (SW3)
3.4.4. Theme 4. Emotional Support and a Complementary Role in Face-to-Face Counseling
Five statements from all three social workers described the perceived value of supportive content and CAREBot’s complementary role in counseling. Social workers suggested that emotional support or calming information could provide an interim resource and help connect patients to face-to-face counseling, without replacing professional assessment or intervention. These statements represent social workers’ perceptions and do not establish an observed improvement in patients’ emotional outcomes. Because the social worker survey did not include a directly corresponding item, this theme is reported separately rather than being forced into the joint display (Table 7).
- •
- “For patients who are in a state of ‘I want to live, but I don’t know how,’ this kind of step is really necessary.” (SW2)
- •
- “When anxiety is explained as a normal reaction, patients tend to accept it better than expected.” (SW3)
- •
- “I think it is very meaningful that CAREBot does not completely replace face-to-face counseling but plays an intermediate role that helps patients connect to actual counseling.” (SW1)
3.4.5. Theme 5. Need for More Personalized Patient Interactions
Three statements from one social worker identified limitations in reflecting individual patient circumstances within standardized interactions and called for more tailored response pathways. These comments related to the survey ratings for alignment with actual counseling workflows (Item 11: mean 3.67, SD 1.15) and reliability of chatbot results and guidance (Item 15: mean 4.33, SD 1.15; Table 7), although the survey items did not directly measure personalization. These findings concern the rule-based CAREBot system evaluated in this study and should not be interpreted as an evaluation of general-purpose large language models.
- •
- “Each patient’s situation is different, and there are limitations in reflecting all of that.” (SW2)
- •
- “It would be more trustworthy if the responses were tailored to the patient’s specific situation rather than being general.” (SW2)
Taken together, this joint display demonstrates how qualitative insights from social workers complement and help explain selected quantitative findings, providing a more comprehensive understanding of CAREBot’s clinical utility and workflow integration in the emergency department setting.
4. Discussion
This pilot study evaluated the feasibility and acceptability of the CAREBot digital intervention for patients at risk of suicide in the emergency department, addressing real-world challenges in delivering standardized assessments and continuous monitoring in a setting characterized by staffing shortages and high workload [6,7]. Importantly, CAREBot was designed not as a replacement for clinicians, but as a supportive clinical tool to assist social workers in risk assessment, monitoring, and case management processes. The significance of this study lies in the implementation of a chatbot-based intervention in an actual clinical emergency setting and in the integrated analysis of perspectives from both patients and social workers.
The study findings suggested that CAREBot was perceived as accessible and usable by participants, with chatbot access logs documenting repeated access during the study period. Previous research has demonstrated the role of emergency department-based crisis intervention teams in supporting patients following suicide attempts [29]. Within this established care context, CAREBot may serve as a digital adjunct to routine case management. However, its effectiveness in improving continuity of care or clinical outcomes was not evaluated in this pilot study.
Within the approximately four-week study period, CAREBot allowed patients to complete repeated psychological assessments between routine case management contacts. This function may complement conventional follow-up practices by providing additional patient-reported information for social workers to review. However, the present study did not establish whether CAREBot improved the frequency, timeliness, or quality of clinical monitoring. The variability in psychological scale score changes across the pre-, mid-, and post-assessment time points reflects the clinical heterogeneity of patients at risk of suicide in the emergency department and underscores the need for ongoing, continuous assessment [30].
CAREBot’s interface and interaction flow were also shaped with attention to end-user experience, informed by iterative clinical review with social workers throughout development. This user-informed approach is consistent with prior work emphasizing co-design and active end-user involvement in the development of mental health chatbots, which has been associated with improved usability [31,32].
From the patient perspective, CAREBot received generally positive evaluations for ease of use and clarity of information delivery. In contrast, items related to emotional comfort and immediate support during crisis situations received relatively lower scores, reflecting limitations inherent to the closed, scenario-based chatbot structure applied in this study. This design choice reflected CAREBot’s intended role as a structured assessment and case management support tool rather than an autonomous provider of emotional counseling. Rule-based architectures allow predefined interaction pathways and consistent information delivery, although their limited flexibility may constrain personalized emotional support [15]. The lower patient ratings for emotional comfort in the present study are consistent with this limitation, but do not establish that rule-based systems are less effective than other chatbot architectures. Accordingly, these findings suggest that digital interventions such as CAREBot may be better suited to supporting basic assessment, risk signal detection, and information provision, rather than replacing the emotional engagement provided by social workers. Nevertheless, both the qualitative interviews (e.g., SW2) and selected survey items (e.g., Items 11 and 15) indicate a potential need for more adaptive and patient-tailored interactions.
The study results suggested a distinction between the chatbot’s functional utility and its capacity for emotional engagement. While participants rated the structured guidance favorably, the lower scores for emotional comfort (Item 4; mean 2.5) suggest that a rigid, rule-based logic may feel impersonal to patients in acute distress. Nevertheless, the choice of a rule-based model for CAREBot was a deliberate design decision. In the context of suicide prevention within an emergency department, ensuring the reliability, predictability, and safety of responses is paramount. Unlike open-ended generative models, rule-based systems restrict responses to predefined pathways and content, allowing greater control over information delivery. However, this approach does not eliminate clinical safety risks, particularly when important risk signals fall outside predefined assessment items or thresholds.
To address the gap in emotional engagement while preserving safety, the integration of LLMs may represent a potential next step, although evidence regarding the appropriateness of chatbot responses to suicidal crises remains limited [15,16]. Our findings suggest that any transition to LLM-based systems should be accompanied by robust clinical guardrails. Such guardrails should not only include toxic content filtering but also incorporate ‘clinical grounding’—a mechanism that ensures the LLM’s empathetic responses remain within the boundaries of validated crisis intervention protocols. For instance, while an LLM can tailor the tone and language to the patient’s narrative, the underlying decision-making for risk escalation must remain governed by deterministic logic. A hybrid architecture that combines the conversational flexibility of LLMs with the deterministic safety framework of rule-based systems may enable a more human-centered yet clinically responsible CDSS for high-risk populations.
The three social workers involved in both CAREBot development and evaluation reported favorable perceptions of its potential utility for automated risk assessment, alerts for high-risk responses, and standardized record-keeping. In the qualitative interviews, social workers perceived CAREBot as potentially helpful for identifying risk-related responses before face-to-face counseling and preparing for subsequent consultations. These perceptions were consistent with their favorable survey ratings; however, they should be interpreted as preliminary, subjective feedback rather than objective evidence of improved clinical workflow or efficiency.
At the same time, the qualitative interviews identified potential under- or overestimation of risk, challenges in prioritizing interventions, and the need for patient-tailored conversations as key areas for improvement. These qualitative findings complement the moderate quantitative scores observed in the domains of risk judgment logic and scenario appropriateness (e.g., Item 13), highlighting limitations of relying solely on scale-based, rule-based classification. In this context, future work could explore the integration of additional algorithmic risk prediction models that incorporate multiple clinical and contextual variables to support more refined and clinically aligned risk stratification [33]. Such an approach may help address social workers’ concerns regarding risk misclassification while further strengthening the role of digital tools in supporting clinical decision-making in emergency settings. Notably, social workers’ perception of CAREBot as an intermediary—bridging emotionally distressed patients to face-to-face interventions rather than replacing counseling—suggests a potentially appropriate role for such tools in ED-based case management.
Importantly, CAREBot should not be interpreted as a replacement for clinical care, but rather as a digitally mediated adjunct within the emergency psychiatric workflow. Within the scope of this pilot implementation, its functions may be understood across four practical domains:
- •
- Screening support, by extending structured risk prompts beyond the single time-point ED assessment;
- •
- Monitoring support, by maintaining limited longitudinal engagement during the post-discharge period;
- •
- Escalation support, by detecting predefined crisis signals and triggering alerts;
- •
- Human-centered intervention, by ensuring that final evaluation and action remain under clinician responsibility.
Framed in this way, CAREBot does not redefine the care continuum but introduces a controlled digital layer that facilitates earlier signal detection and structured escalation. This intermediary positioning may represent a pragmatic and safer pathway for integrating AI-assisted tools into emergency psychiatric care.
In summary, this pilot study provides preliminary evidence of the feasibility and perceived acceptability of CAREBot as an adjunct to emergency department-based case management for patients at risk of suicide. However, the favorable evaluations reported by the three social workers should be interpreted cautiously given their involvement in system development and the small number of evaluators. These findings suggest that integrating digital tools into existing face-to-face and telephone-based emergency suicide prevention services may improve repeated assessment, consistency of record-keeping, and early detection of high-risk responses [9,14].
Building on these findings, future versions of CAREBot could be developed to support continued outpatient follow-up for patients at risk of suicide who cannot be admitted to hospital, complementing existing clinician-led case management. However, the limited emotional support and conversational flexibility observed in this pilot study highlight the need to enhance patient-centered interactions while incorporating robust clinical safety guardrails. Future development and clinical evaluation will be necessary to determine whether such refinements can improve continuity of care in resource-constrained clinical settings.
Beyond these quantitative and qualitative findings, our experience of developing and operating CAREBot in a real clinical setting offers additional insight relevant to future system design. As described in the Development Process, CAREBot’s assessment content, scenario flow, and alert logic were shaped through an iterative process involving biweekly review with social workers and clinical advisors throughout development. This close involvement of end users may have contributed to the relatively high usability and workflow-fit scores reported by social workers, as several design decisions—such as presenting risk levels rather than raw psychological scale scores—were directly informed by early clinical feedback; however, because the same social workers also evaluated CAREBot, these favorable ratings may also partly reflect social desirability or familiarity bias. At the same time, once CAREBot was deployed and used under real clinical conditions, some aspects of patient interaction diverged from what had been anticipated during development: the closed, scenario-based structure intended to ensure safety and predictability was, in practice, perceived by some patients as less emotionally responsive than expected, and social workers similarly noted a need for more adaptive, patient-tailored conversations to accommodate the variability of real patient presentations. These observations suggest that pre-deployment clinical review, while valuable for ensuring content accuracy and safety, cannot fully anticipate the range of needs that emerge once a system is used in practice. Based on these real-world observations, we plan to revise CAREBot’s scenario architecture in future iterations to allow more flexible, patient-tailored interaction pathways while preserving the safety and predictability required in high-risk settings. Future studies should prospectively evaluate whether such a redesigned, user-informed version of CAREBot improves emotional engagement without compromising the reliability of risk assessment. Beyond its preliminary findings, this study highlights the potential of chatbot-based approaches to support structured assessments and continuity of care within existing suicide prevention services. Such approaches may become increasingly relevant as healthcare systems seek scalable and sustainable strategies to support clinical workflows.
However, this study has several limitations. As a pilot-scale, single-center study, the sample size was limited, and the study was conducted using an observational design without a control group. In addition, the study sample was predominantly female (90%) and consisted largely of participants in their twenties (70%). This demographic imbalance further limits the generalizability of the findings to patients of different genders and age groups.
An additional limitation concerns the discrepancy between self-reported chatbot usage and log-derived access frequency. Participants may have perceived multiple URL accesses associated with a single episode of use as one interaction, whereas the system recorded separate access events. However, this interpretation could not be verified because additional in-depth interviews regarding patients’ perceptions of chatbot use were not conducted, considering the vulnerability of the study population. Therefore, URL access counts should not be interpreted as equivalent to distinct chatbot conversations or meaningful engagement.
The numbers of patients assessed for eligibility and those who declined participation were not systematically recorded, preventing calculation of the recruitment rate and limiting assessment of potential selection bias. Furthermore, although three participants were excluded from the final analysis because they did not maintain continuous weekly CAREBot use, their safety outcomes following study discontinuation were not systematically collected. Consequently, the study cannot determine whether discontinuation of CAREBot participation was associated with subsequent safety-related outcomes. In addition, some participants reported the minimum SBQ-R score or provided different responses to items assessing historical suicidal experiences across assessment time points. Although SBQ-R scores were automatically calculated using the standard scoring guidelines, the consistency of participants’ self-reported historical experiences could not be verified against individual clinical records. Therefore, changes in SBQ-R scores should not be interpreted as indicating changes in lifetime suicidal history, and the reasons for these response patterns remain uncertain.
Another important limitation concerns the risk detection and alert system. Because automated alerts were triggered solely by total psychological scale scores, clinically significant responses to individual items, particularly PHQ-9 item 9, could remain undetected when the total score was below the predefined threshold. Therefore, the alert function should not be considered sufficient for comprehensive suicide risk assessment, and future versions of CAREBot should incorporate item-level detection to strengthen patient safety monitoring.
In addition, although CAREBot was available to patients at any time, automated alerts generated outside the center’s operating hours were not reviewed until the next working period. Because no dedicated after-hours monitoring or emergency response service was available through CAREBot, the system could not ensure immediate clinical intervention in response to emerging suicide risk. This limitation further underscores the need to use CAREBot as an adjunct to routine clinical case management rather than as a substitute for emergency psychiatric care.
Additionally, psychological assessments were completed at varying frequencies and time points according to participant preferences. The pre-, mid-, and post-assessment scores used for descriptive analysis were selected retrospectively from available records rather than collected according to a strictly standardized assessment schedule. This may have introduced selection bias and limited the comparability of psychological score trajectories across participants. Therefore, the observed changes in psychological assessment scores should be interpreted cautiously and cannot establish the clinical effectiveness of CAREBot. Because participants continued to receive routine case management and no control group was included, the observed score changes cannot be attributed specifically to CAREBot and may reflect ongoing clinical support, regression to the mean, or natural recovery from an acute crisis.
Another limitation is that the three social workers who evaluated CAREBot had also participated in its development by providing clinical feedback. This overlap between development and evaluation roles may have introduced social desirability bias, potentially influencing their survey ratings and interview responses. Therefore, the social worker-reported findings should be interpreted cautiously, and future evaluations should involve independent professionals who were not involved in system development. The qualitative analysis was based on researcher-documented feedback statements rather than complete verbatim interview transcripts, potentially limiting the contextual depth and completeness of the findings.
Furthermore, formal pre-deployment usability testing—such as structured heuristic evaluation or a dedicated pilot phase involving a small user subset—was not conducted prior to implementation; system refinement instead relied on internal developer testing followed by iterative clinical review. Future development efforts should incorporate formal usability testing protocols prior to full-scale deployment. In addition, the closed, rule-based chatbot structure—designed to prioritize safety and standardization—has inherent limitations in fully capturing individualized emotional responses [15]. Future research should include large-scale, multi-center validation studies and further evaluate the clinical utility of hybrid digital intervention models that combine emergency department-community linkage frameworks with both rule-based structures and open-ended AI response generation [5,9,15].
5. Conclusions
This mixed-methods study suggests that a rule-based, safety-first chatbot can be feasibly integrated into post-emergency department care for patients at risk of suicide as a structured adjunct rather than a replacement for clinical services. Within this pilot implementation, CAREBot was designed to support structured psychological assessment, post-discharge monitoring, and the identification of predefined risk signals while maintaining clinician oversight. Although the participating social workers perceived potential benefits for case management, the small number of evaluators and their involvement in system development limit the generalizability of these findings. The clinical effectiveness of CAREBot, including its impact on risk detection and workflow efficiency, remains to be established in future studies. Larger controlled studies are warranted to evaluate the clinical effectiveness of CAREBot and to investigate the feasibility and safety of integrating LLMs with rule-based mechanisms in high-risk psychiatric settings.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare14223981/s1, Supplementary Table S1: URL access events by participant.
Author Contributions
Conceptualization, H.C. (Hyunsu Choi); methodology, H.C. (Hyunsu Choi); software, S.H. (Sunjin Hwang); validation, J.A., S.H. (Sejin Heo), T.K. and M.K.; formal analysis, H.C. (Hyunsu Choi); investigation, H.C. (Hyunsu Choi) and C.J.; resources, C.J., J.A., S.H. (Sejin Heo), T.K. and M.K.; data curation, S.H. (Sunjin Hwang) and C.J.; writing—original draft preparation, H.C. (Hyunsu Choi); writing—review and editing, H.C. (Hansol Chang); supervision, H.C. (Hansol Chang); project administration, J.Y. and W.C.; funding acquisition, J.Y., W.C. and H.C. (Hansol Chang). All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea, grant number RS-2024-00335925.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Samsung Medical Center (protocol code SMC 2025-05-164-003; date of approval: 22 August 2025).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data presented in this study are available from the corresponding author upon reasonable request. The data are not publicly available to protect participant privacy, as they include participant-level mental health assessment data.
Acknowledgments
The authors thank all participants for their involvement in this study. They gratefully acknowledge the case management team of the Samsung Lifeline Center at Samsung Medical Center for their support in the implementation of this study. During the preparation of this manuscript, the authors used ChatGPT (GPT-5; OpenAI, San Francisco, CA, USA) for language editing and improving the clarity and readability of the manuscript. The authors reviewed and edited the output and take full responsibility for the content of this publication.
Conflicts of Interest
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Abbreviations
The following abbreviations are used in this manuscript:
| API | Application Programming Interface |
| CDSS | Clinical Decision Support System |
| COREQ | Consolidated Criteria for Reporting Qualitative Research |
| GAD-7 | Generalized Anxiety Disorder 7-item scale |
| GRAMMS | Good Reporting of A Mixed Methods Study |
| LLMs | Large Language Models |
| PHQ-9 | Patient Health Questionnaire-9 |
| SBQ-R | Suicidal Behaviors Questionnaire-Revised |
References
- Miller, I.W.; Camargo, C.A., Jr.; Arias, S.A.; Sullivan, A.F.; Allen, M.H.; Goldstein, A.B.; Manton, A.P.; Espinola, J.A.; Jones, R.; Hasegawa, K.; et al. Suicide prevention in an emergency department population: The ED-SAFE study. JAMA Psychiatry 2017, 74, 563–570. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Boudreaux, E.D.; Camargo, C.A., Jr.; Arias, S.A.; Sullivan, A.F.; Allen, M.H.; Goldstein, A.B.; Manton, A.P.; Espinola, J.A.; Miller, I.W. Improving suicide risk screening and detection in the emergency department. Am. J. Prev. Med. 2016, 50, 445–453. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Inagaki, M.; Kawashima, Y.; Kawanishi, C.; Yonemoto, N.; Sugimoto, T.; Furuno, T.; Ikeshita, K.; Eto, N.; Tachikawa, H.; Shiraishi, Y.; et al. Interventions to prevent repeat suicidal behavior in patients admitted to an emergency department for a suicide attempt: A meta-analysis. J. Affect. Disord. 2015, 175, 66–78. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- World Health Organization. Suicide Worldwide in 2021: Global Health Estimates; World Health Organization: Geneva, Switzerland, 2025. [Google Scholar]
- O’Keeffe, S.; Suzuki, M.; Ryan, M.; Priebe, S.; Byng, R.; Simpson, A.; Araújo-Soares, V.; Albert, R.; Fialho, R.; Walker, N.; et al. Enhanced psychosocial assessment and rapid follow-up care for people presenting to emergency departments with self-harm and/or suicidal ideation: The Assured feasibility study and internal pilot trial. Pilot Feasibility Stud. 2025, 11, 20. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Asarnow, J.R.; Babeva, K.; Horstmann, E. The emergency department: Challenges and opportunities for suicide prevention. Child Adolesc. Psychiatr. Clin. N. Am. 2017, 26, 771–783. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Bell, J.; Davies, B.; Walsh, C.; Knowlman, H.; Hefford, T.; Kuipers, P. The role and potential of social worker involvement in hospital emergency departments: A practice-based scoping review. Int. J. Soc. Work 2018, 5, 79. [Google Scholar] [CrossRef] [Scilit]
- Pellatt, R.A.; Painter, D.R.; Young, J.T.; Kõlves, K.; Keijzers, G.; Kinner, S.A.; Heffernan, E.; Crilly, J.; Broadbent, M.; Johnston, A.; et al. The risk of repeated self-harm and suicide after emergency department presentation with self-harm in mental health presenters: A retrospective cohort study with data linkage in Queensland, Australia. Lancet Reg. Health West. Pac. 2025, 54, 101263. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Stanley, B.; Brown, G.K.; Brenner, L.A.; Galfalvy, H.C.; Currier, G.W.; Knox, K.L.; Chaudhury, S.R.; Bush, A.L.; Green, K.L. Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry 2018, 75, 894–900. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Firth, J.; Torous, J.; Nicholas, J.; Carney, R.; Pratap, A.; Rosenbaum, S.; Sarris, J. The efficacy of smartphone-based mental health interventions for depressive symptoms: A meta-analysis of randomized controlled trials. World Psychiatry 2017, 16, 287–298. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Fitzpatrick, K.K.; Darcy, A.; Vierhile, M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial. JMIR Ment. Health 2017, 4, e19. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Li, H.; Zhang, R.; Lee, Y.C.; Kraut, R.E.; Mohr, D.C. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. npj Digit. Med. 2023, 6, 236. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Casu, M.; Triscari, S.; Battiato, S.; Guarnera, L.; Caponnetto, P. AI chatbots for mental health: A scoping review of effectiveness, feasibility, and applications. Appl. Sci. 2024, 14, 5889. [Google Scholar] [CrossRef] [Scilit]
- Philippe, T.J.; Sikder, N.; Jackson, A.; Koblanski, M.E.; Liow, E.; Pilarinos, A.; Vasarhelyi, K. Digital health interventions for delivery of mental health care: Systematic and comprehensive meta-review. JMIR Ment. Health 2022, 9, e35159. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Hua, Y.; Siddals, S.; Ma, Z.; Galatzer-Levy, I.; Xia, W.; Hau, C.; Na, H.; Flathers, M.; Linardon, J.; Ayubcha, C.; et al. Charting the evolution of artificial intelligence mental health chatbots from rule-based systems to large language models: A systematic review. World Psychiatry 2025, 24, 383–394. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Pichowicz, W.; Kotas, M.; Piotrowski, P. Performance of mental health chatbot agents in detecting and managing suicidal ideation. Sci. Rep. 2025, 15, 31652. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Seyedsalehi, A.; Bailey, J.; Ogonah, M.G.T.; Fanshawe, T.R.; Fazel, S. Prediction models for self-harm and suicide: A systematic review and critical appraisal. BMC Med. 2025, 23, 549. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- McCarthy, J.F.; Cooper, S.A.; Dent, K.R.; Eagan, A.E.; Matarazzo, B.B.; Hannemann, C.M.; Reger, M.A.; Landes, S.J.; Trafton, J.A.; Schoenbaum, M.; et al. Evaluation of the Recovery Engagement and Coordination for Health–Veterans Enhanced Treatment Suicide Risk Modeling Clinical Program in the Veterans Health Administration. JAMA Netw. Open 2021, 4, e2129900. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Campbell, R.J. The Five Rights of Clinical Decision Support: CDS Tools Helpful for Meeting Meaningful Use. J. AHIMA 2013, 84, 42–47. [Google Scholar] [PubMed]
- Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Fetters, M.D.; Curry, L.A.; Creswell, J.W. Achieving integration in mixed methods designs—Principles and practices. Health Serv. Res. 2013, 48, 2134–2156. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef] [Scilit]
- Tong, A.; Sainsbury, P.; Craig, J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int. J. Qual. Health Care 2007, 19, 349–357. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- O’Cathain, A.; Murphy, E.; Nicholl, J. The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 2008, 13, 92–98. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Kroenke, K.; Spitzer, R.L.; Williams, J.B. The PHQ-9: Validity of a brief depression severity measure. J. Gen. Intern. Med. 2001, 16, 606–613. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Spitzer, R.L.; Kroenke, K.; Williams, J.B.; Löwe, B. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch. Intern. Med. 2006, 166, 1092–1097. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Osman, A.; Bagge, C.L.; Gutierrez, P.M.; Konick, L.C.; Kopper, B.A.; Barrios, F.X. The Suicidal Behaviors Questionnaire-Revised (SBQ-R): Validation with clinical and nonclinical samples. Assessment 2001, 8, 443–454. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Kim, T.H.; Song, K.J.; Shin, S.D.; Hong, K.J.; Lee, J.K. National implementation of emergency department-based follow-up program for suicidal attempts. J. Korean Med. Sci. 2022, 37, e245. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ahn, E.; Kim, J.; Moon, S.; Ko, Y.H.; Cho, H.; Park, J.H.; Song, J.H.; Kim, H.N.; Jee, J.Y.; Han, R.Y. Effect of a crisis intervention team for suicide attempt patients in an emergency department in Korea. Hong. Kong J. Emerg. Med. 2020, 27, 92–98. [Google Scholar] [CrossRef] [Scilit]
- Inagaki, M.; Kawashima, Y.; Yonemoto, N.; Yamada, M. Active contact and follow-up interventions to prevent repeat suicide attempts during high-risk periods among patients admitted to emergency departments for suicidal behavior: A systematic review and meta-analysis. BMC Psychiatry 2019, 19, 44. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Thunström, A.O.; Ali, L.; Krage Carlsen, H.; Bohm, M.; Wesén, L.; Wrede, O.; Sarajlic Vukovic, I.; Hellström, A.; Larson, T.; Steingrimsson, S. Behavior Emotion Therapy System and You: Co-design and evaluation of a mental health chatbot and digital human for mild to moderate anxiety in healthy participants. JMIR Form. Res. 2025, 9, e66163. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Wrightson-Hester, A.R.; Anderson, G.; Dunstan, J.; McEvoy, P.M.; Sutton, C.J.; Myers, B.; Egan, S.; Tai, S.; Johnston-Hollitt, M.; Chen, W.; et al. An artificial therapist (Manage Your Life Online) to support the mental health of youth: Co-design and case series. JMIR Hum. Factors 2023, 10, e46849. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Fernandez-Quijano, I.; Herrera-Peco, I.; Lopez-Espuela, F.; Suarez-Llevat, C.; Moreno-Sanchez, R.; Ruiz-Nunez, C. Artificial intelligence in suicide prevention: A systematic review of randomized controlled trials on risk prediction, fully automated interventions, and AI-guided treatment allocation. Psychiatry Int. 2025, 6, 143. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.



