User Evaluation of Technology-Based Interventions Developed to Address Falls in an Inpatient Ward

Abstract

Preventing inpatient falls remains challenging for healthcare institutions globally, including in Singapore. Integrating technological innovations into fall prevention measures may optimize inpatient care and improve health outcomes. A multiphase study was conducted from 2019 to 2022, employing a human-centred design (HCD) approach to develop a technology-based inpatient fall prevention system (IFPS). The four phases include (1) pre-design observations and focus groups, (2) feature prioritization and wireframe development, (3) prototype testing and safety assessments, and (4) post-design staff training and feedback collection. The developed IFPS integrated artificial intelligence (AI) video analytics for bed-exit prediction with communication devices and autonomous commode delivery to facilitate ward communication and reduce staff workload. This paper describes the development process and user evaluation of the IFPS to assess its operational usability and safety. Potential users of the IFPS, such as ward nurses and patients, suggested features for the IFPS during the pre-design phase and thereafter evaluated the system through focus group discussions and/or feedback surveys. Pre-design focus group participants (n = 24) emphasized durability and user-friendliness requirements, informing system design. When evaluating the system, nurse users (n = 39) perceived the IFPS as effective in reducing falls (65%), enabling them to perform other duties (85%), and allowing them to remain with patients without searching for a commode (64%). Patient users (n = 21) found pre-recorded messages effective (91%), though communication clarity varied. Engaging healthcare workers in IFPS development offered valuable context-based insights, highlighting the importance of addressing technology acceptance factors early to promote adoption of fall prevention technologies in acute care settings.

1. Introduction

Each year falls account for 38 million disability-adjusted life years (DALYs) lost globally [1]. Falls continue to be among the top sentinel events, defined by “a patient safety event that results in death, permanent harm, or severe temporary harm”, with most incidents occurring in hospital settings [2]. In Singapore, a recent study conducted in 2020 reported an inpatient fall rate of 1.15 per 1000 patient days at a community hospital [3]. Among those who fell, 27% to 72% sustained injuries such as skin abrasions, bruises, lacerations, sprains, fractures, dental or head injuries, which could lead to longer hospital stays or disabilities [4]. Inpatient falls impose significant burden on both patients and healthcare systems. A study by Wong et al. (2002) found that older adults with fractures were hospitalized for an average of 17 days, with hospitalization costs ranging from Singapore dollars 6335 to 10,515 [5]. Beyond financial costs, patients experience diminished quality of life, while healthcare facilities must allocate substantial resources to manage fall-related conditions.

While strategies to prevent falls in healthcare facilities are well-established, inpatient falls remain a persistent challenge worldwide, including Singapore. Fall prevention strategies are multifaceted, targeting various levels, from policy, organization, and individuals [6,7]. In Singapore, inpatient fall prevention efforts are guided by the national clinical practice guidelines available. These efforts include conducting fall risk assessments at the time of hospitalization and throughout the patient’s hospital stay, identifying and monitoring high fall risk patients, improving environmental safety, assisting with patients’ daily activities, and educating patients, families, and hospital staff [8,9]. However, several of these fall prevention efforts rely heavily on healthcare providers, such as nurses, for continuous patient monitoring and care [10,11,12]. Increasing staff-to-patient ratios for monitoring is not economically viable in the long term and is further complicated by the global shortage of healthcare workers [13].

There is a growing body of research on integrating technology into fall prevention strategies to optimize care within the ward. Current sensor-based fall prevention technologies, such as wearable sensors and pressure sensors, are primarily designed for fall risk assessment, fall detection, bed-exit detection, and patient monitoring. Some of these technologies are commercially available and provide fall detection through assistive devices that alert caregivers when a fall occurs [14,15,16]. Recent advancements in video analytics and the increasing availability of affordable high-resolution cameras and powerful computing units have also improved the feasibility of conducting real-time remote patient monitoring without requiring human effort to manually watch and review video footage [17,18,19,20,21,22,23,24]. Nevertheless, many technology-based solutions that could improve patient outcomes and health system efficiency remain unadopted [25,26,27,28].

Human-Centred Design (HCD), defined as “an iterative, collaborative, and people-centred approach for designing products, services, and systems”, provides a methodology for developing systems that align with clinical workflows and user needs [29,30,31,32]. HCD prioritizes early and continuous engagement with stakeholders and end-users to ensure solutions address real-world healthcare challenges and increase the likelihood of successful implementation.

This study employed a HCD approach to develop an inpatient fall prevention system (IFPS) comprising a redesigned inpatient ward workflow that incorporates predictive video analytics, assistive robotics, and communicative devices to enhance patient safety and staff productivity. To the best of our knowledge, the development and adoption of our proposed IFPS components to address inpatient falls remained relatively nascent, with limited evidence on their feasibility in hospital setting. In our effort to address this gap, our paper describes the development process and reports on user evaluation of the IFPS to assess its safety, appropriateness, and acceptability prior to real-world implementation and effectiveness trials. User evaluation was conducted through focus group discussions and feedback surveys administered after users tested the system. As this was a formative evaluation study, objective performance data and clinical effectiveness outcomes were not reported.

2. Methods

A multiphase study was conducted at an acute public hospital in Singapore from May 2019 to September 2022. The study site selected for the deployment of an IFPS was a 43-bedder medical subsidized ward located at Changi General Hospital. The ward accommodates both male and female adult patients with diverse medical conditions and varying ambulation capabilities, thereby presenting a range of fall risks.

Development of the IFPS

Firstly, a research team was formed comprising medical doctors, nurses, physiotherapists, health service researchers, and academic collaborators specializing in engineering product development, robotics automation, and wireless communications. The research team worked collaboratively to ideate, develop, and evaluate prototypes for the proposed IFPS. This iterative process also involved active engagement with the primary users of the IFPS, such as nurses and physiotherapists, given their integral roles in patient care in the ward and collecting inputs and frequent feedback across the different study phases. The key phases of the study are summarized in

Table 1. Overview of study phases guided by Sanders and Stappers co-design framework [33].

Phases	Purpose	Research Activities Conducted
Pre-design	To understand context-specific experiences and prepare people to participate in the co-design process	Onsite observations and time-motion studies to understand existing workflows and identify challenges in preventing inpatient falls Rapid review to identify the main components of new IFPS and characterize patient profiles who would use the technology solution Focus group discussions with stakeholders to gather insights on expectations for the technology solution and experiences with fall-risk patients
Generative	To produce ideas, insights, and concepts	Developed a list of functions and features for IFPS components through stakeholder discussions with engineering and product development experts Categorized features as ‘Good to have’, ‘Must have’, or ‘Critical to have’ based on budget and timeline constraints Created wireframes for user interface workflow with nursing stakeholders and mapped functionalities between devices and users Finalized envisioned system through multiple iterations and feedback sessions with stakeholders
Evaluative	To assess the effectiveness, usefulness, or usability of products, systems, or services	Built physical prototypes and programmed software using an iterative development approach aligned with user requirements Conducted multiple rounds of testing in mock-up ward environments and virtual ‘Gazebo’ simulations Trialled prototypes in the study ward with nurses reviewing functionality against planned workflow Focus group discussions with nurses and physiotherapists to assess operational feasibility and identify critical safety issues
Post-Design	To understand context-specific experiences and prepare people to participate in the co-design process	Deployed a complete IFPS in the study ward as part of a feasibility study Developed a manual package with troubleshooting guides and workflows for each prototype Provided training to ward nurses and managers on the expected use of prototypes Conducted surveys with patients and nurses to gather user feedback on deployed system

Note: Bolded text represents the categorical groupings used for the row and column headings.

, guided by Sanders and Stappers (2014) co-design framework [33].

2.1. Study Phases

2.1.1. Pre-Design Phase

In the first phase, the study team conducted onsite observations within the general inpatient wards to understand the existing workflows, identify challenges in preventing inpatient falls, and scope the patient profile. These observations were recorded through time-motion studies and field notes. Simultaneously, a rapid review was undertaken to identify the main components of the new IFPS and to characterize the profiles of patients who would prospectively use the technology-based solution.

Focus Group Discussions (FGDs) were held in person to gather insights into participants’ expectations for the technology-based solution and proposed new workflow for inpatient fall prevention, and their experiences in caring for fall risk patients. At the beginning of each FGDs, participants were shown an overview of the proposed project scope, which consisted of video analytics to monitor patients’ movement, nurse wearables and bedside communicators, and assistive devices that would autonomously travel to the patient’s bedside. Participants were also introduced to user personas, consisting of fictional presentations of patient profiles commonly encountered in a ward, which served as stimuli to provide contextual grounding during the FGDs. A total of three FGD sessions, each lasting one hour, were conducted in person during this phase.

2.1.2. Generative Phase

The findings from the pre-design phase generated a list of functions and features for the various components of the IFPS. Discussions with collaborators, including those with engineering and product development expertise, were held to closely examine and determine features and functions that were ‘Good to have’, ‘Must have’, and ‘Critical to have’ based on descriptions of what is essential and desirable. While these categories facilitated discussions as to how the features and functions will be prioritized, further considerations were made to the project’s budget and timeline constraints. After shortlisting potential features and functions for the prototypes, a wire framing of the user interface (UI) workflow was developed with the nursing stakeholders, the future direct users, by sketching the integration of workflow with prototypes and processes to illustrate the envisioned fall prevention system. This was to ensure that the UI of the prototypes would complement the overarching workflow of the system. Thereafter, the research team mapped the functionalities and interactions between devices and users, including the expected activation of functions and utilization within the proposed workflow.

The envisioned system (Figure 1), including both the workflow and prototypes, was finalized through discussions and feedback sessions with key healthcare stakeholders in ensuring that each prototype met operational demands and user expectations, and conceptually feasible for deployment at the study site. The IFPS components consist of (1) video analytics, (2) bedside communicator, (3) nurse communicator, and an (4) autonomous commode (Figure 1). Several iterations were made prior to finalization of the system to refine its anticipated implementation in different hypothetical but real-world scenarios.

2.1.3. Evaluative Phase

Once the envisioned system was finalized, the form and hardware of each prototype were built, and the software was programmed. An iterative approach to innovation was employed during prototype development to ensure alignment with user requirements. Each prototype version underwent multiple rounds of testing in a mock-up ward environment and robotic simulation in a ‘Gazebo’ virtual environment. This allowed collaborators to build and test the robots using realistic physics and sensor models. Where applicable, additional testing was conducted to ensure that the prototypes complied with industry standards, such as the electronic components. Once the prototypes were assessed for readiness, they were trialled for operability in the study ward. Ward nurses and nurse clinicians reviewed the prototypes during the onsite trials and confirmed the functionality of each prototype against the planned workflow, finalizing the prototype versions for evaluation.

An in-person session was held with nurses and physiotherapists to present the final versions of the prototypes and collect feedback on the user interface (look) and user experience (feel). This allows primary users to assess the operational feasibility and identification of critical safety issues to be corrected before the actual deployment of the prototypes in the ward. Additionally, insights into potential error scenarios and their acceptable troubleshooting methods and workarounds were gathered to address any issues the IFPS might encounter during the full deployment phase.

2.1.4. Post-Design Phase

In the final phase, the whole IFPS, comprising both the prototypes and workflow, was set up in the study ward for deployment, allowing end-users to experience the system. The research team developed a manual package containing troubleshooting guides and workflows for each prototype. Training was provided to ward nurses and managers on the expected use of the prototypes. Nurse leaders were kept informed of the study’s progress and prototype deployment, to ensure that key stakeholders could address any serious concerns if an adverse event occurred. A survey was then conducted with both patients and nurse participant groups to gather user feedback.

2.2. Participants

Participants were recruited using purposive sampling to ensure that meaningful insights into inpatient fall safety and the relevant work processes were gathered. There were two groups of participants, healthcare professionals and patients. Healthcare professionals would participate in FGDs and feedback surveys, whereas patients would complete the surveys only. Most government hospitals in Singapore practice multi-disciplinary, team-based care, meaning that patients in the ward often receive care from doctors, nurses, and allied health professionals [34]. Therefore, the research team identified a mix of healthcare professionals, such as nurses, occupational therapists, physiotherapists, and doctors, for the pre-design FGD. The inclusion criteria for the healthcare professional group were, (1) above the age of 21, and (2) had experience with caring for patients across various ward disciplines. The inclusion criteria for the patient group were, (1) above the age of 21 and (2) not assessed to be bedbound, to ensure that they can exit the bed and activate the IFPS.

Healthcare professionals were recruited through, internal advertisements, referrals, and word-of-mouth. Ward nurses were informed about the study during roll call or briefing sessions held before or after their work shifts, and they were provided with copies of the informed consent forms. Nurses who expressed interest in participating in the FGDs notified the research team. All nurses working during IFPS deployment in the study ward were asked to complete feedback surveys at the end of their participation.

Patients were recruited upon hospitalization in the study ward. After screening for eligibility, the research team informed patients or their legally authorized representatives about the study and obtained written informed consent from those who agreed to participate. Patient participants then experienced the system and completed a feedback survey at the end of their participation.

2.3. Conducting FGDs

FGDs were conducted during the pre-design phase to systematically engage participants in exploring perceptions and ideas aimed at developing an end-to-end integrated solution for preventing inpatient falls and improving staff productivity at a public hospital in Singapore [35].

The FGD sessions were held in a hospital meeting room and scheduled to accommodate participants’ availability. The sessions were facilitated by nurse clinicians with extensive training, qualifications, and experience in leading discussions. These facilitators played an instrumental role in creating an open and conducive environment for discussion, ensuring that all participants had the opportunity to share their experiences and insights. Their combined efforts facilitated a robust collection of qualitative data, aligning with the study’s objectives of understanding the nuances of patient care from different professionals’ perspectives.

The development of semi-structured guides for the FGDs was informed by a literature review on healthcare innovation and refined by collaborators with expertise in engineering and product development. The guide was designed to elicit responses pertaining to the physical form and function of prototypes, workflow of the proposed system, and user perspectives on the solution’s effectiveness. The guide was reviewed by a multidisciplinary research team for content validity. Participants were also encouraged to share their experiences in caring for a diverse range of patients at risk of falling. They discussed how the proposed solution may apply to the fictional representations reflected in the user personas. The final guide was tested and revised within the research team and amongst colleagues working in the same hospital.

2.4. User Feedback Survey

Surveys were developed specifically for this study to gather user feedback on each IFPS component. As the IFPS was considered novel, survey items were designed de novo based on the system’s intended functions and key themes or priorities identified during the pre-design FGD. Separate surveys were developed for nurse and patient participants, to reflect their distinct roles and interactions with the system. The survey was reviewed by the multidisciplinary research team and non-participating ward nurses to ensure content validity and relevance to the clinical and study context. Due to the formative nature of this evaluation and context-specific IFPS, formal psychometric validation was not conducted and may present a limitation.

Nurses and patients were recruited during the post-design phase to participate in the evaluation. A self-administered paper-based survey was distributed to the nurses at the conclusion of the deployment period. Nurse survey respondents were nurses who had provided direct patient care and participated in the use of IFPS at the study site during the deployment period. The research team provided nurse survey respondents the survey form to fill after explaining the purpose of survey during roll call. For the recruited patients, research team members assisted with administering the paper-based survey. Before patients were discharged or transferred to another location, research team members provided the survey form to the patient participants and offered assistance with translation and recording their responses in cases of language barriers, vision impairments, or limited hand mobility.

2.5. Data Analysis

Each FGD session was audio-recorded and transcribed verbatim for analysis. Following each FGD, a research team member played back the audio recording and manually transcribed the discussion. Analysis was conducted manually using Microsoft Excel, with no qualitative data analysis software used. For familiarization, research team members had an initial read of the three transcripts, generated from the FGDs, to gain an overall sense of the discussions and identify potential themes. The research team members then extracted relevant excerpts from the discussions and organized them systematically before conducting classical content analysis to cluster codes in the data and form thematic categories. This involves line-by-line open coding of the transcripts and categorizing data to IFPS components, system requirements for fall prevention, user concerns, and implementation considerations. These are then grouped based on content similarity, to form broader themes. Reading and coding of data was done iteratively. Investigator triangulation was used to confirm findings and the different perspectives [36]. Disagreements identified during the analysis were discussed with the study lead who was facilitating resolution and consensus-building.

Survey responses were analyzed using descriptive statistics and summarized using proportions. The 4-Likert scale responses were dichotomized into ‘Agree’ and ‘Disagree’ categories, combining ‘Strongly Agree’ with the ‘Agree’ responses and similarly for ‘Strongly Disagree’ and ‘Disagree’. Dichotomization was performed to provide a clear summary statistic of overall user acceptability towards the IFPS. This aligns with the study’s objective of assessing whether the IFPS have met the minimum acceptability threshold necessary before it is deployed for pilot or effectiveness testing.

Participants completed survey items that were relevant to the devices they used during the study. They could select a ‘not applicable’ option if they did not experience the use of the device. Therefore, data analysis included survey responses that were provided by experienced users only, excluding those who had selected ‘not applicable’. Due to the short study period, there were low survey responses regarding the use of commodes among patients (n ≤ 2) and therefore, were not included in the analysis.

3. Results

Twenty-four healthcare professionals participated in the FGDs, comprising nurses (n = 20), allied health professionals (n = 3), and a medical doctor (n = 1). All participants had experience in acute ward patient care, with diverse experiences in caring for adult to geriatric patients with varying physical limitations and co-morbidities.

Following the FGDs and subsequent discussions with engineering collaborators, features and functions were prioritized as ‘Critical’, ‘Must have’, or ‘Good to have’ (Table S1). Ideas and suggestions regarding IFPS components and workflow (Figure 1) generated from the FGDs informed the design and development of prototypes and workflows (Table S2 and Figure S1). After prototype development and workflow refinement, nurses and physiotherapists tested and appraised the prototypes. The developed prototypes were confirmed to have incorporated critical features identified in the FGDs and would not disrupt nurses’ duties. Thus, the IFPS was deployed in the study ward for two months to trial its operational feasibility. Survey responses were then gathered from nurses (n = 39) and patients (n = 21) who had experienced or used the IFPS. All participants approached had agreed to complete the feedback survey.

Survey responses by the nurses and patients are presented in

Table 2. Nurses feedback regarding the IFPS’ camera and deployment and use of communicators and commodes.

IFPS Component	Questions	(n = )	Agree (%)	Disagree (%)
Camera	The camera allowed me to perform my other ward duties without being distracted by the need to monitor high fall risk patients who were recruited to the study	27	85	15
Camera	I felt my privacy was protected by the masking of my face in the video recording captured by the camera	28	93	7
Camera	The camera was accurate in predicting bed exits by patients	28	68	32
Camera	The camera offered alerts with adequate response time for me to intervene and prevent patients from getting out of their beds	28	71	29
Communicator	Trouble shooting of the iPod Touch was easy with the provided guide	28	93	7
Communicator	I received adequate training on how to do recording of customized message on the iPod Touch	30	100
Communicator	It was easy to record customized message using the iPod Touch	23	87	13
Communicator	Trouble shooting of the communicator was easy with the provided guide	29	97	3
Communicator	I received adequate training on how to use the nurse communicator	31	97	3
Communicator	The camera and bedside communicator are effective enablers to ward staff in preventing fall risk patients from exiting beds independently	29	72	28
Communicator	I find the nurse communicator’s broadcast function useful	30	97	3
Communicator	It was easy and intuitive to use the nurse communicator	30	93	7
Communicator	The audio quality of the communicator was good when I was using it in the ward	30	83	17
Communicator	The other party who was using the communicator was able to hear me clearly when I was using the communicator in the ward	30	73	27
Communicator	I received an alert via nurse communicator whenever the walking frame/commode was not available for activation	13	85	15
Communicator	I received an alert via nurse communicator when the walking frame/commode had reached the bed side of a target patient upon activation	13	85	15
Commode	When the autonomous commode was deployed, it was parked at a bedside location which is within the reach of most patients in the ward	14	86	14
Commode	The number of autonomous commodes was enough to meet the patients’ needs in the ward	15	73	27
Commode	It was easy to operate the autonomous commode	16	81	19
Commode	The autonomous commode allowed me to accompany a patient at the bedside without having to walk away and look for the commode	14	64	36
Commode	The autonomous commode was able to reach bedside of a patient in less than 3 min upon activation	12	92	8
Commode	The autonomous commode’s alert was useful to prompt me that patient has changed from sitting on the commode to a standing position	12	100
Commode	I received adequate training on how to use the autonomous commode	20	95	5
Commode	Trouble shooting of autonomous commode was easy with the provided guide	16	88	13
Commode	In the new workflow, the deployment of camera, communicator, autonomous walking frame and autonomous commode is more effective in reducing inpatient falls compared to the previous workflow where such devices are not deployed	26	65	35

and

Table 3. Patient’s feedback regarding the IFPS’ camera and deployment and use of communicators.

IFPS Component	Questions	(n = )	Agree (%)	Disagree (%)
Camera	I felt my privacy was protected by the masking of my face in the video recording captured by the camera during my inpatient stay in the ward	21	100	-
Camera	I felt that my safety was enhanced with the automated monitoring by the camera	20	85	15
Communicator	The pre-recorded message was effective in reminding me to seek assistance before getting out of bed	11	91	9
Communicator	I was attended to by a nurse through the communicator in less than one minute	7	57	43
Communicator	When I used the bedside communicator to converse with a nurse, I could hear the nurse clearly	3	100	-
Communicator	The nurse could hear me clearly when I converse with a nurse through the bedside communicator	3	67	33
Communicator	The information I received on how to use the bedside communicator was adequate	18	95	5
Communicator	The bedside communicator allowed me to communicate with ward nurses in a timely manner	5	60	40

respectively. The findings from the FGDs and surveys were summarized under broad, overarching themes to highlight healthcare-specific needs and priorities for context-specific innovation.

3.1. Ease of Use and Durability

Pre-design Phase FGD requirements:

Focus group participants identified ease of use and durability as essential prerequisites for successful technology implementation. They emphasized that devices must be intuitive to operate.

“nurse wearable communicators…..lightweight and easy to wear, and durable…..lightweight in a sense that it is not going to be heavy on them when they move around. Something that they can actually still do their work without hindering the nurses”

—N20, nurse participant

Survey evaluation:

Post-deployment survey responses indicated that the ease-of-use requirement were largely met. Nurses found the communicators (93%) and autonomous commode (81%) easy to use. The communicator’s audio quality was perceived to be good (83%) and indicated that the other party could hear them clearly (73%) when using the communicators in the ward (Table 2). Participants also reported that it was easy to troubleshoot the communicators (97%) and autonomous commode (88%) using the guides provided. While patients reported being able to hear the nurse clearly through the bedside communicator (100%), only 67 percent felt that the nurses could hear them clearly in return (Table 3).

Long-term durability was not assessed during the two-month evaluation period and should be examined in extended studies in the future.

3.2. Safety and Privacy

Pre-design requirements

Patient safety was one of the more prominent themes in the discussions. Participants raised several concerns about patient–device interactions in the context of a diverse inpatient population with varying health needs. Participants felt that patients should be supervised with the use of assistive devices, as patients with limited mobility may require more support from nurses. A patient’s health may also fluctuate rapidly, necessitating careful assessments of a patient’s cognitive and physical capabilities before using the prototypes. Prompt alerts or early notifications with sufficient lead time were requested to allow nurses to attend to patients before the prototypes reach patients’ bedside. A consistent emphasis was placed on prioritizing patient safety throughout the process, particularly through early nurse notification, enabling timely intervention and preventing incidents such as unsupervised bed exits.

“I’m just thinking whether so the current like commode is also like you make sure that the patient sits all the way in, so sometimes I mean I don’t know so maybe you want to suggest some smart feature of it is that making sure that the patient is really seated all the way in because sometimes maybe they are urgent but they are not properly positioned so make sure they really sit all the way in deeply and then or like some kind of alert”

—N23, therapist participant

Survey evaluation

Survey findings revealed slightly divergent perspectives between nurses and patients regarding safety outcomes. Most patients felt that their safety was enhanced with the automated monitoring by the camera (85%) (Table 3), but less than two-thirds of the nurses perceived the bed-exit predicting capabilities to be accurate (68%) and may offer adequate response time for nurses to intervene a patient’s bed-exit (71%) (Table 2). This suggests that while the IFPS concept aligns with the safety requirements identified in the FGDs, prediction accuracy and timing of alerts need to be optimized to meet operational safety standards.

Most nurses agreed that the autonomous commode was deployed in accordance with the planned workflow (86%), capable of reaching the patient’s bedside within three minutes of activation (92%) and provided useful alerts indicating when patients attempted to stand (100%) (Table 2). However, only two-thirds of nurses perceived that the autonomous commode enabled them to accompany a patient without needing to search for a commode (64%).

Both nurse (93%) and patient (100%) participants found that their privacy was protected despite having a camera placed at the beds due to measures put in place, such as face-masking. Although privacy-related matters were not raised during the pre-design FGDs, it was considered an important feature to ensure participant anonymization.

3.3. Perceived Usefulness and Effectiveness

Survey evaluation

Most nurses found the communicator’s broadcast function useful (97%), cameras monitoring high-risk patients allowed them to perform other ward duties (85%), and that the system alerted the nurses in accordance with the planned IFPS workflow (85%) (Table 2). Additionally, 72 percent of nurses perceived the cameras and communicators as effective enablers in preventing fall-risk patients from independently exiting their beds. However, only 65 percent of the participants perceived the overall IFPS to be more effective in reducing inpatient falls compared to the existing workflow (65%).

From the patient perspective, the pre-recorded message was considered effective (91%) in reminding them to seek nurse assistance before exiting the bed (Table 3). At the same time, almost half of the patients did not agree that the bedside communicators allowed for timely communication with ward nurses (40%) or that nurses attended to their call through the communicator within one minute (43%)

3.4. User Willingness to Use the System Components

Pre-design requirements

Focus group participants emphasized that device usage may be influenced by individual patient preferences and comfort level. Furthermore, it would be important to assess patient suitability to use the IFPS, as patients may have varying medical conditions which could limit their ability to use the devices.

“Some elderly they might not know, they tend to forget also. At least with the buttons, they can still press rather than just touch the screen”

—N14, nurse participant

Additionally, participants suggested incorporating features that may support utility of devices and patient care. Some of the features included two-way communication between patients and nurses and multilingual capabilities to accommodate patients from different ethnic groups who speak other languages and dialects.

“I think that communicator need to translate whatever the nurse says into the language that the patient can understand. Just like the iCOMM, I speak inside there and the other side patient communicator will just translate into the language that patient can understand. Something like that”

—N19, nurse participant

4. Discussion

This study employed an HCD approach to develop an integrated IFPS, addressing a gap in the use of fragmented single-technology approaches for fall prevention. The IFPS integrates AI video analytics, real-time communication devices, and autonomous commode delivery, creating a closed-loop workflow that promotes predictive fall prevention rather than reactive intervention.

Overall, the healthcare providers engaged in this study demonstrated a positive attitude towards the concept of an IFPS incorporating AI and robotics, showing no explicit resistance to its development. During the FGDs, participants identified important factors that could influence the system’s successful integration and adoption in the study ward. Though there was a plan to integrate AI to assist with patient bed-exit monitoring in the IFPS, participants in the pre-design phase were still understandably wary that patients’ health conditions can fluctuate, possibly resulting in greater physical or cognitive impairments, making it unsafe for patients to independently use the prototypes. In this case, the IFPS may serve as a monitoring tool, alerting healthcare providers when patients are about to exit their beds and enabling timely assistance. Additional considerations were made to ensure the safe deployment of the prototypes, particularly in managing environmental risks and hazards associated with deploying the autonomous commode in a ward where there is high human traffic.

This integrated approach demonstrates a notable shift from the conventional fall management strategies. To contextualize the contributions of our integrated IFPS within the broader landscape of existing fall management strategies, Table S3 compares our IFPS with recent fall detection studies. Unlike the wearable-based approach of Nooruddin et al. (2020) [37] and the proximity-based system of Wong et al. (2014) [38], our IFPS eliminates the need for patients to wear devices, addressing common compliance issues whilst maintaining continuous monitoring capabilities. The integration of multiple components—video analytics, real-time communication, and autonomous commode delivery—creates a comprehensive workflow that extends beyond simple alerting to facilitate coordinated care responses. Whilst existing systems focus primarily on detecting falls after bed exits occur, our IFPS anticipates patient needs and enables preventive interventions, addressing the underlying causes of risky bed exits rather than merely detecting them.

These design considerations emerged through extensive early stakeholder engagement and communication, including regular progress updates and gathering qualitative feedback, which proved to be crucial for the development of technology-based interventions [39,40,41]. Healthcare providers possess valuable clinical and patient experience that enhances the appropriateness of technology-based solution [41,42,43]. Findings from the FGDs offered valuable insights into context-based needs and potential challenges, allowing for adjustments to be made to the proposed workflow and ensuring the safe deployment of IFPS in the study ward for testing.

One important consideration made was to allow sufficient lead time from the point of system activation to the arrival of autonomous commodes at the patient’s bedside. This would enable healthcare providers to reassess the patients’ condition before they use the commodes. Fulfilling requests to allow nurses to attend to patients before the device prototypes reach patient’s bedside partially addresses the coordination challenges inherent in non-integrated systems. Healthcare professionals were also able to identify patient-related barriers that could be addressed through design considerations. This is important in healthcare as patient may have varying physical abilities, such as hearing and mobility impairments, and multilingual capabilities, whereby patients are not fluent in English. The FGD participants suggested incorporating modular or customizable features, such as recording instructional or alert messages in different local languages for users to select, so that the system can accommodate a wider range of patients [16].

The broad themes of the findings, to a certain extent, resonate with findings of other healthcare innovation research. Concerns were raised about environmental challenges, operational safety, and patient-related needs [44,45,46]. Though existing literature has reported concerns about data privacy regarding technology-based solutions [45,47,48], these concerns were notably not present in our study. Based on the user feedback survey in the post-design phase, both nurses and patients found that their privacy was protected as they were informed that people’s faces were obscured when they came into the camera’s view.

While most potential end users provided positive feedback regarding the perceived safety, productivity, and usability of the IFPS, several areas for improvement were identified. The nurses’ feedback highlighted the need to enhance the accuracy of AI for bed exit prediction and to investigate factors influencing the overall effectiveness of the IFPS. Additionally, based on patients’ feedback, refinement is needed to improve the audio clarity when using bedside communicators and timely communication between patients and nurses. Healthcare innovation will be increasingly essential to improve patient outcomes and address operational and logistical issues faced by the healthcare system, such as healthcare workforce shortages. The current shortage of healthcare workers exacerbates staff productivity and impacts work conditions [49]. Singapore has a high reliance on foreign healthcare workforce, which would not be a viable solution for workforce shortages in the long run, and requires implementation of innovative measures similar to our IFPS, which could help to reduce the workload of existing healthcare workforce [49].

The strong innovative culture embedded within Singapore’s healthcare system and the presence of a supportive organizational structure play a significant role in fostering the innovative drive and behaviour among healthcare providers, enabling them to engage meaningfully throughout the different co-design phases as observed in our study [28,42,43,50]. Nevertheless, there were several limitations to this study. Firstly, the study took place during the COVID-19 pandemic, which introduced significant logistical challenges. The pandemic caused delays in the procurement of prototype parts, which in turn hindered the timely deployment of prototypes. Additionally, restrictions on hospital visits and inpatient interactions limited the ability for external collaborators to explore the clinical setting. Although study extension was granted to account for the delays, the feasibility study had to be condensed to a one-month period. The shortened period made it difficult to adequately evaluate the IFPS effectiveness when deployed for patients’ use, limiting the depth of insights from real-world testing.

Secondly, this study evaluated user acceptance and operational usability but did not assess clinical effectiveness. Controlled studies comparing fall rates between IFPS and standard care are needed to evaluate the system’s effectiveness in preventing inpatient falls and to demonstrate quantitative advantages of the integrated approach. Thirdly, the small sample size and use of purposive sampling from a single inpatient ward introduces potential selection bias. Participants recruited through this approach may have differing technology acceptance to other non-participants. Additionally, findings from this context-specific evaluation may not generalize to other specialized wards, hospital settings, or countries with different healthcare infrastructure and workflows. The HCD approach used in this study may need to be adapted for other innovation projects, especially those targeting the vulnerable population. When engaging with the vulnerable population, considerations around power dynamics, agency, and cultural sensitivities are crucial to ensure effective engagement [29,41]. Lastly, the dichotomization of survey responses was performed to present the overall user acceptability of the IFPS. However, this approach may result in a loss of information as it potentially masks variations in the strength of user opinions and attitudes towards the system.

5. Conclusions

As healthcare demand continues to rise in a rapidly ageing society, coupled with the increasing burden of non-communicable disease, it is critical to develop solutions and interventions that can address both the healthcare needs of the population and the current shortage of the healthcare workforce [51]. In the development of such solutions, following the HCD approach is helpful for ensuring that they meet the complex needs of healthcare practices. This is evident in the development of our IFPS, where our active engagement of healthcare workers and potential end users offered valuable insights into context-based needs and anticipated challenges, which in turn allowed us to explore and address factors that would affect technology acceptance early on. Though the user evaluation study of our IFPS offered overall positive sentiments, there were improvement opportunities that need to be worked on to enhance its operability in inpatient fall prevention. Further studies are needed to evaluate the effectiveness of IFPS in a real-world setting.