Empowering Clinical Engineering and Evidence-Based Maintenance with IoT and Indoor Navigation ^†

Abstract

The OHIO (Odin Hospital Indoor cOmpass) project received funding from the European Union’s Horizon 2020 research and innovation action program, via ODIN–Open Call, which is issued and executed under the ODIN project and focuses on enhancing hospital safety, productivity, and quality by introducing digital solutions, such as the Internet of Things (IoT), robotics, and artificial intelligence (AI). OHIO aims to enhance the productivity and quality of medical equipment maintenance activities within the pilot hospital, “Le Scotte” in Siena (Italy), by leveraging internal informational resources. OHIO will also be completely integrated with the ODIN platform, taking advantage of the available services and functionalities. OHIO exploits Bluetooth Low Energy (BLE) tags and antennas together with the resources provided by the ODIN platform to develop a complex ontology-based IoT framework, which acts as a central cockpit for the maintenance of medical equipment through a central management web application and an indoor real-time location system (RTLS) for mobile devices. The application programmable interfaces (APIs) are based on REST architecture for seamless data exchange and integration with the hospital’s existing computer-aided facility management (CAFM) and computerized maintenance management system (CMMS) software. The outcomes of the project are assessed both with quantitative and qualitative methods, by evaluating key performance indicators (KPIs) extracted from the literature and performing a preliminary usability test on both the whole system and the graphic user interfaces (GUIs) of the developed applications. The test implementation demonstrates improvements in maintenance timings, including a reduction in maintenance operation delays, duration of maintenance tasks, and equipment downtime. Usability post-test questionnaires show positive feedback regarding the usability and effectiveness of the applications. The OHIO framework enhanced the effectiveness of medical equipment maintenance by integrating existing software with newly designed, enhanced interfaces. The research also indicates possibilities for scaling up the developed methods and applications to additional large-scale pilot hospitals within the ODIN network.

1. Introduction

Contemporary healthcare facilities are complex and broad environments that satisfy the needs of both in-patients and out-patients through a wide range of services. The widespread integration of advanced medical technologies is observed across various specialties, necessitating continuous interventions for effective administration. However, the rapid advancement of medical technologies has not consistently corresponded with a concurrent adaptation in hospital infrastructures. Numerous hospitals persist with outdated designs, posing navigational challenges for users and staff members [1]. The integration of digital solutions, including the Internet of Things (IoT) [2], robotics [3], cloud computing [4], computer vision [5], and artificial intelligence (AI) [6], is increasingly pivotal in virtually every aspect of healthcare. These technologies aim to optimize daily duties by relieving workers from stressful and time-consuming tasks. IoT, robots, and AI can assume a critical role in supporting human workers, particularly in high-stress situations, by managing non-essential tasks, thereby contributing to enhancing overall performance [7].

ODIN—a European initiative funded through Horizon 2020, which is the EU’s Research and Innovation program dedicated to advancing top-tier science—embodies a noteworthy project with a primary focus on augmenting safety, productivity, and quality within hospital settings [8]. The central objective of ODIN is to develop an open digital platform that facilitates services leveraging robotics, IoT solutions, and specialized AI. These resources undergo comprehensive testing across seven clinical use cases in leading hospitals spanning six European countries: Spain, France, Germany, Poland, the Netherlands, and Italy. The implementation is structured into three fundamental reference areas for hospital interventions: eWorkers, eRobots, and eLocations.

The ’eWorkers’ initiative concentrates on endowing hospital staff with technologies designed to alleviate them from arduous and time-consuming tasks, thereby enhancing routine activities. Concurrently, eRobots seeks to automate hospital processes through the incorporation of robotics technology, enabling human workers to concentrate on core responsibilities while the technology manages ancillary tasks. Lastly, eLocations allow medical settings to intelligently support hospital processes through appropriate instrumentation. These medical locations are equipped with sensors and technologies that engage with humans, along with high connectivity, to facilitate efficient communication among workers, robots, and devices. Collectively, these intervention areas address a wide spectrum of hospital aspects, encompassing logistics, robotics, IoT, and disaster management [9].

1.1. Unmet Needs

Planned and corrective maintenance interventions require technical personnel to access internal department spaces to locate targeted devices. These interventions are susceptible to potential failures due to various reasons, highlighting a series of unmet needs (UNs):

• UN 1: Technicians require assurance that the devices slated for maintenance are not in use by healthcare staff upon their arrival.
• UN 2: Technicians need the certainty that the devices are easily locatable; however, instances often arise where the devices are untraceable or loaned to another department.
• UN 3: Maintenance of devices must not impede the interaction between healthcare and technical staff.
• UN 4: Suppliers and biomedical technicians (BMETs) need accurate information on how to reach the assigned devices. Outdated knowledge of department layouts poses challenges for external personnel navigating within the structures.

Currently, healthcare organizations struggle to adopt effective medical equipment maintenance management without suitable systematic procedures and planning. Maintenance, service, and restoration of this equipment are time-consuming procedures. Thus, if the maintenance activity is performed ineffectively, the equipment may remain unavailable for an extended time [10]. A well-organized approach is required to guarantee that the equipment selected for inspection or repair is available at a designated hot-spot collection area inside the department. In the realm of clinical engineering (CE), the absence of a universally agreed-upon “Electro-medical Equipment Hub” (EEH), which refers to an area or room within wards where devices are handed over on the intervention day, necessitates that technicians disrupt healthcare processes to access devices.

The maintenance of medical equipment becomes more expensive every year, and hospital management structures are constantly looking for solutions to extend the life of the equipment to optimize maintenance programs and reduce the total cost of ownership through the efficient use of available resources [11]. Maintenance activities are broadly categorized into “corrective maintenance” (CM) and “scheduled maintenance” (SM). The former is a type of maintenance task performed after an equipment failure, referring to all activities that restore failed or broken-down assets to their normal working condition, while the latter involves scheduled activities performed before equipment failure, grouping all activities that maintain assets and prevent them from failure or breakdown. In this context, real-world data (RWD), i.e., observational data associated with outcomes in real-world settings, can be leveraged to generate real-world evidence (RWE). This evidence aids in assessing the effectiveness and safety of health technologies by examining both the intended and unintended consequences of their use. Unfortunately, computerized maintenance management system (CMMS) software often only contains descriptions of failures, repair procedures, and spare parts used, lacking information about measures needed to prevent failures [12]. On the other hand, evidence-based maintenance (EBM), a systematic process that examines the root causes of equipment malfunctions, can improve maintenance methodologies continuously by harnessing empirical RWD and scientific research. EBM identifies the most effective maintenance strategies for medical devices, to bolster efficiency, dependability, and safety [13].

1.2. Objectives

The OHIO (Odin Hospital Indoor cOmpass) project, funded under the European Union’s Horizon 2020 research and innovation initiative through ODIN–Open Call, implemented as part of the ODIN project (GA 101017331), is led by the Department of Medical Biotechnologies at the University of Siena, Italy. Its primary objective is to enhance the safety, productivity, and quality of maintenance of medical equipment within hospitals [14] by proposing a novel approach that significantly expedites the maintenance process, increases the success rate of interventions, and minimizes disruptions to healthcare operations, through the definition and the introduction of EEHs. The OHIO project exploits the custom hospital’s CAFM system named SPOT [15], and leverages the adoption of an EBM-based approach to medical equipment maintenance by offering a standard classification of the possible failures and assessing the actual quality of the maintenance by calculating objective and comparable key performance indicators (KPIs).

The proposed IoT framework represents a significant innovation in the field of clinical engineering, both for the solutions created from scratch (such as the backend interfaces and the entire IoT framework for indoor navigation) designed for maintenance, as well as the interoperability guaranteed through the use of protocols and components from the ODIN platform, along with the ODIN ontology [16], which are also further extended by the proposed solution.

2. Related Works

The manufacturer’s maintenance recommendations have been subjected to a critical analysis process in recent years, which has prompted clinical engineers and health technology management (HTM) specialists to adopt evidence-based procedures to preserve the dependability and safety of medical equipment while using their resources wisely. Most hospital maintenance reports just outline the failures, repair procedures, and spare components used. These reports never mention any procedures that may have been taken to prevent the failure. The EBM-based approach allows a knowledge of the history of a failure and enables the monitoring and improvement of the current maintenance strategy so that the most appropriate approach can be found [17].

A real-time locating system (RTLS) is used to automatically identify and track the locations of objects or people in real time. An RTLS is made up of specialized fixed receivers or readers that receive wireless signals from small badges or tags attached to items of interest and/or people to establish their location within a building or another confined indoor or outdoor space [18]. Tracking the position of portable MDs automatically improves a hospital’s ability to manage medical equipment and respond to emergencies. The key benefit of adopting tracking technologies is that they show promise for enhancing operational efficiency by knowing the precise location of MDs [19,20]. However, some drawbacks prevent such solutions from being deployed on a large scale in hospitals [21]. Current RTLS solutions prove inadequate in meeting the logistical support requirements of CE, as they are primarily designed for tracking patients and visitors [22]. Particularly, the usage of RTLS for asset/device tracking presents limitations concerning cost-effectiveness and scalability to large-scale manufacturing [23].

There are currently no industry-standard solutions that enable positioning in every kind of indoor environment [24]. The literature presents several solutions based on different fingerprinting and multimodal approaches, using technologies such as RFID (radio frequency identification), UWB (ultra wide-band), infrared, ZigBee, WiFi, and Bluetooth Low Energy (BLE) [25]. BLE is a wireless personal area network (WPAN) technology created and managed by the Bluetooth special interest group. The synergy between good performance, low power consumption, and ubiquitous diffusion (BLE is available in all laptops, tablets, and smartphones) makes BLE an excellent candidate for a great variety of applications, including e-health applications, automotive applications, voice communications, and domotics for healthcare environments and smart houses [26]. Although RFID is flexible regarding tag size, the cost of RFID readers and unstable received signal strength make it less effective compared to BLE [27]. Moreover, RFID shows the drawback of requiring several RFID readers, while a WiFi-based approach offers a higher range but suffers from a lack of accuracy. Compared to traditional Bluetooth, BLE is designed to consume significantly less power while retaining a comparable communication range and accuracy (2–3 m) [28].

3. Materials and Methods

The case study is the hospital “Le Scotte” in Siena (Italy), which covers 8375 rooms across 13 buildings, covering an area of 155,314 square meters, accommodating 43 departments, 220 operative units, 689 beds, and 3071 employees. The premise also includes 270,772 assets, with 44,353 of these assets identified as electro-medical equipment. The hospital serves as a reference for approximately 900,000 citizens in southeast Tuscany, accommodating more than 21,000 ordinary hospitalizations and about 3,200,000 outpatient services per year. The management of the medical equipment and its maintenance is entrusted to the Health Technology Department of the Regional Health Service Body of Tuscany (ESTAR). Dedicated internal clinical engineers and technicians take care of designing scheduled and corrective maintenance protocols according to the directives provided by manufacturers, while the maintenance actions are entrusted to an external global service provider. Work orders (WOs) on both SM and CM are stored inside the hospital CMMS.

The Department of Medical Biotechnologies developed a computer-aided facility management (CAFM) information system, SPOT, used at the hospital “Le Scotte” in Siena. The web-based software is designed to meticulously map the internal organization of the hospital, including the designated purposes of various areas and performed activities, providing quantitative, qualitative, and graphical reports. It is grounded on a Microsoft SQL server DBMS (database management system) linked to other hospital databases (Figure 1). A standalone main application outlines the current state of the premises, encompassing details such as the number of beds, square meters, designated purposes, functional areas, and other pertinent information for each room.

The main software is provided with additional modules such as a custom web search engine, a document manager, and a mobile application for indoor navigation named HiWay [24] providing two distinct navigation modes contingent on the device’s actual positions: on-site and off-site. In the on-site mode, users are directed to their intended destination, the coordinates of which are retrieved via SPOT through dedicated APIs (application programming interfaces). The off-site mode is for path planning, facilitating the addition of multiple destinations to generate complex route maps, as illustrated in Figure 2. During the development of the first release of the application, the design team considered which real-time location system (RTLS) was best suited for the use-case hospital in terms of employed technologies and cost-effectiveness. The outcomes are detailed and reported in the work by Luschi et al. [24]. The employed RTLS, which consumes positioning information, is IndoorAtlas [29]; it utilizes multiple technologies through sensor fusion, incorporating alterations in magnetic fields, variations in WiFi and Bluetooth signals from installed routers, signals from BLE (Bluetooth Low-Energy) beacons, and position changes detected by accelerometers and gyroscopes. BLE was preferred above the other available IPS technologies because it is the de facto communication protocol for the IoT [30]. It is also the widest-used solution for IPS in hospitals [25,26] as it provides an inexpensive solution and less critical technology in terms of electromagnetic interference (EMI) [31], allowing smooth compatibility for smartphones, tablets, and other Bluetooth-enabled devices while ensuring inexpensiveness and accuracy. The RTLS provides the coordinates of a smartphone running HiWay, allowing for real-time navigation within the hospital venue. This multifaceted approach ensures sharper and faster universal positioning, optimizing estimation timing and accuracy.

3.1. Deployment

The OHIO framework utilizes a REST architecture to enhance data exchange and facilitate seamless integration with third-party applications. It relies on two primary front-end applications: a web application and an upgraded and redesigned version of the HiWay mobile application. Both applications are deployed within a complete IoT framework specifically designed to optimize the maintenance process of medical equipment for clinical engineering services. The whole framework has been specifically developed to overcome the identified unmet needs (Section 1.1). In particular, the upgraded version of HiWay is now able to perform more complex and advanced features (see Section 4.2) than the previous release, which was developed only for patient and visitor navigation. The management web application and HiWay interact with a set of APIs to share data with the underlying back-end technologies and infrastructures. Specifically, the dedicated management module interacts with the hospital’s CMMS to retrieve work orders (WOs) associated with the selected medical equipment, streamlining maintenance activities. This module cross-checks data about the medical equipment and its maintenance schedule with precise device location information provided by SPOT. Communication with SPOT allows the APIs to dynamically retrieve up-to-date destination location information, ensuring instant reflection of any structural or organizational changes in HiWay. Apache Cordova is used as the mobile development framework for HiWay, chosen for its continuity with the previous version [24].

The Health Technology Manager identified specific Electro-medical Equipment Hubs (EEHs) as collection and maintenance areas for each ward of the building in the pilot hospital used for initial testing. This strategy restricted the deployment area and facilitated quicker data collection. The positions of mapped medical devices are retrieved using BLE tags attached to them. Bluetooth antennas were strategically installed in the identified collection areas to capture the available BLE signals (Figure 3). BLE antennas and tags were provided by the external technology vendor BlueUp [32]. Only movable equipment was tagged (fixed installation devices, such as computed tomography (CT) and magnetic resonance imaging (MRI) scans, were excluded because they do not need to be moved to the EEH for maintenance as their positions are time-invariant and known by technicians). The initial tests in the pilot hospital focused solely on ultrasound (US) machines, totaling 14 devices. US scans were chosen because the Health Technology Manager believed they would benefit most from the proposed framework, given the number of losses in previous years. The identified hubs were marked in SPOT and associated with the related medical equipment to facilitate BMETs in locating and maintaining equipment without the need for rescheduling due to missing intervention. When a recognized tag enters or exits the reach of an antenna, both its MAC (Media Access Control) address and the antenna’s address are transmitted to the main OHIO server within the hospital’s network. The new features of HiWay, together with the functionalities provided by the management application, ensure that the status of the equipment that must undergo maintenance (collected in the EEH or not) is known in real time. This allows for better scheduling of the maintenance operations, preventing the identified unmet need from occurring (Section 1.1). A detailed description of both the management web application and the new version of HiWay is provided in Section 4.

Figure 4 shows the fingerprinting quality related to WiFi, magnetic, and BLE mapping for a given story of the hospital. The WiFi and magnetic coverage alone are not able to provide a sufficient level of environmental quality, which is considerably improved by the installation of the BLE beacons.

3.2. Integration with the ODIN Platform

An instance of the ODIN platform (Figure 5) was deployed on a dedicated server within the hospital network, ensuring its communication with both the SPOT and OHIO servers. The modules of the ODIN platform that OHIO exploited include the IoT gateway (an Eclipse Mosquitto MQTT Broker) [33], the semantic triple store (Apache Jena Fuseki [34]) for the ODIN ontology, and the enterprise service bus (ESB, namely Apache Kafka [35]). The technologies are part of the inner architecture of the ODIN platform as described in the deliverable D3.11 of the ODIN project [36]. The MQTT Broker collects the incoming messages from the BLE antennas, published via MQTT (message queuing telemetry transport) protocol; Apache Jena Fuseki is responsible for serving resource description framework (RDF) [37] data over HTTP (hypertext transfer protocol) performing semantic translations to the ODIN ontology; Apache Kafka is an open-source distributed event streaming platform for high-performance data pipelines, streaming analytics, and data integration, ensuring syntactic transport.

Connectors were developed to allow communication between the above-mentioned OHIO applications (management web application and HiWay) and the local deployment of the ODIN platform, allowing them to be registered as key enabling resources (KERs). The first deployed connector translates the payload message from the MQTT protocol to a standardized ODIN-compliant JSON format and subsequently publishes it on a dedicated topic on the ESB. This information is then retrievable via another MQTT connector subscribed to the same Kafka topic, which exchanges information with the OHIO APIs. The development of the web application, APIs, and ODIN connectors was executed within the Microsoft .NET 6 framework, leveraging the MQTTnet [38] and Confluent’s .NET client [39] libraries. OHIO also interfaces with the ODIN ontology (Figure 6), ensuring the desired level of abstraction. Various aspects of information about the healthcare facility, including buildings, stories, and rooms, are systematically mapped using the building topology ontology. The inner organization of the hospital, encompassing departments and operative units, is represented through an organizational ontology. Employee data are classified using the National Cancer Institute Thesaurus (NCIT) ontology for medical and allied health occupations and the friend of a friend (FOAF) ontology for non-medical personnel. Data related to medical devices are structured using the OdinEMDN ontology. Additionally, an extension of the ODIN ontology incorporating the Dublin Core Ontology (DC) [40] has been proposed and implemented. This extension integrates digital documents by building abstractions for a variety of file types, including images, texts, videos, and more.

OHIO interacts with the ODIN ontology by storing information about performed maintenance on the Apache Fuseki semantic triple store. To ensure that relational data stored in the SPOT Microsoft SQL Server DBMS are accessible through the ODIN platform via semantic SPARQL queries, they need to be exposed through the ODIN ontology. R2RML (RDB to RDF mapping language) is a language used for expressing customized mappings from relational databases to RDF datasets. Such mappings enable the viewing of existing relational data within the RDF data model, expressed in a structure and target vocabulary chosen by the mapping author. R2RML mappings are themselves RDF graphs and are written in Turtle syntax [41]. Therefore, an R2RML connector was developed, utilizing the R2RML4net library. This connector processes the created R2RML mapping graph against the SPOT SQL database. This approach ensures seamless integration of information from the SPOT relational DBMS into the ODIN ontology, facilitating data retrieval through semantic SPARQL queries. The schema in Figure 7 summarizes all the developed connections among the various components of the project as described so far.

3.3. Impact Assessment

OHIO actively supports the goals of the ODIN project by proving the ODIN platform’s replicability and scalability in a large and complex pilot scenario. Additionally, it expands the existing hospital’s IoT capabilities by incorporating new technologies. The project’s outcomes are evaluated using a specifically designed set of KPIs. Furthermore, the project’s outcomes are subject to a semi-quantitative assessment, employing usability questionnaires to evaluate user satisfaction with both the web and mobile applications. Fallback modalities have been designed to mitigate the potential negative impact of malfunctions in the proposed IoT framework. Specifically, if the RTLS fails, a QR code-based system allows users to retrieve the current position by scanning QR codes printed on the doorplates of rooms, which exchange information with the CAFM software. If the IoT Gateway is unreachable or a BLE antenna/tag malfunctions, the last known position of the equipment is displayed to the users, along with a message indicating that the information has not been updated.

3.3.1. Key Performance Indicators

Table 1. List of proposed KPIs used for assessing the impact of the OHIO project.

#	Name	Formula	U.M.	Target
KPI 1	Lag	$(InterventionDate-ScheduledDate)$	[days]	20% decrease
KPI 2	Duration	$(W{O}_{closed}-W{O}_{accepted})$	[dd:mm:ss]	10% decrease
KPI 3	Downtime	${\displaystyle \frac{T_{na}}{RT}}\ast 100$ $T_{na}$ = Time of non-availability $RT$ = Required time per year	[%]	10% decrease
KPI 4	SM with failure	${\displaystyle \frac{N_{SMwithfailure}}{N_{SM}}}\ast 100$	[%]	10% decrease
KPI 5	Devices per technician	${\displaystyle \frac{N_{devices}}{N_{technicians}}}$	[number]	10% increase

shows the implemented set of KPIs for the impact assessment. The expected targets were set by the health technology manager of the hospital.

KPI 1 and 2 were intentionally designed to comprehensively capture the impact of the management web application and the utilization of the IPS mobile application (HiWay) on maintenance timings. The other three indicators were drawn from prior works by the authors in the domain of EBM [12], which was designed following the UNI EN ISO 15341:2007 standard [42]. Specifically, from the original list, only the KPIs relevant to SM were chosen for assessment. It is noteworthy that no indicators related to costs were included, given that maintenance at the pilot hospital operates within a multi-year contract at a fixed fee.

1.

Lag. The KPI measures the delay in maintenance operations by calculating the difference between when the maintenance was conducted and when it was scheduled. This metric is critical for assessing the responsiveness of the maintenance team. Delays can lead to increased downtime and potentially more severe equipment issues. By monitoring this KPI, organizations can identify bottlenecks in their scheduling processes and improve their planning and execution to ensure timely maintenance, thus minimizing the impact on equipment availability.

2.

Duration. This KPI assesses the time management of maintenance tasks by tracking the period from when a work order is accepted by the technician to when it is officially closed after the maintenance is complete. This duration is a direct reflection of the maintenance process’s efficiency. It includes the time taken for diagnosis, repair, testing, and any other associated tasks. By evaluating this metric, an organization can pinpoint inefficiencies and improve the turnaround time, which is crucial for maintaining high equipment availability and reliability.

3.

Downtime. It quantifies the proportion of time that equipment is not operational and available for use compared to the total required operational time within a year. It is a significant indicator of equipment reliability and maintenance effectiveness. High downtime not only affects the immediate availability of the equipment but also impacts potential service loss to patients, which can influence the quality of care. Reducing downtime through effective maintenance can increase the operational time of equipment, thereby enhancing the overall productivity of the medical facility.

4.

SM with failure. It investigates the percentage of maintenance actions that resulted in identifying a fault versus all scheduled maintenance activities. This indicator helps to evaluate the quality of the maintenance program. A high number of SMs with failures might indicate that the equipment is prone to issues or that the preventative maintenance schedule is not as effective. It can drive improvements in maintenance procedures or strategies, such as enhancing the predictive maintenance capabilities or revising the maintenance frequency and procedures.

5.

Devices per technician. The last KPI evaluates the average number of devices each technician is responsible for within a given class of biomedical equipment. It gives insights into workload distribution and staffing adequacy. If the ratio is too high, it may suggest that technicians are overburdened, which could lead to rushed maintenance tasks and potential errors. Conversely, a low ratio might indicate that there are too many technicians, which could be less cost-effective. By monitoring this KPI, an organization can balance its staffing levels to ensure effective maintenance without incurring unnecessary costs.

3.3.2. User Satisfaction

User satisfaction was evaluated by administering a custom questionnaire at the end of the testing phase, one month after the implementation began, to the five involved BMETs. Currently, there are no validated guidelines [43] or formalizations in the implementation and administration of usability tests [44], so the employed questionnaire was derived from a prior usability protocol designed by Denisova et al. [44]. The answers given to each of the proposed questions were scored according to an agreement scale ranging from 1 (strongly disagree) to 5 (strongly agree).

Table 2. List of questions for the user satisfaction questionnaire.

#	Question
1	The software/application exhibits intuitive usability.
2	The software/application has the potential to enhance the effectiveness of my work.
3	No issues or bugs were encountered in the utilization of the various features of the software/application.
4	No difficulties were detected in utilizing the various functionalities provided by the software/application.
5	The various functionalities of the software/application are clear and comprehensible.
6	The graphical interface and various icons are intuitive and clear.
7	Any error messages provided are sufficiently informative.
8	The software/application provides clear feedback that aids in understanding whether an activity has been completed.
9	I would like to regularly utilize this software/application.
10	I believe no training or initial support is necessary for the utilization of the software/application.
11	I felt comfortable in the utilization of the software/application.
12	No issues were encountered in the utilization of the navigation module (only for the HiWay mobile application).

shows the 12 questions listed in the questionnaire.

The outcomes of user satisfaction will allow the development of upgraded and improved GUIs for future releases of the applications.

4. Results

4.1. The OHIO Management Web Application

Figure 8 shows the main page of the OHIO management web application. Maintenance WOs can be imported inside the application by the CE department via a standard CSV (comma-separated value) template, which enables communication between the hospital CMMS software and OHIO. The provided text format enables software-independent data transferring, which can be implemented despite the specific used CMMS.

Once the user selects a specific working date via the calendar input at the top of the web page, the ID, assigned technician, starting and closing dates and times of the maintenance, priority, maintenance fault class, and notes are displayed for each WO. The ID refers to the actual WO identification number as it appears inside the hospital CMMS; the Assigned Technician and WO Starting Date time are set once a BMET takes charge of the WO with HiWay (see Section 4.2); WO Closing Date time is automatically set by HiWay when a BMET closes a WO (see Section 4.2); the CE department can specify a Priority in the web application to notify BMETs that certain interventions are more critical than others; Fault Class represents the fault classification code assigned to the maintenance by the BMET once a WO is closed (with possible additional notes displayed in the field Notes). The failure codes come from a work by Iadanza et al. [12] and are shown in

Table 3. Failure Codes.

Code	Description
NPF	No problem found
BATT	Battery failure
ACC	Accessory failure (including supplies)
NET	Failure related to network
USE	Failure induced by use (e.g., abuse, accident)
UPF	Unpreventable failure caused by normal wear and tear
PPF	Predictable and preventable failure
SIF	Induced by service (i.e., caused by a technical intervention not properly completed or premature failures of a part just replaced)
ENV	Environmental causes (e.g., construction yards, cleaning services, flooding)
UTIL	Induced by utilities (e.g., electrical power, medical gas or vacuum, ventilation)

.

The interface also shows the list of the devices related to the WO (by clicking on the Device blue button) and a green check mark is displayed whenever the equipment has been collected in the associated EEH.

4.2. HiWay Mobile Application

The previous version of HiWay provided navigation to fixed points of interest for patients and external users [24]. The upgraded version includes new features specifically designed for managing medical equipment maintenance in conjunction with the OHIO management web application. HiWay now allows BMETs to access the WOs, take charge of them, navigate to the medical equipment, and classify the maintenance. Specifically, the new GUI (graphical user interface) of the application displays the assigned WOs once a technician has logged into the app (see Section 4.3). For each device, the basic information is displayed (inventory number and description) as well as the room code of the designed EEH, highlighted in green if the medical equipment is currently collected (Figure 9a). BMET can access any documentation related to the equipment and stored inside the SPOT document manager thanks to the data exchange features provided by HiWay and its communication with the other KERs through the ESB of the ODIN platform (Figure 10a). The IndoorAtlas RTLS integrated into the app, along with the deployed BLE beacons, allows technicians to navigate in real time to the selected device (Figure 9b). Once the maintenance is completed, BMETs can close the intervention by clicking the red stop icon. The user interface then asks for a fault class to be selected (Figure 10b) while the current date and time are saved inside the mobile device. When all the maintenance duties are completed, technicians are asked to upload the locally stored information to the OHIO server for storage and analysis.

4.3. Authentication and Data Protection

While authorization with the RTLS third-party solution is demanded on their inner architecture [29], the authentication of HiWay with OHIO relies on the basic authentication security protocol: an HTTP POST method is executed to a dedicated API controller, sending the Base64-encoded username and password of the current user together with a secret API key within the request header. If the authentication is successful, the system returns a JWT (JSON Web Token) that is valid for the next 24 h. The application then exploits the JWT for future authentications through the bearer authentication security protocol. Basic authentication is simple and convenient, and it has been chosen over other protocols because confidentiality, integrity, authenticity, and anonymity are guaranteed by the hospital’s virtual private network (VPN), under which the whole framework is installed. VPN technologies are enough to ensure robust authentication and data privacy [45].

4.4. KPIs and User Satisfaction

Table 4. Values for the proposed KPIs for the years 2021, 2022, 2023 (OHIO not deployed), and 2024 (OHIO deployed and in use).

Year	KPI 1 Lag (Avg)	KPI 2 Duration (Avg)	KPI 3 Downtime (Avg)	KPI 4 SM with Failure	KPI 5 Devices per Technician
2021	17 days	01 h 00 m 00 s	0.0344%	11.11%	2.67
2022	109 days	00 h 53 m 45 s	0.0283%	35.71%	2.20
2023	−2 days	00 h 57 m 30 s	0.0323%	21.43%	3.00
2024	0 days	00 h 33 m 55 s	0.0110%	14.29%	3.50

shows the values for the proposed set of KPIs (see Section 3.3) for the last four years. During 2021, 2022, and 2023 OHIO was not set up and deployed as yet, while data for 2024 were collected after the SM had been performed by exploiting the OHIO framework.

The results show that the preset targets for each KPI (Table 1) were reached:

• KPI 1 decreased from an average of 41 days in the last three years to 0 days.
• KPI 2 decreased from an average value of about 57 min to 33 min and 55 s (around 40% lower).
• KPI 3 has changed from 0.0317% on average to 0.0110% while using the proposed framework (a reduction of around 65%).
• KPI 4 decreased from an average value of 19.05% to 14.29% representing a decrease of 25%.
• KPI 5 increased from an average value of 2.62 devices per BMET to 3.50 (34% more).

Figure 11 reports the statistical description of user satisfaction as the frequency count and percentage for each of the 12 assessing questions.

5. Discussion

All identified scenarios that could potentially lead to failures (see Section 1.1) were observed during the testing phase. The OHIO management web application played a crucial role in satisfying the UN 1. It achieved this by proactively notifying healthcare staff responsible for a medical device scheduled for maintenance, prompting them to collect the equipment in the EEH. When a device designated for clinical use was required within the scheduled maintenance period, the OHIO management application promptly reported this information to the CE department well in advance. This preemptive communication enabled a timely rescheduling of the maintenance, avoiding excessive time waste. Despite the theoretically designated collecting area, HiWay effectively shows the real-time positioning of tagged medical equipment within the hospital. During the initial testing, a technician was able to locate the device effortlessly, even though it had been collected in a different “filter zone” due to a prior loan between departments the CE department was unaware of (UN 2). The collection of medical equipment in the designated EEH, their specific position, and the precise information of the day and time of intervention provided to the healthcare personnel by the OHIO management web application allows maintenance operations to remain separate from medical ones (UN 3). Moreover, the indoor navigation capabilities embedded in HiWay empower each technician to access the collecting area without requiring a comprehensive understanding of the hospital layout, thus fulfilling the UN 4. This situation has also already occurred during the tests, and HiWay effectively prevented an issue where a newly hired technician assigned to the SM of a tagged US machine would have struggled to independently reach the device without the assistance of HiWay. Therefore, OHIO successfully mitigated all of the identified UNs. Table 4 highlights that all the KPIs improved after the introduction of OHIO.

KPI 1 (Lag) changed from an average of 41 days over the previous three years to 0 days, meaning that each SM has been performed (on average) on the same days it was initially planned.

KPI 2 (Duration) reduced by approximately half in comparison to the previous three years, reaching a value of 33 min and 55 s on average (the minimum recorded time was 8 min and 32 s). HiWay played a pivotal role in reducing KPIs 1 and 2 because it provided a faster and more certain way to reach the equipment.

KPI 3 (Downtime) for 2024 is about one-third compared to the previous years. In the calculation, the $RT$ (required time per year) was set to 8760 h/year for the medical equipment in use 24/7 activities (e.g., ICU, ER, and ward), and to 3000 h/year (12 h a day for 250 working days per year) for the others (e.g., outpatient clinics, non-emergency imaging). The certainty of finding the devices in the EEH assured by the OHIO framework (both software and IoT hardware) contributes to reducing both KPIs 2 and 3. It is worth noting that data extracted from the hospital CMMS showed that at least one piece of medical equipment out of the 14 identified items was not found during the SMs in 2021, 2022, and 2023 (one item in 2021, and two items both in 2022 and 2023). As a result, the SM could not be performed even though the maintenance was rescheduled several times during the year. OHIO mitigated this problem, as the BMETs promptly located the previously unfindable medical equipment using HiWay, ensuring that all 14 identified US scans were found and underwent maintenance.

KPI 4 (SM with failure) is the only KPI that is not strictly correlated to the introduction of OHIO, as it is related to the equipment itself rather than to the maintenance actions. However, the introduced failure classification (Table 3) helped the technicians to better categorize and assess the nature of the identified failures.

KPI 5 (Devices per technician) shows that the introduction of OHIO allows each BMET to perform more SMs in comparison to the past (3.50 in 2024 vs. 3.00 in 2023), which is strictly correlated to the reduction of KPI 2 (less time needed for single maintenance reflects into more maintenance activities for each technician).

The user satisfaction questionnaire, which has been administered to the five BMETs that have used and tested OHIO, shows a global positive evaluation of the usability of both the management web application and HiWay, as well as their GUIs (questions no. 4, 5, 8, and 11 received all positive scores). The globally positive score to question no. 9 (60% of answers) also highlights that the technicians would promote the use of the OHIO infrastructure. However, even though the calculated KPIs demonstrate that OHIO brought an actual improvement in the effectiveness of the SM, this improvement is not perceived by the BMETs, as only 20% of the answers to question no. 2 refer to a positive score. When asked about this, the tested technicians revealed that the low scores stem from a general skepticism in believing that such a framework could be extended to the whole hospital and that the positive collaboration with the medical staff in collecting the medical equipment in the EEH within the required timeframe could be maintained outside the boundaries of testing purposes.

6. Conclusions

The primary objective of the OHIO project is to enhance the management capabilities of a large pilot hospital, specifically the “Le Scotte” hospital in Siena, Italy, with a focus on clinical engineering and logistics. This involves augmenting the existing CAFM system (SPOT) and the IPS mobile application (HiWay) to improve the effectiveness of maintenance activities through the integration of established software, newly designed advanced interfaces, and seamless information sharing through the ODIN platform. The research provides CE services with a novel set of tools that aims to increase equipment reliability, reduce downtime, and mitigate failures. These efforts help provide a safer, more efficient, and higher-quality healthcare environment for both patients and healthcare providers. The designed IoT framework revealed significant outcomes regarding the improvements of SM activities for the analyzed medical equipment. The proposed failure codes and the list of KPIs offer a structured, standardized, and measurable approach for data collection and analysis, promoting the adoption of EBM practices in CE services and expanding the informative content of CMMS software by allowing for an immediate classification of maintenance interventions. The impact assessments based on KPIs and usability post-test questionnaires indicate promising results in improving the effectiveness of the SM while offering simple and intuitive GUIs. This underscores the project’s capacity to positively influence hospital and SM operations by addressing previously unmet needs. The solution outlined by the OHIO Project not only represents a significant innovation in the field of clinical engineering and provides valuable insights into the maintenance of specific medical devices, but also offers a flexible and transferable framework that can be applied to a wide range of healthcare environments and equipment categories.