Impact of Hospital Employees’ Awareness of the EMR System Certification on Interoperability Evaluation: Comparison of Public and Private Hospitals

: This study examined the awareness of the EMR certification system among employees of public and private hospitals that have obtained EMR certification. It also assessed how this awareness impacted the evaluation of EMR interoperability. The objective of this study is to contribute to the stable adoption and further development of EMR system certification in Korea. Data were collected through 3600 questionnaires distributed over three years from 2021 to 2023. After excluding 24 questionnaires owing to missing values or insincere responses, 3576 responses were analyzed. The analysis involved descriptive statistics, cross-tabulation, t -tests, ANOVA, and multiple regression using SPSS 26.0. The significance level ( α ) for statistical tests was set at 0.05. This study revealed differences in awareness of EMR system certification and interoperability among hospital employees. In both public and private hospitals, awareness of the EMR system certification positively influences the evaluation of interoperability.


Introduction
The rapid development of medical technology and evolution of disease patterns are driving the transition toward an information-centric society.Particularly, advancements in information and communications technology (ICT) within the medical field have had a significant impact on patient treatment and the organizational culture of medical institutions [1,2].At the core of this transformation was the introduction of electronic medical record (EMR) systems.An EMR system refers to a system that inputs, organizes, and stores records of patients' treatment, surgeries, and tests at medical institutions based on computerization [1].These systems play a pivotal role in enhancing the quality of medical services, improving accessibility, and boosting operational efficiency.This contributes to the acceleration of the informatization of medical services by providing patient information more safely and rapidly.Furthermore, the accessibility and efficiency of health and medical information have been enhanced, enabling the rapid and effective management of medical services.It also plays a crucial role in reducing documentation errors and ensuring consistency in patient management, thereby enhancing the accuracy of medical records [3].In other words, the EMR system has enhanced the efficiency of medical services and patient management.It has also significantly reduced errors in medical information by facilitating the management and exchange of medical data, thereby improving patient care [4,5].
Countries worldwide have introduced various certification systems to effectively implement and manage EMR systems.In the United States, the dissemination of these systems is being promoted through the Health Information Technology for Economic and Clinical Health Act (HITECH Act).The United States incentivizes medical institutions by providing financial rewards when they meet specific criteria [5].The European Union (EU) regulates EMR systems with an emphasis on protecting personal data; these regulations significantly influence the overall design and operation of these systems [5].Australia is advancing the adoption of EMR systems through its national e-health strategy, which aims to manage medical data more effectively and enhance the quality of patient care [5].Since the onset of the COVID-19 pandemic, restrictions on face-to-face medical services have intensified, leading to a heightened demand for non-face-to-face medical services.Accordingly, efficient management of digital health records, including EMR systems, is increasingly important.In this context, the standardization and certification of EMR systems are critical factors for determining the quality of medical services [3].
In Korea, despite the high adoption rate of EMR systems, there are disparities in the quality management of medical information among medical institutions attributable to variations in EMR systems.Accordingly, in accordance with Article 23-2 (Standardization of Electronic Medical Records, etc.) of the Medical Service Act, the EMR system certification has been implemented since June 2020.The purpose of this system is to encourage the development of standard products through national standards and verification of suitability for EMR systems to support patient safety and continuity of care, and to improve quality by ensuring interoperability of systems.Through this, the focus is on ensuring interoperability of medical information between medical institutions and improving continuity of care and patient safety [6].Through domestic and international cases, the EMR system certification has been shown to play a key role in improving the quality of medical services and improving the efficiency of medical data management.Therefore, the introduction of the EMR system certification is expected to go beyond simply meeting technical requirements and contribute to improving patient safety and the quality of medical services [7].As of December 2023, about 50 tertiary general hospitals and 300 general hospitals have obtained certification [8].
The certification criteria for Korea's EMR systems are categorized into functionality, interoperability, and security.Interoperability refers to the ability of two or more EMR systems to exchange information and effectively utilize it post-exchange [8].Given that the primary goal of EMR system certification is to ensure interoperability and enhance the quality of medical information, it is essential to ensure interoperability within EMR systems.
Korea's medical delivery system is predominantly led by the private sector, and most public hospitals have fewer competitive facilities and personnel than private hospitals [21].Therefore, confirming the performance of public and private hospitals in introducing EMR systems will be an important basis for building an advanced hospital information system in the future.At this point, four years have passed since the introduction of the EMR system certification in Korea, so there is a need to check the awareness and performance of the EMR system certification.So far, after the introduction of the EMR system certification in Korea, there have been studies confirming the awareness of the EMR system certification among medical institution workers [3,22], but there is no research comparing public hospitals and private hospitals.Therefore, through this study, we confirmed the awareness of the EMR system certification among workers in public and private hospitals, and confirmed the impact of the perception of the EMR system certification on the evaluation of EMR system interoperability to determine future EMR systems.Based on these findings, we aim to contribute to the stable establishment and development of the EMR system certification in the future.

Data Collection and Research Participants
This study targeted workers at hospitals that had obtained certification for the use of EMR systems, and their occupations were limited to doctors, nurses (including nursing assistants), medical technicians, health information managers, and administrators who frequently used EMRs because of the nature of their work.The survey was divided into three years, from 2021 to 2023, and questionnaires were distributed by sampling though convenience sampling for job category in hospitals who wished to participate in the survey.A total of 1200 questionnaires were collected each year for a total of 3600 questionnaires.The collected questionnaires were stored in a locked cabinet to protect personal information.Of these, 3576 were included in the final analysis, after excluding 24 with missing values or incomplete responses (Table 1).The total number of participants in this study was 3576, of whom 2146 worked in public hospitals and 1430 in private hospitals.Regarding the general characteristics of all the study participants, the number of workers in general hospitals was 2370 (66.3%), which was higher than that in tertiary general hospitals (1206 [33.7%]).Regarding gender, there were 2615 women (73.1%), which was higher than 961 men (26.9%).In terms of age, the distribution was in the following order: 1101 people in their 20s (30.8%), 1058 people in their 30s (29.6%), 898 people in their 40s (25.1%), and 519 people in their 50s or older (14.5%).Regarding the educational level, 2478 (69.3%) had a college degree, 551 (15.4%) had a graduate degree or higher, and 547 (15.3%) had a college degree or lower.The participants occupations were as follows: nursing: 1829 (51.1%); administrative workers: 600 (16.8%); medical technicians: 553 (15.5%); health information managers: 341 (9.5%); and doctors: 253 (7.1%).Years of service: 875 people (24.5%) had 5 to less than 10 years; 805 people (22.5%) had more than 20 years; 736 people (20.6%) had 10 to less than 20 years; 658 people (18.4%) had less than 3 years; and 3 years to less than 5 years was found to be the largest group at 502 (14.0%) (Table 2).

Recognition of EMR System Certification
To measure the awareness of the EMR system certification, a research tool was developed by consulting four experts (health information managers) working in the health and medical information team, the main department responsible for EMR system certification.The research tool consisted of a total of 5 questions and was evaluated on a 5-point Likert scale with 1 point being 'not at all like this', 2 points being 'not like that', 3 points being 'average', 4 points being 'agree', and 5 points being 'extremely agree'.The higher the score, the higher the awareness of EMR system certification.Specific questions included 'I know about the EMR certification system', 'I know that your hospital has received EMR certification', 'The quality of work has improved since EMR certification', 'Work efficiency has increased since EMR certification' and 'I realized the need for an EMR system certification'.Cronbach's α in this study was 0.907.

Interoperability of EMR
The interoperability of EMR systems is one of the EMR system certification criteria and refers to the ability to exchange information between two or more EMR systems and utilize the exchanged information [8].In this study, the interoperability of the EMR system was assessed with a total of 6 questions on a 5-point Likert scale: 1 point for 'not at all', 2 points for 'disagree', 3 points for 'average', 4 points for 'agree', and 4 points for 'very much'.'Yes' was evaluated on a scale of 5, and the higher the score, the better the evaluation of the interoperability of the EMR system.In this study, Cronbach's α was 0.947.

Ethical Considerations
Prior approval was obtained from the D University IRB (1041493-A-2021-008, 1041493-A-2022-003, and 1041493-A-2022-016) and the study was conducted according to approved research procedures.The department head of each hospital explained the purpose of this study and recruited participating research subjects with their cooperation.The recruited research subjects were notified of the purpose of this study, principles of anonymity and confidentiality, and the possibility of withdrawal from participation.Data were collected after voluntarily agreeing to a personal information use agreement, and gifts were provided in return.

Research Method
The collected data were analyzed using SPSS Statistics ver 26.0, and the significance level (α) for statistical testing was set at 0.05.The analysis method was as follows.First, descriptive statistics and cross-analyses were conducted on the sociodemographic characteristics of the study subjects.Second, t-tests and ANOVA were conducted to determine differences in the recognition of the EMR system certification and the evaluation of interoperability according to the general characteristics of the study subjects, and the Scheffe test was used for post hoc analysis.Third, a correlation analysis was conducted to determine the correlation between the recognition of the EMR system certification and evaluation of the interoperability of the EMR system.Fourth, multiple regression analysis was conducted to verify the factors influencing the recognition of the EMR system certification in evaluating the interoperability of the EMR system.

Differences in Recognition of EMR System Certification according to General Characteristics of Research Participants
Awareness of the EMR system certification was higher in private hospitals, at 3.57 ± 0.756 points, than in public hospitals, at 3.51 ± 0.890 points (t = −2.264,p = 0.024).
The difference in awareness of the EMR system certification according to the general characteristics of research subjects in public hospitals was found to be higher for general hospitals, with 3.57 ± 0.852 points, than for tertiary general hospitals, with 3.42 ± 0.938 points, in terms of medical institution classification (t = −3.852,p < 0.001).In terms of age, those in their 50s or older showed the highest score, at 3.94 ± 0.829 points, those in their 40s at 3.67 ± 0.823 points, those in their 30s at 3.37 ± 0.903 points, and those in their 20s at 3.32 ± 0.867 points (F = 47.932,p < 0.001).In terms of occupation, health information managers scored highest at 3.75 ± 0.681 points, administrative workers at 3.61 ± 0.800 points, medical technicians at 3.51 ± 0.935 points, nurses at 3.49 ± 0.883 points, and doctors at 3.07 ± 1.105 points (F = 12.962, p < 0.001).In terms of number of working years, 20 years or more was the highest with 3.89 ± 0.839 points, followed by 3 years to less than 5 years with 3.49 ± 0.757 points, 10 years to less than 20 years with 3.47 ± 0.979 points, 5 years to less than 10 years with 3.41 ± 0.778 points, and less than 3 years with 3.23 ± 0.965 points (F = 34.350,p < 0.001).
The difference in awareness of the EMR system certification according to the general characteristics of the research subjects in private hospitals was found to be higher for women, at 3.61 ± 0.735 points, than for men, at 3.48 ± 0.801 points (t = −2.789,p = 0.005).
In terms of age, those in their 50s or older showed the highest score at 3.76 ± 0.739 points, those in their 40s at 3.71 ± 0.679 points, those in their 30s at 3.51 ± 0.815 points, and those in their 20s at 3.41 ± 0.736 points (F = 15.999,p < 0.001).In terms of educational level, those with graduate school or higher were found to be highest at 3.73 ± 0.688 points, those who were college graduates or lower were 3.64 ± 0.763 points, and those who were college graduates were 3.52 ± 0.762 points (F = 6.970, p = 0.001).In terms of number of working years, 20 years or more was the highest with 3.82 ± 0.698 points, followed by 10 years to less than 20 years with 3.64 ± 0.736 points, 3 years to less than 5 years with 3.46 ± 0.734 points, less than 3 years with 3.44 ± 0.733 points, and 5 years to less than 10 years with 3.42 ± 0.792 points (F = 17.136, p < 0.001) (Table 3).

Differences in Evaluation of EMR System Interoperability according to General Characteristics of Research Participants
The EMR system interoperability evaluation showed that public hospitals scored 3.58 ± 0.760 points, which was higher than private hospitals' 3.45 ± 0.753 points (t = 4.903, p < 0.001).
The difference in the evaluation of EMR system interoperability according to the general characteristics of research subjects in public hospitals was 3.74 ± 0.830 points for those in their 50s or older; 3.64 ± 0.705 points for those in their 20s; and 3.51 ± 0.734 points for those in their 30s; it was found to be higher, in the order of 3.50 ± 0.798 points, for those in their 40s (F = 10.120,p < 0.001).In terms of occupation, the scores were highest in the following order: administrative 3.65 ± 0.715 points, medical technician 3.62 ± 0.769 points, nursing 3.57 ± 0.756 points, health information manager 3.53 ± 0.826 points, and doctor 3.43 ± 0.746 points (F = 2.447, p = 0.045).In terms of years of service: less than 3 years-3.69± 0.729 points; 3 years to less than 5 years-3.67 ± 0.657 points; more than 20 years-3.56 ± 0.843 points; 10 years to less than 20 years-3.52 ± 0.784 points; for 5 years to 10 years scores below 3.51 ± 0.735 were found to be highest (F = 5.076, p < 0.001).
The differences in the evaluation of EMR system interoperability according to the general characteristics of research subjects in private hospitals were 3.54 ± 0.787 points for those in their 20s; 3.46 ± 0.741 points for those in their 30s; 3.44 ± 0.759 points for those in their 50s and above; and 3.44 ± 0.759 points for those in their 40s.It was found to be highest at 3.35 ± 0.714 points (F = 4.358, p = 0.005) (Table 4).

Correlation between Recognition of EMR System Certification and Evaluation of EMR System Interoperability
A correlation analysis was conducted to examine the correlation between the recognition of the EMR system certification and EMR system interoperability evaluation.A significant positive (+) correlation was found between recognition of the EMR system certi-fication and the evaluation of EMR system interoperability in public (r = 0.439, p < 0.001) and private hospitals (r = 0.402, p < 0.001) (Table 5).
Table 5. Correlation between recognition of EMR system certification and evaluation of EMR system interoperability.unit: r(p).

Impact of Recognition of EMR System Certification on Evaluation of EMR System Interoperability
To determine the factors influencing the perception of the EMR system certification of public and private hospitals on the evaluation of the interoperability of the EMR system, EMRs were provided according to the general characteristics of the research subjects, including their type.Age, occupation, years of service, and awareness of the EMR system certification, which showed significant differences in EMR system interoperability, were used as independent variables to identify factors affecting the evaluation of EMR system interoperability.
In the case of public hospitals, the tolerance of all independent variables was 0.192~0.918and the VIF was 1.089~5.201,ruling out the possibility of multicollinearity.The Durbin-Watson result was 1.313, indicating no autocorrelation.Age was found to have a significant positive (+) effect on those in their 50s or older (β = 0.086, p < 0.001) and awareness of the EMR authentication system (β = 0.475, p < 0.001).The type of medical institution was hospital (β = −0.075,p < 0.001), the length of service was 3 to less than 5 years (β = −0.055,p = 0.026), and 5 to less than 10 years (β = −0.118,p < 0.001), significant negative effect in 10 to less than 20 years (β = −0.120,p < 0.001), and over 20 years (β = −0.253,p < 0.001).This effect was found to be significant.The explanatory power of the model was found to be 23.0%(F = 50.403,p < 0.001).

Discussion
To secure interoperability and improve the quality of medical information, Korea introduced, and is operating, an EMR system certification in June 2020.This study confirmed the awareness of the EMR system certification among public and private hospital employees who have obtained EMR system certification, and confirmed the impact of the perception of the certification system on the evaluation of EMR system interoperability.An attempt was made to contribute to the stable establishment and development of the EMR system certification in the future.
This study found a difference in the perception of the EMR system certification and interoperability between public and private hospital employees; however, in both hospitals, the perception of the EMR system certification had a confirmed positive (+) effect on the interoperability evaluation.
An EMR system certification is necessary because in the era of big data and AI, the demand for structured and standardized medical information utilization and medical information exchange is increasing.The purpose of this study is to verify the national standards and suitability of the EMR system through a certification system and to encourage the development of standard products to ensure interoperability between systems [24].By securing interoperability, we can expect to revitalize medical information exchange.The impact of medical information exchange on medical quality and patient safety is significant and directly related to the certification of the EMR system.For example, the exchange of medical information in the emergency medical system is directly related to the efficient treatment of emergency patients, and the accuracy and rapid delivery of medical information can be guaranteed through certification of the system [10].However, when medical information exchange is disabled, the risks of medical errors and patient-safety accidents increase [25].Therefore, among the EMR system certification and standards identified in this study, interoperability can be considered an important means of solving these problems.In the US, UK, Taiwan, Australia, and Canada, financial support is provided to hospitals that operate EMR system certification [5,26].However, in Korea, there is no realistic support other than the inclusion of EMR system certification as a pilot indicator in the Ministry of Health and Welfare's medical quality evaluation.Therefore, for the EMR system certification to become active and stable, it should be accompanied by a national-level support plan that includes financial support.
Few previous studies are similar to this one; therefore, a direct comparison with other studies is difficult.However, the implications of this study's results are as follows.First, the results confirmed that the higher the hospital employees' awareness of the EMR system certification, the more influential the evaluation of the EMR system interoperability, which strengthens the positive perception and effectiveness of EMR system certification for hospital employees.It is necessary to develop a method to achieve this.Second, it is necessary to devise a plan to continuously evaluate the effectiveness of the EMR system certification.Through this, problems with the certification system can be identified early, and certification standards can be improved.These implications are expected to contribute to the stable establishment of an EMR system certification in Korea and to accelerate the exchange of medical information between hospitals by improving interoperability.
In Korea, an EMR system certification has been implemented since June 2020.This study is significant in that it is the first to confirm the awareness and interoperability of the EMR system certification among hospital employees who obtained certification over a period of three years after the introduction of the EMR system certification.In particular, it is thought to be more meaningful as it comprehensively includes various occupations, such as doctors, nurses, health information managers, medical technicians, and administrative staff, that are directly related to the use of EMR systems at work.
However, this study had some limitations.First, although the survey questions were developed by collecting opinions from experts in related fields based on certification system standards, the inherent limitations of the survey cannot be ruled out.Second, because the data were collected using self-administered survey questions, the possibility that respondents' subjective opinions influenced the results cannot be ruled out.Third, there are limitations in generalizing the research results because the participating hospitals did not target all hospitals that have obtained certification.Therefore, in follow-up research, it will be necessary to address these limitations, develop more objective research tools, and apply a research methodology that can investigate the effects of the EMR system certification.

Conclusions
This study confirmed the awareness of the EMR system certification among public and private hospital employees who obtained EMR system certification and the impact of the perception of the certification system on the evaluation of EMR system interoperability.An attempt was made to contribute to the stable establishment and development of the EMR system certification in the future.In this study, although there was a difference in the perception of the EMR system certification and interoperability between public and private hospital employees, it was confirmed that the perception of the EMR system certification in both hospitals had a positive effect on the interoperability evaluation.Based on the results of this study, to establish a stable EMR system certification in Korea, it is necessary to prepare measures to strengthen the positive perceptions and effectiveness of EMR system certification among hospital employees.Through this, we will be able to ultimately improve interoperability and promote the provision of quality medical services and promotion of public health through the exchange of medical information between hospitals.Furthermore, we hope to contribute to the international standardization of health and medical information.

Table 1 .
Data collection methods.

Table 2 .
Cont.Presidential Decree in accordance with Article 2, Paragraph 3 of the Public Health and Medical Services Act, with the primary purpose of providing public health and medical care.Among the hospitals that participated in the survey based on the 2021 Status of Public Medical Institutions published by the Ministry of Health and Welfare and the National Medical Center [23], the Regional Medical Center, Korea Workers' Compensation & Welfare Service Hospital, and National University Hospital were classified as public hospitals, while the other hospitals were classified as private hospitals.
2.2.Key Terms and Research Instruments 2.2.1.Public and Private Hospitals A public hospital is a healthcare institution established and operated by the state, a local government, or a joint organization prescribed by a

Table 3 .
Differences in recognition of EMR system certification according to general characteristics of research participants.

Table 4 .
Differences in evaluation of EMR system interoperability according to general characteristics of research participants.

Table 6 .
Impact of recognition of EMR system certification on evaluation of EMR system interoperability.