Evaluation of Patient No-Shows in a Tertiary Hospital: Focusing on Modes of Appointment-Making and Type of Appointment

No-show appointments waste resources and decrease the sustainability of care. This study is an attempt to evaluate patient no-shows based on modes of appointment-making and types of appointments. We collected hospital information system data and appointment data including characteristics of patients, service providers, and clinical visits over a three-month period (1 September 2018 to 30 November 2018), at a large tertiary hospital in Seoul, Korea. We used multivariate logistic regression analyses to identify the factors associated with no-shows (Model 1). We further assessed no-shows by including the interaction term (“modes of appointment-making” X “type of appointment”) (Model 2). Among 1,252,127 appointments, the no-show rate was 6.12%. Among the modes of appointment-making, follow-up and online/telephone appointment were associated with higher odds of no-show compared to walk-in. Appointments for treatment and surgery had higher odds ratios of no-show compared to consultations. Tests for the interaction between the modes of appointment-making and type of appointment showed that follow-up for examination and online/telephone appointments for treatment and surgery had much higher odds ratios of no-shows. Other significant factors of no-shows include age, type of insurance, time of visit, lead time (time between scheduling and the appointment), type of visits, doctor’s position, and major diagnosis. Our results suggest that future approaches for predicting and addressing no-show should also consider and analyze the impact of modes of appointment-making and type of appointment on the model of prediction.


Introduction
No-shows occur when a patient fails to attend a scheduled appointment with no prior notification to the hospital [1]. Terms identified as corresponding to "no-show" in existing literature included appointment breaking, nonattendance, dropping out, missed appointment, and appointment failures [2]. No-show appointments not only decrease the sustainability of care for individuals but also pose several challenges for health care systems, including wasted resources, longer wait times, and concomitant threats to future patient satisfaction [3,4]. It is a well-known fact that no-show decreases the service provider's productivity and efficiency, increases healthcare costs, and limits the medical institution's effective capacity [5]. Hence, from the perspectives of sustainable medical services and operational efficiencies, managing patients' no-show behavior is essential [6].

Subjects and Data Source
This study used data of patients who made an outpatient appointment at a tertiary hospital located in Seoul, Korea. The hospital is one of the largest general hospitals in Korea, operating 63 departments, and thus allowed a large number of and diverse patients to be included in the study sample. Data were collected from medical records and administration records of patients who had reserved outpatient care appointments from 1 September 2018 to 30 November 2018. In order to protect personal information, the patients' personal identification number was used instead of the hospital registration number, which was also anonymized in the initial extraction stage. During this period, the total number of outpatients was 256,011, and the total number of appointments given was 1,252,127. This study was reviewed by the Yonsei University Health System Institutional Review Board and was ruled exempt (IRB number: Y-2019-0097). Written informed consent from patients was waived as this was a secondary data analysis using de identified data.

Dependent Variable
We examined the outpatient nursing department records for each patient to check the no-show status. All patients' appointments were categorized as either "no-show" or "show-up." No-show was defined as a patient who did not attend outpatient care on the day of the appointment [10].

Independent Variables Demographic Characteristics
The demographic characteristics included were gender, age, region (based on the patient's residence), and types of insurance. Patients were divided into seven groups based on their age (≤19, 20-29 years, 30-39 years, 40-49 years, 50-59 years, 60-69, and ≥70 years). We divided the regions of patient residence into three groups: Seoul, Incheon Gyeonggi, and other areas. This categorization was employed as the study hospital is located in Seoul, the largest city of Korea; the second group, Incheon and Gyeonggi, represents the two of the most populated regions in Korea following Seoul; lastly, the remaining 14 cities and do-provinces of Korea are categorized as other areas. We divided types of insurance into four groups: National Health Insurance, Medical Aid, Industrial Accident Compensation Insurance and Automobile Insurance, and International Insurance and Private Insurance.

Appointment Related Characteristics
Among the appointment related characteristics, we divided the modes of appointmentmaking into three groups: Follow-up, online or telephone, and walk-in. Follow-up appointments meant appointment for the next schedule after previous medical treatment in the hospital. Walk-in appointments indicate that a patient visited the hospital in person to make an appointment. We divided time of visit into six groups: before" 9:00 a.m., 9:00 a.m.-11:00 a.m., 11:00 a.m.-13:00 p.m., 13:00 p.m.-15:00 p.m., 15:00 p.m.-17:00 p.m., and after 17:00 p.m. Days of a week were from Monday to Sunday, and Sundays included an examination reservation in addition to a treatment reservation. Lead time was the time between scheduling and the appointment and, in this study, was divided into less than 8 days, 8-14 days, 15-21 days, 22-28 days, 29-56 days, 57-84 days, and more than 85 days. Seven days represent one week; however, to avoid overlap among the periods, the weeks were indicated as days.

Practice-Related Characteristics
The practice-related characteristics included the types of visit, type of appointment, department, doctor's positions, and patient's major diagnosis. We divided type of visit into three groups: A (new patients at the study hospital), B (patients who had visited the clinical department before), and C (new patients at the clinical department, but those who had visited the study hospital before). There were three major appointment types: Consultation, examination, and treatment and surgery. We divided clinical departments into 10 groups: Internal medicine, surgery department, obstetrics and gynecology (OBGYN), pediatrics, ophthalmology, otolaryngology, dermatology, urology, neuropsychiatry, and others. The positions of the doctors were categorized into professional, fellow, and training positions. The categorization followed the progression of years and training required for a doctor to be specialized in a particular field in Korea as his or her seniority increases within the hospital. For instance, a doctor who is in a fellow position refers to a person who trains for 1-2 years at a department of his or her major after obtaining the license to become a resident doctor. Patients' major diagnoses were classified according to the 22 major diagnoses based on the Korean standard classification of diseases-7 codes.

Statistical Analysis
A three-step analysis was performed. First, for all categorical variables, we used chi-square tests to calculate the distribution of patient characteristics according to no-show status. This test is commonly used to test association between two or more categorical variables. Second, multivariate logistic regression analysis was used to assess the factors associated with no-show (Model 1). Finally, the interaction term ("modes of appointmentmaking" x "types of appointment") was included in Model 2. Multiple logistic regression was used for two main reasons. First, the no-show status, the dependent variable of this study, was a binary outcome (Show-up: 0; No-show: 1). Second, logistic regression is appropriate for handling relationships among outcome variables and independent variables. To control for multiple appointments by the same patient, we incorporated repeated measures by using the "repeated subject" option in the generalized estimating equation. Statistical significance was established at p < 0.05. SAS software (ver. 9.4; SAS Institute, Cary, NC, USA) was used for all calculations and analyses. Table 1 shows the comparison of characteristics between no-show and show-up. During the study period, the number of outpatient appointments was 1,252,127, and the no-show rate was 6.12%. Regarding demographic characteristics, the group of subjects who failed to show-up for their appointments comprised mostly men (6.4%), those ≤19 years (7.2%), patients residing in Seoul (6.5%), and recipients of Medical Aid (8.7%).

Comparison of Characteristics between No-Shows and Show-Ups
For appointment-related characteristics, regarding the modes of appointment-making, the no-show rate was higher for walk-in appointments (10.6%). In terms of the time of the visit, the no-show rate was higher for 9:00-11:00 a.m. category (8.7%) than at other times. Regarding the day of week, the no-show rate was higher on weekends (9.6%) than on the other days. Regarding, on lead time, the following groups had the highest no-show rates: Less than 8 days (8.1%), 8-14 days (6.9%), and 57-84 days (6.7%).
For practice-related characteristics, regarding types of visits, the no-show rate was the highest for re-visits (6.5%). Based on types of appointment, the following groups had the highest no-show rates: Examination (30.1%), treatment and surgery (16.6%), and consultation (4.5%). The department category others had the highest no-show rate of 8.1%, followed by dermatology (7.3%) and otolaryngology (6.7%). With regard to doctor's position, the no-show rate was the highest for the fellow position (6.5%). Regarding patient's diagnosis, diseases of the nervous system (G00-G99) had the highest no-show rate at 11.3%, followed by injury, poisoning, and certain other consequences of external causes (S00-T98) (10.2%), and diseases of the genitourinary system (N00-N99) (10.1%). Table 2 shows the results of the multivariate logistic regression (Model 1) identifying the factors associated with no-shows. Regarding the modes of appointment-making, follow-ups had a lower odds ratio of no-shows (OR = 0.86) than walk-in. The odds ratio for online/telephone-based appointments was not statistically significant. Patients who underwent planned examination, treatment, and surgery had a much higher odds ratio of no-shows than those with planned consultation (examination OR = 9.09; treatment and surgery OR = 4.51).

Factors Associated with No-Shows
In the demographic characteristics of Model 1, for gender, the odds ratio was higher in males (OR = 1.05). Based on age, the odds ratio for other age groups was lower than for the group aged ≤19 years (30-39 OR = 0.91; 40-49 OR = 0.81; 50-59 OR = 0.80; 60-69 OR = 0.78; ≥70 OR = 0.92). The odds ratio for 20-29 groups was not statistically significant. For region, the odds ratio for patients residing in the Incheon·Gyeonggi area (OR = 1.07) was lower than that for patients residing in Seoul. Based on the type of insurance, the odds ratio for Medical Aid (OR = 1.29) and International Insurance and Private Insurance (OR = 1.45) were higher than those for National Health Insurance. In terms of appointment related characteristics of Model 1, based on the time of visit, the odds ratio for other times was higher than that for 9:00-11:00 a.m. (OR = 0.88) were lower than that for less than 8 days. However, the odds ratios for 29-56 days (OR = 1.04) and 57-84 days (OR = 1.05) were higher. From practice-related characteristics, based on the type of visits, the odds ratio for C (new patients at the clinical department, but those who had visited the study hospital before) (OR = 0.30) was lower than that for A (new patients at the study hospital). Based on the department, the odds ratios for surgery, ophthalmology, otolaryngology, dermatology, urology, and neuropsychiatry (OR = 1.15 to 2.15) were higher than that for internal medicine, however the odds ratios for OBGYN and others (OBGYN OR = 0.94; others OR = 0.71) were lower. For the doctor's position, the odds ratios for training position (OR=1.24) and fellow position (OR = 1.16) were higher than for professional position. From the diagnosis point of view, compared to neoplasms (C00-D48), diseases of the nervous system (G00-G99) (OR = 1.15), diseases of the ear and mastoid process (H00-H59) (OR = 1.12), diseases of the respiratory system (J00-J99) (OR = 1.11), diseases of the digestive system (K00-K93) (OR = 1.12), diseases of the genitourinary system (N00-N99) (OR = 1.07), pregnancy, childbirth, the puerperium, and certain conditions originating in the perinatal period (O00-P96) (OR = 1.11), and symptoms, signs, and abnormal clinical and laboratory findings, NEC (R00-R99) (OR = 1.23) had higher odds ratios of no-shows. While several diseases had lower odds ratios than neoplasms (C00-D48), no diagnosis (OR = 0.44) was the lowest.

The Interaction Term
The interaction term is shown in Table 2. The interaction term between the modes of appointment-making and types of appointment was positive and statistically significant. As shown in Figure 1, for a follow-up patient, examination had a higher odds ratio of no-shows (OR = 12.59); however, treatment and surgery had a lower odds ratio (OR = 0.39). Patients with online telephone, treatment, and surgery had the highest odds ratio of no-shows (OR = 11.89).

Discussion
Identifying factors based on outpatient no-shows represents an important potential win-win for patients and hospitals by improving the sustainability of care and reducing wastage of resources. Our study demonstrates that demographic, appointment-related, and practice-related characteristics, which are factors readily available in the HIS data, can be successfully used to evaluate no-shows. Our study examined various factors associated with no-shows, focusing on the modes of appointment-making and types of appointment. The no-show rate in the study, 6.12%, is within the reported range of 5-25% according to the literature [10][11][12]. However, these studies had different timing of examination, examination period, and analysis perspective. Thus, it is difficult to directly compare the results of these studies with those of this study.
Several factors found to be influential for no-shows based on the literature review were also confirmed to be important factors in this study. The summary of the comparison between the factors associated with no-shows in this study and those in existing studies is as follows. First, our study showed that the odds ratio of no-shows for follow-up appointments in Model 2 considering interaction term. This finding might be related to how their needs are not addressed properly. Follow-up appointments are often scheduled after a patient has received a treatment, but it is scheduled to fit the department's availability rather than the patients. However, in cases where patients themselves with medical conditions made online, telephone, or in-person appointments, the no-show ratio was lower because the appointment compliance was carried out actively by the patients according to their situation. Hospitals may need to prioritize appointment and medical service management with utmost consideration because high no-show ratio of follow-up patients has a direct and detrimental effect on their health [30][31][32].
Further, our study identified various factors including gender, age, region, types of insurance, time of visit, day of week, lead time, types of visit, types of appointment, department, doctor position, and major diagnosis as potential factors associated with no-shows.
In examining types of insurance, our study showed that the odds ratios for no-show for Medical Aid and International Insurance and Private Insurance were higher than that for National Health Insurance. This is similar to previous studies that showed that low-income patients covered by public assistance programs had higher odds of no-show [12,33,34], and patients with International Insurance and Private Insurance had higher odds of noshows [12]. Medical institutions of Korea are categorized into three tiers: (1) Tertiary hospitals that provide specialized medical services, (2) primary medical institutions for basic medical services, and (3) clinics that have general practitioners for providing outpatient services [33]. A medical referral form issued by a sub-tier institution is required for patients to visit a tertiary hospital [35]. It is not difficult to obtain a referral at most medical institutions [35]. Patients with National Health Insurance visiting tertiary hospitals must pay 60% out-of-pocket expenses, while those with Medical Aid I pay a fixed amount of KRW 2000 and Medical Aid II pay 15% out-of-pocket expenses [36]. Thus, because Medical Aid patients can easily access high-tier medical institutions by paying relatively lower out-of-pocket expenses, they prefer tertiary hospitals that provide specialized medical services [35]. This phenomenon causes problems as the no-show rate increases. Moreover, owing to having no penalties-either financial or concerning future reservations-for no-shows, most patients are not watchful of no-shows. A no-show problem by Medical Aid patients has been brought up consistently as a major operative challenge for hospitals, suggesting the necessity of Medical Aid patient management.
Odds ratios for no-show were higher during Friday and weekends (Saturdays and Sundays), a finding similar to previous results that no-shows were most frequent on Fridays and Saturdays [37][38][39]. This result can be explained with the existing study that found more likelihood of no-shows in Friday reservations due to various events. On the contrary, some studies identified no-show were more likely to occur for Monday reservations than for weekend reservations. According to a study by Kwon et al. (2015) [12], which was conducted in the same study hospital, the days and likelihood of no-shows differed according to the types of appointment. In the case of consultation appointment, noshow was most frequent on Mondays, while Saturdays experienced the highest number of no-shows for a test, treatment, and surgery appointment. In the case of the study hospital, during weekends, especially on Sundays, most reservations are for diagnostic examinations, such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT), and Positron Emission Tomography (PET), other than a consultation. Accordingly, contradictory results between this study and the existing study can be explained by differences in the type of appointment according to the days of the week. Moreover, this may occur partially because such time slots (i.e., Sunday) are generally less used. Thus, further studies should be conducted considering different days of the week. Increase in lead time, which is the time between the booking of the appointment to the actual day of the appointment, seems to raise the odds ratio of no-shows. The further the date of appointment, the more likely the patient is to not show up. It is possible that with increased lead time, patients are more likely to forget their appointment, have a scheduling conflict, or visit another hospital or physician's office that gives them a closer appointment date [10,33]. According to Athenahealth, "an analysis of 4.2 million appointments scheduled in 2016 by 13,000 providers found that shorter appointment lead times can be critical to getting new patients in the door" [40]. Specifically, Athenahealth found that "on average, a new patient who waits more than a month for a first appointment is more than twice as likely to cancel and not reschedule as a new patient who is scheduled within a week." [40].
Lower odds ratios for established patients compared to new patients also have important implications. Established patients may have a reason to be attached to a specific hospital. They may have experienced the warmth of the staff, care of the nurses, and insightfulness of the physicians. However, to a new patient, a hospital clinic may still be just a name on a page. This is not unique to healthcare; it is a concept well studied in behavioral economics called the endowment effect [41].
In terms of departments, odds ratios of no-shows were higher for surgery department, pediatrics, ophthalmology, otolaryngology, dermatology, urology, and neuropsychiatry. This implies that tertiary hospitals, in managing ambulatory care, should invest greater efforts to improve patient compliance.
Furthermore, this study also proved that other major diagnoses had a higher noshow rate compared to neoplasms (C00-D48) along with their higher no-show ratio of examination, and treatment, and surgery appointments. This result could be related to patients' needs. Tertiary care requires patients to have a medical referral form issued by primary or secondary care facilities, and these documents are issued quite easily by most medical institutions. Therefore, easy access to top medical institutions has resulted in more options for patients, which then leads to them favoring tertiary hospitals that provide specialized medical services [34,35]. Inevitably, this creates a problem of higher no-show rates for subordinate medical institutions as patients flock to tertiary medical institutions [33]. Considering the long waiting period in tertiary hospitals, patients with mild medical conditions who do not require immediate care tend to make appointments at several hospitals simultaneously [33]. This finding is important as any intervention aimed at reducing no-shows should include the effects of the patient's diagnosis.
Our variables of key interest were modes of appointment-making, types of appointments, and their interactions. The finding that both follow-up and online/telephone appointments had higher odds ratios compared to walk-in appointments may suggest that relatively easier modes of appointment-making increase the likelihood of no-shows. Among the types of appointments, treatment and surgery had a much higher odds ratio of no-show compared to consultation. Missed treatment or surgery disrupts schedules, and potentially leaves doctors and nurses with gaps during the workday. When examining the interaction between the modes of appointment-making and types of appointment, followup and examination appointment had a higher odds ratio of no-shows while treatment and surgery had a lower odds ratio.
Not every examination will lead to an operation or admission, but doctors could lose out on service time for other patients when patients miss potentially critical examination appointments. Hospitals faced with a patient who does not show up for an examination may lose revenue, but the greater financial impact could be the possible necessary service that never occurs.
Another interaction term indicates that for patients who made appointments for treatment or surgery online or by phone had much higher odds of no-show. One might assume that providing direct or easy scheduling will result in better patient access, and thus, a reduction in no-shows. However, we could not find results that indicate that direct or easy scheduling improves patient access. Rather, the results suggest that direct or telephone scheduling decreases patients' frequency of access to care.
Our study suggests that it is important for hospital managers to manage no-show with a focus on two aspects. First, we recommend that hospitals design interventions to reduce no-shows of follow-up patients who visit for examination. In addition, it is helpful for hospitals to decrease no-shows of online or telephone appointment patients who visit for treatment and surgery. Second, hospital managers must treat the issue seriously and design effective interventions for managing no-shows. In addition to satisfying the basic requirements of fundamental medical services, hospitals can provide other supportive services, for example, more comfortable waiting rooms and personalized accompanying services for patients.
This study has several limitations. First, this study was based on data on outpatient reservations at a single tertiary hospital for three months. Thus, the research results cannot be generalized to all tertiary hospitals. Second, restrictions on data extraction may have led to exclusion of certain variables that may be a factor in no-shows. In addition, the analysis did not include medical expenses, history of no-shows, and reason for no-shows, which were found to be associated with no-shows in the previous study. Therefore, to overcome the limitations of existing studies and this study, future studies need to include multiple medical institutions, as well as variables such as past no-show history and reservation change history. Despite these limitations, our study is unique in that it considered the modes of appointment-making and types of appointments simultaneously to examine noshows and established the need to address patients' medical needs. In addition, the target hospital responsible for this study is one of the largest tertiary hospitals in South Korea, which made it possible to analyze multiple and large groups of patients' medical cases.

Conclusions
No-shows contribute to reduced scheduling efficiency and lowered effectiveness of the medical services delivered [42]. Sustainability of care is essential to ensure maximum health benefits for patients [43]. There is a need to develop interventions that will improve clinic attendance among clients [43]. Our results suggest that future approaches for predicting and addressing no-shows should also consider and analyze the impact of the modes of appointment-making and types of appointment on the model of prediction. Still, because no-shows constitute a complex issue and are affected by cultural differences and varying hospital systems, it is desirable to consider a conservative approach when applying the intervention, considering the limited generalizability of the present findings.  Informed Consent Statement: Patient consent was waived as this was a secondary data analysis using de identified data.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest.