Optimizing Daily Service Scheduling for Medical Diagnostic Equipment Considering Patient Satisfaction and Hospital Revenue

Abstract

Under the background of the unbalanced supply and demand of medical diagnostic equipment and rising health care costs, this study aims to optimize the service scheduling for medical diagnostic equipment so as to improve patient satisfaction by ensuring the equipment utilization rate and hospital revenue. The finite horizon Markov Decision Process (MDP) was adopted to solve this problem. On the basis of field research, we divided patients into four categories: emergency patients, inpatients, appointed outpatients, and the randomly arrived outpatients according to the severity of illness and appointment situations. In the construction of the MDP model, we considered the possibility of cancellation (no-show patients) in scheduling optimization. Combined with the benefits and costs related to patient satisfaction, based on the value iteration algorithm, we took patient satisfaction and hospital revenue as the objective functions. Results indicated that, compared with the current scheduling strategy, the integrated strategy proposed in this study has a better performance, which could maintain the sustainable usage rate of large medical resources and patient satisfaction.

1. Introduction

In recent years, large-scale medical equipment has become an increasingly important tool for hospitals to provide high quality medical services. Large-scale medical examination facilities such as positron emission computed tomography (PET-CT), digital subtraction angiography (DSA), nuclear medicine single photon emission computed tomography (SPECT), and magnetic resonance imaging (MRI) are important medical resources. Especially, the diagnosis and the treatment of the most common illnesses are based on large medical equipment. However, the purchase and maintenance costs of medical equipment are quite high. For example, a MRI facility costs more than 10 million Chinese Yuan (CNY), and the annual maintenance costs of MRI are up to more than 20 million CNY. Combined with the purchase quota of large medical equipment, it is difficult to meet the patients’ demand in densely populated Shanghai hospitals. Under the circumstances, how to more effectively improve the utilization rate of medical equipment with the limited number of devices has become an urgent problem for hospitals in mega cities in China.

Due to the large population base, increasing life pressures and unhealthy lifestyles, there is a growing demand for health care as well as for medical facilities. The high maintenance costs of medical image diagnostic equipment result in the fact that small and medium-sized medical institutions seldom equipped themselves with such medical diagnostic equipment, but the inspection results from the equipment are highly important in the diagnosis process. Therefore, based on the reasons above, a large number of patients need the medical image diagnosis equipment, implying that the effective scheduling on how to utilize the limited resources to check patients is an urgent problem to be solved. Physiologically, emergency patients have urgent demand for treatment; inpatients, appointed outpatients and randomly arrived outpatients are less urgent for diagnosis and treatment. In the study of doctor–patient contradiction, we find that medical time is one of the important factors that has caused the doctor–patient contradictions. This kind of contradiction refers to patients’ feelings of resentment or aggressive behaviors which results from the length of medical time for making an appointment and treatment. In addition, hospitals need to meet their revenue targets on the basis of comprehensive consideration of patients’ physiological and psychological conditions.

Furthermore, in many radiology departments, the emergency patients, inpatients and outpatients can be examined by the same MRI facilities; however, there exist large differences among the priorities of the patients’ diseases and the cost–income values. With the rapid development of modern medical imaging technology and the growing medical demand, hospitals will face more challenges and dilemmas in terms of time scheduling.

Hospital managers under cost pressures are usually dedicated to improve the resource utilization rates of these large-scale medical facilities, which leads to a long waiting time for the patients to use these resources for the diagnosis and treatment. There is still need for further optimization on the appointment model construction.

“First come, first served (FCFS)” is one of the most commonly adopted strategies in public hospitals in China, that is, when outpatients come to make an appointment for examination, hospital staff usually make an earliest appointment for patients on the waiting list. However, because the patients’ conditions are different, and the tolerance of waiting time is limited, when the hospital staff arrange an appointment for the patients, they need to consider the likelihood that a more urgent patient might come, and the problem of leaving equipment unused as a result of delaying the arrangement of patients on the waiting list while a more urgent patient does not show up as expected; meanwhile, the waiting time for inspection may extend when the number of urgent patients is more than expected, in which case even hospitals cannot optimize the problem.

Therefore, in this study, a systematic approach is adopted to weigh the key factors of scheduling by taking MRI as the research object so as to scientifically schedule and effectively arrange the patients by considering the tradeoff between patient satisfaction and hospital income.

The contributions of this study are summarized as follows:

First, we consider the satisfaction of patients. Specifically, on the basis of the original linear satisfaction, we obtain patient’s satisfaction according to the actual situations. In addition, we fully consider the different satisfactions of the four types of patients. The economic and social benefits of hospitals as well as the doctor–patient relationship can be improved through the improvement of patient satisfaction.

Second, we fully consider the actual situation of patients who require large equipment inspection making appointments in advance according to the actual investigation. Considering that some patients might not be able to make appointments, the hospitals can ensure that randomly arrived outpatients are examined by proper arrangements. Therefore, the proposed model will not only improve patient satisfaction, but also increase the equipment utilization rate, which can make the optimization strategy more realistic.

Third, most of the existing research focuses on equipment reservation scheduling problems on the day before the service day. This study investigates the equipment inspection scheduling problem on the service day. Considering that the arrivals of various patients on the service day are highly random, the difficulty and dimensions of this research can be enhanced.

2. Literature Review

In recent years, there have been increasing concerns about patient scheduling, and most of the research has been concentrated on outpatient appointment scheduling problems [1,2,3,4]. Kaandorp and Koole [5] studied the problem of optimal outpatient appointment scheduling, and took average waiting time of patients and idle and tardiness as the study objectives. Muthuraman and Lawley [6] constructed a stochastic overbooking model, developed the relevant appointment scheduling policy, and conducted the performance analysis of the proposed scheduling policy. In order to achieve a higher level of medical service, Vermeulen et al. [7] proposed an adaptive scheduling for CT equipment. Patric et al. [8] adopted the MDP to dynamically schedule the multi-priority patients for the diagnostic facilities. Green et al. [9] constructed a finite horizon MDP model for diagnostic service appointment schedule. On the basis of this model, Kolisch and Sickinger [10] adopted the MDP model and considered two CT-scanners and three patient groups for this problem. Yan et al. [11] proposed an appointment scheduling model that considers patient satisfaction using patient choice and service fairness indices. Zhu et al. [12] built an MDP model to maximize the expected revenue for scheduling the outpatients taking two types of outpatients into consideration. Ho and Lau [13] considered various rules to minimize the total cost for outpatient appointments scheduling. Green and Savin [14] adopted a single-server queuing model to reduce delay for patient appointments. LaGanga et al. [15] proposed an appointment scheduling model that considered patient no-shows, and developed an effective solution procedure. Robinson and Chen [16] compared traditional and open-access appointment scheduling policies. Sch¨utz and Kolisch [17] proposed a model for capacity allocation in the service industry. Gupta and Wang [18] provided an MDP model for the appointment scheduling problem. Ma et al. [19] proposed capacity planning decisions and appointment scheduling methodology for new patient demand. Gocgun and Puterman [20] developed an MDP model for chemotherapy patient appointment scheduling that considers due dates and time windows, and compared the performance with heuristic decision rules under different scenarios. Patrick and Puterman [21] developed a methodology to improve the utilization of diagnostic industry and reduce the wait times of patients. Taking computed tomography (CT) as an example, Gocgun et al. [22] developed a finite-horizon MDP model for multi-category patient scheduling. Du et al. [23] proposed a clinical pathways scheduling model and employed the hybrid genetic algorithm to solve the problem. Parizi and Ghate [24] developed an MDP model for multi-class and multi-resource patient scheduling that considers factors such as no-shows, cancellations and overbooking. Astaraky and Patrick [25] proposed the combined simulation and approximate dynamic programming method for surgical scheduling with multi-classes and multi-resources. Barz and Rajaram [26] discussed the elective patient admission problem with different resource constraints. Geng et al. [26,27,28,29,30] proposed relevant model and optimal control-policies for managing MRI examination for stroke patients. Luo et al. [31] developed an MDP model for medical examination appointment problem. In addition, a simulation methodology was employed for real-time medical equipment scheduling optimization problem [32].

In the literature, according to the cost structure and arrival time, most scholars believe that patients can be divided into some categories: outpatients, inpatients and emergency patients, they pay attention to the scheduling of different types of patients, and make their waiting time tied to the hospital income [33,34,35]. In addition, some scholars linked patient satisfaction with economic benefits of hospitals, and considered the patients’ time preferences and doctors’ preferences. Another research direction is taking the sensitivity for waiting time of patients into consideration.

Meanwhile, the existing literature has theoretical reference values as well as certain limitations, such as paying more attention to hospitals’ economic benefits while less consideration to social benefits (patient satisfaction level); because the integrated queuing coordination planning model is unsure, it is difficult to balance the equilibrium point of over-hypothesis. The difficulty of modeling leads to a rigorous hypothesis and an over-simplified model. In addition, it lacks dynamic analysis on the input of the queuing system, and there is an optimization of the waiting process of examination from the perspective of medical image diagnostic devices. The research fields of many studies have deep theoretical explorations, but they do not pay much attention to the practical applications of these theories.

Compared with other existing studies, the most important research significance of this study is to consider a variety of complex factors comprehensively and introduce patient satisfaction as the new measurement standard of hospital decision-making. Using the Markov decision process as a practical scheduling tool, we take both patient satisfaction and hospital income into consideration for equilibrium principle. Based on the hospital filed research, we verify the new decision under the principle of patients queueing up for receiving MRI equipment tests. The result shows that this new decision principle can guarantee the hospital income, as well as improve patient satisfaction.

This study establishes a finite horizon MDP model to solve the scheduling problem of patients under four categories (emergency patients, inpatients, outpatients, and the randomly arrived outpatients) for MRI examinations. The goal is to realize the maximized benefits of hospitals and take the patient satisfaction into account. In addition, this study is expected to combine these theories with the actual conditions of top-three hospitals in Shanghai, China and provide decision-making programs for the hospital managers to optimize the medical diagnostic devices scheduling.

3. Data Collection and Presentation

3.1. The Questionnaire Design

In order to understand the basic appointment and scheduling management situations of medical equipment, the design of questionnaires is based on the field research of Shanghai Sixth People’s Hospital, and the questionnaires are divided into doctors’ edition and patients’ versions. The respondents of questionnaires (doctors’ edition) are doctors, nurses and hospital staff in the first-class (Sanjia) hospitals in Shanghai; and the respondents of questionnaires (patients’ edition) are mainly the patients of different ages who went the first-class (Sanjia) hospitals. In addition, we first interviewed the medical device operators, doctors and managers in hospitals at Shanghai, and then made appropriate modifications on the final versions of questionnaires.

Questionnaires in this study can be used to analyze patients of different ages, at different income levels, or with different living conditions and with educational levels. It can also analyze the current situations of medical equipment reservation mechanisms at all stages, and explore the factors affecting queuing time when using medical equipment on the service day and the improvements which are accepted by patients. On the other hand, questionnaires of doctors’ edition are mainly concerned with the utilization of medical equipment, problems existing in the current medical equipment reservation mechanisms, and the improvements which are accepted by doctors.

The data processing and expected objective descriptions of the questionnaires about medical equipment appointments (brief/overview) are as follows:

• Questionnaires of Patients’ Edition

  ✓

  The satisfaction degrees of different groups on the medical equipment reservations and scheduling mechanisms (The main analysis factors: outpatients or inpatients, age, hospitals’ grade, income levels, educational levels, etc.).

  ✓

  The understanding of the mechanisms of medical equipment reservations and scheduling by different groups (The main analysis factors: age, educational levels, etc.).

  ✓

  Waiting time for appointments of different medical devices (The main analysis factors: outpatients or inpatients, etc.).

  ✓

  The endurance limits of different groups on the appointment time of medical equipment (The main analysis factors: age, income levels, educational levels, etc.).

  ✓

  The influence of different groups on the waiting time of medical equipment (The main analysis factors: outpatients or inpatients, age, income levels, educational levels, etc.).

• Questionnaires of Doctors’ Edition

  ✓

  The satisfaction degrees of doctors and nurses on the medical reservations and scheduling mechanisms.

  ✓

  The utilization of different medical devices.

  ✓

  The problems of medical equipment reservations and scheduling mechanisms.

  ✓

  The improvement measures for doctors and nurses.

3.2. Data Collection and Analysis

Considering the large number of medical service units in Shanghai, this survey selected a few representative regions to carry out in-depth research. We randomly submitted questionnaires to medical institutions of Putuo District, Xuhui District, Minhang District and Changning District in Shanghai, and conducted deep research on doctors, nurses and hospital managements in Shanghai No. 6 People’s Hospital, Huashan Hospital, Shanghai fifth People’s Hospital and other relevant hospitals. We issued 401 questionnaires to patients at the aforementioned hospitals, and the number of effective questionnaires was 332 after eliminating invalid samples. The number of effective questionnaires for doctors who work at Shanghai Sixth People’s Hospital is 108.

The profiles of respondents (patients) are summarized in Figure 1, Figure 2 and Figure 3:

In the process of questionnaire design and filed research, we also considered the severity of patients’ conditions, patient-hospital travel distance, the degrees of patients who would keep their appointments, the preferences and habits of patients, the transparency of hospital’s information (whether or not to inform patients in advance), whether there exists the problem of excessive medical treatment and so on.

The main problems are listed as follows:

From Figure 4, we can see that the four most time-consuming parts regarded by the patients are all related to the waiting time.

In addition, we find that the four processes with the lowest degree of satisfaction are matched with the corresponding waiting time. Thus, we can conclude that one of the core problems in the process of making an appointment is the waiting time (shown in Figure 5).

3.3. Analysis of Major Problems

For patients who queued up for the payment within 5 min, were satisfied with the service (rating score ranged from 5 to 10 points). However, there was still a number of patients (13.2%) who were extremely dissatisfied with the waiting time for payment. Most patients had moderate degrees of satisfaction when the waiting time ranged from 5 to 20 min, while the number of dissatisfied patients was less than that in the former case. When the waiting time was more than 20 min, more patients (16.2%) were dissatisfied. The sensitivity is not strong because the queuing time for payment was generally much less than that of the treatment. Therefore, it would be suitable to control the queuing time for payment between 5 and 10 min if the hospitals were able to do the coordination.

Waiting time is one of the most prominent issues in the process of making appointments of medical facilities. From queuing time for reservation, to queuing time for payment, and then queuing time for medical check, each process may result in the waste of time and resources. Based on the investigation results, time of queuing up for an appointment is the longest. Specifically, 186 patients voted for the queuing time for payment, 162 patients voted for the waiting time for seeing the doctor, 161 patients voted for the waiting time for making an appointment. Based on the above analysis, the location arrangement, the complexity of the appointment process has a relatively weak influence on waiting time. From the corresponding satisfaction point of view, the corresponding satisfaction rates for the above processes were only 5.73, 5.76 and 5.31 (the rating rate ranges from 1 to 10).

✓

Longer time for waiting will reduce patient satisfaction.

For each reservation process, patients have to spend much time for waiting, which to a large extent, would affect other personal time arrangements. According to the survey data, more than half of the patients (56.43%) had to leave early because of other time conflicts. That is to say, to a certain extent, the longer waiting time for the appointments affects patients’ work and life and even good mood, which will result in an increase in rates of missing appointments.

✓

Longer time for waiting exposes the problems of medical management.

Waiting for a long time may also lead to leaving the medical equipment idle, resulting in a waste of medical resources and lower efficiency. Therefore, shortening the time for waiting is an important issue for medical management.

✓

The severity of patients’ conditions, patient–hospital travel distance and travel time would have impacts on patient satisfaction.

Compared to inpatients, appointed patients, and the randomly arrived patients, emergency patients have higher requirements for instant treatments. The severity of patients’ conditions would result in different expectations on patient satisfaction. Patient–hospital travel distance and travel time would also influence patient satisfaction.

✓

Insufficient information transparency would reduce patient satisfaction.

Results indicate that insufficient information transparency reduces patient satisfaction. In the responses to the question “there is no information about the number of people queued up”, most of the patient satisfaction scores range from 0 to 5. Based on the field research, we find that providing queuing number information has a greater impact on patients’ noncompliance to clinic appointments.

Among the patients who have higher degrees of satisfaction, most are satisfied with the relevant measures to improve transparency of hospitals on queuing information. Specifically, the rate of patients who would like to know the specific time for waiting is 80.3%, higher than the number of patients who would like to know the specific people who are in the waiting list (69.1%). The transparency of information is also reflected in the usefulness of the information (the value of the information) provided to patients. Therefore, how to convert public information into more valuable information for patients is also an important issue for hospitals to consider.

✓

The problem of excessive medical treatment would reduce patient satisfaction.

The issue of excessive medical treatment in China cannot be ignored. The excessive medical treatment refers to the inappropriate, irregular, and unethical medical behaviors, including over-examination, over-treatment, and over-medication. In particular, the problem of over-examination would result in the unnecessary occupation of medical equipment, which would further prolong patient waiting time and make the problem of long waiting time more serious. Meanwhile, unnecessary over-examination also indirectly aggravates the problem of mistrust between doctors and patients.

4. Methodology

4.1. Research Method

The following algorithms [36] are commonly adopted for the scheduling optimization during the diagnosis process (as shown in

Table 1. Algorithm comparisons.

	Convenience	Rapidity	Computation	Randomness	Optimality
Greedy Algorithm Based on Information	Strong	Strong	Small	Incapable	Poor
Backtracking Algorithm with Rollout Strategy	Strong	Strong	Small	Incapable	Poor
Heuristic AND/OR map (AO*) Search Algorithm Based on Least Test Cost	Weak	Weak	Large	Incapable	Good
Depth-first Search Method of Dual Heuristic Function	Weaker	Weaker	Smaller	Incapable	Good
Markov Decision Process (MDP)	Stronger	Stronger	Large	Capable	Good

):

The selection of MDP as the research method in this study is mainly based on: First, the hospital scheduling optimization problem has high requirements on the result of strategy optimization. Second, the visiting behaviors of patients and the decision-making behaviors of hospitals are highly random and computationally intensive. Third, diagnosis of MRI equipment optimization scheduling problem on the service day belongs to a periodic observation of dynamic decision process, which would be applied to study the MDP.

The application of MDP is as follows: patients’ satisfaction is introduced, and patient satisfaction is embedded in the finite horizon Markov Decision Process (MDP), the original reward (r)—net benefit is replaced. Take the patient satisfaction as a new form of reward for making decision action a_v based on Sv.

The capability of the MRI equipment is measured by time slots, which are the base of time; for example, each time slot in an MRI is set as 15 min.

4.2. Problem Definition

Both hospitals and patients are in urgent need for solving the problem of long waiting time during the healthcare appointment reservations and treatment processes. However, most of the existing studies optimized the schedule based on hospital revenue while ignored patient satisfaction. To make up for this gap, by adopting the finite horizon MDP and considering patient satisfaction and hospital revenue, we scheduled patients with large medical diagnostic equipment as an example, so as to obtain a better decision-making theory and practice at the same time. Reservation scheduling is dynamic and complex. In order to better reflect the reality, the definitions and assumptions are listed below:

Condition 1.

Patients are divided into four categories: emergency patients (type A); Appointed inpatients (category B); Appointed outpatients (category C); randomly arrived outpatients (class D).

Emergency patients are in urgent need for immediate equipment examination. Inpatients are less urgent than emergency patients, because they have obtained proper treatment and need to monitor their physical conditions through facilities for further diagnosis and treatment. Appointed outpatients are less urgent than appointed inpatients, because they have been examined by doctors, and they only need the check by diagnostic equipment for further diagnosis. The randomly arrived outpatients are less urgent than appointed outpatients, because only some of them would need further physical examination after the examination by doctors. Among them, type B and type C patients have already reserved appointments from the hospital reservation systems. Patients can choose whether to accept these periods of time or not. If not, they can cancel them from the scheduling system of the day.

Condition 2.

Different levels of priorities are divided according to the severity of the disease, the appointment status and the different target preferences. That is, when there are multiple patients waiting at the same time, the hospital will choose the order of patients to be examined according to the priority levels.

Condition 3.

Factors influencing the service time of the device, such as doctors’ operating proficiency and the interaction between the operator and the patient, are ignored. For the sake of simplification, we assume that each inspection period is equal.

Therefore, in the case of fixed number of equipment and working hours, the number of time slots per day and the number of served patients are also fixed. According to the principle of queuing theory, the maximum number of each type of patient at each time is one.

Condition 4.

Due to the existence of reservation conditions, types B and C patients can arrive on time or break the contracts (including not arriving on time and no-show).

Condition 5.

Randomness of arrival of various types of patients is subject to Bernoulli distribution.

4.3. Model Construction

Based on the above principles and assumptions, the patient waiting state (s) model, the hospital decision action (a) model, and the reward function (r) model in the MDP are distributed and constructed.

Based on the assumption that the inspection time of the MRI device is fixed, the day is now divided into N time slots. Each time (stage) is a one-time time slot in the MDP, the state at the n (n = 1, 2, … N − 1) moment can be expressed as:

$$\overrightarrow{{\displaystyle S}}=\left(W_A,W_B,W_C,W_D,d_B,d_C\right)$$

(1)

$W_j$ represents number of class j patients waiting at time n; ∀_j ∈ {A, B, C, D}.

$d_B$, $d_C$ denote class B or C patients arrive on time, d_B = 1 or d_C = 1; d_B = 0 or d_C = 0 otherwise. (the appointment will be examined at time n and will arrive at time n − 1 to time n).

Here, under the existing MDP patients state representation model [22], we use the number of waiting patients at each time slot to represent the state of MDP, and expand the model by using the binary number (binary variables) d_B and d_C, respectively, to represent whether the studied patients arrive ahead of time or arrive on time, which adds the situation of breaching a booking into the model.

1. Decision-making action of MDP

(1) decision-making actions in each state can be expressed as

$$\overrightarrow{{\displaystyle a}}=\left(a_A,a_B,a_C,a_D,a_W\right)$$

(2)

$a_j\in \{A,B,C,D\}$: the number of class j patients examined at time n + 1,

$a_W$: a binary variable, $a_W=1$ if a random patient required to be examined accept the appointment at time n+1, otherwise, $a_W=0$ (2) the constraints

$$a_A+a_B+a_C+a_D\le G$$

(3)

$$a_A=W_A$$

(4)

$$a_j\le w_j{\forall }_j\in \{A,B,C,D\}$$

(5)

$$a_j\ge 0{\forall }_j\in \{A,B,C,D\}$$

(6)

The above constraints can be interpreted as: the number of patients examined at the next time shall not exceed the number of equipment G; emergency patients must be examined as early as possible due to the instability of the condition. The number of patients selected to be examined should not exceed the number of waiting patients.

2. Transition probability of MDP

Under the status of the n-th time S, according to the decision strategy, under the constraint condition, a feasible decision action (namely to choose what type of patients should receive inspection, which are considered as the “action”) is chosen, after which, the state is changed, and the “state transfer” make it to the state of n + 1 time S’ [22,37].

$$S^{\prime }=\left(d_A^{\prime },W_B-a_B+d_B^{\prime }+Z_B^{\prime },W_C-a_c+d_C^{\prime }+Z_C^{\prime },W_D-a_D+d_W\ast a_w,d_B^{\prime },d_C^{\prime }\right)$$

(7)

$d_A^{\prime }$: if there is a class A patient arriving at time n to n + 1, then $d_A^{\prime }$ = 1, otherwise 0;

$d_B^{\prime }$ if there is a class B patient arriving at time n to n + 1, then $d_B^{\prime }$ = 1, otherwise 0;

$d_C^{\prime }$: if there is a class C patient arriving at time n to n+1, then $d_C^{\prime }$ = 1, otherwise 0;

$d_W^{\prime }$: if a class D patient arrives randomly on the service day and receive the recommended time of examination by the hospital, then $d_W^{\prime }$ = 1; otherwise, it is 0.

$Z_B^{\prime }$: if class B patients who have made an appointment for inspection delay to arrive in n to n + 1 time, the $Z_B^{\prime }$ = 1, otherwise 0.

$Z_C^{\prime }$: if C patient make an appointment for inspection at a time of n and delay to arrive in n to n + 1 time, then $Z_C^{\prime }$ = 1, otherwise 0.

As mentioned above, since the study is based on the schedule on the service day, it is necessary to construct the state of patients at the next time (n + 1) in terms of whether the appointment is made and whether the appointment is delayed. Therefore, in the next time state representation model of MDP patients [36,38,39,40,41,42,43,44,45], two additional binary variables $Z_B^{\prime }$ and $Z_C^{\prime }$ are added to explain the delayed arrival of patients on appointments.

Therefore, the transition probability from S to S’ under decision action a (in this case, “partial transfer probability” in MDP) is defined as the following equation through mathematical deduction [37]:

$$P\left(S^{\prime }S,a\right)=P\left(d_A^{\prime }\right)\left\{\{\begin{array}{r}{p}_2\ast p_3,d_C^{\prime }=1\\ 1-p_2\ast p_3,d_C^{\prime }=0\end{array}\right\}P\left(d_C^{\prime }\right)\left\{\{\begin{array}{r}0,Z_B^{\prime }=1,d_B=0\\ 1,Z_B^{\prime }=1,d_B=0\\ \frac{\left(1-p_2\right)p_5}{1-p_2{p}_5},Z_B^{\prime }=1,d_B=0\\ \frac{1-p_2}{1-p_2{p}_5},Z_B^{\prime }=1,d_B=0\end{array}\right\}P\left(Z_C^{\prime }{d}_C\right)\left(1-P^{\prime }\left(d_W\right)a_W\right)$$

(8)

According to the definition of the above question, p₂ represents the probability of appointed inpatients come to the hospital, and p₃ represents the probability of appointed inpatients arrive at the hospital on time. P ($Z_B^{\prime }$ = 1, inpatient d_B = 1) represents the reserved inpatients arriving in the n − 1 to n moment and the probability of reserved inpatients for inspection in the n time delay to n + 1 time; then, according to the actual situation, P ($Z_B^{\prime }$ = 1, d_B inevitable to 0 = 1) must be 0.

3. Satisfaction function

At present, the satisfaction function method is widely used in the research of multiple corresponding optimization problems, and its concept has been widely used in decision-making analysis. However, there are few reports on the application of medical decision making based on doctor–patient preference differences.

The satisfaction function introduced in this study, i.e., making decision av under the state Sv in the decision-making process of Markov in discrete time, will obtain the corresponding satisfaction. In order to measure satisfaction, the patient satisfaction function is constructed (

Table 2. Discrete function.

Category	Type A	Type B	Type C	Type D
Benefits from patient satisfaction	800	400	300	900
Cost resulted from fulfilling patient satisfaction	1000	800	700	200

).

According to the MDP, for every decision (acceptance or rejection) the hospital makes, there is a patient satisfaction feedback. When the hospital accepts a certain type of patients, it will bring satisfaction benefits to the hospital. When a hospital rejects a certain type of patient, there is a satisfaction cost. Combined with the actual investigation, a discrete satisfaction function based on patient priority was constructed. For emergency patients (type A patients), when the hospital chooses to accept such patients for equipment examination, it will bring 700 patient-satisfaction benefits to the hospital. When hospitals choose not to accept such patients for equipment examination, it will result in 200 patient satisfaction losses. Because of the timeliness, urgency and necessity of equipment examination for emergency patients, hospital acceptance of such patients will not yield a high satisfaction benefit; however, once the hospital rejects emergency patients, it will lead to a high satisfaction cost.

5. Experimental Results

The data for empirical analysis in this study are collected from Shanghai Sixth People’s Hospital, an affiliated hospital to Shanghai Jiao Tong University as a teaching base for the School of Medicine, which attained the rank of “Grade 3, Class A” in Chinese hospitals. The collected sample data showed that there were large differences among the arrival probability of different types of patients. According to the actual situation of the hospital, he working time per day for the MRI scanner is 8 h. Therefore, the period during 8:00 a.m. to 17:00 p.m. is divided into many time slots. The arrival probabilities of the four types of patients are obtained from the statistical data of Shanghai Sixth People’s Hospital. Specifically, in each time slot, the probability of arrival for different types of patients equals the total demand over the total number of days in the statistical period.

5.1. State Space

In the case study, the maximum number of each type of patients at each time is one, that is, the number of patients waiting in each class is a binary variable with a value of 0 or 1. As follows, the waiting states of all possible patients at each time are exhaustive to form a state space of 16 states (shown in

Table 3. States space.

State SpaceA, (WB, WC, WD)	Remarks
(0,0,0,0)	No waiting of patients
(1,0,0,0)	One emergency patient is waiting
(0,0,0,1)	One randomly arrived outpatient is waiting
(0,1,0,0)	One appointed inpatient is waiting
(0,0,1,0)	One appointed outpatient is waiting
(1,0,0,1)	One emergency patient and one randomly arrived outpatient are waiting
(0,1,0,1)	One appointed inpatient and one randomly arrived outpatient are waiting
(0,0,1,1)	One appointed inpatient and one randomly arrived outpatient are waiting
(1,1,0,0)	One emergency patient and one appointed inpatient are waiting
(1,0,1,0)	One emergency patient and one appointed outpatient are waiting
(0,1,1,0)	One appointed inpatient and one appointed outpatient are waiting
(0,1,1,1)	One appointed inpatient, one appointed outpatient and one randomly arrived outpatient are waiting
(1,0,1,1)	One emergency patient, one appointed outpatient and one randomly arrived outpatient are waiting
(1,1,0,1)	One emergency patient, one appointed inpatient and one randomly arrived outpatient are waiting
(1,1,1,0)	One emergency patient, one appointed inpatient and one appointed outpatient are waiting
(1,1,1,1)	One emergency patient, one appointed inpatient, one appointed outpatient and one randomly arrived outpatient are waiting

).

5.2. State Transition Probability

According to the actual situations based on survey, the actual arrival probabilities of all kinds of patients are shown in

Table 4. Probability of arrival for various types of patients.

Probability of Arrival for Various Types of Patients	Probability Value
p1	0.2
p2	0.7
p3	0.6
p4	0.5

.

By substituting the actual arrival probability of various patients into the constructed model, the state transition probabilities under different strategies can be calculated, and the state transition probability arrays can be obtained.

5.3. Comparison of Optimal Strategies under Different Objective Functions

Combined with field investigations, the objective bias of daily service MRI equipment inspection can be divided into patient satisfaction bias, hospital benefit bias, patient satisfaction degree, and hospital benefit balance bias. According to different target preferences, this study carries out three types of reward matrix setting, which depicts the benefits (or costs) brought by accepting (or rejecting) a certain type of patients. The value iteration algorithm is used to solve MDP, and then the optimal strategy under three kinds of target bias can be solved. Comparison of optimal strategies as illustrated in Figure 6.

The remuneration setting is explained as follows:

Based on the consideration of quantified satisfaction, the unit of uniform revenue, and cost, the value of [0, 10] was set as an indicator to measure the acceptance benefit and rejection cost based on the actual situations in the survey.

Payment setting based on patient satisfaction is shown in

Table 5. Payment setting based on patient satisfaction.

	Emergency Patients	Appointed Inpatients	Appointed Outpatients	Randomly Arrived Outpatients
Benefit	8	4	3	9
Cost	−10	−8	−7	−2

.

(1)

Emergency treatment: because the emergency patient is in an urgent condition, whether the hospital accepts or rejects the emergency patient’s behavior will have a great impact on the satisfaction of the emergency patients. Because there is the possibility of loss of life, the rejection cost caused by refusing the emergency patient’s request is the highest.

(2)

Treatment for appointed patients: the conditions of appointed inpatients are generally more serious, so once they have been rejected, their lengths of hospital stay will be extended. Therefore, this condition may cost more money, and the absolute value of accepting revenue and rejecting cost is greater than the appointment reservations.

(3)

Service for randomly arrived outpatients: generally, the conditions of randomly arrived patients are not quite serious, and under the conditions of medical equipment tensions, if the request of randomly arrived outpatients can be accepted, the satisfaction degree could be further improved; however, even if the requests of randomly arrived outpatients were refused, the costs would not be high.

Payment setting based on the income of the hospital is shown in

Table 6. Payment setting based on the income of the hospital.

	Emergency Patients	Appointed Inpatients	Appointed Outpatients	Randomly Arrived Outpatients
Benefit	7	10	6	4
Cost	−4	−8	−6	−3

.

(1)

Emergency patients: the treatment of emergency patients will consume more resources from hospitals, which enables hospitals to obtain more benefits; however, due to the certain probability of death, the absolute value of accepting and rejecting expectations will be below the reservation patients.

(2)

Appointed inpatients: appointed inpatients tend to be in serious condition, therefore, variations of hospitalization costs and other checking costs would be more than the examination costs of appointed outpatients. To summarize, both accepting revenue and rejecting cost are quite high.

(3)

Randomly arrived outpatients: comparatively, the conditions of randomly arrived clinic patients are not quite serious, therefore, the project income from other inspection projects are less. In conclusion, the revenue and cost to accept and reject randomly arrived outpatients are the lowest among these 4 types of patients.

Based on the analysis above, payment setting of equilibrium bias is illustrated in

Table 7. Payment setting of equilibrium bias.

	Emergency Patients	Appointed Inpatients	Appointed Outpatients	Randomly Arrived Outpatients
Benefit	7.5	7	4.5	6.5
Cost	−7	−8	−6.5	−2.5

.

The weighted average algorithm reflects the balanced consideration of satisfaction and hospital benefits. The formula is as follows:

Acceptance benefit balance =0.5* acceptance benefit satisfaction +0.5* acceptance benefit of the hospital

Rejection cost balance =0.5* rejection cost due to satisfaction +0.5* rejection cost due to hospital benefits

1. Three reward matrices

According to the three kinds of reward settings, three kinds of reward matrix corresponding to the three kinds of strategies are obtained for further calculations.

2. Solutions for the optimal strategy

Using Matlab, the value iteration and strategy iteration algorithm were used to obtain the optimal strategy response under the three target orientations, and the specific implementation process project data were given in detail.

3. The policy set is explained as follows (shown in

Table 8. Policy set.

Optimal Policy	Decision-Making Actions
1	(0,0,0,0)
2	(1,0,0,0)
3	(0,1,0,0)
4	(0,0,1,0)
5	(0,0,0,1)

):

Under the target pursuit of patient satisfaction, according to the MDP result, we suggest that the optimal strategy of the hospital should be strictly based on the cost of patient satisfaction, and the selection (acceptance or rejection) of decision-making actions should be determined according to the sequence of various types of patient satisfaction costs. Satisfaction costs are set according to the first type of remuneration: A>, B>, C>, D, corresponds exactly to the priority levels of patients. Therefore, in any waiting states, hospitals should give preference to type A patients (emergency patients) for examination. When there is no optimal grade of patients, sub-optimal type B patients (patients who are scheduled to be hospitalized) are selected for examination, and the optimal strategy is obtained by analogy.

Under the target pursuit of the hospital’s revenue, the optimal strategy of the hospital is strictly based on the patients’ benefits, and the corresponding decision-making actions are determined according to the benefits brought by the acceptance of various types of patients and the costs brought by the rejection of various types of patients. According to the acceptance benefits of its reward matrix, class B > class A > class C > class D. Therefore, in any waiting states, as long as there are type B patients (reserved inpatients) waiting, the hospital must give preference to type B patients for inspection. According to the rejection cost of its reward matrix: class B > class C > class A > class D, the hospital must give preference to type C patients (appointed patients) if there were no type B patients waiting.

Under the target of ensuring patient satisfaction and hospitals’ benefit balance, two reward matrices under satisfaction goal and hospital benefit target are given 50% weightings, and the benefit matrix under equilibrium condition is obtained through the weighted average. The optimal strategy results show that the decision actions of the hospital take into account patient satisfaction and hospital benefits. The satisfaction cost of type-A patients is the highest. As long as there are type-A patients waiting and no type-B patients waiting, the hospital will give preference to type-A patients for examination. Therefore, under the balanced strategy, the hospital not only considers the patient satisfaction, but also optimizes the decision-making actions under the condition of maximized hospital’s benefits.

In terms of equipment utilization rate, through the simulation method, we find that when the slot = 12, the equipment utilization rates remain constant under three different strategies. That is, under any states, equipment utilization rates are equal on the service day based on the results of three different decision-making principles. Results show that the average device vacancy rate is 4.91% per day; in other words, the device utilization rate is 95.09% per day. Therefore, results indicate that the device utilization rate has reached the highest possible level with the proposed method in this study.

6. Discussion

In this section, we will evaluate the optimal strategy effect under different target orientations. The actual simulations of the various patients waiting for equipment inspection on the day of 1000 service days, and the corresponding decisions for the patients waiting for the simulation are based on the three major optimal strategies.

Meanwhile, this study further improves the authenticity of simulation simulations and the effectiveness of evaluation by changing the number of time slots (slot = 6, slot = 12, slot = 18, slot = 24). The slots can be understood as the number of times a device can be checked by patients within one device service day. In fact, there are different device inspection arrangements in each major hospital.

1. Optimal strategy effect

According to the Figure 7, the optimal strategy (strategy 1) for patient satisfaction target is set, the optimal strategy (strategy 2) for hospital revenue target is set, and the optimal strategy (strategy 3) for a balanced target is set. After 1000 service days, the average benefits of the three optimal strategies in the four slots under their respective measurement indexes are obtained.

2. Optimal decision-making effect

Based on 1000 simulations and changes in the number of time slots, the effect of optimal decision making can be evaluated from three perspectives: average hospital benefit, patient satisfaction, and number of patients examined under the guidance of the three optimal strategies (Figure 8).

First of all, according to the results, if there were more slots, there would be more average benefits bring to hospitals, especially in the case of slot =24. Results indicate that the three optimal strategies can all bring positive average benefits to the hospital.

Secondly, the results show that there was not much difference between the average benefit of strategy 3 under the guidance of balanced target bias and the average benefit of strategy 2 under the guidance of hospital benefit bias. Even in 1000 simulations, the average benefit of strategy 3 exceeds that of strategy 2. This indicates that strategy 3, which takes both hospital benefits and patient satisfaction into account, has a good guiding effect on decision-making actions of hospitals, reflecting the operability and practical significance of future promotion of this balanced strategy.

As shown in Figure 9, the optimal strategy under the guidance of patient satisfaction will have a negative situation in any slots. This result is also consistent with the actual situation, and patient satisfaction is one of the aspects that hospitals seldom consider when making decisions and choices, which is the focus and difficulty of this study.

According to the evaluation results, strategy 1, which is based on the patient satisfaction bias, can bring the highest level of patient satisfaction, which is also the ideal target preference pursued by the public. Of course, under the current situation, it is necessary to consider the benefits of the hospital, the optimal decision-making actions guided by the equilibrium bias can bring higher patient satisfaction on the basis of guaranteeing the benefits of the hospital, compared with the optimal decision-making actions guided by the pure revenue bias.

This study takes 12 slots as an example and carries out the simulation of all kinds of patients waiting for device inspection for 100 service days, which effectively characterizes the actual number of patients for checking. The bubble charts were used to show the actual number of patients under the guidance of three optimal strategies. The larger the bubble, the greater the proportion of patients who are examined by the device within a day. The actual number of patients examined is affected by both the target bias and the actual arrival probability of patients.

As shown in Figure 10, Strategy 1 (or policy 1) tends to select type B patients (appointed inpatients) and type-A patients (emergency patients). This is because it is a strategy based on the preference of patient satisfaction. If type A patients are not selected, the rejection cost of patient satisfaction would be quite high. The actual arrival probability of type B patients was the highest, and the proportion of patients selected in the actual situation was also higher.

Strategy 2 (or policy 2) tends to select type B patients (appointed inpatients) and type C patients (appointed outpatients). This is because strategy 2 is formulated from the perspective of the hospitals’ revenue target. The economic benefits of receiving type B patients is the highest, and the cost of rejecting type C patients is the highest. From the perspective of hospitals’ economic benefits, since the economic benefits brought to the hospital by type A and type D patients are relatively low, fewer patients of these two types would be selected by the hospital.

Under the condition that hospital benefits are guaranteed, patient satisfaction has been comprehensively considered. Therefore, the proportions of type B and type C patients are relatively high. Meanwhile, compared with strategy 2, the proportion of type A patients also increased.

As shown in Figure 11, type A patients (emergency patients) are more likely to be selected under strategy 1. The proportion of type A patients selected under strategy 2 was lower. This difference is due to the fact that strategy 1 is biased towards patient satisfaction, and type A patients must be selected for facility examination whenever they showed up. However, strategy 2 will not give priority to type-A patients with the constraint of hospital revenue.

The proportion of type B patients (appointed inpatients) selected under the strategy 3 is quite high, which is related to the actual arrival probability of type B patients. Among the four types of patients, type B patients have a higher probability of actual arriving due to their characteristics, and therefore, a higher proportion of type B patients are selected.

The proportions of type C patients (appointed outpatients) selected under strategy 2 and strategy 3 are higher than that under strategy 1. This is because type C patients bring higher acceptance benefits and rejection costs to the hospital, therefore, under the guidance of strategy 2 and strategy 3, which take the hospital benefits into consideration, the proportion of type C patients has been increased.

Type D patients (randomly arrived patients) were selected with a lower proportion under the three strategies due to the lowest rejection cost. However, type D patients are more likely to be selected under strategy 1, which is because the satisfaction benefit of such patients is the highest among the four types of patients.

The probability of actual arrival of appointed patients is calculated according to the actual data of the Shanghai Sixth People’s Hospital. Moreover, the assumptions of the division of patient priority as well as the utilization rate of equipment are based on the actual situations of hospitals in China. Therefore, the empirical results of this study are practical and applicable, which can provide a valuable reference for the systematic optimization of scheduling for medical equipment inspection. The proposed model in this study can be put into the field research of several hospitals and can be further enriched and enhanced to ensure the maximum utilization of equipment and make more scientific arrangements for patient examination. In this context, the waiting time of patients can be shortened, the patient satisfaction can be improved to alleviate the intense doctor–patient relationship, and thereby improve national happiness. On the basis of existing research, we not only considered the hospital’s income, but also explored the scheduling model to improve patient satisfaction from the perspective of patients, in order to achieve a win–win situation between doctors and patients.

7. Conclusions and Implications

Combined with the actual situations, depending on the classification of patients based on their emergency degrees, according to four different types of patients and the examination facility of MRI, a limited time MDP model is put forward for dynamic scheduling.

By ensuring hospital benefits, we take the satisfaction of patients into account. The patient satisfaction of being accepted for examination and the loss of satisfaction caused by rejection are included to approximate the satisfaction of different situations. In the case of ensuring equipment utilization rate, the optimal strategy pursued by different hospitals is obtained through value iteration.

The longer waiting time has negative effects on patient satisfaction. Usually, VIP (Very Important People) patients can receive on-demand urgent care services in private hospitals. Therefore, the priority of medical treatment (e.g., the access to appointments and treatment without waiting) is determined by patient types (VIPs or non-VIP patients) in private hospitals. However, considering that public hospitals dominate in China’s hospital system, and medical pricing and management are regulated by Chinese government, “VIP. Syndrome” is not included in this study. In other words, the waiting time in this study is determined by the emergency of treatment other than the VIP Syndrome. Specifically, type-A patients (emergency patients) have higher priority than type-B patients (who make appointments to be hospitalized), and type B patients have higher priority than type-C patients (appointed outpatients), and type-C patients have higher priority than type D patients (randomly arrived patients).

Through simulations and comparisons of the optimal strategies under three different objectives, we find that different objectives would affect the inspection decisions of the hospital. Patient satisfaction is biased downwards, the optimal strategy of the hospital is strictly based on the cost of patient satisfaction, the priority sequence of patients is selected as: Type A, Type B, Type C, Type D. The hospital’s income is biased downwards, and the hospital’s optimal strategy is strictly based on the hospital’s income and rejection costs, the priority sequence of selecting patients is: Type B, Type A, Type C, Type D; according to the rejection cost of the reward matrix, the priority sequence of patients is as follows: Type B, Type C, Type A, Type D, in the absence of Type B patients waiting, the hospital will choose to reject the higher-cost type C patients for examination; the equilibrium is biased downward, the hospital’s decision-making actions comprehensively consider the patient satisfaction and the hospital’s income, and the type B patients and the type A patients are the key decision-making targets of the hospital. Moreover, the balance between patient satisfaction and hospital benefits is still one of the difficulties in the process of making the inspection decision. Based on the model proposed by this study, the hospital can improve its strategy according to its actual situations and the pursued targets. While considering the benefits, the hospital also pays attention to patient satisfaction. Future studies can consider the following research directions. First, more examination items should be considered, because a higher proportion of patients receive more facility examinations than MRI. Second, consider more time slots or the increased number of patients examined on the service day. Third, consider the same kind of multiple devices at the same time for inspection [46,47,48].