1. Introduction
According to the State Statistics Bureau, there were 2.37 million traffic accidents nationwide between 2012 and 2020, 21% of which were caused by fatigue. Therefore, it is crucial to study fatigue driving detection methods. Driver fatigue monitoring systems have been extensively studied. Domestic and international efforts to monitor driving fatigue are characterized by two main aspects: driving behavior and physiological features. Driver fatigue monitoring systems detect driver fatigue using behavioral features such as the driver’s speed [
1] and steering wheel motion [
2]. However, the recognition accuracy is not high because of the influence of vehicle type, road conditions, and personal driving habits. Therefore, many researchers have studied driver fatigue monitoring systems using physiological signals such as electroencephalogram [
3], electrooculography, electrocardiography, and eye and mouth movements. Although the accuracy of these physiological signals for fatigue monitoring is high, most physiological data measurements are intrusive, and the signals are challenging to extract. A suitable solution is to use less intrusive and more easily extractable physiological signals as auxiliary markers to monitor fatigue.
As an important physiological index of the human body, the temperature can be obtained through non-contact to monitor fatigue. Theoretical studies have found that driving fatigue is closely related to arousal level, which decreases when the driver is fatigued. Researchers [
4,
5] have found that the temperature of parts of the human body, such as the nose and fingers, can reflect the activity of the autonomic nervous system and be used to track arousal levels. Because the autoregulatory system regulates the cardiovascular system, including heart rate (HR) and the peripheral vasomotor system, skin temperature changes occur in the fatigue state. Moreover, in Jagannath’s [
6] simulated driving fatigue test, subjects were found to have altered HR and blood pressures before and after fatigue.
To obtain skin temperature, it is necessary to build a biological heat transfer model. Mathematical heat transfer models of biological systems have been the subject of extensive research by various biologists, physicians, mathematicians, and engineers. Many bioheat transfer models have been used to analyze heat transfer in skin tissue in previous studies, and the bioheat transfer model of Pennes is the most widely used model for studying temperature distribution. In 1948, Pennes presented a solution for biological heat transfer in the forearm. Wissler [
7], Wyndham and Atkins [
8] extended the model to the entire body. Multi-segment models [
9,
10,
11,
12,
13] usually include explicit simulations of heat transport and thermal regulation processes in the human body, considering characteristics such as body area, heat capacity, and tissue stratification [
14]. Tanabe et al. [
10] modeled the heat transfer process of biological tissues in layers, providing separate heat transfer parameters and establishing energy balance equations for different tissue layers. However, Tanabe’s blood perfusion model depends on local skin and core temperatures. With the continuous development of cardiovascular modeling, a cardiovascular system (CVS) model with lumped parameters [
11,
13] has been used to improve the accuracy of tissue blood perfusion.
Human heat transfer models have been widely used; however, most of these approaches [
15,
16] have only used empirical models to assess blood flow and heat transfer from the body core to the body surface without considering the real closed-loop cardiovascular system. Blood flow is an essential factor that influences body heat diffusion and skin temperature, and external conditions can affect skin microcirculation and lead to changes in skin temperature [
17,
18]. Owing to the complexity of cardiovascular networks, the lumped parameter method is the best choice for building a CVS model due to its simplicity of modeling, ease of solution, and low computational effort. The lumped parameter model (LPM) of cardiovascular systems can be used not only to study the hemodynamics of the cardiovascular system but also to explain the link between hemodynamics and heat transfer in the human body [
12,
19].
In this study, an integrated full-body thermal model based on lumped parameters was built to predict the temperature of different body parts and research changes in the cardiovascular system during fatigue. Infrared thermography temperature measurements were conducted before and after fatigue in eight young adults using fatigue induction experiments. The hand and face temperatures of subjects under normal waking conditions were used to deduce tissue layer thickness and skin resistance. Changes in different subjects’ heart rates and skin resistance were inversely evaluated using the post-fatigue hand and face temperatures considering the skin characteristics of each subject. Finally, changes in the cardiovascular system before and after fatigue were compared.
2. Materials and Methods
The primary objective of this study was to construct a simulated model of flow heat transfer in the human body using an LPM. In contrast to previously published studies, the cardiovascular system was considered in the tissue heat transfer model to provide blood perfusion. In this study, a full-body thermal model with CVS was developed. The entire body was divided into four segments (the head, torso, upper limbs, and lower limbs). Owing to the varied tissue composition in different body parts, the head and torso segments were divided into five layers (core, bones, muscle, fat, and skin), and the upper and lower limbs segments were divided into four layers (bones, muscle, fat, and skin). A conceptual illustration of the integrated model is shown in
Figure 1. Heat is exchanged between the body and the environment by convection, radiation, and evaporation. Individual segments exchanged heat through conduction. Blood perfusion was distributed in different proportions across various tissue layers.
2.1. Heat Transfer Tissue Model
The relationship between skin temperature and local blood perfusion has been an interesting topic for researchers. Due to the presence of different structures and distinct physical properties of each tissue layer, it is challenging to study the thermal response of living skin tissues. For anisotropic tissue, Anders et al. [
20] explored the relationship between two layers of skin and blood perfusion using an electrical analog model, which showed the thermal response in human skin.
2.1.1. LPM Model of the Heat Transfer in Tissue
Considering tissue thickness, surface area, and physical properties, the process of heat transfer in tissues was simulated using circuit elements. To simplify the calculation, each tissue layer was considered as the same material with effective properties (specific heat
c, thermal conductance
λ, tissue density
ρ, and tissue thickness
l). The properties of each layer are listed in
Table 1 [
21]. As shown in
Figure 2, the voltage can be represented as an electrical analog measure of temperature. In this model, each layer consists of a serial connection of two elements, each of which is constituted by heat resistance
r =
l/
λ and heat capacity component
C =
cρl.
l represents tissue thickness, and the blood perfusion,
q ml/cm
2, generates the heat flux,
ΦA =
qρc(
Tb −
Ti) W/cm
2. From this expression, the perfusion thermal resistance can be defined as
REq = 1/
qρc. Moreover,
Vi simulates the arterial temperature, and
Ve simulates the environmental air temperature. The heat resistance, Re, and REL account for inevitable heat loss during heat isolation and air thermal resistance on the skin surface, respectively. The model parameters are listed in
Table 1.
2.1.2. Heat Transfer by Blood Perfusion
It is necessary to determine the blood perfusion of the tissue layer to evaluate heat exchange between tissues and blood. Because the cardiovascular system model can only determine total blood perfusion, the blood perfusion of each tissue layer (BP) must be assigned. In this study, BP was determined by summarizing the relative ratio of blood perfusion of each tissue to total blood perfusion reported in previous research [
21]. Thus, the ratio of the blood perfusion of each tissue to the total blood perfusion was calculated using a weighting method.
where
i is the number of body segments, including the head, torso, and upper and lower limbs;
m is the number of tissues; and
j is the number of tissue layers, including the skin, fat, muscle, bone, and core.
Total perfusion was distributed according to the proportion of blood perfusion in each tissue layer, as described above. The blood perfusion in each tissue layer was calculated by
ωij =
ẟij·
qi, where
qi represents the total flow through the arterioles of the tissue.
ẟij of each part is shown in
Table 2. The tissue layer innermost to the muscle layer is specified as the core tissue layer, whereas the fat and dermis are skin tissue layers. The calculated blood perfusion is used to calculate the core and skin blood flow resistance and considered core resistance and skin resistance, respectively. Blood flow through these resistances is referred to as core blood flow (CBF) and skin blood flow (SBF), respectively. The specific resistance values are shown in
Table 3.
2.1.3. Environmental Resistance
Heat transfer between the environment and body surfaces involves conduction, convection, and radiation. Φ (W/cm
2) is the heat exchange rate and is described by Equation (2).
ht (W/cm
2 °C) is the total heat transfer coefficient from the skin surface to the environment and is expressed by Equation (3). Tsk and Ta represent skin and ambient environment temperatures, respectively.
hr and
hc are the convective and radiant heat transfer coefficients, respectively. The environmental resistance was defined as REL = 1/
ht.
2.2. Cardiovascular System Model
The cardiovascular system (CVS) facilitates the blood perfusion required by the human body and helps maintain a normal skin temperature. As skin temperature and blood perfusion are closely related, modeling the cardiovascular system is beneficial for understanding the mechanism between heat transfer and the cardiovascular system. Hemodynamics in the cardiovascular system can be represented by a set of electrical elements based on the Windkessel model theory, considering the sophistication of the cardiovascular networks.
Yang’s left heart model [
22] was modified to predict blood flow in the entire human body. To obtain arteriolar flow in various parts of the body, blood circulation at the end of the body was enriched based on Liang’s model [
23]. In this model, the CVS was divided into six circulation types (heart, cerebral, upper limbs, torso, lower limbs, and peripheral), where the torso and lower circulations were simplified. The CVS model was built using the parameters
R,
L, and
C, where
R represents blood viscous resistance,
L represents blood inertia, and
C represents vessel wall elasticity. These values can be determined based on the anatomy of the vascular segments.
where
μ and
ρ represent blood viscosity and density, respectively;
E and
h denote vascular wall elasticity and thickness, respectively; and
l and
r express vascular length and diameter, respectively.
A complex electrical analog circuit was constructed for the CVS, as illustrated in
Figure 3. The model begins with a pressure source P
pu, which refers to a constant perfusion pressure in the pulmonary capillaries. Pulmonary capillary and venous compliances are lumped into a single capacitance C
pvc, which fills the left atrium (la) via L
pv and R
pv. The preload of the left ventricle (lv) comes through the mitral valve (mv). The afterload of lv consists of a lumped-parameter representation for ascending aorta (aa), the upper limb circulation (ulb), the cerebral circulation (cer), the thoracic aorta (ta), and descending aorta (da). Meanwhile, blood flows through the thoracic aorta to the lower limbs circulation (llb), including torso circulation (tor) and peripheral vascular circulation (pc). The blood flow in each vessel compartment of the upper limb circulation and cerebral circulation is in analogy with resistance, capacitance and inductance of an electric circuit in series, where R, S, C and L represent blood flow resistance, viscous part of the vessel, compliance of the vessel and blood inertia respectively. The additional subscripts of
a,
al,
c,
v, and
vv denote different vessels, including arteries (a), arterioles (al), capillaries (c), veins (v), and venous beds (vv). The RCL parameters were initially determined based on vascular anatomy. In the later stages, the RCL values were partially adjusted by matching blood flow and blood pressure from the relevant literature. The final RCL values are listed in
Table 4. As the subcutaneous vascular network is highly complex, the tissue blood perfusion rate is controlled by subcutaneous arterioles. The arterial flow from the cardiovascular system is distributed to the corresponding skin tissue to enable the coupling of the cardiovascular system and skin tissue heat transfer.
2.3. Coupled Computation
In this paper, the CVS model provides the blood flow information for the 0D human tissue heat transfer model to calculate the tissue temperature. The 0D cardiovascular blood circulation model was coupled to the human tissue heat transfer model through the thermal resistance of blood perfusion (
REq). The specific computational process is shown in
Figure 4.
2.4. Experimental Setup
Figure 5 shows a schematic of the experimental system for measuring physiological signals such as skin temperature and blood perfusion signals. In this study, fatigue experiments were conducted using infrared thermal imager temperature measurement methods. An infrared thermal imager was used to collect the infrared thermal images of the face and a single hand in the same frame as the subject (
Figure 5). A laser Doppler flowmeter (AD Instruments) was used to monitor blood perfusion in the index finger. Eight men volunteered to participate in this study. Their mean age, weight, and height were 24 ± 1.1 years, 73 ± 10.5 kg, and 1.7 ± 0.07 m, respectively. The experimental procedure was strictly followed for all subjects to ensure that the data could be compared without interference. All experimental procedures were performed in a controlled laboratory environment.
A small circularly-moving ball was used as the induction source in the infrared thermographic temperature measurement experiment. As the experiment lasted for only 10–15 min, the self-measurement method interfered with the fatigue experienced by the subject. Experimental videos were recorded, and the evaluator assessed the fatigue level of a subject based on the subject’s sleepiness, referring to the NEDO method [
24]. The NEDO method is a fatigue-level evaluation criterion based on facial expressions. The fatigue evaluation criteria divided the fatigue into five levels, as listed in
Table 5. In this study, the state of the subjects was evaluated by the raters every 30 s. To ensure that the evaluation results were not affected by individual differences among raters, the assessment results of the three raters were averaged to obtain final state evaluation results for each subject. Levels 1–2 were classified as awake, and levels 3–5 were classified as fatigued.
3. Data Processing
Infrared thermal imaging cameras only obtain image and temperature matrix information; therefore, it is necessary to extract temperature information from the desired part of the thermal image. The low resolution of the thermal imaging camera and blurred edges of the image result in poor target tracking. The adopted method combines visible-light images with target-tracking methods using computer vision to extract the temperature of the region of interest in the thermal image. The specific methods are as follows:
This paper uses the open-source library Dlib to perform facial feature point recognition on visible face maps. The open-source library uses the HOG method to identify face position recognition in the input vector, calls the already trained facial feature point detector to identify 68 facial feature points (see
Figure 6b), and outputs the position coordinates of the feature points. The identified feature points are named from minimum to maximum as f_0, f_1, …, f_67, then f_30 is selected as the center of the tip of the nose, and r1 is the rounded area with a radius of r1, which requires the dynamic calibration of r1 in each frame.
In this study, the open-source library Mediapipe was used for hand-feature point recognition on visible light maps. Similar to Dlib, an open-source library used for facial recognition, the palm region is detected first. Then, a trained hand feature point detector is invoked to recognize 21 hand feature points (see
Figure 6a), and the location coordinates of the feature points are output. The identified feature points are denoted as h_0, h_1, …, and h_20, from minimum to maximum. Next, a circular region of radius r2 with h_8 as the center is used as the index fingertip region, where r2 is dynamically calibrated again in each frame.
The first step combined the visible map (
Figure 6c) with Dlib and Mediapipe to extract the feature points traced to the region of interest (
Figure 6d).
As shown in the previous step, the feature points of the face and hand in the visible image were recognized using Dlib and Mdiapipe, respectively, and the position coordinate sequences of the feature points in the visible image were obtained. Then, we aimed to apply the feature points of the visible image to the corresponding positions of the infrared thermal image with high precision. The visible light camera of the binocular infrared camera used in this study was placed directly above the infrared camera; however, the relative positions of the two cameras remained unchanged. Although the positions of the visible and infrared thermal images are not the same, the difference between the two images is slight. The two images correspond to the same location in space based on image alignment. The dimensions of the visible and thermal images were (1440, 1080) and (384, and 288), respectively. To achieve a better alignment effect, the visible light image size was reduced to the same size as that of the thermal image. The relative offsets (dx and dy) in the horizontal and vertical directions of the two images were then considered for alignment. Due to the lens arrangement of the binocular thermal imaging camera used in this study, the orientation offset direction was considered only in the longitudinal direction. To ensure that the selected image offsets were valid for non-specific target distances, the results of image alignment offsets were tested at different distances from the camera. Finally, 0.5 m spacing and (14, 2) pixel image offset were determined as the optimal settings.
Precise mapping of the visible feature points onto the corresponding positions of the thermal image was achieved in Step 2. Then, automatic tracking and calibration of the region of interest were required. As the subject moved to varying degrees during the experiment, the field of view of the face and hand presented in the image changed, which caused the face and hand area in the entire image to change, thus changing the pixel distances r1 and r2 that determine the region of interest. Therefore, it was necessary to determine the field of view change rates for the two frames before and after imaging. When the field of view changes, the distance between adjacent even feature points also changes; therefore, the absolute distance ratio between two points within the current frame and the previous image was calculated using facial feature points f_27 and f_30 as the field-of-view change rate to adjust the size of the region of radii r1 and r2 to improve region localization.
The final step involved calculating the average temperature inside the localized region of interest. After localizing the thermal image region of interest, the average temperature of the region of interest was obtained by loading a CSV file of the temperature matrix extracted using the software with a thermal imaging camera.
Figure 7 shows the algorithm flow for skin temperature extraction within the region of interest based on visible image-assisted thermal images.
6. Discussion
Fatigue is a complex physiological phenomenon that affects both physical and mental activity. This study attempted to establish a biological thermal model considering CVS parameters to analyze changes in HR, skin blood flow, and blood pressure in the fatigue stage. It is found that changes in peripheral resistance and HR caused by fatigue are the primary reasons for skin temperature reduction, leading to changes in the cardiovascular system.
Previous studies have shown that the physiological differences between individuals and skin blood perfusion have a significant influence on heat transfer [
25,
26,
27]. Therefore, this paper proposes a method for establishing a personal biological thermal model based on experimental personal data that thoroughly considers differences in tissue layers and skin blood perfusion. Based on the model, some results can be obtained by varying HR and skin resistance. There is something to discuss the simulation results of skin blood perfusion and blood pressure.
For skin blood perfusion, increased peripheral resistance and reduced blood flow induce changes in skin temperature during fatigue. The reduced skin blood perfusion rates in the hands and head after fatigue were consistent with the reduced Laser Doppler blood flow signal components in the fingers identified in the experiment. This suggests that reduced blood perfusion rate after fatigue leads to a decrease in skin temperature, which is consistent with previously reported results obtained from simulated driving experiments [
28]. The skin resistance values are expected to change more significantly due to fatigue in the hand than that in the head because skin temperature changes in the hand are strongly influenced by blood perfusion. Moreover, the high oxygen consumption in the head and the stability in blood flow are crucial to maintaining oxygen consumption in the brain, so temperature fluctuations are expected to be less significant in the head.
For blood pressure, we calculated changes in HR and blood pressure after fatigue. HR decreased at fatigue onset, which is consistent with the results of other studies on simulated driving fatigue [
6,
29]. Regarding the change in blood pressure, a marked decrease in both systolic and diastolic blood pressure was observed before and after fatigue. Our results exhibited consistent trends with Jagannath et al. However, our results contradicted those reported by Fumio [
30] and Yamakoshi [
28], where the blood pressure increased in their experiments. This was caused by differences in driving environments. In Fumio’s experiment, the subjects studied were city drivers where the urban traffic environment was worse, which results in tremendous stress and can lead to nervousness or anxiety, thus potentially increasing blood pressure. In Yamakoshi’s [
28] experiment, the simulated road conditions were highway environments. However, subjects were given clear demands, such as “keep an eye on surroundings“ and asked to perform ongoing monotonous tasks under strained conditions. Therefore, the demand for prolonged attention causes stress in drivers and may increase blood pressure.
Furthermore, the blood pressure response is influenced by local conditions in the working muscles, which are peripherally regulated by a muscle chemical reflex of metabolites in the trapped muscles and remain continuously elevated as long as the occlusion persists [
31]. Therefore, during previous experiments, there is a possibility that peripheral regulation by muscle chemical reflexes may be responsible for the increase in blood pressure. However, the present experiment did not exceed 1 h in length, and the subjects were not asked to pay constant attention to their surroundings, which allowed them to reach a state of fatigue in a safe situation. Therefore, the subjects’ blood pressure was not expected to increase in response to tension.
By extension, drivers experience higher stress levels under sustained mental stress. Moreover, as mental fatigue increases, HR decreases, and HR variability (HRV) increases. Increased HRV implies that the autonomic nervous system inhibits vagus nerve activity and increases sympathetic nerve activity, which may be related to the protection of physiological coherence [
32]. In addition, during fatigue, excitatory sympathetic activity activates the renin-angiotensin system (RAS), which produces angiotensin that acts on vasoconstriction [
33], leading to the contraction of skin capillaries, a decrease in tissue blood perfusion, and a decrease in skin temperature. In future studies, wavelet analysis could be applied to analyze changes in endothelial and neural frequency segments to explore the reasons for the decrease in blood flow due to the increase in peripheral resistance. When peripheral resistance increases, blood pressure increases, and the increase in blood pressure stimulates blood pressure receptors, which can reduce HR. Not only does the blood perfusion rate in the superficial layer decrease, but the decrease in HR also leads to a decrease in blood perfusion in the core layer of the tissue. The effects of fatigue on the cardiovascular system are characterized by a decrease in HR as well as a decrease in blood pressure and a decrease in aortic root flow before and after fatigue, suggesting that HR and blood pressure influence the magnitude of blood flow to the peripheral body [
34]. Therefore, in a fatigued state, the blood supply to the brain may be reduced, and the brain requires more oxygen to function correctly. Slight hypoxia may also be a cause of fatigue in the body, and a reduction in saturated blood oxygen to the brain has been reported in previous experiments [
35].
Finally, although the experimental results of only 8 subjects are presented in this paper, a lot of work has been done to verify the model. In the early experiments, the thermocouple was used to collect skin temperature signals and obtain data from 13 subjects. Based on these results, we found that hand and head temperature decreased after fatigue, which is consistent with infrared measurements. Since the infrared thermal imager can measure temperature without contact, which is more suitable for our research purposes, we used the infrared thermal imager to capture temperature in the later experiment. And the infrared thermal imager can get the temperature information of the whole face space. In the following work, it is necessary to further expand the experimental data of the infrared thermal imager and improve the reliability of the model.