Change of Blink Rate in Viewing Virtual Reality with HMD

: The eye blink rate, a major human physiological response, directly affects ocular diseases, such as keratitis and dry eye syndrome. It has been shown that the eye blink rate in normal eyes has a certain frequency for individuals, from 6–30 times/min. It was suggested in a previous study that the eye blink rate can be decreased during the viewing of high-intensity and realistic content. Therefore, in this paper, we examine the change of the eye blink rate during the HMD (head-mounted display) viewing of VR (virtual reality) contents; accordingly, we propose an algorithm to measure the eye blink rate as well as compare and analyze this rate in three different environments (natural, monitor, and HMD). We conﬁrmed that IPD (interpupillary distance) and phoria affected the eye blink rate in each environment. In this experiment, 21 subjects (28.38 ± 6.87 years) were selected, and a paired t -test was performed for changes in the eye blink rate over 1 min for each environment. The IPD and phoria effects on the eye blink rate were conﬁrmed using the Spearman’s correlation coefﬁcient. In this experiment, the eye blink rate was decreased in the monitor and HMD environments compared with the natural environment, while that in the HMD environment was decreased compared with the monitor environment. The results of the correlation analysis of far IPD and the eye blink rate show no statistical signiﬁcance or correlation. The correlation analysis of near IPD and the eye blink rate showed a strong positive correlation of the eye blink rate in the monitor environment. The correlation analysis of distance phoria and the eye blink rate showed a strong negative correlation of the eye blink rate in the HMD environment. The correlation analysis of near-ﬁeld phoria and the eye blink rate showed a strong negative correlation of the eye blink rate in the HMD environment. It is expected that the results of this study will be used as a VR-viewing recommendation.


Introduction
Humans accept information from the external environment through various sensory organs. Among them, vision is higher than the other senses and plays an important role in the support of the other senses. Human beings' acquisition of external information frequently occurs via eye transmission [1,2].
Eye blinking, one of the main human physiological reactions, is caused by the interaction between the levator palpebrae superioris muscle, which lifts the palpebral; and the orbicularis oculi muscle, which closes the eyelid. When the normally activated levator palpebrae superioris muscle is deactivated, and the orbicularis oculi muscle contracts, the eye closes. When the contraction of the orbicularis oculi muscle ceases and the levator palpebrae superioris muscle contracts, a mechanism that opens the eye results in eye blinking [3][4][5].
In the normal vision of individuals, spontaneous eye blinking occurs at a certain frequency. The influential factors affecting eye blinking are eyelid condition, eye condition, disease status, contact-lens presence, psychological state, surrounding environment, drugs, and other stimuli. The blinking frequency is from 6-30 times/min [6,7].
The eye blink response directly affects ophthalmic diseases, such as keratitis and dry eye [8][9][10]. In addition to the ophthalmic factors, the tear film on the surface of the eyeball is evenly dispersed to remove corneal irregularities and prevent corneal dehydration and conjunctiva. Furthermore, the eye blink rate is decreased during the viewing of realistic content; therefore, it is necessary to study the relationship between the viewing of such realistic media and eye blink patterns.
In general, the eye blink rate tends to decrease during work rather than at rest [11,12]. In addition, the eye blink rate tends to decrease during reading or demanding tasks, as these are more visually strict than the typical visual activities [13]. The higher the focal degree regarding an object, the lower the eye blink rate [14,15]. An increase in the interest induction from media viewing can result in a decrease in the eye blink rate, and this can cause ocular pathological effects.
In the study of Sheedy et al., ocular dryness is suggested as one of the external symptoms of stable fatigue [16], and xerophthalmia can be induced by a reduced eye blink rate [8]. Ocular dryness is accompanied by visual disturbances, inflammation of the ocular surface, and ocular discomfort [17], thereby causing symptoms such as irritation, burning sensation, and pink eye (also called conjunctivitis) [9]. This paper proposes an algorithm for the measurement of the eye blink rate, and the authors also studied the effects of each of the realistic media environments on the eye blink pattern in the following three environments: natural, monitor, and head-mounted display (HMD).

Participants
The subjects of the experiment possessed a near-far correctional visual acuity of 0.8 or greater, had not been diagnosed with any special ophthalmologic, systemic, or mental illnesses, and were not inhibited by the Worth four-dot test. Twenty-one subjects in the age segments of the 20 s and 30 s (28.38 ± 6.87 years old) were selected after they demonstrated that they understood and agreed with the experiment contents.

Detection for Eye Blink
Natural and Monitor: The measurement of the eye blink rate can be evaluated using visual observation after the completion of a video recording. In this case, however, there is a probability that mistakes will occur when measuring eye blink rate in visual observation. Therefore, in this paper, we propose an algorithm for efficient experiments in the natural and monitor environments, as shown in Figure 1. •

Face Detection
The Haar cascade classifier [18] and optical flow algorithms were applied for face detection and face tracking, respectively [19]. Figure 2 shows the Haar features for face detection. The image frame with templates of different sizes and orientations is convolved for the Haar feature computation. Local movement X = local movement X + f_xy × x, where local movement X and local movement Y are the local changes in the x coordinate and the y coordinate for slight facial movements, respectively; x and y are the actual changes that lead to changes in local movement; and the global movements of X and Y are achieved using the opticaltracking method. •

Eye-Region Extraction
In this step, the ocular region of the facial image is identified on the basis of certain geometrical dependencies and then divided into these boxes according to the traditional proportion rules [20], as

• Face Detection
The Haar cascade classifier [18] and optical flow algorithms were applied for face detection and face tracking, respectively [19]. Figure 2 shows the Haar features for face detection. The image frame with templates of different sizes and orientations is convolved for the Haar feature computation.
Local movement X = local movement X + f_xy × x, Local movement Y = local movement Y + f_xy × y, where local movement X and local movement Y are the local changes in the x coordinate and the y coordinate for slight facial movements, respectively; x and y are the actual changes that lead to changes in local movement; and the global movements of X and Y are achieved using the optical-tracking method. •

Face Detection
The Haar cascade classifier [18] and optical flow algorithms were applied for face detection and face tracking, respectively [19]. Figure 2 shows the Haar features for face detection. The image frame with templates of different sizes and orientations is convolved for the Haar feature computation. Local movement X = local movement X + f_xy × x, Local movement Y = local movement Y + f_xy × y, where local movement X and local movement Y are the local changes in the x coordinate and the y coordinate for slight facial movements, respectively; x and y are the actual changes that lead to changes in local movement; and the global movements of X and Y are achieved using the opticaltracking method. • Eye-Region Extraction In this step, the ocular region of the facial image is identified on the basis of certain geometrical dependencies and then divided into these boxes according to the traditional proportion rules [20], shown in Figure 3. The located ocular region is subsequently extracted from the face and used as a template for further eye tracking by means of template matching. •

Eye Blink Detection and Contours
Generally, an interval of 2-10 seconds occurs between each adult eye blink, and after face detection, the system will detect the eye and wait for its disappearance, followed by a calculation of the time interval between the disappearance and the next appearance of the pupil in the frame. The eye-detection algorithm only detects both of the eyes if they have been opened and closed. Using this information, it is possible to assess whether the user's eyes are closed or open and to count the number of times the user blinks; a single eye is sufficient for the detection of the eye blink rate.
HMD (head-mounted display): The accuracy of our proposed algorithm for HMD is much more efficient than the traditional algorithm [21][22][23]. For HMD, the FOVE HMD infrared camera (FOVE, Inc., San Mateo, CA, USA) is used. There are various applications of a histogram [24], and our method uses a histogram for eye blink detection using HMD. The flow chart of the proposed algorithm in HMD is shown in Figure 4. The proposed algorithm works according to a series of steps for HMD environments, which are as follows:

•
Frame Capture and Histogram Analysis

• Eye Blink Detection and Contours
Generally, an interval of 2 -10 seconds occurs between each adult eye blink, and after face detection, the system will detect the eye and wait for its disappearance, followed by a calculation of the time interval between the disappearance and the next appearance of the pupil in the frame. The eye-detection algorithm only detects both of the eyes if they have been opened and closed. Using this information, it is possible to assess whether the user's eyes are closed or open and to count the number of times the user blinks; a single eye is sufficient for the detection of the eye blink rate.
HMD (head-mounted display): The accuracy of our proposed algorithm for HMD is much more efficient than the traditional algorithm [21][22][23]. For HMD, the FOVE HMD infrared camera (FOVE, Inc., San Mateo, CA, USA) is used. There are various applications of a histogram [24], and our method uses a histogram for eye blink detection using HMD. The flow chart of the proposed algorithm in HMD is shown in Figure 4. The proposed algorithm works according to a series of steps for HMD environments, which are as follows: shown in Figure 3. The located ocular region is subsequently extracted from the face and used as a template for further eye tracking by means of template matching. •

Eye Blink Detection and Contours
Generally, an interval of 2-10 seconds occurs between each adult eye blink, and after face detection, the system will detect the eye and wait for its disappearance, followed by a calculation of the time interval between the disappearance and the next appearance of the pupil in the frame. The eye-detection algorithm only detects both of the eyes if they have been opened and closed. Using this information, it is possible to assess whether the user's eyes are closed or open and to count the number of times the user blinks; a single eye is sufficient for the detection of the eye blink rate.
HMD (head-mounted display): The accuracy of our proposed algorithm for HMD is much more efficient than the traditional algorithm [21][22][23]. For HMD, the FOVE HMD infrared camera (FOVE, Inc., San Mateo, CA, USA) is used. There are various applications of a histogram [24], and our method uses a histogram for eye blink detection using HMD. The flow chart of the proposed algorithm in HMD is shown in Figure 4. The proposed algorithm works according to a series of steps for HMD environments, which are as follows:  • Frame Capture and Histogram Analysis First, the recorded video was captured by HMD, and image frame of HMD was converted into gray scale using Visual Studio C++. Next, histogram analysis was performed on the opened eye and closed eye image frames, as shown in Figure 5. We compared the open eye histogram in Figure 5a with the closed eye histogram in Figure 5c and observed that the number of pixels at higher pixel values were greater in the closed eye than in opened eye. This is because the black pupil always lowers the number of pixels at higher pixel levels.
First, the recorded video was captured by HMD, and image frame of HMD was converted into gray scale using Visual Studio C++. Next, histogram analysis was performed on the opened eye and closed eye image frames, as shown in Figure 5. We compared the open eye histogram in Figure 5a with the closed eye histogram in Figure 5c and observed that the number of pixels at higher pixel values were greater in the closed eye than in opened eye. This is because the black pupil always lowers the number of pixels at higher pixel levels. •

Eye Blink Detection and Counter
We adjusted the threshold value between the histogram of the closed eye and that of the open eye. If the value exceeds the threshold, a blink was detected, and if the pixel was lower than the threshold, the same process was repeated. Using this information, it is possible to assess whether the user's eyes are closed or open. Finally, after successful blink detection, the number of blink detections was counted.

Experiment Environment
Before the participation of the subjects in the experiment, the subject factors that could affect the experiment were checked in a preliminary interview. The experiment conditions were measured in the natural, monitor, and HMD viewing environments, and the experiment order was set randomly to prevent the order effects. Before the measuring of the eye blink rate, the subjects' far and near interpupillary distances (IPD) and near phoria were measured. The IPDs were measured using the BRT-II pupillary distance (PD) meter, and the phoria was measured using the Howell phoria card (Bernell Corporation, Mishawaka, IN, USA). The eye blink rate was measured for a total of 2 min for stabilization and then for 1 min, except for 30 s before and afterward. For measurement accuracy, the

• Eye Blink Detection and Counter
We adjusted the threshold value between the histogram of the closed eye and that of the open eye. If the value exceeds the threshold, a blink was detected, and if the pixel was lower than the threshold, the same process was repeated. Using this information, it is possible to assess whether the user's eyes are closed or open. Finally, after successful blink detection, the number of blink detections was counted.

Experiment Environment
Before the participation of the subjects in the experiment, the subject factors that could affect the experiment were checked in a preliminary interview. The experiment conditions were measured in the natural, monitor, and HMD viewing environments, and the experiment order was set randomly to prevent the order effects. Before the measuring of the eye blink rate, the subjects' far and near interpupillary distances (IPD) and near phoria were measured. The IPDs were measured using the BRT-II pupillary distance (PD) meter, and the phoria was measured using the Howell phoria card (Bernell Corporation, Mishawaka, IN, USA). The eye blink rate was measured for a total of 2 min for Symmetry 2018, 10, 400 6 of 15 stabilization and then for 1 min, except for 30 s before and afterward. For measurement accuracy, the subject was not informed about the measurement of the eye blink rate. The eye blink rate was evaluated using the proposed algorithm.
The eye blink rate in the natural state was measured using a TD20 camera (Sony Corporation, Tokyo, Japan) in a situation where the subject did not recognize the measurement. The measurement of the eye blink rate in the viewing monitor was performed using 24-in Ultron 2457 Ultra monitor (Hansung, Seoul, South Korea) The experiment was conducted at a viewing distance of 1.0 m. The eye blink rate in the viewing HMD was measured using the FOVE0 HMD (FOVE, Inc., San Mateo, CA, USA). The use of the FOVE0 HMD in the experimental environment is shown in Figure 6. In this experiment, we used a VR demo game, called "Grandmother's Doll", provided by Fove. This game was used for both the monitor and HMD. subject was not informed about the measurement of the eye blink rate. The eye blink rate was evaluated using the proposed algorithm. The eye blink rate in the natural state was measured using a TD20 camera (Sony Corporation, Tokyo, Japan) in a situation where the subject did not recognize the measurement. The measurement of the eye blink rate in the viewing monitor was performed using 24-in Ultron 2457 Ultra monitor (Hansung, Seoul, South Korea) The experiment was conducted at a viewing distance of 1.0 m. The eye blink rate in the viewing HMD was measured using the FOVE0 HMD (FOVE, Inc., San Mateo, CA, USA). The use of the FOVE0 HMD in the experimental environment is shown in Figure 6. In this experiment, we used a VR demo game, called "Grandmother's Doll", provided by Fove. This game was used for both the monitor and HMD.

Data Analysis
The data analysis was performed using the Wilcoxon signed-ranked test and a simple correlation analysis (Spearman's correlation coefficient) of the SPSS software, ver. 18.0 for Windows (SPSS, Chicago, IL, USA). In this analysis, the 95% confidence interval was considered to have a statistical significance of p < 0.05.

Comparison of the Blink Rate in the Natural State and with the Viewing Monitor
Table 1 and Figure 7 shows a comparison of the measured eye blink rate per minute in the natural state and with the viewing monitor. The eye blink rate was comparatively decreased (Z = −3.218, p = 0.001) at a statistically significant level during the monitor viewing.

Data Analysis
The data analysis was performed using the Wilcoxon signed-ranked test and a simple correlation analysis (Spearman's correlation coefficient) of the SPSS software, ver. 18.0 for Windows (SPSS, Chicago, IL, USA). In this analysis, the 95% confidence interval was considered to have a statistical significance of p < 0.05. Table 1 and Figure 7 shows a comparison of the measured eye blink rate per minute in the natural state and with the viewing monitor. The eye blink rate was comparatively decreased (Z = −3.218, p = 0.001) at a statistically significant level during the monitor viewing.   Table 2 and Figure 8 shows a comparison of the measured eye blink rate per minute in the natural state and with the viewing HMD. The eye blink rate was comparatively decreased (Z = −3.984, p < 0.001) at a statistically significant level during the HMD viewing.   Table 3 and Figure 9 shows the measurement results of the eye blink rate per minute with the viewing monitor and the viewing HMD. The comparison of the results shows that the eye blink rate decreased at a statistically significant level (Z = −2.214, p = 0.027) during the HMD viewing.  Table 2 and Figure 8 shows a comparison of the measured eye blink rate per minute in the natural state and with the viewing HMD. The eye blink rate was comparatively decreased (Z = −3.984, p < 0.001) at a statistically significant level during the HMD viewing.   Table 2 and Figure 8 shows a comparison of the measured eye blink rate per minute in the natural state and with the viewing HMD. The eye blink rate was comparatively decreased (Z = −3.984, p < 0.001) at a statistically significant level during the HMD viewing.   Table 3 and Figure 9 shows the measurement results of the eye blink rate per minute with the viewing monitor and the viewing HMD. The comparison of the results shows that the eye blink rate decreased at a statistically significant level (Z = −2.214, p = 0.027) during the HMD viewing.  Table 3 and Figure 9 shows the measurement results of the eye blink rate per minute with the viewing monitor and the viewing HMD. The comparison of the results shows that the eye blink rate decreased at a statistically significant level (Z = −2.214, p = 0.027) during the HMD viewing.  Wilcoxon signed-ranked test.

Analysis of the Correlation between Far IPD and the Eye Blink Rate
The results of the analysis of the correlation between far IPD and the eye blink rate in Table 4 and Figure 10 show the correlation between remote IPD and the eye blink rate in each environment.

Analysis of the Correlation between Far IPD and the Eye Blink Rate
The results of the analysis of the correlation between far IPD and the eye blink rate in Table 4 and Figure 10 show the correlation between remote IPD and the eye blink rate in each environment. The correlation results of far IPD and the eye blink rate are as follows: A weak positive correlation (0.3 ≥ r ≥ 0.1) in the natural environment, a strong positive correlation (0.7 ≥ r ≥ 0.3) in the monitor environment, and a strong positive correlation (0.7 ≥ r ≥ 0.3) in the HMD environment. However, there is no statistical significance (r = 0.193, 0.393, and 0.319; p = 0.402, 0.078, and 0.159). Table 4. Analysis of the correlation between far IPD (interpupillary distance) and the eye blink rate (N = 21).

Variable
Far  Table 5 and Figure 11 shows the correlation between near IPD and the eye blink rate in each environment. The correlation analysis results of the near IPD and the eye blink rate are as follows: a weak positive correlation (0.3 ≥ r ≥ 0.1) in the natural environment, a strong positive correlation (0.7 ≥ r ≥ 0.3) in the monitor environment, and a strong positive correlation (0.7 ≥ r ≥ 0.3) in the HMD environment. The statistical significance in the blink rate was identified (r = 0.280, 0.526, and 0.307; p = 0.220, 0.014, and 0.176).   Table 5 and Figure 11 shows the correlation between near IPD and the eye blink rate in each environment. The correlation analysis results of the near IPD and the eye blink rate are as follows: a weak positive correlation (0.3 ≥ r ≥ 0.1) in the natural environment, a strong positive correlation (0.7 ≥ r ≥ 0.3) in the monitor environment, and a strong positive correlation (0.7 ≥ r ≥ 0.3) in the HMD environment. The statistical significance in the blink rate was identified (r = 0.280, 0.526, and 0.307; p = 0.220, 0.014, and 0.176).  3.1.6. Analysis of the Correlation between Far Phoria and the Eye Blink Rate Table 6 and Figure 12 shows the correlation between far phoria and the eye blink rate in each environment. The correlation analysis results of far phoria and the eye blink rate are as follows: no correlation in the natural environment (0.1 ≥ r ≥ −0.1), a strong negative correlation in the monitor environment (−0.3 ≥ r ≥ −0.7), and a strong negative correlation (−0.3 ≥ r ≥ −0.7) in the HMD environment. Among them, the HMD blink rate is statistically significant (r = −0.011, −0.302, and −0.470; p = 0.964, 0.183, and 0.031).   3.1.6. Analysis of the Correlation between Far Phoria and the Eye Blink Rate Table 6 and Figure 12 shows the correlation between far phoria and the eye blink rate in each environment. The correlation analysis results of far phoria and the eye blink rate are as follows: no correlation in the natural environment (0.1 ≥ r ≥ −0.1), a strong negative correlation in the monitor environment (−0.3 ≥ r ≥ −0.7), and a strong negative correlation (−0.3 ≥ r ≥ −0.7) in the HMD environment. Among them, the HMD blink rate is statistically significant (r = −0.011, −0.302, and −0.470; p = 0.964, 0.183, and 0.031).   Table 7 and Figure 13 shows the correlation between near phoria and the eye blink rate in each environment. The correlation analysis results of the near phoria and the eye blink rate are as follows: a negative correlation (−0.1 ≥ r ≥ −0.3) in the natural environment, a weak negative correlation (−0.1 ≥ r ≥ 0.3) in the monitor environment, and a strong negative correlation (−0.3 ≥ r ≥ −0.7) in the HMD environment. Among them, the HMD blink rate showed statistical significance (r = −0.102, −0.248, and −0.493; p = 0.659, 0.278, and 0.023).    Table 7 and Figure 13 shows the correlation between near phoria and the eye blink rate in each environment. The correlation analysis results of the near phoria and the eye blink rate are as follows: a negative correlation (−0.1 ≥ r ≥ −0.3) in the natural environment, a weak negative correlation (−0.1 ≥ r ≥ 0.3) in the monitor environment, and a strong negative correlation (−0.3 ≥ r ≥ −0.7) in the HMD environment. Among them, the HMD blink rate showed statistical significance (r = −0.102, −0.248, and −0.493; p = 0.659, 0.278, and 0.023).

Discussion
The eye blink rate was decreased as the observation degree was strengthened beyond the typical natural state. Decreases in the eye blink rate require great care because they lead to ocular pathological effects. The experiment of this study showed that the eye blink rate was decreased during the monitor and HMD viewing compared with the natural state. Moreover, the comparison of the monitor and HMD conditions shows that the eye blink rate was decreased during the HMD viewing, thereby showing that the interest rates of the monitor and HMD videos were increased along with the increase of the observation degree.
The eye blink rate directly affects ocular diseases, such as keratitis and dry eye. In addition, tears are evenly distributed on the surface of the eyeball to form a tear film. This removes the irregularities of the cornea and allows the light to refract. It is also an important factor in preventing the dehydration of the cornea and conjunctiva. Because the individual's eye blink rate habit is so important in eyeball physiology, observation of the eye blink rate, which can be changed by using realistic content, should be observed as a major visual item.
The results of this study show the same tendency as previous studies; that is, the eye blink rate was decreased during activities with a higher attention intensity compared with those with the typical visual activity. In general, near-field work, such as reading, leads to visual fatigue, and the degree of fatigue varies depending on the environment; in this paper, the same tendency was observed for the eye blink rate in the HMD environment, near the physical viewing distance. It is known that the eye blink rate is affected by the cognitive load as well as visual fatigue. In Kim's study [25], regarding the change in the eye blink rate according to the reading environment, the eye blink

Discussion
The eye blink rate was decreased as the observation degree was strengthened beyond the typical natural state. Decreases in the eye blink rate require great care because they lead to ocular pathological effects. The experiment of this study showed that the eye blink rate was decreased during the monitor and HMD viewing compared with the natural state. Moreover, the comparison of the monitor and HMD conditions shows that the eye blink rate was decreased during the HMD viewing, thereby showing that the interest rates of the monitor and HMD videos were increased along with the increase of the observation degree.
The eye blink rate directly affects ocular diseases, such as keratitis and dry eye. In addition, tears are evenly distributed on the surface of the eyeball to form a tear film. This removes the irregularities of the cornea and allows the light to refract. It is also an important factor in preventing the dehydration of the cornea and conjunctiva. Because the individual's eye blink rate habit is so important in eyeball physiology, observation of the eye blink rate, which can be changed by using realistic content, should be observed as a major visual item.
The results of this study show the same tendency as previous studies; that is, the eye blink rate was decreased during activities with a higher attention intensity compared with those with the typical visual activity. In general, near-field work, such as reading, leads to visual fatigue, and the degree of fatigue varies depending on the environment; in this paper, the same tendency was observed for the eye blink rate in the HMD environment, near the physical viewing distance. It is known that the eye blink rate is affected by the cognitive load as well as visual fatigue. In Kim's study [25], regarding the change in the eye blink rate according to the reading environment, the eye blink rate was decreased