The E ﬀ ect of Depth Information on Visual Complexity Perception in Three-Dimensional Textures

: Visual complexity, as an attribute of images related to human perception, has been widely studied in computer science and psychology. In conventional studies, the research objects have been limited to the traditional two-dimensional (2D) patterns or images. Therefore, if depth information is introduced into this scenario, how will it a ﬀ ect our perception of visual complexity of an image? To answer this question, we developed an experimental virtual reality system that enables control and display of three-dimensional (3D) visual stimuli. In this study, we aimed to investigate the e ﬀ ect of depth information on visual complexity perception by comparing 2D and 3D displays of the same stimuli. We scanned three textures with di ﬀ erent characteristics to create the experimental stimuli and recruited 25 participants for the experiment. The results showed that depth information signiﬁcantly increased the visual complexity perception of the texture images. Moreover, depth information had di ﬀ erent degrees of impact on visual complexity for di ﬀ erent textures. The higher the maximum depth introduced in the 3D image, the more signiﬁcant the increase in visual complexity perception. The experimental virtual reality system used in this study also provides a feasible experimental tool for future experiments.


Introduction
Visual complexity is one of the subjective impressions associated with visual information of stimuli and plays an important role in human perception and recognition of patterns and images. The study of visual complexity is significant for visual understanding and aesthetic evaluation. It is useful in many applications, such as image segmentation, content-based image retrieval, and user interface design. It has been shown that visual complexity also affects human emotions, memory, and active movement [1]. However, it is challenging to describe human sensitivity to perception of visual complexity quantitatively. The definition of visual complexity also varies according to the research field and application [2][3][4].
In the field of computer science, researchers have attempted to describe, measure, and model image complexity based on mathematical and computational approaches, such as information theory [5], spatial information [6], visual attention [7], independent component analysis [8], and machine learning [9][10][11]. Computer scientists are usually interested in the objective description of image complexity rather than the subjective perception of visual complexity. In the field of psychology, researchers are more interested in exploring attentional models and identifying factors that affect Appl. Sci. 2020, 10 visual complexity perception [12][13][14][15]. Previously, several studies have attempted to correlate the computational measures and the perception of visual complexity [16][17][18]. Quantitative analyses have been conducted to investigate the relationship between the perception of visual complexity and the factors influencing it [19,20]. The stimuli and target images used in previous studies of visual complexity perception include abstract patterns [14,21], outline images and hieroglyphs [22], texture images [23][24][25], grayscale images [26], paintings [16,19,20], webpages [27,28], and real-world images [8,13,18]. Thus, in conventional studies of visual complexity perception, the research objects have been limited to two-dimensional (2D) images. Due to the rapid development of technologies such as three-dimensional (3D) measurement, range imaging, and stereoscopic display, users now have more opportunities to appreciate 3D or RGB-depth (RGB-D) images that convey depth information. In this study, we aimed to explore how depth information affects visual complexity perception. It should be noted that the depth information we introduced in this study refers to spatial changes in depth on the continuous image surface. Structural complexity in 3D modeling lies outside the scope of this study.
Virtual reality (VR) technology has been widely used as a research tool in psychological studies of visual perception over the last few decades [29,30]. One of the advantages of VR in psychological experiments is that it can display visual stimuli with controlled parameters in 3D. The impression of depth can be reproduced by presenting binocular vision cues using a head-mounted display (HMD) [31]. VR technology also provides an immersive environment for studies on multimodal sensory interactions [32][33][34].
In this study, we constructed a VR system and applied it to investigate the visual complexity perceptions of texture images in 2D and 3D conditions. One of the reasons we chose textures as the stimuli in this study is that the properties of textures are more accessible and easier to manipulate than natural images [35]. Moreover, the visual perception of textures can be represented in a lower feature space. Three to five textural features have been commonly used to describe perceptual dimensions of textures [24,25,36,37]. We scanned three textures with different visual complexities from physical objects using photogrammetry. We investigated the relationship between visual complexity perception and depth information, and found that depth information is essential in the visual complexity perception of 3D textures.

Participants
Twenty-five college students participated in the experiment. The ages of the participants ranged from 19 to 22 years (M = 19.88, SD = ±0.67). All participants had normal or corrected-to-normal vision and none of them had participated in any similar study previously. All participants provided written informed consent before taking part in the experiment. The experiment was conducted in accordance with relevant guidelines and regulations. The study was conducted in accordance with the Declaration of Helsinki, and the study protocol was approved by the Ethics Committee of the National Institute of Technology, Gunma College (project identification code: H28-2, date of approval: 11 July 2016).

Stimuli
In this study, we selected three textures with different characteristics as the stimuli for the experiment. These textures and their characteristics were as follows:

1.
Door (a Japanese room divider consists of sheets on a lattice frame): High regularity, high directionality, low roughness, low density, high depth variation, and expected to have medium visual complexity.

2.
Dot (truncated domes): High regularity, medium directionality, low roughness, low density, low depth variation, and expected to have low visual complexity.

3.
Stone (stone wall): low regularity, medium directionality, high roughness, high density, high depth variation, and expected to have high visual complexity.
The texture images were scanned in 3D from real objects using photogrammetry (Agisoft Metashape, Version 1.5.0 build 7492, Agisoft LLC, St. Petersburg, Russia, 2018). Outliers in the scanned 3D models were manually removed. The 3D texture images were then normalized in a 100 × 100 × 100 mm bounding box while retaining their original aspect ratios. The corresponding 2D textures were obtained by parallel projection of the 3D textures. The resolution of the 2D and 3D textures was down sampled to 512 × 512 pixels.
The three texture stimuli used in the experiments are shown in Figure 1.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 3 of 9 The texture images were scanned in 3D from real objects using photogrammetry (Agisoft Metashape, Version 1.5.0 build 7492, Agisoft LLC, St. Petersburg, Russia, 2018). Outliers in the scanned 3D models were manually removed. The 3D texture images were then normalized in a 100 × 100 × 100 mm bounding box while retaining their original aspect ratios. The corresponding 2D textures were obtained by parallel projection of the 3D textures. The resolution of the 2D and 3D textures was down sampled to 512 × 512 pixels.
The three texture stimuli used in the experiments are shown in Figure 1.

Apparatus
The VR experimental environment was built using Unity and displayed on an HTC VIVE pro HMD (HTC Corporation, New Taipei City, Taiwan) (resolution: 2880 × 1600 pixels combined; field of view: 110 degrees; refresh rate: 90 Hz). The 3D rendering was performed on a personal computer (PC) with the following specifications: Intel i7-6700HQ CPU, 8 GB of RAM, and NVIDIA GeForce GTX 1070 8 GB.
The experimental system had the following functions: 1. Select and display the textures. 2. Switch between 2D and 3D textures. In our experiment, functions (3) to (7) were set to fixed values. The background color was set to RGB = (89, 89, 89). The shader was disabled in Unity. Figure 2 shows the experimental system used in this study.

Apparatus
The VR experimental environment was built using Unity and displayed on an HTC VIVE pro HMD (HTC Corporation, New Taipei City, Taiwan) (resolution: 2880 × 1600 pixels combined; field of view: 110 degrees; refresh rate: 90 Hz). The 3D rendering was performed on a personal computer (PC) with the following specifications: Intel i7-6700HQ CPU, 8 GB of RAM, and NVIDIA GeForce GTX 1070 8 GB.
The experimental system had the following functions: 1. Select and display the textures.

3.
Change the size of the textures.

4.
Change the depth of the 3D textures.

5.
Change the initial position of the textures. 6.
Change the background color. 7.
Move the observation position.
Appl. Sci. 2020, 10, 5347 4 of 9 In our experiment, functions (3) to (7) were set to fixed values. The background color was set to RGB = (89, 89, 89). The shader was disabled in Unity. Figure 2 shows the experimental system used in this study.

Procedure
The participants were seated on a chair built for the experiment and wore an HMD. Before the experiment, we orally confirmed that each participant could correctly perceive stereoscopic vision. The texture stimuli were presented in front of the participant at a distance of 1.00 m in the VR space. It subtended 53.13 degrees of visual angle horizontally and vertically. The distance was set as the most comfortable distance at which a participant could properly observe the stimuli according to the results of a preliminary experiment involving five participants. The participants who participated in the preliminary experiment did not participate in the present experiment. First, the participants were asked to view all the six textures once. Then, they were asked to observe each texture again and rate its visual complexity based on their perception, on a 7-point Likert scale ranging from 1 (very simple) to 7 (very complex). The duration of exposure of the stimuli and the participant's response time were not limited. The stimuli were presented randomly. After the experiment, the participants were asked to describe the characteristics of the stimuli which they used for determining the visual complexity for each stimulus. Multiple answers were allowed.

Statistical Analysis
In order to analyze each parameter statistically, we used two-way, repeated-measures analysis of variance (ANOVA) on depth (depth level: 2D, 3D) and texture (door, dot, stone). The main effect and interaction effect of each parameter was computed by a repeated-measure function in the IBM SPSS Statistics (V.26, IBM, Armonk, New York, USA, 2019). A post-hoc test was performed by using pairwise comparisons with Bonferroni correction. Additionally, we calculated the mean (M) and standard deviation (SD) of the data. Mauchly's sphericity test was performed to validate the results of the ANOVA. As shown in Tables 1 and 2, we considered differences to be statistically significant for p < 0.05, and highly significant for p < 0.01. The effect sizes shown in Table 2 were estimated by using Cohen's d.

Procedure
The participants were seated on a chair built for the experiment and wore an HMD. Before the experiment, we orally confirmed that each participant could correctly perceive stereoscopic vision. The texture stimuli were presented in front of the participant at a distance of 1.00 m in the VR space. It subtended 53.13 degrees of visual angle horizontally and vertically. The distance was set as the most comfortable distance at which a participant could properly observe the stimuli according to the results of a preliminary experiment involving five participants. The participants who participated in the preliminary experiment did not participate in the present experiment. First, the participants were asked to view all the six textures once. Then, they were asked to observe each texture again and rate its visual complexity based on their perception, on a 7-point Likert scale ranging from 1 (very simple) to 7 (very complex). The duration of exposure of the stimuli and the participant's response time were not limited. The stimuli were presented randomly. After the experiment, the participants were asked to describe the characteristics of the stimuli which they used for determining the visual complexity for each stimulus. Multiple answers were allowed.

Statistical Analysis
In order to analyze each parameter statistically, we used two-way, repeated-measures analysis of variance (ANOVA) on depth (depth level: 2D, 3D) and texture (door, dot, stone). The main effect and interaction effect of each parameter was computed by a repeated-measure function in the IBM SPSS Statistics (V.26, IBM, Armonk, New York, USA, 2019). A post-hoc test was performed by using pairwise comparisons with Bonferroni correction. Additionally, we calculated the mean (M) and standard deviation (SD) of the data. Mauchly's sphericity test was performed to validate the results of the ANOVA. As shown in Tables 1 and 2, we considered differences to be statistically significant for p < 0.05, and highly significant for p < 0.01. The effect sizes shown in Table 2 were estimated by using Cohen's d.   Table 3 shows the mean and standard deviation of the visual complexity score for each texture. We analyzed the relationship between texture and depth in determining visual complexity using two-way, repeated-measures ANOVA. The analysis of the differences in the perceived complexity of the textures revealed significant key impacts of depth (F(1, 24) = 22.41, p < 0.001, partial η 2 = 0.48) and texture (F(1.58, 38) = 10, p = 0.001, partial η 2 = 0.29), and a significant interaction between depth and texture (F(2, 48) = 5.13, p = 0.01, partial η 2 = 0.18) ( Table 1). This indicated that the characteristics of the texture and depth information affected visual complexity perception. Further, depth information had different degrees of impact on visual complexity perception for different textures.

Results
We indicated the pairwise comparison for interaction between the depth and texture factors, corrected using a Bonferroni adjustment. As shown in Table 2, there was a statistically significant difference between the complexity scores for each texture in 2D and 3D. The results revealed that the participants seemed to recognize the complexity in the 3D textures more (M = 3.67, SD = ±1.54) than in the 2D textures (M = 2.61, SD = ±0.94) ((t(74) = 4.732, p < 0.001, Cohen's d = 0.546)). Figure 3 shows the result of the pairwise comparison for mean complexity between 2D and 3D textures. For all textures, the mean complexity scores in 3D were significantly higher than those in 2D. partial η 2 = 0.29), and a significant interaction between depth and texture (F(2, 48) = 5.13, p = 0.01, partial η 2 = 0.18) ( Table 1).
This indicated that the characteristics of the texture and depth information affected visual complexity perception. Further, depth information had different degrees of impact on visual complexity perception for different textures.
We indicated the pairwise comparison for interaction between the depth and texture factors, corrected using a Bonferroni adjustment. As shown in Table 2, there was a statistically significant difference between the complexity scores for each texture in 2D and 3D. The results revealed that the participants seemed to recognize the complexity in the 3D textures more (M = 3.67, SD = ±1.54) than in the 2D textures (M = 2.61, SD = ±0.94) ((t(74) = 4.732, p < 0.001, Cohen's d = 0.546)). Figure 3 shows the result of the pairwise comparison for mean complexity between 2D and 3D textures. For all textures, the mean complexity scores in 3D were significantly higher than those in 2D.
In our results, for door, a significant difference (p < 0

Discussion
The experimental results revealed that both texture characteristics and depth information, had significant impacts on the evaluation of visual complexity in a VR space. Depth, roughness, threedimensionality, directionality, regularity, and understandability of the textures were the factors that

Discussion
The experimental results revealed that both texture characteristics and depth information, had significant impacts on the evaluation of visual complexity in a VR space. Depth, roughness, three-dimensionality, directionality, regularity, and understandability of the textures were the factors that most influenced the visual complexity perception, as found by analyzing the participants' responses to the questionnaire given after the experiment.
We compared the visual complexity of three sets of texture images under 2D and 3D conditions. All three sets of texture images indicated greater visual complexity by approximately one score under 3D conditions. This may have been due to the introduction of additional depth information in the 3D textures, which implied that the amount of information in a 3D texture was greater than that in its corresponding 2D texture, thereby resulting in increased visual complexity perception.
It is worth noting that the understandability of a texture has been considered to be a major factor affecting visual complexity [24,25]. Understandability represents how easily a participant can comprehend a particular stimulus. Textures with higher understandability usually have lower perceived visual complexity. When additional depth information is introduced to a 3D texture, the additional information may help the participants to comprehend the texture more easily, thereby reducing the visual complexity of the 3D texture. However, the experimental results did not support this hypothesis. This may be because of the characteristics of the texture stimuli which we selected for the experiment. The understandability of these texture stimuli was relatively high, and the additional depth information did not significantly improve the understandability. We plan to conduct more experiments on other texture stimuli in the future in order to explore this further.
The experimental results also revealed that textures with higher complexity had higher standard deviations, and 3D textures with depth information had higher standard deviations than 2D textures, as shown in Figure 3. This can also be attributed to additional depth information provided in 3D textures. The additional depth cues might have resulted in more factors that affect visual complexity, making it more difficult for participants to obtain consistent results in evaluation of complexity. Furthermore, the experimental results shown in Table 3 indicate that the mean difference between the complexity of a texture in 2D and 3D was proportional to the maximum depth of its 3D image (door: maximum depth = 21.07 mm, dot: maximum depth = 7.89 mm, stone: maximum depth = 18.01 mm). There was a significant interaction between the depth and texture in visual complexity perception. This indicates that for different textures, depth information had different effects on perception of visual complexity. In the future, we aim to perform more experiments to verify whether the interaction between depth and texture in visual complexity perception is caused by the characteristics of different textures (directionality, density, regularity, etc.) or is only related to the maximum depth of a texture.

Conclusions
In this study, we conducted an assessment of visual complexity perception for three sets of texture stimuli with different characteristics and analyzed the effect of depth information on visual complexity. The experiment was carried out in a VR space with real-world scanned texture stimuli. The results showed that depth information significantly increased visual complexity perception for the texture images. This indicates that along with the RGB information about the texture, depth information also has an essential impact on the perception of visual complexity. This study suggests that in the case of visual perception and analysis, it is necessary to consider the factor of depth information.
Virtual reality has been successfully used as an experimental tool in many psychological experiments [29,30]. The VR system and experimental protocol in this study demonstrated their potential for studies of visual complexity with 3D stimuli. In future studies, we will make full use of the advantages of virtual reality technology in 3D stimulus presentation, parameter control, and real-time interaction, and conduct visual complexity experiments on more real-world scanned or synthetic stimuli.