The completion times are investigated in this section. The largest time is 1168.43 s, while the smallest is 7.9 s (although it is impossible to solve 10 spatial ability problems in this small time). The mean of all test times is 200.388 s, while the dispersion is 123.279 s. Before the analysis, the distribution of completion times is investigated. To do so, the Kolmogorov-Smirnov test was used, and its results prove that the data does not follow a Gauss-distribution (p-value < 2.2 × 10−16).
To analyze these test times, the effect of display parameters and devices are examined on them: the possible factors are investigated by one in the first subsection, while they are investigated in pairs, triplets, quartets, and a quintet in the second, third, fourth, and fifth in the following subsections.
4.1. The Effect of Each Factor
In this section, each factor (camera type, its FoV, rotation, contrast ratio, a display device, and whether shadows are turned on) was investigated with regression analysis. This means that each factor was compared to the basis of the variable, its coefficient is named Intercept in each case and the level itself Intercept Level. The results of every significant comparison are presented with 95% confidence intervals (CIs) in
Figure 2. It should be noted that confidence intervals can be visualized well: it is possible to observe the effects of variables, but if the confidence interval belonging to a non-basis level variable reaches zero, then there is no significant effect of this level on the test completion related to the basis.
First, the effect of the camera type was analyzed. It has to be noted that a virtual camera has two types: orthogonal and perspective. The former uses orthographic projection, while the latter is similar to the human eye. 1291 tests were conducted with the former and 1418 with the latter. The former also was the Intercept variable with an estimate of 201.081 s. According to the results, the camera type does not significantly affect the completion times (p-value = 0.78). Therefore, the camera type was omitted from further analysis.
Next, the influence of camera FoVs was examined. This factor had 5 levels in this study: 45°, 60°, 75°, 90° and undefined. The latter also means that an orthogonal camera was used since the FoV is undefined in that case. 1049, 120, 134, 115, and 1291 tests were done with all these levels, respectively. Here, the Intercept level was 45° with an estimate of 205.586 s. As shown by the results of the regression analysis, only the 60° FoV has a significant effect (p-value = 0.00788): the times are decreased in this case, on average. Out of the remaining levels, 90° FoV was the closest to being significant (p-value = 0.10340). These insignificant factors also decreased the completion times, albeit slightly.
Afterward, the camera rotation was investigated. 7 levels existed in this case: −45°, −30°, −15°, 0°, 15°, 30°, and 45°. 294, 294, 106, 1251, 312, 313, and 139 tests were done with these levels, respectively. The Intercept level was −15° with an estimate of 236.1102 s. As can be seen, two variables have significant effects which decrease completion times: 0° (p-value < 2 × 10−16), and −45° (p-value = 0.044). This means that in most cases, significantly smaller test completion times can be reached without camera rotation.
The following variable to examine was the contrast ratio: there were 5 levels in this case: 1.5:1, 14:1, 21:1, 3:1, and 7:1. Respectively, the number of tests was 1066, 164, 191, 167, and 1121. 1.5:1 was the Intercept level with an estimate of 148.378 s. As can be seen, every level has a significant influence, and each increases the completion times. The significances are the following: 3:1 (p-value = 1.13 × 10−5), 7:1 (p-value < 2 × 10−16), 14:1 (p-value = 0.00391), and 21:1 (p-value = 4.45 × 10−7). The best case is contrast ratio 7:1. It means, that the contrast should be increased to a certain level, but if it is too strong, its effect decreases.
Next, the existence of shadows in the scene was assessed. Therefore, there were two levels: shadows are turned on (1414 tests), and shadows are turned off (1295 tests). The latter was the Intercept level with an estimate of 193.355 s. According to the results, when shadows are turned on, the completion times are significantly increased (p-value = 0.00447). It means, that the lack of shadow is better. Its existence could be confusing.
Lastly, the used display device was assessed. Similarly, there were two levels in this case: desktop display (2160 tests) and the Gear VR (549 tests). The former was the Intercept level with an estimate of 188.784 s. Due to the results of the regression analysis, it can be concluded that the use of the Gear VR significantly increased the completion times (p-value < 2 × 10−16). It means that people need more time in immersive circumstances, which can be explained by the unusual conditions.
4.2. The Effect of Factor Pairs
After the factors (variables) were analyzed one by one, the investigation continued with pairs. All possible combinations were made and were compared to each other. These combinations and their Intercept variables are the following as shown in
Table 1:
As can be seen in
Table 1, the pair of 45° FoV & −15° camera rotation has the largest estimate of seconds, while the pair of desktop display & 1.5:1 contrast ratio and the desktop display have the smallest. If the pairs are looked at, it can also be suspected that when the −15° camera rotations are paired with another factor, the estimates become quite large. These estimates are the largest ones. Similarly, when the 1.5:1 contrast ratio is paired with another factor, the estimates decrease. These estimates are the smallest ones.
The significant pairs resulting from the comparison can be seen in
Figure 3 in the form of 95% CIs. From this point onward, the following new abbreviations are used: ROT for camera rotation, CR for contrast ratio, DD for desktop display, GVR for the Gear VR, SH ON for turned on shadows, and lastly, SH OFF for turned off shadows. As illustrated in
Figure 3, there are 75 pairs with significant decreases and increases in completion times. 23 pairs decrease test completion times significantly, while 52 increase them. The largest significant decrease is in the case of 75° FoV and −45° camera rotation, while the largest significant increase is in the case of the Gear VR and 7:1 contrast ratio.
Next, the interactions between all pairs of factors were assessed. To conserve space, only the significant interactions are shown in
Table 2. Inside the “Estimate” column, the estimates of the Intercept variables are presented between brackets, respectively.
The data presented in
Table 2 shows that the pair of FoV and camera rotation has the largest number of significant interactions: there are seven of them. Contrarily, the pairs of FoV & Display device; Camera rotation & Display device; Camera rotation & Contrast ratio; and lastly, Contrast ratio & Display device have only one significant interaction in them. The largest increase can be found in the case of 75° FoV and 0° camera rotation, while the largest decrease exists in the case of 90° FoV & 7:1 Contrast ratio with an estimate of 149.376 s. It can also be observed in
Table 2 that when the camera rotation is paired with an undefined FoV, the estimates of interactions significantly decrease. A similar phenomenon happens when it is paired with the Gear VR.
Also, pairs that do not have interactions exist in the model. These are the following: FoV & Shadows; Camera rotation & Shadows; Display device & Shadows; and lastly, Contrast ratio & Shadows. This means that every pair that has the factor of shadows in them, do not have significant interactions.
4.3. The Effect of Factor Triplets
The next step was to analyze the triplets of factors. Similar to before, all possible combinations of factors were created. These combinations and their Intercept levels are the following as shown in
Table 3, while the significant pairs resulting from the comparison can be seen in
Figure 4 in the form of 95% CIs.
As can be seen in
Table 3, when the contrast ratio of 1.5:1 is present in the intercept levels, the estimates become smaller. Usually, when either the desktop display or the turned off shadows factor is in the intercept levels, the estimates also become smaller. However, when either the 45° FoV or the −15° camera rotation is beside them, the estimate increases. Therefore, it can be suspected from
Table 3, that the 45° FoV and −15° camera rotation can increase completion times when they are in a triplet.
According to the results, there are 179 significant differences when triplets are involved. There are 23 significant decreases in completion times, while the remaining 156 differences are significant increases. The largest significant decrease with an estimate of −132.228 s is when 75° FoV and −45° camera rotation is used with turned on shadows (p-value = 0.029173), while the largest significant increase with an estimate of 420.88 s on average is when 75° FoV, 45° camera rotation and 3:1 contrast ratio are used (p-value = 0.000768).
Afterward, the interaction among triplets was examined. Before assessing each triplet, the ANOVA variance analysis was used to identify which model is optimal out of three: regression models with only additive properties (I), regression models in which the interaction of pairs is allowed (II), and regression models in which the interaction of triplets is allowed (III). The results of this analysis are presented in
Table 4 and the optimal models are also shown in it.
As can be seen in
Table 4, when comparing model I and model II, 8 significant differences exist among them. This means that model II proved to be superior in 8 cases. In the remaining ones, the optimal model is the additive model (I) as there was no significant interaction found among the factors. When comparing model I and model III, 7 significant differences were found, but when comparing model II and model III, the latter did not prove to be better than the former.
Ultimately, model III is not better than model II. Therefore, this means that in 8 cases, model II is the most appropriate of all three models. Model I should be used in the case of the remaining 2 cases. In the end, according to the optimal models, there are 45 significant interactions among pairs of factors.
4.4. The Effect of More than Three Factors
In this section, the effect of more than three factors is analyzed. It should be noted that due to a large number of quartets and quintets, many of them contain small (≤10) sample sizes due to the used randomization technique as was mentioned in
Section 3. Therefore, the results presented in this subsection should be interpreted with caution.
As before, the models were compared to each other to find the optimal one. This means that during the investigation, every model (regression models with only additive properties (I), regression models in which the interaction of pairs is allowed (II), regression models in which the interaction of triplets is allowed (III), regression models in with the interaction of quartets are allowed (IV), and regression models in which the interaction of quintets are allowed (V)) was compared to each other with the ANOVA variance analysis.
Naturally, there were four models in the case of quartets and five in the case of quintets. In both cases, model II proved to be the best. Even if the interactions of at least variable triplets are permitted, no model is better than II.
4.4.1. The Effect of Factor Quartets
When investigating the effect of factor quartets all possible (864) combinations are made of them. Afterward, they were compared to each other. These combinations and their Intercept variables are the following as presented in
Table 5.
According to the results, 230 significant quartets were found out of all possible 864 quartets. This means that 26.62% of the combinations are significant. However, only 6 combinations decrease the completion times significantly, while the remaining ones increase it. The greatest significant decrease is when 60° FoV, 45° camera rotation, a desktop display is used without shadows.
4.4.2. The Effect of Factor Quintets
Lastly, the effect of factor quintets was investigated. As there were five factors, in the end, only one quintet could be examined: Camera FoV & Camera rotation & Display device & Contrast ratio & Shadows. Similarly, regression analysis was used for the investigation. The Intercept variable was 45° FoV, −15° ROT, DD, 1.5:1 CR, and SH OFF with an estimate of 69.780 s on average and a standard error of 54.784.
According to the results, 110 significant differences exist among quintets. It can also be seen that there are no significant decreases in completion times. The largest significant increase is when the quintet is 75° FoV, 45° ROT, DD, 3:1 CR, and SH ON. In this case, the increase is 420.880 s on average, which is approximately 7 min. If the whole CI is looked at, the increase in completion times can reach approximately 10.5 min. Contrarily, the smallest significant increase is when the quintet is 45° FoV, 0° ROT, GVR, 1.5:1 CR, and SH ON. The increase is 112.517 s on average which is almost 2 min. While not significant, there are 11 quintets which could decrease the completion times. These can be seen in
Figure 5.
While the quintets in
Figure 5 can decrease completion times, they can also slightly increase them. As can be seen, most of these completion times are received using a desktop display, while there is only one with the use of the Gear VR.
Regarding significant interactions, 14 exist of them. Significant ones only occur among the pairs of FoV & ROT; FoV & CR; and ROT & CR. The most significant interaction is in case of undefined FoV & 30° camera rotation (p-value = 6.01 × 10−5).