This section presents the results of the experiments, focusing on the statistically significant results (p < 0.05).
3.1. Test Time
The main descriptive statistics of the test time differentiating VI and sighted subjects are shown in
Table 3. The mean execution time of the test section was 7 min 22 s (5 min 52 s for sighted subjects and 9 min 42 s for VI subjects). The minimum test time for both types of subjects was 3 min; moreover, the maximum test time was 12 min for sighted subjects, but this was double, 24 min, for VI subjects. The mean test time needed for those subjects that used simple vibrations was 8 min 30 s, while the mean time taken when using complex vibrations was 6 min 14 s.
A two-way ANOVA (ANalysis Of VAriance) was conducted to examine the effect of the visual capacity and the type of vibration on the time taken to perform the test (
Table 4).
There was no statistically significant interaction between the effects of both parameters on the time taken (F (2.46) = 0.465, significance = 0.499 > 0.05). However, simple main effects analysis showed that the vibration type significantly affects the test time (F (1.46) = 4.672 and significance = 0.036 < 0.05). The same behavior is observed regarding the visual capacity of the subject, which also affects the recognition time (F (1.46) = 11.583, significance = 0.01 < 0.05). In fact, the visual capacity has the main influence on the test time as VI subjects need more time than sighted subjects to complete the test.
3.2. Recognition Rate
The two-way ANOVA test shows a statistically significant interaction between the effects of the visual capacity of the subjects and the type of vibration (simple or complex) on the recognition rate (
Table 5; F (2.54) = 8.397, significance = 0.006 < 0.05).
Simple main effects analysis shows that significant differences exist between subjects with different visual capacities (F (2.46) = 15.15, significance = 0.0 < 0.05) and the type of vibrations (F (2.46) = 30.96, significance = 0.0 < 0.05).
As shown in
Figure 5, the recognition rate is considerably improved when using complex and different vibration patterns for VI subjects, increasing from 0.47 to 0.83. Moreover, the recognition rate is slightly increased (0.12) for sighted subjects when using complex vibrations. Consequently, it can be deduced that the use of different and complex vibration patterns associated with the icons of applications significantly improves (0.36) the recognition rate for VI users, while it slightly increases the recognition rate of sighted subjects. This conclusion agrees with the results obtained by the one-way ANOVA test of the influence of the use of simple or complex vibrations on VI (F (1.18) = 29.641, significance = 0.0 < 0.05) and sighted (F (1.28) = 4.521, significance = 0.043 < 0.05) subjects.
The recognition rate per haptic icon as a function of the type of vibration pattern (with unique and simple vibrations compared to complex and different vibrations) and without, considering the type of subject (VI or sighted), is shown in
Figure 6.
The recognition rate increases for all haptic icons when using complex vibration patterns compared to the results obtained using simple vibrations. This is the expected behavior as complex vibrations have more information than simple vibrations. However, in spite of this predictable result, assistive applications such as TalkBack for Android do not use this advantage. The most noticeable improvements are for the Internet (reaching 1.0), Facebook and Twitter, which have single-click vibrations with different intensities and durations, as well as the icons with pulses and buzz-bump vibrations. Considering the single-click vibrations, it is expected that these will be more distinguishable from the simple vibrations (using the Facebook vibration pattern) as it is a way to differentiate them. On the other hand, the buzz-bump vibration patterns (Calendar, Line, Phone and Camera) are very different from the rest of the vibration patterns, hence, they are expected to be more discernable. Finally, the Email haptic icon also improves as the associated pulse vibration pattern is the most different from the rest of the vibration patterns in terms of duration and structure.
Figure 7 compares the recognition rate of sighted subjects using simple and complex vibration patterns. The figure shows that for most of the haptic icons, the recognition rates are slightly increased except for the Internet, Facebook and Twitter, where the increase is more noticeable, as shown in
Figure 6. The LinkedIn (buzz) and Email (pulses) haptic icons also perform much better. However, the SMS (double-click) and Google+ (ramp-up) icons slightly reduce in their recognition rates.
The same study (results presented in
Figure 7) is repeated but considering the recognition rate per haptic icon of the VI subjects as a function of the use of simple or complex vibration patterns, with the results shown in
Figure 8. It can be observed that the recognition rate is considerably increased when using complex vibration patterns, especially for the SMS, Calendar and Camera haptic icons. It is worth mentioning that the Calendar haptic icon obtained only a 0.2 recognition rate when using simple vibrations and obtained a 1.0 recognition rate (it is always recognized) when using complex vibrations. In previous work [
29], some VI subjects pointed out that they had more problems recognizing the haptic icons when they were associated with applications they did not use regularly. Consequently, it is expected that applications such as SMS (that has marginal use nowadays) or the Camera will obtain low recognition rates for VI people.
Figure 9 compares the haptic icon recognition rates of VI and sighted subjects when using complex and different vibration patterns. It can be observed that the recognition rates obtained by VI subjects are very close to or even better than those obtained by the sighted subjects for most of the haptic icons. For VI subjects, the Internet and Calendar haptic icons are fully recognized; the same behavior is observed for sighted subjects for the Internet (sharp click), Facebook (strong click), Play Store (double bump click) and Calculator (long buzz) haptic icons. Sighted people obtain better results than VI people at recognizing the Facebook (strong click), Calculator (long buzz) and LinkedIn (short buzz) haptic icons. On the other hand, VI subjects better recognize the SMS (double sharp click), Calendar (transition bump), Line (transition bounce 100) and Camera (transition bounce 33) haptic icons.
In our experiment, the haptic icons’ locations were fixed and, hence, it may be possible that the memorization of their position was related to their relative location. That is, haptic icons situated in certain locations could have been more easily remembered because of their position instead of their vibration patterns.
Figure 10 represents, using a heatmap graph, the same information shown in
Figure 9 but taking into consideration the haptic icons’ positions. The center of the figure shows the haptic icons’ locations, while the left and right graphs show the recognition rates for VI and sighted people, respectively, using complex vibration patterns. The colors represent, according to the legend shown in the right part of the figure, the recognition rate, using colors close to green for values close to 1.0 and red for values those close to 0.0. The displayed values for VI people do not show any pattern that could infer that there is a relationship between the recognition rate and the haptic icon location. Nevertheless, sighted people seem to better recognize or remember the first row of haptic icons. The Internet haptic icon is perfectly recognized by both types of users because it is the first one and it is thus easier to remember.
Finally,
Figure 11 represents the recognition rates of VI and sighted people considering the types of vibration patterns classified in
Section 2.3., that is, click, double-click, triple-click, buzz, ramp, pulse and buzz-bump vibration. There was no significant difference between the VI and sighted users’ performances. Sighted users usually obtained a better recognition rate, except for pulses and buzz-bumps, for which VI users outperformed sighted users, but these variations were insignificant. The noticeable difference was for buzz vibrations (Calculator and LinkedIn), where sighted users outperformed VI people by more than 0.20. Unfortunately, with the collected data, we cannot infer which vibration was selected when an icon was incorrectly recognized, only whether it was recognized.