1. Introduction
A program-based puzzle for which a solution can be easily found by human subjects, and at the same time, hardly found by machines, is known as CAPTCHA test. The goal of the CAPTCHA is the same as in the standard Turing test—to test if the computer can simulate the human behavior. A human subject and the computer in the Turing test have to answer a set of questions. The human judge evaluates the obtained answers. If the machine can answer the questions in the same way as a human, then it is said that the machine has intelligence. In the CAPTCHA test, the evaluator of the answers is not a human, but a machine (computer). That is the reason the CAPTCHA is sometimes called a reverse Turing test.
The bots are computer programs which simulate the human behavior. There are many different algorithms which can be incorporated into the bots [
1], such as speech recognition algorithms, Optical Character Recognition (OCR) algorithms, etc. There are many types of CAPTCHA, but many of them are not in use because of a poor security level in the practical use, due to attacks made by bots.
A successful CAPTCHA must operate in the area where the human ability is stronger than the computers, such as: (i) image analysis; (ii) video processing; and (iii) puzzle solving. The most promising ones are CAPTCHAs which are based on a puzzle. Although real puzzles are based on recognizing images, the puzzle-based CAPTCHA does not include image elements. This CAPTCHA needs a longer time to be solved and it has no easy solution for the users. On the other side, finding the solution for this CAPTCHA with the bots is almost impossible.
Finding the most influencing factors on the CAPTCHA solution is very useful. Accordingly, Brodić et al. [
2] used traditional statistical analysis in terms of Mann–Whitney
U test for detecting the user’s factors affecting the Dice CAPTCHA solution time among age, gender and education level. The goal was to detect if the Dice CAPTCHA could be compliant to the“ideal” model (a solution to the CAPTCHA should be provided in short time—lower than 30 s—and the time spent to find the solution should not be influenced by personal user’s features [
3]). Brodić et al. [
4] proposed to extend this statistical analysis with new user’s factors, including the Internet experience, type of device on which the Dice CAPTCHA is solved and number of attempts for obtaining a correct solution. They explored the influence of the co-occurrence of the different user’s features on the Dice CAPTCHA solution time by association rule mining. There are different aspects which are not considered in this investigation. In particular, association rule mining only provides unsupervised analysis of this dependence, missing the aspect of predicting the solution time given the user’s factors. To overcome this limitation, Amelio et al. [
5] proposed an artificial neural network model for predicting the solution time to the Dice CAPTCHA from the user’s age, Internet experience and device type on which the Dice CAPTCHA is solved.
In this study, we extended the previous analysis on 197 subjects who use the Internet, characterized by age, education level, Internet use, and number of attempts for successfully solving the Dice CAPTCHA. The solution time was measured for the whole group of Internet users. The investigation was performed on a laptop or tablet for a given number of attempts. This work analyzed the combination of user’s features influencing the time to correctly solve the Dice CAPTCHA using: (i) association rule mining (unsupervised method); and (ii) prediction by artificial neural network (supervised method).
To summarize, the main contributions of this work vs. the literature are the following:
Differently from the authors of [
2,
4,
5], a more complete experiment was performed, involving both an unsupervised method (association rule mining) and a supervised method (artificial neural network).
A traditional statistical analysis as in [
2] makes preliminary assumptions on the data. By contrast, the association rule mining does not need any initial assumption on the data, and is able to capture dependences of multiple user’s factors on the Dice CAPTCHA solution time.
The set of the adopted user’s features is different from the set in [
2]. It includes age, education level, Internet use, device type on which Dice CAPTCHA is solved and number of attempts for obtaining a correct solution. Gender is omitted since it has no influence in both association rule mining and artificial neural network analysis.
Differently from Amelio [
5], the artificial neural network model was extended with the number of attempts for successfully solving the Dice CAPTCHA as a new input parameter. It brings new results completing the analysis in [
5].
The rest of the paper has the following organization.
Section 2 makes an overview of the related works, while
Section 3 describes the basics of the Dice CAPTCHA. The experimental part is given in
Section 4 as well as the explanation of the association rule mining and artificial neural network. The results of the investigations together with the discussion are given in
Section 5 and
Section 6, respectively. Finally, the conclusions and guidelines for the future work are presented in
Section 7.
2. Related Work
Different works on the usability of the CAPTCHA can be found in the literature. Singh and Pal [
6] investigate the drawbacks of different types of CAPTCHA. In particular, text-based CAPTCHAs are usually hard to solve because it is difficult to correctly identify the characters. The users have problems in solving image-based CAPTCHAs when their vision is impaired, or when the images presented are blurred. Audio-based CAPTCHAs are usually presented in English language, which is a limitation for non-native English speakers or people who do not comprehend English, while for the video-based CAPTCHAs, the users have issues with downloading and finding the correct CAPTCHA. In the end, the CAPTCHAs based on puzzles are more difficult to be solved since usually the solution time is longer, and the user needs to correctly identify the solution to the puzzle.
Fidas et al. [
7] investigated users’ perceptions, preferences and usage of the CAPTCHA. The authors used a survey to collect responses, and concluded that the CAPTCHAs are hard to be solved by humans. From 210 collected surveys, the authors concluded that every other participant needs more than one try to solve the CAPTCHA. Moreover, the background patterns are identified as the main barrier when solving the CAPTCHA.
In [
8], usability and usability issues of the CAPTCHA design were investigated. The authors proposed a framework for investigating the usability of the CAPTCHA, consisting of three dimensions: (1) distortion; (2) content; and (3) presentation. Based on this framework, the following usability issues were identified. First, foreigners have some difficulty to find a solution to CAPTCHAs based on text due to the language barrier. Second, the use of the color in a CAPTCHA affects both its usability and security. Lastly, the ability to predict the CAPTCHA sequence may have serious implications on the usability of the CAPTCHA.
Beheshti and Liatsis [
9] used a survey which consisted of 13 questions to evaluate the users’ experience and performance when solving the reCAPTCHA. Users’ age, gender, vision impairment, and monitor type were considered in the analysis. Their results showed that, from 100 participants, 61% solved the reCAPTCHA in one try, while 28% of the users solved the reCAPTCHA in two attempts, and the rest of the users needed three attempts to correctly solve the CAPTCHA. Moreover, most of the users solved the reCAPTCHA in less than 5 s, while only 5% of them needed more than 10 s to solve it. The results also showed that a high character distortion leads to a longer solution time. In addition, most of the participants evaluated the ambiguity level of the CAPTCHA characters as moderately clear, moderately unclear, and very unclear.
In [
10], the Dynamic Cognitive Game (DCG) CAPTCHA was evaluated from a perspective of usability and security. The gender, age, and education of the participants were taken into account when the authors performed the analysis of the solution time, user experience, and success rate of solving the CAPTCHA, but no meaningful relation was found. The results show that this type of CAPTCHA remains secure in terms of completely automated attacks.
In addition, Conti et al. [
11] introduced a new image-based CAPTCHA called CAPTCHaStar!, based on the identification of different shapes in a confused environment. A usability analysis involving a population of 281 users was performed on the proposed CAPTCHA in terms of success rate and solution time. The obtained results prove that CAPTCHaStar! has a higher than 90% success rate.
The first large scale assessment of the CAPTCHA test was provided in [
12] for evaluating the difficulty level of solving different types of CAPTCHA. The analysis involved more than 318,000 CAPTCHA tests of 21 different types, including 13 image-based and 8 audio-based CAPTCHAs. The obtained results show that humans have difficulties in solving the CAPTCHA test, in particular the audio-based CAPTCHA. In addition, for non-native English speakers, the solution to English-based CAPTCHA types can be slower and less accurate.
Brodić et al. [
13] investigated the influence of the CAPTCHA based on image and text on the users’ solution time, based on their age, gender, level of education, and Internet experience. The obtained results prove that younger users solve the CAPTCHA faster, while no statistically significant differences in solution time were found between male and female users. Moreover, users with a level of higher education are faster in solving the CAPTCHA. Lastly, this research showed that users with a higher Internet experience solve the CAPTCHA slightly more quickly than users with less Internet experience. Brodić et al. [
2] investigated the aspects of usability in the Dice CAPTCHA solved on a laptop and tablet using traditional statistical analysis (Mann–Whitney
U test). Specifically, the analysis explored the user’s factors influencing the Dice CAPTCHA solution time. The authors concluded that the Dice CAPTCHA can be considered as very close to an “ideal” test, i.e. the CAPTCHA does not depend on the user’s age, education and gender, and can be solved in less than 30 s [
3]. The same authors [
4] extended the previous analysis using association rule mining, which explored the dependence of co-occurrence of the user’s factors on the Dice CAPTCHA solution time. Finally, Amelio et al. [
5] analyzed the prediction ability of the user’s factors on the Dice CAPTCHA solution time using an artificial neural network model. Both works [
4,
5] investigated which Dice CAPTCHA type (among the analyzed ones) is closer to the “ideal” model.
Author Contributions
Conceptualization, A.A. and D.T.; methodology, A.A., D.T., I.R.D., and R.J.; and validation, A.A. and R.J.
Funding
This work was supported by the Mathematical Institute of the Serbian Academy of Sciences and Arts (Project III44006).
Acknowledgments
The authors are fully grateful to the voluntary participants for anonymously providing their data. This paper is dedicated to Darko Brodić with full gratitude.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
References
- Ye, G.; Tang, Z.; Fang, D.; Zhu, Z.; Feng, Y.; Xu, P.; Chen, X.; Wang, Z. Yet Another Text Captcha Solver: A Generative Adversarial Network Based Approach. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS ’18, Toronto, ON, Canada, 15–19 October 2018; pp. 332–348. [Google Scholar]
- Brodić, D.; Amelio, A.; Draganov, I.R. Statistical Analysis of Dice CAPTCHA Usability. arXiv 2017, arXiv:1706.10177. [Google Scholar]
- Von Ahn, L.; Blum, M.; Hopper, N.; Langford, J. CAPTCHA: Using hard AI problems for security. In Advances in Cryptology—EUROCRYPT 2003, Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques, Warsaw, Poland, 4–8 May 2003; Biham, E., Ed.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2003; Volume 2656, pp. 294–311. [Google Scholar]
- Brodić, D.; Amelio, A.; Draganov, I.R.; Janković, R. Exploring the Usability of the Dice CAPTCHA by Advanced Statistical Analysis. In Artificial Intelligence: Methodology, Systems, and Applications, Proceedings of the 18th International Conference AIMSA 2018, Varna, Bulgaria, 12–14 September 2018; Agre, G., van Genabith, J., Declerck, T., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2018; Volume 11089, pp. 152–162. [Google Scholar]
- Amelio, A.; Janković, R.; Tanikić, D.; Draganov, I.R. Predicting the Usability of the Dice CAPTCHA via Artificial Neural Network. In Digital Libraries: Supporting Open Science, Proceedings of the 15th Italian Research Conference on Digital Libraries, Pisa, Italy, 31 January–1 February 2019; Manghi, P., Candela, L., Silvello, G., Eds.; Communications in Computer and Information Science; Springer: Berlin/Heidelberg, Germany, 2019; Volume 988, pp. 44–58. [Google Scholar]
- Singh, V.P.; Pal, P. Survey of different types of CAPTCHA. Int. J. Comput. Sci. Inf. Technol. 2014, 5, 2242–2245. [Google Scholar]
- Fidas, C.A.; Voyiatzis, A.G.; Avouris, N.M. On the necessity of user-friendly CAPTCHA. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, 7–12 May 2011; pp. 2623–2626. [Google Scholar]
- Yan, J.; El Ahmad, A.S. Usability of CAPTCHAs or usability issues in CAPTCHA design. In Proceedings of the 4th Symposium on Usable Privacy and Security, Pittsburgh, PA, USA, 23–25 July 2008; pp. 44–52. [Google Scholar]
- Beheshti, S.M.R.S.; Liatsis, P. CAPTCHA Usability and Performance, How to Measure the Usability Level of Human Interactive Applications Quantitatively and Qualitatively? In Proceedings of the International Conference on Developments of E-Systems Engineering (DeSE), Duai, UAE, 13–14 December 2015; pp. 131–136. [Google Scholar]
- Mohamed, M.; Gao, S.; Sachdeva, N.; Saxena, N.; Zhang, C.; Kumaraguru, P.; Van Oorschot, P.C. On the security and usability of dynamic cognitive game CAPTCHAs. J. Comput. Secur. 2017, 25, 205–230. [Google Scholar] [CrossRef] [Green Version]
- Conti, M.; Guarisco, C.; Spolaor, R. CAPTCHaStar! A Novel CAPTCHA Based on Interactive Shape Discovery. In Applied Cryptography and Network Security, Proceedings of the International Conference on Applied Cryptography and Network Security, Guildford, UK, 19–22 June 2016; Manulis, M., Sadeghi, A.-R., Schneider, S., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2016; Volume 9696, pp. 611–628. [Google Scholar]
- Bursztein, E.; Bethard, S.; Fabry, C.; Mitchell, J.C.; Jurafsky, D. How Good Are Humans at Solving CAPTCHAs? A Large Scale Evaluation. In Proceedings of the 2010 IEEE Symposium on Security and Privacy, Berkeley/Oakland, CA, USA, 16–19 May 2010; pp. 399–413. [Google Scholar]
- Brodić, D.; Amelio, A.; Janković, R. Exploring the influence of CAPTCHA types to the users response time by statistical analysis. Multimed. Tools Appl. 2017, 77, 12293–12329. [Google Scholar] [CrossRef]
- Wilkins, J. Strong CAPTCHA guidelines v1.2 Retrieved Nov. Available online: http://www.123seminarsonly.com/Seminar-Reports/008/47584359-captcha.pdf (accessed on 25 June 2019).
- Sullivan, D.G. Data Mining V: Preparing the Data; Boston University: Boston, MA, USA, 2013. [Google Scholar]
- Bradley, P.S.; Mangasarian, O.L.; Street, W.N. Clustering via Concave Minimization. In Advances in Neural Information Processing Systems, Proceedings of the Conference on Neural Information Processing Systems, Princeton, NJ, USA, 1–7 December 1996; ACM: New York, NY, USA, 1996; pp. 368–374. [Google Scholar]
- Tan, P.-N.; Steinbach, M.; Kumar, V. Introduction to Data Mining, 1st ed.; Addison–Wesley Longman Publishing Co.: Boston, MA, USA, 2005. [Google Scholar]
- Tanikić, D.; Marinković, V.; Manić, M.; Devedžić, G.; Randelović, S. Application of response surface methodology and fuzzy logic based system for determining cutting temperature. Bull. Pol. Acad. Sci. Tech. Sci. 2016, 64, 435–445. [Google Scholar] [CrossRef]
- LeCun, Y. A Theoretical Framework for Back-Propagation. In Proceedings of the Connectionist Models Summer School, Pittsburgh, PA, USA, 17–26 June 1988; pp. 21–28. [Google Scholar]
- Levenberg, K. A Method for the Solution of Certain Non-Linear Problems in Least Squares. Q. Appl. Math. 1944, 2, 164–168. [Google Scholar] [CrossRef]
- SPSS Tutorials: Pearson Correlation. Available online: https://libguides.library.kent.edu/SPSS/PearsonCorr (accessed on 25 June 2019).
- Iantovics, B.; Corina, R.; Muaz, N. MetrIntPair—A novel accurate metric for the comparison of two cooperative multiagent systems intelligence based on paired intelligence measurements. Int. J. Intell. Syst. 2018, 33, 463–486. [Google Scholar] [CrossRef]
Figure 1.
The two types of Dice CAPTCHA: (a) Dice 1; and (b) Dice 2.
Figure 2.
Distribution of: (a) users’ age; (b) Internet experience in number of years; and (c) daily Internet usage in number of hours.
Figure 3.
Distribution of the CAPTCHA solution time for: (a) Dice 1; and (b) Dice 2.
Figure 4.
(a) ANN proposed structure; and (b) ANN learning principle.
Figure 5.
Scatter plot of the ARs given: (i) support; (ii) confidence; and (iii) lift, for Dice 1 and 2. Each coloured point represents an AR identified by its numerical ID (see
Table 1 and
Table 2). The position
of each point depends on the support and confidence values of the corresponding AR. The colour of each point represents the lift value of the corresponding AR.
Figure 6.
Distribution of the discretized solution time in Dice 1 and 2 datasets.
Figure 7.
Pearson’s correlation coefficient R between the target and predicted solution time for one training, validation, and test part of the Dice 1 dataset and for the whole dataset varying the number of neurons () in the hidden layer of the ANN model.
Figure 8.
Pearson’s correlation coefficient R between the target and predicted solution time for one training, validation, and test part of the Dice 2 dataset and for the whole dataset varying the number of neurons () in the hidden layer of the ANN model.
Figure 9.
(a) Histogram of the error (target—predicted solution time) for one training, validation, and test part of the Dice 1 dataset ( = 45); (b) trend of the target vs. predicted solution times for the Dice 1 dataset; (c) histogram of the error (target—predicted solution time) for one training, validation, and test part of the Dice 2 dataset ( = 20); and (d) trend of the target vs. predicted solution times for the Dice 2 dataset.
Figure 10.
Trend of the error computed as target—predicted solution time over the Internet users.
Figure 11.
Regression results (target vs. predicted solution time) for one training, validation, test part of the Dice 1 dataset and for the whole dataset ( = 45).
Figure 12.
Regression results (target vs. predicted solution time) for one training, validation, test part of the Dice 2 dataset and for the whole dataset ( = 20).
Table 1.
The set of the extracted association rules for Dice 1 CAPTCHA. The number of attempts in the consequent is 1 for all ARs (consequently, it is omitted).
Id. | Ant. | Cons. | S | C | L | |
---|
1 | > 35, Tablet, High Int. experience | Interm. | 0.06 | 0.44 | 1.99 | 1.40 |
2 | Middle Int. experience | Quick | 0.24 | 0.45 | 1.34 | 1.21 |
3 | Middle Int. experience, Middle Int. daily usage | Quick | 0.13 | 0.52 | 1.55 | 1.39 |
4 | < 35 | Quick | 0.11 | 0.44 | 1.31 | 1.19 |
5 | Middle Int. experience, < 35 | Quick | 0.08 | 0.53 | 1.60 | 1.43 |
6 | Tablet, < 35 | Quick | 0.08 | 0.42 | 1.26 | 1.15 |
7 | > 35, Middle Int. experience | Quick | 0.17 | 0.42 | 1.25 | 1.14 |
8 | Middle Int. experience, Low Int. daily usage | Quick | 0.09 | 0.45 | 1.34 | 1.21 |
9 | > 35, Middle Int. experience, Low Int. daily usage | Quick | 0.06 | 0.43 | 1.29 | 1.17 |
10 | > 35, Middle Int. experience, Middle Int. daily usage | Quick | 0.10 | 0.49 | 1.45 | 1.30 |
11 | Laptop | Quick | 0.20 | 0.40 | 1.20 | 1.11 |
12 | Middle Int. experience, Laptop | Quick | 0.17 | 0.51 | 1.54 | 1.37 |
13 | > 35, Middle Int. experience, Laptop | Quick | 0.14 | 0.48 | 1.44 | 1.28 |
14 | Middle Int. experience, Laptop, Low Int. daily usage | Quick | 0.06 | 0.50 | 1.49 | 1.33 |
15 | > 35, Middle Int. experience, Laptop, Low Int. daily usage | Quick | 0.05 | 0.46 | 1.37 | 1.23 |
16 | > 35, Laptop | Very quick | 0.18 | 0.42 | 1.94 | 1.36 |
17 | > 35, Laptop, Low Int. daily usage | Very quick | 0.08 | 0.40 | 1.83 | 1.30 |
18 | > 35, Middle Int. experience, Laptop, Low Int. daily usage | Very quick | 0.05 | 0.42 | 1.91 | 1.34 |
19 | Laptop, Middle Int. daily usage | Quick | 0.10 | 0.45 | 1.36 | 1.22 |
20 | Middle Int. experience, Laptop, Middle Int. daily usage | Quick | 0.10 | 0.58 | 1.72 | 1.57 |
21 | Laptop, High Int. experience | Very quick | 0.08 | 0.64 | 2.93 | 2.17 |
22 | > 35, Laptop, High Int. experience | Very quick | 0.07 | 0.67 | 3.05 | 2.34 |
23 | > 35, Laptop, Middle Int. daily usage | Quick | 0.09 | 0.47 | 1.41 | 1.26 |
24 | > 35, Middle Int. experience, Laptop, Middle Int. daily usage | Quick | 0.09 | 0.57 | 1.69 | 1.53 |
Table 2.
The set of the extracted association rules for Dice 2 CAPTCHA. The number of attempts in the consequent is 1 for all ARs (consequently, it is omitted).
Id. | Ant. | Cons. | S | C | L | |
---|
1 | Middle Int. experience | Quick | 0.22 | 0.41 | 1.11 | 1.07 |
2 | Middle Int. experience, Tablet | Quick | 0.09 | 0.44 | 1.18 | 1.12 |
3 | Tablet, Middle Int. daily usage | Quick | 0.07 | 0.41 | 1.11 | 1.07 |
4 | < 35 | Quick | 0.11 | 0.44 | 1.19 | 1.12 |
5 | Middle Int. experience, < 35 | Quick | 0.06 | 0.46 | 1.25 | 1.17 |
6 | Tablet, < 35 | Quick | 0.08 | 0.42 | 1.14 | 1.09 |
7 | > 35 | Very quick | 0.32 | 0.43 | 1.16 | 1.10 |
8 | > 35, Low Int. daily usage | Very quick | 0.15 | 0.45 | 1.21 | 1.14 |
9 | > 35, High Int. experience | Very quick | 0.10 | 0.42 | 1.11 | 1.07 |
10 | Low Int. daily usage, High Int. experience | Very quick | 0.05 | 0.40 | 1.06 | 1.04 |
11 | > 35, Middle Int. experience, Middle Int. daily usage | Quick | 0.08 | 0.41 | 1.11 | 1.07 |
12 | > 35, Tablet, Middle Int. daily usage | Quick | 0.05 | 0.42 | 1.12 | 1.08 |
13 | Middle Int. experience | Very quick | 0.22 | 0.41 | 1.09 | 1.06 |
14 | Middle Int. daily usage | Very quick | 0.16 | 0.41 | 1.09 | 1.06 |
15 | Middle Int. experience, Middle Int. daily usage | Very quick | 0.11 | 0.44 | 1.16 | 1.11 |
16 | > 35, Middle Int. experience | Very quick | 0.20 | 0.51 | 1.35 | 1.26 |
17 | Middle Int. experience, Low Int. daily usage | Very quick | 0.10 | 0.47 | 1.26 | 1.19 |
18 | > 35, Middle Int. experience, Low Int. daily usage | Very quick | 0.10 | 0.63 | 1.69 | 1.70 |
19 | High Int. daily usage | Quick | 0.08 | 0.42 | 1.14 | 1.09 |
20 | Middle Int. experience, High Int. daily usage | Quick | 0.05 | 0.58 | 1.56 | 1.49 |
21 | Tablet, High Int. daily usage | Quick | 0.06 | 0.43 | 1.16 | 1.10 |
22 | > 35, Middle Int. daily usage | Very quick | 0.14 | 0.45 | 1.20 | 1.14 |
23 | > 35, Middle Int. experience, Middle Int. daily usage | Very quick | 0.09 | 0.46 | 1.23 | 1.16 |
24 | Laptop | Very quick | 0.27 | 0.55 | 1.45 | 1.38 |
25 | Middle Int. experience, Laptop | Very quick | 0.18 | 0.54 | 1.45 | 1.37 |
26 | > 35, Laptop | Very quick | 0.26 | 0.60 | 1.60 | 1.56 |
27 | > 35, Middle Int. experience, Laptop | Very quick | 0.17 | 0.59 | 1.56 | 1.51 |
28 | Laptop, Low Int. daily usage | Very quick | 0.12 | 0.53 | 1.42 | 1.34 |
29 | > 35, Laptop, Low Int. daily usage | Very quick | 0.12 | 0.57 | 1.53 | 1.47 |
30 | Middle Int. experience, Laptop, Low Int. daily usage | Very quick | 0.08 | 0.58 | 1.53 | 1.47 |
31 | > 35, Middle Int. experience, Laptop, Low Int. daily usage | Very quick | 0.08 | 0.62 | 1.66 | 1.66 |
32 | Laptop, Middle Int. daily usage | Very quick | 0.13 | 0.57 | 1.51 | 1.44 |
33 | > 35, Laptop, Middle Int. daily usage | Very quick | 0.12 | 0.63 | 1.68 | 1.69 |
34 | Middle Int. experience, Laptop, Middle Int. daily usage | Very quick | 0.09 | 0.54 | 1.45 | 1.37 |
35 | > 35, Middle Int. experience, Laptop, Middle Int. daily usage | Very quick | 0.09 | 0.57 | 1.51 | 1.44 |
36 | Laptop, High Int. experience | Very quick | 0.08 | 0.60 | 1.60 | 1.56 |
37 | > 35, Laptop, High Int. experience | Very quick | 0.08 | 0.71 | 1.90 | 2.18 |
Table 3.
Pearson’s correlation coefficient for the whole dataset from Dice 1 and Dice 2. The best values are marked.
R | 5 | 10 | 15 | 20 | 25 | 30 | 35 | 40 | 45 | 50 |
Dice 1 | 0.733 | 0.675 | 0.728 | 0.724 | 0.777 | 0.553 | 0.723 | 0.774 | 0.789 | 0.553 |
Dice 2 | 0.796 | 0.752 | 0.657 | 0.803 | 0.795 | 0.560 | 0.300 | 0.569 | 0.321 | 0.726 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).