A Comparative Assessment of Graphic and 0–10 Rating Scales Used to Measure Entrepreneurial Competences
Abstract
:1. Introduction
2. Measurement Scales
3. Method
3.1. Measurement of Entrepreneurial Competences
3.2. Measurement Quality Assessment
- t1: I can easily interpret correctly other people’s moods and reactions (empathy).
- t2: I am able to change my working habits to meet public interest (self-control).
- t3: I wish to repeatedly perform a task forcing me to strive to succeed (self-motivation).
- t4: I act quickly and determinedly whenever opportunities and crises appear (self-motivation).
- t5: I aim to analyze and correct my mistakes in order to improve my performance in the future (cognitive capacity).
- t6: One must take immediate action when perceiving a possibility for success (self-motivation).
- by the trait, computed as λ2j ϕii and referred to as valid variance or error-free variance,
- by the method, computed as ϕMjj and referred to as invalid variance or method effect variance,
- and random error variance (θij),
3.3. Data Description
4. Results
5. Discussion
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Zadeh, L.A. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
- Linares-Mustarós, S.; Ferrer-Comalat, J.C.; Cassú-Serra, E. The assessment of cash flow forecasting. Kybernetes 2013, 42, 736–753. [Google Scholar] [CrossRef]
- Jaini, N.; Utyuzhnikov, S. A fuzzy trade-off ranking method for multi-criteria decision-making. Axioms 2017, 7, 1. [Google Scholar] [CrossRef] [Green Version]
- López, C.; Linares, S.; Viñas, J. Evolutionary positioning of outsourcing in the local public administration. Intang. Cap. 2019, 15, 157–170. [Google Scholar] [CrossRef]
- Farrar, J.T.; Young, J.P., Jr.; LaMoreaux, L.; Werth, J.L.; Poole, R.M. Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001, 94, 149–158. [Google Scholar] [CrossRef]
- Riedl, C.; Blohm, I.; Leimeister, J.M.; Krcmar, H. Rating scales for collective intelligence in innovation communities: Why quick and easy decision making does not get it right. In Proceedings of the Thirty First International Conference on Information Systems, St. Louis, MO, USA, 24 November 2010. [Google Scholar]
- Gonzalez Campos, J.A.; Manriquez Penafiel, R.A. A method for ordering of LR-type fuzzy numbers: An important decision criteria. Axioms 2016, 5, 22. [Google Scholar] [CrossRef]
- Hernández, P.; Cubillo, S.; Torres-Blanc, C.; Guerrero, J. New order on type 2 fuzzy numbers. Axioms 2017, 6, 22. [Google Scholar] [CrossRef] [Green Version]
- Piasecki, K. Revision of the Kosiński’s theory of ordered fuzzy numbers. Axioms 2018, 7, 16. [Google Scholar] [CrossRef] [Green Version]
- Tuana, N.A. Developing a generalized fuzzy multi-criteria decision making for personnel selection. Fuzzy Econ. Rev. 2018, 23, 27–41. [Google Scholar]
- Zadeh, L.A. The concept of a linguistic variable and its application to approximate reasoning-I. Inf. Sci. 1975, 8, 199–249. [Google Scholar] [CrossRef]
- Curry, B.; Lazzari, L. Fuzzy consideration sets: A new approach based on direct use of consumer preferences. Int. J. Appl. Manag. Sci. 2009, 1, 420–436. [Google Scholar] [CrossRef]
- Dey, A.; Pradhan, R.; Pal, A.; Pal, T. The fuzzy robust graph coloring problem. In Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA) Paniora, Odisha, India, 14 November 2014; Springer: Cham, Switzerland, 2015; pp. 805–813. [Google Scholar]
- Dey, A.; Pal, A.; Pal, T. Interval type 2 fuzzy set in fuzzy shortest path problem. Mathematics 2016, 4, 62. [Google Scholar] [CrossRef] [Green Version]
- Dey, A.; Son, L.H.; Kumar, P.K.; Selvachandran, G.; Quek, S.G. New concepts on vertex and edge coloring of simple vague graphs. Symmetry 2018, 10, 373. [Google Scholar] [CrossRef] [Green Version]
- Ferrer-Comalat, J.C.; Linares-Mustarós, S.L.; Corominas-Coll, D. A formalization of the theory of expertons. Theoretical foundations, properties and development of software for its calculation. Fuzzy Econ. Rev. 2016, 21, 23–39. [Google Scholar] [CrossRef]
- Linares-Mustarós, S.; Ferrer-Comalat, J.C.; Corominas-Coll, D.; Merigó, J.M. The ordered weighted average in the theory of expertons. Int. J. Intell. Syst. 2019, 34, 345–365. [Google Scholar] [CrossRef]
- Alfaro-Calderón, G.G.; Godínez-Reyes, N.L.; Gómez-Monge, R.; Alfaro-García, V.; Gil-Lafuente, A.M. Forgotten effects in the valuation of the social well-being index in Mexico’s sustainable development. Fuzzy Econ. Rev. 2019, 24, 67–81. [Google Scholar]
- Linares-Mustarós, S.; Ferrer-Colomat, J.C. Teoría de conjuntos clásica versus teoría de subconjuntos borrosos. Un ejemplo elemental comparativo. Tribuna Plural: La Revista Científica 2014, 2, 485–500. [Google Scholar]
- Miller, G.A. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol. Rev. 1956, 63, 81–97. [Google Scholar] [CrossRef] [Green Version]
- Baddeley, A. The magical number seven: Still magic after all these years? Psychol. Rev. 1994, 101, 353–356. [Google Scholar] [CrossRef]
- Gil-Aluja, J. Fuzzy Sets in the Management of Uncertainty; Springer Science & Business Media: Berlin, Germany, 2004. [Google Scholar]
- Linares-Mustaros, S.; Cassu-Serra, E.; Gil-Lafuente, A.M.; Ferrer-Comalat, J.C. New practical tools for minimizing human error in research into forgotten effects. In Computational Data Analysis Techniques in Economics and Finance; Doumpos, M., Zopounidis, C., Eds.; Nova: Hauppauge, NY, USA, 2015; pp. 231–248. [Google Scholar]
- DeCastellarnau, A. A classification of response scale characteristics that affect data quality: A literature review. Qual. Quant. 2018, 52, 1523–1559. [Google Scholar] [CrossRef]
- Bosch, O.J.; Revilla, M.; DeCastellarnau, A.; Weber, W. Measurement reliability, validity, and quality of slider versus radio button scales in an online probability-based panel in Norway. Soc. Sci. Comput. Rev. 2019, 37, 119–132. [Google Scholar] [CrossRef]
- Cook, C.; Heath, F.; Thompson, R.L.; Thompson, B. Score reliability in web- or Internet-based surveys: Unnumbered graphic rating scales versus Likert-type scales. Educ. Psychol. Meas. 2001, 61, 697–706. [Google Scholar] [CrossRef]
- Gartner, W.B. What are we talking about when we talk about entrepreneurship? J. Bus. Ventur. 1990, 5, 15–28. [Google Scholar] [CrossRef]
- Dey, A.; Pradhan, R.; Pal, A.; Pal, T. A genetic algorithm for solving fuzzy shortest path problems with interval type-2 fuzzy arc lengths. Malays. J. Comput. Sci. 2018, 31, 255–270. [Google Scholar] [CrossRef] [Green Version]
- Dey, A.; Pal, A.; Long, H.V. Fuzzy minimum spanning tree with interval type 2 fuzzy arc length: Formulation and a new genetic algorithm. Soft Comput. 2019, 2019, 1–12. [Google Scholar] [CrossRef]
- Moayedi, H.; Tien Bui, D.; Gör, M.; Pradhan, B.; Jaafari, A. The feasibility of three prediction techniques of the artificial neural network, adaptive neuro-fuzzy inference system, and hybrid particle swarm optimization for assessing the safety factor of cohesive slopes. ISPRS Int. J. Geo-Inf. 2019, 8, 391. [Google Scholar] [CrossRef] [Green Version]
- Polishchuk, V.; Kelemen, M.; Gavurová, B.; Varotsos, C.; Andoga, R.; Gera, M.; Christodoulakis, J.; Soušek, R.; Kozuba, J.; Hospodka, J.; et al. A fuzzy model of risk assessment for environmental start-up projects in the air transport sector. Int. J. Environ. Res. Public Health 2019, 16, 3573. [Google Scholar] [CrossRef] [Green Version]
- Castillo, O.; Valdez, F.; Soria, J.; Amador-Angulo, L.; Ochoa, P.; Peraza, C. Comparative study in fuzzy controller optimization using bee colony, differential evolution, and harmony search algorithms. Algorithms 2019, 12, 9. [Google Scholar] [CrossRef] [Green Version]
- Kaufmann, A. Expert appraisements and counter-appraisements with experton processes. In Proceedings of the First International Symposium on Uncertainty Modeling and Analysis, College Park, MD, USA, 3 December 1990; IEEE Computer Society Press: Washington, DC, USA, 1990; pp. 619–624. [Google Scholar]
- Vizuete Luciano, E.; Gil-Lafuente, A.M.; García González, A.; Boria-Reverter, S. Forgotten effects of corporate social and environmental responsibility. Kybernetes 2013, 42, 736–753. [Google Scholar] [CrossRef]
- Bass, B.M.; Cascio, W.F.; O’Connor, E.J. Magnitude estimations of expressions of frequency and amount. J. Appl. Psychol. 1974, 59, 313–320. [Google Scholar] [CrossRef]
- Osgood, C.E.; Suci, G.J.; Tannenbaum, P.H. The Measurement of Meaning; University of Illinois Press: Chicago, IL, USA, 1975. [Google Scholar]
- Lodge, M.; Tursky, B. On the magnitude scaling of political opinion in survey research. Am. J. Political Sci. 1981, 25, 376–419. [Google Scholar] [CrossRef]
- Saris, W.E. Variation in Response Functions: A Source of Measurement Error in Attitude Research; Sociometric Research Foundation: Amsterdam, The Netherlands, 1988. [Google Scholar]
- Fowler, F.J., Jr. Survey Research Methods; Sage: Thousand Oaks, CA, USA, 2013. [Google Scholar]
- Groves, R.M.; Fowler, F.J., Jr.; Couper, M.P.; Lepkowski, J.M.; Singer, E.; Tourangeau, R. Survey Methodology; Wiley: Hoboken, NJ, USA, 2011. [Google Scholar]
- Saris, W.E. Continuous Scales in the Social Sciences: An Attractive Possibility; Sociometric Research Foundation: Amsterdam, The Netherlands, 1987. [Google Scholar]
- Couper, M.P.; Tourangeau, R.; Conrad, F.G.; Singer, E. Evaluating the effectiveness of visual analog scales: A web experiment. Soc. Sci. Comput. Rev. 2006, 24, 227–245. [Google Scholar] [CrossRef]
- De Pijper, W.M.; Saris, W.E. The Formulation of Interviews Using the Program INTERV; Sociometric Research Foundation: Amsterdam, The Netherlands, 1986. [Google Scholar]
- Saris, W.E.; Munnich, Á. The Multitratit-Multimethod Approach to Evaluate Measurement Instruments; Eötvös University Press: Budapest, Hungary, 1995. [Google Scholar]
- Cape, P. Slider scales in online surveys. In Proceedings of the CASRO Panel Conference, New Orleans, LA, USA, 2 February 2009. [Google Scholar]
- Sikkel, D.; Steenbergen, R.; Gras, S. Clicking vs. dragging: Different uses of the mouse and their implications for online surveys. Public Opin. Q. 2014, 78, 177–190. [Google Scholar] [CrossRef]
- Aramo-Immonen, H.; Bikfalvi, A.; Mancebo, N.; Vanharanta, H. Project managers’ competence identification. Int. J. Hum. Cap. Inf. Technol. Prof. 2011, 2, 37–51. [Google Scholar] [CrossRef]
- Shapero, A.; Sokol, L. The social dimensions of entrepreneurship. In Encyclopedia of Entrepreneurship; Kent, C.A., Sexton, D.L., Vesper, K.H., Eds.; Prentice Hall: Englewood Cliffs, NJ, USA, 1982; pp. 72–90. [Google Scholar]
- Ajzen, I. The theory planned behavior. Organ. Behav. Hum. Decis. Process. 1991, 50, 179–211. [Google Scholar] [CrossRef]
- Kantola, J.; Karwowski, W.; Vanharanta, H. Managing managerial mosaic: The Evolute methodology. In Electronic Globalized Business and Sustainable Development through IT Management: Strategies and Perspectives; Ordóñez de Pablos, P., Lytras, M.D., Karwowski, W., Lee, R.W.B., Eds.; IGI Global: Hershey, PA, USA, 2006; pp. 77–89. [Google Scholar]
- Peterman, N.; Kennedy, J. Enterprise education: Influencing students’ perceptions of entrepreneurship. Entrep. Theory Pract. 2003, 28, 129–144. [Google Scholar] [CrossRef]
- Palolen, E. Tricuspoid-The Competence Evaluation Application for Entrepreneurs; Tampere University of Technology: Tampere, Finland, 2005. [Google Scholar]
- Coenders, G.; Saris, W.E. Systematic and random method effects. Estimating method bias and method variance. In Developments in Survey Methodology; Ferligoj, A., Mrvar, A., Eds.; Metodološki Zvezki 15, FDV: Ljubljana, Slovenia, 2000; pp. 55–74. [Google Scholar]
- Campbell, D.T.; Fiske, D.W. Convergent and discriminant validation by the multitrait multimethod matrices. Psychol. Bull. 1959, 56, 81–105. [Google Scholar] [CrossRef] [Green Version]
- Van Meurs, A.; Saris, W.E. Memory effects in MTMM studies. In Multitrait Multimethod Approach to Evaluate Measurement Instruments; Saris, W.E., Munnich, A., Eds.; Eötvös University Press: Budapest, Hungary, 1995; pp. 89–103. [Google Scholar]
- Andrews, F.M. Construct validity and error components of survey measures. A structural modeling approach. Public Opin. Q. 1984, 48, 409–442. [Google Scholar] [CrossRef]
- Coenders, G.; Saris, W.E. Testing nested additive, multiplicative and general multitrait-multimethod models. Struct. Equ. Model. 2000, 7, 219–250. [Google Scholar] [CrossRef]
- Saris, W.E.; Gallhofer, I.N. Design, Evaluation, and Analysis of Questionnaires for Survey Research; Wiley: Hoboken, NJ, USA, 2014. [Google Scholar]
- Yuan, K.H.; Bentler, P.M. Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. In Sociological Methodology; Sobel, M.E., Becker, M.P., Eds.; American Sociological Association: Washington, DC, USA, 2000; pp. 165–200. [Google Scholar]
- Muthén, L.K.; Muthén, B.O. Mplus User’s Guide, 8th ed.; Muthén & Muthén: Los Angeles, CA, USA, 2017. [Google Scholar]
- Roster, C.A.; Lucianetti, L.; Albaum, G. Exploring slider vs. categorical response formats in webbased surveys. J. Res. Pract. 2015, 11, 1–16. [Google Scholar]
- Saris, W.E.; Oberski, D.L.; Revilla, M.; Zavala Rojas, D.; Gallhofer, L.; Lilleoja, I.; Gruner, T. Final Report about the Project JRA3 as Part of ESS; RECSM Working Paper, No. 24; RECSM/UPF: Barcelona, Spain, 2011. [Google Scholar]
- Toninelli, D.; Revilla, M. How mobile device screen size affects data collected in web surveys. In Advances in Questionnaire Design, Development, Evaluation and Testing; Beatty, P., Collins, D., Kaye, L., Padilla, J.L., Willis, G., Wilmot, A., Eds.; Wiley & Sons: Hoboken, NJ, USA, 2020; pp. 349–373. [Google Scholar]
- Höhne, J.K.; Revilla, M.; Lenzner, T. Comparing the performance of agree/disagree and item-specific questions across PCs and smartphones. Methodology 2018, 14, 109–118. [Google Scholar] [CrossRef]
- Couper, M.P.; Antoun, C.; Mavletova, A. Mobile web surveys. In Total Survey Error in Practice; Biemer, P.P., de Leeuw, E.D., Eckman, S., Edwards, B., Kreuter, F., Lyberg, L.E., Eds.; Wiley & Sons: Hoboken, NJ, USA, 2017; pp. 133–154. [Google Scholar]
- Tourangeau, R.; Maitland, A.; Steiger, D.; Yan, T. A framework for making decisions about question evaluation methods. In Advances in Questionnaire Design, Development, Evaluation and Testing; Beatty, P., Collins, D., Kaye, L., Padilla, J.L., Willis, G., Wilmot, A., Eds.; Wiley & Sons: Hoboken, NJ, USA, 2020; pp. 47–73. [Google Scholar]
Mean | SD | Min | Q1 | Q2 | Q3 | Max | |
---|---|---|---|---|---|---|---|
t1m1 | 7.0 | 2.26 | 0.0 | 5.6 | 7.3 | 8.8 | 10.0 |
t2m1 | 6.4 | 2.21 | 0.1 | 5.1 | 6.6 | 8.1 | 10.0 |
t3m1 | 5.2 | 2.51 | 0.0 | 3.5 | 5.3 | 7.1 | 10.0 |
t4m1 | 6.5 | 2.10 | 0.2 | 5.1 | 6.7 | 8.1 | 10.0 |
t5m1 | 7.1 | 1.87 | 1.3 | 5.8 | 7.3 | 8.6 | 10.0 |
t6m1 | 6.8 | 2.02 | 0.3 | 5.4 | 6.9 | 8.4 | 10.0 |
t1m2 | 7.4 | 1.91 | 0.0 | 7.0 | 8.0 | 9.0 | 10.0 |
t2m2 | 7.0 | 1.77 | 1.0 | 6.0 | 7.0 | 8.0 | 10.0 |
t3m2 | 5.7 | 2.35 | 0.0 | 4.0 | 6.0 | 7.0 | 10.0 |
t4m2 | 6.9 | 1.70 | 0.0 | 6.0 | 7.0 | 8.0 | 10.0 |
t5m2 | 7.7 | 1.49 | 2.0 | 7.0 | 8.0 | 9.0 | 10.0 |
t6m2 | 7.7 | 1.72 | 1.0 | 7.0 | 8.0 | 9.0 | 10.0 |
Estimate | LCL | UCL | ||
---|---|---|---|---|
Bias assessment: | τ2 | 0.41 | −0.56 | 1.38 |
λ2 | 1.02 | 0.88 | 1.17 | |
Trait variance: | ϕ11 | 2.66 | 1.92 | 3.39 |
ϕ22 | 2.09 | 1.49 | 2.70 | |
ϕ33 | 3.17 | 2.45 | 3.90 | |
ϕ44 | 2.10 | 1.58 | 2.61 | |
ϕ55 | 1.30 | 0.91 | 1.68 | |
ϕ66 | 1.55 | 1.13 | 1.98 | |
Error variance: | θ11 | 1.84 | 1.26 | 2.42 |
θ21 | 2.12 | 1.61 | 2.63 | |
θ31 | 2.90 | 2.20 | 3.60 | |
θ41 | 1.78 | 1.39 | 2.17 | |
θ51 | 1.56 | 1.22 | 1.89 | |
θ61 | 2.35 | 1.83 | 2.88 | |
θ12 | 0.88 | 0.30 | 1.45 | |
θ22 | 1.04 | 0.62 | 1.46 | |
θ32 | 2.07 | 1.34 | 2.80 | |
θ42 | 0.65 | 0.30 | 0.99 | |
θ52 | 0.88 | 0.60 | 1.16 | |
θ62 | 1.14 | 0.75 | 1.52 | |
Method variance: | ϕM11 | 0.47 | 0.31 | 0.63 |
ϕM22 | 0.06 | −0.10 | 0.22 |
Trait | Random Error | Method | |
---|---|---|---|
(Quality) | (Lack of Reliability) | (Lack of Validity) | |
t1m1 | 53% | 37% | 10% |
t2m1 | 45% | 45% | 10% |
t3m1 | 49% | 44% | 07% |
t4m1 | 48% | 41% | 11% |
t5m1 | 39% | 47% | 14% |
t6m1 | 35% | 54% | 11% |
t1m2 | 75% | 23% | 02% |
t2m2 | 67% | 31% | 02% |
t3m2 | 61% | 38% | 01% |
t4m2 | 76% | 22% | 02% |
t5m2 | 59% | 38% | 03% |
t6m2 | 58% | 40% | 02% |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Vall-Llosera, L.; Linares-Mustarós, S.; Bikfalvi, A.; Coenders, G. A Comparative Assessment of Graphic and 0–10 Rating Scales Used to Measure Entrepreneurial Competences. Axioms 2020, 9, 21. https://doi.org/10.3390/axioms9010021
Vall-Llosera L, Linares-Mustarós S, Bikfalvi A, Coenders G. A Comparative Assessment of Graphic and 0–10 Rating Scales Used to Measure Entrepreneurial Competences. Axioms. 2020; 9(1):21. https://doi.org/10.3390/axioms9010021
Chicago/Turabian StyleVall-Llosera, Laura, Salvador Linares-Mustarós, Andrea Bikfalvi, and Germà Coenders. 2020. "A Comparative Assessment of Graphic and 0–10 Rating Scales Used to Measure Entrepreneurial Competences" Axioms 9, no. 1: 21. https://doi.org/10.3390/axioms9010021
APA StyleVall-Llosera, L., Linares-Mustarós, S., Bikfalvi, A., & Coenders, G. (2020). A Comparative Assessment of Graphic and 0–10 Rating Scales Used to Measure Entrepreneurial Competences. Axioms, 9(1), 21. https://doi.org/10.3390/axioms9010021