# Gamified Evaluation in STEAM for Higher Education: A Case Study

^{1}

^{2}

^{*}

^{†}

^{‡}

## Abstract

**:**

## 1. Introduction

## 2. Gamification of Evaluation

## 3. Gaming Environment

#### 3.1. Methodology

#### 3.2. Architecture

#### 3.3. Competence Profiles

#### 3.4. Aggregation of Streaming Evaluation

## 4. Virtual Models

#### 4.1. Design Concepts

- Visual design––the models are expected to be 3D models with relatively sufficient graphical effects, like texturing and lighting. The models themselves should resemble actual objects from our everyday life, so that students can easily understand their expected functionalities.
- Interactional design—the models should provide game-like interactivity, this includes walking around the models, getting close to them and getting away from them. Additionally, all models provide their own specific interactivity, because they represent virtual devices and mechanisms, which can be controlled.
- Pedagogical design—this design encompasses four distinct features: (i) the model should provide sufficient clues that help students, but should still provoke decision making and problem solving efforts as each problem could be solved in different ways; (ii) the model is generated with some randomness of its configuration, however, it is possible to define the desired level of difficulty by limiting the scope of the randomness of individual parameters; (iii) each model should report back general data about the gameplay, like how many clicks are done, how much time is used, how accurate is the solution, and so forth, and (iv) models can be used for evaluation and assessment of students’ theoretical knowledge and practical skills as well as they develop the sense-and-feeling soft-skill in the domain of computer graphics.

#### 4.2. The CMY Reservoir

#### 4.3. The Euler’s Grill

- touching tunnels introduce an odd Euler’s characteristic (as opposed to even);
- tunnels through tunnels decrease the Euler’s characteristic to negative values;
- wide or colliding tunnels that split the object into separate, non-connected objects increase the Euler’s characteristic;
- non-touching indentations and protrusions do not change the Euler characteristic, although they increase the number of faces, edges and vertices.

#### 4.4. The Cohen-Sutherland’s Thimble

#### 4.5. The Matrix Carousel

- provide a unified way to represent various transformations—they are all expressed as matrices;
- allow packing several transformations into a single matrix by multiplying matrices in advance;
- control working in local or global coordinate systems by premultiplication or postmultiplication;
- support nested transformations by using a matrix stack;
- are fast to compute as there is hardware support for matrix operations in GPUs and shader languages.

## 5. Preliminary Results

#### 5.1. Automatically Collected Data

#### 5.2. Students’ Survey

- Are there any technical issues with using mobile devices?
- Are there any difficulties caused by the user interface?
- Are gaming evaluations suitable for university students?
- Are there any suggestions for improvement of the models?

## 6. Conclusions and Further Work

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

CMY | Cyan, magenta, yellow |

CS | Computer Sciences |

FCG | Fundamentals of Computer Graphics |

FMI | Faculty of Mathematics and Informatics |

GBL | Game-Based Learning |

GPU | Graphical Processing Unit |

IBL | Instruction-Based Learning |

LMS | Learning Management System |

PBL | Project-Based Learning |

STEAM | Science, Technology, Engineering, the Arts and Mathematics |

STEM | Science, Technology, Engineering and Mathematics |

WebGL | Web Graphics Library |

## References

- Ortiz Rojas, M.E.; Chiluiza, K.; Valcke, M. Gamification in higher education and stem: A systematic review of literature. In Proceedings of the 8th International Conference on Education and New Learning Technologies (EDULEARN), Barcelona, Spain, 4–6 July 2016; IATED—International Academy of Technology, Education and Development: Valencia, Spain, 2016; pp. 6548–6558. [Google Scholar]
- Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in Education: A Systematic Mapping Study. J. Educ. Technol. Soc.
**2015**, 18, 75–88. [Google Scholar] - Kim, S.; Song, K.; Lockee, B.; Burton, J. Gamification Cases in Education. In Gamification in Learning and Education; Springer: Berlin, Germany, 2018; pp. 117–123. [Google Scholar]
- Fotaris, P.; Mastoras, T.; Leinfellner, R.; Rosunally, Y. Climbing up the Leaderboard: An Empirical Study of Applying Gamification Techniques to a Computer Programming Class. Electron. J. E-Learn.
**2016**, 14, 94–110. [Google Scholar] - Piteira, M.; Costa, C.J. Gamification: Conceptual framework to online courses of learning computer programming. In Proceedings of the 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Lisbon, Portugal, 14–17 June 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–7. [Google Scholar]
- Ibáñez, M.B.; Di-Serio, A.; Delgado-Kloos, C. Gamification for engaging computer science students in learning activities: A case study. IEEE Trans. Learn. Technol.
**2014**, 7, 291–301. [Google Scholar] [CrossRef] - de Almeida Souza, M.R.; Constantino, K.F.; Veado, L.F.; Figueiredo, E.M.L. Gamification in software engineering education: An empirical study. In Proceedings of the 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE&T), Savannah, GA, USA, 7–9 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 276–284. [Google Scholar]
- Iosup, A.; Epema, D. An experience report on using gamification in technical higher education. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education, Atlanta, GA, USA, 5–8 March 2014; ACM: New York, NY, USA, 2014; pp. 27–32. [Google Scholar]
- Villagrasa, S.; Duran, J. Gamification for learning 3D computer graphics arts. In Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality, Salamanca, Spain, 14–15 November 2013; ACM: New York, NY, USA, 2013; pp. 429–433. [Google Scholar]
- O’Donovan, S.; Gain, J.; Marais, P. A case study in the gamification of a university-level games development course. In Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference; ACM: New York, NY, USA, 2013; pp. 242–251. [Google Scholar]
- van Roy, R.; Zaman, B. Why gamification fails in education and how to make it successful: Introducing nine gamification heuristics based on self-determination theory. In Serious Games and Edutainment Applications; Springer: Berlin, Germany, 2017; pp. 485–509. [Google Scholar]
- Buckley, P.; Doyle, E. Gamification and student motivation. Interact. Learn. Environ.
**2016**, 24, 1162–1175. [Google Scholar] [CrossRef] - Freeman, S.; Eddy, S.L.; McDonough, M.; Smith, M.K.; Okoroafor, N.; Jordt, H.; Wenderoth, M.P. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. USA
**2014**, 111, 8410–8415. [Google Scholar] [CrossRef] [PubMed][Green Version] - Hamari, J.; Eranti, V. Framework for Designing and Evaluating Game Achievements. In Proceedings of the 2011 Digra Conference, Hilversum, The Netherlands, 14–17 September 2011. [Google Scholar]
- Heilbrunn, B.; Herzig, P.; Schill, A. Tools for gamification analytics: A survey. In Proceedings of the 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, London, UK, 8–11 December 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 603–608. [Google Scholar]
- Menezes, C.C.N.; Bortoli, R.d. Potential of gamification as assessment tool. Creat. Educ.
**2016**, 7, 561–566. [Google Scholar] [CrossRef][Green Version] - Gañán, D.; Caballé, S.; Clarisó, R.; Conesa, J.; Bañeres, D. ICT-FLAG: A web-based e-assessment platform featuring learning analytics and gamification. Int. J. Web Inf. Syst.
**2017**, 13, 25–54. [Google Scholar] [CrossRef] - Zichermann, G.; Cunningham, C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2011. [Google Scholar]
- Winnicka, A.; Kȩsik, K.; Połap, D.; Woźniak, M.; Marszałek, Z. A Multi–Agent Gamification System for Managing Smart Homes. Sensors
**2019**, 19, 1249. [Google Scholar] [CrossRef] [PubMed][Green Version] - Boytchev, P.; Boytcheva, S. Evaluation and assessment in TEL courses. AIP Conf. Proc.
**2018**, 2048, 020035. [Google Scholar] - Boytchev, P.; Boytcheva, S. Gamified Evaluation in STEAM. In International Conference on Information and Software Technologies; Springer: Berlin, Germany, 2019; pp. 369–382. [Google Scholar]

**Figure 2.**Examples of 3D models in Meiro—central projection (

**left**) and multilayered objects (

**right**).

**Figure 8.**A snapshot of the CMY reservoir (

**left**) and disappearing plate reaching a solution (

**right**).

**Figure 9.**The cube (

**left**), the dodecahedron (

**center**) and the icosahedron (

**right**) all have $\chi =2$.

**Figure 10.**A snapshot of the Euler’s grill (

**left**) and examples of different topological complexities (

**right**).

**Figure 11.**Bitmasks of the nine areas defined by the framing lines (

**left**), extended approach to convex non-rectangular area with only some of the masks shown (

**right**).

1. Mathematics | 2. Computer Sciences | 3. Physics | 4. Art |
---|---|---|---|

1.1. Math objects | 2.1. Rasterization | 3.1. Physics laws | 4.1. Colours |

1.2. Equations | 2.2. Geometrical data | 3.2. Simulations | 4.2. Shapes |

1.3. Parameters | 2.3. Animation | 3.3. Phenomena | 4.3. Orientation |

1.4. Relations | 2.4. Graphical objects | 4.4. Synchronization | |

1.5. Approximation | 4.5. Graphical effects |

Parameter | Description | Example |
---|---|---|

ACTION | Type of the log record | END |

MODEL | Name of the 3D model | T001 |

DIFFICULTY | Level of difficulty | 2 |

MAX_SCORE | Maximal score | 0.9 |

SCORE | Achieved score | 0.84 |

TIME | Number of seconds | 38 |

CLICKS | Number of clicks | 14 |

COMPETENCES | Vectored score | 0.84, 0, 2.53, 0.84, 2.53, 0, 0.84, 0, 0, 0, 1.68, 0, 4.21, 0, 0.84, 0, 2.53 |

IP | IP address | 188.254.xxx.xxx |

TIMESTAMP | Local time | 2020-01-05 18:35:00 |

Parameter | Logged-In | Anonymous | Total |
---|---|---|---|

Unique users | 121 | 65 | 186 |

Models played | 17,663 | 1386 | 19,049 |

Time played (hours) | 342 | 42 | 384 |

Clicks and taps | 1,048,982 | 89,049 | 1,138,031 |

Incomplete records | 1543 | 392 | 1935 |

Model | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | Total |
---|---|---|---|---|---|---|---|---|---|---|---|

Answers | 91 | 87 | 85 | 74 | 71 | 85 | 78 | 69 | 83 | 71 | 794 |

Edits | 104 | 103 | 109 | 88 | 82 | 104 | 95 | 80 | 107 | 85 | 957 |

Words | 4581 | 3384 | 4298 | 2677 | 2401 | 3242 | 3519 | 2982 | 4242 | 2829 | 34,155 |

Section | Section 4.2 | Section 4.3 | Section 4.4 | Section 4.5 |

Soft Skill | Examples |
---|---|

Seeing similarity of colours | Model 1 (The CMY reservoir) does not provide a numeric representation of the current liquid ink proportions, so students may only inspect the colour visually. |

Estimating positions in space | Model 8 (The Loop’s torus) requires the students to calculate or guess positions on a curved surface without a detailed coordinate grid to guide them. |

Using non-visual senses | In model 6 (the Pick’s polygon) students may solve the problem more easily by listening to the sound effects—they resemble a Geiger counter depending on how accurate the solution is. |

Motion synchronization | Model 2 (The Bouncing balls) requires the students to compare and identify differences in bouncing frequencies of several balls, that bounce at different time offsets. |

Spacial memory | The interior and exterior of Model 5 (The Cohen-Sutherland’s thimble) cannot be seen at the same time, so students must alternate between two viewpoints and remember objects’ positions in space. |

Chaining transformations | In model 9 (The flight of the butterfly) students construct a chain of geometrical transformations in order to define motion along a path—they do this by imagining the effect of the transformations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Boytchev, P.; Boytcheva, S. Gamified Evaluation in STEAM for Higher Education: A Case Study. *Information* **2020**, *11*, 316.
https://doi.org/10.3390/info11060316

**AMA Style**

Boytchev P, Boytcheva S. Gamified Evaluation in STEAM for Higher Education: A Case Study. *Information*. 2020; 11(6):316.
https://doi.org/10.3390/info11060316

**Chicago/Turabian Style**

Boytchev, Pavel, and Svetla Boytcheva. 2020. "Gamified Evaluation in STEAM for Higher Education: A Case Study" *Information* 11, no. 6: 316.
https://doi.org/10.3390/info11060316