Next Article in Journal
AES–CP–IDABE: A Privacy Protection Framework against a DoS Attack in the Cloud Environment with the Access Control Mechanism
Previous Article in Journal
Studying the Influences of Bus Stop Type and Specifications on Bicycle Flow and Capacity for Better Bicycle Efficiency
Previous Article in Special Issue
Prediction Framework with Kalman Filter Algorithm
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Implementation of Gamification in Polish Companies—Stages, Elements, Ethics

Department of Organizational Behaviors, Cracow University of Economics, 31-510 Kraków, Poland
Information 2020, 11(8), 371; https://doi.org/10.3390/info11080371
Submission received: 3 June 2020 / Revised: 17 July 2020 / Accepted: 18 July 2020 / Published: 22 July 2020
(This article belongs to the Special Issue Cloud Gamification 2019)

Abstract

:
Business gamification has been gaining in popularity in Poland in recent years and is indeed appearing in companies, especially large ones. However, the implementation of game-based solutions is still not sufficiently described. The technology allows the use of solutions such as AI or Machine Learning, but gamification is not only an IT project. The aim of the article is to determine the stages of implementation of business gamification according to various models, describe the existing differences and confront the results with business practice in Poland. To this end, a scoping review on the subject was carried out in terms of the existing methodologies for the implementation of gamification solutions. In the next stage, a scenario was created to conduct individual in-depth interviews (IDI) with companies implementing gamification projects in business. As a result of the research, the practice of implementing business gamification in Poland was described against the background of the methodologies proposed in the literature. This has led to the identification of several significant differences in implementation stages both between theory and practice and among the implementations proposed by the companies participating in the interviews. An attempt was made to explain these differences by taking the type of IT solution as a criterion.

1. Introduction

Extensively quoted Gartner’s report “Gamification 2020: What Is the Future of Gamification?” indicated that gamification at the beginning of the 2020s would be used primarily in employee training, education, customer engagement and personal development. The author predicted that the hype around gamification would stop, but it would become a tool that would be slowly incorporated into everyday life [1]. The supporters of gamification indicate that it is a new, irreversible trend in motivating contemporary workers, and its elements are used in management by most large Western companies [2]. The authors point out that more and more contemporary organizations are introducing gamification elements into their activities in order to engage their staff to perform their tasks more efficiently and effectively [3]. In the literature and statements of practitioners, there are more and more business areas for which using gamification has been indicated as reasonable. This includes knowledge sharing [4], human resources management [5], marketing, sales, project management [6], team management [7]. It is even pointed out that with the help of gamification, it is possible for the employees to be occupied by their work, just like a player is while playing [8].
On the other hand, solution skeptics see more risks than benefits. Some authors perceive gamification only as an additional source of control of employees [9] or their exploitation, calling the projects exploitationware [10]. Others accuse such solutions of interfering with users’ privacy by collecting personal information about employees [11]. Gamification projects are also accused of being forced to experience positive emotions [12] or of being too trivial and naive, calling this pathology of projects a playfication [13]. On the other hand, some projects are accused of focusing only on technical aspects of the game, such as points, levels, badges, rankings, use of schematic models, which can be described as punctification [14]. Implementation of gamification, in particular in a business context, is a very complex project that should also take into account aspects such as risk assessment, industrial safety, and business innovation [15,16]. Therefore, some authors indicate that the gamification practice is not sufficiently developed and people approach it too optimistically based on limited research results [17,18,19].
Similar indications appear in Polish sources. The literature of the subject suggests a small recognition of the tool in Poland [20]. It is indicated that in the practice of Polish companies, gamification projects are implemented experimentally [21] and are still at an early stage of development. One of the important points is the issue of matching elements of gamification to employees. According to F. Tondello et al., linking gamification to the motivation of individual participants is more effective than creating a gamification campaign generally addressed to an unidentified audience [22]. However, it is emphasized that gamification as a new fashion has not yet gained sufficient empirical confirmation proving its effectiveness. Little research has been carried out to indicate the relationship between the game elements used (e.g., points and progress, competition, story and narrative) and the environment and context in which the elements are to be implemented [23].
In order to achieve the aim of the article, which is to determine the stages of implementation of business gamification taking into account its key elements, the following research questions were asked:
Q1.
What methodologies for implementing business gamification are present in the literature and what are their main features?
Q2.
What stages of implementation of gamification for business purposes can be distinguished?
Q3.
What stages of implementation can be distinguished in projects implemented in Poland by companies implementing business gamification?
The structure of the article is as follows: Section 2. “Materials and Methods” presents the methodology for conducting a scoping review in Section 2.1 and in-depth interviews in Section 2.2. Section 3. “Results and discussion” describes the results of conducted research and the resulting conclusions. Section 3.1. “Implementation of gamification based on literature” contains results on scoping review. Section 3.2. “Implementation of gamification in business practice” presents the results of in-depth interviews and a comparison of methodologies for implementing gamification in enterprises to framework identified in the literature.

2. Materials and Methods

The research was divided into two stages. To verify the research questions Q1, Q2 a scoping review approach was used. Subsequently, an in-depth interview method was performed to answer the Q3 question. It was decided to combine the methods due to the complex nature of the studied area. Scoping review alone allows to isolate important elements and stages of gamification implementation from the point of view of the literature on the subject. The results will be used to construct an outline of the scenario of interviews conducted with enterprises that implement projects based on game elements. Individual interviews were chosen because of the freedom of expression, the ability to share full information without fear of duplication by the competition.

2.1. Scoping Review

To identify relevant gamification project methodologies, papers were distinguished based on works indexed in Scopus, Web of Science and Google Scholar. The aim was to identify global gamification models, with an emphasis on solutions proposed in Polish conditions. The search base were words “gamification” and “model”. To narrow the search and get results with models regarding the implementation of gamification in the company, the words “framework”, “enterprise” “internal” “implementation” were added to create keywords:
-
Gamification model framework,
-
Enterprise gamification model,
-
Internal gamification model,
-
Implementation gamification model.
The results were as follows: 532 papers from Scopus, 349 papers from Web of Science, 1342 papers from Google Scholar. The methodologies searched were not narrowed down to any time frame. In the next step, the results were narrowed down to open access only, which gives 1183 articles in total. Successively, all education-related results were rejected, resulting in 511 articles for further analysis, 206 articles were obtained after excluding duplicates. Models that did not meet the objectives of the study were excluded from the review. These requirements relate to the ability of the model to be used in general or business conditions, the model should include a partial or total gamification project. Within the framework of the scoping review, nineteen gamification implementation models were identified, and briefly described with the specification of their implementation stages or key elements for the implemented gamification. After analyzing the proposed solutions, they have been divided into several categories:
  • Approach: general: (the author/authors point to the universality of the solution and the possibility of its use for both business and social or educational purposes) or business (the author/authors indicate that the designed models are dedicated to the implementation of defined goals in organizations),
  • Scope: elements of the gamification project or implementation stages,
  • Tools: tools and methods that the authors indicate to use during the implementation of gamification,
  • Theories: theories, concepts or methodologies, which the author/authors indicate as the basis for their concept.

2.2. In-Depth Interviews

Stages of implementation of the gamification project and its elements identified during scoping review were used to prepare the scenario for an individual in-depth interview. The questions referred to the course of implementation, the degree of project personalization, factors influencing the costs and time of implementation, as well as people involved in the project. They also assume loose comments from respondents. The IDI scenario is attached as Appendix A to this article. The target group of the interviews were companies carrying out general or dedicated gamification projects operating in Poland. The aim was to obtain a sample close to the full population. Due to the lack of possibility to distinguish such entities on the basis of the Polish Classification of Activities (PKD), a group of respondents was used to identify the group:
  • Previous experience of the author (companies identified during the implementation of the previous project—“Scientific Inspirations—Interesting and Creative”, project ID I/1166/EK/1497/19),
  • Google search engine using keywords: gamification implementation, gamification company,
  • The snowball method, asking at each interview about the proposal of subsequent respondents.
To answer the Q3 research question, sixteen companies were identified in which the main subject of activities is the implementation of gamification. The group of identified respondents excluded consulting companies that do not participate in gamification implementations but only carry out gamification training. Companies implementing only serious games or simulations are also not included. In-depth interviews were conducted with representatives of five of them during the set period of research (January–April 2020): Gamfi, Gamehill, Great Interactive, GrywIT and Oskar Wegner. Interviews were conducted by the phone with the owners or managers of each company. Each call lasted about an hour. Companies have from 4 to more than 10 years of experience in implementing solutions based on the game elements. Each of them has experience in implementing gamification in a B2B model. For greater clarity of the article, the testing procedure is presented in Scheme 1.

3. Results and Discussion

3.1. Implementation of Gamification Based on Literature

3.1.1. Characteristics of the Models

Nineteen models identified in the first part of the study along with a division into categories are presented in Table 1. The list contains the models and their authors, ranked according to the year of publication, from the most recent proposals.
Considering the general or business approach of the models, some of the authors point to the universality of the solution and the possibility of its use for both business and social or educational purposes [24,25,27,31,32,33,34,36,41,42]. In other papers, it is indicated that the designed models are dedicated to the implementation of defined goals in organizations [26,28,29,30,35,37,38,39,40].
Considering theories, concepts or methodologies, which the authors indicate as the basis for their concept, most of the papers highlight psychological issues, particularly the motivations and needs of users, indicating the need to include SDTs (Self Determination Theory) [43] as a basis for shaping internal and external motivation [25,26,32,38,41,42]. Some of the solutions are based on the flow concept [44] when designing gamification solutions [25,32,34]. Few models are largely based on the solution proposed by K. Werbach and D. Hunter [28,33,39]. Several authors draw attention to ethical issues of the project [29,37,39,40]. Some articles indicate the possibility of alternatively using the MDA framework [30,34] or DMC pyramid [26,33].
Examining tools and methods that the authors indicate to use during the implementation of gamification, authors focus on tools to identify and distinguish different types of users such as Bartle Player Types test [32,33,36,40], Hexad User Types test [26,32] or Personas [24,31]. Some authors also indicate the need to evaluate projects among users by using questionnaires [27,31,41] or other techniques [29,31,41]. There is a certain insufficiency in the selection of game mechanisms and IT tools. Some authors suggest using a social engagement loop in building the game mechanics [25,31,35,37,40] or PBL’s elements [33].
Considering scope: elements of the gamification project or implementation stages, three of the identified gamification models focus on the elements of the gamification project [30,34,36], the others indicate the stages of solution implementation. Additionally, answering the first research question (Q1. What methodologies for implementing business gamification are present in the literature and what are their main features?), short characteristics of identified models were prepared, highlighting stages of implementation of gamification or its elements distinguished in them.
GameLog Model proposed by M. Klevers M, M. Sailer, W.A. Günthner consists of three stages: Analysis and Exploration, Design and Realization and Evaluation and Reflection. The analysis and exploration stage consists of such elements as: problem description, goal definition, analysis of basic conditions. This requires a list of all the steps that are taken by the employee during the process, so that after the implementation of the solution, the employee can choose whether he wants to participate in a gamified or traditional form of the process. Additionally, at this stage, the organizational structure and currently used IT solutions are analyzed. In the design and realization stage, one can distinguish between game mechanics selection, game elements selection and realization. The choice of gamification mechanisms in this context consists of the selection of mechanisms that are tailored to the goals set and are intended to change user behavior in this direction. Mechanics can be understood as, for example, cooperation, competition, feedback. The next step is to choose the elements of the game that reproduce the desired mechanics. These may include points, badges, leaderboards. The realization consists of the development of a game framework development, taking into account the goals and mechanics. At this stage, the dynamics of gamification and the storyline are established—a subject adapted to the employees’ interests. Another element is the selection of IT tools and physical elements of the project, which will be integrated with the existing IT environment. The last element of this stage is the implementation of the system, including tests, during which changes in the scope of IT solutions may take place and the inclusion of the running application in the business process. The implementation is completed by the third stage: evaluation and reflection. The aim of this stage is to measure the assumed results on the basis of key measures of success and acceptance of the project among employees based on surveys [27].
The model proposed by M. Raftopoulos, Sustainable Gamification Design (SGD), consists of five parts: A: discover, B: reframe, C: envision, D: create E: values/ethics. Stage A consists of such elements as: establishing the needs and objectives of the project and the ethical basis, defining the methods and expected results of the project, as well as the characteristics of the project participants and the expectations of the stakeholders. The first stage also aims to familiarize participants and stakeholders of the gamification project. It also establishes the values and the ethical framework, which are a reference point during the project implementation. As examples of methods used to achieve the objectives of this stage the author gives a Motivations–Methods–Outcomes framework (Berdichevsky and Neuenschwander 1999); Conceptual–Empirical–Technical method (Friedman et al. 2008); Envisioning Cards (Friedman and Hendry 2012); Method Cards (IDEO 2003); Grow A Game Cards (Flanagan and Nissenbaum 2007). Stage B consists of creative problem solving and idea creation through participation and co-design. The aim of this stage is to analyze the information collected in the first stage and to create a framework for potential solutions on its basis. The methods proposed to achieve the objectives of this stage are Ideation, Scenarios, Storyboarding, Gamestorming, Envisioning Cards (Friedman and Hendry 2012), Method Cards (IDEO 2003), Grow A Game Cards (Flanagan and Nissenbaum 2007), Values and Ethics Checklist (Berdichevsky and Neuenschwander 1999). In stage C, the most appropriate technology options used for the project are determined, the gameplay is determined, and the game mechanics are selected. The aim of this stage is to select the preferred solution and define its scope. The author proposes to use the following methods for this stage: Storyboarding, conceptual modeling, wireframing, game design document, Game Design Cards (Schell 2008), Grow A Game Cards (Friedman and Hendry 2012), Values and Ethics Checklist (Berdichevsky and Neuenschwander 1999). Stage D includes prototyping, piloting, iterating and launching the gamification application. The aim of this stage is to design and launch a gamification solution. The author suggests using such methods as: Agile methodologies in software and game design need to be applied here; values and ethics checklist. Systems dynamics methodologies also provide valuable tools to endure that effective gamification systems are developed. Stage E, i.e., values and ethics, are at the heart of the model. Defined in stage A, they are visible in each subsequent phase of the project [29].
The Canvas Model Gamification consists of nine sections containing, according to S. Jimenez, key elements of gamification, such as: Revenues, Players, Behaviors, Aesthetics, Dynamics, Components, Mechanics, Platforms, Costs. Revenues is about describing the economic or social effect of a gamification solution. The Players stage refers to the description of the target group for which we want to develop behavior. Behaviors concerns the description of behavior or actions that are necessary for the players to achieve a return from the project. Aesthetics should describe the desired emotional reactions of the player when interacting with the game, the author points out the link between this point and the fun while playing. Within Dynamics, the changes of mechanics affecting the participant during the project should be described, the author points out the link between this element and motivation. Within Components, the elements or characteristics of the game should be described in order to create mechanics or to determine how to give feedback to players. Mechanics describes the rules of the game, which together with its components create dynamics. The Platforms element refers to IT tools that will allow the implementation of game mechanics. The Costs stage should include the key costs of project implementation [30].
D. DiTommaso distinguishes seven stages of the gamification project: Why gamify, Player profile, Goals and objectives, Skills and actions, A look through lenses of interest, Desired outcomes, Play-test and Polish. In the first stage the business objectives of gamification, actions, behaviors to be performed by the participants to achieve the goals are defined. In a second stage, the profile of the participant is defined, in a third stage, intermediate objectives are defined. The fourth stage consists in drawing up a list of desired behaviors, skills acquired by the participants and defining the ways of measuring them. The fifth stage defines the mechanics of the game, and in the sixth stage, the author points out the key role of feedback in achieving the main goal of the project. The seventh stage involves testing the project, implementation and continuous improvement [42].
The model created by K. Werbach and D. Hunter, Six steps to Gamification, consists of six elements and is commonly known as 6D. First, the model assumes the definition of business objectives, then the definition of desired behaviors and actions of users, together with the definition of ways to measure them. In the third stage, the model assumes the description of players. The fourth element is to create an engagement loop, a mechanics that will engage users. The fifth element focuses on fun, the authors propose to consider what will motivate users to participate in the project after eliminating external prizes. The last element assumes the selection of IT tools and re-verification of the assumed business objective [40].
K. Julius and J. Salo rely on this model in their model—a framework for gamification suited for marketing, adding the Market research stage to the distinguished 6D stages [39].
Similarly, B.M. De Paz’s “8 steps to gamification” model is similar to K. Werbach, D. Hunter’s “Six steps to gamification” model. The author distinguishes three stages: Prepare, Design, Implement and Maintain, assuming successively: in the first stage: setting business goals, defining the desired effects of the project and describing the users to whom the project is addressed. The second stage involves designing the solution, including determining the desired behaviors, components, dynamics and mechanics of the game; the last stage refers to building a real application, implementation, data collection and continuous system customization [33].
A. Marczewski’s model “GAME Framework Planning and Building a gamified solution” consists of four stages: Gather, Act, Measure, Enrich. The first one includes the collection of information about gamification goals, users, and measures to verify the project. Then the model assumes creating a solution based on the information obtained and the project test. In the third stage, the author points out the need to measure users’ activity and achieve the assumed goals, implement improvements. The last stage involves supporting and enriching the project during its duration [32].
In 2017, Marczewski described changed assumptions of gamification projects. The new methodology—Gamification Design Framework, consists of three stages: Define, which consists in identifying the problem, the user, and the measures of success; Project, which includes a description of the user’s interaction with the system—the user’s journey, as well as social engagement loops; and Build and Refine, which describes the implementation and improvement of the system [25].
A. Francisco-Aparicio F. Luis Gutiérrez-Vela, J.L. Isla-Montes, J.L. González Sanchez in their model distinguish four stages of gamification implementation: Identification of the main objective, Identification of the transversal objective, Selection of game mechanics support and effectiveness analysis. The method assumes the following: In the first stage, identify the business objective to be achieved by implementing the gamification project. The second stage involves setting one or more objectives that are important for the participants in the project. In the third stage, project elements are selected on the basis of established objectives, taking into account the needs arising from the SDT theory. The fourth stage assumes the implementation of the project and the analysis of its effectiveness through questionnaires, expert analysis with the use of heuristic techniques or comparison of the achievement of business objectives before and after the implementation of the gamification solution [41].
C. Marache-Francisco and E. Brangie in their gamification design process are relying on Human–Computer Interaction (HCI) principles. They define three dimensions, which, according to the authors, should be used to define a clear framework for the project: sensory-motor dimensions, emotions, motivation and involvement and the cognitive dimension of interaction. On the basis of these dimensions they divide the gamification project into two stages: context analysis (user-oriented design) and creation of an iterative project concept [31].
D. Robinson and V. Bellotti distinguish six basic categories of gamification design elements in their taxonomy: General Framing—the purpose of the gamification project outside the game, for example, social, business and storyline goals; General Rules and Performance Framing—goals of the game that clearly define what a good score means in the context of a given gamification project; Social Features—elements enabling or facilitating interaction between user, Incentives—external prizes and motivators, which are to be perceived by the user as internal; Resources and Constraints—time constraints and other boundaries that the user must observe; Feedback and Status Information indicating the user’s current position, what else he must do, or how he/she compares with other participants in the gamified system [34].
H. Jacobs distinguishes five stages in the design of a gamification solution: setting business goals, bringing the participants’ profiles closer, considering the user and the role of social media in the project, feedback and analysis, and launching a social engagement loop [35].
J. Kumar describes the design of gamification solutions in eight steps: The first is to understand the player, the next step is to understand the mission of the project, then to determine the motivation of users, to apply the game mechanics, to establish the rules of the game, to define the engagement loop, the next step includes management, monitoring and measurement, the last step is to consider legal and ethical issues [37].
B.K. Neeli has created a model for designing gamification for business purposes, assuming that the project is implemented in six stages: (1) setting the project’s objective and subobjectives, (2) determining the project’s challenges and motivation for its implementation, (3) analyzing users’ motivation, (4) designing a gamified system, (5) implementing, measuring effects and improving the system, (6) adding additional elements involving users [28].
Key elements of gamification solutions according to Y-k. Chou are (1) epic meaning and calling (2) development and accomplishment, (3) creativity and feedback, (4) ownership and possession (5) social influence and relatedness, (6) scarcity and impatience, (7) unpredictability and curiosity (8) loss and avoidance [36].
A gamification model created by D. Gears and K. Braun consists of six elements: objectives, business rules, behavioral norms, preconditions, subjects and course of action [38].
The model created by E. Wiklund and V. Wakerius is based on existing gamification models in the literature. It assumes seven stages of creating a gamification solution: defining business objectives, identifying target, matching player with motivation, building the experience, considering journey, testing, control and adjusting [26].
A. Uchoa, E. Fernandes, B. Fonseca, R. de Mello, C. Barbosa, G. Nunes, A. Garcia, L. Teixeira distinguish six stages of the gamification project in their model. First stage—Preparation System, which assumes setting gamification goals, prioritizing them, describing precisely and rationally justifying the goals. The second stage—User Analysis, characterizes the recipients of the created gamification system, determines their needs and motivations, and the third stage—Context Analysis, characterizes the existing system in terms of IT solutions, process flow, key features and possibilities of their inclusion in the gamification project. The fourth stage—Requirements Elicitation consists of establishing and describing the functional and non-functional requirements of the gamification system, defining the rules of the game and how to incorporate them into the current system. The fifth phase—Gamification Design consists in designing the assumptions of the gamification system, the game elements used and linking them to the previous assumptions. The last stage—Software Design, aims to create an interface, a project layer visible to the user [24].

3.1.2. Gamification Implementation Stages

Within the proposed solutions, the authors distinguish from two [31] to eight [37] stages of the gamification project. It is worth noting that these stages differ in terms of the level of detail. However, after the analysis, it can be concluded that there is a clear similarity between them in the order of the implemented activities, which allows the answering of research question Q2. What stages of implementation of gamification for business purposes can be distinguished? On the basis of the presented models, it is possible to distinguish following general implementation stages:
  • Goal setting,
  • Identification of users,
  • Substantive and graphic design of the game play,
  • Selection of IT solutions,
  • Testing and implementation,
  • Sustaining.
It should be noted that the order of the stages varies depending on the model, and three of them do not describe the sustaining stage, as described in more detail later in the article.
All methodologies assume the establishment of project goals outside the goal of the game, which results directly from the definition of gamification. Most of them indicate this as the first stage of the implementation of gamification (apart from J. Kumar’s proposal, which assumes first of all to describe users [37]). Some of the proposals assume the establishment of superior goals and intermediate goals (related, for example, to the change of employees’ behavior). The authors suggest that in the first stage, the main goal of the project, specific goals and the way to measure them, for example, using KPIs (Key Performance Indicators), should be established [27].
Each model also assumes the need to characterize the system end users, employees, customers or other target groups. There is only one proposal to describe the users before setting gamification goals [37]. Most models assume the UCD (User Centered Design) approach, but this is not visible in the rules or limitations on design or IT tools, but only emphasizes that man should be at the center of gamification projects. Some models assume the description of end users taking into account the characteristics of individuals [25,26,32,40]. For this purpose, the authors suggest using such tools as the Hexad user type test or Bartle Player type test. They motivate their approach by the fact that not everyone can be effectively motivated by the same mechanisms. They suggest personalization as an approach to game design, emphasizing, however, that this has not been sufficiently researched so far [45]. The remaining models assume a general description of the target group with an average of age, needs or motivation or the creation of Persona [24,31].
The next stage in the implementation of gamification based on the literature can be defined as designing a gamification solution that takes into account the establishment of the rules of the game, selection of elements and mechanics taking into account the business objectives and/or users, definition of an engagement loop. This stage is most different in the presented solutions, because the authors base on different models of game design or do not directly describe the methodology chosen. Some authors refer to the MDA [30,34] (Mechanics–Dynamics–Aesthetics) or the DMC pyramid described in the 6D model (Dynamics, Mechanics, Components) [26,33].
The next stage focuses on choosing an IT solution. It is rather briefly described in the models, two of them emphasizing the need to integrate the solution with the existing IT system in the organization [24,27]. One of the models proposes an analysis of the possibilities of using existing IT solutions [24].
The next stage involves solution testing and implementation. Some solutions stop at the stage of a tested gamification project because they refer to a design framework or gamification elements rather than a holistic implementation model [30,34,36].
Emphasizing that gamification solutions require continuous control after implementation and maintenance for part or the entire duration of the project, the literature indicates as the last stage of implementation the maintenance and ongoing analysis of user activity and achievement of the assumed objectives. Some models provide for the assessment of users by conducting surveys among them [24,41].
It is worth quoting here the works of other authors who have previously made similar comparisons of gamification models by narrowing them down according to different criteria [46,47,48].

3.2. Implementation of Gamification in Business Practice

The next part of the article presents conclusions concerning the implementation of gamification, formulated on the basis of conducted interviews. This part attempts to answer research question Q3. What stages of implementation can be distinguished in projects implemented in Poland by companies implementing business gamification? The presented results refer to the implementation of cloud gamification solutions in the business context.
For research purposes it was assumed that the implementation time is counted from the positive decision to introduce gamification by the customer. Based on the interviews, several stages of gamification implementation have been identified, which are described below, highlighting the differences between them.
As the first stage of the implementation of the gamification project, we can distinguish the establishment of the main goal of gamification, generally understood as, for example, increasing sales results, shortening the process of onboarding, changing the way of communication, changing the organizational culture. At this point, a workshop is usually conducted with the client, which allows to get acquainted with the operational teams responsible for the project, specify the objectives, set the desired levels of their implementation and analyze the target group. In some cases, workshops are omitted, and the client provides the target levels of goal accomplishment, which can be set individually for each employee. Parallel to this work, legal and formal issues, such as agreements on the entrustment of personal data, may be carried out on the customer side.
In the next stage, a draft of the gamification project is prepared, including the strategy for engaging users, basic mechanisms and/or the story proposal. Such a draft, once accepted by the client, is used to plan the project in detail: to determine the story and types of tasks to be performed by the user. Some of them may not be directly related to the business purpose, but only serve to have fun or introduce the story.
The next stage is related to the platform and at this point the most differences between individual implementations appear, it can be stated that the differences in the implementation stages of gamification are mainly due to the IT solution used; we can distinguish:
  • Dedicated solution—creating an online platform to meet customer needs,
  • Ready-made IT solution—implementation of the designed content.
Dedicated solutions create the need to distinguish the technical stage including such activities as: platform creation, IT solution testing, IT infrastructure configuration. With higher levels of integration with existing customer systems, it may be necessary at this stage to create two-way communication between the solution created and the customer systems. In the case of ready-made IT solutions, this stage is omitted.
The prepared solution is then tested on-site.
The stage that is not featured in the literature should be emphasized but has been identified by some companies as a separate one—the marketing stage, also called the recruitment stage, which involves promoting the solution among the target group. This is usually done using the customer’s existing communication channels. Although participation is declared voluntary, it should be stressed that some activities or business processes require participation due to the fact that an alternative, old version of a given process ceases to exist; especially in the case of processes such as onboarding or employee training.
In the next stage, the platform is launched, and the first users log in. However, this step cannot be clearly considered as the completion of implementation. In the vast majority of cases, the launch is only a step, followed by the maintenance stage, which includes the current operation of the platform, analysis of the implementation of the assumed business objectives. Some solutions even assume designing only a fragment of the gamification and then adding the story, and selecting tasks depending on the analysis of user activity during the project. The success of the implementation of gamification is measured individually for each project. Its primary determinant is whether business objectives have been achieved. However, they are not always quantified, but only in general terms, for example, as an increase in sales, without percentages. The most basic measures of the project’s success can then be considered:
  • Percentage of the target group that has registered on the platform or entered it at least once;
  • Percentage of the target group that completed the gamification path or was active within the given project duration.
In addition, in the case of long-term gaming projects, success is measured by the number of actions taken by users, and evaluation surveys are used among gaming users.
In the following part of the article, additional assumptions and key elements of the gamification projects are presented, which were separated on the basis of conducted interviews.
The comments on the use of the story in gamification projects are divergent. Each of the companies has made a story implementation, according to some remarks, the enclosing of the project in a story is a key element of gamification, according to others, the validity of the story depends on the specificity of the project. In this case, companies recommend using it mainly when the tasks to be performed within the project are repetitive and tedious, in order to make them more attractive, for example, if the aim is to increase the efficiency of work in the customer service department. According to some of the interlocutors, the choice of the story should take into account such aspects as: business goals, client’s organizational culture, basic characteristics of the target groups and their everyday environment.
The identification of the user target group is indicated as one of the key elements for the successful implementation of gamification, but not at the level of an individual person, but of the entire user group or subgroups. Data such as age, gender and habits are important and form the basis for choosing a communication strategy. For some projects, cultural differences due to nationality are also important. There was one comment regarding the choice of game mechanisms in terms of demographic data, for example, tasks stimulating competition selected for the target group where men predominate.
It can be said that for the interlocutors the key to choosing the types of tasks and other elements of the game are selected mechanisms such as competition, cooperation, which depend on the desired user behavior in the real world. These behaviors, in turn, directly result from the established business objectives of the gamification project. The characteristics of the target user are less important in the selection of game elements.
According to the interlocutors, every action or business process in which a person participates can be gamified. Not in every case, however, will this be justified. Ethical issues are an important aspect of gamification implementations. Some of the objectives to be achieved through the implementation of gamification may be questionable. An example is the sales performance improvement project, in which office workers who do not sell products or services in their daily duties are also to be involved through gamification.
According to the interviews, most often implementations are performed in large enterprises, which is due to relatively high implementation costs, which range on average from PLN 150,000 to PLN 500,000 per year. A cheaper solution may be to buy access to a ready-made IT solution, a platform that can be used to create gamification, but without substantive support and additional functionalities. The annual cost of access is several thousand PLN for up to 50 users. There are also implementations in the SME sector using EU grants by the customer for this purpose. The second reason for the popularity of implementations in large entities is the willingness to engage employees, provide them with feedback on the work done combined with the need to standardize solutions. Polish companies’ awareness of gamification is assessed as very low, most of the people taking part in the project on the client’s side at the beginning of the implementation do not fully understand what such solutions are about. An interesting solution is to purchase only access to the ready-made gamification platform. However, this raises the question about the effectiveness of such solutions. Earlier studies co-created by the author on large enterprises in Poland also indicate that gamification is a little-known notion, only by 20.6% of respondents. An external company was used in the companies where it was implemented. It would be interesting to compare the effectiveness of gamification implemented by an external company and projects implemented in terms of their content entirely by the client, only with the purchase of access to the platform, both in terms of achieved business objectives and the involvement of the target group in the project.
Two-phase sequential (facilitation) studies were conducted, in which the results of one study are grounds for the implementation of the next study. Although, the limitation of the study is the lack of quantitative methods. The proposals for the implementation stages themselves require verification, which could take place through participatory studies.

4. Conclusions

The implementation of gamification in practice by companies implementing business solutions in Poland shows some similarities but also a few significant differences to models described in the literature.
In terms of highlighted implementation stages, business practice includes in projects all phases described in the literature. The main difference in this aspect between the literature on the subject and the practice is the marketing/recruitment stage, which is separated by the companies and includes the promotion of the solution among the target group. Literature mentions the need to encourage employees to participate in the project, but does not distinguish an additional phase for this purpose. The second important difference is the course of the IT solution selection stage. This phase is described quite generally in the literature. In business practice, more emphasis is placed on it, because the choice of a standardized or dedicated IT solution significantly affects the level of project complexity, which impacts on costs and duration of implementation. Researchers devote more space in publications to the stages related to the issue of motivation and theories underlying their implementation methodologies.
In terms of methods used, the implementations in business practice are not entirely based on any of the gamification methodologies available in the literature. Companies emphasize that they rely mainly on lessons learned, experience and intuition of their employees. However, the influence of some theories, methods or tools (SDT, flow and Personas) is clearly visible.
In terms of the overall approach both in literature and business practice, the man is highlighted as the center of the project. In practice, however, no tools are used to analyze player types. The personalization of solutions is done in terms of the general characteristics of the target group. Matching some elements of the game to individual people takes place already during the maintenance stage of the project based on the analysis of their activity, using AI and machine learning.

Funding

Publication was financed from the funds granted to the Cracow University of Economics, 120/ZZZ/2020/PRO.

Conflicts of Interest

The author declares no conflict of interest.

Appendix A

  • Describe the implementation phases—are they similar for every implementation or depend on some factors?
  • What elements are taken into account during all of your gamification implementations (e.g., corporate culture, personality of users)?
  • Are there any elements considered only in specific cases?
  • Who is involved in gamification implementation from enterprise and from Gamfi side (positions, responsibilities)?
  • What is the average time of implementation—what are the key elements affecting implementation time?
  • Which elements of the enterprise and its culture are taken into account during designing the gamification project (e.g., existing knowledge base, intranet)?
  • Based on your experience, to which of the business areas or processes gamification shouldn’t or cannot be implemented?
  • How do you developed the methodology for implementing gamification—is it based on your experience or any scientific research, if so, which ones?
  • How do you choose the components of the gamification (e.g., badges, points, leaderboards)? What the selection depends on?
  • Do you describe the behaviors or actions necessary to develop in players in order to get a return on the project (e.g., watch video, answer survey)?
  • Do you describe who and what the people are like to whom you want to develop behaviors? What are the key factors (e.g., habits, age, personality)?
  • How relevant is the story in the gamification implementation? How do you create the story? Is it universal or tailored? (What is it tailored to, e.g., user, corporate culture)?
  • What budget (in average) a company needs to implement gamification? Describe the main cost categories.
  • How do you measure the success of the gamification?
  • Do you increase the difficulty of mechanics over time?
  • When you consider the implementation complete (e.g., is it the time when the platform/gamification is launched or after agreed time)?
  • Do you have any other thoughts about implementing gamification?

References

  1. Burke, B. Gamification 2020: What Is the Future of Gamification? Gartner: Stamford, CT, USA, 2012; pp. 1–7. [Google Scholar]
  2. Osowiecka, M. Grywalizacja. Polityka 2014, 35, 22–24. [Google Scholar]
  3. Klub Trenerów Biznesu. Available online: www.klubtrenerowbiznesu.pl/grywalizacja.html (accessed on 20 April 2020).
  4. Swacha, J. Gamification in knowledge management: Motivating for knowledge sharing. Pol. J. Manag. Stud. 2015, 12. [Google Scholar]
  5. Bombiak, E.; Cisek, M. Grywalizacja jako innowacja w obszarze funkcji personalnej. In Współczesne Wyzwania w Zarządzaniu Zasobami Ludzkimi; Domańska-Szaruga, B., Bombiak, E., Eds.; Wydawnictwo Naukowe Uniwersytetu Przyrodniczo-Humanistycznego: Siedlce, Poland, 2019; pp. 29–48. [Google Scholar]
  6. Bielecki, W.T. Gamification-Learning by Doing and Fun? Eкoнoмiчний Нoбeлiвcький Вicник 2014, 1, 35–40. [Google Scholar]
  7. Cybulski, A. Czym nie jest Grywalizacja? Available online: https://www.linkedin.com/pulse/czym-nie-jest-grywalizacja-arkadiusz-cybulski (accessed on 10 March 2020).
  8. Cybulski, A. Grywalizacja jako Sposób na Zaangażowanego Pracownika? Available online: http://hrpolska.pl/hr/narzedzia/grywalizacja-w-firmie-odpowiedzi-na-kryzys-zaanga-owania-zespo-u.html (accessed on 15 April 2020).
  9. Lopez, S. Disneyland Workers Answer Is “Electronic Whip”. Los Angeles Times. Available online: http://articles.latimes.com/2011/oct/19/local/la-me-1019-lopez-disney-20111018 (accessed on 10 March 2020).
  10. Bogost, I. Gamification Is Bullshit: My Position Statement at the Wharton Gamification Symposium. 2011. Available online: http://www.bogost.com/blog/gamification_is_bullshit.shtml (accessed on 10 February 2020).
  11. Whitson, J.R. Gaming the quantified self. Surveill. Soc. 2013, 11, 163–176. [Google Scholar] [CrossRef]
  12. Mollick, E.; Rothbard, N. Mandatory fun: Consent, gamification and the impact of games at work. Whart. Sch. Res. Pap. Ser. 2014. [Google Scholar] [CrossRef]
  13. Czerska, I. Grywalizacja miejsca pracy jako innowacyjne podejście w motywowaniu personelu. Mark. Manag. 2017, 1, 253–260. [Google Scholar] [CrossRef] [Green Version]
  14. Robertson, M. Can’t Play, Won’t Play. Available online: http://kotaku.com/5686393/cant-play-wont-play (accessed on 12 February 2020).
  15. Algashami, A. Gamification Risks in Collaborative Information Systems: Identification and Management Method. Ph.D. Thesis, Bournemouth University, Bournemouth, UK, 2019; pp. 150–153. [Google Scholar]
  16. Pamfilie, R.; Procopie, R.; Giușcă, S.; Vasilcovschi, A. A Case Study of Business Model Innovation: Building Value through Gamification. In Proceedings of the 6th BASIQ International Conference on New Trends in Sustainable Business and Consumption, Messina, Italy, 4–6 June 2020; Pamfilie, R., Dinu, V., Tăchiciu, L., Pleșea, D., Vasiliu, C., Eds.; ASE: Bucharest, Romania, 2020; pp. 1257–1262. [Google Scholar]
  17. Hamari, J.; Koivisto, J.; Sarsa, H. Does gamification work? A literature review of empirical studies on gamification. In Proceedings of the 47th Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 6–9 January 2014. [Google Scholar]
  18. Huotari, K.; Hamari, J. Defining gamification—A service marketing perspective. In Proceedings of the 16th International Academic MindTrek Conference (17–22), Tampere, Finland, 3–5 October 2012; ACM Press: New York, NY, USA, 2012. [Google Scholar]
  19. Witoszek, A. Dobór mechanizmów gier do zwiększania zaangażowania i motywacji pracowników. J. Manag. Financ. 2019, 17, 35–47. [Google Scholar]
  20. Kozłowska, I. Gamifikacja–specyfika wykorzystania narzędzia w Polsce. Studia Ekonomiczne 2016, 255, 37–45. [Google Scholar]
  21. Winnicka-Wejs, A. Grywalizacja jako przykład eksperymentu w obszarze. In Eksperyment w Obszarze HR; Lipka, A., Waszczak, S., Winnicka-Wejs, A., Eds.; CeDeWu: Warsaw, Poland, 2015. [Google Scholar]
  22. Tondello, F.; Wehbe, R.; Diamond, L.; Busch, M.; Marczewski, A.; Nacke, L. The Gamification User Types Hexad Scale. Available online: https://dl.airtable.com/MxccwJaeSeazUxqlZahG_The-Gamification-User-Types-Hexad-Scale.pdf (accessed on 14 February 2020).
  23. Skok, K. Punktyfikacja, playifikacja, hazardyzacja… Analiza skuteczności mechanizmów motywacyjnych stosowanych w procesie gamifikacji. Homo Ludens 2017, 1, 193–216. [Google Scholar]
  24. Uchoa, A.; Fernandes, E.; Fonseca, B.; de Mello, R.; Barbosa, C.; Nunes, G.; Garcia, A.; Teixeira, L. On Gamifying an Existing Healthcare System: Method, Conceptual Model and Evaluation. In Proceedings of the 2019 IEEE/ACM 1st International Workshop on Software Engineering for Healthcare (SEH), Montreal, QC, Canada, 27 May 2019. [Google Scholar] [CrossRef]
  25. Gamified.UK. Available online: www.gamified.uk/downloads/gamification-design-framework-workbook/ (accessed on 10 February 2020).
  26. Wiklund, E.; Wakerius, V. The Gamification Process: A Framework on Gamification. Master’s Thesis, Jonkoping University, Jonkoping, Sweden, 2016. [Google Scholar]
  27. Klevers, M.; Sailer, M.; Günthner, W.A. Implementation Model for the Gamification of Business Processes: A Study from the Field of Material Handling. In Simulation and Gaming in the Network Society; Translational Systems Sciences; Kaneda, T., Kanegae, H., Toyoda, Y., Rizzi, P., Eds.; Springer: Singapore, 2016; Volume 9, pp. 173–184. [Google Scholar] [CrossRef]
  28. Neeli, B.K. Gamification in the Enterprise: Differences from Consumer Market, Implications, and a Method to Manage Them. In Gamification in Education and Business; Springer International Publishing: Cham, Switzerland, 2015; pp. 489–511. [Google Scholar]
  29. Raftopoulos, M. Towards gamification transparency: A conceptual framework for the development of responsible gamified enterprise systems. J. Gaming Virtual Worlds 2014, 6, 159–178. [Google Scholar] [CrossRef]
  30. Gamasutra. 2013. Available online: https://www.gamasutra.com/blogs/SergioJimenez/20131106/204134/Gamification_Model_Canvas.php (accessed on 15 April 2020).
  31. Marache-Francisco, C.; Brangier, E. Process of gamification. From the consideration of gamification to its practical implementation. In Proceedings of the 6th International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services CENTRIC, Venice, Italy, 27 October–1 November 2013. [Google Scholar]
  32. Marczewski, A. Gamification Frameworks. Available online: https://www.slideshare.net/daverage/gamification-frameworks (accessed on 16 April 2020).
  33. de Paz Merino, B. Gamification: A Tool to Improve Sustainability Efforts. Master’s Thesis, University of Manchester, Manchester, UK, 2013. [Google Scholar]
  34. Robinson, D.; Bellotti, V. A preliminary taxonomy of gamification elements for varying anticipated commitment. In Proceedings of the CHI 2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences, Paris, France, 28 April 2013. [Google Scholar]
  35. Jacobs, H. Gamification: A Framework for the Workplace. Ph.D. Thesis, Universityof Liverpool, Liverpool, UK, 2013. [Google Scholar]
  36. Chu, Y.-K. Octalysis—The Complete Gamification Framework. Available online: http://www.yukaichou.com/gamification-examples/octalysis-complete-gamification-framework/ (accessed on 1 January 2020).
  37. Kumar, J. Gamification at Work: Designing Engaging Business Software. In Lecture Notes in Computer Science, Proceedings of the Design, User Experience, and Usability. Health, Learning, Playing, Cultural, and Cross-Cultural User Experience DUXU 2013, Las Vegas, NV, USA, 21–26 July 2013; Marcus, A., Ed.; Springer: Berlin, Germany, 2013; Volume 8013, pp. 528–537. [Google Scholar] [CrossRef]
  38. Gears, D.; Braun, K. Gamification in Business: Designing Motivating Solutions to Problem Situations. In Proceedings of the CHI 2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences, Paris, France, 28 April 2013; pp. 3263–3266. [Google Scholar]
  39. Julius, K. Designing Gamification. Master’s Thesis, University of Oulu, Oulu, Finland, 2013. [Google Scholar]
  40. Werbach, K.; Hunter, D. For the Win: How Game Thinking Can Revolutionize Your Business, 1st ed.; Wharton Digital Press: Philadelphia, PA, USA, 2012; pp. 70–83. [Google Scholar]
  41. Francisco-Aparicio, A.; Luis Gutiérrez-Vela, F.; Isla-Montes, J.L.; González Sanchez, J.L. Analysis and application of gamification. In Proceedings of the 13th International Conference on Interacción Persona-Ordenador, Elche, Spain, 3–5 October 2012. [Google Scholar] [CrossRef]
  42. Di Tommaso, D. Beyond Gamification: Architecting Engagement through Game Design Thinking. 2011. Available online: https://www.slideshare.net/DiTommaso/beyond-gamification-architecting-engagement-through-game-design-thinking/48-STEP_1_WHY_GAMIFY_CRITICAL2011 (accessed on 14 February 2020).
  43. Ryan, R.; Deci, E. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 2000, 55, 68–78. [Google Scholar] [CrossRef] [PubMed]
  44. Csikszentmihalyi, M. Przepływ: Psychologia Optymalnego Doświadczenia [Flow: Psychology of Optimal Experience]; Wydawnictwo Moderator: Wałbrzych, Poland, 2005; pp. 134–150. [Google Scholar]
  45. Busch, M.; Mattheiss, E.; Orji, R.; Marczewski, A.; Hochleitner, W.; Lankes, M.; Nacke, L.E.; Tscheligi, M. Personalization in serious and persuasive games and gamified interactions. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, CHI PLAY 2015, London, UK, 5–7 October 2015; ACM: New York, NY, USA, 2015; pp. 811–816. [Google Scholar]
  46. Mora, A.; Riera, D.; Gonza, C.; Arnedo-Moreno, J. Gamification: A systematic review of design frameworks. J. Comput. High Educ. 2017, 29, 516–548. [Google Scholar] [CrossRef]
  47. Obaid, I.; Shoaib Farooq, M.; Abid, A. Gamification for Recruitment and Job Training: Model, Taxonomy, and Challenges. Inst. Electr. Electron. Eng. 2020, 8, 65164–65178. [Google Scholar] [CrossRef]
  48. Zainuddina, Z.; Kai Wah Chua, S.; Shujahata, M.; Jacqueline Perera, C. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educ. Res. Rev. 2020, 30, 100326. [Google Scholar] [CrossRef]
Scheme 1. Testing procedure.
Scheme 1. Testing procedure.
Information 11 00371 sch001
Table 1. Gamification frameworks—business-centered and general approach.
Table 1. Gamification frameworks—business-centered and general approach.
NameAuthorsApproachTheoriesToolsScopeYear
Gamification Conceptual Model [24]A. Uchoa, E. Fernandes, B. Fonseca, R. de Mello, C. Barbosa, G. Nunes, A. Garcia, L. TeixeirageneralBusiness Process Model and Notation (BPMN)Personasstages2019
Gamification Design Framework [25]A. MarczewskigeneralRelatedness, Autonomy, Mastery and Purpose (RAMP), Self-determination theory, Flow theorySocial engagement loopstages2017
Wiklund and Wakerius Framework [26]E. Wiklund, V. WakeriusbusinessRelatedness, Autonomy, Mastery and Purpose (RAMP), Self-determination theory, Dynamics, Mechanics, Components (DMC)Hexad User Types teststages2016
GameLog Model [27]M. Klevers, M. Sailer, W.A. GünthnergeneralThe Waterfall Model, Deming CycleKPI, Questionnairesstages2016
Neeli Framework [28]B.K. Neelibusiness6D-stages2015
Sustainable Gamification Design SGD [29]M. RaftopoulosbusinessAgile Methodologies, Motivations-Methods-Outcomes framework,Conceptual-Empirical-Technical method, Envisioning Cards, Grow A Game Cards, Values and Ethics Checkliststages2014
Gamification Model Canvas [30]S. JimenezbusinessMechanics, Dynamics, Aesthetics (MDA framework), Business Model Canvas -elements2013
Gamification Design Process [31]C. Marache-Francisco, E. Brangiegeneral-Observations, Interviews, Questionnaires, Diaries, Focus groups or Personas, Social engagement loopstages2013
GAME Framework Planning and Building a gamified solution [32]A. MarczewskigeneralSelf-determination theory, Flow theoryHexad User Types test, Bartle Player Types teststages2013
8 Steps to Gamification [33]B.M. De Pazgeneral6D, Dynamics, Mechanics, Components (DMC), Gamified System Feedback CycleBartle Player Types test, Points, Badges and Leader-boards (PBLs)stages2013
Robinson and Bellotti taxonomy [34]D. Robinson, V. BellottigeneralFlow theory, Mechanics, Dynamics, Aesthetics (MDA framework),-elements2013
Gamification Framework model [35]H. JacobsbusinessGoal-Model Design, Agile Methodologies Social engagement loopstages2013
Octalysis: Complete Gamification Framework [36]Y-k. Chougeneral-Bartle Player Types testelements2013
Player Centered Design Methodology [37]J. KumarbusinessUser-centered designSocial engagement loopstages2013
Role-Motivation-Interaction Framework (RMI) [38]D. Gears and K. BraunbusinessSelf-determination theory, 16 basic desires-stages2013
A framework for gamification suited for marketing [39]K. Juliusbusiness6D-stages2013
Six steps to Gamification (6D) [40]K. Werbach, D. Hunterbusiness-Bartle Player Types test, Social engagement loopstages2012
Francisco-Aparicio et al. framework [41]A. Francisco-Aparicio F. Luis Gutiérrez-Vela, J.L. Isla-Montes, J.L. González SanchezgeneralSelf-determination theoryHeuristic Techniques, Questionnaires stages2012
A Framework for Success. [42]D. DiTommasogeneralSelf-determination theory-stages2011

Share and Cite

MDPI and ACS Style

Witoszek-Kubicka, A. Implementation of Gamification in Polish Companies—Stages, Elements, Ethics. Information 2020, 11, 371. https://doi.org/10.3390/info11080371

AMA Style

Witoszek-Kubicka A. Implementation of Gamification in Polish Companies—Stages, Elements, Ethics. Information. 2020; 11(8):371. https://doi.org/10.3390/info11080371

Chicago/Turabian Style

Witoszek-Kubicka, Aleksandra. 2020. "Implementation of Gamification in Polish Companies—Stages, Elements, Ethics" Information 11, no. 8: 371. https://doi.org/10.3390/info11080371

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop