Negotiating Virtually and Face-to-Face: Experience from a Serious Game Conducted in Person and via Smartphone Application

Abstract

Serious games and negotiation simulations such as the Phosphorus Negotiation Game (P-Game) are increasingly used to support sustainability-oriented education. To broaden accessibility, a smartphone-based version of the face-to-face P-Game was developed and is presented here. A comparative design integrating quantitative pre–post survey measures with analysis of open-ended responses was employed to examine self-reported knowledge gains and learning experiences among participants who completed the P-Game in face-to-face workshops and those who played the virtual version. Both formats were associated with significant increases in participants’ perceived understanding of phosphorus science and negotiation science/practice. Self-reported knowledge of phosphorus science increased by 92.3% (global face-to-face), 70.7% (Hungarian face-to-face), and 88.4% (online), with comparable gains observed in negotiation science and practice across groups. Qualitative findings complemented these results, indicating that while learning gains were broadly similar, the modes differed in experiential emphasis: face-to-face delivery elicited performance-oriented and socially embedded reflections, whereas the online format was more frequently described in terms of structured participation and reflective processing. User satisfaction with the virtual P-Game was high, reflected by a System Usability Scale (SUS) score above 80. Overall, the findings suggest that the virtual P-Game represents a viable and accessible complement to traditional face-to-face implementation, maintaining educational impact while extending reach. Further research with larger and more diverse participant samples is recommended to strengthen generalizability and explore long-term learning outcomes in sustainability contexts.

1. Introduction

Serious games [1,2,3,4] and negotiation simulations [5,6,7,8] are becoming increasingly popular teaching tools to foster sustainable environmental education by providing an interactive and engaging learning environment. These tools offer participants a practical way to explore complex socio-environmental issues, encouraging them to analyze different perspectives, develop critical thinking skills, and practice decision-making in simulated real-world scenarios. Such approaches align with the growing need for innovative educational methods that promote experiential learning and address global sustainability challenges.

Most recently, researchers from the University of Continuing Education Krems in Austria [9] introduced the Phosphorus Negotiation Game (P-Game), a face-to-face serious negotiation game designed to support science-policy decision-making regarding sustainable phosphorus fertilizer production. Phosphate ore, one of the five most-mined materials globally, is an essential raw material for agricultural production due to its phosphorus (P) content. However, phosphate ores often contain relevant concentrations of radiotoxic uranium, which, if not recovered during processing, can largely transfer to the final fertilizer product [10,11,12]. The presence of uranium poses potential environmental and health risks, making the decision to recover or leaving it in the fertilizer a complex policy and economic challenge. In the P-Game, participants take on stakeholder roles to debate the advantages and disadvantages of recovering uranium during mineral fertilizer production. The goal of the game is to persuade others using well-researched arguments while navigating scientific, economic, and political considerations.

Since its inception, the P-Game has been successfully conducted with 788 participants across more than 20 countries, providing valuable insights into the decision-making processes of diverse stakeholder groups. To further expand its accessibility and impact, a virtual reality platform [13] and a smartphone application were developed, allowing the game to reach a broader audience beyond the constraints of in-person participation. Such digital adaptations of negotiation simulations could potentially enhance learning by offering a more flexible and inclusive approach, accommodating diverse geographical locations and enabling participants to engage asynchronously at their own pace [6,14]. In addition, a virtual setting may enable the inclusion of artificial participants [15].

Most negotiation classes or training courses have traditionally been conducted in person; however, the rapid advancement of technology has led to a significant increase in online or virtual negotiation courses. These formats have gained popularity due to their inherent flexibility [16,17], cost and time efficiency [18,19], and continuous technological improvements that increasingly simulate in-person interactions with higher fidelity [20]. While the trend towards virtual learning already existed before the COVID-19 pandemic, the global crisis accelerated the shift to online education, prompting a critical examination of its effectiveness in comparison to face-to-face methods [21].

A key debate in educational research concerns the effectiveness of virtual teaching and negotiation training, which was recently summarized by Oehlschläger and Merz [22]. One of the theoretical frameworks often referenced in this debate is the Media Richness Theory [23,24], which suggests that learning outcomes may be influenced by the richness of the communication medium used. According to this theory, richer media, such as in-person communication, can provide more effective learning experiences because they convey multiple cues (e.g., facial expressions, gestures, tone of voice) that enhance understanding and reduce ambiguity [25,26]. In contrast, virtual environments may require participants to invest more cognitive effort in navigating the interface and managing digital communication, potentially diverting cognitive resources from the actual learning process.

The P-Game is grounded in British Parliamentary Style (BPS) debating, an interactive debating format that fosters engagement by enabling direct interaction between students and instructors [27]. This format encourages active participation, critical analysis, and structured argumentation while allowing for constructive feedback and reinforcement of contributions [28]. The question remains whether such an interactive approach can be effectively transferred to a virtual setting without losing the essential elements that drive engagement and learning outcomes. The present study seeks to explore this issue by comparing self-reported learning gains and participant experiences across face-to-face and virtual versions of the P-Game.

Early observations suggest that while in-person debates benefit from real-time interactions and immediate feedback, virtual formats offer advantages in terms of accessibility, flexibility, and opportunities for reflection. Comparisons between the effectiveness of these formats may provide insights similar to those observed in previous studies on telephone negotiations [29], where the absence of non-verbal cues influenced negotiation dynamics but did not necessarily reduce overall outcomes.

Overall, the P-Game serves as a valuable tool for advancing sustainability, offering an interactive learning platform for participants to develop skills and knowledge relevant to complex environmental challenges. The objective of this work is to examine whether the educational benefits observed in face-to-face implementation are similarly reported when the P-Game is delivered via a smartphone application. To address this question, identical pre- and post-game survey instruments were used across both formats to assess self-reported knowledge gains, complemented by qualitative analysis of open-ended responses to explore how participants describe their learning experiences across delivery modes.

2. Materials and Methods

2.1. The Development of the Online Version of the P-Game

For the online version of the P-Game, Flutter (Google, Mountain View, CA, USA) [30], Dart (Google, Mountain View, CA, USA) [31] and Firebase (Google, Mountain View, CA, USA) [32] were used. Flutter is an open-source framework that can be used for building natively compiled applications for multiple platforms from a single codebase. Flutter was selected for its accessibility features, modern user interface (UI) capabilities, high performance, and extensive ecosystem of development packages. Dart offers high performance due to native code compilation, cross-platform support, strong typing for early error detection, and a rich library ecosystem. Firebase is a mobile and web application development platform. It provided backend services for application development, user authentication, and real-time data management. The following key Firebase services were also used: Cloud Firestore, Cloud Functions, Authentication, and Cloud Storage.

For the development, the Spiral Model was used. It was chosen due to its ability to handle risks effectively by combining elements of evolutionary, waterfall, and prototyping models. This iterative approach allowed for continuous refinement based on feedback, making it suitable for complex projects such as the P-Game. The Business Logic Component (BLoC) architecture was used to separate the presentation layer from the business logic and data layers, enhancing maintainability and scalability. The BLoC library’s Cubit class provided state management. Additionally, Google’s Material Design language ensured a consistent look and feel across platforms.

Several P-Game specific features were integrated into the online adaptation. Voice chat was used as the primary communication tool during debates to reduce the influence of visual cues on argument evaluation. Asynchronous participation allowed users to engage in debates at their time of convenience. This promoted inclusivity and enabled participants to prepare more thoroughly before contributing. The developed smartphone application also allows participation from different locations (globally), although this feature was not utilized in the present assessment.

2.2. Details of the Application

Before entering the application, the users accessed a landing page. Here, the users could either log in or sign up before the game. An account could also be created using an already existing Google account. The landing page was accessed via a smartphone application or using a browser on a computer. Both can be seen in Figure 1.

After (signing up and) logging in, the users were taken to the lobby where they could see different debate rooms. They could also see whether a room was currently used, or whether the debate had already finished in a certain room. Users could add new public and private debate rooms by pressing the “+” button in the bottom right corner of the screen. If the users wanted to join a private debate room, they needed to enter a password. The lobby can be seen in Figure 2.

After entering a selected room, the users were taken to the debating screen (shown in Figure 3), where voice messages could be recorded or replayed. In Figure 3, details about the debate are shown at the top of the screen. According to these details, the debate concerned phosphorus mining. A short introduction about the subject was also provided to help the participants understand the specific topic chosen for the debate. After the game had started, users could send voice recordings to each other, simulating the BPS format that was also used for the in-person P-Game.

2.3. Data Collection

A mixed-methods design was employed, combining quantitative pre–post measures with qualitative open-ended responses to examine both reported knowledge change and participants’ experiential framing of learning.

Data were collected on the gaming experience and the system usability. To better understand the gaming experience of participants and to compare the virtual P-Game experience with participant experiences from playing the face-to-face P-Game, the same pre- and post-game surveys were used as previously applied in the in-person P-Game [9]. Minor modifications were made to the survey questionnaire to include additional clarification for the Likert scale and open-ended questions. The content of the pre- and post-game surveys, as illustrated in Figure 4, remained unchanged. Despite the virtual P-Game being conducted on smartphones, participants were requested to complete printed copies of the surveys to maintain consistency with the face-to-face P-Game approach.

Because the study examines perceived knowledge acquisition, negotiation confidence, and experiential engagement, constructs that involve internal cognitive and reflective processes, participants’ self-reports represent an appropriate and widely used method in serious game and negotiation simulation research. While self-assessment measures may be influenced by social desirability bias or individual response tendencies, the pre/post design of the surveys strengthens interpretability by focusing on within-participant change rather than absolute scores. Furthermore, the integration of qualitative open-ended responses allows examination of how the participants articulated learning, providing complementary evidence beyond numerical self-ratings.

The surveys therefore included both quantitative and qualitative elements to provide a comprehensive overview of learning outcomes and game experience. Pre- and post-game measures assessed changes in participants’ self-reported understanding of phosphorus science and negotiation science/practice. Open-ended questions enabled participants to express their reflections in their own words and were analyzed to identify patterns of meaning that extended beyond scaled responses. Future studies may further strengthen evaluative robustness by incorporating objective knowledge tests or structured performance-based indicators alongside self-reported data.

In addition to the pre- and post-game surveys used for the face-to-face P-Game, a standard System Usability Scale (SUS), shown in

Table 1. Standard System Usability Scale (SUS) that was used in this work.

No.		Strongly Disagree				Strongly Agree
1	I think that I would like to use this system frequently.	○	○	○	○	○
2	I found the system unnecessarily complex.	○	○	○	○	○
3	I thought the system was easy to use.	○	○	○	○	○
4	I think that I would need the support of a technical person to use this system.	○	○	○	○	○
5	I found the various functions in this system were well integrated.	○	○	○	○	○
6	I thought there was too much inconsistency in this system.	○	○	○	○	○
7	I would imagine that most people would learn to use this system very quickly.	○	○	○	○	○
8	I found the system very cumbersome to use.	○	○	○	○	○
9	I felt very confident using the system.	○	○	○	○	○
10	I needed to learn a lot of things before I could get going with this system.	○	○	○	○	○

, was used to evaluate the usability of the online P-Game [33].

Testing of the virtual P-Game was conducted in two phases. First, an initial pilot test was conducted with a small group of participants from a course related to this work to identify and subsequently address major technical issues. Results from this pre-pilot testing study are not reported here. After these preliminary tests, full-scale testing was conducted with a group of 29 voluntary participants in Hungary at the University of Pannonia.

2.4. Data Analysis

Data from playing the virtual P-Game with 29 participants at the University of Pannonia in Veszprem in 2024 were compared to data from face-to-face P-Game sessions conducted with a different cohort of 30 participants in Hungary (19 students from the University of Pannonia in Veszprem in 2023 and 11 students from the University of Debrecen in 2024). Additionally, the obtained results were analyzed alongside data from all reported face-to-face P-Game rounds, which were conducted with a total of 788 participants across 22 countries between 2022 and 2024 [9].

The comparative analysis included descriptive statistics, percentage change calculations with uncertainty propagation [34], and effect size assessments using Hedges’ G parameter [35]. These analyses were computed using Microsoft Excel 365 (Microsoft Corporation, Redmond, WA, USA). Statistical significance was assessed using two-way ANOVA in GraphPad Prism version 10.4.1 (GraphPad Software, Inc., La Jolla, CA, USA) to determine differences in self-reported knowledge scores across participant groups, delivery modes, gender, and academic backgrounds.

For comparisons between post-game differences relative to the pre-game baseline within each subgroup, Dunnett’s multiple comparisons test was used, as it is well-suited for comparing multiple conditions against a common control. For within-group pre- vs. post-game comparisons, Šídák’s multiple comparisons test was applied, allowing for robust pairwise analysis while controlling for the family-wise error rate. This combined approach ensured statistical validity while providing detailed insights into learning outcomes.

Data obtained from the SUS are presented separately, since such data were not obtained during the previous face-to-face P-Game rounds.

2.5. Qualitative Analysis of Open-Ended Responses

To complement the quantitative pre/post measures, open-ended survey responses from both delivery modes were analyzed using Reflexive Thematic Analysis (RTA) [36,37]. RTA was selected because the data comprised short form written reflections, where the aim was to identify patterned meanings in participants’ accounts of learning and experience, rather than to quantify response categories or frequency counts.

An inductive data-driven coding approach was employed, allowing themes to emerge through iterative engagement with the dataset rather than being imposed a priori. No pre-defined codebook was used. Coding focused on semantic content, including explicit statements regarding perceived knowledge gains, negotiation skills, engagement, and platform evaluation. Analysis followed Braun and Clarke’s six phases: familiarization with the dataset, initial coding, theme development, theme review, theme definition, and interpretive synthesis. An independent researcher completed the coding and analysis of the qualitative data.

Two qualitative corpora were examined:

• In-person Hungary pre- and post-game responses (n = 22);
• Online software pre- and post-game responses and usability comments (n = 19–21).

A comparative matrix was constructed to examine:

• Pre–post shifts within each delivery mode;
• Differences in thematic emphasis between face-to-face and online formats.

This comparative approach enabled interpretation of how the delivery context shaped participants’ descriptions of learning, negotiation practice, and engagement. Qualitative findings were interpreted alongside quantitative results to enable methodological triangulation and a more nuanced understanding of reported learning outcomes across delivery modes.

3. Results and Discussion

3.1. Comparison of the Face-to-Face and Virtual P-Game Experience

Table 2. Self-reported data of the participants from the pre-and post-game surveys.

	Virtual P-Game Hungary (n = 29, This Study)	Face-to-Face P-Game Hungary (n = 30)	Face-to-Face P-Game Globally (n = 788)
Background of the participants
Educational background
Level of education
Gender
Age
Self-reported knowledge on phosphorus science
Pre-game	1.72 ± 1.36	1.50 ± 0.86	1.56 ± 0.98
Post-game	3.24 ± 1.61	2.56 ± 0.94	3.00 ± 1.04
%Change 1	+88.4 ± 82.5%	+70.7 ± 48.1%	+92.3 ± 66.2%
Hedges’ G	1.02	1.16	1.43
Self-reported knowledge on negotiation science/practice
Pre-game	2.34 ± 1.39	1.97 ± 1.07	1.83 ± 1.03
Post-game	3.64 ± 1.38	3.03 ± 0.96	3.12 ± 1.02
%Change 1	+55.6 ± 39.1%	+53.81 ± 33.83%	+70.5 ± 45.9%
Hedges’ G	0.94	1.04	1.26

¹ The SD of the % Change was calculated applying the principles of uncertainty propagation [34].

provides an overview of the participants’ backgrounds. The self-reported educational background of the virtual P-Game group in Hungary differed somewhat from that of the face-to-face P-Game group in Hungary, while it was more similar to that of the global face-to-face P-Game group. With regard to level of education, the virtual P-Game group from Hungary showed the largest share of BSc students, which was also reflected in the younger age distribution of the participants. The gender distribution was in fairly good agreement across all three groups.

It is worth mentioning that the participants of the in-person games self-reported a considerable number of nationalities, resulting in potentially diverse cultural backgrounds. The P-Game was inspired by the Mercury Game [38], and the researchers developing and conducting the Mercury Game recommended including participants with diverse cultural backgrounds to understand whether the developed game methodology can successfully be used with participants from different contexts. This was achieved in the global face-to-face P-Game, which was played in 22 countries with participants self-reporting more than 60 nationalities (Austria, Bangladesh, Belize, Bosnia and Herzegovina, Bulgaria, Cape Verde, China, Congo, Croatia, Czech, Egypt, Ethiopia, France, Germany, Ghana, Greece, Haiti, Hungary, Indian, Indonesia, Iranian, Iraq, Ireland, Italy, Japan, Kenya, Kingdom of Eswatini, Lebanon, Lesotho, Lithuania, Malaysia, Mexico, Mongolia, Moroccan, Myanmar, Nepal, Nigeria, Nigerian, Pakistan, Palau, Palestine, Philippines, Poland, Romania, South Africa, Russia, Rwanda, Serbia, Slovakia, Slovenia, South Sudan, Syria, Taiwan, Tanzania, Thailand, Tunisia, Uganda, Ukraine, United States of America, Vietnam, Zambia, Zimbabwe). The face-to-face P-Game in Hungary was also played with a relatively diverse group of students that self-reported 14 nationalities (Bangladesh, Cape Verde, China, Egypt, Hungary, Indonesia, Iraq, Kenya, Lebanon, Malaysia, Morocco, Palestine, Syria, Thailand). In contrast, the focus of the virtual P-Game was not on recruiting participants with diverse cultural backgrounds, and the virtual game was ultimately conducted with a more homogeneous group, largely (>96%) consisting of Hungarian students.

Despite these differences, comparable improvements in self-reported knowledge of phosphorus science and negotiation science/practice were observed across all three groups. Specifically, the self-reported knowledge of phosphorus science increased by 88.4 ± 82.5% in the virtual P-Game group from Hungary, 70.7 ± 48.1% in the Hungarian face-to-face P-Game group, and 92.3 ± 66.2% in the global face-to-face P-Game group, with corresponding Hedges’ G effect sizes of 1.03, 1.18, and 1.43, respectively. The relatively large standard deviations across groups suggest substantial variability in reported learning gains, which may reflect differences in prior knowledge, engagement levels, and individual learning styles. Nevertheless, the consistently positive mean improvements indicate that both the virtual and face-to-face P-Game formats support learning about complex challenges such as the dilemma of unconventional uranium recovery from phosphate ores in mineral fertilizer production.

The Hedges’ G values suggest a strong impact of the intervention across all groups. Values greater than 0.8 are generally considered large, indicating that the P-Game was associated with substantial improvements in participants’ self-reported knowledge [39].

The self-reported knowledge improvements in negotiation science/practice showed a slightly different trend. The virtual P-Game group reported an increase of 55.6 ± 39.1%, whereas the face-to-face groups reported increases of 53.81 ± 33.83% (Hungary) and 70.5 ± 45.9% (global), with corresponding Hedges’ G values of 0.94, 1.04, and 1.26. The relatively high pre-game survey values in the virtual P-Game group suggest that participants may have had prior exposure to the content, potentially explaining the observed differences in reported knowledge increases. This may also be attributed to information sharing from previous face-to-face P-Game rounds. This highlights the need for further studies with larger and more diverse samples to better understand the impact of the virtual format on negotiation knowledge gains. Conducting the P-Game at different institutions could also help clarify whether the observed differences stem from prior exposure or other contextual factors.

Overall, the Hedges’ G values for both phosphorus science and negotiation science/practice are closely aligned, indicating a consistently strong effect of the P-Game intervention across both domains. This suggests that the P-Game supports learning in both areas, with variations likely attributable to contextual differences between participant groups rather than a fundamental disparity in the game’s educational impact.

Both the online and face-to-face P-Game formats demonstrated their capacity to enhance participants’ understanding of phosphorus science and negotiation skills. From a facilitation perspective, the face-to-face format was characterized by higher levels of direct engagement and motivation among participants. The qualitative evidence indicated that face-to-face debates fostered a competitive and dynamic environment, encouraging active participation and immediate feedback loops. This heightened interaction may contribute to stronger motivation and engagement, reinforcing the importance of interpersonal interaction in educational settings.

Conversely, the virtual P-Game format offered distinct advantages, which were highlighted in the feedback received from the participants. The flexibility of asynchronous participation allowed users to engage with the material at their own pace, facilitating reflection and, in some cases, a more thorough engagement with the topic. This flexibility may be particularly beneficial for participants with scheduling constraints or those in different time zones. Additionally, the virtual format’s location independence provides opportunities for broader participation, potentially fostering a wider exchange of ideas and perspectives that could enhance the learning experience through cross-cultural insights.

The motivational aspects of both formats differed. Face-to-face participants often cited the energy and immediacy of live discussions as key motivational factors. In contrast, online participants appreciated the ability to prepare more thoroughly for their speeches, as well as the reduced pressure associated with not having to perform in person in front of peers, which may lessen performance-related anxiety for some participants. Communication patterns also differed: face-to-face interactions benefitted from non-verbal cues and immediate feedback, while online interactions relied primarily on voice recordings. The latter potentially reduced immediacy but allowed for more considered and structured responses.

3.2. Comparative Analysis of the Pre- and Post-Game Surveys

The bar charts (Figure 5) illustrate self-reported pre- and post-game knowledge (P and Neg) across three participant groups: in-person global, in-person Hungarian, and online participants. The analyses presented in Figure 5 are broken down into overall averages, gender, and academic fields, with statistical significance assessed using a two-way ANOVA followed by Dunnett’s multiple comparisons test. Significance levels are indicated by * (p < 0.05), ** (p < 0.01), *** (p < 0.001), and **** (p < 0.0001).

It is important to note that the in-person global study included 788 participants from 22 countries, while the Hungarian in-person and online groups each consisted of approximately 30 participants. Pre- vs. post-game comparisons were conducted exclusively within the online group, and Hedges’ G effect size values provided additional context regarding the magnitude of observed differences.

For pre-game phosphorus science knowledge (Figure 5a), the two-way ANOVA revealed no significant differences across any subcategories, indicating that participants across all groups, regardless of delivery mode, gender, or academic field, began with a comparable baseline self-reported knowledge of phosphorus science. In contrast, pre-game negotiation science/practice knowledge (Figure 5c) showed significant differences, with the online group reporting higher scores overall and particularly within the social sciences subgroup (****). These findings suggest that online participants, especially those in social sciences, may have had higher prior exposure to or confidence in negotiation-related knowledge. However, given the small sample size of the online group, these results should be interpreted cautiously.

The post-game results for phosphorus science knowledge (Figure 5b) revealed significant differences in the science and social sciences subgroups, where online participants reported higher post-game scores than either of the in-person groups. Similarly, post-game negotiation science/practice knowledge (Figure 5d) showed that the online group reported higher post-game scores overall (*) and within the social sciences subgroup (**). These results suggest that the online delivery mode may offer specific advantages for social sciences participants in improving phosphorus- and negotiation-related knowledge. However, the generalizability of these findings is limited by the smaller size of the online cohort compared to the global in-person dataset.

The pre-game vs. post-game comparisons (Figure 5e,f), conducted exclusively within the online group, revealed significant gains in both phosphorus science and negotiation science/practice knowledge within most subcategories. The observed improvements, supported by statistical analysis, reinforce the effectiveness of the online P-Game format in supporting learning across diverse demographic groups.

For overall knowledge on phosphorous science, a large effect size (g = 1.025) suggests strong learning gains, with the largest gains observed in science participants (g = 1.44) and the smallest in social sciences participants (g = 0.68) (

Table 3. S Hedges’ G Effect Sizes for self-reported knowledge gains across different participant demographics in the online P-Game format, based on the pre- and post-game surveys.

Group	Knowledge on Phosphorous Science	Knowledge on Negotiation Science/Practice
All	1.03	0.94
Male	1.04	0.86
Female	0.91	0.77
BSc	1.08	1.06
Science	1.44	1.36
Social Science	0.68	0.63

). Similarly, for knowledge on negotiation science/practice, a large effect size (g = 0.94) was observed, with science participants showing the largest improvement (g = 1.36) and social sciences participants the lowest (g = 0.63). These effect sizes suggest that while the activity was effective across all groups, the magnitude of improvement varied, with science participants benefiting the most.

These findings highlight that the online format may be particularly effective for participants in science fields, as evidenced by the large effect sizes for both phosphorus science and negotiation science/practice knowledge improvements in this subgroup. Social sciences participants, while reporting smaller effect sizes, still demonstrated meaningful gains. The small sample size of the online group (approximately 30 participants) remains a limitation for generalizability, while the global in-person study’s larger cohort of 788 participants from 22 countries provides a more robust dataset for analysis. Overall, the results emphasize the activity’s adaptability and effectiveness across different delivery modes, with notable benefits observed in the online format for specific subgroups, particularly those in science fields.

3.3. Qualitative Thematic Findings from Open-Ended Responses

To complement the quantitative pre–post measures, open-ended responses were analyzed using Reflexive Thematic Analysis (RTA) [36,37]. The analysis explored how participants described learning, engagement and negotiation experience across face-to-face and online delivery modes. While both delivery formats showed positive self-reported learning gains, thematic patterns indicated that participants framed their learning experiences differently depending on mode of delivery. The qualitative findings largely supported the quantitative trends, providing interpretive depth regarding how learning was experienced across modes.

3.3.1. In-Person Delivery Themes

Five themes characterized responses from the in-person Hungary workshops.

Theme 1: Rapid conceptual orientation

Participants frequently described entering with minimal baseline knowledge and leaving with a foundational understanding of the topic (e.g., “now I feel I have the basics”). This suggests that the face-to-face format supported accelerated entry into an unfamiliar socio-technical domain.

Theme 2: Memorable consequence-linked scientific learning

Learning claims were often anchored to vivid high-salient scientific details, particularly those involving uranium recovery, heavy metals, and environmental or health risks. These “sticky” recall anchors suggest that scientific learning was framed through consequence, trade-off, and uncertainty.

Theme 3: Negotiation learning as embodied performance

Negotiation learning was described in terms of confidence, persuasion, rebuttal, and argument craft. Participants emphasized speaking under pressure, structuring arguments, and responding to opposing claims (e.g., “if you are confident, you can sell even weaker statements”). This indicates that negotiation learning was experienced as enacted performance rather than abstract theory.

Theme 4: Critical realism about power and politics

Several participants reflected on how debate and negotiation function in real institutions, including strategic positioning, persuasion norms, and the influence of power. These reflections suggest that the simulation encouraged systems-level awareness beyond procedural debating skills.

Theme 5: Affective and social engagement as learning catalysts

Enjoyment, challenge, and social participation were frequently described as meaningful aspects of the learning experience (e.g., “really rewarding activity,” “I had fun,” “develop my social skills”). These responses highlight that affective engagement and group participation may contribute to learning outcomes in face-to-face delivery.

3.3.2. Online Delivery Themes

Five themes characterized responses from the online smartphone-based implementation.

Theme A: Baseline uncertainty and conceptual ambiguity

Pre-test responses frequently expressed unfamiliarity with the topic (e.g., “What is this?”, “Never heard of it”). Compared to the in-person dataset, online baseline responses were often shorter and more minimal, suggesting variability in engagement or interpretation of the task.

Theme B: Generalized learning gains with fewer specific anchors

Post-test responses frequently reflected broad evaluative statements (e.g., “learned a lot,” “new perspective”), with fewer specific scientific details compared to the in-person responses. However, occasional concrete anchors were present (e.g., uranium as a by-product of phosphate mining).

Theme C: Negotiation framed as listening and structure

Negotiation learning in the online mode was described more metacognitively, including attentive listening, understanding the structure of formal debate, and incremental improvement in confidence. Compared to face-to-face delivery, fewer responses emphasized real-time persuasion and rebuttal craft.

Theme D: The platform as a pedagogical actor

Online participants explicitly evaluated the interface and software features (e.g., comments on graphical user interface, usability, and suggestions for enhancements). This indicates that the technology itself became part of the learning experience, shaping engagement and reflection.

Theme E: Engagement polarization

Some responses reflected strong engagement, while others indicated confusion or disengagement. This polarization suggests that online delivery may require clearer onboarding and scaffolding to ensure consistent participation and interpretation of the game context.

3.3.3. Cross-Mode Comparison

Overall, the thematic comparison suggests that face-to-face delivery elicited more vivid consequence-linked scientific recall and more performative descriptions of negotiation learning, while online delivery was more frequently associated with reflective structure-oriented negotiation learning and explicit evaluation of the software platform. These findings support the quantitative results by providing insights into how learning experiences were interpreted and articulated across delivery formats.

3.4. Results from the System Usability Scale (SUS)

The SUS score of this first group of participants was slightly higher than 81. According to standard SUS evaluation guidelines [33], a score above 68 is considered acceptable, whereas scores exceeding 80.3 are deemed excellent. The relatively high usability score suggests that the online platform is user-friendly and effective in delivering a positive game experience. However, it is important to acknowledge that this initial evaluation was conducted with technologically proficient BSc students, which may have influenced the overall score.

Participants from diverse educational and technical backgrounds may perceive and evaluate the usability of the platform differently. Therefore, continuous improvements and refinements will be essential to maintain and enhance usability, especially as new features and updates are introduced to the platform. Future assessments should include broader participant demographics to ensure a more comprehensive understanding of the platform’s accessibility and effectiveness across different user groups.

4. Conclusions, Limitations and Outlook

The development of the virtual P-Game, as evidenced by self-reported knowledge gains, effect sizes, usability scores, and qualitative thematic findings, represents a meaningful and successful extension of its face-to-face counterpart. Both delivery formats were associated with substantial increases in participants’ perceived understanding of phosphorus science and negotiation science/practice. The qualitative analysis of the open-ended responses suggests that these gains were not merely numerical shifts but were reflected in how participants articulated their learning experiences. In-person participants frequently described vivid consequence-linked scientific insights and embodied negotiation performance, whereas online participants more often emphasized reflective understanding, listening skills, and structured argumentation. Together, these findings indicate that while learning outcomes were broadly comparable across modes, the experiential texture of learning differed in meaningful ways.

The virtual format also expanded accessibility and flexibility, allowing participation independent of location and enabling asynchronous engagement. This increased accessibility broadens the reach of the P-Game and enhances inclusivity for participants who may be unable to attend in-person sessions. At a broader level, the virtual P-Game demonstrates how digital transformation can extend sustainability education while maintaining core pedagogical integrity. Rather than replacing the face-to-face format, the online version appears to offer a complementary pathway, supporting similar learning gains through different interactional affordances.

However, significant differences were observed in pre- and post-game self-reported knowledge across groups. The online participants, particularly in the social sciences subgroup, reported higher pre-game scores in negotiation science/practice knowledge, suggesting potential prior exposure to the topic, which may have influenced post-game knowledge increases. This underscores the need for future studies to account for pre-existing knowledge differences when interpreting learning outcomes.

While both formats produced statistically significant improvements, qualitative patterns suggest that the face-to-face format may foster heightened affective engagement and real-time persuasive performance, whereas the online format may encourage reflective processing and structured contribution. These distinctions align with established theories of social presence and mediated communication and indicate that the delivery mode shapes not only outcomes but also how participants experience and describe their learning [28,40,41].

A key limitation of this study is that learning outcomes were assessed primarily through self-reported survey measures rather than objective performance-based indicators. Although the pre–post design reduces the influence of individual response tendencies by focusing on within-participant change, self-report data may still be affected by social desirability bias, differences in interpretation of scale items, or variability in response effort. Future research could strengthen evaluation by incorporating objective knowledge checks (e.g., short quizzes on phosphorus concepts), behavioral indicators (e.g., participation frequency, response length, turn-taking patterns), or structured rubrics for assessing negotiation quality (e.g., evidence use, rebuttal effectiveness, argument coherence). Such measures would provide a valuable additional layer of validation and allow closer comparison between perceived and demonstrated learning outcomes.

The nature of the written qualitative responses collected across the two delivery modes represents a related limitation. Differences in social presence and facilitation between face-to-face gameplay and asynchronous online participation may shape not only learning outcomes but also how participants articulate their experience. The online cohort showed more variability in the depth and detail of open-ended responses, which is a recognized feature of self-administered survey formats [42,43,44,45] and may reflect differences in engagement or in the interpretation of the prompts.

A further limitation concerns the sample size and the lack of diversity among participants. The online format was tested with a relatively small sample of fewer than 30 participants, predominantly from Hungary, which does not represent a globally diverse population. Expanding the sample size and including participants from more diverse geographical, educational, and cultural backgrounds in future studies could yield more generalizable results. Another limitation is the relatively short testing period. A longer duration could facilitate deeper interactions, potentially leading to more comprehensive insights into learning dynamics and knowledge retention.

The face-to-face format allowed for immediate observable feedback and dynamic social interaction, which were less directly replicated in the online environment. The qualitative responses suggest that in digital settings, onboarding clarity, interface design, and platform usability become integral components of the pedagogical experience.

Additionally, technological barriers, such as access to reliable internet and device compatibility, could have influenced participants’ ability to fully engage with the virtual P-Game. Differences in technology access may introduce variability into the user experience that was not captured in this study. Future studies should aim to enhance the platform’s support for various devices and connectivity conditions to ensure a more consistent and inclusive experience for all users.

In conclusion, the findings suggest that both virtual and in-person formats can effectively support sustainability-oriented learning in phosphorus science and negotiation practice. While the modes differ in interactional dynamics and experiential emphasis, the virtual P-Game demonstrates strong potential as an accessible and scalable complement to traditional implementation. Further research with larger more diverse samples and multimodal evaluation strategies will be essential to fully understand how delivery context shapes learning processes and long-term educational impact.