What Could Possibly Go Wrong? Exploring Challenges and Mitigation Strategies of Applying a Living Lab Approach in an Innovation Project ^†

Abstract

The living lab methodology is widely used in innovation projects to drive user-centered development. While its benefits, such as co-creation and real-world validation, are well known, its implementation presents challenges that remain underexplored. This study examines these challenges by using the Horizon 2020 Möbius project as a case study. While the Möbius project itself aimed to modernize European book publishing through an immersive reading application and a data visualization tool, this study reflects on the implementation process of the living lab approach within that context, using an action research approach. After project completion, a structured brainstorming session reviewed identified challenges and mitigation strategies. Findings highlight three key challenges. First, misalignment between assumed and actual stakeholder needs hindered industry engagement. Second, recruitment was complicated by the ambiguous use of “prosumer”, causing confusion among participants. Third, communication gaps and personnel changes disrupted the integration of user feedback into development cycles. These challenges underscore the need for early and continuous stakeholder alignment, adaptive communication, and structured knowledge management. Based on these findings, the study proposes strategies to improve engagement and integrate user insights more effectively, ultimately enhancing the impact of living lab-based innovation projects.

1. Introduction

Due to the complexity and high pace of changes in industry, organizations are turning from traditional research and development processes to more collaborative and open innovation methods [1,2,3]. Living labs have gained popularity amongst researchers, policymakers, and practitioners as a tool for practical and collaborative innovation in various industries [4,5,6]. Living labs offer a cooperative setting for innovation in both public and private sectors [7,8]. They are defined as “a design research methodology aimed at co-creating innovation through the involvement of aware users in a real-life setting” [9] (p. 139). Living labs can be divided into two main categories: permanent living lab organizations and temporary project-based living labs. Permanent living lab organizations function as ongoing innovation ecosystems, supporting continuous collaboration and co-creation activities with no fixed end date. In contrast, project-based living labs are designed to achieve specific goals and objectives within a limited duration, characterized by their time-bound, goal-oriented innovation outcomes within a defined scope [10]. The Horizon 2020 Möbius Project, a specific type of innovation project to support innovation in European markets, is an example of a living lab as a project approach. The Möbius project (for more information, visit: https://mobius-project.eu/, accessed on 10 April 2025) was funded by the European Commission through the Horizon 2020 research and innovation program, which had a total budget of nearly EUR 80 billion and ran from 2014 to 2020. Horizon 2020 aimed to stimulate sustainable innovation, scientific excellence, and industrial competitiveness across Europe. Within this framework, Möbius focused on modernizing European book publishing by revising traditional value chains and business models. The main goal was to explore cross-media storytelling and encourage readers and creators to actively contribute to content creation. This approach aimed to promote cultural innovation, enrich reading experiences, and stimulate diversity in narrative formats. At the macro-social and macro-economic levels, Möbius sought to support the digital transformation of the publishing sector and stimulate investment in the creative industries in Europe. The focus on delivering concrete results through multidisciplinary collaboration demonstrates the practical application of living lab principles. The consortium of the Möbius project consisted of universities, technical developers, and industry partners, with the aim to combine research and development in iterative cycles. Two products were developed: an immersive book (Möbius book, prototype development version) and a data visualization tool (Prosumer Intelligence Toolkit—PIT, prototype development version). The Möbius book consists of an immersive reading app (the Möbius Player) for readers as well as a tool for writers (the Möbius Creator, prototype development version) to create a story and include multi-media content. The second product, the PIT, consists of a dashboard that aims to present publishers with big data insights from online communities, such as platforms for readers and fan fiction platforms (aside from these products, another focus of the project was on prosumer business models, immersive book experiences, and mobile immersive book experiences. But, these products/innovations were not developed using the living lab approach, and are, therefore, not included in this paper).

While implementing the living lab methodology in the Möbius project, we encountered several pitfalls that could be of interest to a wider research community. This paper aims to examine the challenges of implementing the living lab methodology in a project-based innovation context using the Möbius project as a case study, and to identify possible strategies for addressing these challenges. More specifically, this article discusses the challenges and mitigation strategies that arose around the PIT. To this end, the theoretical framework begins with a definition of living labs, an exploration of their impact and outcomes, and an analysis of both their advantages and challenges. We then outline how the living lab methodology was implemented in the Möbius project. The results section describes three main challenges encountered during the project. In the discussion, we propose possible strategies for addressing these challenges. Finally, we summarize the findings and link the challenges identified in the literature review and in the results chapter.

Although the living lab methodology has gained popularity as an innovation model in both academic and policy circles, the existing literature mainly emphasizes its conceptual frameworks and advantages, such as user involvement, testing in practice, and co-creation [11]. However, a recurring limitation in the literature is the lack of detailed empirical research on the application of living labs in project-based contexts with a fixed duration, particularly with regard to implementation issues such as stakeholder misalignment, complexity of recruitment, and integration of feedback. In their systematic review of more than 190 articles, Hossain, Leminen, and Westerlund [4] explicitly note that most studies on living labs remain descriptive and do not critically examine methodological or organizational frictions. Leminen et al. [10] also distinguish between permanent living labs and temporary, project-based living labs, but emphasize that the implementation dynamics of the latter are under-explored. This article fills that gap by offering an action research-based reflection on a European living lab project. Although the Möbius project was funded under the Horizon 2020 Programme, the challenges identified are likely to resonate in other (European and non-European) short-term project-based innovation environments, particularly those involving co-creation in cultural and digital domains.

2. Literature

2.1. Definition and Benefits of a Living Lab Methodology

In this section, we focus on defining living labs, and the advantages and challenges of implementing a living lab methodology in innovation projects.

Typically, successful innovation projects bring together three perspectives: technological, business, and end-user [12,13]. A multidisciplinary approach is necessary to combine optimal technological functionalities, economic value, and applicability in the user context [11,12,14]. Living labs are ideally suited for this purpose, as they give stakeholders more power by involving them in the development process at an early stage [15,16].

The literature shows that there are different ways of defining and identifying living labs. Schuurman et al. [6] make a distinction between different levels of analysis to look at living labs, namely the macro, meso, and micro level. On the macro level, we have living labs as constellations, where different stakeholders in a public–private–people partnership come together to organize and carry out living lab research and projects. On the meso level, we have living lab projects, which are innovation projects carried out within the living lab constellation, and on the micro level, we have the living lab research activities that provide the methodology used in living lab projects to involve and gather user feedback. It is clear that a main definition for living labs is that it is a constellation in which living lab projects and activities are carried out together with users to create new innovations. Examples of such living lab constellations include ENoLL [17], TAMK Living Lab [18], and Botnia Living Lab [19]. Here, living labs have an organized approach where innovation happens through real-life experimentation involving active users and stakeholders, such as public organizations, private companies, and citizens [6].

There is also the definition of living labs as a project, where a type of living lab constellation is set up for one single innovation project within a limited time frame, which merges the macro and meso level of analysis [6,20]. In these projects, the living lab approach is utilized to guide and aid the development of (innovative) products and/or technologies [20]. In this definition, the activities and processes of innovation that take place within said environments are emphasized, and the focus lies more in how participants are contributing to innovation processes through different types of activities [4]. The main difference between living labs as a structure and living labs as a project is that in the prior, there is a need for a more permanent infrastructure where collaborations are set up throughout for different purposes, while in the latter, a fixed living lab constellation is not always given, and living labs are rather utilized as an approach.

In line with this, Dell’Era and Landoni [9] (p. 139) defined a living lab as “a design research methodology aimed at co-creating innovation through the involvement of aware users in a real-life setting”. Furthermore, Eriksson et al. [21] (p. 4) defined a living lab as “a user-centric research methodology for sensing, prototyping, validating and refining complex solutions in multiple and evolving real-life contexts”. Based on these definitions, three core elements emerge, as follows: it functions as a research methodology, operates within a practical environment, and ensures that users are consciously and actively involved in the innovation process. Ballon and Schuurman [11] expand the concept by defining “users” as a diverse group of stakeholders, including technology suppliers, institutional entities, and both professional and private end-users. They consider involvement in the living lab to be inherently multi-methodical, using techniques from disciplines such as ethnography, psychology, sociology, and strategic management. Furthermore, they emphasize that co-creation must take place through iterative design cycles involving different stakeholder groups throughout the process.

For both definitions of living labs, it is clear that several and significant advantages are offered by focusing on developing products or services that are validated with and by relevant user groups, ensuring alignment with their needs and expectations. This collaborative approach thus maximizes the likelihood for successful uptake. By integrating business, technology, and end-user perspectives, the most comprehensive set of requirements can be created to meet the diverse needs of all stakeholders [12]. Engaging stakeholders early in the development process also uncovers deep underlying values and new use cases can be discovered, enabling decision-makers to make timely adjustments during the development and production phases [22]. Furthermore, real-life testing in the natural context of the use case helps identify genuine societal needs and concrete future user behaviors. In addition, living labs contribute significantly to social value, particularly through their cultural impact [23]. For instance, the Möbius project has enhanced cultural experiences by developing interactive learning technologies that incorporate multimedia elements such as images, videos, and audio. This approach has proven especially effective for fostering literacy and making reading more appealing and engaging [24].

2.2. Outcomes and Impacts of Living Labs

The outcomes of living labs can be classified into two primary categories: tangible outcomes and intangible outcomes. Tangible outcomes encompass physical or material innovations, including designs [25], products [26], prototypes [26], and systems [26]. Intangible outcomes, on the other hand, refer to conceptual or knowledge-driven innovations, such as concepts [26], ideas [26], intellectual property rights [27], knowledge [27], and services [27]. As for permanent living labs, the outcomes are often long-term and systematic, with a focus on sustainable knowledge creation, ongoing stakeholder engagement, and incremental innovations that evolve over time. These living labs are well suited to fostering systematic innovations due to their stability and continuous nature. Project-based living labs, by contrast, tend to achieve short-term targeted outcomes, such as specific products, prototypes, or services. These outcomes are more focused and directly related to the project objectives within a limited timeframe [10]. However, in the context of living labs, the distinction between tangible and intangible results is often blurred. Depending on the specific situation, results often fall into both categories. Living labs are crucial for facilitating a range of innovations beyond these two main categories [4]. Small, gradual updates or improvements to existing goods, services, or procedures that gradually increase efficiency or effectiveness are known as incremental innovations [26]. Incremental innovations are more often associated with permanent living labs because of their continuous development processes, while project-based living labs often focus on radical innovations within the scope of the project [10].

Radical innovations are breakthrough discoveries that bring about completely new markets, technologies, or approaches. These developments often involve significant risks, but also have the potential to radically change established systems [26]. Social innovations, which focus on developing new concepts or methods to increase social well-being, address social problems, and create social value, are another important category [28]. Technological innovations focus on the development or use of new technologies to address problems or create new opportunities [28]. Finally, systemic innovations are adaptations that impact entire systems; to produce significant and lasting effects, they often require close coordination and collaboration between different sectors, stakeholders, or technologies [29]. This innovation spectrum shows how living labs play a dynamic and multidimensional role in advancing different fields.

Examining the benefits and limitations of living labs also reveals the positive impact they can have as well as the methodological challenge of effectively evaluating their impact. A major contribution of living labs is their ability to generate economic value, by, e.g., improving the product to better fit the market as well as foster business development. Ballon et al. [30] emphasize that living labs provide a unique platform for companies, where they can incorporate external viewpoints in their innovation processes. By including the viewpoints of the general public or specific stakeholders in the innovation process, the design and usability of products and other offerings are enhanced, which might not have been possible in more traditional R&D settings [20,30]. Eventually this leads to increased investments, employment, and business turnover. Thus, living labs may create value for both businesses and end-users [20]. Permanent living labs often have long-term social impacts through ongoing operations and stable relationships with stakeholders. These impacts often include sustainable economic growth, policy influence, and community development. In contrast, project-based living labs tend to have more immediate and localized impacts that are closely linked to the specific objectives of the project. Their impact can be intense but is often limited in duration and scope compared to permanent living labs [10]. Niitamo et al. [31] discuss how living labs provide open and neutral platforms that foster technological and business model innovation, and that they provide an environment that is instrumental for rapid innovation cycles. Thus, by integrating social, environmental, and economic sustainability principles, living labs have the potential to drive socio-economic dynamism.

To a large extent, living labs are deemed as beneficial and as having positive impacts based on their embodied characteristics. For example, Ståhlbröst [20] proposes five guiding principles for assessing the impact of living labs: (1) value, considering whether living labs create benefits for all partners involved as well as external users, such as end-users and customers; (2) sustainability, evaluating to what extent and how living labs manage environmental, social, and economic effects created within projects; (3) influence, evaluating the extent to which the input by external viewpoints contribute to the development process of an innovation or product; (4) realism, ensuring that the results generated within the living lab reflect real market conditions as well as realistic use situations by understanding users’ needs and behaviors; and, finally, (5) openness, which assesses whether the innovation ecosystem fosters collaboration and knowledge sharing among stakeholders involved. However, Paskaleva and Cooper [16] argue that there is a lack of empirical evidence to substantiate these claims of success. Although it is the aim of living labs to achieve these objectives, there is little data publicly available to assess whether this is achieved in practice. Thus, stakeholders fund living labs without necessarily requiring robust evaluations of the impacts and outcomes, which has led to an increase in living labs that operate without a standardized criteria for assessing themselves in these aspects. Bronson et al. [32] highlight that there is a lack of robust methodologies of assessing long-term outcomes of the impact of living labs. As living lab evaluations often rely on comparative case studies, the focus is mostly on improving internal processes rather than measuring broader impacts, which makes it difficult to generalize findings across contexts as evaluation methods are tailored to specific living labs [32].

In summary, both permanent and project-based living labs contribute to the innovation ecosystem, but their outcomes and impact vary depending on their structural and operational frameworks. The structural framework refers to whether the living lab operates continuously (permanent) or within a certain time frame (project-based), while the operational framework refers to stakeholder engagement, co-creation methods, and approaches to evaluate outcomes [10,20]. Evaluation practices also differ, e.g., permanent living labs may lack standardized methods for assessing long-term impacts, while project-based living labs often focus on short-term outcomes without robust methods for broader impact assessment [16,32]. Recognizing these differences is crucial for stakeholders to align their strategies with the goals of the living lab. This means stakeholders need to adjust their expectations, resource allocation, and level of engagement based on whether the living lab is focused on continuous, long-term societal impact (as in permanent living labs) or on specific, time-bound project outcomes (as in project-based living labs). For example, companies may focus on sustainable partnerships in permanent living labs, while in project-based living labs they prioritize rapid product development and short-term market validation [30,31].

2.3. Challenges in Living Lab Approaches

Although the living lab methodology in projects holds a lot of advantages, it may also present some challenges. At the beginning of a project, objectives and goals of said project are outlined. In innovation projects, it is important to be as specific as possible about innovation and development goals and objectives to ensure funding for such a project. However, when using a living lab approach, a more flexible framework that leaves room for adjustments is set up, which also encourages iterative collaboration between stakeholders where the outcomes of co-creation steer the development process [33]. These set objectives and goals can appear ambiguous and/or unclear, characterizing the ‘fuzzy front end of innovation’, which can make governance in living lab projects difficult [14], especially as the project may take on a different course than initially thought. This is where living lab projects differ from conventional project-based innovation development, especially in terms of objectives, control points, the roles of project managers and users, how resources are used, and what management tools are used to control and monitor the project [33]. The main difference is that the latter have a more controlled and already determined trajectory, while the prior is subject to change as the project goes on.

As living labs are focused on co-creation, iterative development, and experimentation, benefits and outcomes are yielded over an extended period of time, which in the context of an innovation project usually spans 1 to 6 years [34]. When involving users to co-create solutions in iterative cycles, the project trajectory is ever evolving and the direction of which the development process takes may change throughout the project lifetime [35], making it difficult to predict final outcomes of an innovation [33]. This also poses the challenge of demonstrating the long-term value of living labs to stakeholders in a project [36], as some stakeholders may become more skeptical about the long-term efficacy of living labs. Due to the limited time frame of these projects, there is the assumption that products created in this context are not disruptive, as they are not able to generate results that significantly disrupt the market; thus, companies are less inclined to improve their products through living lab approaches [15].

In living lab projects, changes are bound to happen due to the diverse and various participants involved. By involving various and diverse participants throughout a project lifecycle, more and new knowledge is continuously being created, which may change the trajectory of development cycles and goals. To account for this, it is important that living labs remain flexible and can guarantee stability [10]. Being flexible means that as changes to participants, objectives, and/or resources occur, the living lab should be able to adjust and adapt to said changes. An example of this is the oPEN Lab project, where during the implementation of the project, they ran into some barriers when it came to the planning of transitioning each city as stakeholder engagement and technical constraints became apparent [37]. To mitigate this, the project thus customized its strategies to the local context (e.g., involving local residents through interviews), modified the ways in which they approached stakeholders to engage (e.g., sending postcards to encourage stakeholders to visit the local information center, but also by issuing open calls for industry partners to get involved), and reallocated resources to address issues in cities where needed [37].

This flexibility also means that participants can participate and leave at any time, which makes maintaining iterative feedback loops more complex, as for each new evaluation moment, new participants need to be recruited, which in itself is not always an easy task due to potential low response rates [38]. The engagement and long-lasting commitment of users proves challenging and can result in there not being enough testers to evaluate the innovation [15]. To avoid spending a lot of time and effort on recruiting new participants for new projects and activities, Veeckman et al. [23] suggest that when setting up a living lab, it can be beneficial at the beginning of the project to work towards establishing (strong) community support. From this community, participants can then be recruited to participate in the project and its activities, and the risk of putting high efforts and potentially losing accumulated knowledge can be reduced. Furthermore, Veeckman et al. [23] argue that by utilizing a simplified communication strategy and by tailoring recruitment initiatives, participant diversity and engagement can be increased. However, this may lead to participant bias [38]. To avoid this, it is necessary to involve and engage potential participants that otherwise would not be actively involved in the community [38]. This proves to be difficult, however, since the participants usually accepting to participate find interest and value in participating and are, thus, biased or are already part of the community [38]. It is important to ensure a diverse user group within the community, as otherwise an inadequate representation of diverse user groups not only reduces the inclusiveness of the living lab process [39], but also limits the generalizability and scalability of the results [4].

As personnel within the living labs may change throughout the lifecycle of a project, the ways in which evaluation activities are carried out for different innovative products may change due to lack of context of previous activities [40]. This is why it is important that living labs guarantee stability, since keeping the accumulated knowledge and skills are crucial in continuing the project while gathering further feedback from (new) participants. In the case of project-based approaches where there is a limited time frame, it can be even more challenging to guarantee stability and alignment. The documentation of knowledge and creation of repositories is, therefore, crucial but may take a lot of time [41]. Without effective strategies to document the knowledge retained and limited time to carry out the work of the project, valuable insights might risk being overlooked or lost [41].

Within living labs and projects utilizing a living lab approach, it appears that stakeholders have little confidence that their opinion really matters for the development of innovations [42]. When there are differing ideas of what the goal or objective of an innovative product is, especially between the user and the developer, a power imbalance ensues [43]. This may occur as those directly involved in the development of an innovative product may have their own predetermined ideas of what the product should do but also of what the users might want and need [38]. To avoid complex iterative negotiations for decision making and delays, alignment of stakeholders to achieve a unified project pathway is crucial [11]. It is equally important in living labs that project personnel keep an open mind to user feedback, be it positive or negative, so that they can base solutions and improvements on real user needs.

3. Materials and Methods

This section describes the process used to identify challenges encountered during the Möbius project and develop risk mitigation strategies. This process incorporated an action research approach [44] alongside brainstorming techniques [45] to support data analysis. To provide the reader with context, this is followed by the methodology employed to foster innovation within the Möbius project.

3.1. Identifying Challenges and Developing Mitigation Strategies

To understand the challenges during the Möbius project and the steps taken to address them, an action research approach was applied according to the framework described by Logghe and Schuurman [44]. This method integrates practical action with reflection and links theoretical concepts with real-world applications. It aims to effectively address pressing challenges [46]. This means that we played a practical role in applying the methodology for the Möbius project, including designing, coordinating, organizing, and facilitating activities as part of the living lab methodology. As workshop coordinators, organizers and moderators, we were able to observe the process closely and reflect on the challenges encountered in setting up the living lab methodology during the project [46]. Feedback from living lab participants and project partners helped further identify specific challenges. Using this input, strategies were developed to mitigate these challenges where possible [44].

After the completion of the project, a structured brainstorming session was held to review the challenges and mitigation strategies, complementing those already implemented with suggestions for future application beyond the Möbius project. This was divided into two phases [45]. In the first phase, the divergent phase, a broad list of challenges and potential strategies was generated without filtering or prioritizing. In the second phase, the convergent phase, these items were restructured (grouping and merging similar and related challenges) and critically reviewed according to their importance and significance for publication purposes [45]. This process resulted in a detailed list of challenges and mitigation strategies, which are described in the following sections of the paper.

3.2. Living Lab in the Möbius Project

The results below are lessons from the Möbius project. For context, we here describe how a living lab methodology was applied within the project, allowing co-creation with users, defined early as ‘prosumers’ (readers, fan fiction writers, and amateur writers) and local (European) publishers. Co-creation workshops, think-aloud sessions, interviews, and surveys were the primary methods used in developing the Möbius book applications and the PIT. Co-creation, as defined by Sanders and Stappers [47], is the shared act of collective creativity, where people collaborate to generate new ideas and solutions. The quadruple helix model emphasizes collaborative innovation, where government, industry, academia, and civil society co-create, test, and refine new products in a real-world setting. This ensures that the outcomes are technically sophisticated, socially relevant, and sustainable [48].

In the Möbius project, academic researchers designed and analyzed the co-creation process. Technical partners developed applications, while communication partners and European publishers managed commercialization, bridging commercial imperatives and user needs. Public authorities, such as the EU, provided funding, and the involvement of civil society—readers, authors, and publishers—ensured the technology met real-world needs. Each proof of concept was developed and tested iteratively, incorporating stakeholder needs. Feedback loops fine-tuned the technology using qualitative and quantitative data. Although methods varied by phase, the same user groups were addressed for different applications. The living lab methodology consisted of four phases: a pre-pilot phase for requirements gathering, a pilot phase with three rounds of iterative development and testing (see Figure 1), and a post-pilot phase demonstrating services at large-scale events.

The pre-pilot phase gathered insights into publishing industry practices to create and update user requirements. Three pilot phases followed, each reaching more users (see Figure 1). The first pilot phase included research activities such as co-creation workshops and surveys. The second pilot phase built on previous insights to update user requirements, using a mixed-method approach including think-aloud sessions, in-depth interviews, and surveys. The third pilot phase gathered feedback on near-final products through interviews, workshops, and surveys. Finding PIT test participants was challenging in all phases, as the intended numbers were not reached (see Figure 1 and

Table 1. Overview of activities in each pilot phase.

	Möbius Player (Prosumers, Readers, Sellers)	Möbius Creator (Writers)	PIT (Publishers)
Phase 1: Paper mock-ups and proxy technologies	3 online co-creation workshops: 18 participants	2 online co-creation workshops: 13 participants	2 online co-creation workshops: 11 participants 2 interviews Survey evaluations: 20 respondents
Phase 2: Clickable mock-ups and prototypes	Survey evaluations: 266 respondents	Survey evaluations: 12 respondents	Think-aloud sessions and interviews: 32 participants
Phase 3: Near-final version of products	User requirement survey: 282 respondents Impact survey: 171 respondents	2 co-creation workshops: 23 participants User requirement survey: 88 participants Impact survey: 111 respondents	6 in-depth interviews 1 co-creation workshop: 18 participants Survey evaluations: 15 respondents

).

For more information on the methodology used in the Möbius project, see following deliverables: D2.4: https://zenodo.org/records/11204340 (accessed on 10 April 2025); D3.5: https://zenodo.org/records/10695588 (accessed on 10 April 2025); D4.5: https://zenodo.org/records/10560772 (accessed on 10 April 2025); D5.3: https://zenodo.org/records/10925433 (accessed on 10 April 2025).

As we aim to highlight the challenges of implementing a living lab methodology within the specific context of this project, the next chapter does not provide a detailed evaluation of the products developed. Instead, the focus is on the main challenges that arose during the implementation of the living lab approach, with specific evaluation results only being referred to when relevant to these challenges.

4. Results and Discussion

This section reflects on the lessons learned from implementing a living lab methodology within the Möbius project, highlighting main challenges encountered and outlining both the strategies adopted and those recommended for addressing them. The first part discusses engagement with the market and industry, emphasizing the critical importance of gaining a comprehensive understanding of industry needs, priorities, available data, and community resources, ideally as early as the proposal development phase. The second part addresses the concept of the prosumer, focusing on the challenges of identifying and recruiting appropriate users for a product or service when the concept is not yet fully tangible or clearly understood by potential participants. Finally, we consider the complexities of communication and coordination with technical partners, offering reflections on the difficulties of effectively translating user feedback into actionable technical developments within innovation projects, along with strategies to mitigate these challenges. These results are reworked but largely based on the results presented at the Open Living Lab Days 2024 [50].

4.1. Stakeholder Misalignment and Data Accessibility Barriers

In any project, it is essential to develop a rapid yet profound understanding of stakeholder needs and concerns, the market context in which they operate, relevant legislative constraints, and the resources at their disposal. In this project, the proposal outlined the use of openly accessible data from fanfiction communities to demonstrate the functionality of the toolkit dashboard. This approach was intended to support the technical partners in configuring the data analysis and visualization components of the tool. It was assumed that fanfiction data would be both relevant to publishers, by providing insights into online prosumer communities, and well-suited to illustrate the potential of the dashboard in terms of analytical and visualization capabilities applicable to their own user communities.

This underscores a key impact of living labs, as identified by Niitamo et al. [31], namely, their ability to generate socio-economic value by stimulating new business opportunities and improving data-driven decision-making processes. Interviews conducted with professional users and publishers during the pre-pilot phase indicated a clear interest in deriving insights from online reading communities. Respondents acknowledged a notable lack of accessible data for anticipating emerging trends, highlighting a reliance on retrospective sales data to inform decisions about future publications. There was a strong expressed need for timely information and access to data that could reveal trending topics. This interest was reiterated by participants throughout the pilot phases:

“[…] I think the past sales of books or the past comments of books indicate the trends for future publishing initiatives. We do that with sales, but we don’t predict it with consumer information.”;

(Publisher EU MS A, pilot phase 1)

“I don’t know if it’s like this everywhere, but in [country] we don’t have access to many numbers from the market… there’s no independent organism that would give you the numbers from the market. So we are always trying to like magically know what is happening around us.”.

(Publisher 5, pilot phase 3)

This highlights an important outcome of living labs as found in the literature, which is the creation of both tangible results, such as prototypes and data visualization tools, and intangible results, such as new knowledge and insights into market trends [26,27].

However, several challenges emerged over the course of the project. During the pre-pilot phases, a marked discrepancy surfaced between the communities that publishers deemed relevant and the fanfiction platforms initially identified as the project’s main community. Throughout the pilot activities, fanfiction writers and enthusiasts were frequently described as “not representative of the reading community” and “not relevant to my business as a publisher”. A key concern raised during workshops in pilot phase 1 was that fanfiction platforms, such as Wattpad, do not reflect the profile of typical book buyers. Participants pointed out that such platforms are largely dominated by fantasy content, which they considered unrepresentative of the broader market’s diverse genre preferences. Additionally, publishers questioned the applicability of fanfiction data due to the prevalence of major intellectual properties (e.g., Harry Potter), arguing that such dominance limits its relevance for smaller publishing houses. They further indicated practical constraints, such as limited resources for managing online communities, risks related to the potential misappropriation of original IP, and the necessity for insights tailored to specific countries or language markets, as reasons for their reluctance to engage with these platforms.

These findings are consistent with what has been reported in the literature. They reflect common challenges observed in living lab projects, particularly regarding the misalignment between stakeholder expectations and project outcomes, as noted by Westerlund and Leminen [33]. In the Möbius project, the lack of stakeholder interest was not rooted in a limited vision towards long-term outcomes, as discussed by Guzman et al. [36]. Instead, it stemmed from a misalignment between the solutions envisioned within the project to address publishers’ challenges and the solutions stakeholders themselves prioritized. This type of misalignment can also be understood through the lens of Herstatt and Verworn’s [14] concept of the ‘fuzzy front end of innovation’, where ambiguity in objectives complicates stakeholder alignment and project management.

A related, major challenge was participant engagement, as highlighted by Nesti [15] and Bergvall-Kåreborn and Ståhlbröst [38]. Publishers’ lack of interest in fanfiction communities made recruiting them for data visualization tools evaluations and demonstrations difficult, particularly due to the inclusion of fanfiction data. Publishers relied on a combination of internal and external data, such as sales figures, reading behavior, consumption figures, and general market data. However, they did not have access to special online communities or the type of data required for the dashboard. Respondents noted that data accessibility issues stemmed from fragmented data sources, making it difficult to effectively support publishing decisions. Many publishers involved in the workshops stated that they were reluctant to start and manage their own online communities due to high costs and low returns in terms of quality content or relevant insights. Publishers often expressed the need for a dashboard that would allow them to compare third party data with respect to relevant markets, as the data they have access to is often fragmented.

The challenges stemming from misalignment between project assumptions and stakeholder priorities highlight the need for a thorough evaluation of market needs from the outset of innovation. Early-stage ambiguity in objectives can complicate stakeholder alignment and project management [12,14]. While some stakeholders were involved in the consortium, their participation did not always provide a deep understanding of industry practices, particularly regarding data use and accessibility. The Innovation Binder Approach (IBA) offers a structured method to integrate user, technology, and business perspectives, fostering alignment and decision-making throughout the project [51]. This approach should not only address early-stage challenges but also adapt to evolving stakeholder needs over time. Maintaining a balance between openness to change and project focus is crucial. For instance, if the team had anticipated publishers’ reluctance to engage with fanfiction data, alternative stakeholder involvement and project directions could have been considered earlier, potentially influencing technological development and collaboration outcomes.

4.2. What’s in a Name? Recruitment of ‘Prosumers’

The concept of the prosumer is not new; Toffler [52] introduced the term in 1980 to describe the dual roles of producing and consuming content. Since then, prosumers have been further conceptualized as individuals who voluntarily and creatively contribute to existing content without direct economic incentives [53,54]. Through this dual role, prosumers have the potential to enrich the content ecosystem by offering their unique perspectives and inputs. Within the context of this project, the term referred to individuals capable of collaboration and value co-creation, with the aim of fostering sustainable interactions that benefit both prosumers and the European publishing industry. Prosumers were broadly understood to include authors, influencers, entrepreneurs, beta-testers, and early adopters.

While the term proved useful within the consortium, it frequently led to confusion among external stakeholders, posing challenges for participant recruitment. This reflects a well-documented issue in living lab environments, where the involvement of diverse user groups is often hindered by complex terminology and the need for more accessible communication strategies [23]. The term ‘prosumer’ also served as the project’s tagline and was prominently featured at public engagement events (see Figure 2). However, it often prompted questions from passers-by, who found the concept too vague to relate to. Although many of these individuals self-identified as self-publishers or amateur (fiction) writers, they did not recognize themselves in the broader label of ‘prosumer.’ As a result, the project team adapted its communication approach, opting for more familiar and specific terms such as ‘readers’ or ‘writers’ to enhance clarity and resonance with target audiences.

These adaptations highlight a key advantage of living labs, as their iterative approach facilitates the integration of real-time feedback, which, in turn, supports the continuous refinement and improvement of communication and engagement strategies [11]. Recognizing that recruitment is not just about reaching the right people but also about framing engagement in a way that resonates with their priorities is essential. By allowing space for trial and error in outreach efforts, projects can better tailor their communication, ultimately improving participation rates. Changing the terminology helped us to better reach our target audience, thus facilitating smoother engagement and collaboration with these communities.

Interviews with publishing industry professionals revealed that participants distinguished between self-publishers and prosumers, emphasizing the motivations and ambitions of each type of creator. Self-publishers were described as individuals who typically aim to become professional authors, retain ownership of their intellectual property, and leverage self-publishing platforms, such as Kindle, as part of an entrepreneurial strategy. In contrast, prosumers, exemplified by fanfiction writers, were seen as creating content primarily out of personal passion, without the intention of commercializing their work. As previously noted, publishing professionals frequently associated such prosumer-generated content with lower quality.

Conversely, respondents from the fanfiction community did consider themselves to align with the definition of a prosumer, though they did not use the term explicitly. They emphasized the diverse motivations behind writing and reading fanfiction, such as a profound appreciation for particular media narratives and a desire to extend the storytelling experience. Some noted that dissatisfaction with the original plotlines also drives them to rewrite or reimagine stories according to their preferences. As a result, participants acknowledged that fanfiction often lacks the polished quality of professionally published works, but viewed this as a strength rather than a limitation. The community particularly values fanfiction for its accessibility and inclusivity:

“[…] readers like the fan fiction community because it’s so easily accessible and everything, and that’s why people like it and why there are so many people reading it”.

(Gaby, fanfic group interview, preparatory phase)

Furthermore, fanfiction is valued for its narrative flexibility, which allows storylines to be rewritten, new characters to be introduced, and new interpersonal dynamics to be created. This emphasis on character and relationship development is seen as an important source of appeal within fan communities, despite the often unprofessional presentation style. This feedback from the respondents serves as a key impact and outcome of living labs as their ability to uncover deep, underlying values, and user-driven innovations that traditional research methods might overlook [22], while generating new knowledge and insights into the motivations and behavior of the target group, which can influence the development of more tailored and effective innovation strategies [27].

4.3. Communication Is Key: Bridging User Requirements and Technological Implementation

The iterative process of co-creation and development presents various challenges, particularly with regard to maintaining clear communication about research results relating to user needs, requirements, and product evaluation. Within the project, the alignment between user insights and technological innovation was further complicated by various factors. These challenges also reflect the inherent complexity of living lab methodologies, where the flexible framework designed to foster iterative collaboration can sometimes result in ambiguous objectives and unclear governance structures [14,33]. Despite repeated efforts to communicate user-driven insights relevant to technological development, through project meetings and updates on the project’s SharePoint platform, a gap persisted in the implementation and follow-up of these findings. Three key factors contributed to the difficulties in effectively communicating and acting upon user feedback (see Figure 3).

In the context of an H2020 project, outcomes are typically disseminated through intermediate and final deliverables. However, instead of aligning user involvement outputs with the project’s iterative development cycles, the consortium opted for a single final deliverable. As a result, findings were primarily shared informally, via meetings and draft documents uploaded to the project’s SharePoint. This approach often gave the impression that the results were still provisional, leading stakeholders to deprioritize their integration. Furthermore, updates regarding user requirements posted on internal platforms such as SharePoint were not always sufficiently emphasized or tracked, increasing the risk that critical details would be overlooked. A further complicating factor was the frequent turnover in project management and among personnel responsible for technological development, which hindered continuity and institutional memory within the team. Although no one is specifically at fault here, it did result in a lack of consistent follow-up. In combination with the aforementioned lack of official deliverables, this made it more difficult to keep the priorities raised by relevant stakeholders on the agenda for technological development. Such changes in project personnel often lead to disruptions in knowledge continuity, which is a common challenge in living labs, where accumulated knowledge and skills are critical to maintaining project coherence [40]. A more structured documentation approach, as emphasized by the IBA, could have helped mitigate this issue. The main point of the IBA is that decisions made throughout a project’s lifetime will be made on a collaborative basis, where interdependencies within the consortium are laid out clearly, and where documentation of decisions made can be used to reflect upon how the goals have been worked towards. The latter is especially important when there are changes in the team structure. Maintaining thorough documentation helps mitigate the challenge of knowledge loss due to personnel changes [41]. By thoroughly documenting why and how decisions were made, new personnel can easily take part of the information and easier adjust to their role in the project. The documentation can also work as a point for the consortium to go back to and see what alternative paths can be taken to overcome potential challenges, by looking at why certain decisions were made at one point that could have been different.

Furthermore, some partners reported limited familiarity with user-centered design methodologies, which contributed to a lack of awareness regarding the outcomes of these processes, specifically, the identification of relevant user needs and requirements. This highlights the need for capacity-building within project teams to strengthen their understanding and application of user-centered approaches, thereby facilitating the effective integration of user insights into technological development [42]. In addition, as partners often remain focused on their own deliverables and timelines, user needs and requirements may not be viewed as a priority. This task-specific orientation can impede the cross-functional collaboration necessary to harness the full potential of living lab methodologies, ultimately constraining opportunities for disruptive innovation [15]. To address these challenges, we argue that the systematic use of project management tools to track and visualize key outcomes, such as user requirements and technological development milestones, can foster mutual understanding, improve communication across teams, and help ensure that user perspectives are consistently taken into account throughout the innovation process.

5. Conclusions and Recommendations

Living labs are widely recognized for facilitating the development of robust innovations grounded in a nuanced understanding of the needs, practices, and contexts of relevant user groups and stakeholders. Existing research highlights several key benefits of the living lab approach, including the ability to make early adjustments during the development process, the empowerment of stakeholders through co-creation, and the validation of outcomes by end-users. However, the literature also identifies a number of challenges associated with this methodology. These include the need for early and sustained alignment with stakeholder expectations and practices, difficulties in achieving truly disruptive outcomes, challenges in integrating user insights during the development phase, and communication barriers between users and project teams. This paper contributes to this body of research by presenting the specific challenges encountered during the implementation of a living lab methodology within the project. Figure 4 provides an overview of the challenges mentioned in the literature, categorized and supplemented with the challenges encountered during the project, together with the proposed mitigation strategies.

The experience gained in this project confirms that aligning stakeholders and developing a deep understanding of market dynamics and industry needs are critical for bridging the gap between stakeholder expectations and the outcomes of living lab processes. However, this study also extends existing knowledge by highlighting the challenges of realigning project objectives when new, complex, or even contradictory user needs emerge during the early phases of implementation. Communication with diverse user groups proved particularly difficult due to the project’s focus on the concept of prosumers, which was not consistently understood or embraced by participants. Moreover, we observed significant challenges in aligning partner priorities, particularly when it came to integrating the outcomes of co-creation and user evaluation activities into ongoing technological development. The findings and mitigation strategies presented in this paper offer a more nuanced understanding of the dynamics at play in living lab environments, especially in the context of large-scale, project-based innovation. They underscore the importance of maintaining flexibility and adaptability to effectively address unanticipated challenges as they arise. In doing so, this study emphasizes the need for continuous stakeholder engagement and the iterative reorientation of project goals to ensure relevance and impact. Thus, three key lessons emerged from our experience:

-

Lesson 1: Several assumptions made during the proposal phase proved to be inaccurate. A more comprehensive understanding of the market landscape, as well as stakeholder needs and practices, either prior to the project’s start or through a formal reorientation following early signals during the pre-pilot phase, would have significantly benefited product development. This underscores the importance of conducting thorough pre-project market analysis and stakeholder consultation to ensure alignment between project objectives and real-world conditions.

-

Lesson 2: Although the term prosumer was accurately applied within the project’s internal and external communications, it caused substantial confusion among both professional publishers and fanfiction or amateur writers. This lack of clarity diminished interest in participating in living lab workshops and created uncertainty among audiences visiting the project booth at conferences and public events. Refining communication strategies to clarify target group definitions proved essential in improving participant recruitment and engagement.

-

Lesson 3: The outcomes of stakeholder and user workshops were not consistently prioritized by all project partners. To ensure effective follow-up, it is crucial to communicate workshop results clearly, engage in in-depth discussions about their implications, and explore how they can be implemented in iterative development phases. Strengthening project management practices to maintain knowledge continuity and systematically integrate user feedback into technological development processes emerged as a key strategy for addressing this challenge.

In addition to practical and methodological insights, the findings also have implications for policy, evaluation frameworks, and funding practices in the context of living lab initiatives. For policymakers, the recurring problems of a lack of alignment between stakeholders and unclear definitions of user groups point to the need for clearer guidelines for stakeholder engagement and communication. Evaluation frameworks would benefit from the inclusion of criteria for adaptability, knowledge integration between partners, and iterative feedback loops. In addition, funding bodies, particularly those managing short-term projects such as Horizon 2020, could provide support for early-stage market validation activities and mechanisms for policy adjustments during project implementation.

Furthermore, this paper underlines the conditional nature of the applicability of the living lab model for social engagement. Although it provides a valuable framework for co-creation and integration of stakeholders [11], our findings show that effective social engagement requires clear, shared terminology and an early joint definition of the roles and expectations of the participants [14,20]. The abstract description of “prosumers” limited the participation of civil society actors and highlighted how a mismatch between the project language and users’ self-identification can undermine the “civil society” pillar of the quadruple helix model [23,48]. To harness the full potential of living labs for societal engagement, future projects must integrate inclusive frameworks and communication strategies from the outset [15].

Ultimately, this paper provides valuable insights for researchers and practitioners seeking to optimize the implementation of living lab methodologies in future projects, contributing to the broader discourse on collaborative innovation and stakeholder-driven development. Based on these findings, the paper proposes various mitigation strategies to address the identified challenges. These include the use of tailor-made methods to promote alignment between stakeholders and project staff, such as the IBA, the implementation of project management systems to monitor development progress and ensure that insights from user workshops are integrated, and the regular refinement of the communication approach during the iterative process and beyond.

This study offers valuable insights into the implementation of a living lab methodology in a project-based environment, but it also has a number of limitations. The analysis is based on a single case, the Möbius project, which means that the findings may not be generally applicable to other sectors or regions. The context-specific nature of publishing and digital storytelling may not provide an accurate picture of the challenges in other domains. Furthermore, although useful for gaining in-depth insight, the use of an action research approach may lead to bias due to the active involvement of the researchers.

Future research should examine how the application of alignment methods such as IBA can influence the challenges associated with the living lab methodology. Such insights could be valuable in understanding how different techniques and processes can contribute to the early identification and resolution of problems and conflicting assumptions that arise during a project. Furthermore, we suggest that future research explore the differences between project-based and long-term living labs. Such a comparison could shed light on the advantages and limitations of each approach and reveal opportunities to refine methodologies. Similar comparative studies would provide valuable perspectives on the flexibility and scalability of living lab approaches. Expanding the research to different sectors and cultural contexts would help validate the broader applicability of the findings, while involving external evaluators could reduce potential bias by researchers actively involved in implementation.

Taking these reflections into account, it is also important to propose relevant research questions that could guide future research. For example, how do stakeholder management strategies within consortia differ from those involving external stakeholders, especially under time constraints? Additionally, what impact do time constraints have on the effectiveness of stakeholder alignment methods? Investigating how the predefined outlines and objectives of specific innovation projects influence the outcomes of living labs could provide further insights. Furthermore, how can project management tools be optimized to improve stakeholder integration and communication effectiveness? Finally, understanding what specific factors contribute to the success or failure of mitigation strategies in both short- and long-term living lab environments would be invaluable for refining existing methods and improving project outcomes.