Next Article in Journal
Towards Sustainable Residential Buildings in Saudi Arabia According to the Conceptual Framework of “Mostadam” Rating System and Vision 2030
Next Article in Special Issue
The Defining Characteristics of Agroecosystem Living Labs
Previous Article in Journal
Sustainability in Railway Investments, a Study of Early-Phase Analyses and Perceptions
Previous Article in Special Issue
Facing Societal Challenges in Living Labs: Towards a Conceptual Framework to Facilitate Transdisciplinary Collaborations
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 2
10.3390/su13020791
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Living Labs: From Niche to Mainstream Innovation Management

by
Katharina Greve
1,2,*,
Riccardo De Vita
2,
Seppo Leminen
3,4,5 and
Mika Westerlund
5
1
Centre for Science, Technology & Innovation Policy, Institute for Manufacturing, University of Cambridge, Cambridge CB3 0FS, UK
2
The Networks and Urban Systems Centre (NUSC), Business School, University of Greenwich, London SE10 9LS, UK
3
Department of Business, Strategy and Political Sciences, University of South-Eastern Norway, 7053 Drammen, Norway
4
Department of Marketing, Aalto University School of Business, FI-00076 Aalto, Finland
5
Sprott School of Business, Carleton University, Ottawa, ON K1S 5B6, Canada
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(2), 791; https://doi.org/10.3390/su13020791
Submission received: 8 December 2020 / Revised: 9 January 2021 / Accepted: 11 January 2021 / Published: 15 January 2021
(This article belongs to the Special Issue Innovation Management in Living Labs)
Browse Figures
Versions Notes

Abstract

:
Living Labs have received increasing attention over the last decade. However, despite their growing popularity and ability to positively impact organisations’ innovation performance, mainstream innovation management literature has overlooked the diverse and promising Living Labs research landscape. In an effort to move the field forward, this study analyses extant Living Labs literature in the domain of innovation management. The study identifies conceptual bases informing Living Labs research, maps the collaboration between scholars in the field, examines prevailing themes influencing the debate and reveals the influence of Living Labs research on other domains. Bibliometric methods of co-authorship, keyword co-occurrence analysis as well as bibliographic coupling are employed on two databases. Database A includes 97 focal journal articles and Database B includes all cited sources of Database A, totalling 500 documents. This study reveals the rapid growth of the scholarly literature on Living Labs in the innovation management domain, driven by a core group of authors. However, other contributions from highly visible scholars have the potential to connect Living Lab research to mainstream innovation management studies. The study also identifies the influence of Living Labs research in different application fields and potential for its further evolution.

1. Introduction

Faced with increasingly complex challenges and accelerating change in their environment, organisations must continuously innovate to remain competitive. The limitations of internal research and development (R&D) processes has led to the adoption of more collaborative and open approaches to innovation, enabling organisations to gain access to scarce and previously inaccessible external resources [1,2,3,4,5]. Not only businesses, but also entire economies are paying closer attention to approaches that facilitate innovation across organisational and national boundaries [6]. The COVID-19 pandemic has highlighted the importance of such collaborative approaches to innovation, involving a range of different stakeholders to co-create new solutions in order to revive the economy and for the survival of many organisations [7].
Living Labs are an increasingly popular collaborative innovation approach that has received growing attention among innovation scholars, practitioners and policy makers [8,9,10]. Living Labs are often initiated and funded by policy makers with regional or national policy goals in mind [11], which positions them as “innovation intermediaries” filling the gap between R&D and market introduction [12]. Living Labs provide a collaborative platform for private and public sector innovation [13,14,15] and include three separate but intertwined layers: the Living Lab organisation, the Living Lab project(s), and the individual Living Lab user and stakeholder activities [16].
Following Westerlund et al. [10], this study considers Living Labs as platforms providing shared resources and bringing together a variety of private and public stakeholders to gather, create, communicate, and deliver new knowledge, validate existing products, services and processes, facilitate professional development and social impact in real-life contexts. Such platforms enable the co-creation process between universities, large organisations, Small and medium-sized enterprises (SMEs), start-ups and users as well other stakeholders [12,17]. This complex network of organisations and individuals, often described as a public–private-people partnership, enables participation through diverse activities and methods [18,19,20]. In Living Labs, close collaborations between different stakeholders and their networks accelerate the innovation process [21,22], as participants contribute heterogeneous resources and knowledge into joint innovation activities [23,24]. In essence, the Living Labs phenomenon is characterised by its open, inclusive, and collaborative approach to solve innovation challenges under realistic use conditions [25]. Unsurprisingly, Living Labs have gained real momentum during the last decade.
Benefits of Living Labs to businesses, policy makers, users and the wider society are diverse [15]. They help conducting experiments and obtaining user feedback by providing a place where co-creation is facilitated [26]. Living Labs are capable of tapping into tacit knowledge that can be used to translate latent user needs into novel products and services or improve existing ones [17,24]. Moreover, Living Labs provide governance and a structure for gathering users’ insights and filtering problems to support user entrepreneurship [27]. Aside from tangible outcomes, including designs, products, prototypes, solutions, and systems, Living Labs also generate intangible outcomes such as concepts, ideas, intellectual property rights, knowledge, and services [28,29,30]. These outcomes are resting on the Living Labs’ capacity to facilitate co-creation and enhance access to knowledge [22]. Co-creation decreases market risk when launching new offerings, increases the return on investment, and accelerates time to market [31].
Despite the growing popularity and the ability of Living Labs to positively impact innovation performance outcomes, including time, cost, quality, and go-to-market [32], the scholarly debate on the topic is dominated by a small number of researchers who actively contribute to the field [15]. In an effort to analyse extant literature and move the field forward, a few studies have begun to shed light on the structure of the Living Lab debate, conceptualisation, and theoretical roots [8,33]. For example, Leminen and Westerlund [33] outlined eight major research avenues that researchers in the Living Lab field have taken to study the phenomena. A more recent study took a cross-disciplinary perspective on the structure, concepts, and theoretical foundations of Living Labs [8]. Findings revealed that Living Lab research in the area of innovation management is a particularly prominent stream that continues to grow. However, the phenomenon is often considered a subfield of other more established and mainstream disciplines and paradigms, such as open and user innovation paradigms [25]. Living Labs are viewed as a link between open innovation and user innovation [19]. Open innovation is now a widely adopted concept in innovation management [34], and user innovation has become “a solid milestone within innovation management studies” [35], p. 7. While the potential to influence mainstream innovation management is evident to Living Labs researchers, the topic has not fully established itself in mainstream innovation management literature [8].
In order to capitalise from and inform mainstream innovation management literature, it is essential to take stock of the current Living Lab research landscape in the field of innovation management. To date, studies revealing the Living Labs debate and its conceptual and theoretical foundations in innovation management are scarce. To fill this gap, this study aims to identify the intellectual contours of the Living Labs field by analysing extant literature in the innovation management domain using a bibliometric approach. The study poses the following research questions: (i) How is the Living Labs research landscape characterised in context of innovation management? (ii) How do the Living Labs debate and key concepts contribute to future research and practice in the context of innovation management?
This study contributes to the Living Lab literature in several ways. Firstly, it unpacks the recent evolution of Living Labs studies and places them in the broader context of innovation management research. Secondly, it analyses the collaborative relationships linking scholars co-authoring papers and identifies a core of prolific authors, as well as transient, but influential contributions. Thirdly, it discusses the prevailing topics characterising the field. Finally, by looking at further research citing the Living Lab literature, this study maps influence and potential development of current research.
Following this introduction to the Living Lab phenomenon and its relevance in innovation management, the second section of this paper discusses the diverse nature of the Living Lab field. The third section describes the sample selection strategy as well as the bibliometric methods used to advance our understanding of the Living Lab debate in innovation management. The findings are presented in three subsections (i) analysing the roots of Living Lab studies, (ii) examining the current Living Lab debate, and (iii) revealing the influence of Living Lab research. To conclude, the theoretical contributions as well as the managerial implications of this study are discussed. Finally, the study’s limitations and future research directions are outlined.

2. Living Labs Research Landscape

The Living Labs field has become more and more salient. Attention to both the field (number of articles directly related to Living Labs) and influence (number of articles that cite Living Labs articles) have grown rapidly over the last decade. However, along with the fast expansion of the research domain, literature on Living Labs has become fragmented and diverse. Although it is not an exhaustive list, Table 1 provides a number of examples to highlight the diverse nature and understanding of Living Labs along seven dimensions: (1) definitions, (2) interpretation, (3) types of Living Labs, (4) stakeholders involved in Living Labs, (5) disciplines and concepts covering Living Lab research, (6) context of Living Labs, (7) perspective and level of analysis to examine the phenomena.
Innovation management scholars and practitioners have provided a number of definitions explaining what a Living Lab constitutes. In an attempt to clarify the concept, Dell’Era and Landoni [29] provided 13 definitions from both the academic and practitioner communities. However, they argued that these definitions ignore “the original new product development approach implied by the Living Lab methodology” (p. 139) and suggested that the focus lies in organisational features of the managing body or environmental characterisation. While these aspects are relevant, Dell’Era and Landoni [29] concluded that the definitions do not fully capture the methodological peculiarities of Living Labs. Revealing further complexities associated with extant Living Labs literature, Leminen [36] highlighted that there is no universally accepted distinction between the terms ‘living lab’, ‘living laboratory’, and ‘living labbing’. Rather, these terms are used interchangeably in the literature. Based on a review of 70 different definitions, Leminen [36] identified four characteristics or perspectives that define Living Labs: (1) real-life environments, (2) stakeholders, (3) approaches, instruments, methods, methodologies, and (4) concepts, conceptualisations and tools. Westerlund, Leminen and Habib [67] applied content analysis to 40 membership applications to the European Network of Living Labs (ENoLL) to reveal nine key constructs that characterise Living Labs and to offer insights that help in providing a definition of Living Labs as innovation platforms. The key constructs include (1) objective, (2) governance, (3) openness, (4) stakeholders, (5) funding, (6) value, (7) communications, (8) infrastructure, and (9) methods. Nevertheless, the large number of different definitions that are proposed in the literature to explain what a Living Lab is [68], and the absence of a widely recognised definition [66,69,70,71] indicate the lack of a common understanding of the concept and its underlying mechanisms [72].
Scholars tend to have different implicit understandings and interpretations of what a Living Lab constitutes and what the most important features are, and so do practitioners. For this reason, there are numerous projects that label themselves Living Labs, but in reality fail to cover elementary components of Living Labs. On the other end of the spectrum, there are initiatives that satisfy and exceed the criteria but do not call themselves Living Labs [73]. Dutilleul, Birrer and Mensink [37] highlighted five different meanings for the Living Labs concept in the literature, including (1) an innovation system consisting of organised and structured multi-disciplinary networks, fostering innovation and collaboration; (2) the in vivo monitoring of a “living” social setting, generally involving experimentation with a technology; (3) an approach for involving users in the product development process; (4) the organisations facilitating a network, maintaining and developing its technological infrastructure, and offering relevant services; and (5) the European Living Labs movement. The ambiguity related to the interpretation of the Living Lab term, despite most Living Labs sharing a common vision, has resulted in studies showing that “there is strong heterogeneity in terms of the interpretation and implementation of that vision, to the point that it is hard to compare actual experiences” [74], p. 4.
In addition to the large number of definitions and varying interpretations of the phenomena, scholars also distinguish between different types of Living Labs. For example, Leminen, Westerlund and Nyström [40] identified four distinct types of Living Labs characterised by open innovation. According to their (ibid.) study, Living Labs can be utiliser-driven, enabler-driven, provider-driven, or user-driven (or user–community-driven). Each of these types has a different actor who adopts the most active and driving role in guiding the innovation activities. The study further explains that Living Labs differ from each other with regards to their activities, structure, organisation, and coordination.
Scholars also emphasise the variety of stakeholders that are involved in Living Labs. These include suppliers, customers, users, competitors, universities, and other institutions and organisations [4]. Schuurman et al. [41] identified distinct roles of Living Lab actors, who include researchers, utilisers, enablers, users, providers. Moreover, Schuurman et al. [41] also explored their respective motivations to engage in a Living Lab. The scholars hypothesised that exploration is the main motive of utilisers to participate in Living Labs, and the role of researchers is seen as an intermediary between utilisers and users. Researchers are motivated by the opportunity to exploit implementable knowledge and explore new knowledge. Instead, the providers are expected to be motivated by technology and/or knowledge exploitation opportunities, whereas users are driven by intrinsic motivations [24]. On the other hand, enablers contribute to the Living Lab with financial support or other assets that allow Living Labs operations to be carried out, and hence, expect the Living Lab to realise some predefined policy objectives.
The diverse and scattered nature of the field [33] is also evident from the variety of contexts in which the phenomena are observed. As indicated in Table 1, existing studies compare projects and experiences across Living Labs in different countries and sectors [40]. For example, the Living Labs approach is studied in relation to information and communication technologies (ICTs) [45], smart cities and digital cities [4,55], health care [47,48,49], agriculture [51] and many more. Living Labs are also investigated in both business-to-consumer as well as business-to-business contexts [42], in developed and developing countries [44]. The application of the concept to a range of different contexts points towards the relevance and potential benefits of Living Labs. However, any attempt to transfer insights across different contexts creates the risk of ignoring specific factors, which may lead to overgeneralisations of research findings.
Further, extant literature highlights different perspectives and level of analysis [56]. For example, with regards to innovation research and innovation management, scholars have analysed Living Labs as innovation intermediaries [58,59], placing Living Labs in the domain of user-driven innovation methodologies [60,61], or as a method in the plethora of design research methodologies [29]. Other research has examined specific cases of Living Lab development [63,64].
Given the diverse areas of application and perspectives, it comes as no surprise that the disciplines and concepts covering Living Labs research are equally varied. Indeed, Bergvall-Kåreborn et al. [65] referred to Living Labs as a multi-disciplinary concept. For example, Living Labs research covers disciplines ranging from innovation management to information science, among others [54,66], adding to the fragmented nature of the field. However, Living Labs research in the area of innovation management is a particularly prominent stream that continues to grow [8]. Indeed, concepts such as open and user innovation are commonly associated with Living Labs studies [19]. Despite its potential to influence mainstream innovation management research as well as provide guidance to practitioners and policy makers, the Living Labs debate has to date not fully established itself in mainstream innovation management literature [8].
Finally, the diverse research landscape, together with ambiguous theorisations, impedes progress in this domain and hinders the development of an integrated conceptual framework and robust empirical inquiries [16]. In an effort to move the field forward to mainstream innovation management, this study analyses the roots of the current Living Labs debate, reveals the influence of Living Labs research on other studies, and discusses how the insights can inform future innovation management research.

3. Methodology

3.1. Sample Selection

A systematic process led to the creation of two databases that were employed in this study. Database A included 97 focal journal articles. Database B included all documents citing any of the papers included in Database A.
Due to its wide coverage and relevance for innovation management research, Web of Science (WoS) was used to retrieve articles for both Databases A and B. As opposed to, for example, Scopus, WoS indexes also the Technology Innovation Management (TIM) Review, which is the journal with the largest number of special issues and articles on Living Labs to date [10]. Moreover, WoS has already been employed to analyse similar domains, such as open innovation [75], technology business incubation [76], and Living Labs [8].
The following steps describe how the sample of 97 focal articles and subsequently the sources citing them were obtained. Firstly, particular search criteria were identified to capture all variations of the Living Labs concept. Therefore, synonyms of “living lab*” such as living laboratory and living labbing were included [8,25]. Following comparable studies [77,78], our study focused only on peer-reviewed journal articles in its analysis of focal articles and, therefore, books and conference papers were excluded from Database A. In line with the objective of this study, only articles on Living Labs published in journals declaring in their aims and scope innovation or innovation management as an area of interest, were considered. The abstract, title, and keywords of papers in Database A were independently reviewed by two of the authors of this study to decide on their inclusion in the analysis [79,80]. When the two authors were not in agreement, further discussions took place and, when necessary, a third author decided on their suitability for inclusion [81].
As a result of this systematic review, articles on Living Labs published in journals outside the innovation and innovation management scope were excluded. Moreover, articles employing the term “living laboratory” in its metaphorical meaning were not considered (see, e.g., [82]). False positive results such as studies referring to “living laboratory animals” in the context of medical research were excluded (e.g., [83]). By the same token, studies investigating “living labour” were excluded (e.g., [84]). At the end of the process, 97 focal articles were included in the sample representing Database A.
This paper took an approach similar to Randhawa et al. [78], trying to capture the roots and impact of the focal articles included in Database A. For this reason, a second database, named Database B, was created. Database B included 500 documents recorded in WoS Core Collection citing at least one of the articles included in Database A, for a total of 500 documents: 419 articles and proceedings, 38 reviews, 29 books, book chapters, and book reviews, and 14 editorial materials. The two databases were analysed using different techniques, which are described in the following section.

3.2. Analysis

To both identify the theoretical bases of Living Labs research and analyse their implications for different disciplines and practice, this study employed a number of techniques to analyse the current Living Labs debate, its conceptual roots, as well as its diffusion. Figure 1 visualises the approach of the study. Following the identification of the 97 focal articles (Database A) representing the current debate around Living Labs in the broader innovation management literature, the theoretical bases of such debate were extrapolated by considering the references cited by papers in the database. Analysis of cited references is often performed to identify the conceptual roots of a specific field (e.g., [85]). In our study, we first identified the most commonly cited references as particularly influential in shaping the Living Labs debate, and then performed a bibliographic coupling analysis. Bibliographic coupling associated papers included in Database A if they cited a common reference. Such an approach was useful to identify intellectual communities [86] in a specific field.
Consistent with recent studies [8], the current debate was then analysed, trying to measure the collaboration within the scholarly community captured through a co-authorship analysis and the focus on specific themes. Such themes were detected performing a keyword co-occurrence analysis. Keyword co-occurrence analysis was also applied to the set of 500 documents citing Database A articles. The objective of the analysis was to identify which other fields and areas were influenced by Living Labs research.
Analysis was performed using the social network analysis and visualisation software Visone [87], Ucinet [88], and VOSviewer [89]. Figure 1 provides an overview of the analysis of Database A as well as Database B highlighting their objectives and focus. In line with the aim of this study, findings of this study were presented with respect to three subsections: (i) analysing the roots of Living Labs studies, (ii) examining the current Living Labs debate, and (iii) revealing the influence of Living Labs research.

4. Results

4.1. Analysing the Roots of Living Lab Studies

The most common citations of the papers included in Database A were identified to uncover the intellectual roots of the studies discussing Living Labs in the context of innovation management. Table 2 presents the 21 citations appearing at least 10 times in the reference lists of the 97 papers included in the database. It has to be noted that it was not possible to download cited references for all the papers in the database. Eighteen articles had to be excluded from this analysis for this reason. The table clearly identifies the influence of specific authors in shaping the debate surrounding Living Labs. It is interesting to observe that one of the most cited references was a PhD dissertation and another one was the famous article by [90] about the generation of theories from case studies. These factors were consistent with the still nascent nature of Living Lab research.
Bibliographic coupling, the sharing of common cited references between two documents, was used to measure how much the papers included in Database A moved from the same theoretical background. Out of the 73 articles sharing at least one reference, only 35 shared more than five; furthermore, such articles were often written by the same author(s) (Figure 2). While such a result can suggest a rather fragmented situation in the field, it is again important to acknowledge the nascent state of the Living Lab literature and of its academic community.

4.2. Examining the Current Living Labs Debate

In total, 97 journal articles were identified through Web of Science that form the basis of this study. The articles were published during the last decade, from 2010 to 2020. Due to the time it takes to index articles in Web of Science, articles published in 2020 may be underrepresented. Taking this into consideration, the data suggested that the Living Labs literature continued to grow, as highlighted in Figure 3. The study was conducted in October 2020 and thus only included articles published and indexed in WoS until then; considering the time required for indexing, articles in 2020 may be underrepresented. Funding associated with the research presented in these articles was largely obtained through the European Commission or national government departments and research councils. However, 82% of the articles had no funding information attached.
The 97 articles were published in 30 different journals. However, 75% of all articles were disseminated by only six journals and the remaining 25% of articles were each published in a different journal. Interestingly, TIM Review represents an essential outlet for Living Labs studies, having published 60 out of 97 articles. Among the highly ranked journals (3 and 4 * according to Chartered Association of Business School (ABS) ranking, Academic Journal Guide (AJG) 2018) that have published Living Labs research are R&D Management [98], Research Policy [99] and Harvard Business Review [100]. The articles published in these journals were very recent, which could be a signal that the Living Labs field is expanding into more mainstream innovation management and high quality academic and practitioner-focused journals.
Despite the limitations of citation metrics as performance measures [101], citations represented a good impact indicator of “the relative scientific significance or ‘quality’ of papers” [102], p. 21. The top 10 articles, based on their total number of citations (the citation count only included the number of times the publication was cited by articles from sources included in WoS; citations from other publications and sources were therefore excluded) as well as their key contributions, are summarised in Table 3. Unsurprisingly, eight out of 10 articles were published in TIM Review, further emphasising the journal’s key role in promoting Living Labs research. The contents of the top cited articles revealed the nascent state of the Living Labs literature as they mainly defined the concept, its characteristics, functions, and outcomes as well as positioned Living Labs in the broader landscape of innovation approaches.
Furthermore, it was possible to observe how some of the most cited articles overlapped with the list of sources identified in Table 2. In other words, several of the references most cited by the articles included in Database A were already part of the database itself, confirming the very recent nature of the field as well as the existence of a close community of scholars who gave birth to the field and still influence its debate. The most cited articles identified through a keyword search, indeed, often also appeared as the most cited sources of the papers resulting from the same search.
Furthermore, current Living Labs research was also analysed exploring patterns of collaboration between authors. Figure 4 connects authors if they co-authored a paper. The thickness of the lines is associated with the number of papers two scholars have co-authored together. The size of the nodes in the network represents the number of articles included in the database published by each author. Finally, the intensity of the colour captures the number of citations received by the articles included in the database written by each author (the number of citations received was based on articles included in the database Web of Science—Core Collection); in other words, the more intense the colour of the nodes, the more often an article was cited by subsequent research.
The 97 focal articles were published by 186 unique authors. A total of 159 authors (85.5%) were associated with only one of the papers included in the database, signalling the lack of a core of permanent contributors to the scholarly community studying Living Labs from the perspective of innovation management. As a result, apart from one specific subgroup of scholars, collaboration was particularly fragmented and often relied on single episodes of collaboration. Such a tendency has been identified by recent studies as a characteristic of the broader Living Labs debate, not only in relation to the innovation management area [‎8].
When considering the number of citations received, it was interesting to observe that authors with a limited number of articles and not necessarily part of the largest connected subgroup, also could attract many citations. An example is provided in Figure 5, which zooms in on the triad highlighted in Figure 4. The triad depicts the collaboration between Wareham, Almirall (who co-authored two papers), and Lee (who is also an author on one of the papers written by Wareham and Almirall). While the number of publications was not as high as the one of other authors, in proportion, the articles attracted a vast number of citations (155 for Wareham and Almirall, 86 for Lee). This is explained by the broader visibility of the authors in the more mainstream scholarly community and their contributions to the open-innovation field, and also through publications in highly ranked journals.
In other words, while there seemed to exist a core group of researcher particularly concerned with the study of Living Labs within the broader field of innovation management, it was also apparent that influential contributions were produced outside of such a group. This could be the result of the prominence of individual authors in cognate fields or the visibility of specific journals where their articles are published. Engaging with highly impactful authors currently occupying peripheral positions in the Living Lab collaboration network is an important opportunity for the Living Lab scholarly community to increase its visibility. At the same time, it is equally vital to integrate such contributions and ensure this will not lead to a further fragmentation in the debate. Table 4 presents the most prominent authors in terms of number of articles published, number of collaborative connections, and citations received, providing further evidence to the considerations developed so far.
After commenting on the whole network, Figure 6 focuses on its main component [‎104]. This area of the co-authorship network was particularly interesting, as it contained the largest number of authors directly or indirectly connected through collaborative relationships. The group involved 37 (almost 20%) of the authors included in the database.
It was also interesting to map the evolution of the network over the past decade. Figure 7 presents the growth of the co-authorship network for the period 2010–2020. Authors who are part of the main component of the network are identified in turquoise. The figure highlights how some of the authors who are part of the largest component have contributed to innovation management studies about Living Labs since the very beginning. It also transpires how such authors progressively attracted a larger number of co-authors over the years, while at the same time strengthening the relationships between themselves. It is also evident how the scholarly community started to grow from 2012 and 2013. In this respect it is important to mention the role of special issues on Living Labs, the majority of which were published by TIM Review, in acting as a catalyst for the academic debate [‎‎8,‎10]. Collaboration linking authors was often associated with geographical proximity: for example, Seppo Leminen and Mika Westerlund collaborated largely with scholars from Finland, whereas Anna Ståhlbröst was part of the research network of Swedish scholars; Dimitri Schuurman and Lieven De Marez often engaged with researchers from Belgium.
The data were also analysed to produce a network showing the co-occurrence of keywords to capture the main themes characterising the debate, as well as their association. Figure 8 identifies author-defined keywords appearing together in the papers included in the database. The data included 240 unique keywords. To improve readability, the figure includes only those keywords appearing at least twice across the 97 papers. Furthermore, the keyword “Living Lab” was also removed from the visualisation, given its prominence. It has to be noted that not all papers included keywords. Furthermore, author-defined keywords were also homogenised. For example, the words “living lab”, “living labs”, “living laboratory” were all identified as “living lab”. Similarly, “co-creation” and “co-creation method” were grouped under the same label. Finally, repetitions of the same keyword in the same paper were disregarded (e.g., “KPIs” and “Key performance indicators”. Keywords were automatically clustered in separate groups identified by different colours by the software VOSviewer [‎89]. The clustering identified the centrality of innovation management concepts such as “open innovation”, “innovation ecosystem” and “co-creation”. At the same time, however, specific subfields and applications emerged: user innovation seemed to be associated with the experimentation phase in order to test an idea prior to the scaling-up; Living Labs found also specific application in the context of smart cities and of start-up initiatives.

4.3. Revealing the Influence of Living Labs Research

The 97 articles discussing Living Labs in the area of innovation management were cited by 500 documents, which were included in Database B. Their analysis allowed to discuss the influence of the 97 focal articles on the following research. The 500 citing documents excluded self-citations. An overview of the top journals, top research areas, and most frequent keywords, characterising documents belonging to Database B, is presented in Table 5.
Considering the most dominant journals, research areas, and keywords among the cited documents, it became apparent that environmental aspects and sustainability-related studies represented a key area that Living Lab studies seemed to influence. Moreover, attention was also paid to Urban Living Labs and Smart Cities. Finally, common terminology related to the characteristics of Living Labs was featured as the most common keywords. These included, for example, innovation, co-creation, open innovation, collaboration, collaborative innovation, innovation ecosystem, and participatory design.
Such considerations were corroborated by the analysis of keyword co-occurrence. Figure 9 presents the co-occurrence of the author-defined keywords of the documents included in Database B. Also, in this case a process of amalgamation of keywords was carried out by the authors of the study. The visualisation only focuses on keywords appearing at least four times and only co-occurrences of values 2 and higher are visualised. The figure summarises the main themes of the documents citing the papers included in Database A and their association. By focusing on the documents citing the 97 focal papers, the figure captures other debates influenced by the study of Living Labs.
It was evident how the idea of “Living Lab” remained as one the keywords (97 occurrences). This was particularly important as it shows how the field maintained its identity. Furthermore, documents citing the original papers often placed Living Labs within the field of open innovation or in relation to specific subdomains (e.g., social innovation, user innovation, collaborative innovation) [19]. The keywords identified suggest that Living Labs are of particular interest, thanks to the involvement of users (user involvement, user experience) in the innovation journey.
An area of application where Living Labs seemed to have been studied with particular interest was the one of smart cities, sometimes in relation to the development of sustainability-related initiatives. The growth of the implementation of Living Labs at the level of cities has led to the emergence of the specific notion of “urban Living Lab”, also captured by the keyword co-occurrence map.
The figure also shows areas that can be further integrated within the Living Labs domain. The 500 documents citing the original database, indeed pointed, for example, to the possibility to explore the potential of Living Labs for SMEs or public initiatives, or as part of broader innovation ecosystems. Finally, the presence of keywords such as “literature review” or “case study” signals how the field is still evolving and has not reached its maturity yet [8].

5. Conclusions

5.1. Theoretical Contributions

This study addressed the need for a better understanding of the evolving Living Labs research landscape. Different from previous studies providing broad overviews of the field of Living Labs [8,25], this work focused tightly on the evolution of the field within the innovation management area. Using a bibliometric approach, the study characterised the academic debate about Living Labs and identified the area’s theoretical roots as well as the potential to influence future research and practice. This research makes several important contributions to the Living Labs literature.
First, the overall picture emerging from the analysis suggests that the research community is partially fragmented, with only a small core of authors consistently publishing in this domain. Such a situation is not surprising, especially when considering the potential applications of Living Labs in many disciplines and the still nascent state of the field [8,10]. Somehow finding similar patterns, for example, Raasch et al. [105] studied open source software as an example of interdisciplinary research field and identified a shift from interdisciplinarity to multi-disciplinarity taking place in less than 10 years. On the one hand, the presence of authors from a multiplicity of fields and disciplines can make the Living Labs phenomenon have an impact in different fields; the analysis of the papers citing the articles included in Database A confirms the potential of Living Labs research. On the other, however, when the community becomes too fragmented and fails to share common theoretical foundations, there is the risk for the field to lose consistency and suffer from the lack of common definitions and understandings.
It is therefore important for the Living Labs research community to mitigate some risks and build a cohesive core that is well placed within the more mainstream innovation literature. The core community has so far contributed to the advancement of Living Labs research through a relatively narrow set of innovation management journals, many of which may not be included in common journal quality lists or are not ranked as top journals in the area. While this can be seen as a common issue for most novel topics, the Living Labs community needs to root the debate on Living Labs in higher ranked and more widely read mainstream innovation management journals to foster further advancement of the debate. Examples discussed in this article highlighted how contributions from “occasional” authors with high visibility in related fields, even if not central to the Living Labs debate, can attract a lot of citations. A potential suggestion for the Living Labs field is to engage with prominent authors in the open innovation field, for example, inviting them as editors of special issues on Living Labs. Special issues, indeed, were found to play a fundamental role in shaping the academic debate and growing the community and scholarly interest in the area.
Maintaining cohesion by reaching out to other researchers in what is still a relatively small community is fundamental to ensure a robust growth of the research domain. Some scholars might find the results presented in this work useful to identify brokering opportunities and to reach out to other colleagues in the field to link otherwise disconnected research teams. In this way, the scholarly community can plan their Living Labs research efforts more strategically, as the field is growing rapidly.
The study also shows the importance of open access journals in supporting the evolution and diffusion of Living Labs research. Indeed, outlets such as TIM Review play a major role in making the scholarly insights available to not only researchers but also a wider array of practitioners and policy makers, or other interested stakeholders. By increasing visibility and access to research on Living Labs through open access journals such as Sustainability, it will be possible to amplify the power to inform, educate, and enlighten a wide audience reaching beyond academia.

5.2. Managerial Implications

Our study maps the Living Labs research landscape in the innovation management domain. Such results have the potential to inform practitioners and managers in diverse fields. Among them, studies extensively discuss the application of Living Labs to a variety of contexts, including but not limited to ICT [45], assisted living [46], health care [47,48,49], media [50], agriculture [51], mobility [52,53], urban and rural areas, smart cities and digital cities, as well as buildings [4,54,55].
Living Labs research not only informs practice, but it is also regularly driven by people directly involved in innovation projects. In their review of the early days of Living Labs, Leminen and Westerlund [54] discussed the key roles of certain researching and practicing individuals in the establishment of Living Labs in the United States and Europe. Indeed, boundaries between Living Labs research and practice can be blurred and intertwined. For example, prominent Living Labs scholars such as Dimitri Schuurman not only publish articles on the topic (e.g., [12] but also manage innovation projects and develop specific Living Labs offerings targeted at entrepreneurs [106]. Similarly, Chris McPhee, previously the editor-in-chief of TIM Review and a person engaged with numerous special issues on Living Labs in the journal, currently acts as innovation management specialist to foster Living Labs research and practice in the agricultural and agri-food sector in Canada. Also, Christofer Daiberl, managing director of JOSEPHS®, a Living Lab in Germany, is combining research in the area of Living Labs with practice (e.g., [107]).
Finally, organisations such as the European Network of Living Labs (ENoLL) play an important role in bridging research-based knowledge with practice and facilitating the process of building greater competence and experience in the Living Labs domain. In order to move the Living Labs field forward, further opportunities to connect communities need to be sought. On one hand, ENoLL has the potential to further strengthen the link between Living Labs scholars, policy makers, and practitioners in the field through, for example, their Living Lab days or other Living Lab-specific events. On the other hand, special tracks and special issues organised through conferences such as ISPIM, OpenLivingLab Days, and others provide opportunities to build bridges between the Living Labs community and scholars as well as practitioners in the broader innovation management field. This study suggests to build on such efforts more systematically in order to attract more attention from innovation management scholars and to grow the Living Labs research community and its core.

5.3. Limitations and Future Research Directions

Like any other study, our research has some limitations. While the research followed a systematic approach, relevant articles discussing Living Labs in the context of innovation management might have been overlooked if not published in any of the journals included in the study or indexed in Web of Science. Furthermore, while Web of Science is widely used to conduct bibliometric research, some information is incomplete or partially inconsistent. Despite the efforts of the authors in ensuring consistency in the data set, the process introduced the possibility of errors; finally, some analyses are limited or impacted by data availability.
Notwithstanding these limitations, the study paves the way for promising future research opportunities. The relevance of co-authorship in the emergent field of Living Labs research, for example, calls for deeper investigation into those factors driving collaboration; furthermore, as the study identified examples of scholars collaborating based on geographical proximity, studies in languages other than English can be analysed to explore the development of Living Labs research in different countries. Finally, an analysis of the full text of the selected papers can provide further insights into the discourse surrounding Living Labs research.

Author Contributions

Conceptualization, K.G., S.L. and M.W.; methodology, K.G. and R.D.V.; software, R.D.V.; formal analysis, R.D.V. and K.G.; data curation, K.G. and S.L.; writing—original draft preparation, K.G.; writing—review and editing, K.G., R.D.V., S.L. and M.W.; visualization, K.G. and R.D.V. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Restrictions apply to the availability of these data. Data was obtained from Web of Science and are available with the permission of Web of Science.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bogers, M.; Zobel, A.; Afuah, A.; Almirall, E.; Dahlander, L.; Frederiksen, L.; Gawer, A.; Haefliger, S.; Hagedoorn, J.; Hilgers, D.; et al. The open innovation research landscape: Established perspectives and emerging themes across different levels of analysis. Ind. Innov. 2017, 2716, 1–33. [Google Scholar] [CrossRef]
  2. Drechsler, W.; Natter, M. Understanding a firm’s openness decisions in innovation. J. Bus. Res. 2012, 65, 438–445. [Google Scholar] [CrossRef]
  3. Leminen, S.; Nyström, A.G.; Westerlund, M. Change processes in open innovation networks—Exploring living labs. Ind. Mark. Manag. 2020, 91, 701–718. [Google Scholar] [CrossRef]
  4. Leminen, S.; Rajahonka, M.; Westerlund, M. Towards third-generation living lab networks in cities. Technol. Innov. Manag. Rev. 2017, 7, 21–35. [Google Scholar] [CrossRef] [Green Version]
  5. Mirvis, P.; Herrera, M.E.B.; Googins, B.; Albareda, L. Corporate social innovation: How firms learn to innovate for the greater good. J. Bus. Res. 2016, 69, 5014–5021. [Google Scholar] [CrossRef]
  6. Lee, S.; Lee, H.; Lee, C. Open innovation at the national level: Towards a global innovation system. Technol. Forecast. Soc. Change 2020, 151, 119842. [Google Scholar] [CrossRef]
  7. Chesbrough, H. To recover faster from Covid-19, open up: Managerial implications from an open innovation perspective. Ind. Mark. Manag. 2020, 88, 410–413. [Google Scholar] [CrossRef]
  8. Greve, K.; Leminen, S.; De Vita, R.; Westerlund, M. Unveiling the diversity of scholarly debate on living labs: A bibliometric approach. Int. J. Innov. Manag. 2020, 1–25, online ready. [Google Scholar] [CrossRef]
  9. Voytenko, Y.; McCormick, K.; Evans, J.; Schliwa, G. Urban living labs for sustainability and low carbon cities in Europe: Towards a research agenda. J. Clean. Prod. 2016, 123, 45–54. [Google Scholar] [CrossRef] [Green Version]
  10. Westerlund, M.; Leminen, S.; Rajahonka, M. A Topic Modelling Analysis of Living Labs Research. Technol. Innov. Manag. Rev. 2018b, 8, 40–51. [Google Scholar] [CrossRef] [Green Version]
  11. Katzy, B.R. Designing viable business models for living labs. Technol. Innov. Manag. Rev. 2012, 2, 19–24. [Google Scholar] [CrossRef]
  12. Schuurman, D.; Herregodts, A.-L.; Georges, A.; Rits, O. Innovation Management in Living Lab Projects: The Innovatrix Framework. Technol. Innov. Manag. Rev. 2019, 9, 63–73. [Google Scholar] [CrossRef]
  13. Criado, J.I.; Dias, T.F.; Sano, H.; Rojas-Martín, F.; Silvan, A.; Filho, A.I. Public Innovation and Living Labs in Action: A Comparative Analysis in post-New Public Management Contexts. Int. J. Public Adm. 2020, 1–14. [Google Scholar] [CrossRef]
  14. Gascó, M. Living labs: Implementing open innovation in the public sector. Gov. Inf. Q. 2017, 34, 90–98. [Google Scholar] [CrossRef]
  15. Greve, K.; Jonas, J.M.; Neely, A.; Möslein, K.M. Unlocking Unique Value through Co-creation in Open Laboratories. In Innovating in the Open Lab: The New Potential for Interactive Value Creation across Organizational Boundaries; Fritsche, A., Jonas, J.M., Roth, A., Möslein, K.M., Eds.; De Gruyter: Berlin, Germany, 2020; pp. 81–92. [Google Scholar]
  16. Schuurman, D. Bridging the gap between open and user innovation? Exploring the value of living labs as a means to structure user contribution and manage distributed innovation. Ph.D. Thesis, Ghent University, Ghent, Belgium and Vrije Universiteit Brussel, Brussels, Belgium, 2015. [Google Scholar]
  17. Franz, Y. Designing social living labs in urban research. Info 2015, 17, 53–66. [Google Scholar] [CrossRef]
  18. Leminen, S.; Westerlund, M. Categorization of innovation tools in living labs. Technol. Innov. Manag. Rev. 2017, 7, 15–25. [Google Scholar] [CrossRef]
  19. Schuurman, D.; De Marez, L.; Ballon, P.; Felton, E. Open innovation processes in living lab innovation systems: Insights from the LeYLab. Technol. Innov. Manag. Rev. 2013, 3, 28–36. [Google Scholar] [CrossRef]
  20. Sutinen, P.; Erkkilä, K.; Wollstén, P.; Hagman, K.; Hirvikoski, T.; Äyväri, A. KYKY Living Lab Handbook for Co-Creation by Schools and Companies; City of Espoo: Espoo, Finland, 2016. [Google Scholar]
  21. Nyström, A.G.; Leminen, S.; Westerlund, M.; Kortelainen, M. Actor roles and role patterns influencing innovation in living labs. Ind. Mark. Manag. 2014, 43, 483–495. [Google Scholar] [CrossRef] [Green Version]
  22. Leminen, S.; Nyström, A.-G.; Westerlund, M.; Kortelainen, M.J. The effect of network structure on radical innovation in living labs. J. Bus. Ind. Mark. 2016, 31, 743–757. [Google Scholar] [CrossRef]
  23. Edwards-Schachter, M.E.; Matti, C.E.; Alcántara, E. Fostering quality of life through social innovation: A living lab methodology study case. Rev. Policy Res. 2012, 29, 672–692. [Google Scholar] [CrossRef] [Green Version]
  24. Leminen, S.; Westerlund, M. Towards innovation in living labs networks. Int. J. Prod. Dev. 2012, 17, 43. [Google Scholar] [CrossRef]
  25. Hossain, M.; Leminen, S.; Westerlund, M. A systematic review of living lab literature. J. Clean. Prod. 2019, 213, 976–988. [Google Scholar] [CrossRef]
  26. Hakkarainen, L.; Hyysalo, S. The Evolution of Intermediary Activities: Broadening the Concept of Facilitation in Living Labs. Technol. Innov. Manag. Rev. 2016, 6, 45–58. [Google Scholar]
  27. Hakkarainen, L.; Hyysalo, S. How do we keep the living laboratory alive? learning and conflicts in living lab collaboration. Technol. Innov. Manag. Rev. 2013, 3, 16–22. [Google Scholar]
  28. Buhl, J.; von Geibler, J.; Echternacht, L.; Linder, M. Rebound effects in Living Labs: Opportunities for monitoring and mitigating re-spending and time use effects in user integrated innovation design. J. Clean. Prod. 2017, 151, 592–602. [Google Scholar]
  29. Dell’Era, C.; Landoni, P. Living lab: A methodology between user-centred design and participatory design. Creat. Innov. Manag. 2014, 23, 137–154. [Google Scholar] [CrossRef]
  30. Evans, J.; Jones, R.; Karvonen, A.; Millard, L.; Wendler, J. Living labs and co-production: University campuses as platforms for sustainability science. Curr. Opin. Environ. Sustain. 2015, 16, 1–6. [Google Scholar] [CrossRef]
  31. Niitamo, V.; Westerlund, M.; Leminen, S. A small-firm perspective on the benefits of living labs. Technol. Innov. Manag. Rev. 2012, 2, 44–49. [Google Scholar]
  32. Dell’Era, C.; Landoni, P.; Gonzalez, S.J. Investigating The Innovation Impacts of User-Centred And Participatory Strategies Adopted By European Living Labs. Int. J. Innov. Manag. 2019, 23, 1950048. [Google Scholar]
  33. Leminen, S.; Westerlund, M. A framework for understanding the different research avenues of living labs. Int. J. Technol. Mark. 2016, 11, 399–420. [Google Scholar]
  34. Bogers, M.; Chesbrough, H.; Moedas, C. Open innovation: Research, practices, and policies. Calif. Manage. Rev. 2018, 60, 5–16. [Google Scholar] [CrossRef]
  35. Schiavone, F. User Innovation in Healthcare: How Patients and Caregivers React Creatively to Illness, 1st ed.; Springer International Publishing: Cham, Switzerland, 2020. [Google Scholar] [CrossRef]
  36. Leminen, S. Living Labs as Open Innovation Networks. Networks, Roles and Innovation Outcomes. Ph.D. Thesis, Aalto University, Espoo, Finland, 2015. [Google Scholar]
  37. Dutilleul, B.; Birrer, F.A.J.; Mensink, W. Unpacking european living labs: Analysing innovation’s social dimensions. Cent. Eur. J. Public Policy 2010, 4, 60–85. [Google Scholar]
  38. Ballon, P.; Schuurman, D. Living labs: Concepts, tools and cases. Info 2015, 17, 1–11. [Google Scholar] [CrossRef]
  39. Mitchell, W.J. Me++: The Cyborg Self and the Networked City; MIT Press: Cambridge, MA, USA, 2003. [Google Scholar]
  40. Leminen, S.; Westerlund, M.; Nyström, A.G. Living labs as open-innovation networks. Technol. Innov. Manag. Rev. 2012, 2, 6–11. [Google Scholar] [CrossRef]
  41. Schuurman, D.; Baccarne, B.; De Marez, L.; Veeckman, C.; Ballon, P. Living labs as open innovation systems for knowledge exchange: Solutions for sustainable innovation development. Int. J. Bus. Innov. Res. 2016, 103, 322–340. [Google Scholar] [CrossRef]
  42. D’Hauwers, R.; Herregodts, A.-L.; Georges, A.; Coorevits, L.; Schuurman, D.; Rits, O.; Ballon, P. Overcoming Barriers to Experimentation in Business-to-Business Living Labs. Technol. Innov. Manag. Rev. 2017, 7, 20–26. [Google Scholar] [CrossRef]
  43. Westerlund, M.; Leminen, S. Managing the challenges of becoming an open innovation company: Experiences from living labs. Technol. Innov. Manag. Rev. 2011, 1, 19–25. [Google Scholar] [CrossRef]
  44. Baelden, D.; Audenhove, L.; Van Audenhove, L. Participative ICT4D and living lab research: The case study of a mobile social media application in a rural Tanzanian University setting. Telemat. Inform. 2015, 32, 842–852. [Google Scholar] [CrossRef]
  45. Følstad, A. Towards a living lab for development of online community services. Electron. J. Virtual Organ. Netw. 2008, 10, 47–58. [Google Scholar]
  46. Panek, P.; Zagler, W.L. A Living Lab for Ambient Assisted Living in the Municipality of Schwechat. LNCS 2008, 5105, 1008–1015. [Google Scholar] [CrossRef]
  47. Kanstrup, A.M.; Bjerge, K.; Kristensen, J.E. A living laboratory exploring mobile support for everyday life with diabetes. Wirel. Pers. Commun. 2010, 53, 395–408. [Google Scholar] [CrossRef]
  48. Agogué, M.; Comtet, G.; Menudet, J.F.; Picard, R.; Le Masson, P. Managing innovative design within the health ecosystem: The living lab as an architect of the unknown. Manag. Avenir Sante 2013, 1, 17–32. [Google Scholar] [CrossRef]
  49. Swinkels, I.C.S.; Huygens, M.W.J.; Schoenmakers, T.M.; Nijeweme-D’Hollosy, W.O.; Van Velsen, L.; Vermeulen, J.; Schoone-Harmsen, M.; Jansen, Y.J.; Van Schayck, O.C.; Friele, R.; et al. Lessons learned from a living lab on the broad adoption of eHealth in primary health care. J. Med. Internet Res. 2018, 20, 83. [Google Scholar] [CrossRef] [PubMed]
  50. Schuurman, D.; De Moor, K.; De Marez, L.; Evens, T. A living lab research approach for mobile TV. Telemat. Inform. 2011, 28, 271–282. [Google Scholar] [CrossRef]
  51. Wolfert, J.; Verdouw, C.N.; Verloop, C.M.; Beulens, A.J.M. Organizing information integration in agri-food-A method based on a service-oriented architecture and living lab approach. Comput. Electron. Agric. 2010, 70, 389–405. [Google Scholar] [CrossRef]
  52. Agerskov, M.L.; Høj, J.C.L. Lessons learned from the Danish EV living lab. In 2013 World Electric Vehicle Symposium and Exhibition (EVS27); IEEE: Barcelona, Spain, 17–20 November 2013; pp. 1–7. [Google Scholar]
  53. Sjöman, M.; Ringenson, T.; Kramers, A. Exploring everyday mobility in a living lab based on economic interventions. Eur. Transp. Res. Rev. 2020, 12, 1–17. [Google Scholar] [CrossRef]
  54. Leminen, S.; Westerlund, M. Living labs: From scattered initiatives to a global movement. Creat. Innov. Manag. 2019, 28, 250–264. [Google Scholar] [CrossRef]
  55. Mulder, I.; Marseille, J. Co-creating sociable smart city futures. In The Routledge Companion to Smart Cities; Willis, K.S., Aurigi, A., Eds.; Routledge: New York, NY, USA, 2020. [Google Scholar]
  56. Shin, D. A living lab as socio-technical ecosystem: Evaluating the Korean living lab of internet of things. Gov. Inf. Q. 2019, 36, 264–275. [Google Scholar] [CrossRef]
  57. Almirall, E.; Wareham, J. Living Labs: Arbiters of mid- and ground-level innovation. Technol. Anal. Strateg. Manag. 2011, 23, 87–102. [Google Scholar] [CrossRef]
  58. Katzy, B.R.; Baltes, G.H.; Gard, J. Concurrent process coordination of new product development by living labs—An exploratory case study. Int. J. Prod. Dev. 2012, 17, 23. [Google Scholar] [CrossRef]
  59. Ståhlbröst, A. A living lab as a service: Creating value for micro-enterprises through collaboration and innovation. Technol. Innov. Manag. Rev. 2013, 3, 11. [Google Scholar]
  60. Ballon, P.; Pierson, J.; Delaere, S. Test and experimentation platforms for broadband innovation: Examining european practice. In Proceedings of the 16th European Regional Conference by the International Telecommunications Society (ITS), Porto, Portugal, 4–6 September 2005; pp. 1–22. [Google Scholar]
  61. Almirall, E.; Lee, M.; Wareham, J.; Schrage, M. Mapping living labs in the landscape of innovation methodologies. Technol. Innov. Manag. Rev. 2012, 2, 12–18. [Google Scholar] [CrossRef]
  62. Ståhlbröst, A.; Holst, M. Reflecting on actions in living lab research. Technol. Innov. Manag. Rev. 2017, 7, 27–34. [Google Scholar]
  63. Bendavid, Y.; Cassivi, L. A “living laboratory” environment for exploring innovative RFID-enabled supply chain management models. Int. J. Prod. Dev. 2012, 17, 94–118. [Google Scholar] [CrossRef]
  64. Bourgault, M. Developing professional competencies using a Living Lab approach: An exploratory study in the field of management education. Int. J. Prod. Dev. 2012, 17, 76. [Google Scholar] [CrossRef]
  65. Bergvall-Kåreborn, B.; Ihlström Eriksson, C.; Ståhlbröst, A.; Svensson, J. A milieu for innovation—defining living labs. In The 2nd ISPIM Innovation Symposium; ISPIM: New York, NY, USA, 2009. [Google Scholar]
  66. Kviselius, N.Z.; Andersson, P.; Ozan, H.; Edenius, M. Living labs as tools for open innovation. Commun. Strateg. 2009, 2, 75–94. [Google Scholar]
  67. Westerlund, M.; Leminen, S.; Habib, C. Key Constructs and a Definition of Living Labs as Innovation Platforms. Technol. Innov. Manag. Rev. 2018, 8, 51–62. [Google Scholar] [CrossRef]
  68. Garcia Robles, A.; Hirvikoski, T.; Schuurman, D.; Stokes, L. Introducing ENoll and Its Living Lab Community; European Network of Living Labs: Brussels, Belgium, 2016. [Google Scholar]
  69. Grotenhuis, F.D.J. Living labs as service providers: From proliferation to coordination. Glob. Bus. Organ. Excell. 2017, 36, 52–57. [Google Scholar]
  70. Mulder, I.; Velthausz, D.; Kriens, M. The living labs harmonization cube: Communicating living lab’s essentials. Electron. J. Virtual Organ. Netw. 2008, 10, 1–14. [Google Scholar]
  71. Tang, T.; Wu, Z.; Karhu, K.; Hämäläinen, M.; Ji, Y. Internationally distributed living labs and digital ecosystems for fostering local innovations in everyday life. J. Emerg. Technol. Web Intell. 2012, 4, 106–115. [Google Scholar]
  72. Bergvall-Kåreborn, B.; Ståhlbröst, A. Living Lab: An open and citizen-centric approach for innovation. Int. J. Innov. Reg. Dev. 2009, 1, 356–370. [Google Scholar] [CrossRef] [Green Version]
  73. Hakkarainen, L. Caring for Technology—Evolving Living Lab Collaboration; Aalto University: Espoo, Finland, 2017. [Google Scholar]
  74. Ballon, P.; Van Hoed, M.; Schuurman, D. The effectiveness of involving users in digital innovation: Measuring the impact of living labs. Telemat. Inform. 2018, 35, 1201–1214. [Google Scholar] [CrossRef]
  75. Dahlander, L.; Gann, D.M. How open is innovation? Res. Policy 2010, 39, 699–709. [Google Scholar] [CrossRef]
  76. Mian, S.; Lamine, W.; Fayolle, A. Technology Business Incubation: An overview of the state of knowledge. Technovation 2016, 50–51, 1–12. [Google Scholar] [CrossRef]
  77. Hausberg, J.P.; Korreck, S. Business incubators and accelerators: A co-citation analysis-based, systematic literature review. J. Technol. Transf. 2020, 45, 151–176. [Google Scholar] [CrossRef] [Green Version]
  78. Randhawa, K.; Wilden, R.; Hohberger, J. A bibliometric review of open innovation: Setting a research agenda. J. Prod. Innov. Manag. 2016, 33, 750–772. [Google Scholar] [CrossRef]
  79. Adunlin, G.; Diaby, V.; Xiao, H. Application of multicriteria decision analysis in health care: A systematic review and bibliometric analysis. Health Expect. 2015, 18, 1894–1905. [Google Scholar] [CrossRef] [Green Version]
  80. Perrier, L.; Lightfoot, D.; Kealey, M.R.; Straus, S.E.; Tricco, A.C. Knowledge synthesis research: A bibliometric analysis. J. Clin. Epidemiol. 2016, 73, 50–57. [Google Scholar] [CrossRef]
  81. Belur, J.; Tompson, L.; Thornton, A.; Simon, M. Interrater Reliability in Systematic Review Methodology: Exploring Variation in Coder Decision-Making. Sociol. Methods Res. 2018, 1–29. [Google Scholar] [CrossRef] [Green Version]
  82. Autili, M.; Di Salle, A.; Gallo, F.; Pompilio, C.; Tivoli, M. A choreography-based and collaborative road mobility system for L’Aquila city. Futur. Internet 2019, 11, 132. [Google Scholar] [CrossRef] [Green Version]
  83. Bové, J.; Prou, D.; Perier, C.; Przedborski, S. Toxin-induced models of Parkinson’s disease. NeuroRX 2005, 2, 484–494. [Google Scholar] [PubMed] [Green Version]
  84. Hartmann, E. The fetish of global competition. Cap. Cl. 2014, 38, 184–196. [Google Scholar] [CrossRef]
  85. Di Stefano, G.; Peteraf, M.; Verona, G. The organizational drivetrain: A road to integration of dynamic capabilities research. Acad. Manag. Perspect. 2014, 28, 307–327. [Google Scholar] [CrossRef] [Green Version]
  86. Chen, K.; Zhang, Y.; Fu, X. International research collaboration: An emerging domain of innovation studies? Res. Policy 2019, 48, 149–168. [Google Scholar]
  87. Brandes, U.; Wagner, D. Analysis and Visualization of Social Networks. In Graph Drawing Software; Springer: Berlin/Heidelberg, Germany, 2004; pp. 321–340. [Google Scholar]
  88. Borgatti, S.P.; Everett, M.G.; Freeman, L.C. Ucinet for Windows: Software for Social Network Analysis; Analytic Technologies: Harvard, MA, USA, 2002. [Google Scholar]
  89. van Eck, N.J.; Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2010, 84, 523–538. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  90. Eisenhardt, K.M. Building theories from case study research. Acad. Manag. Rev. 1989, 14, 532–550. [Google Scholar]
  91. Leminen, S. Coordination and participation in living lab networks. Technol. Innov. Manag. Rev. 2013, 3, 5–14. [Google Scholar] [CrossRef]
  92. Veeckman, C.; Schuurman, D.; Leminen, S.; Westerlund, M. Linking living lab characteristics and their outcomes: Towards a conceptual framework. Technol. Innov. Manag. Rev. 2013, 3, 6–15. [Google Scholar]
  93. Almirall, E.; Wareham, J. Living labs and open innovation: Roles and applicability. Electron. J. Virtual Organ. Networks 2008, 10, 21–46. [Google Scholar]
  94. Chesbrough, H. Open Innovation: The New Imperative for Creating and Profiting from Technology; Harvard Business School Press: Boston, MA, USA, 2003. [Google Scholar]
  95. Schuurman, D.; De Marez, L.; Ballon, P. The impact of living lab methodology on open innovation contributions and outcomes. Technol. Innov. Manag. Rev. 2016, 6, 7–16. [Google Scholar]
  96. Mulder, I. Living labbing the Rotterdam way: Co-creation as an enabler for urban innovation. Technol. Innov. Manag. Rev. 2012, 2, 39–43. [Google Scholar] [CrossRef]
  97. Leminen, S.; Westerlund, M.; Nyström, A.G. On becoming creative consumers—user roles in living labs networks. Int. J. Technol. Mark. 2014, 9, 33–52. [Google Scholar] [CrossRef]
  98. De Silva, M.; Wright, M. Entrepreneurial co-creation: Societal impact through open innovation. R D Manag. 2019, 49, 318–342. [Google Scholar] [CrossRef]
  99. Engels, F.; Wentland, A.; Pfotenhauer, S.M. Testing future societies? Developing a framework for test beds and living labs as instruments of innovation governance. Res. Policy 2019, 48, 103826. [Google Scholar] [CrossRef]
  100. Furr, N.; O’Keeffe, K.; Dyer, J.H. Managing Multiparty Innovation. Harv. Bus. Rev. 2016, 94, 76–83. [Google Scholar]
  101. Aksnes, D.W.; Langfeldt, L.; Wouters, P. Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open 2019, 9, 1–17. [Google Scholar] [CrossRef] [Green Version]
  102. Cole, J.R.; Cole, S. Social Stratification in Science; The University of Chicago Press: Chicago, IL, USA, 1973. [Google Scholar]
  103. Juujärvi, S.; Pesso, K. Actor Roles in an Urban Living Lab: What Can We Learn from Suurpelto, Finland? Technol. Innov. Manag. Rev. 2013, 3, 22–27. [Google Scholar] [CrossRef]
  104. Wasserman, S.; Faust, K. Social Network Analysis: Methods and Applications; Cambridge University Press: Cambridge, UK, 1994; Volume 8. [Google Scholar]
  105. Raasch, C.; Lee, V.; Spaeth, S.; Herstatt, C. The rise and fall of interdisciplinary research: The case of open source innovation. Res. Policy 2013, 42, 1138–1151. [Google Scholar] [CrossRef]
  106. Research Group for Media, Innovation and Communication Technologies, Ghent University. 2020. Available online: https://www.ugent.be/mict/en/team/affiliated_members/dimitri-schuurman.htm (accessed on 1 December 2020).
  107. Daiberl, C.; Roth, A. Driving Service Productivity of Open Innovation Labs; Fritzsche, A., Jonas, J.M., Roth, A., Möslein, K.M., Eds.; Innovating in the Open Lab. De Gruyter Oldenbourg: Berlin, Germany, 2020; pp. 165–182. [Google Scholar]
Sustainability 13 00791 g001 550
Figure 1. Overview of data analysis.
Figure 1. Overview of data analysis.
Sustainability 13 00791 g001
Sustainability 13 00791 g002 550
Figure 2. Bibliographic coupling—relations identify the sharing of more than five common references.
Figure 2. Bibliographic coupling—relations identify the sharing of more than five common references.
Sustainability 13 00791 g002
Sustainability 13 00791 g003 550
Figure 3. Articles per year.
Figure 3. Articles per year.
Sustainability 13 00791 g003
Sustainability 13 00791 g004 550
Figure 4. Collaboration among scholars in the Living Labs research community (names of the authors removed to improve readability).
Figure 4. Collaboration among scholars in the Living Labs research community (names of the authors removed to improve readability).
Sustainability 13 00791 g004
Sustainability 13 00791 g005 550
Figure 5. Influential Living Labs authors not included in the main component of the network.
Figure 5. Influential Living Labs authors not included in the main component of the network.
Sustainability 13 00791 g005
Sustainability 13 00791 g006 550
Figure 6. Co-authorship network: main component.
Figure 6. Co-authorship network: main component.
Sustainability 13 00791 g006
Sustainability 13 00791 g007 550
Figure 7. The evolution of the co-authorship network, 2010–2020.
Figure 7. The evolution of the co-authorship network, 2010–2020.
Sustainability 13 00791 g007
Sustainability 13 00791 g008 550
Figure 8. Keyword co-occurrence network.
Figure 8. Keyword co-occurrence network.
Sustainability 13 00791 g008
Sustainability 13 00791 g009 550
Figure 9. Keyword co-occurrence—citing documents.
Figure 9. Keyword co-occurrence—citing documents.
Sustainability 13 00791 g009
Table
Table 1. Revealing the diverse Living Labs landscape.
Table 1. Revealing the diverse Living Labs landscape.
Living Labs ResearchExamples Revealing Diverse Nature and Understanding of Living Labs
Definitions
  • 70 different Living Labs definitions [36]
  • 13 different definitions [29]
Interpretations
  • (1) An innovation system consisting of organised and structured multi-disciplinary networks, fostering innovation and collaboration; (2) the in vivo monitoring of a “living” social setting, generally involving experimentation with a technology; (3) an approach for involving users in the product development process; (4) the organisations facilitating a network, maintaining and developing its technological infrastructure, and offering relevant services; and (5) the European Living Labs movement [37]
  • European Living Labs are characterised by five basic elements: (1) active user involvement, (2) a real-life setting, (3) multiple stakeholder participation, (4) a multi-method approach, and (5) co-creation [38]
  • American Living Labs are an extension of laboratory experiments [39]
Types
  • Utiliser-driven, enabler-driven, provider-driven, and user-driven (or user–community-driven) Living Labs [40]
Stakeholders
  • Suppliers, customers, users, competitors, universities, and other institutions and organisations [4]
  • Researchers, utilisers, enablers, users, providers [41]
Context
  • Business-to-consumer as well as business-to-business [42]
  • Virtual and physical Living Labs [43]
  • Living Labs in developed and developing countries [40,44]
  • Information and communication technology (ICT) [45], assisted living [46], health care [47,48,49], media [50], agriculture [51], mobility [52,53], urban and rural areas, smart cities and digital cities, buildings [4,54,55]
Perspective and Level of Analysis
  • Single-level analysis and multi-level analysis [56]
  • Living Labs as innovation intermediaries [57,58,59], Living Labs in the domain of user-driven innovation methodologies [60,61], Living Labs as a method in the design research methodologies [29,62], specific cases of Living Lab development [63,64]
Disciplines and Concepts
  • Multidisciplinary concept [65]; innovation management, user-centred design, entrepreneurship, cognitive science, organisation theory, management models, context awareness, human–computer interaction, information science, social computing, among many others [54,66]
Table
Table 2. Most cited sources.
Table 2. Most cited sources.
Cited ReferenceCitations
Leminen, S., Westerlund, M., Nyström, A.G., 2012. Living labs as open-innovation networks. Technol. Innov. Manag. Rev. 2, 6–11. [40]30
Almirall, E., Lee, M., Wareham, J., Schrage, M., 2012. Mapping living labs in the landscape of innovation methodologies. Technol. Innov. Manag. Rev. 2, 12–18. [61]25
Dell’Era, C., Landoni, P., 2014. Living lab: A methodology between user-centred design and participatory design. Creat. Innov. Manag. 23, 137–154. https://doi.org/10.1111/caim.12061 [29]19
Almirall, E., Wareham, J., 2011. Living Labs: arbiters of mid- and ground-level innovation. Technol. Anal. Strateg. Manag. 23, 87–102. [57]18
Følstad, A., 2008. Towards a living lab for development of online community services. Electron. J. Virtual Organ. Networks 10, 47–58. [45]18
Bergvall-Kåreborn, B. Ståhlbröst, A., 2009. Living Lab: an open and citizen-centric approach for innovation. Int. J. Innov. Reg. Dev. 1, 356–370. [65]17
Leminen, S., 2013. Coordination and participation in living lab networks. Technol. Innov. Manag. Rev. 3, 5–14. [91]16
Nyström, A.G., Leminen, S., Westerlund, M., Kortelainen, M., 2014. Actor roles and role patterns influencing innovation in living labs. Ind. Mark. Manag. 43, 483–495. [21]16
Veeckman, C., Schuurman, D., Leminen, S., Westerlund, M., 2013. Linking living lab characteristics and their outcomes: Towards a conceptual framework. Technol. Innov. Manag. Rev. 3, 6–15. [92]16
Almirall, E., Wareham, J., 2008. Living labs and open innovation: Roles and applicability. Electron. J. Virtual Organ. Networks 10, 21–46. [93]14
Westerlund, M., Leminen, S., 2011. Managing the challenges of becoming an open innovation company: Experiences from living labs. Technol. Innov. Manag. Rev. 19–25. [43]14
Chesbrough, H., 2003. Open innovation: The new imperative for creating and profiting from technology. Harvard Business School Press, Boston, Massachusetts. [94]13
Schuurman, D., De Marez, L., Ballon, P., 2016. The impact of living lab methodology on open innovation contributions and outcomes. Technol. Innov. Manag. Rev. 6, 7–16. [95]12
Schuurman, D., 2015. Bridging the gap between open and user innovation? Exploring the value of living labs as a means to structure user contribution and manage distributed innovation. PhD Thesis, Ghent University and Vrije Universiteit Brussel (VUB). [16]12
Dutilleul, B., Birrer, F.A.J., Mensink, W., 2010. Unpacking European living labs: Analysing innovation’s social dimensions. Cent. Eur. J. Public Policy 4, 60–85. [37]11
Leminen, S., Westerlund, M., 2017. Categorization of innovation tools in living labs. Technol. Innov. Manag. Rev. 7, 15–25. [18]11
Mulder, I., 2012. Living labbing the Rotterdam way: Co-creation as an enabler for urban innovation. Technol. Innov. Manag. Rev. 2, 39–43. [96]11
Leminen, S., Westerlund, M., Nyström, A.G., 2014. On becoming creative consumers—user roles in living labs networks. Int. J. Technol. Mark. 9, 33–52. [97]11
Eisenhardt, K.M., 1989. Building theories from case study research. Acad. Manag. Rev. 14, 532–550. [90]10
Leminen, S., Westerlund, M., 2012. Towards innovation in living labs networks. Int. J. Prod. Dev. 17, 43. [24]10
Ståhlbröst, A., 2013. A living lab as a service: Creating value for micro-enterprises through collaboration and innovation. Technol. Innov. Manag. Rev. 3. [59]10
Table
Table 3. Top 10 articles by total number of citations.
Table 3. Top 10 articles by total number of citations.
#AuthorsPublication YearArticle TitleJournalN. of Citations
1Leminen, S; Westerlund, M; Nyström, AG [40]2012Living Labs as Open-Innovation NetworksTechnology Innovation Management Review90
Key Contributions: This study identifies Living Labs as four different types of networks characterised by open innovation: utiliser-driven, enabler-driven, provider-driven, and user-driven.
2Almirall, E; Lee, M; Wareham, J [61]2012Mapping Living Labs in the Landscape of Innovation MethodologiesTechnology Innovation Management Review86
Key Contributions: This study offers insights into the most common European Living Labs approaches and positions them in the landscape of user-contributed innovation methodologies.
3Dell’Era, C; Landoni, P [29]2014Living Lab: A Methodology between User-Centred Design and Participatory DesignCreativity and Innovation Management73
Key Contributions: This study proposes a new Living Lab definition, positioning the methodology among other design methodologies and highlighting its peculiarities. Furthermore, four different types of Living Labs, based on the openness of the user involvement and the adopted platform technology, are identified.
4Almirall, E; Wareham, J [57]2011Living Labs: arbiters of mid- and ground-level innovationTechnology Analysis & Strategic Management69
Key Contributions: This study identifies four common Living Lab functions. First, Living Labs function at the low- and mid-level innovation strata; second, Living Labs are technologically agnostic; third, Living Labs use context-based experience to surface new, socially constructed meanings for products and services; and finally, Living Labs are equally focused on exploration and exploitation.
5Veeckman, C; Schuurman, D; Leminen, S; Westerlund, M [92]2013Linking Living Lab Characteristics and Their Outcomes: Towards a Conceptual FrameworkTechnology Innovation Management Review43
Key Contributions: This study examines how the different building blocks of Living Lab environments contribute to the outputs of innovation projects launched within the lab. The article provides practical guidelines on how Living Labs should be managed on the levels of community interaction, stakeholder engagement, and methodological setup.
6Westerlund, M; Leminen, S [43]2011Managing the Challenges of Becoming an Open Innovation Company: Experiences from Living LabsTechnology Innovation Management Review41
Key Contributions: This study identifies four distinct steps in becoming an open innovation company, which is based on research into firms’ experiences with Living Lab experiments in the information and communication technology (ICT) sector. The article describes these phases and illustrates the divergent roles that users play in each one. The study also provides insight into the differences between the management challenges of conventional development projects versus the open innovation model.
7Juujarvi, S; Pesso, K [103]2013Actor Roles in an Urban Living Lab: What Can We Learn from Suurpelto, Finland?Technology Innovation Management Review38
Key Contributions: This study examines the characteristics and success factors of urban Living Labs based on a case study of Suurpelto, Finland.
8Leminen, S [91]2013Coordination and Participation in Living Lab NetworksTechnology Innovation Management Review36
Key Contributions: This study offers a framework for analysing coordination (i.e., top-down versus bottom-up) and participation (i.e., inhalation-dominated versus exhalation-dominated) approaches in Living Lab networks. The framework reveals opportunities for practitioners of innovation with respect to coordination and participation in Living Lab networks.
9Mulder, I [96]2012Living Labbing the Rotterdam Way: Co-Creation as an Enabler for Urban InnovationTechnology Innovation Management Review33
Key Contributions: This study elaborates on “living methodologies”, methods and tools necessary in “living labbing”.
10Schuurman, D; De Marez, L; Ballon, P [95]2016The Impact of Living Lab Methodology on Open Innovation Contributions and OutcomesTechnology Innovation Management Review33
Key Contributions: This study suggests that a real-life intervention and a multi-method approach—both of which are methodological characteristics of Living Lab projects—increase the chance of generating actionable user contributions for the innovation under development. Moreover, the results also suggest that a Living Lab project yields maximal value when evolving from concept towards prototype. This article also demonstrates that Living Lab projects are a perfect “playground” to test and validate assumptions from the open innovation literature.
Table
Table 4. Most prominent authors.
Table 4. Most prominent authors.
Most Prominent Authors
by N. of Articles		Most Prominent Authors
by N. of Collaborative
Connections 		Most Prominent
Authors by N. of
Citations Received
Leminen, S. (16)Schuurman, D. (33)Leminen, S. (323)
Westerlund, M. (15)Leminen, S. (28)Westerlund, M. (287)
Schuurman, D. (14)Westerlund, M. (28)Schuurman, D. (172)
Ståhlbröst, A. (6)Georges, A. (17)Almirall, E. (155)
De Marez, L. (5) De Marez, L. (13) Wareham, J. (155)
Georges, A. (5)Ballon, P. (12)Nyström, A. (123)
Ballon, P. (4)Dupont, L. (11)De Marez, L. (102)
7 authors (3)Rits, O. (11)Lee, M. (86)
Ståhlbröst, A. (10)2 authors (74)
Table
Table 5. Journals, research areas, and keywords of citing documents (without self-citations).
Table 5. Journals, research areas, and keywords of citing documents (without self-citations).
Top 3 Journals by number of citing publications (in %)1Sustainability (6%)
2Technology Innovation Management Review (4%)
3Journal of Cleaner Production (3%)
Top 9 Research Areas (at least 5%)1Business Economics (30%)
2Environmental Sciences Ecology (17%)
3Science Technology Other Topics (16%)
4Engineering (16%)
5Computer Science 16%)
6Public Administration (10%)
7Social Sciences Other Topics (6%)
8Education Educational Research (5%)
9Urban (5%)
Top 17 Keywords and Phrases (occurring at least eight times)1Living Lab (97)
2Innovation (47)
3Open Innovation (46)
4Co-creation (42)
5Smart City (39)
6Social Innovation (18)
7Sustainability (16)
8Collaboration (12)
9Urban Living Lab (10)
10Entrepreneurship (9)
11ICT (9)
12Sustainable development (9)
13Case study (8)
14Collaborative innovation (8)
15Innovation ecosystem (8)
16Participatory design (8)
17Sustainability transitions (8)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Greve, K.; Vita, R.D.; Leminen, S.; Westerlund, M. Living Labs: From Niche to Mainstream Innovation Management. Sustainability 2021, 13, 791. https://doi.org/10.3390/su13020791

AMA Style

Greve K, Vita RD, Leminen S, Westerlund M. Living Labs: From Niche to Mainstream Innovation Management. Sustainability. 2021; 13(2):791. https://doi.org/10.3390/su13020791

Chicago/Turabian Style

Greve, Katharina, Riccardo De Vita, Seppo Leminen, and Mika Westerlund. 2021. "Living Labs: From Niche to Mainstream Innovation Management" Sustainability 13, no. 2: 791. https://doi.org/10.3390/su13020791

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop