Next Article in Journal
The Electrification of Ships Using the Northern Sea Route: An Approach
Previous Article in Journal
A New Optimization Technique for the Location and Routing Management in Agricultural Logistics

Responsible Research and Innovation in Enterprises: Benefits, Barriers and the Problem of Assessment

Faculty of Business Management, Vilnius Gediminas Technical University, LT-10223 Vilnius, Lithuania
J. Open Innov. Technol. Mark. Complex. 2020, 6(1), 12;
Received: 27 December 2019 / Revised: 4 February 2020 / Accepted: 5 February 2020 / Published: 6 February 2020


In the paper, the author takes stock of the conceptual reflection and empirical studies described in the current scientific literature on responsible innovation in the context of the emergence of the Responsible Research and Innovation (RRI) concept. RRI has been promoted in the European Union as a part of the Europe 2020 strategy with the objective of making research and innovation more sustainable and inclusive. As more than half of the EU’s firms declare they are conducting innovation activities, RRI problems are more relevant than ever. There remain many open questions, unresolved dilemmas and empirical white spots that call for more research in this field. This paper’s focus is the problem of RRI acceptance as a global framework for responsible innovation and the scarcity of suitable instruments that may help industries understand and adopt this concept. The main contributions of this paper include critical analysis of the RRI concept and its implications for industry, proposing a concept of an RRI index for innovating enterprises.
Keywords: responsible research and innovation; responsibility; innovation; assessment; technology assessment; foresight responsible research and innovation; responsibility; innovation; assessment; technology assessment; foresight

1. Introduction

Responsible Research and Innovation (RRI) is an emerging framework for governing the R&D activity and innovation in both the public and private sectors. It is considered to be “the ongoing process of aligning research and innovation to societal values, needs and expectations” [1]. It has been promoted in the European Union as a part of the Europe 2020 strategy with the objective of making research and innovation more sustainable and inclusive. Von Schomberg’s seminal definition states that:
Responsible Research and Innovation is a transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products (in order to allow a proper embedding of scientific and technological advances in our society) [2].
In recent years, a significant number of research projects and industry-oriented initiatives have attempted to address RRI from different angles, in different geographical areas and with different methods.
There remain many open questions, unresolved dilemmas and empirical white spots that call for more research in this field [3]. As more than half of the EU’s firms declare they are conducting innovation activities [4], RRI problems have become more relevant than ever. This paper’s main focus is the problem of RRI acceptance as a global framework for responsible innovation and the scarcity of suitable instruments that may help industries understand and adopt this concept.
In the first part of this work, the author analyses the current scientific discourse on RRI. Bibliometric analysis is carried out together with a critical analysis of most common RRI-related subjects, lines of thought, and methodologies. The critical analysis approach is also used in the second part in which the issue of RRI costs and benefits are synthesized and discussed. Next, the problematic of RRI assessment is presented and deconstructed. Various approaches to measuring RRI in public and private entities are discussed. The paper concludes with a discussion of the results, identification of the limitations of the study, and suggestions for the possible future research directions.

2. Responsible Research and Innovation—Overview of Current Discourse

Scientific production on the topic of RRI has been steadily increasing since 2009—the year of publication of Robinson’s important work that used the term “responsible research and innovation” for the first time [5]. The same may be said about the diffusion of the concept in non-academic sources. Media intelligence tools that analyze public news items online suggest a steady societal uptake of RRI from 2010 to 2017; however, the absolute numbers of news items with the term “responsible research and innovation” are not impressive (356 news items globally in 2017) [6].
Query in the Scopus database on the documents that contain phrases “responsible research and innovation” or “responsible innovation” in their title, abstract or keywords, returned 841 documents (mostly articles, conference papers and book chapters; including 127 papers from the Journal of Responsible Innovation) published between 2009 and 2019. A clear growing tendency in the number of published documents may be observed; however, the tendency seems to slow down, as shown in Figure 1.
The slowdown is a new observation compared to previous analyses of this kind by Cuppen, van de Grift and Pesch [7] Timmermans [8] and Nazarko [9]. It remains to be seen if the slowdown constitutes a definite break in the sharp upward trend registered so far or if it is only temporary. Some of Timmermans’ conclusion in his mapping study [8] provide an argument for the former hypothesis. He observes that most of the RRI related activities took place in Western Europe and were funded by the European Union or public institutions in Western European member states. More than 100 million EUR from the Horizon 2020 Framework Program (2014–2020) was spent on more than 40 projects that dealt explicitly with RRI [9]. It should be noted, however, that customizing and contextualizing of RRI has been taking place in many (if not all) H2020 program lines, without necessarily mentioning the term RRI (terms such as broader impacts criterion, value-sensitive design, technology acceptance, social-technological alignment, ethical impact assessment, human readiness level, social implications of technology, stage-gate system, upstream engagement, etc., were used instead). The living lab and comparative evaluation approach developed in the framework of the NewHoRRIzon project sought to recognize and invent RRI relevance in many different sections of H2020 to provide a systematic approach for addressing complex social challenges related to RRI [10].
Since the RRI-related activity may not be motivated by a market demand of Research and Innovation (R&I) milieu, but rather has been pushed by policy-makers [8], the future extent of RRI-related scientific production is heavily dependent on the available public funding for RRI activities. As the EU funding cycle within its Framework Program for Research and Technological Development (Horizon 2020) is coming to an end, the number of RRI-related documents generated as the results of the H2020 projects is going down. For the time being, the EU planning documents for the next Framework Program (Horizon Europe, 2021–2027) do not mention RRI explicitly and do not foresee a separate funding stream similar to the current “Science with and for Society” (SwafS) program. There seems to be more emphasis on excellence rather than on responsibility [11]. Strategic implementation of RRI in future R&I policy at the EU level remains uncertain [1]. Such developments may herald stagnation in the research work on RRI unless the industry significantly increases its interest and involvement, which has not happened so far (only 12.5% of consortia members in RRI-related projects were for-profit organizations [9]). Owen and Pansera [12] also admit that RRI faces an uncertain future as an EU policy agenda. They take into account the possible shift in the framing of RRI toward relating it to the ’3 Os’: Open Innovation [13,14], Open Science and Openness to the world [15]. Indeed, a more explicit connection to the Open Innovation paradigm that envisages the creative interaction of business [16], industry, government and civil society in a quadruple helix [17], and offers some practical tools [18], may be a remedy for a low uptake of the RRI concept in industries.
Moving from the quantitative to the qualitative dimension, several categories of topics may be distinguished by analyzing the keywords of the RRI-related documents indexed by Scopus. Visualization with VOSviewer has resulted in the identification of eight clusters of terms that co-occur most frequently (Figure 2). The clusters (labelled with different colors in Figure 2), however, are neither internally coherent nor qualitatively distinctive. It is, therefore, necessary to decompose the set of the most frequent keywords and make an attempt to form distinctive thematic categories based on substance and not on the co-occurrence.
Thematic analysis of keywords in RRI related literature resulted in the identification of 9 categories of topics. The categories are named in Table 1. Relevant terms are assigned to categories (including terms that have been omitted in Figure 1 for the sake of clarity).
The obtained categories reveal the dominance of theoretical and conceptual themes in the RRI literature. A bulk of papers and book chapters concern the epistemological foundations of RRI together with many dilemmas, ambiguities and contradictions. Much attention is also devoted to localizing RRI among more established concepts, such as Corporate Social Responsibility or sustainability. RRI principles [19] and RRI policy agendas are discussed against that epistemological and conceptual background. Moreover, RRI evokes new voices with regards to the (re)conceptualization of entrepreneurship and innovation. A considerable section of the works deals with RRI agendas for particular technologies and innovation spheres. Public engagement and science education topics are also prominent. Finally, the European Union (more specifically, Horizon 2020 Framework Program) is mentioned frequently, which is an indication of the policy-push evident in the current discourse. Generally, the works are conceptual rather than practical and process-oriented rather than outcome-oriented, which corresponds with the findings of Cuppen, van de Grift and Pesch [7] and Timmermans [8].

3. Promised Benefits and Criticism of RRI

RRI remains a concept and a policy framework connected to the selected funding streams in the European Union. As such, it has not been translated into new regulatory measures at the level of particular countries. A unified, comprehensive international regulatory framework related to RRI does not seem viable in the foreseeable future. Thus, the uptake of the RRI is dependent on the effective communication of the advantages that may accompany its broad adoption. Such is the approach adopted in the US in the framework of the Socio-Technical Integration Research (STIR) project that aimed at encouraging and equipping innovators with tools to reflect more deeply and creatively on societal aspects of their work [20].
In Table 2 and Table 3, a synthesis of the current discussion on both benefits and problems related to the implementation of RRI in enterprises are presented.

4. Towards an RRI index for Innovating Organizations

4.1. One RRI Index Does Not Fit All

When attempting to compile metrics and indicators or to design indices that allow the comparison of different units from the perspective of RRI, the following facts become apparent.
Firstly, separation of “research” and “innovation” within the RRI concept seems appropriate since these two terms may be ascribed to different phenomena/processes with different actors involved [22,33]. “Research”, as it is normally described in RRI literature, belongs to universities, laboratories, research institutes and research funding agencies. Research, in this sense, is associated with the production of new knowledge, often on the basis of public funding. “Innovation”, in turn, is a process of turning existing or new knowledge into a marketable good, improved process, or organizational arrangement. It is mostly associated with market context and the for-profit activity. In consequence, it is more suitable to speak about Responsible Research in the context of public institutions and Responsible Innovation (RI) in the corporate/industrial context. This is not to say that research entities do not innovate or that the private sector does not conduct research; however, the above distinction is helpful in making sense of differences between R&I in the non-profit/public and for-profit contexts. Such delineation may also be observed in the recent work by von Schomberg [34] in which he drafts two separate models of step-up processes for ’good’ open research and ’good’ open innovation (three steps: credible—responsive—responsible).
Secondly, as a consequence of the above observation, different ways of assessing RRI are proposed for public entities (research conducting and research funding bodies) and for innovating enterprises. In Table 4 and Table 5, an overview of RRI assessment criteria proposed in the literature is presented.
Thirdly, it requires much scientific effort both theoretically and empirically to evaluate the appropriateness of the RRI assessment criteria presented below. As in the case of Corporate Social Responsibility (CSR), despite higher maturity of the concept, there is still much doubt as to whether the available CSR metrics and indicators are suitable, optimal and transparent [35], and what is the relationship between CSR and firm performance [36].
Metrics and indicators presented in Table 4 are of a very diverse nature. Some are quantitative and objective, others are qualitative and more arbitrary. Some are based on data already available, others require new surveys. It is quite obvious that performance in some RRI dimensions (like gender, open access) is easier to capture with the use of indicators than in others (ethics, governance). Many of the indicators offer information about formal settings and structures established at research performing or research funding institutions. However, it requires a much deeper look to see how those formal settings influence researchers, innovators and wider society [6]. Categorizing the assessment criteria according to the defined RRI policy agendas also has its implications. Much emphasis is put on the process but less on the outcome of R&I. The presented metrics seem to miss the Grand Challenges and sustainability dimension of the RRI discourse [39]. Moreover, some indicators relate to the democratization of science and technology decision-making, which could be a serious barrier in the adoption of RRI in non-democratic political systems.
Assessment criteria for enterprises presented in Table 5 are clearly different from the ones presented in Table 4. This confirms the argument for a separate approach to RRI in for-profit, market-driven entities and in public research performing and research funding bodies. The indicators suitable for enterprises are predominantly based on subjective judgement and self-assessment. Moreover, it should be noted that not all criteria are relevant to the activity of each innovating organization. Additionally, certain criteria may have more importance in a particular sector/technological domain than in others. All these conclusions have implications for the design of an assessment framework in the form of a RRI index presented below.

4.2. Proposal of RRI Index for Innovating Organizations

There is a clear tension between the need to measure and quantify the responsibility aspect of the innovation activity (“you can only manage what you can measure”) and the broad, heuristic and often ambiguous nature of the RRI concept. “Monitoring RRI by utilizing closed response surveys, questionnaires, and other highly quantified methodologies risks serving to reify narrow interpretations of its application and constitution” writes van de Klippe [43]. Nevertheless, if RRI were ever to become a widely accepted paradigm of corporate responsibility and governance, the industry would expect academia to provide concrete solutions and to propose concrete metrics against which RRI could be measured. The conceptual work presented below has been inspired by existing approaches to sustainability evaluation [44].
The proposed RRI index for innovating organizations has the following features:
  • It is based on respondent’s subjective/arbitrary judgement (semi-quantitative nature);
  • Its ingredients may be customized according the needs of a particular sector or a group of enterprises;
  • Components of the index are weighted. Weights are also arbitrarily determined by the users of the index.
The first stage of the development of the RRI index for enterprises consists of conducting a survey among innovating companies. Companies judge their own performance in the RRI relevant fields with the use of a simple 5-point scale. They make their judgement on the basis of what they think is the best practice/performance in that area. As a result, each company obtains its score in RRI-relevant fields compared to the highest score in the group. Examples of RRI fields that may be assessed are presented in Figure 3.
The second stage consists of determining the weights of the index components (E1, E2, G1, G2, Gov1, Gov2, O1, O2, P1, S1, S2). Weights could be determined arbitrarily by a researcher or policy maker or they may be determined collectively by the enterprises participating in the survey. The sum of weights should be equal to 1.
n = 1 N w n = 1
wn, - weight of the nth component
N - total number of components
The third stage is carried out separately for each enterprise. The distance between the enterprise’s score (E1) and the leader’s score (E1max) should be calculated for each component; the difference will have a negative value. In components where the studied company is actually the leader, the distance between its score (E2) and the score of the second-best enterprise (E2max) is calculated; the difference will have a positive value (Figure 4).
The fourth stage consists of summing up the distances calculated in the third stage, taking into account the weights determined in the second stage. The enterprises may be ranked according to the obtained result. The higher the number, the better the RRI performance. Normalization procedures may be applied if needed.
R R I   I n d e x = n = 1 N w n ( C n C n   m a x )
wn, - weight of the nth component
N - total number of components
Cn - enterprise’s score for n-th component
Cn max - leader’s score for n-th component (alternatively, second best score if the studied company is the leader)
The advantage of the proposed RRI index is its flexibility and simplicity, which makes it understandable, usable and easy to interpret. Organizations do not need to sacrifice much time and many resources to fill in the score sheet. The time needed to fill in the sheet decreases even further if the exercise is performed on a regular basis (to monitor changes). It should be noted that the optimal use of the index consists in benchmarking, i.e., in calculating the index with the same components and the same weights for a number of similar organizations. Such exercises may help identify leaders and followers in RRI, thus spreading best RRI practices. If used by only one organization, the RRI index may help with tracking changes over time.

5. Discussion and Conclusions

The complexity and dynamism of innovation ecosystems is growing on regional, national and global levels. The impact of new technologies, products and services on society, economic activity, natural environment, the evolution of culture and values is increasingly difficult to predict as the world is increasingly characterized by volatility, uncertainty, complexity and ambiguity (VUCA). In this context, classical perspectives on responsibility that focus on its individual dimension are insufficient to reflect the complexity and multifaceted reality of modern socio-technical systems [24]. That calls for a new, relational and networked understanding of responsibility which transits from “responsibility in and to society” towards “responsibility for and with society” [29].
Gianni’s position that RRI is essentially a “performative framework that can never be accomplished” [46] carries both risks and opportunities. On one hand, its unfinished nature allows for a big deal of flexibility and adaptability to different contexts. On the other, a lack of one common understanding of the concept poses serious difficulties in its operationalization and buy-in from the industry [47].
The distinctive feature of RRI approach consists in the aspiration to take stakeholders’ needs, concerns and values before a product, service or technology enters the market; to go beyond “not doing wrong” (for example by introducing environmental management systems [48]) towards “doing good”. Thus, anticipation, in addition to regulation, becomes a new critical element of innovation governance [49]. Even more than before, innovators are encouraged to practice reflexivity in the form of foresight [50], Future-Oriented Technology Analysis [51,52] or innovative forms of Technology Assessment [53,54]. This is easier said than done, as anticipation of future impacts of innovation (often unintended and ambiguous) falls into the Collingridge dilemma: “When change is easy, the need for it cannot be foreseen; when the need for change is apparent, change has become expensive, difficult, and time-consuming” [55].
The main limitation of the presented study comes from the lack of a representative empirical component. It is therefore the author’s ambition to continue studies in that direction. Two avenues seem especially enticing: investigating RRI adoption and testing RRI assessment tools in selected sectors (for example food industry [56] or machinery building industry [57]) and exploring the territorial aspect of RRI in less-studied European regions (namely, Eastern and Central Europe [58]).
In the author’s opinion, further research may be directed towards two goals. First, attempts should be made to overcome the Eurocentrism of the RRI and to increase its global reception. After all, RRI should foster innovation that tackles major societal challenges that are global by nature: i) tightening supplies of energy, water and food; ii) pandemics and public health issues; iii) ageing societies [59]; iv) security; and v) climate change [60]. Analyzing innovation systems of countries such as South Africa and China with a view of verifying the adaptability of RRI concept in those contexts is a step in the right direction [61]. It should not be impossible to establish a certain set of social desirability criteria necessary for RRI to go global [62].
Secondly, context-sensitive, business-relevant and user-friendly tools for RRI evaluation should be further developed and continuously improved to increase industry acceptance of the framework. Development of valid measures of the responsibility of innovation processes and outcomes are perhaps the most important challenge for researchers and innovation stakeholders alike. Innovation responsibility ratings of enterprises could be strong tools capable of influencing ethically-sensitive investors and mobilizing consumers, potential employees and social movements. As awkward as it sounds, there is a big deal of responsibility attached to responsibility assessment.
A universally-accepted set of social desirability criteria would be a very good starting point for global responsibility benchmarking surveys [63] and analyses. The application of more sophisticated analytical instruments would be desirable. For example, Data Envelopment Analysis (DEA) could be used to evaluate how efficiently innovating organizations transform their resources and time (input) into socially desirable “outputs” [64,65]. Such “RRI productivity” approach could also be used at the strategic planning stage to choose between competing technology roadmaps [66,67,68] with different anticipated inputs and outputs.


This project has received funding from European Social Fund (project No 09.3.3-LMT-K-712-02-0155) under grant agreement with the Research Council of Lithuania (LMTLT).


The author wishes to express his appreciation to professor Borisas Melnikas (Vilnius Gediminas Technical University) for the support and consultation during the post-doctoral research project entitled “Technology Management in the context of Responsible Research and Innovation”.

Conflicts of Interest

The author declares no conflict of interest


  1. Pathways Declaration. The Future of Responsible Research and Innovation (RRI) in “Horizon Europe”. Available online: (accessed on 22 December 2019).
  2. Von Schomberg, R. Prospects for Technology Assessment in A Framework of Responsible Research and Innovation. In Technikfolgen abschätzen lehren: Bildungspotenziale transdisziplinärer Methoden; Dusseldorp, M., Beecroft, R., Eds.; Springer VS: Wiesbaden, Germany, 2012; pp. 39–61. [Google Scholar]
  3. Ferri, F.; Dwyer, N.; Raicevich, S.; Grifoni, P.; Altiok, H.; Andersen, H.T.; Laouris, Y.; Silvestri, C. Preface. In Governance and Sustainability of Responsible Research and Innovation Processes. Cases and Experiences; Ferri, F., Dwyer, N., Raicevich, S., Grifoni, P., Altiok, H., Andersen, H.T., Laouris, Y., Silvestri, C., Eds.; Springer: Cham, Switzerland, 2018; pp. v–x. [Google Scholar]
  4. Eurostat. Community Innovation Survey 2016; Eurostat: Brussels, Belgium, 2019. [Google Scholar]
  5. Robinson, D. Co-evolutionary scenarios: An application to prospecting futures of the responsible development of nanotechnology. Technol. Forecast. Soc. Chang. 2009, 76, 1222–1239. [Google Scholar] [CrossRef]
  6. Peter, V.; Maier, F.; Spaini, C.; Woolley, R.; Meijer, I.; Costa, R.; Bloch, C.; Mejlgaard, N. Monitoring the Evolution and Benefits of Responsible Research and Innovation. The evolution of Responsible Research and Innovation—The Indicators Report; European Commission: Brussels, Belgium, 2018. [Google Scholar]
  7. Cuppen, E.; van de Grift, E.; Pesch, U. Reviewing responsible research and innovation: lessons for a sustainable innovation research agenda? In Handbook of Sustainable Innovation; Boons, F., McMeekin, A., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2019; pp. 142–164. [Google Scholar]
  8. Timmermans, J. Mapping the RRI Landscape: An Overview of Organisations, Projects, Persons, Areas and Topics. In Responsible Innovation 3. A European Agenda? Asveld, L., van Dam-Mieras, R., Swierstra, T., Lavrijssen, S., Linse, K., van den Hoven, J., Eds.; Springer: Cham, Switzerland, 2017; pp. 21–47. [Google Scholar]
  9. Nazarko, L. Responsible research and innovation—A conceptual contribution to theory and practice of technology management. Bus. Theory Pract. 2019, 20, 342–351. [Google Scholar] [CrossRef]
  10. Nielsen, M.; Mejlgaard, N.; Alnor, E.; Griessler, E.; Meijer, I. Ensuring Societal Readiness: A thinking tool. Available online: (accessed on 6 February 2020).
  11. Horizon Europe. The next EU Research & Innovation Programme (2021–2027). Available online: (accessed on 22 December 2019).
  12. Owen, R.; Pansera, M. Responsible Innovation and Responsible Research and Innovation. In Handbook of Research on Public Policy; Simon, D., Kuhlmann, S., Stamm, J., Canzler, W., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2019; pp. 26–48. [Google Scholar]
  13. Yun, J.J.; Park, K.; LM, C.; Shin, C.; Lee, S. Dynamics of Social Enterprises—Shift from Social Innovation to Open Innvation. In Proceedings of the SOItmC & CSCOM 2016 Conference, San Jose State University, San Jose, CA, USA, 31 May–3 June 2016. [Google Scholar]
  14. Yun, J.; Egbetoku, A.; Zhao, X. How Does a Social Open Innovation Succeed? Learning from Burro Battery and Grassroots Innovation Festival of India. Sci. Technol. Soc. 2019, 24, 122–143. [Google Scholar] [CrossRef]
  15. Shin, C.; Park, J. How Social Entrepreneurs’ Value Orientation Affects the Performance of Social Enterprises in Korea: The Mediating Effect of Social Entrepreneurship. Sustainability 2019, 11, 5341. [Google Scholar] [CrossRef]
  16. Walecka-Jankowska, K.; Zimmer, J. Open innovation in the context of organisational strategy. Eng. Manag. Prod. Serv. 2019, 11, 86–95. [Google Scholar] [CrossRef]
  17. Yun, J.; Liu, Z. Micro-and-Macro-Dynamics of Open Innovation with a Quadruple-Helix Model. Sustainability 2019, 11, 3301. [Google Scholar] [CrossRef]
  18. Bergvall-Kareborn, B.; Stahlbrost, A. Living Lab: An open and citizen-centric approach for innovation. Int. J. Innov. Reg. Dev. 2009, 1, 356–370. [Google Scholar] [CrossRef]
  19. Owen, R.; Stilgoe, J.; Macnaghten, P.; Gorman, M.; Fisher, E.; Guston, D. A framework for responsible innovation. In Responsible Innovation: Managing the Responsible Rmergence of Science and Innovation in Society; Owen, R., Bessant, J., Heintz, M., Eds.; John Wiley & Sons Inc.: Chichester, West Sussex, UK, 2013; pp. 27–50. [Google Scholar]
  20. Fisher, E.; Guston, D. Making Responsible Innovators. In Does America Need More Innovators? Wisnioski, M., Hintz, E., Kleine, M., Eds.; The MIT Press: Cambridge, UK, 2019; pp. 345–366. [Google Scholar]
  21. Nazarko, L. Responsible Research and Innovation in Industry: from Ethical Acceptability to Social Desirability. In Corporate Social Responsibility in the Manufacturing and Services Sectors; Golinska-Dawson, P., Spychała, M., Eds.; Springer: Cham, Switzerland, 2019. [Google Scholar]
  22. Ceicyte, J.; Petraite, M. Networked Responsibility Approach for Responsible Innovation: Perspective of the Firm. Sustainability 2018, 10, 1720. [Google Scholar] [CrossRef]
  23. Gurzawska, A.; Makinen, M.; Brey, P. Implementation of Responsible Research and Innovation (RRI) Practices in Industry: Providing the Right Incentives. Sustainability 2017, 9, 1759. [Google Scholar] [CrossRef]
  24. Timmermans, J.; Yaghmaei, E.; Stahl, B.C.; Brem, A. Research and innovation processes revisited - networked responsibility in industry. Sustain. Account. Manag. Policy J. 2017, 8, 307–334. [Google Scholar] [CrossRef]
  25. Wiek, A.; Foley, R.; Guston, D.; Bernstein, M. Broken promises and breaking ground for responsible innovation—Intervention research to transform business-as-usual in nanotechnology innovation. Technol. Anal. Strateg. Manag. 2016, 28, 639–650. [Google Scholar] [CrossRef]
  26. Blok, V.; Hoffmans, L.; Wubben, E.F.M. Stakeholder engagement for responsible innovation in the private sector: Critical issues and management practices. J. Chain Netw. Sci. 2015, 15, 147–164. [Google Scholar] [CrossRef]
  27. Pavie, X. Responsible Innovation: From Concept to Practice; World Scientific: Hackensack, NJ, USA, 2014. [Google Scholar]
  28. Stilgoe, J.; Owen, R.; Macnaghten, P. Developing a framework for responsible innovation. Res. Policy 2013, 42, 1568–1580. [Google Scholar] [CrossRef]
  29. Owen, R.; Macnaghten, P.; Stilgoe, J. Responsible research and innovation: From science in society to science for society, with society. Sci. Public Policy 2012, 39, 751–760. [Google Scholar] [CrossRef]
  30. Stahl, B. Responsible research and innovation: The role of privacy in an emerging framework. Sci. Public Policy 2013, 40, 708–716. [Google Scholar] [CrossRef]
  31. Nazarko, L.; Melnikas, B. Responsible Research and Innovation in Engineering and Technology Management: Concept, Metrics and Assessment. In Proceedings of the 2019 IEEE Technology & Engineering Management Conference, Atlanta, GA, USA, 12–14 June 2019. [Google Scholar]
  32. Inzelt, A.; Csonka, L. The Approach of the Business Sector to Responsible Research and Innovation (RRI). Foresight Sti Gov. 2017, 11, 63–73. [Google Scholar] [CrossRef]
  33. Long, T.B.; Blok, V. Integrating the management of socio-ethical factors into industry innovation: Towards a concept of Open Innovation 2.0. Int. Food Agribus. Manag. Rev. 2018, 21, 463–486. [Google Scholar] [CrossRef]
  34. Von Schomberg, R. Why Responsible Innovation. In International Handbook on Responsible Innovation: A Global Resource; Schomberg, R.V., Hankins, J., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2019; pp. 12–32. [Google Scholar]
  35. Chatterji, A.K.; Levine, D.I.; Toffel, M.W. How Well Do Social Ratings Actually Measure Corporate Social Responsibility? J. Econ. Manag. Strategy 2009, 18, 125–169. [Google Scholar] [CrossRef]
  36. Ratajczak, P.; Szutowski, D. Exploring the relationship between CSR and innovation. Sustain. Account. Manag. Policy J. 2016, 7, 295–318. [Google Scholar] [CrossRef]
  37. Ravn, T.; Nielsen, M.W.; Mejlgaard, N. Metrics and Indicators of Responsible Research and Innovation. Progress Report D3.2; MoRRI Project: Woluwe-Saint-Pierre, Belgium, 2015. [Google Scholar]
  38. Woolley, R.; Rafols, I. Progress Report D6: Definition of Metrics and Indicators for RRI Benefits; MoRRI Project: Woluwe-Saint-Pierre, Belgium, 2016. [Google Scholar]
  39. Mishra, S.; Singh, R. Responsible Innovation: A New Approach to Address the Theoretical Gaps for Innovating in Emerging E-Mobility Sector. In Governance and Sustainability of Responsible Research and Innovation Processes. Cases and Experiences; Ferri, F., Dwyer, N., Raicevich, S., Grifoni, P., Altiok, H., Andersen, H.T., Laouris, Y., Silvestri, C., Eds.; Springer: Cham, Switzerland, 2018; pp. 93–99. [Google Scholar]
  40. RRI self-reflection tool. Available online: (accessed on 6 February 2020).
  41. Kupper, F.; Klaassen, P.; Rijnen, M.; Vermeulen, S.; Broerse, J. Report on the Quality Criteria of Good Practice Standards in RRI. Available online: (accessed on 6 February 2020).
  42. Čeičytė, J. Implementing Responsible Innovation at the Firm Level; Kaunas University of Technology: Kaunas, Lithuania, 2019. [Google Scholar]
  43. Van de Klippe, W. From MoRRI to SUPER_MoRRI: Monitoring as reflection and learning, not representation and control. Available online: (accessed on 6 February 2020).
  44. Hedstrom, G.S. Sustainability: What It Is and How to Measure It; Walter de Gruyter Inc.: Boston, MA, USA, 2019. [Google Scholar]
  45. Nazarko, L.; Melnikas, B. Operationalising Responsible Research and Innovation—Tools for enterprises. Eng. Manag. Prod. Serv. 2019, 11, 21–28. [Google Scholar] [CrossRef]
  46. Gianni, R. Responsibility and Freedom: The Ethical Realm of RRI; Wiley: London, UK, 2016. [Google Scholar]
  47. Dreyer, M.; Chefneux, L.; Goldberg, A.; Von Heimburg, J.; Patrignani, N.; Schofield, M.; Shilling, C. Responsible Innovation: A Complementary View from Industry with Proposals for Bridging Different Perspectives. Sustainability 2017, 9, 1719. [Google Scholar] [CrossRef]
  48. Ejdys, J.; Matuszak-Flejszman, A.; Szymanski, M.; Ustinovichius, L.; Shevchenko, G.; Lulewicz-Sas, A. Crucial factors for improving the ISO 14001 environmental management system. J. Bus. Econ. Manag. 2016, 17, 52–73. [Google Scholar] [CrossRef]
  49. Stahl, B.C.; Wright, D. Ethics and Privacy in AI and Big Data: Implementing Responsible Research and Innovation. IEEE Secur. Priv. 2018, 16, 26–33. [Google Scholar] [CrossRef]
  50. Nazarko, J.; Ejdys, J.; Halicka, K.; Magruk, A.; Nazarko, L.; Skorek, A. Application of Enhanced SWOT Analysis in the Future-oriented Public Management of Technology. Procedia Eng. 2017, 182, 482–490. [Google Scholar] [CrossRef]
  51. Halicka, K. Main Concepts of Technology Analysis in the Light of the Literature on the Subject. Procedia Eng. 2017, 182, 291–298. [Google Scholar] [CrossRef]
  52. Winkowski, C. Classification of forecasting methods in production engineering. Eng. Manag. Prod. Serv. 2019, 11, 23–33. [Google Scholar] [CrossRef]
  53. Nazarko, L. Future-Oriented Technology Assessment. Procedia Eng. 2017, 182, 504–509. [Google Scholar] [CrossRef]
  54. Pearson, J.; Gianni, R.; Ikonen, V.; Haick, H. From technology assessment to responsible research and innovation (RRI). In Proceedings of the 2016 Future Technologies Conference (FTC), San Francisco, CA, USA, 6–7 December 2016. [Google Scholar]
  55. Collingridge, D. The Social Control of Technology; St. Martin’s Press: New York, NY, USA, 1980. [Google Scholar]
  56. Garst, J.; Blok, V.; Jansen, L.; Omta, O. Responsibility versus Profit: The Motives of Food Firms for Healthy Product Innovation. Sustainability 2017, 9, 2286. [Google Scholar] [CrossRef]
  57. Turovets, Y.; Vishnevskiy, K. Patterns of digitalisation in machinery-building industries: Evidence from Russia. Eng. Manag. Prod. Serv. 2019, 11, 7–22. [Google Scholar] [CrossRef]
  58. ROSIE Project. Available online: (accessed on 22 December 2019).
  59. Halicka, K. Gerontechnology—The assessment of one selected technology improving the quality of life of older adults. Eng. Manag. Prod. Serv. 2019, 11, 43–51. [Google Scholar] [CrossRef]
  60. Lund Declaration. Available online: (accessed on 22 December 2019).
  61. Dijkstra, A.; Schuijff, M.; Yin, L.; Mkansi, S. RRI in China and South Africa: Cultural Adaptation Report: Deliverable 3.3; NUCLEUS project: Enschede, The Netherland, 2017. [Google Scholar]
  62. Schroeder, D.; Kaplan, D. Responsible inclusive innovation: Tackling grand challenges globally. In International Handbook on Responsible Innovation: A Global Resource; Hankins, J., Schomberg, R.V., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2019; pp. 308–324. [Google Scholar]
  63. Nazarko, J.; Kuźmicz, K.A.; Szubzda-Prutis, E.; Urban, J. The general concept of benchmarking and its application in higher education in Europe. High. Educ. Eur. 2009, 34, 497–510. [Google Scholar] [CrossRef]
  64. Chodakowska, E.; Nazarko, J. Environmental DEA method for assessing productivity of European countries. Technol. Econ. Dev. Econ. 2017, 23, 589–607. [Google Scholar] [CrossRef]
  65. Nazarko, J.; Chodakowska, E.; Kaplinski, O.; Paslawski, J.; Zavadskas, E.; Gajzler, M. Measuring productivity of construction industry in Europe with Data Envelopment Analysis. Innov. Solut. Constr. Eng. Manag. Flex. Approach 2015, 122, 204–212. [Google Scholar] [CrossRef]
  66. Berner, S.; Derler, H.; Rehorska, R.; Pabst, S.; Seebacher, U. Roadmapping to Enhance Local Food Supply: Case Study of a City-Region in Austria. Sustainability 2019, 11, 3876. [Google Scholar] [CrossRef]
  67. Jou, G.; Yuan, B. Utilizing a Novel Approach at the Fuzzy Front-End of New Product Development: A Case Study in a Flexible Fabric Supercapacitor. Sustainability 2016, 8, 740. [Google Scholar] [CrossRef]
  68. Schwerdtner, W.; Siebert, R.; Busse, M.; Freisinger, U. Regional Open Innovation Roadmapping: A New Framework for Innovation-Based Regional Development. Sustainability 2015, 7, 2301–2321. [Google Scholar] [CrossRef]
Figure 1. The number of publications that contain the phrases “responsible research and innovation” or “responsible innovation” in their title, abstract or keywords, indexed in Scopus database (source: own elaboration on the basis of Scopus bibliographic data).
Figure 1. The number of publications that contain the phrases “responsible research and innovation” or “responsible innovation” in their title, abstract or keywords, indexed in Scopus database (source: own elaboration on the basis of Scopus bibliographic data).
Joitmc 06 00012 g001
Figure 2. Visualization of the network of keywords in RRI-related literature (source: own elaboration on the basis of Scopus data with use of VOSviewer software (ver. 1.6.14, Centre for Science and Technology Studies (CWTS) of Leiden University, Netherlands, 2020)).
Figure 2. Visualization of the network of keywords in RRI-related literature (source: own elaboration on the basis of Scopus data with use of VOSviewer software (ver. 1.6.14, Centre for Science and Technology Studies (CWTS) of Leiden University, Netherlands, 2020)).
Joitmc 06 00012 g002
Figure 3. Stage 1 of the RRI index development from the perspective of a single enterprise (source: our own elaboration on the basis of Reference [45]).
Figure 3. Stage 1 of the RRI index development from the perspective of a single enterprise (source: our own elaboration on the basis of Reference [45]).
Joitmc 06 00012 g003
Figure 4. Stage 3 of the RRI index development from the perspective of a single enterprise.
Figure 4. Stage 3 of the RRI index development from the perspective of a single enterprise.
Joitmc 06 00012 g004
Table 1. Thematic categories of RRI-related literature.
Table 1. Thematic categories of RRI-related literature.
1Epistemological foundations of RRIvalues, Grand Challenges, ethics, human rights
2Concepts and frameworks related to RRICorporate Social Responsibility, sustainability, sustainable development, social responsibility, value-sensitive design
3Discussion on RRI principlesresponsiveness, anticipation, participation, engagement
4Analysis of RRI as a policy frameworkresearch policy, anticipatory governance, participation and science governance, governance, innovation policy, science policy
5Conceptualizing entrepreneurship and innovation in line with RRI principlesentrepreneurship, social entrepreneurship, social innovation, open innovation
6Sectoral approach to RRIindustry, emerging technologies, nanotechnology, artificial intelligence, big data, human brain project, ICT, synthetic biology, technology
7Educational and participatory aspect of RRIscience communication, public engagement, science education, stakeholder engagement, citizen science, science and society, stakeholders
8Tools and methods for RRIforesight, technology assessment, risk assessment
9Institutional origin and source of fundingHorizon 2020
Table 2. RRI principles and promised benefits.
Table 2. RRI principles and promised benefits.
RRI PrinciplePromised Benefits
1AnticipationAwareness of possible future legislation
Increased foresight competences and better risk management
“First mover” advantage
2ReflexivityHigher quality of innovation outcomes due to third-party critical appraisal
Higher probability of innovation success
3TransparencyBetter ability to interpret available information thanks to the culture of sharing knowledge
Higher effectiveness and efficiency of collaboration initiatives
Better ability to match social expectations
4ResponsivenessIncreased trust from customers
Improved corporate image and increased public trust in offered goods
Better insight into needs and preferences of customers
Source: own elaboration on the basis of [21,22,23,24,25,26,27,28].
Table 3. RRI criticism and identified barriers to its implementation.
Table 3. RRI criticism and identified barriers to its implementation.
Problematic IssueImplications for Innovating Organizations
1RRI concept under constant development; different views and understandings of RRI framework in the literatureIndifference towards the RRI idea
Little comprehension of the notion of RRI
2Western Eurocentrism of the RRI conceptLittle chance of buy-in at the global level
Regulatory and cultural differences between countries even within EU
3No clear “division of labor” in the sphere of responsibility in the innovation activity as a consequence of the “shared responsibility” [29] or “meta-responsibility” [30] conceptsNo clear indication of what is actually expected and at what stage from innovating businesses
4Tension between “excellence” and “responsibility” both in science and businessTreating all activities related to RRI as an additional burden that makes innovation more costly and time consuming
5Insistence on transparency and open accessCorporate strategies of intellectual property management not aligned with the open access paradigm
6Shortage of understandable and easy-to-use tools to measure responsible innovation in businessLittle interest in filling out long surveys and exhausting self-reflection forms
Source: own elaboration on the basis of [21,31,32].
Table 4. Examples of RRI assessment criteria for public entities.
Table 4. Examples of RRI assessment criteria for public entities.
RRI Policy AgendaNames of Indicators
1Gender equalityshare of organizations with gender equality plans, share of female researchers, share of female authors and inventors, gender wage gap
2Science literacy and science educationimportance of societal aspects of science in science curricula, availability of RRI-related training at higher education institutions, number of citizen science publications in Scopus, organizational membership in European Citizen Science Association
3Public engagementmodels of public involvement in science and technology (S&T) decision-making, Active information search about controversial technologies innovation democratization
4Open Accessshare of Open Access publications, citation scores for Open Access publications, incentives and barriers for data sharing
5Ethicspresence and performance of research ethics structures at research performing and research funding organizations
6Governanceuse of science in policy making, RRI-related governance mechanisms within research-funding and performing organizations
Source: own elaboration on the basis of [6,37,38].
Table 5. Examples of RRI assessment criteria for enterprises.
Table 5. Examples of RRI assessment criteria for enterprises.
RRI Policy AgendaNames of Metric/Indicator
1Gender equalityImplementation of a gender equality plan and practices regarding team composition, management positions, work spaces, salary levels, contract conditions
Systematic evaluation of the state of gender equality in the organization
Provision of gender equality training
Identification of gender stereotypes present in R&I activity
2Science literacy and science educationSupporting citizens in making informed decisions
Increasing stakeholder awareness that R&I can create solutions that have impact on their lives
Using different outreach channels and adapting contents according to the target group
3Public engagementEngagement of relevant stakeholders in the innovation process (civil society organizations, local government, education community, customers, patients, families, etc.)
Conducting outreach activities and reflecting on them
Addressing conflicts of interests
4Open AccessEnsuring transparency and open access throughout the innovation process
Clear traceability of ownership and authorship
5EthicsAnticipation of the benefits and risks of innovation project (including long-term side effects)
Ensuring project outcomes are used responsibly even after the project
Alignment with the Code of Conduct for Research Integrity
Encouragement of critical peer review and internal discussion on ethical issues throughout the process
Consultation with external ethics experts or committees
6GovernanceOpenness to emerging societal needs
Readiness to change the research plan or innovation project in response to unforeseen results or as a result of a dialogues with the stakeholders
Providing time for reflection during the innovation processes
Appointment of a staff RRI expert
Providing RRI training to employees
Source: own elaboration on the basis of [40,41,42].
Back to TopTop