An Investigation into Risk Perception in the ICT Industry as a Core Component of Responsible Research and Innovation

Abstract

This paper makes an original contribution to the responsible research and innovation (RRI) discourse, with an inquiry into the extent to which risk, risk assessment, or risk management, including ethical and social issues, is relevant to companies. As a core component of the higher or “meta-responsibility” of RRI, an investigation of practices and attitudes towards risks can provide us with a window into companies’ attitudes towards responsible innovation that is rooted in real-world experiences. Drawing upon data from 30 in-depth interviews and a large Delphi study, we reveal different underlying attitudes towards risk governance for individuals working in the information and communication technology (ICT) industry. For some companies, there is already an obvious degree of alignment with RRI values. For others, framing of the RRI discourse in terms of ethical and societal risks may help to promote understanding and uptake. Results from the interviews suggest that lack of awareness of the full extent of ethical and societal risks associated with research and innovation in the ICT industry may act as a barrier to engagement with RRI, and educational activities may be needed to rectify this situation. Results from the Delphi survey suggest that when presented with simple information about potential ethical and societal risks, industry personnel can easily recognise the main risks and provide clear opinions about how they should be addressed. The relationship between risk governance and RRI warrants further investigation as it is an essential facet of RRI.

1. Introduction

Information and communication technologies (ICTs) yield numerous social and economic benefits, but they can also raise ethical, societal and environmental concerns. Companies that undertake research and innovation (R&I) in ICTs have an important responsibility in identifying, understanding and addressing potential benefits and hazards [1]. However, despite the crucial role for companies in effecting socially acceptable uses of technology, attention is not routinely paid to the ethical and social implications of their R&I activities. A large body of research has investigated questions of business ethics for corporate activities, such as finance, professional integrity, workers’ rights, and so on. In contrast, the question of how R&I activities fit into the overall picture of corporate responsibilities has received less attention [2].

Our investigation is inspired by the discourse surrounding responsible research and innovation (RRI). RRI is a field of study and practice that has gained prominence over the last decade, its aim being to ensure that R&I activities are socially acceptable, desirable, and sustainable [3]. A number of research funding organisations, including the European Commission, have adopted RRI as both a subject of study and a condition of funding [4]. However, to date, the promotion of RRI has focused predominantly on publicly funded research, omitting a substantial proportion of the company-based innovation activities, occurring at a more advanced technology readiness level in preparation for market entry [5].

In this study, we focus exclusively upon R&I in the private arena, specifically in the ICT industry. Our underlying aim is to seek insight into what might motivate private ICT companies to conduct their R&I activities in a responsible manner. The answer to this question goes to the heart of companies’ self-perception and their role in society. It is also important because, to a large degree, this answer drives organisational practices.

The premise for our empirical inquiry is that an investigation into attitudes and behaviours associated with risk will help to shed light on the debate about the relevance and motivating factors for RRI in commercial settings. Given that a primary aim of RRI is to ensure that R&I activities are ethically and socially acceptable, an exploration of companies’ attitudes towards ethical and societal risks in their R&I activities could help us to assess whether the introduction of RRI might be viewed as necessary or helpful. Additionally, an exploration of the motivations and systems for addressing and managing such risks might in turn help us to understand potential willingness for adoption of an RRI framework.

Such an investigation cannot provide us with a complete understanding of potential company motivations for engaging with RRI, not least because the concept of RRI goes way beyond the traditional understanding of governance based upon formal risk assessment [3]. The European Commission makes this clear with their declaration that RRI implies better alignment of the processes and outcomes of R&I with the values, needs and expectations of society [6]; it is not simply about avoiding harm.

However, the concept of RRI is new for the majority of people working in industry, and hence, investigation into attitudes towards RRI might only reveal purely theoretical considerations. On the other hand, investigation into attitudes and behaviours towards ethical and societal risks of R&I could help to reveal more deeply embedded opinions that are rooted in real world experience.

This paper makes an original contribution to the RRI discourse by investigating the extent to which consideration of ethical and societal risks, risk assessment, or risk management, is relevant to companies. This is important to understand because companies tend to have established risk management processes. If existing practices and processes are in some ways already aligned with the key objectives of RRI, then the integration of RRI into company procedures will be more straightforward, thus aiding promotion and uptake of RRI.

The paper proceeds as follows: We begin with a short review of the key underpinning concepts for the investigation, namely RRI, risk assessment and the specific risks of ICTs. On this basis, we then introduce the methodology of the study that draws upon two methods of data collection for the analysis, namely, 30 in-depth interviews and a large Delphi study. Following our presentation of the findings from these two methods, the subsequent discussion highlights our key insights, leading to our conclusions.

2. Background

2.1. Responsible Research and Innovation

Use and discussion of the term “RRI” has been gathering momentum since around 2010 [7,8,9], and the level of attention awarded to the topic of RRI has now increased to the point where it has its own dedicated journal. Furthermore, RRI has been adopted in slightly different variants by a number of research funders, such as the European Commission [10], and the UK Engineering and Physical Sciences Research Council [11].

In general terms, RRI can be thought of as a move towards greater inclusion and responsiveness in the governance of research and innovation [3]. While discussion and debate about the precise nature of RRI is ongoing [12], there are certain recurring themes that emerge from the RRI discourse. These include matters such as the need for alignment of R&I with societal needs, the need to anticipate and be responsive to ethical, environmental and societal concerns, and the need to enhance these efforts through engagement with a broad range of stakeholders, including members of the public [13,14].

The term “RRI” may be relatively new, but the concept rests upon and furthers earlier work in areas such as science and technology studies, technology assessment, and technology ethics [15]. For example, in order to anticipate and be responsive to ethical, environmental and societal concerns, efforts must be made to actively seek knowledge about the future consequences of R&I activities [16]. This might be achieved through the integration of foresight and future studies [17,18], incorporating activities such as risk assessment [19], or a variety of impact assessments, such as social impact assessment [20], environmental impact assessment [21], or privacy impact assessments [22].

Many of the component activities of RRI, such as risk management or public engagement, are well established and have been around for decades. There are long-standing debates and practices in areas, such as technology assessment [15], that have been promoting the integration of foresight and public engagement in science and technological development for decades. Hence, it is reasonable to ask what is novel about the concept of RRI.

One response to this crucial question is that responsible innovation extends a more conventional ethical review of research, and can even be viewed as “creating opportunity” [13]. According to Owen et al., the departure point for RRI is that adoption of responsible innovation compels us to reflect on what sort of future(s) we want science and technology to bring into the world, and how the aims and objectives of R&I can be identified in an ethical, inclusive, and equitable manner.

Whilst we accept that this explanation makes a perfectly valid and important point, our response to the question about the novelty of RRI comes from another perspective, suggesting that RRI is located at a different level to existing activities. The notion that RRI can be viewed as a type of ‘meta-responsibility’, as proposed by Stahl [2], implies that responsible innovation can best be conceptualised as a higher-level responsibility that encompasses existing responsibilities. Numerous responsibilities are already ascribed and defined for R&I activities. For instance, individual researchers are responsible for the integrity of their work, research institutions are responsible for provision of research infrastructure, funders are responsible for competent administration grants. RRI, as a meta-responsibility, encompasses these existing responsibilities, and serves to ensure that they are aligned and synergetic, and that they promote shared aims or outcomes. Viewed in this way, work in RRI can start by mapping and understanding existing responsibilities and their practical implementations. In practice, responsibilities are never individual but always embedded in networks or ecosystems of responsibility. Understanding the relationship between these responsibilities is a key condition of successfully modifying and developing them to achieve desired outcomes.

In this study, we explore one particular aspect of RRI, or one particular aspect of responsibility, namely its link to risks, risk assessment, and risk management. This approach is in keeping with our intention to investigate potential links between RRI and industry. To date, the RRI literature has focused primarily upon publicly funded research and in so doing has neglected the fact that key decisions about R&I are often made elsewhere, notably in the private sector [23,24]. This is particularly relevant, because there could be marked conflicts between the adoption of RRI and commercial interests [25]. It is therefore important to think about whether, and to what degree, RRI could be relevant to private companies. One avenue that has been proposed and pursued elsewhere [5,26,27] is that RRI is closely linked to corporate social responsibility (CSR). We believe that this is appropriate, and the arguments in favour of CSR, namely moral obligation, sustainability, license to operate, and reputation [28], can be broadly applied to RRI.

However, in this study, we approach the topic from a slightly different perspective, to investigate existing attitudes, practices and responsibilities associated with risk. Our approach has been inspired by the observation that many of the aspects that are addressed by RRI can easily be framed as risks. This is not to say that the concept of RRI can be reduced in a simplistic manner to notions of risk, rather, we are holding that this aspect of responsibility is an essential component of the meta-responsibility of RRI. Many of the ethical issues associated with ICT developments carry significant risks for organisational success. In addition, one can observe that the early discussion of RRI was often framed in terms of risk and risk management [8]. The precautionary principle, which has close links to RRI, is a way of dealing with large-scale risks [29,30]. Some of the ways in which RRI may implemented, such as via privacy impact assessments, have clear risk management characteristics [31]. Risk assessment has played an important role in the development of responsible attitudes and procedures for R&I in industry, as touched on in the following section.

2.2. Risk Assessment

Industrial activities, including R&I activities, can lead to outcomes that were unintended and/or unexpected. Sometimes these outcomes have hugely damaging effects upon individuals, communities, or the environment. As the consequences of industrial activity become more well known, so has the requirement to identify and evaluate the impacts become more pressing [32]. Citizens of advanced technological societies demand a degree of certainty about the benefits of science and technology. Even when innovation is conducted largely in the private sector, governments are expected to ensure that corporate profit motives will not lead to unreasonable harms [33]. Set against this backdrop, the field of risk assessment has emerged, along with the development of a wide variety of tools. Numerous tools are now available to support the various phases of the risk management process [34].

As well as uncertainty about the nature of adverse consequences, risk can also involve uncertainty about the timing or magnitude of outcomes. Covello and Merkhoher [35] define risk assessment as “a systematic process for describing and quantifying the risks associated with hazardous substances, processes, actions or events” (p. 3). Risk assessment, for many, has become a systemised process that is reliant upon tools, and nothing more than a “tick-box” or checklist activity [36]. According to Kasper [32], measures of risk fall into two categories:

• Those that observe or calculate the risk of a process.
• Those that rely upon the perceptions of individuals.

Measures in the first category normally rely upon tools that are often quantitative in nature, and involve sophisticated statistical analysis, often framed in probabilities. Here, there will be an attempt to weigh the potential for benefits against the potential for harm in an objective manner. Measures in the second category are of a more subjective nature, and they tell us what people think the risks of a particular activity might be. Whilst this may describe a traditional and broadly accepted approach to risk assessment, it is clear that effective risk assessment involves more than a simplistic probability analysis of potential for harms. For example, as Jasanoff [37] maintains, the meaning of risk varies from one cultural context to another; risk concepts are culturally and politically conditioned ways of interpreting both our relationship to the world around us, and our obligations to others on the planet.

This inquiry looks beyond simple beneficent and maleficent concerns that are key to standard risk assessment procedures, to enquire about attitudes towards broader ethical and societal concerns. Furthermore, we accept that concepts of risk may be embedded within, and influenced by, societal and industrial cultures. The specific challenges for effective risk assessment in the ICT industry are acknowledged in this study. In a fast-moving field like ICT, developments easily outpace the refinement of tools. Similarly, risk assessment that relies upon the experience of individuals must be of limited value when working with cutting-edge developments. Some of the distinctive risks posed by R&I in the ICT industry can have pervasive effects as described in the following section.

2.3. Ethical and Societal Risks Associated with R&I in ICT

RRI and its predecessors, such as technology assessment, science and technology studies, or technology ethics, originally focused upon technologies and developments that were associated with the most obvious ethical issues and risks. One high profile example of such a technology, often evoked as a motivation for developing RRI, is that of genetically modified organisms [38]. Other examples of technologies with high stakes and high risks include nuclear power, or the outputs of the chemical or pharmaceutical industries.

Unlike these high profile examples, innovations in ICT do not generally raise significant fears in society. In fact, one could argue that the opposite is true; the use of ICT has become so ubiquitous that many aspects are hardly questioned. ICT developments are integrated into all aspects of personal, professional and social life, from the individual use of social networks via mobile phones, to global supply chain planning systems that are used worldwide. The ubiquity and pervasiveness of ICTs have, however, rendered them a primary target for RRI promotors, including in-depth consideration of RRI for ICT and security [39]. Additionally, ICTs have a number of other characteristics that, whilst not necessarily unique, can combine to generate consequences that make the applicability of RRI to ICTs seem eminently reasonable. In addition to their ubiquity and pervasiveness [40,41], these characteristics of ICT include their speed of innovation, the problem of many hands, their logical malleability, and their interpretive flexibility, as further described below.

Firstly, the speed of innovation in ICTs set them apart from other technologies. Through use of existing ICT infrastructures, a researcher or innovator can devise new applications that have the potential to go viral and become available worldwide within an extremely short timescale. Secondly, the “problem of many hands” [42] stems from the issue that the development of ICTs is cumulative; new ICT developments often build upon and incorporate existing technologies. This is most visible in software where new code is built upon already existing codes. ICT research and development is often distributed in ways that render it impossible to trace particular characteristics or behaviours back to an individual researcher or designer. Thirdly, the logical malleability [43,44] of ICTs means that prediction of all future consequences can be impossible; these technologies are typically designed as multi-purpose machines where even the designers cannot predict the uses to which they will be put. Lastly, interpretive flexibility, a feature of all technologies [45], means that the technology itself does not determine its use. While this is not a challenge that is exclusive to ICTs, when combined with their inherent logical malleability, it becomes a particularly pertinent challenge for this field.

Because of this combination of characteristics, and because of their unique function of transmitting, processing and storing data, ICTs give rise to a large number of ethical and societal concerns [46,47]. A recent review of the literature identified a broad range of such issues [48]. Unsurprisingly, the most prominent, and by far most widely discussed issues, are those of privacy and data protection. However, it is important to underline that many other issues were identified, ranging from professionalism, changes in the workplace, security and digital inclusion, to challenges for autonomy, agency, trust, and identity. In their scope, these issues encompass a wide spectrum, from the very specific and tangible, to those of a more avant-garde nature, such as the possibility of super-human artificial intelligence.

From the perspective of a company involved in R&I in ICT, these issues can be perceived as risks. If customers, end-users, or communities reject novel technologies because of worries about their possible consequences, then this can reduce the company’s profits, and even turn into a threat for organisational survival. Companies therefore need to be cognizant of, and engage with, these issues. At the same time, many of the traditional mechanisms for dealing with ethical concerns are problematic for ICT. A good example of this type of problem might be the application of individual informed consent procedures to data protection issues. Informed consent is a cornerstone of biomedical ethics and based on sound ethical reasoning [49]. However, its application to large-scale data is not always seen as successful [50,51]. Current data analytics technologies can be fundamentally exploratory, rendering obsolete the idea that the data subject might make an informed decision about consent for use of their data. Furthermore, the sheer scale of the data can obstruct or preclude the ability to gather consent from all in question.

Given the risks that are posed by ICTs, and the fact that novel governance structures may be called for, we sought to explore the question of how companies might relate to RRI via their attitudes and behaviours towards risk management. For our exploration, we implemented an empirical investigation as described in the following section.

3. Methodology

A multinational approach was utilised in this far-reaching, two-stage investigation, in which the findings from 30 in-depth interviews were compared and contrasted with those from a two-phase Delphi study that included 35 industry representatives.

Findings from the first phase of this study (30 interviews), concerning potential drivers and obstacles for RRI, are reported elsewhere in this special edition of Sustainability [12]. For the purpose of this investigation, the rich data set was completely reanalysed to reveal information related to risks and risk assessment.

In the second phase of our study, 165 stakeholders participated in a large, pan-European Delphi survey. Of the 165 respondents, 35 held key positions in the ICT industry. The entire body of data from these industry representatives has been extracted and analysed to provide an industry perspective.

Hence, the data for our investigation is derived from industry representatives through two distinct methods: from interviews that were purely qualitative in nature, and a Delphi study that was primarily quantitative.

3.1. Sampling Method

Purposeful sampling was implemented in both stages of the study, to ensure that participants had the relevant experience, and were able to provide a variety of perspectives from different countries and different-sized corporations. It was vital that all had experience of working in ICT industries that are actively involved in research and innovation.

3.2. The Interviews

In the first stage of the inquiry, qualitative, in-depth interviews were conducted with 30 industry representatives from across Europe to ensure a diversity of cultural perspectives. The interviews were conducted by a number of researchers who were located in different countries; this helped to ensure that spoken language was not a barrier to participation. To aid consistency between the interviews, a semi-structured interview schedule was used to guide the topics of investigation. While some questions were pre-set, this also allowed for exploration of other unforeseen subjects. The interview questions covered a range of topics that related to the implementation of RRI in industry. These included questioning about efforts that are made to predict the impact of developments on the environment, society, and the wellbeing of users (see Appendix A).

Potential participants were sent an invitation to participate together with an information sheet and consent form in their own language. Interviews were arranged for those that provided consent, and they were conducted either in person, where possible, or via web-conferencing or telephone. Each interview lasted approximately 45–60 min; they were audio recorded, transcribed, and translated into English where necessary. Recruitment continued until the target number of 30 interviews had been reached. All interviews were conducted in line with ethical boundaries, such as informed consent, respect of privacy, and avoidance of harm and deception [52,53].

Analysis of the transcripts was undertaken centrally. Anonymised transcripts were ascribed an identification code (1–30) and analysed through a stepped process of content thematic analysis with the aid of NVivo version 10, qualitative data analysis software. The first stage of open coding was followed by thematic coding, during which all emerging themes associated with awareness or assessment of risks were compared, contrasted, and gradually refined. The final stage of the analysis consisted of in-depth comparison with the results from the Delphi stage. This included a search for potential links between the findings and industry size, type, and location.

3.3. Two-Round Delphi Study

In the second stage of the inquiry, a Delphi study was undertaken with experts in the field of ICT for an ageing society from across Europe. Delphi studies were originally developed to help organise debate, to collect and synthesise opinions, and to achieve a degree of convergence as an alternative to open discussions where the views of some may be lost amongst those who are more vocal [54]. In the classic Delphi studies, an anonymised, iterative, multistage survey process is used to gather opinions from all participants (mainly experts in the field) which are then combined, ideally until group consensus or stalemate is reached. The exchange of opinions over several rounds has the advantage that feedback processes are possible, which encourage participants to re-examine their own evaluations. The data is evaluated between rounds by use of appropriate statistical methods. As a rule, the spectrum of assessments is reduced, trends become clearer, and convergence of opinions is reached.

Since the method was first proposed at the Rand Corporation in the early 1950s [55], variations of the Delphi method have evolved, in an effort to meet the specific needs of different decision makers. There are now many different types of Delphi exercises and they can be implanted in a wide range of scenarios. For example, Delphis can be constructed to help identify and prioritise policy goals. Delphis are no longer simply about achieving consensus, they are more often used to test whether there is already consensus [56]. Some versions of the Delphi survey are explicitly designed to identify different clusters of opinion [56]. In practice, modern Delphis do not make much, if any, use of iterations of the questionnaire. These Delphi surveys employ only two rounds of survey, inviting a deepening of exploration in the second round rather than aiming for consensus of the group. Thus, an individual can express a distinctly different opinion to the group perspective. This implementation of the Delphi exercise was adopted for the present study where multiple perspectives of different stakeholders are recommended for decision-making [57,58]. It means that the responses from a selected group of the participants, in this case the industry personnel, can be extracted and analysed in isolation, without compromising the integrity of the results.

In our two-round study, the results from the previous qualitative interviews were used to inform development of the largely quantitative initial Delphi questionnaire (see Delphi stage 1 in Supplementary Materials), covering issues such as:

• Awareness of RRI
• Integration of RRI into the product value chain
• Responsible governance
• Inclusion of RRI dimensions in the ICT for ageing society area.

The survey was delivered via an online platform (surveymonkey.com) that is commonly used for scientific and commercial surveys.

The first questionnaire was sent to 500 experts who had been identified as being interested and/or involved in areas that are relevant to the topic of RRI. The potential participants spanned a wide geographical area and had a variety of roles within their organisations. Of the 500 who were contacted, 165 completed the online questionnaire.

Following descriptive analysis of the results from the first round, the findings were used as a guide for the formulation of a second different questionnaire. This questionnaire included new, more qualitative and in-depth questions, to enable further elaboration of the most salient aspects emerging from the first round (see Delphi stage 2 in Supplementary Materials). In the second round of the survey, the respondents to the first questionnaire were invited to get acquainted with the summarised opinions that had been expressed in the first round, and were encouraged to express their opinions on debatable topics and make recommendations for effective operationalisation of RRI in industry.

Of the 165 Delphi respondents to the first questionnaire, 35 were from industry. Responses from industry representatives were extracted from the whole collection of completed questionnaires and analysed separately, enabling direct comparison with the results from interviews.

Quantitative analysis of the anonymised data from the two on-line surveys was undertaken with the aid of the Survey Monkey software, which offers the possibility of filtering the data by property (such as size of the organisation, role of respondent, country, etc.). The qualitative data from responses to the open questions was organised with the aid of NVivo software, and thematic analysis applied in a similar manner to the interviews.

4. Results

The results begin with a summary of participant details for both the interviews and the Delphi study. This is followed by our analysis of results from the interviews and then outcomes from the Delphi study. Here we present the emerging themes from the interviews, the analysed data from the Delphi study, and a comparison between them on matters related to the perception and avoidance of risk. Direct quotes from the participants are identified only by their code number. Results from the Delphi study are identified as arising from the first or second round of the survey.

4.1. Summary of Participants

In total, 30 interviews were conducted with participants located in 11 different countries, and 35 representatives from industry, located in 14 different countries, participated in at least one round of the Delphi study. All participants were holding, or had previously held, key positions in a large, medium, or small, ICT company; they were all working with projects that in some way related to the use of ICT for health and wellbeing. In both the interviews and the Delphi study, the majority of the participants were from small or medium sized enterprises (SMEs) (66% and 71%, respectively) and approximately one third of the participants were from large companies of more than 250 employees. To ensure anonymity, individual names of interviewees and their companies cannot be identified. Hence, a composite overview of participant characteristics is summarised in

Table A1. Anonymised Overview of the Interview Participant Details

Code	Country of Origin	Type of Business	Size	Position Held
1	Cyprus	Medical technology	SME	Founder/Chief executive
2	Cyprus	Research/technology	SME	Employee
3	UK	Data management	SME	Partner
4	UK	Consultancy/e-health	SME	Director
5	France	Health robotics	Large	Marketing
6	Switzerland	Medical technology	Large	Vice president
7	UK	Medical data solutions	SME	National manager
8	Germany	Medical technology	Large	Chief executive officer
9	Germany	Medical technology	Large	Development manager
10	Spain	ICT	Large	Project manager
11	Spain	Telemedicine	SME	Chief technology officer
12	Spain	ICT	SME	Chief executive officer
13	Finland	Green IT	SME	Global sourcing
14	Germany	Engineering	SME	Head of programme
15	Finland	Telecommunications	Large	Head of Innovation
16	NL	R&I	SME	Researcher
17	NL	R&I	SME	President
18	NL	Technology	Large	Research director
19	Italy	Telecommunications	Large	Project manager
20	Italy	ICT	SME	Information systems manager
21	Italy	Web healthcare apps	SME	Project worker
22	Denmark	ICT	SME	Head of research
23	Denmark	Technology	SME	Business development manager
24	Denmark	Technology	SME	Business development manager
25	Finland	ICT	SME	Development manager
26	Spain	Immersive technologies	SME	Research & development Manager
27	UK	Health NGO	SME	Programme lead
28	Spain	Technology	Large	International director
29	UK	Technology for elderly	Large	Chief executive officer
30	Sweden	ICT	Large	Vice president

and

Table A2. Anonymised Overview of Delphi Study Participant Details

Delphi Code	Country of Origin	Type of business	Size	Position Held
1	Switzerland	IT sector	SME	Business Consultant
2	Switzerland	IT services and solutions	SME	Project Manager
3	Turkey	e-Health	SME	Researcher
4	Finland	Sustainable development	SME	Director
5	Italy	Biotechnology/healthcare	SME	Manager
6 *	Finland	Healthcare	SME	Owner
7	Germany	Healthcare	Large	Team Leader
8 *	Spain	Home care and telecare	SME	CEO
9	Netherlands	Innovation	Large	Innovation Consultant
10 *	Austria	e-Health	SME	Project leader
11 *	Italy	Telecommunications	Large	Project manager
12 *	Sweden	IT solutions for the elderly	SME	CEO
13 *	Greece	ICT (e-Health)	SME	Manager
14 *	Italy	Visualisation products	SME	Innovation Manager
15	Cyprus	R&I	SME	Manager
16 *	Italy	IT solutions	SME	Director
17 *	Germany	IT-Automation	SME	CEO
18	Norway	Ambient Assisted Living	SME	CEO
19 *	Germany	Healthcare	Large	Sales strategy manager
20	Germany	Healthcare	Large	Information systems worker
21 *	Germany	Home fitness	SME	Manager
22	Finland	Sustainable Development	SME	CEO
23	Sweden	Technology	SME	Manager
24	Sweden	Telecare for the elderly	SME	Product manager
25	-	-	Large	-
26	Cyprus	Innovation	SME	Research Engineer
27 *	Belgium	Care for the elderly	Large	Director
28 *	Italy	e-Healthcare	Large	Project Assistant
29 *	Greece	Telemedicine	Large	Project Manager
30	Finland	Welfare innovations	SME	CEO
31	Finland	Architectural innovations	Large	Director
32	UK	-	SME	General Manager
33	Finland	IT services and solutions	SME	Business Development manager
34	Finland	Financial services	SME	Researcher
35 **	Germany	ICT	SME	Project Leader

Respondents from 1 to 34 participated in the 1st Delphi Survey. Participants in both Delphi Surveys are indicated by *. One respondent (N° 35) participated only in the 2nd Delphi Survey.

in Appendix B and Appendix C.

4.2. Results from the Interviews

When analysed from the perspective of risk awareness and assessment, three broad categories emerged from the interview data (profit assurance, profit plus, and data management) as detailed in

Table 1. Categorisation of the interview data related to risk awareness and explanation of category name.

Category	Explanation	Company Sizes
Profit assurance	Risk assessment is viewed primarily as profit-related; stakeholder engagement will help to increase product/service acceptance and associated sales.	SME = 10 Large = 4 Total = 14
Profit plus	Risk assessment is necessary to ensure profit but is also important for addressing broader societal needs (such as environmental concerns etc.)	SME = 6 Large = 4 Total = 10
Data management	Risk assessment is primarily concerned with data management and protection.	SME = 2 Large = 1 Total = 3

(For 3 of the interviewees the topic is either not relevant or the interviewee had very little to say on the matter of risk) Small or medium sized enterprise (SME).

and explained further below.

4.2.1. Profit Assurance

For almost half of those who were interviewed (n = 14), the topic of risk assessment was primarily associated with user engagement. When prompted, all interviewees, without exception, acknowledged both the need for, and benefits of, stakeholder involvement in the development of products and services. There was broad acceptance that the end-users are key drivers and must be involved. However, this is, first and foremost, to ensure appropriate and relevant performance of the product or service; it concerns quality and acceptance, as opposed to more general responsible motivations. If products and services are not received well, then reputation and profit are ultimately affected:

“It would be almost stupid to do something without having all the stakeholders involved … we would probably be out of business if we don’t deliver what works or what solves real need.”

(13)

For these companies, the broad impact or long-term consequences of R&I activities were of secondary or minor concern. While they may have formal governance structures and processes in place, these were of limited scope. For example:

“There are standards in place … but those standards refer mainly to performance and quality control as opposed to having any specific ethical content.”

(28)

Furthermore, eight interviewees declared that there was no consideration of longer-term consequences at all, with statements such as this:

“We do not use specific procedures to evaluate the risk of unintended consequences of our product development.”

(19)

Where there is some evidence of broader ethical and societal consideration, this appears to be undertaken in an ad hoc manner, with reliance upon personal experience for assessment of ethical risks:

“In general, inside our company, we do not have any professional ethical code. It is sufficient for us to use our moral intuitions, based on our experience, to evaluate if the data are sensitive or not. We ask our customers to sign a waiver to discharge us from legal liability.”

(20)

For companies like the ones in this category, it is clear that their current approaches, systems, and processes, are not currently aligned with RRI values. Furthermore, recommendations for RRI are unlikely to be welcomed with open arms, unless they convey the potential for (economic) benefit.

4.2.2. Profit Plus

In contrast to those in the previous category, one third of the interviewees described motivations and activities that extend beyond the pursuit of profit. Of course, profit is necessary for the survival of any private company, but this does not necessarily have to be the sole purpose, or raison d’être; other broader societal concerns are also of fundamental importance. The types of broader concerns may be specifically to do with enhancing the health or wellbeing of the end-users:

“Our research and development work is always carried out with the end-users, like persons with dementia and their relatives … The most significant action in our development work is to involve the end-users into process. All research and development should be based on actual needs and have significant meaning to the target group.”

(5)

Or they may be much broader. For example, consideration of effects at all stages of the value chain:

“(Risk assessment) is a part of the consumer design in the first place. So you have to understand the effects of the products, the effects of manufacturing the products, the effects from the raw materials and of course the effects can be social, they can be physical, they can be medical … following how we make our products, what we make our products from and so on.”

(15)

Even extending to different localities:

“We include worldwide risk assessment also providing information as to which risks they are within different countries, different geographical areas, and for different kinds of all the vendors that we have with regards to labour and the environment and their level of effects.”

(24)

Engagement in this type of in-depth risk assessment warrants more than just lip service. The products of research and innovation, even those that have been in development for some time, may be abandoned following the findings from risk assessment:

“Because the idea was great but the outcome, during development, shows that this potentially could not be useful or used in the market because of the potential risks that are discovered.”

(6)

4.2.3. Data Management

For those in the final category, risk assessment appeared to be concerned primarily with issues of data management and data protection. Perhaps unsurprisingly, when dealing with representatives from the ICT industry, concerns such as this were raised:

“The issue, in terms of responsible research and innovation, has come with the use of actual patient records technology and sharing the data across organisations of individuals and health care professionals and who has secondary rights to that data for the purposes of research and how to manage the governance around that.”

(3)

However, more surprising, is that matters concerning data management and protection were only highlighted by a minority of participants in the interviews. For those who did raise it as a major concern, the need for compliance with regulations and ethical guidelines was extremely important:

“We have to handle and manage a massive amount of sensitive data and information in health care. We have in place all the protocols that are necessary to guarantee data protection and safety.”

(21)

4.3. Results from the Delphi Study

The semi-structured narrative elicited from the interviews demonstrated wide variety in the extent to which potential risks were discussed or even acknowledged. By contrast, the Delphi study included direct, open and closed questions about participants’ opinions and practices related to societal and ethical risk assessment, which enabled specific exploration of the subject. In total, seven questions in stage 1 of the Delphi study covered topics related to opinions on what the main risks are, when they need to be addressed, by whom they should be addressed, and how they should be addressed. In stage 2 of the Delphi study, the analysed responses to these questions were summarised and presented alongside new, more probing, purposive or confirmatory questions. The opinions/positions of all stakeholders in the 1st stage were taken into account in the questionnaire for the 2nd stage, as a crucial element that industry should consider when pursuing RRI.

Our analysis of responses to these questions from Delphi stages 1 and 2 is presented below.

4.3.1. What the Risks Are

In Delphi stage 1, participants were asked to rate the level of societal and ethical risks arising from various ICT technologies. Responses from this analysis (detailed in Figure 1) show quite clearly that all were considered risky from an ethical and/or societal point of view, with a risk level ranging between 2.6/5 (medium) and 3.5/5 (high).

The highest ethical risks were assigned to the following factors:

• Transmission of data to a third party (3.5/5)
• Reasoning systems for privacy–sensitive data analysis (for example, noise analysis for activity recognition) (3.4/5)
• Brain–computer interface (3.4/5).

The particular domains that were considered most susceptible to ethical and societal risks in the design and development of ICT products are represented in Figure 2, with the most risky by far being identified as “Individual rights and liberties” (privacy, rights to freedom of movement, etc.).

4.3.2. When to Address Risks

When asked at what stage ethical and societal risks should be addressed, many stages were thought to be important, as shown in Figure 3.

The two most important stages for risk assessment were deemed to be, the early planning stage (50% agreement) and the entire value chain (41% agreement). In Delphi stage 2, participants were asked to comment upon the pros and cons of these two options, and the responses from this open question indicate that participants can see potential problems and benefits arising from both.

The main benefit identified for the assessment of risks and ethical issues along the entire value chain was that this allows for a comprehensive and “global” or “holistic” assessment. However, while this option was described as “an ideal”, in reality, this approach can be expensive and time consuming, slowing the pace of development.

The main identified benefits for assessment of ethical and societal risks at an early stage were that it is much less time consuming, less expensive, and that it allows for changes to be made at an early stage before too much investment has been made. However, a number of participants were concerned that assessment at an early stage is not sufficient to detect all risks; there may be unforeseen issues that were not imagined, especially once products and services are actually being used by the end-users in their own homes. As one participants expressed:

“I disagree with this approach—no amount of early planning can prepare the delivery organisations for the perception issues they must deal with when going into the market.”

(D13)

4.3.3. Who Should Address the Ethical and Societal Risks?

In Delphi stage 1, participants were asked which departments/people within a company should be responsible for assessment of ethical and societal risks. The responses indicate that central management were considered the best placed for addressing the risks related to R&I by the majority of respondents (see Figure 4). Research and development (50%), the CSR (38%), and the legal departments (29%) were the other most voted for options.

In Delphi stage 2, participants were asked to consider this issue further and to indicate what they believed should be the main tasks for the four most voted-for departments in addressing ethical and societal risks. In addition, participants were asked to consider whether there should be any distinctions made between SMEs and large enterprises in relation to these tasks.

In their responses, there was general agreement that the legal departments provide the framework for ensuring compliance with relevant regulations, and that R&D staff perform ethical and societal risk assessment for new applications, identify solutions that monitor/limit risks, and conduct tests of new products with end-users. The management is believed to have a key role in compliance with RRI principles in the following ways: raise awareness and set the vision, ensure commitment/accountability of the organisation, adopt risk governance tools, and create an ethical culture amongst employees. If there is management “buy in”, then the other departments will follow.

Several respondents described the need for all departments to work together in the evaluation of ethical and societal risks. This may be challenging in practice, so proactive mechanisms need to be put in place that allow this to happen. Interestingly, there were no indications from the participants that tasks should be any different for those in SMEs and those in large enterprises.

When asked about who should be involved in the identification and evaluation of the potential ethical and societal risks from outside of the organisations, participants in Delphi stage 1 voted as shown in Figure 5. The most commonly selected were Ethics Committees (79.4%), Civil Society Organisations (53%), Consumer Organisations (50%), and Research Organisations (50%).

Questioning on this topic probed deeper in Delphi stage 2, as participants were asked to consider exactly who should be involved at each of the risk assessment stages. The results from this (shown in Figure 6) suggest that research organisations are thought to be the most important for hazard identification; civil society organisations, for making decisions about who might be harmed and why; policy makers and research organisations are equally important for evaluating risks and deciding on precautions; and ethics committees are thought to be most important for regular review and update of risk assessment procedures.

4.3.4. How to Address the Risks

In Delphi stage 1, participants were asked for their opinions about procedures that should be used to identify and evaluate societal risks and ethical aspects of research and innovation in advance of development. The following four factors achieved the most consensus:

• Ethical assessment (78%)
• Risk assessment procedures (71%)
• Pilot studies for evaluating different scenarios (44%)
• List of moral values (44%).

When presented with these findings, the majority of respondents in Delphi 2 agreed that these approaches would be adequate, with only one person asserting that they would be insufficient, commenting:

“I believe the failures are typically in the execution of the methodologies, rather than in the methodologies themselves.”

(D2)

As previously described, respondents in Delphi stage 1 were in agreement that all technologies used for the development of products, systems and services for an ageing society, are risky from an ethical and/or societal point of view. From the results of Delphi stage 2, it is clear that the majority (80%) of the respondents also agreed that the current regulatory framework is not adequate for addressing these risks. Only 7% of the respondents were of the opinion that the current regulatory framework is adequate for addressing ethical and social risks in the design and development of ICT products, systems and services; the remaining 13% being in the “don’t know” category.

When asked to identify shortcomings in current regulatory framework from a range of possible factors, the following four concerns achieved the greatest consensus:

• Need for rules on personal data protection that are valid for all companies, regardless of their establishment, inside or outside Europe (75%)
• Need for a single pan-European regulation for personal data protection (67%)
• Insufficient in terms of personal data protection (58%)
• Insufficient in terms of specific legislation on e-Health, including mobile health practices (50%)

In consideration of the domains most susceptible to ethical and societal risks in the design and development of ICT products/systems/services for an ageing society that had been identified in Delphi stage 1, participants ranked a range of suggested initiatives/measures for suitability in addressing the risks in these domains, with results as shown in Figure 7.

Some respondents also made suggestions for other initiatives that could help to address risks. These were primarily to do with the need for clearer legislation for data protection issues, such as the nature of third parties, the full and informed consent of the provider of the data, the reasons and scope of the data gathering, and attention in the employment of company personnel. Additionally, it was felt that users should have the easy ability to forbid intrusions categorically.

The final question in Delphi stage 1 that concerned risk, asked the participants how they would involve the general public and other relevant stakeholders to identify and evaluate risk factors. The most agreed upon tools for engagement of the public were the formation of networks (64%), and the organisation of focus groups (64%) and workshops (51%).

The particular design methods believed to be most helpful for assurance of ethically acceptable and socially desirable ICT products/systems were considered to be participatory design [59] (60%), and human centred design [60] (60%), followed by prototype trial (33%), and human driven design [61] (30%).

In Delphi stage 2, respondents made further suggestions about how to improve societal engagement and public trust in ICT products for an ageing society that included greater involvement of care delivery organisations, the use of printed media (believed to be more acceptable to the elderly), showroom and public demonstrations, and articles in magazines and on television.

4.3.5. A Culture of Responsible Research and Innovation

Aside from the particulars of the what, when, who, and how of risk assessment, the results from the Delphi study convey a more general message about the perceived need for RRI values to be embedded within the culture of an organisation, in particular, the need for raising awareness of ethical issues amongst all personnel. As one participant expressed:

“There is no real need for new tools or more bureaucracy, but, instead, for a change of thinking. RRI should be part of the company philosophy. The management should be responsible for propagating this philosophy throughout the company, creating a company culture in which every department and employee act responsibly.”

(D21)

The same participant pointed out that most of the workers in ICT companies have never had training in ethics, so it may be difficult for them to appreciate the problems with some of the consequences of their developments:

“Most people are not aware of the risks of new technologies, as, for example, seen by the imprudent use of social networks and cloud services. Therefore, we need more education and an open discussion of technology impacts and societal responsibility.”

(D21)

In Delphi stage 1, the need for ethics education was confirmed by the high percentage of respondents (87%) who agreed about the need to consider RRI issues as an aspect of continuous professional development for researchers and innovators, and the 70% who believed that RRI issues should be incorporated into training and education at all levels. In Delphi stage 2, there was 93% agreement about the need to promote a culture of social and ethical responsibility in R&I. However, there was a degree of scepticism about whether this is possible:

“I agree from a general point of view, but social aspects, ethics, and responsibility cannot be taught to adults with years of experience.”

(D11)

Others, it seems, were taking a longer-term view with recommendations that ethics should be a mandatory component of education and training right from school, through to university and beyond, and that an ethical culture should permeate industry.

5. Discussion

We began this investigation with the assumption that RRI can be viewed as a meta-responsibility [11], encompassing many lower level responsibilities, including risk assessment and management in its broadest sense. As an abstract and relatively new concept, it would be of limited value to seek experience and opinions of RRI from those working in industry. However, as a core component of RRI, investigation of practices and attitudes towards risk provide us with data that is rooted in real-world experience and that might offer us a window into the perceived relevance of RRI for those in our study.

The two methods of data collection in this study have revealed different types of information about attitudes towards risk, that, when viewed as a whole, help to reveal the complexity underpinning potential motivations for responsible innovation in the ICT industry.

In the interviews, participants were questioned in an open manner, allowing them the freedom to focus upon the issues that were of most relevance to them. Typically, in this type of interview situation, people speak most enthusiastically about the things they feel most strongly about [62]. Interestingly, even though the interviewees were asked directly about their procedures for assessment of risks, there is comparatively little acknowledgement, from some, of the potential risks associated with their R&I activities. It is clear that both attitudes towards, and awareness of risks, varied greatly. On the one hand, we have participants who report that there is no formal consideration, at all, of the consequences of their R&I activities, and on the other hand, we have those who describe well-developed systems and motivations. Additionally, we hear that risk assessment is a big problem for R&I in ICT because of the lack of relevant ethical guidelines for all new and emerging technologies.

In our analysis of the interview data, we noted a difference in understanding and interpretation of what “risk” means. It appears that “risk” is often perceived primarily as “economic risk” rather than, more broadly, as inclusive of ethical and societal risk factors. This is consistent with our interview category, “profit assurance”; for the individuals in this category, risk assessment is something that is undertaken primarily as a means of increasing product or service acceptability, thereby increasing potential for profit. The potential for enhanced economic benefits from effective stakeholder engagement in innovation is well accepted [63]. We infer from this, that some companies could be motivated to engage in RRI-related activities purely for pragmatic reasons, to limit or reduce the economic risks, or to attract investments, by improving the acceptability of their product/services. Certainly, the idea that engagement with responsible innovation might ultimately be of financial benefit has been postulated by others [64,65].

However, as previously stated, the concept of RRI goes way beyond the traditional understanding of governance based upon risk assessment [3]. The adoption of RRI additionally includes proactive consideration of “what kind of future we want innovation to bring into the world” [66] (p. 758). This conceptualisation is more aligned with those in our category of “profit plus”. For participants in this category, it appears that motivations included more virtuous/idealistic factors; they pay attention to the ethical and social implications associated with their developments for the good of humanity, society, or the environment. This does not preclude attention to economic factors but they do not constitute the primary or sole motivation. We surmise from our analysis that these companies are already more in tune with RRI values than those in the “profit assurance” category.

At first glance, this diversity in attitudes and opinions does not seem to be replicated in the Delphi study; when faced with a list of potential risks in the Delphi questionnaire clearly related to ethical and societal issues, all participants had opinions about risk. In the Delphi, there was a notable consensus that measures to limit ethical and societal risks must be taken by the enterprises both at technical level (limitation of data transmission, design of systems that avoid external intrusion, etc.) and at organisational level (careful selection and ethical training for the personnel who may access personal data). Moreover, it was acknowledged that the impact of ICT products and services on the users’ quality of life should be regularly assessed.

Disparity between interview and Delphi findings might have arisen because awareness or appreciation of the risks that are relevant for RRI was not at the forefront of consciousness for those in our interviews, while those in the Delphi were “forced” to consider these issues. As previously indicated in our introduction (Section 2.3), the use of ICTs has become so ubiquitous that many aspects are hardly ever questioned, even, it seems, by those who work with them.

However, it appears that, when laid out clearly for them, ICT industry representatives are able to recognise potential risks and express opinions about when, how, and by whom these should be addressed. Additionally, indications about how to proceed and who should be responsible for addressing ethical and societal risks have emerged quite clearly from the investigation.

This finding has important implications for those who wish to promote the uptake of RRI. The potential ethical and societal risks from innovation, and the potential benefits from RRI, are not going to be obvious to everyone who works in industry, even when they are asked to reflect upon them. Education may be necessary to improve awareness of ethical and societal issues, but even the use of a simple tool like our Delphi questionnaire might be effective in raising awareness and promoting broader reflection.

Notwithstanding the discrepancies between the two methods of data collection, the acknowledgement by industry of the importance of adopting a responsible approach has emerged quite clearly. This could be, at least in part, because the ICT sector chosen for the study combines essential aspects for RRI; the ICT industry is research and technology intensive and it looks for innovative and sometimes futuristic solutions that can raise significant social and ethical dilemmas that need to be addressed during product development [44,67]. In other sectors, that are not as close to the end-user, the perceived importance of ethical and societal risk assessment could be more limited. A clear perception of the issues at stake (people, values, principles) for the specific sector/application is essential for effective industry engagement with RRI.

Detailed and useful suggestions were made by the Delphi respondents about the tasks of the departments that should work together to address ethical and societal risks inside the enterprise. The most important role was assigned to the management who should create an “ethical culture” amongst the personnel and take responsibility for establishing the appropriate procedures for monitoring and dealing with societal and ethical risks, along (where possible) the entire value chain. The key role of leadership in this task was also highlighted by Campbell, in her investigation into whether effective risk management is a signal of virtue-based leadership. Campbell hypothesises that making decisions and taking actions to manage risk is a continuous process that requires moral discipline in looking to the interests of others and acting in service to those interests above self-interest [68]. Findings from her extensive investigation suggest that the ability of a nation’s government to manage risk is largely viewed as a leadership issue, more so than simply a governance or regulatory compliance issue. Whilst Campbell’s investigation was at a governmental level, there are parallels to be drawn at the organisational level where the role of the CEO is not to implement the individual risk management strategies, but rather to cultivate and support the development of a risk culture [69]. Our findings seem to support the notion that virtue-based leadership or management may facilitate the type of organisational culture in which RRI-related principles can flourish.

In our assessment of related factors, data from both the interviews and the Delphi study were examined for evidence of trends. Contrary to our expectations, we did not find clear links between attitudes towards ethical and societal risks and size of industry, although there is some evidence in the interviews that the larger and more organised companies are already using CSR tools and/or aligning with global initiatives and codes. There were, however, sufficient indications to suggest that associations may exist for two other factors:

(1)

Degree of innovation

The enterprises that develop products which are at the forefront of R&I are more responsive to ethical and societal issues.

(2)

Interface with end-users

Enterprises that develop products or services that involve direct interface with end-users are more likely to undertake activities for engagement/involvement of end-users to increase acceptability/desirability of the products/services.

Whilst we cannot assert that there is firm evidence for the above associations, they may be worthy of further investigation.

The interviews and the Delphi exercise focused on representatives of ICT industry for an ageing society. The risks and the regulatory framework in place are specific of this sector, but the highlighted motivations might also have some relevance for other fields. Further research in other fields would be needed to confirm this.

The number and selection criteria of our respondents mean that our findings cannot claim to be generalisable. To gain generalisable insights, a larger study or repeated studies would need to be undertaken to test the validity and reliability of our findings. Moreover, our investigation focused on ICT, and in particular, on the application of ICT to health, demographic change, and wellbeing. This application area is associated with particular ethical issues, and our findings may therefore be coloured by shared industry perceptions that may not be applicable in other parts of the industry (like hardware design, social networks, or gaming). Again, additional studies would be needed to answer these questions.

One further issue worthy of consideration is the potential for confusion or clashes between personal opinion and corporate policy on the topic of risk assessment or implementation of RRI. In our study, we specifically asked individuals about their own experiences and opinions. Our analysis, particularly of the interviews, reveals that personal opinion can sometimes be at odds with existing corporate practice. Individuals working in the field of ICT for an ageing society are regularly encountering and discussing ethical issues in their daily practice, and often working closely with end-users. This is likely to contribute to their opinions about the relevance of ethical and social impacts of their R&I activities, even beyond the general practices and policies of their company.

6. Conclusions

In this paper, we investigated the attitudes and opinions of individuals working in the ICT industry, describing their view of an essential component of RRI through the lens of risk. In order to be relevant and have wide scale impact, RRI needs to be adopted by industry. For our investigation, we selected the ICT industry because these technologies raise a number of concerns and risks that are difficult to address with established governance mechanisms. Through our empirical study, we generated a rich dataset of a qualitative and quantitative nature that provided insights into the way in which the participants perceive the risks of ICTs and how this might align with RRI.

The investigation has indicated that motivations for adopting RRI could be wide ranging, some more pragmatic and others more idealistic or virtuous. Both are important, but companies that are currently functioning on a purely pragmatic level cannot realise the full potential of RRI as a tool for imagining the kind of innovation that society wants and needs. Findings from the Delphi study suggest that for this to change, full promotion of an ethical culture and instilling of RRI values and principles throughout the company might be necessary. In this way, RRI values could become embedded in the governance of a company that, in turn, might result in improved integration of the aims of the company personnel with those of the corporate policy. Functioning at a meta-level, RRI could serve to ensure strategic alignment of all facets of responsibility for R&I. However, “buy in” of the RRI approach by the management of the company would be essential for this.

While our study leaves numerous avenues for further research, we are confident that it does make an important contribution to knowledge. We have shown that individuals working in the ICT industry are open to concepts related to responsible innovation and understand that it can be beneficial for them. Employing the theoretical lens of risk might, in particular, aid those companies that find the concept of RRI challenging to connect with. This could provide a means of engagement, through a language that is familiar to them, in a pragmatic “first step” approach.

This study also shows clearly that there are different attitudes to risk that might be associated with different views of responsibility. The notions of risk that emerged from analysis of our interview data reveal that economic considerations are fundamental for private companies, but, even so, for many companies, broader ethical and societal concerns are also of fundamental importance. This is of theoretical importance to the RRI discourse; the link between RRI and the various facets of risk may warrant further exploration to strengthen the concept overall.

We do not claim that approaching industry through the terminology of risk is either the only or the best way of communicating about RRI. What we have shown, however, is that it may be a suitable way to do so and one that can open doors to further communication. Of course, RRI is not only a risk management tool, and governance of modern research and innovation processes is much too complex to be reduced to one aspect. However, having demonstrated the relevance of risk in this context, we hope to contribute to the broadening of the discussion of RRI, and, by implication, to the aim of ensuring that processes and products of research and innovation will be increasingly acceptable, desirable, and sustainable.