Responsible Research and Innovation: Using the Requirements Tool for Stakeholder Engagement in Developing a Universal Design for Learning Guidelines for Practice

: Responsible research and innovation (RRI) is growing in importance, and alongside this growth is an acknowledgement that for research and innovation projects to be successful, stakeholders must be involved from the outset. When developing guidelines for practice, stakeholders will often be presented with a document to ratify rather than one to develop or revise. This gap in stakeholder engagement has been recognised and addressed by the development of the requirements tool. This tool was originally created to provide a systematic approach to the development of guidelines for the governance of RRI, but it was quickly recognised that the tool can bridge the gap and involve stakeholders from the outset, thereby increasing the likelihood of buy-in. This paper presents the second validated use of the tool that was used to inform the revision of guidelines for the introduction of a universal design for learning (UDL) at a UK University. The resulting revised guidelines for practice and their adoption by those tasked with producing them provide further evidence of the value and ﬂexibility of the tool and its potential for its continued use in the future development or revision of guidelines.


Introduction
The concept of responsible research and innovation (RRI) as a discrete approach to research and innovation is relatively recent but has its roots in early 20th century discussions about 'value free' science and its dominance of social science thinking through the ideas of Max Weber. The idea that science, and social sciences in particular, should be conducted without a value judgements, and pay no concern to the potential consequences or outcomes, in the search of quantifiable truths. The potential dangerous and damaging nature of ethically neutral science [1,2], and its detachment from its stakeholders, has long been challenged [3][4][5] alongside concerns about ethics and the importance of involvement in the decision-making of those likely to be affected.
This has led to a relatively recent acknowledgement within the science community that 'The approach used by normal science to manage complex social and biophysical systems as if they were simple scientific exercises has brought us to our present mixture of intellectual triumph and socio-ecological peril' [6] and so science and innovation actors can no longer avoid responsibility for their activities as being necessary for science to be considered rigorous and therefore valid [7][8][9][10]. As scientific endeavours reveal ever greater complexity and interconnected dependencies, not including context, engaging with stakeholders, whether human or otherwise, and involving other external actors where possible, is actually irresponsible, unsustainable, and likely to result in poor science. This De Montfort University (DMU) introduced the principles of universal design for learning into their teaching and learning activities in 2016 [39]. The initiative aimed at addressing some of the funding gaps created when the British government [40] removed disability funding. This initiative also aimed to provide a more inclusive and flexible approach to teaching and learning (universal design for learning, UDL) for students [41]. As part of the project, a set of guides were produced to help lecturers and those developing curricula to embed the principles of UDL into their activities and approach. These guides were produced with no stakeholder involvement and were not widely accepted as having relevance to particular subject areas by staff. The requirements tool that had been used in the project 'Governance for Responsible Innovation' (GREAT) to create guidelines was utilised to revise the guides for UDL practice so that they were designed by those asked to use them, with direct relevance to their own particular area of expertise.
Applying the requirements tool within this organisational change context would also provide evidence of the flexibility and potential for further practical application of the requirements tool across different sectors and in different contexts.
In the following sections, I will first present the underlying philosophy of RRI adopted by the GREAT project, notwithstanding the historical and complimentary aspects of Science and Technology Studies, Corporate Social Responsibility and the social license to operate, highlighted above. It explains how understanding of the literature underpinned the approach to the development of the requirements tool and, subsequently, the guidelines for RRI. Next, the requirements gap is discussed, which explains how the lack of engagement early in the process of change may lead to ineffective or subsequent failure of new initiatives, and how identification of this gap highlighted that, for an RRI project to utilise RRI in practice, a solution to this gap needed to be developed. This paper then goes on to explain the process of development and the outcomes of the resulting guidelines being used in practice. The next section explains the rationale for utilising the requirements tool as part of a review of existing guidelines within the context of organisational change and the introduction of the universal design for learning at a UK University. Details of the process and use of the tool are provided, and the outcomes presented. Finally, there is a discussion of the future potential to embed RRI into organisational change initiatives and how the use of the requirements tool can help in achieving UN sustainable development goals 16 and 17 [23,24] and to increase the likelihood of successful adoption of new innovations and initiatives in industry, education, and beyond, through greater and earlier stakeholder engagement.
In order to understand the philosophical and theoretical landscape of RRI and the discourse between science and society the GREAT project undertook a Horizon scanning literature review in 2013 [42], alongside a further review of the theory and philosophical underpinnings of RRI thinking through to 2016 [43], to paint a theoretical landscape of RRI. Through this work, five key ingredients of RRI, which were largely inclusive of RRI thinking at the time, were identified: • Anticipation-which tries to predict societal impact from technological development, • Transparency-to guarantee openness to include dissemination and discussion of all outcomes. • Responsiveness-to be willing and able to act to adapt and revise practice, procedure and behaviour as circumstances and needs change.

•
Reflexivity-Two orders of reflexivity include considering how far something could be adapted or changed to improve outcomes and to acknowledge how researchers are framing their work and the assumptions and unconscious bias that may affect their decisions.

•
Participation-ensures that all stakeholders should be engaged in meaningful involvement from the outset and throughout a project.
This work informed the entire philosophical approach for the project and, by extension, the process by which the identification of the requirements for RRI guidelines would be developed. To produce the guidelines themselves, however, involved further understanding of how they could, alongside other training and development activities, lead to new and more responsible behaviours becoming normalized in practice.
It is clearly important that any new set of guidelines, rules, or new procedures should be informed by the context in which they would be applied [44]. In this way, a transition is more likely to be achieved so that new norms and ways of working become tacitly embedded into everyday practice and are less likely to be subject to resistance by workers. As part of this approach, [45] explains the need for reflexivity and for 'doing' RRI; the author explains that 'engagement with ethical questions will require the development of reflective processes within research, so that norms, their context and application can be understood, predicted and influenced'. Further, perceptions of effective governance identify a need to resolve the conflicts between 'scientific accuracy, policy effectiveness and political legitimacy' [46] in order to be successfully embedded in everyday working, but fail to recognise the requirement for acceptance and user adoption of any new initiatives. For effective governance in any sphere, it seems only sensible to include those being governed, and expected to adopt organisational change in their daily lives, in the entire development process. Whilst there is some reluctance to adopt RRI in healthcare due to concerns about patient involvement [47], there is some evidence of its adoption in science education [37,48].
RRI requires that there is due consideration of the impact of organisational change on individuals and smaller groups, such as those within and between those organisations or teams that are at the forefront of such change [49] The introduction of new processes and procedures, therefore, can have a huge impact on those both directly and indirectly affected by them and if they are done without due diligence, involvement, and reflection [50,51]; then, these processes and procedures are unlikely to be fully effective, or may result in unexpected negative outcomes.

The Requirements Gap
When creating guidelines for any purpose, where there is stakeholder engagement, it invariably involves the inclusion of those stakeholders at some point. The necessity to include stakeholders in decision-making is well-established [49,50,52,53]. The inclusion of stakeholders into the analysis of the impact of industry in local areas [54] and the need to involve patients in decision-making [55] provide good examples of where stakeholder engagement is both important and necessary. However, particularly where there are internal organisational changes or new innovations, it can be the case that new initiatives, such as guidelines for practice, do not get to the stakeholder engagement phase until they have already been developed. Unless there are serious objections, it is expected at this stage to only require tweaks and minor changes before the new initiative can be rolled out to all. The stakeholders in this case are rarely involved in the creation of the original guidelines and are highly unlikely to be asked what they would like to see in the guidelines from the outset, or how they could be made to be useful and relevant.
This gap means that the stakeholder requirements for those guidelines have already been decided upon by those creating them, thereby severely limiting the options stakeholders have to shape those guidelines in such a way that they are fit for purpose from the outset [56]. This lack of stakeholder consultation and inclusion at the design phase of guidelines creation means that it is then much more difficult for organisations to know if what is being provided to stakeholders [22] and if it is what they would have wanted or found helpful, had they been asked in the first place. It is understood that when there is limited time or existing documents, then stakeholders may be presented with a draft, but the key to addressing the gap is to provide the opportunity for stakeholders to rip it up and start again.

The Requirements Tool
To address this engagement gap, the requirements tool was developed (see Appendix A) and subsequently validated in practice to enable RRI principles to be included in developing guidelines and recommendations [50,56]. The tool was developed to identify what the stakeholder's requirements were, before going on to develop the guidelines themselves. Utilising RRI principles in the development of the tool itself, through stakeholder engagement from the outset, meant that the participants were involved at all stages of the development of the requirements tool, which in-turn led to the identification of the requirements for the resulting guidelines for RRI [57].
The requirements tool was developed during an EU project tasked with understanding and driving the governance of RRI, which included the production of a set of guidelines for practice informed by a strong theoretical base [43]. The aim was to 'develop an empirically based and theoretically sound model of the role of responsible research and innovation governance' [38] and bring about new and more responsible norms of behaviour in research and innovation and to provide researchers with guidelines to help them to implement RRI into their working practices.
The guidelines aimed to clarify and provide practical answers to RRI that could then be incorporated into both the process and outcomes of research and innovation projects. However, once the team started to directly engage with the process of developing guidelines, utilising a 'practice what you preach' RRI approach, it soon became apparent that there was a disconnection between those being provided with guidelines and the production of those guidelines. A key element of RRI is engagement [21,58], so those most likely to be expected to use the guidelines were invited to provide input into what kind of things should and should not be included, what format would be most helpful, and so on. The process of engagement to discover the requirements for guidelines led to the development of the requirements tool [59]. The guidelines for RRI resulting from the use of the tool have successfully been applied in several key EU projects [60,61] and are further presented in use within the context of organisational change.
The tool, in the form of a table, works by asking questions regarding general requirements, content specific requirements, those specific to the project, areas of enquiry or stakeholder groups, and issues and concerns that need to be addressed prior to the development of guidelines. In this way, the tool provides a reflective, robust and flexible approach to the development of guidelines or to changing processes within an organisation. The expectation of reflexivity that is built into the tool helps people to understand how guidelines for a research or innovation project or a change to working practices should be structured and what those guidelines should contain to achieve desirable outcomes for that particular project or initiative.
It was recognised quite early on in the development process, and from its application in practice, that the requirements tool could be useful, not only for RRI guidelines, but also for stakeholder engagement activities across all sectors and disciplines. At a minimum, however, the requirements tool in practice led to an understanding that when a set of guidelines are required, the discipline in which those guidelines are used, is of less importance than the effectiveness and perceived usefulness of the guidelines themselves [56,57,62].

A Guide for Universal Design for Learning
The opportunity to test the tool and its flexibility to apply it in a different context came about during the implementation of a University-wide project to embed universal design for learning (UDL) principles into teaching and learning practice. UDL is a flexible approach that encourages universal access to resources, assessment and services which encompasses the needs of all students, including those with disabilities, and supports the student's learning throughout their course [63]. The aim is to level opportunities rather than merely providing specialist provision [41,64].
The University decided to adopt a new approach to teaching and learning in order to better support all their students. As a potentially far reaching and innovative organisational change to teaching and learning practice, the University was keen to ensure that all staff engaged with the process of that change [65].
UDL Champions, including the author of this paper, were seconded from each faculty to support staff in implementing the new requirements, and a series of workshops and training sessions introduced staff to the key aspects of UDL and how it would impact on their curriculum development. Examples included recording lectures, providing materials in a range of different formats, devising flexible, student centred assessments, and so on. In addition, as part of the process of embedding UDL practices into curriculum development within the University and the provision of learning opportunities for students, a set of guidelines including advice on how to apply them, were developed and distributed to staff.
The faculty is diverse and wide-ranging with three schools offering different teaching and learning experiences for their students from journalism to cyber-security, computer science to mechanical engineering. The main rationale for introducing UDL across the whole University was to widen student access to teaching and learning tools and resources, and thereby improve outcomes for all students [66]. The UDL approach is flexible and inclusive enough to accommodate all learning styles and learning differences by removing barriers rather than providing alternative provision [67]. Therefore, it was anticipated that this would improve student outcomes.
As the UDL champion for the faculty of Technology, the author was required to undertake a number of training and development workshops to promote UDL and to help people include the principles into their teaching practice. This process also provided an opportunity to informally gain feedback on the guidelines that had already been developed. Although largely anecdotal, it quickly became apparent that the guides were only generally useful and lacked direct discipline specific relevance. Further, it was also apparent that a failure to engage with the guides that were provided to facilitate this organisational transformation could mean that it would be more difficult to achieve the desired outcomes of embedding UDL into everyday teaching and learning practice. In light of the feedback received therefore, permission was gained to re-shape and tailor the guidelines to each of the schools within the faculty.
This also provided an opportunity to test the flexibility of the requirements tool to improve an existing set of guidelines, (as previously it had only been used to identify requirements for a new set of guidelines), and to utilise an RRI approach in the process by engaging with the stakeholders in a co-creation exercise from the outset.
The task would result in either a complete re-making or at least a revision of the guides for UDL that focus on the specific needs of each of the schools. An RRI approach was taken from the outset, and the AREA framework was adopted to systematically ensure that an ethos of RRI was embedded into the project throughout.

Identifying and Engaging with Participants
The target group for participants in the project were those within each school who were in positions that not only would require them to undertake curriculum development activities themselves, but also might be supporting other staff in their teaching development. Within each of the three schools these participants are further sub-divided into four subject groups, so an email was sent to each subject group leader asking for nominations of 1-2 people to be invited to be involved in the project. From this process of nomination and invitation, 6 participants volunteered from two of the schools, and 8 from the third, one of whom left the project after one meeting.
Participants comprised: It was fortunate that there was a good range of staff members across each of the schools, as well as a mix of senior and junior staff, included in the project. All participants were directly affected by the requirements of UDL and would need to use the guidelines. All of the volunteer staff had undergone some UDL training and were keen to utilise the principles in their teaching and learning activities, hence their agreement to be involved. This meant that these stakeholders were particularly well-placed to engage with and value the aims of the project.

Workshops
The workshops were designed to provide an opportunity for affected staff to make the decisions about what should be included in the revised guidelines for UDL in each school in the faculty. The stakeholders were, therefore, engaged with the process and were informed from the outset that, if they wished, they could completely reject the guidelines and work to create a new set of guidelines. The participants were all happy to revise rather than start from scratch. It may be that the prospect of creating a new set of guidelines would have involved more time and commitment than they were able to commit; thus, the existing guidelines were to be revised rather than replaced, but that was the decision of the participants.
Thematic analysis was used to identify key themes and issues, and where individuals gave feedback or provided perspectives that the other group members agreed with, those themes were also taken into account. The results of the analysis were then used to firstly adapt the requirements tool to a project specific focus, and then to use the tool to analyse and identify changes in the guidelines.

Workshop 1-Adapting the Requirements Tool
The first workshop included all of the volunteers for the project as the aim was to adapt the requirements tool to replace RRI with UDL approaches, which would then be used to guide the re-creation of the guidelines that were school specific in a subsequent workshop.
When validating a tool, it would normally be presented ready for use by stakeholders. However, the core principle of the tool, its development, and its validity is the engagement process being at the outset. So, involvement of stakeholders in framing and adapting the requirements tool is the first step in its use to bridge the requirements gap. This meant that when using the revised requirements tool, participants were already invested in its usefulness to revise the guidelines.
Each participant was sent a copy of the original requirements tool two weeks before the workshop with information regarding how it had been developed and used in the GREAT project. They were also provided with information about the aims of the workshop and advised about how they could reflect upon the tool and how it could be adapted so that it could be used to re-develop the guidelines for UDL. Copies of the existing guides for UDL were also sent to all participants. Consent forms were also provided for participants to sign before the workshop.
The workshop specifically intended to make direct changes to the tool, not just to gain feedback so that existing tool was re-purposed to aid in the revision of the guidelines for UDL. At the start of the workshop, the organiser explained the purpose of the tool and asked each participant to briefly give their own perspectives on the tool and the UDL guides. Then, in informal, tabled groups, they were asked to further reflect on, discuss, and review the existing tool, and to identify changes that would help guide them towards school specific revisions. After 20 min of group discussion, each group was asked to present their conclusions. This process was followed by a full group discussion for 10 min, where the changes to the tool were agreed upon.
The resulting revisions were sent to participants, resulting in the revised requirements tool (Table 1 below), which would then be used in the project to guide the revision of the UDL guidelines. The aim of each of the school specific workshops was to identify the changes that were required to make the guides more effective by using the UDL requirements tool to guide the decision-making process. The tool enabled a systematic and focused approach to identify what was needed and how it should be presented. In all workshops, the original look and feel of the guides was retained, as they were felt to be easy to understand; the guides contained a checklist and other easy to use sections.
The main complaint from the participants was the generic nature of the content and the lack of consultation and involvement in their creation.
During the sessions, the participants emphasised the importance of incorporating teaching and learning practices of the specific subject area into the guidelines for UDL. In addition, within the generic guides, some explanations were provided, but it was felt that they needed improvement.
The workshops for revisions of the guidelines involved a smaller number of participants, as these were done with just the members of each school. The participants were asked to provide their initial perspectives before being split into two groups to consider how to revise the guides. The requirements tool was used to guide and structure their deliberations and to help them to provide concrete changes. The resulting requirements from the participants were written on flip-charts. After 15 min, the charts were exchanged and the perspectives re-examined, revised, and added to. Finally, the whole group discussed and reflected on the ideas presented, to identify gaps and duplications, as well as any ideas that were subsequently rejected or retained.
During this discussion, it was also recognised that often the teaching staff were already including UDL into their curriculum development, but were not always able to identify it as such. By making the UDL guides and their accompanying notes specific to the subject areas, it would become much easier for staff to identify what was meant by the term UDL when applied to their experiences of teaching engineering, journalism, or computing, and to recognise that what they were doing was already incorporating a UDL approach.

Review and Inclusion of New Requirements into Existing Guidelines
Discussion regarding the original guides strongly indicated that whilst the guides were easy to follow due to their relative simplicity, they were also highly generic, overly prescriptive, and in some cases, not relevant, which meant that they lacked guidance and were more useful as a very basic checklist.
The researcher then collected all of the notes and flip charts from the sessions and converted then to a word document. This document was then analysed to identify the ideas, requirements, and perspectives of the participants. These findings then informed the changes to the guides to make them more specific to the subject area in which they were to be used.
The revised guides for each school had a cleaner, simpler format. Additional information and access to resources and links were now provided in a separate guidance notes document. Participants had expressed concern about the examples provided in the guides that were unclear, not relevant to them, or overly prescriptive. Stripping out much of this problematic material to present the content in a more detailed information document which included links to further resources meant that the guides themselves could act as tools to support incorporating UDL, and the guidance notes provided helpful examples and advice. This document included many of the more detailed requirements identified through the use of the requirements tool, such as links to further information and sources of other UDL tools, and techniques, all of which meant that the new set of documents retained their original simplicity, whilst providing access to greater detail if required.
The resulting guides and explanatory notes were then sent to the workshop participants for further review and feedback. This led to some minor revisions and the addition of case study examples to the supporting document. After the second round of changes were made, no further changes were requested by the participants, and approval for the final versions was obtained. The resulting guidelines and explanatory notes were then sent to each head of school to be distributed to all staff involved in curriculum, module, or programme planning to aid their future adoption of UDL in their teaching and learning.
Feedback from the participants regarding the process and outcomes indicated that not only were the guides now perceived to be much more useful, particularly for those who were new to curriculum development, but participants indicated they had found both the process and the tool to be helpful to enable them to focus their activities and to feel genuinely part of the process.
Whilst it is clear that when undergoing change, the inclusion of stakeholders is important, within an academic environment, where there is a high level of expertise and professional integrity, it can be argued that managers and professors are not always utilising the existing resources at their disposal. The process of revising the guidelines and the subsequent feedback from participants illuminated the value of both the tool and the engagement process.

The Requirements Tool in Practice
The requirements tool has now been used in two practical examples. In the first, a new set of guidelines for RRI was developed. The requirements tool was devised to close the stakeholder engagement gap in an RRI project, by helping to identify what those who would use a set of guidelines for the governance of RRI would need in those guidelines in order for them to be effective. The resulting guidelines from that process have been successfully adopted within a number of RRI related projects. The second practical example of the use of the requirements tool is provided by the UDL case, whereby the requirements tool demonstrated its flexibility. The tool was adapted according to the area in which it was to be applied, i.e., UDL, and then used to guide the next step, involving the revision and tailoring of an existing set of guides to make them more appropriate to those to which they would be applied [56]. The common factor in the successful development of the guides is the bridging of the stakeholder requirements gap.
That the speed that the tool can be adopted and adapted for specific needs makes it highly usable, particularly if guidelines are already in existence, and revision is needed urgently to avoid disengagement by stakeholders, if, for example, there is disillusionment about the first set of guidelines. The tool, because its core parameters are already in place, can be adapted quickly, with the insertion of indicative words in key areas as a starting point, to be then adapted further if required during the initial stakeholder engagement process.
While there is still significant work to do to further validate the requirements tool, the evidence is growing that this tool can be useful to bridge the stakeholder requirements gap, something that has been largely neglected when developing guidelines for practice and when undertaking organisational or social change. In fact, it could be surmised that this gap has not been previously identified in the literature. The adaptation and adoption of the tool in research and innovation and within industry would strengthen engagement by involving stakeholders much earlier in the research process. For example, in industry, to obtain a 'social licence to operate' [26], or to undertake mining operations, local communities would be engaged at the initial concept stage so that the most important concerns and assurances can be identified and addressed earlier. In this way, communities are able to directly influence local industrial development thereby minimising opposition and building relationships [27] Within research and innovation projects, the wants, needs, and concerns of stakeholders can be better understood [22,36,50,54] rather than assumed or imposed, thereby avoiding the development of inappropriate or useless artefacts, imposing unworkable policy, or devising unsatisfactory working practices.
The experience of revising the guides for UDL also highlighted further uses for the tool in an educational environment. Student engagement, experience, and active involvement in the development of their courses is growing, with feedback at all levels being elicited from students, who in turn inform curriculum development. The flexibility of the requirements tool means that it could also be used to engage with students so that courses are developed that reflect their needs or changes in learning styles, whilst at the same time enabling students to know that their input is having a direct impact on the way that their courses are developed.
One of the core principles of RRI is reflexivity, or the ability to consider not only the context in which a decision is made, but also the pre-conceived ideas, world view, and personal context of the decision-makers. The requirements tool provides an opportunity for open dialogue, with the objectives clearly defined but is flexible and adaptable to any situation requiring change. By providing the initial topics for reflection and discussion, the tool avoids some of the potential conflicts that may arise. The likelihood of success is dependent on the buy-in of those tasked to implement or who will be affected by it, and the progress is being aimed globally, with initiatives such as the UN sustainable development goals, proposing early and democratic involvement of stakeholders [18], and the requirements tool provides a first step in linking the needs of people to the needs of future research, innovation, and development.
However, sometimes there is a perception from management and those in disciplines where 'value free' approaches remain dominant [4], that issues of concern raised are actually just pushing back against change or progress. The requirements tool enables important discussions around management of change or the conduct of research and innovation to be had, and to turn the process into one that involves stakeholders being part of the decision-making process [12] and avoids the formation of negative perceptions. The reflective and transparent nature of the tool also increases the likelihood of the adoption of new approaches, guidelines, and practices by those who have been involved in their development.
There is a push towards introducing and developing RRI and better governance of science [68] across all sectors, from industry to academia, and from the public to the private. The development of tools to aid organisations across disciplines to adopt RRI in ways that provide positive outcomes is a valuable aim. For example, researchers [16] have developed a maturity model that helps industries to assess their readiness and ability to incorporate RRI into their Research and innovation processes, and thereby provide insight and goals for industry. The requirements tool is still in its infancy with regards to its use in practice, and so taking opportunities to test its practical application and flexibility will benefit not only the stakeholders engaged in guidelines development but will also add to the ever growing toolkit towards the adoption of RRI. In order to develop an effective tool, it was important initially to examine how guidelines were being created elsewhere, and to evaluate the effectiveness of their approach.
In the first instance, therefore, all of the publicly available documentation from two large multi-disciplinary projects that had included the development of guidelines in their deliverables, were examined in detail. The projects were selected due to their direct relevance to both RRI and the production of guidelines, and represented key projects from two research funding bodies. CONSIDER (Civil Society Organisations in Designing Research Governance) was a European Commission (EC) FP7 project that had created a set of stakeholder specific RRI guidelines for engagement with Civil Society organizations. Framework for Responsible Research and Innovation in Information and Communication Technologies (FRRICT), was a project funded by the EPSRC (Engineering and Physical Sciences Research Council), which had created a framework and tools for RRI in Information and communication technologies (ICT). Through the review of the approaches taken by these projects, it was possible to identify 14 outline proposals for requirements for guidelines that would act as the starting point for discussion with the stakeholders.
Academics and researchers from a range of disciplines were invited to attend a workshop for the purposes of understanding and identifying the requirements for guidelines. The engagement process at the very beginning of the process was intended to help to bridge the requirements gap identified above. The process enabled the researchers to understand the particular context and needs to produce a set of guidelines, and the participants were able to shape the nature and scope of guidelines by identifying their own requirements from the outset.
The process also enabled both the participants and researchers to reflect on what was being created and the rationale for its creation, as well as being part of the creation process.
The workshop focused on problem-centred group discussions moderated by the researcher [62]. The discussion centred on reflecting on the initial set of 14 outline proposals, to provide their views and input. The participants were encouraged to be highly critical and reflexive throughout, to consider how their context specific needs could best be represented in a set of requirements at the same time as being asked to 'think about their own ethical, political or social assumptions underlying and shaping their roles and responsibilities in research and innovation as well as in public dialogue' [43], in a process involving two orders of reflexivity.
This meant that one of the most active discussions was around perceptions about having guidelines for RRI governance at all, and the need and likelihood of acceptance and adoption in a broader sense. This discussion highlighted a generally dismissive approach to the idea of guidelines in any form. It was acknowledged by the participants that this perception of guidelines was in-part due to previous experiences of guidelines being imposed with little or no consultation, which merely served to highlight the need to take a different approach. A more detailed description of the methodology used in the development of the requirements tool can be found in [50,56,59].
The table below provided the identification of requirements that enabled the development of guidelines for the governance of RRI [69]. These guidelines were validated by their adoption in the EU Horizon2020 project 'Progressive Standards in ICT for active and healthy ageing' (PROGRESSIVE) as its internal RRI guidelines document for its activities. Use a common language that overlaps all disciplines.
May require use of some discipline specific language (Glossary D2.1) for clarity and precision.
Over simplification or too much complexity in the discipline specific language may result confusion or the evasion of the 'spirit' of RRI in practice.

2
Be concise and ensure it is practical and usable (bullet points etc.) Shorter documents are more likely to be read and understood.
Some projects, areas of enquiry or stakeholder groups may need detailed descriptions to ensure value and relevance.
If the guidelines are too brief they may lead to lack of clarity and detail.

3
Use good style to enhance readability (colours, diagrams, pictures other types of media). Make it attractive and easy to understand.
Could trivialise or over complicate the message. To inform from the specific project, area of enquiry or stakeholder group as well as more broadly. Provide access to multiple sources and perspectives and provide a contextualised approach.
Information overload and accessibility issues.

5
Provide a pitch to grab attention for example, a cover page with the key points.
Graphics and wording to aim at specific target audience Tone may be inappropriate, over simplification 6 Provide a single definition of RRI and other key terms and provide a description of meaning, scope, and complexity within the document through links to the Glossary (D2.1).
May be too narrow, overly prescriptive, or irrelevant in stakeholders' particular contexts 7 Provide links to different definitions of RRI to broaden awareness of RRI principles and to encourage the use of RRI theory to relate to user's own practice.
Perspectives included in definitions may be limited or conversely, information overload may create confusion or possible narrowing of perspectives. Case studies may be limited or not sufficiently relevant.

9
Provide links to RRI and governance tools to provide information and explanations on existing RRI approaches and resources including web links and references. This will provide examples for discussion leading to organisational/individual learning.
Facilitate awareness and understanding of domain specific approaches e.g., FRRIICT strengthens perception of the legitimacy of the guidelines in ICT.
Tools may not be useful or relevant. May be confusing if too many options are available. Existing approaches may be limited, too complex or unsuitable 10 Provide methods to re-asses and challenge the guidelines including a regular review.
If too flexible, may have little impact as the guidelines may be too weak to be effective.

11
Respond to EC framework, e.g., intervention logic model (relevance, effectiveness, efficiency, utility) and relate the benefits and problems of RRI to EC framework.
Identify and respond to aspects of EC framework that are specific to project, area of enquiry, stakeholder group.
May be too prescriptive. May not be accepted in other geographical regions. May narrow scope and constrain innovation or may create confusion due to contradictions. May lead to superficial or 'tick-box' compliance.

12
Utilise the findings from the GREAT project empirical work to inform development of the guidelines. This will not be included in the guidelines themselves but a link to the project website will be provided.
May be too narrow in scope.

Appendix A.2. The Requirements Tool-Requirements for Guidelines for RRI
A tool such as this can be limited in its usefulness if it is unable to be effective beyond its original purpose. Validation of the guidelines themselves was enabled when they were adopted within another EU project (PROGRESSIVE) to guide its internal research processes. However, the requirements tool was not validated directly in this project. The built-in flexibility of the requirements tool, however, meant that it was designed to be suitable for use in different projects and contexts and its successful adoption and validation in a different context would indicate its usefulness for the future development of guidelines within different sectors.