Co-producing Progression Criteria for Feasibility Studies: A Partnership between Patient Contributors, Clinicians and Researchers

There is a lack of guidance for developing progression criteria (PC) within feasibility studies. We describe a process for co-producing PC for an ongoing feasibility study. Patient contributors, clinicians and researchers participated in discussions facilitated using the modified Nominal Group Technique (NGT). Stage one involved individual discussion groups used to develop and rank PC for aspects of the trial key to feasibility. A second stage involving representatives from each of the individual groups then discussed and ranked these PC. The highest ranking PC became the criteria used. At each stage all members were provided with a brief education session to aid understanding and decision-making. Fifty members (15 (29%) patients, 13 (25%) researchers and 24 (46%) clinicians) were involved in eight initial groups, and eight (two (25%) patients, five (62%) clinicians, one (13%) researcher) in one final group. PC relating to eligibility, recruitment, intervention and outcome acceptability and loss to follow-up were co-produced. Groups highlighted numerous means of adapting intervention and trial procedures should ‘change’ criteria be met. Modified NGT enabled the equal inclusion of patients, clinician and researcher in the co-production of PC. The structure and processes provided a transparent mechanism for setting PC that could be replicated in other feasibility studies.


Introduction
Complex health interventions involve multiple context-dependent and inter-acting components [1,2]. Consequently, they can be challenging to evaluate and implement. Feasibility studies explore the viability of a definitive randomised controlled trial (RCT) for complex interventions [2,3]. These equal participation of all group members, promoting the sharing of power and the ability for all to take a key role [14,[21][22][23].
The method can be adapted for a range of purposes [22,[24][25][26]. We introduced a two-stage process ( Figure 1) that facilitated the inclusion of a greater number of stakeholders and allowed us to provide brief face-to-face education sessions that provided group members with sufficient background information to participate equally and allowed them to seek further clarification throughout the process if required. background information to participate equally and allowed them to seek further clarification throughout the process if required.

Group Members and Settings
Potential patient contributors were identified through pre-existing patient and public involvement (PPI) groups within the University of Leicester and University Hospitals of Leicester NHS Trust, and invited to be involved based on their experience of living with frailty and receiving haemodialysis.
Clinicians and researchers were identified via contacts within the University of Leicester, University Hospitals of Leicester NHS Trust and Lancashire Teaching Hospitals NHS Foundation Trust, and invited based upon experience related to renal disease, frailty or exercise provision, or because they represented professions most likely to be involved in delivering a definitive RCT or implementing the final intervention within clinical practice in the future.
Groups were held at the University of Leicester, Leicester General Hospital and Royal Preston Hospital at a time convenient to members. Patient contributors were reimbursed for their time and travel, in line with best practice guidance from INVOLVE, a national advisory group within the UK that aims to support active patient and public involvement in NHS, public health and social care research [27]. They lasted 60-90 minutes and were moderated by two facilitators (HY and SG), one of whom (HY) was involved in the FLEX-HD study. Moderators aimed to have minimal influence at all stages [28,29]. Group discussions were audio-recorded to provide an aide memoire to the research team. Verbal consent for the discussions to be audio-recorded was sought, but recordings were neither transcribed nor analysed [30]. The proformas used for developing the progression criteria and voting (described in Sections 2.3 and 2.4) formed the basis of the results described.

Stage One. Initial Discussion Groups
Plain English written materials, including an outline of the purpose and procedures used within the groups and a glossary of terms, were given to members before group attendance [31]. Prior understanding was not assumed, and all groups were provided with the same information. Greater explanations of medical terminology were provided to patient contributors and the language used adapted in consideration of possible negative connotations surrounding the term 'frailty' [32]. Large print versions of the materials were produced for visually impaired members.
Members of the initial discussion groups were grouped based upon whether they were a clinician, researcher or patient contributor to promote free discussion. Sessions began with a brief education session that provided the background information necessary to actively contribute [33]. The content of the session is outlined in Table 1. The written materials and the content of the education sessions were developed by researchers (HY, SG, KP, ER), and a PPI member with experience in health research and education (MM). Patient and clinician groups were also provided with an everyday analogy to facilitate their understanding of progression criteria (Figure 2) [9]. As most groups had a variety of experience of research and trials, following the education session, the key aspects of the feasibility study that typically require progression criteria were outlined by the group moderators [5,8]. These were eligibility and recruitment rates, intervention and outcome acceptability, and loss to follow-up. These terms are ill-defined within the literature [7], and therefore groups were also provided with plain English definitions of these concepts, which are outlined in Table 2. Group members were also invited to suggest any additional areas of the study where progression criteria may be indicated specifically for the FLEX-HD study.
Patient and clinician groups were also provided with an everyday analogy to facilitate their understanding of progression criteria (Figure 2) [9]. As most groups had a variety of experience of research and trials, following the education session, the key aspects of the feasibility study that typically require progression criteria were outlined by the group moderators [5,8]. These were eligibility and recruitment rates, intervention and outcome acceptability, and loss to follow-up. These terms are ill-defined within the literature [7], and therefore groups were also provided with plain English definitions of these concepts, which are outlined in Table 2. Group members were also invited to suggest any additional areas of the study where progression criteria may be indicated specifically for the FLEX-HD study. Figure 2. Example analogy to explain progression criteria and how 'stop', 'change' and 'go' thresholds might be applied to aid decision-making. Table 2. Areas requiring progression criteria, and plain English explanations provided for each during the groups.

Eligibility
The number of patients who can take part in the study, whether they later agree to or not.

Recruitment
The number of patients who agree to take part in the study Intervention acceptability Whether participants can stick to the exercise programme

Outcome acceptability
Whether participants can complete the assessments at the start and the end of the study. These assessments can tell us whether intervention might be beneficial.

Loss to follow-up
The numbers of participants who drop out or were 'lost' (lost to follow-up) Members were then asked to suggest progression criteria for each of these aspects of the study, in turn. They were prompted to consider these using three basic questions relating to the 'stop', 'change' and 'go' criteria, which were adapted according to the specific aspect of the trial being addressed. An example is provided within Figure 3.

Eligibility
The number of patients who can take part in the study, whether they later agree to or not.

Recruitment
The number of patients who agree to take part in the study Intervention acceptability Whether participants can stick to the exercise programme Outcome acceptability Whether participants can complete the assessments at the start and the end of the study. These assessments can tell us whether intervention might be beneficial.
Loss to follow-up The numbers of participants who drop out or were 'lost' (lost to follow-up) Members were then asked to suggest progression criteria for each of these aspects of the study, in turn. They were prompted to consider these using three basic questions relating to the 'stop', 'change' and 'go' criteria, which were adapted according to the specific aspect of the trial being addressed. An example is provided within Figure 3.
For each aspect of the trial, members were provided with a proforma to document their ideas for progression criteria for both 'stop' and 'go' thresholds, expressed as a percentage [22]. They were also informed that the 'change' criteria would fall between the 'stop' and 'go' thresholds and asked to document what changes could be made that might increase feasibility or acceptability. Supplementary Materials Table S1 provides a generic proforma that may be adapted for use in other studies. Members had three minutes to complete the task for each aspect of the trial. [22,23]. Typically, NGT requires participants to generate ideas for only one or two questions, but as our process required at least five aspects of the trial to be considered, the generation of ideas was time-limited [23]. For each aspect of the trial, members were provided with a proforma to document their ideas for progression criteria for both 'stop' and 'go' thresholds, expressed as a percentage [22]. They were also informed that the 'change' criteria would fall between the 'stop' and 'go' thresholds and asked to document what changes could be made that might increase feasibility or acceptability. Supplementary Materials Table S1 provides a generic proforma that may be adapted for use in other studies. Members had three minutes to complete the task for each aspect of the trial. [22,23]. Typically, NGT requires participants to generate ideas for only one or two questions, but as our process required at least five aspects of the trial to be considered, the generation of ideas was time-limited [23].
No discussion was allowed, but clarification could be sought, if required [22]. All the progression criteria were collated, discussed and combined where appropriate [23]. In accordance with conventional NGT methodology, the intention was for the groups to then rank the progression criteria for each part of the trial in order of preference, to arrive at a set of final 'preferred' progression criteria [22].

Stage Two. Final Discussion Group
The aim of the final discussion group was to bring together contributors of all backgrounds to review the progression criteria generated by the initial groups and decide upon the final preferred progression criteria [34]. Members of the mixed final group were asked not to discuss their backgrounds to reduce potential power differentials [29]. The education session was repeated, and the anonymised progression criteria developed by the initial groups presented. Instances where multiple initial groups had come up with the same progression criteria were highlighted [25]. Fifteen minutes was allocated for group members to review and refine these until there was a condensed list. Group members were then asked to vote for their 'top three' preferred criteria from this list, using a proforma. A generic version of this proforma is included within Supplemental Materials Table S2. Voting was completed for each aspect of the trial in turn, without conferring [23]. Members awarded their preferred option the highest score (1) and continue ranking to 3, for their least preferred option. Moderators scored the votes for each potential 'stop' and 'go' criteria. Progression criteria voted in first place were awarded a score of 3, the second a score of 2 and the last a score of 1. The highest scoring criteria became those used in the FLEX-HD study [25]. In instances where scores were tied, further discussion and an additional vote was allowed until a preferred criterion was identified [34]. All contributors were given a written summary of the findings once all the groups were completed.

Impact Evaluation
The specific impact of the involvement of the patient contributors was not formally evaluated. We aimed for all members to work in partnership and singling out one group to determine the impact No discussion was allowed, but clarification could be sought, if required [22]. All the progression criteria were collated, discussed and combined where appropriate [23]. In accordance with conventional NGT methodology, the intention was for the groups to then rank the progression criteria for each part of the trial in order of preference, to arrive at a set of final 'preferred' progression criteria [22].

Stage Two. Final Discussion Group
The aim of the final discussion group was to bring together contributors of all backgrounds to review the progression criteria generated by the initial groups and decide upon the final preferred progression criteria [34]. Members of the mixed final group were asked not to discuss their backgrounds to reduce potential power differentials [29]. The education session was repeated, and the anonymised progression criteria developed by the initial groups presented. Instances where multiple initial groups had come up with the same progression criteria were highlighted [25]. Fifteen minutes was allocated for group members to review and refine these until there was a condensed list. Group members were then asked to vote for their 'top three' preferred criteria from this list, using a proforma. A generic version of this proforma is included within Supplemental Materials Table S2. Voting was completed for each aspect of the trial in turn, without conferring [23]. Members awarded their preferred option the highest score (1) and continue ranking to 3, for their least preferred option. Moderators scored the votes for each potential 'stop' and 'go' criteria. Progression criteria voted in first place were awarded a score of 3, the second a score of 2 and the last a score of 1. The highest scoring criteria became those used in the FLEX-HD study [25]. In instances where scores were tied, further discussion and an additional vote was allowed until a preferred criterion was identified [34]. All contributors were given a written summary of the findings once all the groups were completed.

Impact Evaluation
The specific impact of the involvement of the patient contributors was not formally evaluated. We aimed for all members to work in partnership and singling out one group to determine the impact of their involvement felt incongruous [35]. All members were, however, invited to give verbal feedback on what worked well and what could be improved in relation to the process at the end of the discussions.

Ethics Approval
The UK Health Research Authority (HRA) toolkit [36] determined that the work was not considered research, as participants were not randomised, and no change to usual care or intervention was made. In addition, although the process described may be transferrable to other studies, the 'results' (i.e., the progression criteria) were specific to the FLEX-HD study, and were used solely to illustrate the process. Furthermore, joint guidance from the HRA and INVOLVE states that ethical approval is only required if patients are conducting research as part of the research team, or if they are involved in the study as participants [37]. In accordance with this national guidance, and following confirmation from the University of Leicester (sponsor of the FLEX-HD study), ethical approval was not required for this PPI activity.

Initial Groups
Eight initial groups were held, involving a total of 52 members: 15 (29%) patient contributors, 13 (25%) researchers and 24 (46%) clinicians. The characteristics of group members involved in the initial discussion groups are outlined in Table 3. The research groups included seven (54%) research associates and six (46%) clinical academics with a median of seven (interquartile range (IQR) [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22] years' experience. The clinician groups included nine (36%) physiotherapists, nine (36%) renal nurses, five (20%) occupational therapists and two (8%) nephrologists with a median of 14 (3-24) years' experience. The patient contributor groups included six (40%) people living with renal disease and nine (60%) older people living with frailty. Across all the groups 37 (71%) were female; 45 (86%) were White British, six (11%) Asian British, one (1.5%) Black British and one (1.5%) did not wish to state their ethnicity. Researchers were a median age of 44 (IQR 26-48) years, clinicians a median age of 38 (IQR 33-49) and patient contributors 63 (50-82) years. In total 32 (62%) had previously been involved in research in any capacity. The percentage progression criteria developed for each of the aspects of the trial by each initial group are outlined in Table 4. Whilst the groups came to consensus over their preferred criteria through discussion (negating the need for the ranking task originally planned), Table 4 clearly demonstrates the wide range of progression criteria produced across the different groups. Additionally, other aspects of the trial, namely the acceptability of the randomisation procedures, intervention fidelity and incidence of adverse events and harms were identified as important by a minority of groups, reflecting the differing priorities and levels of knowledge amongst the groups. As there was uncertainty around whether these warranted progression criteria, they were not subject to voting and ranking within the final group. Table 4. "Stop" and "Go" progression criteria decided by individual discussion groups. Progression criteria are expressed as a percentage; blank fields indicate that the group did not set progression criteria for that aspect of the trial.

Characteristics of Final Groups
The final joint discussion group involved 8 members: two (25%) patient contributors, one (13%) nephrologist, one (13%) researcher, two (25%) physiotherapists and two (25%) occupational therapists. Four (50%) were female, five (63%) were White British and three (38%) Asian British, with a median age of 45 (IQR 29-64) years. Six (80%) members had been involved in an initial group and four (50%) had previously been involved in research in any capacity. Table 5 outlines the scores given to each of the condensed progression criteria following the voting task. All the criteria ranked in first place received 67 to 100% of the votes.

Final Progression Criteria
The final preferred 'stop' and 'go' progression criteria selected are presented within Figure 4. Greater flexibility was afforded to exercise adherence to allow for periods of ill-health, hospitalisation and holidays over the duration of the intervention. The thresholds for outcome measure completion were more stringent, in recognition of the high levels of data completion required to make robust conclusions about efficacy in a definitive RCT. The potential need for different 'stop' and 'go' thresholds for different types of outcome measures was highlighted. For example, physical function measures were time-consuming, burdensome and subject to greater influence from symptoms and illness than patient-reported measures, thus requiring more flexible thresholds for completion. Four groups stated that loss to follow-up should be assessed at interim, and one group indicated that progression criteria established for loss to follow up should be applied to both the intervention and control arms of the study separately. progression criteria established for loss to follow up should be applied to both the intervention and control arms of the study separately.

Change Criteria
Patient contributors, clinicians and researchers across groups identified multiple different ways that the trial and intervention could be amended, should results fall between the 'stop' and 'go' thresholds, and indicate that 'change' is required. These are summarised within Figure 5.

Change Criteria
Patient contributors, clinicians and researchers across groups identified multiple different ways that the trial and intervention could be amended, should results fall between the 'stop' and 'go' thresholds, and indicate that 'change' is required. These are summarised within Figure 5.
Thirteen (25%) initial group members identified that eligibility rates could potentially be boosted by widening the eligibility criteria, particularly in relation to frailty levels. Regular contact with those screening for eligibility, reminding them about the study and checking that screening was being completed appropriately was recommended. The therapist groups recommended that eligibility criteria for the exercise intervention be revisited should the design and delivery of the programme change significantly. For example, some participants who might not be eligible to exercise at home unsupervised may be able to exercise safely within a supervised environment, and the eligibility criteria should reflect this difference.
All groups indicated that the intervention would be the primary area of the trial to review should strategies to boost recruitment be required. Simplifying the intervention by reducing the number of visits required, the length of the intervention and making it enjoyable to participate was important.
The approach taken during recruitment was also an area highlighted for review. Explaining the study face-to-face and, where possible, exploring potential individual barriers to participation during the recruitment process was felt to be especially relevant in a population of older people living with frailty, whose participation may be limited by external constraints such as reliance upon packages of care and family assistance. By organising the trial procedures around these considerations, groups felt that more people may be encouraged to participate. When discussing the study, all groups also indicated that highlighting the potential benefits for the individual was important. Thirteen (25%) initial group members identified that eligibility rates could potentially be boosted by widening the eligibility criteria, particularly in relation to frailty levels. Regular contact with those screening for eligibility, reminding them about the study and checking that screening was being completed appropriately was recommended. The therapist groups recommended that eligibility criteria for the exercise intervention be revisited should the design and delivery of the programme change significantly. For example, some participants who might not be eligible to exercise at home unsupervised may be able to exercise safely within a supervised environment, and the eligibility criteria should reflect this difference.
All groups indicated that the intervention would be the primary area of the trial to review should strategies to boost recruitment be required. Simplifying the intervention by reducing the number of visits required, the length of the intervention and making it enjoyable to participate was important.
The approach taken during recruitment was also an area highlighted for review. Explaining the study face-to-face and, where possible, exploring potential individual barriers to participation during the recruitment process was felt to be especially relevant in a population of older people living with frailty, whose participation may be limited by external constraints such as reliance upon packages of Involving the family in the recruitment process and enlisting the support of patient advocates and peers who had taken part in the study and giving potential participants the chance to observe the types of exercise they may be asked to participate in was also identified as important across all groups. Patient groups suggested the use of an overview booklet be used in lieu of a more traditional patient information sheet, with images alongside limited simple text. More detailed information could then be provided if interest in taking part is expressed.
Sixteen (25%) of all members across all groups felt that varying the mode of exercise offered would increase participant engagement with the intervention by potentially making it more enjoyable. Fifteen (23%) of all participants across all groups also suggested reviewing the intensity of the exercise programme in this population. Fatigue was identified as a key factor limiting exercise adherence for people living with frailty and receiving haemodialysis, and therefore all groups recommended lowering the exercise intensity, providing participants with a longer-run in period and offering interval training (where short bouts of moderate intensity are interspersed with those of lesser intensity). Increasing the support available from exercise professionals was also identified as important. Supplementing this practical support with regular reviews to check participant progress and including techniques to support and maintain behaviour change were also suggested.
Members postulated that reducing the number of measures used, and selecting those which were simple, quick and less intrusive but also meaningful to participants, would help to increase completion. Prioritising the measures key to the trial, conducting them at a time and location that was most convenient to the individual was also highlighted as important. Effective communication about what the measure is assessing, alongside support for outcome completion was additionally a frequently identified means of increasing outcome completion.
It was evident during discussion that several aspects of the trial were interlinked, and that changing one part might have a positive impact on another. For example, adapting the intervention might not only increase acceptability, but might also improve recruitment and retention. Finally, all group members emphasised the importance of understanding the barriers to feasibility from the perspectives of both participants and clinicians. They deemed this crucial to identifying those amendments most likely to result in successful delivery of the future definitive trial.

Resource Implications
The complete process took four months (including preparation time) and a total time of 9.5 hours to complete. The total running costs for the groups (refreshments, room hire, expenses and payments to some patient contributors) was £975.

Impact and Feedback
Group members valued being actively involved in setting progression criteria that would have a direct impact upon the FLEX-HD study; however, they found the application of abstract concepts and theoretical percentages challenging. They suggested that more time be allowed, and that members of the final discussion group be given an outline of the results from the individual groups to allow greater time for reflection prior to discussion.

Discussion
This work describes a transparent and collaborative process for developing progression criteria that are a critical part of determining whether a definitive RCT is feasible [3]. Some components were study specific, but the overall structure and process could be replicated within other feasibility studies. This represents a first step in addressing the paucity of guidance detailing how progression criteria should be developed [2,7]. A template for generating and subsequently voting on progression criteria for use in other feasibility studies is included within Supplementary Materials Tables S1 and S2.

Involving Patients, Clinicians and Researchers Equally in the Co-Production of Progression Criteria
The results of the initial groups highlight tensions between the priorities and perspectives amongst diverse groups of patient contributors, clinicians and researchers. Involving all stakeholders in the process recognises that what is feasible within a 'real world' clinical setting may be at odds with what is possible within a highly standardised research one [7,10,14,38]. These issues were considered and resolved in a transparent and equally inclusive manner, leading to the development of progression criteria that are meaningful within both settings. The inclusion of clinicians and patients was also particularly important within the context of the FLEX-HD study, as there is little prior research in a frail group in the field of renal rehabilitation to guide the development of progression criteria. As people living with frailty represent a 'seldom heard' group, who can be challenging to engage in research, their perspectives are crucial when determining feasibility [38][39][40] and working in collaboration facilitated a detailed understanding regarding how to adapt trial processes in a manner acceptable to this patient group, and the context in which they receive care.
The use of co-production has been linked with the generation of multiple ideas for innovative intervention development and service improvement [41]; in keeping with this, the breadth of suggestions for amendments to the trial in the event of the 'change' thresholds being met strongly underlines the value of involving multiple stakeholders in the co-production of progression criteria. The combination of the group members' different types of knowledge and experience provided numerous innovative ways in which an RCT could be adapted to increase feasibility, which moved beyond the scope of those suggested by the research groups alone. The application of these potential solutions may inform whether or not to proceed to a definitive trial and would complement the decision-making process outlined by Bugge et al. [1]. When considering these amendments, the challenge for the study team is three-fold. Firstly, which of the suggestions are most likely to enhance feasibility? One option would be to compare the suggestions made with the views of the study participants if a mixed-methodology approach has been employed. Secondly, what (and how many) changes constitute major alterations to the trial and intervention? Numerous or major alterations may indicate that further development and feasibility testing is required [2], but to date there is no guidance on what amount of change would suggest that further testing is warranted [7]. Thirdly, the research team should consider how proposed changes might impact both scientific rigour and resource requirements, which could also influence feasibility [8]. For example, if outcome completion levels fall within the 'change' threshold, providing participants with support is one means of increasing completion rates, but may also necessitate additional researchers blinded to intervention groups to provide this without introducing bias.

The Challenges of Co-Producing Progression Criteria
Research literacy is a key component of co-production, because knowledge increases parity and participation [7,18,42,43]. Different levels of understanding were evident across all groups, and there were different levels of PPI and research participation experience amongst contributors. The ability of all these individuals to articulate their ideas and contribute within the group setting demonstrates inclusivity. However, there were some challenges to ensuring that all felt able to do this, and to focus those more experienced members away from issues less pertinent to feasibility studies (e.g., the need for statistical power). Whilst the education session offered aimed to ensure that all involved could contribute equally, this needed to be balanced with not overburdening group members or removing their lived or clinical experience from the forefront of their decision-making [11,19,44,45]. Similar challenges have been identified within research literacy training offered as part of community-based participatory research [46][47][48]. Evaluations of these programmes conclude that theoretical constructs may best be explained through practical examples [46]. A strength of our approach was introduction of a brief education session, which was developed with PPI members to include everyday analogies and jargon-free content to facilitate understanding [44,49,50]. Despite this, some members found it difficult to fully comprehend the abstract concept of a percentage-based progression criteria, which may, in part, be reflected in the wide range of criteria suggested in the initial groups. Visual methods of learning are an accessible and engaging way of increasing comprehension and encouraging creative thinking [41,50,51]. The use of simple graphical representations (e.g., pie charts using Microsoft Excel), or physical counters representing participant numbers, used during the groups, may have enhanced understanding of the impact of one progression criteria over other aspects of the trial. Members also fed back that additional time would be beneficial in future groups, and this may also have supported the use of visual representations.

Limitations
Whilst flexibility is a key feature of the NGT method, the modifications made may have influenced the criteria obtained. The omission of the ranking stage in the initial groups occurred iteratively and may have made it more difficult for some members to express their views. However, this process was carefully moderated and occurred in groups separated by background, minimising the potential impact. Some level of subjectivity is unavoidable in consensus methodology, and the inclusion of different members or additional groups may have resulted in different criteria [33,52]. The outcomes of this process are not generalisable, nor are they intended to be, as they have been specifically developed for the FLEX-HD study. Consensus methods are grounded within a pragmatic constructivist philosophy and therefore we seek to illustrate the rigour of the process used [53].
Finally, given the resource implications outlined, and the two-stage process used, the process may be challenging to incorporate into conventional TSC structures. Patient representation on TSCs is usually limited, and there is a perceived limited impact of patient representation, questioning whether the co-production of progression criteria is possible in this setting [7,12,[18][19][20]. More recently there has been a shift towards involving patient advisory panels, comprising both 'expert' patient contributors as well as less experienced lay members, in specific tasks responsive to the needs of the trial. Wide representation is important from all stakeholders in the development of progression criteria [20,45,54] and therefore the development of progression criteria might sit better with a patient advisory panel, alongside the research team and involved clinicians. The process could be streamlined, and resource implications reduced, by completing fewer initial groups, or by having only one mixed group inclusive of greater numbers complete both stages of the consensus processes in one sitting.

Conclusions
The use of modified NGT within diverse, structured discussion groups enabled the perspectives of patients, clinicians and researchers to be included in the co-production of progression criteria that reflected both trial and 'real-world' feasibility. Importantly, the process also allowed for the identification of numerous areas where the trial could be amended to increase feasibility if required. The main challenge to the process was reconciling different levels of knowledge and understanding amongst the contributors. Future groups would benefit from incorporating creative interactive and visual methods to facilitate understanding. The structure and processes used provide a transparent and flexible means of setting progression criteria that could be simplified, and replicated for use in other studies, with some adaptation to the address the specific focus of the trial required.