A Systematic Review of Arts Practice-Based Research Abstracts from Small and/or Specialist Institutions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper problematises abstract writing to highlight inherent weaknesses in existing arts-based research. The systematic review of 27 abstracts demonstrate sensitivity to how institutional contexts might shape research and writing in the arts. Within the context of small and specialist HEIs in the UK, the paper contributes timely insights on how practice-based research can be framed, enhanced, and evaluated through repository metadata.

While some of the reviewed outputs were assessed in the most recent REF cycle, the paper does not elaborate on the latest REF requirements. Beyond references to REF2021, the manuscript would benefit from clearer articulation of current units of assessment, evaluative criteria, and working methods specific to arts and practice-based disciplines.

The paper would be further strengthened by: (i) clearer methodological transparency regarding specific search terms and the UK repositories that were selected; (ii) sharper conceptual clarification of what “fit for purpose” means in relation to abstracts and for whom; (iii) a more explicit description of Critical Appraisal Skills Programme for systematic review, and which aspects of the CASP checklist was adapted; (iv) clarification of abbreviations in the text (such as CT - cannot tell?).

Also, both the introduction and conclusion would benefit from revision to appear less “transactional”, and more persuasive in addressing a wider readership of arts- and non-arts based practitioners and researchers, including those beyond the UK context.

Author Response

Comment 1. The paper would be further strengthened by: (i) clearer methodological transparency regarding specific search terms and the UK repositories that were selected; 

Thank you for your helpful comments. The explanation, “Data was collected from 20 research repositories containing practice-based art and de-sign research outputs in the UK (see appendix 1 point 7). The data was collected by searching each repository manually by APBO item type. The item types searched included: artefact, show/exhibition, composition, performance, image, digital video, audio, and creative project. Data meeting the above parameters was collected and arranged in a dataset with the following fields: Code for the institution; Date; Item Type; Title; Abstract; License and Item URL. From this initial search outputs were identified (n=352). (3.5. Search strategy, page 8, paragraph 1, lines 305-311). Individual Repositories and institutions not named to protect institutional reputation as requested in ethical approval. 

Comment 2. Sharper conceptual clarification of what “fit for purpose” means in relation to abstracts and for whom.

Many thanks for this feedback. The words “In other words, the abstracts would be deemed fit for purpose if they contributed to-wards their Findablility, Accessibility, Interoperability and Reuse in other research (GoFAIR, 2016) and represented the underpinning research in a useful manner. “ was included to add more explanation. (page 2, para 4, lines 55-57)

Comment 3. A more explicit description of Critical Appraisal Skills Programme for systematic review, and which aspects of the CASP checklist was adapted.

Thank you for this suggestion. More detail has been provided about the CASP tool and the modifications to align with APBOs. Section 3.7. Quality assessment (CASP check list modified) has been substantially expanded to provide more explanation and the decision-making process. (pages 9-10, lines 310-402) . 

Comment 4. Clarification of abbreviations in the text (such as CT - cannot tell?).

This has been resolved please see: Y= Yes, N= No, CT= Cannot Tell (page 12, line 435)

Comment 5. While some of the reviewed outputs were assessed in the most recent REF cycle, the paper does not elaborate on the latest REF requirements. Beyond references to REF2021, the manuscript would benefit from clearer articulation of current units of assessment, evaluative criteria, and working methods specific to arts and practice-based disciplines.

Thank you for this observation and we have included more references from the REF team that are more current, the detailed and final  guidance for REF 2029 has yet to be published but we have included the words:

“Areas assessed at the last REF in 2021 included Outputs (60%), Impact (25%), and Environment (15%). REF 2021 was split into four assessment Main Panels (A, B, C, and D), who oversee Units of Assessment (UoAs) corresponding to discipline areas within the panel member’s expertise. Each assessment Main Panel has different set criteria and working methods reflecting the discipline areas it assesses. In 2021, the UoA for Art and Design: History Practice and Theory (UoA 32) fell under the purview of REF Main Panel D.  For the next REF in 2029, outputs will be submitted under the same output item types as in REF 2021 and will be assessed within the  same UoAs and Main Panels.” (page 5, paragraph 2, lines 189-197)

Comment 6. The introduction and conclusion  need to be less transactional to persuade readers of arts and non-art audiences and international people.

Thank you for this advice we have included the words, “This would enhance the presentation of the research and show the researchers’ work off to the best advantage. It would make the work more accessible to academics and those out-side of academia, widening the audience for practice-based research. By increasing the visibility of arts-based practice-based research, it is hoped that the insights derived from this work will inform new work and be cited as the starting point for new investigations. (page 2, para 1, lines 43-48)

We have also expanded the benefits of FAIR, “This is because the abstracts provide a textual element to APBOs, which are often non-textual items. Having a text-based element in the publicly accessible metadata of an APBO deposit contributes towards its potential Findablility, Accessibility, Interoperability and Reuse in other research (GoFAIR, 2016). Higher quality abstracts provide those accessing APBOs on a repository a clearer understanding of the research process and the new knowledge which was generated through the APBO. They also communicate the specificities of the research which are not evident from the non-textual item on its own (REF, 2019, p58).” (page 3, para 1, lines 91-98 )

The opening section of the conclusion discusses the benefits of abstracts on repositories, “Higher quality abstracts are beneficial to the academics who have undertaken the re-search, those who are accessing the research on a repository, and society as a whole. This is because the research undertaken is represented in the best possible way for the re-searcher; it improves the Findability, Accessibility, Interoperability, and Reusability (FAIR) of the research; and means that other researchers and members of the public can access current artistic research which is represented in a meaningful way. “(page 21, para 6, lines 665-670)

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors, 

I really appreciate your work on innovative and creative research methods and on how the abstracts of the outputs are written and classified in the repository. Your paper deserves to be published, but I think you should do two minor revisions (as you find in the attached file):

1) Clarify what "design practice-based research" is. As part of an innovative and creative research method, a half-page on what the object of your research is would help the reading flow more smoothly. Please insert literature on innovative and creative research methods.

2) Improve your contribution to the literature in the last paragraph, especially by giving a draft of recommendations on how to:

• developing protocols on how to write the abstracts (giving criteria and an example of best practice)
• designing and project repositories that fit the output of APBO,
• writing specific quality assurance criteria and best practices.

 

Comments for author File: Comments.pdf

Author Response

Comment 1. In this paragraph, please insert an half-page on what APBO is. In fact, I have had difficulties on understanding, (while reading) what kind of scientific product the authors are talking about. 

Thank you for your helpful and supportive comments. In response, we have added a concise definition at the first point where the term is introduced in the Introduction (Section 1), “APBOs are derived from innovative and creative research methods often employed by researchers from the art and design field. As this is different from mainstream social research, the approach requires some further explanation.” Then further explanation is given in the next paragraphs. (page 2, para 5, lines 61-63). Please also see 2.2 where it is revisited page 5, para 3, lines 200-209.

Comment 2. Please, increase the literature on APBO. This is not the mainstream social research. It can. be unknown by the readers or little known.

The literature has been increased please see (Page  2, para 5, lines 61- 83).

Comment 3.  Please, report the scientific area of expertise of these reviewers.

Thank you for this request. The words “The reviewers were one researcher with expertise in leading inclusive research cultures in art and design discipline areas and one in art theory and curatorial theory and practice and a repository manager with expertise in Open Research and arts meta-research.” (Page 7 para 4, lines 293-295).

Comment 4. Please, justify this range of time? Why not 10 years, or more?

The justification was given, “The date range (2019-2024) was employed because during this time APBOs had gained recognition in the wider research environment. “(Page 1, para 1, lines 22-25).

Comment 5. 

1. . I think that this last part can be improved a lot, giving detailed recommendations on:

1) developing protocols on how to write the abstracts (giving criteria and an example of best practice),

2) how designing and projecting repositories which fit the output of APBO,

3) writing specific quality assurance criteria and best practice.

Thank you for your kind comments and helpful suggestions. 

1. We have added the following text: 1.Training through continuous professional development for arts researchers and repository managers who need to write and review abstracts. The findings demonstrate a need for targeted continuous professional development (CPD) for both arts researchers and repository managers who are responsible for writing, reviewing, and curating 300-word abstracts for APBOs. Unlike text-based outputs, APBO abstracts are frequently written at the point of repository deposit and must fulfil multiple functions simultaneously: articulating a clear research topic or line of inquiry, explaining how creative practice functioned as a research method, and communicating insights or contributions to knowledge that may not be immediately legible from the creative output itself. CPD initiatives should therefore focus on developing shared, sector-wide understandings of what constitutes a high-quality APBO abstract, with particular emphasis on distinguishing research articulation from promotional or marketing language.

Training could be delivered through workshops, written guidance, and peer-led sessions, and supported by anonymised examples of effective abstracts drawn from high-quality repository records. Such provision would benefit from being aligned with well-designed abstract templates that prompt authors to address core research elements consistently. CPD should also recognize the critical mediating role of repository managers, equipping them with the confidence and expertise to support researchers at the point of deposit and to apply consistent standards across records.

2.Development of protocols for archiving arts-based outputs on repositories and promoted by sector bodies. The review reveals a lack of consistent protocols governing how APBOs are archived and described across institutional repositories. To address this, clearer protocols are required to guide both abstract writing and the structuring of repository records for practice-based research. Based on the analysis presented in this study, effective protocols for APBO abstracts should specify core criteria, including:
 

• clear articulation of the research topic and line of inquiry;
  • explanation of how creative practice functioned as a research method;
  • concise identification of insights or contributions to knowledge, including those emerging through process;
  • contextual information regarding dissemination, without allowing indicators of prestige to dominate the account.

In addition, protocols should address repository design considerations to ensure that infrastructures are capable of accommodating the multimodal and processual nature of APBOs. This includes guidance on the use of templates for consistent metadata entry, the presentation of multiple associated files (such as images, audio, video, or process documentation) in coherent sequences, and clearer differentiation between item types, for example exhibitions as research outputs versus exhibitions as dissemination formats. Sector-level initiatives such as SPARKLE and related projects (including PRVoices and work towards shared repository infrastructures for practice-based research) provide important reference points for how such protocols might be developed and promoted collaboratively. Endorsement by sector bodies would support wider adoption and greater consistency across institutions.

3. Quality assurance checks on outputs in repositories are undertaken by repository staff on a regular basis. The high rate of exclusion observed during the systematic review highlights the importance of routine quality assurance checks on APBO records. While the intrinsic quality of creative research cannot be assessed solely through metadata, minimum standards for repository records are essential to ensure research integrity, usability, and discoverability. Drawing on the adapted CASP-based tool developed for this study, quality assurance checks might include verifying that abstracts:

• contain an identifiable research topic and research question or inquiry;
  • explicitly reference practice-based methods;
  • articulate insights or knowledge generated through the research;
  • provide accessible and functional links to associated materials;
  • correctly attribute collaborators and acknowledge ethical considerations where relevant.

Best practice would involve embedding these checks within clear repository workflows, including review at the point of deposit and periodic auditing of records. An annual repository review process, undertaken by trained repository staff, could help identify incomplete metadata, broken links, or inconsistencies in abstract quality, and support ongoing improvement. Such processes, when combined with appropriate infrastructure and sustained training for both researchers and repository managers, would strengthen the reliability of repositories as infrastructures for practice-based research. Best practice would involve embedding these checks within clear repository workflows, including review at the point of deposit and periodic auditing of records. An annual repository review process, undertaken by trained repository staff, could help identify incomplete metadata, broken links, or inconsistencies in abstract quality, and support ongoing improvement. Such processes, when combined with appropriate infrastructure and sustained training for both researchers and repository managers, would strengthen the reliability of repositories as infrastructures for practice-based research. (page 22, from paragraph 2, lines 697-761). 

Comment 6. Clarify what "design practice-based research" is. As part of an innovative and creative research method, a half-page on what the object of your research is would help the reading flow more smoothly. Please insert literature on innovative and creative research methods.

Thank you for this request we have written 4 paragraphs defining creative methods that lead to non-traditional outputs.

Yjank you for this request. I have put this in the beginning of the introduction because I think readers need to understand straight away what creative research is and how it leads to APBOs.

“APBOs are derived from innovative and creative research methods often employed by researchers from the art and design field. As this is different from mainstream social research, the approach requires some further explanation.

The research insights gained from this method come from the creative practices such as art-making , designing, curating or film-making . The arts offer ‘ways of knowing’ derived from sensory perception leading to aesthetic, emotional and intellectual responses to the world. These different ways of knowing can potentially enhance a researcher’s comprehension of complex human and non-human interactions.

The research that produces APBOs is practice-focussed and driven by the making process. New knowledge is gleaned from the researcher’s reflection on the process and the outcomes of the practice. The outcomes of practice are not those traditionally associated with other disciplines, such as text-based articles and book chapters but could be exhibitions,  digital videos  or creative projects (all examples of APBOs) (Candy, 2006; Rolling, 2014; Candy et al., 2021).

Dallow (2003) noted that the practice-based approach , ‘allows the research work of the creative practitioner to ask questions not only...about work...but through work’ (p. 59).This is achieved through considered introspection that can be shared with other researchers (Brown & Patterson, 2021; Dallow, 2003; Xue & Desmet, 2019).

Frayling et al. (1997) argued that as part of the dissemination of the research, practice-based researchers are, ‘obliged also to map for his or her peers the route by which they arrived at their product/s’ (p. 13).  In other words, arts practice-based researchers whose research outcomes are APBOs need to explain their processes and insights and this can be done through the associated 300-word abstract.” (Page  2, para 5, lines 61- 83).

 

Reviewer 3 Report

Comments and Suggestions for Authors

Abstract

“The qualitative systematic reviews asks…” = The review itself doesn’t ask anything; you do this as authors.

I find the research question rather unusual, based on the abstract. I hope I find a rationale as to why this research is necessary in the introduction/ literature review.

I would also suggest to look at a few best practices in the field to getter a gasp of appropriate language. For example, ‘outputs’ are scientific articles, chapters, etc. In addition, you are referring—with that part of the sentence—to a time frame. Why not explicitly state that the review is covering 2021-2026? Moreover, why the last five years? Why is that necessary? I would suggest to rephrase these sentences.

The 300-word supporting statement is an abstract?

“The data repositories from small and/or specialist higher education institutions” = What is the rationale for this?

The last sentence of the abstract: What do you mean with this?

p.1.32–35 I do not see how the archiving of research output is linked to abstracts. Wouldn’t it make more sense to examine archiving practices of personnel rather than abstract in the output?

p.2.39 But you use the ‘300-word supporting statement’ in the abstract.

p.2.48–50 I am glad the rationale is appropriate and presented clearly. I would suggest to take another look at the abstract. That is not capturing this rationale sufficiently.

p.2.53–55 “Small and/or specialist institutions in the UK generally have fewer than 3000 students and many have specialist provision in a particular subject area such as the arts, music, or teacher training (Bols, 2023)” = And how does that specialist provision translate to the abstracts?

p.2.56–59 This argument is invalid. Other research is also transparent and systematic.

The text on page 2 is lacking coherence (these are all separate bits of information). Can you aim for more coherence? This would also strengthen your argumentation.

p.2.78 See my comment about reporting on a time frame. Ideally you list the month as well.

p.2.79 Can you list examples of those item types?

p.3.81 I am also wondering why you list exclusion criteria here rather than in the methodology. List examples of practice-based research (be specific enough). You are listing your review as being transparent but the information presented here isn’t detailed enough.

p.3.82/83 Thus, this is an exclusion criterion.

The header “Context” comes as a surprise. Is this your literature review? If so, please display a more informative header (and subheaders).

p.3.115 After the “et al” you need to insert a full stop/period. Thus: et al. Apply this consistent throughout your manuscript.

p.5.189 Avoid back-to-back brackets: “(SAR) (ARMA, nd; UKRN, 2025; Drake, 2025)” = (SAR; ARMA, nd; UKRN, 2025; Drake, 2025).

p.5.193/194 Three or more authors can be listed with the fist author + et al.

p.5.205 This hyphen is not an em dash. Please revise. In a similar vein, p.12.375 does not display em dashes (these are en dashes). Please revise. Check the remainder of your work to correct this.

p.5.213–215 To what unlawful or harmful materials are you referring to? Moreover, how did you check this? Walk the reader through the quality checks.

p.6.227 This time frame is different than the one you describe earlier in your work. That is why I suggested writing the years rather than a description ‘of the last five years’. Please revise accordingly.

The in- and exclusion criteria have to be mentioned in the methodology. As I stated before, you will need a lot more detail to describe these criteria. You current version isn’t transparent enough. In addition, for systematic reviews you present both inclusion AND exclusion criteria.

Moreover, on p.6.241 you refer to duplications = duplicates. Furthermore, you need to list how many duplicates you excluded, in light of transparency. Please add this information.

The checklist (p.6.246) also requires more detail. Each question should have a clarification. For example, the clear lines of research questions: is a RQ mentioned? Moreover, how do you determine if research insights have been clearly articulated? Again, this requires more detail to contribute to scientific rigor and the transparency you are talking about.

p.7.278 Are you referring to the checklist you adopted (earlier)? If so, write that. I would suggest to merge these sections for more clarity.

p.7/8.288–293 This text is redundant if you present Table 1. The reviewers probably have more experience than other (external) reviewers. It would be relevant to ‘test’ your coding scheme after extended revisions in the form of more details.

Figure 1 Please insert the exclusion criteria in each stage to provide a more transparent workflow, as you have emphasized in the first pages of your manuscript.

p.9.312 This is not an em dash. Please revise.

Based on Table 2, I notice a major difference between the three reviewers: R1 said ‘yes’ to every output with exception of one. Does this person have more experience than the others? With revised in- and exclusion criteria, you were probably able to capture some disagreement beforehand. I am a bit surprised that the criteria you follow are general (and, therefore, not in line with how systematic reviews have to be conducted).

p.12.335/336 “The types of information that was considered” = ‘types’ is plural.

p.12.345 The themes need more explanation. It is a bit odd that you present this in your discussion rather than in your methodology. If you restructure your manuscript, you can place your results alongside existing research and actually discuss your results/findings. This would also allow for more engagement with existing research. 

p.13.391 “festivals aren’t just quality indicators, they” = Do not use contractions.

Author Response

Comment 1. The qualitative systematic reviews asks…” = The review itself doesn’t ask anything; you do this as authors.

Thank you for your observation, The abstract was amended to “the authors ask to what extent is the 300-word abstract fit for purpose in representing art and design practice-based research outputs on small and/or specialist institutional repositories?”(page 1, para 1 lines, 11-13.

Comment 2. I find the research question rather unusual, based on the abstract. I hope I find a rationale as to why this research is necessary in the introduction/ literature review.

Thank you, the rational is explained in the introduction, “Despite its centrality, little is known about how effectively these statements communicate research processes and insights across the sector. This gap provides the rationale for the present study, which undertakes a qualitative systematic review of APBO abstracts to examine how they are currently functioning as research summaries within small and/or specialist institutional repositories. Small and/or specialist institutions in the UK generally have fewer than 3000 students and many have specialist provision in a particular subject area such as the visual arts, music, or teacher training (Bols, 2023).” (Page 3, para 1 lines 98-105)

Comment 3. I would also suggest to look at a few best practices in the field to getter a gasp of appropriate language. For example, ‘outputs’ are scientific articles, chapters, etc. In addition, you are referring—with that part of the sentence—to a time frame. Why not explicitly state that the review is covering 2021-2026? Moreover, why the last five years? Why is that necessary? I would suggest to rephrase these sentences.

Thank  you. This section has been amended. The abstract is an important part of the metadata when an Arts Practice-Based Output (APBO) is deposited on a repository. APBOs are non-traditional item types resulting from creative/artistic research processes. Examples include exhibitions, designs and digital videos.  Little is known about how effectively these abstracts communicate research pro-cesses and insights across the art and design sector.  This study aims to investigate how well the abstract communicates information about the arts practice-based research through a systematic review of APBOs. The eligibility criteria for inclusion in the review were: APBOs must be from the past 5 years; be an item type where the 300-word abstract is required; the statement must be part of the publicly-available metadata for the item and outputs are practice-based and from the art and design field. The date range (2019-2024) was employed  because during this time APBOs had gained recognition in the wider re-search environment. APBOs from the reviewers’ institutional repository were not included in the study to avoid bias that could skew the results of the review. (page 1, abstract, 11-26).

Comment 4. The 300-word supporting statement is an abstract?

Thank you for observing this, We have changed  the word supporting statement to abstract .

Comment 5. The data repositories from small and/or specialist higher education institutions” = What is the rationale for this?

Thank you for this question we have included the words “The data repositories from small and/or specialist higher education institutions in the United Kingdom were searched for outputs which appeared to meet the eligibility criteria. These types of institution prioritise and produce more of these output types.” (page 1, abstract, lines 26-29)

Comment 6. The last sentence of the abstract: What do you mean with this?

We thank the reviewer for highlighting the ambiguity in this sentence. We have revised the final sentences of the abstract " Findings suggest that the 300-word abstracts contained information about the quality indicators such as whether the project got funding; the identities of prestigious collab-orators and/or dissemination vehicles and the international recognition of the research. Other identified themes were methodologies, contribution to knowledge, subject matter and item type.” (page 1 abstract, lines 31-35).

Comment 7. I do not see how the archiving of research output is linked to abstracts. Wouldn’t it make more sense to examine archiving practices of personnel rather than abstract in the output?

  We hope these words answer your question, “The abstract is an important part of the metadata when an Arts Practice-Based Output (APBO) is deposited on a repository. APBOs are non-traditional item types resulting from creative/artistic research processes. Examples include exhibitions, artefacts and digital videos.” (Page 1 abstract lines 13-16).

Comment 8. But you use the ‘300-word supporting statement’ in the abstract.

Thank you this was resolved by changing word in abstract.

Comment 9. I am glad the rationale is appropriate and presented clearly. I would suggest to take another look at the abstract. That is not capturing this rationale sufficiently.

Thank you for this advice, we have rewritten the abstract to match the introduction. “Through this qualitative systematic review, the authors ask to what extent is the 300-word abstract fit for purpose in representing art and design practice-based research outputs on small and/or specialist institutional repositories? The abstract is an important part of the metadata when an Arts Practice-Based Output (APBO) is deposited on a repository. APBOs are non-traditional item types resulting from creative/artistic research processes. Examples include exhibitions, designs and digital videos.  Little is known about how effectively these abstracts communicate research processes and insights across the art and design sector.  This study aims to investigate how well the abstract communicates information about the arts practice-based research through a systematic review of APBOs. The eligibility criteria for inclusion in the review were: APBOs must be from the date range January 2021 to January 2024; be an item type where the 300-word abstract is required; the statement must be part of the publicly-available metadata for the item and outputs are practice-based and from the art and design field. The date range (2019-2024) was employed because during this time APBOs had gained recognition in the wider research environment. APBOs from the reviewers’ institutional repository were not included in the study to avoid bias that could skew the results of the review. The data repositories from small and/or specialist higher education institutions in the United Kingdom were searched for outputs which appeared to meet the eligibility criteria. These types of institution prioritise and produce more of these output types. A quality tool appropriate for creative/artistic research was applied to the identified dataset of APBOs. The resulting 27 APBOs’ 300-word abstracts were analysed using a thematic approach. Findings suggest that the 300-word abstracts contained information about the quality indicators such as whether the project got funding; the identities of prestigious collaborators and/or dissemination vehicles and the international recognition of the research. Other identified themes were, methodologies, contribution to knowledge, subject matter and item type.” (page 1 abstract lines 11-35)

Comment 10. Small and/or specialist institutions in the UK generally have fewer than 3000 students and many have specialist provision in a particular subject area such as the arts, music, or teacher training (Bols, 2023)” = And how does that specialist provision translate to the abstracts?

A sentence has been added to clarify how specialist provision influences the prevalence and characteristics of APBO abstracts in the repositories studied.

‘This specialist provision is reflected in the types of research outputs deposited in their institutional repositories, with a high proportion of arts and design programmes generating practice-based research outputs (APBOs) whose abstracts foreground creative processes, modes of dissemination, and disciplinary contexts specific to these fields.’ (page 3 para 1 lines 102-109)

Comment 11. This argument is invalid. Other research is also transparent and systematic..

Thank you for this observation, we have discussed further the attributes of systematic review,

“This approach was chosen because it  is a method whereby the reviewers could systematically search, critically appraise, and synthesise the repository data (Temple University Libraries, n.d).  The process is also accountable and replicable and can be updateable to ensure reports reflect current research (EPPI, 2019).  “ Page 3, para 2 lines 112-115).

Comment 12. The text on page 2 is lacking coherence (these are all separate bits of information). Can you aim for more coherence? This would also strengthen your argumentation.

We have revised the Introduction to improve coherence and strengthen the argumentative flow. In particular, we have clarified the conceptual progression from defining arts practice-based research outputs, to outlining their implications for institutional repositories, and finally to establishing the methodological rationale for the study. These revisions were implemented through clearer signposting and transitions between paragraphs, and the section has been updated accordingly.

‘These characteristics of arts practice-based research have direct implications for how such work is represented, discovered, and assessed within institutional research infrastructures.’ (page 3, para 1 lines 83-85)

‘Given this focus on repository-based representations of practice-based research, a systematic review was identified as an appropriate method for examining patterns across institutions and outputs.’ (page 3, para 2 lines 110-112).

Comment 13. See my comment about reporting on a time frame. Ideally you list the month as well. 

This has been done. 

Comment 14. Can you list examples of those item types?

Thank you for this request examples are given, “The item types searched included: artefact, show/exhibition, composition, performance, image, digital video, audio, and creative project. (page 8, para 1 lines 307-309).

Comment 15. I am also wondering why you list exclusion criteria here rather than in the methodology. List examples of practice-based research (be specific enough). You are listing your review as being transparent but the information presented here isn’t detailed enough.

We thank the reviewer for this comment. We briefly included the inclusion criteria here so that an overall summery of the article is presented  for reader as part of the introduction. Later it is explained in more detail, in the Materials and Methods section, eligibility criteria are presented in relation to the review process and screening stages. We therefore consider the information provided to be sufficient and appropriately located for the purposes of transparency, and no changes have been made at this point.

Comment 16. Thus, this is an exclusion criterion.

Yes, this constitutes an exclusion criterion applied at the first stage of the study. Outputs from the reviewers’ institutional repository were intentionally excluded in order to mitigate potential bias and conflicts of interest. This decision was made to preserve the independence and integrity of the systematic review and is consistent with good practice in qualitative review methodology.

Please see in the materials and methods section “As previously mentioned, outputs from the reviewers’ institutional repository were not included in the study to avoid bias that could skew the results of the systematic review” (page 7, para 5 line 300-302).

Comment 17. The header “Context” comes as a surprise. Is this your literature review? If so, please display a more informative header (and subleaders).

Thank you for your comment we have changed the title to give more clarity.“Context of APBO policy and practice in the UK “ (page 4 line 152).

Comment 18. After the “et al” you need to insert a full stop/period. Thus: et al. Apply this consistent throughout your manuscript.

Thank you for observing this error. This has been changed  throughout the article.

Comment 19. Avoid back-to-back brackets: “(SAR) (ARMA, nd; UKRN, 2025; Drake, 2025)” = (SAR; ARMA, nd; UKRN, 2025; Drake, 2025).

The acronym SAR has been deleted as it is unnecessary.

Comment 20. Three or more authors can be listed with the fist author + et al.

Thank you for your observation, this has been amended in the text. 

Comment 21. This hyphen is not an em dash. Please revise. In a similar vein, p.12.375 does not display em dashes (these are em dashes). Please revise. Check the remainder of your work to correct this.

The hyphens have been changed to an emdash where appropriate throughout the document. 

Comment 22.To what unlawful or harmful materials are you referring to? Moreover, how did you check this? Walk the reader through the quality checks.

Thank you for your question. We have rewritten the point,

“The data gathered went through two quality checks before being included in the dataset, including filtering any unlawful or harmful material present in the repository records. Examples would include breaches of the UK Equality Act 2010. No in-stances of unlawful or harmful material were found in the data collected.” (page 7 para 2 line 280-284)

The quality checks are described in more detail in section 3.7. Quality assessment (CASP check list modified) and ethics is in question 5. (Page 8 para 4 line 321-396)

Comment 23.This time frame is different than the one you describe earlier in your work. That is why I suggested writing the years rather than a description ‘of the last five years’. Please revise accordingly.

The time frames have been amended – 1 January 2021 to 1 January 2024. (Page 7 para 5 line 297).

Comment 24. The in- and exclusion criteria have to be mentioned in the methodology. As I stated before, you will need a lot more detail to describe these criteria. You current version isn’t transparent enough. In addition, for systematic reviews you present both inclusion AND exclusion criteria.

Thank you for this advice, we have re-written section 3.7 to show the decision making process. “A tool was created to assess the quality of the remaining database of outputs drawing on the CASP. Each reviewer independently proposed a series of questions in a shared document to guide the assessment and ensure the outputs met an appropriate standard of quality. These questions were informed by the structure of the CASP checklist for qualitative studies (CASP, 2018) but modified to reflect the specific characteristics of practice-based art and design research.  This modification involved removing questions that addressed general qualitative methods that were based on participant contributions because these were not relevant to practice-based arts research. The term ‘valid’ was replaced with the terms authentic/trustworthy/ and meaningful because these are attributes associated with practice-based arts research (Leavy, 2015; Sinner et al., Bassey).  The three reviewers met to reach a consensus on the final checklist, agreeing on seven questions and to support consistency and transparency in the application of the checklist, the reviewers agreed shared interpretive guidance for each question: 

1. Are there clear lines of inquiry or research questions? Yes/no/cannot tell

An important  quality of research is that it is driven by clear research questions and/or lines of inquiry. Clear lines of inquiry or research questions’ were identified where the abstract explicitly stated, or clearly implied, a guiding research question, problem, or investigative focus If the research question(s) were clearly explained  then the APBO would be scored 2 (yes). If the research questions were unclear and difficult to identify the score would be 1 (cannot tell). If there were no research questions mentioned in the abstract then the APBO would be scored a 0 (no).

2. Have the research method/s been clearly explained? Yes/no/cannot tell

Question 2 referred to whether the abstract described how creative practice functioned as a method of inquiry.  A good understanding of the methods employed in the research means that readers know how the work can be repeated if necessary. They can also support the evaluation of how meaningful the insights are in relation to the method.  If the research method were clearly explained  to the reviewers, then the APBO would be scored 2. If the method is not written in a transparent way and is difficult to understand then the APBO would be scored a 1 . If the abstract only described the topic  and not the method then the APBO would be scored a 0.

3. Have the research insights been clearly articulated? Yes/no/cannot tell

Whether or not a contribution to knowledge has been made indicates the quality of research underpinning an output. Research insights that are clearly articulated required explicit statements of what was learned, revealed, or generated through the research process/method.  If the research insights were to the reviewers, then the APBO would be scored 2. If the insights were confused and not expressed well then, the APBO would be scored a 1. If the abstract only described the process and made no attempt at explaining the insights then the APBO would be scored a 0.

4. Are the insights authentic/trustworthy/meaningful? Yes/no/cannot tell

Authenticity and trustworthiness indicate that any stated insights were grounded in the described practice and context rather than asserted without support. If the insights were reasonable and the reader could see how they had been derived from the method and the context in which the research was undertaken then the score would be 2. If the links between the insights and the method were not made explicit then the score 1 was given. If the insights were not present or were not derived from the method that was described then the score given would be 0.

5. Does the research raise any ethical concerns? Yes/no/cannot tell

Ethical consideration is part of the rigour of an output. Ethical considerations were noted where relevant, particularly in relation to collaboration, representation, or engagement with participants or communities. To ascertain whether or not ethical research had been undertaken the abstract and related materials were read. A score of 2 was given if there had been an explicit mention made about ethics in the record. If the discussion about ethics was not present then the score 1 was given. A score of 0 was attributed  to the APBO if there were clear breaches in ethical practice apparent in the work and these had not been addressed.

6. Are there associated materials that are accessible and support the abstract, or is there just a metadata record? Yes/no/cannot tell

In order to judge the quality of the APBO there needed to be accessible materials (images, films, additional texts, websites) available through links in the repository record. Associated materials were considered accessible where links were functional and provided meaningful contextual support. If there were accessible materials  that could be viewed by the reviewers then a score of 2 was given. If the links did not work or additional passwords were required to view the materials then a score of 1 was ascribed to the output. Where there were no additional materials and the APBO was only recorded as a meta-data field then the APBO was scored a 0.

7. How well does the output’s method and/or insights transfer to other artistic or scientific practices, pedagogical contexts, and an informed readership? Yes/no/cannot tell

This question ascertains whether the research design and or the contribution of knowledge could be the starting point or basis for other work done in similar of different disciplines, and therefore has value.  Transferability referred to whether the abstract indicated potential relevance beyond the specific case under review.  The decision to score the APBO as 2 was based on statements that described the potential influence of the work. If the APBO was unclear the score of 1 was given and if there was no mention of how the work could be applied in future research the score of 0 was given. “(Page 8 para 4 line 321-396).

Comment 25. Moreover, on p.6.241 you refer to duplications = duplicates. Furthermore, you need to list how many duplicates you excluded, in light of transparency. Please add this information.

Thank you this has been done and the sentence, “At this stage duplicates were also removed . In total, there 2.8% of outputs collected were duplicated items (n=10).” Has been added. (page  8 para 2 lines 14-15).

Comment 26.The checklist (p.6.246) also requires more detail. Each question should have a clarification. For example, the clear lines of research questions: is a RQ mentioned? Moreover, how do you determine if research insights have been clearly articulated? Again, this requires more detail to contribute to scientific rigor and the transparency you are talking about.

We thank the reviewer for this comment and agree that greater clarity is required. We have expanded Section 3.7 to explain how each checklist question was interpreted and applied, including how the presence, absence, or ambiguity of key research features (such as research questions, methods, and insights) informed the assessment. This additional detail strengthens transparency and clarifies how judgements were made across the dataset. Please see section 3.7 (Page 8 para 4 line 321-396).

Comment 27. Are you referring to the checklist you adopted (earlier)? If so, write that. I would suggest to merge these sections for more clarity.

Thank you we have made a series of changes to this section. At the beginning of section 3.8  we have added the words, “The adapted CASP tool was devised to include or exclude outputs from the study based on the reviewers’ total scores for each APBO.” (page 10 para 2 lines 398-399 )

We have also included a more detail diagram Table 1: A table of the scoring for each CASP question per reviewer per APBO abstract  (page 10 line 406).

Comment 28. This text is redundant if you present Table 1. The reviewers probably have more experience than other (external) reviewers. It would be relevant to ‘test’ your coding scheme after extended revisions in the form of more details.

This section has been rewritten and more detail has been described in Table 1: A table of the scoring for each CASP question per reviewer per APBO abstract. 

Comment 29. Figure 1 Please insert the exclusion criteria in each stage to provide a more transparent workflow, as you have emphasized in the first pages of your manuscript.

Thank you for your suggestion we have moved the table to the end of section  3.8 and included the text above it,

“The diagram below (Figure 1) represents the different stages of the systematic review undertaken. The exclusion criteria for the Data Collection stage can be found at sub-section 3.5 of this article. The exclusion criteria for the Systematic Review stage can be found at sub-sections 3.7 and above in 3.8.” (page 13 para 2 line 444-447) The diagram was amended for clarity (page 14 line 448)

Comment 30. This is not an em dash. Please revise.

This has been done. 

Comment 31. Based on Table 2, I notice a major difference between the three reviewers: R1 said ‘yes’ to every output with exception of one. Does this person have more experience than the others? With revised in- and exclusion criteria, you were probably able to capture some disagreement beforehand. I am a bit surprised that the criteria you follow are general (and, therefore, not in line with how systematic reviews have to be conducted).

Thank you for your observation and questions. In order to clarify the process we have included a new table: Table 1: A table of the scoring for each CASP question per reviewer per APBO abstract (page 10 line 406).

Comment 32.The types of information that was considered” = ‘types’ is plural. 

Changed  ‘was’ to ‘were’

Comment 33. The themes need more explanation. It is a bit odd that you present this in your discussion rather than in your methodology. If you restructure your manuscript, you can place your results alongside existing research and actually discuss your results/findings. This would also allow for more engagement with existing research.

We thank the reviewer for this comment. We have expanded the description of the themes in the Results and Analysis section to provide clearer definitions of each analytic category.  We need to show the qualitative approach to and process of thematic analysis.

The words were added, “The themes agreed between the three researchers were identified as:

• Quality indicators statements, referring to references to funding, institutional partnerships, prestige of venues, awards, or forms of recognition used to signal research quality;
• Methodologies, encompassing descriptions of how creative practice functioned as a method of inquiry, including processes, collaborations, and experimental approaches;
• Contribution to knowledge within the arts and in other disciplines referring to how abstracts articulated new insights, understandings, or relevance beyond the immediate project;
• Subject matter and item type, capturing both the thematic focus of the work and the form of the output (e.g. exhibition, performance, artefact). “ (Page 17 para 3 lines 483-495)

We have also clarified the distinction between analysis and discussion by explicitly indicating that the themes are identified through thematic analysis and are subsequently interpreted in relation to existing literature in the Discussion section.  (page 17 para 4 lines 496 -501) This strengthens transparency while retaining the overall structure of the manuscript.

Comment 34. festivals aren’t just quality indicators, they” = Do not use contractions.

We have re-written the phrase. 

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

The introduction still suffers from incoherence. A paragraph is at least three sentences. As a result, information needs to be integrated. The use signal words and/or repeating information will help you with that.

Information comes as a surprise, again due to fragmentation. 

Keep the term for abstract consistent (the term 'supporting statement' becomes redundant'). 

This construction isn't correct: "The reviewers were one researcher". Moreover, later you talk about two reviewers? 

"ll three reviewers then met to dis-319 cuss the outcome" = And what happened after? 

"Data was collected" = Data were collected (data, in this case, is plural).

The questions as headers do not really work. It contributes to the fragmentation that is present throughout the manuscript. It also undermines a compelling story. It is common to add these questions (+answers) to an Appendix. 

The use of the English language needs attention. Some sentences are oddly structured. It can be relevant to contact a Native English speaker to improve the use of the English language, or ask a senior colleague to go over your manuscript (I would list this as a requirement for publication).

I am unsure about the first recommendation. This is something that art researchers already need to possess. If not, intervention earlier in their careers is then needed. The quality assurance you list under 3 is common for practice in various studies and solidified during PhD trajectories. 

The timeline from the abstract does not match the time line from the abstract. 

In your figure it is still unclear how you ent from 156 to 27 abstracts. This is the reason why such a figure needs to be detailed--including the reasons as to why you exclude the work. I do appreciate the amountof details in the previous section (table 1). 

 

Author Response

Comment 1.

The introduction still suffers from incoherence. A paragraph is at least three sentences. As a result, information needs to be integrated. The use signal words and/or repeating information will help you with that. Information comes as a surprise, again due to fragmentation.

Thank you for this observation, we have decreased fragmentation  in the introduction. (Page 2-4, line 41-149)

Comment 2. Keep the term for abstract consistent (the term 'supporting statement' becomes redundant').

Thank you for this comment. We have hanged statement to abstract when discussing ABPOs on repositories. (When writing about REF guidance we need to keep the word statement).

Comment 3. This construction isn't correct: "The reviewers were one researcher". Moreover, later you talk about two reviewers?

Thank you for your comment. Added the words,  "The group of reviewers comprises …." and  "Two of the reviewers (the third reviewer carried out the quality check described in 3.6"  (Page 7, para 3, line 291 and Page 8, para1, line 311)

Comment 4. "All three reviewers then met to discuss the outcome" = And what happened after?

Thank you for this question, we added, " All three reviewers then met to discuss the outcome, where agreement was reached over the third reviewer’s assessment that it was correct to remove the 19 from the sample therefore the dataset remained at n=156." (Page 8, para 2, lines 319-321)

Comment 5. "Data was collected" = Data were collected (data, in this case, is plural).

Thank you for this observation we have changed ‘was’ to ‘were’ (Page 4, para 1, line 129)

Comment 6. The questions as headers do not really work. It contributes to the fragmentation that is present throughout the manuscript. It also undermines a compelling story. It is common to add these questions (+answers) to an Appendix.

Reference to Appendix B for list of questions. (Page 8, para 3, line 334)

Questions added to the paragraph explaining each of the questions and how they were scored. These were requested by the other reviewers. (Page 8, para 4, line 338; Page 8, para 5 line 347; Page 9 para 2 line 357; Page 9, para 3, line 366; Page 9, para 4, line 375; Page 9, para 5, line 386; Page 9, para 6, line 395.

Comment 7. The use of the English language needs attention. Some sentences are oddly structured. It can be relevant to contact a Native English speaker to improve the use of the English language, or ask a senior colleague to go over your manuscript (I would list this as a requirement for publication).

The manuscript has been read by a senior scholar and small grammatical errors have been rectified throughout the document.

Comment 8. I am unsure about the first recommendation. This is something that art researchers already need to possess. If not, intervention earlier in their careers is then needed. The quality assurance you list under 3 is common for practice in various studies and solidified during PhD trajectories.

Thank you for your queries, we have added the following clarifications , 

"The findings demonstrate a need for PhD programmes to train arts researchers in writing APBO abstracts. They indicate a need for targeted continuous professional development (CPD) for both early career arts researchers without PhDs and repository managers who are responsible for writing, reviewing, and curating 300-word abstracts for APBOs. While arts researchers often possess highly developed skills in articulating their work through artistic, curatorial, and critical forms, the writing of 300-word APBO abstracts constitutes a specific research genre shaped by repository infrastructures, REF requirements, and open access mandates." (Page 22, para 2, lines 696-705 )

"It is also important to recognise that arts research careers are frequently non-linear. Many academics enter higher education following extended professional practice as artists, designers, or curators and may not hold a PhD, or may undertake doctoral study later in their careers. Even where doctoral training has been completed, abstract writing for practice-based outputs—particularly in forms aligned with REF assessment and repository metadata—is rarely an explicit or consistently embedded component of arts PhD programmes. For these reasons, CPD represents an appropriate and inclusive intervention that complements, rather than replaces, early-career research training." (Page 22, para 2, lines  710-717)

"Quality assurance processes are well established within institutional repositories and routinely applied to text-based research outputs. However, the findings of this review indicate that such processes are not always consistently adapted to the specific characteristics of APBOs, which often involve multiple files, non-textual materials, and process-based documentation." (Page 23, para 4, lines 753-758)

Comment 9. The timeline from the abstract does not match the time line from the abstract. Make sure the timeline is consistent

Thank you for your observation, we have amended to January 2019 to January 2024. (Abstract, Page 1, line 21)

Comment 10. In your figure it is still unclear how you went from 156 to 27 abstracts. This is the reason why such a figure needs to be detailed--including the reasons as to why you exclude the work. I do appreciate the amount of details in the previous section (table 1).

Thank you for your request, figure 1 has been amended with decision-making process included. (Page 14, line 456)

Author Response File: Author Response.docx