Changing the Academic Gender Narrative through Open Access

Round 1

Reviewer 1 Report

The authors of this article use publicly available datasets and run statistical analyses to explore the relationships between women in higher education and research output in open access. The article is well written and the authors have done good work in providing details of the data source. I think the size of the raw data and the amount of work put into the statistical analyses make this article a worthy contribution to the literature.

I have three minor issues for the draft:

1. Methods and data (P3, L111) This paragraph actually describe the method of literature review rather than data analysis. I think the readers will appreciate the authors making it clear in the first sentence or so.
2. Figure 2 (P8, L352) Why there is a slightly negative correlation between percentages of academic women and Green OA? It does not make sense as Green OA is free and I expect a positive correlation. The authors should address this question in the Discussion.
3. P10, L425. Why is this pattern only observed for Universities in Australia but not in the UK?

Author Response

Reviewer 2 Report

Summary: This is a fantastic paper. I was looking forward to reading it when I received the abstract and the paper itself was excellent. I am hopeful that it is published soon, so that I can share it with the people who I know will be interested in this that came to mind when I was reading it.

General Comments:

Introduction: Very thorough and clear positioning of the article and the Open Science evolution and terminology to make this approachable to a non specialist.

Methods and Data: I really appreciated the conversation around exclusions and limitations here - particularly around the use of binary data. Also, the noting of excluded contract types in the data set was very welcome - every element and the caveats it requires have been clearly thought through and the limitations surfaced excellently. This was also picked up further in the later section on "Name gender dismbiguation" (line 301-303 had me cheering that this is being acknowledged in work of this kind).

Existing Literature: The existing literature on this topic is limited, but the authors engaged with all the key outputs and authors. 

Conclusions: While there are undoubtedly many factors at play in this discussion and, as the authors suggest, many more avenues for exploration using the Institutional Demographic data is a new approach and adds new data into this field. 

Specific Comments: Table 1: "Advantage to Women" as a header didn't work for all the findings - consider changing to "Findings/Conclusion"? Also line 4 of the table "btw" should be in full. 

Author Response

Reviewer 2 comments                                      Responses

Reviewer 3 Report

The present article aims to build on previous studies on gender biases in research, to put forward different, and more nuanced, perspectives on gender and research performance to the dominant narratives: namely, that addressing gender biases within the framework of engagement with open research allows us move away from notions of 'excellence' and 'impact' and in doing so, reveal that women play, in fact, a leading role in open practices. The conclusions draw both from a review of existing literature and from analysis of data from Australian and UK universities related to engagement with open access.

This is a very interesting angle which, provided that it is supported by the results, could indeed have important implications for both the role of women in research, the frameworks we choose to address 'success' or 'progress' in research, and the role of open research in transforming the culture of higher education. 

 Precisely because this is a powerful idea with interesting implications, I feel that elements of the structure and, particularly, the analysis of the data should be reconsidered. Below I outline areas of strength and suggestions for improvement.

Structure

• The review of the existing literature on the deficit narrative, gender biases, and 'open advantage' for women is overall comprehensive. The authors expose the confounding factors that come into play in studies suggesting a disadvantage for women. When it comes to the 'open advantage', the authors acknowledge that the evidence from the literature is still limited. Perhaps a clearer distinction would be helpful here between studies that show women's authorship with 'open science' research (lines 218-227) and studies looking at 'Green' or Gold' activity; the latter may be much less compelling that the former: as the authors themselves acknowledge, 'Gold' engagement in particular is likely to be affected by institutional processes and wealth, and less so by women being more engaged with open access.
• The authors revisit these studies in the Discussion section (section 7). I feel that Table 1 might fit better in the literature review section earlier on, as it is really a summary; the studies can still be discussed in the end. 
• The article would also benefit from a distinction between the literature review (section 2, 111-118; sections 3 and 4) and the authors' own analysis of their data (section 2, 119-137; brief mention of the data during the literature review: 199-203). This would be a much clearer narrative if the review came first, exposed the gaps in existing research and paved the way for the methods and results. In short: - Introduction. - Literature review: a. the prevailing narrative and ongoing biases; the openness advantage - methods: paragraph on definition of women (102-109); paragraphs on analysis: 119-137); section on name disambiguation (section 5). Followed by Results (section 6) and Discussion (section 7).

Methods and analysis

• My next point is more of a conversation with the authors, with some suggestions, rather than a criticism of the study. One aspect that I find problematic (but acknowledge that the study can only be based on data currently available!) is the dependent variable chosen. Open research/open science, as the authors point out, is such a vast, diverse set of behaviours and practices (sharing data, sharing open educational materials, engaging in open education more broadly, citizen science projects, study preregistration, uptake of narrative CVs, study preregistration, open licensing...). Narrowing the focus on OA publishing may be the sensible first step to take, given the scarcity of data/difficulty of identifying other practices in other areas; but still, in some ways in weakens the argument, as OA publishing is confounded by academic status and wealth (237-238; 263; 267-268) and Green OA by the presence of mandates. Ideally (perhaps in future studies?) the focus should be on the wider uptake of open practices by women, beyond open access. But for the time being, I was wondering if the authors could look specifically at Gold OA versus hybrid OA? It would also be interesting to look at women's engagement with fully OA journals/more innovative 'Diamond' journals. It may be possible to get the data from Unpaywall and information on OA journals from DOAJ.
• Name gender disambiguation: the arguments put forward by the authors are compelling: there are indeed limitations to using disambiguation methods. Still, I have some reservations/questions around the alternative method used by the authors. Instead of looking at numbers of women that engage with Gold OA, the analysis looks at demographic data: ranked data showing percentages of women in different institutions. Essentially, for each data point (article) we are looking at the likelihood that the paper was authored by a woman, instead of looking at whether the article was indeed authored by a  woman.  This likelihood will decrease as the percentage of women in an institution moves nearer 50%; thus weakening any conclusions: what we can say from the data is that institutions that have more women than men produce more or fewer Gold papers, for example. It is possible that in an institution with 70% women most of the publications in the sample were in fact authored by men.
• Further, there are confounding factors that do need to be taken into account: namely, subject (discipline) and publication volume. Correlations alone do not tell us much.
• I also have questions on the choice of 'events' as a variable, as these are more measures of impact. What would we expect from these results, and what do the results tell us about the role of women in open research?
• I wonder if the authors would consider using author disambiguation (perhaps in a future study?). Despite its limitations, this approach should allow a direct comparison between men and women; identification of articles authored by women; and the possibility of more robust analysis: possibly a logistic regression that uses subject (mentioned by the authors as a limitation: 463)  and institutional income (Appendix) as covariates.

Reporting and transparency

• Spearman correlations should have the p value reported
• Sample size should be reported: how many articles were included in the analysis? 
• How was each article credited to an institution? By corresponding author/first author? To all institutions listed in the affiliations (risking counting some publications more than once, e.g. if co-authored by two or more institutions included in the study?
• Data availability statement: this should include a statement about the dataset of publications from CrossRef/Unpaywall. At present it only includes the demographic data.

General/concept

The article makes a call for a transformed research culture, where openness is valued more than 'excellence' and where women, already practising openness (if the evidence is compelling) thrive. I couldn't agree more with this suggestion. The paper certainly opens an interesting conversation, and possibilities for further research. 

If the authors do have access to data (through the OKI project) that will help address the comments above, it would be definitely worth rethinking the choice of variables and analysis.

A clear hypothesis to test might be that in the UK there would be no difference in engagement between men and women because the 'stick' masks the 'carrot': in other words, mandates, transitional agreements and block funds give all genders no choice but to comply. It would be interesting to compare with Australian institutions with different processes in place. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 4 Report

The paper entitled “Changing the Academic Gender Narrative through Open Access” explores relationships and correlations between academic gender and research output from Universities in Australia and the United Kingdom. Although the aim of the study is very relevant, the analysis that is made is too superficial to warrant publication as it is.

The main concern arises from the statistical method used. Correlation, used by authors, measures the strength of a linear relationship between two variables and it can be a good starting point for research. However, an important pitfall of the correlation coefficient is that it cannot be interpreted as causal. It is of course possible that there is a causal effect of one variable on the other, but there may also be other possible explanations that the correlation coefficient does not take into account. Authors should adjust for such confounding effects, for example by using multivariable regression.

To the above it should be added that rank correlation coefficients estimated by authors between percentages of women in academic roles and (Gold and Green) Open Access output in the United Kingdom and Australia universities are weak or negative. These results question the following statement made by the authors: “Our analysis suggests that by choosing the open access route women can change the narrative of deficit and build advantage in academia.”

In addition, there are some more comments for the authors to refine their work:

• Contents included in Section 5 (Name gender disambiguation) would have more sense if they were referred to in Section 2 (Methods and data).
• Section 6 (Correlation and statistics analysis) should be consistent with the aim of the article. Authors analyze a set of indicators not directly related to the aim of the study, that is, the open access route of female academics.
• Section 6 (Correlation and statistics analysis) should be exclusively dedicated to the analysis of findings, keeping the discussion in light of other studies in the Discussion and Conclusions’ section.
• Finally, the Discussion and Conclusions’ section should explain whether the research questions or Hypothesis have been confirmed or refuted. For this aim, I would recommend the authors to first include these in the article, just before the Method section, and then rewrite the Conclusions accordingly. This section should also include a thorough discussion of findings based on previous studies.

Author Response

Please see attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Thank you for addressing the previous comments.

I find the structure and the presentation of the arguments much clearer; particularly the aims of the study to approach gender narratives in academic research from an Open Research perspective.

As mentioned in the previous review, the implications of showing that the narrative around women's contributions to research is different when examined within a different framework (i.e. when we think of open and collaborative research rather than 'excellent' research) is potentially a powerful one. The paper discusses literature pointing to this direction. The paper also presents an analysis that aims to lend evidence to this position, too.  

This is where I am still hesitant around this paper. I understand the arguments against using gender disambiguation (that section of the paper is a very compelling one) but am still uncomfortable around the alternative method used (i.e. the demographic data) and around the correlations analysis. I am not convinced that this is a sound approach in terms of design and analysis, which in turn makes the conclusions less compelling, too. Having said that, you make it clear in the discussion that you are aware of this and that the aim of article is partly to provoke further discussion and further research on the topic.

This, as well as the fact that you have shared the underlying data, makes me think that the paper should be published: particularly with it being open access, and with reviews and data being open, too, it should open the way to some fascinating discussions and to further work on an important topic. 

Thank you again for the opportunity to read and discuss it.

Reviewer 4 Report

I would like to thank the author for reviewing the manuscript.