A Re-Evaluation of the Grievance Studies Affair

Round 1

Reviewer 1 Report

This is an important article which maintains that the authors of what is generally referred to as the "Grievance Studies" hoax falsely claimed that they had managed to get sections of Hitler's Mein Kampf published in, or reviewed by, academic journals, by using word substitution of "Jews" for "grievance studies" terms. The author also documents other exaggerated or false claims made by the hoax authors in relation to the public reporting of their academic methodology, in which they likewise claimed (falsely) to have had accepted for publication papers containing absurdities, or containing endorsements of abhorrent practices (e.g., chaining students to the floor of a classroom). The article represents a further exploration--through careful documentation--of the falsity of the public reports made by the hoax authors, continuing a series of articles on the topic by various scholars, which the author scrupulously documents in the article. It is, in short, an excellent article, meeting most of the highest standards of argumentation and evidence. Furthermore, it states its case in relation to inaccurate reporting of the hoax methodology by its perpetrators in terms that are judicious and measured.

Unfortunately, I have to argue for a minor "revise and resubmit" in this case. There are two grounds, one stronger, and one weaker. The author, who seems to fully understand the importance of intellectual integrity and the absolutely rigorous presentation of a research case will, I think, understand the need for these changes, especially in an instance where the legitimacy of humanities inquiry has been (quite unfairly and misleadingly) placed in question by the hoax authors.

The stronger ground is this. The author cannot, in my view, simply assert that the hoax perpetrators suffered from false memory syndrome. Such an explanatory claim must be demonstrated, and such a demonstration takes more than a general citation from Anderson and Hanslmayr. The assertion may be true, and the author may be an expert in evaluating such things, but assertions, especially about unobservable processes inside persons' memories, cannot stand in such an article. The claim has to be established in relation either to participant testimony, or it has to be rendered highly plausible by a differential elimination of other possible explanations. 

However, at present, it is an unnecessary speculation about a causal explanation for inconsistencies between the hoax papers and the reporting of the hoax, and I think that it should be cut from the paper. I therefore also think that it does not belong in the title of the paper. The originality claim of the paper is satisfied by its new discussion of certain claims about the hoax method, especially the Mein Kampf papers, and this should be reframed as the centrepiece of the article. An explanation of the conduct of the hoaxers is, in my view, a different paper, and really one that belongs in a psychology (or similar) journal, where it can receive the expert scrutiny belonging to that field which it deserves.

The minor ground for revision concerns the ever-so-slight disjunction between the substance of the paper's topic (hoaxers who just made up "evidence" in support of their claims) and the methodology of the present article (a patient discussion of certain inconsistencies, which, however, often relies on reporting the results of cheat-buster software, or frequency counts, or reporting elements of specific sentences). My discomfort is this. I don't think that this author is making anything up. I think that the author's statements are true reports. But the general reader has a right, and in this case, perhaps, a need, to make up their own mind. Specifically, I think that the reader has a right to inspect selections of text that provide context, so as to satisfy themselves beyond reasonable doubt that, for instance, the hoaxers used anodyne sentences only vaguely reminiscent of Mein Kampf, and then falsely claimed that these were, in fact, sentences written by Hitler with "the Jews" removed and other groups inserted. 

I therefore recommend that in relation to every case study type, the author provide at least one extended quote of 40-50 words from the target text (and any relevant co-texts, eg, Mein Kampf), so as to, so to speak, saturate the space of proof in the mind of the reader. 

Author Response

The author cannot, in my view, simply assert that the hoax perpetrators suffered from false memory syndrome. Such an explanatory claim must be demonstrated, and such a demonstration takes more than a general citation from Anderson and Hanslmayr. The assertion may be true, and the author may be an expert in evaluating such things, but assertions, especially about unobservable processes inside persons' memories, cannot stand in such an article. The claim has to be established in relation either to participant testimony, or it has to be rendered highly plausible by a differential elimination of other possible explanations. 

However, at present, it is an unnecessary speculation about a causal explanation for inconsistencies between the hoax papers and the reporting of the hoax, and I think that it should be cut from the paper. I therefore also think that it does not belong in the title of the paper. The originality claim of the paper is satisfied by its new discussion of certain claims about the hoax method, especially the Mein Kampf papers, and this should be reframed as the centrepiece of the article. An explanation of the conduct of the hoaxers is, in my view, a different paper, and really one that belongs in a psychology (or similar) journal, where it can receive the expert scrutiny belonging to that field which it deserves.

Author’s Reply:

Firstly, thank you for the time you have taken on evaluating the paper.

It is reasonable to assume that there can only be two fundamental reasons for the mismatch between what the papers actually stated and how Lindsay et al. later described them. They either deliberately lied or they simply forgot what they had said. I was very wary of accusing the authors of outright deceit so I went with the misremembering explanation. Indeed, the onus must be on an accuser to show deceit as opposed to showing misremembering. So one can argue that the default position has to be the latter. I also do actually believe this. I do think that in their rush to get the story out they did not carefully check what exactly their own papers had stated. This misremembering account also explains why Boghossian did not seem to know how many Mein Kampf papers had been written when interviewed by Josh Rogan. There were three not two, as he stated (I now include this point in the revision). If the reviewer insists I will remove the speculation on how the exaggerations occurred. The revised ms is however very explicit in stating that the misremembering explanation is purely speculative. As requested, I have also omitted the misremembering aspect from the title. I have also removed it from the abstract. The idea now only appears in one section, rather than being present throughout. 

The minor ground for revision concerns the ever-so-slight disjunction between the substance of the paper's topic (hoaxers who just made up "evidence" in support of their claims) and the methodology of the present article (a patient discussion of certain inconsistencies, which, however, often relies on reporting the results of cheat-buster software, or frequency counts, or reporting elements of specific sentences). My discomfort is this. I don't think that this author is making anything up. I think that the author's statements are true reports. But the general reader has a right, and in this case, perhaps, a need, to make up their own mind. Specifically, I think that the reader has a right to inspect selections of text that provide context, so as to satisfy themselves beyond reasonable doubt that, for instance, the hoaxers used anodyne sentences only vaguely reminiscent of Mein Kampf, and then falsely claimed that these were, in fact, sentences written by Hitler with "the Jews" removed and other groups inserted. 

I therefore recommend that in relation to every case study type, the author provide at least one extended quote of 40-50 words from the target text (and any relevant co-texts, eg, Mein Kampf), so as to, so to speak, saturate the space of proof in the mind of the reader. 

Author’s Reply:

Interestingly, I have these quotes and sections requested as part of a full length book I recently completed entitled “The Grievance Studies Affair: The hoax that never was”. So I was able to easily insert these into the Mein Kampf section. I only mention the book to illustrate that I have spent a lot of time evaluating the whole Lindsay et al. project.

Reviewer 2 Report

 I appreciated the opportunity to review this paper. The main argument is that the Grievance Studies articles were misrepresented by the authors of the original articles, but the response to these publications has largely accepted this misrepresentation. I would like to note that as part of the the review process, I sought out and read the Grievance Studies papers that were referenced within this article, as I believed this to be an imperative part of preventing further misrepresentations from spreading. I have found that the current article accurately represents the content of the original Grievance Studies articles. 

Major notes:

1. While the Fat bodybuilding response section is accurate, it appears to me to be the weakest counter-argument/correction of all of those presented. For rhetorical reasons, I would strongly suggest shifting this around so it is the last one presented, or at least not the first. To explain a bit further, I think that, while entirely accurately presented in the paper, the differences between the terms "fat" and "morbidly obese" may not be as large in the mind of the average readers as the author(s) assume. In fact, the only difference in Lindsay et al.'s explanation and the paper in this case seems to be a difference in these terms. The current author seems to work in this area, based on an anonymized reference including in this section, perhaps skewing their idea of how well the average person understands these terms. While this change in terms is certainly problematic, my initial reaction was that this was a fairly small difference, and was underwhelming. I think I may have reacted differently if I'd read the other examples before this one, as they create a bigger impression. 

2. In the Baldwin section, the author challenges anyone to defy the comments included in the Hooters paper are problematic. Personally, I absolutely concur that the comments were problematic, but presumably there are people who are sexist and they may not find or be able to identify these comments as problematic. Over the years, I have occasionally heard people make just such comments, so I could imagine a theoretical reader who would not see the problem with such comments. I think this section might benefit from a slightly more contextualized and less all or nothing claim. For example, it might note that while some may consider such comments innocuous, many (most?) working in academia as journal editors would find this type of male behavior to be problematic and/or sexist. 

3. On page 9, the author states and then goes on to discuss the idea that Lindsay et al. "did not deliberately fabricate the story of what their own hoax papers said." However, there doesn't seem to be any strong evidence presented to back this up, and I'm honestly not sure why we should assume they didn't do this deliberately in light of their deliberate fabrication of this entire body of research. They seem to have lost the benefit of the doubt in that regard, so I think discussion either needs to be reworked or further evidence needs to be presented. At the most simple level, perhaps you could simply state you're giving them the benefit of the doubt rather than making the claim that they didn't deliberately do this. 

Minor notes:
1. On page 1, a paragraph seems to be split in two by another paragraph beginning with "1) Although..." I believe this should be formatted as a footnote possibly? In any case, it needs to be corrected. 

2. I believe a word may be missing in the last sentence to begin on page 4. "Although the method does not seem to have developed in..." should perhaps be "Although the method does not seem to have been developed in..." or some other variation. 

3. On page 8, the author(s) write that "...there were note three people working on the hoax project. There were 83." While I understand the important point being made, this is a rhetorical sleight of hand much like those the author accuses Lindsay et al. of. There were not actually 83 people working on the project, bur rather the equivalent of 83, as explained later. This should be clarified here. 

4. I'd really like to see a link to the archive of the materials posted by Lindsay et al. for quick reference. 

I believe these changes will significantly improve the paper, and that this will add important context to the understanding of this series of events. 

Author Response

1. While the Fat bodybuilding response section is accurate, it appears to me to be the weakest counter-argument/correction of all of those presented. For rhetorical reasons, I would strongly suggest shifting this around so it is the last one presented, or at least not the first. To explain a bit further, I think that, while entirely accurately presented in the paper, the differences between the terms "fat" and "morbidly obese" may not be as large in the mind of the average readers as the author(s) assume. In fact, the only difference in Lindsay et al.'s explanation and the paper in this case seems to be a difference in these terms. The current author seems to work in this area, based on an anonymized reference including in this section, perhaps skewing their idea of how well the average person understands these terms. While this change in terms is certainly problematic, my initial reaction was that this was a fairly small difference, and was underwhelming. I think I may have reacted differently if I'd read the other examples before this one, as they create a bigger impression. 

Author’s Reply:

Firstly, thank you for the time you have taken on evaluating the paper.

Interestingly, given the reviewer’s comment, the description of each article/claim was intentionally ordered from the least exaggerated first to the most exaggeration last. My rationale was to progress the narrative, building it up to the most extravagant claim. I must admit that I do still like this progression and, unless the reviewer insists, I have largely kept the order. I have however explicitly stated this in the Introduction. The text reads: “The exaggerations are presented in reverse order of degree. Thus, whereas the Fat Bodybuilding and Progressive Stack papers were only subtly, but importantly, exaggerated, the notion that such a large number of papers can be generated in the humanities within a matter of months was the largest exaggeration”. I do agree that the difference in terms in small but do think this subtle change from ‘fat’ to ‘morbidly obese’ is very illustrative of an exaggeration. Note that when thinking about the rank order of exaggerated claims I decided that the Hooters paper should go after the Progressive Stack paper.

In reference to one other point the reviewer made, although I have published one previous paper on the Grievance Studies affair (plus a Reply article), I don’t actually work in the field of body representation. I am a vision scientist but undertook a lot of research on the whole Lindsay et al. project.

2. In the Baldwin section, the author challenges anyone to defy the comments included in the Hooters paper are problematic. Personally, I absolutely concur that the comments were problematic, but presumably there are people who are sexist and they may not find or be able to identify these comments as problematic. Over the years, I have occasionally heard people make just such comments, so I could imagine a theoretical reader who would not see the problem with such comments. I think this section might benefit from a slightly more contextualized and less all or nothing claim. For example, it might note that while some may consider such comments innocuous, many (most?) working in academia as journal editors would find this type of male behavior to be problematic and/or sexist. 

Author’s Reply:

Yes, I was looking at the described behaviour through my sensitive academic lens. I have now revised the paragraph in accordance with the reviewer’s suggestion. It now reads, “It is not at all surprising that the reviewers and editors of the journal accepted the manuscript. Whilst some may consider the kind of behaviour and comments innocuous, many (perhaps most) working in academia as journal editors would find these problematic and/or sexist.

3. On page 9, the author states and then goes on to discuss the idea that Lindsay et al. "did not deliberately fabricate the story of what their own hoax papers said." However, there doesn't seem to be any strong evidence presented to back this up, and I'm honestly not sure why we should assume they didn't do this deliberately in light of their deliberate fabrication of this entire body of research. They seem to have lost the benefit of the doubt in that regard, so I think discussion either needs to be reworked or further evidence needs to be presented. At the most simple level, perhaps you could simply state you're giving them the benefit of the doubt rather than making the claim that they didn't deliberately do this. 

Author’s Reply:

Reviewer 1 also raised this issue. As I stated in reply to them, it is reasonable to assume that there can only be two fundamental reasons for the mismatch between what the papers actually stated and how Lindsay et al. later described them. They either deliberately lied or they simply forgot what they had said. I was very wary of accusing the authors of outright deceit so I went with the misremembering explanation. Indeed, the onus must be on an accuser to show deceit as opposed to showing misremembering. So one can argue that the default position has to be the latter. I do actually believe that in their rush to get the story out they did not carefully check what exactly their own papers had stated. This misremembering account also explains why Boghossian did not seem to know how many Mein Kampf papers had been written when interviewed by Josh Rogan. There were three not two, as he stated (I now include this point in the revision). The revised ms is however very explicit in stating that the misremembering explanation is purely speculative, only a possibility. It has also been removed from the title and abstract. As advised by the reviewer, I have now included the point about giving the author’s the benefit of the doubt.

Minor notes:
1. On page 1, a paragraph seems to be split in two by another paragraph beginning with "1) Although..." I believe this should be formatted as a footnote possibly? In any case, it needs to be corrected. 

Author’s Reply:

Now corrected.

2. I believe a word may be missing in the last sentence to begin on page 4. "Although the method does not seem to have developed in..." should perhaps be "Although the method does not seem to have been developed in..." or some other variation. 

Author’s Reply:

Now inserted.

3. On page 8, the author(s) write that "...there were note three people working on the hoax project. There were 83." While I understand the important point being made, this is a rhetorical sleight of hand much like those the author accuses Lindsay et al. of. There were not actually 83 people working on the project, bur rather the equivalent of 83, as explained later. This should be clarified here. 

Author’s Reply:

Yes, this was not literally correct. I included it for effect and intrigue. The sentence now includes the word “equivalent” and reads, “There was the equivalent of 83”.

4. I'd really like to see a link to the archive of the materials posted by Lindsay et al. for quick reference. 

Author’s Reply:

The link is now included in the section on why the exaggerations occurred.

I believe these changes will significantly improve the paper, and that this will add important context to the understanding of this series of events.