A Comparative Corpus Study of Epistemic Culture and Discourse in Computer Science, the Humanities, and the Digital Humanities

Abstract

This article advances understanding of the interdisciplinary components of the digital humanities, and how this hybrid field relates to some of its contributing epistemic traditions. It builds its approach by drawing upon recent empirical work on the values expressed within AI research, extending from that work’s focus on justificatory chains in computer science scholarship to explore these and related discursive strategies in a comparative, computational context across corpora drawn from computer science, traditional and digital humanities. It also looks at how these different disciplines position themselves epistemically, agentially and discursively. In this way, the paper presents a comparative perspective on the values and approaches, in particular, the divergences among these fields, and points toward some of the specific areas needing to be addressed if they are to work more productively together in the development toward more socially and culturally integrated engineering practices.

1. Introduction

Journalists [1] and scientific researchers [2] alike are calling for increased participation from humanistic fields in the development of artificial intelligence (henceforth AI), an activity more commonly understood as anchored in the field of computer science. In theory, this integration makes sense, as AI applications are having an ever more marked impact on everyday cultural practices. Interdisciplinarity is always challenging, however, and particularly so when the fields in question stand at such a considerable distance from each other in terms of epistemic cultures [3], which reflect the diversity not just in scientific approaches and tools, but in the sublimated habits, practices, organisational principles, values and relationships to evidence and achievement. But how far apart are the cultures of AI research and the humanities in reality, and in what ways? What are the most significant potential points of conflict we need to be aware of in fostering this convergence? And, given that a methodological community with the promising, border-crossing name of ‘digital humanities’ (or DH for short) is already well-established, how might its experiences and practices offer a useful perspective or starting point upon which to build field-spanning bridges toward this end goal?

While the methods and approaches of the digital humanities may well be hybrid, it remains the fact that the most common research questions in DH originate from within the humanities disciplines, creating knowledge more often for and with them than other contributing disciplines, such as computer or information science (though of course they do that as well). Indeed, it is often the very challenges and constraints brought by the positioning of DH as grounded in the humanities that make the field so vibrant, and its purposeful investigations of cultural matters so fruitful as points upon which to build interdisciplinary bridges. This grounding does come at a cost, however, one that is paid, perhaps ironically, in the currency of words. This claim refers not to a generic lack of a vocabulary to frame the content of humanities disciplines, or of their digital offshoots, for that, of course, it has, but of a very specific gap in capacity to precisely describe at a macro level the processes, acts, and distinguishing characteristics of humanities research. Historian of science Lorraine Daston reflected on this state of affairs in her 2016 lecture to the Warburg Institute, entitled “Exempla and the Epistemology of the Humanities” [4], referring to the tendency for disciplines to possess “a rich epistemological vocabulary in which to reflect critically on their various ways of knowing.” In contrast, her observation of the humanities disciplines brings her to claim that one can almost say there is “no epistemology of the human sciences,” a situation that stymies reflection and drives these fields toward descriptors developed within and optimised for other disciplines. Regardless of the degree to which Daston may or may not be fully accurate in her assessment, the phenomenon she describes does have an impact on the ability to clearly see, share and analyse humanities research processes, and to foster their interdisciplinary convergence.

The term ‘humanities’ as a descriptor is perhaps itself already too broad to be useful, encompassing a very wide range of disciplines, epistemic cultures, values, and ways of knowing. The manner in which its traditions only seem to acquire precision of self-expression (as opposed to rigour of method, which is not in question here) when described through the vocabulary of other fields of research creates a further source of tension and barrier to their description and documentation. The project of humanistic AI meets significant resistance here. How can we know what impact a shift in method or a move toward interdisciplinary integration has if we do not have stable terminologies to express the original state from which this change began? Similarly, how can we describe, and in doing so explain, the appropriate point of intersection between computer science and the humanities, of the convergence of a method requiring strict quantification with a culture of staunch qualification? We must accept that such answers will always be unbalanced and imperfect but also that they will by necessity need to borrow vocabulary and conceptual frameworks from other fields.

A further complication with respect to the terminological challenges of humanistic AI is the mobility and capaciousness of the term ‘artificial intelligence,’ which is widely used in policy and public discourse to mean something akin to ‘human like’ [5], but which is largely absent in technical discourse, where it is supplanted by more precise referents, such as machine learning, robotics or neural networks. Even when viewed specifically as a branch of or concern within the research field of computer science, a multiplicity of possible referents exists, including computer vision, robotics, large language models, and machine learning generally, to name only a few.

The fact that we do not necessarily have stable explicit vocabularies to speak about either the humanities or artificial intelligence research (and of course the digital humanities as somehow suspended between them) does not mean that these approaches do not reveal themselves through implicit language. Indeed, the discourses used in the scientific writing of these fields speak volumes about the values and assumptions that underpin them. What follows is an attempt to expose some of these deeply embedded differences between the fields through a comparative corpus-based approach. Building upon previous investigation into the discourses of machine learning research (which I, following the authors of that study, take as a meaningful proxy for the wider field of AI research) and, in particular, on the framing of ‘justificatory statements’ in this work, a number of significant discursive differences can be documented, indicating underlying value positioning that would need to be managed carefully so as not to run the risk of constraining productive exchange. In particular, the analysis looks at the expression of researcher values in the bodies of work, differing approaches to the construction of scientific agency across the three fields (humanities, digital humanities and machine learning), the characterisation of knowledge creation as a key element in the “justificatory chains” presented in the corpora, and finally at the manner in which the digital humanities positions itself as an interdiscipline that draws from both the humanities and computer science.

2. Materials and Methods

The method applied in this investigation takes its inspiration from a recent article [6] (hereinafter referred to as Birhane et al.), which applies a discourse analysis approach to extract statements of researcher ‘value commitments’ within a corpus of papers from the field of machine learning. Discourse analysis distinguishes itself from other applications of corpus linguistics in its goal to “shed light on how speakers indicate their semantic intentions, how hearers interpret what they hear, and the cognitive abilities that underlie human symbolic use” [7]. The corpus underlying this analysis is not large, focusing on 100 highly cited papers from two prestigious conference proceedings (NeurIPS and ICML), and its method was labour intensive, with humans annotating and manually encoding the abstract, introduction, discussion, and conclusion sections of each paper in the search for statements of researcher value positionings and ‘justificatory chains,’ that is the lines of reasoning by which each paper makes its case as a contribution to scholarship. The work ultimately concludes that the underlying values expressed within this corpus are out of step with social goods, following instead a tacit political and social agenda that centralises technological power and maintains existing hierarchies.

What is interesting about this work is the manner in which it is able to extract tacit positioning from discourse that was not necessarily intended to be read for these signals. It was this desire not to be led by assumptions that guided Birhane et al. to adopt the labour-intensive dual human encoding method they used to indicate justificatory claims, a slightly different goal from the investigation that follows, which is more concerned with how research is perceived. And yet, taking Birhane et al.’s questions and dataset as a starting point allows for a highly instructive set of comparisons to be made, in particular, as the dataset underpinning their work has been made openly available for further analysis. What follows in this paper, therefore, presents some of the methods applied by Birhane et al. and compares them with equivalent corpora drawn from both a traditional humanities (taking literary studies as an example) and a digital humanities journal. The resulting evidence establishes a framework for clarifying the nuances of these different epistemic cultures, and the nature of their current and potential future state of convergence, from the perspective of discourses related to values, perceptions and approaches, highlighting, in particular, the divergences among these fields. This paper transforms the original authors’ single-discipline dataset into a cross-disciplinary one, adapting the original method in the following ways:

2.1. Moving from a Single to a Comparative Corpus

In Birhane et al.’s original paper, the curation of the data was based upon the notion of papers being influential because they are highly cited, and being presented at certain high-profile conferences, which are indicative of community influence in the computer science disciplines. The use of citations as a proxy for influence does not operate in the same way in the arts and humanities, however, and has been widely criticised when deployed as such (see, for example, [8,9,10]). This is not to say that there are not more and less prominent, established journals, however, and ones which, in their own way, indicate a similar status of representing the voice of a research community. Conferences do not have this status across the disciplines, however, but what humanities disciplines have instead are conferences and associated journals that are published by the large membership associations. The editorial processes of these organs, which are responsive to the voices of the members, can on this basis be understood as a good proxy for community norms and practices.

On this basis, my comparator corpora have been drawn from two long-running and respected journals, which may be said to speak for the norms of the community due to their positioning as the voices of prominent subject associations. The Publications of the Modern Language Association (established in 1884) is a community organ shared by the over 20,000 members of the Modern Language Association, an organisation that presents itself as a leading advocate for the humanities in general, but more specifically for the study of languages, literatures, and culture. Just as ML research stands in as a proxy in Birhane et al.’s work for the wider AI community, the MLA can stand as a coherent representative of a specific sub-community with broad representation in the wider field of the humanities.

While the challenge with the humanities lies in the presence of a number of established and distinct subfields, finding the ‘voice’ of the digital humanities presents a very different, if no less complex, problem. Kathleen Fitzpatrick’s early and inclusive definition of DH as “the humanities, done digitally” (an extension of the preexisting theory/practice divide long known in the humanities) [11]. But the simplicity of this formulation overlays a significant body of work that proposes myriad different, sometimes contradictory, definitions of the field, as seen for example in [12] and in the very existence of a long-running and popular series of volumes known as the Debates in the Digital Humanities (edited by Matthew Gold and Lauren Klein). In spite of this heterogeneity, for the digital humanities, there is an equivalent journal to the PMLA overseen by a prominent subject association in Digital Scholarship in the Humanities (DSH). First published in 1986 (as Literary and Linguistic Computing), the journal has long served as the outlet of the Alliance of Digital Humanities Organisations (ADHO), an umbrella organisation representing over a dozen regional DH associations. As both of these journals are registered in the Web of Science database, abstracts for a similarly sized sample of recently published (2022–2023) articles could be assembled into parallel corpora to be used alongside that of Birhane et al. An overview of the size and complexity of the three corpora appears in

Table 1. Overall size of the three corpora.

Source	Abstracts	Tokens	Unique Terms
Birhane et al.	100	14,610	2724
PMLA	103	15,109	4151
DSH	101	15,372	3346

below.

The papers were selected as a continuous run, starting in 2022 and ending when the desired number of abstracts (+/− 3) had been reached. The of slight variation in the number of entries in each corpus represents the best possible equivalence between the three sources, balancing overall corpus size and number of entries with the desire to include the full contents of any release batches included in the sample. The data was inspected to remove possible biasing factors, such as special issues or reviews, and the full metadata and abstract text extracted into an Excel spreadsheet, where duplicates were removed, missing data (such as publication dates) added, and any obvious errors in spelling or data organisation (such as data in the wrong columns) corrected. From here, the abstract text was copied into a .txt file able to be processed by the corpus query software package being used.

These journals cannot be understood as wholly representative of the wider fields they are drawn from, but as illustrative proxies that provide a useful and good, but not perfect, representation of wider trends. Just as Birhane et al. abstract from data reflecting the Machine Learning community to wider conclusions about AI in general, these corpora also draw from snapshots of humanities and digital humanities research activity, with a bias toward the English language and Northern/Western perspectives, toward the specific editorial policies and procedures of the journals, and, in particular, the decision to feature a subject association anchored in literary studies to represent the humanities. This is not to say that these proxies do not lose something with their essentially metonymical nature, taking a part as representative of a much larger whole, and perhaps missing wider trends in the broader fields as we tend to conceptualise them. It should be noted, however, that the first real challenge here lies in the ease with which we use terms like ‘artificial intelligence’ and ‘the humanities’ in a way that seems to describe something cohesive and clear, but which actually may incorporate a very wide range of practices. Future research might add further corpora, for example drawn from the proceedings of the American Historical Association, or the journals of some of the regional subject associations that operate under the umbrella of the European Association for the Digital Humanities. This approach, however, would potentially introduce further biases and complexities, while also not abiding by the desire to mirror the original structure presented by Birhane et al.. For this reason, in spite of their limitations, these two journals were ultimately chosen as the best possible matches for the goals of this study. The second hard challenge related to the degree to which these outlets can be seen as comparable, seeing as two of them publish papers submitted directly, whereas the original dataset is comprised of conference papers. My response to this potential critique is twofold: first, in fact, a conference paper in the humanities would be likely to be far more different from a computer science conference paper, as humanities conferences tend to accept or reject not upon full papers, but upon short abstracts. The greater comparability in process and form is reflected in the choices made for this analysis, in particular as pertains to critical aspects such as selection criteria and depth of peer review. Second, we must always remember that investigations of interdisciplinary work require that we suspend some more exacting conceptualisations of comparability: disciplines do, to some extent, inhabit their own epistemic cultures, and to compare between them always involves some flattening of specificities. If we allow this perception of differences to dominate, however, then we close doors to methods that might enhance our understanding of how knowledge creation ecosystems might continue to evolve in conversation with each other.

First, please clarify the comparability of the three corpora, particularly differences in publication type, venue, selection criteria, and citation status.

2.2. Creating Comparability Among Divergent Formal Norms

Second, I narrow the focus within the texts to only the abstract of each paper as a locus of value statements and justificatory positioning. Although introductions and conclusions are also used by Birhane et al. in their original work, I removed these sections from their dataset because these are not always clearly indicated as sections in humanities work, and would have been difficult to extract consistently according to any clear and stable criteria. For practical reasons, therefore, abstracts—which are in all three corpora clearly identifiable and distinct—have been chosen to serve as proxies for the articles as a whole. While the differences between the language of article full texts and that of article abstracts have been well documented [13], the differences that have been discovered, such as sentence length and presence of parentheticals, are not particularly relevant for the questions of how authorship is presented and perceived, which are at the heart of the current investigation. Indeed, their formalised structure, consisting of what some scholars characterise as a specific sequence of ‘moves’ (e.g., [14,15,16]), and their status as metadiscourse with a conscious goal of capturing the content and contribution of the article as a whole may well enhance the signals they present. These characteristics render them interpersonal, interactional [17] and evaluative [18] in a way that article texts may not be. That said, it must be recognised that the abstracts do not have the richness of the full texts, and do, therefore, carry some limits as well as an object of study.

2.3. Revisiting Goals and Reviving a Quantitative Approach

Finally, I choose to deploy a mixed-methods, but largely automated analysis workflow, rather than the qualitative and human-encoded one employed by Birhane et al. Such an approach had been considered and rejected by Birhane et al.’s team out of a desire to ensure that the values that were identified were not led by confirmation bias. A number of factors made such an approach a more reasonable choice for this comparative study, however. First of all, a focus on habits of discourse, rather than more complex statements of values, meant that specific linguistic features could be more telling than they might have been for Birhane et al. Furthermore, observation of the annotated data produced by Birhane et al. showed that the value positions extracted could generally be mapped onto particular discrete linguistic features, supporting an analysis that is evidence-based, but ultimately more descriptive than inferential. Reading through the justificatory chains identified in the original dataset demonstrated some very clear patterns. For example, in the examples featured in their article, the value of “efficiency” was always flagged with specific use of this word. Other values, such as novelty, hewed to relatively narrow ranges of expression. Given the focus on discourse rather than underlying values, the automated approach rejected by Birhane et al. could play a larger role here, particularly due to the presence of specific intention statements made by the authors, clearly indicated with the use of the personal pronoun “we” (as in “in this paper, we show …”).

The basis of my analysis then became precisely this: to extract computationally from the three parallel abstract corpora such specific statements of authorial intent, and to compare them in their form, but also in terms of the implications of the words used to characterise the epistemic claims being made. The workflow implemented for this was relatively straightforward, using Microsoft Excel to clean and verify data, and the LancsBox [19] toolkit to apply tags (grammatical and semantic), process and analyse data samples largely using KWIC queries to find and identify the function of single word agency statements and multi-word characterisations of the actions of these agents (active verb following the noun or pronoun reflecting the perceived agent). This feature selection was verified manually, with any ambiguous cases (such as the humanistic use of the word ‘we,’ discussed below) captured and considered for their weight within the interpretive framework. As the corpus was small, all manual encoding was carried out by the author.

What follows below are the observations facilitated by this analysis process and its results, grouped under four different headings: first, a consideration of the values found by Birhane et al. and their lack of resonance in the comparator corpora; second, the nature of the agent making an epistemic claim in a given paper from a given field; third, the discourses of the justificatory chain, in particular, the nature of the action of research being portrayed and finally, the specific positionality of the digital humanities in terms of its blending of disciplinary discourses and cultures.

3. Results

3.1. Can Values Be Compared?

One of the most interesting aspects of Birhane et al.’s findings is in the surprising nature of the tacit value positioning the research identifies, with the top entries being the following: performance, generalisation, efficiency, and novelty and building on past work. Birhane et al. characterise these values as appearing in the place of others that might relate to societal need, but it is interesting to note that such positioning is not present in the humanities/literary and digital humanities corpora either. Research, it seems, serves its own needs, addressing the norms of a discrete community of practice rather than society as a whole. That said, the top entries in Birhane et al.’s list are also largely foreign to the discourse of the humanities and digital humanities. In particular, performance, generalisation, and efficiency appear very much as a foreign language to these corpora, except in those cases where an ML approach is being used in a digital humanities paper. The word ‘efficient’ and its derivatives never appear in the PMLA corpus; ‘novel’ appears 36 times, but never in the sense of something being new, referring instead to the literary form. The DSH corpus does contain three instances of ‘efficient’ and 12 of ‘novel’ being used in the sense of new, reflecting the mixed epistemic culture and values of the journal.

What is interesting, however, is the fact that novelty and new discovery are very much a value across all of the research, even if the discourse surrounding it varies among disciplines. As will be discussed below, the language in which this positioning of novelty is expressed is far more varied in the PMLA corpus than in the ML one. Rather than directly citing the novelty of a method or finding, the humanities research community seems much more circumspect, speaking of gaps or reexaminations, hedging statements with the particle ‘yet,’ or asking rhetorical questions. This finding bears out earlier research (based on research interviews with digitally engaged humanities scholars, see [20] for a fuller account), which established the importance of the concept of a ‘gap’ in humanities-led digital research. These findings will be briefly described here as they illuminate the way in which a convergence in values may be obscured behind a divergence in language.

As befits an approach that is based upon the continual layering of differently conceived pieces of information and knowledge, the researchers interviewed would often speak about ‘gaps’ in the knowledge landscape. Rather than being a negative term, however, the ‘gaps’ were generally the areas of most interest, the places where research questions could be found. A good research question is one “that’s reasonably broad. Also, one that fills a gap in the scholarship. I think that has to be primary; it needs to fill some sort of gap” (INT 3). While not always expressed using this exact term (though very often it was this precise word), the feeling of having located the gap was very often met with incredulity or a mix of something like delight (at having identified a gap) or derision (that this had not been noticed before): “I remember having this odd feeling: why hadn’t somebody made this obvious point before?” (INT 1) or “this collection of essays…had so thoughtlessly excluded drama in polite fashion” (INT 5). These gaps are perceived as something individual, unique to the specific process that has built the scholar’s personal apparatus, making them want to offer a way to “fill in gaps that only I see, and that I think other people should see” (INT 2). In part because of this individual specificity, the finding and addressing of a gap can become quite an exciting prospect:

Let’s say it’s an idea and you can trace its impact and evolution and then there seems to be a moment there where you’re missing a piece of the story and which is often an interesting thing to happen, because often that means there is something unexpected to you [that] has happened, and filling that in can be quite interesting: what primary texts should I be reading to try and track the primary texts, can I trace it from a different route, can I follow it through this branch of authors that I know, the guys I’m working on you know…it gives you courage to sort of stand back and see a bigger picture and see if there are patterns there that can explain what doesn’t make sense there when you’re focussing on the small bit of the canvas that you’re working on.

(INT 8)

…but there’s always something new to say but so argument will come from that [sic] and an awareness of what’s out there what’s not done and what is done which only comes from a significant amount of reading, and just an awareness of what other people are working on I suppose which is sort of insider knowledge, and then use your creativity and that’s the fun part, that’s where your own subconscious urges and fascinations come up—if you start digging around in this stuff it gets very weird.

(INT 2)

A related concept to the ‘gap’ seems to be the ‘hook,’ a sort of catchy ‘edge’ to the gap where other related pieces of information can be built up. In some cases these ‘hooks’ are rhetorical: “I’m annoying, I have to have a hook for the title as well, once I’ve thought of a title and usually it’s some stupid pun, or something, as well” (INT 4). In other cases, however, the ‘hook’ represents that bit of what is known around the gap that allows it to be addressed, providing a way “into the subject.”

The values underlying this desire to address gaps and discover hooks map to the ML researchers’ desire to ensure their work is novel, but the specificity of expression can be very different between communities. This linguistic diversity will be discussed in further detail later, but even as creating new knowledge seems to be a point of overlap for the ML corpus and the humanities one, the values identified by Birhane et al. seem to be otherwise largely useless as a basis for methodological convergence.

3.2. Who (or What) Lays Epistemic Claim to an Act of Research?

Having observed the gap between the humanistic and ML values positioning, I turn to what the parallel corpora can tell us about the question of epistemic agency: who speaks for the findings of a research paper, and how does this differ between fields? The first major adaptation required to accommodate the particularities of the two new datasets was the expansion of the linguistic field understood to express this concept within justificatory chains. Within the original, computer science-derived dataset, nearly all statements identified took the form of a ‘we’ statement (although a small number of instances are present of passive voice being used, as in the following: ”a proof of convergence is presented”). This is indicative of both the strong norming of co-authorship within machine learning research as well as the generally explicit manner of expression that might be expected in a field in which nuances of academic writing style are less of a focus than in the humanities. That said, the consideration of both ‘I’ and ‘we’ statements only encompassed a still limited percentage of agency statements found, in particular, in the PMLA corpus. For this reason, the following possible agents also need to be considered as well: “essay,” “study,” “paper,” and “article.”

The breakdown in use of these various elements across the computer science and humanities representative corpora is given in

Table 2. Distribution of agency statements, by corpus excluding non-justificatory usage.

Source	I	We	Essay	Study	Paper	Article	Passive	Total
Birhane et al.	0	258	0	0	15	0	18	291
PMLA	52	9	76	3	1	15	1	157

. The comparatively significant tendency to depersonalise the agency of the author in humanities research (e.g., by placing the justificatory statements at a distance from the author’s personal agency through the use of a subject term like ‘essay’ or ‘paper’) might be read as indicative of a number of possible value stances. One interpretation could be that this is a reflection of a longer tradition of presenting an act of hermeneutics as separate from the subjectivity of the individual interpreter. On the other hand, this culturally embedded discursive strategy may also be seen to represent a more measured understanding of the individual journal article as an act of scholarship (bound as it is to specifics of time, form, evidence and other contextual constraints), which is clearly entangled, but not coterminous, with the subjectivity of the individual(s) who created it.

The effect of these differences makes for two very distinctive experiences of reading the scholarship. In the PMLA corpus, one senses the scholarship itself as perpetually active, with a wide variety of compelling formulations presenting the epistemic act: “…this essay brings/develops/asks/shows” etc. While the variation is not quite so wide as pertains to how the epistemic acts of the authorial ‘we’ in the machine learning corpus are represented, many of these very same active verbs do appear, and a close equivalent does appear in the form of ‘paper,’ although this formulation has only a limited presence and even more limited range (12 terms) of active verbs associated with it, headed by the very neutral form ‘presents’ (3 occurrences).

Interesting as well in this respect is the manner in which the pronoun ‘we’ actually does appear in the PMLA humanities corpus. In fact, the word appears 25 times, but only 9 of those could be categorised as authorial statements of a justificatory nature. Instead, the remaining usages seemed much more rhetorical, intending to imply a wider social context for the work being presented, as in phrases such as “we need new modes of reading,” “how do we as scholars,” or “what can we learn?” (Indeed, the use of agency pronouns in this article seems very much in line with the positionality of its (humanities trained, digital humanities identifying) author, with 17 instances of ‘I’ (which I use to avoid the temptation toward a passive voice that seems indicative of a Harawayian ‘God trick’) and 14 instances of a largely humanistic ‘we.’) Such references to some kind of shared understanding or condition uniting the author (or their work) and the audience invites, again, a possibly dual fold interpretation, in which the author either imposes their unconscious biases on their audience or, alternatively, points toward the social embedding of their work (something which, according to the conclusions of Birhane et al., is actually very rare in the machine learning corpus).

In these humanistic discursive strategies we can see what Donna Haraway characterises as the methodology of thinking practice that is like the child’s game of cat’s cradle [21], threading together corroborative arguments rather than seeming to propose a streamlined, targeted notion of science. This approach to knowledge creation recognises that “all entities take shape in encounters, in practices” and furthermore, that the humanities are well-placed for “inquiring into all the oddly configured categories clumsily called things like science, gender, race, class, nation, or discipline.” This is also reminiscent of what Haraway terms “situated knowledges” [22], in which the partiality of any perspective must always form a basis to contextualise objectivity as something other than what she calls a “God Trick.”

3.3. Convergence and Divergence in the Discursive Act of Making a Justificatory Claim

Across the corpora, the largest number of explicit contributions to the creation of justificatory chains was found in Birhane et al.’s original corpus, with 291 total statements identified, as opposed to 157 in the PMLA corpus (I will return to the DSH corpus in this respect in Section 3.4).

In spite of this large variation in overall count of references, the number of unique words (after lemmatisation) in each of the corpora is far less divergent. Of interest, however, is that although the Birhane et al. corpus and the PMLA corpus contain roughly the same number of verbs being used in a justificatory context, the precise elements within those two sets are quite different, with 78 unique items in the Birhane et al. corpus (average of ca. 4 appearances per word) and 69 in the PMLA (average of ca. 2.25 appearances). For comparison, the top six most common words in the PMLA and ML corpora are shown in Table 2 and

Table 3. Most common verbs in justificatory chain, Birhane et al. vs. PMLA.

Word	Occurrences Birhane et al.	Occurrences PMLA	Log-Likelihood
show	32	11	11.43
propose	26	3	21.69
present	20	0	28.40
develop	12	1	11.34
introduce	12	0	17.04
prove	12	0	17.04

:

As the log-likelihoods given above show, in almost all of the cases, the differences between the presence of these words in their respective corpora are statistically significant to a high degree of confidence (p-value of 0.05 or better). The lower relative diversity in Birhane et al.’s corpus does not tell the whole story, however, as the results are highly skewed toward particular, seemingly standardised, expressions. In fact, the top six most common such words account for almost 40% of all of the justificatory statements, as per Table 2. Although the word “show” clearly represents a semantic field where the two sets of discourses overlap, the correlation beyond that is much weaker. It is also interesting, however, that almost all of the occurrences of ‘show’ in the PMLA corpus are prefaced with the subject ‘I’, positioning it as a function of the author, whereas in Birhane et al.’s corpus, the ’show’er can either be the authoring team (‘we’), or the experiment, evidence, or method applied in the research.

The overlap in the most commonly used justificatory verbs between the corpora is, therefore, limited, a pattern which actually extends to the overall vocabulary of justification, with the Birhane et al. corpus containing 45 unique, significant words that appear only in that corpus, and the PMLA corpus containing 35. Given the overall number of terms in question here, the divergence in these scientific vocabularies is striking and, I would contend, invites interpretation and consideration of the Harawayian perspectives presented above. Words like ‘examine,’ ‘explore,’ ‘argue,’ and ‘read’ all feature heavily in the PMLA corpus, and seem to emphasise the nature of the humanities as often interpretive, rather than declarative, as common terms in the Birhane et al. corpus such as ‘present,’ ‘prove,’ and ‘propose,’ could be seen to indicate.

3.4. Digital Humanities and the Discourse of an Interdiscipline

Given that the above discussion indicates clear differences in the discursive strategies these two epistemic cultures use to characterise their knowledge creation processes, we can now ask the question of where, as an interdiscipline, digital humanities positions itself. The answer seems to be, as one might expect, that it falls somewhere in the middle.

If we first extend the table showing how explicit justificatory statements are made, this positioning begins to become clear, as is shown in

Table 4. Most common verbs in justificatory chain, PMLA vs. Birhane et al.

Word	Occurrences PMLA	Occurrences Birhane et al.	Log-Likelihood
examine	17	1	16.7
argue	13	2	8.65
show	11	32	11.43
read	8	0	10.82
call	6	5	0.06
explore	6	3	0.92

. The abstracts in DSH contain more explicit justificatory claims than PMLA overall, but there are some, if not many, indications of a single author personalising their justificatory chain with the use of ‘I’ rather than ‘we’ (though the plural sign of collaborative authorship is clearly much more common). At the same time, the move toward a depersonalisation, which was introduced in the discussion of the humanistic use of the term ‘essay,’ also seems to pertain here, with a much closer ratio in the digital humanities of work-based (also including active subjects of ‘paper,’ ‘study’ and ‘article’) to author-based statements (using ‘I’ or ‘we’) (83:107) compared with the computer science corpus (15:258).

Similarly, when we look at the top six verbal expressions of scholarly intent, we find (as per

Table 5. Distribution of agency statements, by corpus, including DSH.

Source	I	We	Essay	Study	Paper	Article	Passive	Total
Birhane et al.	0	258	0	0	15	0	18	291
PMLA	52	9	76	3	1	15	1	157
DSH	6	101	5	0	33	45	0	190

,

Table 6. Most common verbs in the justificatory chain, DSH vs. PMLA.

Word	Occurrences DSH	Occurrences PMLA	Log-Likelihood
propose	27	3	21.67
demonstrate	10	3	3.86
use	7	3	1.58
show	6	11	1.58
examine	5	17	7.12
explore	5	6	0.11

and

Table 7. Most common verbs in the justificatory chain, DSH vs. Birhane et al.

Word	Occurrences DSH	Occurrences Birhane et al.	Log-Likelihood
propose	27	26	0.01
demonstrate	10	0	13.69
use	7	4	0.78
show	6	32	19.98
examine	5	1	2.84
explore	5	3	0.47

) that the digital humanities corpus contains a very strong presence of the same top two justificatory verbs: ‘show’ and ‘propose,’ but it also features prominent humanistic tendencies in the presence of ‘examine’ and ‘explore.’ As the log-likelihood scores show, the use patterns for shared lexical items tend not to be significantly different from either the literary studies or the machine learning corpus. Interestingly, however, at a deeper level, we once again see a distinctive vocabulary being used, with a total of 35 words appearing in the DSH corpus that do not feature in either of the other two (though the occurrences of these words are very low). From a discursive point of view, the digital humanities can, therefore, be seen to be borrowing from both sides of the epistemic divide, but also apparently forging its own discursive space as well.

It is interesting to ponder what might be both the cause and the impact of this characteristic. Certainly the participation of teams and individuals from different backgrounds in DH publishing would be a major factor in the variation. If we return to the most broadly shared lexical item, ‘show,’ we do see one example of the word being used in line with the conventions of the PMLA, with the authorial ‘I’ as the subject. The remaining five occasions either use ‘we’ or describe some aspect of method or experimental outcome as having ‘shown’ something. Though a small sample, this would seem to indicate a closer affinity with the discursive practices of machine learning, but the picture is more complex if you look across the words deployed, with the more declarative ‘demonstrate’ featuring as the top result, but the rest of the list seeming far more humanistically interpretive, positional and hedged, with words such as ‘propose,’ ‘examine,’ and ‘explore’ setting a very different tone.

4. Conclusions and Directions for Future Work

This study was conducted as an initial exploration of how the implicit values of machine learning scholarship, as defined by Birhane et al., might provide a basis upon which to commence development of a more concrete comparison of the differences between the discursive and epistemic cultures of a branch of computer science, one from the humanities and the digital humanities as a possible merging of the two. If this exploration can be seen as a lens through which to understand the potential (and necessary enablers) for a humanities-inflected form of AI research, then we can start to understand just how far apart the humanities and machine learning cultures of research and communication are.

The humanities will be hard-pressed to contribute meaningfully within ML research unless the values of scholarship can be shifted to some kind of common ground (which may or may not be found in and through the digital humanities). The discussion above provides preliminary evidence that these disciplines are doing more than speaking different languages: if this snapshot of their discourses is to be believed, they seem instead predicated on entirely different concepts of what scholarship is and does, how an argument is made, and how new knowledge is generated. This irreconcilability is a perception with a very long history, but relatively little actual concrete evidence for what this means has been gathered in that time. Though it would require further inspection to prove, the digital humanities may combine these perspectives at a field level, but less so at the level of individual acts of scholarship. As such, they provide a possible meeting place, more than an established common ground, in line with the concept of a “third wave” of AI ethics [23], which takes a more holistic view of the interaction between human and technical systems.

In addition, however, the manner in which different disciplines construct the agency and positionality, in article abstracts at least, of the researcher gives pause. The distributed depiction of agency of the ML author, compared with the confidence of their language of discovery, might be seen as an opening for just the kind of gap between what a researcher might perceive as good quality work (from the standpoint of the values of quality, performance, etc.) and good quality applications of that work (in which the ethical and social considerations would come to the fore). As we know, however, computer scientists tend not to receive much ethical training, or value ethical training much [24,25], and often draw boundaries around their sphere of influence even when the tenets of popular movements such as “privacy by design” might seem to grant them significant responsibility [26]. The more socially integrated and positional ‘we’ of the humanist represents an approach to one’s epistemological achievements that could enrich the human-centredness of ML.

The present experiment resonates with, and provides evidence for, the commonly held belief that there are deeply set differences between disciplines in the perception of what scholarship is and does, and, by extension on the basis of Birhane et al.’s findings, the values of those scholars who do it, visible just below the surface of the record of published research. Such an evidence base can be a starting place for more productive discussion of how disciplines might cooperate to build better approaches to AI across the epistemic divides. Digital humanists find themselves well-placed to negotiate in and around this gap, incorporating, as they do, elements of both of the epistemic cultures that contribute to the field. More work to explore the capability of this field to create functional boundary languages contributing to a more human- or life-centred AI development trajectory would, therefore, be very worth the effort and investment required to foster it, capitalising on the discourses and practices of a discipline that sits between established poles of scholarship.

The findings in this paper also indicate that the fashion for interdisciplinary research at scale may rest upon weaker foundations than it might, if greater attention was paid not just to the coming together of disciplines for a purpose, but to the fundamental meaning of disciplinary convergence. The research system tends to reward deep specialism, but the methodological breadth required to broker conversations about values, evidence and the nature of knowledge itself needs also to be recognised, fostered, and rewarded if we are to see disciplines such as machine learning and literary studies converge on more than the most superficial level, and in more than the most capricious and personality-based manner.

Finally, it is hoped that this paper in itself might stand as an example of how computational and interpretive traditions might be brought together in a way that harnesses the affordances of each without leaning too heavily on the values or assumptions of either. Its intention is not to provide a last word on the subject, but rather to demonstrate a way in which we might begin to build a more rigorous, informed approach to the interdisciplinarity assumed to be available to foster human-centred AI. Although it uses a relatively small corpus of abstracts only, and may be constrained by the limitations that accompany such a framing, this article does achieve its aim of demonstrating a method and making some initial conclusions that can be followed up with further work that might close the gap between what an experiment of this scale can prove, and the broader contextual reflections it instigates. A computational or data-driven approach can be an end in itself, or it can be a starting point for the negotiation of meaning and understanding between complementary ways of knowing. Aligning justificatory chains may be, like Haraway’s game of cat’s cradle, a way to weave knowledge from differing, uneven fibres of warp and weft, resulting in imperfect, and yet more integrated, results.