Strategies and Recommendations for the Management of Uncertainty in Research Tools and Environments for Digital History

Round 1

Reviewer 1 Report

The paper is well argued and well written. This article focuses on the role and meaning of uncertainty for humanities research, an attribute of humanistic epistemology which is often not made explicit, taken for granted or even taken for granted in a specific form. 

In the age of interdisciplinary work, to make field-specific epistemological aspects (field meaning here a sub level to a discipline) is in particular valuable, not to say crucial to enabling successful knowledge transfer.

The author presents the reader with a well-written essay, carefully selected examples, instructively engaging with the existing literature and cumulating in - what I would call - a check list of issues for adequate epistemology in the humanities. 

The only point I would like to made, is that the paper would win, if the author would unfold this plan to the reader more explicitly it is done in the quotation taken from the end of section 1. “This paper will therefore look at how this differentiation  between humanistic ‘fact’ and interpretation shapes the nature of humanistic research questions and  attitudes toward sources in the face of such uncertainty. It will look at this phenomenon first in its analogue manifestations, then in the context of digital tools and data-driven research, thereby  exposing some of the challenges inherent in designing digital systems to reduce uncertainty in  research in the humanities. “

For instance, in the transition between section 1 and section 2, suddenly the author refers to one specific project, please explain how you constructed  your ‘case studies’, and give an overview how you selected your evidence. 

section 2 gives an illustration of the kind of uncertainty and the consequences for source seeking and use  of sources. While reading, I thought the flow of the rhetorical argument throughout all sections could be made stronger, i.e. by binding each section back to the question put out at the beginning of the essay

Could the author give the types of uncertainty listed on page 3 not only descriptions but a label? This is not a request, more a question or challenge.

If you talk about Farge, you could emphasis that here you deal with the prior announced off-line examples. For a reader not that familiar with archives, that might not be self-evident.

section 4: Motivations and Mechanisms for Managing Uncertainty 

As the author writes: each  research question starts with uncertainty. The author claims “..will be very different in these cases than in those involving historical research. “  Is this really the case? Could the types you introduce in section 3 not been matched into Petersen’s matrix? Where would such mapping go amiss? 

section 5: make the structure of your argument explicit, you speak of your “third case study” what have been case study 1 and 2. I assume the positions of researchers you name in dominantly in section 2 and 3, right? Make this more explicit, already in the outset. 

Author Response

Dear Reviewer, 

Many thanks for your considered and useful comments.  In the revised manuscript, I have responded to them as follows (changes made described in line with your comments):

The paper is well argued and well written. This article focuses on the role and meaning of uncertainty for humanities research, an attribute of humanistic epistemology which is often not made explicit, taken for granted or even taken for granted in a specific form. 

In the age of interdisciplinary work, to make field-specific epistemological aspects (field meaning here a sub level to a discipline) is in particular valuable, not to say crucial to enabling successful knowledge transfer.

The author presents the reader with a well-written essay, carefully selected examples, instructively engaging with the existing literature and cumulating in - what I would call - a check list of issues for adequate epistemology in the humanities. 

The only point I would like to made, is that the paper would win, if the author would unfold this plan to the reader more explicitly it is done in the quotation taken from the end of section 1. “This paper will therefore look at how this differentiation  between humanistic ‘fact’ and interpretation shapes the nature of humanistic research questions and  attitudes toward sources in the face of such uncertainty. It will look at this phenomenon first in its analogue manifestations, then in the context of digital tools and data-driven research, thereby  exposing some of the challenges inherent in designing digital systems to reduce uncertainty in  research in the humanities. “

I have endeavored in the revision to strengthen this, including by adding a paragraph at the end of section one outlining the flow of the argument across the sections of the paper.  I also have expanded a bit to make clear what I think the value of the paper and the approach are.

For instance, in the transition between section 1 and section 2, suddenly the author refers to one specific project, please explain how you constructed  your ‘case studies’, and give an overview how you selected your evidence. 

This has now been clarified, at the point you mention and where I turn to a second set of cases.

section 2 gives an illustration of the kind of uncertainty and the consequences for source seeking and use  of sources. While reading, I thought the flow of the rhetorical argument throughout all sections could be made stronger, i.e. by binding each section back to the question put out at the beginning of the essay

See comment above, this has been added

Could the author give the types of uncertainty listed on page 3 not only descriptions but a label? This is not a request, more a question or challenge.

Yes, I have tried to make this clearer

If you talk about Farge, you could emphasis that here you deal with the prior announced off-line examples. For a reader not that familiar with archives, that might not be self-evident.

I am not sure exactly what you mean by 'prior off-line examples' but have clarified my use of Farge and the contribution I think she makes

section 4: Motivations and Mechanisms for Managing Uncertainty 

As the author writes: each  research question starts with uncertainty. The author claims “..will be very different in these cases than in those involving historical research. “  Is this really the case? Could the types you introduce in section 3 not been matched into Petersen’s matrix? Where would such mapping go amiss? 

I have added a discussion of this example specifically.  I have also revised the introduction significantly to support this.

section 5: make the structure of your argument explicit, you speak of your “third case study” what have been case study 1 and 2. I assume the positions of researchers you name in dominantly in section 2 and 3, right? Make this more explicit, already in the outset.

This was a linguistic infelicity, now changed.

With my thanks again.

Reviewer 2 Report

The author presents a positional work in which he/she discusses the treatment of uncertainty in history and literature from a complete and broad perspective with interesting ideas and contributions from paradigms and approaches.

With respect to the ideas transmitted, although I think they are correctly argued, I have some comments:

- The approach of giving some guidelines for developers supports (perhaps without the explicit intention of the author) an idea of humanities projects unidirectional or less interdisciplinary : please clarify the position on this. In my opinion, the best projects in this field are those in which all the researchers, both engineers and humanists, are the ones who implement and decide the models of uncertainty.

- I observe in the argumentation a certain position of "debate between the two cultures: sciences vs. humanities". For example, pp. 1 it is argued that in natural sciences or similar "We consider them certain, not requiring questioning when at the basis of a new investigation". Or. "Humanities research questions can be quite complex, with multiple interdependent parts"

There are countless examples of complex investigations with questions similar to those made in humanities in natural science disciplines, especially in frontier research." Please qualify these kind of affirmations. However, in the examples of projects in section 2 are not references for the research hypothesis related with the goal of the differetne project, an d importnatn concept here. I think that the idea that perhaps in humanities several complex hypotheses are considered and developed from the examples would be more appropriate.

I observe throughout the argumentation some unprecised use of terms: certainty, bias, purpose, motivation. Check please.

- In section 6, it is argued:" By and large, humanistic researchers are not looking for tools that change what they study how they undertake their investigations, so much as an enhancement of and supplement to their already heterogenous sources and adaptable methods". I do not believe that this assessment is correct: any object of study is likely to be influenced by methodology or study tools, not only in the digital field.

- I think the guidelines and conclusions are quite valuable for digital humanitites field, and I agree with most ofthem. Maybe will be interesting for complete the paper to add some references for the work done in some of the lines detailed. It is posible to find some references to specific proposal for fuzzy vocabularies or other ideas proposed by the author. For example, 

- Tolle, K.; Wigg-Wolf, D. Uncertainty . . . ? ECFN Meeting 2014—Basel Goethe University 2014. Available online: http://ecfn.fundmuenzen.eu/images/Tolle_Wigg-Wolf_Uncertainty.pdf (accessed on 6 May 2019).

Hedges, M. Grid-enabling humanities datasets. Digit. Humanit. Q. 2009, 3, 4

- Martin-Rodilla, Patricia, and Cesar Gonzalez-Perez. "Representing Imprecise and Uncertain Knowledge in Digital Humanities: A Theoretical Framework and ConML Implementation with a Real Case Study." Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, 2018.

Please referred some of them.

However, my biggest concern about the paper is the adaptation to the journal of this type of work. The first part is a very interesting revision of paradigms, whose debate is current and necessary, but perhaps in a less technical publication than this magazine. I leave of course at the discretion of the editorial team if this type of papers, more positional, are within the scope of the special issue that concerns us. If so, I believe that authors should review and qualify the comments described above. In addition, I consider that, although the guidelines that the author intends to offer are of great reflective and interesting depth, their final implementation in contexts of software development and digital projects in general is very far away. That is, for readers of the journal it would be interesting to develop these guidelines more to give more concrete proposals that can be included as part of their methodologies.

Author Response

Dear Reviewer, 

Thank you for your comments on my submission.  I have responded to them in my revision as follows (see comments in line):

The author presents a positional work in which he/she discusses the treatment of uncertainty in history and literature from a complete and broad perspective with interesting ideas and contributions from paradigms and approaches.

With respect to the ideas transmitted, although I think they are correctly argued, I have some comments:

- The approach of giving some guidelines for developers supports (perhaps without the explicit intention of the author) an idea of humanities projects unidirectional or less interdisciplinary : please clarify the position on this. In my opinion, the best projects in this field are those in which all the researchers, both engineers and humanists, are the ones who implement and decide the models of uncertainty.

I agree with you that ideally collaborative, interdisciplinary research is of reciprocal benefit.  It has not been my experience that this is always the case.  I am very often asked to translate my needs as a humanist into technical model, but I feel that there is also plenty of room remaining in this debate for informing the kinds of models used.  I have tried to clarify this in the revision.

I observe in the argumentation a certain position of "debate between the two cultures: sciences vs. humanities". For example, pp. 1 it is argued that in natural sciences or similar "We consider them certain, not requiring questioning when at the basis of a new investigation". Or. "Humanities research questions can be quite complex, with multiple interdependent parts"There are countless examples of complex investigations with questions similar to those made in humanities in natural science disciplines, especially in frontier research." Please qualify these kind of affirmations. However, in the examples of projects in section 2 are not references for the research hypothesis related with the goal of the differetne project, an d importnatn concept here. I think that the idea that perhaps in humanities several complex hypotheses are considered and developed from the examples would be more appropriate.

I actually agree with you more than my paper would perhaps have made clear.  Rightly or wrongly, I have all too often had to try and explain the nature and impact of epistemic cultures to researchers new to the digital humanities and been astonished at the lack of appreciation for this.  I was all too happy to have your encouragement to make this section more robust, however. The introduction is entirely changed, to include threads that I have hopefully interwoven effectively throughout the argument

I observe throughout the argumentation some unprecised use of terms: certainty, bias, purpose, motivation. Check please.

I have done so and glossed where it seemed unclear

In section 6, it is argued:" By and large, humanistic researchers are not looking for tools that change what they study how they undertake their investigations, so much as an enhancement of and supplement to their already heterogenous sources and adaptable methods". I do not believe that this assessment is correct: any object of study is likely to be influenced by methodology or study tools, not only in the digital field.

This has been clarified

I think the guidelines and conclusions are quite valuable for digital humanitites field, and I agree with most ofthem. Maybe will be interesting for complete the paper to add some references for the work done in some of the lines detailed. It is posible to find some references to specific proposal for fuzzy vocabularies or other ideas proposed by the author. For example, 

- Tolle, K.; Wigg-Wolf, D. Uncertainty . . . ? ECFN Meeting 2014—Basel Goethe University 2014. Available online: http://ecfn.fundmuenzen.eu/images/Tolle_Wigg-Wolf_Uncertainty.pdf (accessed on 6 May 2019).

Hedges, M. Grid-enabling humanities datasets. Digit. Humanit. Q. 2009, 3, 4

- Martin-Rodilla, Patricia, and Cesar Gonzalez-Perez. "Representing Imprecise and Uncertain Knowledge in Digital Humanities: A Theoretical Framework and ConML Implementation with a Real Case Study." Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, 2018.

Please referred some of them.

I now reference two of the three of these.  I have also added further examples of what I consider exemplary practice in section 6.

However, my biggest concern about the paper is the adaptation to the journal of this type of work. The first part is a very interesting revision of paradigms, whose debate is current and necessary, but perhaps in a less technical publication than this magazine. I leave of course at the discretion of the editorial team if this type of papers, more positional, are within the scope of the special issue that concerns us. If so, I believe that authors should review and qualify the comments described above. In addition, I consider that, although the guidelines that the author intends to offer are of great reflective and interesting depth, their final implementation in contexts of software development and digital projects in general is very far away. That is, for readers of the journal it would be interesting to develop these guidelines more to give more concrete proposals that can be included as part of their methodologies.

I personally believe for an interdisciplinary topic such as the the one proposed for this collection, it is sometimes necessary to engage in quite fundamental discussions, and that the disciplines implicit in the collaboration must be empowered to discuss the issues at hand in their own epistemic 'mother tongues,' from their own disciplinary perspectives and ways of knowing.  Otherwise, it is not interdisciplinary or indeed dialogue at all.  If this perhaps seems too 'far from implementation' then perhaps that reflects the reality of where we are, and offers an opportunity for innovation.  That said, I have tried to clarify the positionality of my work in the opening section, its value for a broad audience, and also given more concrete examples of how such values can be put into practice.  Perhaps this will address your concerns.

With my thanks again.

Round 2

Reviewer 2 Report

The author kindly pointed out and clarified
the comments made in the review.

If the journal committee considers
that the article is appropriate
for the scope of the journal then I recommend it for publication


Best regards,