Information Retrieval and Knowledge Organization: A Perspective from the Philosophy of Science

Round 1

Reviewer 1 Report

- pp.3-4 query transformation is discussed. Perhaps another interpretation of the information retrieval process would be the transformation of the research questions into search strings and the selection/evaluation of the results that scientifically answer the question (regardless whether they include the terms or not). Of course, in this process KO should play a big part in this kind of "best match" (statistical as in probabilistic as in science) system. 

- On p.7 the author discusses the problem of popularity in Google using the example of rare diseases.  This is completely true. However, if we accept that Google Scholar works under the same principles as Google (with the difference of just filtering sources by their origin regardless their content/semantics), would popularity be still a problem for retrieval? This same question applies for other systems for retrieving documents such as Web of Science, etc. and also library catalogs that are including relevance criteria to sort results. I believe this question would be important not also for IR but also for the evaluation of science. How to justify or show that one scientific view is more relevant than another one at a given moment? (even if the answer was clear, the question here would be how to express it in the search results for a given query in a specific moment, i.e. how to sort)

- end of p.11 and beginning of p.12 "Compared to classification systems and thesauri they mostly have a much higher level of granularity and are closely related to actual scientific research in the domain. In practice, they tend to be more explicit and precise in their definitions,17"

Here, it would be helpful if the author can develop more on why ontologies are closely related to actual scientific research. In fact another interpretation of footnote 17 could be that ontologies impose the constraints (sometimes simplifications) required by computer logic and the definition of classes, inheritance and hierarchy that are proper of object oriented programming, while the scope notes and the "logical inconsistencies" of thesauri could present a more flexible opportunity to explain and represent the complexities of science in a more humanistic/verbal way. From my point of view, constraining all the complexities of what is being organized into a set of relationships as the ones expressed by Smith et al (2005) on p.13 is just making up reality, an illusion of reality for the sake of efficiency and consistency. Even if the number of relationships used and construed has no limit as the author on lines 422-423, it could be argued that unless techniques such as fuzzy logic or paraconsistent logic are used, it is not only the formal definition of relationships but also the logic that is used what constraints the representation.

• Minor spelling typos were detected (e.g. p.2, line 67: "You just start construing a classification and stops when it seems to suit the purpose", p.4 footnote 5 "The study of how different parts of bot the", p.8 footnote 12 "who found that important work is more")

-On p.16. Herre's own position and the disagreements with Smith’s might not be clear enough for the reader. It would be beneficial for the text if the author includes a brief explanation here.

Author Response

Comments and Suggestions for Authors

- pp.3-4 query transformation is discussed. Perhaps another interpretation of the information retrieval process would be the transformation of the research questions into search strings and the selection/evaluation of the results that scientifically answer the question (regardless whether they include the terms or not). Of course, in this process KO should play a big part in this kind of "best match" (statistical as in probabilistic as in science) system. 

REPLY BY AUTHOR: I believe I have already included this view, although very briefly, writing (in a note): “Unknown documents of relevance for a given inquiry may have to be searched in a context and with concepts as well as symbolic systems unknown for the searcher. This is in particularly the case if there have been paradigm shifts in the field of enquiry.” (NO ACTION TAKEN)

- On p.7 the author discusses the problem of popularity in Google using the example of rare diseases.  This is completely true. However, if we accept that Google Scholar works under the same principles as Google (with the difference of just filtering sources by their origin regardless their content/semantics), would popularity be still a problem for retrieval?

REPLY BY AUTHOR: Yes, I cannot see why Google and GoogleScholar should be different in this respect.  GoogleScholar is, as the reviewer says, based on more selected sources, but we are here discussing IR principles, which may be applied to different collections, but still are based on the same principles containing the same strengths and weaknesses. (NO ACTION TAKEN)

This same question applies for other systems for retrieving documents such as Web of Science, etc. and also library catalogs that are including relevance criteria to sort results.

REPLY BY AUTHOR:  WoS is a “set based” (or exact-match) system. Therefore, these “popularity” issues does not apply to the system as a whole (although searching for or limiting the search to highly cited documents is a “popularity” mechanism very similar to Google’s.

Library catalogs are mainly based on Boolean techniques, and thus “exact match”, and not popularity measures However, library systems (often termed “library discovery systems”) are increasingly adding other technologies, including “best match” technologies. (NO ACTION TAKEN)

I believe this question would be important not also for IR but also for the evaluation of science. How to justify or show that one scientific view is more relevant than another one at a given moment? (even if the answer was clear, the question here would be how to express it in the search results for a given query in a specific moment, i.e. how to sort).

REPLY BY AUTHOR: Yes, I agree: Different “paradigms” have different views of the world, which research questions are raised, and which documents are relevant to answer the questions. This is a main point of the present article, but from the perspective of IR and KO this is not addressed in itself.  (NO ACTION TAKEN)

- end of p.11 and beginning of p.12 "Compared to classification systems and thesauri they mostly have a much higher level of granularity and are closely related to actual scientific research in the domain. In practice, they tend to be more explicit and precise in their definitions,17"

Here, it would be helpful if the author can develop more on why ontologies are closely related to actual scientific research.

REPLY BY AUTHOR: Well, the question could be turned around: Why are other kinds of KOS not closely related to actual research. If they are considered important infrastructures, they ought to be! Formerly documentation systems such as the UDC used to be based on a narrow cooperation between subject specialists and information specialists (and reflect updated scientific knowledge) today it no longer has access to these kinds of resources.

 Many ontologies, on the other hand (e.g., the Gene Ontology) are considered important parts of research infrastructures and have better access to qualified manpower (although, as described by Leonelli (2016, 32) “Curators are well aware that the success of their products depends on how useful they prove to be to biologists, as this determines the levels of funding and community support that they will receive. Their careers depend at least in part on their ability to identify, embrace, and constructively engage with as many epistemic cultures in biology as possible.” (Although this perhaps can be understood as a need among curators to broaden their metadata of economic reasons rather than provide optimal systems for highly specialized research fields). However, I have no good idea on what to add to the article on this point. (NO ACTION TAKEN)

In fact another interpretation of footnote 17 could be that ontologies impose the constraints (sometimes simplifications) required by computer logic and the definition of classes, inheritance and hierarchy that are proper of object oriented programming, while the scope notes and the "logical inconsistencies" of thesauri could present a more flexible opportunity to explain and represent the complexities of science in a more humanistic/verbal way. From my point of view, constraining all the complexities of what is being organized into a set of relationships as the ones expressed by Smith et al (2005) on p.13 is just making up reality, an illusion of reality for the sake of efficiency and consistency. Even if the number of relationships used and construed has no limit as the author on lines 422-423, it could be argued that unless techniques such as fuzzy logic or paraconsistent logic are used, it is not only the formal definition of relationships but also the logic that is used what constraints the representation.

REPLY BY AUTHOR: These are highly interesting questions. I have added two footnotes about these issues (Footnote 21 and Footnote 23)

- Minor spelling typos were detected (e.g. p.2, line 67: "You just start construing a classification and stops when it seems to suit the purpose", p.4 footnote 5 "The study of how different parts of bot the", p.8 footnote 12 "who found that important work is more")

REPLY BY AUTHOR: These errors have been corrected, and the whole paper proof-read again.

-On p.16. Herre's own position and the disagreements with Smith’s might not be clear enough for the reader. It would be beneficial for the text if the author includes a brief explanation here.

REPLY BY AUTHOR:  this question is now addressed in Footnote 34

Reviewer 2 Report

Research by BH in Knowledge Organization is extremely prominent. Although this work reviews previous studies by the author and many other relevant authors, I reckon that every single approach to KO and IR is well argumented and explained. It is a very convenient summary of the research in both fields and, therefore, very useful for academics, information professionals and advanced students.

Author Response

Comments and Suggestions for Authors

Research by BH in Knowledge Organization is extremely prominent. Although this work reviews previous studies by the author and many other relevant authors, I reckon that every single approach to KO and IR is well argumented and explained. It is a very convenient summary of the research in both fields and, therefore, very useful for academics, information professionals and advanced students.

Reply by author: Thanks for the kind words. (No action taken)

Reviewer 3 Report

The paper is of relevant and exciting topic for the readers of journal Information. It is discussing a really timely, actual, and interesting issue, namely the relationship of information retrieval and knowledge organization from a philosophical perspective.

The paper is supported by theoretical foundations well, we get a quite good insight into the context. Providing the philosophical approach with the toolset of conceptualization is one of the strengths of the paper.

I recommend the paper for publication but after some improvements and modifications only:

• There is no introduction in the paper; it starts with a kind of mixture of theoretical foundation and literature review (while separated, comprehensive literature review isn't in the text). Writing an introduction at the beginning of the paper is a must: here the goal of research, the research questions could be set and the context, limitations could be highlighted.
• In the conclusions, it should be demonstrated, why this research is useful for contemporary science and practice, what is the relevance of the results on our nowaday's circumstances?
• The article is too long, it should be shortened by few pages.
• The technical quality of the figures is very low; these should be reedited and resized in order to fit the journal's requirements and for the aesthetic appearance.

Author Response

Comments and Suggestions for Authors

The paper is of relevant and exciting topic for the readers of journal Information. It is discussing a really timely, actual, and interesting issue, namely the relationship of information retrieval and knowledge organization from a philosophical perspective.

The paper is supported by theoretical foundations well, we get a quite good insight into the context. Providing the philosophical approach with the toolset of conceptualization is one of the strengths of the paper.

I recommend the paper for publication but after some improvements and modifications only:

• There is no introduction in the paper; it starts with a kind of mixture of theoretical foundation and literature review (while separated, comprehensive literature review isn't in the text). Writing an introduction at the beginning of the paper is a must: here the goal of research, the research questions could be set and the context, limitations could be highlighted.

REPLY BY AUTHOR: I have added an introduction and the goal of the article is added here.

• In the conclusions, it should be demonstrated, why this research is useful for contemporary science and practice, what is the relevance of the results on our nowaday's circumstances?

REPLY BY AUTHOR: In the existing conclusion, I have now added that the article has met the goal formulated in the introduction.

• The article is too long, it should be shortened by few pages.

REPLY BY AUTHOR: Unfortunately, I have not been able to shortening the paper (on the contrary, as two reviewers demanded some additions). My widespread use of footnotes is, however, an attempt to remove stuff from the article itself in order to improve its readability.

• The technical quality of the figures is very low; these should be reedited and resized in order to fit the journal's requirements and for the aesthetic appearance.

REPLY BY AUTHOR: I have improved three of the four figures (and adjusted the size of Figure 2 ).