Next Article in Journal
Student Preferences about University Education Department Instructor Attire
Previous Article in Journal
Transformation for a Post-Pandemic World: Exploring Social Innovations in Six Domains
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Brief Report

Knowledge Development in Artificial Intelligence Use in Paediatrics

Peter Kokol
Helena Blažun Vošner
2,3,4 and
Jernej Završnik
Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia
Community Healthcare Centre Dr. Adolf Drolc Maribor, 2000 Maribor, Slovenia
Alma Mater Europaea—ECM, 2000 Maribor, Slovenia
Faculty of Health and Social Sciences Slovenj Gradec, 2380 Slovenj Gradec, Slovenia
Faculty of Natural Sciences and Mathematics, University of Maribor, 2000 Maribor, Slovenia
Science and Research Center Koper, 6000 Koper, Slovenia
Author to whom correspondence should be addressed.
Knowledge 2022, 2(2), 185-190;
Submission received: 1 February 2022 / Revised: 5 March 2022 / Accepted: 18 March 2022 / Published: 30 March 2022


The use of artificial intelligence in paediatrics has vastly increased in the last few years. Interestingly, no historical bibliometric study analysing the knowledge development in this specific paediatric field has been performed yet, thus our study aimed to close this gap. References Publication Years Spectrography (RPYS), more precisely Cited Reference Explorer (CRE) software tool was employed to achieve this aim. We identified 28 influential papers and domain experts validation showed that both, the RPYS method and CRE tool performed adequately in the identification process.

1. Introduction

The use of artificial intelligence (AI) in medicine can be traced back to 1963 when Rand Corporation published its memorandum on AI and brain mechanisms [1]. The descriptions of some of the first practical applications were published in 1968. In these applications AI was used to support diagnosing [2], and using decision trees in prognosing drug effects in children [3]. Few years later Shortliffe et al. [4] presented the expert system Mycin, developed in 1971, which was able to identify bacteria causing severe blood infections and to recommend antibiotics. Despite the fact that Mycin outperformed members of the Stanford medical school in the accuracy of diagnosis it was never used in practice due to a legal issue -who do you sue if it gives a wrong diagnosis? In the same year when Mycin was developed Bellanti & Schlegel [5] published a study in which they presented a paediatric AI application supporting the diagnosing of immune deficiency diseases. However, the use of AI in medicine become really popular only in 2016, when the IBM AI platform Watson correctly and spectacularly diagnosed and proposed an effective treatment for a 60-year-old woman’s rare form of leukemia [6]. Since then more than 12,000 papers presenting AI application in paediatrics were published covering themes such as use of machine learning in public health, deep learning in image and signal processing, diagnosing, emotion recognition and serious games for health education [7], indicating that AI become a hot research topic also in paediatrics.
Some recent review papers including a bibliometric based review analysed the use of AI in paediatrics in general [8,9,10] and further ones reviewing the AI use in paediatrics subspecialities such as: radiology [11,12], prematurity [13,14], surgery [15,16] or cardiology [17,18]. Even though studying the history of and developing knowledge in medical fields is widely recognized [19], the above publications failed to present a structured and systematic analysis of how the use of AI in paediatrics has progressed. Namely, understanding the history of knowledge development. enables medical professionals to learn from past experiences and influence the practice in clinical environments in more efficient ways. The aim of our current study was to close this gap.
Knowledge development and its impact on the society in general can be studied in various ways. The traditional approach is to analyze citation patterns [20]. A more recent approach emerging lately is called Altmetrics. Altmetrics is a set of quantitative and qualitative metrics, which complements citation metrics. Altmetrics scans mentions and discussions in social and research networks such as Mendeley, Twitter, Facebook, mainstream media, research blogs, reference manager bookmarks and similar [21,22]. Despite Altmetrics raising popularity and impact it is argued that Altmetrics is not sufficient to determine the scientific value of the research output and is also more biased (for example double tweeting [23]) than traditional bibliometrics based approaches [24,25]. Additionally, it has also been shown that tweeting and consequently Altmetrics influence citation rates [26], Due to above reasons we decided to employ the traditional knowledge development analysis in our study.

2. Materials and Methods

The notion of historical roots as important or influential publications in a specific research area (SRA) was introduced by Robert K. Merton in 1985 [27]. Since than, historical roots and influential papers identification become part of the scientific discourse in scientometrics and bibliometrics research. Simply, the number of citations that SRA publications received seems to be an obvious measure to identify SRA’s historical roots. However, SRA publications might be cited (frequently) also by publications outside the SRA, which suggests that such publication may have impact beyond SRA, but not in the SRA in question. Moreover, the SRA’s cited publications may not be indexed in bibliographic databases at all. Consequently, a method called References Publication Years Spectroscopy (RPYS) has been developed to overcome this problem. One of the core software tools implementing RPYS is the Cited Reference Explorer (CRE; 10 February 2022) [28,29], which we employed in our study. The method has been already successfully used in different medical SRAs [30,31]. CRE analyses references’ publication years and aggregate the number of cited references over time on a spectrogram. Pronounced peaks indicates the years when historical roots/influential papers were published. In addition to the spectrogram CRE offers some other tools to identify influential publications. In our study we used the CRE tabular output to identify historical roots for the early period and the CRE spectrogram for the recent period. Additionally, CRE indicators, namely N_TOP10, which reveals publications that were among the 10% most cited publications over a longer period and Sleeping beauties (SB) indicators were also used. An SB is a publication that goes unnoticed (sleeps) for a long time and then almost suddenly become highly cited and turns out to be interesting (awakens). SBs frequently presented important innovations and their awakening can be associated with important paradigm shifts in science [32,33]. The identified influential publications were reviewed by five domain experts coming from computer and health science fields to informally asses the accuracy of the identification [34].
As part of the analysis, we created a corpus of publications related to the application of AI in paediatrics using the SCOPUS bibliographic database employing the search string (“machine learning” or “rough sets” or (“decision tree*” and (induction or heuristic)) or “artificial neural network*” or “support vector machines” or “rough sets” or “deep learning” or “intelligent systems” or “artificial intelligence”) and (newborn* or toddler* or child* or adolescent*) and exported their metadata to the CRE. The search was performed on 3rd of October 2021.

3. Results

The search resulted in 12,360 publications citing 977,812 references. After removing duplicates 781,526 references remained and were analysed with the CRE. The analysis resulted in 28 historical roots/influential papers, which are presented in Figure 1 as peaks in the blue line and compiled in Table 1. The oldest cited reference, namely Hobbes Leviathan [35] was published in 1651. Regarding the content, the references can be divided into three groups:
  • Publications concerned with general intelligence, emotions, psychology, behaviour and functioning of brains,
  • Publications concerning statistical and mathematical methods, and
  • Computer algorithms, tools and languages used for building artificial intelligent systems.
Knowledge development vent trough several stages, namely (1) researching general intelligence, emotions and attitudes, (2) establishing statistical and mathematical foundations, (3) brain and neural networks research, (4) computing machinery and intelligence (5) development of AI algorithms and their applications in paediatrics, and (6) Deep learning applications in paediatrics.

4. Discussion

During the expert review it was concluded that all of the identified influential publications might have contributed to the development of Artificial intelligence in general and its use in paediatrics, however some interesting observations have been made, which could lead to further investigations:
  • Robotic surgery is quite routinely used in medicine, and “emotion teaching” or “stress reduction robots” are widely used in paediatrics, but none influential papers related to robotics or history of robotics were identified. It could be the case that earliest papers on robotics were not scientific papers, but less known technical reports. Earlier mentions of Greek, Persian and Medieval automata are also not documented (or that documentation has been destroyed or lost) and become known only recently in various histories of AI.
  • Statistical and mathematical papers did not directly contribute to the development of AI, however advanced statistics and mathematics are used in proving of AI algorithms, especially in machine learning.
  • There are a lot of influential papers related to prominent machine learning algorithms, such as neural networks, deep learning, decision trees and fuzzy sets, and also in paediatrics rarely used algorithms like nearest neighbors, however influential publications on popular algorithms such as support vector machines or Bayes are missing.

5. Conclusions

In terms of methodologically, RPYS supported by CRE proved to be a viable approach to identify historical roots or influential papers. Addtionaly. experts also agreed that the analysis of knowledge development based on historical roots identification can significantly contribute to the understanding specific research fields, provide learning from past experiences, or provide motivation for further research and consequently enable health care professionals to influence the everyday practice settings using historical evidence.

Author Contributions

Conceptualization, P.K. and J.Z.; methodology, P.K.; validation, P.K., J.Z. and H.B.V.; formal analysis, P.K.; investigation, H.B.V.; resources, J.Z.; data curation, P.K.; writing—original draft preparation, P.K.; writing—review and editing, J.Z., H.B.V.; visualization, H.B.V.; supervision, J.Z. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Maron, M.E. Artificial Intelligence and Brain Mechanisms; RAND Corporation: Santa Monica, CA, USA, 1963. [Google Scholar]
  2. Paycha, F. Diagnosis with the aid of artificial intelligence: Demonstration of the 1st diagnostic machine. Presse Therm. Clim. 1968, 105, 22–25. [Google Scholar] [PubMed]
  3. Klein, D.F.; Honigfeld, G.; Feldman, S. Prediction of Drug Effect by Diagnostic Decision Tree. Dis. Nervous Syst. 1968, 29, 159–187. [Google Scholar]
  4. Shortliffe, E.H.; Davis, R.; Axline, S.G.; Buchanan, B.G.; Green, C.C.; Cohen, S.N. Computer-Based Consultations in Clinical Therapeutics: Explanation and Rule Acquisition Capabilities of the MYCIN System. Comput. Biomed. Res. 1975, 8, 303–320. [Google Scholar] [CrossRef]
  5. Bellanti, J.A.; Schlegel, R.J. The Diagnosis of Immune Deficiency Diseases. Pediatr. Clin. N. Am. 1971, 18, 49–72. [Google Scholar] [CrossRef]
  6. Brusco, S. IBM’s Watson Diagnosed A Rare Condition That Left Doctors Stumped—Medical Product Outsourcing. Available online: (accessed on 11 October 2021).
  7. Kokol, P.; Završnik, J.; Blažun Vošner, H. Artificial Intelligence in Paediatrics: Benefits and Pitfalls. In Proceedings of the Europaeditrics Conference, Healthmanagement, Zagreb, Croatia, 7–9 October 2021. [Google Scholar]
  8. Jamnik, J. Artificial Intelligence in Paediatrics: A Checkup. Medicine 2019, 96, 27–29. [Google Scholar]
  9. Kokol, P.; Završnik, J.; Blažun Vošner, H. Artificial Intelligence and Pediatrics: A Synthetic Mini Review. Pediatr. Dimens. 2017, 2, 1–5. [Google Scholar] [CrossRef] [Green Version]
  10. Shu, L.-Q.; Sun, Y.-K.; Tan, L.-H.; Shu, Q.; Chang, A.C. Application of Artificial Intelligence in Pediatrics: Past, Present and Future. World J. Pediatr. 2019, 15, 105–108. [Google Scholar] [CrossRef] [Green Version]
  11. Desai, S.B.; Pareek, A.; Lungren, M.P. Current and Emerging Artificial Intelligence Applications for Pediatric Interventional Radiology. Pediatr. Radiol. 2021, 2, 1. [Google Scholar] [CrossRef]
  12. Otjen, J.P.; Moore, M.M.; Romberg, E.K.; Perez, F.A.; Iyer, R.S. The Current and Future Roles of Artificial Intelligence in Pediatric Radiology. Pediatr. Radiol. 2021, 2021, 7. [Google Scholar] [CrossRef]
  13. Reid, J.E.; Eaton, E. Artificial Intelligence for Pediatric Ophthalmology. Curr. Opin. Ophthalmol. 2019, 30, 337–346. [Google Scholar] [CrossRef]
  14. Scruggs, B.A.; Chan, R.V.P.; Kalpathy-Cramer, J.; Chiang, M.F.; Campbell, J.P. Artificial Intelligence in Retinopathy of Prematurity Diagnosis. Trans. Vis. Sci. Technol. 2020, 9, 5. [Google Scholar] [CrossRef] [Green Version]
  15. Gödeke, J.; Muensterer, O.; Rohleder, S. Künstliche Intelligenz in der Kinderchirurgie. Chirurg 2020, 91, 222–228. [Google Scholar] [CrossRef] [PubMed]
  16. Wang, E.; Brenn, B.R.; Matava, C.T. State of the Art in Clinical Decision Support Applications in Pediatric Perioperative Medicine. Curr. Opin. Anaesthesiol. 2020, 33, 388–394. [Google Scholar] [CrossRef] [PubMed]
  17. Gaffar, S.; Gearhart, A.S.; Chang, A.C. The Next Frontier in Pediatric Cardiology. Pediatr. Clin. N. Am. 2020, 67, 995–1009. [Google Scholar] [CrossRef] [PubMed]
  18. Gearhart, A.; Gaffar, S.; Chang, A.C. A Primer on Artificial Intelligence for the Paediatric Cardiologist. Cardiol. Young 2020, 30, 934–945. [Google Scholar] [CrossRef] [PubMed]
  19. Feller, I.; Stern, P.C. A Strategy for Assesing Science, Behavioral and Social Research on Aging, N.R.C. (US) C. on A.B. and S.S.R.; National Academies Press: Washington, DC, USA, 2007. [Google Scholar]
  20. Garfield, E. Citation Indexes for Science; a New Dimension in Documentation through Association of Ideas. Science 1955, 122, 108–111. [Google Scholar] [CrossRef] [PubMed]
  21. What Are Altmetrics? Altmetric 2015. Available online: (accessed on 10 February 2022).
  22. Rosenkrantz, A.B.; Ayoola, A.; Singh, K.; Duszak, R., Jr. Alternative Metrics (“Altmetrics”) for Assessing Article Impact in Popular General Radiology Journals. Acad. Radiol. 2017, 24, 891–897. [Google Scholar] [CrossRef]
  23. Cao, R.; Geng, Y.; Xu, X.; Wang, X. How Does Duplicate Tweeting Boost Social Media Exposure to Scholarly Articles? J. Inf. 2022, 16, 101249. [Google Scholar] [CrossRef]
  24. Sener, H.; Polat, O.A. Altmetric analysis of the most-cited 100 articles on the retina published between 2010 and 2020. Retina 2022, 42, 283–289. [Google Scholar] [CrossRef]
  25. Kunze, K.N.; Manzi, J.E.; Polce, E.M.; Vadhera, A.; Bhandari, M.; Piuzzi, N.S. High Social Media Attention Scores Are Not Reflective of Study Quality: An Altmetrics-Based Content Analysis. Int. Emerg. Med. 2022, 2022, 3. [Google Scholar] [CrossRef]
  26. Luc, J.G.Y.; Archer, M.A.; Arora, R.C.; Bender, E.M.; Blitz, A.; Cooke, D.T.; Hlci, T.N.; Kidane, B.; Ouzounian, M.; Varghese, T.K., Jr.; et al. Does Tweeting Improve Citations? One-Year Results From the TSSMN Prospective Randomized Trial. Ann. Thor. Surg. 2021, 111, 296–300. [Google Scholar] [CrossRef] [PubMed]
  27. Merton, R. On the Shoulders of Giants: A Shandean Postscript; The Vicennial Edition; Harcourt Brace Jovanovich: San Diego, CA, USA, 1985. [Google Scholar]
  28. Thor, A.; Marx, W.; Leydesdorff, L.; Bornmann, L. Introducing CitedReferencesExplorer (CRExplorer): A Program for Reference Publication Year Spectroscopy with Cited References Standardization. arXiv 2016, arXiv:1601.01199. [Google Scholar] [CrossRef] [Green Version]
  29. Thor, A.; Bornmann, L.; Marx, W.; Mutz, R. Identifying Single Influential Publications in a Research Field: New Analysis Opportunities of the CRExplorer. Scientometrics 2018, 116, 591–608. [Google Scholar] [CrossRef] [Green Version]
  30. Blažun Vošner, H.; Kokol, P.; Železnik, D.; Završnik, J. Identifying historical roots of knowledge development in cardiovascular nursing using bibliometrics. Int. J. Nurs. Pract. 2019, 25, e12726. [Google Scholar] [CrossRef]
  31. Kokol, P.; Blažun Vošner, H.; Završnik, J. Application of Bibliometrics in Medicine: A Historical Bibliometrics Analysis. Health Inf. Librar. J. 2021, 38, 125–138. [Google Scholar] [CrossRef]
  32. Kokol, P.; Blažun Vošner, H.; Završnik, J. Do Simultaneous Inventions Sleep? A Case Study on Nursing Sleeping Papers. Scientometrics 2020, 125, 2827–2832. [Google Scholar] [CrossRef]
  33. van Raan, A.F.J. Sleeping Beauties in Science. Scientometrics 2004, 59, 467–472. [Google Scholar] [CrossRef]
  34. Kokol, P.; Zavrsnik, J.; Blažun Vošner, H. Identifying Historical Roots in Paediatric Echocardiography Using RPYS. arXiv 2021, arXiv:2108.00789. [Google Scholar]
  35. Hobbes, T. Leviathan or the Matter, Forme, & Power of a Common-Wealth Ecclesiastical and Civill; Edward White and David Widger: London, UK, 1651. [Google Scholar]
Figure 1. Paediatric historical roots in area of using Artificial intelligence in paediatrics for the period 1950 (the year when A. Turings, Computing machinery and intelligence was published) to 2021. Peaks in blue curve represent years when important publications were published.
Figure 1. Paediatric historical roots in area of using Artificial intelligence in paediatrics for the period 1950 (the year when A. Turings, Computing machinery and intelligence was published) to 2021. Peaks in blue curve represent years when important publications were published.
Knowledge 02 00011 g001
Table 1. Historical roots in the area of using Artificial intelligence in paediatrics.
Table 1. Historical roots in the area of using Artificial intelligence in paediatrics.
Hobbes, T1651Leviathan Oldest
Darwin, C., 1872The expression of the emotions in man and animalsTabular
James, W., 1890Principles of PsychologyTabular
Pearson, K.,1901On lines and planes of closest fit to systems of points in space Tabular
Spearman, C.1904General intelligence, “objectively determined and measured”Tabular
Spearman, C.1904The proof and measurement of association between two things Tabular
Likert, R.1932A Technique for the Measurement of Attitudes Tabular
Hotelling, H.1933Analysis of a complex of statistical variables into principal components Tabular
Fisher, R.A1936The use of multiple measurements in taxonomic problems Tabular
Kanner, L.1943Autistic disturbances of affective contact Tabular
McCulloch, W.S., Pitts, W1943A logical calculus of the ideas immanent in nervous activity Tabular
Dice, L.R.,1945Measures of the amount of ecologic association between species Tabular
Hebb, D.O., (1949) 1949The Organization of BehaviourTabular
Turing, A.M.1950Computing machinery and intelligence Spectrogram
Zadeh, L.A1965Fuzzy sets Spectrogram
Cover, T., Hart, P1967Nearest neighbour pattern classification Spectrogram
Jennett, B., Bond, M1975Assessment of outcome after severe brain damage Spectrogram
Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.1984Classification and Regression TreesSpectrogram
Quinlan, J.R.1986Induction of decision trees NTop10
Rumelhart, D.E., Hinton, G.E., Williams, R.J.1986Learning representations by back-propagating errors NTop10
Quinlan, J.R.1993C4.5: Programs for Machine LearningNTop10
Bishop, C.M1995Neural Networks for Pattern RecognitionNTop10
Pedregosa, F.2011Scikit-learn: Machine learning in Python SB
Krizhevsky, A., Sutskever, I., Hinton, G.E.2012Imagenet classification with deep convolutional neural networks SB
LeCun, Y., Bengio, Y.2015Deep learning Spectrogram
Esteva, A., Kuprel, B., Novoa, R.A.2017Dermatologist-level classification of skin cancer with deep neural networks Spectrogram
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kokol, P.; Vošner, H.B.; Završnik, J. Knowledge Development in Artificial Intelligence Use in Paediatrics. Knowledge 2022, 2, 185-190.

AMA Style

Kokol P, Vošner HB, Završnik J. Knowledge Development in Artificial Intelligence Use in Paediatrics. Knowledge. 2022; 2(2):185-190.

Chicago/Turabian Style

Kokol, Peter, Helena Blažun Vošner, and Jernej Završnik. 2022. "Knowledge Development in Artificial Intelligence Use in Paediatrics" Knowledge 2, no. 2: 185-190.

Article Metrics

Back to TopTop