Artificial Endoscopy and Inflammatory Bowel Disease: Welcome to the Future

Artificial intelligence (AI) is assuming an increasingly important and central role in several medical fields. Its application in endoscopy provides a powerful tool supporting human experiences in the detection, characterization, and classification of gastrointestinal lesions. Lately, the potential of AI technology has been emerging in the field of inflammatory bowel disease (IBD), where the current cornerstone is the treat-to-target strategy. A sensible and specific tool able to overcome human limitations, such as AI, could represent a great ally and guide precision medicine decisions. Here we reviewed the available literature on the endoscopic applications of AI in order to properly describe the current state-of-the-art and identify the research gaps in IBD at the dawn of 2022.


Introduction
Crohn's disease (CD) and ulcerative colitis (UC) are chronic inflammatory bowel disease (IBD), with increasing incidence all around the world and a great impact on general well-being, social functioning, and utilization of healthcare resources [1,2]. The diagnosis of IBD is a daily challenge for physicians, being based on different elements such as clinical data, biochemical values, radiology, endoscopy, and histology [3]. Among them, endoscopy represents a cornerstone in the diagnosis and follow-up of CD and UC [4,5].
In the last five years, the concept of endoscopy has evolved from a traditional one to a new idea based on artificial intelligence (AI). AI is defined as any machine that has cognitive functions mimicking humans for problem solving or learning [6]. AI has already been tested in several fields of endoscopy, such as in the detection of Barrett's esophagus [7] or the evaluation of adenoma detection rate during colonoscopy [8,9].
Attention has shifted to the potential role of AI in the field of IBD where endoscopic activity is based on several scores, such as the Mayo endoscopic subscore (MES), the Ulcerated Colitis Endoscopic Index of Severity (UCEIS), the Crohn's Disease Endoscopic Index of Severity (CDEIS), the Lewis score, and the Capsule Endoscopy Crohn's Disease Activity Index (CECDAI) [10][11][12][13][14]. The reason for this large number of scores lays in the need for establishing a strict definition of disease activity, thus reducing the interobserver variability and having a solid comparative analysis of different patients or studies [15]. In this context AI could be a great step forward in the research of homogeneity and reproducibility of endoscopic data. This article aims to summarize the literature data on AI endoscopic applications in the field of IBD, underlining the strengths and limitations of the currently available tools at the dawn of 2022.

What Is Artificial Intelligence and Its Current Application in Endoscopy?
AI-assisted endoscopy is based on computer algorithms that perform as human brains do [16]. They react (output) to what they receive as information (input) and what they have learned when built. The fundamental principle of this technology is "machine learning" (ML) [17].
There are many different ML methods ( Table 1) and one of the most popular is the use of artificial neural networks (ANN) [18]. ANN is based on multiple interconnected layers of algorithms, which process data in a specific pattern and feed data so that the system can be trained to carry out a specific task [19]. Another diffuse ML method is the Supportvector machine (SVM), which is used for classifying data sets by creating a line or plane to separate data into distinct classes [20]. An evolution of ML is deep learning (DL): a complex, multilayer neural network architecture learns representations of data automatically by transforming the input information into multiple levels of abstractions [21,22]. An evolution of the simpler ANN is the convolution neural network (CNN), inspired by the response of human visual cortex neurons to a specific stimulus and being able to convolve the input and pass its result to the next layer [19,23]. Table 1. Algorithms involved in machine learning process.

Supervised
The algorithm is trained by labeling data tagged with the correct answer

Semisupervised
The algorithm is trained without marking the training data

Unsupervised
The algorithm is structured on a large amount of unlabeled data based on a small amount of labeled data Based on this technology, three kinds of tools have been generated to support endoscopy in each part of its activity [24][25][26]: -Computer-aided detection (CADe), which detects gastrointestinal lesions; -Computer-aided diagnosis (CADx), which characterizes gastrointestinal lesions; -Computer-aided monitoring (CADm), which evaluates the procedure and the endoscopist, thus improving the quality of endoscopy.
In particular, CADe and CADx are the best developed systems with many experiences around the world demonstrating their better performance than the human eye [9,[27][28][29]; for example, the GI-Genius Medtronic system reached a sensibility of 99.7% in polyps' detection as shown by Hassan et al. [27]. The application fields of AI are expanding rapidly and IBD is the next target of this innovative technology.

AI in the Diagnosis of IBD
One of the first applications of AI has been the attempt to facilitate the diagnosis of IBD and the differential diagnosis between CD and UC. In the model of Mossotto [30], three supervised ML models were developed utilizing endoscopic data only, histological only, and combined endoscopic/histological with an accuracy of 71.0%, 76.9%, and 82.7%, respectively [30]. The model combining endoscopic and histological data was tested on a statistically independent cohort of 48 pediatric patients from the same clinic, with an accuracy of about 83.3% in patients' classification.
Quénéhervé and colleagues [31] tried to design a model to diagnose IBD and establish differential diagnoses between CD vs. UC. They based their study on confocal laser endomicroscopy (CLE), which is an adaptation of light microscopy whereby focal laser illumination is combined with pinhole limited detection to geometrically reject out-of-focus light [32]. The authors built a score based on 14 functional and morphological parameters to perform a quantitative analysis of the mucosa called cryptometry and detect a diagnosis of IBD with a sensitivity and a specificity to near 100%. Moreover, this study reached a sensitivity of 92.3% and a specificity of 91.3% in the differential diagnosis between CD and UC.
Diagnosis of IBD can be a complex and challenging procedure due to its heterogeneous presentation. It is generally believed that making a correct diagnosis requires information on the endoscopic and histological features, together with clinical and biochemical data. AI support may be helpful in the diagnostic process by combining all suggestive features intelligently.

AI in UC, State-of-the-Art
As previously underlined, endoscopy plays a fundamental role in the diagnosis and assessment of IBD activity [5]. According to this concept, endoscopy should guarantee an exact staging of the disease and a high level of concordance between different operators. Indeed, the definition of recurrence or the assessment of remission are cornerstones in the disease management, thus guiding the next clinical or surgical decisions [33,34].
In the study of Ozawa, the authors designed a CAD system using a CNN and evaluated its performance in the identification of normal or inflamed mucosa, using a large dataset of endoscopic images from patients with UC [35]. The performance of this new tool was valuable, with areas under the receiver operating characteristic curves (AUROCs) of 0.86 and 0.98 in the identification of MES 0 (completely normal mucosa) and MES 0-1 (mucosal healing state), respectively [35]. In a similar experience from Stidham et al. [36] a CNN showed an AUROC of 0.96 in distinguishing endoscopic remission (MES = 0 or 1) from moderate to severe disease (MES = 2 or 3), with a good weighted κ agreement between the CNN and the adjudicated reference score for identifying exact MES (κ = 0.84; 95% CI, 0.83-0.86). The application of this CNN to the entirety of the colonoscopy videos had high accuracy in identifying moderate to severe disease with an AUROC of 0.97 [36].
Moreover, Gottlieb and colleagues [37] developed another recurrent neural network able to predict MES and UCEIS from entire endoscopy videos and not only from images. The system automatically selected the frame to be analyzed and scores were calculated on the colon section, showing high agreement with the human central reader score [37]. Similarly, a fully automated video analysis system was developed to assess the grade of UC activity and predicted MES in 78% of videos (κ = 0.84). In external clinical trial videos, reviewers agreed on MES in 82.8% of videos (κ = 0.78) [38]. Automated MES grading of clinical trial videos (often low resolution) correctly distinguished remission (MES = 0 or 1) vs. active disease (MES = 2 or 3) in 83.7% of videos. Not only were automated systems able to assess endoscopic activity from still images [39], but they were also able to predict a binary version of the MES directly analyzing a raw colonoscopy video, resulting in a high level of accuracy (AUC of 0.94 for MES ≥ 1 and 0.85 for MES ≥ 2 and MES ≥ 3) [40]. Looking forward, it seems that AI can also guide real-time therapy decisions in patients with UC in clinical remission by helping to stratify the relapse risk one year after AI-assisted colonoscopy [41].
Other experiences pushed forward the application of AI in the prediction of histology. Indeed, Takenaka and colleagues [42] designed a deep neural network algorithm, defined as DNUC, based on more than 40,000 images from colonoscopies and 6000 biopsies of 875 patients prospectively collected. AI system evaluations were matched with the UCEIS score expressed for each image by three expert endoscopists and with the Geboes score determined by pathologists [43]. The DNUC revealed an accuracy of 90.9% and 92.9% in the detection of endoscopic and histological remission, respectively. In addition, Maeda et al. [44] developed a CADx system to predict persistent histological inflammation using endocytoscopy in 187 retrospectively collected patients. Endocytoscopy is one of the most valuable technologies, although it is not widely available in endoscopic departments. Providing ultra-high-resolution white light images (520x), endocytoscopy allows the socalled virtual histology or optical biopsy [45]. The results obtained by the CAD algorithm were compared with the Geboes score defined by five expert pathologists, blinded from endoscopist results. The algorithm showed a sensitivity of 74% and a specificity of 97%, with high level of reproducibility and interobserver agreement (κ value = 1).
Honzawa and colleagues [46] moved forward with the AI-application in trying to differentiate between MES 0 and MES 1 in patients with UC in clinical remission. The authors investigated the correlation among the so-called MAGIC score (Mucosal Analysis of Inflammatory Gravity by i-scan TE-c Image), MES, and histological Geboes score. Interestingly, the MAGIC score, based on the level of mean inflammation derived from all the pixels, was significantly higher in the MES 1 group than in the MES 0 group (p = 0.0034), with a significant correlation with histology (p = 0.015).
Similar to the color map of the MAGIC score, a validation study [47] elaborated an operator-independent, computer-based tool, named Red Density (RD), that determined disease activity in UC according to a redness map and vascular pattern recognition. The RD score, which is different from the previous exposed experiences as it is based on pure physics parameters, significantly correlated with the histological scoring systems (Robarts Histopathology index, r = 0.74) and with MES and UCEIS endoscopic scores with r = 0.76 and 0.74, respectively. Some weak points of this work are the monocentric experience, the small population (29 patients), and the analysis being performed only on the single picture and not on the entire colonic segment. However, this study represents an important application of AI as testified by the high level of performance. Notably, the algorithm structure does not require as much information as the CNN system due to the possibility of sequential modulation of the algorithm during the development.
Finally, a multicenter study in inactive patients with UC (PRognOstiC valuE of rEd Density in Ulcerative Colitis: PROCEED-UC; NCT04408703) is planned to assess the predictive value of the RD score for sustained clinical remission. It is plausible that the RD score might be used in the future as the first objective operator-independent endoscopic target in a treat-to-target strategy in UC. The main characteristics of the studies on endoscopic AI application in IBD are summarized in Table 2.

AI in CD, State-of-the-Art
In the field of CD, AI has been mostly applied on video capsule technology (Table 3), which has been assuming an important role both in the diagnosis and assessment of mucosa healing in the small bowel [48]. In the current European Crohn's and Colitis Organisation (ECCO) guidelines, patients suspected to have CD but with a negative endoscopy should undergo a second level diagnostic method such as magnetic resonance imaging (MRI) or video capsule endoscopy [4]. Moreover, even in cases of normal imaging tests, such as MRI and clinical signs suspicious of small bowel CD (e.g., elevated calprotectin and/or unexplained iron deficiency anemia), video capsule endoscopy is indicated to exclude small bowel involvement [4]. However, the use of video capsules has some limitations, such as the collection of a huge amount of data and the duration of the analysis [48]. AI may overcome these barriers by selecting the frame or the part of video needed for the assessment and cutting off the time for diagnosis, thus requiring a limited amount of data to store. The first experience was conducted about 10 years ago. Girgis et al. [49] built a system that identified the inflamed regions after a SVM training, with an accuracy of 87%, sensitivity of 93%, and specificity of 80%. Two years later, Kumar et al. [50] developed a similar system with a precision of about 90% in detecting CD lesions. Lately, several studies have been conducted for the development of systems able to automatically detect ulcers and/or aphthae and to grade mucosal damage.
A novel filtering process, called hybrid adaptive filtering (HAF), was proposed for efficient extraction of lesion-related characteristics using wireless capsule endoscopy. This system was trained on 800 images collected by 13 different patients and offered high performances in the detection of severe lesions (93.8% of accuracy, 95.2% of sensitivity, 92.4% specificity, and 92.6% of precision) [51]. The group of Klang provided two experiences in this direction [52,53] . The former showed an AUC of 0.99 with an accuracy ranging from 95.4% to 96.7% in classifying images into either normal mucosa or mucosa with ulcers [52]. The latter exhibited a good accuracy of 93.5% [±6.7%] in classifying strictures vs. nonstrictures [53] .
A CNN was trained to detect erosions and ulcers, demonstrating performances comparable with the activity of two expert gastroenterologists, with an AUC of 0.96 for the detection of abnormalities [54]. Interestingly, a consensus reading was used to train another CNN in automatic grading of images of CD ulcers. The resulting algorithm was tested against capsule readers, showing high accuracy in classifying severe ulcers (0.91 for grade 1 vs. grade 3 ulcers compared to 0.6 for grade 1 vs. 2) [55].
DL methods for autonomous detection and classification of CD lesions have also been applied to panenteric capsule endoscopy system that is now available allowing simultaneous investigation of the small bowel and colon. AI technology has increased the diagnostic yield and reduced interobserver variability in this integrated procedure [56,57].
Not only did AI show a high level of performance, but also a significantly faster reading with an average time of 3.5 minutes against 50 minutes for a full video of capsule endoscopy [52,58].
Some limitations of these works warrant attention. Firstly, they were made on single images and not on the entire video so that the analysis was not able to provide an overall evaluation of the validated scores for video capsule (e.g., the Lewis score). Moreover, they are retrospective cohort studies based on restricted samples of patients.
Nevertheless, all these experiences could give a great impulse to capsule endoscopy in CD. The inflammation in the proximal bowel is correlated with a worst prognosis and a higher surgical risk [59], therefore a modern method of analysis with high sensitivity and specificity is eagerly awaited in clinical practice [60].

AI for the Detection of Neoplasms in Long-Standing IBD
Given the increased risk for developing colorectal neoplasia, surveillance colonoscopy plays an important role in the management of UC [61]. The gold standard method for dysplasia surveillance is chromoendoscopy, which utilizes indigo carmine or methylene to better define the superficial gastrointestinal mucosa [62]. New endoscopic imaging technologies such as virtual chromoendoscopy, autofluorescence imaging, CLE, and endocytoscopy are now emerging, but there are only a few reports about the application of AI-assisted colonoscopy techniques for the early diagnosis of colorectal cancer [5].
The AI capacity has been tested in the detection of colorectal neoplasia (Figure 1) but not specifically in patients with IBD.
The first experience is a case report of Maeda and colleagues [63] where the Endo-BRAIN eye system was tested for detecting dysplasia in a patient with long-standing UC. This system is able to identify colorectal lesions with high accuracy in general population [64], but in this case it proved to support endoscopists in the identification of UC-associated dysplasia, which is not always easy to detect due to its flat appearance and unclear boundaries. methylene to better define the superficial gastrointestinal mucosa [62]. New endoscopic imaging technologies such as virtual chromoendoscopy, autofluorescence imaging, CLE, and endocytoscopy are now emerging, but there are only a few reports about the application of AI-assisted colonoscopy techniques for the early diagnosis of colorectal cancer [5].
The AI capacity has been tested in the detection of colorectal neoplasia (Figure 1) but not specifically in patients with IBD. The first experience is a case report of Maeda and colleagues [63] where the Endo-BRAIN eye system was tested for detecting dysplasia in a patient with long-standing UC. This system is able to identify colorectal lesions with high accuracy in general population [64], but in this case it proved to support endoscopists in the identification of UCassociated dysplasia, which is not always easy to detect due to its flat appearance and unclear boundaries.
Another example of AI-support in the detection of dysplasia was reported by Fukunaga [65]. In this case report, EndoBRAIN system helped endocitoscopy in the detection of high-grade dysplasia in a patient with long-standing UC who subsequently underwent an endoscopic submucosal dissection. To note, colitis-associated colorectal cancer may be generally difficult to diagnose due to consequences of inflammation on mucosal appearance ( Figure 2) and the use of EndoBRAIN could help non-expert endoscopists to identify lesions. These experiences underline the potential and future role of AI in the colitis-associated dysplasia and neoplasia detection during IBD surveillance. Another example of AI-support in the detection of dysplasia was reported by Fukunaga [65]. In this case report, EndoBRAIN system helped endocitoscopy in the detection of high-grade dysplasia in a patient with long-standing UC who subsequently underwent an endoscopic submucosal dissection. To note, colitis-associated colorectal cancer may be generally difficult to diagnose due to consequences of inflammation on mucosal appearance ( Figure 2) and the use of EndoBRAIN could help non-expert endoscopists to identify lesions. These experiences underline the potential and future role of AI in the colitis-associated dysplasia and neoplasia detection during IBD surveillance.

Conclusions and Future Perspectives
AI is a cornerstone revolution in endoscopy. In the field of IBD, its primary applications are providing great results in the diagnosis and staging of the disease. In a field of medicine where the current mantra is the treat-to-target strategy and where treatment directions are guided by endoscopic remission, a sensible and specific tool able to overcome human limitations could represent a great ally. High-performing diagnostic aids with low variability are useful in the detection and standardization of results and in the targets' assessment. Moreover, if mucosal healing could be perceived as a realistic target, a concept that moves forward and takes to the extreme the previous idea is disease clearance. Even though a clear definition is still lacking, this objective includes simultaneous clinical, endoscopic, and histological remission of disease. It follows that the modern algorithms presented in the current review could help in the detection of this ambitious goal.
All the reported experiences improved the awareness about AI potential strengths

Conclusions and Future Perspectives
AI is a cornerstone revolution in endoscopy. In the field of IBD, its primary applications are providing great results in the diagnosis and staging of the disease. In a field of medicine where the current mantra is the treat-to-target strategy and where treatment directions are guided by endoscopic remission, a sensible and specific tool able to overcome human limitations could represent a great ally. High-performing diagnostic aids with low variability are useful in the detection and standardization of results and in the targets' assessment. Moreover, if mucosal healing could be perceived as a realistic target, a concept that moves forward and takes to the extreme the previous idea is disease clearance. Even though a clear definition is still lacking, this objective includes simultaneous clinical, endoscopic, and histological remission of disease. It follows that the modern algorithms presented in the current review could help in the detection of this ambitious goal.
All the reported experiences improved the awareness about AI potential strengths and limitations. Most were nonrandomized and retrospective with small sample sizes. In addition, very limited studies were conducted to test AI support in the detection of dysplasia and neoplasia in patients with IBD. We believe these limitations should be overcome before AI becomes part of real-life practice.
In the context of AI and big data, a future perspective is the creation of algorithms for diagnosis and monitoring of IBD based not only on endoscopic, but also on clinical and histological data in order to have a complete overview of all disease features.