Colorectal Polyp Image Detection and Classification through Grayscale Images and Deep Learning

Colonoscopy screening and colonoscopic polypectomy can decrease the incidence and mortality rate of colorectal cancer (CRC). The adenoma detection rate and accuracy of diagnosis of colorectal polyp which vary in different experienced endoscopists have impact on the colonoscopy protection effect of CRC. The work proposed a colorectal polyp image detection and classification system through grayscale images and deep learning. The system collected the data of CVC-Clinic and 1000 colorectal polyp images of Linkou Chang Gung Medical Hospital. The red-green-blue (RGB) images were transformed to 0 to 255 grayscale images. Polyp detection and classification were performed by convolutional neural network (CNN) model. Data for polyp detection was divided into five groups and tested by 5-fold validation. The accuracy of polyp detection was 95.1% for grayscale images which is higher than 94.1% for RGB and narrow-band images. The diagnostic accuracy, precision and recall rates were 82.8%, 82.5% and 95.2% for narrow-band images, respectively. The experimental results show that grayscale images achieve an equivalent or even higher accuracy of polyp detection than RGB images for lightweight computation. It is also found that the accuracy of polyp detection and classification is dramatically decrease when the size of polyp images small than 1600 pixels. It is recommended that clinicians could adjust the distance between the lens and polyps appropriately to enhance the system performance when conducting computer-assisted colorectal polyp analysis.


Introduction
Colorectal cancer is globally the third most prevalent cancer and mainly derived from pre-malignant colorectal polyps [1]. Colonoscopy screening and endoscopic resection of colorectal polyps can reduce the incidence and mortality of colorectal cancer [2]. Imageenhanced colonoscopy with contrast enhancement through dyes or optical strategies facilitates the detection and observation of colorectal lesions [3]. However, the accuracy of the detection and diagnosis of colorectal polyps varies depending on the experience of endoscopists, thus influencing the effectiveness of colonoscopy in reducing the incidence and mortality of colorectal cancer [3,4].
For detailed observation of the colorectal lesions, endoscopists can apply dyes and/or virtual chromoendoscopy [5]. Colorectal lesions are stained with dyes such as indigo carmine, crystal violet or methyl blue and are observed by colonoscopy with or with- Examples (a) Type 1 (b) Type 2 (c) Type 3 In 2011, the American Society of Gastrointestinal Endoscopy (ASGE) proposed a guideline for colorectal polyp diagnosis through a new optical endoscopic technique [11]. For diminutive polyps of the sigmoid colon and the rectum, the diagnose-and-leave or resect-and-discard strategy can be adopted if the correct diagnosis is made based on real- In 2011, the American Society of Gastrointestinal Endoscopy (ASGE) proposed a guideline for colorectal polyp diagnosis through a new optical endoscopic technique [11]. For diminutive polyps of the sigmoid colon and the rectum, the diagnose-and-leave or resect-and-discard strategy can be adopted if the correct diagnosis is made based on real-time endoscopic optics. The diagnose-and-leave strategy is applicable when the endoscopic optical diagnosis is highly precise (i.e., a negative predictive value of 90% and above for adenoma). Pathological biopsy is not required for such polyps. When the resect-and-discard strategy is adopted to remove diminutive colorectal polyps identified by high-confidence optical diagnosis, clinicians must combine optical diagnosis of rectosigmoid diminutive polyps with histopathological diagnosis for colorectal polyps >5 mm to determine the post-polypectomy surveillance interval. All colorectal polyps are sent for pathologic interpretation to ensure a consistency of 90%. Cost-effectiveness analyses have revealed that employing the innovative endoscopic optical technology to diagnose colorectal polyps and adopting either the diagnosis-and-leave or resect-and-discard strategy are cost-effective [12][13][14][15][16][17]. Nevertheless, the current challenge in the use of innovative endoscopic optical technology to diagnose colorectal polyps is that not all endoscopists meet the standards recommended by the ASGE. In fact, of the endoscopists in community hospitals, only 25% could achieved a colorectal polyp diagnosis accuracy of 90% by using the NBI-assisted optical diagnosis method [18], with the consistency rate of postoperative polyp resection monitoring intervals being only 85% and 81% [4,19]. Therefore, identifying the effective use of computer-aided diagnostic systems in helping clinicians decide the polyp treatment methods is imperative.
Different computer-aided diagnostic systems for colonoscopy have been proposed [20][21][22]. The color, shape, texture, and spatial features of the images are analyzed through image processing techniques (e.g., noise removal, image sharpening, and image segmentation) and according to the optical characteristics captured using virtual chromoendoscopy (i.e., color of colorectal polyps, structure of capillaries, and morphology of capillary surface). The images are further classified or identified using grouping algorithms, and an identification accuracy and sensitivity of approximately 72-89% [4] and 90% [23], respectively. Through texture analysis and wavelet transformation, Hafner and Kwitt [23,24] computed the red-green-blue (RGB) and LAB models by using a color wavelet cross-cooccurrence matrix and analyzed the polyp classifications by applying K-nearest clustering and feature vectors. Gross et al. [25] employed nonlinear filters and Canny edge detectors for tumor contouring. Tamaki [26] extracted image feature vectors by using the scale-invariant feature transform and performed identification using support vector machine classification. However, determining a standard or single feature vector to classify different types of polyp images by using the said methods is difficult.
In most computer-aided diagnostic systems, analysis is performed based on the magnification of endoscopic images. However, in clinical practice, non-magnified high-resolution colonoscopy images are most commonly used mainly because magnification endoscopes are more expensive, difficult to operate, and unsuitable for routine practice. Therefore, they are not preferred by general hospitals and clinicians and only used in medical centers for clinical research purposes. Several studies have proposed the application of deep learning to computer-aided diagnostic systems for colonoscopy, published data that conducted based on magnification endoscopic images yielded a diagnostic accuracy over 90% [21,27]. However, studies using non-magnification colonoscopic images for colorectal polyps classified by deep learning system is lacking.
Deep learning has been increasing used in the medical field [28][29][30][31][32][33]. Many studies have applied a convolutional neural network (CNN) to the identification and classification of colorectal polyps [27,34]. Chen et al. [35] employed the deep neural network to differentiate between NBI images of neoplastic and hyperplastic polyps. Park et al. [36] used the CNN to classify the polyps in colonoscopy images. Ribeiro et al. [37] increased data sets through small patches and conducted polyp classification according to the features extracted using a CNN. Shin et al. [38] proposed the false-positive and offline learning method based  [42] used VGG16 and VGG19 models to conduct analyses through 7-fold validation. Zheng et al. [43] used the you-only-look-once (YOLO) model to identify the polyp location in WL and NBI images. All these methods were adopted to locate polyps and assist clinicians for the classification of polyps. In most of these methods, training and testing were based on RGB images. However, because RGB images, expressed by three primary color layers, are multidimensional images, the computation cost involved is higher. In addition, the network frameworks were composed of a considerable number of layers, which makes the process time-consuming.
Currently, many medical images are processed into grayscale images, such as ultrasound, computed tomography (CT), and magnetic resonance (MR) images. Tan et al. [44] applied a gray-level cooccurrence matrix and CNN to CT images for polyp diagnosis. Zhang et al. [45] compress the three-channel color images of chest CT to grayscale images. A five-layer deep CNN with stochastic pooling is used to diagnose chest-based COVID-19. Xie et al. [46] mentioned that in deep learning, colors are not the key features influencing accurate image classification. They also discovered that in X-ray image classification, the speed and accuracy of processing grayscale images were considerably higher than those of processing RGB images. The grayscale method used was ITU-R Recommendation BT.601 [47]. Moreover, misjudgments were easily made in the presence of excessive intestinal wall textures, when polyps were too small ( differentiate between NBI images of neoplastic and hyperplastic polyps. Park et al. [36 used the CNN to classify the polyps in colonoscopy images. Ribeiro et al. [37] increased data sets through small patches and conducted polyp classification according to the fea tures extracted using a CNN. Shin et al. [38] proposed the false-positive and offline learn ing method based on a region-based CNN (R-CNN), successfully improving the existin polyp locating method. By using Faster R-CNN, Ren et al. [39] developed a learnin method by drawing a bounding box around a polyp. On the basis of the Faster R-CNN o the visual geometry group-16 model, Mo et al. [40] trained 16 colonoscopy images. Wan et al. [41] employed the SegNet model for examining colonoscopy images. Urban et a [42] used VGG16 and VGG19 models to conduct analyses through 7-fold validation Zheng et al. [43] used the you-only-look-once (YOLO) model to identify the polyp location in WL and NBI images. All these methods were adopted to locate polyps and assist clini cians for the classification of polyps. In most of these methods, training and testing wer based on RGB images. However, because RGB images, expressed by three primary colo layers, are multidimensional images, the computation cost involved is higher. In addition the network frameworks were composed of a considerable number of layers, which make the process time-consuming.
Currently, many medical images are processed into grayscale images, such as ultra sound, computed tomography (CT), and magnetic resonance (MR) images. Tan et al. [44 applied a gray-level cooccurrence matrix and CNN to CT images for polyp diagnosis Zhang et al. [45] compress the three-channel color images of chest CT to grayscale images A five-layer deep CNN with stochastic pooling is used to diagnose chest-based COVID 19. Xie et al. [46] mentioned that in deep learning, colors are not the key features influenc ing accurate image classification. They also discovered that in X-ray image classification the speed and accuracy of processing grayscale images were considerably higher tha those of processing RGB images. The grayscale method used was ITU-R Recommendatio BT.601 [47]. Moreover, misjudgments were easily made in the presence of excessive intes tinal wall textures, when polyps were too small (   In this study, we proposed a colorectal polyp detection and classification method b using grayscale images and a lightweight deep learning framework, which provided th same or similar detection effect as RGB images but incurred lower computing costs. T increase the applicability of the proposed method in clinical diagnosis, we developed precise, computer-aided diagnostic system compatible with mobile devices.
The rest of this study is organized as follows: Section 2 explains the materials and In this study, we proposed a colorectal polyp detection and classification method by using grayscale images and a lightweight deep learning framework, which provided the same or similar detection effect as RGB images but incurred lower computing costs. To increase the applicability of the proposed method in clinical diagnosis, we developed a precise, computer-aided diagnostic system compatible with mobile devices.
The rest of this study is organized as follows: Section 2 explains the materials and methods used in the colorectal polyp image detection and classification system; Section 3 details the experiments performed on the system, and Section 4 presents the discussion and conclusions.

Materials and Methods
In this study, we employed colonoscopy images provided by the CVC Clinic [48,49] and the Department of Gastroenterology and Hepatology, Chang Gung Medical Hospital (CGMH) ( Table 2). The CVC Clinic data set comprises 612 continuous images extracted from 29 WL images, among which 592 images had one polyp and the remaining 30 images had more than two polyps. The image resolution was 384 × 288 pixels. For image marking, the upper, lower, left, and right boundaries of the white part were set as the four sides of the bounding box according to the binarized ground truth (GT) obtained from the CVC Clinic data set. The endoscope models used were CF-260AI/AL and CF-290AI/AL (Olympus Optical Co, Ltd., Tokyo, Japan), and the endoscope host employed was EVIS LUCERA (Olympus Medical Systems) (Figures 4 and 5). In both WL and NBI images, the parts with polyps were extracted. In all, 1000 polyp image data were captured, and 2160 image data were extracted from the experimental data, including 80 images of the same polyp from different angles. First, we selected one WL and one NBI image each from the 900 WL and 900 NBI colonoscopy images, respectively (1800 colonoscopy images in total). The image resolution was 640 × 480 pixels. Each image in the data set was manually marked by clinicians, and the file format used was tagged image file format (TIFF). In addition, we selected another 360 polyp images for a polyp classification test to improve classification accuracy.      Figure 6 presents the architecture for the colorectal polyp detection and diagnosis system. The system is capable of polyp detection and classification. During colorectal polyp detection, the input colonoscopy images were preprocessed. The images in the data set were converted to grayscale images during preprocessing and applied for feature extraction and training to detect the location of polyps. The detected polyps were subsequently classified into neoplastic polyps (including adenomas and deep submucosal invasive cancer) and hyperplastic polyps.

Polyp Detection
The main polyp detection procedures involve the conversion of images to grayscale and labeling the location of polyps. During gray scaling, RGB images were converted to grayscale images with pixel values ranging from 0 to 255. RGB pixels were converted to gray pixels by using ITU-R BT.601 [47]: where R, G, and B are the pixel values of the red, green, and blue colors in color images, respectively (Figures 7 and 8).  Figure 6 presents the architecture for the colorectal polyp detection and diagnosis system. The system is capable of polyp detection and classification. During colorectal polyp detection, the input colonoscopy images were preprocessed. The images in the data set were converted to grayscale images during preprocessing and applied for feature extraction and training to detect the location of polyps. The detected polyps were subsequently classified into neoplastic polyps (including adenomas and deep submucosal invasive cancer) and hyperplastic polyps.   Figure 6 presents the architecture for the colorectal polyp detection and diagnosis system. The system is capable of polyp detection and classification. During colorectal polyp detection, the input colonoscopy images were preprocessed. The images in the data set were converted to grayscale images during preprocessing and applied for feature extraction and training to detect the location of polyps. The detected polyps were subsequently classified into neoplastic polyps (including adenomas and deep submucosal invasive cancer) and hyperplastic polyps.

Polyp Detection
The main polyp detection procedures involve the conversion of images to grayscale and labeling the location of polyps. During gray scaling, RGB images were converted to grayscale images with pixel values ranging from 0 to 255. RGB pixels were converted to gray pixels by using ITU-R BT.601 [47]: where R, G, and B are the pixel values of the red, green, and blue colors in color images, respectively (Figures 7 and 8).

Polyp Detection
The main polyp detection procedures involve the conversion of images to grayscale and labeling the location of polyps. During gray scaling, RGB images were converted to grayscale images with pixel values ranging from 0 to 255. RGB pixels were converted to gray pixels by using ITU-R BT.601 [47]: where R, G, and B are the pixel values of the red, green, and blue colors in color images, respectively (Figures 7 and 8).   Furthermore, we adopted a CNN model for feature extraction (Figure 9). The analytical process included convolution, batch normalization (BN), rectified linear unit (ReLU) function implementation, and max pooling operations. The input colonoscopy images were scaled to 128 × 128 pixels, and a convolution filter was used for feature map extraction. The mean and standard deviation of the feature map were calculated in batches to normalize each channel of the input feature map. Subsequently, two trainable parameters, normalized through scaling and translation, were added to the model to effectively transfer the extracted feature map to the next layer, thereby increasing the training speed and enhancing the network stability. Threshold values of the normalized feature map that were less than 0 were set to 0. Through the filter, the maximum value of the feature map was obtained, thus highlighting the features: where I is the input colonoscopy image; K denotes the filter; p and q are the horizontal and vertical axis coordinates of the original image, respectively; and x and y are the horizontal and vertical axis coordinates of the output feature map, respectively: where is the normalized output value; xi is the input value; μB is the mean of the batches; σB 2 is the standard deviation of the batches; ε is the minimum value and ε = 10 −5 in this study; yi is the output value of the normalized value; γ is the scaling parameter; and β is the translation parameter. The ReLU function performs threshold operations on the features in the upper layer, changing all values less than 0 to 0, which can clarify the features and facilitate network training. Threshold values of the normalized feature map that were less than 0 were set to 0: Furthermore, we adopted a CNN model for feature extraction (Figure 9). The analytical process included convolution, batch normalization (BN), rectified linear unit (ReLU) function implementation, and max pooling operations. The input colonoscopy images were scaled to 128 × 128 pixels, and a convolution filter was used for feature map extraction. The mean and standard deviation of the feature map were calculated in batches to normalize each channel of the input feature map. Subsequently, two trainable parameters, normalized through scaling and translation, were added to the model to effectively transfer the extracted feature map to the next layer, thereby increasing the training speed and enhancing the network stability. Threshold values of the normalized feature map that were less than 0 were set to 0. Through the filter, the maximum value of the feature map was obtained, thus highlighting the features: where I is the input colonoscopy image; K denotes the filter; p and q are the horizontal and vertical axis coordinates of the original image, respectively; and x and y are the horizontal and vertical axis coordinates of the output feature map, respectively: wherex i is the normalized output value; x i is the input value; µ B is the mean of the batches; σ B 2 is the standard deviation of the batches; ε is the minimum value and ε = 10 −5 in this study; y i is the output value of the normalized value; γ is the scaling parameter; and β is the translation parameter. The ReLU function performs threshold operations on the features in the upper layer, changing all values less than 0 to 0, which can clarify the features and facilitate network training. Threshold values of the normalized feature map that were less than 0 were set to 0: where BNy is the output value after BN, namely the input value of the ReLU function; The max pooling operation performs dimension reduction on the features to extract their maximum value. Through the filter, the maximum value of the feature map was obtained, thus highlighting the features: where a i is the input value; p is the pooling regions; n is the pooling size; and a j is the output value after the maximum value is obtained. Polyp detection was then conducted in the bounding box transform layer. The prediction result of the feature map was obtained in the last layer and was output through a 1 × 1 convolutional layer by using a pixel-based approach and converted to a bounding box mode for display [50]. There are eight groups of bounding boxes to locate the polyps: (16,16), (32,32), (24,48), (48,24), (60, 80), (108, 72), (216, 144), and (180, 180). The cutoff value of polyp detection is 0.2. The size of the predicted bounding box was adjusted by converting it to the range of a true-value bounding box. Table 3 displays the framework of the CNN model. Figures 10 and 11 show the results of WL and NBI polyp detection, respectively, where the first column is the RGB image; the second column is the grayscale image; the red box is the prediction box; and the green box is the actual polyp location marked by the clinician, namely the GT.
thus highlighting the features: where ai is the input value; p is the pooling regions; n is the pooling size; and aj is the output value after the maximum value is obtained. Polyp detection was then conducted in the bounding box transform layer. The prediction result of the feature map was obtained in the last layer and was output through a 1 × 1 convolutional layer by using a pixel-based approach and converted to a bounding box mode for display [50]. There are eight groups of bounding boxes to locate the polyps: (16,16), (32,32), (24,48), (48,24), (60, 80), (108, 72), (216, 144), and (180, 180). The cutoff value of polyp detection is 0.2. The size of the predicted bounding box was adjusted by converting it to the range of a true-value bounding box. Table 3 displays the framework of the CNN model. Figures 10 and 11 show the results of WL and NBI polyp detection, respectively, where the first column is the RGB image; the second column is the grayscale image; the red box is the prediction box; and the green box is the actual polyp location marked by the clinician, namely the GT.

Polyp Classification
According to the biopsy reports, we classified colorectal polyps into neoplastic polyps and hyperplastic polyps by using a CNN model ( Table 4). The cutoff value of polyp classification is 0.5.

Statistical Analysis
Categorical data were presented as rates and proportions and 95% confidence intervals (CI) were calculated. The McNemar's test was used to compare the accuracy of polyp detection between RGB and NBI images and grayscale images for deep learning system. A two-tailed p value less than 0.05 is considered statistically significant. The statistical analysis was performed by using IBM SPSS Statistics 22 (SPSS Inc., Chicago, IL, USA).

Experimental Results
We divided the experiment data into three parts: the CVC Clinic images and the WL images and NBI images provided by the Chang Gung Medical Foundation. The images were divided based on data sets and different light sources to verify whether the dissimilarity and compatibility of images differed due to different light sources. The RGB and grayscale images in each data set were separately trained. This system was implemented using software, i.e., MATLAB version 2020b (The MathWorks, Inc., Natick, MA, USA).

Polyp Detection Results
The data sets for polyp detection were divided into five groups (Table 5), and the data were input to the colorectal polyp detection system through a 5-fold cross validation for training and testing. During testing, calculations were performed based on the marked-up parts and the prediction box and according to the standard used in the 2015 endoscopic vision challenge held by the Medical Image Computing and Computer-Assisted Intervention Society (MICCAI-2015) [30,51]. Tables 6-14 presented the detection accuracy of our models for the tested data of the CGMH-WL, CGMH-NBI, and CGMH-WL+ CGMH-NBI images. The accuracy value of polyp detection is the number of correct detected polyp images divided by the total number of polyp images. The results indicated that the detection accuracy of grayscale images reached almost 95%, which was higher than of the RGB images when using WL image dataset (Tables 6 and 7). However, there is no difference in the accuracy of polyp detection between NBI images and grayscale transformation images when using NBI image dataset (Tables 8 and 9). Among RGB images, detection errors were noted in situations where polyp protrusions were nonobvious or had a texture similar to that of the intestinal wall, images had excessive noise, or the polyps were too small. Particularly, polyps with a size less than 1600 pixels (40 × 40 pixels) in the image were likely to be misjudged. Moreover, we conducted a deep learning for detection of polyp size divided into greater and smaller/equal than 0.5 cm (Table 12). The identification result of the gray scaled CNN exhibited no significant difference. The accuracy of the images with a polyp size both greater and smaller than 0.5 cm ranged between 94.4% and 96.5% (Tables 13 and 14). That is, image size was the key factor affecting the detection accuracy. We used 900 normal colonoscopy images as the testing data to verify the performance of polyp detection. The accuracy of normal colonoscopy images detection was 870/900 = 96.56%. We found that some RGB polyp images that were not fully focused were likely to be undetected if the images were not transformed to grayscale. It is a common phenomenon in clinicians during colonoscopy screening and usually happened in the dynamic colonoscopy images. In addition, some unfocused NBI images may also be the possible cause of misjudgment for polyp detection.

Polyp Classification Results
We classified 2072 WL and NBI colorectal polyp images provided by the Chang Gung Medical Hospital according to the results of pathological interpretation. The dataset was randomly split into training, validation, and testing group. None of the image shared between groups. The number of polyp images used for classification training, verification, and testing was 1370, 342, and 360, respectively (Table 15). The verification and testing data comprised the images that were accurately detected, and polyp images different from the preceding ones were used as the verification data. The 5-fold cross validation was also used for training, testing, and validation of polyp classification. Table 16 presents the performance index in which true positive (TP) is the situation where the center point of the prediction box falls at any GT; false positive (FP) is the situation where the center point of the prediction box does not fall at any GT; and false negative (FN) is the situation where no prediction box appears. True negative (TN) was not included because all the images in the data set had at least one polyp, implying that they were not negative images. Tables 17 and 18 present the confusion matrices for the classification of WL and NBI grayscale images, respectively. Table 19 presents the performance evaluation of polyp classification. Figure 12 shows the ROC curve and the AUC value is 0.93. The results indicated that NBI images facilitated more accurate polyp identification than WL images. During polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious, polyps were smaller than 1800 pixels, or when the images had an excessively low resolution. However, hyperplastic polyps were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.  Table 17. Confusion matrix of WL polyp classification.

Neoplastic (Predicted) Hyperplastic (Predicted)
Neoplastic(Actual) 118 6 Hyperplastic(Actual) 25 31 the prediction box does not fall at any GT; and false negative (FN) is the situation where no prediction box appears. True negative (TN) was not included because all the images in the data set had at least one polyp, implying that they were not negative images. Tables  17 and 18 present the confusion matrices for the classification of WL and NBI grayscale images, respectively. Table 19 presents the performance evaluation of polyp classification. Figure 12 shows the ROC curve and the AUC value is 0.93. The results indicated that NBI images facilitated more accurate polyp identification than WL images. During polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious, polyps were smaller than 1800 pixels, or when the images had an excessively low resolution. However, hyperplastic polyps were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.   In Table 18, three neoplastic polyp images that were less than 1800 pixels were classified as hyperplastic polyps. The NPV for adenoma by using the proposed system is 84% (31/37), which did not reach the threshold of ≥90% for implementation of the diagnoseand-leave strategy. However, if the polyp images less than 1800 pixels were excluded from the study, the NPV is 91% (31/34), which achieved the threshold set by ASGE for diagnose-and-leave policy.

Discussion and Conclusions
For polyp detection, the mean accuracy of grayscale images was 95.1%, which was higher than that of RGB/NBI images (94.1%). Detection errors in RGB images were caused when polyp protrusions were nonobvious or had a texture similar to that of the intestinal wall ( Figure 13), images had excessive noise (Figure 14), or the polyps had an excessively small pixel size ( Figure 15). Particularly, polyps with a size less than 1600 pixels (40 × 40 pixels) in the image were likely to be misjudged. Therefore, we recommend that in polyp detection, the length and width of polyp images must be at least 40 pixels in the lens regardless of the actual size of the polyps. This is likely to reduce misjudgment. We also found that when the colorectal polyps image is more complex, such as with more wrinkles or light spots, the use of color image deep learning for polyp detection may be more accurate. In addition, when the vascular lines of colorectal polyps were less obvious, such as unfocused or residual images, it would be better to use gray-scale image deep learning for polyp detection. Next step we plan to use a hybrid deep learning model that combines color and grayscale images for polyp detection in the dynamic colonoscopy videos. Maybe it can take the advantages of both methods as well as improve the detection accuracy.
In Table 18, three neoplastic polyp images that were less than 1800 pixels were classified as hyperplastic polyps. The NPV for adenoma by using the proposed system is 84% (31/37), which did not reach the threshold of ≥90% for implementation of the diagnoseand-leave strategy. However, if the polyp images less than 1800 pixels were excluded from the study, the NPV is 91% (31/34), which achieved the threshold set by ASGE for diagnose-and-leave policy.

Discussion and Conclusions
For polyp detection, the mean accuracy of grayscale images was 95.1%, which was higher than that of RGB/NBI images (94.1%). Detection errors in RGB images were caused when polyp protrusions were nonobvious or had a texture similar to that of the intestinal wall ( Figure 13), images had excessive noise (Figure 14), or the polyps had an excessively small pixel size ( Figure 15). Particularly, polyps with a size less than 1600 pixels (40 × 40 pixels) in the image were likely to be misjudged. Therefore, we recommend that in polyp detection, the length and width of polyp images must be at least 40 pixels in the lens regardless of the actual size of the polyps. This is likely to reduce misjudgment. We also found that when the colorectal polyps image is more complex, such as with more wrinkles or light spots, the use of color image deep learning for polyp detection may be more accurate. In addition, when the vascular lines of colorectal polyps were less obvious, such as unfocused or residual images, it would be better to use gray-scale image deep learning for polyp detection. Next step we plan to use a hybrid deep learning model that combines color and grayscale images for polyp detection in the dynamic colonoscopy videos. Maybe it can take the advantages of both methods as well as improve the detection accuracy.   In polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious (Figure 16), images had an excessively low resolution, or when polyps were smaller than 1800 pixels ( Figure 17). Hyperplastic polyps, however, were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.  In polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious (Figure 16), images had an excessively low resolution, or when polyps were smaller than 1800 pixels ( Figure 17). Hyperplastic polyps, however, were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels. In polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious (Figure 16), images had an excessively low resolution, or when polyps were smaller than 1800 pixels ( Figure 17). Hyperplastic polyps, however, were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.
In polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious (Figure 16), images had an excessively low resolution, or when polyps were smaller than 1800 pixels ( Figure 17). Hyperplastic polyps, however, were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.  In this study, we employed a deep learning framework for colorectal polyp detection by using grayscale images. The marked polyp data in the public database of the CVC Clinic and the clinical data provided by the Chang Gung Medical Hospital were collected for verification and testing. Rather than only using an analytic method for single-source In polyp classification, neoplastic polyps were likely to be misjudged as hyperplastic polyps when polyp textures were not obvious (Figure 16), images had an excessively low resolution, or when polyps were smaller than 1800 pixels ( Figure 17). Hyperplastic polyps, however, were likely to be misjudged as neoplastic polyps because of the WL around polyps in the images and when polyps were smaller than 1800 pixels.  In this study, we employed a deep learning framework for colorectal polyp detection by using grayscale images. The marked polyp data in the public database of the CVC Clinic and the clinical data provided by the Chang Gung Medical Hospital were collected for verification and testing. Rather than only using an analytic method for single-source In this study, we employed a deep learning framework for colorectal polyp detection by using grayscale images. The marked polyp data in the public database of the CVC Clinic and the clinical data provided by the Chang Gung Medical Hospital were collected for verification and testing. Rather than only using an analytic method for single-source data, we collected colonoscopy images captured by different brands and models of colonoscopes. This study had the following characteristics: first, the use of grayscaled RGB images facilitated the computation of the deep learning model. Second, in the deep learning framework, we used three convolutional filters (3 × 3, 1 × 3, and 3 × 1) in the convolutional layer to extract features. Different from conventional CNN models where single-feature filters were used, the proposed approach was capable of extracting more features for computation. We also compared the result with those obtained using other models and confirmed that the proposed approach could effectively increase the polyp detection accuracy. Third, grayscale images could reduce the dimensions and size of RGB images. Image deviation and noise caused by the difference in image gradient and endoscope lens could also be avoided, which increases the accuracy of polyp detection and classification. The results of the experiment indicate that in the detection of diminutive polyps-those smaller than 0.5 cm-the accuracy considerably decreased when polyps were smaller than 1600 pixels. The differentiation between neoplastic polyps and hyperplastic polyps became difficult when polyps were smaller than 1800 pixels. Hence, we recommend that clinicians could adjust the distance between the lens and polyps to enhance the detection results when conducting computer-assisted colorectal polyp analysis, which ensures that the polyps in the images have a minimum size of 1600 pixels.
Grayscale images achieve an equivalent or even higher accuracy of polyp detection than RGB/NBI images for lightweight computation. When NBI images were larger than 1800 pixels, our model could achieve high NPV (>90%) for neoplastic polyps. The colorectal polyp detection technique developed in this study can help clinicians rapidly locate polyps, whereas the colorectal polyp classification technique can assist clinicians in determining whether polypectomy or surgery is required, which may effectively improve patient outcome, avoid unnecessary polypectomy, and reduce misdiagnosis rate of endoscopists. In addition, hardware equipment is no longer limited by computation costs. The incorporation of simple mobile devices for edge computing and telemedicine services through lightweight computation and the hand-held colorectal polyp detection technique based on 5G wireless communication technology can substantially reduce the cost of medical equipment and increase medical standards.