Fractional Order Processing of Satellite Images

: Nowadays, satellite images are used in many applications, and their automatic processing is vital. Conventional integer grey-scale edge detection algorithms are often used for this. This study shows that the use of color-based, fractional order edge detection may enhance the results obtained using conventional techniques in satellite images. It also shows that it is possible to ﬁnd a ﬁxed set of parameters, allowing automatic detection while maintaining high performance.


Objective and Contribution
Image processing has always been an essential tool in remote sensing. Processed images taken by satellites allow extracting relevant information in fields like land monitoring, routine mapping and surveillance [1].
Edge detection represents an important role in remote sensing. The conventional edge detectors are based on integer derivatives. With the introduction of fractional calculus, some new non-integer edge detectors were created and the existent integer ones adapted [2].
The aforementioned detectors were developed for grey-scale images. Since colored images are often available, a simple solution is to convert them to grey-scale, but this may compromise the performance of edge detection and discards potentially relevant information. Thus, color-based edge detection detectors were created [3,4].
Algorithms for edge detection methods often involve first or second order derivatives, which can be generalised using a fractional order derivative instead [5]. The goal of this paper is to apply fractional edge detection to satellite images, verifying how appropriate these algorithms are for the job. To assess this, results obtained with fractional order algorithms applied to color images are compared with those of conventional integer order algorithms, for both color and grey-scale images. Another important point to assess performance is whether or not it is possible to use fixed parameters to treat all the images. Experiences with medical images, for which optimal parameters have very disparate values, and are hardly predictable in advance, suggest that this may be an issue [6]. So the contribution of this paper is to show that fractional derivatives achieve good results in image treatment of satellite images with parameters automatically set.

Previous Work on Fractional Image Processing
Fractional derivatives, defined below in Section 2.2, are studied in depth in [7][8][9]. Their application to image processing began in 2002 with the CRONE method [10]. Since then, a few already extant image treatment methods have been adapted to include fractional derivatives (instead of usual, integer order derivatives), such as the fractional Roberts operator [11], the fractional Sobel method [12,13], the fractional Canny method [6], or the fractional Laplacian of Gaussian method [14]. These methods will be addressed below in section 2 and details can be found in [5].
Fractional derivatives can be used in image treatment for purposes other than edge detection, such as denoising, encription, or compression [2]. One of the most frequent applications is to medical imaging [15,16], in which difficulties in parameter tuning were felt [6], as already mentioned.
A state of the art of satellite image processing falls outside the scope of this paper; reviews covering not only edge detection but also segmentation and classification can be found in [17,18].

Satellite Image Data
In this paper, forty-three random images with low nebulosity from ESA's Sentinel-2 satellite [19] were used. The images retrieved from the website were analysed and a ground truth was manually taken using GIMP. The satellite images database, as well as the detailed extended results of this study, are available in an online repository [20].
Due to space reasons, from the forty three figures, four were chosen to illustrate performance in this paper. The selected images are numbers 12, 18, 27 and 38. These pictures were selected since they present different levels of global performance due to their heterogeneity and are shown in Figure 4 together with their corresponding ground truth.  . 2021, 1, 1 2 of 20 derivatives (instead of usual, integer order derivatives), such as the fractional Roberts operator [11], the fractional Sobel method [12,13], the fractional Canny method [6], or the fractional Laplacian of Gaussian method [14]. These methods will be addressed below in section 2 and details can be found in [5]. Fractional derivatives can be used in image treatment for purposes other than edge detection, such as denoising, encription, or compression [2]. One of the most frequent applications is to medical imaging [15,16], in which difficulties in parameter tuning were felt [6], as already mentioned.
A state of the art of satellite image processing falls outside the scope of this paper; reviews covering not only edge detection but also segmentation and classification can be found in [17,18].

Satellite Image Data
In this paper, forty-three random images with low nebulosity from ESA's Sentinel-2 satellite [19] were used. The images retrieved from the website were analysed and a ground truth was manually taken using GIMP. The satellite images database, as well as the detailed extended results of this study, are available in an online repository [20].
Due to space reasons, from the forty three figures, four were chosen to illustrate performance in this paper. The selected images are numbers 12, 18, 27 and 38. These pictures were selected since they present different levels of global performance due to their heterogeneity and are shown in Figure 4 together with their corresponding ground truth.

Paper Organisation
This paper is organised as follows. Section 2 explains the theoretical formulations for the edge detectors tested. Section 3 presents performance metrics and results. Conclusions are drawn in Section 4.

Definitions and Formulations
This section presents and compares the image treatment algorithms.

Main Algorithm Description
A main script was developed where the different edge detectors were introduced: • The image to be processed is read. • A pre-processing step is performed where the figure is scanned for blue color pixels. A mask is formed attributing value zero to all blue pixels found. This mask is later used in post-processing ( Figure 5), both for the case of grey-scale detection operators, and the case of color detection operators.

Paper Organisation
This paper is organised as follows. Section 2 explains the theoretical formulations for the edge detectors tested. Section 3 presents performance metrics and results. Conclusions are drawn in Section 4.

Definitions and Formulations
This section presents and compares the image treatment algorithms.

Main Algorithm Description
A main script was developed where the different edge detectors were introduced: • The image to be processed is read. • A pre-processing step is performed where the figure is scanned for blue color pixels. A mask is formed attributing value zero to all blue pixels found. This mask is later used in post-processing ( Figure 5), both for the case of grey-scale detection operators, and the case of color detection operators.

Paper Organisation
This paper is organised as follows. Section 2 explains the theoretical formulations for the edge detectors tested. Section 3 presents performance metrics and results. Conclusions are drawn in Section 4.

Definitions and Formulations
This section presents and compares the image treatment algorithms.

Main Algorithm Description
A main script was developed where the different edge detectors were introduced: • The image to be processed is read. • A pre-processing step is performed where the figure is scanned for blue color pixels. A mask is formed attributing value zero to all blue pixels found. This mask is later used in post-processing ( • One of the edge detection operators described in the next subsection is applied to the image. If the operator is to be applied to a grey-scale image, it must be converted first; Figure 6 shows the case when the image is converted to grey-scale for comparison purposes. • After contour detection it is necessary to close the contours, so that in the end segmentation of land and water is evident. For that, several closing morphological operations are performed. • Firstly, using lines in 0 • , 45 • , 90 • and 135 • directions a closing operation is carried out in order to close pixel gaps between nearby edges that the detector did not encounter. • Next, in order to fill the parts of land that were not yet detected by the operator and the closing operations, a MATLAB built-in function is used (imfill) . • The blue mask obtained in the pre-processing is then applied to the result of the previous operations. As one can see in Figure 6, after the edge detection there is still a lot of noise in the water. The blue mask serves as a filter that eliminates this noise and guarantees better performance in segmentation. • One last closing operation is performed. This is due to the fact that sometimes the color filter identifies small areas of land as blue. Using this closing operation, these small areas are set to white and are correctly identified. This step finalizes the processing part of the algorithm. An example of a processed image is shown in Figure 7.
• One of the edge detection operators described in the next subsection is applied to the image. If the operator is to be applied to a grey-scale image, it must be converted first; Figure 6 shows the case when the image is converted to grey-scale for comparison purposes. • One of the edge detection operators described in the next subsection is applied to the image. If the operator is to be applied to a grey-scale image, it must be converted first; Figure 6 shows the case when the image is converted to grey-scale for comparison purposes. • After contour detection it is necessary to close the contours, so that in the end segmentation of land and water is evident. For that, several closing morphological operations are performed. • Firstly, using lines in 0 • , 45 • , 90 • and 135 • directions a closing operation is carried out in order to close pixel gaps between nearby edges that the detector did not encounter. • Next, in order to fill the parts of land that were not yet detected by the operator and the closing operations, a MATLAB built-in function is used (imfill) . • The blue mask obtained in the pre-processing is then applied to the result of the previous operations. As one can see in Figure 6, after the edge detection there is still a lot of noise in the water. The blue mask serves as a filter that eliminates this noise and guarantees better performance in segmentation. • One last closing operation is performed. This is due to the fact that sometimes the color filter identifies small areas of land as blue. Using this closing operation, these small areas are set to white and are correctly identified. This step finalizes the processing part of the algorithm. An example of a processed image is shown in Figure 7. • After contour detection it is necessary to close the contours, so that in the end segmentation of land and water is evident. For that, several closing morphological operations are performed. • Firstly, using lines in 0 • , 45 • , 90 • and 135 • directions a closing operation is carried out in order to close pixel gaps between nearby edges that the detector did not encounter. • Next, in order to fill the parts of land that were not yet detected by the operator and the closing operations, a MATLAB built-in function is used (imfill). • The blue mask obtained in the pre-processing is then applied to the result of the previous operations. As one can see in Figure 6, after the edge detection there is still a lot of noise in the water. The blue mask serves as a filter that eliminates this noise and guarantees better performance in segmentation. • One last closing operation is performed. This is due to the fact that sometimes the color filter identifies small areas of land as blue. Using this closing operation, these small areas are set to white and are correctly identified. This step finalizes the processing part of the algorithm. An example of a processed image is shown in Figure 7. • After post-processing, performance analysis is done. For this, the ground truth jpg image is read and converted to a logical array. Then, by comparison between the resultant processed image and the ground truth, four metrics of performance are computed. The results of this analysis are stored in an array and are ready for analysis.

Edge Detection Operators
All the edge detection operators used in the algorithm above make use of the Grünwald-Letnikoff definition of fractional derivative: where combinations of a things, b at a time, are given by [21] The operators employed were the following. They are further detailed in [5].
• Canny Edge Detector. In this popular edge detection algorithm [22], the image is first convolved with a Gaussian Filter, and then a derivative operator is applied to the smoothed image to compute gradients. This is done determining the first partial derivatives of the smoothed image in both the x and y directions for each color channel, defining the following Jacobian [3]: • After post-processing, performance analysis is done. For this, the ground truth jpg image is read and converted to a logical array. Then, by comparison between the resultant processed image and the ground truth, four metrics of performance are computed. The results of this analysis are stored in an array and are ready for analysis.

Edge Detection Operators
All the edge detection operators used in the algorithm above make use of the Grünwald-Letnikoff definition of fractional derivative: where combinations of a things, b at a time, are given by [21] The operators employed were the following. They are further detailed in [5].
• Canny Edge Detector. In this popular edge detection algorithm [22], the image is first convolved with a Gaussian Filter, and then a derivative operator is applied to the smoothed image to compute gradients. This is done determining the first partial derivatives of the smoothed image in both the x and y directions for each color channel, defining the following Jacobian [3]: The direction in the image along which the largest variation in the chromatic image function occurs is represented by the eigenvector of J T J corresponding to the largest eigenvalue.
• Sobel Edge Detector. The integer Sobel operator performs a derivative operation on an image and so it highlights regions where there are sudden increases of pixel intensity which correspond to edges, and consists of the two masks [23] The fractional Sobel operator given in [13] has the mask A novel color-based fractional Sobel was implemented by applying the same algorithm of the Canny deterctor, with the mask in (7). • Roberts Edge Detector. The integer Roberts operator [24] consists of a pair of 2 × 2 masks [23]: The fractional Roberts detector presented in [11] and based on the truncated coefficients of the GL definition applies derivatives of arbitrary order α: This mask is applied after the convolution of an image with the conventional integer Sobel. A color-based version of this detector was developed using the same approach as the Sobel algorithm. Since the fractional operator requires two convolutions, the color-based computations were applied only to the first one with the integer mask. After this, the result is convolved with (9). • Laplacian of Gaussian (LoG) Detector. Three possible Laplacian operators are To tackle the sensitivity of second order derivatives to noise, in the LoG the image is Gaussian smoothed before applying the Laplacian filter reducing high frequency noise. The smoothing filter can also be convolved first with the Laplacian kernel and only then is the result convolved with the input image. In the fractional case [14], For color images [25], a pixel is considered as part of an edge if zero-crossings are found in any of the color channels. Using this definition, an algorithm that detects edges in color images was developed. The algorithm convolves each channel of the input color image with the LoG mask in (11). Then, a search for zero-crossings in the results for each channel is performed. If any zero-crossings are found, the corresponding pixels are flagged as part of an edge. • CRONE. The CRONE operator [10] uses the coefficients of (1): In order to detect edges on images, the formulated detector can be used in two dimensions with mask and its transpose. Color is handled as for the Canny operator. • Fractional Derivative Operator. Fractional mask (9) from the Roberts operator was used in this paper by itself as a fractional edge detector. For color images, the Jacobian is reduced to a vector and the magnitude of the gradients is given by

Results
In this section, the overall relevant results of the iterations with different parameters will be presented, and illustrated with the four images shown in Section 1.3. Again, the results for all the forty-three images in the database are available in repository [20].

Performance Assessment
In order to check performance, quantification has to be made. For that, the ground truth is compared to the output of the algorithm by scanning all pixels within the image. Four instances may occur: True Positives, False Positives, True Negatives and False Negatives (Table 1). Using the number of occurrences of each instance the following metrics were computed: During the study it was concluded that the Jaccard coefficient (J) is a good overall performance metric since the results with the highest J corresponded to the ones with maximum mean performance regarding the four metrics. Thus, the Jaccard coefficient is used as overall performance indicator.

Performance Analysis of Fractional vs. Integer Edge Detection
The first step of performance analysis is to understand, in each edge detection algorithm, if its fractional order based adaptation outperforms the conventional integer version. Therefore, all the algorithms formulated in Section 2 were tested. The performance was evaluated for all 43 images in the database, converted for simplicity purposes to grey-scale, and using the metrics introduced in Section 3.1. The best performances of the different detectors for the selected images are presented in Figures 8-23 and Tables 2-5. Using the number of occurrences of each instance the following metrics were computed: During the study it was concluded that the Jaccard coefficient (J) is a good overall performance metric since the results with the highest J corresponded to the ones with maximum mean performance regarding the four metrics. Thus, the Jaccard coefficient is used as overall performance indicator.

Performance Analysis of Fractional vs. Integer Edge Detection
The first step of performance analysis is to understand, in each edge detection algorithm, if its fractional order based adaptation outperforms the conventional integer version. Therefore, all the algorithms formulated in chapter 2 were tested. The performance was evaluated for all 43 images in the database, converted for simplicity purposes to grey-scale, and using the metrics introduced in Section 3.1. The best performances of the different detectors for the selected images are presented in Figures 8-23 and Tables 2-5.    Using the number of occurrences of each instance the following metrics were computed: During the study it was concluded that the Jaccard coefficient (J) is a good overall performance metric since the results with the highest J corresponded to the ones with maximum mean performance regarding the four metrics. Thus, the Jaccard coefficient is used as overall performance indicator.

Performance Analysis of Fractional vs. Integer Edge Detection
The first step of performance analysis is to understand, in each edge detection algorithm, if its fractional order based adaptation outperforms the conventional integer version. Therefore, all the algorithms formulated in chapter 2 were tested. The performance was evaluated for all 43 images in the database, converted for simplicity purposes to grey-scale, and using the metrics introduced in Section 3.1. The best performances of the different detectors for the selected images are presented in Figures 8-23 and Tables 2-5.

Grey-Scale Edge Detectors vs. Color Based Edge Detectors
As explained before, color edge detection algorithms were implemented and tested with the whole data set due to their good results in other applications. All the six methods were adapted and applied in the main algorithm in order to perform color-based edge detection. The best results of the performance assessment for these versions of the algorithms are presented above and are compared with the corresponding best grey-scale result, using index ∆J: The results of this comparison are presented under the form of histograms (in semilogarithmic plots) for each detector in Figure 24.

Grey-Scale Edge Detectors vs. Color Based Edge Detectors
As explained before, color edge detection algorithms were implemented and tested with the whole data set due to their good results in other applications. All the six methods were adapted and applied in the main algorithm in order to perform color-based edge detection. The best results of the performance assessment for these versions of the algorithms are presented above and are compared with the corresponding best grey-scale result, using index ∆J: The results of this comparison are presented under the form of histograms (in semilogarithmic plots) for each detector in Figure 24.

Overall Performance with Varying Parameters
To analyse the overall performance of detectors, the mean Jaccard of the best results for each edge detection operator (for all images in the data set) was computed. The results are shown in Table 6 and Figure 25 .

Overall Performance with Varying Parameters
To analyse the overall performance of detectors, the mean Jaccard of the best results for each edge detection operator (for all images in the data set) was computed. The results are shown in Table 6 and Figure 25 .

Overall Performance with Fixed Parameters
One crucial objective of this work is to assess if it is possible to find and easily tune parameters, in an automatic way, without loosing performance. A search for optimal fixed parameters was conducted sorting performance results using fixed parameters for all the images in the data set. Table 7 and Figure 26 present the best fixed parameters results for each detector; they are discussed in the next section.

Overall Performance with Fixed Parameters
One crucial objective of this work is to assess if it is possible to find and easily tune parameters, in an automatic way, without loosing performance. A search for optimal fixed parameters was conducted sorting performance results using fixed parameters for all the images in the data set. Table 7 and Figure 26 present the best fixed parameters results for each detector; they are discussed in the next section.

Discussion and Conclusions
In this paper, an automatic coasts edge detection tool using five state-of-the-art edge detection versions and seven novel detector versions was presented. The conventional integer versions of the algorithms were also tested and compared to fractional solutions.
Forty-three high definition (10,980p × 10,980p) satellite images were tested using the different versions of the operators in a wide range of parameters.
The grey-scale Fractional Derivatives operator was the detector that provided the best average scores regarding all figures, with varying parameters. The mean Jaccard coefficient for this operator was 0.9623, i.e. the algorithm identified correctly in average 96.23% of the pixels in the input images.
In remote sensing, it is important not only that the algorithm can reach high efficiency but also that the parameters required are possible to implement for an automatic solution. Despite the exceptional performance achieved with varying parameters, this Jaccard coefficient is only possible using a great spread of parameters impossible to reach in automatic solutions. Thus, the search for a solution with fixed parameters was conducted. In this case, the color-based Fractional Derivatives operator was the method that provided the best average scores regarding all figures in the data set, with a Jaccard metric of 0.9436. The result obtained sets a fixed-parameter solution that allows automatic detection of coasts with a decrease in performance compared to the aforementioned varying-parameters of less than 2%.

Discussion and Conclusions
In this paper, an automatic coasts edge detection tool using five state-of-the-art edge detection versions and seven novel detector versions was presented. The conventional integer versions of the algorithms were also tested and compared to fractional solutions.
Forty-three high definition (10,980p × 10,980p) satellite images were tested using the different versions of the operators in a wide range of parameters.
The grey-scale Fractional Derivatives operator was the detector that provided the best average scores regarding all figures, with varying parameters. The mean Jaccard coefficient for this operator was 0.9623, i.e. the algorithm identified correctly in average 96.23% of the pixels in the input images.
In remote sensing, it is important not only that the algorithm can reach high efficiency but also that the parameters required are possible to implement for an automatic solution. Despite the exceptional performance achieved with varying parameters, this Jaccard coefficient is only possible using a great spread of parameters impossible to reach in automatic solutions. Thus, the search for a solution with fixed parameters was conducted. In this case, the color-based Fractional Derivatives operator was the method that provided the best average scores regarding all figures in the data set, with a Jaccard metric of 0.9436. The result obtained sets a fixed-parameter solution that allows automatic detection of coasts with a decrease in performance compared to the aforementioned varying-parameters of less than 2%.
Further analyzing overall performance Tables 6 and 7, as well as the performance data in [20], one may draw a few more conclusions. The usage of fractional derivatives in edge detection for this application matched or improved, in most cases, the performance of the conventional integer methods, which is not surprising since more information is available to be used. The color based algorithms allowed to equal or improve the performance of grey-scale methods in most cases. However, often the increase in performance is low in percentage. Nevertheless, and since we are dealing with images that are composed of more than 120 million pixels, a percentage of 1% increase corresponds to more than 1 million pixels correctly identified.
To summarize, in this study a successful automatic tool was developed that identifies and segments coasts in satellite images using both grey-scale and novel color-based fractional edge detectors. Subsequent parameters that allow the automation of this tool were also found.