State-of-the-Art Document Image Binarization Using a Decision Tree Ensemble Trained on Classic Local Binarization Algorithms and Image Statistics

Abstract

Image binarization algorithms reduce the original color space to only two values, black and white. They are an important preprocessing step in many computer vision applications. Image binarization is typically performed using a threshold value by classifying the pixels into two categories: lower and higher than the threshold. Global thresholding uses a single threshold value for the entire image, whereas local thresholding uses different values for the different pixels. Although slower and more complex than global thresholding, local thresholding can better classify pixels in noisy areas of an image by considering not only the pixel’s value, but also its surrounding neighborhood. This study introduces a local thresholding method that uses the results of several local thresholding algorithms and other image statistics to train a decision tree ensemble. Through cross-validation, we demonstrate that the model is robust and performs well on new data. We compare the results with state-of-the-art solutions and reveal significant improvements in the average F-measure for all DIBCO datasets, obtaining an F-measure of 95.8%, whereas the previous high score was 93.1%. The proposed solution significantly outperformed the previous state-of-the-art algorithms on the DIBCO 2019 dataset, obtaining an F-measure of 95.8%, whereas the previous high score was 73.8%.

1. Introduction

Image binarization is a vital preprocessing step in document digitalization and other applications of computer vision. It is the process of classifying pixels into either background or foreground, representing them with either black or white. For a thresholding-based approach, pixel values are classified as black if they are below a certain threshold and as white if they are above it. If the threshold is the same for all pixels in the image, then the method falls in the global thresholding family. However, if the threshold can take different values based on pixel position and the surrounding neighborhood, then the method falls into the local thresholding family. Global thresholding methods are faster, but they do not perform well on images with poor lighting conditions, stains, faint ink, ink bleeding through from the other side of the paper, holes, creases, or other types of noise. Local thresholding methods overcome these limitations at the cost of increased computational complexity and the risk of introducing noise into what should be uniform areas in the image.

Transforming the original image into a bitonal image improves the results of most optical character recognition (OCR) engines and layout analysis algorithms (including line and separator detection) to such an extent that they usually include a binarization algorithm in their processing pipeline. Several document image datasets containing low-quality images that highlight the caveats of the previously used thresholding methods are available, which has sparked the development of new thresholding methods to address these caveats. Most documents contain dark text on a bright background, which is why most datasets provide ground-truth images, where black represents the text and white represents the background.

The main contribution of this paper is that we propose a machine learning model, more specifically, a decision tree ensemble that leverages classical local thresholding algorithms and image statistics to significantly improve the classification performance of the leveraged algorithms, achieving a performance comparable to the state of the art.

Related Works

Otsu [1] introduced a classic global thresholding algorithm. Moghaddam and Cheriet [2] proposed a local version of Otsu’s method using the following formula:

$$\begin{array}{c}{O}_A\left(x\right)=\Theta \left(\left|O_G-O_L\left(x\right)\right|/R-1\right) \left(O_{Gi}-O_L\left(x\right)\right)+O_L\left(x\right),\end{array}$$

(1)

where $O_G,O_{Gi}$, and $O_L$ are the thresholds obtained by applying Otsu’s thresholding method to the entire image, the inverted image, and a window centered on the pixel $x$, $\Theta$ is the unit step function, and $R$ is an adjustable parameter used to compensate for the deviation between the local and global thresholds. In a later study [3], they proposed a parameterless and adaptive generalization of Otsu’s method that uses the following formula:

$$\begin{array}{c}{O}_A\left(x\right)=ϵ+\left(O_L\left(x\right)-ϵ\right)\Theta \left(\sigma \left(x\right)-k_{\sigma }{\sigma }_{EB}\left(x\right)\right),\end{array}$$

(2)

where ϵ is a small value that ensures numerical stability, $\sigma \left(x\right)$ and ${\sigma }_{EB}\left(x\right)$ are the standard deviations of the neighborhood centered on $x$ in the original image and the estimated background image, and $k_{\sigma }$ is a factor between 1 and 2 used to modify the behavior of the unit step function $\Theta$. However, applying Otsu’s algorithm to each window requires local histogram computation, making it significantly more time-consuming compared with other methods.

Bloechle et al. [4] proposed a two-stage algorithm that applies adaptive Otsu thresholding after estimating and subtracting the background. The background estimation was further refined in [5], where only global Otsu was used after subtracting the background.

Howe [6] proposed an algorithm with three steps. First, the image is binarized by minimizing a global energy function, inspired by a Markov random field model. Next, the Laplacian of the pixel intensities is used to distinguish the foreground from the background. Finally, edge discontinuities are incorporated into the smoothness term of the global energy function, and the final binarized image is generated.

Jia et al. [7] computed local thresholds for neighborhoods centered on structurally symmetric pixels and used a voting mechanism to classify pixels as either foreground or background.

Calvo-Zaragoza and Gallego [8] proposed SAE, a convolutional auto-encoder that transforms the original image into a probability map, which can then be thresholded to produce the binarized image.

He and Schomaker [9] proposed DeepOtsu, a method that uses deep learning to iteratively remove noise from input images and then applies Otsu’s method on the enhanced images. They proposed two iterative methods: recurrent refinement and stacked refinement. Recurrent refinement uses the same network in all iterations, whereas stacked refinement uses different networks in different iterations. They later proposed CT-NET [10], a chained-cascade network based on a dual task, T-shaped network that can perform both image enhancement and image binarization.

Peng et al. [11] proposed MRAM, a deep learning solution where a multi-resolution attention model generates a probability map, which is then fed into a convolutional random field to produce the binarized image.

Mondal et al. [12] proposed 2DMN, a bidimensional morphological network that uses dilation and erosion to eliminate noise in the original image. The network was trained on old handwritten documents using backpropagation.

De et al. [13] proposed DD-GAN, a dual discriminator generative adversarial network that consists of a network that processes the image globally and a network that processes the image locally. The generated image is then thresholded to produce the binarized image.

Zhao et al. [14] proposed cGANs, harnessing conditional generative adversarial networks to combine multi-scale information. The method consists of two stages: one that extracts text pixels from the original image using various scales and one that combines the multi-scale binarized images to produce the final binarized image.

Souibgui and Kessentini [15] proposed DE-GAN, an end-to-end deep learning framework that uses conditional generative adversarial networks to binarize images. The method was trained on severely degraded document images.

Suh et al. [16] proposed a two-stage deep learning approach using generative adversarial networks. The first stage consists of four color-independent networks that extract color information from the original image. The second stage consists of two independent networks that process the images generated in the first stage, producing the final binarized image.

Yang and Xu [17] proposed D2BFormer, a deep learning semantic segmentation approach that produces a two-channel feature map with a probability sum of one, one channel corresponding to the foreground and the other to the background. Then, they used a weighted combination of the binary cross-entropy loss function and the Sørensen–Dice coefficient loss function to further train the model. They used the resulting final feature map to binarize the image and argued that avoiding the conversion of pixel values into binary values within the model improves the binarization results.

Biswas et al. [18] proposed DocBinFormer, a two-level transformer network based on vision transformers. The transformer encoder operates directly on pixel patches and feeds their latent representation to the decoder, which then generates the binarized image.

Yang et al. [19] proposed GDB, a method based on gated convolutions, arguing that regular convolutions tend to extract stroke edges imprecisely. Their method was based on two sub-networks: a coarse sub-network trained to obtain more precise feature maps and a refinement sub-network cascaded to refine the output of the coarse sub-network using gated convolutions. They also presented a version of GDB with multi-scale operations (GDBMO) that combines local and global features to further improve the results.

2. Materials and Methods

In this section, we present the thresholding algorithms that we used as features in our machine-learning model, other notable approaches, the tools we used for this study and, finally, the proposed solution.

2.1. Features

As features, we used the results of ten thresholding algorithms that can be computed globally, using convolution (i.e., Gaussian blur), or per pixel in constant time, using summed area tables. Three thresholds were elementary, and seven were from the scientific literature. The elementary local thresholds are the average pixel value of the window, the average of the maximum and minimum pixel values of the window, and the Gaussian blur value, computed using the following formula:

$$T=\sum _{x=-w}^w\sum _{y=-h}^h{e}^{-{\displaystyle \frac{1}{2}}\left({\left({\displaystyle \frac{x}{{\sigma }_x}}\right)}^2+{\left({\displaystyle \frac{y}{{\sigma }_y}}\right)}^2\right)}I(x,y),$$

(3)

where $w$ is the half-width of the window, $h$ is the half-height of the window, and ${\sigma }_x=0.3\left(w-1\right)+0.8$ and ${\sigma }_y=0.3\left(h-1\right)+0.8$ are the variances of the Gaussian kernel. The formulas for the standard deviations were taken from the Gaussian kernel generator function defined in OpenCV 4.13.

Bernsen [20] proposed an algorithm based on the range of pixel values within a window. In their method, to reduce the number of misclassified pixels in the very dark or very bright windows, the difference between the maximum and minimum values within the window is compared with a contrast threshold. If the difference is lower than this threshold, then the binarization threshold is the average of the maximum and minimum possible pixel values. Otherwise, the result is the average of the maximum and minimum pixel values in the window.

White and Rohrer [21] proposed a dynamic threshold algorithm comprising a two-dimensional running average, $z\left(x\right)$, and a bias function, $h(x,z(x\left)\right)$, between $z\left(x\right)$ and the pixel being compared, $x$. Without this bias function, noise fluctuations would heavily influence the threshold selection in areas with low variance.

Niblack’s method [22] uses the local mean $\mu$ and standard deviation $\sigma$ to compute a local threshold $T=\mu +k\sigma$, where $k$ is an adjustable parameter with a negative value. The author recommends $k=-0.2$.

Khurshid et al. [23] proposed the NICK algorithm, which is a modified version of Niblack’s method that shifts the binarization threshold down to improve the results for “white- and light-page” images. The formula for the NICK algorithm is $T=\mu +k\sqrt{\left(\sum p_i^2-{\mu }^2\right)/N}$, where $k$ is an adjustable parameter, $p_i$ are the gray-level values of the pixels within the local window, and $N$ is the number of pixels within the local window. The authors recommend $k=-0.2$.

Sauvola and Pietikainen [24] modified Niblack’s method by introducing a new term $R$, which is the dynamic range of the standard deviations of every sliding window in the entire image. The formula for Sauvola’s threshold is $T=\mu \left(1+k\left(\sigma /R-1\right)\right)$, where $k$ is an adjustable parameter that gets positive values. The authors recommend a value of 0.5 for $k$.

Wolf and Jolion [25] modified Sauvola’s method, shifting the binarization condition from gray levels towards contrast by introducing M, which is the minimum value of the gray levels of the whole image. The formula for Wolf’s method is $T=\left(1-k\right)\mu +kM+k\left(\mu -M\right)\sigma /R$, where $k$ is an adjustable parameter that controls the incertitude around the mean value. The authors recommend a value of 0.5 for $k$.

Phansalkar et al. [26] modified Sauvola’s method to improve the performance in low-contrast dark regions without significantly affecting the performance in high-contrast light regions. The formula for Phansalkar’s method is $T=\mu \left(1+p{e}^{-q\mu }+k\left(\sigma /R-1\right)\right)$, where $k$, $p$, and $q$ are adjustable parameters. The authors recommend $k=0.25,p=3,q=10$.

We also used image statistics as features, including average, standard deviation, skewness, kurtosis, Sarle’s Bimodality Coefficient (BC) [27], minimum, and maximum. All these statistics were computed both on the whole image and on each window, resulting in a global and local set of image statistics for each pixel. The last feature was a global normalized histogram computed using five bins.

Computing the image statistics on each window might seem to have a high computational cost, but it can be performed in constant time ($O\left(1\right)$) using summed area tables and substituting the local minimum and maximum with Lehmer means. The following should be noted:

$$\begin{array}{c}\underset{p\to \mathrm{\infty }}{\lim }{\displaystyle \frac{\sum x^{p+1}}{\sum x^p}}=\max \left(x\right)\\ \underset{p\to -\mathrm{\infty }}{\lim }{\displaystyle \frac{\sum x^{p+1}}{\sum x^p}}=\min \left(x\right),\end{array}$$

(4)

for any distribution $x$. The powers for the Lehmer means were chosen on an image-by-image basis as the highest possible value for which overflow did not occur. This ensured sufficient precision for all 256 possible values of the local minimum and maximum, introducing no approximation errors on any of the tested images. For example, on a window where the maximum value is 231 and the result of the Lehmer mean is 230.8, it would be correctly rounded to 231 without loss.

2.2. Tools

ML.NET [28] is a framework developed and maintained by Microsoft that allows users to train and evaluate a variety of machine-learning models. It allows automatic exploration of various hyperparameter configurations for these models, using one of several search strategies, such as random search, grid search, and Cost Frugal Tuner (CFT) [29]. Several binary classification model types are available in AutoML, but LightGBM [30] always produced the best results in our experiments.

LightGBM is a gradient boosting decision tree implementation that harnesses Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB) to improve efficiency and scalability. GOSS excludes a considerable proportion of data with small gradients, thereby obtaining an accurate estimate of the information gain while reducing the data size. EFB bundles mutually exclusive features to reduce the number of features without significantly affecting the accuracy of split point determination.

Models trained and tested on the same dataset are prone to maximizing the performance of the model on that dataset at the cost of reducing the performance on new data. This problem is called overfitting and is usually addressed by separating a dataset into training and testing subsets. The model is fitted to the training set and evaluated on the test set, resulting in one set of performance metrics. According to Berrar [31], 10-fold cross-validation is one of the most widely used methods for simulating the behavior of a model when introducing new data. To perform $k$-fold cross-validation, the dataset is split into $k$ equally sized disjunct partitions, and each partition forms one testing subset, whereas the other partitions form the corresponding training subset. Each test set has its own set of performance metrics that can be averaged over all testing sets to obtain the performance metrics of the model. The results on the $k$ test partitions are then aggregated. For example, 3-fold cross validation works by training the model on partitions 1 and 2 and testing on 3, then training on 1 and 3 and testing on 2, and, finally, training on 2 and 3 and testing on 1, producing a total of 3 trained models, each of them with metrics computed on their respective test partitions, which can be aggregated by computing their mean value, their standard deviation, etc.

2.3. Proposed Solution

We trained a binary classification model using the 10 local thresholds presented in Section 2.1, local and global statistics (average, standard deviation, skewness, kurtosis, minimum and maximum values), and a global histogram computed with five bins. Four scenarios were explored, using the following feature sets:

• Only local thresholds;
• Only local and global image statistics;
• All features except the global histogram;
• All features.

No matter the scenario, the feature set is computed per pixel and contains pixel-specific values for local thresholds and statistics, and image-specific values for global statistics and histogram.

The DIBCO datasets [32,33,34,35,36,37,38,39,40,41] were used because they provide a wide variety of hard-to-binarize document images, and many other binarization algorithms in the scientific literature use at least one of them, allowing us to compare our results with those of other state-of-the-art methods. The dataset was split into ten partitions, one for each DIBCO dataset, and the models were trained using 10-fold cross-validation. This means that, for each DIBCO dataset, the model was trained on all other DIBCO datasets and validated on the respective DIBCO dataset.

The F-measure ($FM$), also known as the $F_1$ score, is used as a metric in many studies on image binarization and is defined as follows:

$$\begin{array}{c}FM=2TP/(2TP+F),\end{array}$$

(5)

where $TP$ is the number of correctly classified foreground pixels and $F$ is the number of incorrectly classified pixels. $FM$ takes values between 0 and 1, with 1 indicating a perfect classifier.

The ML.NET binary classification experiment was executed with a memory limit of 28 GB and a time limit of 48 h on a system with an Intel 14600K processor and 32 GB of RAM. $FM$ was used as the optimizing metric, and CFT was used as the search strategy. The 10 $FM$ values obtained from the 10-fold cross-validation were aggregated into an average value, ${\mu }_{FM}$, and a standard deviation, ${\sigma }_{FM}$. The best model was considered to be the model with the highest value for ${\mu }_{FM}-{\sigma }_{FM}$, thus encouraging the robustness of the selected model. For example, a model that achieved 95% $FM$ on 9 partitions and 80% on the 10th would be considered worse than a model that achieved 90% on all 10 partitions, because a robust model would achieve consistent results on any dataset.

3. Results

After training all the model types available in ML.NET, the best models found after each training session were instances of the LightGBM model. The experiment was then limited to training only LightGBM instances.

Table 1. Scenario comparison.

$\mathit{F}\mathit{M}$ (%)	Scenario 1	Scenario 2	Scenario 3	Scenario 4
${\overline{FM}}_D$	91.775	94.3657	95.846	95.675
${\overline{FM}}_I$	91.774	94.3652	95.845	95.672
${\overline{FM}}_O$	95.357	96.92	99.41	99.21

compares the best results obtained in scenarios 1–4. ${\overline{FM}}_D$ represents the mean value for all the datasets, ${\overline{FM}}_I$ represents the mean $FM$ value for all the images, and ${\overline{FM}}_O$ represents the mean value for all the images after retraining the model on all datasets. ${\overline{FM}}_D$ and ${\overline{FM}}_I$ differ because the datasets did not contain the same number of images. ${\overline{FM}}_I$ and ${\overline{FM}}_O$ differ because ${\overline{FM}}_I$ was the average obtained using the results of the cross-validation models for their respective test datasets, whereas ${\overline{FM}}_O$ was obtained using a single model trained and tested on all the datasets.

The importance of each feature used in a decision tree ensemble can be determined by permuting the values of that feature in the dataset and measuring the impact on the evaluation metric [42].

Table 2. Feature importance.

Feature	$\mathit{F}\mathit{M}$ Difference	Scenarios Using the Feature
Pixel Value	−0.76909	1, 2, 3, 4
White	−0.02847	1, 3, 4
Bernsen	−0.04733	1, 3, 4
Niblack	−0.07923	1, 3, 4
Sauvola	−0.0711	1, 3, 4
Wolf	−0.05156	1, 3, 4
Phansalkar	−0.07106	1, 3, 4
Nick	−0.04388	1, 3, 4
Gaussian Blur	−0.38731	1, 3, 4
Average of Local Minimum and Maximum	−0.05427	1, 3, 4
Local Average	−0.01125	1, 2, 3, 4
Local Standard Deviation	−0.14666	2, 3, 4
Local Skewness	−0.08837	2, 3, 4
Local Kurtosis	−0.07781	2, 3, 4
Local BC	−0.14658	2, 3, 4
Local Minimum	−0.15828	2, 3, 4
Local Maximum	−0.06222	2, 3, 4
Global Average	−0.02384	2, 3, 4
Global Standard Deviation	−0.02445	2, 3, 4
Global Skewness	−0.04725	2, 3, 4
Global Kurtosis	−0.00972	2, 3, 4
Global BC	−0.01169	2, 3, 4
Global Minimum	−0.00799	2, 3, 4
Global Maximum	−0.00867	2, 3, 4
First Bin of Global Histogram	−0.05454	4
Second Bin of Global Histogram	−0.01896	4
Third Bin of Global Histogram	−0.01805	4
Fourth Bin of Global Histogram	−0.01445	4
Fifth Bin of Global Histogram	−0.00761	4

shows the impact on $FM$ of permuting each feature used in the four scenarios. Higher absolute values correspond to more important features and values closer to 0 correspond to less important features. It may be noted that the features exclusive to scenario 4 have significantly lower importance compared with the others, which explains the very small difference between the $FM$ values obtained in scenarios 3 and 4.

Retraining the model on all the datasets significantly improved the average $FM$, but this might have been caused either by overfitting or by increasing the size of the training set. Therefore, it makes more sense to compare our cross-validation results with state-of-the-art solutions because they also excluded the test set from the training process.

Table 3. State-Of-The-Art Comparison.

Method	${\overline{\mathit{FM}}}_{9\mathit{D}}$ (%)	${\overline{\mathit{FM}}}_{9\mathit{I}}$ (%)	${\overline{\mathit{FM}}}_{19}$ (%)	${\overline{\mathit{FM}}}_{\mathit{D}}$ (%)	${\overline{\mathit{FM}}}_{\mathit{I}}$ (%)
Otsu	78.74	78.75	63.87	77.25	76.56
Sauvola	79.72	80.05	63.82	78.13	77.66
DIBCO winners	91.33	91.30	72.88	89.49	88.59
Howe	90.93	90.97	48.20	86.66	84.68
Jia	89.94	89.93	55.87	86.53	84.92
DeepOtsu	91.68	91.57	60.75	88.59	87.04
SAE	92.98	93.04	58.78	89.56	88.01
DD-GAN	92.71	92.82	58.01	89.24	87.70
MRAM	91.93	91.90	49.53	87.69	85.67
2DMIN	84.46	87.49	42.76	80.29	78.61
cGANs	93.19	93.18	62.29	90.10	88.64
CT-Net	94.03	94.05	-	-	-
DE-GAN	93.33	94.13	56.03	89.60	88.52
Suh	93.33	93.25	70.63	91.06	89.92
GDB	93.84	93.82	73.84	91.84	90.89
GDBMO	94.10	94.20	73.50	92.04	91.16
D2BFormer	96.02	96.05	66.69	93.08	91.73
DocBinFormer	95.76	95.87	60.31	92.22	90.64
Scenario 1	91.776	91.777	91.763	91.775	91.775
Scenario 2	94.367	94.368	94.351	94.366	94.365
Scenario 3	95.846	95.845	95.846	95.846	95.845
Scenario 4	95.677	95.675	95.658	95.675	95.673

compares the best average $FM$ obtained using the proposed method in all four scenarios with state-of-the-art methods based on deep learning and traditional methods. It might seem counterintuitive to compute ${\overline{FM}}_I$ and ${\overline{FM}}_D$ for the winners of the DIBCO competitions, because they are entirely different solutions, but the scores for the other solutions were also computed for different model instances obtained using different training and test sets. The results for DIBCO 2019 are highlighted because previous state-of-the-art methods performed significantly worse on that dataset, most likely owing to the increased variance of the images compared with the previous DIBCO datasets. ${\overline{FM}}_{19}$ represents the average $FM$ for the DIBCO 2019 dataset, ${\overline{FM}}_{9D}$ represents the average $FM$ for the nine previous DIBCO datasets, and ${\overline{FM}}_{9I}$ represents the average $FM$ for all the images from the previous nine DIBCO datasets.

Figure 1 compares the binarization results for image 9 from the DIBCO 2019 dataset, which is difficult to binarize because of the ink bleeding from the other side of the page. Figure 2 compares the binarization results for image 13 from the DIBCO 2019 dataset, which is difficult to binarize because of the missing parts of the papyrus on which the text is written. In both figures, green signifies false positives (background pixels classified as foreground), and magenta signifies false negatives (foreground pixels classified as background).

The proposed solution is compared with a classic global thresholding method (Otsu), a classic local thresholding method (Sauvola), and a state-of-the-art deep learning method that achieved the highest average $FM$ for the DIBCO 2019 dataset (GDB). Both figures graphically illustrate that the proposed solution performed significantly better than the classical and state-of-the-art methods for exceedingly difficult images.

Appendix A contains high-resolution versions of the proposed solution binarization for both images.

4. Discussion

The proposed solution outperformed the state-of-the-art solutions by a significant margin on average, especially for the DIBCO 2019 dataset, which proved to be exceedingly difficult for all the compared solutions.

The performance of the proposed solution on DIBCO 2019 was remarkably close to the average over all the datasets, making our solution the most robust method presented in this study.

Although D2BFormer outperformed the proposed solution on the first nine DIBCO datasets, its performance on DIBCO 2019 was so low that, on average, the proposed solution performed better in three of the four scenarios. The poor performance of previous state-of-the-art solutions on the DIBCO 2019 dataset is most likely caused by bleed-through on 6 of the 20 images (see Figure 1) and by missing paper sections on 10 of the 20 images (see Figure 2).

The proposed method uses a decision tree ensemble, which is significantly less demanding to train compared with any deep learning architecture, being able to run in low memory conditions and using only CPU. Thus, the user is able to quickly and easily incorporate any additional datasets in the training phase, making the tuning of the model to specific use cases more accessible. A deep learning architecture incorporating the same features might yield better results, but finding a suitable architecture and tuning the hyperparameters would likely take much longer. Further experimentation is required.

Refitting the model to the full dataset produced significantly better results. However, this may have been caused by overfitting. Therefore, further investigations are required. In practical applications, we would recommend using the model fitted on the whole dataset. However, if it performs poorly, one of the cross-validation models could be used, or they could all be combined using a voting system, minimizing the risk of overfitting, therefore guaranteeing similar performance on new data. The practical application we would recommend would be using the proposed model to binarize images as part of an OCR pipeline. However, the proposed method is not limited to this use case. It can easily be adapted to deploy practical applications in other fields, such as industrial counting, medical imaging, pattern recognition, etc., by training on data specific to those fields.

Future work could include using more datasets or adding more features to the model, such as higher-order moments, higher-order generalized bimodality coefficients [43], other local thresholding algorithms, and global thresholding algorithms. The proposed method is not limited to document images; other uses of binarization may be tested using domain-specific datasets.

Another viable way to improve the results would be to use nested cross-validation [44] to increase the confidence that the best model found is not prone to overfitting, allowing us to retrain the model on the entire dataset for even better results. However, this would multiply the training time by the number of outer cross-validation folds, making it very time-consuming.

A more straightforward method of improving the results would be to run the experiment on a machine with more memory and a faster processor, or to increase the time limit.