MLFAnet: A Tomato Disease Classification Method Focusing on OOD Generalization

Abstract

Tomato disease classification based on images of leaves has received wide attention recently. As one of the best tomato disease classification methods, the convolutional neural network (CNN) has an immense impact due to its impressive performance. However, better performance is verified by independent identical distribution (IID) samples of tomato disease, which breaks down dramatically on out-of-distribution (OOD) classification tasks. In this paper, we investigated the corruption shifts, which was a vital component of OOD, and proposed a tomato disease classification method to improve the performance of corruption shift generalization. We first adopted discrete cosine transform (DCT) to obtain the low-frequency components. Then, the weight of the feature map was calculated by multiple low-frequency components, in order to reduce the influence of high-frequency variation caused by corrupted perturbation. The proposed method, termed as a multiple low-frequency attention network (MLFAnet), was verified by the benchmarking of ImageNet-C. The accuracy result and generalization performance confirmed the effectiveness of MLFAnet. The satisfactory generalization performance of our proposed classification method provides a reliable tool for the diagnosis of tomato disease.

1. Introduction

Tomato is one of the most representative crops widely planted in China. Due to the characteristics of tomato planting, it is inevitable to encounter diseases in the process of tomato growth. The impact of various diseases is one of the reasons for tomato yield loss. To avoid the serious economic losses caused by tomato yield loss, early diagnosis is an efficient way to prevent the spread of tomato disease. Therefore, early and accurate diagnosis of tomato diseases is of utmost importance to tomato yield increase and national economic development.

In the old days, the main early diagnosis method of tomato disease was visual examination. However, this method relies on the professional diagnosis experts, non-specialists are highly prone to error while analyzing diseases. Owing to the visual disparity on leaves in the early stage of tomato diseases, analyzing diseases by computer vision is a reliable method for early diagnosis. Over the past decade, a significant amount of research has addressed various methods for tomato disease classification in computer vision systems. These methods measured the discrepancy of leaf images by feature extraction, then visualized the differences with reproducible results. Prior to diagnosis, approaches of computer vision focused on the manual feature extraction. Ropelewska et al. [1] extracted the texture parameters to discriminate the cultivars of tomatoes. Several classical machine learning methods, such as HoeffdingTree and BayesNet, were also proved to be effective in classification [2]. The limitation of manual feature extraction methods is the dependence of a reasonable feature-design algorithm. To overcome this challenge, deep learning methods have attracted considerable interest in research, due to their convenience of extracting features from huge data sets instead of hand-crafted features. In addition to the advantage mentioned above, it indicates that deep learning methods could also offer a better accuracy performance [3,4,5].

As one kind of deep learning method, the convolutional neural network (CNN) is inspired by the animal visual cortex, primarily employed for object recognition. Various CNN architectures are proposed and improved for specific tasks. In the field of tomato disease classification, Tm et al. [6] proposed a CNN classification method to diagnose tomato disease, it offered a better performance than classical machine learning methods. Tan et al. [7] also compared deep learning methods with classical machine learning methods in tomato disease classification, the experiments of their research indicated that deep learning networks were all better than measured machine learning algorithms. Rangarajan et al. [8] improved diagnosis precision by applying transfer learning methods on CNN. The attention mechanism has proved to be effective for CNN. The attention-based CNN architectures are proposed in order to provide high-performance discrimination of tomato disease [9]. Several CNN architectures are designed by lesser parameters [10], these light-weight CNN architectures could be applied on mobile devices for real-time tomato disease diagnosis.

Most of the previous deep learning methods for tomato disease classification focused on verifying by independent identical distribution (IID) samples, i.e., both the training samples and the testing samples are divided from the same original dataset. However, in real-world diagnosis tasks, the testing dataset tends to be infinite, and most testing samples are provided with out-of-distribution (OOD) samples. The existing study neglected the generalization of the model. The accuracy performance of these proposed state-of-the-art methods may drop steeply when testing by OOD samples, such as images with serious noises or samples from another dataset. The issues mentioned above have attracted a lot of attention from several research as well. Kc et al. [11] explored the method of VGG architecture could achieve an accuracy of 99.53% on public datasets but dipped to 33.27% in real-world detection tasks. Qiao et al. [12] observed the accuracy of classification on a database of the Mixed National Institute of Standards and Technology (MNIST) would decrease significantly from 100% to 27.83% by replacing the validation set with Street View House Number (SVHN) dataset. The challenge of OOD generalization must be overcome before deep-learning classification methods translate into practice.

An OOD sample refers to a sample with a significant distribution discrepancy with other samples in its category. In computer vision, the factors which cause the discrepancy include but are not limited to background shifts, corruption shifts, texture shifts and style shifts [13]. Background shifts mean the change of background, which may bias the prediction of a deep learning method [14]. Corruption shifts stand for the vicinal impurities mixed in samples, which are caused by different shooting equipment, photograph conditions (i.e., illumination) or image processing (i.e., segmentation) [15]. Texture shifts refer to the destruction of texture features [16]. Style shifts often reflect in multiple concept changes such as transforming the image of a real object to a cartoon or painting style [17]. The OOD Generalization ability of a deep learning classification method is measured by the IID/OOD generalization gap [18], which can be defined as

$$Gap(f,c;D_{iid},D_{ood})=\frac{1}{\left|D_{iid}\right|}{\displaystyle \sum _{(x,y)\in D_{iid}}\mathbb{I}}\left(c\left(f\left(x\right)\right)=y\right)-\frac{1}{\left|D_{ood}\right|}{\displaystyle \sum _{(x,y)\in D_{ood}}\mathbb{I}}\left(c\left(f\left(x\right)\right)=y\right),$$

(1)

where $x$ is the input image and $y$ is the corresponding label. $c\left(\cdot \right)$ and $f\left(\cdot \right)$ represent the classifier and feature encoder. $n_{iid}$, $n_{ood}$ denote the number of IID and OOD data and $D_{iid}={\left\{(x_i,y_i)\right\}}_{i=1}^{n_{iid}}$, $D_{ood}={\left\{(x_j,y_j)\right\}}_{j=1}^{n_{ood}}$. $\mathbb{I}$ is the indicator function.

In the real-world diagnosis of tomato disease, background shifts and corruption shifts are relatively common. There have been many methods proposed for background shifts, i.e., image segmentation is an efficient method to improve the generalization performance of background shifts [19,20,21]. However, the work of corruption shifts generalization for tomato disease classification is still at a conceptual stage. Therefore, the major challenge of tomato disease OOD detection is to improve the generalization performance of corruption shifts.

Corruption shifts mainly incorporate image noise, image blur, weather change and digital transformation [22], all these issues are frequently encountered while diagnosing tomato disease by image in the real world. Recent works for corruption shift generalization focused on data augmentation, domain alignment and adversarial training methods. Data augmentation methods create a larger training set by transformed images [23] or generative adversarial networks (GAN) images [24]. The limitation of these methods is the domain of the test dataset tends to be infinite when the model is applied in the real world. It is impossible to expand the training set to contain all the unknown domains. The purpose of domain alignment is to align the distribution of the source domain and target domain [25]. This method needs to obtain the distribution of target domain samples. However, the distribution of target domain samples is actually unknown in the real world. Adversarial training methods apply perturbation to the training process in order to improve the robustness of the deep learning model. These methods could achieve better prediction performances than standard training approaches on OOD samples [26,27,28]. Nevertheless, compared to standard training approaches, these adversarial training methods perform poorly on IID samples [29].

In this article, we investigate the corruption shifts generalization of CNN to propose a method for OOD tomato disease classification. The proposed method could improve the classification performance of corruption shifts samples while retaining the high-accuracy performance of the original images. Our main contributions are summarized as follows:

• To the best of our knowledge, this is the first study to investigate the corruption shifts’ generalization of tomato disease classification.
• We investigated the impacts of attention mechanism on generalization improvement. The channel attention mechanism is improved by using multiple low-frequency components instead of the lowest frequency component. Moreover, depth convolution is applied to reduce the complexity of the mechanism.
• A CNN architecture named multiple low-frequency attention network (MLFAnet) is proposed for tomato disease classification. The experiment result demonstrates that the proposed architecture could consistently improve the performance of generalization ability.

This paper is organized as follows. After the introduction, Section 2 introduces the dataset, the principle of the method, the proposed MLFA block and the architecture of the proposed MLFAnet. Section 3 designs the experiments. Then, the results of Section 3 are discussed in Section 4. Finally, Section 5 makes a conclusion of the study.

2. Materials and Methods

2.1. Dataset

The database employed in our experiments contained 16,484 original tomato leaf images obtained from the PlantVillage dataset [30]. It was divided into 9 classes, including one healthy class and 8 kinds of disease classes. Figure 1 shows the samples of these different diseased leaf images.

The whole images of the database, with 9 classes, were divided into three parts: a training set, a validation set and an original testing set, as shown in

Table 1. Number of samples for each disease category in dataset.

Samples	Training	Validation	Testing	Total
Healthy	1193	298	100	1591
Bacterial Spot	1622	405	100	2127
Early Blight	720	180	100	1000
Late Blight	1447	362	100	1909
Leaf Mold	682	170	100	952
Mosaic Virus	218	55	100	373
Septoria	1337	334	100	1771
Target Spot	1043	261	100	1404
Yellow Leaf	4206	1051	100	5357

. We first randomly selected 100 tomato leaf images from each disease category to create the original testing set. Then, training set was set up by 80% of the rest of the images. The other images were validation set samples. Resizing method was applied on these samples to ensure the size of each image was $224\times 224\times 3$.

Then, we created two additional testing sets: the low-frequency leaf testing (LLT) set and the corruption-shift leaf testing (CLT) set. Samples of each additional set were generated by the original testing set images. The generation method of LLT set is presented in reference [31]. In CLT set, samples were produced by the benchmark of ImageNet-C [22].

The purpose of LLT set is to investigate the relationship between layer depth of the CNN and low-frequency identification bias, in order to design an efficient CNN architecture with reasonable layer depth. Discrete cosine transformation (DCT) was first applied onto samples of original testing set to achieve the frequency spectrums. Then, the frequency spectrums were divided into low-frequency components and high-frequency components by different manual thresholds. At last, the low-frequency reconstructed images and high-frequency reconstructed images were generated by equation:

$${\mathbf{X}}_{\mathrm{L}}\left[i,j\right]=\frac{1}{K}{\displaystyle \sum _{h=0}^{K-1}{\displaystyle \sum _{w=0}^{K-1}\mathbf{F}\left[0,0\right]}}+\frac{2}{K}{\displaystyle \sum _{h=0}^{K-1}{\displaystyle \sum _{w=0}^{K-1}\mathbf{F}\left[h,w\right]}}\cos \left(\frac{\pi (2i+1)h}{2K}\right)\cos \left(\frac{\pi (2j+1)w}{2K}\right),$$

(2)

$${\mathbf{X}}_{\mathrm{H}}\left[i,j\right]=\frac{2}{K}{\displaystyle \sum _{h=K}^{223}{\displaystyle \sum _{w=K}^{223}\mathbf{F}\left[h,w\right]}}\cos \left(\frac{\pi (2i+1)h}{2K}\right)\cos \left(\frac{\pi (2j+1)w}{2K}\right),$$

(3)

where ${\mathbf{X}}_{\mathrm{L}}\left[i,j\right]$, ${\mathbf{X}}_{\mathrm{H}}\left[i,j\right]$ are the elements of low-frequency and high-frequency reconstructed image and $K$ is the hyperparameter of threshold.

An example of the reconstructed images was shown in Figure 2.

To verify the generalization performance of each classification method for samples affected by corruption shifts, we created CLT set. It was composed of samples with 15 kinds of corruption shifts including Gaussian noise, shot noise, impulse noise, defocus blur, glass blur, motion blur, zoom blur, fog, frost, snow, brightness change, contrast transformation, elastic transformation, pixelation and JPEG, as shown in Figure 3.

2.2. Methods

Our proposed tomato disease classification method was based on an improved convolutional network. It was designed to take more focus on low-frequency components of the feature maps, and we named it MLFAnet. The whole architecture of the MLFAnet was shown in Figure 4a, and the detail of MLFA block was illustrated in Figure 4b.

2.2.1. Generalization Gap and Frequency Components

In a conventional CNN classification task, the overall objective to train the model could be defined as:

$$({\widehat{\theta }}_f,{\widehat{\theta }}_c)=\underset{{\theta }_f,{\theta }_c}{\arg \min }{\mathcal{L}}_y\left({\sigma }_{sf}\left(c\left(f(x);{\theta }_f\right);{\theta }_c\right),y\right),$$

(4)

where $f(\cdot ;{\theta }_f)$ denotes the feature encoder with parameters of ${\theta }_f$, $c(\cdot ;{\theta }_c)$ denotes the classifier encoder which parameters are ${\theta }_c$, ${\sigma }_{sf}$ is the softmax function, ${\mathcal{L}}_y$ denotes the loss function and it is generally based on the cross-entropy method. $({\widehat{\theta }}_f,{\widehat{\theta }}_c)$ are the optimal parameters learned by the CNN architecture.

Suppose the corruption shifts are caused by additive noise $n$, the CNN model with the parameters $({\widehat{\theta }}_f,{\widehat{\theta }}_c)$ may achieve a generalization gap which is affected by the factor $\delta$ and $\delta$ is calculated by the following equation:

$$\delta =\left|{\mathcal{L}}_y\left({\sigma }_{sf}\left(c\left(f(x+n);{\widehat{\theta }}_f\right);{\widehat{\theta }}_c\right),y\right)-{\mathcal{L}}_y\left({\sigma }_{sf}\left(c\left(f(x);{\widehat{\theta }}_f\right);{\widehat{\theta }}_c\right),y\right)\right|.$$

(5)

To obtain the minimized value $\delta$, Ganin et al. [32] demonstrate $({\widehat{\theta }}_f,{\widehat{\theta }}_c)$ could be optimized by an extra loss function ${\mathcal{L}}_d$, then the total loss could be improved by the following equation:

$$\mathcal{L}={\mathcal{L}}_y+\lambda {\mathcal{L}}_d,$$

(6)

where ${\mathcal{L}}_d$ is domain confusion loss, $\lambda$ is a hyper-parameter which is determined by $\delta$. However, these references suppose the distribution of $n$ is known. Due to the randomness of $n$ in a generalization problem, it is difficult to optimize $({\widehat{\theta }}_f,{\widehat{\theta }}_c)$ in spatial domain with the domain loss function ${\mathcal{L}}_d$, thus we transform the image and noise to the frequency domain by DCT [33], which can be defined as

$$\begin{array}{l}\mathbf{F}\left[h,w\right]={\alpha }_h{\alpha }_w{\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}\mathbf{X}\left[i,j\right]}}\cos \left(\frac{\pi (2i+1)h}{2H}\right)\cos \left(\frac{\pi (2j+1)w}{2W}\right)\\ {\alpha }_h=\left\{\begin{array}{c}1/\sqrt{H}, h=0\\ \sqrt{2/H}, 1\le h<H \end{array}\right.,{\alpha }_w=\left\{\begin{array}{c}1/\sqrt{W}, w=0\\ \sqrt{2/W}, 1\le w<H \end{array}\right.\end{array},$$

(7)

where $\mathbf{X}\in {ℝ}^{H\times W}$ is a two-dimensional input signal and $\mathbf{X}\left[i,j\right]$ denotes the element at row $i$ and column $j$ in $\mathbf{X}$, $H$ is the height of input signal, the width of it is $W$ and $\mathbf{F}\left[h,w\right]$ is the frequency component corresponding to $\mathbf{X}\left[i,j\right]$.

Another frequency-transform method is discrete Fourier transform (DFT). We applied DCT instead of DFT by the reason of DCT’s higher degree of spectral compaction. It was indicated in Figure 5. Lower frequency components were concentrated in four directions of DFT frequency spectrum, as shown in Figure 5b. Unlike DFT, lower frequency components are gathered in top left corner, as shown in Figure 5c.

The corruption shifts image are generally caused by pixel change. The change could be seen as a kind of noise image, which has the equal dimensions with the input image. We use ${\mathcal{X}}_f\in {ℝ}^{H\times W\times C}$ and ${\mathcal{X}}_n\in {ℝ}^{H\times W\times C}$ to represent the input image and image noise, in which $C$ denotes the channel number. Let ${\mathbf{X}}_f^{N-1}={\mathcal{X}}_f[:,:,N-1]$, ${\mathbf{X}}_n^{N-1}={\mathcal{X}}_n[:,:,N-1]$. Then, the frequency components of $N$th channel in ${\mathcal{X}}_f$ are ${\mathbf{F}}_f^{N-1}$, ${\mathbf{F}}_n^{N-1}$. As most of the image noise focus on the high-frequency components [34], there may exist the thresholding factors $\Omega$ and $\epsilon$ which has the conditions for

$$\left\{\begin{array}{c}{\mathbf{F}}_n^{N-1}\left[h,w\right]<\epsilon \mathrm{if} 0\le h<\Omega , 0\le w<\Omega \\ {\mathbf{F}}_n^{N-1}\left[h,w\right]\ge \epsilon \mathrm{otherwise} \Omega \le h<H , \Omega \le w<W\end{array}\right.,$$

(8)

where $\epsilon$ denotes the thresholding factor of generalization gap caused, which means the CNN model will occur the generalization gap problem if it focuses on the over-threshold frequency components. Thus, the method to improve the generalization gap can be regarded as making the model pay more attention to the frequency components under threshold $\Omega$.

2.2.2. Low-Frequency Attention for Domain Generalization

In the $t$th convolutional layer of a CNN architecture, the feature map was calculated by the convolution kernels and features in $(t-1)$th layer. To take a frequency point of view, it was created by filtering frequency components of features in $(t-1)$th layer, and the convolution kernels were various of filters to fulfil high-pass, low-pass, brand-pass and band-stop.

To improve the performance of IID/OOD generalization gap, we increased the weight of feature maps which are created by low-pass filters, and the weight was learned by improved channel attention mechanism. The conventional channel attention algorithm could be defined as

$${\mathsf{\omega }}_{se}={\sigma }_{sg}\left({\mathbf{W}}_2\left({\mathbf{W}}_1\left(\frac{1}{HW}{\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathcal{X}}_f\left[i,j,:\right]}}\right)\right)\right),$$

(9)

where ${\mathbf{W}}_1\in {ℝ}^{C\times C/r}$ and ${\mathbf{W}}_2\in {ℝ}^{C/r\times C}$ are learned by the multi-layer perceptron (MLP) and dimensionality is reduced by $r$ for low complexity of the model. ${\sigma }_{sg}$ is the sigmoid function and ${\mathsf{\omega }}_{se}\in {ℝ}^C$ is the weight vector of channel-wise to evaluate the importance of each channel.

Through Equation (7), we have

$${\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathbf{X}}_f^{N-1}}}\left[i,j\right]={\mathbf{F}}_f^{N-1}\left[0,0\right]\times \sqrt{HW},$$

(10)

$$\begin{array}{ll}{\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathcal{X}}_f}}\left[i,j,:\right]& =\left[{\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathbf{X}}_f^0}}\left[i,j\right], {\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathbf{X}}_f^1}}\left[i,j\right], \cdots , {\displaystyle \sum _{i=0}^{H-1}{\displaystyle \sum _{j=0}^{W-1}{\mathbf{X}}_f^C}}\left[i,j\right]\right]\\ & =\sqrt{HW}\left[{\mathbf{F}}_f^0\left[0,0\right], {\mathbf{F}}_f^1\left[0,0\right], \cdot \cdot \cdot , {\mathbf{F}}_f^C\left[0,0\right]\right]\\ & =\sqrt{HW}\mathcal{F}\left[0,0,:\right],\end{array}$$

(11)

where $\mathcal{F}\in {ℝ}^{H\times W\times C}$ is the frequency component of a feature tensor. Thus, from Equations (9) and (11), the weight vector of channel-wise ${\mathsf{\omega }}_{se}$ can be defined as

$${\mathsf{\omega }}_{se}={\sigma }_{sg}\left({\mathbf{W}}_2\left({\mathbf{W}}_1\left(\frac{1}{\sqrt{HW}}{\mathcal{F}}_f\left[0,0,:\right]\right)\right)\right).$$

(12)

Equation (12) shows that channel attention algorithm is a method to pay attention to the lowest frequency component of feature maps. It was proved experimentally in Section 3.2 that the method improved the IID/OOD generalization gap as well.

However, from Equation (8), we know that the lowest frequency component of a feature map ${\mathbf{F}}^{N-1}(0,0), N\in (0, 1, \cdot \cdot \cdot , C)$ is but not limited to the determining factors because all the frequency components under threshold $\Omega$ may cause the generalization gap and should not be discarded. Thus, we calculated the weight by multiple low-frequency components instead of the lowest one. The low-frequency components tensor $\mathcal{H}$ can be defined as

$$\begin{array}{l}\mathcal{H}=\left[\begin{array}{cccc}{\mathcal{F}}_f\left[0,0,:\right]& {\mathcal{F}}_f\left[0,1,:\right]& \cdots & {\mathcal{F}}_f\left[0,P-1,:\right]\\ {\mathcal{F}}_f\left[1,0,:\right]& {\mathcal{F}}_f\left[1,1,:\right]& \cdots & {\mathcal{F}}_f\left[1,P-1,:\right]\\ ⋮& ⋮& \ddots & ⋮\\ {\mathcal{F}}_f\left[P-1,0,:\right]& {\mathcal{F}}_f\left[P-1,1,:\right]& \cdots & {\mathcal{F}}_f\left[P-1,P-1,:\right]\end{array}\right],\\ \mathrm{s}.\mathrm{t}. 0\le P-1\le \min \left\{H,W\right\}\end{array}$$

(13)

where $\mathcal{H}\in {ℝ}^{P\times P\times C}$ and $P$ is a hyperparameter to adjust the threshold of significant low- frequency components.

2.2.3. Multiple Low-Frequency Attention Network Design

An overview of the proposed architecture MLFAnet was presented in Figure 4a. An input RGB image was first forward to the multiple low-frequency attention (MLFA) block, which was shown in Figure 4b. Several bottleneck blocks proposed by ResNet architecture [35] were applied afterwards, the output size of each bottleneck block was $\frac{H}{4}\times \frac{W}{4}\times 64$, $\frac{H}{8}\times \frac{W}{8}\times 128$, $\frac{H}{16}\times \frac{W}{16}\times 256$ and $\frac{H}{32}\times \frac{W}{32}\times 512$, respectively. The output resolutions were forward to the dense layer which mixes the resulting output channels.

MLFA block was based on the proposed attention algorithm. As mentioned previously, the block was applied after an RGB input image. Feature map ${\mathcal{X}}_i\in {ℝ}^{H\times W\times C}$ was derived with multiple filters in the CNN layer of the block, and it was split into $16\times 16$ patch, thus the feature dimension ${\mathcal{X}}_p$ after patch partitioning was ${\mathcal{X}}_p\in {ℝ}^{16\times 16\times (\frac{H}{16}\times \frac{W}{16}\times C)}$. Then the feature dimension ${\mathcal{X}}_p$ was transformed to frequency component by DCT algorithm. The low-frequency tensor ${\mathcal{X}}_{lf}\in {ℝ}^{P\times P\times (\frac{H}{16}\times \frac{W}{16}\times C)}$ was derived by a manual threshold $P$.

Depthwise convolution was applied on each channel of ${\mathcal{X}}_{lf}$. The size of convolution kernel was $P\times P\times 1$. Then, the output vector was forward to MLP to obtain the weight vector ${\mathsf{\omega }}_{mf}$. The updated feature dimension ${\mathcal{X}}_q\in {ℝ}^{16\times 16\times (\frac{H}{16}\times \frac{W}{16}\times C)}$ was calculated by ${\mathcal{X}}_p$ and ${\mathsf{\omega }}_{mf}$ through the method of weighting with ${\mathsf{\omega }}_{mf}$ on the channel wise of ${\mathcal{X}}_p$. The final output tensor ${\mathcal{X}}_o$ was generated by merging the patch in the updated feature dimension ${\mathcal{X}}_q$, and the size of ${\mathcal{X}}_o$ is ${\mathcal{X}}_o\in {ℝ}^{H\times W\times C}$ which was same as the feature map ${\mathcal{X}}_i$.

The number of bottleneck blocks in proposed MLFAnet is 1, 1, 1, 1, respectively. From accuracy performance perspective, deeper convolutional neural network was an effective method for classification and that was pointed out by [3,4,5]. To be specific, a deeper convolutional neural model could extract more abstract features from the deep layers [36], these abstract features are considered as high-frequency components and they improve the accuracy performance of the model [31]. Thus, various methods for tomato disease classification were based on a very deep-layer architecture and the efficiency is confirmed by the experiments [37]. We investigated the correlation between depth of model and impacts of corruption shifts sample generalization in tomato-disease classification, it had been found experimentally in Section 3.1 that a deeper model achieved a pool IID/OOD generalization gap though it improved the IID performance a little. Due to the reason above, the proposed MLFAnet deprecated stacking a large number of bottleneck blocks. The architecture is shown in

Table 2. Architecture of proposed MLFAnet.

Input	Operator	Output	Parameters
224, 224, 3	MLFA (k = 7, s = 2, p = 1, c = 32, P = 5)	112, 112, 32	388,864
112, 112, 32	Bottleneck (k = 3, c = 24)	56, 56, 64	7488
56, 56, 64	Bottleneck (k = 3, c = 48)	28, 28, 128	29,952
28, 28, 128	Bottleneck (k = 3, c = 96)	14, 14, 256	119,808
14, 14, 256	Bottleneck (k = 3, c = 192)	7, 7, 512	440,832
7, 7, 512	GAP	1, 1, 512	0
1, 1, 512	Dense	1, 1, 9	4617
	Softmax

, where hyperparameter $k$ denotes the kernel size of the convolution, $s$ denotes the stride, $p$ is the padding, $c$ represents the number of filters and P is the threshold of low-frequency patch.

2.3. Network Training and Evaluation

We carried out the experiments with an NVIDIA GeForce RTX 2080 GPU, an I7-9700k CPU and 16 GB of memory. The experimental environment was Python 3.7 (Guido van Rossum, Haarlem, Netherlands) and PyTorch 1.5.1 (Soumith Chintala, New York, NY, USA). The batch size, learning rate, number of training epochs and initial weights for all these networks were the same. The number of training epochs was set to 100, the batch size was 32 and learning rate was the exponential-decay learning rate.

3. Results

3.1. Relevance between Layer Depth and Low-Frequency Attention

To investigate the relationship between a number of network layers and low-frequency identification bias, we trained several different CNN architectures with similar ResNet and VGG backbones. The CNN architectures were ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-152, VGG-11, VGG-13, VGG-16 and VGG-19, respectively. Each of them has a diverse set of convolutional layers or bottleneck blocks. If the model takes more focus on low-frequency information, the accuracy performance on low-frequency reconstructed samples of the architecture will be better than others.

Table 3. Accuracy result of original samples and reconstructed samples.

Architecture	Ori.	$\mathit{K}=28$			$\mathit{K}=14$			$\mathit{K}=7$
		LFR	HFR	Gap	LFR	HFR	Gap	LFR	HFR	Gap
ResNet-18	98.7	87.2	11.1	11.5	66.8	11.1	31.9	52.9	11.0	45.8
ResNet-34	98.6	85.8	11.4	12.8	64.4	14.7	34.2	51.7	14.8	46.9
ResNet-50	99.0	86.0	11.6	13.0	63.1	5.9	35.9	50.2	9.8	48.8
ResNet-101	99.2	83.9	12.2	15.3	61.2	14.3	38.0	50.1	11.6	49.1
ResNet-152	99.4	82.4	5.2	17.0	60.4	10.1	39.0	49.8	12.6	49.6
VGG-11	96.7	82.5	12.4	14.2	59.2	11.7	37.5	48.1	11.1	48.6
VGG-13	96.4	74.6	11.1	21.7	55.6	9.1	40.7	47.4	12.9	49.0
VGG-16	97.2	77.4	19.0	19.8	58.1	17.8	39.1	47.7	18.3	49.5
VGG-19	97.2	75.6	14.0	21.6	56.7	13.3	40.5	47.3	14.7	49.9

shows the comparison results of the models on the original testing set and LLT set. Ori. is the accuracy performance of samples in the testing-1 set. LFR indicates the accuracy results of low-frequency reconstructed samples in the LLT set, and the samples are reconstructed by frequency components under threshold $K$. HFR represents the accuracy of high-frequency reconstructed samples. Gap denotes the generalization gap between Ori and LFR.

3.2. Accuracy Performance

We compared our method with the other conventional deep CNN architectures including ResNet-50 [38], DenseNet-121 [39], EffecientNet-b3 [40], ShuffleNet-v2 [41] and MobileNet-v3 [42]. Each of them was proved to be efficient in tomato disease classification. We trained the different CNN architectures on the same training set. Then, we tested and verified them by the original testing set and CLT set, respectively. Figure 6 shows the accuracy performance of each method on training and validation sets. Confusion matrices provided by different CNN architectures are shown in Figure 7.

The accuracy results of CNN architectures on the original testing set and CLT set are shown in

Table 4. Performance of CNN architecture under corruption shifts, including noises and blurs. The bold type represents the best classification performance of the architecture under a certain type of corruption.

Networks	Original	Noise				Blur
		Gauss.	Shot	Impulse	Avg.	Defocus	Glass	Motion	Zoom	Avg.
ResNet-50 [38]	99.0	67.6	56.4	36.3	53.4	59.2	82.0	83.1	73.1	74.4
DenseNet-121 [39]	99.6	84.2	75.0	68.1	75.8	84.8	91.1	94.9	89.6	90.1
Eff.Net-b3 [40]	99.9	61.4	50.8	39.9	50.7	81.3	94.8	95.8	92.1	91.0
ShuffleNet-v2 [41]	99.0	73.2	65.0	53.3	63.8	87.3	94.9	95.7	93.2	92.8
MobileNet-v3 [42]	99.4	65.0	58.8	44.3	56.0	78.2	95.2	96.6	89.9	90.0
Proposed	99.4	96.4	93.0	92.4	93.9	91.3	98.4	98.3	96.2	96.0

and

Table 5. Performance of CNN architecture under corruption shifts, including weathers and digital transforms. The bold type represents the best classification performance of the architecture under a certain type of corruption.

Networks	Weather					Digital
	Bright.	Fog	Frost	Snow	Avg.	Contrast	Elastic	JPEG	Pixel	Avg.
ResNet-50 [38]	95.6	30.1	49.3	64.7	59.9	34.6	94.0	91.8	96.4	79.2
DenseNet-121 [39]	88.9	17.8	39.7	66.7	53.3	27.2	96.6	94.4	96.8	78.6
Eff.Net-b3 [40]	94.8	32.8	60.2	85.3	68.3	39.8	97.4	89.4	94.7	80.3
ShuffleNet-v2 [41]	88.9	19.9	44.4	73.4	56.7	39	95.2	96.8	97.5	82.1
MobileNet-v3 [42]	95.3	32.7	59.3	79.2	66.6	39.2	96.9	94.0	97.1	81.8
Proposed	92.9	33.1	56.9	74.7	60.7	39.8	98.6	98.8	99.0	84.0

. Original represents the accuracy results on the original testing set. Gauss., Shot, Impluse, Defocus, Glass, Motion, Zoom, Bright, Fog, Frost, Snow, Contrast, Elastic, JPEG and Pixel represent the accuracy performance of different corruption factors on the CLT set.

3.3. Experiments for Optimized Manual Thresholds

To investigate the optimized manual threshold of the MLFA block, we reconstructed the low-frequency sample as Equation (2). The size of the original image was $224\times 224$, thus when the value of $K$ equaled 112, the sample was reconstructed by a quarter of the feature spectrum. The CNN architecture was modified by MLFAnet, which applied a conventional bottleneck block instead of an MLFA block. The accuracy results of low-frequency reconstructed samples with different thresholds were compared in

Table 6. Accuracy result of low-frequency reconstructed samples with different thresholds.

K = 98	K = 84	K = 70	K = 56	K = 42	K = 28
95.9	95.9	95.6	94.3	92.1	85.9

.

As shown in Table 6, the accuracy decreases as the manual threshold hyperparameter reduces, and it breaks down dramatically while $K$ is under 70. By observing Figure 5c, we conjectured the reason was that the low-frequency reconstructed sample traded off many low-frequency components which limited the model’s prediction precision. Thus, the hyperparameter $P$ of the MLFA block in Table 2 was set to 5, due to the $16\times 16$ patch size.

4. Discussion

The experiment results in Section 3.1 indicated that for tomato disease classification, both architectures with small-number layers (ResNet-18, VGG-11) and architectures with large-number layers (ResNet-152, VGG-19) could achieve high degrees of accuracy on original samples. As the high-frequency components decreased, the prediction accuracy of low-frequency reconstructed samples reduced more dramatically for the model with large-number layers. It indicated that a deeper model would be biased to neglect the low-frequency components. Low-frequency components are significant to the model’s generalization performance, this discovery could be applied to design a more efficient CNN architecture with reasonable layer depth.

In Section 3.2, the accuracy performances indicated that EffecientNet-b3 architecture, which achieved the highest accuracy rate (99.9%) in the original testing set, performed best on the IID tomato disease classification task. The other architectures, with accuracies of more than 99%, achieved excellent classification performance on IID testing samples as well. However, on corruption shifts samples of CLT set, the accuracy performances break down dramatically. It can be calculated in Table 4 and Table 5 that the average accuracy result of EffecientNet-b3 was 74.0%. The generalization gap was 25.9% (the difference value of 99% and 74.0%). The proposed MLFAnet achieved an average accuracy of 84.0% on corruption shifts samples, and the better generalization gap performance was 15.4%. To easily compare the generalization ability between different architectures, visual representations were shown in Figure 8.

Based on Figure 8, we could have the following observations. First, the average accuracy performance of our proposed method on corrupted samples classification was better than other conventional CNN architecture, as shown by the blue histogram. Second, the method had a better generalization performance because it achieved the smallest generalization gap, while the accuracy on original IID samples was almost identical to others, as shown by the orange histogram. It illustrates that the prediction model we proposed for tomato disease, which has a stronger generalization ability on corruption shifts samples, is a better fit than previous studies in practical application.

We further explored why our model achieved a strong generalization ability. Figure 9 showed the feature maps of the original sample and corruption shifts samples. The kinds of corruption were Gaussian noise and motion blur. The features were the first convolution layer outputs of conventional ResNet-50 architecture and the proposed MLFAnet architecture. It observed that feature maps extracted by ResNet-50 changed sharply as the samples were corrupted by Gaussian noise and motion blur. On the contrary, the diversity among the feature maps was less under the proposed MLFAnet architecture. The example demonstrated that the feature extractor of MLFAnet was more robust on corruption shifts. It helped to improve the accuracy performance of the classifier on corruption shifts samples.

Comprehensive experiments showed that the MLFAnet performed well on benchmarks of corruption shift generalization. Furthermore, the prediction of noise and blur corruption shifts samples, in particular, had a dramatically improved. It indicated that the MLFAnet may provide support for the real-world diagnosis of tomato disease, i.e., it can diagnose the disease accurately from a blur leaf image which is captured by a shaking camera or a moving device. However, when the leaf images are affected by weather corruption and contrast transformation, the classification performance of MLFAnet is not outstanding. It shows that accurate diagnosis of MLFAnet is limited in complex weather conditions.

5. Conclusions

In this work, we investigated the OOD generalization of CNN architectures and proposed an efficient classification method for tomato disease. The method with a better capacity of generalization could diagnose tomato disease more accurately from corruption shifts images while retaining the satisfying classification performance of original images.

From our study and simulation results, we may obtain the following conclusions: (1) CNN may degrade the generalization ability while focusing on high-frequency components. (2) For a basic classification task, such as tomato disease with nine categories, the accuracy precision on IID samples of a shallow CNN architecture is similar to that of DCNN models. However, shallow-layer CNN architecture has a better OOD generalization performance. (3) Results verify the hypothesis that attention mechanisms are capable of improving the generalization performance of deep CNN models. For these reasons, an improved attention algorithm was proposed by calculating the weight of feature maps with multiple low-frequency components, and it has been demonstrated the effectiveness of corruption shifts generalization. Moreover, we provided an efficient CNN method named MLFAnet to improve the generalization on the diagnosis of tomato disease. An interesting question to make further progress would be verifying the effectiveness of MLFAnet on other fruits or vegetables. Though we consider our proposed method is based on a general CNN system, it can be applied to most of the image classification tasks with a high probability. However, extra work and experiments should be designed to prove it for further study.

Our work on corruption shifts sample generalization could scale to the real-world problem, which is left for future investigation.