Color-Sensitive Sensor Array Combined with Machine Learning for Non-Destructive Detection of AFB₁ in Corn Silage

Abstract

Aflatoxin B₁ (AFB₁) contamination in corn silage poses significant risks to livestock and human health. This study developed a non-destructive detection method for AFB₁ using color-sensitive arrays (CSAs). Twenty self-developed CSAs were employed to react with samples, with reflectance spectra collected using a portable spectrometer. Spectral data were optimized through seven preprocessing methods, including Standard Normal Variate (SNV), Multiplicative Scatter Correction (MSC), first-order derivative (1st D), second-order derivative (2nd D), wavelet denoising, and their combinations. Key variables were selected using five feature selection algorithms: Competitive Adaptive Reweighted Sampling (CARS), Principal Component Analysis (PCA), Random Forest (RF), Uninformative Variable Elimination (UVE), and eXtreme Gradient Boosting (XGBoost). Five machine learning models were constructed: Light Gradient Boosting Machine (LightGBM), XGBoost, Support Vector Regression (SVR), RF, and K-Nearest Neighbor (KNN). The results demonstrated significant AFB₁-responsive characteristics in three dyes: (2,3,7,8,12,13,17,18-octaethylporphynato)chloromanganese(III) (Mn(OEP)Cl), Bromocresol Green, and Cresol Red. The combined 1st D-PCA-KNN model showed optimal prediction performance, with determination coefficient ($R_p^2$ = 0.87), root mean square error (RMSEP = 0.057), and relative prediction deviation (RPD = 2.773). This method provides an efficient solution for silage AFB₁ monitoring.

1. Introduction

Silage corn feed, valued for its high nutritional content and superior storability, has emerged as a critical roughage source for ruminant livestock. However, suboptimal production practices—including inadequate compaction and improper sealing—can lead to fungal proliferation and mycotoxin biosynthesis [1,2]. Mold contamination in corn silage typically refers to the secondary fermentation triggered by exposure to air during feed retrieval, leading to the production of mycotoxins such as aflatoxins, zearalenone, and deoxynivalenol (vomitoxin). Among these, aflatoxin B₁ (AFB₁) is the most prevalent and toxic, recognized as one of the most potent naturally occurring carcinogens [3]. The ingestion of contaminated feed by livestock may compromise immune function, induce pathological conditions, and ultimately pose zoonotic risks through bioaccumulation in the food chain [4,5]. Current quality assessment protocols for silage corn feed primarily rely on organoleptic evaluation, with chemical and microbiological analyses serving as supplementary methodologies [6]. While sensory examination facilitates rapid quality determination through visual (moisture content, coloration) and olfactory indicators, this approach suffers from significant subjectivity. Chemical assays provide quantitative measurements of pH, organic acid profiles, and nutritional composition with enhanced precision, whereas microbiological testing evaluates populations of beneficial (e.g., Lactobacillus spp.) and pathogenic microorganisms (e.g., toxigenic molds). Nevertheless, these laboratory-based methods are constrained by protracted analysis timelines and substantial operational costs [7]. The predominant mycotoxins identified in silage corn include aflatoxin B1, deoxynivalenol (vomitoxin), and T-2 toxin—all exhibiting high toxicity thresholds. Conventional detection techniques such as enzyme-linked immunosorbent assay (ELISA) and high-performance liquid chromatography (HPLC), despite their respective advantages, present practical limitations including methodological complexity, elevated reagent costs, and time-intensive protocols [8]. Therefore, the development of a rapid, accurate, and scientific method for the detection of mycotoxins in corn silage feed has become a key issue to be solved.

Hyperspectral and near-infrared spectroscopy techniques have made significant progress in the field of mycotoxin detection in agricultural and sideline products [9]. Earlier studies [10] confirmed that this technique can effectively capture the molecular vibrational information of mycotoxins, laying the foundation for subsequent applications. Among the chemometric methods, the Partial Least Squares Regression (PLS) model was outstanding in the detection of fumonisin B1 in beans (R² = 0.92) [11], while the detection error of aflatoxin B1 in rice could be controlled at 2.691 μg/kg by the Standard Normal Variate (SNV)-PLS combination [12]. A more in-depth study [13] developed an online corn mold monitoring system by fusing hyperspectral and computer vision. The application of machine learning algorithms further enhanced the detection performance, with Support Vector Machine (SVM) achieving an accuracy of 3.57 μg/kg in the detection of maize aflatoxin precursor Ver A [14], and the Uninformative Variable Elimination (UVE) algorithm enabling the peanut aflatoxin detection model to achieve R² = 0.9577 [15]. Recent breakthroughs include the 95.7% classification accuracy of the Short-Wave Infrared (SWIR)-SVM combination for toxins in maize [16] and the 98.6% composite detection accuracy of the Random Forest algorithm for five toxin classes [17]. Existing studies have shown that short-wave hyperspectral [14] and spectral preprocessing [18] can significantly improve model stability, but the detection of complex structures such as Fusarium toxins is still challenging [17]. In the future, novel algorithms need to be developed, and a standardized framework needs to be established to optimize the detection performance.

The color-sensitive sensor technology generates RGB signals through the specific color responses of arrayed materials, which are combined with pattern recognition algorithms to achieve rapid detection of mycotoxins [19]. At the technical application level, the olfactory visualization system obtained 100% training set discrimination accuracy in wheat mold detection through the synergy of principal component dimensionality reduction and machine learning algorithms (e.g., SVM) [20]. Innovative applications of electronic nose technology have shown that its combination with lateral flow immunoassay can effectively identify aflatoxins (81% accuracy) and fumonisins (85% accuracy) in maize, and the multivariate discrimination model has a combined determination accuracy of 65% of the contamination status [21]. Leggieri et al. used an electronic nose system with artificial neural network (ANN) algorithms to achieve a high level of accuracy in the identification of aflatoxin B1 and fumonisins, with improved identification accuracy of 78% and 77%, respectively [22]. The latest technological breakthrough is reflected in the development of nanocomposite colorimetric sensor arrays, which have significantly improved the identification of AFB₁/AFB₂-producing strains through characteristic metabolite capture combined with Principal Component Analysis (PCA)/Linear Discriminant Analysis (LDA) modeling [23]. Existing studies have shown that the color-sensitive sensor technology has demonstrated good potential for application in the field of aflatoxin B₁ (AFB₁) detection, but the relevant studies for the specific substrate of corn silage feed are still relatively limited.

Different detection techniques present significant differences in technical characteristics in the field of mycotoxin detection. Andueza et al. [24] found that the calibration model established based on dried samples has significant limitations in silage applications, which suggests the importance of matrix specificity for the detection method. The nanoparticle-based colorimetric assay developed by Sheini [25] innovatively realizes the simultaneous detection of the three mycotoxins, and its high sensitivity and anti-interference properties provide a new solution for food and feed safety screening. In the comparison of spectroscopic techniques, Kim et al. [26] systematically showed that both fluorescence-QSVM and SWIR-LSVM combinations could achieve 95.7% detection accuracy at the 10 μg/kg threshold and maintain zero false negatives, and this performance was significantly better than the other combinations of techniques. The study of Deng [27], on the other hand, through the SVM model constructed by Particle Swarm Optimization (PSO)-Combined Moving Window (CMW) optimization algorithm, achieved high-precision quantitative detection of maize AFB₁ (R² = 0.9707), providing a feasible solution for field detection. With optimized machine learning algorithms, spectral technology attains sufficient detection accuracy to enable practical on-site feed analysis applications.

While researchers worldwide have successfully applied hyperspectral imaging, near-infrared spectroscopy, colorimetric sensors, and electronic nose technologies to mycotoxin detection in agricultural products (e.g., maize, wheat), studies on feedborne mycotoxins remain scarce. To address this gap, we developed a non-destructive detection method based on color-sensitive sensing technology for quantitative prediction of AFB₁ content. The main work of this study is as follows: (1) Screening 20 color-sensitive dyes to prepare 4 × 5 sensor arrays. (2) After the reaction of the self-developed color-sensitive sensor arrays (CSAs) with silage samples, the Maya 2000 portable spectrometer was used to collect the spectral reflectance data of silage maize feed, and the values of aflatoxin B₁ (AFB₁) content of different samples were obtained by physicochemical tests. (3) For the spectral data of different chromatic sites, seven preprocessing methods—SNV, Multiplicative Scatter Correction (MSC), first-order derivative (1st D), second-order derivative (2nd D), wavelet denoising (WD), SNV+1st D, and MSC+WD—were used to determine the optimal preprocessing methods for different chromatic sites. (4) After the implementation of the seven preprocessing methods for the different chromogenic loci after reaction with silage samples, the prediction analysis of AFB₁ content was carried out based on the three machine learning models—Support Vector Regression (SVR), Random Forest (RF), and K-Nearest Neighbor (KNN). Next, the model performance indexes were systematically evaluated (the coefficient of determination for the calibration set ($R_c^2$), root mean square error of calibration (RMSEC), prediction set coefficient of determination ($R_p^2$), root mean square error of prediction (RMSEP), and relative predictive deviation (RPD)). Finally, three dye components with significant specific response characteristics to AFB₁ were screened out. (5) Two preprocessing methods, MSC and 1st D, were selected. After preprocessing, the features were screened by five algorithms: Competitive Adaptive Reweighted Sampling (CARS), PCA, RF, UVE, and eXtreme Gradient Boosting (XGBoost), and the optimal feature data were outputted by five algorithmic models: Light Gradient Boosting Machine (LightGBM), KNN, XGBoost, RF, and SVR. (6) The best combination of AFB₁ toxin content prediction models was screened by comparing the $R_p^2$, RMSEP, and RPD as the main evaluation parameters.

2. Materials and Methods

2.1. Sample Acquisition

Inner Mongolia is rich in straw resources and has a lot of silage. This experiment took the silage corn feed from dairy farms in Inner Mongolia as the research object, and selected silage corn feed from seven dairy farms in Baotou City (latitude 40°15′~42°43′ N, longitude 109°15′~110°26′ E), Ulanqab City (latitude 40°29′~40°32′ N, longitude 112°28′~112°30′ E), and Hohhot City (39°58′~40°41′ N, 111°26′~112°18′ E). A total of five batches of silage maize feed were tested in two groups per batch, with each sample from the first two batches coming from a different farm, and the last three batches coming from the same farm in each batch. The corn silage at the farm was pit-stored in September 2023 after being chopped to 1–2 cm particle size, compacted to a density of ≥700 kg/m³ fresh matter, and sealed with oxygen-barrier film. The silage was unsealed in March 2024 following a 6-month fermentation period. The corn silage at the farm was pit-stored in September 2023 and unsealed in March 2024. The trial period was from April to July 2024. After the silage was delivered to the laboratory, it was divided into polyethylene boxes (approximately 4 kg per box, 2 boxes per sample) using the tetrad method [28], which ensures the stability of the samples by using polyethylene’s chemical inertness and lightness; the lids of the boxes were covered with a perforated aluminum foil to meet the needs of evaporation prevention and air permeability. Each batch was subjected to a 7-day aerobic exposure test (0D–6D), with 3 samples taken daily from each box (total of 6 samples/batch), divided into two portions for the color sensor response and two portions for the refrigerated retention samples. At the end of each batch (7 days), the refrigerated samples were removed for testing of AFB₁ toxin levels. A total of 42 samples were prepared from each batch, for a total of 210 samples from the five batches for subsequent testing. The test and data analysis processes are shown in Figure 1.

2.2. Preparation of CSAs

Color-sensitive sensors achieve detection by producing color changes through the interaction of the materials with the target. After several rounds of comparison before the test, 11 metal porphyrins and their derivatives and 9 PH indicators—a total of 20 color-sensitive materials—were finally selected for this test. Information on these 20 color-sensitive materials is shown in

Table 1. Color-sensitive dyes required for making the color-sensitive sensor array.

Number	Name
1	2,3,7,8,12,13,17,18-Octaethyl-21H,23H-porphine
2	5,10,15,20-Tetrakis(4-methoxyphenyl)-21H,23H-porphine iron (III) chloride
3	5,10,15,20-Tetrakis(4-methoxyhenyl)-21H,23H-porphine
4	5,10,15,20-Tetrakis(4-methoxyhenyl)-21H,23H-porphine cobalt (II)
5	5,10,15,20-Tetraphenyl-21H,23H-porphine
6	5,10,15,20-Tetraphenyl-21H,23H-porphine zinc
7	5,10,15,20-Tetraphenyl-21H,23H-porphine copper (II)
8	5,10,15,20-Tetraphenyl-21H,23H-porphine iron (III) chloride
9	5,10,15,20-Tetraphenyl-21H,23H-porphine manganese(III) chloride
10	5,10,15,20-Tetraphenyl-21H,23H-porphine palladium(II)
11	(2,3,7,8,12,13,17,18-octaethylporphynato)chloromanganese(III)
12	Bromocresol Green
13	Bromothymol Blue
14	Bromophenol blue
15	Congo red
16	Methyl Red—Ethanol
17	Bromocresol Purple
18	Neutral Red
19	Cresol Red
20	Bromothymol Blue

.

The selected 11 metal porphyrins and their derivatives (8 mg each) were dissolved in 4 mL of dichloromethane, and 9 pH indicators (8 mg each) were dissolved in 4 mL of ethanol, all formulated into 2 mg/mL solutions. After ultrasonic treatment for 30 min, the solutions were stored in the dark for later use. Subsequently, within a fume hood, 0.1 μL capillaries were used to spot each of the 20 solutions onto C2 reversed-phase silica gel plates (4 cm × 5 cm), controlling the diameter of each colorimetric spot to be 3 mm, arranged in a 4 × 5 matrix. After drying the sensor for 15 min and allowing the colorimetric sensor array to stabilize, they were individually sealed in sample bags for future use.

2.3. Spectral Data Acquisition from Color-Sensitive Sensor Array

This study utilized a custom-built spectral acquisition system consisting of a computer, spectrometer, Y-type optical fiber, single-hole cold light source, and a self-made, door-opening dark box. The spectrometer used was the Maya2000Pro produced by Ocean Optics, with a spectral collection range from 200 nm to 1100 nm, a resolution of up to 0.035 nm, and an adjustable integration time from 17 ms to 10,000 ms. The light source was a single-hole halogen lamp. Spectral data acquisition was carried out using Ocean Optics’ Ocean View 2.0.16 software. For spectral data collection, 50 ± 0.1 g of silage corn feed samples were placed in a beaker with a diameter of 60 mm. The back side of the color-sensitive sensor array was fixed to the cling film using double-sided tape, ensuring its front faced the silage corn feed sample. The beaker was then sealed with cling film, allowing the silage corn feed to react with the color-sensitive sensor for 18 min. After the reaction, the spectral data of the color-sensitive sensor array was collected using the self-built spectral acquisition system. Before collection, the spectrometer and light source were preheated for 30 min at 34–38 °C to reach a stable state. The lights were turned off, and the curtains were drawn to ensure the room was in a dim state. The dark box was installed, and the color-sensitive sensor, which had reacted with the sample, was placed on the carrier platform. The fiber optic probe was fixed in position using a clamp, maintaining a constant height of 1.5 cm from the color-sensitive sensor, and the spectral data of the 20 dye points on the reacted color-sensitive sensor were collected sequentially.

2.4. Determination of AFB₁ Content in Silage Corn Feed

According to the standards set by the Codex Alimentarius Commission (CAC), the limit for AFB₁ in corn and its products is 20 μg/kg. This study, based on the standard GB/T 17480-2008 “Determination of Aflatoxin B₁ in Feed—Enzyme Linked Immunosorbent Assay,” employed the enzyme-linked immunosorbent assay (ELISA) method using the Pribolab^® ELISA test kit to determine the AFB₁ content in 210 samples of silage corn feed prepared during the experiment. The limits of quantification (LOQ) were 1 μg/kg, and the detection limit (LOD) was 0.3 μg/kg. The microplate wells (12 strips × 8 wells) were placed in the ELX808 microplate reader (BioTek Instruments, Winooski, VT, USA) for absorbance measurement.

Before conducting the ELISA assay, a series of AFB₁ standard solutions was prepared according to the instructions provided with the Pribolab^® ELISA test kit. A stock solution of AFB₁ was diluted with the appropriate diluent to obtain standard concentrations of 0, 0.5, 1, 2, 5, and 10 μg/kg. These standard solutions were used to generate a standard curve, which was used to calculate the AFB₁ concentration in the corn silage samples.

Before the determination of AFB₁ content, sample pretreatment was performed. Firstly, 2.50 ± 0.01 g of sample was accurately weighed using an electronic balance with an accuracy of 0.01 g, and placed in a 15 mL centrifuge tube for spare use. Then, the sample was transferred to a homogenizer preheated to 37 °C, and 20 mL of 60% methanol solution was added, and homogenization was carried out for 30–45 s until the sample was completely homogenized (a homogeneous green suspension was observed by the naked eye). In order to ensure that the sample was completely recovered, the probe and residues on the wall of the homogenizer were rinsed with 5 mL of methanol solution of the same concentration. In order to ensure the complete recovery of the sample, 5 mL of the same concentration of methanol solution was used to rinse the probe of the homogenizer and the residues on the wall of the tubes; the centrifuge tubes were sealed and placed in a constant temperature shaker at 25 ± 1 °C for 5 min to promote the full dissolution of the solutes; finally, the centrifuge was centrifuged at 4000 rpm for 5 min, and the supernatant was filtered through a Whatman No.1 filter paper and then collected in a 10 mL glass test tube. The AFB₁ toxin assay was performed according to the ELISA method. Firstly, 200 μL of filtrate was mixed with 300 μL of diluent I to prepare the test solution, and the pH was adjusted to the range of 6.0–8.0, then 50 μL of the test solution, the enzyme marker and the antireagent were sequentially added to the plate equilibrated at room temperature for a 30-min room-temperature reaction, and then 300 μL of the wash solution was used to wash the plates for 3–4 times to remove the unbound material. After the reaction was completed, the reaction was repeated 3–4 times with 300 μL/well of washing solution to remove the unbound material, then 100 μL of color development solution was added to the plate to avoid light for 15 min, and the reaction was terminated by adding 50 μL of termination solution, and then the absorbance value was determined by using an enzyme marker that had been pre-warmed for 30 min, and the actual amount of AFB₁ in the samples to be tested was calculated according to the standard curve.

2.5. Spectral Data Preprocessing

Due to the performance issues of the spectrometer or light source, resulting in large threshold ranges and unsmooth bands in the initial and final spectral bands, this study only selects the stable band of 450–1000 nm for analysis and modeling. The spectral range is concentrated in the visible and shortwave near-infrared bands, encompassing 1252 dimensions of information within this range. With a sample size of 210, an initial data matrix of (210, 1252) is formed. Initially, Z-score standardization is applied to the spectral matrix data to eliminate the dimensional differences between features and improve the performance and stability of the model. The formula for the Z-score calculation is as follows:

$$s_k^{\prime }={\displaystyle \frac{s_k-\mu }{\delta }}$$

(1)

where $s_k^{\prime }$ represents the standardized data, $s_k$ represents the original data, $\mu$ represents the mean, and $\delta$ represents the standard deviation.

Locate the coordinates where the absolute value of the Z-score standardized values is greater than 3, and replace the values at these coordinates in the original data with the column mean. After modifying all outliers, complete the initial data cleaning process. Seven preprocessing methods commonly used for processing spectral data, SNV, MSC, 1st D, second-order derivative, wavelet denoising, SNV+1st D, MSC+wavelet denoising, were used to preprocess the 20 dye points, respectively, which included two combination methods. The preprocessed data were output through SVR, RF, and KNN3 models, respectively, which were used to filter out the best dye points.

2.6. Data Preprocessing, Feature Selection, and Model Establishment Algorithms

Due to the characteristics of the data in this experiment, which include a small sample size and high dimensionality, it is necessary to perform 5-fold cross-validation to compensate for the insufficiency of training data. In each iteration, 80% of the data is used for training and 20% for validation. The final performance is averaged, reducing the chance of random bias caused by a single data split and more reliably detecting whether the model is overfitting. For high-dimensional spectral data, this study adopts a data screening strategy to reduce the dimensionality in order to improve the model efficiency and prediction accuracy. In terms of algorithm selection, the principle of diversity is followed, covering different types of unsupervised learning and supervised learning, kernel methods and non-kernel methods, integrated models, and monolithic algorithms. By selecting a combination of algorithms with different principles, it is more conducive to finding the optimal modeling scheme. The following is a brief description of each of the five feature screening algorithms selected for this study.

CARS is a spectral feature wavelength selection method based on the Partial Least Squares Regression (PLS) model, with its core principle simulating a “survival of the fittest” competition mechanism. It employs an exponential decay function and adaptive reweighting techniques to progressively eliminate wavelength points with low information content, ultimately retaining the subset of features most strongly correlated with the target variable [29]. The advantages of this algorithm lie in its effectiveness in overcoming collinearity issues in high-dimensional spectral data, significantly enhancing the model’s prediction accuracy and robustness, while dynamically optimizing the wavelength selection process to balance computational efficiency with the reliability of feature selection.

UVE distinguishes informative variables from uninformative ones by introducing artificial noise variables as references and utilizing the stability analysis of regression coefficients (such as the mean/standard deviation ratio). It then sets thresholds based on the stability distribution of the noise variables to eliminate irrelevant features. The advantages lie in its efficiency in reducing the dimensionality of high-dimensional data like spectra, enhancing model computational efficiency and prediction accuracy, improving anti-interference capability through noise comparison, and enabling automated feature selection without prior knowledge. However, it is important to note the limitation that highly correlated variables may be mistakenly removed.

PCA is an unsupervised dimensionality reduction algorithm that projects high-dimensional data onto a lower-dimensional principal component space with the largest variance through orthogonal transformation. Its core principle involves performing eigenvalue decomposition on the data’s covariance matrix and retaining the top k eigenvectors with the highest cumulative contribution rate as new feature axes. The advantages of this algorithm lie in its effectiveness in eliminating data redundancy, reducing computational complexity, while maximally preserving the variability of the original information.

RF is a machine learning algorithm based on the integration of decision trees. Its core principle involves generating diverse training subsets through Bootstrap sampling and randomly selecting a subset of features for splitting during the construction of each tree, thereby enhancing the diversity of the model [30]. The algorithm performs classification prediction through a collective voting mechanism of multiple decision trees or conducts regression prediction by averaging, possessing an inherent resistance to overfitting. Its main advantages include the effective handling of high-dimensional data, strong robustness against outliers and missing values, and the ability to provide feature importance assessment. This algorithm is widely applicable to classification and regression tasks, but it also has limitations such as higher computational cost and the potential to generate overly complex tree structures in noisy data [31,32].

XGBoost, as an advanced gradient-boosted ensemble learning framework, constructs decision tree models through multiple rounds of iterations at its core. During the optimization process, it simultaneously considers first-order gradient information and second-order Hessian matrices, coupled with meticulously designed regularization terms to continuously enhance prediction accuracy. The algorithm innovatively implements a parallelized computing architecture and incorporates built-in feature importance evaluation capabilities, endowing it with significant advantages in engineering practice. XGBoost achieves efficient data processing through block storage and cache optimization, employs regularization constraints to prevent overfitting, and offers extensible interfaces that support custom loss functions and missing value handling.

This study selected five algorithms for model output. Among them, RF and XGBoost have already been introduced in the feature selection section, so they will not be repeated here. The other three algorithms are SVR, KNN, and LightGBM. Below is a brief introduction to these three algorithms.

SVR is a regression method based on the SVM framework. By introducing the ε-insensitive band and kernel function techniques, it maps the input space to a high-dimensional feature space for linear regression, achieving high-precision fitting of nonlinear relationships while ensuring the sparsity of the model solution [33]. Its core advantages lie in its strong robustness against outliers, effective handling of high-dimensional small sample data, and ensuring good generalization performance through structural risk minimization, making it particularly suitable for scenarios with high requirements for regression accuracy.

KNN is a type of instance-based supervised learning method. Its core principle involves calculating the distance metrics, such as Euclidean distance or Manhattan distance, between the target sample and the training samples in the feature space. It then selects the K nearest neighboring samples and uses a majority voting mechanism for classification prediction or calculates the mean for regression analysis [34]. As a representative of lazy learning algorithms, it does not require an explicit model training process and makes no prior assumptions about the data distribution. It has strong adaptability and naturally supports multi-class classification tasks. This method is particularly suitable for problem scenarios with distinct local feature patterns. However, its computational complexity grows linearly with the sample size, and it is relatively sensitive to high-dimensional data and imbalanced samples. In practical applications, it is necessary to carefully choose the value of K and the distance metric method.

LightGBM is an efficient machine learning framework based on gradient boosting decision trees. Its core principle involves the use of a histogram algorithm combined with a depth-limited Leaf-wise growth strategy to accelerate training, and it optimizes computational efficiency through one-sided gradient sampling and mutually exclusive feature bundling techniques. It has significant advantages, such as being more than ten times faster in training speed, low memory consumption, support for direct input of categorical features, efficient parallel computation, and maintaining high accuracy while handling large-scale data. However, it may be prone to overfitting on small sample datasets [35,36].

2.7. Quantitative Forecasting Model Evaluation Metrics

This study evaluates the quantitative prediction model through $R_c^2$, RMSEC, $R_p^2$, RMSEP, and RPD. The relevant calculation formulas are as follows:

$$R_c^2=1-{\displaystyle \frac{{{\displaystyle \sum }}_{i=1}^{n_c}{(y_i-\widehat{y_i})}^2}{{{\displaystyle \sum }}_{i=1}^{n_c}{(y_i-\overline{y_c})}^2}}$$

(2)

where $n_c$ is the number of samples in the calibration set, $\widehat{y_i}$ is the predicted value of the ith sample in the calibration set, $y_i$ is the measured value of the ith sample in the calibration set, and $\overline{y_c}$ is the average of the measured values of all samples in the calibration set.

$$RMSEC=\sqrt{{\displaystyle \frac{1}{n_c}}{{\displaystyle \sum }}_{i=1}^{n_c}{\left(y_i-\widehat{y_i}\right)}^2}$$

(3)

where $n_c$ is the number of samples in the calibration set, $\widehat{y_i}$ is the predicted value of the ith sample in the calibration set, and $y_i$ is the measured value of the ith sample in the calibration set.

$$R_p^2=1-{\displaystyle \frac{{{\displaystyle \sum }}_{i=1}^{n_p}{(y_i-\widehat{y_i})}^2}{{{\displaystyle \sum }}_{i=1}^{n_p}{(y_i-\overline{y_p})}^2}}$$

(4)

where $n_p$ is the number of samples in the prediction set, $\widehat{y_i}$ is the predicted value of the ith sample in the prediction set, $y_i$ is the measured value of the ith sample in the prediction set, and $\overline{y_p}$ is the average of the actual measured values of all samples in the prediction set.

$$RMSEP=\sqrt{{\displaystyle \frac{1}{n_p}}{{\displaystyle \sum }}_{i=1}^{n_p}{\left(y_i-\widehat{y_i}\right)}^2}$$

(5)

where $n_p$ is the number of samples in the prediction set, $\widehat{y_i}$ is the predicted value of the ith sample in the prediction set, and $y_i$ is the measured value of the ith sample in the prediction set.

$$RPD={\displaystyle \frac{SD}{RMSEP}}$$

(6)

where SD is the standard deviation of the prediction set, and the RMSEP is the root mean square error of the prediction set.

The model’s predictive capability is mainly evaluated using $R_p^2$, RMSEP, and RPD [37,38]. The closer the value is to 1, the higher the goodness of fit between the model’s predicted values and the actual observed values, and the lower the RMSEP value, indicating higher prediction accuracy. RPD reflects the generalization performance of the evaluation model on unknown data. If RPD < 1.5, the model cannot achieve sample prediction. When 1.5 ≤ RPD < 2.0, the model can perform a rough assessment of the samples; when 2.0 ≤ RPD < 3.0, the model can make excellent predictions on the samples.

3. Analysis of Test Results

3.1. AFB₁ Content Analysis

By collecting samples from each batch for chemical testing, the AFB₁ content of each sample was measured, and a heatmap illustrating the changes in AFB₁ content was created as shown in Figure 2. In the figure, warm colors represent high concentrations, while cool colors indicate low concentrations.

3.2. Data Preprocessing and Selection of Optimal Dye Points

Due to the significant differences in toxin values among the five batches of samples and the uneven data distribution, randomly selecting test samples may not ensure that samples with different concentrations are included in the training, leading to poor training results. Therefore, the data sampling is first sorted by time period, with every 42 samples belonging to the same time period, dividing the samples into five time periods. Within each time period, stratified random sampling is conducted. The test set and prediction set are divided at a ratio of 4:1, totaling 210 samples, with 147 samples in the test set and 63 samples in the prediction set. At the same time, 5-fold CV cross-validation is carried out to compensate for the defect of having few samples.

Twenty dye points were preprocessed using seven methods: SNV, MSC, 1st D, 2nd D, WD, SNV+1st D, and MSC+WD. The original unprocessed data was used as a reference. After preprocessing, the data was analyzed using three models: SVR, RF, and KNN to generate outputs. The $R_c^2$, RMSEC, RMSEP, $R_p^2$ and RPD for each preprocessing method were collected. The model outputs for filtering different dye points show significant differences in evaluation metrics, with the predictive performance of most dye points being significantly insufficient ($R_p^2$ < 0.5). Therefore, the 19th dye point, which performed the best, was selected for comparative analysis of indicators.

Table 2. Comparison of the performance of AFB₁ content prediction models for dye point 19 under different spectral preprocessing methods.

Model	Preprocessing Methods	Calibration		Prediction
		${\mathit{R}}_{\mathit{C}}^2$	RMSEC	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD
SVR	Raw Data	0.8556	0.0547	0.7321	0.0766	1.9427
	SNV	0.9076	0.0436	0.7674	0.0712	2.0952
	MSC	0.9077	0.0435	0.7688	0.0710	2.1041
	1st D	0.8995	0.0455	0.6391	0.0886	1.6849
	2nd D	0.9000	0.0454	0.4847	0.1062	1.4013
	WD	0.8215	0.0608	0.7144	0.0790	1.8814
	SNV+1st D	0.8982	0.0457	0.6819	0.0832	1.7951
	MSC+WD	0.9070	0.0438	0.7531	0.0733	2.0413
RF	Raw Data	0.9287	0.0385	0.7580	0.0729	2.0329
	SNV	0.9368	0.0362	0.7804	0.0694	2.1339
	MSC	0.9669	0.0262	0.8217	0.0626	2.3682
	1st D	0.9461	0.0335	0.6361	0.0894	1.6577
	2nd D	0.8834	0.0492	0.3677	0.1178	1.2576
	WD	0.9303	0.0381	0.7440	0.0750	1.9763
	SNV+1st D	0.9394	0.0355	0.7430	0.0751	1.9724
	MSC+WD	0.9450	0.0338	0.7898	0.0679	2.1811
KNN	Raw Data	0.9993	0.0264	0.7010	0.0733	1.8289
	SNV	0.9993	0.0264	0.8425	0.0532	2.5197
	MSC	0.9993	0.0264	0.8103	0.0584	2.2962
	1st D	0.9993	0.0264	0.8662	0.0472	2.8420
	2nd D	0.9993	0.0264	0.6805	0.0758	1.7690
	WD	0.9993	0.0264	0.7014	0.0733	1.8300
	SNV+1st D	0.9993	0.0264	0.8326	0.0549	2.4441
	MSC+WD	0.9993	0.0264	0.8013	0.0598	2.2436

Note: SVR (Support Vector Regression), RF (Random Forest), KNN (K-Nearest Neighbor), SNV (Standard Normal Variate), MSC (Multiplicative Scatter Correction), 1st D (first-order derivative), 2nd D (second-order derivative), WD (wavelet denoising), $R_C^2$ (the coefficient of determination for the calibration set), RMSEC (root mean square error of calibration), $R_P^2$ (prediction set coefficient of determination), RMSEP (root mean square error of prediction), RPD (relative predictive deviation).

summarizes the AFB₁ prediction indicators for the 19th dye point under seven preprocessing conditions, as output by the SVR, RF, and KNN models. The optimal preprocessing methods and indicator values under each model are shown in bold.

According to the data analysis in Table 2, the optimal preprocessing method for Dye 19 is MSC in both SVR and RF models, while 1st D preprocessing yields the best results in the KNN model. It is noteworthy that the KNN model exhibits the highest predictive performance on this dye point, with a determination coefficient $R_P^2$ of 0.8662. In contrast, the 2nd D preprocessing method performs suboptimally across all models, possibly because it significantly amplifies high-frequency noise while eliminating baseline drift, leading to a reduced signal-to-noise ratio.

The study also found that the predictive performance of combined preprocessing algorithms did not outperform that of a single preprocessing method. This result indicates that, for this dataset, using a single preprocessing algorithm is sufficient to fully extract the key features of the data, while the combination algorithm not only increases computational complexity but may also introduce unnecessary noise or information redundancy. Based on the above analysis, and according to the evaluation index data of each dye point under different preprocessing algorithms, the optimal dye points corresponding to each algorithm model were selected. Dye points with an $R_P^2$ greater than 0.6 and an RPD greater than 1.5 after each preprocessing method were prioritized, and a selection chart for the best dye points was drawn, as shown in Figure 3.

As shown in Figure 3, dye spots No. 11, No. 12, and No. 19 were ultimately selected as the best, corresponding to (2,3,7,8,12,13,17,18-octaethylporphynato)chloromanganese(III)(Mn(OEP)Cl), Bromocresol Green, and Cresol Red, respectively. One of these is a metal porphyrin derivative, and two are pH indicators. This indicates that these dyes exhibit excellent spectral response characteristics within a specific wavelength range, enabling more precise detection of minute changes in AFB₁ concentration. The actual screening process for selecting the dye spots is depicted in Figure 4.

List the optimal pretreatment methods and $R_P^2$ and RPD index values after the optimal dye points are output by SVR, RF, and KNN models, as shown in

Table 3. Optimal preprocessing method and $R_P^2$ and RPD metric values for the optimal dye point after three model outputs of SVR, RF, and KNN.

Model	Dye Point	Method	${\mathit{R}}_{\mathit{P}}^2$	RPD
SVR	19	MSC	0.7688	2.1041
	12	MSC+WD	0.6868	1.7880
	11	MSC	0.6629	1.7256
RF	19	MSC	0.8217	2.3682
	12	MSC	0.6779	1.7621
	11	WD	0.6971	1.8169
KNN	19	1st D	0.8662	2.8420
	12	SNV	0.7839	2.1513
	11	1st D	0.7695	2.0827

Note: Abbreviations as defined in Table 2.

.

Analysis of the optimal preprocessing methods in Table 3 indicates that MSC is the most frequently used, appearing a total of five times, including four instances on its own and one instance within a combined preprocessing algorithm. The 1st D appears a total of two times, while SNV, WD, and MSC+WD each appear once. Notably, the preprocessing method corresponding to the maximum $R_P^2$ value after the output of the SVR and RF models is MSC, and the preprocessing method corresponding to the maximum $R_P^2$ value after the output of the KNN model is 1st D. This is because MSC processing not only eliminates physical interference but also retains the key chemical information of the samples [36]. After MSC preprocessing, the performance indicators are high, suggesting that the data processed by MSC more directly reflects the chemical composition differences of the samples, enabling the model to more accurately establish a quantitative relationship between the spectrum and the target properties. After preprocessing with the 1st D, the KNN model outputs the highest $R_P^2$ value for dye No. 19, reaching 0.8662. Based on the above analysis, the subsequent feature selection and modeling will, respectively, adopt the MSC method, which appears most frequently, and the 1st D method, which has the highest $R_P^2$ value.

The original spectra of dyes No. 11, No. 12, and No. 19 are shown in Figure 5. The spectra after 1st D processing are displayed in Figure 6, and the spectra after MSC preprocessing are shown in Figure 7. In all figures, the x-axis represents wavelength, and the y-axis represents reflectance.

3.3. Feature Selection Algorithms and Determination of the Optimal Model

Read the data of dye points 11, 12, and 19, and divide the test set and prediction set in a 4:1 ratio. The 210 samples are divided into five time periods, and stratified random sampling is carried out within each time period, while performing 5-fold cross-validation. The data is preprocessed with either MSC or 1st D, and then subjected to feature selection using five algorithms: CARS, PCA, RF, UVE, and XGBoost. The feature selection process uses GridSearchCV with 5-fold cross-validation for parameter search, with the $R_p^2$ as the evaluation metric to select the optimal number of features and determine the best model parameters. The optimal feature data is then output through five algorithm models: LightGBM, KNN, XGBoost, RandomForest, and SVR. Statistics $R_p^2$, RMSEP, and RPD are used as the main evaluation parameters, and the optimal number of features is also output. The above algorithms and evaluation metrics are summarized in

Table 4. Performance comparison of hybrid feature selection-modeling approaches under MSC and 1st D preprocessing.

Feature Selection Algorithm	Preprocessing Methods	Number of Best Features (Where PCA Refers to the Number of Principal Components	Model	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD
CARS	MSC	1001	LightGBM	0.76	0.077	2.042
			KNN	0.765	0.077	2.063
			XGBoost	0.734	0.082	1.937
			RF	0.73	0.082	1.923
			SVR	0.718	0.084	1.882
	1st D	218	LightGBM	0.722	0.083	1.896
			KNN	0.866	0.058	2.733
			XGBoost	0.71	0.085	1.857
			RF	0.698	0.087	1.82
			SVR	0.667	0.091	1.733
PCA	MSC	76	LightGBM	0.758	0.078	2.031
			KNN	0.782	0.074	2.141
			XGBoost	0.695	0.087	1.812
			RF	0.727	0.083	1.915
			SVR	0.721	0.084	1.892
	1st D	67	LightGBM	0.652	0.093	1.696
			KNN	0.87	0.057	2.773
			XGBoost	0.58	0.102	1.544
			RF	0.564	0.104	1.515
			SVR	0.706	0.086	1.844
RF	MSC	792	LightGBM	0.757	0.078	2.029
			KNN	0.779	0.074	2.128
			XGBoost	0.744	0.08	1.977
			RF	0.718	0.084	1.883
			SVR	0.601	0.1	1.583
	1st D	1122	LightGBM	0.719	0.084	1.886
			KNN	0.722	0.083	1.895
			XGBoost	0.723	0.083	1.9
			RF	0.683	0.089	1.776
			SVR	0.608	0.099	1.597
UVE	MSC	187	LightGBM	0.699	0.087	1.822
			KNN	0.747	0.08	1.986
			XGBoost	0.692	0.088	1.802
			RF	0.696	0.087	1.815
			SVR	0.663	0.092	1.723
	1st D	187	LightGBM	0.744	0.08	1.978
			KNN	0.756	0.078	2.026
			XGBoost	0.735	0.081	1.944
			RF	0.7	0.087	1.825
			SVR	0.662	0.092	1.721
XGBoost	MSC	1252	LightGBM	0.761	0.077	2.045
			KNN	0.747	0.08	1.988
			XGBoost	0.725	0.083	1.908
			RF	0.718	0.084	1.884
			SVR	0.591	0.101	1.563
	1st D	1252	LightGBM	0.728	0.083	1.916
			KNN	0.664	0.092	1.726
			XGBoost	0.725	0.083	1.909
			RF	0.688	0.088	1.789
			SVR	0.608	0.099	1.598

Note: CARS (Competitive Adaptive Reweighted Sampling), PCA (Principal Component Analysis), UVE (Uninformative Variable Elimination), XGBoost (eXtreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine). For other abbreviations, see Table 2.

.

In order to increase the visualization effect and facilitate the comparison of evaluation indexes, the radar chart of different model output evaluation indexes under each feature screening is drawn, as shown in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17 below. The MSC preprocessing group and the 1st D of the two types of preprocessing groups are set out of the chart for comparison, respectively. In the chart, the longer the $R_P^2$ and RPD lines, the better the model is, and the shorter the RMSEP line is, the better the model is.

Analyzing the results of the above figure, it can be seen that the feature screening results based on the CARS algorithm show that different spectral preprocessing methods have a significant impact on the selection of the optimal number of features. When MSC preprocessing is used, the optimal number of features determined by the system’s automatic optimization is 1001. Meanwhile, when 1st D preprocessing is used, the optimal number of features decreases to 218. It is worth noting that the increase in the number of features did not significantly improve the model prediction performance; on the contrary, the optimal prediction was obtained in the combination of the 1st D-CARS-KNN algorithm ($R_P^2$ = 0.866).

In the UVE feature screening algorithm, the system stably selects 187 optimal features regardless of the preprocessing method, showing strong consistency in feature selection. In contrast, the XGBoost algorithm selects the maximum number of features (1252) under all preprocessing conditions, while the feature selection results of the Random Forest (RF) algorithm show large fluctuations: 792 under the MSC preprocessing, increasing to 1122 under the 1st D preprocessing. It should be noted in particular that the performance of the prediction models constructed by all three algorithms (UVE, XGBoost, RF) did not reach the desired level ($R_P^2$ < 0.8).

The PCA dimensionality reduction analysis showed that 76 principal components were extracted by the MSC preprocessing and 67 principal components were extracted by the 1st D preprocessing. Under both preprocessing conditions, KNN was the optimal modeling method, with the 1st D-PCA-KNN combination achieving the best prediction ($R_P^2$ = 0.87). This result further validates the important impact of the synergy between feature selection methods, preprocessing techniques, and modeling algorithms on model performance.

4. Discussion

4.1. Performance Comparison of CARS-Based Feature Selection Under Stepwise Preprocessing Strategies

Experimental results indicate that the 1st D-CARS-KNN model achieves an $R_P^2$ of 0.866. To investigate whether model performance could be further improved by applying dye-specific preprocessing followed by CARS-based feature selection, we systematically evaluated three dye datasets, each preprocessed with their respective optimal methods (determined via KNN performance) before CARS screening. While maintaining the automated parameter optimization mechanism of KNN, key performance metrics ($R_P^2$, RMSEP, RPD) were analyzed with dynamically selected optimal feature numbers.

Table 5. Performance comparison of CARS-based feature selection across diverse preprocessing strategies.

Model	Preprocessing Methods	Optimal Number of Features	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD	Optimal Parameters
CARS-KNN	11, 19-1st D 12-SNV	779	0.695	0.087	1.812	metric = manhattan, n_neighbors = 5, weights = ‘uniform’
	1st D	218	0.866	0.058	2.733	metric = manhattan, n_neighbors = 3, weights = distance

Note: Abbreviations as defined in Table 2 and Table 4.

compares the model performance between stepwise-optimized preprocessing and uniform 1st D preprocessing under CARS feature selection.

Analysis of Table 5 reveals that when applying dye-specific optimal preprocessing methods, the CARS algorithm selected 779 optimal features (with the KNN optimal neighbor parameter adjusted to 5), significantly exceeding the 218 features obtained under uniform preprocessing. Notably, this approach resulted in degraded predictive performance (R = 0.695). The underlying mechanism may stem from dye-specific preprocessing inducing heterogeneous distributional shifts across datasets, which could obscure or distort the “globally significant wavelengths” identified by CARS, ultimately compromising prediction accuracy. This interaction between preprocessing and feature selection highlights a critical trade-off: while localized optimization enhances feature representation, it must be balanced against the need for global data consistency to maintain model robustness.

4.2. Performance Comparison of PCA-Based Feature Selection Under Stepwise Preprocessing Strategies

Experimental results show that the 1st D-PCA-KNN model achieved an $R_P^2$ of 0.87, prompting examination of potential performance improvement through dye-specific preprocessing combined with PCA-based feature selection. As a linear dimensionality reduction method, PCA performs an orthogonal transformation to project data along maximal variance directions. Considering PCA’s mathematical constraints, two approaches were evaluated for processing three independently preprocessed dye datasets: Merge-then-PCA Screening (consolidating data before PCA reduction) and PCA-then-Merge Screening (applying PCA individually, then merging).

Table 6. Model performance comparison: stepwise optimized preprocessing with Merge-then-PCA screening vs. uniform 1st-D preprocessing.

Model	Preprocessing Methods	Best Primary Score	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD	Optimal Parameters
PCA-KNN	11, 19-1st D 12-SNV	20	0.657	0.093	1.707	metric = Manhattan, n_neighbors = 5, weights = distance
	1st D	67	0.87	0.057	2.773	metric = Manhattan, n_neighbors = 3, weights = distance

Note: Abbreviations as defined in Table 2 and Table 4.

compares the model performance between the stepwise-optimized preprocessing with the Merge-then-PCA screening strategy and the uniform 1st D preprocessing method.

Table 6 demonstrates that when employing the Merge-then-PCA strategy, the optimal number of principal components decreased to 20 (with KNN neighbor parameter simultaneously optimized to 5), yet the predictive performance metrics deteriorated significantly. This indicates that while the method maintains spectral data continuity, it introduces non-negligible noise interference due to multi-source data merging, confirming the infeasibility of this approach. Shifting to analyze the PCA-then-Merge method, its advantage lies in preserving the spectral specificity of each dye, but it faces the technical challenge of inconsistent feature dimensions after dimensionality reduction. This leads to a critical methodological choice—whether to output datasets with unified dimensions or maintain independent dimensional structures. This study first verifies the feasibility of the unified dimension approach by enforcing all dye points to undergo dimensionality reduction using the same number of principal components, ensuring feature space consistency.

Table 7. Performance metrics of stepwise-optimized preprocessing combined with PCA-then-Merge strategy under unified dimensionality output.

Harmonization of Principal Components	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD	Optimal Parameters
99	0.806	0.069	2.274	metric: euclidean, n_neighbors: 5, weights: distance

Note: Abbreviations as defined in Table 2.

presents the model performance metrics of the stepwise-optimized preprocessing combined with the PCA-then-Merge strategy under unified dimensionality output.

Table 7 demonstrates a performance metric ($R_P^2$ = 0.809) that significantly surpasses the Merge-then-PCA approach, with optimized KNN parameters shifting to Euclidean distance metric (n_neighbors = 5), though still not exceeding the 1st D-PCA-KNN model’s predictive capability. Subsequent analysis examines independent dimensionality output performance, where zero-padding aligns datasets to the maximum principal component count among three dyes while preserving individual dimensionality characteristics, with

Table 8. Performance metrics of stepwise-optimized preprocessing with PCA-then-Merge strategy under independent dimensionality output.

The Number of Principal Components for Dye 19	The Number of Principal Components for Dye 12	The Number of Principal Components for Dye 11	Maximum Filling Dimension	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD
114	99	120	120	0.779	0.074	2.131

Note: Abbreviations as defined in Table 2.

detailing each dye’s principal component numbers and corresponding model performance metrics under this independent dimensionality condition.

According to the data analysis in Table 8, it can be seen that when using PCA for screening, the predictive ability of independent dimensions is not as good as that of unified dimensions. The reason for this may be that under independent PCA, the individual dimensionality reduction for each dye point disrupts the global correlation of the spectral data, leading to the loss of collaborative features across dye points. It could also be that zero-filling introduces invalid dimensions, diluting the concentration of effective features.

4.3. Performance Comparison of KNN-Based Feature Importance Screening Under a Stepwise Preprocessing Strategy

Since KNN achieves the best predictive performance regardless of whether CARS or PCA is used for feature selection, this study examines whether KNN-based direct feature selection can further improve model performance metrics. The validation proceeds as follows:

First, all three dye datasets are preprocessed using the first-derivative transformation. Feature importance is evaluated via KNN distance, where the average KNN distance of each wavelength across samples is calculated and converted to a normalized importance score (0–1 range). An automated test selects features from 50 to full dimensionality, using cross-validated R² as the criterion to dynamically determine the optimal number of top-N features.

The model optimization process utilizes 5-fold cross-validated GridSearchCV with R² as the primary evaluation metric. Two distinct strategies are systematically compared for scenarios necessitating dye-specific preprocessing: the Filter-then-Merge approach, which independently selects features per dye dataset through permutation importance assessment (quantified by performance degradation following feature value randomization), retaining the top 20% of features with prioritized inclusion of both stable features (exhibiting cross-dye consistency) and local features (dye-specific high-importance wavelengths); and the Merge-then-Filter strategy, which initially consolidates all datasets prior to feature selection, preserving the globally optimal 20% of features based on consensus importance while strategically incorporating dye-specific critical wavelengths to mitigate potential information loss. Comprehensive performance metrics and optimized parameters for these two methodologies, along with the baseline KNN model, are quantitatively compared and presented in

Table 9. Performance comparison of KNN-based feature selection across different preprocessing strategies.

Model	Preprocessing Methods	Data Processing Methods	Optimal Number of Features	${\mathit{R}}_{\mathit{P}}^2$	RMSEP	RPD	Optimal Parameters
KNN-KNN	1st D		216	0.796	0.087	1.807	Metric: manhattan, n_neighbors: 3, weights: distance
	11, 19-1st D 12-SNV	Merge-then-Filter	251	0.812	0.068	2.306	metric: manhattan, n_neighbors: 3, weights: distance
		Filter-then-Merge	96	0.721	0.083	1.893	metric: manhattan, n_neighbors: 3, weights: distance

Note: Abbreviations as defined in Table 2.

, thereby facilitating a rigorous evaluation of their respective predictive capabilities under varying preprocessing configurations.

Analysis of Table 9 reveals that the Merge-then-Filter approach achieves the highest predictive performance ($R_P^2$ = 0.812). In contrast, applying a unified first-derivative preprocessing yields an $R_P^2$ of 0.796, demonstrating moderate predictive efficacy. The Filter-then-Merge method performs the worst, likely because merging data prior to screening preserves the global distribution characteristics across dye samples, mitigating local biases induced by individual preprocessing steps. Additionally, KNN’s distance-based mechanics benefit from merged data, as neighborhood calculations incorporate comprehensive spatial relationships. Premature screening in Filter-then-Merge may disrupt critical feature correlations, leading to erroneous neighborhood assessments.

The analysis demonstrates that predictive performance marginally surpasses uniform preprocessing only when employing KNN-based feature importance with dye-specific preprocessing followed by the Merge-then-Filter strategy. Notably, dye-specific preprocessing consistently underperforms uniform preprocessing across all feature selection methods (CARS or PCA). Specifically, Merge-then-Filter achieves superior results with KNN-driven screening, whereas Filter-then-Merge proves more effective for PCA-based screening.

5. Conclusions

This study employed a colorimetric sensor array combined with spectral analysis to verify the feasibility of quantitative detection of AFB₁ in silage corn feed. Seven preprocessing methods were applied to the spectral data from different colorimetric points, and model performance parameters were compared using the SVR, RF, and KNN models. It was found that single preprocessing methods outperformed combined preprocessing methods, with the 1st D preprocessing method showing the best performance. Preprocessing analysis also successfully identified three dye components—Mn(OEP)Cl, Bromocresol Green, and Cresol Red—that exhibited significant specific responses to AFB₁. By employing various typical machine learning algorithms for spectral data screening and modeling, and comparing different model evaluation metrics, it was discovered that the combination of CARS and PCA for feature selection with the KNN model output resulted in the smallest error in predicting AFB₁ toxin content. The optimal model for predicting AFB₁ toxin content was found to be 1st D-PCA-KNN, which can effectively and rapidly predict the AFB₁ content in silage feed. This study systematically compared the strategies of unified preprocessing and stepwise optimization preprocessing for the first time, confirming that adopting the optimal preprocessing methods for different chromogenic sites and using KNN importance screening has improved the model performance parameters. It provides new methodological guidance for the fusion analysis of multi-source spectral data, filling the research gap in this field.

Currently, due to the spectral data collected directly from samples not being timely introduced into the analysis, the performance indicators of the model can be steadily improved in the later stage by increasing the number of training samples. In addition, if this research method is to be used for on-site detection, it is also necessary to study the stability and repeatability of colorimetric sensors. These are all directions that our team will strive to research in the future.