1. Introduction
Product quality and efficiency are of great importance in the design of drying processes for exotic fruits or other sensitive biological foodstuffs. This is of great interest, since on the one hand, energy is becoming more and more expensive and the accessibility of energy cannot be guaranteed always and everywhere. On the other hand, exotic fruit demonstrate great health benefits like vitamins, sugars, or acids, which could be destroyed in poor drying processes. For measuring the retention of these however, complicated chemical analyses are necessary. In modern approaches in drying technology, a substitution of these analysis techniques is sought. The use of spectrometers, RGB (Red, Green, Blue) cameras or combinations of both (e.g., hyperspectral imaging) bear chances of being this substitution.
Much research has been done in this field with various products. An overview of the different technologies used in food processing and their advantages as well as disadvantages are given by Zhang, et al. [
1]. They describe which measuring techniques are commonly used in which applications and for which products. Further, they explain common methods for image correction and spectral compensation. Gomez et al. used VIS (visible)- and NIR (near infrared) spectroscopy techniques to measure the acidity, strength and amount of total soluble solids in mandarins [
2]. Martinsen and Schaare analyzed the distribution of soluble solids in kiwifruit using NIR spectroscopy [
3] and Wang et al. worked on predicting the strength and soluble solids content of pears using VIS and NIR spectroscopy [
4]. Vejarano et al. provide an overview of various analytical and optical methods for the detection of biological contaminants in food and then focus on the use of hyperspectral imaging [
5]. They highlight different contaminants that are critical for food safety and show how hyperspectral imaging can be used for early detection of contamination. Detailed work on the use of spectral and hyperspectral imaging in drying has been performed on tea [
6], carrots [
7], soy beans [
8], and apples [
9,
10,
11,
12]. The use of hyperspectral imaging in the drying of mangoes has been studied by Pu and Sun, who investigated the changes in moisture content when drying mango slices in a microwave [
13]. The results of all their work showed that it is possible to predict the intrinsic properties of fruits using spectroscopic methods.
It was also shown that each method has disadvantages that have to be dealt with when wanting to use the techniques for inline measuring in food processing or drying processes: Spectrometers deliver big amounts of data that have to be saved, processed, analyzed, and interpreted. At the same time, they provide information only point by point and some fruits require spatial resolution. Point spectrometers also have the disadvantage of a fixed measuring aperture in which the measured value is averaged. With textured surfaces such as fruits, this averaging process leads to a systematic error through which the spectral texture information is lost. Pure RGB cameras, which contain spatial information, hold too little spectral information and do not cover wavelengths in the ultra violet or infrared regions. Hyperspectral cameras, on the other hand, are expensive and very complicated to operate. Further, hyperspectral imaging systems often need moving products, which cannot always be realized in manufacturing processes.
In this work, the use of a novel multi-spectral snapshot imaging system is tested in drying processes. The system acquires simultaneously 12 distinct spectral image channels. Based on a pre-defined calibration, spectral surface reflectance data can be estimated on a pixel-by-pixel basis from the image data. Spectral reflectance is a physical surface property describing how an incident spectrum of light is reflected from a surface. Hence, this is a property that is independent of the scene lighting, which is a limitation of conventional color measurement with RGB cameras. Using the proposed multi-spectral imaging workflow, device independent spectral color measurement is possible.
With the use of this multi-spectral imaging system and the spectra that are calculated out of the images, links to changes in the chemical components can be predicted.
3. Results and Discussion
The overall goals of the work were to find out if the multi-spectral imaging system could be used for quality control in drying and to acquire information about the relationship between the estimated surface reflectance spectra and the measured chemical values.
3.1. Multi-Spectral Imaging System
Using the multi-spectral image data, pixel-wise surface reflectance data were estimated and averaged over an area that corresponds to the spectroradiometer measurement spot.
Figure 3 shows the RGB pictures of mangoes and their changes over time (0 min up to 390 min) from the preliminary experiments for reference. It can be seen that the color changes slightly but uniformly over the drying time. This behavior can also be found in the changes in the calculated spectral reflectance (
Figure 4).
Details about the imaging systems can be found in the work of Zirkler, et al. [
16].
The results for the production of quality criteria are shown in the following tables and graphs.
3.2. Moisture Content Prediction
The data of the multi-spectral imaging system were then used to investigate if a prediction of the samples’ quality criteria based on the spectral information was possible. When using PLS and PCA/PCR highest R
2pred were obtained when predicting the relative moisture content
xwb of the samples. As can be seen from the data in
Table 2 and
Figure 5, the measured and the predicted results have a low RSME
pred and high R
2pred of the prediction of the test data.
With PLS the predicted values are very close to the measured ones after the first quarter of the prediction and have a slight “under-prediction” in the second quarter, where the predicted values are lower than the measured values. For the PLS modelling, the difference between prediction and measurement gets smaller again in the last two quarters, whereas for the PCA/PCR prediction, the difference gets bigger after the first quarter. This is also shown in
Figure 5.
The predicted values of the PLS and of the PCA/PCR both show a similar behavior regarding the curves’ shapes. The PCA/PCR prediction also shows lower values than the ones actually measured. For the prediction of the moisture content, the PLS gives better results than the PCA/PCR. When using the PLS for the prediction of the moisture content xwb out of the spectra measured, the results show a good fit to the measured xwb. This can be seen in the graph as well as in a high R2 and a low RSME. Especially for low moisture contents, which are of great interest when wanting to know when a product is dry, the values of prediction and measurement show the same tendency.
The results obtained therefore showed that it is possible to use the multi-spectral imaging system for the prediction of the moisture content xwb. The problems of the high predicted values in the first quarter and the low values of the PCA/PCR prediction have to be further analyzed. Reasons for this might lay in the use of PLS and PCA/PCR for prediction, when other models might be better for this problem. Further, it might make sense to not predict one complete experiment out of the data of four other full experiments, but to mix all measured values and then use random 80% of the data for the prediction of the other 20%.
3.3. Prediction of Other Chemical Criteria
For the prediction of the TSS and the pH-values, less data were acquired (only one measurement every 30 min instead of one every 3 min). Therefore, the amount of data for the modelling and the prediction was much lower. The prediction of the pH-values with the use of the acquired data was not possible since no relation of pH-value and spectral data was found on basis of the acquired data when using PLS and PCA/PCR. Therefore, no further prediction model was developed for these parameters.
The results for the prediction of TSS are shown in
Table 3 (prediction for the rehydrated product),
Table 4 (prediction for the rehydration water) and
Figure 6 (prediction of rehydration water and rehydrated product).
For the TSS of the rehydrated products, the PLS prediction showed much higher R2 and therefore more similar values to the measurement than the PCA/PCR. Especially for the first two measurements, the predicted value and the measured one were almost the same. For the other three, the prediction was a bit higher than the measured values, but with an overall R2pred of 0.93, the prediction is still good. The prediction with the PCA/PCR gave much lower values for the first three measurements than the measured values and gives a low R2pred.
When predicting the TSS of the rehydration water, the PCA/PCR showed a slightly higher R2 and therefore more similar values to the measurement than the PLS. For the first two measurements, both methods gave good predictive values. For the measurements 3 to 5 however, the values that were predicted with the PLS were much lower than the measured ones. The results of the PCA/PCR’s predicted values lay also close to the measured ones for these three measurements, which can also be seen in the R2pred of 0.96.
4. Summary, Conclusions and Outlook
The use of the multi-spectral imaging system presents many opportunities for its use in drying technology. As compared to RGB systems, the spectral estimation performance is superior [
16]. While 3-channel systems can generally be used to predict device independent color coordinates (e.g., in CIE-
L*a*b* color space) up to a certain degree of accuracy, such systems typically perform low in predicting spectral reflectance curves. This is done pixel by pixel and not averaged over the entire measured area, as it is done when using point spectrometers. Since the region of interest can be freely adapted to any shape, even highly structured surfaces can be measured and analyzed without losing information. This is important, for example, for fruits with seeds that do not change their spectral properties during drying, such as kiwis, or when shadows are cast, as it happens in pineapple drying. Different areas of the pictures can be easily used or not used for analysis.
Further, spectral reflectances can be computed from multi-spectral image data and indications for links between those spectra and different chemical criteria were found. Through the help of PLS and PCA/PCR it was possible to model these links and to use this information for the prediction of the chemical criteria. Very good results were obtained for the prediction of the product moisture content xwb, where a prediction of moisture contents with a RSMEpred of 0.05 and a R2pred of 0.96 was achieved when using PLS for modelling and prediction. For the total soluble solids TSS in the rehydration water the use of PCA/PCR showed the highest R2pred with 0.96 and for the TSS in the rehydrated products the prediction with PLS gave an R2pred of 0.93. For the pH values, no correlation between spectral reflectance and measured values could be found when using PLS and PCA/PCR.
A further advantage of the system developed in this work is that, unlike in hyperspectral imaging techniques, the product does not have to move during measurement. This makes it possible to examine exactly the same samples at all times and obtain very accurate test results. In addition, a rigid system is more stable because there are no wear parts or moving parts. Since the amount of data produced is also much less than with hyperspectral imaging, faster measurements are possible.
In drying technology, the chance to predict information from a small amount of relevant image data can be used to stop drying processes as soon as the products reach a desired moisture content. Drying products to an exact final state can save time and energy. Further, products of higher quality can be produced.
The most important advantages of using this combination of the multispectral camera and the prediction of criteria with PLS and PCA/PCR are that the measurements are non-destructive during the process, but it is still possible to look into the product at any given time during the process, while only a few process parameters have to be measured. These measured variables can then be integrated into closed loop control systems. This is a particular advantage compared to hyperspectral imaging, because the simpler a system is, the more error-resistant it is and the more uncomplicated it is to operate. Since in addition, certain regions of interest can be examined by using the software filters, the system developed in this work is suitable for sensitive processes and products where, for example, the process cannot be interfered with, as well as for simple applications.
In rural regions for example, central measuring points can be set up which are used by many small farmers. The stable conditions regarding temperature and air humidity that the measuring system needs could be realized with a modular box system. When small farmers currently dry their harvest themselves, they proceed as they see fit or as it has been handed down. The farmers cannot directly determine the drying quality; they “hope” that the drying has been successful. In the case of poorly dried food, some of it will spoil and pests can spread to the unspoilt products that are stored together with the partly undried food. If, on the other hand, the products are overdried for safety reasons, time and energy will be wasted. By setting up a central measuring station with a multispectral measuring system, farmers can have their dried food analyzed and find out if and how well their products can be stored. This gives them two advantages: firstly, they can exchange information with other farmers who have better drying results. Thus, the ability to dry well is spread quickly and their own drying processes are constantly improved. Secondly, poorly dried products do not have to spoil because it is known that they do not have a good shelf life. Therefore, they can be consumed in time, processed elsewhere or re-dried.
Another application would be in processes in which certain ingredients must not fall below or exceed specified nominal values. During the processing of fruit valuable components might be degraded; the combination of multispectral image analysis and PLS or PCA/PCR developed in this work makes it possible to monitor this degradation and stop drying before their content falls below a specified minimum value. To automate the processes, the system can be integrated into a control loop.
In industrial applications, multispectral or hyperspectral analysis systems are often used for sorting tasks where it is decided whether a product is “good” or “bad”. For example, nuts and their shells can be separated or those analysis systems can be used in grain handling to distinguish between grain and impurities [
21,
22]. These are classification problems, and the products examined are divided into classes. The combination with PLS and PCA/PCR, however, can also be used for regression tasks. This means that the data collected can be used to draw conclusions about the unambiguous state of the product at any time and not just about its category, which allows a much more detailed interpretation of the collected data.
The multispectral camera system developed in this work is therefore an innovation: it combines the advantages of RGB cameras, such as simplicity, robustness, high measurement speed and rather low price, with those of point spectrometers or hyperspectral imaging techniques, which can provide a great amount of information at once. At the same time, it is able to work with a small amount of data due to the only 12 channels, which can be further reduced individually for each application. This makes the system fast, stable and a widely applicable tool in quality assurance in food processing.
The methods of prediction PLS and PCA/PCR have proven to be generally suitable for finding correlations between product properties and spectral data. Further work is necessary however, because no cross-validation was done in this work. No thorough statistical analysis of the chemical data was done and only a small amount of data was used. It was also found that there could be other prediction methods that lead to better correlations and a better interpretation of data, especially for higher moisture contents or chemical criteria. The use of the machine learning algorithms like decision trees is now being investigated.
Further research could also be done regarding other changes that happen in the product, like changes in vitamins, or textural changes. Additionally, it could be of interest to have a closer look at changes in the ultra violet or the infrared spectrum and to include information from these wavelengths in the prediction of quality criteria. Overall, the results of this work already offer many possibilities for improving post-harvest processes, increasing product quality and thus reducing post-harvest losses.