High End Quality Measuring in Mango Drying through Multi-Spectral Imaging Systems

: In modern fruit processing technology, non-destructive quality measuring techniques are sought for determining and controlling changes in the optical, structural, and chemical properties of the products. In this context, changes inside the product can be measured during processing. Especially for industrial use, fast, precise, but robust methods are particularly important to obtain high-quality products. In this work, a newly developed multi-spectral imaging system was implemented and adapted for drying processes. Further it was investigated if the system could be used to link changes in the surface spectral reflectance during mango drying with changes in moisture content and contents of chemical components. This was achieved by recovering the spectral reflectance from multi-spectral image data and comparing the spectral changes with changes of the total soluble solids (TSS), pH-value and the relative moisture content x wb of the products. In a first step, the camera was modified to be used in drying, then the changes in the spectra and quality criteria during mango drying were measured. For this, mango slices were dried at air temperatures of 40–80 °C and relative air humidities of 5%–30%. Samples were analyzed and pictures were taken with the multi-spectral imaging system. The quality criteria were then predicted from spectral data. It could be shown that the newly developed multi-spectral imaging system can be used for quality control in fruit drying. There are strong indications as well, that it can be employed for the prediction of chemical quality criteria of mangoes during drying. This way, quality changes can be monitored inline during the process using only one single measuring device.


Introduction
Product quality and efficiency are of great importance in the design of drying processes for exotic fruits or other sensitive biological foodstuffs. This is of great interest, since on the one hand, energy is becoming more and more expensive and the accessibility of energy cannot be guaranteed always and everywhere. On the other hand, exotic fruit demonstrate great health benefits like vitamins, sugars, or acids, which could be destroyed in poor drying processes. For measuring the retention of these however, complicated chemical analyses are necessary. In modern approaches in drying technology, a substitution of these analysis techniques is sought. The use of spectrometers, RGB (Red, Green, Blue) cameras or combinations of both (e.g., hyperspectral imaging) bear chances of being this substitution.
Much research has been done in this field with various products. An overview of the different technologies used in food processing and their advantages as well as disadvantages are given by Zhang, et al. [1]. They describe which measuring techniques are commonly used in which applications and for which products. Further, they explain common methods for image correction and spectral compensation. Gomez et al. used VIS (visible)-and NIR (near infrared) spectroscopy techniques to measure the acidity, strength and amount of total soluble solids in mandarins [2]. Martinsen and Schaare analyzed the distribution of soluble solids in kiwifruit using NIR spectroscopy [3] and Wang et al. worked on predicting the strength and soluble solids content of pears using VIS and NIR spectroscopy [4]. Vejarano et al. provide an overview of various analytical and optical methods for the detection of biological contaminants in food and then focus on the use of hyperspectral imaging [5]. They highlight different contaminants that are critical for food safety and show how hyperspectral imaging can be used for early detection of contamination. Detailed work on the use of spectral and hyperspectral imaging in drying has been performed on tea [6], carrots [7], soy beans [8], and apples [9][10][11][12]. The use of hyperspectral imaging in the drying of mangoes has been studied by Pu and Sun, who investigated the changes in moisture content when drying mango slices in a microwave [13]. The results of all their work showed that it is possible to predict the intrinsic properties of fruits using spectroscopic methods.
It was also shown that each method has disadvantages that have to be dealt with when wanting to use the techniques for inline measuring in food processing or drying processes: Spectrometers deliver big amounts of data that have to be saved, processed, analyzed, and interpreted. At the same time, they provide information only point by point and some fruits require spatial resolution. Point spectrometers also have the disadvantage of a fixed measuring aperture in which the measured value is averaged. With textured surfaces such as fruits, this averaging process leads to a systematic error through which the spectral texture information is lost. Pure RGB cameras, which contain spatial information, hold too little spectral information and do not cover wavelengths in the ultra violet or infrared regions. Hyperspectral cameras, on the other hand, are expensive and very complicated to operate. Further, hyperspectral imaging systems often need moving products, which cannot always be realized in manufacturing processes.
In this work, the use of a novel multi-spectral snapshot imaging system is tested in drying processes. The system acquires simultaneously 12 distinct spectral image channels. Based on a predefined calibration, spectral surface reflectance data can be estimated on a pixel-by-pixel basis from the image data. Spectral reflectance is a physical surface property describing how an incident spectrum of light is reflected from a surface. Hence, this is a property that is independent of the scene lighting, which is a limitation of conventional color measurement with RGB cameras. Using the proposed multi-spectral imaging workflow, device independent spectral color measurement is possible.
With the use of this multi-spectral imaging system and the spectra that are calculated out of the images, links to changes in the chemical components can be predicted.

Dryer and Drying Process
The drying experiments were conducted in a single layer through-flow dryer, whose function is shown in Figure 1, and a Binder KBF 720 (Binder GmbH, Tuttlingen, Germany) drying chamber. Preliminary experiments guaranteed that the products from both dryers had the same drying kinetics. During those tests, changes in product mass, moisture content and surface temperature as well as the total drying times and final moisture content of the products were analyzed [14]. Therefore, samples with the same moisture content xwb could be compared despite being dried in different dryers. The samples of the single layer dryer were used for analysis of the spectra and the relative moisture content xwb, whereas the samples from the drying chamber were used for analysis of the relative moisture content xwb, total soluble solids (TSS) and pH-value. In this single layer dryer, a constant airflow with the air velocity vair is achieved with the help of a fan (1). The air is blown through a water bath (2) with controlled dew point temperatures ϑair. ϑDP is kept stable through coordinated heating (2b) and cooling (3) of the water bath. After completing its path through the water bath, the air reaches a humidity of 100%, which is checked with a psyochrometer (4). The air is then heated up to air temperatures ϑair of up to 120 °C (5), leading to different relative humidities RH. Afterwards, the air is vertically blown through a sieve with one layer of product (7) into the lower drying chamber. The product mass is constantly measured with a scale. At the point when the air passes the product, it has velocities between 0.2 m/s and 3 m/s, depending on the settings at the fan. For all experiments in this work, the air velocity was set to 1 m/s. The lighting of the product is done using a Corona II lighting system from Chromasens GmbH [15] in a 45% angle with respect to the surface normal (9). This LED based line-light was used offfocus to achieve a spatially homogeneous lighting over the entire field-of-view of the camera. Active temperature control assures the stability of illumination with respect to its spectral properties over time. The measuring chamber (8b) is air-cooled with a set of fans and divided from the drying chamber (8a) by a glass plate to keep the imaging systems from heating up. Multi-spectral images, an RGB image, and the spectroradiometric reference measurements are acquired at ~0° with respect to the surface normal from the top of the chamber (10).
The drying chamber is basically an insulated cabinet with controlled air temperature and air humidities. In a simulation done previously, the airflow as well as the temperature and moisture distribution in the drying chamber was checked to make sure that the flow around all samples was the same [16]. This showed that the airflow, and therefore the air temperature, air humidity, and air velocity distribution, in the drying chamber was mainly independent from the horizontal position of the samples in the chamber. The few positions where the airflow was much higher or lower could be left empty when distributing the samples in the chamber. To prevent an influence of the vertical position, all samples were placed on the same tray.
Five runs with mango samples were planned and carried out. The experimental design was randomized before being used, and this is shown in Table 1. This means that the order of the experiments was set randomly. In an un-randomized experimental design, all experiments with the same setting, e.g., the same air temperature, would be done first, then all with the next setting. This might lead to a wrong interpretation of data, when changes that occurred over to time are interpreted as changes due to a certain setting of parameters.  1  80  5  120  79  2  42  21  360  194  3  40  13  300  121  4  59  5  210  112  5  80  20 120 91 * with "Drying time" describing the drying time until the final moisture content was reached, "DC" being the drying chamber and "SLD" the single layer dryer.
Each experiment was done simultaneously in both dryers, with samples from the same actual product. The parameters that were changed were air temperature and air humidity, while the air velocity was kept constant. Since the main goals of this work were to find out if the measuring system was applicable to drying and if changes in product quality could be predicted, it was decided to only use two different parameters in the experimental design. In preliminary experiments, air humidity and air temperature were shown to have a greater influence on changes in properties than the air velocity. Further, it was easier to keep these two parameters at the same level in both dryers than the air velocity. Minimum and maximum air temperatures for the complete experimental design were 40 °C and 80 °C and minimum and maximum relative air humidities were 5% and 30%. These values were chosen on the one hand due to the technical limitations of the dryers and the measuring equipment, on the other hand on logical reasons: too high air humidities would lead to very high final moisture contents of the products. The goal was to dry the products up to a final product moisture content of less than 15% [17]. To be sure that this goal was reached, samples were slightly overdried. After calculating the point of time where the final moisture content was first reached, only data up to this point was used. For high air humidities at low air temperatures, the final moisture content was not reached, because the cool, wet air could not take any more moisture from the samples. In those cases, slightly higher final moisture contents of about 20% were achieved (experiments 3 and 14). For the prediction of chemical criteria based on the spectral measurements, these experiments could still be used.

Products and Sample Preparation
Kent mangoes from the Dominican Republic were used. The fruits were purchased at a local market, which guaranteed a stable quality of the fresh products. This was very important for being able to measure the impact of the drying parameters on quality rather than the variations in the products themselves.
Two to three mangoes were used per run. They were cut into slices with a thickness of 4 mm with a Graeff C20EU slicing machine. Out of these, one big slice of dout = 70 mm and several small slices of dout = 20 mm were punched by using round punches. The big slice was used for color and spectral analysis, the small ones for the determination of the drying kinetics, the moisture contents and for chemical analysis. The rest of the mango was used for the determination of the total soluble solids (TSS) in %, pH-value and the initial moisture content of the fresh samples. The slices were then placed on black sieves in the preheated dryers.

Analysis of Color and Spectra
In a set of preliminary experiments, the quality of the new multi-spectral imaging system in drying processes was analyzed. The general set-up is described in the chapter "Dryer and Drying Process"; more detailed information is given in Figure 2. Reference measurements were acquired with the spectroradiometer (OceanOptics USB 2000+ with a 600 µm glass fiber OceanOptics QP600-2-UV and a 50 mm lens by Ocean Optics, Ostfildern, Germany) and compared with estimated spectral reflectance measurements obtained from the 3-channel RGB CMOS camera (IDS uEye CP GV-5260 CP by IDS, Obersulm, Deutschland) as well as the 12-channel multi-spectral imaging system. For the spectroradiometric measurements, the measurement spot is in the center of the sample and has a diameter of approximately 10 mm (also shown in Figure 3: dotted circle in picture 0 min). The wavelengths of the multi-spectral imaging system that are analyzed in this work lie between 380 nm and 730 nm. The field-of-view of the camera covers a larger area of 220 mm by 220 mm. For the spectral measurements derived from the multi-spectral image, the measurement spot was selected from the images to correspond to the spectroradiometer.
Since the multi-spectral imaging system achieved a very low distance between colors ΔE2000 of 0.21 and root mean square error (RSME) of 0.011 when compared to a given color sample in the application-specific spectral calibration, in this paper only data from the multi-spectral imaging system was used. Further information about the multi-spectral imaging system can be found in the work of Zirkler et al. [18]. Figure 2 shows the set-up of the devices in the drying chamber.  During the drying process, images of the samples with a black background for color and spectral analysis were taken using the multi-spectral camera. Pictures were taken every 3:20 min. The data were collected and analyzed with the help of a program developed by Chromasens GmbH and the Institute of Applied Thermo-and Fluid Dynamics at the University of Applied Sciences, Konstanz.
The images of the multi-spectral camera were then used for spectral reflectance estimation through which it was possible to calculate the pixel-wise spectral reflectance as follows.
The estimation is based on a pre-defined color and geometric calibration of the system. All image data are pre-processed by a two-point radiometric calibration of the system to correct dark current and spatial inhomogeneity of the illumination. Geometric calibration is the process of mapping image data from all 12 channels such that corresponding object points are aligned spatially in a data cube. The spectral color calibration is subsequently obtained from image data of a proprietary color chart by Chromasens GmbH and corresponding spectral measurements acquired by a reference spectrodensitometer of type FD-7 by Konica Minolta (Marunouchi, Japan) in the range of 380-730 nm. The color chart contains 660 uniform color patches that sample a large color gamut. The 12channel camera response data corresponding to each color patch and the spectral reference measurements are used to fit a linear system of equations using regression. This system obtained from training data can then be used to estimate spectral reflectances from multi-channel image data that are not contained in the training set [18].

Chemical Analysis
For chemical analysis, 12 samples were taken from the drying chamber every 30 min. Two slices were weighed and used for the determination of the moisture content xwb, the rest were frozen and kept at −17 °C until analyzed further. Chemical analysis was done for the total soluble solids (TSS), the pH-value and the product's moisture content xwb. All dried and frozen samples were rehydrated for TSS and pH measurement. For this, samples were placed in water at room temperature for 3 h. Then the water used for rehydration as well as the rehydrated products were analyzed. This was done because some sugars might dissolve into the rehydration water and changes in TSS in the water could therefore be used to describe changes in the products that occurred during drying more thoroughly [19].
For the TSS measurement, the °Brix and therefore the % of TSS were measured with a Krüss DR101-60 manual refractometer. A few drops of the fruit juice or the rehydration water were placed on the refractometer and after waiting for about 2 min for letting the temperatures assimilate, the TSS values in % were measured. The pH-values of the fruits' juice and the rehydration water were measured using a WTW ph3110 pH-meter (Xylem Analytics Germany Sales GmbH & Co. KG WTW, Weilheim, Germany).
The moisture content of the samples xwb,i at any given point of time i in % was calculated using Formula (1) for both driers. The dry matter mdm of the samples was determined by drying the samples for another 48 h at 70 °C in a drying chamber. The weight at the end of the 48 h was used as the dry matter mdm [20].
where mi is the mass of the sample at the point of time i in g and mdm is the dry matter of the sample in g. xwb was always determined separately for the single layer dryer and the drying chamber and samples with the same xwb values were then compared.

Prediction of Quality Criteria
To analyze if quality criteria could generally be predicted out of the spectral data partial leastsquares regression (PLS) and principal component analysis (PCA) with subsequent Principal Component Regression (PCR) were used. The link between the spectral data and the chemical criteria was modelled for one set of data. Then, the chemical criteria were predicted from spectral data of another set of data. For this, the measured values at all points of time from run 1-4 (160 values for xwb, 36 values for TSS and pH) were used as training data for the modelling; the ones from run 5 (28 values xwb, 5 values for TSS and pH) were used as test data for the prediction. For the modelling and prediction of xwb data from the single layer, the dryer was used; for the modelling and prediction of TSS and pH-value data from, the drying chamber at the same xwb were used. In this first check, no cross-validation was done. To analyze the quality of the predicted values, R 2 pred of the test set as well as RSMEpred of the xwb data were calculated.

Results and Discussion
The overall goals of the work were to find out if the multi-spectral imaging system could be used for quality control in drying and to acquire information about the relationship between the estimated surface reflectance spectra and the measured chemical values.

Multi-Spectral Imaging System
Using the multi-spectral image data, pixel-wise surface reflectance data were estimated and averaged over an area that corresponds to the spectroradiometer measurement spot. Figure 3 shows the RGB pictures of mangoes and their changes over time (0 min up to 390 min) from the preliminary experiments for reference. It can be seen that the color changes slightly but uniformly over the drying time. This behavior can also be found in the changes in the calculated spectral reflectance (Figure 4). Details about the imaging systems can be found in the work of Zirkler, et al. [16]. The results for the production of quality criteria are shown in the following tables and graphs.

Moisture Content Prediction
The data of the multi-spectral imaging system were then used to investigate if a prediction of the samples' quality criteria based on the spectral information was possible. When using PLS and PCA/PCR highest R 2 pred were obtained when predicting the relative moisture content xwb of the samples. As can be seen from the data in Table 2 and Figure 5, the measured and the predicted results have a low RSMEpred and high R 2 pred of the prediction of the test data.
With PLS the predicted values are very close to the measured ones after the first quarter of the prediction and have a slight "under-prediction" in the second quarter, where the predicted values are lower than the measured values. For the PLS modelling, the difference between prediction and measurement gets smaller again in the last two quarters, whereas for the PCA/PCR prediction, the difference gets bigger after the first quarter. This is also shown in Figure 5.  The predicted values of the PLS and of the PCA/PCR both show a similar behavior regarding the curves' shapes. The PCA/PCR prediction also shows lower values than the ones actually measured. For the prediction of the moisture content, the PLS gives better results than the PCA/PCR. When using the PLS for the prediction of the moisture content xwb out of the spectra measured, the results show a good fit to the measured xwb. This can be seen in the graph as well as in a high R 2 and a low RSME. Especially for low moisture contents, which are of great interest when wanting to know when a product is dry, the values of prediction and measurement show the same tendency.
The results obtained therefore showed that it is possible to use the multi-spectral imaging system for the prediction of the moisture content xwb. The problems of the high predicted values in the first quarter and the low values of the PCA/PCR prediction have to be further analyzed. Reasons for this might lay in the use of PLS and PCA/PCR for prediction, when other models might be better for this problem. Further, it might make sense to not predict one complete experiment out of the data of four other full experiments, but to mix all measured values and then use random 80% of the data for the prediction of the other 20%.

Prediction of Other Chemical Criteria
For the prediction of the TSS and the pH-values, less data were acquired (only one measurement every 30 min instead of one every 3 min). Therefore, the amount of data for the modelling and the prediction was much lower. The prediction of the pH-values with the use of the acquired data was not possible since no relation of pH-value and spectral data was found on basis of the acquired data when using PLS and PCA/PCR. Therefore, no further prediction model was developed for these parameters.
The results for the prediction of TSS are shown in Table 3 (prediction for the rehydrated product), Table 4 (prediction for the rehydration water) and Figure 6 (prediction of rehydration water and rehydrated product).
For the TSS of the rehydrated products, the PLS prediction showed much higher R² and therefore more similar values to the measurement than the PCA/PCR. Especially for the first two measurements, the predicted value and the measured one were almost the same. For the other three, the prediction was a bit higher than the measured values, but with an overall R 2 pred of 0.93, the prediction is still good. The prediction with the PCA/PCR gave much lower values for the first three measurements than the measured values and gives a low R 2 pred.  When predicting the TSS of the rehydration water, the PCA/PCR showed a slightly higher R² and therefore more similar values to the measurement than the PLS. For the first two measurements, both methods gave good predictive values. For the measurements 3 to 5 however, the values that were predicted with the PLS were much lower than the measured ones. The results of the PCA/PCR's predicted values lay also close to the measured ones for these three measurements, which can also be seen in the R 2 pred of 0.96.

Summary, Conclusions and Outlook
The use of the multi-spectral imaging system presents many opportunities for its use in drying technology. As compared to RGB systems, the spectral estimation performance is superior [16]. While 3-channel systems can generally be used to predict device independent color coordinates (e.g., in CIE-L*a*b* color space) up to a certain degree of accuracy, such systems typically perform low in predicting spectral reflectance curves. This is done pixel by pixel and not averaged over the entire measured area, as it is done when using point spectrometers. Since the region of interest can be freely adapted to any shape, even highly structured surfaces can be measured and analyzed without losing information. This is important, for example, for fruits with seeds that do not change their spectral properties during drying, such as kiwis, or when shadows are cast, as it happens in pineapple drying. Different areas of the pictures can be easily used or not used for analysis.
Further, spectral reflectances can be computed from multi-spectral image data and indications for links between those spectra and different chemical criteria were found. Through the help of PLS and PCA/PCR it was possible to model these links and to use this information for the prediction of the chemical criteria. Very good results were obtained for the prediction of the product moisture content xwb, where a prediction of moisture contents with a RSMEpred of 0.05 and a R 2 pred of 0.96 was achieved when using PLS for modelling and prediction. For the total soluble solids TSS in the rehydration water the use of PCA/PCR showed the highest R 2 pred with 0.96 and for the TSS in the rehydrated products the prediction with PLS gave an R 2 pred of 0.93. For the pH values, no correlation between spectral reflectance and measured values could be found when using PLS and PCA/PCR.
A further advantage of the system developed in this work is that, unlike in hyperspectral imaging techniques, the product does not have to move during measurement. This makes it possible to examine exactly the same samples at all times and obtain very accurate test results. In addition, a rigid system is more stable because there are no wear parts or moving parts. Since the amount of data produced is also much less than with hyperspectral imaging, faster measurements are possible.
In drying technology, the chance to predict information from a small amount of relevant image data can be used to stop drying processes as soon as the products reach a desired moisture content. Drying products to an exact final state can save time and energy. Further, products of higher quality can be produced.
The most important advantages of using this combination of the multispectral camera and the prediction of criteria with PLS and PCA/PCR are that the measurements are non-destructive during the process, but it is still possible to look into the product at any given time during the process, while only a few process parameters have to be measured. These measured variables can then be integrated into closed loop control systems. This is a particular advantage compared to hyperspectral imaging, because the simpler a system is, the more error-resistant it is and the more uncomplicated it is to operate. Since in addition, certain regions of interest can be examined by using the software filters, the system developed in this work is suitable for sensitive processes and products where, for example, the process cannot be interfered with, as well as for simple applications.
In rural regions for example, central measuring points can be set up which are used by many small farmers. The stable conditions regarding temperature and air humidity that the measuring system needs could be realized with a modular box system. When small farmers currently dry their harvest themselves, they proceed as they see fit or as it has been handed down. The farmers cannot directly determine the drying quality; they "hope" that the drying has been successful. In the case of poorly dried food, some of it will spoil and pests can spread to the unspoilt products that are stored together with the partly undried food. If, on the other hand, the products are overdried for safety reasons, time and energy will be wasted. By setting up a central measuring station with a multispectral measuring system, farmers can have their dried food analyzed and find out if and how well their products can be stored. This gives them two advantages: firstly, they can exchange information with other farmers who have better drying results. Thus, the ability to dry well is spread quickly and their own drying processes are constantly improved. Secondly, poorly dried products do not have to spoil because it is known that they do not have a good shelf life. Therefore, they can be consumed in time, processed elsewhere or re-dried.
Another application would be in processes in which certain ingredients must not fall below or exceed specified nominal values. During the processing of fruit valuable components might be degraded; the combination of multispectral image analysis and PLS or PCA/PCR developed in this work makes it possible to monitor this degradation and stop drying before their content falls below a specified minimum value. To automate the processes, the system can be integrated into a control loop.
In industrial applications, multispectral or hyperspectral analysis systems are often used for sorting tasks where it is decided whether a product is "good" or "bad". For example, nuts and their shells can be separated or those analysis systems can be used in grain handling to distinguish between grain and impurities [21,22]. These are classification problems, and the products examined are divided into classes. The combination with PLS and PCA/PCR, however, can also be used for regression tasks. This means that the data collected can be used to draw conclusions about the unambiguous state of the product at any time and not just about its category, which allows a much more detailed interpretation of the collected data. The multispectral camera system developed in this work is therefore an innovation: it combines the advantages of RGB cameras, such as simplicity, robustness, high measurement speed and rather low price, with those of point spectrometers or hyperspectral imaging techniques, which can provide a great amount of information at once. At the same time, it is able to work with a small amount of data due to the only 12 channels, which can be further reduced individually for each application. This makes the system fast, stable and a widely applicable tool in quality assurance in food processing.
The methods of prediction PLS and PCA/PCR have proven to be generally suitable for finding correlations between product properties and spectral data. Further work is necessary however, because no cross-validation was done in this work. No thorough statistical analysis of the chemical data was done and only a small amount of data was used. It was also found that there could be other prediction methods that lead to better correlations and a better interpretation of data, especially for higher moisture contents or chemical criteria. The use of the machine learning algorithms like decision trees is now being investigated.
Further research could also be done regarding other changes that happen in the product, like changes in vitamins, or textural changes. Additionally, it could be of interest to have a closer look at changes in the ultra violet or the infrared spectrum and to include information from these wavelengths in the prediction of quality criteria. Overall, the results of this work already offer many possibilities for improving post-harvest processes, increasing product quality and thus reducing post-harvest losses.