Next Article in Journal
Feasibility of Determination of Foodborne Microbe Contamination of Fresh-Cut Shredded Cabbage Using SW-NIR
Previous Article in Journal
Influence of Osmotic Dehydration on Mass Transfer Kinetics and Quality Retention of Ripe Papaya (Carica papaya L) during Drying
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
AgriEngineering
Volume 1
Issue 2
10.3390/agriengineering1020017
agriengineering-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Classification of Soymilk and Tofu with Diffuse Reflection Light Using a Deep Learning Technique

by
Kenta Itakura
1,
Yoshito Saito
2,
Tetsuhito Suzuki
2,
Naoshi Kondo
2 and
Fumiki Hosoi
1,*
1
Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1, Yayoi, Bunkyo-ku, Tokyo 113-8657, Japan
2
Graduate School of Agriculture, Kyoto University, Kitashirakawa-Oiwakecho, Sakyo-ku, Kyoto 606-8502, Japan
*
Author to whom correspondence should be addressed.
AgriEngineering 2019, 1(2), 235-245; https://doi.org/10.3390/agriengineering1020017
Submission received: 2 April 2019 / Revised: 6 May 2019 / Accepted: 13 May 2019 / Published: 15 May 2019
Browse Figures
Versions Notes

Abstract

:
Tofu is an ancient soybean product that is produced by heating soymilk containing a coagulation agent. Owing to its benefits to human health, it has become popular all over the world. An important index that determines the final product’s (tofu’s) quality is firmness. Coagulants such as CaSO4 and MgCl2 affect the firmness. With the increasing demand for tofu, a monitoring methodology that ensures high-quality tofu is needed. In our previous paper, an opportunity to monitor changes in the physical properties of soymilk by studying its optical properties during the coagulation process was implied. To ensure this possibility, whether soymilk and tofu can be discriminated via their optical properties should be examined. In this study, a He–Ne laser (Thorlabs Japan Inc., Tokyo, Japan, 2015) with a wavelength of 633 nm was emitted to soymilk and tofu. The images of the scattered light on their surfaces were discriminated using a type of deep learning technique. As a result, the images were classified with an accuracy of about 99%. We adjusted the network architecture and hyperparameters for the learning, and this contributed to the successful classification. The construction of a network that is specific to our task led to the successful classification result. In addition to this monitoring method of the tofu coagulation process, the classification methodology in this study is worth noting for possible use in many relevant agricultural fields.

Graphical Abstract

1. Introduction

Tofu, also known as soybean curd, is an ancient soybean product that is produced by heating soymilk containing a coagulation agent [1,2,3,4]. Tofu is widely consumed in many Asian countries. Further, Western populations have shown interest in soybean products as a result of the influence of Asian immigrants and the possible benefits of soybeans for human health [5,6,7]. As one of the best sources of plant protein, tofu is also rich in beneficial lipids, vitamins, and minerals as well as other bioactive compounds such as isoflavones. Therefore, tofu reduces the risk of many diseases, including hypertension [4,8].
An important index that determines the final tofu product quality is firmness [9], which is mainly determined by the soymilk coagulation process. Hence, control of the coagulation process is crucial [10]. In this process, coagulants such as CaSO4 and MgCl2 affect the firmness [11]. However, manipulation during the coagulation process is dependent on the intuition and experience of skillful artisans [12]. With the increasing demand for tofu, a monitoring methodology that ensures high-quality tofu is desirable [13].
A monitoring method that can detect the coagulation of soymilk by electrical impedance spectroscopy (EIS) has been developed [14,15,16]. However, the method requires the insertion of two electrodes into the tofu. Thus, this destructive method is unsuitable. Saito et al. [13] emitted a He–Ne laser at a wavelength of 633 nm on tofu, and a significant correlation between the reduced scattering coefficient and the firmness of soymilk and tofu was observed. This offers the opportunity to monitor changes in the physical properties of soymilk via its optical properties during the coagulation process [13]. To ensure this possibility, first, whether soymilk and tofu can be discriminated via diffuse reflection light should be examined. If the discrimination can be done accurately, then the detailed tofu coagulation process can be monitored. For example, it would be possible to evaluate the extent of the coagulation of the soymilk from 1 to 10 non-destructively. The technique leads to a precise control of tofu firmness, resulting in high-quality tofu.
Recently, a convolutional neural network (CNN), which is a deep learning technique, achieved high classification accuracy in image analysis [17,18,19]. In a CNN, relevant contextual features in image categorization problems are automatically discovered, and the use of CNNs is gaining attention [20]. CNNs are being used in relevant agricultural fields such as plant species and disease classification [20,21,22,23], remote sensing [24,25,26], and classification of cattle behavior patterns [27]. Using an image analysis technique to monitor tofu and soymilk may be possible. For example, paying attention to the attenuation of the light from the emitted light would lead to the discrimination of tofu and soymilk due to the difference of the edge of the scattered light. However, to define the difference quantitatively is not easy. Edge detection or the calculation of the standard deviation could be considered as a method for defining the difference, but to build the features from lots of potential images is very tedious and difficult. On the other hand, a CNN conducts the feature extraction automatically, so the use of a CNN is appropriate here in this study.
However, the construction of a new neural network with appropriate architecture and weights is difficult. A technique called transfer learning has recently gained attention. This approach utilizes existing knowledge of some related task or domain in order to increase the learning efficiency of the problem under study by fine-tuning pretrained models [28,29,30]. In a previous study, Itakura and Hosoi [31] classified flower images accurately with transfer learning. In another significant work, Suh et al. [32] utilized transfer learning for the classification of sugar beets and volunteer potatoes under field conditions. Although transfer learning is effective in classification, it is essential to modify the transferred model into a network that is specific to each task; that is, just using an existing network (programming codes) for our task is not sufficient. Additional operations such as customizing the network to the task and the optimization of hyperparameters are needed.
In this study, we constructed a tailor-made deep neural network using a transferred framework by adjusting the network structure and hyperparameters for an accurate classification of soymilk and tofu through their diffusely scattered light images. A He–Ne laser with a wavelength of 633 nm was emitted to soymilk and tofu. Then, the image of the scattered light was taken. Whether the image was from soymilk or tofu was estimated using a pretrained network. After adjusting the network architecture and hyperparameters, the classification was conducted.

2. Materials and Methods

2.1. Sample Preparation

Tofu was made from soymilk (manufactured by Sujahta Meiraku Co. Ltd., Nagoya, Japan, 2016) by adding 58.1 mL of coagulant (CaSO4·1/2H2O) and heating for 20 minutes. Then, the tofu was kept in a refrigerator (4 °C) for more than 2 hours. To diversify the tofu firmness, the coagulation temperature was set from 50 to 80 °C. The tofu was made in a tube with a volume of 80 mL and diameter of 18 mm.

2.2. Experimental Setup

Figure 1a shows the experimental setup. A He–Ne laser with a wavelength of 633 nm (Thorlabs Japan Inc., Tokyo, Japan, 2015) was emitted to samples in a dark room (temperature: 20 °C), and an image was taken by a digital camera (Canon, EOS Kiss X2). Figure 1b and c show the soymilk and tofu images, respectively. The focal (F) number was set to 3.5 and 6.3, and the shutter speed was set to 1/10 s, 1/20 s, 1/40 s, 1/80 s, and 1/160 s. A total of 70 and 148 images of soymilk and tofu were obtained, respectively.

2.3. Training of Classifier via Deep Learning Technique

2.3.1. Data for Classification

The number of images for training and validation during the training and test phases was 200, 40, and 58 (soymilk: 30 and tofu: 28), respectively. In the training process, the weights and biases for the neural network were determined. In the validation process, how the network was adopted to the classification task was predicted. If the adaptation is not enough, more training is necessary. If the network adapts to the task too much, it leads to the over-fitting to the training data, resulting in low accuracy in the test data set. Therefore, the validation set can contribute to a high accuracy. The training process was performed with the images in the training dataset. At the same time, the classification accuracy was calculated during the training process using the validation dataset. As the training process goes on, the classification accuracy with the training images increases, however, the network might be over-fitted to the training data, in which case the classification accuracy with training images is high while the accuracy with other image data sets is low. Thus, a validation dataset was prepared to escape the over-fitting, and the classification with the validation set was calculated. If the classification accuracy with the training dataset increased and the accuracy with the validation set stopped increasing or started decreasing, the training process terminated, preventing the over-fitting of the network, as the network seemed to have started to over-fit the training dataset. Here, the number of training images was 200 as mentioned, and the weights and biases were updated seven times in one epoch as explained in Table 1. The number of maximum epochs was 10, meaning the update of the weights and biases was made a maximum of 70 times. The accuracy assessment with the validation dataset was conducted every three updating processes. The high frequency of the assessment with the validation dataset provided early detection of over-fitting, while leading to a longer time for the training process. Samples were randomly selected, and the training below was conducted five times. Then, the average accuracy was calculated. For soymilk, images that were horizontally flipped [33,34] were added in order to double the number of images. This technique, called data augmentation, is effective in teaching the network the desired invariance and robustness properties when the number of training samples is limited [35,36,37].

2.3.2. Network Architecture

The pretrained network GoogLeNet, which was made by Szegedy et al., was utilized [38,39]. This network introduces a new module called Inception, which concatenates filters of different sizes and dimensions into a single new filter [40]. GoogLeNet is not a series network but a directed acyclic graph network. The network performed at very high accuracy in the image classification of Imagenet, which is an image database [41]. However, GoogLeNet is a network for 1000 classes; accordingly, it cannot be used for the current task, i.e. two-class discrimination. A new structure for the current work was needed. To create a network for the two-class discrimination, the final fully connected softmax and output layers were cut off, and only the remaining network was transferred, as shown in Figure 2a,b. Here, the network structure that was not cut off was exactly the same as GoogLeNet. Then, new layers for two-class discrimination were built and directly connected to the last part of the transferred network, as Figure 2c shows. Then, training was implemented with this customized network. Constructing the CNN from scratch is not needed in this transfer learning method, and it is comparatively easy to conduct the classification with the pre-trained deep neural networks.
This network consisted of 144 layers. As shown in Figure 2b, the weights and bias from the first to the 110th layer were fixed, and the weight and bias values were the same as before the transfer (i.e., learning rate = 0). Then, only the rest of the network was used for the learning, as shown in Figure 2b,c. The learning rate in the 142nd layer was set at 10 times as much as the rest of the layers after the 110th layer. With the method, it is likely that the processing time of the training for fixing the weight and bias values can be reduced. To ensure this, the times when the parameters were fixed and not fixed were compared.

2.3.3. Hyperparameter Settings

The optimizer was stochastic gradient descent (SGD) with momentum [42]. The momentum, weight decay [43], and initial learning rate that directly influenced the result were determined using Bayesian optimization [44,45]. Owing to the optimization, the determination of the parameters could be done automatically and appropriately [46,47,48,49]. Other hyperparameters were determined manually. The parameters used are listed in Table 1. This study defined the parameter of validation frequency. In the learning process, the accuracy of the validation set (as explained in Section 2.3.1.) was calculated every three iterations. Then, if the accuracy did not increase five times in a row, the training process stopped because it arrived at the critical point where the learning does not proceed further. This reduced the training time and is called early stopping [43].

3. Results and Discussion

3.1. Classification Accuracy

The estimation accuracy of soymilk and tofu was 99.0 ± 0.94%, indicating that the classification could be done very accurately. In the coagulation process, molecular interactions between the substances such as water soluble protein, particle protein, lipid sphere, phytin, and salt ions are involved, which results in the three-dimensionally complicated structure of tofu. Then, the structure of tofu and soymilk changes from colloid to card throughout the interaction. However, the interaction was not investigated in detail for the classification of soymilk and tofu in this study. It is said that the change of the optical properties originating from the structural alternation would lead to non-destructive discrimination [13]. Therefore, the classification was tried with the scattering light image before and after the coagulation. The tofu surface is uneven on the microscale, resulting in a scattered reflection when the laser beam is emitted. Then, the scattered light waves interfere with different phases between them. As a result, when images of the scattered light are taken by cameras, images with random contrast are obtained [50]. Thus, a subtle difference between the soymilk and tofu images was observed around the edge of a red circle.
However, generally, accurate classification is difficult when the shapes and colors of the images are similar among the classes to be classified. This study imported a pretrained network that could extract powerful and informative features, and utilized it as a starting point to learn a new task (transfer learning). Then, the network made by the authors was connected to the last layers of the transferred network. This customized network contributed to the high accuracy. This method is especially effective when the amount of data is comparatively less, as in this study [28,51]. Further, a network composed of many filters and layers like the network in this study learns increasingly higher-level features [52], and these factors result in a higher accuracy. Moreover, the learning rate of the last softmax layer (142nd) that directly influences the classification was 10 times larger than other layers, resulting in a high classification accuracy.
In addition to a well-designed network architecture, appropriate image acquisition conditions and optimization of the hyperparameters are essential when determining the effectiveness of transfer learning. In this study, the diffused light was located around the center of the image, and the configuration was not much different throughout the dataset. This unbiased localization of the target contributed to the high accuracy [37]. In addition, the appropriate parameters for the network could be determined based on Bayesian optimization.
It is desired to utilize this transfer learning method in agriculture-related tasks. In addition to GoogLeNet, other frameworks that possess advantages such as high accuracy and smaller network sizes have been introduced, including VGG named after Visual Geometry Group [53], squeeze net [54], resnet [55], and DenseNet-201 [56]. By selecting a good framework that is familiar with each task, more difficult tasks can be solved.

3.2. Monitoring of Accuracy and Weight Change during Training Process

Figure 3 shows the relationship between the number of epochs (horizontal axis), the classification accuracy in each epoch (first vertical axis), and the weight change in the last softmax layer (second vertical axis). The weight change is the average value of the absolute change rate of the weight from the previous epoch. The weight of the last softmax layer, in particular, influences the classification. Thus, the value of the layer is tracked. As indicated by Figure 3, as the learning proceeded and the number of epochs increased, the accuracy also increased. Simultaneously, the weight of the layer became optimized. Then, the change rate became stable. By connecting the transferred network to the customized layers, the training was done appropriately and the weight values were optimized. Therefore, classification with a high accuracy was accomplished.
Observation of the weight value in the layers close to the input layer is also effective. Thus, the occurrence of a gradient vanishing problem can be determined [57,58], and a better network corresponding to the specific task can be created.

3.3. Activation Map

Figure 4 illustrates a part of the image used for the classification. It was considered that each image had an important part that contributed to the prediction. Figure 4a–c was created based on the method proposed by Zhou et al. [59]. The first row (a) shows the raw images of soymilk and tofu. In the second row (b), the parts that were highly utilized for the classification are indicated in white. The first and second rows overlap in the last row. The part that contributes to the classification and one that does not contribute are highlighted in purple and green, respectively. In this figure, the edges in the larger circle in the image were utilized for the classification. The centers of the images were saturated, and the retrieved information was limited. Thus, the observation around the edges and not the center part was reasonable.
Note that the part near the edges in the smaller circle, which is the central saturated circle in tofu, is also required for the classification as shown in Figure 4c. We can see that the edges of the smaller circle are also uneven; the edge line of the saturated part can be informative for the classification. This feature was not recognized by the authors until this figure was created. It is worth noting that a feature that cannot be noticed by tofu manufacturers was found. In addition, only half of the lower or upper side of the image was used for the classification. Further, as the coagulation proceeds, a microstructure forms in the tofu [60,61], resulting in an increase in the diffuse coefficient [13]. As the diffused scattering coefficient increases, the slope of the attenuation curve of the diffusely scattered light intensity increases away from the emitted point [62]. This attenuation might contribute to the classification.

3.4. Reduction in Processing Time

We tuned the weights of the layers after the 110th layer only. The processing time with the weights and biases up to the 110th layer was 414 s (fixed) and 1163 s (not fixed), as shown in Figure 5. In Figure 5, “Freezing” and “Not-freezing” represent the “sped-up version” and normal one, respectively.
If the parameters are fixed, the process of updating the parameters in the layers before the 110th layer is not needed. Then, a backpropagation to the layers was not performed, resulting in a significant decrease in the processing time. Furthermore, owing to the early stopping explained in Section 2.3.3., the processing time can be reduced.
The value of max epochs was 10, but the training was stopped in the middle of the 7th epoch owing to early stopping. The processing times with and without early stopping were 355 and 570 s, respectively, indicating a reduction in the processing time with a high accuracy of 99%. This technique is also effective for halting the training when it is not appropriately conducted owing to non-optimized parameters and network architecture. Moreover, it is also effective in detecting overfitting. If the training loss is decreasing and validation loss is increasing, the training should be stopped because the classifier learns a detail of the training data unrelated to the features of the classes.

4. Conclusions

In this study, a He–Ne laser with a wavelength of 633 nm was emitted to soymilk and tofu. Then, an image of the scattered light was taken. Whether the image was from soymilk or tofu was estimated using a type of deep learning—transfer learning. First, a pretrained network, GoogLeNet was imported. Next, the last three layers of GoogLeNet were cut off and new ones were added. Then, our customized network was built. As a result, the images of the scattered light of soymilk and tofu were classified with an accuracy of about 99%. We adjusted the network architecture and hyperparameters for the learning, and this adjustment contributed to the successful classification. The construction of a network that was specific to our task led to the successful classification result. To date, the coagulation process has been monitored using confocal laser scanning microscope (CLSM), atomic force microscope (AFM), and mathematical modelling [63,64,65,66,67]. In our future work, how the proposed optical method in this study can reflect the changes in the coagulation process in the microscale should be explored. Our final goal is to predict the extent of the coagulation of the soymilk and tofu. If we can reveal that changes in the microscale can be monitored using the present method, the potential for the application of this method and its reliability will become clearer. In addition to this monitoring method of the tofu coagulation process, the classification methodology in this study is worth noting for possible use in many relevant agricultural fields such as disease detection and classification, estimation of best harvest time, and early detection of plant wilting. In our future work, the same experiment should be done with different kinds of soymilk and tofu. Additionally, the feasibility of this classification with other conditions such as a different light environment, cameras, and wavelength of emitted light should be investigated.

Author Contributions

K.I. conducted the analysis with the assistance of Y.S., T.S., and N.K. Y.S. conducted the experiments. This paper was written by K.I. N.K., and F.H. supervised this research.

Funding

This work was supported by ACT-I, Japan Science and Technology Agency.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sakai, N.; Hagiwara, T.; Hui, Y.; Tojo, A. Heat transfer analysis of tofu coagulation process. Nippon Shokuhin Kagaku Kogaku Kaishi 2001, 48, 733–737. [Google Scholar] [CrossRef]
  2. Kawaguchi, T.; Kita, R.; Shinyashiki, N.; Yagihara, S.; Fukuzaki, M. Physical properties of tofu gel probed by water translational/rotational dynamics. Food Hydrocoll. 2018, 77, 474–481. [Google Scholar] [CrossRef]
  3. Maurya, S.B.; Shukla, S.S.; Gour, L. Studies on physical and hunter colour of gamma irradiated tofu (soy paneer). Int. J. Curr. Microbiol. Appl. Sci. 2018, 7, 2008–2018. [Google Scholar] [CrossRef]
  4. Zhang, Q.; Wang, C.; Li, B.; Li, L.; Lin, D.; Chen, H.; Liu, Y.; Li, S.; Qin, W.; Liu, J. Research progress in tofu processing: From raw materials to processing conditions. Crit. Rev. Food Sci. Nutr. 2018, 58, 1448–1467. [Google Scholar] [CrossRef] [PubMed]
  5. Yang, A.; James, A.T. Effects of soybean protein composition and processing conditions on silken tofu properties. J. Sci. Food Agric. 2013, 93, 3065–3071. [Google Scholar] [CrossRef] [PubMed]
  6. Peng, X.; Ren, C.; Guo, S. Particle formation and gelation of soymilk: Effect of heat. Trends Food Sci. Technol. 2016, 54, 138–147. [Google Scholar] [CrossRef]
  7. Kamizake, N.K.K.; Silva, L.C.P.; Prudencio, S.H. Impact of soybean aging conditions on tofu sensory characteristics and acceptance. J. Sci. Food Agric. 2018, 98, 1132–1139. [Google Scholar] [CrossRef]
  8. Onodera, Y.; Ono, T.; Nakasato, K.; Toda, K. Homogeneity and microstructure of tofu depends on 11S/7S globulin ratio in soymilk and coagulant concentration. Food Sci. Technol. Res. 2009, 15, 265–274. [Google Scholar] [CrossRef]
  9. Ito, Y. Easy measurement method of “Tofu” hardness. J. Iwate Ind. Res. Inst. 2007, 14, 44–46. [Google Scholar]
  10. He, P.; Zhou, Z.; Wang, N. Development of a novel fiber optic sensor for on-line monitoring of soymilk coagulation. Appl. Mech. Mater. 2011, 52, 1703–1708. [Google Scholar] [CrossRef]
  11. Mine, Y.; Murakami, K.; Azuma, K.; Yoshihara, S.; Fukunaga, K.; Saeki, T.; Sawano, E. A comparison of various coagulants in tofu-forming properties. Nippon Shokuhin Kogyo Gakkaishi 2005, 52, 114–119. [Google Scholar] [CrossRef]
  12. Ohara, T.; Sakai, T.; Matsuhashi, T. Computerized measurement of minimum coagulant necessary for coagulation of soymilk by electric conductivity meter. Nippon Shokuhin Kogyo Gakkaishi 1992, 39, 406–411. [Google Scholar] [CrossRef]
  13. Saito, Y.; Konagaya, K.; Suzuki, T.; Kondo, N. Determination of optical coefficients of tofu using spatially resolved diffuse reflectance at 633 nm. Eng. Agric. Environ. Food 2018, 11, 38–42. [Google Scholar] [CrossRef]
  14. Li, X.S.; Toyoda, K.; Ihara, I. Characterization of tofu coagulation process by impedance spectroscopy. J. Jpn. Soc. Agric. Mach. 2008, 70, 265–266. [Google Scholar]
  15. Toyoda, K.; Li, X. Monitoring of coagulation process of soymilk by an integrated electrical sensing and control system. In Proceedings of the CIGR XVIIth World Congress, Quebec City, QC, Canada, 13–17 June 2010; pp. 1–10. [Google Scholar]
  16. Li, X.S.; Toyoda, K.; Ihara, I. Coagulation process of soymilk characterized by electrical impedance spectroscopy. J. Food Eng. 2011, 105, 563–568. [Google Scholar] [CrossRef]
  17. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25, Proceedings of the Neural Information Processing Systems 2012, Lake Tahoe, NV, USA, 3–8 December 2012; Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q., Eds.; NIPS: Lake Tahoe, NV, USA, 2012; pp. 1106–1114. [Google Scholar]
  18. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
  19. Kamilaris, A.; Prenafeta-Boldú, F.X. Deep learning in agriculture: A survey. Comput. Electron. Agric. 2018, 147, 70–90. [Google Scholar] [CrossRef]
  20. Dyrmann, M.; Karstoft, H.; Midtiby, H.S. Plant species classification using deep convolutional neural network. Biosyst. Eng. 2016, 151, 72–80. [Google Scholar] [CrossRef]
  21. Barbedo, J.G.A. Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant disease classification. Comput. Electron. Agric. 2018, 153, 46–53. [Google Scholar] [CrossRef]
  22. Barbedo, J.G.A. Factors influencing the use of deep learning for plant disease recognition. Biosyst. Eng. 2018, 172, 84–91. [Google Scholar] [CrossRef]
  23. Sibiya, M.; Mbuyu, S. A computational procedure for the recognition and classification of maize leaf diseases out of healthy leaves using convolutional neural networks. AgriEnging 2019, 1, 9. [Google Scholar] [CrossRef]
  24. Maggiori, E.; Tarabalka, Y.; Charpiat, G.; Alliez, P. Convolutional neural networks for large-scale remote-sensing image classification. IEEE Trans. Geosci. Remote Sens. 2017, 55, 645–657. [Google Scholar] [CrossRef]
  25. Liu, X.; Tian, Y.; Yuan, C.; Zhang, F.; Yang, G. Opium Poppy Detection Using Deep Learning. Remote Sens. 2018, 10, 1886. [Google Scholar] [CrossRef]
  26. Itakura, K.; Hosoi, F. Estimation of tree structural parameters from video frames with removal of blurred images using machine learning. J. Agric. Meteorol. 2018, 74, 154–161. [Google Scholar] [CrossRef]
  27. Peng, Y.; Kondo, N.; Fujiura, T.; Suzuki, T.; Yoshioka, H.; Itoyama, E. Classification of multiple cattle behavior patterns using a recurrent neural network with long short-term memory and inertial measurement units. Comput. Electron. Agric. 2019, 157, 247–253. [Google Scholar] [CrossRef]
  28. Pan, S.; Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 2010, 22, 1345–1359. [Google Scholar] [CrossRef]
  29. Nakayama, H. Image feature extraction and transfer learning using deep convolutional neural networks. IEICE Tech. Rep. 2015, 115, 55–59. [Google Scholar]
  30. Kamishima, T. Deep Learning; Kindaikagakusha: Tokyo, Japan, 2015; p. 177. [Google Scholar]
  31. Itakura, K.; Hosoi, F. Background and foreground segmentation in plant images with active contour model and plant image classification using transfer learning. ECO Eng. 2018, 30, 81–85. [Google Scholar]
  32. Suh, H.K.; Ijsselmuiden, J.; Hofstee, J.W.; Van Henten, E.J. Transfer learning for the classification of sugar beet and volunteer potato under field conditions. Biosyst. Eng. 2018, 174, 50–65. [Google Scholar] [CrossRef]
  33. Saito, K. Deep Learning from Scratch; Oreilly Japan: Tokyo, Japan, 2016; p. 245. [Google Scholar]
  34. Bargoti, S.; Underwood, J. Deep fruit detection in orchards. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May–3 June 2017; pp. 3626–3633. [Google Scholar]
  35. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada, 10–14 September 2015; pp. 234–241. [Google Scholar]
  36. Cui, X.; Goel, V.; Kingsbury, B. Data augmentation for deep neural network acoustic modeling. IEEE/ACM Trans. Audio Speech Lang. Process. 2015, 23, 1469–1477. [Google Scholar]
  37. Itakura, K.; Saito, Y.; Suzuki, T.; Kondo, N.; Hosoi, F. Estimation of citrus maturity with fluorescence spectroscopy using deep learning. Horticulturae 2019, 5, 2. [Google Scholar] [CrossRef]
  38. Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A.A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–10 February 2017. [Google Scholar]
  39. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
  40. Shin, H.C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I. Deep convolutional neural networks for computer aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 2016, 35, 1285–1298. [Google Scholar] [CrossRef] [PubMed]
  41. Deng, L.; Hinton, G.; Kingsbury, B. New types of deep neural network learning for speech recognition and related applications: An overview. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 8599–8603. [Google Scholar]
  42. Qian, N. On the momentum term in gradient descent learning algorithms. Neural Netw. 1999, 12, 145–151. [Google Scholar] [CrossRef]
  43. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press Cambridge: London, UK, 2016. [Google Scholar]
  44. Močkus, J. On Bayesian methods for seeking the extremum. In Proceedings of the Optimization Techniques IFIP Technical Conference; Springer: Berlin/Heidelberg, Germany, 1975; pp. 400–404. [Google Scholar]
  45. Mockus, J.; Vytautas, T.; Antanas, Z. The application of Bayesian methods for seeking the extremum. Towards Glob. Opt. 1978, 2, 117–129. [Google Scholar]
  46. Brochu, E.; Cora, M.; de Freitas, N. A Tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv 2010, arXiv:1012.2599. [Google Scholar]
  47. Snoek, J.; Hugo, L.; Adams, R.P. Practical Bayesian Optimization of Machine Learning Algorithms. In Advances in Neural Information Processing Systems 25, Proceedings of the Neural Information Processing Systems 2012, Lake Tahoe, NV, USA, 3–8 December 2012; Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q., Eds.; NIPS: Lake Tahoe, NV, USA, 2012; pp. 2951–2959. [Google Scholar]
  48. Zhang, Y.; Sohn, K.; Villegas, R.; Pan, G.; Lee, H. Improving object detection with deep convolutional networks via bayesian optimization and structured prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 249–258. [Google Scholar]
  49. Kano, M.; Yoshizaki, R. Operating condition optimization for efficient scale-up of manufacturing process by using Bayesian optimization and transfer learning. J. Soc. Instrum. Control Eng. 2017, 56, 695–698. [Google Scholar]
  50. Yoshida, A.; Adachi, M.; Amano, M. Coagulation monitoring of gel material using reflected speckle pattern. In Proceedings of the JSPE Semestrial Meeting, Utsunomiya, Japan, 2006; pp. 699–700. [Google Scholar]
  51. Oquab, M.; Bottou, L.; Laptev, I.; Sivic, J. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 1717–1724. [Google Scholar]
  52. Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; van der Laak, J.A.W.M.; Van Ginneken, B.; Sánchez, C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88. [Google Scholar] [CrossRef] [PubMed]
  53. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
  54. Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5 mb model size. arXiv 2016, arXiv:1602.07360. [Google Scholar]
  55. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
  56. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
  57. Hochreiter, S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 1998, 6, 107–116. [Google Scholar] [CrossRef]
  58. Pascanu, R.; Mikolov, T.; Bengio, Y. Understanding the exploding gradient problem. arXiv 2012, arXiv:1211.5063. [Google Scholar]
  59. Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 2921–2929. [Google Scholar]
  60. Ono, T. New techniques for stable colloidal solution and mixed solid from study on soymilk and tofu. Food Ind. 2008, 51, 50–56. [Google Scholar]
  61. Ono, T. Formation of soymilk colloid for tofu or drinks. Nippon Shokuhin Kogyo Gakkaishi 2017, 64, 220–225. [Google Scholar] [CrossRef]
  62. Farrell, T.J.; Patterson, M.S.; Wilson, B. A diffusion theory model of spatially resolved, steady-state diffuse reflectance for the noninvasive determination of tissue optical properties in vivo. Med. Phys. 1992, 19, 879–888. [Google Scholar] [CrossRef] [PubMed]
  63. Sow, L.C.; Chong, J.M.N.; Liao, Q.X.; Yang, H. Effects of κ-carrageenan on the structure and rheological properties of fish gelatin. J. Food Eng. 2018, 239, 92–103. [Google Scholar] [CrossRef]
  64. Sow, L.C.; Kong, K.; Yang, H. Structural modification of fish gelatin by the addition of gellan, κ-carrageenan, and salts mimics the critical physicochemical properties of pork gelatin. J. Food Sci. 2018, 83, 1280–1291. [Google Scholar] [CrossRef] [PubMed]
  65. Sow, L.C.; Tan, S.J.; Yang, H. Rheological properties and structure modification in liquid and gel of tilapia skin gelatin by the addition of low acyl gellan. Food Hydrocoll. 2019, 90, 9–18. [Google Scholar] [CrossRef]
  66. Sow, L.C.; Toh, N.Z.Y.; Wong, C.W.; Yang, H. Combination of sodium alginate with tilapia fish gelatin for improved texture properties and nanostructure modification. Food Hydrocoll. 2019, 94, 457–467. [Google Scholar] [CrossRef]
  67. Zhou, Y.; Yang, H. Effects of calcium ion on gel properties and gelation of tilapia (Oreochromis niloticus) protein isolates processed with pH shift method. Food Chem. 2019, 277, 327–335. [Google Scholar] [CrossRef]
Agriengineering 01 00017 g001 550
Figure 1. Experimental setup and images obtained from soymilk and tofu. Notes: Panels (a) Show the experimental set up. The laser beam was emitted to the sample and its diffuse reflection light was taken by the camera. Panels (b) and (c) show diffuse reflection light from soymilk and tofu, respectively.
Figure 1. Experimental setup and images obtained from soymilk and tofu. Notes: Panels (a) Show the experimental set up. The laser beam was emitted to the sample and its diffuse reflection light was taken by the camera. Panels (b) and (c) show diffuse reflection light from soymilk and tofu, respectively.
Agriengineering 01 00017 g001
Agriengineering 01 00017 g002 550
Figure 2. Network architecture. (I) GoogLeNet and its last three layers when cut off; (II) customized network for this study, composed of transferred network from GoogLeNet and new layers for two-class discrimination.
Figure 2. Network architecture. (I) GoogLeNet and its last three layers when cut off; (II) customized network for this study, composed of transferred network from GoogLeNet and new layers for two-class discrimination.
Agriengineering 01 00017 g002
Agriengineering 01 00017 g003 550
Figure 3. Accuracy and weight change at each epoch in training process. Weight change represents mean absolute value of difference of weight when compared to value in last epoch.
Figure 3. Accuracy and weight change at each epoch in training process. Weight change represents mean absolute value of difference of weight when compared to value in last epoch.
Agriengineering 01 00017 g003
Agriengineering 01 00017 g004 550
Figure 4. Class activation map in two-class discrimination. The method for the class activation map was introduced by Zhou et al [59]. The first row (a) shows the raw images of soymilk (I) and tofu (II). In the second row (b), the parts that were highly utilized for the classification are indicated in white. The first and second rows overlap in the last row (c). The part that contributes to the classification and the one that does not contribute are highlighted in purple and green, respectively.
Figure 4. Class activation map in two-class discrimination. The method for the class activation map was introduced by Zhou et al [59]. The first row (a) shows the raw images of soymilk (I) and tofu (II). In the second row (b), the parts that were highly utilized for the classification are indicated in white. The first and second rows overlap in the last row (c). The part that contributes to the classification and the one that does not contribute are highlighted in purple and green, respectively.
Agriengineering 01 00017 g004
Agriengineering 01 00017 g005 550
Figure 5. Processing time for training when freezing and not freezing weights in forward layers. When freezing, weight values in layer before 110th were fixed and not changed in training process.
Figure 5. Processing time for training when freezing and not freezing weights in forward layers. When freezing, weight values in layer before 110th were fixed and not changed in training process.
Agriengineering 01 00017 g005
Table
Table 1. Parameters used for training.
Table 1. Parameters used for training.
ParameterValue or Name
OptimizerStochastic gradient descent
Momentum0.9
Weight decay1.0 × 10−4
Initial learn rate1.0 × 10−4
Max epochs10
Minibatch size30
Validation frequency during learningEvery 3 iterations

Share and Cite

MDPI and ACS Style

Itakura, K.; Saito, Y.; Suzuki, T.; Kondo, N.; Hosoi, F. Classification of Soymilk and Tofu with Diffuse Reflection Light Using a Deep Learning Technique. AgriEngineering 2019, 1, 235-245. https://doi.org/10.3390/agriengineering1020017

AMA Style

Itakura K, Saito Y, Suzuki T, Kondo N, Hosoi F. Classification of Soymilk and Tofu with Diffuse Reflection Light Using a Deep Learning Technique. AgriEngineering. 2019; 1(2):235-245. https://doi.org/10.3390/agriengineering1020017

Chicago/Turabian Style

Itakura, Kenta, Yoshito Saito, Tetsuhito Suzuki, Naoshi Kondo, and Fumiki Hosoi. 2019. "Classification of Soymilk and Tofu with Diffuse Reflection Light Using a Deep Learning Technique" AgriEngineering 1, no. 2: 235-245. https://doi.org/10.3390/agriengineering1020017

Article Metrics

Back to TopTop