# Determination of the Optimal State of Dough Fermentation in Bread Production by Using Optical Sensors and Deep Learning

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. State of the Art

#### 2.1. Fermentation Monitoring

#### 2.2. Robust Object Recognition

## 3. Materials and Methods

#### 3.1. Dough Volume Monitoring

#### 3.2. Data Processing

#### 3.3. Preparation of the Dough Pieces and Fermentation Process

## 4. Results

^{3}for the three dough pieces during the fermentation period of 90 min, which resembles our measurement of the static test objects. Furthermore, the relative volume difference with regards to the dough volume decreased with increasing fermentation time, which resulted in more accurate volume estimation when the process approaches the optimal state. The method for estimating the volume of watermelons by means of image processing proposed in [33] achieves a deviation of approximately 7.7 % on average. Approximately the same deviation (7.8 %) is stated by the authors in [34] while estimating the volume of kiwi fruits using image processing techniques. Compared to those, our method performs slightly weaker at the beginning with a small object volume, but with increasing fermentation time we obtain more accurate volume estimations because the relative volume error decreases.

## 5. Conclusions and Future Work

## 6. Patents

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Zentralverband des Deutschen Bäckerhandwerks e. V. Entwicklung Beschäftigte & Betriebe: Beschäftigtenzahlen im Bäckerhandwerk. Available online: https://www.baeckerhandwerk.de/baeckerhandwerk/zahlen-fakten/entwicklung-beschaeftigte-betriebe/ (accessed on 13 December 2018).
- HuffPost Deutschland. Konkurrenz für McDonald’s und Burger King: Das ist der überraschende Spitzenreiter bei Schnell-Restaurants. Available online: https://www.huffingtonpost.de/2015/05/04/schnell-restaurant-spitzenreiter_n_7205306.html (accessed on 15 March 2019).
- Wing, D.; Scott, A. The Bread Builders: Hearth Loaves and Masonry Ovens; Chelsea Green Publishing: White River Junction, VT, USA, 1999; ISBN 1890132055. [Google Scholar]
- Ivorra, E.; Amat, S.V.; Sánchez, A.J.; Barat, J.M.; Grau, R. Continuous monitoring of bread dough fermentation using a 3D vision Structured Light technique. J. Food Eng.
**2014**, 130, 8–13. [Google Scholar] [CrossRef] - Benz, F. Backwarenherstellung. Fachkunde für Bäcker, 6; Dr Schroedel: Hannover, Germany, 1986; ISBN 350791414X. [Google Scholar]
- Priyadharshini, K.; Akila, R. A Survey on Computer Vision Technology for Food Quality Evaluation. Int. J. Innov. Res. Comput. Commun. Eng.
**2016**, 4, 14860–14865. [Google Scholar] - Lösche, K.; Reichenbach, A. Aerosole & Bio-Additive in der Bäckerei: Innovative Wege zur Verbesserung von Qualität, Haltbarkeit und Energieeffizienz. In Bäckereitechnologie: Forschung und Innovationen; f2m Food multimedia Gmbh: Hamburg, Germany, 2015; pp. 50–59. [Google Scholar]
- BIBA—Bremer Institut für Produktion und Logistik GmbH, ttz Bremerhaven. Vorrichtung und Verfahren zur Prozessüberwachung mehrerer Teiglinge in einer Prozesskammer (Pending), 2018; 102018124378.2.
- Elmehdi, H.M.; Page, J.H.; Scanlon, M.G. Monitoring Dough Fermentation Using Acoustic Waves. Food Bioprod. Process.
**2003**, 81, 217–223. [Google Scholar] [CrossRef][Green Version] - Skaf, A.; Nassar, G.; Lefebvre, F.; Nongaillard, B. A new acoustic technique to monitor bread dough during the fermentation phase. J. Food Eng.
**2009**, 93, 365–378. [Google Scholar] [CrossRef] - Bajd, F.; Serša, I. Continuous monitoring of dough fermentation and bread baking by magnetic resonance microscopy. Magn. Reson. Imaging
**2011**, 29, 434–442. [Google Scholar] [CrossRef] [PubMed] - Pour-Damanab, A.S.; Jafary, A.; Rafiee, S. Monitoring the dynamic density of dough during fermentation using digital imaging method. J. Food Eng.
**2011**, 107, 8–13. [Google Scholar] [CrossRef] - Scholz-Reiter, B.; Thamer, H. Multi-View Sensor Fusion of Synthetic ToF Images for Object Recognition of Universal Logistic Goods; GfaI: Berlin, Germany, 2011. [Google Scholar]
- Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- He, K.; Gkioxari, G.; Dollar, P.; Girshick, R. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision, ICCV, Venice, Italy, 22–29 October 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 2980–2988, ISBN 978-1-5386-1032-9. [Google Scholar]
- Fang, X.S. The extruded generalized cylinder: A deformable model for object recovery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 21–23 June 1994; IEEE: Piscataway, NJ, USA, 1994. [Google Scholar]
- Biederman, I. Recognition-by-components: A theory of human image understanding. Psychol. Rev.
**1987**, 94, 115–147. [Google Scholar] [CrossRef] [PubMed] - Barr, A.H. Superquadrics and angle-preserving transformations. IEEE Comput. Graph. Appl.
**1981**, 1, 11–23. [Google Scholar] [CrossRef] - Solina, F.; Bajcsy, R. Recovery of parametric models from range images: The case for superquadrics with global deformations. IEEE Trans. Pattern Anal. Mach. Intell.
**1990**, 12, 131–147. [Google Scholar] [CrossRef] - Thamer, H.; Scholz-Reiter, B. 3D-Objekterkennung von heterogenen Stückgütern—Flexible Automatisierung basierend auf 3D-Bildverarbeitung. Ind. Manag.
**2014**, 31, 35–38. [Google Scholar] - West, J.; Ventura, D.; Warnick, S. Spring Research Presentation: A Theoretical Foundation for Inductive Transfer; Brigham Young University, College of Physical and Mathematical Sciences: Provo, UT, USA, 2007; Volume 1. [Google Scholar]
- Lin, T.-Y.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C.L.; Dollár, P. Microsoft COCO: Common Objects in Context. 2014. Available online: http://arxiv.org/pdf/1405.0312v3 (accessed on 11 October 2019).
- Perumal, L. Quaternion and its application in rotation using sets of regions. Int. J. Eng. Technol. Innov.
**2016**, 1, 35–52. [Google Scholar] - Whaite, P.; Ferrie, F.P. From uncertainty to visual exploration. In Proceedings of the Third International Conference on Computer Vision, Osaka, Japan, 4–7 December 1990; Kak, A.C., Tsuji, S., Eklundh, J.-O., Eds.; IEEE: Piscataway, NJ, USA, 1990; pp. 690–697, ISBN 0-8186-2057-9. [Google Scholar]
- Bardinet, E.; Ayache, N.; Cohen, L.D. Fitting of iso-surfaces using superquadrics and free-form deformations. In Proceedings of the IEEE Workshop on Biomedical Image Analysis, Seattle, WA, USA, 24–25 June 1994; IEEE Computer Society Press: Washington, D.C., USA, 1994; pp. 184–193, ISBN 0-8186-5802-9. [Google Scholar]
- Duncan, K.; Sarkar, S.; Alqasemi, R.; Dubey, R. Multi-scale superquadric fitting for efficient shape and pose recovery of unknown objects. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, 6–10 May 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 4238–4243, ISBN 978-1-4673-5643-5. [Google Scholar]
- Biegelbauer, G.; Vincze, M. Efficient 3D Object Detection by Fitting Superquadrics to Range Image Data for Robot’s Object Manipulation. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Rome, Italy, 10–14 April 2007; IEEE: Piscataway, NJ, USA, 2006; pp. 1086–1091, ISBN 1-4244-0602-1. [Google Scholar]
- Wächter, A.; Biegler, L.T. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program.
**2006**, 106, 25–57. [Google Scholar] [CrossRef] - Jaklič, A.; Leonardis, A.; Solina, F. Superquadrics and their geometric properties. In Segmentation and Recovery of Superquadrics; Springer: Berlin/Heidelberg, Germany, 2000; pp. 13–39. [Google Scholar]
- Sinha, N. Handbook of Food Products Manufacturing, 2 Volume Set; John Wiley & Sons: Hoboken, NJ, USA, 2007; ISBN 0470113545. [Google Scholar]
- Kerker, M. The Scattering of Light and other Electromagnetic Radiation: Physical Chemistry: A Series of Monographs; Academic Press: Cambridge, MA, USA, 2013; ISBN 1483191745. [Google Scholar]
- Bhattacharya, M.; Hanna, M.A. Viscosity modelling of dough in extrusion. Int. J. Food Sci. Technol.
**1986**, 21, 167–174. [Google Scholar] [CrossRef] - Koc, A. Determination of watermelon volume using ellipsoid approximation and image processing. In Postharvest Biology and Technology; Elsevier: Amsterdam, The Netherlands, 2007; pp. 366–371. [Google Scholar]
- Rashidi, M.; Gholami, M. Determination of kiwifruit volume using ellipsoid approximation and image-processing methods. Int. J. Agric. Biol.
**2008**, 10, 375–380, ISSN 1560-8530. [Google Scholar]

**Figure 2.**Volume development during fermentation (own illustration based on [5]).

**Figure 3.**Schematical illustration of the measurement system (

**left**) and real measurement module (

**right**).

**Figure 6.**Architecture of Mask R-CNN (own illustration based on [15]).

**Figure 8.**Superimposition of dough surface with (pink) and without induced aerosol (purple) of three different dough objects (

**a**–

**c**).

**Figure 10.**Processing steps: (

**a**) Measured topography point cloud of the fermentation chamber inside; (

**b**) Segmented dough pieces; (

**c**) Model-fitted dough pieces.

**Figure 11.**Course of width (

**upper left**) and height (upper right) of three dough objects, the quotient W/h (

**lower left**) and calculated volume (

**lower right**).

Test dough Piece | Shape | Real Dough Volume | Calculated Dough Volume | Deviation [%] |
---|---|---|---|---|

1 | Round | 70 | 78 | 11.4 |

2 | Round | 70 | 76 | 8.6 |

3 | Long | 140 | 119 | 15.0 |

4 | Long | 140 | 122 | 12.9 |

5 | Long | 90 | 79 | 12.2 |

6 | Round | 70 | 73 | 4.3 |

7 | Long | 90 | 81 | 10.0 |

8 | Long | 70 | 62 | 11.4 |

9 | Round | 140 | 143 | 2.1 |

10 | Round | 70 | 76 | 8.6 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Giefer, L.A.; Lütjen, M.; Rohde, A.-K.; Freitag, M. Determination of the Optimal State of Dough Fermentation in Bread Production by Using Optical Sensors and Deep Learning. *Appl. Sci.* **2019**, *9*, 4266.
https://doi.org/10.3390/app9204266

**AMA Style**

Giefer LA, Lütjen M, Rohde A-K, Freitag M. Determination of the Optimal State of Dough Fermentation in Bread Production by Using Optical Sensors and Deep Learning. *Applied Sciences*. 2019; 9(20):4266.
https://doi.org/10.3390/app9204266

**Chicago/Turabian Style**

Giefer, Lino Antoni, Michael Lütjen, Ann-Kathrin Rohde, and Michael Freitag. 2019. "Determination of the Optimal State of Dough Fermentation in Bread Production by Using Optical Sensors and Deep Learning" *Applied Sciences* 9, no. 20: 4266.
https://doi.org/10.3390/app9204266