Recent Trends in Computer Vision for Cheese Quality Evaluation ^†

Abstract

A review of current trends in CVSs (computer vision systems) regarding hardware and software components, in the context of cheese quality evaluation, is presented in this paper, and some directions for future development in the field are discussed. The application of methods of computer vision for evaluating the quality indicators of different types of cheese is explored in detail regarding systems for data gathering and algorithms for their processing. We discuss opportunities for the usage of advancements in AI (artificial intelligence) for a fast and effective quality control of food production. Modern computer-based concepts are viewed in the context of food quality control such as CAFE (computer-aided food engineering) and digital twins. Methods for further enhancement of the quality of human life are highlighted in the context of sustainability related to contemporary computer-based technologies.

1. Introduction

Over the last twenty years, we have observed advancements in computer technics and technologies based on them. These processes influence many aspects of human activities as well as our everyday life. As a significant part of our life, food and food technologies also are involved in contemporary digitalization processes, and thus, the production of high quality foods is related to computer-based technologies and different methods of information processing, such as computer vision methods, AI (artificial intelligence) and IoT (Internet of Things) [1,2,3]. In manufacturing, many automation systems are used in order to control the production process regarding the sustainability and quality of produced foods, using modern concepts of Industry 4.0. These systems are computer-based and complex, and their components use a lot of input devices and sensors that collect data for signals in different ranges. The ultrasound signals as well visible and infrared ranges of the electromagnetic spectrum are sources of valuable information for controlling all stages of the production process and quality control of produced foods. The control of food quality is part of the real-time production process and also part of offline quality control. Registering signals in visible and other ranges provides an input flow for, i.e., CVSs (computer vision systems), which have to extract information for specific characteristics of examined objects and to produce data useful for other systems responsible for decision making (such as AI-based systems).

As one of the most popular milk products, cheese has a long-term history and traditions for its production all over the world. Its nutrition composition, ability for long-term storage, health benefits, and variety of tastes and flavors motivate a lot of food specialists to prepare recipes, diets, lists of recommendations and rules for healthy food consumption that include many different types of cheese [4,5]. Increasing the demand of cheese production involves the utilization of computer technologies for fast, effective and sustainable quality control. The trends in Industry 4.0 related to the dairy industry are discussed in the context of high-level technology concepts such as AI, big data, IoT, robotics, Blockchain and 3D printing [6]. Thus, the motivation for the current study is to explore and to summarize research results that indicate the application of computer-based technologies, especially methods (algorithms) of computer vision, for cheese quality evaluation and to highlight their advantages, disadvantages and some opportunities for future development in more detail with respect to specifics in data acquisition and processing.

2. Computer Vision Systems’ (CVS) Application in Cheese Quality Evaluation

The main structure of CVS is related to three basic components necessary in every data processing system—a sub-system for data acquisition, a sub-system for data processing and a module for results’ presentation (Figure 1). A computer system equipped with the necessary system resources (hardware and software components) is mainly used for data processing and visualization. Some implementations of CVSs are based on embedded systems which use specialized microprocessors and peripheral modules. In specific industrial systems, CVSs could be included as a part of the whole system that controls the production process; thus, CVSs have to communicate with microcontrollers in a heterogeneous environment. Therefore, the relations among CVSs, different types of users and other systems becomes vital in the context of a contemporary digital world in order to support the production of high-quality foods.

In the field of cheese quality assessment, signals from ultrasound, near-infrared (NIR), radio and visible waves are exploited in order to receive information about the surface and internal structure of cheese, and specific sensors for registering information and sources for emitting information are used, respectively.

2.1. Using Ultrasound for Cheese Quality Evaluation

Ultrasound signals have a wide range of applications in the dairy industry in two main directions—firstly, as a tool for the microbiological enhancement of milk during different processes of its transformations (high-intensity ultrasound signals), and secondly, as a tool for the quality assessment of different milk products (low-intensity ultrasound signals) [7].

Some parameters of ultrasound signals, such as amplitude and velocity, are informative for the assessment of the structural characteristics of Swiss-types of cheese, Cheddar, Torta del Casar, and Bulgarian white cheese in brine [8,9,10,11]. The advantages of ultrasound technology are related mainly to its non-destructive nature regarding the structure of the examined cheese; on the other hand, its disadvantages are related to its high sensitivity to environmental parameters, such as temperature, humidity, etc. Some researchers indicate that the ultrasound signal which is registered after its passing through examined cheese samples (through-transmission mode) carries significant information about the cheese structure [10]. Crespo et al. used contact ultrasound technology for the quality control of soft types of cheese (Torta del Casar cheese) during ripening, and they report a high correlation (p < 0.01) between several ultrasonic parameters (related to signal velocity) and the physico-chemical and textural properties of examined cheese samples. Based on the research results, the authors offer a model for predicting the defective cheese samples during process of their ripening using discriminant analysis. Earlier, Benedito et al. also used an ultrasound transducer and receiver that contacted the examined cheese (Cheddar) [9] in order to assess its textural properties related to the state of cheese maturity. Other specific elements in the cheese structure such as “eyes” or other cavities that present some ripening defects also could be monitored using ultrasound signals, as reported by Eskelinen et al. for Swiss-type cheeses [8]. Using contactless ultrasound technology, Shopov et al. classified white cheese in brine into two categories: a first class for cheese produced from cow milk and a second class for cheese produced from sheep milk [11]. High classification accuracy is achieved due to essential differences in the water and fat content (and related to them the density) of cheese samples that are produced from different types of milk [12,13].

2.2. Using NIR for Cheese Quality Evaluation

In the last few years, due to the continuous evolution of sensors technologies, near-infrared (NIR) signals became more and more exploited in the field of fast and accurate on-line measures of some food quality parameters also in the field of the dairy industry for cheese quality evaluation using portable devices [14]. Advantages of NIR technology are related mainly to its viability in context to provide precision and a fast speed of measurement in comparison with traditional laboratory analysis based on manual sampling; on the other hand, some disadvantages could be noted, such as the costs of implementation and support.

Solberg et al. report their results based on a case study of some physico-chemical quality indicators for the large-scale production of Gouda cheese using NIRS (Near-Infrared Spectroscopy) [14]. They monitor the dry matter content in two control points during the production process, and all collected data are analyzed in order to predict this parameter (dry matter content) for the produced Gouda cheese. The power spectral density (PSD) of the NIR signal is used to measure the dry matter content, and the results form the basis for better understanding the parameter variation in the production process of Gouda cheese. The quality indicators of yellow cheese during its storage are successfully measured based on the NIR signal data, and a regression model is proposed for yellow cheese shelf life prediction based on changes in its active acidity [15]. As a part of the experimental setting, a diode strip is used as a source of light (the highest intensity of emitted light is at 850 nm) by image capturing with a video camera. Typical for Turkey, Ezine cheese is examined using NIR and MIR spectroscopy (mid-infrared) in order to be propose accurate models for prediction of the physico-chemical parameters of cheese quality [16]. This type of cheese (Ezine) is a complex biotechnological product due to its composition based on a mixture of bovine, caprine, and ovine milk in different proportions, which is the main reason for the variety of the physico-chemical parameters of Ezine cheese produced from different regions (dairy farms). Ayvaz et al. report that using data extracted from NIR signals, effective regression models are developed for the prediction of total protein, fat, salt, dry matter, moisture and ash content in a semi-hard full-fat cheese (Ezine). Other research studies present effective models based on NIR data and pattern recognition for prediction of the stage of ripening of Turkish white brined cheese [17]. Yaman et al. report results of monitoring the concentration of organic acids, free fatty acids and free amino acids in white cheese during the ripening period using standard analyses and NIRS for solutions prepared using samples of the examined cheese and three different solvents (water, ethanol and methanol). The stage of ripening for cheese prepared from cow, ewe and goat milk in different proportions is preciously predicted using NIR data and ANNs (Artificial Neural Networks) designed with three levels using two popular software tools for ANN modeling: Java Neural Networks Simulator and Matlab [18]. Soto-Barajas et al. report that using ANN and data for fatty acid concentration, the prediction accuracy is 80%, whereas the prediction accuracy for an ANN model and data of NIRS is 100%. Thus, NIRS has become preferred for many different analyses related to cheese ripening.

2.3. Using MRI for Cheese Quality Evaluation

MRI (Magnetic Resonance Imaging) is a branch of science that is typically related to medicine, and its nature is based on using radio and magnetic waves. The application of MRI for cheese quality evaluation is related mainly to monitoring the cavities in the cheese structure. The main advantages of this technology are related to its ability to present the internal structure of an examined object without any mechanical impact; i.e., MRI technology is non-invasive and non-destructive, but the necessary equipment is too expensive.

As a popular cheese with mold in its structure, blue-veined cheese is produced all over the world, and its production is related to a control of mold growth based on the evolution of cavities and their oxygen content; thus, the development of technologies that effectively support this process is vital. In this context, the application of MRI technology for quality control by the manufacturing the blue cheese is an actual topic. The structural analysis of two groups of blue-veined cheese samples produced in different manufacturing conditions is performed using a Magnetom Open Siemens scanner [19]. Magnetic resonance images are normalized and segmented using the SOM (Self-Organizing Map) technique, which is an approach based on neural networks. Onea et al. report results from their preliminary study of opportunities for the application of MRI in order to recognize the type of used manufacturing technology and stage of ripening for soft blue cheese. Using a statistical approach (discriminant analysis), researchers concluded that MRI technology is appropriate for assessing the quality of blue cheese during its manufacturing and regarding cavities evolution. A special type of cavities named “eyes” are typical for some semi-hard types of cheese, and their size (volume in cheese pieces) and distribution are important for the overall quality assessment of the product. A microCT system and a computer tomography reconstruction are used as a referent measurement in order to demonstrate the accuracy of MRI technology for assessment eyes in a cheese structure [20]. Musse et al. report that the volume of eyes (including overlapping eyes) could be measured with high accuracy using MRI and image processing algorithms implemented in a Scilab environment. The eyes measurement is performed during the ripening period for all samples using MRI and 3D reconstruction before and after measurement with a microCT system. The measurement results are compared in order to determine the accuracy of MRI-based measurement, and a mean error of about 1% is calculated. Other research presents the opportunity for application of the MRI technology for the assessment of a specific Italian cheese: Fiore Sardo [21]. This type of cheese is registered as a PDO (Protected Designation of Origin) product, and its production has started in Sardinia as an artisanal cheese produced from raw milk. Nowadays, Fiore Sardo cheese is produced industrially using thermal processed milk and traditionally using raw milk in small farms. Anedda et al. report the results of their experiments that prove the applicability of MRI technology for the recognition of two types of Fiore Sardo cheese: one that is produced using thermal processed milk and second type that is produced using raw milk. Using a DNN (Deep Neural Network), they achieve about 93% accuracy in the classification of the cheese samples into two classes: one for Fiore Sardo cheese produced from thermized milk and second for cheese produced the classic way using raw milk.

2.4. Using Digital Images for Cheese Quality Evaluation

In the visible range of the electromagnetic spectrum, a lot of registering devices have been developed, and a wide range of image-processing algorithms are known that have application in cheese quality evaluation regarding the possibility of examining some of the visual parameters of cheese quality and the related structural characteristics [22]. The advantages of CVS using signals that are visible waves are mainly related to the contactless and non-destructive nature of the registering process, and the possibilities of using a multipurpose digital camera and additional low-cost equipment. The evolution of gas holes in a traditional Portuguese cheese named “Queijo de Nisa” during its ripening is monitored using images processing, and a strong correlation with physico-chemical parameters is registered [23]. The cheese samples are captured at the beginning of ripening, on the 15^th day, and on the 35^th day of ripening with a Canon M6 digital camera, and all the collected images are processed with ImageJ software version 1.52d (National Institute of Health, USA) in order to define the number of holes, as well as their perimeter, the minimum and maximum Feret diameter, the area occupied by the holes and their percentage of the cheese sample. Dias et al. conclude that changes in the area of gas holes and the percentage of this area of the sample correspond with changes in the pH and moisture content of “Queijo de Nisa” cheese during the ripening period. The soft cheese Mozzarella is one of the most popular type of cheese in Italy, and its color is an important characteristic that influences consumer preferences because this type of cheese is often used as an ingredient in popular foods such as pizza and lasagna. A comparison between the colors of Mozzarella cheese measured with a spectrophotometer and with CVS using five different digital cameras is performed in order to assess the ability of computer vision to register the color of Mozzarella cheese [24]. The designed computer vision system has a vision cabinet equipped with two luminescent lightening sources and a port for stable mounting the camera. All of the cameras used in the experiments have CCD or CMOS sensors and a resolution in the range 0.92 to 16 MP. All captured images are processed in the Scilab environment in order to extract the color of the cheese in RGB and La*b* color systems. The comparison of cheese colors measured with a spectrophotometer and with a designed CVS is based on the color difference calculated for measured colors, which is significant only when the camera with the lowest resolution is used. Minz and Saini report that the measured colors for the examined cheese samples using the developed CVS and spectrophotometer are equivalent, and thus, the CVS could be an inexpensive alternative for monitoring the quality of Mozzarella cheese regarding its color. Other researchers present an application of computer vision methods for evaluation of the quantity and distribution of specific ingredients in cheese that are optically visible in its structure [25,26]. Jeliński et al. use a flatbed scanner for image acquisition and Matlab (Mathworks, 1992) for image processing in order to evaluate the quantity and distribution of vegetable ingredients in pasteurized cheese [25]. The identification of ingredients is based on images thresholding using Otsu’s method, and the distribution analysis is performed using four quarters of every image. The reported results indicate that about 88% accuracy is achieved for prediction of the quantity and distribution of garlic and parsley using images processing in comparison with a sensory evaluation for these parameters, and about 81% and 71% accuracy is achieved for the distribution and quantity of pepper and parsley, respectively, compared with the assessment of panelists (sensory evaluation). Ganchovska et al. use a CVS module for data acquisition built from a digital camera (Canon EOS 2000D) and luminescent lightening positioned in a photo-box, and they process all the collected images of blue cheese in an NI LabVIEW environment [26]. The processing of blue cheese images is performed in order to evaluate the quantity of mold and its distribution on the cheese cut surface. An evaluation of mold distribution is implemented using different size grids, and the results are compared with experts’ assessment through correlation analysis. It is concluded that the 4 × 4 grid size (i.e., the image is divided into 16 pieces) and one of algorithms for automatic thresholding- metric, clustering or inter-class variance are preferred in order to achieve a very high (above 90%) correlation with experts’ assessment regarding the quantity and distribution of Penicillium roqueforti mold in blue cheese. A special characteristic of Bulgarian white cheese in brine related to its structure is effectively evaluated using images processing in HSI (Hue–Saturation–Intensity) color space [27]. This characteristic is named “Porcelanov lom”, and it is related to the presence of specific areas that look like parts of a broken porcelain cup on the broken surface of the block of cheese. The presented algorithm for images processing in an HSI color space extracts coefficients for four colors into segmented images that are used for comparison with experts’ assessment for the presence of “Porcelanov lom” in examined white sheep cheese in brine, and the results prove the effectiveness of the developed algorithm (very strong correlation is achieved). Other research presents an application of images segmentation through the SRM (Statistical Region Merging) algorithm for complex assessment of the structure of Bulgarian white brined cheese in an RGB color system [28]. The processing of images is performed using ImageJ software, and the results indicate a strong correlation with experts’ assessment. The authors conclude that a calculated correlation coefficient (above 0.85) indicates that the usage of a median filter and segmentation algorithm (SRM) provides a basis for the development of an intelligent software tool for automatic assessment of the quality and the structure of a cut surface for white cheese in brine. Other research presents an application of algorithms for images processing for assessment of the distribution of gas holes in Swiss cheese [29]. The authors define the factor of gas holes’ even distribution, and using statistical analysis, they conclude that a 4 × 4 grid size is preferred for distribution analysis, and a high value of even distribution corresponds with moderate-to-even distribution determined by experts for the examined samples. The hyperspectral imaging is a technology that uses an additional lens in order to extend the standard visible range that is registered by a digital camera and to collect more spectral data for every pixel in the image. Collected hyperspectral data are useful for the identification of specific elements in cheese and cheese-based products [30,31]. Some studies report that the starch content in fresh cheese and mold growth in cheese lets could be effectively registered using hyperspectral imaging in the ranges of 200 to 1000 nm and 400 to 1000 nm, respectively. Using Matlab for image processing and statistical analysis, the authors conclude that hyperspectral imaging is appropriate for a rapid and contactless analysis of the quality of cheese and cheese-based products [30,31].

3. Discussion and Future Trends

As a common a term, “computer vision” is related to opportunities to use hardware and software resources in order to generate some statements about an examined object that could be interpreted as a result of information perception about the explored object by the computer. The parallel between human vision and computer vision often leads to understanding that computer systems have to register information from the visible range of the electromagnetic spectrum in order to implement its vision. Regarding the evolution of technologies, the computer goes from opportunities to abilities for the implementation of its vision based on signals from many different ranges—not only from the visible range. Thus, the contemporary computer systems are able to support the evaluation of many quality parameters for foods, and in particular cheese, using a lot of different signals. As a result, the term “computer vision” became wider, including the perception and processing of signals not only in the visible range. Modern concepts such as CAFE (computer-aided food engineering) are introduced in order to complement traditional approaches [32]. In the field of cheese production, CAFE could provide enhancements in manufacturing due to its nature based on food physics. Cloud-based technologies, AI (Artificial Intelligence) and big data analysis also have potential to improve the production of cheese with high quality, but the effectiveness of all of them is related to the acquisition of valuable data for objects and for processes. Thus, the development of robust and resilient technologies for the extraction of data with highly informative characteristics has become more important. The current study is focused on recent research in the field of application of computer vision for cheese quality evaluation. Different type of cheese and technologies for data acquisition are used in order to assess the physico-chemical, structural and sensory parameters of cheese quality (

Table 1. Cheese quality parameters evaluated using CVS.

	Type of Cheese	Quality Indicator	Used Technology	Source
Regional cheese	Torta del Casar	fat and ash content, firmness	Ultrasound	Crespo et al., 2020 [10]
	Fiore Sardo	internal structure	MRI *	Anedda et al., 2021 [21]
	Queijo de Nisa	gas holes area	CV *	Dias et al., 2020 [23]
Cheese with mold	Blue cheese	cavities evolution	MRI *	Onea et al., 2003 [19]
		mold distribution	CV *	Ganchovska et al., 2022 [26]
Cheese with holes	Swiss type of cheese	maturation stage	Ultrasound	Eskelinen et al., 2007 [8]
		eyes distribution	CV *	Bosakova-Ardenska et. al., 2021 [29]
	Semi-hard cheese	eyes volume	MRI *	Musse et al., 2014 [20]
White cheese	Bulgarian white cheese in brine	type of raw milk	Ultrasound	Shopov et al., 2020 [11]
		presence of “Porcelanov lom”	CV *	Danev et al., 2019 [27]
		structure	CV *	Bosakova-Ardenska et al., 2022 [28]
	Turkish white cheese in brine	organic, free fatty and free amino acids	NIR *	Yaman et al., 2023 [17]
	Ezine cheese	protein, fat, salt, dry matter, moister and ash content	NIR *	Ayvaz et al., 2020 [16]
	Cheese from milk mixture	fatty acid, type of raw milk	NIR *	Soto-Barajas et al., 2013 [18]
Cheese	Mozzarella	color	CV *	Minz and Saini, 2021 [24]
	Cheddar	moisture	Ultrasound	Benedito et al., 2000 [9]
	Yellow cheese	active acidity	NIR *	Baycheva et al., 2023 [15]
	Fresh cheese	starch content	CV (HSI) *	Barreto et al., 2018 [30]
	Cheese with vegetable ingredients	amount and distribution of ingredients	CV *	Jeliński et al., 2007 [25]
	Gouda	dry matter content	NIR *	Solberg et al., 2023 [14]

* NIR—Near-Infrared Technology; MRI—Magnetic Resonance Imaging; CV—computer vision (visible light); CV (HSI)—computer vision (hyperspectral imaging).

).

Earlier studies of some visible structural quality parameters of cheese using ultrasound or MRI technologies have provided a basis for developing image processing algorithms (CV) for their further assessment. Other observations are related to cheese color, which is a rarely studied parameter because it depends on a very wide range of factors related to using raw milk and other ingredients. On the other hand, the color of specific ingredients (such as mold, vegetables and others) is used for their identification in the cheese structure. In the last few years, another visible trend is related to studies for the application of CVS for the evaluation of specific regional types of cheese (PDO cheese also). These studies are motivated by trends known as “green thinking” that induct preferences to artisanal cheese due to its health benefits.

The continuous development of portable and inexpensive devices for measuring the quality indicators of cheese will motivate a lot of studies based on some technologies that are typical for medicine [33]. For example, the development of low-cost smartphone-based colorimetric devices is not only a dream [34]. The quality of digital cameras that are built into smartphones has provided the opportunity to collect information about the optical parameters of cheese quality in an easy and user-friendly way. Furthermore, the wide range of abilities for data sharing through mobile devices such as smartphones provides a basis for complex analysis using remote computational and data resources. Some new computer-based concepts such as “digital twins” are used for the development of a conceptual framework for modeling the process of food production in order to perform a retrospective or predictive analysis of some quality indicators [35]. Krupitzer and Noack present their ideas to design an intelligent (self-aware) system to produce food with high-quality thinking about a variety of physico-chemical parameters of raw materials and their influence on the quality of produced foods. These ideas could be implemented in cheese production and also be motivated by technologies that are already implemented in the dairy industry, which provides a basis for future development regarding the physico-chemical parameters of raw milk, which is a complex biological product.

4. Conclusions

In the last decade, the scientific interests in the field of computer vision have increased significantly, which has been proven by the increasing number of scientific works with the term “computer vision” in the title, abstract or keywords sections (Figure 2). On the other hand, the scientific interests in the field of food science and particularly cheese production science have also increased but at a slower tempo. It is obvious that the evolution of computer techniques has led to the development of methods and approaches to improve quality control in cheese production. We did not notice an increasing trend in the number of occurrences of the terms “computer vision” and “cheese” for the current year (2023), because we conducted our information retrieval before the end of the year.

In the last few years, the concept of “Sustainable ICT” (Sustainable Information and Communication Technology) has become more actualized, and thus, its principles founded on the reduction in electronic waste, energy consumption, etc. are being implemented in every field of human activity, including food production and quality control (scientific interest regarding the terms “sustainable” and “food quality” has increased about sixfold since 2013, which could be noted using search tools in scientific databases). Regarding these principles, the usage of one device or system that is designed for a specific purpose to support a variety of manipulations in cheese production and quality control corresponds to these trends. Thus, the highest-level equipment produced for medical purposes could be used for cheese quality control also, and a multipurpose device such as a digital camera could be used for monitoring the quality indicators of cheese, too. In the future, our computer systems are going to be more intelligent, and some ideas that are part of science fiction at present could be a part of the real world in very near future. Thereby, “computer vision” will continue its evolution, and the benefits of such technologies are growing in all aspects of human life and in the process of quality control of food and especially cheese also. The development of AI-based software tools for computers, smartphones and mobile systems is going to provide intelligent digital assistants, which will help us interact with the environment as a part of our personal and professional activities. In the near future, our smartphones will be also our portable scanners for the quality control of food and many other objects that exist in our environment.