Vision System in Product Quality Control Systems

Abstract

The development of technology for manufacturing products and machines carrying out technological operations is closely linked to developing systems for tracking and controlling product and production process parameters. This paper shows how the development of quality control and production management systems such as TQM and MES is related to the development of imaging and image analysis methods used in industry. The development of imaging methods is discussed in the context of developing product quality control capabilities. It is also shown as to what extent image analysis can be used to observe manufacturing parameters and process management capabilities. It was noted that the use of vision systems as an industrial measurement-quality control system would still increase. Due to the increase in imaging resolution, there was an increase in the imaging frequency, growth in the spectral range of imaging, and the dynamic development of three-dimensional and hybrid imaging methods. Based on experience from industrial applications and tasks described in scientific publications, areas where vision systems will play a key role in inspection tasks have been identified. This is the introductory article for the Special Issue “Image Analysis for Product Quality Control” on using vision systems in various industries to execute production quality control tasks.

1. Introduction

The paper discusses the development of the use of vision systems and image analysis methods in the implementation of inspection and measurement tasks for assessing the quality of product workmanship and conducting the production process. The observed technical and technological development of enterprises influences the construction of modern production stations, machines, and production lines. A characteristic feature of modern production lines is their increasing structural and functional complexity in technology and production process control. There is a growing proportion of electronic systems, drive systems, control systems, safety systems, and specialized software for managing machine operation. Most machines are also equipped with subsystems that allow for automatic diagnostics of operating parameters. Each subsystem’s operating time and parameters are controlled, enabling the collection and analysis of historical and current data in production systems. Based on these parameters, the timing of periodic inspections and overhauls are determined as well as the degree of wear of the components operating in the machines [1,2]. The results of the operation of the diagnostic subsystems installed in the machines make it possible to shorten the service response time and reduce the time for dismantling. The development of systems for monitoring the operation of machines is currently focused on predicting the possibility of failure and wear events and using artificial intelligence in these tasks

Machines and production stands performing operations in the manufacturing process are also a part of the company’s production and manufacturing quality management system. This involves equipping machine control systems and control and measurement systems installed in machines with interfaces enabling communication with systems: TPM (Total Productive Maintenance), TQM (Total Quality Management), MES (Manufacturing Execution System), and ERP (Enterprise Resource Planning). Observation and analysis of parameters of the production process carried out on an ongoing basis are currently necessary from the point of view of the management of the production process, connected with the management of enterprise resources and the control and steering of product quality [3,4,5,6,7,8,9,10]. This has led to the dynamic development of the architecture of industrial information systems used in production process control [11]. The next step was developing the “Industry 4.0” strategy, the foundation of which is the so-called “Smart Factory”, encompassing the integration of technical means of production and cyber-physical systems supporting operational processes and management. Technical means of production include machines and production stands, production lines, industrial infrastructure, and means of transport [12]. Introducing the “Smart Factory” concept aims to achieve autonomy of control operations, monitoring, and product quality control through direct data exchange between machines and measurement systems using network infrastructure and data analysis algorithms and artificial intelligence solutions [13].

This is related to the many years of systematic development of controlling technological operations directly within the conducted operation. Quality control operations at separate workstations are being abandoned to introduce measuring and control systems as critical equipment for machines and manufacturing stations. This requires extending the tasks carried out in machine control systems. It is also necessary to select appropriate product control methods and measure product parameters in industrial conditions. At the same time, it is required to consider the equipment and design of the machine, the installation of network infrastructure and equipment enabling the implementation of measurements, and analysis of the measurement data. The result of the conducted control decides whether to refer the semi-finished product to further processing or remove it from the process due to noncompliance with the parameters accepted in the manufacturing documentation. The product’s measurement parameters are sent to cooperating machines and master systems and used in the production management process.

It is also important to link the parameters describing the product quality with the parameters of the manufacturing process [14,15,16]. Increasing the requirements for developing machine functions toward the realization of diagnosis, analysis, and the reporting of parameters describing machine operation force the development of cyber-physical systems [16,17,18,19,20]. These systems are responsible for process control using sensor networks and production monitoring using the digital network [21]. Additionally, cyber-physical systems enable access to distributed industrial data [22]. Cyber-physical systems should be seen as solutions that integrate sensor networks with machine and robot control systems that continuously use industrial networks to collect and archive production data.

It is currently assumed that the quality of product workmanship should be assessed based on recording and evaluating the manufactured product’s parameters in the technological operation time. In many cases, quality control carried out after the technological process does not allow for detecting all defects arising from the technological operations [23,24]. In the context of developing the “Smart Industry” concept, the development of the control and measurement capabilities of machines should be connected with introducing new measurement methods on machines currently used only in laboratory control. In many measurement tasks, this involves the development of new hardware solutions and measurement methods, taking into account the specification of the machine workspace. Designers of measurement systems used in machines reach for various technologies, enabling the realization of tasks related to the assessment of product parameters [25].

The paper aimed to indicate the relationship between the possibilities of developing production systems, production management systems, and the development of the capabilities of industrial vision systems. The range of imaging methods, the content of tasks performed by modern vision systems in relation to industrial sectors, and the level of robotization level are indicated. In addition, the development of measurement and diagnostic capabilities is presented based on reports on the market for vision systems observed in recent years, and forecasts for the development of this technology in the coming years.

2. Vision System in the Quality Control Systems

Industrial production is inextricably linked to the continuous evaluation and improvement of “product quality”. However, what is “product quality”? The product can be evaluated in two stages. The first stage includes the control of the product manufacturing parameters and technical parameters of the product itself. In the next stage, product quality assessment was carried out by product users/customers. Product quality assessment carried out in both stages was carried out on the basis of other parameters.

Functional design quality is the parameter that describes the project in the context of the function being implemented. Proper preparation of the product requires experience in selecting a solution that fulfils the assumed function of the product. Next, it requires considering the issues related to the preparation of the product technology. The quality of the project determines the degree of satisfaction of the recipient with the manner and scope of the functions performed included in the prepared product design.

Product performance quality is the degree of compliance of product parameters with the parameters assumed in the project documentation. It is assumed that the quality of the product is correct if the parameters describing the product are within the tolerances adopted in the design documentation. In many cases, the parameters adopted in the design documentation are also accompanied by descriptive parameters defining the “aesthetics of the product”. All noncompliance with the assumed parameters is indicated as defects of the product.

By defining the role of vision systems in implementing quality control tasks, we are talking about tasks related to controlling the product performance quality. This quality can be described by parameters describing selected features of the product. The task of the vision system is to indicate features that, in the parametric evaluation, do not meet the design assumptions. It should be noted that the quality features will be strongly related to the industry [26,27,28]. We speak of “manufacturing defects” or “manufacturing deficiencies” in industrial terminology. Their formation is most influenced by malfunctions of machines and equipment, defects in raw materials and materials, errors made by operators, organization of production, and incorrect or insufficient technical control. The quality of product workmanship can be determined on the basis of measurements and tests carried out in the production process in manufacturing quality control operations. Functional quality and product workmanship are described together as the potential quality of the product. This is verified by users who determine the actual degree of satisfaction of their needs.

Vision systems are currently taking over control and measurement operations by operators on production lines. This is due to the significant increase in the productivity of production lines and the expansion of inspection tasks in many industries. In many cases, the operator cannot effectively carry out the required control tasks. This is mainly due to the increase in inspection frequency associated with increased production efficiency. The lack of repeatability of inspections due to the monotony of the performed task and operator fatigue also greatly influence the outcomes. There is also a need to observe phenomena in the environment that are aggressive for humans and those that the human eye cannot register (e.g., due to the wavelength range dangerous for health conditions of technological operations) [29,30]. Applying a vision system to perform tasks within product quality control can be divided into research and implementation.

As part of the research and preparation design of the vision system, research should be carried out in the field of recognition, description of the measurement task, and the development of a feasibility study [31]. Research should be carried out taking into account the specific operating conditions of the designed vision system. Research in the field of identification and optical modeling not necessary at the stage of preparation of correct lighting conditions should be included. Then, preliminary algorithms for image analysis should be developed and tested. Algorithms make it possible to extract the features used for measuring the parameters describing the product from the image. The result of the research work is to develop a “task feasibility study”. This study should include a description of the scope of control tasks that can be performed and their costs. It should also indicate control tasks that cannot be performed under given production conditions. After the research stage, based on the results obtained in the research part of the task, a decision can be made to start implementation work. It is only within their implementation framework that mechanical, electrical, and IT projects connected with installing the system in machines or on production lines begin [32,33].

Preparing a successful industrial project requires interdisciplinary knowledge and implementation experience from the engineering staff. There is a need for the cooperation of mechanics, electronics, optics, and computer science specialists. Engineers leading the manufacturing process and technologists responsible for product manufacturing must directly cooperate.

The most challenging stage in implementing a vision-based production quality control system is transferring a solution developed under laboratory conditions to the production line. A significant challenge is to achieve the assumed accuracy of the measurement of selected product characteristics in the presence of industrial disturbances. The following should also be taken into account related to the operation of the production line, with the natural variability of production processes and the technical and quality level of process automation [34].

Another challenge is to combine the vision control system with the control systems operating in plants, production data acquisition and analysis systems, and production and plant management systems. Tracing back throughout history, the development of manufacturing systems has also forced the development of methods to assess the manufacturing process. Research conducted by Walter Shewart on statistical methods for describing production variability allowed for the preparation of assumptions for SPC systems of statistical process control. These studies were continued by W. Edwards Deming and supplemented by continuous process improvement, which led to the development of management systems through product and process quality—Total Quality Management (TQM). This method of company management is aimed at constantly increasing the product quality, both in terms of production and product function.

Vision systems play an essential role in collecting parameters describing the quality of product manufacture. This is due to the possibility of controlling many product parameters in one system based on product image analysis. At the same time, vision systems make it possible to link parameters describing product quality with product serial numbers and technological parameters at which the product was made. The collection of data relating to product quality and process parameters is sent directly to TQM systems and used to evaluate the process and react to changes in it.

The food industry, for example, is seeing the introduction of vision systems at every stage of production. The reception of raw materials into production is currently carried out using sorting systems for materials classification. For example, sorting fruit and vegetables described in the literature often occurs during harvesting. The first sorters worked using classical image analysis algorithms on monochrome and color images [35]. Subsequent solutions used infrared imaging [36]. With the development of neural networks, solutions using this method for fruit quality classification have appeared [37]. All systems, regardless of the imaging method used or the image analysis algorithm employed, work within the TQM framework and ensure the quality of the raw material received for production. A similar development of vision technology has been observed in the woodworking industry. The first scientific papers and implementations discussed using 2D monochrome and color images for wood surface analysis and surface defect detection using classical image analysis algorithms [38]. Subsequent studies extended the measurement methods to 3D solutions [39]. These were used, among others, to study wood growth and assess the wood surface defects [40,41]. The next phase of developing vision systems in the industry saw implementations using machine learning methods [42]. However, the most dynamic development of vision systems was observed in the automotive sector. It can be assumed that every newly designed production line and the technological station is equipped with a set of vision systems controlling the manufactured product and process parameters. Vision systems (2D and 3D) are used together with industrial robots to locate semi-finished products in bins and conveyor belts. They are therefore involved in the production process. In addition, they are also used in product inspection operations, where they carry out measurement tasks [43,44]. Within the scope of 3D imaging, control and measurement systems based on laser triangulation, stereovision, and the TOF method are applied in production processes [45,46,47,48] (Figure 1).

The food and waste sorting industry has also seen a very rapid development of hyperspectral imaging in implementing inspection tasks in the production process. This imaging method has made it possible to observe the product in multiple ranges of electromagnetic radiation and to identify correlations between selected features of the object and the image in the selected imaging range. Waste sorting is an exciting industry where the combination of hyperspectral imaging with artificial intelligence methods has been observed [49,50,51]. Additionally, combined with AI solutions, hyperspectral imaging will be a key technology in detecting and assessing defects in materials and products.

The trend of introducing vision measurement and control systems to production lines observed in recent years will also continue in the years to come. According to the Frost and Sullivan report [52], the automotive industry will still have the largest share. However, the second and third places will be taken by the pharmaceutical and medical industries. This information is referred to in the year 2020 as the base (Figure 2). One of the constraints on the development of vision-based quality control systems is the variability and intermittent difficulties in the availability of high-resolution matrices with the parameters needed to build measurement systems. For example, the impact of COVID-19 on the production of components for vision systems can be seen in 2021–2022.

Currently, the role of vision systems as data sources in the automation of quality control is growing. This is particularly noticeable in factories using database systems and process data analysis algorithms working within Smart Industry. The advantage of this solution is that the image analysis and description of the product quality in the form of parameters are carried out directly by the vision system on a production line.

A report by C. Barnes and Co [53] on the development of the vision systems industry in 140 countries indicated an increase in the use of vision systems due to the need for the automation of manufacturing processes. This is connected to the necessity for the realization of automatic product control, process control, and robotization. The World Robotics 2021 report and the experience gained during implementations indicate that the number of robots is growing in practically every manufacturing industry. The term VGR—Vision Guided Robotics—means that robots will be equipped with a vision system whose task is to control the robot based on information read from the environment by the vision system. This applies to process robots that perform technological operations—manipulation—and transport robots—mobile robots that perform logistical functions in enterprises. Figure 3 shows an increase in the use of robots in selected branches of production. The World Robotics 2021 report published by the International Robotics Federation [IFR] showed a dynamic growth in the use of robots in factories around the world, which was an increase of 10% [54]. Despite the global pandemic, sales of new robots increased by 0.5%, with 384,000 new robots delivered worldwide in 2020. The increase in robot use directly influences the growth in the use of vision systems to support process operations.

At the same time, the approach to the construction of quality control algorithms used in vision systems is changing. There is a very high demand for new AI solutions that enable a change in the approach to preparing a product quality control algorithm. Based on the use of AI algorithms, deep learning is being used in the preparation of applications that enable the identification of product defects. The development of such algorithms can be seen in the historical data and development forecasts available in the report published by Grand View Research Inc. [55]. Since 2018, an increase in the development of software-based deep learning has been observed (Figure 4).

The most significant increase was visible and predicted in the design, construction, and industrial implementation of a ready vision system on the production line in quality tasks (Figure 5).

3. Discussion

Vision systems support automation across a whole range of tasks performed on production lines. They are a source of data about the quality of the product for TQM quality management systems and MES production management. Their rapid development has been noticeable both in the scope of the realized control functions and the efficiency of image analysis. The development of 2D image analysis methods and the development of 3D image acquisition and analysis methods have resulted in the expansion of the number of measurement and control tasks that can be performed in industrial conditions. A simultaneous increase in the set of controlled parameters and a reduction in inspection time were observed. The automation of inspection operations using vision systems ensures the maintenance of the repeatability of the assessment of parameters describing the quality of a product and improves the effectiveness of the evaluation in relation to a visual inspection carried out by operators. The development of vision systems toward equipping them with I/O systems, communication protocol HMI interfaces, and databases makes it possible to integrate them directly into industrial automation systems and quality and production tracking and management systems.

Building an industrial product vision system for 2D imaging involves selecting a sensor array and optics to achieve the desired imaging field and measurement resolution. Based on research and engineering knowledge, quality control and measurement algorithms can be implemented for the system. For 3D imaging systems, however, there are some limitations due to the availability of hardware such as high-resolution ToF arrays. Limitations also arise from the imaging method itself. For each inspection task, a feasibility study of the task must be carried out and a suitable imaging method selected (Figure 6). When building hybrid imaging systems, the limitations of all methods used in the construction of the product image must be taken into account.

The increasing use of vision systems in quality control systems results from the dynamic growth observed in many fields of science and technology. The most significant influence on the development of all imaging techniques and their use in systems of product quality control are as follows:

-

Development of image registration methods is defined by the development and availability of a wide range of 2D sensors allowing for the registration of visible light, infrared, and ultraviolet including hyperspectral cameras after combining the fields. The development of 3D imaging technologies include laser triangulation, stereovision, structured light, TOF, SFF methods, and stereo photometry;

-

Developments in optics result in lenses for different sensor formats that also enable imaging at various resolutions and the availability of lenses specialized for post-measurement tasks (e.g., telecentric, custom-made for a selected field of view);

-

Development of technical lighting methods and lightning sources for vision systems. The large availability of illuminators allows for the selection of a solution to the control and measurement task;

-

Development of processor technology and communication interfaces including in industrial systems, mainly USB Vision, Giga Vision, and Camera Link technologies, allow for the increase in the frequency of the data registration and analysis;

-

Development of image analysis algorithms available in various types of post-programming for building industrial vision projects and in the form of vision libraries available in the currently used programming languages.

In what has been described as Industry 4.0, there has been a strong emphasis on introducing cyber-physical systems to monitor production processes. The aim is to create a virtual copy of real objects for data analysis and decision-making. In the Internet of Things, cyber-physical systems make it possible to build communication links that allow different systems and people to work together. At the same time, these activities require large financial investments (e.g., introducing surveillance and production control systems and constructing IT infrastructure).

It seems that the manufacturing industry is moving toward developing Industry 5.0, which will be autonomous and extensively use human–machine interfaces while at the same time placing great emphasis on preventing the creation of waste in production structures, the use of recycling, and at the same time, significantly extending the product life cycle.

Both increasing production efficiency and enhancing the product life cycle by increasing product quality are linked to developing control and measurement systems. This means an even greater saturation of production lines with equipment that enables multi-criteria evaluation of the manufactured product.

The most significant development in the construction of industrial quality control systems will be seen in the high-speed inspection; multispectral/hyperspectral imaging in defect/flaw detection [56]. The use of 3D imaging methods in multi-dimensional product inspection tasks is also growing. Additionally emerging in industrial needs is the construction of hybrid inspection and measurement systems that combine different imaging methods (e.g., 2D imaging combined with 3D images and multidimensional product image analysis).

Vision system implementations can be divided into two groups of applications. The first group includes simpler applications based on the use of off-the-shelf systems supplied by manufacturers. These applications are based on using smart cameras or vision sensors with varying degrees of sophistication. For example, these could be systems for reading barcodes or OCR characters. These systems are optimized by manufacturers and work very well in many industries. They are massively deployed on production lines and, in most applications, require no maintenance.

The second group of vision systems includes custom-built systems designed for specific inspection and post-measurement tasks. However, the effectiveness of these systems is strongly dependent on the cooperation between the company building such a system and the company using it. In preparing such a vision system, which is often very innovative and uniquely designed, a very good exchange of information between the designer and the user of the vision system is needed.

The first step needed to prepare an effective manufacturing quality control system is to conduct preliminary studies and develop a ‘feasibility study of a system taking into account customer needs’. The completed research and the initial implementation quote should show the user to what extent the control task will be performed. At the same time, it should indicate the difficulties in implementing a vision system for a specific inspection task. Here, it should be made clear for what reason the developed vision system can only partially fulfill the customer’s requirements. The question here is what reasons may affect the system’s effectiveness.

One reason may be that not all new imaging technologies are ready for industrial applications. The problem may be the very different operating parameters of vision systems. For example, imaging resolutions that are too low (e.g., ToF imaging in industrial applications), is currently limited to a resolution of 640 × 480. This is still too low a resolution in many applications. Another reason is the lack of immunity of the cameras or imaging methods to industrial disturbances (e.g., operating temperature, vibrations, dust, and excessive humidity). Another reason is that the vision system is not equipped with communication interfaces for data exchange with industrial controllers. A seemingly minor difficulty involves the need to convert signals via intermediate devices, which increases the implementation costs while reducing the reliability of the production control system.

Another obstacle can be the attitude of the engineering staff at production plants. This is due to long-standing experience working with machinery and equipment suppliers for production lines. This is particularly important in the case of machine suppliers using vision systems. Too high a failure rate of the prepared solution or the inability to quickly rectify a fault in the operation of the vision system generates concrete financial losses. This situation has made many companies perceive that vision systems are challenging to maintain and are unreliable. Recipients are, therefore, cautious and view vision solutions with considerable incredulity. A big difference in the evaluation of the performance of laboratory and industrial systems is apparent here. For example, the 80–90% efficiency of AI-based production vision systems is still unacceptable in many industries. This is due to the possibility of misjudging more than 10% of production, which can be too great a financial loss.

Therefore, it is imperative to carry out preliminary studies and produce a report describing the system’s operating parameters and limitations. On this basis, it is possible to determine the conditions needed for the effective operation of the vision inspection system and to decide on its construction.

The subsequent work on implementing the vision inspection system is also based on cooperation between the client and the designer. This is due to the high variability in the relevant materials, technologies, and components observed on the production lines. All of this information should be fed back to the system designer, who must incorporate this information into the solution on an ongoing basis. Implementing a vision system on a production line very often requires not only the commissioning of a vision system, but also the updating of the operating programs of many devices (e.g., feeder machines or product sorting systems).

The final success of implementing innovative and cutting-edge production vision systems is therefore dependent on many parameters and the cooperation of the engineers developing the expected solution.

4. Conclusions

We found a large body of information on various solutions in the literature on imaging technologies and image analysis. The direction of developing quality control systems using imaging technologies appears to be very dynamic. This is evident in many scientific publications, particularly regarding the combination of imaging technologies, the use of new types of matrices, the use of new lighting technologies, and the use of innovations in applied optics. In image analysis methods, a very dynamic development of algorithms based on artificial intelligence is evident. However, many industrial applications are still based on conventional algorithms that have been used for many years in vision systems.

For the use of vision-based inspection systems in standard applications, hardware and software solutions enable the inspection process to be carried out with almost 100% efficiency.

However, such a high efficiency cannot be achieved for inspection tasks with a higher degree of complexity. This is particularly noticeable in applications where new hardware and software solutions are being developed in preparation for innovative solutions. Improving the performance of a vision-based control system takes place gradually in the improvement and optimization of the implemented solutions.

This is particularly important in industrial settings and in the presence of variable disruptions resulting from production lines. A solution prepared even for the same industry but implemented on a different production line often requires a fundamental redesign. Due caution must therefore be exercised, and the operating conditions of the vision-based production control system must be well-recognized.

However, it seems that regardless of the strategy to be adopted in the further development of production processes (i.e., Industry 4.0, Industry 5.0, Smart Industry, or some new concept), vision systems will be the solution for observing the product and assessing its manufacturing parameters. Their most significant advantage is combining different product parameter information in the image, resulting from the ability to combine imaging methods. At the same time, it is possible to determine multiple product parameters from the analysis of a single image.