Development of a Didactic Plant for Material Classification Using an Intelligent Sensor ^†

Abstract

Object tracking and recognition are key technologies in Industry 4.0, enabling automated systems to perform tasks such as quality control, defect detection, material identification, and process monitoring. Many industrial vision solutions rely on high-end hardware, centralized processing units, or cloud-based services, which increase the cost of implementation and restrict their adoption in educational environments. For laboratories that already have conventional automation equipment and lack the resources to acquire such commercial intelligent sensing systems, such a limitation is highly pertinent. This work presents a low-cost didactic platform oriented toward Industry 4.0 education, integrating embedded computer vision with legacy programmable logic controllers. The proposed system performs classification tasks locally by means of a Faster Objects, More Objects (FOMO) network, reducing latency, minimizing infrastructure requirements and eliminating the need for external servers. In addition to cost reduction, this approach increases the technological lifespan of existing equipment, allowing legacy controllers to remain useful in activities involving modern concepts of artificial intelligence. The device communicates with the PLC through TCP/IP and also supports digital and analog outputs, providing compatibility with more traditional industrial interfaces. Two vision models were evaluated: the cylinder model achieved an F1-score of 99.47% on the test dataset with an inference time of approximately 220 ms, while the object model achieved an F1-score of 96.41% with an inference time of approximately 300 ms. The cylinder configuration was selected for classroom implementation because it combines vision-based color identification with conventional sensor-based height detection. Two student groups successfully implemented and operated the proposed classification process. Overall, this work highlights the potential of accessible AI-powered sensors to democratize intelligent automation, reduce barriers to Industry 4.0 education and promote the reuse of existing automation infrastructure.

1. Introduction

Industry 4.0 (I4.0) represents a significant shift in the manufacturing paradigm, characterized by the integration of digital technologies, connectivity and cyber–physical systems capable of operating autonomously. Since its introduction as an industrial strategy in Germany, I4.0 has been reshaping production chains, business models and decision-making processes through the incorporation of technologies such as the Internet of Things (IoT), advanced data analytics, and the integration between machines and computing infrastructures [1,2]. In this context, production systems increasingly rely on real-time data to optimize their operations, anticipate failures, and improve manufacturing flexibility.

Within this scenario, distributed computing technologies, particularly edge computing, have gained prominence. They shift data processing from remote or centralized infrastructures toward devices located near the data sources, reducing communication latency, bandwidth requirements, and dependence on external computational resources [3,4]. These characteristics are especially relevant in industrial applications in which timely responses are required, such as visual inspection, collaborative robot control, and predictive maintenance [5,6].

Embedded computer vision systems are particularly pertinent in this context because they enable the extraction of complex object features without physical contact and can support applications involving quality inspection, object detection and classification, material identification, and process monitoring. Advances in low-cost embedded platforms have enabled computer vision and machine-learning inference to be performed directly on resource-constrained devices, expanding the applicability of artificial intelligence (AI) in industrial and educational environments [7,8].

Despite these advances, adopting machine vision and intelligent sensing technologies remains challenging in environments with limited financial and computational resources. Commercial machine vision solutions often require dedicated industrial hardware, centralized processing units, or external computational infrastructure. However, acquisition, deployment and maintenance costs may restrict their adoption in educational laboratories and small-scale automation environments [9]. At the same time, external processing infrastructure may increase system complexity and hinder integration with existing automation equipment.

Several studies have explored complementary approaches for introducing Industry 4.0 technologies into educational and legacy automation environments. Pugliese et al. [10] developed a low-cost didactic plant based on microcontrollers and ultrasonic sensors, while Pajpach et al. [11] proposed an educational platform using machine learning and computer vision for quality inspection. At the system level, Krushnan and Schrödel [12] developed a low-cost laboratory-scale Industry 4.0 plant using industrial components, whereas Madera-Bermeo et al. [13] modernized an existing learning factory through IIoT, MQTT, NFC-based traceability, and enterprise-level integration. From a retrofit perspective, Di Carlo et al. [14] proposed a framework for modernizing existing process plants instead of replacing them. However, limited attention has been given to combining the reuse of an existing PLC-based didactic plant with low-cost embedded AI sensing, local inference, conventional industrial sensors, and classroom application within the same platform.

In view of this scenario, this work proposes a low-cost didactic plant oriented toward Industry 4.0 that integrates an intelligent embedded computer vision sensor with existing industrial automation equipment. Rather than replacing the installed infrastructure, the proposed approach incorporates local AI-based processing and multiple communication interfaces into the existing system, thereby extending its technological lifespan and enabling conventional automation equipment to be used in activities involving modern Industry 4.0 technologies.

The main contributions of this work are: (i) the development of a low-cost intelligent vision sensor capable of performing embedded object detection and classification without requiring external computational resources; (ii) the integration of the intelligent sensor with existing PLC-based automation systems through TCP/IP communication, digital I/O and analog signaling; (iii) the implementation of a modular didactic platform that combines conventional industrial automation with embedded artificial intelligence; and (iv) the development of an architecture that enables existing automation infrastructure to be reused and technologically extended rather than replaced.

2. Materials and Methods

2.1. Didactic Plant Architecture

The developed didactic plant combines conventional industrial automation with embedded computer vision. The existing system comprises conveyor belts, industrial sensors, pneumatic actuators and a PLC, while the intelligent sensing module performs image acquisition, preprocessing, object detection and classification locally.

The overall architecture is shown in Figure 1. The ESP32-S3 processes images acquired by the OV5640 camera and provides the classification results to the PLC through TCP/IP, digital signals, or an analog interface. The PLC combines this information with conventional sensor signals to execute the plant control logic.

Figure 1. Overall architecture of the proposed didactic plant, showing the embedded computer vision sensor, local inference, communication interfaces, PLC, conventional sensors and actuators.

2.2. Intelligent Vision Sensor and Local Inference

The intelligent vision sensor was implemented using an ESP32-S3 N16R8 with 8 MB of PSRAM and 16 MB of flash memory, connected to an OV5640 camera. Image acquisition, preprocessing and inference are performed locally on the microcontroller. Therefore, no external computer, cloud service, or inference server is required during normal operation. Network communication is used to transfer classification results and system information to the PLC and receive commands from the PLC.

2.3. Embedded Model

The embedded model was configured using the Faster Objects, More Objects (FOMO) architecture, employing a MobileNetV2 convolutional neural network. The model input was defined as 96 × 96 pixels using squash resizing. Unlike a conventional image classification network that assigns a single class to an entire image, the adopted FOMO architecture performs object localization and classification, allowing regions containing objects of interest to be identified within the captured frame. For this reason, the term object detection and classification is adopted throughout this work when referring to the operation performed by the embedded vision model, whereas the term material classification refers to the higher-level function performed by the didactic plant.

The system exhibits greater sensitivity to objects presenting well-defined contrast relative to the background and lower sensitivity under conditions involving nonuniform illumination, intense reflections, or colors similar to the background. These conditions require greater standardization during image acquisition.

Two distinct datasets were used to train separate instances of the embedded vision model. The cylinder dataset comprised three classes that referred to the colors of the objects (Blue, Green, and Red), with 378 images used for training and 94 for testing. The object dataset comprised White, Green, and Red Cubes, as well as Green and Red Pyramids, with 450 images used for training and 101 for testing. For the cylinder configuration, the vision model classifies only object color, while height is determined by capacitive and inductive sensors positioned at different vertical levels. The performance of both models was evaluated using precision, recall, F1-score, and processing time metrics, with the quantitative comparison presented in the Results section.

The image datasets were acquired directly from the didactic plant, using an embedded image-capture interface with Wi-Fi data transfer. As the images were captured while the objects moved forward and backward along the conveyor belt, multiple object positions were captured within the camera field of view. The dataset was initially divided into training and testing subsets using an approximately 80/20 ratio for each class. Furthermore, during model training, 20% of the training subset was internally reserved for validation. The partitioning was performed at the image level rather than by physical object or acquisition session, which means that different images of the same physical objects acquired under similar experimental conditions may appear across the subsets.

2.4. PLC Communication and Electrical Interface

To ensure compatibility with existing PLC-based systems, TCP/IP communication was adopted as the primary network interface. The sensor establishes a TCP connection with the PLC and sequentially transmits structured information generated by the embedded inference process. This communication scheme enables intelligent sensing information to be incorporated into existing automation architectures without requiring replacement of the controller.

In addition to TCP/IP communication, digital input and output signals are employed for direct interaction with the automation system. These signals provide a conventional interface that can be used independently of higher-level network communication. An analog communication channel was also implemented based on the conversion of a PWM signal generated by the ESP32-S3. The PWM signal is transmitted through fast-switching optocouplers and subsequently filtered to obtain a continuous voltage level proportional to the identified class. A second-order passive RC filter is employed for this purpose.

2.5. Embedded Firmware

The behavior of the intelligent sensing module is coordinated by an embedded firmware responsible for image acquisition, processing, inference, user interaction, and communication with the automation system. It was structured in a modular manner to facilitate maintenance, testing, and future expansion.

The embedded firmware is organized into initialization, inference, user-interface and integration functions. System peripherals, including the camera, OLED display, digital I/O, PWM output and communication interfaces, are configured during initialization. Captured images are processed locally by the FOMO model, either continuously or upon an external trigger. A state-machine-based interface allows the selection of operating modes and communication settings. The classification results are then provided to the PLC through digital outputs, the analog interface, and TCP/IP communication.

2.6. Classroom Experimental Procedure

The plant was subsequently applied in a laboratory class with two student groups. The activity required the students to develop and implement the control logic of the complete classification process. The system combined vision-based identification with conventional sensor-based detection to automatically route different objects to predefined outputs.

The students configured the PLC, implemented the sequential control logic and integrated the classification sensor with the controller. The implemented control logic also included operational constraints ensuring correct positioning before pneumatic actuation and handling cases in which an object was not initially detected by the classification sensor.

3. Results and Discussion

Figure 2 presents the final plant and its main components integrated into the system. It consists of two conveyor belts, each equipped with an optical sensor, an inductive sensor, a capacitive sensor and a pneumatic actuator. The computer vision sensor is positioned at the beginning of the first conveyor belt.

Figure 2. Developed didactic plant.

Two embedded vision models were evaluated using the cylinder and object datasets. Table 1 summarizes the validation and test results. For the cylinder model, the validation F1-score was 99.35%, while the test F1-score reached 99.47%. The object model achieved F1-scores of 94.81% and 96.41% for validation and test, respectively. In both test sets, the model achieved 100% precision, indicating that the main source of error was missed detections rather than incorrect class assignments.

Table 1. Performance comparison of the two embedded vision models.

Model	Dataset	Precision	Recall	F1-Score	Inference
Cylinders	Validation	100.00%	98.70%	99.35%	∼220 ms
Cylinders	Test	100.00%	98.94%	99.47%	∼220 ms
Objects	Validation	100.00%	90.12%	94.81%	∼300 ms
Objects	Test	100.00%	93.07%	96.41%	∼300 ms

The normalized confusion matrices for both test datasets are shown in Figure 3. The cylinder model exhibited only one missed detection in the Green class, while the object model showed a larger number of missed detections, particularly for the White Cube class. No relevant confusion between object classes was observed, with most errors corresponding to objects classified as background.

Figure 3. Normalized confusion matrices obtained for the test datasets: (a) cylinder classification model and (b) object classification model.

The cylinder model presented both higher detection performance and lower inference time than the object model, which can be attributed to the simpler classification task, since the cylinder model distinguishes objects by color, whereas the second model must distinguish combinations of color and geometry. For the object dataset, the lowest test recall was observed for the White Cube (82.35%), while the Green Pyramid achieved 100% recall, and the overall F1-score remained above 96% on the test dataset.

Considering the intended educational application, the cylinder configuration was selected for implementation in the laboratory activity. In addition to its higher performance and lower inference time, this configuration provides a more representative example of heterogeneous industrial sensing: the embedded vision system determines object color, while conventional capacitive and inductive sensors positioned at different heights provide dimensional information.

The selected configuration was subsequently applied in a classroom laboratory activity. Two student groups followed the proposed experimental procedure and successfully operated the material classification system. The activity allowed students to observe the interaction between embedded artificial intelligence, conventional industrial sensors, PLC-based control and the physical sorting process within the same platform. However, since the classroom evaluation involved only two student groups, these results should be interpreted as a feasibility demonstration rather than as an assessment of learning effectiveness.

Throughout the experiments, communication between the embedded sensor and the industrial controller remained stable and the classification results were correctly incorporated into the PLC control logic. The measured inference time of approximately 220 ms was compatible with the operating speed of the didactic plant, enabling reliable execution of the proposed classification process.

Table 2 highlights that the investigated approaches address complementary aspects of Industry 4.0 modernization. While previous studies demonstrate the feasibility of retrofitting existing equipment or developing educational platforms with modern communication and automation technologies, the proposed system combines these characteristics with embedded artificial intelligence, allowing an existing PLC-based plant to incorporate AI-assisted sensing without replacing the controller or requiring an external inference server.

Table 2. Qualitative comparison with related Industry 4.0 approaches.

Work	Existing Plant Reuse	Educational Application	Embedded AI	Local AI Inference
Di Carlo et al. [14]	Yes	No	No	No
Madera-Bermeo et al. [13]	Yes	Yes	No	No
Krushnan and Schrödel [12]	No	Yes	No	No
Proposed system	Yes	Yes	Yes	Yes

4. Conclusions

This work presented a low-cost didactic platform that extends an existing PLC-based automation system through an embedded intelligent vision sensor capable of performing object detection and classification locally. The proposed architecture preserves the conventional industrial control infrastructure while introducing embedded artificial intelligence, multiple communication interfaces and complementary conventional sensing.

Two FOMO-based models were evaluated. The cylinder model achieved an F1-score of 99.47% on the test dataset with an inference time of approximately 220 ms, while the object model achieved an F1-score of 96.41% with an inference time of approximately 300 ms. Based on its higher performance, lower inference time and ability to complement conventional sensor-based height detection, the cylinder configuration was selected for the classroom implementation.

The complete system operated reliably with the existing PLC, and the classification information was successfully incorporated into the automation logic. The platform was subsequently employed in a laboratory activity in which two student groups successfully implemented and operated the proposed classification process. These results demonstrate the feasibility of combining embedded AI, conventional industrial sensing and legacy automation equipment within the same educational environment, extending the technological lifespan of existing laboratory infrastructure.