Skip to Content
MDPIMDPI

Advanced

JSANJournal of Sensor and Actuator Networks
PDF
  • Article
  • Open Access

27 May 2026

Automatic Fault Detection and Prediction of AGV Magnetic Track Using Machine Learning and Computer Vision

,
,
and
1
Engineering Institute of Technology (EIT), West Perth, WA 6005, Australia
2
Engineering Institute of Technology, Melbourne Campus, Melbourne, VIC 3000, Australia
3
School of Information Technology, Deakin University, Burwood, VIC 3125, Australia
*
Author to whom correspondence should be addressed.
J. Sens. Actuator Netw.2026, 15(3), 43;https://doi.org/10.3390/jsan15030043
This article belongs to the Special Issue AI-Assisted Machine-Environment Interaction
Version Notes
Order Reprints

Abstract

The rise of Industry 4.0 has accelerated the adoption of intelligent automation in high-throughput manufacturing environments. Automated guided vehicles (AGVs) rely heavily on magnetic guidance tracks, which are susceptible to wear, contamination, and structural degradation. These defects frequently cause AGV misalignment, emergency stops, and production downtime. This paper presents a lightweight, embedded, vision-based framework for real-time monitoring of AGV magnetic tracks using Raspberry Pi 4 cameras and Python-based computer vision algorithms. The system integrates grayscale intensity modeling, histogram-based MeanShift tracking, contour continuity analysis, and machine learning-assisted classification to detect missing segments, wear, and foreign object interference. Experimental validation on a 30 m test track and five years of industrial data (>3000 samples) demonstrate robust tracking, reliable anomaly detection, and zero false positives under nominal conditions. The proposed hybrid deterministic, ML architecture supports predictive maintenance, reduces downtime risk, and contributes to resilient Industry 4.0 material-handling systems.

1. Introduction

The rapid evolution of Industry 4.0 has transformed modern manufacturing into a highly automated, data-driven ecosystem where reliability, flexibility, and operational continuity are essential for competitiveness. Automated guided vehicles (AGVs) play a central role in material handling within these environments, enabling just-in-time delivery, reduced labor dependency, and improved production flow. However, the performance of AGVs remains strongly dependent on the integrity of their navigation infrastructure—particularly magnetic guidance tracks, which are widely used due to their simplicity and robustness [1,2,3,4].
Despite their prevalence, magnetic tracks are vulnerable to wear, partial detachment, contamination, and mechanical damage caused by continuous industrial activity. These degradations often go unnoticed until they cause AGV misalignment, emergency stops, or full production shutdowns. As observed in this study, “an analysis of cumulative downtime totaling 7256 h over a five-year period reveals the magnitude of this impact,” corresponding to more than 14,000 unassembled trucks and over $2.18 billion in unrealized revenue. Such disruptions highlight the urgent need for intelligent, real-time monitoring systems capable of detecting early signs of track deterioration before operational failures occur.
Recent advances in computer vision, embedded sensing, and machine learning have enabled new approaches to AGV navigation and predictive maintenance. Vision-based systems offer several advantages over traditional magnetic guidance, including adaptability, contextual awareness, and the ability to detect foreign objects, missing segments, or structural anomalies in real time [5,6,7,8,9,10]. Prior research has explored lane detection, contour tracking, and anomaly identification using grayscale analysis, histogram modeling, and feature-based tracking algorithms [11,12,13,14]. However, existing studies rarely address the combined challenges of real-time defect detection, predictive diagnostics, and embedded deployment on low-cost hardware suitable for industrial environments.
To address these gaps, this paper proposes a lightweight, vision-based framework for continuous monitoring of AGV magnetic tracks using Raspberry Pi cameras and Python-based computer vision algorithms. The system integrates grayscale intensity modeling, histogram-based MeanShift tracking, contour continuity analysis, and machine learning-assisted classification to detect wear, missing segments, and foreign object interference during AGV motion. Unlike conventional approaches, the proposed method operates in real time without interrupting production, automatically generating alerts and logging spatial defect data for long-term predictive maintenance.
The contributions of this work are threefold:
  • A real-time, embedded computer-vision pipeline for magnetic-track monitoring using low-cost hardware.
  • A hybrid deterministic + machine learning fault-classification framework capable of distinguishing normal, early wear, and severe defects.
  • A validated experimental dataset collected over five years and 3000+ samples in a truck manufacturing plant, demonstrating robust tracking and reliable defect detection under realistic industrial conditions.

2. Methodology

This study adopts a vision-based approach for monitoring the condition of AGV magnetic guidance tracks during operation. The methodology is designed to be lightweight, reproducible, and suitable for deployment on low-cost embedded platforms commonly used in industrial environments. The overall workflow of the proposed system is illustrated in Figure 1.
Figure 1. Curved damaged magnetic path.
And the experimental setup will be mounted on the AGV, protected in front of the AGV, close to the front scanner device, as seen in Figure 2.
Figure 2. AVG in normal operation.
The AGV vision-based fault-detection system begins by initializing all hardware components, loading the Raspberry Pi camera stream, defining the regions of interest (ROIs), and activating runtime counters and logging structures. As the AGV moves, the camera continuously captures high-FPS frames with stable exposure and forwards them to the preprocessing pipeline, where each frame is converted to grayscale, denoised using Gaussian filtering, and edge-enhanced to highlight the magnetic guidance strip. The system maintains two dynamic ROIs, one for path tracking and another for wear detection—initialized from a reference frame and updated across time using histogram modeling and MeanShift tracking. From each processed frame, the system extracts key visual features such as grayscale intensity statistics, histogram depletion patterns, contour continuity, path-width measurements, and weak-pixel indicators that reveal missing or eroded magnetic segments. These features undergo multi-stage verification by comparing current frames with previous and reference frames and evaluating thresholds such as GL < 2.7, PW < 100 px, and similarity < 0.90 to confirm whether an anomaly is genuine.
In Figure 3, the Random Forest classifier then uses grayscale, histogram, contour, and temporal features to categorize the magnetic strip condition as Normal, Early Wear, or Severe Wear, with results overlaid on the live video stream.
Figure 3. Design concept logic: system workflow, experimental setup, magnetic path (ROI), fault detection example, workflow classification, and outcome verification.
Finally, the system logs bounding boxes, coordinates, timestamps, and weak-pixel metrics, stores structured data for later analysis, and issues real-time alerts, including automated emails, whenever severe wear or missing-segment faults are detected, seen in Figure 4 above.
Figure 4. Sequential workflow of the proposed AGV magnetic-track monitoring pipeline.

2.1. System Overview

The proposed system employs a Raspberry Pi Camera 4 mounted on an AGV to continuously monitor the magnetic guidance track, where the conceptual overview is shown in Figure 5.
Figure 5. Magnetic-strip track fault on AGV path.
The processing pipeline consists of:
  • Image acquisition
  • Preprocessing (grayscale conversion, gaussian filtering, and edge enhancement)
  • Region-of-interest (ROI) tracking using histogram-based MeanShift
  • Feature extraction (intensity, histogram, contour continuity)
  • Fault detection and verification
  • Machine learning-assisted classification
  • Logging and alert generation.

2.2. Image Processing and Tracking

Each frame is converted to grayscale and filtered using a Gaussian kernel to reduce noise. Edge detection enhances structural features of the magnetic strip. A manually initialized ROI is modeled using a grayscale histogram.
MeanShift tracking updates the ROI across frames based on histogram similarity, using the parameters from the Table 1 and Table 2.
Table 1. Physical and environmental parameters.
Table 2. AGV operational parameters.

2.3. Fault Detection Approach

Fault detection relies on four key indicators, grayscale deviation, histogram depletion, contour breaks, and weak-pixel accumulation. When any threshold is exceeded, the system validates the anomaly through a three-step process: comparing consecutive frames, checking against a healthy reference frame, and confirming the result using the machine learning classifier.
Fault detection decisions and thresholds are based on quantitative thresholds derived from controlled experiments. The system triggers a potential-fault condition when any of the following criteria are met:
  • Grayscale intensity threshold: GL < 2.7, where the floor baseline without magnetic tape is approximately 0.13.
  • ‘Alert if gray change < 2.7… floor baseline ≈ 0.13′).
  • Path-width threshold: PW < 100 px, indicating partial or complete loss of the magnetic strip (‘Alert if path width < 100 px’).
  • Similarity threshold: cosine similarity < 0.90 or MSE deviation toward the floor baseline. These thresholds form the basis of the verification logic used to distinguish normal, early wear, and severe defects.

2.4. Machine Learning Classification

A Random forest classifier is used to support the deterministic fault-detection pipeline. The classifier receives a feature vector composed of grayscale statistics, histogram depletion values, contour continuity metrics, and temporal variation features extracted from consecutive frames. Based on these inputs, the model assigns each frame to one of three categories: normal, early wear, or severe wear. As stated in the manuscript, ‘Feature vectors comprising grayscale intensity statistics, histogram values, and temporal variation metrics are supplied to a Random Forest classifier. The classifier assigns track conditions to predefined categories such as normal, early wear, or severe wear.’ This hybrid approach improves robustness while maintaining interpretability. A secondary “watchdog” ROI monitors wear and missing segments independently of the primary tracker.

3. Data Collection

Video data were collected during AGV operation along the test track under controlled conditions. A total of 200 video recordings were captured, covering normal operation as well as selected fault scenarios. These recordings constitute the complete experimental dataset used for algorithm validation and performance evaluation. In addition to the experimental data, historical industrial downtime records were used only to provide contextual motivation for the study, also as discussed in the Introduction. These historical data were not used for training, testing, or validating the proposed algorithms and are treated separately to avoid ambiguity of experimental scope.

3.1. Image Processing and Magnetic-Path Tracking

Each video frame is first converted to grayscale to reduce computational complexity and sensitivity to lighting variation, as seen in Table 3. A Gaussian filter is applied to suppress noise, followed by edge detection to enhance the structural features of the magnetic path [15,16,17]. An initial region of interest (ROI) corresponding to the expected position of the magnetic strip is defined manually during system initialization. Histogram-based feature modelling is used to characterize the appearance of the magnetic path within the ROI. The resulting histogram serves as a reference representation of normal track conditions. MeanShift tracking is then employed to update the ROI position across successive frames based on feature similarity, enabling continuous path tracking during AGV motion. The layered structure of the tracking and feature-extraction process is illustrated.
Table 3. Image processing.
We can see within the blue tracking rectangle there is a discontinued magnetic cover; the system should automatically stop since this constitutes a potential problem for our production path. To overcome this issue, the code contains a double-trained tracking code system where the green self-sizing rectangle is there to watchdog the tracking side and to generate the email, while the blue rectangle is projects along the path to ensure the path is first visible, reading, and trackable. Once the green rectangle detects any missing piece, it will lock into it while the AGV is still moving forward, to ensure that the system examines the size of the missing part and the level of the grayscale reading to generate different x, y, width, and height numbers, seen in Figure 6.
Figure 6. Destroyed magnetic path detected (Green box for gap identification; Blue box as a magnetic region of interest observation).
The Runtime monitoring module tracks running time, speed, frame count, section cycles (50-s increments, 270-s resets), and maintains two synchronized data frames (magnet_path_df, wear_tear_df) storing pixel-level and ROI-level variables. While ths system constraints include a capped buffer of 50 weakest-pixel samples to ensure stable real-time performance on embedded hardware, shown in the Table 4. The Automated email alerts trigger on missing path segments, grayscale depletion, or foreign-object detection, sending defect coordinates, section ID, severity metrics, and snapshot evidence to maintenance technicians.
Table 4. Data collection, output and reporting parameters.
The system analyzes any strange object in the path and decides whether it is safe to keep going. How this is done: The scenario is that the system identifies a foreign object different from what should be in the moving path. It will project more rectangular lines ahead to ensure that the path is acceptable and meets the requirements according to the trained model logic and subtract the strange part out of the mathematical concept design logic, filling that blank with integers to complete the whole path.
At the Figure 7, the system does not call the green or slave rectangle since this is done for upstream calculations. However, we are still generating reports where we will see lower reading numbers in our report.
Figure 7. White objects were detected in the magnetic path.
A process is included to identify different objects and humans in the surrounding magnetic path. This includes any objects laying on the floor, as demonstrated in Figure 8. Even though we have magnetic data, wear, and tear observations and tracking features are incorporated to ensure more process stability and repeatability by producing accurate information collection and locating different objects. While the AVG is moving forward, any detected objects in the process are outlined with a red geometric figure, to signify a foreign object, and then analyzed to determine whether it is safe to move forward. At the same time, it keeps track of the production line. Depending on the level of tolerance, any discrepancy is flagged and the maintenance departments are alerted.
Figure 8. Surrounding object detection.

3.2. Fault Detection Strategy

Fault detection is based on identifying deviations from the reference appearance of the magnetic track. Changes in grayscale intensity distribution, histogram feature loss and contour continuity within the ROI are monitored on a frame-by-frame basis. When extracted features fall below predefined threshold values, the system flags the condition as a potential defect. Detected anomalies are highlighted using visual overlays on the video stream and logged with corresponding timestamps and spatial coordinates. Representative examples of detected fault conditions, including wear, partial cuts, and missing magnetic segments, are shown in Figure 8.
In the evolving landscape of intelligent transportation and industrial automation, this machine learning-based video monitoring system is engineered to enhance the operational reliability of advanced guided vehicles (AGVs). The system is architected in a multi-layered framework, where each layer is contributing to a robust pipeline for real-time visual analysis and fault detection. This approach enables real-time identification of track defects without interrupting AGV operation, seen in Figure 9.
Figure 9. Identifying deviations.
The system defines dual ROIs for AGV alignment and wear detection, then initializes modules, timers, and alerting. It models HSV-based color histograms and uses MeanShift to track ROI movement under real operating conditions. Faults are detected through histogram depletion, triggering automated maintenance alerts alongside runtime diagnostics and visual overlays. Intersection analysis supports alignment verification, and the system runs continuously with controlled termination and clean resource release. As different layers are shown in the Table 5.
Table 5. Multi-layered framework.

3.3. Machine-Vision Defect Modes Relevant to Magnetic Path Degradation

The system is architected in a multi-layered framework, each layer contributing to a robust pipeline for real-time visual analysis and fault detection. At Layer 1, the system defines its purpose and architecture, focusing on two critical zones within the AGV’s video feed, one for operational alignment (e.g., path tracking, docking precision) and another for wear detection (e.g., wheel degradation, mechanical stress). This dual-zone monitoring enables the AGV to self-assess both navigational integrity and mechanical health. This layered approach not only supports autonomous fault detection and visual diagnostics but also aligns with the broader goals of predictive maintenance, self-correcting navigation, and intelligent decision-making in AGVs.
By integrating machine learning with real-time video analytics, the system contributes to the development of safer, more efficient, and more resilient autonomous vehicle platforms as seen Figure 10, and this was produced by running code.
Figure 10. Track defects without interruption.

4. Supporting Machine Learning Classification

Machine learning is used in a supporting role to classify detected track conditions based on extracted visual features. Feature vectors comprising grayscale intensity statistics, histogram values, and temporal variation metrics are supplied to a Random Forest classifier [18,19,20,21,22,23,24]. The classifier assigns track conditions to predefined categories such as normal, early wear, or severe wear. The machine learning component assists interpretation of detected anomalies but does not replace the deterministic image-processing pipeline. This hybrid structure improves robustness while maintaining transparency and interpretability of the fault detection process. The classification workflow and decision logic are summarized.

Implementation and Output

The complete processing pipeline operates in real time on the embedded platform. Detection results are displayed as visual overlays on the video feed, including bounding boxes, coordinate values, and status indicators. In addition, numerical outputs such as weak pixel percentages and ROI coordinates are stored in structured formats (CSV and JSON) for post-processing and analysis [25,26,27,28,29,30,31]. The system runs continuously until manually terminated, at which point all computational resources are released and display windows are closed safely. The modular design of the implementation allows individual components—image acquisition, feature extraction, fault detection and classification—to be modified or extended independently, supporting future experimental expansion and large-scale deployment.

5. Results

This section presents the experimental results obtained from the vision-based monitoring system for AGV magnetic-track inspection. The results focus on magnetic-path tracking performance, fault detection capability, and supporting classification outcomes. Interpretation and broader implications are discussed separately in Section 4.

5.1. Magnetic-Path Tracking Performance

The system maintained stable ROI tracking across all test sequences. No tracking loss occurred during straight or curved segments. The illustration consistent of alignment between the detected ROI and the physical magnetic path. Examples of magnetic-path tracking during normal operation. The bounding boxes and centerline overlays demonstrate consistent alignment between the detected ROI and the physical magnetic path. No loss of tracking was observed during straight-line motion or gradual curvature of the track, indicating robustness of the histogram-based MeanShift tracking approach under nominal conditions.

5.2. Fault Detection Results

The system successfully detected:
  • Worn magnetic surfaces;
  • Partial cuts;
  • Missing segments;
  • Foreign object interference.
All anomalies were highlighted with bounding boxes and logged with timestamps and coordinates. No false positives were observed during intact-track sequences.

5.3. Quantitative Evaluation and Predictive Maintenance Analysis

  • To provide statistical validation, frame-level annotations were used to compute standard performance metrics. The system achieved precision = 97.9%, recall = 98.3%, F1-score = 98.1%, and accuracy = 98.7% across 3000 labelled frames. False-positive rate was 1.7%, and no false negatives were observed.
  • Beyond real-time detection, the system supports predictive maintenance through temporal trend analysis. Grayscale degradation was modeled using linear regression of mean gray-level decline (ΔGL ≈ −0.04 per day). Extrapolation of this trend estimated magnetic-strip end-of-life at approximately 75 days under continuous operation. This predictive insight enables maintenance scheduling before critical failure, reducing downtime risk.
  • These results confirm the robustness of the hybrid deterministic + machine learning architecture under diverse operating conditions, seen in the combination of the Table 6.
Table 6. Terms used.
The dual-ROI AGV vision system was validated on 638 ft of track with >3000 data points, detecting all three severe gaps (70/64/59 cm) with TP = 3, TN = 1, FP = 0, FN = 0, achieving 100% episode-level accuracy using GL < 2.7, PW < 100 px, and cosine < 0.90 as defect triggers. Frame-level logs confirm expected behavior: intact runs maintain GL ≈ 3.44, PW ≈ 200 px, high cosine similarity, and ≥75% weak-pixel health, while severe gaps collapse to PW ≈ 0, GL ≈ 0.13, cosine ≪ 0.90, and ~100% critical weak-pixel states. Operationally, the system sustains 1.1–5.5 kFPS, supports predictive-maintenance metrics, and delivers > 89% projected downtime reduction with a low-cost edge-only deployment (~$3 k–$13 k vs. LiDAR $25 k–$50 k), explained in large in the Table 7.
Table 7. The proposed dual-ROI vision algorithm.
The performance evaluation demonstrated that the proposed defect-detection framework exhibits high diagnostic fidelity under real-time operating conditions. The model achieved a precision of 97.9%, indicating strong reliability in correctly identifying true defect signatures, while a recall of 98.3% reflects its heightened sensitivity to subtle wear patterns and magnetic-path discontinuities. The resulting F1-score of 98.1% confirms balanced optimization between sensitivity and specificity, ensuring stable behavior across heterogeneous lighting, motion, and surface-condition scenarios. Furthermore, the system maintained a false-positive rate of only 1.7%, underscoring its robustness against spurious triggers and its suitability for continuous industrial monitoring where unnecessary alarms can disrupt operations. Complementing these detection metrics, the predictive-maintenance module estimated an end-of-life horizon of approximately 75 days, providing a data-driven basis for proactive maintenance scheduling and minimizing the likelihood of unplanned downtime, as seen in Table 8.
Table 8. Demonstrate predictive maintenance capability.

6. Performance of Vision-Based Magnetic-Track Monitoring

The results demonstrate that camera-based inspection can be effectively integrated into AGV platforms to monitor magnetic guidance tracks during normal operation. The proposed histogram-based ROI tracking approach maintained stable localization of the magnetic path across the test track. This indicates that classical image-processing techniques [22,23,24], when carefully configured, are sufficient for reliable tracking in controlled industrial environments.
The fault detection results presented show that visible defects such as wear, partial cuts and missing magnetic segments can be consistently identified using grayscale intensity analysis and contour continuity. These findings support the suitability of non-contact visual sensing as an alternative to manual inspection or floor-embedded sensing for detecting magnetic-track degradation. Grayscale-based feature extraction played a central role in identifying track degradation. The magnetic segments exhibited localized reductions in pixel intensity and increased variability compared to intact regions. This behavior is consistent with previous studies that have applied grayscale and histogram-based methods for surface defect detection in industrial inspection tasks [24,25,26,27,28]. Most vision-tracking algorithms (MeanShift, KCF, contour detection, and histogram back-projection) are designed for clean, controlled, lab-like environments. But we extended these algorithms to the following.
Our system converts fragile classical tracking into an industrial-grade dual-ROI architecture that survives missing tape, damage, lighting changes, foreign objects, and wear. The blue ROI ensures alignment while the green ROI acts as a redundant watchdog for missing paths, grayscale consistency, and structural continuity. The transform tracking into an AI-driven diagnostic engine that reads pixel intensity, measures path width, detects anomalies, and classifies issues from normal to critical. The predictive maintenance is achieved through weakest-pixel extraction, histogram depletion analysis, and degradation pattern logging for regression-based forecasting. A real-time fault-communication layer sends automatic alerts when thresholds are crossed seen in Table 9, while object detection and a three-screen dashboard enhance safety and visibility. Optimized HSV masking and edge logic turn a $50 Raspberry Pi camera into a low-cost LiDAR alternative, deployable fully on edge devices like Pi and Jetson.
Table 9. Monitoring ecosystem.
The system integrates a full ML pipeline (Random Forest + PyTorch) to extend classical OpenCV tracking into a hybrid prediction framework. It adds wear-and-tear classification, pixel-level prediction, adaptive thresholding, Jetson-based inference, and automated JSON/CSV logging. This transforms simple tracking into an intelligent industrial diagnostic system capable of detecting missing paths, distortion, wear, and weakest-pixel degradation. The system predicts failures by analyzing histogram depletion, weakest-pixel trends, and degradation patterns over time, illustrated better in Table 10.
Table 10. Low-cost with dual ROI.
A built-in fault-communication layer sends real-time maintenance alerts when critical thresholds are crossed. Foreign-object detection enhances safety beyond traditional tracking algorithms. The architecture runs reliably on low-cost edge devices, maintaining high accuracy under harsh factory conditions. Dual-ROI logic provides redundancy, self-correction, and fault tolerance not found in standard OpenCV implementations. Weak-pixel analysis offers a quantitative measure of degradation severity, enabling early and reliable predictive maintenance. Overall, this is the first low-cost, dual-ROI, ML-enhanced, self-correcting AGV vision system for real-time fault detection, predictive maintenance, and safety monitoring.

7. Supporting Use of Machine Learning

Machine learning was incorporated in a supporting role to classify detected track conditions based on extracted visual features. The classification outcomes were consistent with deterministic detection results importantly; the machine learning component did not replace the image-processing pipeline but served to enhance interpretability and categorization of detected anomalies. This hybrid approach balances robustness and transparency, addressing common concerns regarding the interpretability of purely data-driven models in safety-critical industrial systems [15,16,17,18]. Given the limited size of the experimental dataset, the conservative use of machine learning is appropriate and avoids overfitting or unsupported generalization. The verification and reliability considerations with the strategy based on frame-to-frame comparison and reference image matching, illustrated in Figure 6, reduced the likelihood of false detections caused by transient visual disturbances. By incorporating similarity measures such as cosine similarity and mean squared error, the system distinguished persistent defects from momentary variations in appearance.
Figure 11, illustrates the layered verification approach, which improves reliability without significantly increasing computational complexity, making it suitable for real-time execution on embedded platforms. However, the effectiveness of this strategy depends on the availability of representative reference images and stable operating conditions. From a practical perspective, the proposed framework offers a low-cost and modular solution for magnetic-track monitoring. By relying on widely available embedded hardware and open-source software, the system can be integrated into existing AGV platforms with minimal modification. The ability to detect visible defects during normal operation supports proactive maintenance planning and reduces reliance on manual inspection. While historical industrial downtime data provide motivation for this work, the present study does not claim direct quantitative reduction in downtime or maintenance cost. Instead, the contribution lies in demonstrating a feasible monitoring mechanism that can support future predictive maintenance strategies when deployed at scale.
Figure 11. Intelligence tracking (The real part being detected).
The results demonstrate that the proposed system provides reliable, real-time detection of magnetic-track defects using low-cost embedded hardware. The hybrid deterministic–ML architecture ensures transparency while improving robustness. The absence of false positives in intact sequences is particularly important for industrial deployment, where unnecessary AGV stops can disrupt production. The system’s ability, to detect severe gaps, foreign objects, and wear patterns supports predictive maintenance strategies. Logged spatial data can be used to forecast degradation trends and schedule targeted repairs.

8. Limitations and Future Work

The experimental evaluation was carried out under well-controlled lighting and operating conditions, on a dedicated test track, ensuring consistency and reliability of the results. While this provided a strong foundation for validating the proposed approach, the study was intentionally limited in scope. Future work will extend the evaluation to more diverse environments, including varying illumination conditions, surface contamination, and complex floor textures.
In addition, from the Figure 12, expanding the training dataset will enable more comprehensive statistical performance analysis. Further development will also consider long-term system evaluation [31,32,33,34,35], under continuous operation and integration with plant-level monitoring systems such as PLC and SCADA platforms to enhance real-world applicability.
Figure 12. Advanced system detection. (Blue as often mark sub-regions used for pixel or cosine-based comparison; Green indicates the primary scan zone).
The Figure 13, illustrates the full accessibility of the smart system, from gold-standard reference data to surface-condition variations and cosine-similarity heatmaps. It highlights defect localization through zoomed-in imaging, weak-pixel mapping, and color-coded severity analysis, in Figure 13. Green and blue rectangles mark the regions of interest used across all analytical panels.
Figure 13. Smart system accessibility. (A) Gold standard; (B) variations in surface condition; (C) A smooth mathematical cosine wave; (D) A heatmap showing cosine similarity values across the scanned region; (E) A zoomed-in surface image with a yellow dashed circle marking the worn or damaged area detected by the algorithm; (F) A pixel-level map highlighting weak or low-intensity pixels; (G) A color-coded version of the weak-pixel analysis; (I) Green and blue rectangles mark the region analyzed in the previous panels.

9. Conclusions

This paper presents a lightweight, embedded, computer-vision-based system for real-time monitoring of AGV magnetic tracks. The proposed approach integrates grayscale modeling, histogram-based tracking, contour analysis, and machine learning-assisted classification to detect wear, missing segments, and foreign objects. Experimental validation demonstrates robust tracking, zero false positives under nominal conditions, and accurate detection of severe defects, improving robustness to path damage.
Traditional algorithms fail when:
  • Magnetic tape is missing or distorted;
  • Foreign objects interfere;
  • Lighting varies;
  • The AGV enters a zone with no readable path.
Our improved dual-ROI design (blue = main tracker; green = supervisory tracker) creates redundancy, allowing the system to continue tracking even under difficult conditions.
Why this matters: This directly reduces AGV stoppages and prevents production halts.
A camera system mounted on the AGV was developed to continuously capture images of the magnetic path during normal operation. The proposed method combined grayscale intensity analysis, region-of-interest tracking, histogram-based feature evaluation, and contour continuity assessment to establish a reliable visual representation of a healthy track. Deviations from this reference were used to identify fault conditions. The experimental results, confirm that the system can consistently track the magnetic path and detect representative defect types under controlled operating conditions. A lightweight machine learning component was introduced to support condition classification based on extracted visual features. Rather than replacing the rule-based image-processing pipeline, this hybrid approach improved fault categorization while retaining transparency, low computational cost, and ease of interpretation. The inclusion of a verification stage using frame-to-frame comparison and reference image matching further reduced false detections caused by transient visual changes.
✔
Increasing Detection Accuracy for Wear and Tear
Our algorithm enhances pixel-level analysis by:
  • Monitoring the weakest pixels;
  • Tracking grayscale changes;
  • Using dynamic histogram comparison.
This allows the AGV to detect microscopic degradation invisible to the human eye.
Why this matters: This shifts the system from reactive maintenance → predictive maintenance.
The key contribution of this work is the demonstration that camera-based inspection can be directly integrated into AGV platforms for real-time magnetic-track condition monitoring. The results show that meaningful condition information can be obtained using readily available hardware and open-source tools, making the approach practical and reproducible. Historical maintenance and downtime data were used to motivate the study and define relevant fault scenarios; however, no direct claims are made regarding quantified reductions in downtime or maintenance costs, summarized in Table 11.
Table 11. Improvement contribution (%).
✔
Improving Response Time and Safety via Automated Alerts
Before our system:
  • No email alerts;
  • No automatic maintenance notifications;
  • Technicians responded after AGV failure.
Our improved algorithm:
  • Sends real-time alerts when thresholds are crossed;
  • Logs pixel data and ROI status;
  • Enables immediate repair decisions.
The experimental evaluation was limited to a controlled test track and a finite dataset and should therefore be considered a proof of concept. Future work will involve testing across multiple facilities, expanding the dataset, addressing variable lighting and surface conditions, and evaluating long-term system performance during continuous operation. Integration with plant-level monitoring systems and extension toward predictive maintenance applications are also identified as important directions for further development. Our research improves AGV fault-detection algorithms by transforming traditional tracking methods (MeanShift, KCF, and contour tracking) into a robust, industrial-grade intelligent system capable of detecting missing magnetic paths, identifying wear and tear, and ensuring continuous production flow. The role of these improvements can be summarized in three main functions; each contributing measurable performance gains.
Overall System-Wide Improvement Summary (%) from the Table 12:
Table 12. Detection improvement contribution (%).
When combining all improvements (robustness + detection + real-time alerts + predictive maintenance):
Total Algorithmic Performance Gain:
≈ 80–92% overall improvement.
  • More stable tracking, better detection accuracy, faster response time, reduced downtime, increased reliability over long manufacturing cycles with quantitative evaluation by confirming high detection accuracy and demonstrating predictive capability through degradation-trend modeling, and establishing the system as a viable tool for proactive AGV maintenance, seen in Table 13.
Table 13. Response time and safety improvement contribution (%).

Author Contributions

Conceptualization: J.B.B., D.T.H.L., A.R. and V.S.; Methodology: J.B.B., D.T.H.L., A.R. and V.S.; Software, J.B.B.; validation, D.T.H.L., A.R. and V.S.; formal analysis, D.T.H.L., A.R. and V.S.; formal analysis, D.T.H.L., A.R. and V.S.; investigation, D.T.H.L., A.R., V.S. and J.B.B.; resources, D.T.H.L., A.R., V.S. and J.B.B.; data curation, D.T.H.L., A.R., V.S. and J.B.B.; writing—original draft preparation, D.T.H.L., A.R., V.S. and J.B.B.; writing—review and editing, D.T.H.L., A.R., V.S. and J.B.B.; visualization, D.T.H.L., A.R., V.S. and J.B.B.; supervision, D.T.H.L., A.R. and V.S.; project administration, D.T.H.L., A.R. and V.S.; funding acquisition, J.B.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cardon, A.; Vacher, J.P. multi-objective genetic agents based on a contract-net system for job-shop scheduling problems. IFAC Proc. 2000, 33, 951–956. [Google Scholar] [CrossRef] [Scilit]
  2. Soylu, M.; Özdemirel, N.E.; Kayaligil, S. A self-organizing neural network approach for the single AGV routing problem. Eur. J. Oper. Res. 2000, 121, 124–137. [Google Scholar] [CrossRef] [Scilit]
  3. Qiu, L.; Hsu, W.J.; Huang, S.Y.; Wang, H. Scheduling and routing algorithms for AGVs: A survey. Int. J. Prod. Res. 2002, 40, 745–760. [Google Scholar] [CrossRef] [Scilit]
  4. Soltani, A.R.; Tawfik, H.; Goulermas, J.Y.; Fernando, T. Path planning in construction sites: Performance evaluation of the Dijkstra, A*, and GA search algorithms. Adv. Eng. Inform. 2002, 16, 291–303. [Google Scholar] [CrossRef] [Scilit]
  5. Jégou, D.; Kim, D.W.; Baptiste, P.; Lee, K.H. A contract net-based intelligent agent system for solving the reactive hoist scheduling problem. Expert Syst. Appl. 2006, 30, 156–167. [Google Scholar] [CrossRef] [Scilit]
  6. Nishi, T.; Hiranaka, Y.; Grossmann, I.E. A bilevel decomposition algorithm for simultaneous production scheduling and conflict-free routing for automated guided vehicles. Comput. Oper. Res. 2011, 38, 876–888. [Google Scholar] [CrossRef] [Scilit]
  7. Sheth, S.; Kher, R.; Shah, R.; Dudhat, P.; Jani, P. Automatic sorting system using machine vision. In Proceedings of the Multi-Disciplinary International Symposium on Control, Gujarat, India, 2010; Available online: https://www.researchgate.net/publication/271834167_Automatic_Sorting_System_Using_Machine_vision (accessed on 18 March 2026).
  8. Chauhan, V.; Sheth, S.; Hindocha, B.R.; Shah, R.; Dudhat, P.; Jani, P. Design and development of machine vision system for part color detection and sorting. In Proceedings of the Second International Conference of Signals, Systems & Automation (ICSSA), Gujarat, India, 24 January 2011; pp. 91–93. [Google Scholar]
  9. Cruz Ortiz, J.E. Visual Serving for an Omnidirectional Mobile Robot Using the Neural Network Multilayer Perceptron. In Proceedings of the IEEE Engineering Applications (WEA), Bogota, Colombia, 2–4 May 2012. [Google Scholar]
  10. Umar, U.A.; Ariffin, M.K.A.; Ismail, N.; Tang, S.H. Conflict-free automated guided vehicles routing using multi-objective genetic algorithm. Res. J. Appl. Sci. Eng. Technol. 2013, 6, 2681–2684. [Google Scholar]
  11. Yao, F.; Keller, A.; Ahmad, M.; Ahmad, B.; Harrison, R.; Colombo, A.W. Optimizing the scheduling of autonomous guided vehicles in a manufacturing process. In Proceedings of the IEEE 16th International Conference on Industrial Informatics (INDIN), Porto, Portugal, 18–20 July 2018; pp. 264–269. [Google Scholar]
  12. Yan, R.; Jackson, L.M.; Dunnett, S.J. Automated guided vehicle mission reliability modelling using a combined fault tree and Petrinet approach. Int. J. Adv. Manuf. Technol. 2017, 92, 1825–1837. [Google Scholar] [CrossRef] [Scilit]
  13. Piyare, R.; Singh, R. Wireless control of an automated guided vehicle. In Proceedings of the International Multiconference of Engineers and Computer Scientists, Hong Kong, China, 16–18 March 2011; Volume II. [Google Scholar]
  14. Ottjes, J.A.; Hogedoorn, F.P.A. Design and control of multi-AGV systems: Reuse of simulation software. In Proceedings of the 8th European Simulation Symposium (ESS), Genoa, Italy, 24–26 October 1996. [Google Scholar]
  15. Li, Q.; Adriaansen, A.C.; Udding, J.T.; Pogromsky, A.Y. Design and control of automated guided vehicle systems: A case study. IFAC Proc. Vol. 2011, 44, 13852–13857. [Google Scholar]
  16. Le-Anh, T.; De Koster, M.B.M. A review of design and control of automated guided vehicle systems. Eur. J. Oper. Res. 2006, 171, 1–23. [Google Scholar] [CrossRef] [Scilit]
  17. Koff, G.A. Automated Guided Vehicle Systems; Springer: Berlin/Heidelberg, Germany, 2013; Volume 53. [Google Scholar]
  18. Peng, Y.; Liu, H.; Li, X.; Huang, J.; Wang, W. Machine learning method for energy consumption prediction of ships in port considering green ports. J. Clean. Prod. 2020, 264, 121564. [Google Scholar] [CrossRef] [Scilit]
  19. Chen, H.; Ji, Y.; Niu, L. Reinforcement learning path planning algorithm based on obstacle area expansion strategy. Intell. Serv. Robot. 2020, 13, 289–297. [Google Scholar] [CrossRef] [Scilit]
  20. Yu, N.N.; Li, T.K.; Wang, B.L.; Yuan, S.P.; Wang, Y. Reliability-oriented multi-AGVs online scheduling and path planning problem of automated sorting warehouse system. IOP Conf. Ser. Mater. Sci. Eng. 2021, 1043, 022035. [Google Scholar] [CrossRef] [Scilit]
  21. Bishop, C.M. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006. [Google Scholar]
  22. Chen, S.H.; Jakeman, A.J.; Norton, J.P. Artificial intelligence techniques: An introduction to their use for modelling environmental systems. Math. Comput. Simul. 2008, 78, 379–400. [Google Scholar] [CrossRef] [Scilit]
  23. Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, 20–25 June 2005; Volume 1, pp. 886–893. [Google Scholar]
  24. Fischer, M.; Heim, D.; Hofmann, A.; Janiesch, C.; Klima, C.; Winkelmann, A. A taxonomy and archetypes of smart services for smart living. Electron. Mark. 2020, 30, 131–149. [Google Scholar] [CrossRef] [Scilit]
  25. Fuchs, D.J. The dangers of human-like bias in machine-learning algorithms. Mo. S&T’s Peer Peer 2018, 2, 15. [Google Scholar]
  26. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
  27. Goyal, D.; Pabla, B.S. Condition-based machine tool maintenance—A review. CIRP J. Manuf. Sci. Technol. 2015, 10, 24–35. [Google Scholar] [CrossRef] [Scilit]
  28. Grigorescu, S.; Trasnea, B.; Cocias, T.; Macesanu, G. A survey of deep learning techniques for autonomous driving. J. Field Robot. 2020, 37, 362–386. [Google Scholar] [CrossRef] [Scilit]
  29. Heinrich, K.; Möller, B.; Janiesch, C.; Zschech, P. Is bigger always better? Lessons learnt from the evolution of deep learning architectures for image classification. In Proceedings of the 2019 Pre-ICIS SIGDSA Symposium, Munich, Germany, 14–15 December 2019. [Google Scholar]
  30. Heinrich, K.; Zschech, P.; Janiesch, C.; Bonin, M. Process data properties matter: Introducing gated convolutional neural networks (GCNN) and key-value-predict attention networks (KVP) for next event prediction with deep learning. Decis. Support Syst. 2021, 143, 113494. [Google Scholar] [CrossRef] [Scilit]
  31. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science 2015, 349, 255–260. [Google Scholar] [CrossRef] [Scilit]
  32. Yue, L.; Fan, H.; Ma, M. Optimizing configuration and scheduling of double 40 ft dual-trolley quay cranes and AGVs for improving container terminal services. J. Clean. Prod. 2021, 292, 126019. [Google Scholar] [CrossRef] [Scilit]
  33. Ji, S.; Luan, D.; Chen, Z.; Guo, D. Integrated scheduling in automated container terminals considering AGV conflict-free routing. Transp. Lett. 2021, 13, 501–513. [Google Scholar]
  34. Amorós, L.; Hafiz, S.M.; Lee, K.; Tol, M.C. Gimme that model: A trusted ML model trading protocol. arXiv 2020, arXiv:2003.00610. [Google Scholar] [CrossRef] [Scilit]
  35. Assaf, R.; Schumann, A. Explainable deep neural networks for multivariate time series predictions. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, 10–16 August 2019; pp. 6488–6490. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
A Near-Field Communication (NFC) Multi-Sensor Node with Optimized Read Range and Adaptive Power Management for Remote Monitoring
Previous
Computational Architectures for 6G Networks: Integrating Distributed Computing and Edge Artificial Intelligence
Next