Search Results (145)

Indoor navigation remains challenging for students with visual impairments because GPS is unavailable indoors and building layouts are often complex. This paper presents a wearable marker-assisted navigation system integrating QR code localization, SSD MobileNet V3 obstacle detection, TFmini-S LiDAR ranging, A*-based dynamic route planning, and audio feedback on a Raspberry Pi 5. The main contribution is an analytical framework relating marker spacing to predicted localization uncertainty and defining a latency budget for obstacle warnings. A confidence-weighted sensor-fusion method is developed analytically but was not implemented in the evaluated prototype, in which the QR code, camera, and LiDAR channels operated independently. The proposed fusion method and the simulated multi-floor planning extension require further experimental validation. Controlled tests produced a mean positioning error below 1.2 m, a LiDAR ranging MAE of 8.3 cm, and an object-detection throughput of 6–9 FPS. A pilot field evaluation covered nine routes totalling 901 m across two buildings and included one participant with self-reported vision loss of approximately 95%. All route trials were completed, although some required researcher assistance. The system remains a proof of concept and has not yet been evaluated against a baseline or with a sufficiently large target-user sample. Full article

Modern autonomous systems rely on heterogeneous sensing modalities—including vision sensors, millimeter-wave radar, and LiDAR—yet each individual sensor exhibits characteristic failure modes in challenging real-world conditions. While LiDAR-vision co-processing has received extensive attention, the synergistic potential of 4D radar paired with monocular optics remains comparatively unexplored. To fill this gap, we develop a graph-enhanced multi-modal architecture that jointly leverages sparse 4D radar returns and high-resolution camera imagery for scene-level 3D perception. The proposed system is organized around four tightly coupled processing stages: (i) an image-guided point densification scheme (SAA) that augments sparse radar clouds with camera-derived pseudo measurements; (ii) a pose-invariant cross-modal fusion layer that harmonizes enriched radar features with image descriptors and object saliency maps; (iii) a dynamic hypergraph assembly stage that captures higher-order inter-object and cross-sensor dependencies; and (iv) a HyperGCN inference module that regresses 3D bounding parameters and class labels on the resulting relational graph. Integrating the temporal velocity cues native to radar with the rich appearance information from cameras, the system delivers reliable perception under diverse environmental conditions. On the View-of-Delft (VOD) evaluation suite, the proposed model records an mAP of 69.3 and mAOS of 59.8. A systematic ablation further quantifies how geometric invariance—across translation, rotation, and scale transformations—individually affects end-to-end detection fidelity. Full article

Modern cryptography depends on the quality of the operating-system entropy pool, and historic failures of that pool have shown that randomness, not algorithm strength, is often the weakest link of a deployed cryptosystem. Hardware True-Random Number Generators (TRNGs) mitigate this risk but are not universally available or trusted. This paper develops a unified four-stage construction, extract → condition → mix → derive, that admits heterogeneous publicly observable dynamic data sources as auxiliary inputs to the operating-system randomness pipeline. The construction’s central claim is a no-degradation defence-in-depth property, formalised as Theorem 1: under a pseudorandom-function assumption on the conditioning function, the composed output is computationally indistinguishable from the operating-system output alone, even when an adversary fully controls the public source. The framework does not claim that public sources supply secret entropy against an adversary who also observes them; it claims only that mixing them in cannot weaken the baseline, and that any uncompromised source supplies residual unpredictability under operating-system-generator compromise. We instantiate the framework in three pipelines—aircraft state vectors from a public sensor network, webcam frame-differencing of a wall of lava lamps, and computer-vision tracking of vehicles on a public traffic-camera feed—and argue soundness by combining the theorem with published evaluations of each source. On a 381 kbit corpus of framework-derived keys, every applicable NIST SP 800-22 test and SP 800-90B entropy estimator finds the framework output statistically indistinguishable from an os.urandom baseline; this is consistent with no-degradation but not in itself evidence of marginal entropy gain, and per-pipeline raw-input measurements and source-ablation runs are deferred to empirical follow-up. Off-the-shelf computer-vision tooling is sufficient to operationalise the framework on commodity hardware. Full article

Autonomous driving systems rely on a sophisticated pipeline of artificial intelligence models to perceive, predict, and plan in dynamic environments. This review presents a systematic analysis of the machine learning and deep learning models underpinning vehicle autonomy, spanning classical convolutional neural networks (CNNs) for object detection and semantic segmentation to recurrent and Transformer-based architectures for trajectory prediction and motion planning. It also provides a critical examination of the autonomous vehicle sensor stack, including cameras, LiDAR, radar, ultrasonics, and GNSS/IMU as data acquisition systems, highlighting modality-specific AI challenges such as monocular depth estimation, 3D point cloud processing, and radar Doppler interpretation. The evolution of perception and decision-making pipelines is reviewed, contrasting modular architectures with end-to-end learning paradigms that directly map raw sensor data to control commands, and discussing their trade-offs in interpretability, safety assurance, and robustness to rare edge cases. We further survey specialized hardware accelerators and heterogeneous automotive SoCs designed to meet stringent real-time and power constraints. Industrial strategies are compared, including multi-modal sensor fusion and vision-centric approaches based on large-scale imitation learning. Finally, we identify open challenges related to robustness under adverse conditions, domain shift, causal ambiguity, and the need for interpretable and certifiable AI in safety-critical autonomous driving systems. Full article

Monitoring the dynamic response of structures subjected to operational loads is a key component of structural health assessment, providing valuable information for safety evaluation and maintenance planning. In the last decade, video-based measurements have received growing attention for modal identification and damage detection applications, offering a promising alternative to traditional sensor-based approaches. Unlike conventional monitoring systems, which provide discrete measurements and often require extensive instrumentation, computer vision techniques enable dense, non-contact measurements while reducing installation costs and accessibility constraints. Moreover, Motion Magnification algorithms can be combined with computer vision-based identification techniques to amplify displacements within selected frequency ranges, facilitating the detection of low-amplitude structural vibrations. In this work, a semi-automated methodology for structural identification is presented and validated through two experimental applications involving vibrating systems monitored with commercial cameras. The proposed framework combines computer vision algorithms, Motion Magnification (MM), correlation analysis, and Principal Component Analysis (PCA), the latter being adopted as a noise-reduction and dimensionality-reduction tool to extract the most informative features from large sets of time-histories. In contrast to previous studies primarily focused on damage detection and frequency evolution tracking, the present work specifically investigates the influence of key user-defined parameters on the reliability of the identified frequencies and provides practical calibration guidelines for future applications. The methodology was validated against reference measurements obtained from an optical monitoring system and it successfully identified the natural frequencies of the analysed structures with errors ranging from 0.84% to 1.75%. Sensitivity analyses performed on the region of interest size and position, as well as on the correlation threshold, demonstrated the robustness of the proposed workflow. The results confirm that the proposed approach represents a reliable, low-cost, and minimally invasive alternative to conventional dynamic monitoring techniques, while providing practical recommendations for its implementation in real-world structural health monitoring applications. Full article

This paper presents a metrology-oriented review of vision-based displacement and deformation measurement for civil structural health monitoring (SHM), with an emphasis on field robustness and uncertainty quantification (UQ). The review focuses on image- and video-based methods that convert visual information into quantitative physical measurements, such as displacement, strain, or derived dynamic indicators. The literature is organized according to the main stages of the measurement chain: image formation, image-plane motion estimation, and geometric conversion to metric motion. Within this framework, measurement pipelines are interpreted through three levels of geometric mapping, namely, a scalar scale-factor model, a planar homography-based model, and a full Jacobian-based model. The review synthesizes major method families, including marker-based and markerless tracking, feature-based tracking, optical flow, digital image correlation (DIC), phase-based motion magnification, edge-based estimators, fixed- and moving-camera configurations, UAV-based acquisition with ego-motion compensation, hybrid vision–sensor fusion, and deep-learning-enhanced pipelines. A structured taxonomy of uncertainty sources is then presented along the processing chain, covering camera geometry and calibration, imaging noise and blur, quantization, timing and synchronization, environmental disturbances, optical turbulence and heat haze, platform motion, algorithmic failure modes, and reference-sensor uncertainty. The paper also compares UQ practices, including GUM-aligned analytical propagation, Monte Carlo methods, DIC-specific error budgets, bootstrap and resampling strategies, and probabilistic deep learning. The main contribution of this review is to connect computer-vision-based displacement pipelines with metrological requirements by explicitly linking measurement models, uncertainty sources, UQ methods, and field-validation evidence within a unified framework. A practical uncertainty-budget template is compiled to support traceable reporting across different pipelines and deployment scenarios. The paper concludes with prioritized research gaps and future directions, including standardized benchmarks and datasets, traceable UQ for moving-camera systems, multi-sensor fusion with end-to-end uncertainty propagation, long-term drift characterization, optical-turbulence and adverse-weather modeling, validated subpixel limits at extreme range, probabilistic deep learning–metrology integration, and standardized reporting practices. Full article

Autonomous Haulage Systems (AHS) have significantly transformed surface mining operations by improving safety, productivity, and operational consistency. Currently, AHS predominantly rely on vehicle-centric perception architectures. Onboard LiDAR, radar, cameras, and Global Navigation Satellite Systems (GNSS) perform sensing, interpretation, and decision-making within individual systems. These processes enable collision avoidance and path tracking. However, they are limited in their ability to consider the broader, dynamic mining environment characterized by dust, terrain degradation, geotechnical instability, heterogeneous traffic, and rapidly evolving operational conditions. This paper presents a systematic review of dynamic vision systems of AHS in surface mining. It critically analyzes the transition from autonomy to interconnected, ecosystem-aware intelligence. The review synthesizes literature from mining automation, robotics, intelligent transportation systems, and multi-agent perception. It assesses sensing technologies, perception algorithms, sensor fusion strategies, and environmental robustness techniques. Attention is focused on the limitations of egocentric perception models in complex surface mining ecosystems. Building on identified gaps, the paper proposes a conceptual framework for Ecosystem-Centric Dynamic Vision (ECDV). Perception is enhanced through integration with fleet communication networks, dispatch systems, digital twins, geotechnical monitoring platforms, and environmental sensing infrastructure. The framework outlines a multi-layer architecture enabling cooperative perception, predictive hazard modeling, and risk-aware decision support at the mine-wide level. The review concludes by outlining a research agenda to transition from vehicle autonomy to ecosystem intelligence in surface mining. It highlights opportunities in cooperative perception, adaptive sensor fusion under degraded visibility, and digital-twin-integrated predictive safety systems. Full article

Low-latency visual perception is essential for interactive machine vision on edge AI devices, but conventional frame-based image sensors impose frame period delays and generate dense image data that increase memory bandwidth and processing latency. Although Dynamic Vision Sensors (DVSs) are known to provide low latency, sparse output, and high dynamic range, these sensor-level properties do not automatically translate into practical application-level latency reduction on resource-constrained edge platforms. This paper presents a latency-driven sensing algorithm co-design approach for DVS-based low-latency machine vision. The main objective is to connect DVS sensor-level characteristics, event representations, task-dependent processing flows, and measured response times on mobile application processors. We first analyze latency requirements for three representative edge AI applications (i.e., person detection, gesture recognition, and Simultaneous Localization and Mapping (SLAM)), which correspond to different latency regimes and processing structures. We then describe the DVS operating principle, pixel-level event latency, and readout latency, showing how asynchronous event generation reduces sensing delay and suppresses redundant static background information before algorithmic processing. In contrast to prior event camera studies that mainly optimize a single task or a specific event representation, this work evaluates three task-specific event processing systems on mobile processors. Person detection achieves 92 ms processing latency on Exynos 7570, gesture recognition based on event-driven 4-DoF motion estimation achieves 20 ms latency on Exynos 5422, and SLAM achieves 15.9 ms latency on Snapdragon 845. These results satisfy the practical latency targets of the corresponding applications and demonstrate that DVS-based sensing can provide not only sensor-level speed advantages but also system-level latency benefits for AIoT, mobile, robotics, and AR/VR machine vision systems. Full article

Traffic object detection based on camera sensors is a critical task for autonomous vehicles. However, in nighttime conditions with adverse lighting, several challenges arise: blurred object edges, large-scale variations, and complex lighting conditions involving both overexposure and underexposure. As a result, it remains difficult for vision-based perception tasks to ensure reliable precision and rapid inference simultaneously. This paper proposes a novel, efficient, and lightweight vision module for detecting traffic objects in challenging nighttime environments, developed by enhancing the YOLOv8n architecture. Firstly, a bidirectional weighted feature fusion method (BiFPN) is incorporated in the path aggregation network, and an additional shallow P2 feature map is introduced to fully utilize key information from features at different scales. Then, the coordinate attention (CA) module is inserted between the end of the feature pyramid and the detection head to capture both semantic and spatial information of the object. Finally, the dynamic upsampler (DySample) is employed to guide the model in focusing on the detailed features of challenging samples, thereby balancing accuracy across different object categories. A subset of nighttime traffic scenes is curated from the BDD100K dataset for the evaluation of the proposed approach. The experiments demonstrate that, relative to the baseline, our method raises the mean average precision (${\mathrm{mAP}}_{50}$) from 51.5% to 56.6%, achieves a 7.3% decrease in parameter quantity, and maintains a fast inference speed of 208 FPS. For the challenging bike and motorbike categories, notable improvements in detection accuracy are achieved. Compared with other advanced YOLO-series models such as YOLOv11, the proposed model also exhibits significant performance advantages with a 3.7% higher ${\mathrm{mAP}}_{50}$. Furthermore, our model demonstrates good generalization performance on the larger BDD100K nighttime partition. The findings confirm that our approach significantly improves detection accuracy without compromising real-time processing, highlighting its potential as a lightweight vision module providing reliable perceptual inputs for autonomous vehicle control and safety actuators in challenging nighttime scenarios. Full article

With the increasing demand for reliable structural safety assessment in service, high-precision, non-contact, and long-term deformation monitoring has become increasingly urgent for large civil engineering structures. To address this need, this study proposes and validates a system-level non-contact monitoring framework that integrates an embedded vision-based deformation sensor with intelligent algorithms. Rather than treating individual techniques as isolated components, the proposed framework integrates high-precision optical imaging, subpixel localization, and intelligent image processing into a unified monitoring workflow. By continuously imaging and tracking targets on the structural surface, high-precision acquisition of two-dimensional dynamic displacements is achieved. To address issues such as image jitter, environmental disturbances, and camera-induced vibrations under long-distance imaging conditions, a hybrid algorithm based on signal processing and image correction is introduced to effectively compensate and filter the monitoring data, thereby significantly improving the stability and accuracy of deflection measurements. In engineering applications, a girder bridge and an integral open-box sluice structure were selected as monitoring objects, and field experiments were conducted over multiple periods under different working conditions. The results indicate that the proposed system can stably capture small structural displacements, achieving sub-millimeter measurement accuracy. The findings verify the feasibility and reliability of the proposed intelligent vision-based deformation monitoring technology in complex engineering environments, and provide a new technical approach for structural safety assessment and operational monitoring of infrastructure such as bridges and hydraulic structures. Full article

Conservation management of large, multi-species landscapes requires integrating heterogeneous data streams—such as satellite imagery, GPS telemetry, camera traps, bioacoustic sensors, weather stations, and field reports—into a unified model capable of simulating ecosystem dynamics and generating actionable recommendations. This paper proposes a tiered, energy-aware AI architecture for constructing ecosystem digital twins that enables prescriptive, rather than merely descriptive or predictive, landscape-scale conservation management. The framework classifies conservation tasks across three computational tiers: classical machine learning for continuous environmental monitoring and species distribution prediction, deep learning for perception-oriented tasks such as computer vision and bioacoustic analysis, and foundation models for cross-domain synthesis and stakeholder interaction. We apply this architecture to a comprehensive digital twin of the Greater Yellowstone Ecosystem, anchored in the ongoing conservation crisis of the Sublette Pronghorn Herd—a population that crashed from 43,000 to 24,000 animals in a single winter due to compounding severe weather and a Mycoplasma bovis outbreak. We formalize a coupled change model linking population dynamics, forage condition, corridor permeability, winter severity, and disease pressure, and demonstrate how a prescriptive recommendations engine can generate goal-conditioned management actions for the herd’s 165-mile “Path of the Pronghorn” migration corridor. A comparative energy footprint analysis, grounded in hardware-level energy measurements using Intel RAPL instrumentation and the CodeCarbon framework, estimates that the tiered architecture reduces computational energy consumption by approximately 34% relative to a deep-learning-everywhere baseline and by over three orders of magnitude relative to a foundation-model-centric baseline. The architecture provides a replicable blueprint for resource-constrained conservation organizations seeking to deploy AI-powered ecosystem management at landscape scale. Full article

Accurate extrinsic calibration of multi-camera systems is a central problem in three-dimensional computer vision, as errors in the relative positioning of sensors directly propagate into geometric distortions that critically degrade the quality of downstream applications. This paper proposes an incremental extrinsic camera parameter initialization method that improves upon the baseline iterative registration algorithm based on the Perspective-n-Point (PnP) problem. Unlike board-based calibration frameworks, the proposed approach operates on individually placed markers with no prior knowledge of their mutual positions, enabling recalibration without dedicated calibration sessions. The accuracy improvement is achieved through the introduction of heuristic weighting of fiducial marker detections using AprilTags, as well as the application of a multi-view triangulation algorithm for dynamic refinement of marker spatial coordinates at each stage of scene expansion. Theoretical analysis demonstrates that the incorporation of these mechanisms does not increase the overall asymptotic computational complexity of the complete calibration cycle (including the global optimization stage), despite the higher computational cost of the initialization stage itself. Empirical validation of the method is performed on both synthetic datasets with known ground-truth camera parameters and real-world capture data through the evaluation of geometric errors and their comparison with the baseline method. Experimental results, supplemented by an ablation study, indicate that the proposed algorithm achieves statistically significant improvements on synthetic data in more than 80% of cases, while on real data it is on average 85% more accurate in terms of reprojection error. Full article

Vision foundation models (VFMs) enable high-quality 2D instance masks, yet their lifted pseudo-point clouds suffer from scale ambiguity, structural noise, and temporal inconsistency, limiting their utility in 3D annotation. Existing automatic labeling methods either rely on expensive light detection and ranging (LiDAR) sensors or fail to enforce physical plausibility in dynamic roadside scenes. This study proposes a LiDAR-free radar–visual auto-labeling framework that leverages cross-modal spatio-temporal consistency between millimeter-wave radar trajectories and visual pseudo-point clouds to self-correct 3D geometry. The method first associates radar points, 2D masks, and pseudo-point clouds into object-centric sequences. Then, an uncertainty-aware pose fusion module combines motion-derived and structure-derived orientations using automatically solved road priors. Finally, the pseudo-point cloud is refined in canonical space by optimizing stable semantic landmarks from temporally consistent masks and propagating their corrections globally. Evaluated on a real-world roadside dataset, the method achieves 49.1% bird’s-eye-view (BEV) intersection over union (IoU) and 43.0% 3D IoU, outperforming a radar–camera fusion baseline by 5.5/5.9 points. Downstream experiments further show that the generated pseudo-labels and semantic enhancement are useful under the evaluated detector configurations, while broader validation remains future work. Full article

Robotic systems are increasingly being deployed in mining operations to support tasks such as inspection, navigation, environmental monitoring, and safety supervision. However, mining environments present significant challenges for robotic perception due to dynamic terrain conditions, poor illumination, airborne dust, and frequent disturbances caused by excavation and heavy machinery. Conventional frame-based vision systems often struggle under these conditions due to motion blur, latency, and limited dynamic range. This study proposes a system-level conceptual framework for integrating event-based sensing into robotic mining systems in order to support perception in highly dynamic and safety-critical environments, with the aim of improving responsiveness and robustness under such conditions. Event-based cameras, inspired by biological vision, asynchronously detect brightness changes at the pixel level and provide microsecond temporal resolution with high dynamic range and low latency. The proposed framework combines event cameras with complementary sensing modalities including LiDAR, inertial measurement units, and RGB cameras to form a multi-sensor perception architecture. The framework is structured into multiple functional layers encompassing environmental sensing, event-driven perception, sensor fusion and AI processing, digital twin integration, and autonomous decision-making. Potential application scenarios including robotic tunnel inspection, autonomous navigation of mining robots, hazard detection, multi-agent cooperation in mining sites, and real-time digital twin updating are also discussed. The proposed framework provides a unified system-level reference architecture intended to guide future implementation and validation. Full article

This study aims to enhance the precision of humanoid robots in imitating complex human “walking–grasping” coordinated movements. Addressing limitations in sample efficiency and reward function design in Generative Adversarial Imitation Learning (GAIL), we propose the Similarity Reward-Augmented Generative Adversarial Imitation Learning (SRA-GAIL) framework. The method integrates plantar thin-film resistive pressure sensors to measure the real-time pressure distribution at four key points on both feet, combined with roll/pitch angle data acquired from JY901S inertial measurement units (IMUs). A Lagrangian constraint optimization strategy is employed to achieve gait stability control based on the zero moment point (ZMP). Simultaneously, a visual similarity evaluation module is established using human demonstration trajectories captured by a Logitech C920E camera, augmented by grip force feedback from flexible thin-film pressure sensors on the hands. This enables the design of a multimodal sensor-fused similarity reward function. By incorporating Lagrangian constraint optimization and a maximum entropy reinforcement learning framework, Similarity Reward-Augmented Generative Adversarial Imitation Learning synchronously optimizes gait stability control—guided by zero moment point (ZMP) and roll/pitch data—and vision-based trajectory similarity evaluation. These components address motion stability constraints and trajectory similarity metrics, respectively, generating biomechanically plausible gait strategies. A spatiotemporal attention mechanism parses human motion trajectory features to drive the end-effector for high-precision trajectory tracking. To validate the proposed method, an imitation learning experimental system was constructed on a physical XIAOLI humanoid robot platform, integrating inertial measurement units (IMUs), plantar pressure sensors, and a vision system. Quantitative evaluations were conducted across multiple dimensions, including robot platform analysis, walking stability, object grasping success rates, and end-effector trajectory similarity. The results demonstrate that, compared to Generative Adversarial Imitation Learning (GAIL) and behavioral cloning, Similarity Reward-Augmented Generative Adversarial Imitation Learning achieves a stable object grasping success rate of 93.7% in complex environments, with a 23.8% improvement in sample efficiency. The method maintains a 96.5% compliance rate for zero moment point (ZMP) trajectories within the support polygon, significantly outperforming baseline approaches. This effectively addresses the bottleneck in robot policies adapting to dynamic changes in real-world environments. Full article