Search Results (19)

Sign language is a complex and dynamic visual language that requires the coordinated movement of various body parts, such as the hands, arms, and limbs—making it an ideal application domain for sensor networks to capture and interpret human gestures accurately. To address the intricate task of precise and expedient SLR from raw videos, this study introduces a novel deep learning approach by devising a multimodal framework for SLR. Specifically, feature extraction models are built based on two modalities: skeleton and RGB images. In this paper, we firstly propose a Multi-Stream Spatio-Temporal Graph Convolutional Network (MSGCN) that relies on three modules: a decoupling graph convolutional network, a self-emphasizing temporal convolutional network, and a spatio-temporal joint attention module. These modules are combined to capture the spatio-temporal information in multi-stream skeleton features. Secondly, we propose a 3D ResNet model based on deformable convolution (D-ResNet) to model complex spatial and temporal sequences in the original raw images. Finally, a gating mechanism-based Multi-Stream Fusion Module (MFM) is employed to merge the results of the two modalities. Extensive experiments are conducted on the public datasets AUTSL and WLASL, achieving competitive results compared to state-of-the-art systems. Full article

Using a commodity RGB-D sensor is a popular and cost-effective way to enable interaction at room scale, as such a device supports body tracking functionality at a reasonable price point. Even though the capabilities of such devices might be enough for applications like entertainment systems where a person plays in front of a television, this type of sensor is unfortunately sensitive to occlusions from objects or other people, who might be in the way in more sophisticated room-scale set-ups. One may use multiple RGB-D sensors and aggregate the collected data to address the occlusion problem, increase the tracking range, and improve accuracy. However, doing so requires the gathering of calibration information with regard to the sensors themselves and also regarding their relative placement on interactable surfaces. Another challenging consequence of relying on multiple sensors is the need to perform skeleton matching and merging based on their respective body tracking data (e.g., so that skeletons from different sensors but belonging to the same person are recognised as such). The present contribution focuses on approaches to tackling these issues. Ultimately, it contributes a working human interaction tracking system, leveraging multiple RGB-D sensors to provide unobtrusive and occlusion-resilient understanding capabilities. This constitutes a suitable basis for room-scale experiences such as those based on wall-sized displays. Full article

Changes in the jet stream not only affect the persistence of climate change and the frequency of extreme weather but are also closely related to climate change phenomena such as global warming. The manual way of drawing the jet stream axes in meteorological operations suffers from low efficiency and subjectivity issues. Automatic identification algorithms based on wind field analysis have some shortcomings, such as poor generalization ability, and it is difficult to handle merging and splitting. A semi-supervised learning jet stream axis identification method is proposed combining consistency learning and self-training. First, a segmentation model is trained via semi-supervised learning. In semi-supervised learning, two neural networks with the same structure are initialized with different methods, based on which pseudo-labels are obtained. The high-confidence pseudo-labels are selected by adding perturbation into the feature layer, and the selected pseudo-labels are incorporated into the training set for further self-training. Then, the jet stream narrow regions are segmented via the trained segmentation model. Finally, the jet stream axes are obtained with the skeleton extraction method. This paper uses the semi-supervised jet stream axis identification method to learn features from unlabeled data to achieve a small amount of labeled data to effectively train the model and improve the method’s generalization ability in a small number of labeled cases. Experiments on the jet stream axis dataset show that the identification precision of the presented method on the test set exceeds about 78% for SOTA baselines, and the improved method exhibits better performance compared to the correlation network model and the semi-supervised method. Full article

The three-dimensional (3D) reconstruction of buildings using photogrammetric point clouds is important for many applications, ranging from digital city construction to urban energy consumption analysis. However, problems such as building complexity and point cloud flaws may lead to incorrect modeling, which will affect subsequent steps such as texture mapping. This paper introduces a pipeline for building surface reconstruction from photogrammetric point clouds, employing a hybrid method that combines connection evaluation and framework optimization. Firstly, the plane segmentation method divides building point clouds into several pieces, which is complemented by a proposed candidate plane generation method aimed at removing redundancies and merging similarities. Secondly, the improved connection evaluation method detects potential skeleton lines from different planes. Subsequently, a framework optimization method is introduced to select suitable undirected polygonal boundaries from planes, forming the basis for plane primitives. Finally, by triangulating all plane primitives and filling holes, a building surface polygonal model is generated. Experiments conducted on various building examples provide both qualitative and quantitative evidence that the proposed hybrid method outperforms many existing methods, including traditional methods and deep learning methods. Notably, the proposed method successfully reconstructs the main building structures and intricate details, which can be further used to generate textural models and semantic models. Experimental results validate that the proposed method can be used for the surface reconstruction from photogrammetric point clouds of planar buildings. Full article

Classifying pathological gaits is crucial for identifying impairments in specific areas of the human body. Previous studies have extensively employed machine learning and deep learning (DL) methods, using various wearable (e.g., inertial sensors) and non-wearable (e.g., foot pressure plates and depth cameras) sensors. This study proposes early and late fusion methods through DL to categorize one normal and five abnormal (antalgic, lurch, steppage, stiff-legged, and Trendelenburg) pathological gaits. Initially, single-modal approaches were utilized: first, foot pressure data were augmented for transformer-based models; second, skeleton data were applied to a spatiotemporal graph convolutional network (ST-GCN). Subsequently, a multi-modal approach using early fusion by concatenating features from both the foot pressure and skeleton datasets was introduced. Finally, multi-modal fusions, applying early fusion to the feature vector and late fusion by merging outputs from both modalities with and without varying weights, were evaluated. The foot pressure-based and skeleton-based models achieved 99.04% and 78.24% accuracy, respectively. The proposed multi-modal approach using early fusion achieved 99.86% accuracy, whereas the late fusion method achieved 96.95% accuracy without weights and 99.17% accuracy with different weights. Thus, the proposed multi-modal models using early fusion methods demonstrated state-of-the-art performance on the GIST pathological gait database. Full article

In this study, superparamagnetic iron oxide nanoparticles (SPIONs) were engineered with an organic coating composed of low molecular weight heparin (LMWH) and bovine serum albumin (BSA), providing heparin-based nanoparticle systems (LMWH@SPIONs). The purpose was to merge the properties of the heparin skeleton and an inorganic core to build up a targeted theranostic nanosystem, which was eventually enhanced by loading a chemotherapeutic agent. Iron oxide cores were prepared via the co-precipitation of iron salts in an alkaline environment and oleic acid (OA) capping. Dopamine (DA) was covalently linked to BSA and LMWH by amide linkages via carbodiimide coupling. The following ligand exchange reaction between the DA-BSA/DA-LMWH and OA was conducted in a biphasic system composed of water and hexane, affording LMWH@SPIONs stabilized in water by polystyrene sulfonate (PSS). Their size and morphology were investigated via dynamic light scattering (DLS) and transmission electron microscopy (TEM), respectively. The LMWH@SPIONs’ cytotoxicity was tested, showing marginal or no toxicity for samples prepared with PSS at concentrations of 50 µg/mL. Their inhibitory activity on the heparanase enzyme was measured, showing an effective inhibition at concentrations comparable to G4000 (N-desulfo-N-acetyl heparin, a non-anticoagulant and antiheparanase heparin derivative; Roneparstat). The LMWH@SPION encapsulation of paclitaxel (PTX) enhanced the antitumor effect of this chemotherapeutic on breast cancer cells, likely due to an improved internalization of the nanoformulated drug with respect to the free molecule. Lastly, time-domain NMR (TD-NMR) experiments were conducted on LMWH@SPIONs obtaining relaxivity values within the same order of magnitude as currently used commercial contrast agents. Full article

The paper describes a method for planning collision-free motions of an industrial manipulator that shares the workspace with human operators during a human–robot collaborative application with strict safety requirements. The proposed workflow exploits the advantages of mixed reality to insert real entities into a virtual scene, wherein the robot control command is computed and validated by simulating robot motions without risks for the human. The proposed motion planner relies on a sensor-fusion algorithm that improves the 3D perception of the humans inside the robot workspace. Such an algorithm merges the estimations of the pose of the human bones reconstructed by means of a pointcloud-based skeleton tracking algorithm with the orientation data acquired from wearable inertial measurement units (IMUs) supposed to be fixed to the human bones. The algorithm provides a final reconstruction of the position and of the orientation of the human bones that can be used to include the human in the virtual simulation of the robotic workcell. A dynamic motion-planning algorithm can be processed within such a mixed-reality environment, allowing the computation of a collision-free joint velocity command for the real robot. Full article

Skeleton data, which is often used in the HCI field, is a data structure that can efficiently express human poses and gestures because it consists of 3D positions of joints. The advancement of RGB-D sensors, such as Kinect sensors, enabled the easy capture of skeleton data from depth or RGB images. However, when tracking a target with a single sensor, there is an occlusion problem causing the quality of invisible joints to be randomly degraded. As a result, multiple sensors should be used to reliably track a target in all directions over a wide range. In this paper, we proposed a new method for combining multiple inaccurate skeleton data sets obtained from multiple sensors that capture a target from different angles into a single accurate skeleton data. The proposed algorithm uses density-based spatial clustering of applications with noise (DBSCAN) to prevent noise-added inaccurate joint candidates from participating in the merging process. After merging with the inlier candidates, we used Kalman filter to denoise the tremble error of the joint’s movement. We evaluated the proposed algorithm’s performance using the best view as the ground truth. In addition, the results of different sizes for the DBSCAN searching area were analyzed. By applying the proposed algorithm, the joint position accuracy of the merged skeleton improved as the number of sensors increased. Furthermore, highest performance was shown when the searching area of DBSCAN was 10 cm. Full article

There is an essential need for a change in the way we build our physical environment. To prevent our ecosystems from collapsing, raising awareness of already available bio-based materials is vital. Mycelium, a living fungal organism, has the potential to replace conventional materials, having the ability to act as a binding agent of various natural fibers, such as hemp, flax, or other agricultural waste products. This study aims to showcase mycelium’s load-bearing capacities when reinforced with bio-based materials and specifically natural fibers, in an alternative merging design approach. Counteracting the usual fabrication techniques, the proposed design method aims to guide mycelium’s growth on a natural rattan framework that serves as a supportive structure for the mycelium substrate and its fiber reinforcement. The rattan skeleton is integrated into the finished composite product, where both components merge, forming a fully biodegradable unit. Using digital form-finding tools, the geometry of a compressive structure is computed. The occurring multi-layer biobased component can support a load beyond 20 times its own weight. An initial physical prototype in furniture scale is realized. Further applications in architectural scale are studied and proposed. Full article

Human bone is a specialized tissue with unique material properties, providing mechanical support and resistance to the skeleton and simultaneously assuring capability of adaptation and remodelling. Knowing the properties of such a structure down to the micro-scale is of utmost importance, not only for the design of effective biomimetic materials but also to be able to detect pathological alterations in material properties, such as micro-fractures or abnormal tissue remodelling. The Brillouin and Raman micro-spectroscopic (BRmS) approach has the potential to become a first-choice technique, as it is capable of simultaneously investigating samples’ mechanical and structural properties in a non-destructive and label-free way. Here, we perform a mapping of cortical and trabecular bone sections of a femoral epiphysis, demonstrating the capability of the technique for discovering the morpho-mechanics of cells, the extracellular matrix, and marrow constituents. Moreover, the interpretation of Brillouin and Raman spectra merged with an approach of data mining is used to compare the mechanical alterations in specimens excised from distinct anatomical areas and subjected to different sample processing. The results disclose in both cases specific alterations in the morphology and/or in the tissue chemical make-up, which strongly affects bone mechanical properties, providing a method potentially extendable to other important biomedical issues. Full article

Graph convolutional networks (GCNs), which model human actions as a series of spatial-temporal graphs, have recently achieved superior performance in skeleton-based action recognition. However, the existing methods mostly use the physical connections of joints to construct a spatial graph, resulting in limited topological information of the human skeleton. In addition, the action features in the time domain have not been fully explored. To better extract spatial-temporal features, we propose a multi-stage attention-enhanced sparse graph convolutional network (MS-ASGCN) for skeleton-based action recognition. To capture more abundant joint dependencies, we propose a new strategy for constructing skeleton graphs. This simulates bidirectional information flows between neighboring joints and pays greater attention to the information transmission between sparse joints. In addition, a part attention mechanism is proposed to learn the weight of each part and enhance the part-level feature learning. We introduce multiple streams of different stages and merge them in specific layers of the network to further improve the performance of the model. Our model is finally verified on two large-scale datasets, namely NTU-RGB+D and Skeleton-Kinetics. Experiments demonstrate that the proposed MS-ASGCN outperformed the previous state-of-the-art methods on both datasets. Full article

We report a reactive molecular dynamic (ReaxFF-MD) study using the newly parameterized ReaxFF-lg reactive force field to explore the initial decomposition mechanism of 3-Nitro-1,2,4-triazol-5-one (NTO) under shock loading (shock velocity >6 km/s). The new ReaxFF-lg parameters were trained from massive quantum mechanics data and experimental values, especially including the bond dissociation curves, valence angle bending curves, dihedral angle torsion curves, and unimolecular decomposition paths of 3-Nitro-1,2,4-triazol-5-one (NTO), 1,3,5-Trinitro-1,3,5-triazine (RDX), and 1,1-Diamino-2,2-dinitroethylene (FOX-7). The simulation results were obtained by analyzing the ReaxFF dynamic trajectories, which predicted the most frequent chain reactions that occurred before NTO decomposition was the unimolecular NTO merged into clusters ((C₂H₂O₃N₄)_n). Then, the NTO dissociated from (C₂H₂O₃N₄)_n and started to decompose. In addition, the paths of NO₂ elimination and skeleton heterocycle cleavage were considered as the dominant initial decomposition mechanisms of NTO. A small amount of NTO dissociation was triggered by the intermolecular hydrogen transfer, instead of the intramolecular one. For α-NTO, the calculated equation of state was in excellent agreement with the experimental data. Moreover, the discontinuity slope of the shock-particle velocity equation was presented at a shock velocity of 4 km/s. However, the slope of the shock-particle velocity equation for β-NTO showed no discontinuity in the shock wave velocity range of 3–11 km/s. These studies showed that MD by using a suitable ReaxFF-lg parameter set, could provided detailed atomistic information to explain the shock-induced complex reaction mechanisms of energetic materials. With the ReaxFF-MD coupling MSST method and a cheap computational cost, one could also obtain the deformation behaviors and equation of states for energetic materials under conditions of extreme pressure. Full article

Graph convolutional networks (GCNs) have brought considerable improvement to the skeleton-based action recognition task. Existing GCN-based methods usually use the fixed spatial graph size among all the layers. It severely affects the model’s abilities to exploit the global and semantic discriminative information due to the limits of receptive fields. Furthermore, the fixed graph size would cause many redundancies in the representation of actions, which is inefficient for the model. The redundancies could also hinder the model from focusing on beneficial features. To address those issues, we proposed a plug-and-play channel adaptive merging module (CAMM) specific for the human skeleton graph, which can merge the vertices from the same part of the skeleton graph adaptively and efficiently. The merge weights are different across the channels, so every channel has its flexibility to integrate the joints. Then, we build a novel shallow graph convolutional network (SGCN) based on the module, which achieves state-of-the-art performance with less computational cost. Experimental results on NTU-RGB+D and Kinetics-Skeleton illustrates the superiority of our methods. Full article

The functionalization of C–H bonds has become a major thread of research in organic synthesis that can be assessed from different angles, for instance depending on the type of catalyst employed or the overall transformation that is carried out. This review compiles recent progress in synthetic methodology that merges the functionalization of C–H bonds along with the cleavage of C–C bonds, either in intra- or intermolecular fashion. The manuscript is organized in two main sections according to the type of substrate in which the cleavage of the C–C bond takes place, basically attending to the scission of strained or unstrained C–C bonds. Furthermore, the related research works have been grouped on the basis of the mechanistic aspects of the different transformations that are carried out, i.e.,: (a) classic transition metal catalysis where organometallic intermediates are involved; (b) processes occurring via radical intermediates generated through the use of radical initiators or photochemically; and (c) reactions that are catalyzed or mediated by suitable Lewis or Brønsted acid or bases, where molecular rearrangements take place. Thus, throughout the review a wide range of synthetic approaches show that the combination of C–H and C–C cleavage in single synthetic operations can serve as a platform to achieve complex molecular skeletons in a straightforward manner, among them interesting carbo- and heterocyclic scaffolds. Full article

Effective 3D tree reconstruction based on point clouds from terrestrial Light Detection and Ranging (LiDAR) scans (TLS) has been widely recognized as a critical technology in forestry and ecology modeling. The major advantages of using TLS lie in its rapidly and automatically capturing tree information at millimeter level, providing massive high-density data. In addition, TLS 3D tree reconstruction allows for occlusions and complex structures from the derived point cloud of trees to be obtained. In this paper, an automatic tree skeleton extraction approach based on multi-view slicing is proposed to improve the TLS 3D tree reconstruction, which borrowed the idea from the medical imaging technology of X-ray computed tomography. Firstly, we extracted the precise trunk center and then cut the point cloud of the tree into slices. Next, the skeleton from each slice was generated using the kernel mean shift and principal component analysis algorithms. Accordingly, these isolated skeletons were smoothed and morphologically synthetized. Finally, the validation in point clouds of two trees acquired from multi-view TLS further demonstrated the potential of the proposed framework in efficiently dealing with TLS point cloud data. Full article