Research on Posture Sensing and Error Elimination for Soft Manipulator Using FBG Sensors

Abstract

Fiber-optic sensors are highly promising within soft robot sensing applications, but sensing methods based on geometry-based reconstruction limit the sensing capability and range. In this study, a fiber-optic sensor with a different deployment strategy for indirect sensing to monitor the outside posture of a soft manipulator is presented. The internal support structure’s curvature was measured using the FBG sensor, and its mapping to the external pose was then modelled using a modified LSTM network. The error was assumed to follow the Gaussian distribution in the LSTM neural network and was rectified by maximum likelihood estimation to address the issue of noise generated during the deformation transfer and curvature sensing of the soft structure. For the soft manipulator, the network model’s sensing performance was demonstrated. The proposed method’s average absolute error for posture sensing was 63.3% lower than the error before optimization, and the root mean square error was 56.9% lower than the error before optimization. The comparison results between the experiment and the simulation demonstrate the viability of the indirect measurement of the soft structure posture using FBG sensors based on the data-driven method, as well as the significant impact of the error optimization method based on the Gaussian distribution assumption.

1. Introduction

Owing to advantages in aspects such as flexibility, safety, and comfort, soft robots have shown to be highly promising in clinical medicine [1,2,3]. With high degrees of freedom and a flexible structure, soft robots provide new ideas and directions for resolving the bottleneck problems of clinical operations [4,5]. However, the soft structure presents a great challenge to the posture sensing of soft robots, because it is hard to juggle accuracy, safety, and portability.

One of the most common sensing methods involves electronic sensors, which are widely used in rigid robots. The technique is also extended to sensing fields of soft robots with the development of material science and nanotechnology. One of the most accurate sensing methods involves the electromagnetic sensor, which can transduce a position signal into an electrical signal [6]. By laying them in the soft robot, the strain and change in the position of particular spots on the robot structure could be monitored. The limiting factors of the electromagnetic sensor are its high price, magnetic susceptibility, and stiffness, which make it hard to integrate enough sensors into robots. Inspired by biological structures, various kinds of flexible electronic sensors have been developed and applied in soft robots [7,8]. Besides common piezoresistive sensors [9,10], some sensors based on carbon fiber material [11], gel or liquid material [12,13], optical fiber [14,15,16,17,18], and other smart materials are also used in soft robot sensing. Among them, fiber Bragg grating (FBG) sensors have attracted significant research attention due to its advantages of softness, safety, chemical stability, and anti-electromagnetic interference.

The application of FBG sensors on soft robots has been studied for years. The FBG sensor is sensitive to physical quantity, temperature, strain, and curvature. By measuring the optical intensity through the fiber gratings, the sensor could detect changes in the curvature and shape. Due to the flexibility and softness of soft robots and FBG sensors, the current study faces difficulties in improving accuracy and reliability. In most studies, researchers have used FBG sensors that collect the wavelength drift in each fiber core, and have built models based on the strain, curvature, and reconstruction of the fiber shape [19]. Moreover, by using the fiber shape information, they can deduce the posture information of the robot. In this process, errors could be introduced from three sources [20,21]:

• The arrangement of optical fibers in the soft robot cannot accurately follow the geometric assumptions. This is caused by the manufacturing error of soft robots and FBG sensors, as well as other aleatoric factors.
• Errors may be introduced in the reconstruction algorithm of FBG sensing as models can never be completely accurate, especially for soft structures. This may be caused by the unexpected impacts of models on sensors and robots, such as body twisting and temperature changes.
• Due to the discrete distribution of sensing points on the fiber, the bending information between the two points cannot be detected. Although the interpolation method is constantly optimized, the error caused by the lack of original information cannot be eliminated.

Many kinds of FBG arrangement plans and sensing methods have been proposed in recent years, which mostly focus on proposing reconstruction algorithms with more practical models and higher accuracy [22] used the inverse finite element method to build a sensing model of FBG sensors to monitor the shape and structural health of a sandwich panel. The authors of [23] considered twisting the FBG sensors and proposed a twisting compensation method for the shape sensing of flexible medical instruments. The authors of [24] used the stretchable optical fiber sensor to monitor the shape of soft arm and designed reconstruction algorithms for the fiber network. Some of these methods work well in reducing the error from Source 2 mentioned above. However, considering the changeability and instability of the soft structure, Source 1 and 3 mentioned above, methods based on geometry deduction are not so usable. Another popular solution is neural networks, which have been proven to be effective in soft robot sensing and modeling [25,26]. With sufficient data for training, neural networks could create a mapping model between FBG sensing information and the posture information. For Source 3, it could be reduced by establishing suitable FBG sensor networks in the soft structure. In this situation with septate structure between sensors and sensing targets, using neural networks to establish the sensing model is a promising strategy [27,28]. Nevertheless, most of the previous studies on neural networks and FBG sensing took the training dataset as an actual value without any error, and only a few of them considered the noise in the dataset. The noise in the dataset leads to aleatoric uncertainty in the measurement results. To deal with the aleatoric uncertainty, the authors of [29,30] used Bayesian neural networks to model the uncertainty of the data. Moreover, the authors of [31] formulated a probabilistic framework that learns to capture the aleatoric and epistemic uncertainty from data via deep learning, but for only one-dimensional motion and fuzzy sensor data.

In this study, we proposed a sensing method to monitor the posture of the soft manipulator using multi-core FBG sensors. To deal with (3) mentioned above, we implanted FBG sensors into the manipulator, meaning that the strain of the manipulator could be transferred from outside to the sensor through the soft structure smoothly and continuously. Furthermore, to deal with (1) and (2), we used a long short-term memory (LSTM) neural network to establish the mapping relations between the sensor data and outside posture information. Moreover, the output layer of the network was modified based on Gaussian distribution, which the unknown noise and error were assumed to obey. Moreover, the deformation and posture were analyzed and simulated by ANSYS software to demonstrate its feasibility. Finally, the output results from the network were compared with the actual value to verify the effectiveness, accuracy, and reliability of the sensing ability of soft manipulator’s posture through FBG sensors.

2. Materials and Methods

2.1. Materials

The soft part of the manipulator was made of Agilus rubber using 3D printing and the joining was made of acrylonitrile butadiene styrene (ABS). The rubber and ABS were purchased from Guangzhou Xingyou Tech Co., Ltd. (Guangzhou, China) The 3M pr100 glue was purchased from Shanghai Niankai Electronic Technology Co., Ltd. (Shanghai, China) The optical fiber was purchased from Changfei Optical Fiber and Cable Co., Ltd. (Wuhan, China) and was written with Bragg gratings in our lab with the phase mask method.

2.2. Structure of the Soft Manipulator

The manipulator used in this paper was formed from two soft parts and one rigid part. The soft part was a pneumatic-based soft continuum tentacle, which is shown in Figure 1. The tentacle had a length of 108 mm and contained three parallel air chambers. The wall of the tentacle was designed with a convex and concave structure to enhance the bending capability and reduce radial expansion. The maximum outer diameter of the tentacle was 26 mm and its minimum inner diameter was 12.5 mm, while the thickness of the chamber walls was 3 mm. The tentacle could bend in different directions by inflating particular air chambers. A plurality of horizontal binding wires was arranged on the surface to keep the stability of the structure in a large deformation. To maintain the structure stability, no more than two chambers could be added at one time due to the air pressure. The fiber-optic sensor was implanted into the slot on the center to gather the curvature information and ensure collision protection.

2.3. FBG Sensing Principle

FBG sensors can obtain curvature information by detecting the change in optical signal due to the strain-sensitive character of the Bragg gratings [32]. Its center wavelength will shift with the strain change, which obeys:

$$\mathrm{\Delta }{\lambda }_B={\lambda }_B(1-P_e)\cdot \epsilon$$

(1)

where ${\lambda }_B$ is the center wavelength of the FBG core, $\mathrm{\Delta }{\lambda }_B$ is the variation of ${\lambda }_B$, $\epsilon$ is the average strain of the sensing point, and $P_e$ is the elasto-optic coefficient of the FBGs which is assumed invariable for one fiber sensor.

When the grating bends, as shown in Figure 2, the relations between the strain and the curvature are described as follows:

$$\epsilon =\frac{ds-dl}{dl}=\frac{\left(\rho +\delta \right)d\theta -\rho d\theta }{\rho d\theta }=\frac{\delta }{\rho }=k\delta$$

(2)

where $k$ is the curvature, $\rho$ is the radius of the curvature, and $\delta$ is the distance between the center of the peripheral fiber core and the neutral axis.

The section of the optical fiber is shown in Figure 3, where φ is the included angle between fiber core a and the neutral axis. This stands for the curvature direction of the FBG. The included angles (β_a, β_b, and β_c) are all 120° and (r_a, r_b, and r_c) are the known parameters. The system of strain equations at any position can be expressed as:

$$\left\{\begin{array}{c}{E}_a=k{\delta }_a=kr\sin (\phi )+E_0\\ E_b=k{\delta }_b=kr\sin (\phi +\frac{2\pi }{3})+E_0\\ E_c=k{\delta }_c=kr\sin (\phi +\frac{4\pi }{3})+E_0\end{array}\right.$$

(3)

where $r=r_a=r_b=r_c$ and E₀ is a constant caused by changes in temperature.

The deflection angle and curvature of the grating point can be obtained by the following:

$$\phi =\frac{\pi }{6}+\arctan \left(-\sqrt{3}\left(\frac{E_a-E_b}{2E_c-E_a-E_b}\right)\right)$$

(4)

$$k=\frac{E_b-E_c}{\sqrt{3}r\cos \left(\frac{\pi }{6}+\arctan \left(-\sqrt{3}\left(\frac{E_a-E_b}{2E_c-E_a-E_b}\right)\right)\right)}$$

(5)

The curvature information obtained above partially describes the posture of the center axis of the manipulator.

2.4. Design of the LSTM Neural Network

As the FBG sensors were implanted in the manipulator, it was hard to build a sensing model because of the impacts of the super elasticity and the unpredictable environment. However, the neural network has a strong ability to conduct information fitting, which makes it possible to deal with the curvature sensing information with greater efficiency. As a result, we proposed a method to deal with the sensor data to obtain the outside posture information by LSTM network, which is especially effective for series data, such as the FBG curvature data of a single optical fiber. Using the neural network to build the sensing model instead of theoretical derivation raises two problems:

• As it is a data-driven method, training errors can be introduced no matter what optimizer, hyperparameter, and network structure can be chosen.
• Because of the “black-box” character of the neural network, the information processing is incomprehensible for people, which makes the sensing results unreliable to a certain degree. The network just gives numbers, and people have to decide whether to believe it blindly.

Noise exists in each step of sensing and introduces errors, which accumulates to the final result following several steps. To quantify and correct the error, in this paper we assumed that the error caused by the aleatoric noise obeys Gaussian distribution. To describe the posture of the manipulator properly, we used a probability distribution

$$p(y|x)=N(\mu (x),{\sigma }^2(x))$$

(6)

to obtain the mapping relations between the sensing data and the posture information. For each dimension of the posture information, a mean value and a variance value were required to ascertain the Gaussian distribution. The aim of the neural network was to obtain a distribution with the mean that was as close as possible to the true posture value and to simultaneously make the variance low enough. The proper loss function could be derived based on the maximum likelihood of Gaussian distribution:

$$\mathrm{L}=\frac{1}{2}{\displaystyle \sum _{k=1}^d{s}_k+\frac{{(y_k-{\mu }_k)}^2}{e^{s_k}}}$$

(7)

where d is the dimension of posture data, and $\mu =\mu (x)$ and $s=\log {\sigma }^2(x)$ are the outputs of the network, representing the vectors of the mean prediction and log predictive variance of the distribution, respectively.

The hidden layer of networks contains three LSTM layers and connects a dense layer to the output. The LSTM network is known for its outstanding performance on time series analysis. In this study, the sensing data were arranged in chronological order and learning by the network in the same order. The whole process of the system is shown in Figure 4.

3. Experiments and Results

3.1. Simulation

In this section we used SolidWorks 3D design software and the Ansys fluid–solid coupling simulation module to complete the finite element simulation of the deformation of the soft structure in order to validate whether the neural network could accurately establish the mapping relationship between the central axis posture and the outside posture of the soft manipulator. The simulation was arranged, as shown in Figure 5. We set some points on the soft actuator and tracked them during the deformation process. The points were then arranged into four columns, as shown in Figure 5a, with one on the central axis and three on the outside surface evenly. Each column on the outside surface contained seven equidistant points to describe the outside posture of the soft actuator. To collect the posture data of the central axis, the central column was divided into five equidistant groups and each group contained three points, 3 mm apart from each other. If we assumed the small segment to be an arc, then the curvature data of each group, including $\phi$ and $k$, could be obtained using the three points on the segment. Above all, according to the position information of the points, the curvature data, and the posture data of each deformation were obtained and recorded. Based on the curvature data and the posture data of the simulation results, we could establish the training and testing datasets for the network model.

The loading conditions in the simulation are as follows. For the training dataset, a random force of 0~0.5 N was applied in the lateral direction at a random external position, and an input air pressure value of 0~6.0 kPa (with an interval of 1.0 kPa) was applied to each of the three air inlets. We made the interval 1.0 kPa to ensure that the training dataset was sparse enough because the dense dataset could lead to model overfitting and make the validation meaningless. Moreover, for the validation and testing datasets, the force load was the same and the pressure produced random values of 0~6.0 kPa. As described above, 180 pieces of data were collected for training, 30 pieces of data were collected for validation, and 90 pieces of data were collected for testing.

We used a common LSTM neural network trained with a root mean square error (RMSE) loss to validate the mapping function of the neural network models. The model was three layers deep with a hidden size of 256-256-128 and a dropout probability of 0.5, and learned the training dataset until convergence. Subsequently, we tested the model by calculating the 3D average absolute errors (MAEs) on the testing dataset. The average MAE on the testing dataset was 3.25 mm, i.e., less than 8% of the maximum deformation. According to the testing results, the neural network could give a practical and feasible mapping model between the curvature information and the posture information.

3.2. Experiment Platform and Devices

The experimental platform for the soft manipulator posture sensing system is depicted in Figure 6. It is composed of four functional modules: the driving module, which regulated the soft manipulator’s input air pressure; the fiber-optic sensing module, which gathered data from the optical sensors; the visual positioning module, which gathered data from the soft manipulator’s posture by the marker points; and the motion module, which contained the main part of the manipulator. Six proportional valves formed part of the driving module. The manipulator consisted of two actuators. The six air inlets of the soft manipulator were connected to the proportional valves by hoses, and the proportional valves were connected to the air compressor via hoses. The multi-core FBG sensor was implanted in the center axis of the soft manipulator. Five FBG sensing points, each with a gate length of 6 mm and center wavelengths of 1532.4076 nm, 1536.2301 nm, 1540.8751 nm, 1544.1413 nm, and 1548.3529 nm, were evenly distributed on the fiber and connected to the demodulator through the fan-in–fan-out module, with a demodulator rate of 35 kHz and a minimum resolution of 0.5 dBm. Two 12.4 mega-pixel industrial cameras were housed in the visual positioning module and mounted at a 45° angle to the front of the flexible manipulator module. On the one side of the flexible manipulator’s surface, seven reflecting marker points were taped, and the V-STARS industrial camera positioning system was utilized to recover the real-time 3D spatial coordinates of the marker points using multiple views. The position information was used to describe the outside posture of the manipulator.

Except for the soft manipulator motion module, the other three modules were connected to the upper computer. The wavelength data of the FBG sensors, the air pressure data of the driving module, and the visual data of the marker points were collected and transmitted to the computer. The aim of the experiment was to predict the 3D coordinates of the seven marker points using the limited sensing data. The multi-vision information of the marker points was collected and converted into 3D coordinates using the V-STARS industrial software. The wavelength drift of the FBG sensing points was converted into curvature information, according to Section 2.3. The air pressure contained six-dimensional information for the six air inlets to the soft manipulator and the FBG sensing contained ten-dimensional information for the $(\phi ,k)$ values of the five FBG sensing points. Both kinds of information were used as the input of the neural network, and the output of the neural network was the mean value and the variance in the Gaussian distribution of 3D coordinates of the marker points.

3.3. Dataset Building and Model Training

To test the LSTM network’s ability to correct errors, datasets were created in this section of experimental results and the models were trained on the datasets based on the simulation results and experimental results.

3.3.1. Dataset Building from the Experiment

Six variation sequences were randomly generated for the six air inlets of the soft manipulator. The sequences were generated with a total duration of 3 h and a frequency of 1 Hz, which was limited to the range of 0–4.8 kPa. Six separated and electronically controlled proportional valves controlled the air pressure variation in the inlets of the flexible manipulator, according to this sequence. The sampling frequency of the FBG sensing data, the driving air pressure data, and the marker point position data were all 1 Hz. The training set of the network was constructed by combining the above data based on the corresponding time series, and a validation set of 10 min and a test set of 30 min were built by the same method.

3.3.2. Model Training

The modified LSTM network was used to train both the simulated and experimental results. LSTM networks in both used the same hidden layer architecture, while the input and output layers were modified for the different information dimensions. For the simulation datasets, the input data contained the center curvature data (calculated by points on the center axis) with 10 dimensions and the output data contained the coordinates of the outside points with 63 dimensions. For the experiment datasets, the input data contained the center curvature (calculated by the wavelength data from the FBG sensor) with 10 dimensions and the output data contained the coordinates of the outside points with 21 dimensions. However, as the model was supposed to give both mean values and variances in the Gaussian distributions of the outside posture, the size of the output layers was doubled.

The appropriate hyperparameters were found by continuously adjusting the hyperparameters and observing the validation results on the validation set. The data in the dataset were normalized between [−1 and 1] before training using the linear normalization technique. The data sequences from the training set were randomly arranged and inputted into the neural network in a batch size of 32 with a sequence length 30, and the network selected the Adam optimizer with an initial learning rate of 0.001.

3.4. Results and Discussions

In this section, we investigate the mean absolute error (MAE) and the root mean square error (RMSE) of the pose sensing findings on the test set as evaluation indices to see if the method proposed in this study effectively corrected the influence of the systematic error based on the accuracy of the model. Meanwhile, the RMSE, i.e., the typical loss function of the neural networks, can not only quantify the performance of neural networks, but also indirectly indicate the dispersion degree of error and the stability of the approach, and the MAE is the most intuitive way of comparing the sensing results with the real values. The calculation formulas are as follows:

$$MAE=\frac{1}{d}{\displaystyle \sum _{k=1}^d\left|y_k-u_k\right|}$$

(8)

$$RMSE=\sqrt{\frac{1}{d}{\displaystyle \sum _{k=1}^d{(y_k-u_k)}^2}}$$

(9)

where y is the real posture data, u is the output data of the model, and d is the dimension of the posture data. d = 21 (the 3D coordinate data for 7 points) for the experimental data and d = 63 (the 3D coordinate data for 21 points) for the simulation data.

For comparison purposes, we used the conventional LSTM-based data-driven method without error correction, utilizing the RMSE as the loss function to fit the results directly to the posture data, and we recorded the MAE and the RMSE of the two approaches on the test set.

Table 1. The MAE and RMSE of models on the simulation and experiment results (Unit: mm).

Dataset	LSTM Network Model	MAE	RMSE
simulation	Unoptimized	2.1	3.6
	Optimized	2.0	3.1
experiment	unoptimized	7.9	10.2
	Optimized	2.9	4.4

displays the results which show that the accuracy significantly improved. The MAE on the test set fell by 63.3% when compared to that of the previous method. The traditional approach’s tendency to overfit due to systematic data noise was substantially corrected, as evidenced by the large reduction in the RMSE, which also reduced the error’s dispersion and increased the stability of the model.

The LSTM network did not significantly optimize the recognition results on the simulated dataset, showing the range of capabilities of the modified LSTM neural network in terms of error correction. Except for the errors caused by underfitting, in real experiments, the equipment and environmental noise and the unknown factors of the soft structure could lead to errors in the training results. The different performances of the LSTM neural networks between the two data sets show that the random errors introduced in the real experiments were well corrected.

We selected continuous pose sensing results from the test set of five minutes in length, from which the 3D position information of the seventh marker point was extracted in comparison with the true value, in order to confirm the pose sensing capability of the method and ascertain whether the variance values were intuitively indicative. The results are displayed in Figure 7. The model’s sensing ability was good in all three directions, and it can be seen that the variance tended to fluctuate roughly synchronously with the model’s accuracy estimation, and with the area corresponding to the variance expanding in the region with bigger errors.

In order to further confirm the validity and accuracy of the sensing results of this method for the manipulator posture, Figure 8 illustrates a comparison of the reconstruction results with the real values under selective driving air pressure. Figure 8 shows that the reconstruction results of the soft manipulator posture using the present method generally correspond with the real situation.

The MAE of the posture sensing results under all air pressure input cases was counted and compared with the conventional uncorrected method in this section, and the statistical information is shown in Figure 9. This section aims to determine the posture sensing performance and error correction capability of this method under different air pressure conditions and further verify the sources of the influencing factors of the system error. The X-axis and Y-axis distributions represent the average values of the air pressure applied to the air entry holes on both sides of the soft manipulator. Figure 8 shows how the model’s capacity to identify the soft manipulator was impacted by the increase in the input air pressure, corroborating the results of the study based on the air-pressure-related error level. The flexible manipulator’s air cavity’s shape and structure varied significantly when the air pressure increased, reducing structural stability and affecting the data accuracy. It is also clear that this method efficiently decreased the variation in the error and enhanced the accuracy and stability of the findings in addition to reducing the posture error through the quantitative correction of the error.

4. Conclusions and Future Development

For the posture measurement of soft manipulators, we proposed a posture measurement method based on the optimized LSTM neural network in this paper. The FBG sensor’s wavelength–curvature transfer model was first constructed in this research, and an LSTM network was used to map it to the soft manipulator’s external position. The systematic error of the sensing information was addressed, and the model training accuracy was increased by approximating the heteroskedasticity Gaussian distribution of the soft manipulator posture information in the output layer. Lastly, the soft manipulator was used to test the method’s effectiveness. The experimental results demonstrate that the proposed method could improve the posture sensing accuracy, as the average absolute error of posture sensing on the test set was 2.9 mm, which was 63.3% lower than that before optimization, and the root mean square error was 4.4 mm, which was 56.9% lower than that before optimization. The comparative results between the experiment and the simulation demonstrate the viability of the indirect measurement of soft structure posture using FBG sensors based on the data-driven method, as well as the significant impact of error optimization method based on Gaussian distribution assumption.

This paper proposed a generalizable error reduction method involving the use of the FBG sensor to monitor the soft structure state. This method also provided an indicated value of uncertainty of the estimation results, which is integral to the reliability and interpretability of the model. In the current experiment, a fiber sensor with five gratings was applied on a soft manipulator with two sections of the actuator. The accuracy of the system can be improved by using devices with higher precision. Furthermore, for soft structures with greater complexity, more efficient and complex sensor networks could be designed and applied correspondingly, and the model could be easily extended to these situations by modifying the input and output layers. The robotic sensing platform proposed in this paper has great application potential in areas such as posture sensing for medical surgical robots and shape monitoring for underwater robots.

The current problem based on whether the complex sensing system could bring data redundancy and performance degradation to the model is yet to be solved. Future studies should focus on optimizing network performance and achieving real-time posture sensing on complex soft structures.