# Force-Sensorless Identification and Classification of Tissue Biomechanical Parameters for Robot-Assisted Palpation

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Simultaneous estimation of robot end-effector forces and velocities, using only joint position sensors and commanded torques.
- Estimation of tissues’ biomechanical parameters based on the estimated forces and velocities.
- A standard robot manipulator model is employed instead of an ad hoc system.
- Classification of tissues is based on the estimated parameters taking into account a linear model, a nonlinear model, and the combination of both models, giving, as a result, a better classification for the last case.

## 2. Materials and Methods

#### 2.1. Robot and Environment Model

**Property**

**1**

**.**The inertia matrix $\mathit{H}\left(\mathit{q}\right)\in {\mathbb{R}}^{n\times n}$ is symmetric and positive definite for all $\mathit{q}\in {\mathbb{R}}^{n}$.

**Assumption**

**1.**

#### 2.2. Velocity and Force Observer

**Assumption**

**2.**

#### 2.3. Parameter Estimation

#### 2.4. Closed-Loop Force Control

#### 2.5. Tissue Classification

## 3. Results

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

FTO | Finite-Time Observer |

BC | Bayesian Classifier |

SBC | Simple Bayesian Classifier |

## Appendix A. Dyamic Model and Parameters for the Touch Manipulator

Parameter | Estimated Value |
---|---|

${\pi}_{1}$ | 0.00027632 |

${\pi}_{2}$ | 0.00056422 |

${\pi}_{3}$ | 0.00084797 |

${\pi}_{4}$ | 0.01706838 |

${\pi}_{5}$ | 0.01920879 |

${\pi}_{6}$ | 0.01622868 |

${\pi}_{7}$ | 0.00357954 |

${\pi}_{8}$ | 0.00551499 |

## References

- Dario, P.; Bergamasco, M. An advanced robot system for automated diagnostic tasks through palpation. IEEE Trans. Biomed. Eng.
**1988**, 35, 118–126. [Google Scholar] [CrossRef] [PubMed] - Talasaz, A.; Patel, R.V. Integration of force reflection with tactile sensing for minimally invasive robotics-assisted tumor localization. IEEE Trans. Haptics
**2012**, 6, 217–228. [Google Scholar] [CrossRef] [PubMed] - Meli, L.; Pacchierotti, C.; Prattichizzo, D. Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation. Int. J. Med. Robot. Comput. Assist. Surg.
**2017**, 13, e1809. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Konstantinova, J.; Cotugno, G.; Dasgupta, P.; Althoefer, K.; Nanayakkara, T. Palpation force modulation strategies to identify hard regions in soft tissue organs. PLoS ONE
**2017**, 12, e0171706. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Salman, H.; Ayvali, E.; Srivatsan, R.A.; Ma, Y.; Zevallos, N.; Yasin, R.; Wang, L.; Simaan, N.; Choset, H. Trajectory-optimized sensing for active search of tissue abnormalities in robotic surgery. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 5356–5363. [Google Scholar]
- Zevallos, N.; Srivatsan, R.A.; Salman, H.; Li, L.; Qian, J.; Saxena, S.; Xu, M.; Patath, K.; Choset, H. A surgical system for automatic registration, stiffness mapping and dynamic image overlay. In Proceedings of the 2018 International Symposium on Medical Robotics (ISMR), Atlanta, GA, USA, 1–3 March 2018; pp. 1–6. [Google Scholar]
- Xiao, B.; Xu, W.; Guo, J.; Lam, H.K.; Jia, G.; Hong, W.; Ren, H. Depth estimation of hard inclusions in soft tissue by autonomous robotic palpation using deep recurrent neural network. IEEE Trans. Autom. Sci. Eng.
**2020**, 17, 1791–1799. [Google Scholar] [CrossRef] - Saracino, A.; Oude-Vrielink, T.J.; Menciassi, A.; Sinibaldi, E.; Mylonas, G.P. Haptic intracorporeal palpation using a cable-driven parallel robot: A user study. IEEE Trans. Biomed. Eng.
**2020**, 67, 3452–3463. [Google Scholar] [CrossRef] [PubMed] - Scimeca, L.; Hughes, J.; Maiolino, P.; He, L.; Nanayakkara, T.; Iida, F. Action augmentation of tactile perception for soft-body palpation. Soft Robot.
**2022**, 9, 280–292. [Google Scholar] [CrossRef] [PubMed] - Yan, Y.; Pan, J. Fast localization and segmentation of tissue abnormalities by autonomous robotic palpation. IEEE Robot. Autom. Lett.
**2021**, 6, 1707–1714. [Google Scholar] [CrossRef] - Gupta, M.; Rao, B.P.; Rajagopalan, V. Brain tumor detection in conventional MR images based on statistical texture and morphological features. In Proceedings of the 2016 International Conference on Information Technology (ICIT), Bhubaneswar, India, 22–24 December 2016; pp. 129–133. [Google Scholar]
- Raja, P.S. Brain tumor classification using a hybrid deep autoencoder with Bayesian fuzzy clustering-based segmentation approach. Biocybern. Biomed. Eng.
**2020**, 40, 440–453. [Google Scholar] [CrossRef] - Chang, Y.; Sharp, G.C.; Li, Q.; Shih, H.A.; El Fakhri, G.; Ra, J.B.; Woo, J. Subject-specific brain tumor growth modelling via an efficient Bayesian inference framework. In Proceedings of the Medical Imaging 2018: Image Processing, Houston, TX, USA, 11–13 February 2018; Volume 10574, pp. 656–661. [Google Scholar]
- Afshar, P.; Mohammadi, A.; Plataniotis, K.N. BayesCap: A Bayesian approach to brain tumor classification using capsule networks. IEEE Signal Process. Lett.
**2020**, 27, 2024–2028. [Google Scholar] [CrossRef] - De Luca, A.; Mattone, R. Sensorless robot collision detection and hybrid force/motion control. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; pp. 999–1004. [Google Scholar]
- Arteaga-Pérez, M.A.; Rivera-Dueñas, J.C.; Gutiérrez-Giles, A. Velocity and force observers for the control of robot manipulators. J. Dyn. Syst. Meas. Control
**2013**, 135, 064502. [Google Scholar] [CrossRef] - Gutiérrez-Giles, A.; Arteaga-Pérez, M.A. GPI based velocity/force observer design for robot manipulators. ISA Trans.
**2014**, 53, 929–938. [Google Scholar] [CrossRef] [PubMed] - Wahrburg, A.; Morara, E.; Cesari, G.; Matthias, B.; Ding, H. Cartesian contact force estimation for robotic manipulators using Kalman filters and the generalized momentum. In Proceedings of the 2015 IEEE International Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden, 24–28 August 2015; pp. 1230–1235. [Google Scholar]
- Cao, P.; Gan, Y.; Dai, X. Finite-time disturbance observer for robotic manipulators. Sensors
**2019**, 19, 1943. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Heredia-Pérez, S.A.; Harada, K.; Padilla-Castañeda, M.A.; Marques-Marinho, M.; Márquez-Flores, J.A.; Mitsuishi, M. Virtual reality simulation of robotic transsphenoidal brain tumor resection: Evaluating dynamic motion scaling in a master-slave system. Int. J. Med. Robot. Comput. Assist. Surg.
**2019**, 15, e1953. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Teodoro-Vite, S.; Pérez-Lomelí, J.S.; Domínguez-Velasco, C.F.; Hernández-Valencia, A.F.; Capurso-García, M.A.; Padilla-Castañeda, M.A. A high-fidelity hybrid virtual reality simulator of aneurysm clipping repair with brain sylvian fissure exploration for vascular neurosurgery training. Simul. Healthc.
**2021**, 16, 285–294. [Google Scholar] [CrossRef] [PubMed] - Ayvali, E.; Ansari, A.; Wang, L.; Simaan, N.; Choset, H. Utility-guided palpation for locating tissue abnormalities. IEEE Robot. Autom. Lett.
**2017**, 2, 864–871. [Google Scholar] [CrossRef] - Beccani, M.; Di Natali, C.; Sliker, L.J.; Schoen, J.A.; Rentschler, M.E.; Valdastri, P. Wireless tissue palpation for intraoperative detection of lumps in the soft tissue. IEEE Trans. Biomed. Eng.
**2013**, 61, 353–361. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Olveres, J.; González, G.; Torres, F.; Moreno-Tagle, J.C.; Carbajal-Degante, E.; Valencia-Rodríguez, A.; Méndez-Sánchez, N.; Escalante-Ramírez, B. What is new in computer vision and artificial intelligence in medical image analysis applications. Quant. Imaging Med. Surg.
**2021**, 11, 3830. [Google Scholar] [CrossRef] [PubMed] - Siciliano, B.; Villani, L. Robot Force Control; Springer Science & Business Media: New York, NY, USA, 1999. [Google Scholar]
- Siciliano, B.; Sciavicco, L.; Villani, L.; Oriolo, G. Robotics: Modelling, Planning and Control; Springer Science & Business Media: London, UK, 2010. [Google Scholar]
- Basafa, E.; Farahmand, F.; Vossoughi, G. A non-linear mass-spring model for more realistic and efficient simulation of soft tissues surgery. Stud. Health Technol. Inform.
**2008**, 132, 23. [Google Scholar] [PubMed] - Shah, R.; Gupta, A. Non-linear cubic spring-mesh model for simulating biological tissues. Trans. Jpn. Soc. Med. Biol. Eng.
**2013**, 51, U-12. [Google Scholar] - Murray, R.M.; Li, Z.; Sastry, S.S.; Sastry, S.S. A Mathematical Introduction to Robotic Manipulation; CRC Press: Boca Raton, FL, USA, 1994. [Google Scholar]
- Spong, M.W.; Hutchinson, S.; Vidyasagar, M. Robot Modeling and Control; Wiley: New York, NY, USA, 2006; Volume 3. [Google Scholar]
- Ljung, L. System identification. In Signal Analysis and Prediction; Springer: New York, NY, USA, 1998; pp. 163–173. [Google Scholar]
- Sucar, L.E. Probabilistic graphical models. In Advances in Computer Vision and Pattern Recognition; Springer: London, UK, 2015; Volume 10, p. 1. [Google Scholar]
- Arteaga, M.A.; Gutiérrez-Giles, A.; Pliego-Jiménez, J. Local Stability and Ultimate Boundedness in the Control of Robot Manipulators; Springer: Cham, Switzerland, 2022. [Google Scholar] [CrossRef]

**Figure 1.**Experimental setup: 3D Systems Omni Touch robot, ATI Nano 17 force sensor (only for validation), and the silicone samples Ecoflex Gel, Ecoflex 00-50, Dragon Skin 10, and Dragon Skin 30 silicone samples.

**Figure 2.**Force tracking and estimation for the Ecoflex 00-50 sample. (

**a**) Forces: desired (- - -), measured (—), and estimated (—). (

**b**) Force tracking error. (

**c**) Force estimation error.

**Figure 4.**Estimation of the linear elasticity coefficients ${k}_{\mathrm{l}}$ in model (3) for the four different rubber samples and their normal probability density functions.

**Figure 5.**Estimation of the nonlinear elasticity coefficients ${k}_{\mathrm{nl}}$ in model (4) for the four different rubber samples and their normal probability density functions.

Parameter | ${\widehat{\mathit{k}}}_{\mathbf{l}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}/\mathbf{m}]$ | ${\widehat{\mathit{b}}}_{\mathbf{l}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}\mathbf{s}/\mathbf{m}]$ | ${\widehat{\mathit{k}}}_{\mathbf{nl}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}/{\mathbf{m}}^{3}]$ | ${\widehat{\mathit{b}}}_{\mathbf{nl}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}\mathbf{s}/\mathbf{m}]$ |
---|---|---|---|---|

Ecoflex Gel | 0.2274 | 0.0047 | 0.013 | $2.02\times {10}^{-9}$ |

Ecoflex 00-50 | 0.4899 | $2.81\times {10}^{-4}$ | 0.1101 | $4.45\times {10}^{-9}$ |

Dragon Skin 10 | 0.5158 | $5.82\times {10}^{-5}$ | 0.1313 | $3.55\times {10}^{-9}$ |

Dragon Skin 30 | 0.7377 | $6\times {10}^{-4}$ | 0.3688 | $3.17\times {10}^{-9}$ |

Parameter | ${\widehat{\mathit{k}}}_{\mathbf{l}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}/\mathbf{m}]$ | ${\widehat{\mathit{b}}}_{\mathbf{l}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}\mathbf{s}/\mathbf{m}]$ | ${\widehat{\mathit{k}}}_{\mathbf{nl}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}/{\mathbf{m}}^{3}]$ | ${\widehat{\mathit{b}}}_{\mathbf{nl}}\phantom{\rule{0.166667em}{0ex}}[\mathbf{N}\mathbf{s}/\mathbf{m}]$ |
---|---|---|---|---|

Ecoflex Gel | 0.0191 | 0.0016 | 0.0013 | $2.11\times {10}^{-9}$ |

Ecoflex 00-50 | 0.0314 | $7.82\times {10}^{-4}$ | 0.0051 | $6\times {10}^{-9}$ |

Dragon Skin 10 | 0.0364 | $4.45\times {10}^{-4}$ | 0.0051 | $3.8\times {10}^{-9}$ |

Dragon Skin 30 | 0.0507 | $3.17\times {10}^{-4}$ | 0.0089 | $3.14\times {10}^{-9}$ |

**Table 3.**Percentage of correct classifications when considering ${\widehat{k}}_{\mathrm{l}}$ as the BC input.

Material | Ecoflex Gel | Ecoflex 00-50 | Dragon Skin 10 | Dragon Skin 30 |

% Correct class. | 100 | 50 | 50 | 100 |

**Table 4.**Percentage of correct classifications when considering ${\widehat{k}}_{\mathrm{nl}}$ as the BC input.

Material | Ecoflex Gel | Ecoflex 00-50 | Dragon Skin 10 | Dragon Skin 30 |

% Correct class. | 100 | 96.88 | 100 | 100 |

**Table 5.**Percentage of correct classifications when considering ${\widehat{k}}_{\mathrm{l}}$ and ${\widehat{k}}_{\mathrm{nl}}$ for the SBC.

Material | Ecoflex Gel | Ecoflex 00-50 | Dragon Skin 10 | Dragon Skin 30 |

% Correct class. | 100 | 100 | 100 | 100 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Gutierrez-Giles, A.; Padilla-Castañeda, M.A.; Alvarez-Icaza, L.; Gutierrez-Herrera, E.
Force-Sensorless Identification and Classification of Tissue Biomechanical Parameters for Robot-Assisted Palpation. *Sensors* **2022**, *22*, 8670.
https://doi.org/10.3390/s22228670

**AMA Style**

Gutierrez-Giles A, Padilla-Castañeda MA, Alvarez-Icaza L, Gutierrez-Herrera E.
Force-Sensorless Identification and Classification of Tissue Biomechanical Parameters for Robot-Assisted Palpation. *Sensors*. 2022; 22(22):8670.
https://doi.org/10.3390/s22228670

**Chicago/Turabian Style**

Gutierrez-Giles, Alejandro, Miguel A. Padilla-Castañeda, Luis Alvarez-Icaza, and Enoch Gutierrez-Herrera.
2022. "Force-Sensorless Identification and Classification of Tissue Biomechanical Parameters for Robot-Assisted Palpation" *Sensors* 22, no. 22: 8670.
https://doi.org/10.3390/s22228670