# Polymer Microgripper with Autofocusing and Visual Tracking Operations to Grip Particle Moving in Liquid

^{*}

## Abstract

**:**

## 1. Introduction

## 2. System Design and Installation

#### 2.1. Micro Gripper System and Moving Manipulator System

#### 2.2. Object Platform

#### 2.3. Autofocusing Stage

^{3}. The stage is driven by stepping motor with 1.8 degree/step through a gear train of 4.54 ratio. The moving resolution of focusing stage in Z axis by stepping motor is 0.28 μm/degree. The detail components of the image-based Autofocusing Stage are shown in the block diagram of Figure 3. A functional block of the Procedure of autofocusing operation is implemented by PC (Personal computer). The image signal of moving Z position is sensed by CCD through microscope and image capture card. The focusing control through PC can be operated in open-loop or closed-loop mode. The closed-loop control is to realize the autofocusing operation with feedback image signal of moving Z position.

^{®}Core™ 2 CPU. The Clock rate of CPU is 2.13 GHz. The human–machine control interface was implemented by employing Microsoft Foundation Classes. The communication between computer and the subsystems is through image capture card, Morphis (MOR/2VD/84) from Matrox (Quebec, Canada), AD/DA cards, PCI-1727U and PCI-1710HG from Advantech (Kaohsiung, Taiwan), and communication interface by RS-232.

## 3. Autofocusing Operations

#### 3.1. Wavelet-Entropy Focusing Function

_{w}, which was formulated as a ratio between high frequency energy and low frequency energy, provided the best performance.

_{i}: i = 1, …, n} and p

_{i}is the probability of outcome x

_{i}. Next, the probability distribution of signal energy relative to the total signal energy can be expressed:

_{i}is a series of wavelet coefficients and ${\Vert X\Vert}^{2}$ is the l

^{2}-norm of X = {x

_{i}: i = 1, …, n}. The functional in Equation (1) utilizing p

_{i}of Equation (2) in Shannon entropy is non-additive. A one-dimensional additive Shannon entropy may be given as

_{THF}

_{1}and E

_{PHF}

_{2}in the gripping applications needs to be tested.

#### 3.2. Experimental Test and Comparison with Other Focusing Functions

#### 3.2.1. Comparison with Common Focusing Functions

_{THF}

_{1}and E

_{PHF}

_{2}are compared with the common focusing functions: Normalized variance function, Entropy function, Energy Laplace function, Tenenbaum Gradient function, and Sum Modulus Difference (SMD) function. In the wavelet-based algorithms, the mother wavelet was Haar function, i.e., db2 or Daubechies 2. In comparing performance, the evaluated results of autofocusing functions were normalized, as shown in Figure 5. Figure 5 reveals that all focusing functions give the maximum functional values at the same Z position. The Z position at 12 is a correct in-focus position. However, the autofocusing functions E

_{THF}

_{1}and E

_{PHF}

_{2}give sharper curves than those of other autofocusing functions. In addition, the performance of the double decomposition E

_{PHF}

_{2}is somewhat better than that of single decomposition E

_{THF}

_{1}.

#### 3.2.2. Comparison of Wavelet-Based Focusing Functions

_{THF}

_{1}and E

_{PHF}

_{2}was compared with the wavelet-based autofocusing functions by M

_{w}, M1, M2 and M3. Here, the autofocusing functions M1, M2 and M3 correspond to W1, W2, and W3 in [30], respectively. In the comparisons of performance, the evaluated results of autofocusing functions were normalized, as displayed in Figure 6. Figure 6 reveals that all focusing functions give the maximum functional values at the same correct Z position. Among the wavelet-based focusing functions, the present double decomposition E

_{PHF}

_{2}gives the best performance.

_{PHF}

_{2}gives the best performance in focusing both microparticles and gripper tips. Therefore, the E

_{PHF}

_{2}is selected and employed for the depth estimation to align microgripper tips and moving particle in the same focus plane.

#### 3.3. Peak Position Identification and Depth Estimation in Visual Servo

#### 3.3.1. Peak Position Identification

#### 3.3.2. Depth Estimation

## 4. Particle Tracking Process

#### 4.1. PCSS Algorithm

- Step 1.
- Set an interrogation pattern I centered on the object particle i with radius of R. In the pattern I, the other particles are ${i}_{m}$, where $m=1,\dots ,M.$
- Step 2.
- Establish a polar coordinate system with the identified centroid of the particle i as the pole. The relative positions of ${i}_{m}$ to i is obtained by their polar radii ${r}_{im}$ and angles ${\theta}_{im}.$
- Step 3.
- Repeat Steps 1 and 2 for another interrogation pattern J centered on the object particle j with radius of R. In the pattern J, the other particles are ${j}_{n}$, where $n=1,\dots ,N,$ and the relative positions of ${j}_{n}$ to j is obtained by their polar radii ${r}_{jn}$ and angles ${\theta}_{jn}.$
- Step 4.
- Define a similarity coefficient between patterns I and J as$${S}_{ij}={\displaystyle \sum _{n=1}^{N}{\displaystyle \sum _{m=1}^{M}H({\epsilon}_{r}-\left|{r}_{im}-{r}_{jn}\right|,{\epsilon}_{\theta}-\left|{\theta}_{im}-{\theta}_{jn}\right|)}}$$$$H(x,y)=\{\begin{array}{l}1,\text{}x0,\text{}y0\\ 0,\text{\hspace{1em}}otherwise\end{array}$$
- Step 5.
- Calculate the S
_{ij}for all candidate particles in the patterns I and J, and find the matched candidate particle i which gives the maximum value of S_{ij}.

_{ij}in tracking algorithm. Therefore, an enhanced PCSS is proposed to incorporate template matching in the tracking algorithm when the value of S

_{ij}is not well above a threshold value ρ.

#### 4.2. Template Matching

## 5. Gripping Microparticle Tests

#### 5.1. Working Space in Gripping Operation

#### 5.2. Fine Adjustment of Tracking in Z axis

_{max}is the maximum diameter of particle, t

_{sampling}is the sampling time, and V

_{x}and V

_{y}are the estimated particle speed moving in the X and Y direction, respectively. The center of the region is located at the centroid position of the identified particle, which is estimated by the tracking algorithm. Then, a refocusing search is initiated and the focusing measure of E

_{PHF}

_{2}is recorded at each time step. The fine focusing operation will not stop until the peak value of E

_{PHF}

_{2}in the next time step is little degraded. The degraded focusing measure can be detected by setting a threshold value $\delta $:

#### 5.3. Pre-Positioning and Approaching Operations

_{max}. The planner region to pre-position the gripper is depicted in Figure 12. The gripper will stop when the following conditions are satisfied:

#### 5.4. Gripping and Releasing Operations

## 6. Conclusions

_{PHF}

_{2}, which revealed the best performance among the surveyed focus functions, was proposed and utilized to estimate the depth for the alignment of microgripper tips and moving particle in the same focus plane. An enhanced PCSS algorithm incorporated with template matching, edge detection method, and circular Hough Transform was implemented for tracking particles under fluctuating and drifting flow. In the performance tests, the images of experimental results in the processes of tracking, gripping, transporting, and releasing 30–50 μm Polystyrene particle in 25 °C water were captured and displayed. For the experimental tests, the time expense from the beginning of the global search for coarse adjustment to the actual gripping of the particle was less than 50 s. The successful gripping rate of the present apparatus was 8/10 when the object platform was moving in 0.6 mm/s. The failure was mainly due to the adhesion of micro particles after fine adjustment which caused the inaccurate estimation of particle’s coordinates. As a result, the proposed apparatus can perform better when the particle concentration is somewhat low.

## Author Contributions

## Acknowledgments

## Conflicts of Interest

## References

- Jager, E.W.H.; Inganas, O.; Lundstrom, I. Microrobots for micrometer-size objects in aqueous media: Potential tools for single-cell manipulation. Science
**2000**, 288, 2335–2338. [Google Scholar] [CrossRef] [PubMed] - Desai, J.P.; Pillarisetti, A.; Brooks, A.D. Engineering approaches to biomanipulation. Annu. Rev. Biomed. Eng.
**2007**, 9, 35–53. [Google Scholar] [CrossRef] [PubMed] - Carrozza, M.C.; Dario, P.; Menciassi, A.; Fenu, A. Manipulating biological and mechanical micro-objects using LIGA-microfabricated end-effectors. In Proceedings of the IEEE International Conference on Robotics & Automation, Leuven, Belgium, 16–20 May 1998; Volume 2, pp. 1811–1816. [Google Scholar]
- Ok, J.; Chu, M.; Kim, C.J. Pneumatically driven microcage for micro-objects in biological liquid. In Proceedings of the Twelfth IEEE International Conference on Micro Electro Mechanical Systems, Orlando, FL, USA, 17 January 1999; pp. 459–463. [Google Scholar]
- Arai, F.; Sugiyama, T.; Luangjarmekorn, P.; Kawaji, A.; Fukuda, T.; Itoigawa, K.; Maeda, A. 3D Viewpoint Selection and Bilateral Control for Bio-Micromanipulation. In Proceedings of the IEEE International Conference on Robotics & Automation, San Francisco, CA, USA, 24–28 April 2000; pp. 947–952. [Google Scholar]
- Beyeler, F.; Neild, A.; Oberti, S.; Bell, D.J.; Sun, Y. Monolithically fabricated microgripper with integrated force sensor for manipulating microobjects and biological cells aligned in an ultrasonic field. J. Microelectromech. Syst.
**2007**, 16, 7–15. [Google Scholar] [CrossRef] - Feng, L.; Wu, X.; Jiang, Y.; Zhang, D.; Arai, F. Manipulating Microrobots Using Balanced Magnetic and Buoyancy Forces. Micromachines
**2018**, 9, 50. [Google Scholar] [CrossRef] - Potrich, C.; Lunelli, L.; Bagolini, A.; Bellutti, P.; Pederzolli, C.; Verotti, M.; Belfiore, N.P. Innovative Silicon Microgrippers for Biomedical Applications: Design, Mechanical Simulation and Evaluation of Protein Fouling. Actuators
**2018**, 7, 12. [Google Scholar] [CrossRef] - Chronis, N.; Lee, L.P. Electrothermally activated SU-8 microgripper for single cell manipulation in solution. J. Microelectromech. Syst.
**2005**, 14, 857–863. [Google Scholar] [CrossRef] - Hériban, D.; Agnus, J.; Gauthier, M. Micro-manipulation of silicate micro-sized particles for biological applications. In Proceedings of the 5th International Workshop on Microfactories, Becanson, France, 25–27 October 2006; p. 4. [Google Scholar]
- Solano, B.; Wood, D. Design and testing of a polymeric microgripper for cell manipulation. Microelectron. Eng.
**2007**, 84, 1219–1222. [Google Scholar] [CrossRef] - Han, K.; Lee, S.H.; Moon, W.; Park, J.S.; Moon, C.W. Design and fabrication of the microgripper for manipulation the cell. Integr. Ferroelectr.
**2007**, 89, 77–86. [Google Scholar] [CrossRef] - Colinjivadi, K.S.; Lee, J.B.; Draper, R. Viable cell handling with high aspect ratio polymer chopstick gripper mounted on a nano precision manipulator. Microsyst. Technol.
**2008**, 14, 1627–1633. [Google Scholar] [CrossRef] - Kim, K.; Liu, X.; Zhang, Y.; Sun, Y. Nanonewton force-controlled manipulation of biological cells using a monolithic MEMS microgripper with two-axis force feedback. J. Micromech. Microeng.
**2008**, 18, 055013. [Google Scholar] [CrossRef] - Kim, K.; Liu, X.; Zhang, Y.; Cheng, J.; Xiao, Y.W.; Sun, Y. Elastic and viscoelastic characterization of microcapsules for drug delivery using a force-feedback MEMS microgripper. Biomed. Microdevices
**2009**, 11, 421–427. [Google Scholar] [CrossRef] [PubMed] - Zhang, R.; Chu, J.; Wang, H.; Chen, Z. A multipurpose electrothermal microgripper for biological micro-manipulation. Microsyst. Technol.
**2013**, 89, 89–97. [Google Scholar] [CrossRef] - Di Giamberardino, P.; Bagolini, A.; Bellutti, P.; Rudas, I.J.; Verotti, M.; Botta, F.; Belfiore, N.P. New MEMS tweezers for the viscoelastic characterization of soft materials at the microscale. Micromachines
**2018**, 9, 15. [Google Scholar] [CrossRef] - Cauchi, M.; Grech, I.; Mallia, B.; Mollicone, P.; Sammut, N. Analytical, numerical and experimental study of a horizontal electrothermal MEMS microgripper for the deformability characterisation of human red blood cells. Micromachines
**2018**, 9, 108. [Google Scholar] [CrossRef] - Malachowski, K.; Jamal, M.; Jin, Q.R.; Polat, B.; Morris, C.J.; Gracias, D.H. Self-folding single cell grippers. Nano Lett.
**2014**, 14, 4164–4170. [Google Scholar] [CrossRef] [PubMed] - Ghosh, A.; Yoon, C.; Ongaro, F.; Scheggi, S.; Selaru, F.M.; Misra, S.; Gracias, D.H. Stimuli-responsive soft untethered grippers for drug delivery and robotic surgery. Front. Mech. Eng.
**2017**, 3, 7. [Google Scholar] [CrossRef] - Wang, W.H.; Lin, X.Y.; Sun, Y. Contact detection in microrobotic manipulation. Int. J. Robot. Res.
**2007**, 26, 821–828. [Google Scholar] [CrossRef] - Inoue, K.; Tanikawa, T.; Arai, T. Micro-manipulation system with a two-fingered micro-hand and its potential application in bioscience. J. Biotechnol.
**2008**, 133, 219–224. [Google Scholar] [CrossRef] [PubMed] - Chen, L.; Yang, Z.; Sun, L. Three-dimensional tracking at micro-scale using a single optical microscope. In Proceedings of the International Conference on Intelligent Robotics and Applications, Wuhan, China, 15–17 October 2008; pp. 178–187. [Google Scholar]
- Duceux, G.; Tamadazte, B.; Fort-Piat, N.L.; Marchand, E.; Fortier, G.; Dembele, S. Autofocusing-based visual servoing: Application to MEMS micromanipulation. In Proceedings of the 2010 International Symposium on Optomechatronic Technologies (ISOT), Toronto, ON, Canada, 25–27 October 2010; p. 11747002. [Google Scholar]
- Subbarao, M.; Tyan, J.K. Selecting the optimal focus measure for autofocusing and depth-from-focus. IEEE Trans. Pattern Anal. Mach. Intell.
**1998**, 20, 864–870. [Google Scholar] [CrossRef][Green Version] - Brenner, J.F.; Dew, B.S.; Horton, J.B.; King, T.; Neurath, P.W.; Selles, W.D. An automated microscope for cytologic research, a preliminary evaluation. J. Histochem. Cytochem.
**1976**, 24, 100–111. [Google Scholar] [CrossRef] [PubMed] - Groen, F.C.A.; Young, I.T.; Ligthart, G. A comparison of different focus functions for use in autofocus algorithms. Cytometry
**1985**, 6, 81–91. [Google Scholar] [CrossRef] [PubMed][Green Version] - Firestone, L.; Cook, K.; Culp, K.; Talsania, N.; Preston, K., Jr. Comparison of autofocus methods for automated microscopy. Cytometry
**1991**, 12, 195–206. [Google Scholar] [CrossRef] [PubMed] - Santos, A.; De Solorzano, C.O.; Vaquero, J.J.; Peña, J.M.; Malpica, N.; Del Pozo, F. Evaluation of autofocus functions in molecular cytogenetic analysis. J. Microsc.
**1997**, 188, 264–272. [Google Scholar] [CrossRef] [PubMed][Green Version] - Sun, Y.; Duthaler, S.; Nelson, B.J. Autofocusing in computer microscopy: Selecting the optimal focus algorithm. Microsc. Res. Tech.
**2004**, 65, 139–149. [Google Scholar] [CrossRef] [PubMed] - Xie, H.; Rong, W.B.; Sun, L.N. Construction and Evaluation of a Wavelet-Based Focus Measure for Microscopy Imaging. Microsc. Res. Tech.
**2007**, 70, 987–995. [Google Scholar] [CrossRef] [PubMed] - Shindler, L.; Moroni, M.; Cenedese, A. Spatial–temporal improvements of a two-frame particle-tracking algorithm. Meas. Sci. Technol.
**2010**, 21, 1–15. [Google Scholar] [CrossRef] - Chang, R.J.; Lai, Y.H. Design and implementation of micromechatronic systems: SMA drive polymer microgripper. In Design, Control and Applications of Mechatronic Systems in Engineering; Sahin, Y., Ed.; InTechOpen: London, UK, 2017. [Google Scholar]
- Yao, Y.; Abidi, B.; Doggaz, N.; Abidi, M. Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images. Proc. SPIE
**2006**, 6246, 62460G. [Google Scholar] [CrossRef][Green Version] - Kou, C.J.; Chiu, C.H. Improved auto-focus search algorithms for CMOS image-sensing module. J. Inf. Sci. Eng.
**2011**, 27, 1377–1393. [Google Scholar] - Mendelsohn, M.L.; Mayall, B.H. Computer-oriented analysis of human chromosomes-III focus. Comput. Biol. Med.
**1972**, 2, 137–150. [Google Scholar] [CrossRef] - Krotkov, E. Focusing. Int. J. Comput. Vis.
**1987**, 1, 223–237. [Google Scholar] [CrossRef] - Subbarao, M.; Choi, T.S.; Nikzad, A. Focusing techniques. J. Opt. Eng.
**1993**, 32, 2824–2836. [Google Scholar] [CrossRef] - Yeo, T.T.E.; Ong, S.H.; Jayasooriah; Sinniah, R. Autofocusing for tissue microscopy. Image Vis. Comput.
**1993**, 11, 629–639. [Google Scholar] [CrossRef] - Nayar, S.K.; Nakagawa, Y. Shape from Focus. IEEE Trans. Pattern Anal. Mach. Intell.
**1994**, 16, 824–831. [Google Scholar] [CrossRef] - Vollath, D. Automatic focusing by correlative methods. J. Microsc.
**1987**, 147, 279–288. [Google Scholar] [CrossRef] - Vollath, D. The influence of the scene parameters and of noise on the behavior of automatic focusing algorithms. J. Microsc.
**1988**, 151, 133–146. [Google Scholar] [CrossRef] - Yang, G.; Nelson, B.J. Wavelet-based autofocusing and unsupervised segmentation of microscopic images. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, NV, USA, 27–31 October 2003; pp. 2143–2148. [Google Scholar]
- Ruan, X.D.; Zhao, W.F. A novel particle tracking algorithm using polar coordinate system similarity. Acta Mech. Sin.
**2005**, 21, 430–435. [Google Scholar] [CrossRef] - Duda, R.O.; Hart, P.E. Use of the Hough Transformation to detect lines and curves in pictures. Commun. ACM
**1972**, 15, 11–15. [Google Scholar] [CrossRef]

**Figure 1.**Micromanipulation system (overlapped by a photo of microgripper with bending flexible arms in liquid well).

**Figure 4.**A sequence of eight images from total 29 images are displayed to illustrate the test of autofocusing functions. Images (

**1**)–(

**12**) are captured in the focusing process while Images (

**13**)–(

**29**) are captured in the defocusing process. Note that the best focusing image is at Image (

**12**).

**Figure 5.**Comparisons of focus measures by wavelet-entropy E

_{THF}

_{1}, E

_{PHF}

_{2}and other common focus functions (SNR of test image = 26 ± 0.2 dB).

**Figure 6.**Comparisons of focus measures by wavelet-entropy E

_{THF}

_{1}, E

_{PHF}

_{2}and other wavelet-based functions (SNR of test image = 26 ± 0.2 dB).

**Figure 9.**Flowchart of autofocusing operation to align microgripper tips and moving particles in the same focus plane (coarse adjustment).

**Figure 12.**Planner area to locate the gripper by the center of gripper tips’ template in pre-positioning operation.

**Figure 14.**Results of tracking and gripping particle in 3-D space: (

**1**) beginning of global searching for coarse adjustment; (

**2**) finishing autofocus adjustment; (

**3**) identifying particle by clicking mouse; (

**4**) approaching particle; (

**5**) gripping particle; and (

**6**) releasing particle after completion of gripping and transporting particle.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chang, R.-J.; Chien, Y.-C. Polymer Microgripper with Autofocusing and Visual Tracking Operations to Grip Particle Moving in Liquid. *Actuators* **2018**, *7*, 27.
https://doi.org/10.3390/act7020027

**AMA Style**

Chang R-J, Chien Y-C. Polymer Microgripper with Autofocusing and Visual Tracking Operations to Grip Particle Moving in Liquid. *Actuators*. 2018; 7(2):27.
https://doi.org/10.3390/act7020027

**Chicago/Turabian Style**

Chang, Ren-Jung, and Yu-Cheng Chien. 2018. "Polymer Microgripper with Autofocusing and Visual Tracking Operations to Grip Particle Moving in Liquid" *Actuators* 7, no. 2: 27.
https://doi.org/10.3390/act7020027