Vision Feedback Control for the Automation of the Pick-and-Place of a Capillary Force Gripper

In this paper, we describe a newly developed vision feedback method for improving the placement accuracy and success rate of a single nozzle capillary force gripper. The capillary force gripper was developed for the pick-and-place of mm-sized objects. The gripper picks up an object by contacting the top surface of the object with a droplet formed on its nozzle and places the object by contacting the bottom surface of the object with a droplet previously applied to the place surface. To improve the placement accuracy, we developed a vision feedback system combined with two cameras. First, a side camera was installed to capture images of the object and nozzle from the side. Second, from the captured images, the contour of the pre-applied droplet for placement and the contour of the object picked up by the nozzle were detected. Lastly, from the detected contours, the distance between the top surface of the droplet for object release and the bottom surface of the object was measured to determine the appropriate amount of nozzle descent. Through the experiments, we verified that the size matching effect worked reasonably well; the average placement error minimizes when the size of the cross-section of the objects is closer to that of the nozzle. We attributed this result to the self-alignment effect. We also confirmed that we could control the attitude of the object when we matched the shape of the nozzle to that of the sample. These results support the feasibility of the developed vision feedback system, which uses the capillary force gripper for heterogeneous and complex-shaped micro-objects in flexible electronics, micro-electro-mechanical systems (MEMS), soft robotics, soft matter, and biomedical fields.


Introduction
Electronics are becoming increasingly sophisticated while decreasing in size. For high-resolution displays, 0201 sized electronic chip parts and micro-LEDs of less than 100 µm have been developed [1]. Microfiber assembly is also one of the most significant technologies for the advancement of information processing instruments [2].
In conventional surface mounting technology (SMT), the suction force generated by compressed air is used mainly to pick up flat components. However, as the size of the chip decreases, a higher mounting speed and accuracy are required to achieve an adequate mounting density. Transfer printing technologies are feasible methods for assembling micro-LEDs [3][4][5]. Mechanical grippers have important applications, such as medical operations on eyeballs [6]. Shape memory polymers have also been investigated as a feasible micromanipulation method [7,8].
These techniques are based on the contact between the micro-object and the gripper, which may damage the targets. In contact-type grippers, it is necessary to control the gripping force while handling fragile objects. A noncontact-type gripper decreases the possibility of damaging the parts being handled.
In liquids, various non-contact micromanipulation methods have been reported, such as using steady streaming [9,10], electric fields [11], and laser tweezers [12,13] for fragile positioning, such as biological cells and microorganisms.
In a previous article, Tanaka et al. reported a double nozzle gripper using two liquid transporting methods: a diaphragm pump and the capillary phenomenon [22]. This study focused on the picking and placing of 1-mm sized objects with various shapes and did not describe a vision feedback control method for decreasing the positioning errors.
In this paper, we describe the details of the image processing procedure and the investigation of a size matching effect that influences final alignment accuracy [28]. We also describe the attitude control by a shape matching effect between the nozzle shape and the picked object [28,29].
The remainder of the paper is structured as follows. We describe the static analysis in Section 2; the mechanical design in Section 3; the experimental setup, as shown in Figure 1, in Section 4; the image processing in Section 5; the success rate, placing error, and attitude control in Section 6; and concluding remarks and future research scope in Section 7. (See also Video S1 "Digest movie" for the outline).
In a previous article, Tanaka et al. reported a double nozzle gripper using two liquid transporting methods: a diaphragm pump and the capillary phenomenon [22]. This study focused on the picking and placing of 1-mm sized objects with various shapes and did not describe a vision feedback control method for decreasing the positioning errors.
In this paper, we describe the details of the image processing procedure and the investigation of a size matching effect that influences final alignment accuracy [28]. We also describe the attitude control by a shape matching effect between the nozzle shape and the picked object [28,29].
The remainder of the paper is structured as follows. We describe the static analysis in Section 2; the mechanical design in Section 3; the experimental setup, as shown in Figure 1, in Section 4; the image processing in Section 5; the success rate, placing error, and attitude control in Section 6; and concluding remarks and future research scope in Section 7.

Approximation of the Capillary Force
In this section, we estimate the capillary forces using the geometrical parameters of the objects when the top surface of the picked-up object is a plane. Figure 2 shows the capillary bridge between the nozzle and the object. Here, the shape of the bottom of the liquid bridge is approximated to be circular. We defined the radius of the interface between the liquid bridge and the nozzle as r2 and the radius of the interface between the liquid bridge and the object as r1. In this case, assuming that the water spreads over the nozzle, we could estimate r2 to be the radius of the nozzle. h is the height of the liquid bridge, as shown in Figure 2. 1 and 2 are the arc-approximated meniscus radii of the liquid bridges caused by the interfacial tension, respectively. In this case, it is the length of the wispiest part of the liquid bridge. Moreover, 1 and 2 are the contact angles between the object and the liquid and the nozzle and the liquid, respectively.

Approximation of the Capillary Force
In this section, we estimate the capillary forces using the geometrical parameters of the objects when the top surface of the picked-up object is a plane. Figure 2 shows the capillary bridge between the nozzle and the object. Here, the shape of the bottom of the liquid bridge is approximated to be circular. We defined the radius of the interface between the liquid bridge and the nozzle as r 2 and the radius of the interface between the liquid bridge and the object as r 1 . In this case, assuming that the water spreads over the nozzle, we could estimate r 2 to be the radius of the nozzle. h is the height of the liquid bridge, as shown in Figure 2. R 1 and R 2 are the arc-approximated meniscus radii of the liquid bridges caused by the interfacial tension, respectively. In this case, it is the length of the wispiest part of the liquid bridge. Moreover, θ 1 and θ 2 are the contact angles between the object and the liquid and the nozzle and the liquid, respectively. The capillary force is determined geometrically if the section curves of the meniscus are approximated as parts of the arc [23,24]. In this case, 1 and 2 are expressed as follows:  The capillary force is determined geometrically if the section curves of the meniscus are approximated as parts of the arc [23,24]. In this case, R 1 and R 2 are expressed as follows: The mean curvature C of the meniscus is determined as follows: where R 2 is positive. When the shape is convex, R 1 is defined as negative. The difference in pressure across the interface, given by Laplace's equation, is as follows: Here, γ is the surface tension, and r 1 is given as follows: Thus, the suction force acting on the flat plane (top surface of the cube) by the Laplace pressure F L is expressed as follows: The pulling force acting on the flat plane owing to the surface tension is given as follows: F T = 2πr 1 γ sin θ 1 .
From (6) and (7), the required capillary force F C , which is the sum of F L and F T , is given as follows:

Design
The gripper consists of a flow device, flow channel, tank, shaft, and valve stem, all fabricated from acrylic. The flow channel was made of a silicon tube. A stainless-steel tube was used at the end of the flow channel, which is called the nozzle. We used the valve stem and the shaft to refill water and form droplets, and we adopted a spring interlocking mechanism, developed by Hagiwara et al. [21], to move them with the Z-stage. Figure 3 illustrates the filling of water in the channel through capillary action. As shown in the left panel, even if the water at the tip of the channel disappears owing to evaporation or experiment conduction, water can again be filled to the tip of the channel by pulling down the valve and opening the channel, as shown in the right panel.   Figure 4 shows the formation of a droplet. When the water channel moves down, the valve closes the channel to prevent backflow (left panel), and the shaft then pushes the channel and forms the droplet at the channel tip using a diaphragm pump mechanism. Since the deformed volume of the diaphragm and the volume displaced from the shaft are almost equal, the volume of the droplet is determined by measuring the length of the shaft pushed in.  channel and forms the droplet at the channel tip using a diaphragm pump mechanism. Since the deformed volume of the diaphragm and the volume displaced from the shaft are almost equal, the volume of the droplet is determined by measuring the length of the shaft pushed in.  Figure 4 shows the formation of a droplet. When the water channel moves down, the valve closes the channel to prevent backflow (left panel), and the shaft then pushes the channel and forms the droplet at the channel tip using a diaphragm pump mechanism. Since the deformed volume of the diaphragm and the volume displaced from the shaft are almost equal, the volume of the droplet is determined by measuring the length of the shaft pushed in. The object is picked up by contacting a formed droplet with the object. The object is placed by first contacting and applying a formed droplet to the place surface, and then contacting the bottom surface of the object picked up by the nozzle with the applied droplet. Table 1 shows the parameters of the micro-objects used in the pick-and-place experiments. In the experiments, we used various sizes and shapes of nozzles for each object. The experimental conditions and parameters used to calculate the estimated value of the capillary force for each condition are organized and shown in Table 2; the estimated capillary forces are also shown. ℎ, 1 , and 2 were typical values measured from the charge coupled device (CCD) camera image during the experiments. The surface tension γ, measured by the ring method, was 72.9 mN/m; we used purified water as the liquid. The object is picked up by contacting a formed droplet with the object. The object is placed by first contacting and applying a formed droplet to the place surface, and then contacting the bottom surface of the object picked up by the nozzle with the applied droplet. Table 1 shows the parameters of the micro-objects used in the pick-and-place experiments.  Figure 3. Mechanism of water filling till the tip of the channel by capillary action. Figure 4 shows the formation of a droplet. When the water channel moves down valve closes the channel to prevent backflow (left panel), and the shaft then pushes channel and forms the droplet at the channel tip using a diaphragm pump mechan Since the deformed volume of the diaphragm and the volume displaced from the s are almost equal, the volume of the droplet is determined by measuring the length of shaft pushed in. The object is picked up by contacting a formed droplet with the object. The obje placed by first contacting and applying a formed droplet to the place surface, and t contacting the bottom surface of the object picked up by the nozzle with the applied dro Table 1 shows the parameters of the micro-objects used in the pick-and-place experiment In the experiments, we used various sizes and shapes of nozzles for each object. experimental conditions and parameters used to calculate the estimated value of the pillary force for each condition are organized and shown in Table 2; the estimated ca lary forces are also shown. ℎ, 1 , and 2 were typical values measured from the cha coupled device (CCD) camera image during the experiments. The surface tension γ, m ured by the ring method, was 72.9 mN/m; we used purified water as the liquid.  Figure 4 shows the formation of a droplet. When the water channel moves down, the valve closes the channel to prevent backflow (left panel), and the shaft then pushes the channel and forms the droplet at the channel tip using a diaphragm pump mechanism. Since the deformed volume of the diaphragm and the volume displaced from the shaft are almost equal, the volume of the droplet is determined by measuring the length of the shaft pushed in. The object is picked up by contacting a formed droplet with the object. The object is placed by first contacting and applying a formed droplet to the place surface, and then contacting the bottom surface of the object picked up by the nozzle with the applied droplet. Table 1 shows the parameters of the micro-objects used in the pick-and-place experiments. In the experiments, we used various sizes and shapes of nozzles for each object. The experimental conditions and parameters used to calculate the estimated value of the capillary force for each condition are organized and shown in Table 2; the estimated capillary forces are also shown. ℎ, 1 , and 2 were typical values measured from the charge coupled device (CCD) camera image during the experiments. The surface tension γ, measured by the ring method, was 72.9 mN/m; we used purified water as the liquid. In the experiments, we used various sizes and shapes of nozzles for each object. The experimental conditions and parameters used to calculate the estimated value of the capillary force for each condition are organized and shown in Table 2; the estimated capillary forces are also shown. h, θ 1 , and θ 2 were typical values measured from the charge coupled device (CCD) camera image during the experiments. The surface tension γ, measured by the ring method, was 72.9 mN/m; we used purified water as the liquid.   Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.  From Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.  From Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.

Experimental Setup
Micromachines 2022, 13, 1270 5 of 13 From Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.

Experimental Setup
Micromachines 2022, 13, 1270 5 of 13 From Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.

Experimental Setup
To increase the success rate of the pick-and-place, we developed an automatic pickand-place with vision feedback function. The organization of the entire setup is shown in Figure 5.

Gravity force 1.5 μN 12 μN
From Table 2, it can be seen that the estimated capillary force had a much larger value than the corresponding gravitational force, and hence, this gripper could grasp the object under all conditions if there were no additional adhesive or electrostatic forces.

Experimental Setup
To increase the success rate of the pick-and-place, we developed an automatic pick-andplace with vision feedback function. The organization of the entire setup is shown in Figure 5.
In this study, the work surface was positioned in the X-and Y-axis with a positioning resolution of 1 μm and a repeatability of ±0.5 μm (YA10A-L1, Kohzu Precision Co., Ltd., Kawasaki City, Japan). The gripper was connected to a linear stage (MMU-40X, Chuo Precision Industrial Co., Ltd., Tokyo, Japan) and moved along the Z-axis. It was also connected to the interlocking mechanism, which controls the shaft and the valve to form a droplet. The traverse limits of the XY-and Z-stages were ±50 mm and ±5 mm, respectively. To measure the position of the samples, the vision feedback system by OpenCV was used. This system imported the image from the CCD camera set on top of the work bench. The camera had a resolution of 2432 × 2050 pixels (CV-H500M, KEYENCE Corp., Osaka, Japan). The CCD camera also attached to the side of the workbench, and it had a resolution of 1216 × 1025 pixels (CV-200C, KEYENCE Corp.). Each camera had a magnification lens (LA-LM510, Keyence Co.), and the depth of field was 1.28 mm.
The pixel resolutions of the top and side cameras were, approximately, 7 μm/pixel and 9 μm/pixel, respectively. We confirmed that the image processing and gravity center detection process in the vision feedback resulted in a final X-and Y-axis measurement resolution of 1 µ m because of the averaging effect. In this study, the work surface was positioned in the X-and Y-axis with a positioning resolution of 1 µm and a repeatability of ±0.5 µm (YA10A-L1, Kohzu Precision Co., Ltd., Kawasaki City, Japan). The gripper was connected to a linear stage (MMU-40X, Chuo Precision Industrial Co., Ltd., Tokyo, Japan) and moved along the Z-axis. It was also connected to the interlocking mechanism, which controls the shaft and the valve to form a droplet. The traverse limits of the XY-and Z-stages were ±50 mm and ±5 mm, respectively.
To measure the position of the samples, the vision feedback system by OpenCV was used. This system imported the image from the CCD camera set on top of the work bench. The camera had a resolution of 2432 × 2050 pixels (CV-H500M, KEYENCE Corp., Osaka, Japan). The CCD camera also attached to the side of the workbench, and it had a resolution of 1216 × 1025 pixels (CV-200C, KEYENCE Corp.). Each camera had a magnification lens (LA-LM510, Keyence Co.), and the depth of field was 1.28 mm.
The pixel resolutions of the top and side cameras were, approximately, 7 µm/pixel and 9 µm/pixel, respectively. We confirmed that the image processing and gravity center detection process in the vision feedback resulted in a final Xand Y-axis measurement resolution of 1 µm because of the averaging effect.
LabVIEW2018 (National Instruments, Austin, TX, USA) was used as the programming language, and OpenCV was used to conduct image processing. The detailed procedure for the image processing is described in Section 5. The pick-and-place process is as follows: (1) The object is placed within view of the top camera on the surface to measure the Xand Y-coordinates; (2) The XY-stage is moved so that the placement position on the work surface is directly below the nozzle; (3) The gripper is moved up and down by the Z-stage, and a droplet is applied to the work surface; (4) The XY-stage is moved, and the object is placed directly under the nozzle; (5) The gripper moves first down and then up to pick up the object; (6) The XY-stage is moved such that the target placement position is directly below the nozzle grasping the object; (7) The gripper moves down to place the picked-up object on the pre-applied droplet of (3); (8) The XY-stage is moved such that the object is within view of the CCD camera to measure the X-and Y-coordinates.
In steps (3), (5), and (7) of the above process, the amount of Z-stage movement is calculated and determined by the image processing program written in Python from images captured by the side camera.

Image Processing
In this section, we explain the vision feedback system. The program used Open CV by Python. The system was used when applying place droplets, picking up, and placing objects. Figure 6 shows the image of the process of the vision feedback system in placement, and this process is described as follows: (1) The side camera captures the image so that both the object and the place droplets are visible ( Figure 6a); (2) The captured image is imported to the PC and the Sobel filter is applied (Figure 6b).
(3) Next, the image is binarized by the threshold (Figure 6c); (4) The image is then used to detect the contour of the object ( Figure 6d); (5) The distance l 1 between the object and the place surface is calculated from the coordinates of the lowest surface of the object ( Figure 6e); (6) In this experiment, the parameter is the liquid bridge height h 2 between the object and the place surface at placing (Figure 6f). Thus, from h 2 and l 1 , the stage descent amount l 2 is determined as follows: (7) The amount of Z-stage descent determined in (6) is fed back to the control (Figure 6g).
below the nozzle; (3) The gripper is moved up and down by the Z-stage, and a droplet is applied to the work surface; (4) The XY-stage is moved, and the object is placed directly under the nozzle; (5) The gripper moves first down and then up to pick up the object; (6) The XY-stage is moved such that the target placement position is directly below the nozzle grasping the object; (7) The gripper moves down to place the picked-up object on the pre-applied droplet of 3); (8) The XY-stage is moved such that the object is within view of the CCD camera to measure the X-and Y-coordinates.
In steps (3), (5), and (7) of the above process, the amount of Z-stage movement is calculated and determined by the image processing program written in Python from images captured by the side camera.

Image Processing
In this section, we explain the vision feedback system. The program used Open CV by Python. The system was used when applying place droplets, picking up, and placing objects. Figure 6 shows the image of the process of the vision feedback system in placement, and this process is described as follows: (1) The side camera captures the image so that both the object and the place droplets are visible ( Figure 6a); (2) The captured image is imported to the PC and the Sobel filter is applied (Figure 6b).
(3) Next, the image is binarized by the threshold (Figure 6c); (4) The image is then used to detect the contour of the object ( Figure 6d); (5) The distance 1 between the object and the place surface is calculated from the coordinates of the lowest surface of the object ( Figure 6e); (6) In this experiment, the parameter is the liquid bridge height h2 between the object and the place surface at placing (Figure 6f). Thus, from h2 and 1 , the stage descent amount 2 is determined as follows: (7) The amount of Z-stage descent determined in 6) is fed back to the control (Figure 6g).  As described in step 6, we used h 2 as a parameter in this experiment. Here, Table 3 shows the set values of h 2 for each experimental condition.
Here, the condition values of (C1)-(C5) are the same as in Table 2. We conducted the pick-and-place experiment 10 times for each condition. We conducted the pick-and-place operation under the five conditions. Figure 7 shows the sequence of pick-and-place operations under the condition of (C3) for h 2 of 50 µm.
As described in step 6, we used h2 as a parameter in this experiment. Here, Table 3 shows the set values of h2 for each experimental condition.  Here, the condition values of (C1)-(C5) are the same as in Table 2. We conducted the pick-and-place experiment 10 times for each condition.

Comparison of Results with Height h2
We conducted the pick-and-place operation under the five conditions. Figure 7 shows the sequence of pick-and-place operations under the condition of (C3) for h2 of 50 μm. In Table 4, the success rates and positioning errors for h2 = 0, 50, 70, 100, and 120 μm are compared. From Table 4 and Figure 7, the newly developed vision feedback system confirmed that the gripper could automatically pick-and-place objects without solid contact. Figure 8 shows the relationship between the positioning errors and h2 under the condition (C3). In Table 4, the success rates and positioning errors for h 2 = 0, 50, 70, 100, and 120 µm are compared. From Table 4 and Figure 7, the newly developed vision feedback system confirmed that the gripper could automatically pick-and-place objects without solid contact. Figure 8 shows the relationship between the positioning errors and h 2 under the condition (C3).   The smallest value of the positioning error was 44 ± 34 μm with h2 = 70 μm (the ratios to the cube side were 4.4 ± 3.4%), although the error with h2 = 50 μm had almost the same value. Figure 9 shows the typical examples for a large positioning error. The water spread to the sides of the object when h2 was too small, as shown in Figure 9a, whereas the cube was easily inclined when h2 was too large, leading to a large positioning error, as shown in Figure 9b.  The smallest value of the positioning error was 44 ± 34 µm with h 2 = 70 µm (the ratios to the cube side were 4.4 ± 3.4%), although the error with h 2 = 50 µm had almost the same value. Figure 9 shows the typical examples for a large positioning error. The water spread to the sides of the object when h 2 was too small, as shown in Figure 9a, whereas the cube was easily inclined when h 2 was too large, leading to a large positioning error, as shown in Figure 9b.  The smallest value of the positioning error was 44 ± 34 μm with h2 = 70 μm (the ratios to the cube side were 4.4 ± 3.4%), although the error with h2 = 50 μm had almost the same value. Figure 9 shows the typical examples for a large positioning error. The water spread to the sides of the object when h2 was too small, as shown in Figure 9a, whereas the cube was easily inclined when h2 was too large, leading to a large positioning error, as shown in Figure 9b.

Evaluation of Size Matching Effect
The positioning errors for each condition are compared in Table 5. We used the result with the smallest error out of the h 2 values set for each condition. The cube moves toward the center position of the droplet on the surface due to surface tension as a self-alignment phenomenon. If the average error is only attributed to the center position of the pre-applied droplet, the errors of (C1) and (C3) should be almost equal because the same nozzle is used for applying the droplet to the surface. For the same reason, that of (C2) and (C4) should also be almost equal. However, twice the difference between (C1) and (C3) and quadruple the difference between (C2) and (C4) can be observed from Table 5. We supposed that the size and shape matching effects also influenced the final alignment errors [29][30][31].
To examine the size matching effect, we defined the diameter of the circular nozzle as D, the positioning error as E, and the side length of the cube as L. We normalized the parameters (D, E) by L as follows: We compared the results in Table 5 using these parameters to verify a suitable size of circular shaped nozzle for each object, as shown in Figure 10.

Evaluation of Size Matching Effect
The positioning errors for each condition are compared in Table 5. We used the result with the smallest error out of the h2 values set for each condition. The cube moves toward the center position of the droplet on the surface due to surface tension as a self-alignment phenomenon. If the average error is only attributed to the center position of the pre-applied droplet, the errors of (C1) and (C3) should be almost equal because the same nozzle is used for applying the droplet to the surface. For the same reason, that of (C2) and (C4) should also be almost equal. However, twice the difference between (C1) and (C3) and quadruple the difference between (C2) and (C4) can be observed from Table 5. We supposed that the size and shape matching effects also influenced the final alignment errors [29][30][31].
To examine the size matching effect, we defined the diameter of the circular nozzle as D, the positioning error as E, and the side length of the cube as L. We normalized the parameters ( , ) by L as follows: * ≡ / , We compared the results in Table 5 using these parameters to verify a suitable size of circular shaped nozzle for each object, as shown in Figure 10. As shown in Figure 10, the closer D* is to 1, the smaller the positioning error becomes. When the nozzle diameter was approximately twice as large as the object as in condition As shown in Figure 10, the closer D* is to 1, the smaller the positioning error becomes. When the nozzle diameter was approximately twice as large as the object as in condition (C1), the distribution of the final alignment position was larger than those of (C2) and (C3). When the nozzle diameter was approximately half of the object, as in condition (C4), the object was often tilted during pick-up, as shown in the photograph of Figure 10 (C4). That is the main reason why (C4) obtained the largest distribution among the four conditions.

Attitude Control Using Shape Matching Effect
We considered that the attitude angle of the picked-up object could be controlled by matching the shape of the nozzle to the picked object.
To examine the shape matching effect [30,31], we conducted the pick-and-place experiment as in condition (S) in Table 4 with a square-shaped nozzle and side length of 1 mm for the 1-mm cube. Figure 11 shows a sequential photograph of the pick-and-place operation.
(C1), the distribution of the final alignment position was larger than those of (C2) and (C3). When the nozzle diameter was approximately half of the object, as in condition (C4), the object was often tilted during pick-up, as shown in the photograph of Figure 10 (C4). That is the main reason why (C4) obtained the largest distribution among the four conditions.

Attitude Control Using Shape Matching Effect
We considered that the attitude angle of the picked-up object could be controlled by matching the shape of the nozzle to the picked object.
To examine the shape matching effect [30,31], we conducted the pick-and-place experiment as in condition (S) in Table 4 with a square-shaped nozzle and side length of 1 mm for the 1-mm cube. Figure 11 shows a sequential photograph of the pick-and-place operation. From Figure 11a,b, the object changed its attitude angle for aligning the nozzle shape; we verified that the shape matching effect is useful for controlling the attitude of the sample. The movement of the object is illustrated in Figure 12. We compared the experimental results with condition (C3) using a circular nozzle to conduct a pick-and-place of the same cubed object. The positioning error and the attitude angle of the object after placement are shown in Table 6, and the image of the object after placement is shown in Figure 13. From Figure 11a,b, the object changed its attitude angle for aligning the nozzle shape; we verified that the shape matching effect is useful for controlling the attitude of the sample. The movement of the object is illustrated in Figure 12.
(C1), the distribution of the final alignment position was larger than those of (C2) and (C3). When the nozzle diameter was approximately half of the object, as in condition (C4), the object was often tilted during pick-up, as shown in the photograph of Figure 10 (C4). That is the main reason why (C4) obtained the largest distribution among the four conditions.

Attitude Control Using Shape Matching Effect
We considered that the attitude angle of the picked-up object could be controlled by matching the shape of the nozzle to the picked object.
To examine the shape matching effect [30,31], we conducted the pick-and-place experiment as in condition (S) in Table 4 with a square-shaped nozzle and side length of 1 mm for the 1-mm cube. Figure 11 shows a sequential photograph of the pick-and-place operation. From Figure 11a,b, the object changed its attitude angle for aligning the nozzle shape; we verified that the shape matching effect is useful for controlling the attitude of the sample. The movement of the object is illustrated in Figure 12. We compared the experimental results with condition (C3) using a circular nozzle to conduct a pick-and-place of the same cubed object. The positioning error and the attitude angle of the object after placement are shown in Table 6, and the image of the object after placement is shown in Figure 13. We compared the experimental results with condition (C3) using a circular nozzle to conduct a pick-and-place of the same cubed object. The positioning error and the attitude angle of the object after placement are shown in Table 6, and the image of the object after placement is shown in Figure 13.   As no target angle was set in these experiments, we focused on the standard deviations (SDs) to discuss the experimental results of the accuracy of the attitude angle. Table  6 shows that the SD of the attitude angle under condition (S) was smaller than condition (C3). Using this nozzle and a rotational stage, we could set the objects at various angles, as shown in Figure 14.

Conclusions and Future Prospects
This paper described a newly developed vision feedback method for improving the placement accuracy for a single nozzle capillary gripper. The developed system was coded in Python using OpenCV.
In the experiment, we automatically picked-and-placed the cubes without any solid contact, and the success rate was 99% (129 out of 130 samples). For the cubes with 1-and 0.5-mm sides, the minimum positioning error ± SD of each size was 44 ± 34 μm and 39 ± 18 μm, respectively; the ratios of the positioning errors to the lengths of the objects were 4.4 ± 3.4% and 7.8 ± 3.6%, respectively. We also verified that the size matching effect worked reasonably; the average positioning error minimized when the size of the nozzle was nearly equal to the side of the cube.
In addition, using a square-shaped nozzle with the same shape of the cube, we succeeded in controlling the attitude; the SD of the attitude angle was reduced from a random value to 7.4°.
To reduce the positioning errors, our future plans are as follows. We plan to evaluate the relationship between the surface shape and properties of the objects as well as the radius and arrangement pattern of the droplets, for effectively applying the self-alignment phenomena. Further work will study the accuracy of contour detection in the vision feedback system. The impact of lighting on positioning will also be studied in the future, for example, the effects of droplet size and color and diffraction and refraction of lighting. In addition, the investigation of the release condition is a significant subject from both theoretical and experimental approaches. As no target angle was set in these experiments, we focused on the standard deviations (SDs) to discuss the experimental results of the accuracy of the attitude angle. Table 6 shows that the SD of the attitude angle under condition (S) was smaller than condition (C3). Using this nozzle and a rotational stage, we could set the objects at various angles, as shown in Figure 14.  As no target angle was set in these experiments, we focused on the standard deviations (SDs) to discuss the experimental results of the accuracy of the attitude angle. Table  6 shows that the SD of the attitude angle under condition (S) was smaller than condition (C3). Using this nozzle and a rotational stage, we could set the objects at various angles, as shown in Figure 14.

Conclusions and Future Prospects
This paper described a newly developed vision feedback method for improving the placement accuracy for a single nozzle capillary gripper. The developed system was coded in Python using OpenCV.
In the experiment, we automatically picked-and-placed the cubes without any solid contact, and the success rate was 99% (129 out of 130 samples). For the cubes with 1-and 0.5-mm sides, the minimum positioning error ± SD of each size was 44 ± 34 μm and 39 ± 18 μm, respectively; the ratios of the positioning errors to the lengths of the objects were 4.4 ± 3.4% and 7.8 ± 3.6%, respectively. We also verified that the size matching effect worked reasonably; the average positioning error minimized when the size of the nozzle was nearly equal to the side of the cube.
In addition, using a square-shaped nozzle with the same shape of the cube, we succeeded in controlling the attitude; the SD of the attitude angle was reduced from a random value to 7.4°.
To reduce the positioning errors, our future plans are as follows. We plan to evaluate the relationship between the surface shape and properties of the objects as well as the radius and arrangement pattern of the droplets, for effectively applying the self-alignment phenomena. Further work will study the accuracy of contour detection in the vision feedback system. The impact of lighting on positioning will also be studied in the future, for example, the effects of droplet size and color and diffraction and refraction of lighting. In addition, the investigation of the release condition is a significant subject from both theoretical and experimental approaches.

Conclusions and Future Prospects
This paper described a newly developed vision feedback method for improving the placement accuracy for a single nozzle capillary gripper. The developed system was coded in Python using OpenCV.
In the experiment, we automatically picked-and-placed the cubes without any solid contact, and the success rate was 99% (129 out of 130 samples). For the cubes with 1-and 0.5-mm sides, the minimum positioning error ± SD of each size was 44 ± 34 µm and 39 ± 18 µm, respectively; the ratios of the positioning errors to the lengths of the objects were 4.4 ± 3.4% and 7.8 ± 3.6%, respectively. We also verified that the size matching effect worked reasonably; the average positioning error minimized when the size of the nozzle was nearly equal to the side of the cube.
In addition, using a square-shaped nozzle with the same shape of the cube, we succeeded in controlling the attitude; the SD of the attitude angle was reduced from a random value to 7.4 • .
To reduce the positioning errors, our future plans are as follows. We plan to evaluate the relationship between the surface shape and properties of the objects as well as the radius and arrangement pattern of the droplets, for effectively applying the self-alignment phenomena. Further work will study the accuracy of contour detection in the vision feedback system. The impact of lighting on positioning will also be studied in the future, for example, the effects of droplet size and color and diffraction and refraction of lighting. In addition, the investigation of the release condition is a significant subject from both theoretical and experimental approaches.
Moreover, we wish to take on the challenge of developing a pick-and-place mechanism for smaller complex-shaped objects of less than 200 µm, such as micro-LEDs, helical-shaped coils, thin wires, gears, soft contact lenses, fragile gels, and biomedical cells with the capillary grippers to cultivate the manipulation of heterogeneous and complex-shaped microobjects in flexible electronics, micro-electro-mechanical systems (MEMS), soft robotics, soft matter, and biomedical fields. In the microminiaturization of an object, we consider the limitations to be determined as follows: the smallest possible droplet size, the sharpest nozzle, the effect of droplet evaporation, and the piezoelectric control of droplet volume.