Autonomous Sewing Technology and System: A New Strategy by Integrating Soft Fingers and Machine Vision Technology

Abstract

The garment manufacturing industry, being labor-intensive, has long faced challenges in automating the sewing process due to the flexibility and deformability of fabrics. This study proposes a novel strategy for automated sewing by integrating soft fingers and machine vision technology. Firstly, leveraging the flexibility and adjustability of soft fingers, combined with the motion characteristics of the sewing machine, a sewing model was established to achieve coordinated operation between the soft fingers and the sewing machine. Experimental results indicate that the fabric feeding speed and waiting time of the soft fingers are significantly correlated with the sewing speed and stitch density of the sewing machine, but not with the fabric properties. Secondly, machine vision technology was employed to inspect the quality of the sewn fabrics, achieving a classification accuracy of 97.84%. This study not only provides theoretical and technical support for the intelligent upgrading of the garment manufacturing industry but also lays the foundation for the automation of complex sewing processes such as quilting. Future research will further optimize the system’s performance and expand its applications in more complex sewing tasks.

1. Introduction

The garment manufacturing industry is a typical labor-intensive sector that has long maintained a traditional development model. In the process of garment production, fabrics need to go through steps such as ironing, cutting, transferring, and sewing [1]. The low-level repetitive labor involved is similar to that of hundreds of years ago and still requires a significant amount of manual labor. In recent years, with the development of the market economy, the garment industry, supported by advanced science and technology, has actively promoted industrial transformation and upgrading. By utilizing digital intelligent equipment, the production efficiency of garment cutting and transferring has been improved. However, the characteristic of garment fabrics being soft and easily deformable has greatly limited the automation process of sewing. Traditional rigid equipment cannot ensure stable fabric grasping and transferring [2]. Moreover, during the sewing process, the fabric is highly prone to deviation due to the sewing motion of the sewing machine. Human eyes and hands are required to synchronously control the deviation to ensure the quality of the finished product. Therefore, the sewing process in the garment industry still relies on skilled manual labor, and the dependence of sewing tasks on labor force is the main bottleneck that restricts the intelligent development of garment manufacturing [3,4].

Sewing involves processes such as material feeding, fabric conveying, and quality inspection of outgoing products. Fabrics are characterized by their flexibility and susceptibility to deformation. During material feeding, human eyes are needed to observe the placement of the stack of cut pieces and the shape characteristics [4]. Human hands are required to perceive the performance characteristics of the fabric, such as its frictional properties. Then, based on the observed position and shape characteristics, human hands determine the placement of both hands and use appropriate grasping force to pick up the cut pieces smoothly and stably, and then place them on the sewing machine table [5,6]. Since sewing often involves stitching two layers of cut pieces together and requires the alignment of the sewing edges of the upper and lower layers, two cut pieces need to be picked up from the stack during material feeding. After placing the two layers of cut pieces on the sewing table, the sewing edges need to be aligned. Finally, the aligned two layers of cut pieces are transferred to the sewing needle position of the sewing machine for feeding and sewing. During fabric conveying, human hands need to control the pressing force on the fabric and the feeding speed of the hands to achieve the coordination between the fabric and the sewing machine head, and to sew the specified stitches. After the fabric conveying is completed, to ensure the quality of sewing, human eyes are also needed to evaluate the sewing effect after fabric conveying, such as the flatness of sewing [7].

At present, material feeding can be solved through intelligent grasping technology. Many scholars have explored and studied this area and have developed a variety of automatic grasping devices for garment cut pieces [8]. Depending on the principles applied in the grasping methods, these devices can be further categorized into pneumatic suction grasping [9,10], electrostatic grasping [11], needle piercing grasping [12], robotic arm grasping [13], and soft finger grasping [14,15]. However, fabric conveying and quality inspection of outgoing products can only rely on simple machinery or manual labor, which greatly reduces the efficiency of the entire sewing process and limits the quality of fabric conveying. For example, when feeding fabric through rollers, the point contact nature of the roller feeding method, combined with the unstable speed and force of the mechanical parts, can cause uneven force on the fabric during transmission, leading to wrinkles and reduced sewing quality. Moreover, rigid feeding devices lack flexibility and are prone to material mismatch quality issues. For example, template sewing machines use templates made of PVC plastic to hold the fabric in place. The hardness of the material can easily leave marks on the surface of the sewn fabric, reducing sewing quality. In addition, the lack of flexibility in the fabric feeding device means that it can only perform fixed sewing processes. For example, when changing the sewing process on a template sewing machine, it is necessary to redesign the template. When feeding fabric through rollers or conveyor belts, only straight-line sewing processes can be completed. To truly achieve the goal of “machine replacing human” in “intelligent sewing,” it is necessary to integrate modern intelligent machine technology to replace the work of human eyes and hands [3,16].

In recent years, the rise of machine vision technology and soft finger robots has brought hope to solving this problem. Machine vision is the use of machines to replace human eyes to achieve detection of physical objects. In the garment field, with the help of machine vision technology, it is possible to achieve garment retrieval, textile inspection, fabric pattern processing [17,18], and other tasks. Both the detection accuracy and speed are higher than those of manual labor. As a type of soft robot, compared with traditional rigid robots, soft fingers, due to their use of highly flexible materials and bionic design, can better simulate the bending and twisting actions of human fingers. For example, they can easily complete high-difficulty tasks such as picking up a cup, holding a cup, and even holding a sewing needle. Moreover, soft finger materials are mostly composite silicones, which are flexible materials. They have good affinity and compatibility with fabrics and have a wide range of applications in the field of handling flexible materials such as accurate grasping and layer-by-layer separation of textile pieces [8]. Against this backdrop, this project aims to use machine vision technology to study an automated sewing method based on the collaborative work of soft fingers and to develop an intelligent sewing system.

Meanwhile, the robotics community has conducted extensive research on robotic handling of flexible fabrics and vision-guided sewing. However, most existing studies focus on isolated stages, struggling to form a complete intelligent sewing solution. Zacharia et al. proposed a fuzzy visual servoing strategy to recognize and flatten single-layer fabrics lying on a table, effectively mitigating the impact of wrinkles during handling. Nevertheless, their system only addresses the preprocessing stage and does not extend to the actual sewing execution, limiting its applicability to real-world sewing tasks [19]. Li et al. approached the problem from a process modeling perspective, decomposing sewing tasks into straight and curved segments. By combining impedance control with geometric trajectory planning, they enabled active robotic guidance throughout the sewing process. While this method shows some adaptability to fabric slippage and tension variation, it still relies on rigid fixtures to secure the fabric and fails to address challenges such as initial alignment of multi-layer plies and real-time compensation for fabric deformation, thus restricting its versatility in handling flexible materials [20]. Tokuda et al. introduced a fixture-free dual-arm robotic sewing system that utilizes visual servoing for edge tracking and stitch-by-stitch guidance, significantly enhancing sewing path flexibility. However, their system heavily depends on vision accuracy, requires manual intervention for initial fabric positioning, and lacks online quality feedback mechanisms, preventing true full autonomy in sewing operations [21].

Specifically, this paper adopts soft finger technology combined with machine vision technology to assist a computerized multifunctional sewing machine and build an intelligent sewing system. In addition, pure cotton woven fabric is selected, and the simplest flat seam type among the eight types of seams is chosen as the research object to study how the constructed intelligent sewing system completes intelligent flat sewing and ensures the quality of intelligent sewing. The research results provide basic data for establishing an automatic sewing process using soft fingers and have theoretical and practical application value for accelerating the intelligent transformation and upgrading of the garment industry. It is expected to provide references for the intelligentization of more complex sewing methods in the future.

2. Theoretical Background

As depicted in Figure 1a, the seam allowance of fabrics is generally 1 cm and is typically sewn along the edge of the fabric. When sewing is performed without manual assistance, the fabric is subjected to a sewing force F₁ from the sewing machine. Since the point of action of this force is located on one side of the fabric, the center of mass of the fabric does not lie on the line of action of F₁. Consequently, a rotational moment τ₁ is generated in the fabric. The perpendicular distance from the point of action A₂ of the sewing force to the center of mass A₁ of the fabric is denoted as d₁. The rotational moment τ₁ can be expressed as Equation (1):

τ₁ = F₁ × d₁

(1)

According to the right-hand rule, the direction of the moment is perpendicular to both the direction of the force and the line connecting the center of mass to the point of action. In this case, the direction of the moment causes the fabric to rotate about its center of mass A₁ during sewing without manual assistance, resulting in skewed stitching lines and sewing deviations, as shown in Figure 1b.

To address this issue of sewing deviation, this study employs soft fingers to push the fabric from the opposite side, with the pushing speed of the soft fingers synchronized with the sewing speed of the sewing machine, thereby assisting the sewing machine in completing the sewing process, as illustrated in Figure 1c. In this process, the soft fingers exert a pushing force F₂ on the fabric. The perpendicular distance from the point of action A₃ of the pushing force to the center of mass A₁ of the fabric is denoted as d₂. The moment τ₂ generated by the pushing force F₂ is shown in Equation (2):

τ₂ = F₂ × d₂

(2)

Since the sewing force and the pushing force act in the same direction but on opposite sides of the fabric’s center of mass, the moments they generate are in opposite directions. Therefore, the net moment T is shown in Equation (3):

Τ = τ₁ − τ₂ = F₁ × d₁ − F₁ × d₁

(3)

Given that F₁ = F₂ and d₁ = d₂, it follows that T = 0. This means that the net moment is zero, and the fabric does not rotate during sewing, as shown in Figure 1d. As a result, the stitching lines remain straight, and no sewing deviations occur [16].

However, in practical applications, it is challenging to measure the sewing force exerted by the sewing machine. Since different forces result in different sewing speeds, and sewing speed is easily adjustable on the sewing machine, the experiments in this study use the sewing speed of the sewing machine as a reference to correspondingly adjust the fabric-feeding speed of the soft fingers. By maintaining synchronization between the fabric-feeding speed and the sewing speed, the quality of automated sewing is ensured. That is to say, when the sewing speed of the sewing machine is known, how to determine the appropriate feeding speed of the soft fingers.

3. Experimental Method

3.1. Material and Devices

3.1.1. Fabric

Fifteen kinds of cotton woven fabrics commonly used in the market were selected in this research work, which were represented by #1 to #15 respectively. Since the gram weight, thickness, stiffness, and friction coefficient of the fabric can affect the grasping effect they were analyzed. The TH-100 g m meter (CHNSpec Technology (Zhejiang) Co., Ltd., Hangzhou, China) was used to test the grams per square meter of fabrics, according to ISO 3801:1977 standard [22]. The YG-141 fabric thickness tester (Wenzhou Darong Textile Instrument Co., Ltd., Wenzhou, China) was used to test the thickness of the fabrics according to ISO 5084:1996 standard [23]. The YG022D automatic fabric stiffness tester (Bonnin Instrument Technology Co., Ltd., Yangzhou, China) was used to measure the stiffness of the fabrics according to ISO 9073-7:1995 standard [24], and the TC-MXD-01 friction coefficient tester (Jinan Labthink Technology Co., Ltd., Jinan, China) was adopted to determine their friction coefficient according to ISO 8295:1995 standard [25]. The samples were placed at 65 ± 2% RH, 21 ± 1 °C (70 ± 2 °F) conditions for at least 24 h before testing. The parameters related to the fabrics are shown in

Table 1. Parameters related to fabrics.

Fabric	Weight [g/m2]	Thickness [mm]	Bending Stiffness [mN·cm]		Friction Coefficient
			Warp	Weft	Static	Dynamic
#1	258.68	0.611	24.23	51.57	0.69	0.66
#2	82.94	0.348	15.87	30.80	0.32	0.31
#3	251.51	0.581	21.17	29.50	0.72	0.55
#4	297.74	0.648	29.77	26.13	0.70	0.68
#5	159.98	0.407	20.20	18.30	0.47	0.39
#6	395.19	0.867	18.10	22.00	0.95	0.66
#7	131.92	0.351	17.23	28.07	0.30	0.18
#8	253.13	0.579	36.27	50.50	0.17	0.18
#9	275.53	0.666	25.03	50.50	0.62	0.54
#10	378.68	0.997	48.47	36.03	1.04	0.88
#11	302.10	0.760	24.60	26.83	1.00	0.70
#12	113.88	0.354	26.10	26.43	0.89	0.65
#13	174.51	0.508	22.87	20.73	1.07	0.84
#14	77.46	0.356	24.13	18.37	0.82	0.72
#15	175.37	0.433	18.43	25.57	0.90	0.69

.

3.1.2. Soft Finger Robot

The soft finger employed in this experiment, which is modeled as GC-BM20006, is manufactured by SR Co., Ltd. (Suzhou City, Jiangsu Province, China). The intricate details of its structure and performance parameters are comprehensively exhibited in

Table 2. Performance parameters of soft finger.

Repetitive Precision	Recommended Load	Lifetime	Safe Pressure	Frequency	External Gripping Force	Contact Temperature
±0.05 mm	30 g	1.5 million times	+80 kPa	6 times/s	0–0.3 N	180 °C

and Figure 2 appended below [16].

The finger is composed of silicone and is powered by pneumatic pressure. During the vacuum state, the soft fingers’ fingertips come together, whereas in the positive pressure state, the fingertips separate. As the air pressure increases, the distance G between the fingertips also increases. The safe air pressure for this device is 80 kPa. If the pressure exceeds this limit, it will negatively impact its lifespan and may even cause it to burst, rendering it unusable.

Based on the sewing theory discussed earlier, Figure 3 illustrates the schematic diagram of the edge sewing process using the soft finger in this study. During sewing, the soft finger presses against the fabric on the opposite side corresponding to the position of the sewing needle, and the direction of fabric feeding by the soft finger (i.e., the pushing direction) is consistent with the sewing direction of the fabric. Through preliminary experiments, it has been determined that a pneumatic pressure of 40 kPa is sufficient for the soft finger during sewing.

The three-dimensional model of the soft finger was imported into the Abaqus software 2021. To observe the deformation effect when the side of the finger presses down, the fabric piece was simplified as a cuboid model for simulation. The analysis was divided into two steps: the first step involved the soft finger opening under positive pressure, and the second step involved the finger pressing down on the fabric model. The motion and mechanical properties of the soft finger under a positive pressure of 40 kPa were simulated, as shown in Figure 4. In it, S represents stress, and LE represents strain. The redder the region in the figures, the higher the value of the corresponding parameter, while blue indicates lower values. The stress and strain distribution maps can reflect whether the finger experiences tearing due to excessive local stress or strain when pressed. In Figure 4, most regions are shown in blue, with a small portion in green. In the stress and strain distribution maps of the soft finger opening under positive pressure, the maximum stress and strain values are 0.693 MPa and 0.3549, respectively. When the soft finger presses on the fabric, the maximum stress and strain values are 4.289 MPa and 1.420 MPa, respectively. Since the stress and strain values do not exceed the tolerance limits of the silicone material, it indicates that the soft finger is appropriately pressed against the fabric without the risk of bursting due to excessive pressure at any location.

3.1.3. Sewing Machine

Selecting a computerized sewing machine can simplify the redundant sewing motions typically performed by hand, thereby improving sewing efficiency and reducing the complexity of the sewing process. The JANOME brand’s computerized sewing machine is equipped with the most advanced intelligent sewing functions, such as an intelligent fabric feed system, 196 types of stitch patterns, and an autonomous programming system. It is capable of producing high-quality straight stitches and knitted elastic stitches. The machine allows for parametric control of the sewing process, ensuring sewing quality while enhancing work efficiency. It is currently one of the most intelligent, efficient, and convenient computerized sewing machines in the world. The model used in this experiment is the JANOME M7 (Figure 5), a multi-functional computerized sewing machine. Its advantages include a variety of intelligent sewing functions, such as automatic start and stop of sewing, automatic thread cutting, and adaptive presser foot pressure adjustment. It also allows for the adjustment of multiple sewing parameters, such as stitch density, stitch tension, and sewing speed. The maximum sewing speed of this machine is 1350 revolutions per minute (r/min) [16].

Figure 6 shows the general view of the experimental set up. To ensure the feeding speed, the mechanical arm used in this experiment is the UR5 six-axis robotic arm. Figure 7 shows the experimental flowchart that was subsequently conducted using this set of equipment. First, the optimal manual-sewing parameters were determined through orthogonal experiments; these settings were then used to guide the soft finger in speed-matching trials. A regression model that predicts the soft-finger speed as a function of sewing speed and stitch density was developed and validated. Finally, sewing quality was inspected by combining deep learning instance segmentation with deep learning classification in a machine-vision framework.

3.2. Method

The purpose of this experiment is to predict the fabric feeding speed of the soft finger based on the set parameters after selecting the sewing machine parameters while ensuring the quality of sewing. Firstly, to ensure sewing quality, manual sewing of the fabric is first conducted, and orthogonal tests are used to determine the optimal sewing compatibility parameters for the fabric. Then, under these optimal sewing parameters, the fabric feeding speed of the soft robotic finger is set. When its sewing effect matches that of manual sewing, the corresponding fabric feeding speed of the soft finger is recorded. Finally, to ensure the sewing quality, it is also necessary to add the process of machine evaluation to determine whether the sewing is qualified.

3.2.1. Manual Sewing

Previous studies have demonstrated that the type of sewing thread, needle size, presser foot pressure, and stitch tension in sewing process conditions significantly affect the appearance of sewn fabrics. Since the sewing machine used in this study features an adaptive presser foot pressure adjustment function, manual regulation of the presser foot pressure is unnecessary. Therefore, this experiment focuses on three factors: the type of sewing thread, needle size, and stitch tension, for an orthogonal test design. Given that the thickness of the 15 types of fabrics used in this study ranges from 0.3 mm to 1.0 mm, three thicknesses of sewing thread (20 s/2, 40 s/2, and 60 s/2) were selected, along with three needle sizes (11, 14, and 16) and three levels of stitch tension (0.0–1.6, 1.6–3.0, and 3.0–4.5) for the experiments. The following are the bases for selecting these parameter values:

(1)

Sewing-thread linear density: Three commercially produced cotton sewing threads (20 s/2, 40 s/2, 60 s/2) were pre-screened in a pilot study. The selection brackets the range recommended by the fabric weight interval (77–395 g·m⁻²) according to ISO 4915:2021 [26].

(2)

Needle size: Sizes 11, 14 and 16 are the most frequently used needles for light-, medium- and heavy-weight cotton fabrics in apparel production [27].

(3)

Stitch tension: The Continental M7’s “stitch tension” is displayed on the screen as a dimensionless relative value ranging from 0.0 to 7.0 (higher numbers give tighter needle thread); it has no physical units such as cN, g, or N. A preliminary “wide-range” test (0–7, step 0.5) on Fabric #5 showed that tension < 1.6 frequently produced looping on the bobbin side, whereas tension > 3.0 occasionally caused puckering. The interval was therefore discretized into three operative ranges: 0–1.6 (low), 1.6–3.0 (medium), 3.0–4.5 (high). These ranges coincide with the supplier’s manual and with the industrial practice reported by Nayak and Padhye [28].

The experimental design used for the 15 pure cotton woven fabrics was based on the levels of established components and followed the orthogonal table L9(3⁴) scheme, as presented in

Table 3. The orthogonal test scheme using table of L₉(3⁴).

	Column	Factor
Test Number		I Sewing Thread	II Needle Size	ERROR	IV Stitch Tension
1		1 (20S/2)	1 (11)	1	1
2		1	2 (14)	2	2
3		1	3 (16)	3	3
4		2 (40S/2)	1	2	3
5		2	2	3	1
6		2	3	1	2
7		3 (60S/2)	1	3	2
8		3	2	1	3
9		3	3	2	1

Column (error) is the dummy column of L9(3⁴) and is used to estimate pure error in the subsequent ANOVA.

. L₉(3⁴) is a standard 4-column, 3-level, 9-run array. When only three factors are investigated, one column must be left blank to serve as the “error” term (dummy column) which is required for ANOVA and for unbiased estimation of variance [29].

Therefore, the first column (column I) is utilized to organize the variable of sewing thread and its three levels. The second column (column II) is to organize the needle size and its three levels. The forth column (column IV) was used to arrange the factor of stitch tension and its 4 levels. For the experiment, every fabric was cut into size of 150 × 150 mm. When sewing, set the seam allowance to 1 cm.

In the orthogonal tests design, a comprehensive quality index (Q) is introduced to evaluate the overall sewing performance. The equation is defined as follows:

Q = P × 0.6 + S × 0.4

(4)

where (P) represents the stitch appearance quality and (S) denotes the sewing smoothness. Both indices are assessed using a five-level grading system with a score range from 0 to 10 and a precision of 0.1, as detailed in

Table 4. The evaluation index Q.

P	S	Score
worst	worst	0–2
worse	worse	2–4
good	good	4–6
better	better	6–8
best	best	8–10

. Figure 8 illustrates the criteria for sewing-performance evaluation. Figure 8a presents the worst and best stitch appearances: the worst shows exposed under-thread and looped upper-thread, whereas the best exhibits uniform thread tension. A zig-zag stitch was adopted to make the defects more visible; therefore, a straight seam is not shown as an example. Figure 8b displays the worst and best sewing smoothness—the worst with severe fabric puckering and the best with no puckering at all.

Since stitch appearance quality directly reflects the stability of the sewing process and the formation of stitches, it has a greater impact on the esthetic and durability of the final product. Therefore, it is assigned a higher weight in the overall evaluation. Preliminary experiments indicated that the sewing machine possesses a certain self-adjusting capability, which allows it to compensate for minor smoothness deviations during stitching. However, it cannot correct defects in stitch appearance. Consequently, the weight of stitch appearance quality is set to 60%, while sewing smoothness is assigned 40%, ensuring that the evaluation focuses on the most critical aspects of sewing quality in practical applications. Additionally, during the experimental process, each type of fabric was sewn three times, and the average of the three scoring results was taken as the final Q value, with the Q value rounded to two decimal places.

The comprehensive index (Q) takes into account both the visual sensitivity to stitch appearance and the flatness of the sewn fabric, demonstrating good representativeness and discriminability. Since the comprehensive quality index Q defined in Equation (4) is already a deterministic “larger-is-better” response, the optimum parameter combination was obtained directly by ranking the average Q values; no additional signal-to-noise ratio was calculated.

3.2.2. Sewing with Soft Finger

During the preliminary experiments, it was found that in addition to the sewing speed of the sewing machine affecting the fabric-feeding speed of the soft finger, the stitch density of the sewing machine also influences the fabric-feeding speed of the soft finger. Under the same sewing machine speed, the higher the stitch density, the slower the fabric-feeding speed of the soft finger. Therefore, in this experiment, two variables (sewing machine speed and stitch density) are selected to explore how the soft finger can achieve intelligent sewing. Specifically, after obtaining the optimal sewing parameters for each fabric through orthogonal tests with manual sewing, the same parameters are selected, and the sewing speed and stitch density of the sewing machine are varied. When the sewing quality achieved by the soft finger’s fabric feeding matches the best quality obtained through manual sewing of the fabric, the fabric-feeding speed of the soft finger at that moment is recorded. The stitch density levels selected are 2 S.P.I., 2.5 S.P.I., 3 S.P.I., 3.5 S.P.I., 4 S.P.I., 4.5 S.P.I., and 5 S.P.I., while the sewing machine speeds selected are 250 r/min, 375 r/min, 690 r/min, 780 r/min, 900 r/min, and 1350 r/min. Therefore, each fabric will undergo 36 speed-matching experiments.

4. Results and Discussion

4.1. Optimal Sewing Parameters

Table 5. Results of the evaluation index (Q) in the orthogonal tests for manual sewing.

Fabric	Test Number
	1	2	3	4	5	6	7	8	9
#1	4.44	5.23	6.50	9.43	6.38	9.22	9.14	6.41	8.10
#2	3.47	4.79	4.93	9.76	8.16	8.52	9.38	8.97	8.92
#3	4.10	4.67	6.49	9.44	8.98	9.00	9.33	5.74	5.59
#4	4.22	5.91	6.13	9.84	6.69	9.68	9.35	6.57	7.05
#5	3.94	5.56	7.02	9.86	8.25	8.36	9.84	6.93	5.11
#6	4.00	6.66	9.14	9.53	7.63	9.03	9.45	7.20	6.19
#7	3.26	6.37	4.73	9.83	4.94	5.39	9.40	7.89	7.91
#8	4.06	5.75	7.23	9.44	7.49	7.06	9.53	7.81	6.52
#9	4.41	5.87	7.24	9.70	8.59	8.07	9.87	5.78	8.29
#10	4.58	5.91	9.63	9.67	8.26	8.84	9.56	7.59	8.92
#11	4.95	6.07	9.74	9.53	8.19	7.49	9.58	5.89	5.18
#12	3.87	5.74	5.27	9.82	5.94	6.27	9.73	6.00	6.32
#13	3.87	5.64	6.03	9.90	7.67	6.45	9.90	8.17	8.29
#14	4.01	5.58	5.85	7.95	7.54	5.72	9.90	6.77	6.22
#15	3.87	5.64	6.03	9.90	5.61	7.71	9.90	6.46	6.78

records the results of the evaluation index (Q) in the orthogonal tests for manual sewing of the 15 types of fabrics.

Taking the orthogonal tests results of Fabric #5 as an example (

Table 6. The process of range analysis using fabric 5 as an example.

Index	Sewing Thread	Needle Size	Stitch Tension
k1	5.507	7.880	5.767
k2	8.823	6.913	7.920
k3	7.293	6.830	7.937
R	3.316	1.050	2.170

), the impact of each factor can be ranked from the most significant to the least significant based on the range value (R) calculations. The ideal combination of levels can then be selected according to the k value, where a higher k value indicates a stronger influence on the test index. From the R values in Table 6, it is evident that the type of sewing thread has the greatest impact, followed by stitch density, and then needle size. This indicates that for this fabric, the choice of sewing thread has the most significant effect on sewing quality, while the needle size has the least impact. Further analysis of the k values reveals that for stitch tension, although the k3 value is the highest, the k2 value is only 0.017 lower than k3. This suggests that for Fabric #5, both levels of stitch tension (1.6–3.0 and 3.0–4.5) yield similar sewing results. Therefore, there are two optimal sewing parameter combinations for Fabric #5: one involves using 40 s/2 sewing thread, a size 11 needle, and a stitch tension of 3.0–4.5; the other involves using 40 s/2 sewing thread, a size 11 needle, and a stitch tension of 1.6–3.0. Correspondingly, in the soft finger sewing experiments, this fabric should be tested using both of these parameter combinations.

Figure 9 provides a concise summary of the range and significance results for the three parameters that influence the sewing quality of 15 different types of textiles. It can be observed that the type of sewing thread has the most significant impact on the fabric, followed by stitch tension, and then needle size. Therefore, for commonly used cotton fabrics, to ensure sewing quality, the first consideration should be the selection of the appropriate sewing thread, followed by adjustment of the thread tension, while the needle size should be chosen to match the sewing thread.

Table 7. Optimal matching of manual sewing parameters for the fifteen fabrics.

Fabric	Sewing Thread	Needle Size	Stitch Tension
#1	40 s/2	16	1.6–3.0
#2	60 s/2	11	3.0–4.5
#3	40 s/2	11	1.6–3.0
#4	40 s/2	11	1.6–3.0
#5	40 s/2	11	3.0–4.5
	40 s/2	11	1.6–3.0
#6	40 s/2	16	3.0–4.5
#7	60 s/2	11	3.0–4.5
#8	40 s/2	11	3.0–4.5
	60 s/2	11	3.0–4.5
#9	40 s/2	11	3.0–4.5
#10	40 s/2	16	3.0–4.5
#11	40 s/2	11	3.0–4.5
#12	40 s/2	11	1.6–3.0
	60 s/2	11	1.6–3.0
#13	60 s/2	11	3.0–4.5
#14	60 s/2	11	1.6–3.0
#15	40 s/2	11	1.6–3.0
	60 s/2	11	1.6–3.0

summarizes the optimal sewing parameters for these 15 fabrics. It can be seen that for heavier fabrics, i.e., those with higher weight or stiffer cotton fabrics, a thicker sewing thread should be selected for sewing.

It should be noted that the primary objective of this study is to establish a robust speed-matching model for the soft finger during automated sewing, rather than to strictly follow traditional Taguchi optimization practices. In this context, the presence of two equally viable optimal parameter combinations for certain fabrics (i.e., fabrics #5, #8, #12, and #15) was not further narrowed down to a single choice. Instead, both combinations were retained and tested independently. This decision was made to increase the volume of experimental data, thereby enhancing the reliability and generalizability of the resulting speed-matching model. Consequently, these four fabrics underwent 72 speed-matching experiments instead of 36, providing a more comprehensive dataset for model training and validation.

4.2. The Sewing Model for the Soft Finger

4.2.1. Sewing Model

Figure 10 illustrates the process of automatic sewing completed by the soft finger using the proposed method, further demonstrating the feasibility of the theoretical approach presented in this paper. Prior to sewing, the soft finger, under a positive pressure of 40 kPa, is positioned directly above the fabric pressing location (Figure 10a). The soft finger is then slowly lowered until it presses against the fabric surface (Figure 10b). Subsequently, the sewing machine commences sewing, with the soft finger synchronously assisting the sewing process at the same speed as the sewing machine (as shown in Figure 10c–e). Figure 10f illustrates the outcome of the straight-line sewing performed by the soft finger, characterized by uniform stitches and excellent sewing quality.

During the experiments, it was observed that the soft finger cannot immediately accelerate to feed the fabric once the sewing machine starts sewing. Instead, it needs to wait for a certain period before synchronizing its fabric feeding with the sewing machine. Otherwise, the soft finger may move too quickly, pulling the fabric out of alignment and causing skewed stitches, which results in substandard sewing quality. Therefore,

Table 8. Experimental results of sewing assisted by soft fingers.

Sewing Speed r/min	Stitch Density	Pushing Speed [mm/s]		Latency Time [s]
		Initial Speed	Final Speed
250	2.0	8.0	8.0	0.3
	2.5	10.0	10.0
	3.0	12.0	12.0
	3.5	13.4	13.4
	4.0	14.5	14.5
	4.5	17.0	17.0
	5.0	18.0	18.0
375	2.0	12.6	12.6	0.3
	2.5	15.5	15.5
	3.0	18.5	18.5
	3.5	20.5	20.5
	4.0	23.5	23.5
	4.5	26.0	26.0
	5.0	28.0	28.0
690	2.0	21.0	21.0	0.5
	2.5	26.0	26.0
	3.0	30.0	30.0
	3.5	34.5	34.5
	4.0	38.5	38.5
	4.5	43.5	43.5
	5.0	46.0	46.0
780	2.0	23.5	31	0.55
	2.5	28	35
	3.0	29.5	42.5
	3.5	31	48.5
	4.0	33	55
	4.5	37	57
	5.0	39	61
900	2.0	26	37.5	0.6
	2.5	28	44
	3.0	30	47
	3.5	32	51
	4.0	34	57
	4.5	36	62
	5.0	38	63.5
1350	2.0	27	43.5	0.7
	2.5	30.5	47
	3.0	33	49
	3.5	35.5	53
	4.0	39	58
	4.5	42	60
	5.0	45	65

not only records the fabric-feeding speed of the soft finger under different sewing machine speeds and stitch densities but also documents the waiting time for the soft finger to begin sewing.

Additionally, it was found that when the sewing machine speed exceeds 690 r/min, the soft finger cannot maintain a single speed to complete the fabric feeding and sewing process. Otherwise, the soft finger’s speed will eventually lag behind that of the sewing machine. To maintain synchronization, the soft finger needs to adjust its speed a second time to match the sewing machine’s speed. Thus, Table 8 also records the initial and final speeds of the soft finger during the fabric-feeding process.

When the sewing machine begins sewing, its sewing speed does not instantly jump from zero to the designated speed but gradually increases to that speed over a certain response time, completing the process from acceleration to steady sewing. In contrast, the robotic arm, with its sufficiently high acceleration (1200 mm/s), can almost instantaneously set the soft finger in motion. Therefore, at the start of sewing, the soft finger must wait for the sewing machine’s acceleration period, which is the waiting time.

Additionally, because the sewing needle begins to descend and sew the moment the sewing machine starts, it does not wait for the sewing machine to fully accelerate before beginning to sew. Thus, when the sewing machine’s set speed is too high, specifically greater than 690 r/min in this study, the sewing machine has not yet fully accelerated by the time the first stitch is sewn. This means the fabric is initially sewn at a slower speed and then sewn at a steady speed once the sewing machine has fully accelerated. This explains why, in this study, the final fabric-feeding speed of the soft finger is greater than the initial fabric-feeding speed when the sewing machine speed exceeds 690 r/min. Conversely, when the sewing machine speed is less than 690 r/min, the sewing machine has already completed its acceleration process before the first stitch is sewn. Therefore, the final fabric-feeding speed of the soft finger is the same as the initial fabric-feeding speed. This indirectly confirms the sewing principle mentioned earlier: the fabric-feeding speed of the soft finger must match the sewing speed of the sewing machine to ensure balanced forces on the fabric during sewing, thereby achieving high-quality straight-line sewing.

Furthermore, once the sewing parameters were fixed, the soft-finger feeding speed and latency stayed virtually unchanged across all fifteen fabric types, and the Pearson coefficients in

Table 9. Results of correlation analysis.

Factors	Initial Pushing Speed		Final Pushing Speed		Latency Time
	Pearson Correlation	Significance	Pearson Correlation	Significance	Pearson Correlation	Significance
Weight	0.000	1.000	0.000	1.000	0.000	1.000
Thickness	0.000	1.000	0.000	1.000	0.000	1.000
Warp bending	0.000	1.000	0.000	1.000	0.000	1.000
Weft bending	0.000	1.000	0.000	1.000	0.000	1.000
Static friction coefficient	0.000	1.000	0.000	1.000	0.000	1.000
Dynamic friction Coefficient	0.000	1.000	0.000	1.000	0.000	1.000
Sewing speed	0.715 **	<0.01	0.824 **	<0.01	0.970 **	0.00
Stitch density	0.534 **	<0.01	0.419 **	<0.01	0.000	1.000

** At level 0.01 (two-tailed), the correlation was significant.

were exactly zero. This apparent independence of fabric identity should, however, be interpreted cautiously: the experiment covered only cotton wovens within a limited range of areal densities (77–395 g m⁻²) and bending stiffnesses, so any subtle, non-linear influence of fabric structure could have been masked. Enlarging the population to include knits, linens, synthetic blends and coated textiles will therefore be a primary objective of our future work, both to verify the universality of the current model and to refine the control strategy if fiber-type or fabric architecture effects emerge. Moreover, Table 9 shows that both the seeding speed of the soft finger and the stitch density are strongly correlated with the initial and final pushing speed. Latency time is significantly correlated with sewing speed (p < 0.01); stitch density, however, shows no significant relationship.

In conclusion, as indicated by Table 8, the initial fabric-feeding speed (IS) and the final fabric-feeding speed (FS) of the soft finger are significantly correlated with both the sewing speed (SP) and stitch density (SD) of the sewing machine, which aligns with the prior analysis results. In contrast, the waiting time (LT) of the soft finger is solely associated with the sewing speed (SP) of the sewing machine.

Let IS represent the initial fabric-feeding speed of the soft finger, FS is the final fabric-feeding speed of the soft finger, LT is the waiting time of the soft finger, SP is the sewing speed of the sewing machine, and SD is the stitch density of the sewing machine. Through linear regression analysis, the following sewing models (Equations (5)–(7)) can be derived:

$$\mathrm{LT} = 0.0004 \times \mathrm{SP} + 0.2013 (R^2 = 0.943)$$

(5)

$$IS =0.02 \times SP +5.288 \times SD - 4.826 (R^2 = 0.797)$$

(6)

$$FS=\left\{ \begin{array}{c}0.02 \times SP+5.288 \times SD - 4.826 (0 < SP \le 690 \mathrm{r}/\mathrm{m}\mathrm{i}\mathrm{n})\hfill \\ 0.009 \times SP+8.810 \times SD+10.453 (SP > 690 \mathrm{r}/\mathrm{m}\mathrm{i}\mathrm{n})\hfill \end{array}\right. (R^2 = 0.941)$$

(7)

Fabric initial alignment at the needle was carried out in two steps. First, the vision-guided drop-off strategy described in our previous work [30] was used to place the ply stack directly on the needle plate with ≤1 mm positional error. Immediately after placement, the soft finger pushed the ply edge forward until it touched the pre-taught point recorded in the teach pendant (XYZ = 0, 0, 0 in the sewing-machine coordinate frame). Once the fabric reached this taught initial position, a digital I/O signal triggered the JANOME M7 to start automatically. During sewing, the UR5 executed a pre-programmed straight-line G-code (start point = taught position, end point = target distance, feed rate = calculated from Equations (5)–(7)) so that the soft-finger speed was synchronized with the sewing-machine feed; no real-time path correction was performed. Consequently, the present implementation is limited to straight seams whose trajectory is defined a priori.

4.2.2. Validation Test

To verify the reliability of the soft finger sewing model, a pure cotton fabric with the parameters listed in

Table 10. Fabric sewing parameters for the validation test.

Weight [g/m2]	Thickness [mm]	Bending Stiffness [mN·cm]		Friction Coefficient		Sewing Thread
		Warp	Weft	Warp	Weft
190.467	0.391	18.26	25.48	0.195	0.205	40 s/2
Needle size	Stitch tension	Sewing speed	Stitch density	IS	FS	LT
14	1.6–3.0	800	3.0	27.038	44.083	0.510

was selected. The sewing parameters for this fabric, based on the proposed sewing model, are also provided. To further investigate the stability of the sewing quality, the fabric was cut into four pieces, each measuring 15 cm × 15 cm, and named FA, FB, FC, and FD. The process involved sewing pieces FA and FB together to form Fabric A, named A and then sewing pieces FC and FD together to form Fabric B, named B. Finally, Fabric A and Fabric B were sewn together. The process of sewing is illustrated in Figure 11. Figure 12 illustrates the actual sewing process of Fabric A and Fabric B using a soft finger.

As shown in Figure 12, the method proposed in this study not only achieves high-quality automatic sewing but also enables the continuous sewing of long straight seams by mimicking the multiple fabric-feeding actions of manual sewing for long pieces of fabric. Specifically, the soft finger first presses the fabric in preparation for sewing (Figure 12a), then feeds the fabric for straight-line sewing in the same manner as shown in Figure 12b. When the fabric-feeding position of the soft finger reaches the limit of the robotic arm, the sewing machine is paused, and the soft finger is lifted (Figure 12c). The soft finger is then adjusted back to the initial sewing position to prepare for the second cycle of sewing (Figure 12d,e). The sewing machine is restarted to continue sewing, with the soft finger assisting in feeding the remaining fabric (Figure 12f,g). Figure 12h shows the intelligent sewing result of fabric A and fabric B, while Figure 12i displays the effect after unfolding the sewn fabric. It can be seen that the stitch quality is qualified, and the seam allowances are well-aligned.

The sewing process of the four fabric pieces shown in Figure 11 is also the basic four-square pattern sewing process in quilting art. Quilting is a process of piecing fabric pieces into functional or artistic works by following a pattern or design using a specific type of seam, namely, flat seam stitching. In this process, repetitive motions account for 95% of the entire production, which consumes a significant amount of physical effort and time from the quilter and is in urgent need of automation assistance. Therefore, the method proposed in this study can not only provide a reference for the automation of garment sewing but also contribute to the intelligent development of quilting.

5. Quality Inspection

Even though the soft finger-assisted sewing can achieve excellent sewing quality through the sewing experiments, it is still necessary to have a machine-based evaluation process to determine whether the sewing quality is qualified.

Machine vision technology [31,32], with its high precision, high-speed, and non-contact inspection capabilities, is well-suited to meet the requirements for sewing quality inspection. As illustrated in Figure 13, an industrial camera (Hikrobot MV-CE200-10GM(Hikvision in shanghai of China)) was mounted on the robot arm to acquire images immediately after each fabric sewing. These images were subsequently uploaded to the AI-training environment within Vision Master.

Vision Master is a zero-code, modular vision platform engineered for industrial deployment. Its deep learning instance segmentation engine, built on a Mask R-CNN architecture with a Feature Pyramid Network (FPN) backbone, couples pixel-level mask prediction with bounding-box regression in a single forward pass. Leveraging the CNN’s hierarchical feature-extraction capacity, the module automatically delineates each individual stitch region—even when seams overlap or intertwine—without requiring hand-crafted edge operators. Training is performed through interactive polygon annotation followed by on-the-fly data augmentation (brightness ±10%, rotation ±5°, horizontal flip), after which the platform exports a runtime model that infers 640 × 480 images in <80 ms on a mid-range GPU, enabling one-shot, template-free segmentation at production line speed [30].

Equally zero-code, Vision Master’s deep learning image-classification engine employs a residual CNN (ResNet-50/101) pretrained on millions of industrial defect images. Transfer learning is initiated by simply dragging two folders—“OK” and “NG”—into the graphical workspace; the network fine-tunes its final fully connected layer through stochastic gradient descent with adaptive learning-rate decay, while an embedded hard-example-mining algorithm automatically re-weights ambiguous samples across epochs. This yields a high-dimensional decision boundary that generalizes to previously unseen fabric faults after only 15–20 min of training on <200 images per class. The resulting .vmp model is seamlessly invoked by the Vision Master runtime, returning a classification confidence and top-1 result within 20 ms, thereby achieving >93% accuracy without any manual feature engineering or Python 3.8 coding and allowing sewing-quality inspection to be deployed or re-targeted in under one shift. During training and model evaluation, the platform automatically tunes its internal parameters—such as thresholds and scale settings—by analyzing low-level image attributes (brightness, texture, and edge content). Advanced users can further refine these hyper-parameters to suit application-specific constraints [30].

The collected image data is then transmitted to an image processing system. The system compares the actual fabric piece with the preset standard templates and parameters. If the system detects that the fabric piece’s position exceeds the allowable deviation range or its shape does not meet the requirements, it will immediately issue an alarm signal to notify the operator for prompt handling.

Specifically, a total of 1850 fabric pieces, which achieved excellent sewing quality under different sewing machine parameters and stitch settings, were photographed using the MV-CE200-10GM industrial camera. Additionally, 300 fabric pieces with substandard sewing quality were randomly photographed (Figure 14). These images were then transferred to the AI training platform of Vision Master for training. Figure 15 illustrates the images gallery setting page of Vision Master.

The training process consists of two steps: first, a deep learning instance segmentation method is used to identify the sewing locations in the images. Figure 16 shows the record of marking the sewing area locations on the fabric using the Vision Master 4.3 software, followed by model training and evaluation using the software’s built-in deep learning unsupervised instance segmentation method.

Table 11. Training report of the deep learning instance segmentation method generated by the Vision Master.

Training Report
Task ID	1732_9_20250906184108903
Model Version	V8
Training Images	1780
Training Time	48 min 38 s
Incremental Model	NO
Release	VM500
Model Type	High-precision
Epochs	89
Rotation Range	180°
Base Learning Rate	0.0001
Batch Size	8
Model Precision Mode	Fast
Model UID	1732_124106
Data Augmentation	Default Configuration

is the training parameter report generated by the system. In the deep learning instance segmentation, the platform automatically split the dataset into a training set of 1780 images and a test set of 370 images. All training hyper-parameters and on-the-fly augmentation strategies were left at their default values: 89 epochs, rotation range of 180°, and a mini-batch size of 8. Model performance was assessed via the confusion matrix reported in

Table 12. Confusion Matrix on the test set of deep learning instance segmentation.

	Predicted Sewing Area	Predicted Background
Actual sewing area	367	1
Actual background	1	0

, which yields a precision of 99.73%.

Then, a deep learning classification method is employed to label the sewing quality of the identified regions, with “OK” indicating qualified sewing quality and “NG” indicating unqualified sewing quality. Figure 17 illustrates the process of labeling. The prediction accuracy of this method for fabric sewing quality is evaluated using the automatic training and prediction functions in Vision Master.

Table 13. Training report of the deep learning classification method generated by the Vision Master.

Training Report
Task ID	2437_1_20250906184058992
Model Version	V3
Training Images	1503
Training Time	28 min 25 s
Incremental Model	NO
Release	VM500
Model Type	High-precision
Epochs	100
Base Learning Rate	0.0001
Batch Size	32
Model UID	2437_123923
Data Augmentation	Default Configuration
ROI Perturbation	Enabled
Width/Height Jitter	[0.9, 1.2]
Rotation Jitter	[−3°, 3°]
Number of Perturbations	3

is the training parameter report generated by the system. For the deep learning classification stage, the dataset was manually partitioned into a training set of 1503 images and a test set of 674 images. Training hyper-parameters and data-augmentation policies were retained at the system defaults: model-capacity set to “high-accuracy”, 100 epochs, mini-batch size 32. ROI-perturbation parameters were width/height jitter [0.9, 1.2], rotation jitter [−3°, 3°], and number of perturbations 3. Figure 18 shows the prediction results. The dashed box represents the prediction results, while the solid box represents the actual results. Additionally, to distinguish between actual and predicted values, the system automatically represents the predicted values as “OK-1” for qualified sewing quality and “NG-1” for unqualified sewing quality. Model performance was assessed via the confusion matrix reported in

Table 14. Confusion Matrix on the test Set of deep learning classification.

	Predicted NG	Predicted OK
Actual NG	41	8
Actual OK	8	592

, which yields a precision of 97.84%.

It should be emphasized that the machine vision system integrated in this study is used solely for off-line quality inspection after the sewing process is completed, and does not participate in real-time alignment, path correction, or closed-loop control during sewing. Throughout the sewing operation, fabric alignment and path guidance rely entirely on initial positioning and the pre-defined motion trajectory of the soft robotic finger. The system is incapable of responding to fabric drift or deformation in real time.

6. Conclusions

This study proposes an automated sewing system based on soft fingers and machine vision technology, aiming to address the challenge of automating the sewing process in garment manufacturing. Through the analysis of the mechanical principles of soft fingers and experimental validation, combined with the high-precision detection capabilities of machine vision technology, high-quality automated sewing has been successfully achieved. The results indicate that the fabric-feeding speed and waiting time of the soft finger are closely related to the sewing speed and stitch density of the sewing machine, but not to the fabric properties. The established sewing model can accurately predict the fabric-feeding speed and waiting time of the soft finger, thereby ensuring stable sewing quality. Additionally, the introduction of the machine vision system further ensures the reliability of sewing quality, achieving a classification accuracy of 97.84%. This study not only provides theoretical and technical support for the intelligent upgrading of the garment manufacturing industry but also lays the foundation for the automation of complex sewing processes such as quilting.

Although the present work demonstrates the feasibility of soft-finger/sewing-machine synchronization and off-line stitch inspection, it must be emphasized that the system still lacks real-time seam-tracking and error-correction capabilities. (1) The fabric has to be manually aligned to a laser-projected 1 cm seam allowance. (2) The soft finger follows a pre-defined straight-line G-code and cannot compensate for instantaneous deviations caused by fabric tension or inter-ply slip. (3) The vision module serves only as a posterior check, providing no closed-loop feedback. Consequently, the current platform is restricted to regular, straight and a priori known stitch lines with accurate initial positioning, and is still far from genuine “autonomous seam placement”.

Future efforts will embed a micro-camera ahead of the needle to detect seam deviation within <100 ms and enable closed-loop trajectory correction; coupled with fiducial markers and multi-modal sensing, the system will transcend straight-line constraints and achieve fully autonomous curved-path sewing, propelling the advancement of intelligent seaming.