# Curve Skeleton Extraction from Incomplete Point Clouds of Livestock and Its Application in Posture Evaluation

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Experimental Data

#### 2.2. Livestock Data Pre-Processing

- The origin of the CCS is set as the centroid of the livestock.
- The Z-axis is perpendicular to the bilateral symmetry plane, and its positive side points to the right side of the body.
- The Y-axis is perpendicular to the ground plane, and its positive direction points to the dorsal of the livestock.
- The X-axis is perpendicular to the plane that consists of the Z-axis and Y-axis, and its positive direction goes from the origin of the CCS to the head of the livestock.

#### 2.3. Curve Skeleton Extraction

#### 2.3.1. Construct the Contours of the Side Views

#### 2.3.2. Skeleton Extraction and Division

- (i)
- Skeleton division based on detection (implement on pigs)

- (ii)
- Skeleton division based on spatial relationships

#### 2.3.3. Calculation of the Leg Skeleton Position

#### 2.3.4. Calculation of the Torso Skeleton Position

#### 2.4. Experimental Data and Posture Evaluation Application

#### 2.4.1. Evaluation of Correct Body Posture Measurement

## 3. Results

#### 3.1. Curve Skeleton Extraction

#### 3.2. Results of Posture Evaluation

#### 3.3. Results and Comparison with Other Animals’ References

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Pezzuolo, A.; Guarino, M.; Sartori, L.; González, L.A.; Marinello, F. On-barn pig weight estimation based on body measurements by a Kinect v1 depth camera. Comput. Electron. Agric.
**2018**, 148, 29–36. [Google Scholar] [CrossRef] - Condotta, I.C.; Brown-Brandl, T.M.; Pitla, S.K.; Stinn, J.P.; Silva-Miranda, K.O. Evaluation of low-cost depth cameras for agricultural applications. Comput. Electron. Agric.
**2020**, 173, 105394. [Google Scholar] [CrossRef] - Kawasue, K.; Ikeda, T.; Tokunaga, T.; Harada, H. Three-dimensional shape measurement system for black cattle using KINECT sensor. Int. J. Circuits Syst. Signal Process.
**2013**, 7, 222–230. [Google Scholar] - Viazzi, S.; Bahr, C.; Van Hertem, T.; Schlageter-Tello, A.; Romanini, C.; Halachmi, I.; Lokhorst, C.; Berckmans, D. Comparison of a three-dimensional and two-dimensional camera system for automated measurement of back posture in dairy cows. Comput. Electron. Agric.
**2014**, 100, 139–147. [Google Scholar] [CrossRef] - Guo, H.; Ma, X.; Ma, Q.; Wang, K.; Su, W.; Zhu, D. LSSA_CAU: An interactive 3d point clouds analysis software for body measurement of livestock with similar forms of cows or pigs. Comput. Electron. Agric.
**2017**, 138, 60–68. [Google Scholar] [CrossRef] - Guo, H.; Wang, K.; Su, W.; Zhu, D.; Liu, W.; Xing, C.; Chen, Z. 3D Scanning of Live Pigs System and Its Application in Body Measurements. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2017**, 42, 211–217. [Google Scholar] [CrossRef] [Green Version] - Salau, J.; Haas, J.H.; Junge, W.; Thaller, G. A multi-Kinect cow scanning system: Calculating linear traits from manually marked recordings of Holstein-Friesian dairy cows. Biosyst. Eng.
**2017**, 157, 92–98. [Google Scholar] [CrossRef] - Wang, K.; Guo, H.; Ma, Q.; Su, W.; Chen, L.; Zhu, D. A portable and automatic Xtion-based measurement system for pig body size. Comput. Electron. Agric.
**2018**, 148, 291–298. [Google Scholar] [CrossRef] - Wang, K.; Zhu, D.; Guo, H.; Ma, Q.; Su, W.; Su, Y. Automated calculation of heart girth measurement in pigs using body surface point clouds. Comput. Electron. Agric.
**2019**, 156, 565–573. [Google Scholar] [CrossRef] - Shuai, S.; Ling, Y.; Shihao, L.; Haojie, Z.; Xuhong, T.; Caixing, L.; Aidong, S.; Hanxing, L. Research on 3D surface reconstruction and body size measurement of pigs based on multi-view RGB-D cameras. Comput. Electron. Agric.
**2020**, 175, 105543. [Google Scholar] [CrossRef] - Salau, J.; Haas, J.H.; Junge, W.; Thaller, G. Automated calculation of udder depth and rear leg angle in Holstein-Friesian cows using a multi-Kinect cow scanning system. Biosyst. Eng.
**2017**, 160, 154–169. [Google Scholar] [CrossRef] - Ruchay, A.; Dorofeev, K.; Kalschikov, V.; Kolpakov, V.; Dzhulamanov, K. Accurate 3D shape recovery of live cattle with three depth cameras. Proc. IOP Conf. Ser. Earth Environ. Sci.
**2019**, 341, 012147. [Google Scholar] [CrossRef] - Ruchay, A.; Dorofeev, K.; Kalschikov, V.; Kolpakov, V.; Dzhulamanov, K. A depth camera-based system for automatic measurement of live cattle body parameters. Proc. IOP Conf. Ser. Earth Environ. Sci.
**2019**, 341, 012148. [Google Scholar] [CrossRef] - Shi, C.; Zhang, J.; Teng, G. Mobile measuring system based on LabVIEW for pig body components estimation in a large-scale farm. Comput. Electron. Agric.
**2019**, 156, 399–405. [Google Scholar] [CrossRef] - Van Hertem, T.; Viazzi, S.; Steensels, M.; Maltz, E.; Antler, A.; Alchanatis, V.; Schlageter-Tello, A.A.; Lokhorst, K.; Romanini, E.C.; Bahr, C.; et al. Automatic lameness detection based on consecutive 3D-video recordings. Biosyst. Eng.
**2014**, 119, 108–116. [Google Scholar] [CrossRef] - Jabbar, K.A.; Hansen, M.F.; Smith, M.L.; Smith, L.N. Early and non-intrusive lameness detection in dairy cows using 3-dimensional video. Biosyst. Eng.
**2017**, 153, 63–69. [Google Scholar] [CrossRef] - Pezzuolo, A.; Milani, V.; Zhu, D.; Guo, H.; Guercini, S.; Marinello, F. On-Barn pig weight estimation based on body measurements by structure-from-motion (SfM). Sensors
**2018**, 18, 3603. [Google Scholar] [CrossRef] [Green Version] - Le Cozler, Y.; Allain, C.; Xavier, C.; Depuille, L.; Caillot, A.; Delouard, J.; Delattre, L.; Luginbuhl, T.; Faverdin, P. Volume and surface area of Holstein dairy cows calculated from complete 3D shapes acquired using a high-precision scanning system: Interest for body weight estimation. Comput. Electron. Agric.
**2019**, 165, 104977. [Google Scholar] [CrossRef] - Song, X.; Bokkers, E.; van Mourik, S.; Koerkamp, P.G.; van der Tol, P. Automated body condition scoring of dairy cows using 3-dimensional feature extraction from multiple body regions. J. Dairy Sci.
**2019**, 102, 4294–4308. [Google Scholar] [CrossRef] [Green Version] - Lu, J.; Guo, H.; Du, A.; Su, Y.; Ruchay, A.; Marinello, F.; Pezzuolo, A. 2-D/3-D fusion-based robust pose normalisation of 3-D livestock from multiple RGB-D cameras. Biosyst. Eng.
**2021**. [Google Scholar] [CrossRef] - Du, A.; Guo, H.; Lu, J.; Su, Y.; Ma, Q.; Ruchay, A.; Marinello, F.; Pezzuolo, A. Automatic livestock body measurement based on keypoint detection with multiple depth cameras. Comput. Electron. Agric.
**2022**, 198, 107059. [Google Scholar] [CrossRef] - Tagliasacchi, A.; Delame, T.; Spagnuolo, M.; Amenta, N.; Telea, A. 3d skeletons: A state-of-the-art report. In Proceedings of the Computer Graphics Forum; John Wiley & Sons Ltd.: Hoboken, NJ, USA, 2016; Volume 35, pp. 573–597. [Google Scholar]
- Bai, X.; Latecki, L.J.; Liu, W.Y. Skeleton pruning by contour partitioning with discrete curve evolution. IEEE Trans. Pattern Anal. Mach. Intell.
**2007**, 29, 449–462. [Google Scholar] [CrossRef] [PubMed] - Bai, X.; Latecki, L.J. Discrete skeleton evolution. In Proceedings of the International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition; Springer: Heidelberg/Berlin, Germany, 2007; pp. 362–374. [Google Scholar]
- Tagliasacchi, A.; Alhashim, I.; Olson, M.; Zhang, H. Mean curvature skeletons. In Proceedings of the Computer Graphics Forum; John Wiley & Sons Ltd.: Hoboken, NJ, USA, 2012; Volume 31, pp. 1735–1744. [Google Scholar]
- Cao, J.; Tagliasacchi, A.; Olson, M.; Zhang, H.; Su, Z. Point cloud skeletons via laplacian based contraction. In Proceedings of the 2010 Shape Modeling International Conference, Aix-en-Provence, France, 21–23 June 2010; pp. 187–197. [Google Scholar]
- Tagliasacchi, A.; Zhang, H.; Cohen-Or, D. Curve skeleton extraction from incomplete point cloud. ACM Trans. Graph.
**2009**, 28, 1–9. [Google Scholar] [CrossRef] [Green Version] - Huang, H.; Wu, S.; Cohen-Or, D.; Gong, M.; Zhang, H.; Li, G.; Chen, B. L1-medial skeleton of point cloud. ACM Trans. Graph.
**2013**, 32, 1–8. [Google Scholar] - Sundar, H.; Silver, D.; Gagvani, N.; Dickinson, S. Skeleton based shape matching and retrieval. In Proceedings of the 2003 Shape Modeling International, Seoul, Korea, 12–15 May 2003; pp. 130–139. [Google Scholar]
- Yan, H.B.; Hu, S.; Martin, R.R.; Yang, Y.L. Shape deformation using a skeleton to drive simplex transformations. IEEE Trans. Vis. Comput. Graph.
**2008**, 14, 693–706. [Google Scholar] [PubMed] - Seylan, Ç.; Sahillioğlu, Y. 3D skeleton transfer for meshes and clouds. Graph. Model.
**2019**, 105, 101041. [Google Scholar] [CrossRef] - Lin, C.; Li, C.; Liu, Y.; Chen, N.; Choi, Y.; Wang, W. Point2Skeleton: Learning Skeletal Representations from Point Clouds. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition; IEEE Computer Society: Los Alamitos, CA, USA, 2021; pp. 4275–4284. [Google Scholar] [CrossRef]
- Garcia, F.; Ottersten, B. Real-time curve-skeleton extraction of human-scanned point clouds. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP 2015), Berlin, Germany, 3–5 May 2015; pp. 54–60. [Google Scholar]
- Barros, J.M.D.; Garcia, F.; Sidibé, D. Real-time Human Pose Estimation from Body-scanned Point Clouds. In VISAPP 2015, Proceedings of the 10th International Conference on Computer Vision Theory and Applications, Berlin, Germany, 11–14 March 2015; Braz, J., Battiato, S., Imai, F.H., Eds.; SciTePress: Setúbal, Portugal, 2015; Volume 1, pp. 553–560. [Google Scholar] [CrossRef] [Green Version]
- Li, R.; Si, W.; Weinmann, M.; Klein, R. Constraint-Based Optimized Human Skeleton Extraction from Single-Depth Camera. Sensors
**2019**, 19, 2604. [Google Scholar] [CrossRef] [Green Version] - Livny, Y.; Yan, F.; Olson, M.; Chen, B.; Zhang, H.; El-Sana, J. Automatic reconstruction of tree skeletal structures from point clouds. In ACM SIGGRAPH Asia 2010 Papers; Association for Computing Machinery: Seoul, Korea, 2010; pp. 1–8. [Google Scholar]
- Wu, S.; Wen, W.; Xiao, B.; Guo, X.; Du, J.; Wang, C.; Wang, Y. An accurate skeleton extraction approach from 3D point clouds of maize plants. Front. Plant Sci.
**2019**, 10, 248. [Google Scholar] [CrossRef] [Green Version] - Lu, X.; Deng, Z.; Luo, J.; Chen, W.; Yeung, S.K.; He, Y. 3D articulated skeleton extraction using a single consumer-grade depth camera. Comput. Vis. Image Underst.
**2019**, 188, 102792. [Google Scholar] [CrossRef] - Au, O.K.C.; Tai, C.L.; Chu, H.K.; Cohen-Or, D.; Lee, T.Y. Skeleton extraction by mesh contraction. ACM Trans. Graph. (TOG)
**2008**, 27, 1–10. [Google Scholar] [CrossRef] - Guo, H.; Li, Z.; Ma, Q.; Zhu, D.; Su, W.; Wang, K.; Marinello, F. A bilateral symmetry based pose normalization framework applied to livestock body measurement in point clouds. Comput. Electron. Agric.
**2019**, 160, 59–70. [Google Scholar] [CrossRef] - Rusu, R.B.; Cousins, S. 3D is here: Point Cloud Library (PCL). In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, 9–13 May 2011. [Google Scholar]
- Moreira, A.J.C.; Santos, M.Y. Concave hull: A k-nearest neighbours approach for the computation of the region occupied by a set of points. In GRAPP 2007, Proceedings of the Second International Conference on Computer Graphics Theory and Applications, Barcelona, Spain, 8–11 March 2007; Braz, J., Vázquez, P., Pereira, J.M., Eds.; Volume GM/R; INSTICC—Institute for Systems and Technologies of Information, Control and Communication: Lisboa, Portugal, 2007; pp. 61–68. [Google Scholar]
- Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv
**2020**, arXiv:2004.10934. [Google Scholar] - Papon, J.; Abramov, A.; Schoeler, M.; Worgotter, F. Voxel cloud connectivity segmentation-supervoxels for point clouds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2027–2034. [Google Scholar]
- Wongsriworaphon, A.; Arnonkijpanich, B.; Pathumnakul, S. An approach based on digital image analysis to estimate the live weights of pigs in farm environments. Comput. Electron. Agric.
**2015**, 115, 26–33. [Google Scholar] [CrossRef]

**Figure 3.**A visual diagram of the processing pipeline for curve skeleton extraction: (

**a**) Contour construction, (

**b**) Skeleton extraction and division, (

**c**) Leg skeleton calculation, (

**d**) Torso skeleton calculation and (

**e**) Final skeleton.

**Figure 4.**The visualization of a contour and a constructed 2D shape are depicted in (

**a**,

**b**). In figure (

**a**), the contour points calculated by the concave hull algorithm are labelled in red. The skeleton extracted from the contour of a side view is shown in (

**c**). The three branches are shown in red, green, and blue. The centre of the skeleton is labelled in purple, and the endpoints of the skeleton are labelled in orange.

**Figure 5.**Virtual image (

**a**) and labelled areas of three body parts (

**b**). The areas of the body, foreleg, and hind leg are labelled in blue, red, and green, respectively.

**Figure 6.**The leg branches moved to the border of the body (

**a**) and the banded shape point set (

**b**). The skeleton points are labelled in green and the banded shape point set is labelled in red.

**Figure 7.**The constructed concave hull and the extracted torso skeleton for the top view of the livestock.

**Figure 9.**Detection errors and connection errors that occurred in the three methods. The results extracted by ${L}_{1}$-median method, Point2skeleton method and our method are listed in the first row (

**a**–

**c**), second row (

**d**–

**f**), and third row (

**g**–

**i**), respectively. Detection errors are labeled in red. Connection errors are labeled in blue.

**Figure 10.**Visualization of a skeleton of a live pig effectively extracted by our method and another extracted by the $L1$-median method. The pictures in the first line show the skeleton extracted by our method, and the pictures in the second line show the skeleton extracted by $L1$-median.

**Figure 12.**Curve skeletons of different species extracted by the three methods. The results of $Point2skeleton$, ${L}_{1}$-median, and our method are displayed in the first, second, and third columns, respectively.

**Table 1.**The parameters discussed in Section 2.3. r is the resolution of the input point cloud. ${x}_{r}$ is the range of the x-coordinate value of the livestock data. ${d}_{s}$ is the maximum segment of the concave hull. ${d}_{t}$ and ${n}_{b}$ denote the stopping threshold and the number of branches of DSE, respectively. ${d}_{f}$ is the leaf size of the octree. ${d}_{l}$ is the distance threshold of the RANSAC algorithm for extracting the leg branch. ${d}_{sr}$ is the seed resolution of the supervoxel algorithm. a, b, and c are weight values of distance $d({p}_{i},{C}_{j})$. ${d}^{*}$ is the distance of the leg branches moving inward.

Parameter | ${\mathit{d}}_{\mathit{s}}$ | ${\mathit{d}}_{\mathit{t}}$ | ${\mathit{n}}_{\mathit{b}}$ | ${\mathit{d}}_{\mathit{f}}$ | ${\mathit{d}}_{\mathit{l}}$ | ${\mathit{d}}_{\mathit{s}\mathit{r}}$ | a | b | c | ${\mathit{d}}^{*}$ |
---|---|---|---|---|---|---|---|---|---|---|

Value | $16r\ast {x}_{r}$ | $0.003$ | 5 | $6r$ | $10r$ | $0.5$ | 1 | $1.5$ | $0.5$ | $4r$ |

Parameters | Hippo | Water Buffalo | Cow | Rhino | Horse | Cattle |
---|---|---|---|---|---|---|

${n}_{b}$ | 5 | 6 | 5 | 5 | 5 | 5 |

${d}_{t}$ | 0.003 | 0.001 | 0.001 | 0.08 | 0.003 | 0.003 |

${d}_{s}$ | 0.07 | 0.07 | 0.08 | 0.08 | 0.2 | 0.06 |

**Table 3.**Results of the comparison between the ${L}_{1}$-median skeleton approach and our curve skeleton extraction method. The percentage represents the percentage of data with errors out of the total data. ANE stands for the average number of errors per erroneous data point.

Method | Detection Error | Connection Error | ||
---|---|---|---|---|

Percentage (%) | ANE | Percentage (%) | ANE | |

$Point2skeleton\left(40\right)$ | 0 | 0 | 52.5 | 2.33 |

${L}_{1}$-median | 20 | 1.3 | 74 | 1.61 |

$PBCS$–S | 8.5 | 1.24 | 11 | 1.14 |

$PBCS$–D | 10.5 | 1.1 | 2.5 | 1.2 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hu, Y.; Luo, X.; Gao, Z.; Du, A.; Guo, H.; Ruchay, A.; Marinello, F.; Pezzuolo, A.
Curve Skeleton Extraction from Incomplete Point Clouds of Livestock and Its Application in Posture Evaluation. *Agriculture* **2022**, *12*, 998.
https://doi.org/10.3390/agriculture12070998

**AMA Style**

Hu Y, Luo X, Gao Z, Du A, Guo H, Ruchay A, Marinello F, Pezzuolo A.
Curve Skeleton Extraction from Incomplete Point Clouds of Livestock and Its Application in Posture Evaluation. *Agriculture*. 2022; 12(7):998.
https://doi.org/10.3390/agriculture12070998

**Chicago/Turabian Style**

Hu, Yihu, Xinying Luo, Zicheng Gao, Ao Du, Hao Guo, Alexey Ruchay, Francesco Marinello, and Andrea Pezzuolo.
2022. "Curve Skeleton Extraction from Incomplete Point Clouds of Livestock and Its Application in Posture Evaluation" *Agriculture* 12, no. 7: 998.
https://doi.org/10.3390/agriculture12070998