# A Framework for Symmetric Part Detection in Cluttered Scenes

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Representing Symmetric Parts

**Figure 1.**Our representation of symmetric parts. (

**a**) The shape of the runner’s body is transformed into its medial axis (red), a skeleton-like structure that decomposes the shape into branch-like segments, e.g., the leg. The leg’s shape is swept out by a sequence of discs (green) lying along the medial axis. (

**b**) The shape of the same leg is composed from superpixels that correspond to the sequence of discs. The scope of this article’s framework is limited to detecting symmetric parts corresponding to individual branches.

**Figure 2.**To compose a part’s shape from superpixels in a given input image (

**a**), we compute superpixels at multiple scales, for which two are shown (

**b**,

**c**). Superpixels from all scales are included in a single set of candidates, which allows the grouping algorithm to group superpixels from different scales into the same part. (

**d**) The sequence-finding algorithm finds the best sequence of superpixels, comprised of different scales.

## 4. Disc Affinity

#### 4.1. Shape Features

**Figure 3.**Improving invariance with a deformable ellipse: given two adjacent candidate discs, the first step is to fit the ellipse parameters to the region defined by their corresponding superpixels (

**a**). The top row shows invariance achieved with a standard ellipse. The ellipse’s fit is visualized with the major axis (

**b**), the region’s boundary edgels before (

**c**) and after (

**d**) warping out the unwanted variations and the resulting spatial histogram of gradient pixels (

**e**). See the text for details. The bottom row shows the corresponding steps (

**f**–

**i**) obtained by the deformable ellipse. Comparing the results, a visually more symmetric feature is obtained by the deformable ellipse, which fits tightly around the region’s boundary as compared with the standard ellipse.

#### 4.2. Appearance Features

## 5. Grouping Discs

#### 5.1. Agglomerative Clustering

**Figure 4.**In our approach, the (

**a**) input image of foreground leaves is oversegmented into superpixels and a (

**b**) weighted graph $\mathcal{G}$ is built that captures the pairwise affinities that are computed among the superpixels. A graph-based grouping algorithm takes as input the graph $\mathcal{G}$, which may contain false positive affinities between the leaves, as shown in (b). In this figure, we illustrate the relative advantage of (

**d**) sequence optimization over (

**c**) agglomerative clustering. In (c), merging the vertices in $\mathcal{G}$ results in a cluster that undersegments the leaves, combining them into a single symmetric part that violates the assumption that a part is composed of a linear sequence of discs. In (d), the branch constraint is built into the sequence-finding algorithm, which prevents symmetric parts from having tree-structured discs and correctly segments the leaves into two distinct parts.

#### 5.2. Sequence Optimization by Dynamic Programming

**Figure 5.**Grouping by dynamic programming: the iterative step of the algorithm grows sequences by extracting a sequence ${D}^{*}$ from the priority queue and returning longer sequences to the queue obtained by extending the end of ${D}^{*}$ with adjacent discs. See the text for details.

## 6. Results

**Figure 6.**Multiple symmetric parts: for each image (

**a**,

**b**), we show the top 15 masks detected as symmetric parts. Each mask is detected as a sequence of discs, whose centres are plotted in green and connected by a sequence of red line segments that represent the medial axis.

#### 6.1. Qualitative Results

#### 6.2. Quantitative Results

**Figure 7.**Example detections on a sample of images from Berkeley Segmentation Database (BSD)-Parts. Columns left to right: input image, ground-truth masks, top four detection masks. Note that many images have more ground-truth masks than detections that can be shown here.

**Figure 8.**Performance curves corresponding to different settings of the components of our approach on (

**a**) BSD-Parts and (

**b**) Weizmann Horse Database (WHD). See the text for details.

## 7. Conclusions

## Acknowledgements

## Conflicts of Interest

## References

- Lee, T.; Fidler, S.; Dickinson, S. Detecting Curved Symmetric Parts using a Deformable Disc Model. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), Sydney, Australia, 1–8 December 2013.
- Levinshtein, A.; Dickinson, S.; Sminchisescu, C. Multiscale Symmetric Part Detection and Grouping. Int. Conf. Comput. Vis. 2009. Available online: http://www.maths.lth.se/matematiklth/personal/sminchis/papers/LevSminIccv2009.pdf (accessed on 20 July 2015).
- Levinshtein, A.; Sminchisescu, C.; Dickinson, S. Multiscale Symmetric Part Detection and Grouping. Int. J. Comput. Vis.
**2013**, 104, 117–134. [Google Scholar] [CrossRef] - Leyton, M. Symmetry, Causality, Mind; MIT Press: Cambridge, MI, USA, 1992. [Google Scholar]
- Wertheimer, M. Laws of organization in perceptual forms. In Source Book of Gestalt Psychology; Harcourt, Brace & World: New York, NY, USA, 1938. [Google Scholar]
- Tyler, C. Human Symmetry Perception and Its Computational Analysis; Psychology Press: London, UK, 2002. [Google Scholar]
- Blum, H. A transformation for extracting new descriptors of shape. In Models for the Perception of Speech and Visual Form; MIT Press: Cambridge, MI, USA, 1967; Volume 19, pp. 362–380. [Google Scholar]
- Binford, T. Visual Perception by Computer. In Presented at the IEEE Conference on Systems and Control, Miami, FL, USA; 1971. [Google Scholar]
- Pentland, A. Perceptual organization and the representation of natural form. Artif. Intell.
**1986**, 28, 293–331. [Google Scholar] [CrossRef] - Biederman, I. Human image understanding: Recent research and a theory. Comput. Vis. Graph. Image Proc.
**1985**, 32, 29–73. [Google Scholar] [CrossRef] - Siddiqi, K.; Shokoufandeh, A.; Dickinson, S.; Zucker, S. Shock graphs and shape matching. Int. J. Comput. Vis.
**1999**, 35, 13–32. [Google Scholar] [CrossRef] - Sebastian, T.; Klein, P.; Kimia, B. Recognition of Shapes by Editing Their Shock Graphs. IEEE Trans. Pattern Anal. Machine Intell.
**2004**, 26, 550–571. [Google Scholar] [CrossRef] [PubMed] - Siddiqi, K.; Pizer, S. Medial Epresentations: Mathematics, Algorithms and Applications; Springer Verlag: Berlin, Germany, 2008. [Google Scholar]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. CVPR
**2014**. Available online: http://www.cs.berkeley.edu/rbg/papers/r-cnn-cvpr.pdf (accessed on 20 July 2015). - Dickinson, S. The Evolution of Object Categorization and the Challenge of Image Abstraction. In Object Categorization: Computer and Human Vision Perspectives; Cambridge University Press: Cambridge, UK, 2009; pp. 1–37. [Google Scholar]
- Dickinson, S.; Levinshtein, A.; Sala, P.; Sminchisescu, C. The Role of Mid-Level Shape Priors in Perceptual Grouping and Image Abstraction. In Shape Perception in Human and Computer Vision: An Interdisciplinary Perspective; Springer Verlag: Berlin, Germany, 2013; pp. 1–19. [Google Scholar]
- Carreira, J.; Sminchisescu, C. CPMC: Automatic object segmentation using constrained parametric min-cuts. IEEE Trans. Pattern Anal. Machine Intell.
**2012**, 34, 1312–1328. [Google Scholar] [CrossRef] [PubMed] - Uijlings, J.; van de Sande, K.; Gevers, T.; Smeulders, A. Selective search for object recognition. Int. J. Comput. Vis.
**2013**, 104, 154–171. [Google Scholar] [CrossRef] - Levinshtein, A.; Sminchisescu, C.; Dickinson, S. Optimal Image and Video Closure by Superpixel Grouping. Int. J. Comput. Vis.
**2012**, 100, 99–119. [Google Scholar] [CrossRef] - Ren, X.; Fowlkes, C.; Malik, J. Cue integration for figure/ground labeling. NIPS
**2005**, 1121–1128. [Google Scholar] - Lee, T.; Fidler, S.; Dickinson, S. Multi-cue mid-level grouping. In Proceedings of the 12th Asian Conference on Computer Vision, Singapore, 1–5 November 2014.
- Pelillo, M.; Siddiqi, K.; Zucker, S. Matching hierarchical structures using association graphs. IEEE Trans. Pattern Anal. Machine Intell.
**1999**, 21, 1105–1120. [Google Scholar] [CrossRef] - Demirci, F.; Shokoufandeh, A.; Dickinson, S. Skeletal shape abstraction from examples. IEEE Trans. Pattern Anal. Machine Intell.
**2009**, 31, 944–952. [Google Scholar] [CrossRef] [PubMed] - Macrini, D.; Dickinson, S.; Fleet, D.; Siddiqi, K. Object categorization using bone graphs. Comp. Vis. Image Underst.
**2011**, 115, 1187–1206. [Google Scholar] [CrossRef] - Macrini, D.; Dickinson, S.; Fleet, D.; Siddiqi, K. Bone graphs: Medial shape parsing and abstraction. Comp. Vis. Image Underst.
**2011**, 115, 1044–1061. [Google Scholar] [CrossRef] - Crowley, J.; Parker, A. A representation for shape based on peaks and ridges in the difference of low-pass transform. IEEE Trans. Pattern Anal. Machine Intell.
**1984**, 6, 156–170. [Google Scholar] [CrossRef] - Shokoufandeh, A.; Marsic, I.; Dickinson, S. View-based object recognition using saliency maps. Image Vis. Comput.
**1999**, 17, 445–460. [Google Scholar] [CrossRef] - Mikolajczyk, K.; Schmid, C. An affine invariant interest point detector. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, 28–31 May 2002.
- Lindeberg, T.; Bretzner, L. Real-time scale selection in hybrid multi-scale representations. In Proceedings of the 4th International Conference, Scale Space 2003, Isle of Skye, UK, 10–12 June 2003.
- Shokoufandeh, A.; Bretzner, L.; Macrini, D.; Demirci, M.F.; Jönsson, C.; Dickinson, S. The representation and matching of categorical shape. Comput. Vis. Image Underst.
**2006**, 103, 139–154. [Google Scholar] [CrossRef] - Tsogkas, S.; Kokkinos, I. Learning-Based Symmetry Detection in Natural Images. In Proceedings of the 12th European Conference on Computer Vision, Florence, Italy, 7–13 October 2012.
- Brady, M.; Asada, H. Smoothed local symmetries and their implementation. IJRR
**1984**, 3, 36–61. [Google Scholar] [CrossRef] - Connell, J.; Brady, M. Generating and Generalizing Models of Visual Objects. Artif. Intell.
**1987**, 31, 159–183. [Google Scholar] [CrossRef] - Ponce, J. On characterizing ribbons and finding skewed symmetries. CVGIP
**1990**, 52, 328–340. [Google Scholar] - Cham, T.J.; Cipolla, R. Symmetry detection through local skewed symmetries. Image Vis. Comput.
**1995**, 13, 439–450. [Google Scholar] [CrossRef] - Cham, T.; Cipolla, R. Geometric saliency of curve correspondences and grouping of symmetric contours. In Proceedings of the 4th European Conference on Computer Vision, Cambridge, UK, 15–18 April 1996.
- Saint-Marc, P.; Rom, H.; Medioni, G. B-spline contour representation and symmetry detection. IEEE Trans. Pattern Anal. Machine Intell.
**1993**, 15, 1191–1197. [Google Scholar] [CrossRef] - Liu, T.; Geiger, D.; Yuille, A. Segmenting by seeking the symmetry axis. In Proceedings of the Fourteenth International Conference on Pattern Recognition, Brisbane, Australia, 16–20 August 1998.
- Ylä-Jääski, A.; Ade, F. Grouping symmetrical structures for object segmentation and description. Comput. Vis. Image Underst.
**1996**, 63, 399–417. [Google Scholar] [CrossRef] - Stahl, J.; Wang, S. Globally optimal grouping for symmetric closed boundaries by combining boundary and region information. IEEE Trans. Pattern Anal. Machine Intell.
**2008**, 30, 395–411. [Google Scholar] [CrossRef] [PubMed] - Fidler, S.; Boben, M.; Leonardis, A. Learning a Hierarchical Compositional Shape Vocabulary for Multi-class Object Representation. 2014. arXiv:1408.5516. Available online: http://arxiv.org/abs/1408.5516 (accessed on 13 July 2015).
- Lowe, D. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis.
**2004**, 60, 91–110. [Google Scholar] [CrossRef] - Loy, G.; Eklundh, J. Detecting symmetry and symmetric constellations of features. In Proceedings of the 9th European Conference on Computer Vision, Graz, Australia, 7–13 May 2006.
- Lee, S.; Liu, Y. Curved glide-reflection symmetry detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2012**, 34, 266–278. [Google Scholar] [PubMed] - Narayanan, M.; Kimia, B. Bottom-Up Perceptual Organization of Images into Object Part Hypotheses. In Proceedings of the 12th European Conference on Computer Vision, Florence, Italy, 7–13 October 2012.
- Felzenswalb, P.; Huttenlocher, D. Efficient graph-based image segmentation. Int. J. Comput. Vis.
**2004**, 59, 167–181. [Google Scholar] [CrossRef] - Felzenszwalb, P.; McAllester, D. A min-cover approach for finding salient curves. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, 2006 (CVPRW ’06), New York, NY, USA, 17–22 June 2006. [CrossRef]
- Shi, J.; Malik, J. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Machine Intell.
**2000**, 22, 888–905. [Google Scholar] - Borenstein, E.; Ullman, S. Class-specific, top-down segmentation. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, 28–31 May 2002.
- Martin, D.; Fowlkes, C.; Tal, D.; Malik, J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the Eighth IEEE International Conference on Computer Vision (ICCV 2001), Vancouver, BC, Canada, 7–14 July 2001.

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lee, T.; Fidler, S.; Levinshtein, A.; Sminchisescu, C.; Dickinson, S.
A Framework for Symmetric Part Detection in Cluttered Scenes. *Symmetry* **2015**, *7*, 1333-1351.
https://doi.org/10.3390/sym7031333

**AMA Style**

Lee T, Fidler S, Levinshtein A, Sminchisescu C, Dickinson S.
A Framework for Symmetric Part Detection in Cluttered Scenes. *Symmetry*. 2015; 7(3):1333-1351.
https://doi.org/10.3390/sym7031333

**Chicago/Turabian Style**

Lee, Tom, Sanja Fidler, Alex Levinshtein, Cristian Sminchisescu, and Sven Dickinson.
2015. "A Framework for Symmetric Part Detection in Cluttered Scenes" *Symmetry* 7, no. 3: 1333-1351.
https://doi.org/10.3390/sym7031333