Influence of 3D Centro-Symmetry on a 2D Retinal Image

: An object is 3D centro-symmetrical if the object can be segmented into two halves and the relationship between them can be represented by a combination of reflection about a plane and a rotation through 180° about an axis that is normal to the plane. A 2D orthographic image of the 3D centro-symmetrical object is always 2D rotation-symmetrical. Note that the human visual system is known to be sensitive to 2D rotational symmetry. This human sensitivity to 2D rotational symmetry might also be used to detect 3D centro-symmetry. If it is, can this detection of 3D centro-symmetry be helpful for the perception of 3D? In this study, the geometrical properties of 3D centro-symmetry and its 2D orthographic and perspective projections were examined to find out whether 3D centro-symmetry plays any role in the perception of 3D. I found that, from a theoretical point-of-view, it is unlikely that 3D centro-symmetry can be used by the human visual system to organize a 2D image of an object in a way that makes it possible to recover the 3D shape of an object from its 2D image.

Symmetry can be defined as an invariant under a group of transformations [13][14][15]. For example, 2D mirror symmetry is invariant under reflection, and 2D N-fold rotational symmetry is invariant under rotation for 360/N° where N is an integer larger than 1. These two types of 2D symmetry are unique, because only they can form a compact 2D figure. A mirror-symmetrical figure can be segmented into two halves by a line, called the "symmetry-axis" that makes the halves reflections of one another. An N-fold rotation-symmetrical figure can be segmented into N parts in which any part is always a rotation of another part for 360/N°. The human visual system is sensitive to both 2D mirror and 2D rotational symmetry [16][17][18][19][20]. This high sensitivity to 2D mirror symmetry is often explained by the 3D mirror symmetry of many objects in our everyday life (e.g., [21]). However, a 2D retinal image of a 3D mirror-symmetrical object is 2D mirror-symmetrical on a spherical retina only if the symmetry plane of the object passes the center of projection of the eye under a perspective projection. Such a view of this kind of a mirrorsymmetrical object is degenerate. This sensitivity to 2D mirror symmetry can be explained by its property as a model-based invariant of a 3D shape [22,23]. The 2D image of a surface-of-revolution in a 3D scene is always mirror-symmetrical under an orthographic projection to an image plane and under a perspective projection to a spherical retina [23][24][25][26].
Two-dimensional rotational symmetry can be also regarded as a model-based invariant of a 3D shape. A boundary contour of a 2D orthographic projection of a 3D centro-symmetrical shape ( Figure  1) is always 2D rotational-symmetrical with 2 folds (Figure 2). A centro-symmetrical shape has a central point. Line segments connecting symmetrical pairs of points of the shape intersect with one another at the central point and are bisected by the point. In this study, this point is referred to as the "symmetry point" and line segments connecting the individual symmetrical pair of points are referred to as the "symmetry line segments". A centro-symmetrical shape can be segmented into two symmetrical halves by a plane with an arbitrary orientation that passes the symmetry point. The relation between these two halves can be represented by a combination of reflection about the plane and rotation for 180° about an axis that is normal to the plane and that passes the symmetry point.   (Figure 1c) with its surface transparent. The 2D image of the 3D centro-symmetrical shape itself is 2D rotational symmetrical with 2 folds, once we ignore the occlusion by the opaque surface of the shape.
Three-dimensional centro-symmetry has been studied in crystallography but not in vision science, despite its potential relevance for understanding human beings' sensitivity to 2D rotational symmetry. In this study, the geometrical properties of 3D centro-symmetry and its 2D orthographic and perspective projections were studied to make it possible to discuss (i) the potential sensitivity of the human visual system to 3D centro-symmetry and (ii) whether this kind of sensitivity can actually aid the perception of 3D.

A 2D Orthographic Projection of 3D Centro-Symmetry
Consider a 3D centro-symmetrical object with an arbitrary orientation in a 3D scene. Set the 3D Cartesian coordinate system of the scene so that the origin of the 3D system is at the symmetry point of the object. Now consider a symmetrical pair of points ] of the object. Their relation can be represented as = − , because a midpoint = ( + ) 2 ⁄ between P i and Q i is at the symmetry point = [0 0 0] . Next, consider a 2D orthographic projection of the 3D centro-symmetrical object. The image plane for the orthographic projection is set so that it passes the origin of the 3D coordinate system. The 2D Cartesian coordinate system is set so that the x-and y-axes of the 2D coordinate system coincide with the X-and Y-axes of the 3D coordinate system of the scene. The Z-axis of the 3D coordinate system is normal to the image plane.
The 2D orthographic projection of the 3D centro-symmetrical object is always 2D rotationsymmetrical with 2-folds. The orthographic projections of P i and Q i can be written as: where p i is the projection of P i and qi is the projection of Qi. The relation between pi and qi can be represented as = − . It is equivalent with a 2D rotation for 180° around = [0 0] to which the symmetry point O3D is projected. This implies that a 2D orthographic image of a 3D centrosymmetrical object is always 2D 2-fold rotation-symmetrical around a projection of the symmetry center of the object. This means that the image is invariant when it rotates 180° around the projection of the symmetry center.

Recovering a 3D Centro-Symmetrical Shape from Its 2D Orthographic Image
It has been shown that a one-parameter family of 3D shapes can be recovered from a single 2D orthographic image of an object if the object is 3D mirror-or rotation-symmetrical [27][28][29]. Starting with an image of an object, another image of the object can be generated, virtually, from a different viewpoint by using the symmetry of the object ( Figure 3). A point in the object can be projected to both the original and a virtual image. The 3D position of such a point can be computed by finding the intersection of two lines of projection of the point from the original and virtual images. The parameter of this family represents the relative orientation between the two views of the original and virtual images [25,30,31]. The veridical 3D shape of the object is included in this family. It is also possible to generate a virtual image of a 3D centro-symmetrical object from its 2D orthographic image, but, here, the virtual image based on 3D centro-symmetry does not facilitate the recovery of the 3D shape of the object. The viewing direction of the virtual image is opposite to the viewing direction of the original image. This means that that the two lines of projection to each point of the object from the two views of the images coincide perfectly with one another.
Note that a 2D orthographic image of a 3D centro-symmetrical object is always 2D 2-fold rotation-symmetrical. Under an orthographic projection, any 2D 2-fold rotation-symmetrical image has an infinite number of 3D centro-symmetrical interpretations. Now, set the 2D Cartesian coordinate system of the image so that its origin is at the symmetry point of the rotation-symmetrical image. Next, set the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation is constructed so that the X-and Y-axes of the 3D coordinate system coincide with the x-and y-axes of the 2D coordinate system of the image. The Z-axis of the 3D coordinate system is normal to the image plane. The relation between a 2D rotation-symmetrical pair of points ] in the image can be represented by a rotation for 180° around = [0 0] : = − . The 3D centro-symmetrical interpretations of bi and di can be constructed as: where Bi is the 3D interpretation of bi, Di is the 3D interpretation of di, and Zi is an arbitrary real number. The midpoint between Bi and Di is at = [0 0 0] . This construction process can be applied to all 2D rotation-symmetrical pairs of points in the image. For each pair of points, the Z coordinates of their constructed 3D points can be set independently from the other pairs of points. When this is done, the set of pairs of constructed 3D points is 3D centro-symmetrical about O3D. Each pair of the constructed points can be connected by a symmetry line segment that is bisected by O3D.

A 2D Perspective Projection of 3D Centro-Symmetry
A 2D perspective projection of a 3D centro-symmetrical object is not 2D rotation-symmetrical unless the object is planar and frontoparallel. Symmetry line segments connecting symmetrical pairs of points in the objects are projected to line segments that intersect with one another at a point to which the symmetry center of the object is projected. The projection of the symmetry point is not at the midpoints of the projection of the symmetry line segments unless the symmetry line segments are frontoparallel.
A model-based invariant of 3D centro-symmetry under a 2D perspective projection is only a common intersection of line segments that connect corresponding pairs of points. This can be proven by showing that a 3D centro-symmetrical interpretation can always be constructed from a 2D image that satisfies this constraint (see also [25,[32][33][34] for analogous theorems of 3D mirror and rotational symmetry). A common intersection in a 2D image is a projection of the symmetry point of the constructed 3D centro-symmetrical interpretation and its symmetry line segments are projected to the line segments that connect the corresponding pairs of points in a 2D image.
Setting the 2D Cartesian coordinate system of a 2D image and the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation can be constructed as follows: (i) the origin of the 3D coordinate system is at the center of the projection, (ii) the Z-axis of the 3D coordinate system coincides with the principal axis and it is perpendicular to the image plane Z = f , where f is the focal distance, (iii) the Z-axis passes the origin of the 2D coordinate system, and (iv) the X-and Y-axes of the 3D coordinate system are parallel to the x-and y-axes of the 2D coordinate system. Now, consider constructing a 3D centro-symmetrical interpretation from a given 2D image that is composed of corresponding pairs of points. Line segments connecting the individual pairs of points in the 2D image pass a point The point cs appears between points of these individual pairs. These line segments and cs are projections of the symmetry line segments and of the symmetry point of the 3D interpretation. Next, consider one of the corresponding pairs (pi, qi). A line segment connecting pi and qi passes cs. This pair of points can be written as: where θi is the orientation of the line segment connecting pi and qi, and and are the distance of pi and qi from cs. Note that the line segment connecting pi and qi is a projection of a symmetry line segment, and cs is a projection of the symmetry point. The symmetry line segment is parallel to a vector Si connecting the center of projection where and | | is the distance of vi from cs. The endpoints of the symmetry line segment and the symmetry point bisecting the symmetry line segment are projected to pi, qi, and cs. Then, the relation between pi, qi, cs, and vi can be represented by the following equation (see [25,35]): From Equation (5): The symmetry line segment is parallel to a vector Si connecting the center of projection [0 0 0] t and its vanishing point vi in the 2D image: Let the 3D centro-symmetrical construction of pi and qi be Pi and Qi and the symmetry point be Cs: where Δs is a free parameter representing the distance of Cs from the center of projection. Note that Pi and Qi in the 3D scene are connected by the symmetry line segment that is bisected by Cs: where |Δi| is the length of the symmetry line segment. From Equations (3) and (7) From Equations (6) and (10), Δi can be computed as: The free parameter Δs is the distance of the symmetry point Cs from the center of projection. It determines the size of the constructed 3D centro-symmetrical interpretation. This construction process can be applied to all of the corresponding pairs of points in the 2D image. Δs should be the same among all of the pairs of points. When this has been done, the set of pairs of constructed 3D points is 3D centro-symmetrical about Cs. Each pair of constructed points can be connected by a symmetry line segment that is bisected by Cs.

Model-Based Invariant of 3D Centro-Symmetry with Planar Contours
Consider a 3D mirror-symmetrical pair of curves. It has been suggested that the planarity of these individual contours can play an important role when the human visual system detects the 3D mirror symmetry from the 2D image of its contours. This can be explained by introducing a modelbased invariant into the 2D image by using 3D mirror symmetry along with the planarity of contours [32,33,36]. Under an orthographic projection, the relationship between the images of the curves can be represented by a sub-group of the 2D affine transformation. Under a perspective projection, the relationship can also be represented by a transformation that includes the same sub-group of the 2D affine transformation.
An analogous model-based invariant can be introduced into the 2D image by using 3D rotational symmetry along with the planarity of contours [25]. Under an orthographic projection, the relationship between the images of planar curves that are 3D rotation-symmetrical to one another can be represented by a sub-group of the 2D affine transformation. Under a perspective projection, the relationship can also be represented by a transformation that includes the same sub-group of the 2D affine transformation. Note that the sub-group of the 2D affine transformation for 3D rotational symmetry is different from the sub-group for 3D mirror symmetry. In the next two sections, the same approach to 3D centro-symmetry will be discussed under both orthographic and perspective projections.

Model-Based Invariant of 3D Centro-Symmetry with Planar Contours under a 2D Orthographic Projection
Under an orthographic projection, the planarity of contours does not play any role. Note that a 2D orthographic image of a 3D centro-symmetrical object is always 2D rotational symmetry and that a 3D centro-symmetrical interpretation can always be constructed from a 2D rotation-symmetrical image (see Section 2.1. A 2D orthographic projection of 3D centro-symmetry). It is always possible to construct a 3D centro-symmetric pair of planar curves from a pair of 2D rotation-symmetrical curves with an orthographic projection.
Consider a pair of curves ϕ and ψ that are 2D rotation-symmetrical around the origin where NX, NY, NZ, and k Φ are free parameters. This construction process can be applied to all corresponding pairs of points on ϕ and ψ. NX, NY, NZ, and k Φ should be the same among all of the pairs of points. When this is done, a 3D centro-symmetrical pair of planar curves Φ and Ψ can be constructed from ϕ and ψ. Equation (12) represents a plane of Φ. The plane of Φ is parallel to a plane of Ψ.

Model-Based Invariant of 3D Centro-Symmetry with Planar Contours under a 2D Perspective Projection
Under a perspective projection, a model-based invariant can be introduced into the 2D image by 3D centro-symmetry used along with the planarity of contours. Now, consider a 3D centrosymmetrical pair of planar contours Φ and Ψ and their 2D perspective projections φ and ψ. Plane Π Φ of Φ and plane Π Ψ of Ψ are parallel to one another. Set the 2D Cartesian coordinate system of a 2D image and the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation will be constructed as follows: (i) the origin of the 3D coordinate system is at the center of projection, (ii) the Z-axis of the 3D coordinate system coincides with the principal axis and it is perpendicular to the image plane Z = f , where f is the focal distance, (iii) the Z-axis passes the origin of the 2D coordinate system, and (iv) the X-and Y-axes of the 3D coordinate system are parallel to the x-and y-axes of the 2D coordinate system. First, consider for simplicity, a case in which Π Φ and Π Ψ are frontoparallel: From Equation (14), the relation between pi and qi is: where = ⁄ [ ] is a perspective projection of CS. This shows that the relation between φ and ψ can be represented by a sub-group of the 2D affine transformation, namely scaling by a factor of −k Ψ /k Φ around cs.
If a pair of curves β and δ in the 2D image satisfy Equation (15), their 3D centro-symmetrical interpretations are a pair of planar contours in the 3D scene. Now, consider n corresponding pairs of points (b1, b2, … bn) on β and (d1, d2, … dn) on δ where n ≥ 3. They satisfy Equation (15): (bi -cβδ) = −kβδ(di -cβδ), where i is an integer (1 ≤ i ≤ n), kβδ is a constant, and cβδ is an intersection of line segments connecting the corresponding pairs of points between β and δ. From Equation (3), bi and di can be written as bi = cβδ + lbi[cosθi sinθi] t and di = cβδ + ldi[cosθi sinθi] t , where θi is the orientation of the line segment connecting bi and di, and | | and | | are the distance of bi and di from cβδ. Note that lbi/ldi is kβδ for all of the corresponding pairs of points, because β and δ satisfy Equation (15). Let Bi and Di be the 3D centro-symmetrical interpretations of bi and di and ZBi and DBi be their Z-coordinates. From Equations (7)-(11), the Z-coordinates of Bi and Di are: where Δs is a free parameter representing the distance of the symmetry point from the center of projection. It should be the same among all of the pairs of points. Note that ( + ) ( − ) ⁄ is also the same among all of the pairs of points, because lbi/ldi is constant (kβδ). Equation (16) shows that the 3D centro-symmetrical interpretations of β and δ are a pair of contours that are individually frontoparallel and planar.
Next, consider a general case in which planes Π Φ and Π Ψ are not frontoparallel: where k Φ , k Ψ , NX, NY, and NZ are constants, and N X 2 + N Y 2 = 1.  Figure 4). The intersection hij of these two lines is a vanishing point of two parallel lines connecting Pi and Pj and connecting Qi and Qj. These lines are on the planes Π Φ and Π Ψ individually, and they are parallel to one another. It follows that h ij is on a line H s (the solid red line in Figure 4) that is a common horizon of Π Φ and Π Ψ . Another vanishing point can be found by drawing a line connecting different points on φ and by drawing a line connecting their corresponding points on ψ. An additional corresponding pair of points on φ and ψ can be determined by drawing a line (the dotted green line in Figure 4) passing cs and finding its intersection with φ and ψ if necessary.
This vanishing point is also on H s . H s can be determined by finding a line that connects these vanishing points. By using the horizon H s of Π Φ and Π Ψ , a vanishing point of a line normal to Π Φ and Π Ψ can be found. Now, draw a line that passes the principal point [0 0] t in the image and that is perpendicular to H s . Let hv be an intersection of this line with H s . Next, find a point v Π on this line such that the visual angle between hv and v Π from the center of projection is perpendicular. The line connecting v Π and the center of projection is normal to Π Φ and Π Ψ and it is parallel to [NX NY NZ] t (Equation (17)). Now, rotate the principal axis and the image plane together in the 3D scene to make the principal axis normal to Π Φ and Π Ψ and Π Φ and Π Ψ become frontoparallel. This rotation is around a vector [NY − NX 0] t and the degree of the rotation is = cos + 1 ⁄ .
A matrix R Π representing this rotation can be formulated by using Rodrigues' rotation formula: where c θ = cosθ, s θ = sinθ, and r θ = 1−cosθ. After the rotation R Π , a point [x y] t in the original image is transformed to [x′ y′] t [37]: Note that Π Φ and Π Ψ become frontoparallel after R Π . With this done, the transformed image satisfies Equation (15). From Equations (15) and (19), the relation between φ and ψ is: where pi′, qi′, and cs′ are the transformations of pi, qi, and cs by using Equation (19). This shows that the relation between φ and ψ can be represented by the transformation of Equation (19) and by a subgroup of the 2D affine transformation that is scaling.

General Discussion
This study examined how, from a theoretical point of view, 3D centro-symmetry can be used to organize a 2D image of an object and to recover its 3D shape from the 2D image. My study showed that 3D centro-symmetry has the following properties: (i) a 2D orthographic image of a 3D centrosymmetrical shape is always 2D rotation-symmetrical, (ii) a 3D centro-symmetrical shape can be recovered from its single 2D perspective projection but not from its 2D orthographic projection, (iii) any pair of 2D curves is consistent with a 3D centro-symmetrical interpretation under a perspective projection, and (iv) additional model-based invariants of 3D rotational symmetry can be introduced under a perspective projection if the 3D centro-symmetrical set of curves are planar, individually.
This study was motivated by the fact that the human visual system is known to be sensitive to 2D rotational symmetry, which is a model-based invariant of 3D centro-symmetry under an orthographic projection but not under a perspective projection. It raises two questions: i) which 3D centro-symmetry or 3D rotational symmetry is actually inferred from 2D rotational symmetry in the human visual system and ii) can any sensitivity to 3D centro-symmetry be helpful for the perception of 3D? The first question can be addressed by testing the human sensitivity to 2D rotationsymmetrical patterns with a 3D centro-symmetrical or with a 3D rotation-symmetrical depth distribution (see Figure 5, see also [38][39][40][41] for studies of the perception of 2D mirror symmetry from a dotted pattern with a depth distribution). Note, however, even if the human visual system is sensitive to 3D centro-symmetry, it is unclear how 3D centro-symmetry can be of any help in the perception of 3D. Note that this study showed that 3D centro-symmetry can be theoretically helpful for the 3D perception with a perspective projection but not with an orthographic projection. This inconsistency is inconvenient for the human visual system, because it prevents it from making effective use of 3D centro-symmetry. When the retinal image of a 3D centro-symmetrical object is small, the image can be approximated well by an orthographic projection, and the human visual system can detect 3D centro-symmetry of the object on the basis of 2D rotational symmetry. However, here, the 3D centro-symmetry is not helpful in perceiving the 3D shape of the object, because the image is too small. When the retinal image of an object is sufficiently large, 3D centro-symmetry is helpful in perceiving the 3D shape of the object from this large image, but the 3D centro-symmetry cannot be detected on the basis of 2D rotational symmetry. Therefore, this theoretical study suggests that 3D centro-symmetry cannot be used by the human visual system to organize a 2D image of an object to make it possible to recover its 3D shape from the 2D image.