Influence of 3D Centro-Symmetry on a 2D Retinal Image

Abstract

An object is 3D centro-symmetrical if the object can be segmented into two halves and the relationship between them can be represented by a combination of reflection about a plane and a rotation through 180° about an axis that is normal to the plane. A 2D orthographic image of the 3D centro-symmetrical object is always 2D rotation-symmetrical. Note that the human visual system is known to be sensitive to 2D rotational symmetry. This human sensitivity to 2D rotational symmetry might also be used to detect 3D centro-symmetry. If it is, can this detection of 3D centro-symmetry be helpful for the perception of 3D? In this study, the geometrical properties of 3D centro-symmetry and its 2D orthographic and perspective projections were examined to find out whether 3D centro-symmetry plays any role in the perception of 3D. I found that, from a theoretical point-of-view, it is unlikely that 3D centro-symmetry can be used by the human visual system to organize a 2D image of an object in a way that makes it possible to recover the 3D shape of an object from its 2D image.

1. Introduction

The human visual system is particularly sensitive to a number of 2D configurations that are composed of local image features in a retinal image [1,2,3,4,5,6,7,8]. Examples of such 2D image features are parallelism, collinearity, perpendicularity, and symmetry. Two-dimensional symmetry is one of the most important of these configurations (see [9,10,11,12] for reviews).

Symmetry can be defined as an invariant under a group of transformations [13,14,15]. For example, 2D mirror symmetry is invariant under reflection, and 2D N-fold rotational symmetry is invariant under rotation for 360/N° where N is an integer larger than 1. These two types of 2D symmetry are unique, because only they can form a compact 2D figure. A mirror-symmetrical figure can be segmented into two halves by a line, called the “symmetry-axis” that makes the halves reflections of one another. An N-fold rotation-symmetrical figure can be segmented into N parts in which any part is always a rotation of another part for 360/N°.

The human visual system is sensitive to both 2D mirror and 2D rotational symmetry [16,17,18,19,20]. This high sensitivity to 2D mirror symmetry is often explained by the 3D mirror symmetry of many objects in our everyday life (e.g., [21]). However, a 2D retinal image of a 3D mirror-symmetrical object is 2D mirror-symmetrical on a spherical retina only if the symmetry plane of the object passes the center of projection of the eye under a perspective projection. Such a view of this kind of a mirror-symmetrical object is degenerate. This sensitivity to 2D mirror symmetry can be explained by its property as a model-based invariant of a 3D shape [22,23]. The 2D image of a surface-of-revolution in a 3D scene is always mirror-symmetrical under an orthographic projection to an image plane and under a perspective projection to a spherical retina [23,24,25,26].

Two-dimensional rotational symmetry can be also regarded as a model-based invariant of a 3D shape. A boundary contour of a 2D orthographic projection of a 3D centro-symmetrical shape (Figure 1) is always 2D rotational-symmetrical with 2 folds (Figure 2). A centro-symmetrical shape has a central point. Line segments connecting symmetrical pairs of points of the shape intersect with one another at the central point and are bisected by the point. In this study, this point is referred to as the “symmetry point” and line segments connecting the individual symmetrical pair of points are referred to as the “symmetry line segments”. A centro-symmetrical shape can be segmented into two symmetrical halves by a plane with an arbitrary orientation that passes the symmetry point. The relation between these two halves can be represented by a combination of reflection about the plane and rotation for 180° about an axis that is normal to the plane and that passes the symmetry point.

Three-dimensional centro-symmetry has been studied in crystallography but not in vision science, despite its potential relevance for understanding human beings’ sensitivity to 2D rotational symmetry. In this study, the geometrical properties of 3D centro-symmetry and its 2D orthographic and perspective projections were studied to make it possible to discuss (i) the potential sensitivity of the human visual system to 3D centro-symmetry and (ii) whether this kind of sensitivity can actually aid the perception of 3D.

2. Theorems and Proofs

2.1. A 2D Orthographic Projection of 3D Centro-Symmetry

Consider a 3D centro-symmetrical object with an arbitrary orientation in a 3D scene. Set the 3D Cartesian coordinate system of the scene so that the origin of the 3D system is at the symmetry point of the object. Now consider a symmetrical pair of points $P_i={\left[\begin{array}{ccc}{X}_{Pi}& Y_{Pi}& Z_{Pi}\end{array}\right]}^t$ and $Q_i={\left[\begin{array}{ccc}{X}_{Qi}& Y_{Qi}& Z_{Qi}\end{array}\right]}^t$ of the object. Their relation can be represented as $P_i=-Q_i$, because a midpoint $M_i=\left(P_i+Q_i\right)/2$ between P_i and Q_i is at the symmetry point $O_{3D}={\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^t$.

Next, consider a 2D orthographic projection of the 3D centro-symmetrical object. The image plane for the orthographic projection is set so that it passes the origin of the 3D coordinate system. The 2D Cartesian coordinate system is set so that the x- and y-axes of the 2D coordinate system coincide with the X- and Y-axes of the 3D coordinate system of the scene. The Z-axis of the 3D coordinate system is normal to the image plane.

The 2D orthographic projection of the 3D centro-symmetrical object is always 2D rotation-symmetrical with 2-folds. The orthographic projections of P_i and Q_i can be written as:

$$\begin{array}{c}{p}_i=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\end{array}\right]P_i={\left[\begin{array}{cc}{X}_{Pi}& Y_{Pi}\end{array}\right]}^t\\ q_i=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\end{array}\right]Q_i={\left[\begin{array}{cc}{X}_{Qi}& Y_{Qi}\end{array}\right]}^t={\left[\begin{array}{cc}-X_{Pi}& -Y_{Pi}\end{array}\right]}^t\end{array}$$

(1)

where p_i is the projection of P_i and q_i is the projection of Q_i. The relation between p_i and q_i can be represented as $p_i=-q_i$. It is equivalent with a 2D rotation for 180° around $O_{2D}={\left[\begin{array}{cc}0& 0\end{array}\right]}^t$ to which the symmetry point O_3D is projected. This implies that a 2D orthographic image of a 3D centro-symmetrical object is always 2D 2-fold rotation-symmetrical around a projection of the symmetry center of the object. This means that the image is invariant when it rotates 180° around the projection of the symmetry center.

2.2. Recovering a 3D Centro-Symmetrical Shape from Its 2D Orthographic Image

It has been shown that a one-parameter family of 3D shapes can be recovered from a single 2D orthographic image of an object if the object is 3D mirror- or rotation-symmetrical [27,28,29]. Starting with an image of an object, another image of the object can be generated, virtually, from a different viewpoint by using the symmetry of the object (Figure 3). A point in the object can be projected to both the original and a virtual image. The 3D position of such a point can be computed by finding the intersection of two lines of projection of the point from the original and virtual images. The parameter of this family represents the relative orientation between the two views of the original and virtual images [25,30,31]. The veridical 3D shape of the object is included in this family.

It is also possible to generate a virtual image of a 3D centro-symmetrical object from its 2D orthographic image, but, here, the virtual image based on 3D centro-symmetry does not facilitate the recovery of the 3D shape of the object. The viewing direction of the virtual image is opposite to the viewing direction of the original image. This means that that the two lines of projection to each point of the object from the two views of the images coincide perfectly with one another.

Note that a 2D orthographic image of a 3D centro-symmetrical object is always 2D 2-fold rotation-symmetrical. Under an orthographic projection, any 2D 2-fold rotation-symmetrical image has an infinite number of 3D centro-symmetrical interpretations. Now, set the 2D Cartesian coordinate system of the image so that its origin is at the symmetry point of the rotation-symmetrical image. Next, set the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation is constructed so that the X- and Y-axes of the 3D coordinate system coincide with the x- and y-axes of the 2D coordinate system of the image. The Z-axis of the 3D coordinate system is normal to the image plane. The relation between a 2D rotation-symmetrical pair of points $b_i={\left[\begin{array}{cc}{x}_{bi}& y_{bi}\end{array}\right]}^t$ and $d_i={\left[\begin{array}{cc}{x}_{di}& y_{di}\end{array}\right]}^t$ in the image can be represented by a rotation for 180° around $O_{2D}={\left[\begin{array}{cc}0& 0\end{array}\right]}^t$: $b_i=-d_i$. The 3D centro-symmetrical interpretations of b_i and d_i can be constructed as:

$$\begin{array}{c}{B}_i={\left[\begin{array}{ccc}{x}_{bi}& y_{bi}& Z_i\end{array}\right]}^t\\ D_i={\left[\begin{array}{ccc}{x}_{di}& y_{di}& -Z_i\end{array}\right]}^t={\left[\begin{array}{ccc}-x_{bi}& -y_{bi}& -Z_i\end{array}\right]}^t=-B_i\end{array}$$

(2)

where B_i is the 3D interpretation of b_i, D_i is the 3D interpretation of d_i, and Z_i is an arbitrary real number. The midpoint between B_i and D_i is at $O_{3D}={\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^t$. This construction process can be applied to all 2D rotation-symmetrical pairs of points in the image. For each pair of points, the Z coordinates of their constructed 3D points can be set independently from the other pairs of points. When this is done, the set of pairs of constructed 3D points is 3D centro-symmetrical about O_3D. Each pair of the constructed points can be connected by a symmetry line segment that is bisected by O_3D.

2.3. A 2D Perspective Projection of 3D Centro-Symmetry

A 2D perspective projection of a 3D centro-symmetrical object is not 2D rotation-symmetrical unless the object is planar and frontoparallel. Symmetry line segments connecting symmetrical pairs of points in the objects are projected to line segments that intersect with one another at a point to which the symmetry center of the object is projected. The projection of the symmetry point is not at the midpoints of the projection of the symmetry line segments unless the symmetry line segments are frontoparallel.

A model-based invariant of 3D centro-symmetry under a 2D perspective projection is only a common intersection of line segments that connect corresponding pairs of points. This can be proven by showing that a 3D centro-symmetrical interpretation can always be constructed from a 2D image that satisfies this constraint (see also [25,32,33,34] for analogous theorems of 3D mirror and rotational symmetry). A common intersection in a 2D image is a projection of the symmetry point of the constructed 3D centro-symmetrical interpretation and its symmetry line segments are projected to the line segments that connect the corresponding pairs of points in a 2D image.

Setting the 2D Cartesian coordinate system of a 2D image and the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation can be constructed as follows: (i) the origin of the 3D coordinate system is at the center of the projection, (ii) the Z-axis of the 3D coordinate system coincides with the principal axis and it is perpendicular to the image plane Z = f, where f is the focal distance, (iii) the Z-axis passes the origin of the 2D coordinate system, and (iv) the X- and Y-axes of the 3D coordinate system are parallel to the x- and y-axes of the 2D coordinate system.

Now, consider constructing a 3D centro-symmetrical interpretation from a given 2D image that is composed of corresponding pairs of points. Line segments connecting the individual pairs of points in the 2D image pass a point $c_s={\left[\begin{array}{cc}{x}_c& y_c\end{array}\right]}^t$. The point cs appears between points of these individual pairs. These line segments and cs are projections of the symmetry line segments and of the symmetry point of the 3D interpretation.

Next, consider one of the corresponding pairs (p_i, q_i). A line segment connecting p_i and q_i passes c_s. This pair of points can be written as:

$$\begin{array}{c}{p}_i=c_s+l_{pi}{\left[\begin{array}{cc}\cos {\theta }_i& \sin {\theta }_i\end{array}\right]}^t\\ q_i=c_s+l_{qi}{\left[\begin{array}{cc}\cos {\theta }_i& \sin {\theta }_i\end{array}\right]}^t\end{array}$$

(3)

where θ_i is the orientation of the line segment connecting p_i and q_i, and $\left|l_{pi}\right|$ and $\left|l_{qi}\right|$ are the distance of p_i and q_i from c_s. Note that the line segment connecting p_i and q_i is a projection of a symmetry line segment, and c_s is a projection of the symmetry point. The symmetry line segment is parallel to a vector S_i connecting the center of projection [0 0 0]^t and the vanishing point v_i of the symmetry line segment. The vanishing point v_i of the symmetry line segment should be collinear with p_i, q_i, and c_s, and it can be written as:

$$v_i=c_s+l_{vi}{\left[\begin{array}{cc}\cos {\theta }_i& \sin {\theta }_i\end{array}\right]}^t$$

(4)

where and $\left|l_{vi}\right|$ is the distance of v_i from c_s. The endpoints of the symmetry line segment and the symmetry point bisecting the symmetry line segment are projected to p_i, q_i, and c_s. Then, the relation between p_i, q_i, c_s, and v_i can be represented by the following equation (see [25,35]):

$$l_{vi}=\frac{2\left(l_{vi}-l_{pi}\right)\left(l_{vi}-l_{qi}\right)}{\left(l_{vi}-l_{pi}\right)+\left(l_{vi}-l_{qi}\right)}$$

(5)

From Equation (5):

$$l_{vi}=\frac{2l_{pi}{l}_{qi}}{l_{pi}+l_{qi}}$$

(6)

The symmetry line segment is parallel to a vector S_i connecting the center of projection [0 0 0]^t and its vanishing point v_i in the 2D image:

$$S_i=\frac{1}{\sqrt{\Vert v_i\Vert {}^2+f^2}}\left[\begin{array}{c}{x}_c+l_{vi}\cos {\theta }_i\\ y_c+l_{vi}\sin {\theta }_i\\ f\end{array}\right]$$

(7)

Let the 3D centro-symmetrical construction of p_i and q_i be P_i and Q_i and the symmetry point be C_s:

$$\begin{array}{c}{P}_i=\left[\begin{array}{c}{X}_{Pi}\\ Y_{Pi}\\ Z_{Pi}\end{array}\right]=\frac{Z_{Pi}}{f}\left[\begin{array}{c}{p}_i\\ f\end{array}\right]\\ Q_i=\left[\begin{array}{c}{X}_{Qi}\\ Y_{Qi}\\ Z_{Qi}\end{array}\right]=\frac{Z_{Qi}}{f}\left[\begin{array}{c}{q}_i\\ f\end{array}\right]\\ C_s=\left[\begin{array}{c}{X}_C\\ Y_C\\ Z_C\end{array}\right]=\frac{{\Delta }_s}{\sqrt{\Vert c_s\Vert {}^2+f^2}}\left[\begin{array}{c}{x}_c\\ y_c\\ f\end{array}\right]\end{array}$$

(8)

where Δ_s is a free parameter representing the distance of C_s from the center of projection. Note that P_i and Q_i in the 3D scene are connected by the symmetry line segment that is bisected by C_s:

$$\begin{array}{c}{P}_i=C_s+\frac{{\Delta }_i}{2}{S}_i\\ Q_i=C_s-\frac{{\Delta }_i}{2}{S}_i\end{array}$$

(9)

where |Δ_i| is the length of the symmetry line segment. From Equations (3) and (7)–(9), we have:

$$\begin{array}{c}\left(\frac{{\Delta }_s}{\sqrt{\Vert c_s\Vert {}^2+f^2}}+\frac{{\Delta }_i}{2\sqrt{\Vert v_i\Vert {}^2+f^2}}\right)\left[\begin{array}{c}{x}_c\\ y_c\\ f\end{array}\right]+\frac{{\Delta }_i}{2\sqrt{\Vert v_i\Vert {}^2+f^2}}\left[\begin{array}{c}{l}_{vi}\cos {\theta }_i\\ l_{vi}\sin {\theta }_i\\ 0\end{array}\right]=\frac{Z_{Pi}}{f}\left[\begin{array}{c}{x}_c+l_{pi}\cos {\theta }_i\\ y_c+l_{pi}\sin {\theta }_i\\ f\end{array}\right]\\ \left(\frac{{\Delta }_s}{\sqrt{\Vert c_s\Vert {}^2+f^2}}-\frac{{\Delta }_i}{2\sqrt{\Vert v_i\Vert {}^2+f^2}}\right)\left[\begin{array}{c}{x}_c\\ y_c\\ f\end{array}\right]-\frac{{\Delta }_i}{2\sqrt{\Vert v_i\Vert {}^2+f^2}}\left[\begin{array}{c}{l}_{vi}\cos {\theta }_i\\ l_{vi}\sin {\theta }_i\\ 0\end{array}\right]=\frac{Z_{Qi}}{f}\left[\begin{array}{c}{x}_c+l_{qi}\cos {\theta }_i\\ y_c+l_{qi}\sin {\theta }_i\\ f\end{array}\right]\end{array}$$

(10)

From Equations (6) and (10), Δ_i can be computed as:

$${\Delta }_i=2{\Delta }_s\frac{\left(l_{pi}+l_{qi}\right)\sqrt{\Vert v_i\Vert {}^2+f^2}}{\left(l_{qi}-l_{pi}\right)\sqrt{\Vert c_s\Vert {}^2+f^2}}$$

(11)

The free parameter Δ_s is the distance of the symmetry point C_s from the center of projection. It determines the size of the constructed 3D centro-symmetrical interpretation. This construction process can be applied to all of the corresponding pairs of points in the 2D image. Δ_s should be the same among all of the pairs of points. When this has been done, the set of pairs of constructed 3D points is 3D centro-symmetrical about C_s. Each pair of constructed points can be connected by a symmetry line segment that is bisected by C_s.

2.4. Model-Based Invariant of 3D Centro-Symmetry with Planar Contours

Consider a 3D mirror-symmetrical pair of curves. It has been suggested that the planarity of these individual contours can play an important role when the human visual system detects the 3D mirror symmetry from the 2D image of its contours. This can be explained by introducing a model-based invariant into the 2D image by using 3D mirror symmetry along with the planarity of contours [32,33,36]. Under an orthographic projection, the relationship between the images of the curves can be represented by a sub-group of the 2D affine transformation. Under a perspective projection, the relationship can also be represented by a transformation that includes the same sub-group of the 2D affine transformation.

An analogous model-based invariant can be introduced into the 2D image by using 3D rotational symmetry along with the planarity of contours [25]. Under an orthographic projection, the relationship between the images of planar curves that are 3D rotation-symmetrical to one another can be represented by a sub-group of the 2D affine transformation. Under a perspective projection, the relationship can also be represented by a transformation that includes the same sub-group of the 2D affine transformation. Note that the sub-group of the 2D affine transformation for 3D rotational symmetry is different from the sub-group for 3D mirror symmetry. In the next two sections, the same approach to 3D centro-symmetry will be discussed under both orthographic and perspective projections.

2.4.1. Model-Based Invariant of 3D Centro-Symmetry with Planar Contours under a 2D Orthographic Projection

Under an orthographic projection, the planarity of contours does not play any role. Note that a 2D orthographic image of a 3D centro-symmetrical object is always 2D rotational symmetry and that a 3D centro-symmetrical interpretation can always be constructed from a 2D rotation-symmetrical image (see Section 2.1. A 2D orthographic projection of 3D centro-symmetry). It is always possible to construct a 3D centro-symmetric pair of planar curves from a pair of 2D rotation-symmetrical curves with an orthographic projection.

Consider a pair of curves ϕ and ψ that are 2D rotation-symmetrical around the origin $o_{2D}={\left[\begin{array}{cc}0& 0\end{array}\right]}^t$ in a 2D image and a corresponding pair of points $p_i={\left[\begin{array}{cc}{x}_{pi}& y_{pi}\end{array}\right]}^t$ on ϕ and $q_i={\left[\begin{array}{cc}{x}_{qi}& y_{qi}\end{array}\right]}^t$ on ψ. Their relation can be represented as $p_i=-q_i$. From Equation (1), their 3D centro-symmetrical interpretations can be constructed as $P_i={\left[\begin{array}{ccc}{x}_{pi}& y_{pi}& Z_i\end{array}\right]}^t$ and $Q_i=-P_i$, where P_i is the 3D interpretation of p_i and Q_i is the 3D interpretation of q_i. The parameter Z_i is computed as:

$$Z_i=\frac{N_X{x}_{pi}+N_Y{y}_{pi}+k_{\Phi }}{-N_Z}$$

(12)

where N_X, N_Y, N_Z, and k_Φ are free parameters. This construction process can be applied to all corresponding pairs of points on ϕ and ψ. N_X, N_Y, N_Z, and k_Φ should be the same among all of the pairs of points. When this is done, a 3D centro-symmetrical pair of planar curves Φ and Ψ can be constructed from ϕ and ψ. Equation (12) represents a plane of Φ. The plane of Φ is parallel to a plane of Ψ.

2.4.2. Model-Based Invariant of 3D Centro-Symmetry with Planar Contours under a 2D Perspective Projection

Under a perspective projection, a model-based invariant can be introduced into the 2D image by 3D centro-symmetry used along with the planarity of contours. Now, consider a 3D centro-symmetrical pair of planar contours Φ and Ψ and their 2D perspective projections φ and ψ. Plane Π_Φ of Φ and plane Π_Ψ of Ψ are parallel to one another. Set the 2D Cartesian coordinate system of a 2D image and the 3D Cartesian coordinate system of a 3D scene in which the 3D interpretation will be constructed as follows: (i) the origin of the 3D coordinate system is at the center of projection, (ii) the Z-axis of the 3D coordinate system coincides with the principal axis and it is perpendicular to the image plane Z = f, where f is the focal distance, (iii) the Z-axis passes the origin of the 2D coordinate system, and (iv) the X- and Y-axes of the 3D coordinate system are parallel to the x- and y-axes of the 2D coordinate system.

First, consider for simplicity, a case in which Π_Φ and Π_Ψ are frontoparallel:

$$\begin{array}{c}{\Pi }_{\Phi }:Z=k_{\Phi }\\ {\Pi }_{\Psi }:Z=k_{\Psi }\end{array}$$

(13)

where k_Φ, k_Ψ are constants. Let the symmetry center of Φ and Ψ be $C_s={\left[\begin{array}{ccc}{X}_C& Y_C& Z_C\end{array}\right]}^t$. From Equation (13), $\left(k_{\Phi }+k_{\Psi }\right)/2=Z_C$. Now, consider a corresponding pair of points P_i on Φ and Q_i on Ψ. P_i and Q_i can be written as: $P_i={\left[\begin{array}{cc}{X}_C+X_i& Y_C+Y_i\hspace{1em}{k}_{\Phi }\end{array}\right]}^t$ and $Q_i={\left[\begin{array}{ccc}{X}_C-X_i& Y_C-Y_i& 2Z_C-k_{\Phi }\end{array}\right]}^t$. Their perspective projections can be written as:

$$\begin{array}{c}{p}_i=\frac{f}{k_{\Phi }}{\left[\begin{array}{c}{X}_C+X_i\hspace{1em}{Y}_C+Y_i\end{array}\right]}^t\\ q_i=\frac{f}{2Z_C-k_{\Phi }}{\left[\begin{array}{c}{X}_C-X_i\hspace{1em}{Y}_C-Y_i\end{array}\right]}^t\end{array}$$

(14)

From Equation (14), the relation between p_i and q_i is:

$$\left(p_i-c_s\right)=-\frac{k_{\Psi }}{k_{\Phi }}\left(q_i-c_s\right)$$

(15)

where $c_s=f/Z_C{\left[\begin{array}{c}{X}_C\hspace{1em}{Y}_C\end{array}\right]}^t$ is a perspective projection of C_S. This shows that the relation between φ and ψ can be represented by a sub-group of the 2D affine transformation, namely scaling by a factor of −k_Ψ/k_Φ around c_s.

If a pair of curves β and δ in the 2D image satisfy Equation (15), their 3D centro-symmetrical interpretations are a pair of planar contours in the 3D scene. Now, consider n corresponding pairs of points (b₁, b₂, … b_n) on β and (d₁, d₂, … d_n) on δ where n ≥ 3. They satisfy Equation (15): (b_i – c_βδ) = −k_βδ(d_i – c_βδ), where i is an integer (1 ≤ i ≤ n), k_βδ is a constant, and c_βδ is an intersection of line segments connecting the corresponding pairs of points between β and δ. From Equation (3), b_i and d_i can be written as b_i = c_βδ + l_bi[cosθ_i sinθ_i]^t and d_i = c_βδ + l_di[cosθ_i sinθ_i]^t, where θ_i is the orientation of the line segment connecting b_i and d_i, and $\left|l_{bi}\right|$ and $\left|l_{di}\right|$ are the distance of b_i and d_i from c_βδ. Note that l_bi/l_di is k_βδ for all of the corresponding pairs of points, because β and δ satisfy Equation (15). Let B_i and D_i be the 3D centro-symmetrical interpretations of b_i and d_i and Z_Bi and D_Bi be their Z-coordinates. From Equations (7)–(11), the Z-coordinates of B_i and D_i are:

$$\begin{array}{c}{Z}_{Bi}=\frac{{\Delta }_{s}f}{\sqrt{\Vert c_{\beta \delta }\Vert {}^2+f^2}}\left(1+\frac{\left(l_{bi}+l_{di}\right)}{\left(l_{di}-l_{bi}\right)}\right)\\ Z_{Di}=\frac{{\Delta }_{s}f}{\sqrt{\Vert c_{\beta \delta }\Vert {}^2+f^2}}\left(1+\frac{\left(l_{bi}+l_{di}\right)}{\left(l_{di}-l_{bi}\right)}\right)\end{array}$$

(16)

where Δ_s is a free parameter representing the distance of the symmetry point from the center of projection. It should be the same among all of the pairs of points. Note that $\left(l_{bi}+l_{di}\right)/\left(l_{di}-l_{bi}\right)$ is also the same among all of the pairs of points, because l_bi/l_di is constant (k_βδ). Equation (16) shows that the 3D centro-symmetrical interpretations of β and δ are a pair of contours that are individually frontoparallel and planar.

Next, consider a general case in which planes Π_Φ and Π_Ψ are not frontoparallel:

$$\begin{array}{c}{\Pi }_{\Phi }:-N_{Z}Z=N_{X}X+N_{Y}Y+k_{\Phi }\\ {\Pi }_{\Psi }:-N_{Z}Z=N_{X}X+N_{Y}Y+k_{\Psi }\end{array}$$

(17)

where k_Φ, k_Ψ, N_X, N_Y, and N_Z are constants, and N_X² + N_Y² = 1. Note that [N_X N_Y N_Z]^t is normal to Π_Φ and Π_Ψ. Let C_S = [X_C, Y_C, Z_C]^t be their symmetry center. Note that a line connecting two of the points P_i and P_j of Φ, and a line connecting their corresponding points Q_i and Q_j of Ψ, are parallel to one another. Let φ, ψ, and c_s be 2D perspective projections of Φ, Ψ, and C_S. This general case can be transformed to a simple case in which Π_Φ and Π_Ψ are frontoparallel by rotating the principal axis and the image plane around the center of projection [0 0 0]^t. The perspective image after this rotation can be computed by using Kanatani’s transformation [37].

Now consider n corresponding pairs of points (p₁, p₂, … p_n) on φ and (q₁, q₂, … q_n) on ψ where n ≥ 3 (Figure 4). They are projections of points (P₁, P₂, … P_n) on Φ and (Q₁, Q₂, … Q_n) on Ψ. Draw a line connecting two of the points p_i and p_j on φ and a line connecting their corresponding points q_i and q_j on ψ (the dashed gray lines in Figure 4). The intersection h_ij of these two lines is a vanishing point of two parallel lines connecting P_i and P_j and connecting Q_i and Q_j. These lines are on the planes Π_Φ and Π_Ψ individually, and they are parallel to one another. It follows that h_ij is on a line H_s (the solid red line in Figure 4) that is a common horizon of Π_Φ and Π_Ψ. Another vanishing point can be found by drawing a line connecting different points on φ and by drawing a line connecting their corresponding points on ψ. An additional corresponding pair of points on φ and ψ can be determined by drawing a line (the dotted green line in Figure 4) passing c_s and finding its intersection with φ and ψ if necessary.

This vanishing point is also on H_s. H_s can be determined by finding a line that connects these vanishing points.

By using the horizon H_s of Π_Φ and Π_Ψ, a vanishing point of a line normal to Π_Φ and Π_Ψ can be found. Now, draw a line that passes the principal point [0 0]^t in the image and that is perpendicular to H_s. Let h_v be an intersection of this line with H_s. Next, find a point v_Π on this line such that the visual angle between h_v and v_Π from the center of projection is perpendicular. The line connecting v_Π and the center of projection is normal to Π_Φ and Π_Ψ and it is parallel to [N_X N_Y N_Z]^t (Equation (17)).

Now, rotate the principal axis and the image plane together in the 3D scene to make the principal axis normal to Π_Φ and Π_Ψ and Π_Φ and Π_Ψ become frontoparallel. This rotation is around a vector [N_Y − N_X 0]^t and the degree of the rotation is $\theta ={\cos }^{-1}\left(N_Z/\sqrt{N_Z{}^2+1}\right)$.

A matrix R_Π representing this rotation can be formulated by using Rodrigues’ rotation formula:

$$R_{\Pi }=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]=\left[\begin{array}{ccc}{c}_{\theta }+N_Y{}^2{r}_{\theta }& -N_Y{N}_X{r}_{\theta }& -N_X{s}_{\theta }\\ -N_Y{N}_X{r}_{\theta }& c_{\theta }+N_X{}^2{r}_{\theta }& -N_Y{s}_{\theta }\\ N_X{s}_{\theta }& N_Y{s}_{\theta }& c_{\theta }\end{array}\right]$$

(18)

where c_θ = cosθ, s_θ = sinθ, and r_θ = 1−cosθ. After the rotation R_Π, a point [x y]^t in the original image is transformed to [x′ y′]^t [37]:

$$\left[\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right]=f\frac{\left[\begin{array}{ccc}{r}_{11}& r_{21}& r_{31}\\ r_{12}& r_{22}& r_{32}\end{array}\right]\left[\begin{array}{c}x\\ y\\ f\end{array}\right]}{\left[\begin{array}{ccc}{r}_{13}& r_{23}& r_{33}\end{array}\right]\left[\begin{array}{c}x\\ y\\ f\end{array}\right]}$$

(19)

Note that Π_Φ and Π_Ψ become frontoparallel after R_Π. With this done, the transformed image satisfies Equation (15). From Equations (15) and (19), the relation between φ and ψ is:

$$\left({p^{\prime }}_i-{c^{\prime }}_s\right)=\frac{k_{\Phi }}{k_{\Phi }}\left({q^{\prime }}_i-{c^{\prime }}_s\right)$$

(20)

where p_i′, q_i′, and c_s′ are the transformations of p_i, q_i, and c_s by using Equation (19). This shows that the relation between φ and ψ can be represented by the transformation of Equation (19) and by a sub-group of the 2D affine transformation that is scaling.

3. General Discussion

This study examined how, from a theoretical point of view, 3D centro-symmetry can be used to organize a 2D image of an object and to recover its 3D shape from the 2D image. My study showed that 3D centro-symmetry has the following properties: (i) a 2D orthographic image of a 3D centro-symmetrical shape is always 2D rotation-symmetrical, (ii) a 3D centro-symmetrical shape can be recovered from its single 2D perspective projection but not from its 2D orthographic projection, (iii) any pair of 2D curves is consistent with a 3D centro-symmetrical interpretation under a perspective projection, and (iv) additional model-based invariants of 3D rotational symmetry can be introduced under a perspective projection if the 3D centro-symmetrical set of curves are planar, individually.

This study was motivated by the fact that the human visual system is known to be sensitive to 2D rotational symmetry, which is a model-based invariant of 3D centro-symmetry under an orthographic projection but not under a perspective projection. It raises two questions: (i) which 3D centro-symmetry or 3D rotational symmetry is actually inferred from 2D rotational symmetry in the human visual system and (ii) can any sensitivity to 3D centro-symmetry be helpful for the perception of 3D? The first question can be addressed by testing the human sensitivity to 2D rotation-symmetrical patterns with a 3D centro-symmetrical or with a 3D rotation-symmetrical depth distribution (see Figure 5, see also [38,39,40,41] for studies of the perception of 2D mirror symmetry from a dotted pattern with a depth distribution). Note, however, even if the human visual system is sensitive to 3D centro-symmetry, it is unclear how 3D centro-symmetry can be of any help in the perception of 3D. Note that this study showed that 3D centro-symmetry can be theoretically helpful for the 3D perception with a perspective projection but not with an orthographic projection. This inconsistency is inconvenient for the human visual system, because it prevents it from making effective use of 3D centro-symmetry. When the retinal image of a 3D centro-symmetrical object is small, the image can be approximated well by an orthographic projection, and the human visual system can detect 3D centro-symmetry of the object on the basis of 2D rotational symmetry. However, here, the 3D centro-symmetry is not helpful in perceiving the 3D shape of the object, because the image is too small. When the retinal image of an object is sufficiently large, 3D centro-symmetry is helpful in perceiving the 3D shape of the object from this large image, but the 3D centro-symmetry cannot be detected on the basis of 2D rotational symmetry. Therefore, this theoretical study suggests that 3D centro-symmetry cannot be used by the human visual system to organize a 2D image of an object to make it possible to recover its 3D shape from the 2D image.