Counting Non-Convex 5-Holes in a Planar Point Set

: Let S be a set of n points in the general position, that is, no three points in S are collinear. A simple k -gon with all corners in S such that its interior avoids any point of S is called a k -hole. In this paper, we present the ﬁrst algorithm that counts the number of non-convex 5-holes in S . To our best knowledge, prior to this work there was no known algorithm in the literature except a trivial brute force algorithm. Our algorithm runs in time O ( T + Q ) , where T denotes the number of 3-holes, or empty triangles, in S and Q that denotes the number of non-convex 4-holes in S . Note that T + Q ranges from Ω ( n 2 ) to O ( n 3 ) , while its expected number is Θ ( n 2 log n ) when the points in S are chosen uniformly and independently at random from a convex and bounded body in the plane.


Introduction
Let S be a set of n points in the plane that are in general position; that is, no three points in S are collinear. For each k ≥ 3, a k-gon in S is a simple polygon with k corners that are members of S. A k-hole in S is an empty k-gon, that is, its interior does not contain any point in S, so is an empty k-gon. In this paper, we focus on the number of k-holes in S for a fixed k ≥ 3, and address the problem of counting non-convex k-holes in a given set S, in particular for k = 5.
The problem of counting or enumerating k-gons or k-holes has been studied for decades by many researchers. It is obvious that counting objects of a particular type is not as hard as enumerating them, since we can count them by enumeration. Dobkin et al. [1] in 1990 presented an algorithm that enumerates all convex k-holes in time O(h 3 (S) + k · h k (S)), where h k (S) denotes the number of k-holes in S. This algorithm, in particular, for k = 3, 4 takes time O(h k (S)), proportional to the number of reported k-holes. Rote et al. [2] studied the problem of counting convex k-gons in S and presented an O(n k−2 )-time algorithm. This was improved to O(kn 3 ) time for k ≥ 6 by Mitchell et al. [3] after Rote and Woeginger [4]. Mitchell et al. [3] also solved the problem of counting convex k-holes in the same time O(kn 3 ), and it was very recent that the second author improved this to O(kh 3 (S)) time [5]. The number h 3 (S) of empty triangles in S ranges between Ω(n 2 ) and O(n 3 ), while the expected value of h 3 (S) is proven to be Θ(n 2 ) if the points in S are uniformly chosen from a convex and bounded body in the plane [6,7].

Our Contribution and Approach
The problem of counting non-convex k-holes, however, has not received enough attention for a long time. Very recently, the authors showed how to count non-convex 4-holes in S in O(h 3 (S)) time [8].
In this paper, we continue our efforts, and present an algorithm that precisely counts non-convex 5-holes in a given set S. Our main contribution is summarized by the following theorem. Theorem 1. Let S be a set of n points, no three of which are collinear. The number of non-convex 5-holes in S can be computed in O(h 3 (S) + h 4 (S)) time, where h 3 (S) denotes the number of empty triangles in S and h 4 (S) denotes the number of non-convex 4-holes in S.
To our best knowledge, prior to this work, there was no known algorithm that counts non-convex k-holes in a given set S of points for k ≥ 5 in the literature, except a trivial exhaustive enumeration algorithm taking O(n k+1 ) time. Our algorithm takes O(h 3 (S) + h 4 (S)) time, where h 4 (S) denotes the number of non-convex 4-holes in S. The number h 4 (S) of non-convex 4-holes in S varies from 0 to O(n 3 ) [9], while its expected number has been known to be Θ(n 2 log n) by Fabilia-Monroy et al. [10] when the points in S were chosen uniformly at random from a convex and bounded body. Hence, our algorithm is expected to run in O(n 2 log n) time for random points S, while its worst-case running time is bounded by O(n 3 ). See the below and Table 1 for more details on the known bounds on the number h k (S) of convex k-holes and the number h k (S) of non-convex k-holes for each k ≥ 3. Table 1. The current best bounds on the minimum, maximum, and expected numbers of convex and non-convex k-holes, denoted by h k (S) and h k (S), and their total number h k (S) in a set S of n points in general position. For the expected number of k-holes, the n points in S are assumed to be chosen uniformly and independently at random from a convex and bounded body.

Minimum
Maximum Expected k = 3 h 3 n 2 − O(n), 1.6196n 2 + o(n) [11] ( n 3 ) = Θ(n 3 ) [easy] Θ(n 2 ) [6] k = 4 h 4 1 2 n 2 − O(n), 1.9397n 2 + o(n) [11] ( n 4 ) = Θ(n 4 ) [easy] Θ(n 2 ) [10] [easy] Θ(n 3 ) [9] Θ(n 2 log n) [10] [11,13,14] ( n k ) = Θ(n k ) [easy] O(n 2 ) [7] h k 0 [easy] Θ(n k−1 ) [12] O(n k−1 ) [12] h k Ω(n 2 ),O(n k+1 2 log k−3 n) [12] ( n k ) = Θ(n k ) [12] O(n k−1 ) [12] In order to count 5-holes in S, we start with a simple fact that any pentagon is decomposed into three triangles. Conversely, a 5-hole can be obtained by attaching three empty triangles that are adjacent side by side. Of these three triangles, the one adjacent to the other two is called a middle triangle of the pentagon. We give a classification of middle triangles of 5-holes into four types according to the relative positions of the other two triangles, and reveal a crucial relation between the number of diagonals of 5-holes and the types of middle triangles. This observation enables us several different ways of counting 5-holes with multiplicity, together with the structural properties of 3-holes (i.e., empty triangles) and 4-holes known by the previous paper [8]. In turn, we obtain a system of linear equations of a constant number of variables, and the number of non-convex 5-holes in S can be computed by solving the system.

Related Work
Combinatorial and algorithmic studies on k-gons and k-holes were initiated by Erdős and Szekeres [15] in 1935, and are still being continued by many researchers today.
In 1978, Erdős also posed an empty version of the Erdős-Szekeres problem which asks the least number H(k) for k ≥ 3 such that any set of H(k) points in general position contains a convex k-hole [20]. Some known results on H(k) are as follows: H(3) = 3 and H(4) = 5 are easy to see, H(5) = 10 is proved by Harborth [21], H(6) is bounded [22,23], and Horton [13] proved that H(k) for k ≥ 7 is unbounded and undefined.
As a variant of the Erdős-Szekeres problem, the possible number of convex k-gons and k-holes in a set of n points has been studied extensively in the literature. Here, we introduce only results on the number of k-holes; for more results on k-gons, we refer to survey papers by Morris and Soltan [24,25].
It is not difficult to see that the maximum possible number of convex k-holes is achieved when the points in S are in a convex position so that h k (S) ≤ ( n k ) = Θ(n k ). Thus, most of the research has focused on their minimum possible numbers and expected numbers when the points in S are chosen uniformly from a certain convex shape in the plane. For the minimum and expected number of convex k-holes, currently, the best bounds are summarized in Table 1. Some important bounds are as follows. Note that h k (S) can be zero for any k ≥ 7 and some sets S of any size n by Horton [13]. Bárány and Füredi [6] showed that the minimum of h k (S) over all sets S of n points is Θ(n 2 ) for k ∈ {3, 4} and is between Ω(n) and O(n 2 ) for k ∈ {5, 6}. There has been constant effort to improve the hidden constants in the O(·) and Ω(·). For more history, see Bárány and Valtr [11], Valtr [26], Dumistrescu [27], Pinchasi, Radoičić, and Sharir [28], and references therein. A recent breakthrough was due to Aichholzer et al. [14] who showed that the minimum possible number of convex 5-holes is at least Ω(n log 4/5 n). Bárány and Füredi [6] showed that the expected value of h 3 (S) is Θ(n 2 ) when the points in S are chosen uniformly from a unit square. Recently, Balko et al. [7] extended this result to any higher-dimensional space and any convex body of unit measure from which points are chosen uniformly at random.
Despite a huge amount of results on convex k-gons and k-holes, it was relatively recent that the first nontrivial bounds on the minimum and maximum numbers of non-convex k-gons and k-holes were proven. Aichholzer et al. [9] and Aichholzer et al. [12] established several new bounds on the maximum/minimum possible number of non-convex k-gons and k-holes for k ≤ 4. Let h k (S) be the number of non-convex k-holes in S, and let h k (S) := h k (S) + h k (S) be the number of k-holes in S, regardless of being convex or non-convex. Specifically, they showed that the minimum of h k (S) is between Ω(n 2 ) and O(n k+1 2 log k−3 n) and the maximum of h k (S) is Θ(n k−1 ) for each k ≥ 4. Fabila-Monroy et al. [10] proved that the expected number of h 4 (S) is 12n 2 log n + o(n 2 log n) when the points in S are chosen uniformly from a convex and bounded body. This also improves the upper bound of the minimum number of h 4 (S) to O(n 2 log n). Again, see Table 1 for the current best bounds on the minimum, maximum, and expected numbers of h k (S) and h k (S).

Organization of the Paper
The rest of the paper is organized as follows. Section 2 introduces some necessary preliminaries and facts on empty triangles and empty quadrilaterals, that is, 3-holes and 4-holes known in the literature, for further discussion on our algorithm that counts 5-holes. Section 3 is devoted to provide crucial properties of 5-holes in terms of their middle triangles. Our counting algorithm is described and analyzed in Section 4. We finally conclude this paper with remarks and open problems in Section 5.

Preliminaries
Throughout the paper, we consider a standard orthogonal coordinate system in the plane R 2 with the xand y-coordinates, from which we may say that a point is to the left, to the right, below, or above another point in a normal sense. For any two points p, q ∈ R 2 , the line segment joining p and q will be denoted by pq.
A polygon with k corners is represented by a cyclic sequence of its corners p 1 p 2 . . . p k . When we write any polygon by its corner sequence p 1 p 2 . . . p k , it is imposed that the corners are traversed in counterclockwise order. A diagonal of a polygon is a segment between its two nonadjacent corners that does not stick out of the polygon. Any convex k-gon has ( k 2 ) = k(k − 1)/2 distinct diagonals, while any non-convex k-gon has a strictly smaller number of those. Let S be a given set of n points in general position in the plane so that no three points in S are collinear. For any k ≥ 3, let h k := h k (S), h k := h k (S), and h k := h k (S) for simplicity. For any p ∈ S, let S p be the cyclic array consisting of points in S \ {p} in counterclockwise order around p. The sorted cyclic arrays S p for all p ∈ S can be obtained in total O(n 2 ) time by constructing the arrangement of n lines dual to the points in S and traversing the arrangement. See Dobkin et al. [1] and Rote et al. [2].

Empty Triangles and Their Structures
For any two distinct points p, q ∈ S, let T pq be the set of all points r ∈ S \ {p, q} such that the triangle pqr is empty, and we obtain p, q, and r in this order when we traverse the boundary of the triangle in the counterclockwise direction from p. Similarly, let T qp be the set of all points r ∈ S \ {p, q} such that the triangle qpr is empty, and we obtain q, p, and r in this order when we traverse the boundary of the triangle in the counterclockwise direction from q. Note that T pq and T qp are separated by the line through p and q, and that T pq ∪ T qp consists of all points r ∈ S such that the triangle constructed with pq and a third corner r is empty. See Figure 1 for an illustration. The following is an immediate identity by a common counting technique.
Proof. Consider any empty triangle abc in S. Then, the triangle abc is involved in exactly three sets T ab , T bc , and T ca in such a way that c ∈ T ab , a ∈ T bc , and b ∈ T ca . Hence, in the sum ∑ p∈S ∑ q∈S p |T pq |, each empty triangle is counted exactly three times, so we have the claimed equation.
The authors have proved the following in a previous paper [8].
Lemma 2 (Sung and Bae [8]). For any pair (p, q) ∈ S × S with p = q, the points in T pq appear in the same counterclockwise order, both around p and q. For all such pairs (p, q), the points in T pq can be computed and sorted in this order in total O(h 3 ) time into a sorted array.
By an abuse of notation, we shall mean by T pq the sorted array of itself obtained by the above lemma. Additionally, for any positive integer i, let T pq [i] denote the i-th point in the array T pq . In Figure 1, the points in T pq and T qp and their indices are depicted.

Empty Quadrilaterals
Fix two points p, q ∈ S with p = q, and any point r = T pq [i] in T pq . Consider the two directed rays emanating from r towards p and q, respectively, denoted by γ 1 and γ 2 , respectively. We associate with T pq [i] the following: Figure 2 for an illustration. These values are essential to count the number of convex and non-convex 4-holes, as shown in the Ref. [8], in which all these values can be computed in O(h 3 ) time. Lemma 3 (Sung and Bae [8]). Let p, q ∈ S with p = q be fixed. For any Consider an empty triangle pqr in S with r ∈ T pq . Any 4-hole with pq as a diagonal can be obtained by gluing pqr and another empty triangle qps with s ∈ T qp along the side pq. The resulting 4-hole psqr can be either convex or non-convex. The authors have shown the following. Let i be such that r = T pq [i], and j be such that . This directly provides a way of counting 4-holes having pq as a diagonal, being separately convex and non-convex. [8]). Let p, q ∈ S with p = q be fixed. Then, psqr is a 4-hole for r, s ∈ S with a diagonal pq if, and only if r ∈ T pq and s ∈ T qp . Moreover, if r = T pq [i] and s = T qp [j] for indices 1 ≤ i ≤ |T pq | and 1 ≤ j ≤ |T qp |, then the following hold:

Lemma 4 (Sung and Bae
. Therefore, the number of convex 4-holes psqr for fixed p, q ∈ S and r = , and the number of those that are non-convex is exactly The statements in Lemma 4 can also be verified in Figure 2. Consider the example depicted in Figure 2 in which i = 4 and r = T pq [4]. Then, any convex 4-hole of the form psqr should choose s ∈ T qp to be between the two rays γ 1 and γ 2 , so if s = T qp [j], then j must be between A pq [4] + 1 and B pq [4]. On the other hand, non-convex 4-holes are constructed by Lemma 4 provides a way of counting convex and non-convex 4-holes, by iterating all p, q ∈ S and r ∈ T pq .

Lemma 5. It holds that
Proof. The above equations immediately follow from Lemma 4.

Middle Triangles of Empty Pentagons
It is obvious and well-known that any simple k-gon, being either convex or nonconvex, can be decomposed into exactly k − 2 triangles by cutting it along two non-crossing diagonals. Such a decomposition is called a triangulation of a polygon. Any pentagon, hence, admits a triangulation of exactly three triangles. This implies that any simple pentagon has at least two diagonals. More generally, we observe the following. Lemma 6. Let P be any simple pentagon. The number of diagonals of any simple pentagon is between two and five. Moreover, a simple pentagon has five diagonals if, and only if it is convex.
Proof. As discussed above, any simple pentagon admits a triangulation of three triangles. Therefore, there must be at least two diagonals.
For each corner v of any simple pentagon, it is adjacent to two other corners, so there are, at most, two corners that can be joined with v by diagonals. This implies that the maximum number of diagonals of a simple pentagon is five, and this can be achieved only when it is convex.
In any triangulation of a pentagon P, there must be a triangle bounded by two diagonals of the pentagon P. Such a triangle in P is called a middle triangle of P. In general, a simple pentagon has at least one middle triangle as it has a triangulation, while it may not be unique.
Consider an empty pentagon, or a 5-hole, P in the given set S of points. Let pqr be a middle triangle of P such that qr and rp are diagonals of P and pq is a side of P. Then, pqr is an empty triangle in S and P can be written as P = pqxry where x, y ∈ S are the other two corners of P. The two triangles pry and qxr are also empty, and hence we have that x ∈ T rq and y ∈ T pr .
Assume without loss of generality that pq is horizontal and r is above the line extending pq. Let y be the directed line through p and towards r, and x be the directed line through q and towards r. Observe that y lies in the left side of y and x lies in the right side of x , since P is a simple pentagon. Let X be the half-plane on the left side of x and Y be the half-plane on the right side of y . Hence, we have x ∈ X and y ∈ Y.
These regions X and Y are further partitioned as follows: The region X is partitioned into three regions X + , X 0 , and X − by two lines x and such that X + = X ∩ Y, X − is the region below , and X 0 = X \ (X + ∪ X − ). Analogously, Y is partitioned into three regions Y + , Y 0 , and Y − by y and such that Y Figure 3 for an illustration. We classify each middle triangle pqr of a 5-hole P = pqxry into one of the following four types, depending on the locations of x and y. We say that the middle triangle pqr of P is of: type 00 if x ∈ X 0 and y ∈ Y 0 , (b) type 0− if either x ∈ X 0 and y ∈ Y − , or x ∈ X − and y ∈ Y 0 , (c) type 0+ if either x ∈ X 0 and y ∈ Y + , or x ∈ X + and y ∈ Y 0 , or (d) type +−, otherwise, if x ∈ X + ∪ X − and y ∈ Y + ∪ Y − . We observe fundamental properties of a 5-hole that has a middle triangle of each specific type.

Lemma 7.
Let P = pqxry be any 5-hole in S such that pqr is a middle triangle of P. Let d be the number of diagonals and m be the total number of middle triangles of P. Then, the following holds according to the type of pqr.

(a)
Suppose that pqr is of type 00, so we have x ∈ X 0 and y ∈ Y 0 . In this case, by Lemma 4, both 4-holes pqry and pqxr are convex, so both yq and xp are diagonals of 5-hole P = pqxry. There are two cases: either xy is a diagonal or not. (b) Suppose that pqr is of type 0−, so either x ∈ X 0 and y ∈ Y − , or x ∈ X − and y ∈ Y 0 . We assume the former case without loss of generality since the other case is symmetric. In this case, by Lemma 4, 4-hole pqxr is convex while 4-hole pqry is non-convex, so xp is a diagonal of P but yq is not. There are two cases: either xy is a diagonal or not.
• If xy is a diagonal of P, then P has four diagonals and three middle triangles: pqr, xrp, and ypx. Observe that pqr and ypx are of type 0−, while xrp is of type 00 type. See the left figure of Figure 4b. • If xy is not a diagonal of P, then P has three diagonals and two middle triangles pqr and xrp. Observe that xrp is of type 0+, while pqr is of type 0−. See the right figure of Figure 4b.
(c) Suppose that pqr is of type 0+, so either x ∈ X 0 and y ∈ Y + , or x ∈ X + and y ∈ Y 0 . We assume the former case without loss of generality, since the other case is symmetric. Similarly to the above case, the 4-hole pqxr is convex while the 4-hole pqry is non-convex, so xp is a diagonal of P but yq is not. In this case, however, xy is always not a diagonal of P, since both x and y lie in a common side of the line through q and r. Thus, P has three diagonals and two middle triangles pqr and xrp.
Observe that xrp is of type 0−, while pqr is of type 0+. See Figure 4c.
Suppose that pqr is of type +−, so x ∈ X + ∪ X − and y ∈ Y + ∪ Y − . Then, none of the segments xp and yq can be a diagonal of P, as both 4-holes pqxr and pqry are non-convex by Lemma 4. Additionally, observe that the segment xy cannot be a diagonal of P, either, by the following reason: if x ∈ X + or y ∈ Y + , then corner r of P is a reflex vertex and, otherwise, if x ∈ X − and y ∈ Y − , then both p and q are reflex vertices of P. See Figure 4d. Therefore, the pentagon P has exactly two diagonals pr and qr, and only one middle triangle pqr of type +−.
This completes the proof of the lemma.
From Lemma 7 and its proof, we further observe the following properties of 5-holes, depending on the number of its diagonals. If d = 2, then P has a unique middle triangle of type +−.
Proof. Let P = pqxry be any 5-hole such that pqr is a middle triangle, and let d be the number of its diagonals. By Lemma 6, we have 2 ≤ d ≤ 5. We consider each case of d = 2, 3, 4, 5. First, assume that d = 5. By Lemma 7, the middle triangle pqr is of type 00, as, otherwise, there is no possibility that P has d = 5 diagonals. Again by Lemma 7 (a), we know that P has five middle triangles all of which are of type 00.
Second, assume that d = 4. Then, by Lemma 7, we have that pqr is either of type 00 or type 0−. In the former case, Lemma 7 (a) states that P has two more middle triangles all of which are of type 0−. In the latter case, Lemma 7 (b) states that P has two more middle triangles, one of which is of type 00, and the other, which is of type 0−. We hence conclude that a 5-hole P with d = 4 diagonals has three middle triangles, two of which are of type 0− and the third of which is of type 00, in either case.
Next, assume that d = 3. Then, by Lemma 7, we have that pqr is either of type 0− or type 0+. In the former case, Lemma 7 (b) states that P has one more middle triangle of type 0+. In the latter case, Lemma 7 (c) states that P has one more middle triangle of type 0−. Thus, a 5-hole P with d = 3 diagonals has two middle triangles, one of which is of type 0−, and the other of which is of type 0+.
Finally, assume that d = 2. In this case, from Lemma 7, we know that pqr must be of type +−, and it is the unique middle triangle.
The above lemma suggests a natural way of classification of pentagons according to the number of diagonals and middle triangles. In the following section, we describe our algorithm that counts the number of 5-holes with d diagonals separately for d ∈ {2, 3, 4, 5} in a given set S of points, so it separately counts convex and non-convex 5-holes in S.

Counting Empty Pentagons
Let S be a given set of n points in general position, as declared above. For d ∈ {2, 3, 4, 5}, denote by z d the number of 5-holes in S that have d diagonals. By Lemma 6, the number of convex 5-holes in S is h 5 = h 5 (S) = z 5 , and the number of non-convex 5-holes is h 5 = h 5 (S) = z 2 + z 3 + z 4 .
In this section, we describe an algorithm that precisely computes z d for each d ∈ {2, 3, 4, 5}. Our algorithm indeed counts the number of middle triangles of 5-holes in S in several different ways, resulting in a sufficient number of linear equations on {z 2 , z 3 , z 4 , z 5 }. Hence, we can finally solve the system of linear equations to specify the values of z d for all d ∈ {2, 3, 4, 5}.
More specifically, for each empty triangle pqr, we count the number of 5-holes that have pqr as a middle triangle of a specific type. This can be done by attaching two other empty triangles to pqr along shared sides.

Attaching Two Empty Triangles to a Middle Triangle
As discussed in Section 3, any 5-hole can be partitioned into three empty triangles, one of which is a middle triangle. Conversely, we can construct all 5-holes by considering all possible ways of attaching two empty triangles to a tentative middle triangle along shared sides.
For our purposes, we consider each pair (p, q) ∈ S × S with p = q. For each i ∈ {1, . . . , |T pq |}, let r = T pq [i]. Then, consider the empty triangle pqr and all the pairs of empty triangles rqx and pry that share sides qr and pr, respectively. Note that x ∈ T rq and y ∈ T pr by definition. The union pqr ∪ rqx ∪ pry is clearly a 5-hole pqxry such that pqr is a middle triangle, unless rqx and pry overlap. This simply implies that the number of those 5-holes pqxry is |T rq | · |T pr |, minus the number of pairs (x, y) ∈ T rq × T pr such that rqx and pry overlap.
We can even think of those 5-holes pqxry such that pqr is a middle triangle of a specific type. Specifically, for each p, q ∈ S and r ∈ T pq , we define Z 00 (p, q; r) to be the number of pairs (x, y) ∈ T pr × T rq such that x ∈ X 0 and y ∈ Y 0 ; Z 0+ (p, q; r) to be the number of those pairs (x, y) such that either x ∈ X 0 and y ∈ Y + , or x ∈ X + and y ∈ Y 0 ; Z 0− (p, q; r) to be the number of those pairs (x, y) such that either x ∈ X 0 and y ∈ Y − , or x ∈ X − and y ∈ Y 0 ; and Z +− (p, q; r) to be the number of those pairs (x, y) such that either x ∈ X + ∪ X − and y ∈ Y + ∪ Y − , where the six regions X + , X 0 , X − , Y + , Y 0 , Y − are as defined in Section 3. See Figure 3.
These quantities are indeed the potential number of 5-holes pqxry for fixed p, q ∈ S and r ∈ T pq such that the triangle pqr is a middle triangle of each specified type, while they count all those pairs (x, y) without considering whether rqx and pry overlap or not.
One good point for the quantities is that they can be computed quite easily. Let X − , X 0 , X + = Y + , Y 0 , Y − be the five regions defined for a fixed p, q, r as in Section 3; see Figure 3. From the definition of the arrays A qr , B qr , A rp , B rp , we observe that the following relation between the region where x or y lies and their indices in T rq and T pr , respectively. See Figure 5 for an illustration.

Lemma 9.
Let P = pqxry be any 5-hole in S such that pqr is a middle triangle of P. Denote by i p and i q the indices such that p = T qr [i p ] and q = T rp [i q ], and by j x and j y the indices such that x = T rq [j x ] and y = T pr [j y ]. We then have This proves the first part of the lemma. The second part for the range of j y can be seen in a symmetric and analogous way.
Lemma 9 directly implies the following.

Lemma 10.
For any p, q ∈ S with p = q and r ∈ T pq , the following equations hold: where i p and i q denote the indices in arrays T qr and T rp , respectively, such that p = T qr [i p ] and q = T rp [i q ].
Proof. By Lemma 9, we get the number of points x ∈ T rq in each of the three regions X − , X 0 , and X + as follows. The number of x ∈ T rq such that x ∈ X − is exactly A qr [i p ]; the number of x ∈ T rq such that x ∈ X 0 is exactly B qr [i p ] − A qr [i p ]; and the number of x ∈ T rq such that x ∈ X + is exactly Similarly, the number of y ∈ T pr such that y ∈ Y − is exactly |T pr | − B rp [i q ]; the number of y ∈ T pr such that y ∈ Y 0 is exactly B rp [i q ] − A rp [i q ]; and the number of y ∈ T pr such that This directly implies the lemma.
Remember that the four quantities Z 00 (p, q; r), Z 0− (p, q; r), Z 0+ (p, q; r), and Z +− (p, q; r) count the number of pairs (x, y) with x ∈ T rq and y ∈ T pr without considering whether the corresponding triangles qxr and pry overlap or not. In the following, we characterize a necessary and sufficient condition when the two triangles overlap. For any ordered triple (a, b, c) of distinct points in the plane, we say that (a, b, c) makes a left turn if c lies on the left side of the directed line through a towards b. Lemma 11. Let p, q ∈ S and r ∈ T pq be fixed. Let x ∈ T rq and y ∈ T pr . The triangles qxr and pry overlap if, and only if x ∈ X + , y ∈ Y + , and (x, r, y) makes a left turn.
Proof. First, we observe that if qxr and pry overlap, then x ∈ X + and y ∈ Y + . Assume for this purpose that x ∈ X 0 ∪ X − . Then, the triangle qxr and the point y are separated by the line through p and r. This implies that qxr and pry cannot overlap. Symmetrically, we have that qxr and pry cannot overlap if y ∈ Y 0 ∪ Y − . Hence, if qxr and pry overlap, then x ∈ X + and y ∈ Y + .
Then, suppose that x ∈ X + and y ∈ Y + . For any x ∈ X + and y ∈ Y + , it holds that y / ∈ qxr and x / ∈ pry since qxr and pry are empty triangles. This implies that qxr and pry overlap if, and only if (x, r, y) makes a left turn. See Figure 6. This concludes the lemma.
p q r p q r x y x y The above lemma implies the following: On one hand, the three quantities Z 00 (p, q; r), Z 0− (p, q; r), and Z 0+ (p, q; r) are indeed the number of 5-holes of the form pqxry with x ∈ T rq and y ∈ T pr such that pqr is a middle triangle of type 00, 0−, and 0+, respectively. On the other hand, the quantity Z +− (p, q; r), however, may not be the same as the number of 5-holes of the form pqxry with x ∈ T rq and y ∈ T pr such that pqr is a middle triangle of type +−, since it may count the cases of pairs (x, y) that cause an overlap of qxr and pry. We let χ(p, q; r) be the number of such pairs (x, y), so Z +− (p, q; r) − χ(p, q; r) is equal to the number of 5-holes pqxry such that pqr is a middle triangle of type +−. Lemma 12. Let p, q ∈ S and r ∈ T pq be fixed. The value of χ(p, q; r) can be precisely computed in time O(|T rq | − B qr [i p ] + A rp [i q ] + 1), provided that the indices i p and i q such that p = T qr [i p ] and q = T rp [i q ] are known.
Proof. Here, we describe an algorithm that computes χ(p, q; r). This can be done by linearly scanning two sorted arrays T rq and T pr in limited ranges.
More precisely, we initially set an index variable j x to be B qr [i p ] + 1, and another j y to be 1. Additionally, prepare a counter-variable c, initially being zero. We perform the following while j x ≤ |T rq | and j y ≤ A rp [i q ]: If (T rq [j x ], r, T pr [j y ]) makes a left turn, then increase c by |T rq | − j x + 1 and increase j y by 1; otherwise, if (T rq [j x ], r, T pr [j y ]) makes a right turn, then increase j x by 1.
Since either j x or j y increases by one after every iteration, the total running time is The correctness of this algorithm is guaranteed by Lemma 11 and the observation that if (T rq [j x ], r, T pr [j y ]) makes a left turn, then so does (T rq [j x ], r, T pr [j y ]) for all j x ≤ j x ≤ |T rq |. Now, consider the sum of each of these values over all p, q ∈ S and r ∈ T pq . Define We then have the following system of linear equations for z 2 , z 3 , z 4 , z 5 .

Lemma 13. It holds that
Proof. The lemma can be shown directly from Lemmas 8, 10 and 11.
By Lemma 11, for each p, q ∈ S and r ∈ T pq , Z 00 (p, q; r), Z 0− (p, q; r), and Z 0+ (p, q; r) count the number of 5-holes such that pqr is a middle triangle of type 00, 0−, and 0+, respectively. This implies that their sums Z 00 , Z 0− , and Z 0+ count all middle triangles of type 00, 0−, and 0+, respectively, of all 5-holes in S.
Consider every 5-hole in S. Then, Lemma 8 implies the following: (a) Every 5-hole with five diagonals contributes 5 counts in Z 00 , every 5-hole with four diagonals contributes 1 count in Z 00 , and none of those with three or less diagonals contributes in Z 00 . (b) Every 5-hole with four diagonals contributes 2 counts in Z 0− , every 5-hole with three diagonals contributes 1 count in Z 0− , and the others contribute nothing in Z 0− . (c) Every 5-hole with three diagonals contributes 1 count in Z 0+ and the others contribute nothing in Z 0+ . (d) Every 5-hole with two diagonal contributes 1 count in Z +− . Therefore, the first three equations are derived by Lemma 11, as discussed above. In order to conclude the fourth equation, Lemma 11 implies that Z +− (p, q; r) − χ(p, q; r) is exactly the number of 5-holes such that pqr is a middle triangle of type +−. Therefore, we have Z +− = z 2 + χ.

Algorithm
Now, we are ready to describe our algorithm that precisely computes the values of z 2 , z 3 , z 4 , and z 5 . A high-level description of our algorithm is shown in Algorithm 1. Our algorithm consists of two phases: the preprocessing phase and the main computation phase. In the main computation phase, it computes the values of Z 00 , Z 0− , Z 0+ , Z +− , and χ as defined above, and solve the system of equations shown in Lemma 13 to obtain the values of z 2 , z 3 , z 4 , and z 5 . In the preprocessing phase, we prepare some data structures for efficient processing of the main computation phase.
Additionally, we build a dictionary structure D on the set of all empty triangles that maps each empty triangle pqr to a triple (i p , i q , i r ) of indices such that p = T qr [i p ], q = T rp [i q ], and r = T pq [i r ]. The dictionary structure D can be implemented by a hash table with a perfect hash function since the set of keys is static, and hence, the access to any key entry can be done in O(1) time [29,30]. Using the dictionary D, one can know in O(1) time the position of each point of any empty triangle pqr in the sorted arrays T qr , T rp and T pq , respectively. The following lemma shows how to prepare the dictionary structure. Proof. After computing all empty triangles in S, we initialize D on the set of all empty triangles that maps each triangle to a triple (0, 0, 0).
In order to update the triples of indices correctly, we iterate all pairs (p, q) ∈ S × S with p = q and linearly scan the sorted array T pq . For each i ∈ {1, . . . , |T pq |}, access the entry of pqr where r = T pq [i] and update the corresponding index i r in the triple to be the value of i. In summary, in the preprocessing phase, we build sorted arrays T pq with associated arrays A pq and B pq for all pairs (p, q) ∈ S × S with p = q, and the dictionary D. All these data structures are built in total O(h 3 ) time. Now, we describe the main computation step, in which we compute the values of Z 00 , Z 0− , Z 0+ , Z +− , and χ. This can be done simply by iterating all p ∈ S, q ∈ S p , and i ∈ {1, . . . , |T pq |} and compute the values of Z 00 (p, q; r), Z 0− (p, q; r), Z 0+ (p, q; r), Z +− (p, q; r) and χ(p, q; r) by Lemmas 10 and 12. In conclusion, Z 00 , Z 0− , Z 0+ , and Z +− can be computed in O(h 3 ) time, while χ can be computed in O(h 3 + h 4 ) time. Proof. For a fixed triple (p, q, r) with p ∈ S, q ∈ S p and r ∈ T pq , it takes O(1) time to specify the triple of indices (i p , i q , i r ) such that p = T qr [i p ], q = T rp [i q ], and r = T pq [i r ] by the dictionary D. Hence, we can compute in O(1) time the values of Z 00 (p, q; r), Z 0− (p, q; r), Z 0+ (p, q; r), and Z +− (p, q; r) by Lemma 10, using the arrays T rq , A qr , B qr , T pr , A rp , and B rp obtained from the preprocessing phase.
By Lemma 12, and in order to compute χ(p, q; r) + χ(q, r; p) + χ(r, p; q) for a fixed empty triangle pqr, we spend time by rearranging the terms. Thus, the total running time is the sum of the above terms over all empty triangles pqr.
First, observe that if we expand the sum then the term |T q p | for any p , q ∈ S with p = q appears exactly |T p q | times. This is because each occurrences of the term |T q p | corresponds to an empty triangle of the form p q r for some r ∈ T p q . Hence, we have that ).
If we expand this sum S 2 , then the term B p q [i] − A p q [i] appears exactly once for any p , q ∈ S with p = q and any 1 ≤ i ≤ |T p q |. This implies that Putting it all together, The total running time to compute the value of χ is where h 4 denotes the number of non-convex 4-holes in S, by Lemma 5.
Finally, we apply Lemma 13 to compute each of z 2 , z 3 , z 4 , and z 5 , as described in Line 6 of Algorithm 1.
The running time of our algorithm is bounded by O(h 3 + h 4 ), as follows: We spend O(h 3 ) time for the preprocessing step. To compute Z 00 , Z 0− , Z 0+ , and Z +− , it takes O(h 3 ) time by Lemma 15, while we spend O(h 3 + h 4 ) time to compute χ, as analyzed in Lemma 16. The last step computing z 2 , z 3 , z 4 , and z 5 takes only O(1) time.
Therefore, we conclude Theorem 1 stated in Section 1.

Concluding Remarks
In this paper, we presented the first algorithm that counts non-convex 5-holes in a given set S of n points in general position. Our counting algorithm took O(h 3 (S) + h 4 (S)) time, proportional to the number of empty triangles and non-convex 4-holes in S. Note that Ω(n 2 ) ≤ h 3 (S) ≤ O(n 3 ) and 0 ≤ h 4 (S) ≤ O(n 3 ), while their expected numbers were Θ(n 2 ) and Θ(n 2 log n), respectively, when S was randomly chosen from a convex and bounded body in the plane. See Table 1 for more details.
One natural question is about the extension of our algorithmic results to non-convex k-holes for k ≥ 6. It seems to us that it is not urgent to generalize our algorithmic approach even to the case of 6-holes. Specifically, a pentagon was decomposed into three triangles in such a way that two were attached to the middle one, while a polygon with k ≥ 6 corners was decomposed into k − 2 ≥ 4 triangles, and the number of methods to attach k − 2 triangles grew exponentially in k. Moreover, the number of cases in which these attached triangles overlapped also grew rapidly as k increased.
Another open question is about the true computational complexity of counting convex or non-convex k-holes. Both variants of the maximum-width empty annulus problem had a lower bound of Ω(n log n) by reducing from the maximum gap problem. There was a huge gap between the lower bound Ω(n log n) and the running times of our algorithms. Thus, can one find an algorithm that counts k-holes for k = 3, 4, 5 in o(n 3 ) time in the worst case, or prove a better lower bound of ω(n log n)? In particular, we are unaware of any algorithm that counts the number h 3 (S) of empty triangles faster than O(h 3 (S)) = O(n 3 ).