Abundance Estimation Using Minimum Order Set Distances in Line Transect Sampling

Abstract

Line transect sampling is widely used for estimating population abundance, but existing nonparametric estimators of detection density at the transect line often suffer from boundary bias and tuning sensitivity. In this paper, we propose two simple tuning-light estimators based on minimum order statistics of perpendicular distances, requiring measurement of only the judged-closest object within each set. Under mild regularity conditions, the proposed estimators are consistent and asymptotically normal, with low bias and variance demonstrated through simulation studies under exponential and half-normal detection models. An application to a wooden stakes transect survey illustrates the practical advantages of the proposed approach for low-effort ecological surveys.

1. Introduction

Estimating population abundance is a central problem in ecological and environmental statistics. Accurate measures of animal or plant density provide the foundation for understanding species distribution, monitoring biodiversity, and guiding conservation policies. Among the most widely used approaches, distance sampling—and, in particular, the line transect method—has attracted extensive attention due to its efficiency and practicality [1,2,3]. In this methodology, observers traverse predefined lines placed randomly in a study area with total length Lm and they record the perpendicular distances $x_i$ from each detected object to the transect line. These distances are then used to model the detection process and infer the abundance of the underlying population.

Let x denote the perpendicular distance from the line to an object of interest, and let $f\left(x\right)$ be its probability density function ($pdf$). At the transect line (i.e., $x=0$), the detection probability is assumed to equal one. In line transect surveys, abundance per unit area D is commonly expressed as

$$D=\frac{E_{n}f\left(0\right)}{2L},$$

where $E_n$ denotes the expected number of detections [2]. Thus, the main statistical challenge lies in estimating the point value $f\left(0\right)$; that is, the density of the observed perpendicular distances at the line.

Recent work has shown that spatial variation at multiple scales can challenge standard line transect modeling assumptions, motivating alternative designs and estimators that are less sensitive to model misspecification [4].

1.1. Background and Limitations of Existing Methods

A widely used estimator, under standard regularity assumptions, can simply be written as (see Buckland et al. [2])

$$\widehat{D}=\frac{n\widehat{f}\left(0\right)}{2L},$$

(1)

such that n is the observed number of detected objects and $\widehat{f}(.)$ is an estimate of f obtained from the measured sample of perpendicular distances.

Two broad classes of estimation techniques exist in the literature to produce $\widehat{f}(.)$. Parametric approaches assume a specific form for the detection function, such as the half-normal or exponential models, and they then estimate the associated parameters via the maximum likelihood or the method of moments [5,6]. While these methods are efficient when the assumed model is correct, they may lead to biased estimates when model assumptions are violated. Alternatively, nonparametric methods, such as histogram and kernel estimators, avoid strong distributional assumptions and offer flexibility in modeling the detection process that can reduce boundary bias or improve robustness when certain conditions are violated [7,8,9,10].

Nevertheless, despite their flexibility, nonparametric estimators of $f\left(0\right)$ are often subject to boundary bias at the line and exhibit sensitivity to ad hoc tuning parameters, such as bandwidth in kernel estimators [2,9]. Related approaches, including pooling strategies, are frequently used to stabilize inference in distance sampling; however, their robustness properties require careful interpretation [3].

As a more principled alternative within the nonparametric paradigm, we have developed an approach that concentrates information near the line through order statistics. By exploiting the closest objects to the transect line and leveraging the efficiency gains afforded by order statistic sampling, we construct two tuning-light estimators based on independent, non-overlapping subsets of ranked distances. This design effectively replaces the arbitrary choice of bandwidth with a more interpretable decision on the number of subsets, thereby yielding smoother density estimates and improved computational performance. Recent theoretical developments have sharpened the basis for line transect inference, particularly by clarifying when design-based estimators remain valid under heterogeneity and what pooling robustness guarantees [3].

1.2. Motivation for and Contribution of This Study

In practical line transect surveys, observers are often able to reliably identify the object that is closest to the transect line using simple visual indicators such as apparent distance, viewing angle, or ease of detection, even when exact measurement of all distances is not feasible. This type of partial ranking information is inexpensive to obtain and is typically more reliable near the transect line, where detection probability is highest. Nevertheless, standard distance sampling methods do not explicitly exploit this information.

The existing parametric and nonparametric estimators of $f\left(0\right)$ generally require either full distance measurements or careful tuning of smoothing parameters, both of which can be difficult to implement in resource-limited surveys. Recent advances in order statistic-based density estimation suggest that concentrating on extreme observations, such as minima, can yield stable and efficient estimators with reduced tuning requirements (Kung et al. [11]; Garg et al. [12]). Although these methods were not originally developed for line transect sampling, they provide important methodological insight for constructing estimators that emphasize near-boundary information.

The primary contribution of this study is the development of two simple, tuning-light estimators for the boundary density $f\left(0\right)$ based on minimum order statistics of perpendicular distances. The proposed methods are constructed using independent, non-overlapping sets of detected objects and require measuring only the judged-closest object within each set. This design enforces independence by construction and replaces arbitrary smoothing choices with an interpretable set size parameter. Under mild regularity conditions, the proposed estimators are shown to be consistent and asymptotically normal. Their finite-sample performance was evaluated through simulation studies under common detection models, and their practical utility is illustrated using a wooden stakes transect survey. The practical relevance of this motivation is further supported by the application results in Section 3.3, which show that the proposed estimators achieve stable and competitive performance while requiring substantially fewer distance measurements than conventional approaches.

Theory has sharpened the basis for line transect inference, clarifying when design-based estimators stay valid under heterogeneity and what “pooling robustness” actually guarantees [3]. Simulations also show that pooling stabilizes totals but can mask group-level bias when detectability differs [13]. This motivates methods that emphasize well-identified near-line information, notably estimators targeting $f\left(0\right).$

Integrated distance sampling (IDS) is increasingly used to combine transect detections with auxiliary data, which can improve precision in large-scale analyses [14,15]. Such frameworks typically benefit from inputs that require minimal tuning, where robust estimators of $f\left(0\right)$ may be useful.

In-field logistics matter where incomplete viewsheds reduce the detectable area with distance and bias inference if ignored; recent guidance shows how to account for this in design and analysis [16]. Comparisons of walked versus road transects likewise highlight coverage–detectability trade-offs that favor estimators with predictable behavior across modes [17].

Beyond the line transect sampling framework, several authors have proposed innovative strategies for general density estimation. For example, Kung et al. [11] introduced a block-based method that exploits localized information from order statistics to construct flexible and computationally efficient estimators of $f\left(x\right)$. Later, Garg et al. [12] extended this perspective by developing density estimators that combine subsampling with order statistic properties, thereby enhancing stability and reducing reliance on strong distributional assumptions. Although these approaches were not originally designed for line transect data, they highlight the potential of order statistic-driven techniques to achieve robustness and efficiency in density estimation, and they provided the methodological insights that motivated our development of subset-based estimators for $f\left(0\right)$.

2. Materials and Methods

The methods proposed in this paper are fundamentally design-driven: their accuracy depends not only on distributional assumptions, but also on how observations are selected, ranked, and recorded in the field (or in repeated sampling). Accordingly, we begin by specifying the study design and sampling framework that generates the minimum-based summaries on which our estimators rely, highlighting the key design parameters that control sampling effort and information content. We then introduce a general theoretical setup that abstracts these mechanisms into a probabilistic model, so that the resulting properties and comparisons of the proposed estimators can be stated and proved in a unified way.

2.1. Study Design and Sampling Framework

In what follows, we demonstrate the practical use of the proposed methodology for abundance estimation in line transect sampling. Within each set, observers employ a simple and efficient proxy to identify the unit judged to be closest to the transect line. At this stage, neither exact perpendicular distances nor a full ranking of all units are required. Instead, only the judged-closest unit is subsequently measured exactly, and these subset minima form the data used to estimate the target density at the line. The new procedure needs a relatively small set size, so ranking through distance comparisons can be performed or judged easily.

Step 1. 

Choose design settings.

Fix a reasonably small set size r, the number of sets m you will form per route or time window (in each “block”), and how many independent routes S you will run. This design yields $n=S.m.r$ detections encountered (not all measured) and $M=S.m$ exact distances measured (one per set).

Step 2. 

Form disjoint sets in the field (block $t=1,2,\dots ,S$). As observers traverse a line, group the next fixed number of successive detections into one set; start a new set for the next group of successive detections, and so on. Do not recycle detections across sets or routes (windows). Therefore, along a given route t, group r successive detections into a set; repeat m times to create m non-overlapping sets. Keep routes so that groups of detections do not overlap in space or time [2].

Step 3. 

Rank efficiently using a cheap indicator. Within each set, use a rapid cheap proxy (e.g., sighting angle, apparent range, cue strength, visibility class, drone image cue) to select the single unit judged closest to the line. No exact perpendicular distances yet; no full ordering is required.

Step 4. 

Measure only the one judged closest. In each set, measure the exact perpendicular distance for the judged-closest unit only; do not measure the rest. This yields one measured value from that set with minimal effort. For the purposes of theoretical development, we consider this minimum as the true minimum representing the perfect ranking. After m sets, a route contributes m measured distances; after S routes, the study collects $M=S.m$ measurements.

Step 5. 

Repeat across the survey. Continue Step 2–Step 4 until forming m sets for the current route and recording one exact distance per set. Then move to the next route until you complete the planned number of sets for the route. Repeat on independent routes to maintain independence. For comparison with conventional line transect sampling, these steps can be repeated c times to obtain the desired sample size.

Step 6. 

Extend the procedure for enhanced efficiency. To produce more efficient estimators, the exact measurement of the unit judged closest in Step 4 can be temporarily withheld. Instead, mark the identified minima without recording their distances, keeping them available for subsequent comparisons across different sets. To facilitate accurate judgment by the observer, flags or markers can be placed on these units. Then, perform a visual comparison of the marked units within each block to select the single unit judged closest to the line, and record its exact perpendicular distance. This procedure should be repeated at the block level to achieve the desired sample size.

Step 7. 

Record keeping. For each set, store the route ID, set ID, quick ranks, which unit was measured, and the exact perpendicular distance. This ensures transparent auditing and reproducibility.

The following example illustrates the proposed set-based sampling construction and clarifies the notation used for the minimum order statistics.

Example 1

(Field construction with perpendicular distances). Throughout, let r denote the set size (number of consecutive detections per set), m the number of sets per block (per route or time window), and S the number of independent blocks. In this example, we take $r=5$, $m=3$, and $S=2$.

Block $t=1$. As the crew walks a transect, detections are grouped into $m=3$ non-overlapping sets of $r=5$ consecutive detections each. Let $X_{i,j}^{\left(t\right)}$ denote the perpendicular distance of the ith detection in set j on block t (labels only; no numbers are needed). The raw sets on block $t=1$ are

$$Se{t}_1^{\left(1\right)}=\left[\begin{array}{c}{X}_{1,1}^{\left(1\right)}\\ X_{2,1}^{\left(1\right)}\\ ⋮\\ X_{5,1}^{\left(1\right)}\end{array}\right],Se{t}_2^{\left(1\right)}=\left[\begin{array}{c}{X}_{1,2}^{\left(1\right)}\\ X_{2,2}^{\left(1\right)}\\ ⋮\\ X_{5,2}^{\left(1\right)}\end{array}\right],Se{t}_3^{\left(1\right)}=\left[\begin{array}{c}{X}_{1,3}^{\left(1\right)}\\ X_{2,3}^{\left(1\right)}\\ ⋮\\ X_{5,3}^{\left(1\right)}\end{array}\right].$$

Within each set, observers use a rapid low-cost visual feature to identify the single unit judged closest to the line; only that unit’s perpendicular distance is then measured exactly. Denote the one measured value from set j on block $t=1$ by $X_{\left(1\right)j}^{\left(1\right)}$, $j=1,2,3$.

Block $t=2$. On an independent pass (different route or time window), the same construction yields three new sets $S_1^{\left(2\right)},S_2^{\left(2\right)},S_3^{\left(2\right)}$ and one exact perpendicular distance per set, $X_{\left(1\right)1}^{\left(2\right)},X_{\left(1\right)2}^{\left(2\right)},X_{\left(1\right)3}^{\left(2\right)}.$

Across the two blocks we encounter $n=Smr=2\times 3\times 5=30$ detections (all counted), but we measure exactly one perpendicular distance per set, producing $M=Sm=6$ exact measurements:

$$\{X_{\left(1\right)1}^{\left(1\right)},X_{\left(1\right)2}^{\left(1\right)},X_{\left(1\right)3}^{\left(1\right)},X_{\left(1\right)1}^{\left(2\right)},X_{\left(1\right)2}^{\left(2\right)},X_{\left(1\right)3}^{\left(2\right)}\}.$$

These measurements are used to construct the first type of our estimators.

To enhance estimator efficiency, the exact measurement of the unit judged closest in Step 4 can be temporarily withheld. Denote the initially identified minima in set j on block t by $X_{\left(1\right)j}^{\left(t\right)};j=1,\dots ,m$, which are marked but not yet measured. These marked units are kept available for visual comparison across sets. To facilitate accurate judgment, flags or markers can be placed on them. Then, within each block the observer performs a visual comparison of the marked units to select the single unit judged closest to the line, whose exact perpendicular distance is then recorded and denoted $X_{\left(1\right)\left(1\right)}^{\left(t\right)}$. This procedure is repeated at the block level to achieve the desired number of exact measurements.

From a practical point of view, it is crucial that the subset size r remains identical across all sets within all blocks, while the number of blocks may be chosen as needed to achieve the desired sample size. Because sets within a block do not overlap and blocks are disjoint in space or time, the M measured perpendicular distances are obtained under a design that ensures independence both across sets and across blocks. This structure yields i.i.d. minimum order statistics, which form the basis of the estimation procedure proposed in this paper.

Relative to the above procedure of element selection, two estimation methods are proposed in this paper for estimating $f\left(0\right)$ and, subsequently, the abundance D. The first method applies estimation based on the set of minimums $\left\{X_{\left(1\right)j}^{\left(t\right)}:j=1,\dots m;t=1,\dots ,S\right\}.$ We refer to this as the first stage of selection. The second method requires an additional step at the block level by selecting the minimum of these minimums. The produced elements are $\left\{X_{\left(1\right)\left(1\right)}^{\left(1\right)},X_{\left(1\right)\left(1\right)}^{\left(2\right)},\dots ,X_{\left(1\right)\left(1\right)}^{\left(S\right)}\right\}$, where each element is defined as $X_{\left(1\right)\left(1\right)}^{\left(t\right)}=\mathrm{Min}\left\{X_{\left(1\right)1}^{\left(t\right)},X_{\left(1\right)2}^{\left(t\right)},\dots ,X_{\left(1\right)m}^{\left(t\right)}\right\};$ for $t=1,\dots ,S$ blocks. We refer to this as the second stage of selection.

2.2. General Theoretical Setup

Let $X_1,\dots ,X_r$ be i.i.d. perpendicular distances with right-continuous density f on $[0,w]$ with absolutely continuous cumulative distribution function (cdf) F, where $w>0$ denotes the truncation distance (i.e., the strip half-width). Then, the minimum order statistics $X_{\left(1\right)j}^{\left(t\right)}:j=1,\dots ,m;t=1,\dots ,S$ are i.i.d. with cdf $F_{1:r}\left(x\right)$ and pdf $f_{1:r}\left(x\right)$, such that

$$f_{1:r}\left(x\right)=C_{1,r}f\left(x\right){[1-F\left(x\right)]}^{r-1},$$

where $C_{1,r}=r!/(r-1)!=r$. Accordingly, the minima of these minimum order statistics $X_{\left(1\right)\left(1\right)}^{\left(t\right)}$ are also i.i.d. with cdf $G_{1:m}\left(x\right)$ and pdf $g_{1:m}\left(x\right)$, such that

$$g_{1:m}\left(x\right)=C_{1,m}{f}_{1:r}\left(x\right){[1-F_{1:r}\left(x\right)]}^{m-1},$$

where $C_{1,m}=m$.

The following lemma, which is analogous to Garg et al. [12], can be applied respectively on $X_{\left(1\right)j}^{\left(t\right)}$ and $X_{\left(1\right)\left(1\right)}^{\left(t\right)}$, the first- and second-stage minimum perpendicular distances, as follows. Suppose that the minimum perpendicular distances, denoted by ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}$, are obtained from either the first- or second-stage sampling procedures described in Section 1.

Lemma 1.

Let ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(1\right)},{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(2\right)},\dots ,{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(S\right)}$ denote i.i.d. copies of the minimum order statistic arising from samples of size η, whose parent distribution has continuous pdf $f\left(x\right)$ and cdf $F\left(x\right)$. Then, as $\eta \to \infty$, for any fixed block t:

1. 

$(\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]⟶\frac{1}{f\left(0\right)}$.

2. 

If f is twice differentiable in a neighborhood of 0 and $f^{″}$ is continuous at 0 then $(\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]⟶\frac{1}{f\left(0\right)}+O\left(\frac{1}{\eta }\right).$

3. 

$\mathbb{V}ar\left[(\eta +1){\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]⟶\frac{1}{f^2\left(0\right)}.$

Proof. 

Define $U_i\stackrel{d}{=}F\left(X_i\right)$, so that $U_1,\dots ,U_m$ are i.i.d. standard uniform random variables. Let $U_{\left(1\right)}=min\{U_1,\dots ,U_{\eta }\}$; then, $U_{\left(1\right)}\sim \mathrm{Beta}(1,\eta )$ with $\mathbb{E}\left[U_{\left(1\right)}\right]=\frac{1}{\eta +1}$ (see David and Nagaraja [18]) (p. 14)). Moreover, assume $Q(.)=F^{(-1)}(.)$ is the quantile function; thus, ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}=Q\left(U_{\left(1\right)}\right)$ by the monotonicity of $F(.)$.

The inverse-function identity $Q^{\prime }\left(u\right)=1/f\left(Q\left(u\right)\right)$ yields $\underset{u\downarrow 0}{lim}\frac{Q\left(u\right)}{u}=\frac{1}{f\left(0\right)}.$

Equivalently, there exists a function $r\left(u\right)$ with $r\left(u\right)=o\left(u\right)$ as $u\downarrow 0$, such that

$$Q\left(u\right)=\frac{u}{f\left(0\right)}+r\left(u\right).$$

(2)

Using ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}=Q\left(U_{\left(1\right)}\right)$ and (2),

$$\begin{array}{cc}\hfill (\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]& =\frac{1}{f\left(0\right)}+(\eta +1)\mathbb{E}\left[r\left(U_{\left(1\right)}\right)\right].\hfill \end{array}$$

Select an arbitrary $\epsilon >0$. Since $r\left(u\right)=o\left(u\right)$, choose $\delta \in (0,1)$, such that $\left|r\right(u\left)\right|\le \epsilon u$ for all $u\in (0,\delta ]$. Therefore,

$$\begin{array}{cc}\hfill (\eta +1)\mathbb{E}\left[\left|r\right(U_{\left(1\right)}\left)\right|\right]& \le (\eta +1)\mathbb{E}\left[\epsilon U_{\left(1\right)}\mathbf{1}\{U_{\left(1\right)}\le \delta \}\right]+(\eta +1)\mathbb{E}\left[|r\left(U_{\left(1\right)}\right)|\mathbf{1}\{U_{\left(1\right)}>\delta \}\right]\hfill \\ \hfill & \le \epsilon (\eta +1)\mathbb{E}\left[U_{\left(1\right)}\right]+(\eta +1){\parallel r\parallel }_{[\delta ,1]}\mathbb{P}(U_{\left(1\right)}>\delta ),\hfill \end{array}$$

where ${\parallel r\parallel }_{[\delta ,1]}=\underset{u\in [\delta ,1]}{\sup }\left|r\left(u\right)\right|<\infty$ by continuity of Q on $[\delta ,1]$. Now $\mathbb{P}(U_{\left(1\right)}>\delta )={(1-\delta )}^{\eta }$, so $(\eta +1){(1-\delta )}^{\eta }\to 0$ exponentially fast. Therefore

$$\underset{\eta \to \infty }{lim}\sup (\eta +1)\mathbb{E}\left[\right|r\left(U_{\left(1\right)}\right)\left|\right]\le \epsilon .$$

Letting $\epsilon \downarrow 0$ gives $(\eta +1)\mathbb{E}\left[r\left(U_{\left(1\right)}\right)\right]\to 0$. Thus,

$$\begin{array}{cc}\hfill (\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]=\frac{1}{f\left(0\right)}+o\left(1\right)& ⟹(\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]⟶\frac{1}{f\left(0\right)}\hfill \end{array}$$

Hint: If a higher-order bias expression is required, such as the second-order bias, under the conditions $f^{″}\left(0\right)=0$ (which follows from the condition commonly referred to as the shoulder condition in line transect sampling) and if $f^{″}$ exists then

$$\begin{array}{cc}\hfill (\eta +1)\mathbb{E}\left[{\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\right]& =\frac{1}{f\left(0\right)}+\frac{f^{″}\left(0\right)}{f^4\left(0\right)}\frac{1}{(\eta +2)(\eta +3)}+O\left(\frac{1}{{\eta }^3}\right)\hfill \\ \hfill & =\frac{1}{f\left(0\right)}+O\left(\frac{1}{{\eta }^2}\right).\hfill \end{array}$$

This completes the proofs of parts (1) and (2). For part (3), Poisson approximation is employed. Hence, $(\eta +1){\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}\Rightarrow \mathrm{Exp}\left(f\left(0\right)\right)$, and it follows that limiting variance is $1/\left(f^2\left(0\right)\right)$.    □

Lemma 1 is derived under the assumption of perfect identification of the minimum within each set, which represents an idealized sampling scenario. In practical field applications, particularly those relying on rapid visual assessment, occasional misclassification of the closest unit may occur. As discussed in the preceding Section 2.3, the probability of correctly identifying the minimum remains high for moderate separation margins, and the difficulty of minimum-only ranking is substantially lower than that of full ranking. Moreover, the robustness study in Section 3.2.3 demonstrates that moderate departures from perfect ranking primarily result in increased variability rather than severe bias inflation. These findings indicate that the theoretical results provide a reliable approximation under realistic field conditions, even when perfect ranking cannot be guaranteed.

Proposition 1.

1. The first-stage minimum order statistics satisfy Lemma 1 by setting ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}=X_{\left(1\right)j}^{\left(t\right)}$ with $\eta =r$.

2. 

The second-stage minimum order statistics satisfy Lemma 1, where ${\mathcal{X}}_{\left(\mathbf{1}\right)}^{\left(t\right)}=X_{\left(1\right)\left(1\right)}^{\left(t\right)}$, and the following hold:

(i) 

$(m+1)\mathbb{E}\left[X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f_{1:m}\left(0\right)},$ which consequently yields $\left(r(m+1)\right)\mathbb{E}\left[X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f\left(0\right)}.$

(ii) 

If $f_{1:m}\left(x\right)$ is twice differentiable in a neighborhood of 0 and $f_{1:m}^{″}$ is continuous at 0 then $\left(r(m+1)\right)\mathbb{E}\left[X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f_{i:m}\left(0\right)}+O\left(\frac{1}{r^2}\right)$, which leads to $\left(r(m+1)\right)\mathbb{E}\left[X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f\left(0\right)}+O\left(\frac{1}{m^2}\right).$

(iii) 

$\mathbb{V}\mathrm{ar}\left[(m+1)X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f_{1:m}^2\left(0\right)},$ which in turn implies $\mathbb{V}\mathrm{ar}\left[r(m+1)X_{\left(1\right)\left(1\right)}^{\left(t\right)}\right]⟶\frac{1}{f^2\left(0\right)}.$

2.3. Challenge of Ranking

The accuracy of our forthcoming estimators for the boundary density $f\left(0\right)$ and, hence, for abundance D, depends on the feasibility of performing accurate and inexpensive ranking of the minima. In this section, we show that minimum perfect ranking can, in some sense, be achieved with relative ease in the first stage, and that it is even simpler in the second stage. This property is essential for the practical usefulness of our new estimation method.

To formalize the ranking difficulty, we adopt a margin-based criterion. The two units $u_1$ and $u_2$ can be correctly ordered if $|u_1-u_2|>\epsilon$ for some preassigned positive $\epsilon$ [19]. Extending this idea, Al-Saleh and Al Kadiri [20] defined a set $u_1,\dots ,u_m$ as perfectly rankable if $[\underset{i\ne j}{min}|u_i-u_j|>\epsilon ],$ so that all pairwise differences exceed the margin.

For random variables $U_1,\dots ,U_m$, the associated probability of full perfect ranking is given by

$$P_m^{\left(k\right)}=Pr\left(\underset{i\ne j}{min}|U_i-U_j|>\epsilon \right),$$

where $k=1,2$ corresponds to the first and second sampling stages.

Table 1 illustrates these probabilities for the standard exponential distribution, as an example, showing that $P_m^{\left(2\right)}>P_m^{\left(1\right)}$ for $m=3,5$ and that the values decrease as $\epsilon$ grows.

Table 1. Illustrative values of full perfectly ranking $P_m^{\left(1\right)}$ and $P_m^{\left(2\right)}$ under the standard exponential distribution. (Source: [20]).

m	$\mathit{\epsilon }$	${\mathit{P}}_{\mathit{m}}^{\left(1\right)}$	${\mathit{P}}_{\mathit{m}}^{\left(2\right)}$	m	$\mathit{\epsilon }$	${\mathit{P}}_{\mathit{m}}^{\left(1\right)}$	${\mathit{P}}_{\mathit{m}}^{\left(2\right)}$
3	0.1	0.7300	0.8008	5	0.1	0.3664	0.4327
	0.5	0.2232	0.2602		0.2	0.1349	0.1508
	1.1	0.0313	0.0361		0.5	0.0019	0.0069

In the new procedure developed in this paper, the practically relevant ranking task is to distinguish only the closest unit from the rest by a preassigned margin $\epsilon >0$. This requirement is strictly easier than “full” perfect ranking, which demands that all pairwise separations exceed $\epsilon$, and, thus, it yields to higher probabilities.

Definition 1

(Minimum-only Separability). Let $U_1,\dots ,U_m$ be random variables with order statistics $U_{\left(1\right)}\le \dots \le U_{\left(m\right)}$. We say the set is minimum-only perfectly rankable at margin ε if $U_{\left(2\right)}-U_{\left(1\right)}>\epsilon .$ For stage $k\in \{1,2\}$ we define the corresponding probability by

$$P_m^{min\left(k\right)}\left(\epsilon \right)=Pr\left(U_{\left(2\right)}-U_{\left(1\right)}>\epsilon \right).$$

Since full perfect ranking implies minimum-only separability, we have $P_m^{\left(k\right)}\left(\epsilon \right)\le P_m^{min\left(k\right)}\left(\epsilon \right)$ for each k and m.

For minimum-only identification, it suffices to ensure that the smallest element is at least $\epsilon$ below all the others. Since $U_{\left(2\right)}$ is the closest competitor to $U_{\left(1\right)}$, we have

$$U_{\left(2\right)}-U_{\left(1\right)}\le U_{\left(j\right)}-U_{\left(1\right)},\mathrm{for}\mathrm{all}j\ge 2.$$

Consequently, the following statements are equivalent:

$$U_{\left(2\right)}-U_{\left(1\right)}>\epsilon ⟺\underset{j\ge 2}{min}\{U_{\left(j\right)}-U_{\left(1\right)}\}>\epsilon ⟺U_{\left(1\right)}+\epsilon <\underset{j\ge 2}{min}{U}_{\left(j\right)}.$$

Hence, the condition $U_{\left(2\right)}-U_{\left(1\right)}>\epsilon$ is sufficient for minimum-only separability at margin $\epsilon$, and no additional pairwise comparisons are required. When the cdf F is continuous, ties occur with probability zero, so strict inequalities are well-defined.

In the following example, we illustrate Definition 1 in the context of the two sampling stages introduced above.

Example 2.

1. 

Ranking minimum in the first stage: If $X_1,\dots ,X_m$ are i.i.d. random variables with pdf f and cdf F then the probability of first stage minimum-only separability at margin ε is

$$P_m^{min\left(1\right)}\left(\epsilon \right)=m\int f\left(x\right){\left[1-F(x+\epsilon )\right]}^{m-1}dx.$$

2. 

Ranking minimum in the second stage: Let $Z_j\sim F_{j:m}$ denote the jth order statistic from an i.i.d. size-m sample with cdf F. In addition, assume $(Z_1,\dots ,Z_m)$ are independent across j (e.g., they arise from different sets or blocks). Then, the probability of second stage separability is

$$P_m^{min\left(2\right)}\left(\epsilon \right)=\int f_{1:m}\left(x\right)\prod _{j=2}^m\left[1-F_{j:m}(x+\epsilon )\right]dx.$$

For Exp(1) density with sample size $m=2$, we have

$$P_2^{min\left(1\right)}\left(\epsilon \right)=e^{-\epsilon },P_2^{min\left(2\right)}\left(\epsilon \right)=\frac{4}{3}{e}^{-\epsilon }-\frac{1}{2}{e}^{-2\epsilon }.$$

For $m>2$, $P_m^{min\left(1\right)}\left(\epsilon \right)=e^{-(m-1)\epsilon }$, while $P_m^{min\left(2\right)}\left(\epsilon \right)$ typically does not admit a simple closed form and must be evaluated numerically. Another example is the standard Half-Normal distribution, for which the corresponding minimum-only separability probabilities are also obtained numerically.

Table 2 presents the numerical values of $P_m^{min,\left(1\right)}\left(\epsilon \right)$ and $P_m^{min,\left(2\right)}\left(\epsilon \right)$ for the $\mathrm{Exp}\left(1\right)$ and Half-Normal$(0,1)$ distributions with $m=3$ and $m=5$. The results indicate that $P_m^{min,\left(2\right)}\left(\epsilon \right)$ consistently exceeds $P_m^{min,\left(1\right)}\left(\epsilon \right)$ for both sample sizes, and that the probabilities vary as a function of $\epsilon$. Furthermore, in comparison with Table 1, which reports the probabilities of full perfect ranking under the same distribution, it is evident that the probability of correctly ranking the minimum observation is uniformly greater than its full perfect ranking counterpart. This indicates that ranking the minimum elements is easier than performing a full ranking of all elements. Moreover, ranking the minima in the second-stage sampling yields a more favorable outcome than ranking the minima in the first-stage sampling.

Table 2. Values of minimum-only probability of correct identification $P_m^{min,\left(1\right)}\left(\epsilon \right)$ and $P_m^{min,\left(2\right)}\left(\epsilon \right)$ for the standard Exponential$\left(1\right)$ and Half-Normal$(0,1)$ distributions.

Dist.	m	$\mathit{\epsilon }$	${\mathit{P}}_{\mathit{m}}^{min,\left(1\right)}\left(\mathit{\epsilon }\right)$	${\mathit{P}}_{\mathit{m}}^{min,\left(2\right)}\left(\mathit{\epsilon }\right)$	m	$\mathit{\epsilon }$	${\mathit{P}}_{\mathit{m}}^{min,\left(1\right)}\left(\mathit{\epsilon }\right)$	${\mathit{P}}_{\mathit{m}}^{min,\left(2\right)}\left(\mathit{\epsilon }\right)$
Exp$\left(1\right)$	3	0.1	0.8198	0.8617	5	0.1	0.6678	0.7259
		0.5	0.3662	0.4159		0.2	0.4456	0.4947
		0.9	0.1633	0.1681		0.5	0.1130	0.1359
Half-Normal$(0,1)$	3	0.1	0.8082	0.8540	5	0.1	0.6769	0.7379
		0.5	0.3051	0.3632		0.2	0.4458	0.5076
		0.9	0.0907	0.0987		0.5	0.1105	0.0991

In practical field surveys, identification of the judged-closest unit may be subject to observer error due to imperfect visual cues, occlusion, or time constraints. Such judgment error may lead to occasional misranking within a set, where the selected unit is not the true minimum. However, the proposed methodology relies only on minimum-only ranking rather than full ranking, which substantially reduces the difficulty of correct identification, as discussed above. As shown by the probabilities in this section, Section 2.3, the likelihood of correctly identifying the minimum remains high even under moderate separation margins. From a statistical perspective, imperfect ranking primarily increases variability rather than introducing systematic bias, provided that misranking occurs symmetrically and with bounded probability. This robustness motivates the use of low-cost visual proxies in rapid field implementations.

3. Results

This section reports the main outcomes of the study. We first present the proposed minimum-based estimators and summarize their implementation in a unified procedure, emphasizing how the design parameters control the information extracted from repeated sampling. We then provide the associated theoretical and empirical results—covering analytic properties, simulation-based performance, and the applied study—to quantify the accuracy, efficiency, and practical robustness across the considered settings.

3.1. Theoretical Results

3.1.1. Proposed Estimators

Using the sampling framework of Section 1 as a foundation, this section proposes two complementary methodologies to estimate the boundary density $f\left(0\right)$ and, hence, the population abundance D. We examine their key properties, including bias, variance, and asymptotic behavior, extending classical order statistic approaches through the distributional structure of minimum samples.

Stage I constructs minimum-based summaries from repeated samples by selecting and retaining the relevant minima under the prescribed sampling design, thereby producing the core statistics that drive the subsequent estimation.

Let $X_{\left(1\right)j}^{\left(t\right)}$ denote the minimum observation within the $j^{th}$ set of block t, for $j=1,\dots ,m$, $t=1,\dots ,S$. The collection $\left\{X_{\left(1\right)j}^{\left(t\right)}\right\}$ consists of i.i.d. copies of the minimum order statistics from samples of size r, with pdf $f_{1:r}\left(x\right)=rf\left(x\right){[1-F\left(x\right)]}^{r-1}$ and cdf $F_{1:r}\left(x\right)$. Here, r denotes the set size, m the number of subsets within each block and S the number of independent blocks, as defined in Section 1. By setting $M=Sm$, we define the empirical mean of these units as

$${\overline{X}}^{*}=\frac{1}{M}\sum _{t=1}^S\sum _{j=1}^m{X}_{\left(1\right)j}^{\left(t\right)}.$$

(3)

Motivated by Proposition 1, our first-stage estimators of the boundary density for ${f^{-1}}_1\left(0\right)$ and $f_1\left(0\right)$ are, respectively:

$${\widehat{f^{-1}}}_1\left(0\right)=(r+1){\overline{X}}^{*},\mathrm{and}{\widehat{f}}_1\left(0\right)=\frac{1}{{\widehat{f^{-1}}}_1\left(0\right)}.$$

(4)

Hint: under the mild regularity conditions satisfied in our setting, the reciprocal of a consistent and asymptotically normal estimator remains consistent and asymptotically normal [11]. Since our primary estimator $\widehat{\theta }$ targets $1/f\left(0\right)$ and possesses these properties, the inverted estimator ${\widehat{f}}_1\left(0\right)=1/\widehat{\theta }$ inherits consistency and asymptotic normality (see Proposition 2).

The corresponding abundance estimator with observed count M and transect length L is

$${\widehat{D}}_1=\frac{M}{2L}\frac{1}{(r+1){\overline{X}}^{*}}.$$

(5)

We now proceed to Stage II, where these minimum-based summaries are combined to yield the final estimator. For each block t, we define the block representative—that is, the block minimum-of-minima—as 

$$X_{\left(1\right)\left(1\right)}^{\left(t\right)}=min\left\{X_{\left(1\right)1}^{\left(t\right)},\dots ,X_{\left(1\right)m}^{\left(t\right)}\right\},t=1,\dots ,S,$$

which are i.i.d. with pdf $g_{1:m}\left(x\right)=m{f}_{1:r}\left(x\right){[1-F_{1:r}\left(x\right)]}^{m-1}$ and cdf $G_{1:m}\left(x\right)$. We define

$${\overline{X}}^{**}=\frac{1}{S}\sum _{t=1}^S{X}_{\left(1\right)\left(1\right)}^{\left(t\right)}.$$

(6)

Using Proposition 1, hence we define the second-stage estimator

$${\widehat{f^{-1}}}_2\left(0\right)=r(m+1){\overline{X}}^{**},{\widehat{f}}_2\left(0\right)=\frac{1}{{\widehat{f^{-1}}}_2\left(0\right)}.$$

(7)

Thus, the abundance estimate is

$${\widehat{D}}_2=\frac{S}{2L}\frac{1}{r(m+1){\overline{X}}^{**}}.$$

(8)

Relative to the first-stage approach, this second-stage method compresses information within each block by considering only the block-minimum statistics. This results in a smaller effective sample size (S instead of $Sm$) but yields a sharper functional dependence on $(r,m)$. Intuitively, the “min-of-minima” amplifies information about the boundary density by pushing samples closer to zero, at the cost of increased variability. Thus, ${\widehat{f}}_2\left(0\right)$ and ${\widehat{D}}_2$ highlight a trade-off between efficiency and variance reduction, depending on how r and m are tuned.

3.1.2. Improving Density Estimation via Nearest-Neighbor

From a general density-estimation perspective, minimum order statistics can be naturally interpreted as nearest-neighbor distances to a target point. In particular, estimating the boundary density $f\left(0\right)$ in line transect sampling can be viewed as a pointwise density-estimation problem, where the most informative observations are those closest to the transect line. Nearest-neighbor methods are well known for their stability and low tuning requirements in such settings, especially near boundaries. In this context, an analogous viewpoint arises in nearest-neighbor methods evaluated at a fixed point. Kung et al. [11] and Garg et al. [12] showed that, under mild regularity conditions, splitting the sample into subsets and averaging absolute distances yields a consistent estimator of the density $f\left(x\right)$. These results directly motivated the following estimation method, which adapts the minimum-distance principle to the line transect setting, targeting $f\left(0\right)$ as the key quantity and, consequently, the abundance density D.

Suppose $X_1,\dots ,X_N$ are i.i.d. perpendicular distances with pdf f, where we seek the value of f at a particular location $x_0$. A convenient way to recast this pointwise task is to work with distances to the target: $Y_i=|X_i-x_0|.$ The variables $Y_i$ are non-negative with density $g\left(y\right)=f(x_0+y)+f(x_0-y);y\ge 0,$ so $f\left(x_0\right)=\frac{1}{2}g\left(0\right).$ In other words, estimating the mass of the folded distribution at zero, $g\left(0\right)$, immediately yields the desired $f\left(x^{★}\right)$. A bonus of this reduction is that g is typically smoother at zero than f is at $x_0$, which stabilizes estimation at the point of interest [12].

From a practical point of view, the analyst chooses a target point $x_0$ at which the density $f\left(x_0\right)$ is to be estimated. Two practically motivated types dominate near the transect: (i) blind-zone anchors and (ii) anti-heaping anchors. A blind-zone anchor sets $x_0>0$, where it is the smallest distance at which detections are reliably made (e.g., beyond a vehicle hood, boat rail, or dense verge). In practice, $x_0$ is defined from a short pilot or calibration pass as the minimal distance with stable detection and measurement—often the maximum of the empirically observed blind width and twice the instrument’s rounding resolution. An anti-heaping anchor sets $x_0>0$ to avoid known rounding bins that inflate $f\left(0\right)$; here, $x^{★}$ is chosen from small, non-heaped offsets (e.g., $0.20$, $0.50$, $1.0$ m), typically guided by a fine near-zero histogram or rug plot. In both cases the minimum local distance mechanics are identical—only the chosen $x_0$ changes.

To produce new density estimators, return to the raw data $X_1,\dots ,X_M$; $M=rm$, from density f before forming $Y_i=|X_i-x_0|$. Randomly partition the data into m subsets of size r each, and repeat the entire procedure S times (block level). At the block level and for each m subsets, let ${\mathbb{D}}_{\left(1\right)j}^{\left(t\right)}(x_0;r)=\underset{1\le j\le m}{min}{\left\{Y_1,\dots ,Y_r\right\}}_j$ be x’s nearest-neighbor distance computed within the $j^{th}$ subset. For each block t, define ${\mathbb{D}}_{\left(1\right)\left(1\right)}^{\left(t\right)}(x_0;m)=\underset{1\le t\le m}{min}{\left\{{\mathbb{D}}_{\left(1\right)1}^{\left(t\right)},\dots ,{\mathbb{D}}_{\left(1\right)m}^{\left(t\right)}\right\}}_t.$

The quantity labeled ${\overline{X}}^{*}$ in (3) is simply the mean of these ${\mathbb{D}}_{\left(1\right)j}^{\left(t\right)}(x_0;r)$ over j values; that is:

$${\overline{\mathbb{D}}}^{*}\equiv {\overline{X}}^{*}=\frac{1}{M}\sum _{t=1}^S\sum _{j=1}^m{\mathbb{D}}_{\left(1\right)j}^{\left(t\right)}.$$

(9)

Similarly, ${\overline{X}}^{**}$ in (6) is also the mean of these ${\mathbb{D}}_{\left(1\right)\left(1\right)}^{\left(t\right)}(x_0;m)$ across the t blocks.

$${\overline{\mathbb{D}}}^{**}\equiv {\overline{X}}^{**}=\frac{1}{S}\sum _{t=1}^S{\mathbb{D}}_{\left(1\right)\left(1\right)}^{\left(t\right)}.$$

(10)

Thus, two additional estimators can be constructed from

$${\widehat{f^{-1}}}_3\left(0\right)=2(r+1){\overline{\mathbb{D}}}^{*}\mathrm{and}{\widehat{f^{-1}}}_4\left(0\right)=2r(m+1){\overline{\mathbb{D}}}^{**}$$

(11)

that are

$${\widehat{f}}_3\left(0\right)=1/{\widehat{f^{-1}}}_3\left(0\right);\mathrm{and}{\widehat{f}}_4\left(0\right)=1/{\widehat{f^{-1}}}_4\left(0\right).$$

(12)

3.1.3. Large Sample Behavior and Convergence of Minima-Based Estimators

Let ${\mu }_{\left(1\right)}=\mathbb{E}\left[X_{\left(1\right)}\right]$ and ${\sigma }_{\left(1\right)}^2=Var\left(X_{\left(1\right)}\right)$; let ${\mu }_{\left(1\right)\left(1\right)}=\mathbb{E}\left[X_{\left(1\right)\left(1\right)}\right]$ and ${\sigma }_{\left(1\right)\left(1\right)}^2=Var\left(X_{\left(1\right)\left(1\right)}\right)$. Under the standing assumptions the density f is continuous at 0 with $f\left(0\right)>0$ and finite derivative $f^{\prime }\left(0\right)$, and it is twice continuously differentiable. For an interior target $x_0>0$, we assume $f\left(x_0\right)>0$ and that $f^{″}\left(x_0\right)$ is finite. Blocks and sets are independent across the design, and within each set of size r, the observations are i.i.d. Thus, we establish the following propositions.

Proposition 2.

Assume that the sizes, $r,m$ and S are as defined in Section 1 and set $M=Sm$. Then:

(i) 

If $r\to \infty$ and $M=Sm\to \infty$ with $r=o\left(M\right)$ then

$$(r+1){\mu }_{\left(1\right)}\to \frac{1}{f\left(0\right)},r(m+1){\mu }_{\left(1\right)\left(1\right)}\to \frac{1}{f\left(0\right)},$$

and

$$\sqrt{M}\left({\widehat{f^{-1}}}_1\left(0\right)-(r+1){\mu }_{\left(1\right)}\right)\stackrel{d}{\to }\mathcal{N}\left(0,{(r+1)}^2{\sigma }_{\left(1\right)}^2\right),$$

$$\sqrt{S}\left({\widehat{f^{-1}}}_2\left(0\right)-r(m+1){\mu }_{\left(1\right)\left(1\right)}\right)\stackrel{d}{\to }\mathcal{N}\left(0,r^2{(m+1)}^2{\sigma }_{\left(1\right)\left(1\right)}^2\right).$$

Consequently, ${\widehat{f}}_k\left(0\right)\stackrel{p}{\to }f\left(0\right)$ for $k=1,2$.

(ii) 

If r- and m for the second stage-are fixed while both $M\to \infty$ and $S\to \infty$ then

$${\widehat{f^{-1}}}_1\left(0\right)\stackrel{p}{\to }(r+1){\mu }_{\left(1\right)},{\widehat{f^{-1}}}_2\left(0\right)\stackrel{p}{\to }r(m+1){\mu }_{\left(1\right)\left(1\right)}.$$

Both the first- and second-stage estimators are asymptotically unbiased for their limits and attain variances of order $O\left(M^{-1}\right)$ and $O\left(S^{-1}\right)$, respectively. When f is differentiable at zero, the bias relative to $1/f\left(0\right)$ decays at rate $O\left(r^{-1}\right)$ and rate $O\left({\left(rm\right)}^{-1}\right)$ for the two estimators, respectively; see Lemma 1.

Proposition 3

(Standard error estimation). Let ${\widehat{\sigma }}_{\left(1\right)}^2$ be the sample variance of the M values of $X_{\left(1\right)j}^{\left(t\right)}$ and let ${\widehat{\sigma }}_{\left(1\right)\left(1\right)}^2$ be the sample variance of $X_{\left(1\right)\left(1\right)}^{\left(t\right)}$. Then

$$SE\left\{{\widehat{f^{-1}}}_1\left(0\right)\right\}\approx \frac{r+1}{\sqrt{M}}{\widehat{\sigma }}_{\left(1\right)},SE\left\{{\widehat{f^{-1}}}_2\left(0\right)\right\}\approx \frac{r(m+1)}{\sqrt{S}}{\widehat{\sigma }}_{\left(1\right)\left(1\right)}.$$

Accordingly,

$$SE\left\{{\widehat{f}}_1\left(0\right)\right\}\approx \frac{SE\left\{{\widehat{f^{-1}}}_1\left(0\right)\right\}}{{\widehat{f^{-1}}}_1{\left(0\right)}^2},SE\left\{{\widehat{f}}_2\left(0\right)\right\}\approx \frac{SE\left\{{\widehat{f^{-1}}}_2\left(0\right)\right\}}{{\widehat{f^{-1}}}_2{\left(0\right)}^2}.$$

To compare the SE of estimators [$\widehat{f}\left(0\right)$ and ${\widehat{f^{-1}}}_1\left(0\right)$] when $r=n^{\lambda }$ and $m=n^{1-\lambda }$ for some $\lambda \in (1/5,1/3)$ leads to the SE optimal rate $O\left(M^{-4/5-\alpha }\right)$ for any non-negative $\alpha$. In smoothing the histogram, the rate is $O\left(M^{-2/3}\right)$; with the Kernel density method, the rate is $O\left(M^{-4/5}\right)$.

Proposition 4

(Rates for minima-based estimators). For a design comprising S blocks and m sets per block, the estimators ${\widehat{f}}_{\left(1\right)}\left(0\right)$ and ${\widehat{f}}_{\left(2\right)}\left(0\right)$, defined in (4) and (7), respectively, satisfy

$$Var({\widehat{f}}_{\left(1\right)})=O({(Sm)}^{-1}),Var({\widehat{f}}_{\left(2\right)})=O(S^{-1}),\mathit{and}MSE({\widehat{f}}_{\left(j\right)})=O(N_{\mathrm{eff}}^{-1}),$$

where $N_{\mathrm{eff}}=Sm$ for ${\widehat{f}}_{\left(1\right)}$ and $N_{\mathrm{eff}}=S$ for ${\widehat{f}}_{\left(2\right)}$. Similarly, the folded estimators at an interior point $x_0>0$, ${\widehat{f}}_{\left(3\right)}\left(x_0\right)$ and ${\widehat{f}}_{\left(4\right)}\left(x_0\right)$, as defined in (12), exhibit the same convergence rates with the corresponding $N_{\mathrm{eff}}$.

The proposition can be established straightforwardly once we note that ${\overline{X}}^{*}-{\mu }_{\left(1\right)}=O_p({\left(Sm\right)}^{-1/2})$ and ${\overline{X}}^{**}-{\mu }_{\left(1\right)\left(1\right)}=O_p\left(S^{-1/2}\right)$. Apply the delta method to $g\left(x\right)=c/x$ with $g^{\prime }\left(\mu \right)=-c/{\mu }^2$:

$$\begin{array}{cc}\hfill {\widehat{f}}_{\left(1\right)}-g\left({\mu }_{\left(1\right)}\right)& =g^{\prime }\left({\mu }_{\left(1\right)}\right)\left({\overline{X}}^{*}-{\mu }_{\left(1\right)}\right)+o_p\left({\left(Sm\right)}^{-1/2}\right),\hfill \\ \hfill {\widehat{f}}_{\left(2\right)}-g\left({\mu }_{\left(1\right)\left(1\right)}\right)& =g^{\prime }\left({\mu }_{\left(1\right)\left(1\right)}\right)\left({\overline{X}}^{**}-{\mu }_{\left(1\right)\left(1\right)}\right)+o_p\left(S^{-1/2}\right).\hfill \end{array}$$

Hence, $Var\left({\widehat{f}}_{\left(1\right)}\right)=\frac{c^2{\sigma }_{\left(1\right)}^2}{{\mu }_{\left(1\right)}^4}{\left(Sm\right)}^{-1}\{1+o\left(1\right)\}$ and similarly for ${\widehat{f}}_{\left(2\right)}$; the second order bias is $O\left(N_{\mathrm{eff}}^{-1}\right)$, so $MSE=O\left(N_{\mathrm{eff}}^{-1}\right)$. For ${\widehat{f}}_{\left(3\right)},{\widehat{f}}_{\left(4\right)}$, replace X by $Y=|X-x_0|$; the same argument applies and the extra factor $1/2$ only changes constants.

Remark 1

(Rates for histogram and KDE at the boundary for fair comparators). Let $N_{\mathrm{eff}}$ be the size of the reduced sample used by the comparator ($Sm$ for the first-stage level; S for the second-stage level). With bin width h and kernel bandwidth h chosen at standard optimal orders,

$$MSE({\widehat{f}}_{\mathrm{hist}}\left(0\right))=O\left(N_{\mathrm{eff}}^{-2/3}\right)\mathrm{for}h\asymp N_{\mathrm{eff}}^{-1/3},MSE({\widehat{f}}_{\mathrm{KDE}}\left(0\right))=O\left(N_{\mathrm{eff}}^{-4/5}\right)\mathrm{for}h\asymp N_{\mathrm{eff}}^{-1/5}.$$

3.2. Simulation Studies

To investigate the finite-sample performance of the developed estimators, we conducted a series of Monte Carlo simulations under both exponential and half-normal detection models. The design followed the structure of Section 1, with set size r, number of sets per block m, and number of independent blocks S varied across scenarios. Unless otherwise noted, each scenario was replicated 5000 times. All our simulations and data analyses were implemented in R (version 4.4.3). Implementation details and the corresponding R code are provided in Appendix A.

3.2.1. Boundary Estimation Under Perfect Ranking

We compared four estimators derived from subset minima as illustrated in Equations (4), (7) and (11): ${\widehat{f}}_1\left(0\right)$, ${\widehat{f}}_2\left(0\right)$ corresponded to the boundary case with perfect detection at the line and folded estimators evaluated at a small interior point $x_0=0.25$, denoted ${\widehat{f}}_3\left(x_0\right)$ and ${\widehat{f}}_4\left(x_0\right)$. Two standard nonparametric benchmarks were also included for comparison: the histogram estimator (Hist) and a boundary-corrected kernel density estimator (KDE with reflection). For fairness, all the comparators were applied to the same reduced samples ($Sm$ or S units) as the proposed methods. Thus, the effective sample sizes were $N_{\mathrm{eff}}=Sm$ for $f_1$ and $f_3$ and $N_{\mathrm{eff}}=S$ for $f_2$ and $f_4$, ensuring a like-for-like comparison that did not advantage the benchmarks by giving them access to the full raw sample of perpendicular distances.

This section focuses on boundary targets, examining the bias–variance patterns of the proposed estimators for the boundary density $f\left(0\right)$ under several detection-distance models representing both non-shoulder and shoulder behavior. Performance was evaluated in terms of Monte Carlo bias and mean squared error (MSE). Table 3 and Table 4 summarize the results. Table 3 focuses on boundary estimation $f\left(0\right)$, while Table 4 reports estimation at an interior target $x_0=0.25$ motivated by blind-zone or anti-heaping adjustments.

Table 3. Estimation of $f\left(0\right)$ using first- and second-stage methods. Results for random samples from Exponential, Half-Normal, and Hazard-Rate Distributions. True values: $f{\left(0\right)}_{\mathrm{exponential}}=\lambda =1$, $f{\left(0\right)}_{\mathrm{Half}-\mathrm{Normal}}=\sqrt{2/\pi }\approx 0.7979$, and $f{\left(0\right)}_{\mathrm{HR}}\approx 0.7386$ for Hazard-Rate$(\sigma =1,b=3)$. Boldface indicates the smallest MSE among the compared methods for the same $(r,m,S)$ setting within each distribution.

				Exponential(1)			Half-Normal(0,1)			Hazard-Rate(1,3)
$\mathit{r}$	$\mathit{m}$	$\mathit{S}$	Method	$\widehat{\mathit{f}}\left(\mathbf{0}\right)$	Bias	MSE	$\widehat{\mathit{f}}\left(\mathbf{0}\right)$	Bias	MSE	$\widehat{\mathit{f}}\left(\mathbf{0}\right)$	Bias	MSE
3	25	8	$f_1\left(0\right)$	0.7539	−0.2461	0.06335	0.7502	−0.0477	0.00437	0.7254	−0.0132	0.0020
			$f_2\left(0\right)$	1.1051	0.1051	0.21227	0.8842	0.0863	0.13120	0.8233	0.0847	0.1393
			Hist	0.8318	−0.1682	0.03533	0.7195	−0.0784	0.01314	0.6702	−0.0684	0.0113
			KDE	0.8260	−0.1740	0.03686	0.7148	−0.0831	0.01295	0.6661	−0.0725	0.0107
3	40	10	$f_1\left(0\right)$	0.7516	−0.2484	0.00631	0.7480	−0.0498	0.00352	0.7243	−0.0143	0.0011
			$f_2\left(0\right)$	1.0786	0.0786	0.14808	0.8617	0.0638	0.09172	0.8021	0.0635	0.0829
			Hist	0.8631	−0.1369	0.02385	0.7335	−0.0644	0.00896	0.6840	−0.0546	0.0073
			KDE	0.8448	−0.1552	0.02819	0.7230	−0.0749	0.00922	0.6748	−0.0638	0.0072
5	24	10	$f_1\left(0\right)$	0.8368	−0.1632	0.02960	0.7751	−0.0228	0.00254	0.7384	−0.0002	0.0017
			$f_2\left(0\right)$	1.0664	0.0664	0.14006	0.8479	0.0500	0.08737	0.7989	0.0603	0.0846
			Hist	0.7525	−0.2475	0.06402	0.6570	−0.1408	0.02270	0.6176	−0.1210	0.0176
			KDE	0.8316	−0.1684	0.03429	0.6956	−0.1023	0.01512	0.6477	−0.0909	0.0126
5	16	15	$f_1\left(0\right)$	0.8382	−0.1618	0.02914	0.7754	−0.0225	0.00248	0.7397	0.0011	0.0017
			$f_2\left(0\right)$	1.0079	0.0079	0.07961	0.8090	0.0111	0.04801	0.7563	0.0177	0.0406
			Hist	0.7515	−0.2485	0.06449	0.6576	−0.1403	0.02231	0.6200	−0.1186	0.0168
			KDE	0.8300	−0.1700	0.03489	0.6975	−0.1004	0.01467	0.6517	−0.0869	0.0118
12	10	10	$f_1\left(0\right)$	0.9342	−0.0658	0.01354	0.7985	0.0006	0.00570	0.7458	0.0072	0.0053
			$f_2\left(0\right)$	1.0216	0.0216	0.13629	0.8098	0.0119	0.07958	0.7465	0.0079	0.0666
			Hist	0.4420	−0.5580	0.31174	0.4165	−0.3814	0.14589	0.4037	−0.3349	0.1126
			KDE	0.8062	−0.1938	0.05022	0.6582	−0.1396	0.02773	0.6118	−0.1268	0.0232
13	20	13	$f_1\left(0\right)$	0.9319	−0.0681	0.00799	0.7957	−0.0022	0.00224	0.7406	0.0020	0.0019
			$f_2\left(0\right)$	1.0331	0.0331	0.09913	0.8225	0.0246	0.06082	0.7690	0.0304	0.0512
			Hist	0.5126	−0.4874	0.23784	0.4720	−0.3259	0.10657	0.4535	−0.2851	0.0816
			KDE	0.8324	−0.1676	0.03379	0.6781	−0.1198	0.01831	0.6284	−0.1102	0.0156

Notes: Histogram uses $h=c_{\mathrm{hist}}{N}_{\mathrm{eff}}^{-1/3}$; KDE uses Silverman’s rule on the reduced sample with reflection at $x=0$. Comparators use the same reduced samples as the proposed estimators: $N_{\mathrm{eff}}=Sm$ for set-minima methods and $N_{\mathrm{eff}}=S$ for block-minima methods.

Table 4. Estimation of $f\left(x_0\right)$ using first- and second-stage methods (folded at $x_0$). Results for random samples from Exponential, Half-Normal, and Hazard-Rate distributions with $x_0=0.25$. True values: $f_{\mathrm{exponential}}\left(x_0\right)=e^{-x_0}\approx 0.7788$, $f_{\mathrm{Half}-\mathrm{Normal}}\left(x_0\right)=\sqrt{2/\pi }exp(-x_0^2/2)\approx 0.7733$, and $f_{\mathrm{HR}}\left(x_0\right)\approx 0.7385$ for Hazard-Rate$(\sigma =1,b=3)$. Boldface indicates the smallest MSE among the compared methods for the same $(r,m,S)$ setting within each distribution.

				Exponential (1)			Half-Normal (0,1)			Hazard-Rate (1,3)
$\mathit{r}$	$\mathit{m}$	$\mathit{S}$	Method	$\widehat{\mathit{f}}\left({\mathit{x}}_{\mathbf{0}}\right)$	Bias	MSE	$\widehat{\mathit{f}}\left({\mathit{x}}_{\mathbf{0}}\right)$	Bias	MSE	$\widehat{\mathit{f}}\left({\mathit{x}}_{\mathbf{0}}\right)$	Bias	MSE
3	40	10	$f_3\left(x_0\right)$	0.5915	−0.1873	0.03638	0.6313	−0.1420	0.02131	0.6158	−0.1227	0.01614
			$f_4\left(x_0\right)$	0.8446	0.0658	0.09316	0.8497	0.0763	0.09763	0.8085	0.0700	0.08467
			Hist	0.6621	−0.1167	0.01526	0.6572	−0.1161	0.01509	0.6341	−0.1044	0.01247
			KDE	0.7106	−0.0682	0.08232	0.7039	−0.0694	0.08261	0.6735	−0.0650	0.00741
5	24	10	$f_3\left(x_0\right)$	0.7112	−0.0676	0.00712	0.7170	−0.0564	0.00540	0.6931	−0.0454	0.00416
			$f_4\left(x_0\right)$	0.8336	0.0548	0.09965	0.8298	0.0564	0.08810	0.7946	0.0561	0.08189
			Hist	0.5362	−0.2426	0.05953	0.5326	−0.2407	0.05859	0.5177	−0.2208	0.04946
			KDE	0.6799	−0.0989	0.01426	0.6759	−0.0975	0.01378	0.6477	−0.0908	0.01233
5	16	15	$f_3\left(x_0\right)$	0.7141	−0.0647	0.00681	0.7181	−0.0553	0.00532	0.6927	−0.0458	0.00409
			$f_4\left(x_0\right)$	0.7951	0.0163	0.04790	0.7880	0.0147	0.04856	0.7518	0.0134	0.04382
			Hist	0.5370	−0.2418	0.05909	0.5327	−0.2407	0.05860	0.5178	−0.2207	0.04936
			KDE	0.6839	−0.0949	0.01365	0.6763	−0.0971	0.01385	0.6452	−0.0932	0.01262
12	10	10	$f_3\left(x_0\right)$	0.7832	0.0044	0.00540	0.7770	0.0036	0.00557	0.7443	0.0058	0.00484
			$f_4\left(x_0\right)$	0.7825	0.0037	0.07077	0.7888	0.0155	0.08372	0.7447	0.0062	0.06303
			Hist	0.2500	−0.5288	0.27964	0.2496	−0.5238	0.27437	0.2482	−0.4903	0.24045
			KDE	0.6424	−0.1364	0.02604	0.6385	−0.1348	0.02604	0.6101	−0.1284	0.02339
13	52	5	$f_3\left(x_0\right)$	0.7812	0.0024	0.00211	0.7743	0.0009	0.00208	0.7397	0.0012	0.00195
			$f_4\left(x_0\right)$	0.9649	0.1861	0.35467	0.9502	0.1769	0.33550	0.9102	0.1718	0.27789
			Hist	0.3038	−0.4750	0.22563	0.3031	−0.4702	0.22113	0.2998	−0.4387	0.19246
			KDE	0.6629	−0.1159	0.01711	0.6587	−0.1146	0.01682	0.6271	−0.1113	0.01576
13	20	13	$f_3\left(x_0\right)$	0.7825	0.0037	0.00213	0.7743	0.0009	0.00208	0.7397	0.0012	0.00179
			$f_4\left(x_0\right)$	0.8100	0.0312	0.05881	0.8040	0.0307	0.06110	0.7581	0.0197	0.05171
			Hist	0.3039	−0.4749	0.22555	0.3031	−0.4702	0.22113	0.2999	−0.4386	0.19240
			KDE	0.6640	−0.1148	0.01677	0.6587	−0.1146	0.01682	0.6268	−0.1117	0.01561

Notes: Histogram uses $h=c_{\mathrm{hist}}{N}_{\mathrm{eff}}^{-1/3}$; KDE uses Silverman’s rule on the reduced sample with reflection at $x=0$. Comparators use the same reduced samples as the proposed estimators: $N_{\mathrm{eff}}=Sm$ for set-minima methods and $N_{\mathrm{eff}}=S$ for block-minima methods.

In line transect sampling, commonly studied detection functions include the Exponential$\left(1\right)$ model, representing a non-shoulder case, the Half-Normal$(0,1)$ model with a moderate shoulder, and the Hazard-Rate$(\sigma =1,b=3)$ model, which allows for a pronounced shoulder. Across these generating models, the pooled-minima estimator ${\widehat{f}}_1\left(0\right)$ exhibited the anticipated bias decay as r increased: for moderate $(r,m,S)$, the averages were around $0.75$–$0.94$ under Exponential$\left(1\right)$ with negative bias that shrank toward zero as r grew, consistent with an $O(1/r)$ bias term. In contrast, the block min-of-minima ${\widehat{f}}_2\left(0\right)$ often attained smaller bias at a fixed configuration but showed larger variance because its effective sample size was only S; the MSE reflected this trade-off and decreased when S grew or when m was large enough to stabilize block-level minima. These patterns are consistent with the theoretical orders $Var({\widehat{f}}_1)=O({\left(Sm\right)}^{-1})$ and $Var({\widehat{f}}_2)=O(S^{-1})$.

Histogram and KDE Benchmarks

On the reduced samples, the boundary-corrected KDE (reflection) reliably dominated the histogram in the MSE, in line with the canonical rates $O\left(N_{\mathrm{eff}}^{-4/5}\right)$ versus $O\left(N_{\mathrm{eff}}^{-2/3}\right)$. For very small $N_{\mathrm{eff}}$ (e.g., large r but small m), both nonparametric benchmarks could be unstable at the boundary, with the histogram particularly sensitive to the first-bin mass at $[0,h)$. As $N_{\mathrm{eff}}$ increased, the KDE improved steadily; however, ${\widehat{f}}_1\left(0\right)$ typically matched or surpassed the KDE once $Sm$ was moderately large, reflecting the $O\left(N_{\mathrm{eff}}^{-1}\right)$ variance rate (up to constants and second-order bias in $r,m$).

3.2.2. Interior-Point Estimation (Folded Estimators)

For interior evaluation at $x_0=0.25$, the folded estimator ${\widehat{f}}_3\left(x_0\right)$ stabilized rapidly as r increased and was frequently near-unbiased in both models, mirroring the boundary behavior of ${\widehat{f}}_1$ but benefiting from smoother local curvature at $x_0>0$. The block-folded estimator ${\widehat{f}}_4\left(x_0\right)$ again showed the expected bias reduction at the cost of higher variance due to its reliance on S block minima. In most of the scenarios, the KDE remained a strong benchmark at the interior points; nevertheless, ${\widehat{f}}_3\left(x_0\right)$ was often competitive or superior in the MSE once $Sm$ was moderate, while the histogram remained sensitive to bin choice.

As seen in Table 5 and Table 6, across $x_0$ values, the folded pooled set estimator $f_3\left(x_0\right)$ was essentially unbiased with small variance, yielding the lowest MSE in all three values. The MSE declined mildly as $x_0$ increased, consistent with greater stability of the folded minima away from the boundary. The block min-of-minima $f_4\left(x_0\right)$ exhibited a persistent positive bias and substantially larger variance, reflecting its smaller effective sample size S and therefore larger MSE.

Table 5. Estimation of interior density $f\left(x_0\right)$ for samples from $\mathrm{Exp}\left(1\right)$ distribution with fixed sample sizes design. Boldface indicates the smallest MSE among a set values of $x_0$.

${\mathit{x}}_0$	r	m	S	True $\mathit{f}\left({\mathit{x}}_0\right)$	Method	$\widehat{\mathit{f}}\left({\mathit{x}}_0\right)$	Bias	Var	MSE
0.15	13	20	13	0.86071	$f_3\left(x_0\right)$	0.846806	−0.013902	0.002943	0.003136
0.25				0.77880	$f_3\left(x_0\right)$	0.781025	0.002224	0.002145	0.002150
0.50				0.60653	$f_3\left(x_0\right)$	0.610442	0.003911	0.001221	0.001236
0.15					$f_4\left(x_0\right)$	0.893531	0.032823	0.072044	0.073121
0.25					$f_4\left(x_0\right)$	0.805722	0.026921	0.057698	0.058423
0.50					$f_4\left(x_0\right)$	0.625649	0.019118	0.034784	0.035150

Table 6. Estimation of interior density $f\left(x_0\right)$ for samples from Half-Normal$(0,1)$ distribution with fixed sample sizes design. Boldface indicates the smallest MSE among the compared values of $x_0$.

${\mathit{x}}_0$	r	m	S	True $\mathit{f}\left({\mathit{x}}_0\right)$	Method	$\widehat{\mathit{f}}\left({\mathit{x}}_0\right)$	Bias	Var	MSE
0.15	13	20	13	0.78896	$f_3\left(x_0\right)$	0.771076	−0.017883	0.002446	0.002766
0.25				0.77336	$f_3\left(x_0\right)$	0.774786	0.001450	0.002004	0.002006
0.50				0.70413	$f_3\left(x_0\right)$	0.705372	0.001241	0.001674	0.001676
0.15					$f_4\left(x_0\right)$	0.817551	0.028592	0.062897	0.063715
0.25					$f_4\left(x_0\right)$	0.806618	0.033282	0.057505	0.058612
0.50					$f_4\left(x_0\right)$	0.728920	0.024790	0.045955	0.046570

3.2.3. Robustness Against Closest-Unit Misclassification

To assess robustness against imperfect closest-unit identification, we conducted a small Monte Carlo study incorporating judgment error in the set-based selection. Within each set of size r, the true minimum was selected with probability $1-p$, while with probability p the second-smallest distance was selected, following the standard ranking-error models used in the ranked set sampling literature. We considered misclassification probabilities $p\in \{0.00,0.05,0.10,0.20\}$. The results show that moderate judgment error led to gradual increases in variability without severe bias inflation, supporting the robustness of the proposed methodology under realistic field conditions.

Bias and MSE were evaluated for the first-stage and second-stage estimators under the Exponential and Half-Normal detection models using the same design settings as in Section 3.2. Table 7 summarizes the results. As expected, increasing misclassification led to gradual efficiency loss, reflected in larger MSE, while the estimators remained well-behaved. In the extreme case of severe misclassification, performance approached that of simple random sampling using the same number of measured distances. Related results in the ranked set sampling literature show that efficient estimation remains feasible under ranking or measurement errors, with performance degrading smoothly rather than failing [21].

Table 7. Robustness of ${\widehat{f}}_1\left(0\right)$ and ${\widehat{f}}_2\left(0\right)$ to closest-unit misclassification for random samples with configuration $r=5,m=24,S=10$. True values: $f\left(0\right)=1$ for standard Exponential and $f\left(0\right)=\sqrt{2/\pi }\approx 0.7979$ for standard Half-Normal distributions. Boldface indicates the smallest MSE among the compared values of p.

		${\widehat{\mathit{f}}}_1\left(0\right)$		${\widehat{\mathit{f}}}_2\left(0\right)$
	$\mathit{p}$	Bias	MSE	Bias	MSE
Exponential	0.00	−0.1633	0.0296	0.0666	0.1098
	0.05	−0.2127	0.0479	0.0105	0.1166
	0.10	−0.2567	0.0683	−0.0441	0.1281
	0.20	−0.3308	0.1113	−0.1486	0.1516
Half-Normal	0.00	−0.0225	0.0025	0.0688	0.0761
	0.05	−0.0621	0.0057	0.0261	0.0823
	0.10	−0.0977	0.0112	−0.0178	0.0936
	0.20	−0.1597	0.0268	−0.1007	0.1065

In conclusion, all of the above results confirm that minima-based estimators provide a computationally light, tuning-robust alternative to standard nonparametric methods, with especially strong gains in scenarios with moderate-to-large set sizes and block counts.

3.2.4. Practical Guidance for Choosing r, m, and S

The performance of the proposed estimators depends on the design parameters r (set size), m (sets per block), and S (independent blocks), which determine the bias–variance trade-off and effective sample size. We summarize our practical recommendations below.

The set size r primarily controls bias: larger r yields minima closer to the transect line and lower bias but increases judgment difficulty and sensitivity to ranking error. The parameters m and S control variance through the effective sample size, with $N_{\mathrm{eff}}=Sm$ for the first-stage estimator and $N_{\mathrm{eff}}=S$ for the second-stage estimator. Increasing S is particularly important for stabilizing the second-stage estimator.

Table 8 provides concise guidance for selecting $(r,m,S)$ under common field constraints, consistent with the theoretical results in Section 3.1.1and the simulation findings in Section 3.2.

Table 8. Practical guidance for selecting r, m, and S.

Constraint	Recommendation	Effect
Limited ranking accuracy	Smaller r	Improves robustness to misclassification
Limited measurement effort	Small–moderate r	Balances bias reduction and field effort
Few survey routes	Larger m	Increases $N_{\mathrm{eff}}=Sm$
Many survey routes	Larger S	Reduces variance, especially for stage two
Bias is primary concern	Larger r	Minima concentrate near the line
Variance is primary concern	Larger m or S	Increases effective sample size

3.2.5. Choice of the Folding Target $x_0$

The folded estimators in Section 3.2.2 require selecting a target point $x_0>0$, motivated by practical issues near the transect line, such as blind zones and distance heaping. In blind-zone settings, $x_0$ may be chosen as the smallest distance beyond which detections are reliably recorded, as identified from a brief pilot or calibration pass. In anti-heaping settings, $x_0$ can be selected as a small offset from zero that avoids rounded or heaped values.

In practice, simple empirical diagnostics (e.g., near-zero histograms or measurement resolution) are sufficient for choosing $x_0$. Typical values are small relative to the truncation distance, and the simulation results in Section 3.2 indicate that the folded estimators are not overly sensitive to modest changes in $x_0$ within this near-line region.

3.2.6. When Should Minima-Based Estimators Be Used?

Minima-based estimators are particularly well suited for surveys where precise measurement of all perpendicular distances is costly, but reliable identification of the closest unit within small sets is feasible. They are advantageous when robustness to tuning parameters is desired, when interest centers on near-line behavior, or when field effort must be minimized.

In contrast, KDE and parametric distance-sampling methods may be preferable when full distance measurements are readily available, detection functions are well specified, or likelihood-based inference is required. In practice, the proposed estimators are best viewed as complementary to the existing methods, offering a low-effort alternative rather than a replacement for standard approaches.

3.3. Real Data Study: Wooden Stakes Transect Survey

3.3.1. Study Design and Data Collection

To illustrate the applicability of the proposed estimators in practice, we designed a field-style study in which artificial wooden stakes were used as a proxy for objects encountered along a line transect, closely resembling the classical Wooden Stakes experiment of Burnham et al. [5]. A straight transect of length $L=1000$ m was laid out randomly across a uniform open field. Wooden stakes $N=350$ were positioned randomly within a strip extending $w=20$ m on either side of the transect line. The stakes were placed flush with the ground to mimic small animal or plant features and to reduce visual bias from above-ground height. Our field team consisted of the principal investigator and a graduate student assistant. The investigator carried a laser rangefinder and a measuring tape to obtain exact perpendicular distances when required. The assistant was responsible for making the initial “closest unit” judgments within each set.

This experiment represented an intentionally idealized setting in which all objects were detectable with certainty. Its primary role in this study was to serve as a proof-of-concept, allowing the proposed estimators to be evaluated under controlled conditions without confounding effects from distance-dependent detection or imperfect observability. By design, this experiment isolated the statistical behavior of the minimum-based estimators and facilitated direct comparison with existing methods. The results should therefore be interpreted as illustrating methodological performance under ideal detection, rather than as a fully realistic ecological survey.

Data Collection Procedure

Following the sampling design outlined in Section 1, we adopted a set-based sampling approach. The transect was partitioned into $S=5$ segments (routes) per cycle, and within each segment $m=4$ sets of candidate objects were defined. Each set contained $r=3$ candidate stakes, positioned naturally within the field of view. The assistant visually judged the nearest stake to the transect line within each set, using simple visual signs (apparent proximity, relative angle, or sighting ease). This was an efficient approach done quickly without ordering the remaining units and with low-cost judgment. Then, the perpendicular distance of the judged-closest stake was measured exactly, using laser rangefinder (or measuring tape). Thus, from each set exactly one distance was measured and recorded.

After completing $m=4$ sets per route and $S=5$ routes, a total of $M=S·m=20$ distances were obtained per cycle. To increase replication, we conducted $c=3$ independent cycles of the procedure, resulting in a final dataset of $M·c=60$ observed set minima.

Extensions. To accommodate block-minima and folded estimators, $f\left(0\right))$ and $(f\left(x_0\right)$, we also considered (i) block minima across each route (yielding $S·c=15$ block minima), and (ii) folded distances around a chosen interior reference $x_0$, reflecting practical cases where interior density evaluation is relevant. In our study we adopted $x_0=0.25$ m. This choice was justified by two practical considerations:

• Blind zone at the line: stakes very close to the line $(<$25 cm) were hard to measure precisely due to grass or ground cover.
• Heaping and measurement error: Measurements extremely close to zero are subject to “rounding” errors, as field assistants tend to record them as exact zeros. Folding at $x_0=0.25$ avoided this artifact and produced a stable estimation point.

These matched directly to the estimators ${\widehat{f}}_1\left(0\right)$, ${\widehat{f}}_2\left(0\right)$, ${\widehat{f}}_3\left(x_0\right)$, and ${\widehat{f}}_4\left(x_0\right)$ defined in Section 3.1.1.

3.3.2. Observed Perpendicular Distances

The following table presents the observed perpendicular distances (in meters) collected using the judged-closest protocol described in Section 3.3.1. The distances are truncated to three decimal places.

3.3.3. Estimation Results

Let ${\overline{X}}^{*}$ denote the mean of the 60 set minima and ${\overline{X}}^{**}$ the mean of the 15 block minima; similarly, ${\overline{\mathbb{D}}}^{*}\left(x_0\right)$ and ${\overline{\mathbb{D}}}^{**}\left(x_0\right)$ denote the means of the folded set and block minima, as defined in Section 3.1.1. From the data in Table 9, we obtained ${\overline{X}}^{*}=2.4293\mathrm{m},$${\overline{X}}^{**}=0.5553\mathrm{m}.$ Accordingly, the four estimators were ${\widehat{f}}_1\left(0\right)=0.1029$, ${\widehat{f}}_2\left(0\right)=0.1200$, ${\widehat{f}}_3\left(x_0\right)=0.0573$, and ${\widehat{f}}_4\left(x_0\right)=0.1037$, with $x_0=0.25$ m.

Table 9. Observed perpendicular distances (judged-closest, one per set) for all cycles from Wooden Stakes Transect Survey. Design: $r=3$, $m=4$, $S=5$.

Cycle	Route	Set 1	Set 2	Set 3	Set 4	Block Min
1	1	1.482	0.736	2.914	4.207	0.736
	2	3.251	1.106	0.284	5.933	0.284
	3	0.512	1.873	0.961	0.788	0.512
	4	6.102	2.390	3.744	1.225	1.225
	5	4.569	7.331	2.006	0.397	0.397
2	1	2.184	0.655	1.742	3.968	0.655
	2	0.322	2.577	1.118	0.904	0.322
	3	5.746	0.711	2.408	1.936	0.711
	4	1.564	0.486	4.215	6.431	0.486
	5	3.297	2.041	0.273	1.807	0.273
3	1	0.441	1.589	2.903	0.862	0.441
	2	7.012	4.603	2.219	1.107	1.107
	3	2.735	0.358	3.116	5.820	0.358
	4	4.921	3.077	0.694	2.488	0.694
	5	1.866	0.129	0.972	2.650	0.129

Our designed protocol, where $N=350$ stakes, $L=1000$ m and $w=20$ m, corresponding to a covered strip area $A=2wL=40,000{\mathrm{m}}^2$, produced the true population density $D_{\mathrm{true}}=N/\left(2wL\right)=8.75\times {10}^{-3}{\mathrm{m}}^2$ and $n_{\mathrm{observed}}=180$ (i.e., 3 cycles with 60 observed distances each). This yielded the design-based reference $f{\left(0\right)}_{\mathrm{design}}=2L{D}_{\mathrm{true}}/n_{\mathrm{observed}},$ which equaled $0.0972.$

As seen in Table 10, the four estimators ${\widehat{f}}_1\left(0\right),{\widehat{f}}_2\left(0\right),{\widehat{f}}_3\left(x_0\right),{\widehat{f}}_4\left(x_0\right)$ all produced values close to this design-based reference, with ${\widehat{f}}_1$ and ${\widehat{f}}_4$ aligning most closely. The folded estimator ${\widehat{f}}_3$ was more sensitive to the choice of $x_0$, and here provided a conservative lower value. This exercise illustrates how the proposed minima-based estimators can be applied to real field-style data, offering valid alternatives to classical full-data methods in situations where only partial observations are feasible.

Table 10. Estimated boundary and interior densities from the wooden stakes dataset. Design: $r=3$, $m=4$, $S=5$ per cycle, $c=3$ cycles ($M=60$ set minima, 15 block minima). Interior folding at $x_0=0.25$ m. ESW = effective strip width $1/\widehat{f}$.

Estimator	Definition	Sample Used	$\widehat{\mathit{f}}$	ESW (m)
${\widehat{f}}_1\left(0\right)$	$\frac{1}{(r+1){\overline{X}}^{*}}$	60 set minima	0.1029	9.72
${\widehat{f}}_2\left(0\right)$	$\frac{1}{r(m+1){\overline{X}}^{**}}$	15 block minima	0.1200	8.33
${\widehat{f}}_3\left(x_0\right)$	$\frac{1}{2(r+1){\overline{\mathbb{D}}}^{*}\left(x_0\right)}$	60 folded set minima	0.0573	17.47
${\widehat{f}}_4\left(x_0\right)$	$\frac{1}{2r(m+1){\overline{\mathbb{D}}}^{**}\left(x_0\right)}$	15 folded block minima	0.1037	9.64

A simple significant note is that the relatively large ESW implied by ${\widehat{f}}_3\left(x_0\right)$ at $x_0=0.25$ m reflects the smoothing effect induced by evaluating the folded estimator away from the boundary, rather than model misspecification; this mild inflation is consistent with the bias–variance trade-off inherent in avoiding near-zero detection irregularities and remains compatible with the known truncation width $w=20$ m.

Complete Distance Data

In addition to the judgment-minimum protocol, we measured all perpendicular distances for the full population of $N=350$ wooden stakes. The complete dataset lets us exploit the survey’s full information content, directly compare the minima-based estimators with conventional line transect estimators that use all detections, and transparently assess the efficiency trade-off between collecting a few targeted distances and undertaking exhaustive measurement. While our proposed estimators are designed to achieve efficiency with minimal effort, the complete dataset provides a benchmark for validation and ensures that comparisons to standard methods are transparent and scientifically rigorous.

4. Discussion

The proposed minima-based estimators are developed under the standard assumptions of line transect sampling, namely perfect detection at the transect line, independence across sets and blocks, and correct identification of the closest unit within each set. These assumptions are shared by most design-based and nonparametric distance-sampling estimators and are therefore not specific to the present approach.

Violations of perfect detection at the line affect the scale of the boundary density $f\left(0\right)$ and consequently induce proportional bias in abundance estimates, a well-known issue in line transect inference. As with existing methods, this can be mitigated through blind-zone anchoring, auxiliary detection modeling, or integrated distance-sampling frameworks, within which the proposed estimators can be readily embedded.

Independence is primarily required at the block level and is enforced by construction through the use of non-overlapping sets and spatially or temporally separated blocks. Because only minimum observations are retained from each set, the estimators are less sensitive to local dependence among detections than methods that rely on full within-set measurements.

Accurate identification of the closest unit corresponds to ranking by judgment in the ranked set sampling literature. As established in that literature, moderate ranking errors primarily reduce efficiency rather than consistency. In the extreme case of complete misranking—where the selected unit within each set is effectively random—the proposed procedure reduces to a standard estimator based on simple random sampling using the same number of measured distances. Thus, ranking errors lead to a gradual loss of efficiency rather than structural bias, and the method remains well-behaved under realistic field conditions.

The results of both our simulation experiments and our field-style wooden stakes survey highlight the practical strengths and limitations of the proposed minima-based estimators. At the boundary, ${\widehat{f}}_1\left(0\right)$ and ${\widehat{f}}_2\left(0\right)$ converge rapidly and are competitive—often superior-to histogram and boundary-corrected KDE. For interior targets, the folded estimators ${\widehat{f}}_3\left(x_0\right)$ and ${\widehat{f}}_4\left(x_0\right)$ perform well and offer a flexible extension. A key advantage is operational, where only one exact distance per set is required after a quick closest judgment, substantially reducing measurement burden.

In the full-data stakes study ($N=350$), minima-based estimates of $f\left(0\right)$ and ESW aligned closely with conventional analyses. Choice of $x_0$ can be guided by blind-zone or anti-heaping considerations, providing stable inference near the line. These findings directly support the motivation of the proposed methodology by demonstrating that reliable abundance estimation can be achieved using minimal measurement effort when only the closest objects to the transect line are recorded.

Overall, the study demonstrates that minima-based estimators can provide accurate, efficient, and operationally simple density estimates in field surveys. While further work is needed to extend the framework to more complex detection functions and clustered populations, the current results provide a clear foundation for practical adoption. A possible refinement would also be to add further selection stages, which may increase the method’s efficiency.