Pollen-YOLO: A Deep Learning Framework for Automated Pollen Identification and Its Application to Palaeoecological Reconstruction on the Tibetan Plateau

Abstract

Automated pollen identification has become an increasingly important tool for palaeoecological research; however, its application to fossil pollen assemblages remains challenging due to complex backgrounds, morphological variability, and taxonomic similarity among pollen types. In this study, we propose Pollen-YOLO, a deep learning-based object detection framework designed for automated pollen identification from microscopic images, and evaluate its performance using the TPPOL23 dataset. The model integrates a tailored backbone architecture with attention-based feature enhancement and class-specific data augmentation strategies to address the characteristics of fossil pollen images. Experimental results indicate that Pollen-YOLO achieves stable and competitive detection performance for most pollen taxa under the tested conditions, particularly for dominant taxa with distinctive morphological features. Model behavior is further examined through ablation experiments and Grad-CAM-based interpretability analysis, which provide insights into feature learning and classification mechanisms. The applicability of the framework is explored using a fossil pollen sequence from the Shaqu profile on the Tibetan Plateau. Automated results show a high level of agreement with manual identification in capturing major stratigraphic trends and vegetation succession patterns, while discrepancies persist for morphologically similar or low-abundance taxa. Overall, this study suggests that object detection-based deep learning approaches have the potential to support fossil pollen analysis and palaeoecological reconstruction. Rather than replacing expert identification, Pollen-YOLO is intended as a complementary, high-throughput tool that may assist large-scale pollen analysis under appropriate quality control when combined with expert verification.

1. Introduction

The Tibetan Plateau, often referred to as the “Roof of the World” and the “Third Pole”, is one of the most climatically sensitive regions on Earth owing to its extreme elevation, complex topography, and strong interactions with the Asian monsoon system. Since the Quaternary, pronounced climatic oscillations, glacial–interglacial cycles, and stepwise uplift have continuously reshaped regional vegetation patterns, exerting far-reaching impacts on hydrology, atmospheric circulation, and ecosystem dynamics across Asia and beyond [1,2]. Reconstructing long-term vegetation and environmental changes on the Tibetan Plateau is therefore critical for understanding both regional ecological evolution and broader climate system behavior.

Fossil pollen preserved in sediments represents one of the most widely used and robust proxies for Quaternary palaeoecological reconstruction. Variations in pollen assemblages provide essential information on past vegetation composition, ecosystem structure, and climate-driven environmental changes [3]. However, the reliability of such reconstructions strongly depends on the accuracy, consistency, and scale of pollen identification. Traditional palynological analysis relies on manual microscopic identification by trained experts, a process that is labor-intensive, time-consuming, and inevitably affected by observer bias, particularly when dealing with large stratigraphic sequences or multi-site datasets.

These challenges are further amplified in high-altitude and high-energy depositional environments such as the Tibetan Plateau. Fossil pollen grains in plateau sediments commonly exhibit deformation, fragmentation, and mineral interference caused by aeolian abrasion, mechanical compression, and post-depositional alteration. Such conditions often obscure fine diagnostic features (e.g., apertures and exine ornamentation), making consistent and reproducible identification difficult even for experienced palynologists. As a result, many Quaternary palaeoecological studies in this region primarily rely on family- or genus-level pollen identification, which has proven sufficient and robust for reconstructing major vegetation types, functional groups, and long-term environmental trends.

In recent years, rapid advances in deep learning, particularly convolutional neural networks (CNNs), have opened new avenues for automated pollen recognition [4,5,6]. CNN-based approaches have demonstrated superior performance over traditional image-processing methods and have been successfully applied to pollen classification, detection, and counting tasks [7]. Object detection frameworks such as the You Only Look Once (YOLO) series further enhance efficiency by enabling simultaneous localization and classification of multiple pollen grains within a single image [8,9,10]. These developments highlight the potential of deep learning to support high-throughput and objective pollen analysis [11].

Nevertheless, most existing automated pollen identification studies focus on modern pollen samples, limited taxonomic scopes, or highly controlled imaging conditions. Their direct applicability to fossil pollen assemblages remains limited, particularly in complex sedimentary contexts where preservation quality, background noise, and morphological variability pose substantial challenges. Moreover, although species-level identification may be achievable for a small number of taxa with highly distinctive pollen morphology, such resolution is not consistently attainable across diverse fossil pollen groups and may reduce reproducibility when applied at scale.

Against this background, the primary aim of this study is to develop a scalable, reproducible, and palaeoecologically meaningful deep learning framework for automated fossil pollen identification, rather than to replace expert-based species-level taxonomy. We introduce Pollen-YOLO, an improved YOLOv11-based detection framework incorporating a ResNet-152 backbone with dual-path attention and dual-pooling mechanisms, specifically designed to enhance sensitivity to subtle morphological features such as apertures and exine ornamentation in fossil pollen images. To support model training and evaluation, we constructed a standardized microscopic image database of common Quaternary pollen taxa from the Tibetan Plateau (TPPOL23), integrating expert annotation and class-specific data augmentation strategies.

To indicate the applicability of the proposed framework in Quaternary palaeoecology, Pollen-YOLO was applied to fossil pollen assemblages from the Shaqu profile on the northeastern Tibetan Plateau. An independent chronological framework was established using optically stimulated luminescence (OSL) dating combined with Bayesian age–depth modeling, allowing direct comparison between model-identified and manually identified pollen assemblages. By focusing on taxonomic levels that are both reliable and palaeoecologically informative, this study aims to provide an efficient tool for large-scale pollen analysis and long-term vegetation reconstruction in complex sedimentary environments.

2. Study Area

The Tibetan Plateau (26°00′–39°47′ N, 73°19′–104°47′ E) is the largest and highest plateau in the world, with a mean elevation exceeding 4000 m and a total area of approximately 2.5 million km². Owing to its unique topographic configuration, it exerts a profound “heat pump effect” on the Asian monsoon system and regulates hydrothermal patterns across Central Asia, making it one of the most climatically sensitive regions on Earth [12]. Since the Quaternary, stepwise uplift combined with climatic oscillations and glacial processes has significantly reshaped the plateau’s environmental gradients, thereby influencing vegetation dynamics, hydrological cycles, and atmospheric circulation on both regional and global scales.

Modern vegetation on the plateau is diverse and includes coniferous forests, mixed coniferous–broadleaf forests, broadleaf forests, shrublands, grasslands, meadows, herbaceous vegetation, alpine vegetation, deserts, and cultivated vegetation (Figure 1) [13]. The spatial distribution of these vegetation types is primarily controlled by hydrothermal conditions: forests and shrublands dominate the monsoon-influenced southeast; alpine meadows and steppe prevail in the central plateau; deserts and cushion-like alpine vegetation are characteristic of the arid northwest; while cultivated vegetation is mainly confined to river valleys and the eastern margins [14].

This hydrothermal–vegetation gradient not only creates high ecological heterogeneity but also strongly influences pollen production, dispersal, and deposition, thereby shaping distinct assemblage signals in lake, peat, and loess sediments. In recent years, regional and continental pollen datasets from East Asia and the Tibetan Plateau, including both surface and fossil samples, have been established. These datasets have enabled quantitative reconstructions of vegetation and climate since the Holocene and the Last Deglaciation [15]. Lake pollen records from different climatic zones of the plateau demonstrate strong sensitivity to variations in precipitation and aridification events, while also revealing considerable spatial heterogeneity across basins and catchments.

It is worth noting that the complex depositional environments of the plateau (e.g., aeolian input, salinization, and enhanced clastic sedimentation) often degrade or obscure diagnostic morphological traits of fossil pollen, such as apertures and exine ornamentation, thus increasing the difficulty of both manual and computer-assisted identification. Therefore, clarifying the linkages between modern vegetation patterns, environmental gradients, and pollen signals provides the essential context for developing automated pollen recognition methods and constructing large-scale image datasets.

3. Materials and Methods

3.1. Laboratory Preparation of Historical Samples

The pollen samples used in this study were derived from three sources:

(1)

Historical sediment profiles and core samples.

The samples were obtained from the Key Laboratory of Surface Processes and Ecological Conservation, Ministry of Education, during the 2016–2024 research program on “Human activities and environmental change on the Tibetan Plateau.” They cover stratigraphic and lacustrine sections from the eastern, central, and northern parts of the plateau. Specific study sites and profiles include the Daiqu Site [16], Laodaqiao Site [16], Donggicuona Lake [17], Shalongka [18], Nankanyan Site [19], Xiadawu Site [20], Eling Lake Site [21], and Zhongda Site [22]. The depositional facies mainly consist of lacustrine clay layers, overbank silt deposits, and aeolian loess horizons. Here, the term “sample” refers specifically to an individual stratigraphic sediment sample (i.e., a depth interval), which was subsequently processed into one or more microscopic slides for pollen analysis, rather than to individual pollen grains or images. Due to the limitations of historical data management, detailed lithological and chronological information of these profiles is summarized in

Table 1. Names and detailed information of pollen sampling sites. “Pollen sample count” indicates the number of sediment samples analyzed for pollen at each site. 14C denotes radiocarbon dating, and OSL denotes optically stimulated luminescence dating.

Section Name	Sediment Type	Stratigraphic Age (Dating Method)	Pollen Sample Count
Daiqu Site	Aeolian loess–fluvial deposit	12.1–10.9 ka (OSL)	65
Laodaqiao Site	Aeolian loess–fluvial deposit	13–3.3 ka (OSL)	97
Donggicuona Lake	Aeolian loess–lacustrine deposit	2.01–8.77 ka (OSL)	45
Shalongka	Aeolian loess–flood deposit	8.5–3.9 ka (14C)	208
Nankanyan Site	Aeolian loess–fluvial deposit	13.7–0.8 ka (OSL)	66
Xiadawu Site	Aeolian loess–fluvial deposit	0–6.3 ka (14C)	62
Eling Lake Site	Aeolian loess–lacustrine deposit	2.1–14.6 ka (OSL)	105
Zhongda Site	Aeolian loess–fluvial deposit	6.2–19 ka (OSL)	31

and Figure 2.

(2)

Modern pollen reference slides.

A series of modern pollen standard slides were produced by our research group (Figure 3). Preparation followed internationally recognized palynological protocols, including pretreatment of samples, chemical removal of organic and mineral impurities, concentration of pollen grains, and permanent slide mounting. To improve comparability and long-term preservation, an improved preparation procedure was adopted to minimize interference from clastic and organic debris.

(3)

Open access pollen resources.

We additionally utilized the Atlas of Common Pollen Morphology of Plants in Eastern Tibet (2020–2025), released by the National Tibetan Plateau Data Center, citing both the associated dataset publication and the online data release [23,24].

For quality assurance, the first two categories of samples strictly adhered to the following criteria:

(1)

Field documentation confirmed exclusion of modern bioturbation layers;

(2)

Microscopic inspection verified that ≥75% of pollen grains retained intact exine ornamentation.

3.1.1. Description of the Natural Profile and Sampling Strategy

The studied natural profile is located in Jiuzhi County, Golog Tibetan Autonomous Prefecture, eastern Tibetan Plateau (33°12′ N, 101°35′ E; 3820 m a.s.l.), and is hereafter referred to as the Shaqu profile. It is exposed on the sidewall of a river valley and reveals a 2.0 m thick sequence of continuous fluvial and aeolian loess deposits (Figure 4). Based on soil sedimentary features and color, three lithological units were distinguished: 0–20 cm of surface soil, 20–60 cm of peat layer, and 60–200 cm of aeolian loess. Pollen samples were collected from 20 cm below the surface at 10 cm intervals downward, yielding a total of 18 samples. These samples were used both for manual identification and for cross-validation of model performance in pollen recognition. The sampling strategy follows standard palynological field protocols.

3.1.2. OSL Dating and Age–Depth Modeling

To establish a reliable chronological framework for the Shaqu profile, six optically stimulated luminescence (OSL) samples were collected from depths of 40–200 cm (SQ-0 to SQ-5). Quartz grains (63–90 µm) were extracted following standard chemical pretreatment procedures, including removal of carbonates with 10% HCl, oxidation of organic matter with 30% H₂O₂, etching with 40% HF for 40 min to remove the outer α-irradiated layer, and purity screening (>95%) by infrared stimulation. Equivalent dose (De) measurements were carried out using the single-aliquot regenerative-dose (SAR) protocol on a Risø TL/OSL-DA-20 automated luminescence reader (DTU Nutech, Roskilde, Denmark) at the Key Laboratory of Plateau Environmental Processes and Ecological Restoration, Qinghai Normal University [25]. Environmental dose rates were determined from U, Th, and K concentrations measured by high-purity germanium γ-spectrometry at the Xi’an Center of Mineral Resources Testing, Ministry of Natural Resources. Cosmic-ray contributions, water content, and the α-efficiency factor (α = 0.04 ± 0.02) were also included in dose–rate calculations.

The obtained OSL ages range from 4485 ± 320 yr BP at 40 cm (SQ-0) to 12,115 ± 780 yr BP at 200 cm (SQ-5), covering an overall timespan of approximately 12–4 ka BP. These ages were subsequently incorporated into a Bayesian age–depth model using the Bacon 2.5 package in R [26,27]. Markov Chain Monte Carlo (MCMC) simulations produced posterior distributions of accumulation rates and calibrated ages, yielding both mean estimates and 95% confidence intervals. The resulting age–depth model indicates relatively stable sedimentation rates in the lower and middle sections of the profile, with greater variability in the upper layers. This chronological framework provides the temporal basis for pollen analysis and palaeoecological reconstruction.

3.1.3. Laboratory Treatment of Natural Profile Pollen Samples

For clarity, it should be noted that three different hierarchical units are involved in this study: (i) stratigraphic sediment samples, (ii) microscopic images, and (iii) individual pollen grains. Stratigraphic samples refer to discrete sediment depth intervals collected from the studied sedimentary sections and processed for pollen extraction. Microscopic images were acquired from prepared slides under standardized optical conditions, and each image may contain multiple pollen grains. Individual pollen grains were manually annotated within each image and constitute the fundamental objects used for model training and evaluation. A total of 18 soil samples were selected for laboratory pollen preparation. Approximately 60 g of dried sediment was processed for each sample, with one Lycopodium clavatum spore tablet (containing 10,315 ± standard deviation spores; Batch #BC-1023) added as a marker for calculating pollen concentration and recovery. Fossil pollen was concentrated using a combination of acid–alkali digestion and heavy liquid separation procedures. Carbonates were removed using 10% hydrochloric acid (HCl, 24 h), and silicate minerals were dissolved with two successive treatments of 40% hydrofluoric acid (HF, 48 h each). Residues were rinsed with deionized water and centrifuged (3000× g rpm, 5 min) until neutral pH (7.0 ± 0.5) was achieved. Organic matter was oxidized by treating with 10% potassium hydroxide (KOH, 70 °C water bath, 30 min) to disperse colloids, followed by 30% hydrogen peroxide (H₂O₂, 24 h, room temperature) to remove humic substances. Grain-size fractionation was performed by passing the residues through a 7 µm nylon sieve to retain pollen grains. Ultrasonic treatment (40 kHz, 10 min) was applied to eliminate clay particles. The purified residues were then mounted and stored in 15 mL of 50% glycerol solution, sealed, and refrigerated at 4 °C until analysis. This workflow is consistent with internationally accepted palynological preparation protocols and incorporates improvements for optimizing pollen preservation and recovery [28,29].

Based on historical laboratory samples (Section 3.1), this study selected 23 common Quaternary fossil pollen taxa from the Tibetan Plateau to construct a pollen image database. Bright-field imaging was performed using an Olympus CX31 biological microscope (Olympus Corporation, Tokyo, Japan) equipped with a 40× objective lens (NA = 0.95) and a resolution of 2048 × 2048 pixels. Tests indicated that under this imaging condition, key micro-morphological features such as germination apertures could be effectively captured. A total of 1129 original microscopic images were acquired, within which 6616 individual pollen grains were manually identified and annotated by experienced palynologists (Figure 5).

Because pollen grains may undergo physical deformation during natural deposition and laboratory processing, and their apparent morphology may vary under the microscope due to orientation and focal differences (e.g., equatorial vs. polar views), class-specific data augmentation strategies were applied to enhance model robustness and generalization [30]. A total of 1129 original microscopic images were used as the source material, within which individual pollen grains were manually annotated by experienced palynologists. Based on these grain-level annotations, image-level data augmentation was subsequently performed using geometric and optical transformations, including flipping, color adjustment, scaling, cropping, Gaussian blur, and perspective transformation [31,32,33] (Figure 6). In this study, the term “original image count” (

Table 2. Composition of the TPPOL23 dataset. “Original image count” denotes the number of raw microscopic images acquired prior to data augmentation for each pollen taxon. These images constitute the primary dataset from which individual pollen grains were annotated and subsequently expanded through class-specific augmentation strategies.

Pollen Type	Original Image Count	Augmentation Factor	Augmented Image Count	Key Augmentation Methods (Intensity Parameters)	Augmentation Purpose
Apiaceae	1084	1.2	1300	Horizontal flip (p = 0.5)	Improve robustness to rotational symmetry features
				HSV color jitter (ΔH = ±15°, ΔS = ±0.1, ΔV = ±0.1)
				Random crop (scale 0.8–1.0)
Asteraceae	135	8	1080	Vertical flip (p = 0.5)	Simulate different section views, enhance radial pattern recognition
				Gaussian blur (σ = 1.5)
				Perspective transform (rotation ± 20°, scaling ± 15%)
Betulaceae	250	4	1000	Random scaling (0.7–1.3×)	Enhance invariance to aperture position
				Color inversion (p = 0.3)
				Local cropping (retain ≥ 70% area)
Brassicaceae	203	6	1218	Perspective transform (tilt ± 15°)	Simulate microscope slide tilt effects
				Gaussian noise (SNR = 25 dB)
				Motion blur (length = 5 px)
Caprifoliaceae	345	4	1380	Color normalization (histogram matching)	Suppress color bias, enhance mesh pattern consistency
				Center crop (80%)
				Random flip (p = 0.5)
Chenopodiaceae	315	4	1260	Elastic deformation (α = 50, σ = 5)	Improve generalization for irregular edge structures
				Color enhancement (saturation ± 20%)
				Multi-scale scaling (0.5–2.0×)
Corylus (Betulaceae)	181	7	1267	Affine transform (translation ± 10%)	Reduce overfitting, improve robustness of aperture ring detection
				Gaussian blur (σ = 0.5–2.0)
				Random occlusion (≤15%)
Cyperaceae	92	8	736	Perspective distortion (grid warp)	Simulate optical diffraction in thin-walled pollen
				Color channel shift (RGB offset ± 3 px)
				Frequency domain filtering (hybrid high/low pass)
Elaeagnaceae	726	1.5	1089	Random flip + crop combination	Improve multi-angle recognition of surface tubercles
				Color balance (white point correction)
				Motion blur (random angle)
Gentianaceae	91	8	728	Scaling + perspective joint transform	Enhance stability of striate patterns across resolutions
				CLAHE (Contrast Limited Adaptive Histogram Equalization)
				Local Gaussian blur
Hippophae	198	6	1188	Color space conversion (LAB enhancement)	Optimize representation of oily layer optical properties
				Random grid deformation
				Directional blur (along long axis)
Juglandaceae	353	4	1412	Non-rigid deformation (thin-plate spline)	Handle morphological variation due to folding
				Color jitter (brightness ± 10%)
				Random rotation (±180°)
Lamiaceae	375	3	1125	Multi-view perspective projection	Enhance symmetry-breaking feature extraction
				Color channel weighted blending
				Anisotropic Gaussian filtering
Lycopodiaceae	312	4	1248	Frequency domain enhancement (high-frequency boost)	Increase contrast of tiny germination structures
				Random cropping (retain spiny structures)
				Color normalization
Ostrya (Betulaceae)	103	8	824	Light Gaussian noise (σ = 0.5)	Prevent overfitting, preserve aperture ring authenticity
				Random flip (p = 0.5)
				Local color perturbation
Picea (Pinaceae)	527	2.5	1317	3D volume rendering view simulation	Correct morphological distortion due to angle in air sac structures
				Multi-scale Gaussian pyramid fusion
				Color channel shift
Pinus (Pinaceae)	177	6.5	1150	Selective enhancement of air sac regions	Improve classification specificity of bisaccate structures
				Polarized light effect simulation
				Non-uniform illumination synthesis
Poaceae	329	3.5	1151	Morphological dilation/erosion (kernel = 3 × 3)	Enhance rotational invariance of monoporate pollen
				Random perspective deformation
				Color space enhancement
Ranunculaceae	103	8	824	Focus stack simulation (multi-plane blending)	Handle randomness in surface granule distribution
				Color inversion (p = 0.2)
				Elastic deformation
Salix (Salicaceae)	238	5	1190	Frequency-spatial joint enhancement	Improve generalization for tricolpate pollen
				Random occlusion (≤10%)
				HSV color space perturbation
Ulmus	135	8	1080	Selective Gaussian blur	Compensate for spherical projection distortion, highlight aperture features
				Elastic deformation
				Spherical projection transform
Zygophyllaceae	209	5.5	1149	Morphological skeleton enhancement	Optimize edge feature extraction of spiny protrusions
				Color channel separation enhancement
				Motion blur synthesis
Daphne (Thymelaeaceae)	135	8	1080	Multi-directional lighting rendering	Enhance recognition of reticulate patterns under varying illumination
				Gaussian-Poisson noise mixing
				Non-rigid deformation

) refers to the number of individual pollen grains manually annotated from the original microscopic images for each taxon, rather than to the number of augmented training samples. As a result, the augmented dataset comprised 25,796 images (Table 2), and the coefficient of variation (CV) of sample distribution decreased from 0.618 to 0.159, indicating a substantial improvement in class balance among the 23 pollen taxa (Equation (1)).

To ensure reliability, 20% of the augmented images from each class were randomly selected for morphological validation by a panel of three palynological experts, and anomalous samples were excluded [34]. The final outcome is the Tibetan Plateau Quaternary Pollen Image Database (TPPOL23), consisting of expert-verified, manually annotated images (Figure 7). This dataset of 25,796 pollen images was subsequently used for deep learning model training.

The Coefficient of Variation (CV) is a statistical measure of relative variability [35]. It is calculated as the ratio of the standard deviation to the mean of a dataset. The formula is expressed as:

$$CV={\displaystyle \frac{\sqrt{\frac{1}{n-1}{\sum }_{i=1}^n{\left(x_i-\mu \right)}^2}}{\frac{1}{n}{\sum }_{i=1}^n{x}_i}}$$

(1)

where $CV$: Coefficient of Variation—a dimensionless measure of relative dispersion; $n$: Number of samples (i.e., the total number of data points or categories); $x_i$: The value of the $i^{th}$ sample (e.g., the number of augmented images in class $i$). $\mu$: The sample mean, calculated as $\mu ={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{x}_i$; ${\sum }_{i=1}^n{\left(x_i-\mu \right)}^2$: Sum of squared differences from the mean, used to compute variance. $\sqrt{{\displaystyle \frac{1}{n-1}}{\sum }_{i=1}^n{\left(x_i-\mu \right)}^2}$: Sample standard deviation, measuring the absolute dispersion of the data.

3.2. Model Improvements Based on YOLOv11

To address the characteristics of fossil pollen images from sedimentary deposits—namely, the subtlety of germinal aperture structures, the diversity of exine ornamentation, and the complexity of morphological distortion—this study proposes a deep residual network with dual-path attention, termed PollenSENet, which replaces the original backbone of YOLOv11. PollenSENet is built upon the ResNet-152 architecture, incorporating an improved channel–spatial collaborative attention module (Dual-Path Attention Block, DPAB) and an adaptive multi-scale pooling strategy [36,37]. These modifications significantly enhance the network’s capacity to capture key morphological features in microscopic pollen images.

3.2.1. Compression Operation

In feature extraction for pollen micro-images, the objective of the compression operation is to achieve efficient dimensionality reduction and weight allocation, thereby enhancing the model’s sensitivity to key discriminatory traits such as subtle germinal-aperture structures and the diversity of exine ornamentation. Unlike the traditional SENet formulation that relies on global average pooling (GAP), we introduce a dual-modal pooling strategy that combines Soft Pooling and Global Max Pooling (GMP) to mitigate the loss of local saliency that GAP can induce during feature compression [38,39]. Our subsequent experiments show that this strategy markedly improves feature retention and model generalization in pollen classification. Specifically, germinal-aperture attributes (e.g., aperture size and position) and ornamentation types (e.g., reticulate or spiny textures) often produce highly localized responses that GAP tends to homogenize. To address this, PollenSENet incorporates Soft Pooling, which dynamically evaluates the importance of each spatial position via exponential weighting. For an input feature map $U\in R^{H\times W\times C}$, the weight for activation $x_i$ is given by the Softmax function:

$$w_i={\displaystyle \frac{e^{x_i}}{{\sum }_{j\in U}{e}^{x_j}}}$$

(2)

thereby amplifying high-response activations and directing attention to salient pollen regions (e.g., aperture edges).

Based on the weights calculated by Equation (2), each weight $w_i$ is multiplied elementwise with the original activation $x_i$, and the results are summed to generate a compressed channel descriptor vector $y_c$ = $R^{1\times 1\times C}$ (Equation (3)).

$$y_c=x^{\prime }=\sum _{i\in U}{w}_i\times x_i$$

(3)

Compared with GAP, Soft Pooling exhibits a markedly reduced loss of discriminative information in pollen datasets, particularly in preserving delicate structural details such as the radial striations observed in Asteraceae pollen.

To compensate for the potential insensitivity of Soft Pooling to sparse features (e.g., isolated spiny structures typical of alpine steppe pollen), PollenSENet concurrently introduces Global Max Pooling (GMP), which enhances spatial invariance. For each channel of the feature map $U$, t the channel-wise descriptor is computed as

$$z_{max,c}=\underset{(h,w)}{max}{U}_{h,w,c},Z_{max}={\left[\begin{array}{c}{z}_{max,1},\dots ,z_{max,C}\end{array}\right]}^{\mathsf{\top }}\in {\mathbb{R}}^{1\times 1\times C}$$

(4)

By extracting local feature extrema, GMP strengthens the model’s robustness against morphological distortion (e.g., ruptured ornamentation due to diagenetic processes) while maintaining spatial invariance. Finally, the outputs of Soft Pooling ($y_c$) and GMP ($Z_{\mathrm{m}\mathrm{a}\mathrm{x}}$) are concatenated and adaptively fused according to:

$$Z_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{d}}=\alpha \cdot y_c+(1-\alpha )\cdot z_{max},\alpha \in [0,1]$$

(5)

where $\alpha$ is a learnable parameter optimized during backpropagation. The fused descriptor $Z_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{d}}$ thus combines the detail sensitivity of Soft Pooling with the noise resistance of GMP, effectively reducing feature dimensionality while enhancing the extraction of key morphological traits. This design provides a robust foundation for the subsequent excitation and calibration operations.

3.2.2. Excitation Operation of PollenSENet

The subtle variations in germinal aperture structures and the diversity of exine ornamentation in pollen micro-images necessitate a mechanism that can dynamically recalibrate channel weights and apply spatial modulation to strengthen discriminative feature responses [40,41]. To this end, PollenSENet embeds a Dual-Path Excitation Block (DPEB) into the residual units of ResNet-152, consisting of two synergistic branches: channel recalibration and spatial attention (Figure 8). The channel descriptor vector $Z\in R^{1\times 1\times C}$, obtained from the compression stage, serves as the input to this module, where $C$ is the number of channels.

The channel path employs a bottleneck structure that reduces and then restores dimensionality, using LeakyReLU and Hard-Sigmoid activations to obtain a sparse yet stable channel-weight vector:

$$Z_{\mathrm{m}\mathrm{i}\mathrm{d}}\&=\varphi (W_{1}Z+b_1),W_1\in {\mathbb{R}}^{{\displaystyle \frac{C\times C}{r}}},\varphi =\mathrm{L}\mathrm{e}\mathrm{a}\mathrm{k}\mathrm{y}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}(0.2)$$

(6)

$$A_{\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}}\&={\sigma }_h(W_2{Z}_{\mathrm{m}\mathrm{i}\mathrm{d}}+b_2),W_2\in {\mathbb{R}}^{C\times {\displaystyle \frac{C}{r}}},{\sigma }_h-\mathrm{S}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}$$

(7)

with reduction ratio $r=16$. The resulting channel-weight vector is broadcast to the spatial dimension and multiplied with the original feature map $U\in R^{H\times W\times C}$:

$$U_{recalibrated}=A_{\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}}\ast U$$

(8)

This recalibration enhances high-weight channels corresponding to germinal-aperture regions, while suppressing low-weight channels such as background sediment. The learned channel-attention distribution is visualized in Figure 9, where the left panel shows the input pollen image and the right panel displays $A_{\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{e}\mathrm{l}}$, highlighting apertures and exine ornamentation.

To further capture local details of exine ornamentation (e.g., reticulate meshes, spiny protrusions) and improve robustness to morphological distortion, the spatial branch applies a depthwise convolution (3 × 3) to $U_{recalibrated}$, yielding a spatial attention map:

$$M_{\mathrm{spatial}}=\sigma \left(f_{\mathrm{dw}}^{3\times 3}\left(U_{\mathrm{recalibrated}}\right)\right),M_{\mathrm{spatial}}\in R^{H\times W}$$

(9)

which is then fused with the recalibrated features:

$$U_{\mathrm{modulated}}=U_{\mathrm{recalibrated}}+\mathrm{Broadcast}\left(M_{\mathrm{spatial}}\right)$$

(10)

Here, $\sigma$ is the Sigmoid function. This dual-path mechanism (channel + spatial) enables dynamic optimization across both dimensions, thereby suppressing noise while preserving discriminative details. The properties of the activation functions (ReLU, LeakyReLU, Sigmoid, Hard-Sigmoid) are summarized in Figure 10 to support the design rationale of Equations (6) and (7).

Through the excitation operation, PollenSENet significantly enhances its ability to distinguish fine-scale pollen features, establishing a high-confidence feature representation for subsequent adaptive calibration.

3.2.3. Adjustment Operation of PollenSENet

On the basis of the multi-scale feature maps output from the excitation module (Figure 11), the goal of the adjustment operation is to perform dynamic weight allocation and feature fusion [42]. This design optimizes the model’s adaptability to morphological distortions in fossil pollen (e.g., exine folding or partial breakage caused by diagenesis) while simultaneously enhancing classification specificity for fine-grained structures such as germinal apertures and ornamentation edges.

Specifically, three groups of feature maps $\left[X_1^{\prime };X_2^{\prime };X_3^{\prime }\right]\in R^{1\times 1\times C}$, corresponding to different receptive scales of germinal-aperture and ornamentation features, are processed by a lightweight $1\times 1$ convolution to produce a weight matrix:

$$W={\mathrm{Conv}}_{1\times 1}!\left(\left[X_1^{\prime };X_2^{\prime };X_3^{\prime }\right]\right),\hspace{1em}W\in R^{3\times C}$$

(11)

The channel-wise importance of each scale is then normalized using an independent Softmax function per channel (scale attention), following the channel–spatial attention paradigm:

$$w_{k,c}={\displaystyle \frac{e^{W_{k,c}}}{{\sum }_{i=1}^3{e}^{W_{i,c}}}},\hspace{1em}k=1,2,3$$

(12)

Finally, the fused multi-scale representation is obtained as:

$$X_{\mathrm{fused}}=\sum _{k=1}^3{w}_{k,c}\cdot X_k^{\prime },\hspace{1em}{X}_{\mathrm{fused}}\in R^{1\times 1\times C}$$

(13)

To handle the common issue of fossil pollen morphological distortion, a deformable feature calibration is applied. A lightweight MLP predicts the sampling offset $\Delta p\in R^{3\times 3\times 2}$, which is used in a deformable convolution:

$$X_{\mathrm{calibrated}}=\mathrm{DeformConv}\left(X_{\mathrm{fused}},\Delta p\right)$$

(14)

The calibrated features are then merged with the original input via a residual connection to stabilize training under the ResNet backbone:

$$X_{\mathrm{final}}=X_{\mathrm{calibrated}}+X_{\mathrm{input}}$$

(15)

The output $X_{\mathrm{final}}$ serves as the input to the subsequent ResNet-152 layers for deeper feature extraction. This operation ensures robustness to deformed or broken exine structures while preserving the discriminability of fine-scale features.

3.3. Integration of PollenSENet with the YOLOv11 Framework

After the proposed adjustments and improvements, PollenSENet (i.e., ResNet-152 integrated with SE-based dual-path attention) was adopted as the backbone of YOLOv11, forming the fused model Pollen-YOLO. In this configuration, the original YOLOv11 backbone was replaced by PollenSENet, while the Neck (PANet) and Head (Detection Head) modules were preserved to ensure the stability of the detection framework. As illustrated in Figure 12, PollenSENet outputs multi-scale feature maps (C3, C4, and C5), which are subsequently fed into PANet for feature fusion. The fused features are then passed to the YOLO Head for bounding-box regression and class prediction.

Experimental evaluations indicate that Pollen-YOLO achieves higher detection accuracy and better generalization ability in Quaternary pollen identification tasks compared to the vanilla YOLOv11 baseline. These results verify the effectiveness of integrating PollenSENet with in the YOLOv11 framework, particularly in handling the morphological complexity of fossil pollen grains.

4. Results

4.1. Model Training Environment and Parameter Settings

In this study, the deep learning models were deployed locally on a 64-bit Windows 11 operating system. The training framework was PyTorch (version 2.1.0), running on an NVIDIA GeForce RTX 4070 Ti (12 GB) GPU with CUDA version 11.2. The implementation was written in Python 3.9, and code development and debugging were performed in PyCharm 2023.3.7.

To ensure experimental consistency, all models were trained without pre-trained weights [43]. The parameter settings were standardized across experiments, as summarized in

Table 3. Training parameters and experimental setup for Pollen-YOLO model.

Parameter	Value
Epochs	200
Batch size	16
Workers	4
Learning rate	0.001
Optimizer	Adam
Input image size	640
Ratio of training set to validation set	8:2

.

4.2. Evaluation Metrics

The model performance was evaluated using five standard metrics: loss function, precision, recall, mean average precision (mAP), and the F1 score (

Table 4. Evaluation metrics used in this study, including their mathematical definitions and analytical purposes.

Metric	Formula	Definition	Purpose
Loss function (L)	$L=-{\displaystyle \sum _{i=1}^C}{y}_i·\mathrm{l}\mathrm{o}\mathrm{g}\left(p_i\right)$	Measures the error between predicted and ground-truth values across C classes. In YOLO, it combines bounding box regression loss, classification loss, and distribution focal loss.	Evaluates training stability and effectiveness of learning rate scheduling.
Precision	$\mathrm{Precision}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$	Proportion of correctly predicted positives among all predicted positives.	Reflects the accuracy of positive predictions.
Recall	$\mathrm{Recall}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$	Proportion of actual positives correctly identified by the model.	Reflects the completeness of positive detection.
Mean Average Precision (mAP)	$\mathrm{mAP}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\mathrm{A}\mathrm{P}}_i$	The mean of AP values across all N classes. Includes ${mAP}_{50}$ (IoU = 0.5) and ${mAP}_{50–95}$ (averaged across IoU thresholds from 0.5 to 0.95).	Provides a comprehensive measure of detection precision and localization.
F1 Score	$F_1=2·{\displaystyle \frac{\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$	Harmonic mean of precision and recall.	Provides a balanced evaluation when precision and recall are uneven.

).

4.3. Ablation Study

To evaluate the effectiveness of the proposed improvements in fossil pollen classification, an ablation study was conducted on the TPPOL23 dataset, with the results systematically recorded for each experiment. YOLOv11 with its original backbone was used as the baseline model. Using a controlled variable approach, we gradually incorporated the dual-modal pooling strategy, the improved excitation module, and the multi-scale adjustment operation, and compared their performance against the standard SE module.

All experiments were carried out under identical conditions to ensure fairness: input image size of 640 × 640, initial learning rate of 0.001 with the Adam optimizer, batch size of 16, and 200 training epochs. Performance evaluation employed standard metrics including precision, recall, F1-score, mAP@0.5, and mAP@0.5:0.95.

Table 5. Ablation study results on the TPPOL23 dataset.

Experiment ID	Model Variant	Precision (%)	Recall (%)	F1-Score (%)	mAP@0.5 (%)	mAP@0.5:0.95 (%)
1	YOLOv11 (Original Backbone)	78.32	88.2	83.26	80.56	70.25
2	+ResNet-152 Backbone	78.59	89.5	84.05	82.32	72.1
3	+ResNet-152 + Dual-Modal Pooling (Ours)	84.5	91.1	87.8	81.69	73.95
4	+ResNet-152 + Standard SE Attention	88.8	90.3	89.55	93.2	73.2
5	+ResNet-152 + DPEB (Ours)	90.1	92.4	91.25	94.8	74.9
6	+PollenSENet (Full, Ours)	90.67	93.16	91.92	95.29	75.79

summarizes the results of the ablation study. The baseline YOLOv11 achieved precision of 78.32%, recall of 88.20%, F1-score of 83.26%, mAP@0.5 of 80.56%, and mAP@0.5:0.95 of 70.25%. Replacing the backbone with ResNet-152 led to moderate improvements, particularly in recall and mAP. Introducing the proposed dual-modal pooling strategy further boosted the F1-score by 3.75%, demonstrating its effectiveness in feature fusion. Incorporating the standard SE attention improved precision but yielded limited gains in mAP@0.5:0.95. In contrast, the proposed Dual-Path Excitation Block (DPEB) provided a more balanced enhancement in precision and recall, raising the F1-score to 91.25%. Finally, the complete PollenSENet (our full model) achieved the best overall performance, with precision of 90.67%, recall of 93.16%, F1-score of 91.92%, mAP@0.5 of 95.29%, and mAP@0.5:0.95 of 75.79%.

4.4. Comparison with Other Models

Comparison experiments were conducted between the proposed Pollen-YOLO and several mainstream object detection models on the TPPOL23 dataset. The compared models included YOLOv3, YOLOv4, YOLOv7, and the lightweight EfficientDet-D3 [44,45,46,47]. All models were trained and validated using the same hyperparameter settings [48]. Performance was evaluated using precision, recall, F1-score, mAP@0.5, and mAP@0.5:0.95.

The quantitative comparison results are presented in

Table 6. Comparison of different models on the TPPOL23 dataset.

Model	Precision (%)	Recall (%)	F1-Score (%)	mAP@0.5 (%)	mAP@0.5:0.95 (%)
YOLOv3	82.5	87.3	84.8	86.2	71.4
YOLOv4	86.9	89.2	88	90.5	73.6
YOLOv7	88.3	90.1	89.2	92.7	74.5
EfficientDet-D3	85.6	88.7	87.1	89.8	72.1
Pollen-YOLO (Ours)	90.7	93.2	91.9	95.3	75.8

. Among the baseline models, the YOLO series exhibited relatively strong detection performance, with YOLOv7 achieving the highest mAP values [49]. The proposed Pollen-YOLO achieved the highest performance across all evaluated metrics, with a precision of 90.7%, recall of 93.2%, F1-score of 91.9%, mAP@0.5 of 95.3%, and mAP@0.5:0.95 of 75.8%.

4.5. Full-Scale Evaluation of Pollen-YOLO

Full-scale experiments were conducted on the TPPOL23 dataset to evaluate the overall performance of Pollen-YOLO [50]. During training and validation, changes in precision and recall were monitored across epochs. Both metrics increased progressively from initial low values and reached higher levels toward the later stages of training, as shown in the evaluation curves.

4.5.1. Training and Validation Losses

During full-scale training, the training and validation losses decreased across epochs. As shown in Figure 13, the training losses (box_loss, cls_loss, and dfl_loss) decreased from 4.79, 11.66, and 3.59 in the first epoch to 0.74, 0.42, and 0.86 at epoch 200. Correspondingly, the validation losses declined from 2.95, 5.28, and 2.17 to 0.89, 0.56, and 0.93 over the same training period. Similar decreasing trends were observed for both training and validation loss components.

4.5.2. Model Performance Metrics

During full-scale training, changes in precision and recall were monitored across 200 epochs (Figure 14a,b). Precision increased from 0.23 at epoch 1 to 0.90 at epoch 200, while recall increased from 0.056 to 0.94 over the same period. The average precision metrics also exhibited increasing trends during training. The mAP@0.5 value rose from 0.01019 at epoch 1 to 0.9529 at epoch 200 (Figure 14c). The mAP@0.5:0.95 curve showed larger fluctuations during early epochs and gradually stabilized toward the later stages of training (Figure 14d).

The F1-score evolution during training is shown in Figure 15. The F1-score increased from 0.09 at epoch 1 to 0.70 within the first 20 epochs. Between epochs 20 and 60, the F1-score varied within the range of 0.60–0.83. From epoch 70 onward, the F1-score remained above 0.85 and continued to increase, reaching approximately 0.88 at epoch 100. In the later training stage (epochs 150–200), the F1-score stabilized and reached 0.919 at epoch 200. The average inference time of Pollen-YOLO on the TPPOL23 validation set was 3.1 ms per image, excluding input/output operations.

4.5.3. Class-Wise Performance on the Validation Set

Figure 16 illustrates the recognition performance of Pollen-YOLO on the validation set of the TPPOL23 dataset (5159 images), displayed as a heatmap in percentage form, where deeper blue indicates higher recognition accuracy for pollen types. These results are calculated based on the test dataset, which comprises 20% of the expert-annotated pollen grains randomly selected from the augmented TPPOL23 dataset and was not used during model training. On the validation set, Pollen-YOLO achieved robust overall performance (Precision = 0.906, Recall = 0.932, mAP@0.5 = 0.953, mAP@0.5:0.95 = 0.761), indicating a good balance between precision and recall. Notably, pollen taxa such as Juglandaceae, Caprifoliaceae, Picea, and Poaceae showed the most outstanding detection results, with Precision and Recall values approaching or exceeding 0.95, with almost no observable omissions. By contrast, taxa such as Lamiaceae, Ranunculaceae, and Ulmus exhibited relatively weaker performance, with mAP@0.5:0.95 only in the 0.5–0.6 range, reflecting the difficulty in distinguishing morphologically similar pollen groups. The confusion matrix further revealed the primary sources of misclassification for these weaker taxa, such as overlaps between Lamiaceae and Apiaceae or between Ranunculaceae and Ulmus, which led to classification errors. Overall, Pollen-YOLO achieved nearly perfect detection results for most taxa, while maintaining rapid inference speed (approximately 3.1 ms per image), underscoring its potential for large-scale applications in Quaternary palynology.

Table 7. Quantitative performance metrics of Pollen-YOLO for individual pollen taxa.

Class	Precision	Recall	mAP50	mAP50-95	F1-Score
Apiaceae	0.864	0.806	0.888	0.514	0.834
Asteraceae	0.872	0.976	0.955	0.742	0.921
Brassicaceae	0.917	0.89	0.952	0.718	0.903
Betulaceae	0.853	0.937	0.951	0.762	0.893
Caprifoliaceae	0.951	0.964	0.992	0.928	0.957
Chenopodiaceae	0.938	0.913	0.968	0.83	0.925
Corylus	0.929	0.902	0.956	0.806	0.915
Cyperaceae	0.903	0.953	0.968	0.827	0.927
Daphne	0.948	0.925	0.981	0.789	0.936
Elaeagnaceae	0.896	0.947	0.937	0.703	0.921
Gentianaceae	0.95	0.923	0.983	0.778	0.936
Hippophae	0.843	0.965	0.906	0.765	0.900
Juglandaceae	0.972	0.997	0.993	0.927	0.984
Lamiaceae	0.762	0.877	0.839	0.512	0.815
Lycopodiaceae	0.968	0.993	0.994	0.784	0.980
Ostrya	0.946	0.913	0.979	0.78	0.929
Picea	0.942	0.996	0.994	0.945	0.968
Pinus	0.953	0.886	0.934	0.809	0.918
Poaceae	0.923	0.98	0.985	0.839	0.951
Ranunculaceae	0.838	0.881	0.919	0.585	0.859
Salix	0.938	0.953	0.979	0.778	0.945
Ulmus	0.842	0.941	0.903	0.581	0.889
Zygophyllaceae	0.902	0.909	0.963	0.81	0.905

summarizes the quantitative evaluation metrics of Pollen-YOLO for each pollen taxon, including Precision, Recall, mAP@0.5, mAP@0.5:0.95, and F1-score. Overall, most taxa achieved Precision and Recall values above 0.90, with F1-scores stabilizing around 0.90, demonstrating balanced performance in single-class pollen recognition. Among them, Juglandaceae, Lycopodiaceae, Picea, and Caprifoliaceae achieved nearly perfect scores (F1-score > 0.96), indicating that the model is highly reliable in detecting and classifying these pollen types. In contrast, taxa such as Lamiaceae, Ranunculaceae, and Ulmus exhibited lower mAP@0.5:0.95 values (approximately 0.51–0.59), and their F1-scores were noticeably lower (<0.90), highlighting these groups as the weak points of the model.

Figure 17 presents examples of YOLO-POLLEN classification results on the TPPOL23 validation dataset. The examples include pollen grains observed under different imaging conditions, such as variations in illumination (bright-field and dark-field), focal clarity, and background complexity. In some cases, pollen grains appear partially visible or exhibit reduced contrast with the surrounding background. The detected bounding boxes indicate that pollen grains are identified across a range of visual conditions, including images containing background debris and laboratory residues.

4.5.4. Grad-CAM Visualization and Interpretability Analysis

Deep learning models for image recognition are often regarded as “black boxes,” with their internal reasoning processes remaining opaque [51]. While the YOLO-POLLEN model showed/exhibited strong performance in pollen classification, its decision-making pathway differs fundamentally from that of experienced palynologists, who rely on direct morphological cues such as size, exine ornamentation, and aperture structures. With the advancement of explainable AI, several techniques have been developed to visualize model attention and interpret feature contributions, including CNN feature visualization, dimensionality reduction methods such as t-SNE and UMAP, as well as perturbation-based saliency approaches such as LIME and SHAP [52,53,54,55,56]. Compared to these perturbation-based methods, Grad-CAM provides more intuitive and spatially coherent localization, making it particularly effective for highlighting class-discriminative regions [57,58].

By applying Grad-CAM to Pollen-YOLO, we reveal how the model focuses on biologically meaningful areas during classification. In full microscope images containing both pollen grains and background noise, high-response regions (shown in red) are concentrated on pollen structures, while background and debris remain in low-response regions (blue), indicating that the model has effectively learned to distinguish pollen from non-pollen information (Figure 18). When applied to individual pollen grains, Grad-CAM highlights critical morphological traits such as germinal apertures and exine ornamentation (Figure 19). These results confirm that Pollen-YOLO not only localizes pollen accurately but also captures discriminative structural features consistent with palynological expertise, thereby validating its biological interpretability and enhancing trust in its predictions.

4.6. Application to the Shaqu Profile

An age–depth model for the Shaqu profile was established based on six optically stimulated luminescence (OSL) dates (Figure 20). The dated horizons range from SQ-0 (4485 ± 320 yr BP at 40 cm) to SQ-5 (12,115 ± 780 yr BP at 200 cm), indicating that the sedimentary sequence spans approximately 12–4 ka BP. The age–depth model shows relatively stable sedimentation rates in the lower (180–200 cm) and middle (124–130 cm) sections of the profile, whereas the upper section (40–60 cm) exhibits greater variability (Figure 20).

Based on this chronological framework, pollen concentration diagrams were constructed for both manually identified and model-identified assemblages to enable direct comparison (Figure 21A,B). The two diagrams display broadly consistent stratigraphic patterns, including herbaceous dominance during the early stage, increased representation of coniferous taxa in the middle stage, and higher proportions of shrub taxa in the late stage [59]. For dominant taxa such as Poaceae, Picea, and Chenopodiaceae, the stratigraphic concentration trends derived from automated recognition closely match those obtained by manual identification (Figure 21).

To quantitatively assess differences between the two identification approaches, normalized relative errors were calculated and visualized as a heatmap (Figure 22). For most dominant taxa, relative errors remain low across the majority of stratigraphic levels. In contrast, larger relative errors are concentrated in morphologically similar taxa and in taxa occurring at low abundances. Notable discrepancies are observed for taxonomic pairs prone to morphological confusion (e.g., Betulaceae versus Corylus), as well as for rare taxa that are occasionally overestimated or omitted in the model-identified assemblages (Figure 22).

5. Discussion

5.1. Mechanisms Underlying Model Performance

The overall strong performance of the Pollen-YOLO framework can be attributed to the combined effects of architectural design, tailored data augmentation, and attention-based feature enhancement. By adopting an object detection paradigm rather than a classification-only strategy, the model is able to simultaneously localize and classify pollen grains in complex microscopic images containing impurities, overlapping particles, and heterogeneous backgrounds [61]. This design choice is particularly important for fossil pollen samples, where idealized single-grain images are rarely available.

The incorporation of class-specific data augmentation further contributes to model robustness. Fossil pollen grains frequently exhibit deformation caused by natural deposition, laboratory processing, and variable observation angles under the microscope. Augmentation strategies designed according to the morphological characteristics of different pollen taxa enable the model to better generalize across such variability, thereby reducing sensitivity to orientation, scale, and focal differences. This mechanism partially explains the stable recognition performance observed across a wide range of taxa.

Performance heterogeneity among pollen taxa highlights the inherent limitations imposed by pollen morphology. Taxa with distinctive ornamentation or aperture features are more readily recognized, whereas morphologically similar groups remain challenging. Explainability analysis based on Grad-CAM indicates that, for highly confusable taxa, the model tends to focus on overlapping surface structures, leading to classification ambiguity. These results suggest that the current model performance is constrained not only by network architecture but also by the intrinsic morphological similarity among certain pollen types.

The mechanisms discussed above are supported by quantitative training dynamics and ablation results. The convergence behavior and performance stability of the model are illustrated by the precision, recall, mAP, and F1-score trends shown in Figure 14 and Figure 15, while class-wise recognition differences are summarized in Figure 16 and Table 7. Ablation experiments (Table 5) show the quantitative contributions of individual architectural components, while Grad-CAM visualizations (Figure 18 and Figure 19) provide insights into the interpretability of the model’s feature learning. Several pathways may further improve discrimination capability in future work. From a data perspective, expanding both the quantity and diversity of annotated fossil pollen samples would allow the model to better capture subtle inter-taxon differences. From a methodological perspective, integrating more advanced multi-scale attention mechanisms or incorporating morphological priors such as pollen size and shape descriptors may enhance feature separability. Finally, from an application standpoint, combining automated recognition with expert verification offers a practical human–AI collaborative framework, in which high-throughput efficiency is complemented by domain expertise to ensure analytical reliability.

5.2. Implications of the Shaqu Application for Palaeoecological Reconstruction

The Shaqu case study indicates that automated pollen recognition can reproduce the primary stratigraphic signals for Quaternary vegetation reconstruction at a level comparable to expert manual identification. These palaeoecological comparisons are visually summarized in Figure 21 and Figure 22, which provide a direct stratigraphic evaluation of agreement and deviation between manual and model-based pollen identification. The close agreement for dominant taxa is particularly important because these taxa largely determine the first-order structure of pollen assemblages and, consequently, the main vegetation succession trends [62,63].

Discrepancies are concentrated in morphologically similar taxa and in taxa occurring at very low abundances (Figure 22), indicating that the practical reliability of automated recognition is taxon- and abundance-dependent. This pattern suggests that model-based identification is well suited for high-throughput reconstruction of major vegetation changes, while fine-grained interpretation of rare or highly confusable taxa may still benefit from targeted expert verification [1,13,64]. Therefore, automated recognition and manual analysis should be considered complementary: the model provides efficiency and objectivity at scale, whereas expert assessment provides calibration and quality control for challenging taxa.

5.3. Comparison with Existing Pollen Recognition Approaches

Automated pollen recognition has been investigated for more than two decades, with methodological paradigms evolving from handcrafted feature extraction to deep learning–based approaches [65]. Early studies typically relied on manually designed geometric or ornamentation descriptors combined with conventional classifiers such as support vector machines and random forests. Although these methods achieved moderate accuracy under controlled conditions, their strong dependence on expert-defined features limited their robustness and scalability when applied to large, taxonomically diverse fossil pollen datasets.

With the adoption of convolutional neural networks (CNNs), automated pollen recognition achieved substantial performance gains, particularly on standardized single-pollen image datasets [66]. However, most CNN-based studies focused on classification-only tasks and required prior manual segmentation or cropping of individual pollen grains [4]. As a result, these approaches did not explicitly address the challenges posed by real microscopic samples, including overlapping grains, impurities, and heterogeneous backgrounds. Consequently, reported accuracies from such studies are not directly comparable with results obtained from object detection frameworks operating on raw fossil pollen images [67].

By integrating localization and classification into a unified framework, we advance existing approaches; evaluation against representative models under identical conditions (Table 6) shows that our Pollen-YOLO achieves consistently superior performance across multiple metrics. More importantly, the model’s effectiveness is further supported by its application to a stratigraphic fossil pollen sequence, where automated results closely reproduce manually identified assemblage patterns (Figure 21) and exhibit low relative errors for dominant taxa across most stratigraphic levels (Figure 22). These results provide empirical evidence that object detection-based pollen recognition can operate reliably beyond idealized experimental settings.

Despite these advances, limitations remain when distinguishing morphologically similar or low-abundance taxa. As illustrated by the error distributions in Figure 22, confusion persists between taxa with overlapping morphological characteristics (e.g., Betulaceae versus Corylus), indicating that the discriminative capacity of automated models is constrained by intrinsic pollen morphology. In this respect, approaches based on deep feature embedding and clustering may offer complementary insights into inter-taxon relationships, although they are often less suited to high-throughput detection in complex images [68,69].

Overall, rather than replacing existing classification-based systems, Pollen-YOLO represents a complementary methodological pathway tailored to fossil pollen analysis under realistic laboratory and archeological conditions. By prioritizing robust detection in complex microscopic images and emphasizing genus- or family-level identification, the proposed framework aligns with the practical requirements of Quaternary palaeoecological reconstruction, where dominant taxa and long-term vegetation trends are of primary interpretative importance.

6. Conclusions

This study presents Pollen-YOLO, a deep learning-based object detection framework for automated pollen identification, and evaluates its performance on fossil pollen images from the Tibetan Plateau. The results indicate that, under the experimental settings adopted in this study, the proposed framework can achieve stable and reliable detection performance for most dominant pollen taxa, while maintaining high computational efficiency. Ablation analysis and interpretability experiments suggest that architectural design choices, attention-based feature enhancement, and class-specific data augmentation contribute to the observed model performance. At the same time, recognition accuracy remains constrained for morphologically similar or low-abundance taxa, reflecting intrinsic limitations associated with pollen morphology rather than model design alone. Application to the Shaqu fossil pollen sequence shows that automated recognition is able to reproduce major stratigraphic patterns and vegetation succession trends derived from manual identification within the examined profile. Taken together, these results suggest that deep learning–based object detection approaches have practical potential for assisting palaeoecological reconstruction, particularly in large-scale or time-intensive pollen analyses. Importantly, automated pollen identification should be regarded as a complementary tool rather than a substitute for expert analysis. Future work may focus on expanding annotated fossil pollen datasets, improving discrimination of morphologically similar taxa, and further integrating automated recognition with expert validation workflows to enhance reliability and interpretability in palaeoecological applications.