Explorable 3D Hyperspectral Models from Multi-Angle Gimballed LWIR Pushbroom Imagery

Highlights

What are the main findings?

• Co-registering gimballed pushbroom hyperspectral imagery with RGB frame camera data enables 3D reconstruction using commercial photogrammetric software.
• A texture-to-image mapping algorithm preserves the link between 3D model coordinates and original hyperspectral pixels, enabling retrieval of multi-angle spectra (8–50 viewing angles) for any point on the reconstructed model.

What is the implication of the main finding?

• Explorable 3D hyperspectral models allow for interactive analysis of how long-wave infrared spectral signatures vary with viewing angle, supporting material identification for non-Lambertian surfaces where single-angle observations may be insufficient.
• The workflow bridges the gap between specialized hyperspectral sensors and widely available photogrammetry tools, making multi-angle LWIR remote sensing more accessible for applications such as chemical detection, geological mapping, and environmental monitoring.

Abstract

Hyperspectral imaging in the long-wave infrared (LWIR) range enables identification of chemical compositions and material properties, but reconstructing 3D models from gimballed pushbroom sensors remains challenging because their unique acquisition geometry is incompatible with conventional photogrammetric software designed for frame cameras. This study presents a workflow for creating explorable 3D models from multi-angle LWIR hyperspectral imagery by co-registering hyperspectral line-scan data with simultaneously acquired RGB frame camera imagery using deep learning-based image matching. The co-registered images are processed in commercial photogrammetric software (Agisoft Metashape), and a texture-to-image mapping algorithm preserves correspondences between 3D model coordinates and original hyperspectral pixels across multiple viewing angles. Quantitative evaluation against reference data demonstrates that co-registration reduces geometric error approaching the accuracy of models built from high-resolution RGB imagery. The resulting models enable the retrieval of 8–50 spectral signatures per surface point, captured from different viewing geometries. This approach facilitates interactive exploration of angular variations in thermal infrared spectra, supporting material identification for non-Lambertian surfaces where single-angle observations may be insufficient for reliable classification.

1. Introduction

Remote sensing of gaseous chemical pollutants released during natural or human-made disasters requires robust methods for detection, tracking, and identification. Hyperspectral imaging in the long-wave infrared (LWIR) range offers unique capabilities for identifying the chemical composition of gaseous clouds or chemical spills. Since direct sampling of such phenomena is usually impossible, remote sensing methods must observe the target through the atmosphere, which contributes to the sensed signal. Removing the atmospheric contribution from the measured signal is necessary to match observed spectra with those in spectral libraries created under laboratory conditions. Beyond chemical identification, measuring the volume of chemical clouds or other 3D objects from thermal infrared hyperspectral imagery presents additional challenges that can be addressed through 3D reconstruction methods based on digital photogrammetry. Methods for spatiotemporal prediction and atmospheric correction of multi-angle hyperspectral data have been explored in previous work [1,2].

When multiple hyperspectral scans are acquired using a gimballed line scanner, these scans intersect and cover approximately the same area on the ground but with different looking angles and azimuthal orientations. When orthorectification or photogrammetric processing is performed, pixels become geometrically distorted. Specifically, corresponding pixels across different scans do not necessarily observe the same ground area. Even in the ideal case of square pixels whose centers point at the same ground location, the rectangular footprint representing each pixel will be rotated relative to those acquired from different azimuthal directions. This rotation causes some pixels to capture energy from adjacent surface areas, resulting in spectrally mixed measurements. Furthermore, for homogeneous materials, the spectra observed from different horizontal angles vary with both the azimuthal looking direction and viewing angle due to non-Lambertian surface properties. This research addresses the first part of this problem—spatial co-registration of different scans acquired with the hyperspectral scanning sensor—while the adjustment of collected spectra for changing pixel geometry remains beyond the current scope.

Most photogrammetric software is designed for frame cameras rather than pushbroom scanners since frame cameras are inexpensive, provide good spatial resolution, and are readily available for capturing visible-light imagery commonly used for 3D reconstruction. Pushbroom sensors are rarely used in photogrammetry, and processing their data requires accounting for more complex geometric models. On the other hand, most hyperspectral sensors are based on a pushbroom design, as it is more practical for sensors operating at high spectral resolution. Although flagship photogrammetric software packages support pushbroom imagery, no commercial or open-source software currently exists that can process imagery from gimballed hyperspectral pushbroom sensors—a capability essential for collecting images from multiple viewing angles. This gap exists because implementations such as the gimballed pushbroom sensor used in this study do not yet exist for commercial applications. Moreover, the existing software is not designed to create explorable 3D models. By explorable 3D models, we mean models that allow the user to identify which source images contribute to each point of the 3D model, enabling retrieval of the spectra from each source image.

Initial attempts to generate 3D models from hyperspectral imaging (HSI) yielded partial success, with disparate models created for each band not merged into a comprehensive model, as suggested in Liang et al. [3]. A more advanced approach was demonstrated by Zia et al. [4], who created a 3D model of a coffee can using a set of visible and near-infrared (VNIR) hyperspectral images. By creating separate models for each band, they successfully merged them into a singular model. But despite this progress, the popularity of hyperspectral photogrammetry among end users remains hindered by the high cost of sensors and the lack of integration of advanced tie point search algorithms (which consider spectral information) in commonly used commercial photogrammetry software. This deficiency is likely why most current research primarily relies on traditional software algorithms to handle hyperspectral imagery, rather than directly constructing 3D models from hyperspectral images. Consequently, the prevalent methods for creating 3D hyperspectral models involve texturing a model acquired via photogrammetry with the most contrast-rich band in the hyperspectral imagery or using visible-light imagery captured simultaneously with a separate RGB sensor or, in more extensive studies, laser scanning.

Mäkeläinen et al. [5] introduced an early prototype of a VNIR HSI drone camera equipped with a Fabry–Perot interferometer. They constructed models using the HSI imagery’s most contrast-rich band, leveraging commercially available photogrammetric software. Thanks to its lightweight nature and frame imaging geometric model, their 2D frame camera implementation demonstrated potential for seamless integration into photogrammetry workflows. Similarly, Markelin et al. [6] utilized the Visual SFM software to construct a 3D model of a forest from VNIR HSI imagery. Edelman and Aalders [7] also adopted this method, utilizing the most contrast-rich band and subsequent model texturing to build a 3D hyperspectral model in the VNIR range for crime scene documentation. A different method of 3D reconstruction using RGB imagery is demonstrated by Jurado et al. [8], who detailed a method for generating UAV-based hyperspectral mosaics using pushbroom sensors, texturing a model crafted from high-resolution RGB imagery. Kirsch et al. [9] and Salehi et al. [10] used terrestrial laser scanning to produce 3D models, which were then textured with HSI imagery. Ferrera et al. [11] demonstrated the application of HSI in underwater mapping, forming 3D models using imagery from an RGB camera and a pushbroom sensor manufactured by Ecotone (Trondheim, Norway).

Hyperspectral long-wave infrared (LWIR) remote sensing is often used to map outcrops and other geological features that are not distinct in VNIR hyperspectral imagery, as suggested by Kirsch [9]. Their study also substituted photogrammetric methods with ground-based laser scanning to create a 3D model. The camera is typically positioned on a land-based platform, such as a tripod or a ship along the coast, as demonstrated in [9,10]. Al-Khafaji et al. [12] enhanced the SIFT algorithm for co-registering hyperspectral images, which could increase the efficiency and precision of 3D reconstruction from hyperspectral images. Ahlberg et al. [13] presented a terrestrial hyperspectral scanning system that integrates a laser scanner and an HSI camera for 3D reconstruction.

Following these advancements, Aasen et al. provided a comprehensive review of the state of the art [14]. Also, a review paper by Kurz and Buckley [15] provided an additional overview of the role of hyperspectral imaging in close-range applications, specifically when building 3D models from hyperspectral imagery.

In addition to hyperspectral-native methods that operate on multiple hyperspectral bands to construct 3D models, numerous attempts have been reported to create 3D models using thermal imaging. Thermal images are, in some sense, similar to hyperspectral images used in this study in terms of low resolution and using the LWIR wavelength range. In these studies, 3D models are either generated by co-registering the high-quality model built from visible-spectrum imagery with the lower-quality models built from thermal imagery [16]. There are also attempts to improve the resolution of low-resolution thermal imagery by co-registering visible-light images taken at the same time with infrared imagery [17].

To address the limitations identified above, this study introduces a co-registration approach that leverages simultaneously acquired RGB frame camera imagery to transform gimballed pushbroom hyperspectral data into a geometry compatible with conventional photogrammetric software. Our primary contributions are: (1) a practical workflow for creating geometrically accurate 3D models from multi-angle LWIR hyperspectral imagery; (2) a texture-to-image mapping algorithm that preserves the link between 3D model coordinates and source hyperspectral pixels; and (3) interactive tools for multi-angle spectral analysis and material identification. We evaluate the geometric accuracy of the resulting models against reference data and demonstrate the utility of explorable 3D hyperspectral models for examining angular variations in thermal infrared spectra.

The remainder of this paper is organized as follows. The Data section introduces the datasets used in this study, including the Blue Heron gimballed hyperspectral sensor, the WAMI RGB camera, and the aiming camera imagery. The Methods section presents our workflow for creating 3D models from the gimballed pushbroom hyperspectral sensor, including the image co-registration process, texture-to-image mapping algorithm, and the material identification tools. The Results section provides both visual and quantitative assessments of the reconstructed models, comparing models built using our proposed method against those created using conventional frame sensor imagery, and demonstrates the practical application of explorable 3D models for multi-angle spectral analysis and material identification. The Discussion section interprets these findings in the context of prior work, examines the implications, and addresses the limitations and sources of uncertainty of the proposed approach. Finally, the Conclusions section summarizes the key findings.

2. Data

During the Nittany Radiance mission, different kinds of imagery were obtained: thermal hyperspectral imagery, 256 × 128 pixels, created by a gimballed, code-named “Blue Heron” line scanner, at specific points of interest over the Penn State campus. Visible-light imagery (RGB) was also captured simultaneously using two cameras: an aiming camera attached to a hyperspectral sensor assembly and a dedicated high-resolution wide-area motion imagery (WAMI) camera. Data collection mission was conducted on 18 April 2019 between 13:38 and 15:45 US Eastern time.

The Blue Heron hyperspectral sensor consists of two sub-sensors, named “side 1” and “side 2”. Side 1 images are in the 10–13 mm wavelength range and side 2 in the 11–14 mm range. Both sides have 256 bands with an increment of 0.1 mm. This sensor is a pushbroom line-scan sensor, but unlike conventional pushbroom sensors, which are attached to a flying platform immovably, this one is mounted on a gimbal, allowing it to image the ground scene at different angles in a short swath of about 400 lines. The scanner array has 350 pixels across the swath resolution. The sensor aims at the objects of interest, and then, the rotating head performs a quick rotating “pass” to capture a swath of interest. It takes about 1 s to capture an image consisting of 450 lines. The data collection mission is designed to collect data from the scene centered at specific geographic locations in the aircraft’s circular orbit. By varying the gimbal elevation angles, we collected overlapping hyperspectral images that display the scene object from different viewing angles. This multi-angle collection is valuable for identifying targets whose spectral signatures change with viewing angle; i.e., they do not closely follow the Lambertian assumption.

WAMI imagery was collected using the CorvusEye 1500CM sensing system made by Excelis Inc. (McLean, VA, USA). This system consists of four sensors that capture the imagery in visible and near infrared light, with a resolution of 6600 × 4400 pixels and a frame rate of 2 frames per second [18].

In addition to the hyperspectral sensors installed on the rotating head, there is a visible-light camera pointing in a direction generally aligned with the hyperspectral sensors. It is a frame-type camera with a resolution of 1920 × 1080 pixels that starts and stops image capture at the beginning/end of a data collection batch, with a framerate of about five frames per second. These types of images are shown in Figure 1.

During a separate data collection mission [19], high-resolution LiDAR data was captured across the entire Penn State campus on 11 July 2019. The LiDAR point density was 43.222 samples/m² with a point spacing of 0.1521 m.

3. Methods

Creating 3D models from hyperspectral imagery presents several challenges. First, most photogrammetric software packages are designed to process images from frame cameras. Those intended for processing line-scan cameras assume that the sensor is nadir-looking and statically fixed to the aircraft, flying at constant velocity. In our implementation, the sensor is gimballed and images the scene in a rapid sequence of short swaths, similar to panoramic line-scan cameras. This setup provides flexibility in image capture, allowing images to be captured at different angles. However, line-scan images lack the perspective distortions necessary for classic photogrammetry methods to work. Modern photogrammetry software, such as Agisoft Metashape (v. 2.2.0) [20], can work with line scanner images to a certain extent. Still, many of the tie points are rejected, as their expected spatial arrangement does not conform to the frame camera model. These differences are demonstrated in Figure 2.

The other issue is the limited resolution and details available in hyperspectral images. As digital photogrammetry relies on tie points, visible in many images, limited spatial resolution and a lack of texture details obstruct tie point search algorithms. There are significantly fewer tie points in the hyperspectral image pair compared to the visible-light image pair.

To overcome the limitation of the line-scan geometry, we have used the following workflow. This workflow is based on referencing and reprojecting hyperspectral images to a geometry that they would have if captured by a frame camera. Example images produced in different stages of the co-registration process are provided in Figure 3.

(1)

Images from both hyperspectral sensors, side 1 and side 2, were mosaicked to a single image to improve spatial resolution. The images have a constant 15-pixel overlap in the x-direction, but their vertical alignment varies with the viewing angle. We attempted to co-register them automatically, but due to the small overlap, some images failed to co-register. As a result, we resorted to manual registration, in which the operator must select the vertical offset for each image pair (106 in total).

(2)

We selected the images from the visible-light cameras that were roughly synchronized in time with the hyperspectral images. Since precise timestamps are not available for the visible-light images, but the visible-light camera starts and stops imaging at approximately the beginning and the end of the hyperspectral image acquisition sequence in the batch, for each hyperspectral image at position $n_{HS}$ within a batch, the corresponding visible-light image index, $i_{VIS}$, was determined by

$$i_{VIS}=n_{HS}\times {\displaystyle \frac{N_{VIS}}{N_{HS}}}$$

where $N_{VIS}$ and $N_{HS}$ are the total number of visible-light and hyperspectral images acquired in the batch, respectively.

This gave us approximate corresponding visible-light images, created at approximately the same time as the hyperspectral images. The full-size visible images were subset to the coordinates (upper left corner x, y and lower right corner x, y) [700:100] and [1300:900], approximately matching the extent of the hyperspectral images.

(3)

Hyperspectral image mosaics were automatically co-registered with the visible image subsets. We used the “MatchAnything” deep learning image registration model [21]. The MatchAnything deep learning model was selected for image co-registration after traditional methods (SIFT/SURF and ENVI’s automatic registration workflow) failed due to our images’ small spatial footprint and cross-modality appearance differences. MatchAnything is built upon two detector-free architectures: ROMA, which uses DINOv2 features with transformer-based dense matching, and ELoFTR, which employs a coarse-to-fine strategy with attention mechanisms. The model was pre-trained on diverse datasets (MegaDepth, ScanNet++, BlendedMVS, DL3DV, SA-1B, Google Landmark) using both multi-view geometry and synthetic cross-modal pairs (visible–thermal, visible–depth, day–night), enabling it to learn fundamental structural correspondences rather than appearance-based features. This approach achieved a significant improvement over baseline methods on cross-modal benchmarks and demonstrated strong generalizability across unseen multi-modal registration tasks. This makes this model well suited for our geospatial imagery, where conventional correspondence detection methods are failing. The model parameters used to run the registration process are listed in

Table 1. Image registration parameters used in “MatchAnything” image registration model.

Parameter	Value	Description
match_threshold	0.1	Minimum matching confidence threshold
extract_max_keypoints	1000	Maximum number of keypoints to extract
keypoint_threshold	0.015	Keypoint detection sensitivity threshold
key	matchanything_roma	Feature matching algorithm
ransac_method	CV2_USAC_MAGSAC	RANSAC outlier rejection method
ransac_reproj_threshold	8	Reprojection error threshold (pixels)
ransac_confidence	0.999	Required confidence level
ransac_max_iter	10,000	Maximum RANSAC iterations
choice_geometry_type	Homography	Geometric transformation model
force_resize	False	Image resizing flag
api_name	/run_matching	API endpoint identifier

.

(4)

Now we have co-registered hyperspectral image mosaics. Because they have frame-camera-conforming geometry, 3D models can be created directly from these images using photogrammetry. Using image mosaics facilitates co-registration between visible-light and hyperspectral images by expanding the image area. However, it also makes it harder to create texture-to-image mappings that link (x, y) coordinates to the source hyperspectral image (x, y) because of the overlap within the mosaic. To unambiguously create the mappings, we have co-registered original hyperspectral images (from side 1 and side 2 separately) to the registered image mosaics. We have used the same image registration settings in MatchAnything as before. The homography matrices that describe the mappings of the source line scanner image to the registered mosaic image were preserved. This allows us to map texture pixels to the original hyperspectral images when building the model from the registered images.

(5)

At this step, we could build models either directly from hyperspectral images, registered single-side images (s1 or s2), or from visible-light images. In our experiments, the best camera alignment is achieved when building the model from visible-light images. After that, camera alignment parameters can be reused to facilitate image matching.

The following models from different imagery sources were created:

• A model from visible-light images, then re-textured using the co-registered hyperspectral images.
• A model from registered mosaiced hyperspectral images. This works to some extent, but the camera alignment is not as good as that of the visible-light images.
• A model from the registered single-sided images. This method performs poorly, with significant issues in camera alignment.
• Models are created from mosaics and single registered images using the exported camera alignment from the visible-light model. Transferring the camera parameters from the visible-light project significantly improves the effectiveness of the tie-point search. By reusing the camera orientations, we could build reasonably accurate models from the “single-side” hyperspectral images and mosaics. The geometry of the model built from the registered mosaics appears to be better (with no false bridges between buildings), whereas using the single-sided registered hyperspectral images results in a rougher building geometry.

(6)

Models were exported in OBJ format for use in an interactive web-based viewer. Also, a mapping linking a texture x, y coordinate to the source hyperspectral image name and x, y coordinates was created using the procedure explained below.

3.1. Creating a Mapping Between Texture (x, y) Coordinates and Source File (x, y) Coordinates

Since we are interested in creating a 3D model from multiple overlapping images, one needs to know which pixels in the source images contributed to each texture pixel. The explorable 3D model works so that when the user clicks any part of the 3D model, we can translate those coordinates into 2D coordinates on the texture map. Texture coordinates are then used to resolve the set of records consisting of the (filename, x, y) in the corresponding source files obtained from the hyperspectral sensor. Since the images in the Agisoft Metashape photogrammetric project are not the original images but are geometrically transformed via image registration, we used the homography matrices saved during image registration to translate coordinates from the registered images back to the original coordinates used in the source images.

A Python script was created to work with Agisoft Metashape Professional (v. 2.2.0) [20]. It searches for pixel-level correspondences between the model texture and the source images. For each triangular face in the mesh, the algorithm extracts texture coordinates and 3D world coordinates of the three vertices, applies model transformations, and then determines visibility by projecting vertices onto each camera’s image plane. A face is visible when all projected vertices are within the image boundaries.

For visible faces, barycentric interpolation assigns texture pixels to the image coordinates. For point P in triangle V₁V₂V₃, coordinates (u, v, w) satisfy P = u·V₁ + v·V₂ + w·V₃ with u + v + w = 1 and u, v, w ≥ 0. Image coordinates are interpolated using the same weights: Pimg = u·I₁ + v·I₂ + w·I₃. The algorithm samples every pixel within each triangle and records texture coordinates, source filename, image coordinates, camera identifier, and camera-to-face distance for all visible cameras. In total, 8–50 cameras observe a typical texture pixel.

The script outputs data gradually into a CSV file, with each row containing eight fields: texture_x, texture_y, image_file, image_x, image_y, camera_id, face_idx, and camera_distance. When models have a texture resolution of 8192 × 8192, the dataset can reach between 50 and 200 million records.

The full procedure is presented in Algorithm 1.

Algorithm 1: Texture-to-image mapping

Input: 3D model M with faces F, cameras C, texture size T Output: Mapping records R 1:  for each face f ∈ F do 2:      Extract texture coordinates (u1,v1), (u2,v2), (u3,v3) 3:      Extract 3D coordinates V1, V2, V3 4:      Apply model transform if present 5:      visible_cameras ← ∅ 6:      for each camera c ∈ C do 7:          Project V1, V2, V3 → I1, I2, I3 using c.project() 8:          if all Iᵢ within image bounds then 9:              d ← distance (c.position, center (V1, V2, V3)) 10:              visible_cameras ← visible_cameras ∪ {(c, I1, I2, I3, d)} 11:          end if 12:      end for 13:      for each (c, I1, I2, I3, d) ∈ visible_cameras do 14:          for each pixel (x, y) in texture triangle do 15:              (u, v, w) ← barycentric ((x, y), texture_coords) 16:              if u, v, w ≥ 0 then 17:                  (img_x, img_y) ← u·I1 + v·I2 + w·I3 18:                  R ← R ∪ {(x, y, c.filename, img_x, img_y, d)} 19:              end if 20:          end for 21:      end for 22:  end for

The image coordinates in the generated CSV table are based on the co-registered images, not on the original pixel coordinates used in the hyperspectral images. To convert them back to the hyperspectral image coordinates, we use inverse homography matrices. For each record, the 3 × 3 inverse matrix H⁻¹ transforms warped coordinates through homogeneous multiplication, [x′, y′, w]^T = H⁻¹[x_warped, y_warped, 1]^T, followed by normalization: x_orig = x′/w, y_orig = y′/w. Since the RGB images are larger than the hyperspectral images, out-of-bound coordinates are rejected. Filenames are parsed to extract the batch identifier and image number for subsequent data access.

3.2. HDF5 Storage Structure

Transformed mapping data are stored in HDF5 format with three components enabling O(1) lookup (

Table 2. Records dataset structure (/records) storing all texture-to-image mappings.

Field Name	Data Type	Description
batch_id	uint32	Batch identifier
image_number	uint16	Image sequence number
image_x	uint16	X coordinate in source image
image_y	uint16	Y coordinate in the source image
camera_distance	float32	Camera-to-face distance (m)

). The records dataset contains all valid mappings sorted by distance within each texture coordinate group.

The coordinate index (

Table 3. Coordinate index structure (/index/).

Array Name	Dimensions	Description
coords	N × 2 (uint16)	Unique texture coordinate pairs (x, y)
start_idx	N (uint64)	Starting index in the records dataset
count	N (uint32)	Number of records per coordinate

) provides compressed indexing through three parallel arrays of length N, where N equals the number of unique texture coordinates with data.

A coordinate index consists of three parallel arrays: coords stores unique texture coordinate pairs (N × 2 uint16), start_idx stores starting positions in the records dataset (N uint64), and count stores record quantities per coordinate (N uint32). A 2D lookup table that matches the texture dimensions provides the method to access the records. For coordinate (x, y), lookup_table [y, x] returns −1 for no data or an index into the coordinate arrays. This enables O(1) access: one lookup determines data existence, and a second retrieves the record range.

ENVI-format hyperspectral cubes are converted to HDF5, with each image stored as a separate group named by the batch and file identifiers. Groups (

Table 4. Hyperspectral HDF5 group structure. Each group is named by batch and file identifiers (e.g., /182235_0000/).

Dataset	Type/Dimensions	Description
/{batch}_{file}/data	H × W × B (float32)	Hyperspectral cube (gzip level 6)
/{batch}_{file}/angle	scalar (float32)	Gimbal elevation angle (degrees)
/{batch}_{file}/shape	3 (int32)	Cube dimensions [H, W, B]

) contain the data array (height × width × bands, float32, gzip level 6), gimbal elevation angle, and cube dimensions. This organization provides 5–8× compression and enables the extraction of single-pixel spectra through HDF5 chunked storage without loading complete images.

3.3. Backend for a Web Application

A Flask backend enables the web viewer to retrieve and visualize the data stored in HDF5 format. When a user clicks the 3D model, the viewer determines texture coordinates and sends them to the backend. The backend retrieves the corresponding multi-angle spectra using the look-up table. A response that includes retrieved spectra, base64-encoded preview images, gimbal angles, and camera distances, serialized to JSON, is returned to the frontend. Client-side JavaScript renders interactive plots with multiple lines, color-coded by viewing angle. HDF5 files remain open throughout the server’s lifetime; frequently accessed datasets are cached in memory. The backend is implemented using Python 3.8+, h5py 3.7+, NumPy 1.23+, and Flask 2.2+. The frontend is implemented using HTML and JavaScript (d3.js) for data visualization.

3.4. Spectral Preprocessing and Library Matching for Material Identification

One practical scenario for explorable 3D models built from multi-angle imagery is that they can be used to identify different materials, not by using a single image showing the targets of interest, but by using many images that intersect at the specific point of interest and capture the same target pixels under different angles. Having multiple spectral signatures of the same object observed in multiple images may increase the confidence of material identification by considering multiple spectra “voting” for the outcome—the material name in question one wants to identify.

To demonstrate material identification across multiple images, we first preprocessed the radiance data. Calibrated at-sensor radiance data was atmospherically corrected using the ISAC in-scene atmospheric correction method [22]. ISAC allows retrieval of an unscaled, atmospherically corrected radiance spectrum. Unscaled means that the spectra are consistent with the spectral shape, but the resulting transmission and upwelling profiles have arbitrary magnitude. This means that one cannot use it to estimate temperature and emissivity directly. Since the spectra have a consistent shape, they can still be used to match them to the sample spectra in spectral libraries. To facilitate this matching, we have transformed the ISAC-corrected radiance into alpha-residual [22,23] spectra space—this transformation allows us to center the input spectra at zero and removes the dependence on the input temperature. This is done using the following equation:

$$\alpha \left(\lambda \right)=-c_2\left[{\displaystyle \frac{1}{T_b\left(\lambda \right)}}-\overline{\left({\displaystyle \frac{1}{T_b\left(\lambda \right)}}\right)}\right]$$

where c₂ is a second radiation constant (approximately 1.4388 × 10⁻² m·K), and T_b(λ) is a brightness temperature at wavelength λ.

The emissive material spectra from the ECOSTRESS spectral library [24] were also converted to the alpha residual spectra using the following equation:

$$\alpha \left(\lambda \right)=\lambda ln\epsilon \left(\lambda \right)-\overline{\lambda ln\epsilon \left(\lambda \right)}$$

where λ is the wavelength, and ε(λ) is the spectral emissivity at wavelength λ. Overbar (¯) notation indicates finding a mean over a spectral region with reliably measured radiance and library emissivity. We used a spectral range of 10–11 microns to calculate the mean for both the ISAC-corrected radiance and the library spectra.

Having transformed both corrected radiance and library spectra to alpha-residual space, the commonly used SAM (spectral angle mapper) [25] metric was used to match the image spectra to the library spectra. SAM is defined as follows:

$$\alpha ={\mathit{cos}}^{-1}\left({\displaystyle \frac{\sum _{i=1}^n{t}_i·r_i}{\sqrt{\sum _{i=1}^n{t_i}^2·\sum _{i=1}^n{r_i}^2}}}\right)$$

where α is the spectral angle between the test spectrum and reference spectrum, tᵢ is a value of the test spectrum (pixel) in band I, rᵢ is a value of the reference spectrum in band I, and n is the total number of spectral bands. I is the band index (ranging from 1 to n). The lower the resulting spectral angle value, the closer the image spectral vector is to the library spectral vector.

4. Results

Using the imagery and methods introduced above, we have created the following sets of 3D models with different textures.

(1)

A hyperspectral model (further referred to as HS) was created from the hyperspectral image data converted to RGB composite images consisting of bands 194, 126, 23 using the Agisoft Metashape Professional (v. 2.2.0) software [20].

(2)

An RGB model (further referred to as RGB) was created using the imagery from the CorvusEye 1500CM camera pointed at the center of the PSU campus (Old Main building).

(3)

An aiming camera model (further referred to as AIM) was created using visible-light imagery from the aiming camera mounted on the same gimballed platform as the Blue Heron hyperspectral sensor. In this model, full-sized (1920 × 1080) images were used.

(4)

A LiDAR model was created by subsetting the extent of the area of interest, consisting of Walker, Deike, ARL, parts of Westgate, and Hoisler buildings from the original LiDAR scanning LAS file 23001935.las. The data was retrieved from the Pennsylvania Spatial Data Access portal (PASDA) [19], and the model was built using Poisson reconstruction in the CloudCompare 2.13.2 Kharkiv software [26].

(5)

A model from the subsets of the images of the RGB aiming camera (referred to as VIS-COREG). This model was built from the subsets of RGB images. The extent of the subsets is roughly equivalent to the extents of the s1 and s2 hyperspectral mosaics by width and about 2× the height. We ensure that the RGB subset fully encompasses each mosaic extent.

(6)

A model from the Blue Heron hyperspectral images (referred to as HS-COREG). Images from both subsensors (side 1 and side 2) were mosaicked together and referenced to subsets of the aiming camera images to correct for geometry. Then, the S1 images were referenced to the referenced mosaics. Camera orientation was imported from the VIS-COREG project to facilitate the search for the tie points.

Models 1–4 were re-textured with RGB or hyperspectral textures, resulting in an HS model with HS and RGB textures, an RGB model with HS and RGB textures, and a LiDAR model with HS and RGB textures. This was accomplished by creating two Agisoft Metashape projects—one from HS images and another from RGB images. First, the HS and RGB models were exported as OBJ files from the corresponding projects. We manually co-registered the HS model with the RGB model, imported the co-registered HS model into the RGB project, and built a texture. The exact process was applied to the remaining models. Generally, we followed this procedure:

(1)

To build RGB texture on HS model: Export HS model—co-register with RGB model—import to RGB project—build texture—export model with RGB texture—apply reverse registration to match exported model with RGB texture with HS.

(2)

To build HS texture on RGB model: Export RGB model—co-register with HS model—import to HS project—build texture—export model with HS texture—apply reverse registration to match exported model with RGB texture with HS.

Models 5 and 6 were retextured more straightforwardly because the visible-light and hyperspectral images were co-registered. In these cases, we were able to point the Agisoft Metashape to the new images (i.e., in case of a model built from the hyperspectral images registered to the aiming camera RGB subset, we replaced them with the RGB image subsets and vice versa).

4.1. Image Alignment

As shown in Figure 4, the flight orbit layout differs. It is expected that all images will be aligned circularly as the aircraft was flying in a circular orbit. We can see in the inset (a) that the images are aligned circularly. Still, there are certain defects in the alignment, such as too many images being wrongly aligned to the same spatial location in the orbit, as well as places where there are no images at all. Next, the model built from the WAMI imagery—whose layout is shown in inset (b), as well as the models constructed from the RGB aiming camera image subsets (c) and images built from the registered hyperspectral imagery (d)—does not show such defects in the image alignment.

4.2. Visual Assessment of the Resulting 3D Models

In Figure 5, one can see a visual depiction of 3D models created by different methods. One would expect the bottom of the models to be relatively flat, as they all represent the flat ground surface with buildings. In inset (a), a model built directly from the hyperspectral images is shown. From the visual inspection, this model looks to be far from reality, with the part folding down where it is supposed to be flat. We can attempt to re-texture this model with the visible-light texture shown in inset (b), but it still shows all the geometric artifacts visible on the original model. In inset (c), we have attempted to re-texture the model built from the high-resolution RGB imagery with the texture from the hyperspectral model. As one can see in inset (d), the RGB model buildings look closer to the expected building shape, but due to inaccurate camera alignment, the hyperspectral texture is missing on some buildings, as evident in inset (c).

Figure 6 depicts the models built from the hyperspectral imagery that were co-registered with the frame camera imagery to conform to the frame sensor model. In inset (a), there is a model built from hyperspectral imagery. The geometry of the buildings looks more reasonable than that of the models constructed directly from the line scanner hyperspectral imagery without co-registration. However, there are still unwanted bridges between buildings. In insets (c) and (d), there is a model built from the visible-light images and retextured with hyperspectral and visible-light textures. These models do not exhibit unwanted geometry artifacts between buildings.

4.3. Qualitative Comparison of Model Reconstruction with Non-Photogrammetric Methods

The methods discussed above are based on classical photogrammetric approaches, where the resulting 3D point locations are computed through triangulation of corresponding features identified across multiple overlapping images. On the other hand, other deep-learning-based methods that directly infer camera poses and point clouds from input imagery are rapidly emerging. We attempted to use the Facebook VGGT [27] model, which is considered a state-of-the-art model for 3D reconstruction, to perform 3D reconstruction from the hyperspectral and visible-light imagery used in this study. The results of the reconstruction via VGGT are presented in Figure 7.

Examining the results, it is evident that, unlike photogrammetric methods, the VGGT model failed to reconstruct a 3D model from the multi-angle imagery. The results obtained from the transformed hyperspectral imagery (referred to as HS above) are entirely unusable. The source scene appears to be organized circularly, as expected given that the source images were collected from a circular orbit; however, the scene appears flat, with points placed in the scene in disjointed layers, as though each source image were simply overlaid onto its neighbors. The reconstruction attempt from the visible-light images acquired by the aiming camera (referred to as AIM above) more closely resembles the actual scene geometry but still suffers from numerous geometric artifacts. Buildings visible across multiple images appear segmented, with seams coinciding at the boundaries between adjacent images. Roads are segmented similarly.

Comparable results were observed with NVIDIA Instant NGP [28]. It appears that these models were trained on conventional RGB imagery from consumer electronic devices such as smartphones and are not well suited for reconstructing scenes from multi-angle aerial imagery.

4.4. Quantitative Assessment of Different Models’ Geometric Reconstruction Accuracy

Model reconstruction quality: The reference model (LiDAR) was generated from the LiDAR data using Gaussian reconstruction in the CloudCompare 2.13.2 Kharkiv software. It was later used as a baseline model for further evaluations (Figure 8).

Topographic profiles extracted along two transects (Profile A and Profile B) showed geometric variations across reconstruction methods (Figure 9).

The direct hyperspectral model (HS) shows noticeable geometric artifacts (see Profile A), with elevation spikes exceeding 100 m between coordinates 550 and 650. In contrast, both co-registered models (HS-COREG and VIS-COREG) demonstrated intermediate accuracy with occasional departures of 10–15 m at terrain transitions. The AIM model closely follows the reference surface, with deviations typically under 5 m, while the RGB WAMI model shows somewhat larger deviations.

Topographic reconstruction accuracy: Visual assessment of the reconstructed digital elevation models shows pronounced differences in surface quality and appearance (Figure 10).

The direct hyperspectral reconstruction (Figure 10a) produced a fragmented topography with abrupt elevation changes ranging from 256 m to 382 m across the scene, whereas the other methods produced relatively smooth surfaces. The RGB model (Figure 9b) generated the most uniform surface with consistent elevation values around 320–340 m. The RGB model (Figure 10b), as well as the co-registered models (Figure 10c,d), preserved major topographic features. In contrast, the directly reconstructed hyperspectral model (Figure 10a) showed a pronounced rough depression in the southern part (upper-right quadrant).

To quantify reconstruction accuracy relative to the reference (LiDAR) model, we calculated point-to-surface distances across all the models (

Table 5. Quantitative evaluation metrics. Comparison of 3D reconstruction quality metrics across different models. AIM (GT) represents the ground truth model. All distance measurements are in meters.

Model	Vertices	Triangles	MAE	RMSE	Std Dev	90th %ile	95th %ile	Mean Signed Dist
LiDAR (GT)	17,056	33,383	0	0	0	0	0	0
HS	12,218	23,853	18.221	23.898	15.462	38.335	46.163	1.240
RGB	4679	9080	7.331	9.380	5.852	14.127	17.159	3.704
AIM	12,731	21,598	3.305	4.806	3.489	4.643	6.474	0.793
HS-COREG	10,785	21,187	5.222	6.999	4.660	9.856	14.076	0.944
VIS-COREG	2744	5265	3.692	4.945	3.289	6.043	7.989	0.856

).

The direct hyperspectral model demonstrates a mean absolute error (MAE) of 18.22 m with root mean square error (RMSE) reaching 23.90 m, which is the worst across all models in our tests. The AIM model achieves the best reconstruction accuracy (MAE: 3.31 m, RMSE: 4.81 m), followed closely by VIS-COREG (MAE: 3.69 m; RMSE: 4.95 m). The RGB model shows moderate deviation (MAE: 7.33 m; RMSE: 9.38 m). Co-registration of hyperspectral imagery reduced distance errors, achieving an MAE of 5.22 m and an RMSE of 7.00 m for HS-COREG.

Error heatmaps (Figure 11) show the spatial distribution of surface differences.

The hyperspectral model exhibited widespread errors exceeding 40 m, concentrated along the northern and eastern scene boundaries. The RGB model maintained errors, mostly below 15 m across most of the scene area.

Statistical error characteristics: Error distribution analysis revealed the following differences in reconstruction behavior (

Table 6. Error distribution analysis. Distribution of point-to-surface distances relative to ground truth. Positive bias indicates points predominantly above the ground truth surface; negative bias indicates points predominantly below.

Model	Points Above GT	Points Below GT	Mean Bias
HS	6467 (52.9%)	5751 (47.1%)	1.240
RGB	3370 (72.0%)	1309 (28.0%)	3.704
AIM	7573 (59.5%)	5158 (40.5%)	0.793
HS-COREG	5583 (51.8%)	5202 (48.2%)	0.944
VIS-COREG	1532 (55.8%)	1212 (44.2%)	0.856

).

The cumulative error distribution showed that 90% of RGB points achieved sub-14.13 m accuracy, while the same percentile for direct hyperspectral reached 38.34 m. Co-registration significantly compressed the hyperspectral error distribution, with 90th-percentile errors reduced to 9.86 m for HS-COREG. The signed distance distribution shows that the direct hyperspectral model exhibited a slight positive bias (+1.24 m mean signed distance) with 52.9% of points above the reference surface. In contrast, HS-COREG showed a slight positive bias (+0.94 m) with 51.8% of points above the reference.

Hausdorff distance measurements are provided in Figure 12 and

Table 7. Extended error metrics, including median, percentiles, and maximum distances. All measurements are in meters.

Model	MAE	RMSE	Median	Std Dev	P90	P95	Hausdorff
HS	18.221	23.897	14.102	15.462	38.335	46.163	148.662
RGB	7.331	9.380	5.616	5.851	14.127	17.159	47.167
AIM	3.305	4.806	2.705	3.489	4.643	6.474	37.923
HS-COREG	5.222	6.999	3.699	4.660	9.856	14.076	60.478
VIS-COREG	3.692	4.945	2.953	3.289	6.043	7.989	39.121

.

The direct hyperspectral model produced extreme outliers with a Hausdorff distance of 148.66 m, roughly three times greater than the RGB baseline (47.17 m). Co-registration reduced the maximum error to 60.48 m for HS-COREG, though it remained about 1.3 times larger than RGB performance. The VIS-COREG approach achieved a Hausdorff distance of 39.12 m.

To provide a clearer picture of the advantages delivered by the proposed co-registration method, we examine the specific improvements that co-registration methods provide compared with building the 3D models directly from unprocessed pushbroom imagery (

Table 8. Relative improvement of reconstruction methods over the direct hyperspectral (HS) baseline. All error reductions are percentages; bias and standard deviation are in meters.

Model	MAE Reduction (%)	RMSE Reduction (%)	P90 Reduction (%)	Std Dev (m)	Mean Signed Dist (m)	Multi-Angle Spectral Retrieval
HS (baseline)	-	-	-	15.46	+1.24	Yes
RGB	59.8	60.7	63.2	5.85	3.7	No
AIM	81.9	79.9	87.9	3.49	0.79	No
HS-COREG	71.3	70.7	74.3	4.66	0.94	Yes
VIS-COREG	79.7	79.3	84.2	3.29	0.86	Yes

).

HS-COREG reduces MAE by 71.3%, RMSE by 70.7%, and 90th-percentile error by 74.3% relative to the HS baseline. It transforms a geometrically unreliable model (MAE: 18.22 m, with elevation spikes exceeding 100 m) into one with accuracy (MAE: 5.22 m) that surpasses the high-resolution RGB WAMI model (MAE: 7.33 m) despite being derived from lower-resolution imagery with a smaller spatial footprint. VIS-COREG achieves even higher accuracy (MAE: 3.69 m; RMSE: 4.95 m), approaching the performance of the AIM model (MAE: 3.31 m), and, in addition, enables multi-angle spectral retrieval.

4.5. Material Identification Using Spectral Signatures Captured in Multi-Angle Imagery Collection

To facilitate user interaction with explorable 3D models and demonstrate their application for multi-angle spectral classification, we have developed a web application for 3D model viewing. A screenshot of its main window is shown in Figure 13. The following capabilities are available: One can select the desired model from the drop-down list and the desired texture. One can zoom in and out, rotate the displayed model, and click any point on the model to retrieve a multi-angle spectrum. When the spectra are retrieved, it is possible to inspect different types of spectra, associated with the specific model location, such as uncorrected radiance, atmospherically corrected radiance spectra (using ISAC), and emissivity spectra (retrieved using the methodology in [2]), as well as alpha residual spectra. The panel in the top-left corner lists all the source hyperspectral images contributing to the model location, along with their names, angles, and pixel coordinates in the original images. The panel in the top-right corner displays image thumbnails of the original images around the location of interest. Below the preview images, there is a multi-angle spectra plot that shows spectra from each image as lines of different colors, along with a legend explaining the color-to-angle correspondence. When the alpha residual spectra product is selected (the Grow tool works with all types of displayed spectra, as will be described in more detail later), the Match, Vote, and Grow tool buttons are available, which cause the corresponding tools to further analyze the displayed extracted spectra.

To enable quick material identification, based on the background provided in the “Methods” section, we have created three tools, called “Match”, “Vote”, and “Grow”.

The Match tool allows spectral matching between the ECOSTRESS library and averaged spectra observed from multiple angles. When the user clicks the specific location in the model, spectra from the overlapping source images are extracted. We compute the median spectrum and then compute SAM between each library spectrum and the extracted median spectrum. The output lists the closest and the top-5 next-closest spectra, ordered by SAM value, as well as a plot showing the identifying spectra and library matches. A screenshot of the Match tool is presented in Figure 14.

Vote tool—allows for spectra matching between the library and individual spectra from multiple angles. Instead of averaging, this tool performs matching for every single spectrum, extracted for the target of interest. After each spectrum is matched, we count “votes”—how many spectra belong to which material. The material with the top votes is considered the “winner”. To speed up the tool, spectra classification is performed in a multi-threaded fashion. A screenshot of the Vote tool is provided in Figure 15.

The “Grow” tool allows for exploration of similar spectra in the source images. After the multi-angle spectra are extracted, and this tool is invoked, the main window of the tool is shown. It shows a list of images that intersect at the clicked point. The location, corresponding to the clicked point in the 3D model, is marked in the source image. Then, we calculate the RMSE between the spectra from the clicked point and every other pixel in the image. Pixels whose RMSE values are below the specified threshold are highlighted in the image. Also, we display a plot showing the number of pixels vs. their RMSE values. The slider above the plot allows one to select the desired RMSE value, which is also displayed in the plot by the red dashed line. A screenshot of the Grow tool is provided in Figure 16.

All of these tools are implemented in the Python backend. They accept inputs and return the results as serialized JSON data so that the outputs can be displayed in the frontend web application.

Figure 17 demonstrates that emissivity spectra extracted through the proposed framework preserve the original spectral characteristics, with individual measurements from different viewing angles (gray lines) showing strong consistency around the mean spectrum (black line). The angular spread varies by material, reflecting inherent directional emissivity differences, while SAM-based library matching confirms the extracted spectra maintain physically meaningful spectral signatures. Spectral features in low-emissivity regions appear less pronounced than in the library spectra due to a known limitation of the ISAC atmospheric correction method, which neglects downwelling radiance.

5. Discussion

5.1. Interpretation of Geometric Reconstruction Accuracy

To conclude these geometric accuracy findings, it appears that reconstruction accuracy depends on both the spatial coverage (footprint area) of individual images and the geometric quality of the input images. Aiming camera imagery, as well as WAMI RGB imagery, has the widest spatial coverage (large footprint). At the same time, the HS sensor produces imagery in narrow, short swaths, combined with its unique sensor geometry, resulting in a spatial footprint much smaller than that of the above-mentioned RGB images. These narrow swaths make it challenging to find sufficient tie points, with most of these points subsequently rejected as non-conforming to the expected frame camera geometry. While the co-registration technique explained above increases the geometric accuracy of the models, the spatial footprint of the images stays the same as for the original HS images, which compromises the resulting geometric accuracy of the co-registered models compared to the models created from large-footprint RGB imagery.

The direct HS model exhibits a nearly symmetric but very wide error distribution (standard deviation: 15.46 m), indicating spatially incoherent errors from poor camera alignment. Co-registration compresses this spread significantly—to 4.66 m for HS-COREG and 3.29 m for VIS-COREG, while preserving near-symmetric, low-bias distributions (mean signed distance: +0.94 m and +0.86 m, respectively). By comparison, the RGB model exhibits a higher positive bias (+3.70 m), with 72.0% points above the reference surface.

The two co-registered models serve complementary use cases. HS-COREG is applicable when visible-light imagery from a co-mounted camera is not available; in such scenarios, a mathematical camera model could potentially transform pushbroom imagery directly into frame-camera-compatible geometry, though this remains a subject for future work. VIS-COREG is applicable when simultaneously acquired high-resolution RGB imagery is available, allowing the model to be built from RGB images and then re-textured with co-registered hyperspectral imagery. Both co-registered approaches, along with the direct HS model, support multi-angle spectral retrieval, which is absent from the RGB and AIM baselines. Therefore, these models can be used for material identification on non-Lambertian surfaces, such as using the discussed Match, Vote, and Grow tools.

5.2. Multi-Angle Spectral Classification

The spectral classification approach employed in this study, averaging multi-angle emissivity spectra or applying majority voting across individual spectral comparisons, was kept straightforward. The primary contribution of this work lies in establishing the geometric correspondence framework between 3D models and multi-angle hyperspectral imagery rather than advancing classification methodology per se. We acknowledge that more sophisticated approaches could enhance classification performance. These might include quality-weighted spectral matching, where individual spectra are weighted by factors such as viewing angle, signal-to-noise ratio, or measurement uncertainty; Bayesian classification frameworks that explicitly incorporate the angular variability of emissivity; or machine learning methods that leverage angular spectral variation patterns as discriminative features. The available emissivity spectral libraries, such as ECOSTRESS, are collected at fixed near-nadir viewing geometries. They do not capture the directional emissivity behavior—ε(λ, θ, φ)—that can be significant for certain materials. Future work could address this by developing angle-resolved spectral libraries, deriving angular change in emissivity from measurements, so the magnitude of angular emissivity variation can be used as a material-discriminating characteristic.

5.3. Strengths and Implications

The innovation of this work extends beyond geometric reconstruction to the creation of explorable 3D hyperspectral models. Through our texture-to-image mapping pipeline, users can interactively query any point on the 3D model to retrieve spectral signatures from all cameras that observed that location, along with their viewing angles and distances. Users can then analyze how spectral signatures change with viewing angle in long-wave infrared data, which could be helpful for material identification under sub-optimal conditions where Lambertian assumptions fail.

The material identification tools (Match, Vote, and Grow) demonstrate the practical utility of multi-angle spectral observations. By considering multiple spectral signatures of the same target acquired from different viewing geometries, these tools can potentially increase confidence in material classification by bringing multiple spectra observed at different angles together, classifying them, and selecting the most probable answer. The ability to combine geometric and spectral information in an interactive, explorable format makes hyperspectral remote sensing imagery more accessible and engaging for end users.

5.4. Limitations and Sources of Uncertainty

Our limitation is reliance on using visible-light imagery to transform the hyperspectral line-scanner images’ geometry into the frame-camera geometry. In theory, it should be possible to develop a geometric model that performs this transformation automatically based on the sensor input geometry, such as tilt, yaw, and pitch. This direct transformation can possibly avoid error accumulation and propagation from multiple passes of automatic (hyperspectral mosaic to visible-light imagery) and manual (mosaicking images from side 1 and side 2 sensors) image registration. Further research is needed to quantify these errors in this study or to develop an appropriate geometric model for image transformation.

Another limitation of the method described in this paper relates to the geometric distortion of pixel footprints in multi-angle acquisitions. When multiple hyperspectral scans are acquired from different looking angles and azimuthal orientations, corresponding pixels across scans do not observe identical ground areas. Even when pixel centers are projected to the same ground location, the rectangular pixel footprints are rotated relative to one another due to the varying azimuthal orientation of each scan. This rotation causes individual pixels to integrate spectral energy from different, partially overlapping ground areas, resulting in spectral mixing that varies with acquisition geometry. For homogeneous materials, this effect may be minimal, but for scenes with spatial heterogeneity at the sub-pixel scale, the observed spectra from different viewing angles include contributions from different surface materials. Our current method addresses only the spatial co-registration of different scans—enabling geometric alignment for 3D model construction—but does not correct the collected spectra for changing pixel geometry. Future work should investigate methods to model and compensate for these pixel footprint variations, potentially through spectral unmixing techniques that account for the angular-dependent pixel geometry.

The varying pixel footprint across viewing angles introduces spectral mixing effects that impact the reliability of the Match and Vote classification tools. At oblique viewing angles, pixels cover larger ground areas and are more likely to include multiple surface materials, resulting in mixed spectra. For the Match tool, which compares an averaged spectrum against the library, mixed pixels introduce spectral features (absorption or emission) not characteristic of the target material, thereby degrading the SAM score and potentially leading to misidentification. The Vote tool exhibits better robustness to this effect due to its majority-voting mechanism: classification remains reliable provided that fewer than half of the extracted spectra are significantly corrupted by mixing. However, when the majority of contributing pixels are mixed—as may occur for small targets or material boundaries observed predominantly at oblique angles—both tools will yield unreliable results. Consequently, the proposed classification approach is best suited for relatively homogeneous surface regions where pixel mixing is minimal. Future work could address this limitation by incorporating spectral unmixing techniques to estimate sub-pixel material fractions, or by implementing angle-dependent quality weighting that down-weights spectra from oblique views where spatial resolution is degraded.

Another limitation is the manual registration step required for mosaicking side 1 and side 2 hyperspectral images. Due to the small overlap between sensor sides (15 pixels horizontally with variable vertical alignment), automatic co-registration was unreliable for some image pairs. We resorted to manual selection of vertical offsets for all 106 image pairs. Based on our observations, the typical alignment uncertainty in this manual step is approximately 1–2 pixels, corresponding to a positional error of 0.4–0.8 m, given the ground sample distance of approximately 0.4 m. This error contribution represents approximately 8–15% of the total HS-COREG model error (MAE: 5.22 m), indicating that manual mosaicking is not the dominant error source, but it still contributes to the overall uncertainty. From a reproducibility standpoint, the manual registration task is relatively simple. It requires a selection of a single integer offset value per image pair and the expected inter-operator variability to remain within the observed 1–2-pixel range. Future work should explore methods for automated sub-pixel co-registration of the two sensor sides.

6. Conclusions

In this study, we have demonstrated a practical data processing pipeline for creating explorable 3D hyperspectral models from gimballed pushbroom sensor imagery. By using co-registered RGB-frame camera imagery to transform multi-angle hyperspectral line-scan data into a geometry compatible with commercial photogrammetric software, we enable the construction of 3D models that preserve the link between the model geometry and multi-angle spectral measurements.

Quantitative evaluation against reference data reveals that while direct reconstruction from hyperspectral line-scan imagery yields poor geometric accuracy (MAE: 18.22 m; RMSE: 23.90 m), the proposed co-registration approach improves the reconstruction accuracy (MAE: 5.22 m; RMSE: 7.00 m for HS-COREG). The VIS-COREG model achieves even better accuracy (MAE: 3.69 m; RMSE: 4.95 m), approaching the performance of the AIM model (MAE: 3.31 m). While the HS-COREG results remain inferior to the RGB model (MAE: 7.33 m) in RMSE, they represent a significant improvement over the model directly reconstructed from hyperspectral images and work well for the spectral identification tasks.