Exploring the Application of UAV-Multispectral Sensors for Proximal Imaging of Agricultural Crops ^†

Abstract

UAV-mounted multispectral sensors are widely used to study crop health. Utilising the same cameras to capture close-up images of crops can significantly improve crop health evaluations through multispectral technology. Unlike RGB cameras that only detect visible light, these sensors can identify additional spectral bands in the red-edge and near-infrared (NIR) ranges. This enables early detection of diseases, pests, and deficiencies through the calculation of various spectral indices. In this work, the ability to use UAV-multispectral sensors for close-proximity imaging of crops was studied. Images of plants were taken with a Micasense Rededge-MX from top and side views at a distance of 1 m. The camera has five sensors that independently capture blue, green, red, red-edge, and NIR light. The slight misalignment of these sensors results in a shift in the swath. This shift needs to be corrected to create a proper layer stack that could allow for further processing. This research utilised the Oriented FAST and Rotated BRIEF (ORB) method to detect features in each image. Random sample consensus (RANSAC) was used for feature matching to find similar features in the slave images compared to the master image (indicated by the green band). Utilising homography to warp the slave images ensures their perfect alignment with the master image. After alignment, the images were stacked, and the alignment accuracy was visually checked using true colour composites. The side-view images of the plants were perfectly aligned, while the top-view images showed errors, particularly in the pixels far from the centre. This study demonstrates that UAV-mounted multispectral sensors can capture images of plants effectively, provided the plant is centred in the frame and occupies a smaller area within the image.

1. Introduction

UAV-remote sensing has revolutionised crop health monitoring by providing remote sensing images on demand for calculating crop health metrics [1,2,3,4,5,6]. The remote sensing images collected from a variety of sensors have made UAV-based remote sensing a mainstay in precision agriculture [7]. Images from RGB sensors provide very-high-resolution images [8]. Data from hyperspectral sensors is highly valuable as it provides detailed spectral information about the crops [9]. However, RGB sensors offer limited spectral information, whereas hyperspectral sensors, while more comprehensive, are often prohibitively expensive [10]. Multispectral sensors offer a good balance between RGB and hyperspectral sensors, providing essential spectral depth at a lower cost [11]. UAVs equipped with multispectral imaging sensors are commonly used to monitor crop growth in fields because they provide advantages over other sensors [12]. The use of these sensors has yielded outstanding results in the analysis of field crops from a UAV [13], but the design limitations have restricted its use for capturing close-up images from low-flying UAVs or for crops growing in pots [14].

Proximal imaging with multispectral sensors can provide valuable information about pests, diseases, and other stresses affecting crops by utilising visual and spectral data [15], while water and nutrient stress in crops can be detected using vegetation indices and other methods that analyse spectral signatures [16]. Crops’ pests and diseases can only be identified through close-range remote sensing images that capture leaf symptoms [17]. These limitations have led the agriculture community to invest in various sensors for different needs, increasing the cost of crop health monitoring and precision agriculture [18]. Removing the limitations of current multispectral sensors could enable their use with UAVs and for close-range imaging, benefiting the entire community [19].

Currently, multispectral sensors feature a rig with multiple cameras, each sensitive to specific parts of the electromagnetic spectrum (EMR). Each camera is placed separately to avoid obstructing the field of view of others. This placement creates a shift in FOV and causes variations in the images captured by the cameras. For example, for the Micasense RedEdge-MX (MX) five-band sensor, five distinct images will be captured by each camera for every individual instance. All the images will have a parallax shift due to the position of the cameras [20]. This change in images complicates the process of stacking individual layers, making it exceedingly challenging [21]. OEMs recommend flying UAVs at altitudes above 15m to ensure that most of the area of interest is at the centre of the images, where shifting errors are typically lower. At close distances, the parallax shift greatly impacts images because the object of interest takes up most of the frame, making the alignment challenging [14]. Addressing the challenges arising due to the sensor’s distance from objects can enhance its usability in agriculture [22].

Computer vision methods are often favoured for aligning images with parallax errors or disorientations compared to complex deep learning models and simple GPS-based alignment procedures [23]. Computer vision-based feature matching approaches are data-agnostic, lightweight, and offer strong generalisation across diverse scenes and spectral conditions [24]. The ORB (Oriented FAST and Rotated BRIEF) algorithm is a computer vision method known for its efficiency, accuracy, and robustness. ORB combines the FAST corner detection algorithm with orientation estimation and BRIEF descriptor extraction, making it invariant to scale and rotation. Its use of binary descriptors and Hamming distance matching enables quick feature comparison while retaining strong discriminative power, even in distorted views [25]. Further, ORB is fully open-source and free of patents, which makes it perfect for scalable research and real-time applications in agriculture and remote sensing [25]. Thus, ORB is a preferred option in the computer vision field for multimodal image registration tasks due to its robustness across different image resolutions, lightweight computational requirements, and strong performance in low-texture scenes, such as vegetation [25]. The Random Sample Consensus (RANSAC) algorithm, often used with ORB, helps remove outliers from feature matches in computer vision [26]. It works by iteratively selecting random subsets and estimating the best-fitting transformation model. This work uses the RANSAC method to improve the matching of features detected in different spectral bands by eliminating mismatches from parallax errors [26]. After finding reliable matches, a homography matrix is created to align each slave image with the reference band geometrically [27]. This transformation preserves spatial consistency across bands, enabling precise layer stacking for further multispectral analysis [27]. The study aims to enhance the use of current UAV-mounted sensors for close-range applications, lessen reliance on specialised equipment, and provide more accessible and affordable crop monitoring solutions in various agricultural situations. The main goal is to create and test a reliable image alignment pipeline using the ORB feature detection and matching technique, along with RANSAC for homography estimation. This approach aims to align individual spectral bands captured at low altitudes to create high-quality stacked multispectral composites for plant health analysis. Using the pipeline, this study evaluates the use of the UAV-mounted MX sensor for close-range crop imaging to address parallax-induced misalignments in multispectral imagery.

2. Study Area

ICAR-Indian Agricultural Research Institute (IARI) is a leading agricultural research centre located in New Delhi, India. It has several essential facilities for research on various agricultural topics. One such facility is the Big Data Analytics Lab (BDL Lab) (28.639541, 77.162105). The BDL lab uses remote sensing data from ground sensors, UAVs, and satellites to evaluate crop health in different ways. The study was conducted at the Big Data Analytics lab, where the MX sensor was used to assess the health of potted plants. Crops in the pots were subjected to multiple treatments to see their reaction to nutrient and water stress treatments. The MX sensor was applied to assess the impact of these treatments as a remote sensing sensor. The problem of parallax error was first observed during the initial image alignment and layer stacking process. The methodology applied here will address the image alignment issues so that the health of these potted plants can be assessed from the MX images. Figure 1 shows the location of the BDL lab inside IARI.

Figure 1. Shows the location of the Big Data Analytics Lab (Credits: Google Maps).

3. Methodology

3.1. Experiment Setup

The MX sensor was used to capture an image of the subject (potted plants) from either the top or side view. For top-view photos, the sensor was placed directly above the plant in the nadir position at a distance of 1.5 m from the plant. For the side view, the camera was placed on a tripod at 1.5 m from the plant. It was ensured that the object was in the centre of the frame. The MX camera was connected to a mobile device using the Wi-Fi generated from the sensors. After the sensor is connected, we can access the control panel for the sensor in any web browser by entering the IP address 192.168.10.254 into the search bar of the browser. From the control panel, all the functions of Micasense, like camera triggering and live image feed visualisation, can be controlled. Once the sensor is ready with all the required settings, images of the crops were captured, and the raw images were used for further analysis. For the top view, the pot is placed on a black uniform background with markers indicating the edge of the crop. Further, the image also shows a calibration panel placed on the flat ground, along with a label panel with the name of the specimen. This image was taken outdoors under natural sunlight. The side-view image was captured indoors under artificial lights. A potted indoor plant was used as a subject, and we did not use any background to see how the alignment works when images have diverse backgrounds. Since computer vision-based ORB methods are being used for image alignment, these diverse backgrounds and lighting conditions, along with plants, will reveal the robustness of the techniques.

3.2. Image Layer Staking

Figure 2 shows the process followed while layer stacking the proximal MX images. Figure 2 shows two sets of images; the first image is taken from the nadir position when the camera is positioned directly above the crop. A potted wheat crop in its early growth stage was used as a subject for this image. There are five images, one for each band. The brightest of the images is the NIR image, followed by rededge and green, due to the crop’s high reflectance in these bands. The red and blue images are dark, suggesting high absorbance in these bands. Similarly, the second set of images shows a potted indoor plant captured from the MX sensor from a side view. These images are promptly labelled and can be visualised in Figure 2.

Figure 2. Shows the steps involved in layer stacking the individual images from the Micasense Rededge-MX sensor using Oriented FAST and Rotated BRIEF (ORB) computer vision techniques.

The images were processed in a Python 3.12 environment. The ORB technique was accessed from the OpenCV library. OpenCV was used for image processing, feature detection, matching, warping and drawing of the images. In addition, the numpy, rasterio, and matplotlib.pyplot libraries were used for numerical operations, generating a multispectral layer stack, and displaying images and matches, respectively. The MX provide individual images in TIFF format; these images are read using the rasterio library. The OpenCV library was used to setup ORB flow for the images. In the first step, the ORB uses the Features from the Accelerated Segment Test (FAST) method, a corner detection algorithm, to quickly detect keypoints in the images. The FAST approach is particularly known for its speed and is generally used for real-time applications like object tracking or image alignment. The FAST approach looks at each pixel and 16 pixels surrounding it (a Bresenham circle). It checks if a certain number of contiguous pixels in that circle are all significantly brighter or darker than the centre pixel. If there is a significant change, the centre pixel is marked as an edge pixel. Figure 2 shows the keypoints detected in each image from the corresponding views. Each edge pixel is indicated by a small circle drawn around it by the FAST detection algorithm.

FAST detects key points but does not provide orientation to the key points. Orientation is a really important feature that tells the dominant direction and rotation angle of a key point. While this is happening, the ORB also uses the Binary Robust Independent Elementary (BRIEF) descriptor to provide a unique ID to individual key points by comparing the intensity differences between pairs of pixels in a patch around the keypoint. These descriptors are used to detect and match similar key points across the images. Each descriptor takes the rotation angle of a key point into consideration while identifying matching key points. This process makes the ORB alignment process rotation invariant. Figure 2 shows the rotation angle of the images in the orientation assignment section. Here, the ORB reads the images at multiple pyramid layers and then identifies the key points, calculates the rotation angle and assigns them to the key points. In this part of the image, the key points are detected and shown by concentric circles (a circle for each pyramid layer). The direction is indicated by the line connecting these concentric circles.

Once the ORB detects orientation, the BRIEF descriptors are generated for each key point detected by the FAST algorithm. The Random Sample Consensus (RANSAC) removes the outlier keypoint that does not have proper matches across all the images. The RANSAC randomly picks some of the inliers to formulate a transformation that can best align the images together. The algorithm then checks how many of the unselected points agree with this transformation. This process is repeated several times to find the best consensus between the transformation and the maximum inliers. Using this transformation matrix, the homography method warps the slave images to match the master image (rededge) so that all the images are aligned perfectly for layer stacking. Finally, the aligned images are layer-stacked to provide a 5-band TIFF image for further assessment.

4. Results and Discussion

The close-up images of the potted plants taken from the MX sensor were processed using the methodology mentioned above. A layer stack of these bands was generated using QGIS 3.40 LTS version; the outcome of the process is shown in Figure 3. Figure 3 shows the layer-stacked true colour composite (TCC) image of the front and side view images before and after alignment. Before the alignment, the shift between the bands is visible in both the front and side view images. After the alignment, too, the top-view image shows a lot of shit between the bands. The image taken from the nadir position has a uniform background with markers to identify the edge, which the feature detection algorithms can use to improve alignment. In addition, the image also has high-contrast edges that can make the alignment process more accurate. But in this case, the ORB failed to provide a proper alignment [28], and from Figure 3, the aligned top-view image looks very similar to the original layer-stacked image, except for some zones.

Figure 3. Shows the comparison between the layer stack generated using the original bands without alignment and the results obtained after the bands are aligned using the ORB approach. Top-view layer stack without alignment (top-left), top-view layer stack after alignment (bottom-left), side-view layer stack without alignment (top-right), and side-view layer stack after alignment (bottom-right).

Though most of the aligned images from the nadir view have lapses, some features, like the reflectance panel, show near-perfect alignment. This indicates that the image went through the alignment process, features were detected, but the alignment failed to produce a perfectly aligned layer stack. One strong reason could be that the key points are mainly concentrated in and around the reflectance panel area of the top-view image. From Figure 2, the key points in the blue, green, red, and red-edge bands clearly show that most of the key points are situated near the panel area due to its high contrast edges. Since FAST detects only the edges and ORB only uses these key points for further processing [29], all the key features are concentrated only in a small zone. Further, RANSAC utilises these features to formulate the transformation matrix for aligning the images (see Figure 2, RANSAC key point matching). Since the algorithm failed to detect a large no of key points on the crop, the alignment process warped the images along the available key points, resulting in a near-perfect reflectance panel layer stack, disregarding the rest of the image. Thus, the lack of contrast between the crop and the background, along with having high contrast edges concentrated in a single zone, could have caused the ORB algorithm to fail in image alignment [30].

To improve the alignment further, we have reduced the key-point threshold to allow RANSAC to select more inliers for better alignment [31]. Figure 4 shows the aligned images after improving keypoint selection and distribution. From Figure 4, the top-left image was the result of the ORB alignment after the key-point selection threshold was reduced. The reduction in the threshold value improved the number of keypoints available for the alignment process. Additionally, the key features near the label sheet (right side of the pot), on the pot edge markers, and the pot itself were also included for alignment. The inclusion of more key points, especially from diverse positions, has drastically enhanced the alignment of the images. The aligned image clearly shows that the panel, label sheet and most of the crop, along with the pot, have aligned, leaving some portions still distorted. Mainly, the leaf edges that are not in the centre and spread across the image with several twists show significant variations in the alignment. The improvement in the alignment is clearly due to the increasing key points for feature matching and their distribution. To improve the alignment further, the RANSAC method was used to select only those key points based on their spatial distribution. The bottom-left image from Figure 4 shows the results after improving the key point selection by checking for distribution. From the figure, the change in the number of key points selected based on the positions was not significant, which affected the alignment process. The bottom-left aligned image in Figure 4 does not show any noteworthy change in the crop alignment from the last output. Thus, clearly showing that the most significant part of the feature-based image alignment is the detection of key points that are numerous and distributed adequately across the image [32]. To improve key point detection, the background and the subject must be in stark contrast. Further, the figures also show that the object must be at the centre of the frame with less spreading towards the corners of the image. The spread of the object was more across the image as the image was captured from the top. If the image was captured from the side view, then the plant may not spread across the image that well, and the image can be appropriately aligned.

Figure 4. Shows the top-view image after alignment using more no of key-points (top-left) and after improving the distribution of key-points (bottom-left). The corresponding key-point matching between the green bands and the red-edge (reference) band can be observed in (top-right) and (bottom-right) positions, respectively.

With these learning, the MX sensor was used to capture the side-view image of an indoor potted plant with diverse backgrounds and bright indoor lighting conditions. Figure 2 shows how the pot looks in different bands. The figure also shows the key points detected in each band. From the figure, the key points in the side view image are spread across the image, with several of them being on the plant and the pot itself. The edges and high contrast features are spread across the side-view images, thus providing well-distributed features at prominent positions. The RANSAC matching also shows highly diverse matching lines connecting the well-distributed features, indicating how the high-quality key points can influence the alignment process. The results of the alignment can be seen from the images displayed in Figure 3 (bottom-left). The figure shows the TCC of the side-view layer stacked image without alignment (top-right) and the layer stacked image after alignment (bottom-right). From the figure, the TCC of the unaligned image shows a significant shift between the bands. In contrast, the TCC of the aligned image show a perfect layer stack where the alignment process was able to align the bands to near perfection. This layer stack image can be used to perform an analysis of choice to extract crop health information. Figure 5 shows the normalised difference vegetation index (NDVI) image generated from the stacked image layer. The image is colour-coded from red to green, where red indicates very low NDVI and green indicate higher NDVI. The minimum and maximum values of NDVI for the plant are around 0.1 and 0.7 [33]. The leaf of the indoor plant has several pale green spots where the chlorophyll is very low, and some areas are still bright green where the chlorophyll content is high. The NDVI image clearly shows a difference between the zones with lower chlorophyll content and the areas with high chlorophyll content in the leaves [34]. Wherever the chlorophyll content is low, the NDVI values are low and vice versa [35]. Since the NDVI is showing all these variations accurately per the plant chlorophyll content, it clearly shows that the alignment and layer stacking of the image worked perfectly. Thus, the MX sensor can be used effectively for assessing crop health from a very close distance, provided some precautions are taken before capturing the images.

Figure 5. Shows the NDVI image generated from the side-view aligned layer stack image.

Though this research was conducted with the Micasense RedEdge-MX sensor, care must be taken to explore what might occur if paired with alternative types of sensors for similar close-proximity imaging applications. Other multispectral sensors (such as the Parrot Sequoia and Sentera 6X) have a different set of spectral bands but, likewise, can be utilised for proximal imaging. In these instances, the alignment approach will need to be modified, as bands count, optics, and camera configurations affect parallax and registration performance [12,13]. Hyperspectral sensors, on the other hand, typically employ pushbroom or whiskbroom scanning, which yields a complete spectral datacube instead of several separate images. This obviates band-to-band alignment requirements, but poses other issues, such as motion Artefact, illumination, and the requirement for accurate calibration at acquisition. RGB cameras, while providing extremely high spatial resolution, do not capture key spectral bands (e.g., red-edge and NIR), and thus are not very effective in capturing early plant stresses [8,10]. Multispectral sensors, therefore, find a middle ground between spectral richness, cost, and data volumes that can be handled [7,11,15]. An initial examination of sensor specifications indicates that proximal imaging is possible across platforms, although the processing pipeline would need to be adapted based on sensor design and acquisition geometry. For additional information, refer to Table 1, which compares various sensors and how the current approach can be applied to these sensors.

Table 1. Comparison of imaging sensors, ranging from RGB to Hyperspectral, to evaluate the feasibility of the current approach.

Sensor Type	Bands (Centre Wavelength ± FWHM)	Spatial Resolution	Data Format	Suitability for Proximal Imaging	Advantages	Limitations
RGB DSLR (generic)	3 bands (Blue ~450 nm, Green ~550 nm, Red ~650 nm)	12–24 MP	Single image	Captures fine detail of leaves, but no spectral depth	Very high spatial resolution; low cost; easy to deploy	No NIR/Red-edge; cannot detect early stress [9,11]
Micasense RedEdge-MX	5 bands (Blue 475 nm, Green 560 nm, Red 668 nm, Red-edge 717 nm, NIR 840 nm)	1280 × 960 px	5 separate TIFF images	Proven in UAV and proximal use; requires band alignment	Balanced cost vs. spectral info; strong vegetation indices	Band separation → parallax; ORB+RANSAC alignment needed (this study)
Parrot Sequoia+	4 bands (Green 550 nm, Red 660 nm, Red-edge 735 nm, NIR 790 nm) + RGB	1280 × 960 px (multi) + RGB camera	4 TIFFs + RGB JPEG	Lightweight; widely used	Simpler (fewer bands); RGB + NIR indices possible	Alignment/parallax issues remain; limited bands [13,14]
Sentera 6X Multispectral	6 bands (Blue 450 nm, Green 530 nm, Red 670 nm, Red-edge 710 nm, NIR 840 nm, Wide NIR 940 nm)	20 MP	6 independent images	High spatial detail + spectral range	High resolution; broad crop stress sensitivity	More demanding alignment across bands
DJI P4 Multispectral	Blue 450 ± 16 nm; Green 560 ± 16 nm; Red 650 ± 16 nm; Red-edge 730 ± 16 nm; NIR 840 ± 26 nm	~2.08 MP per band; plus RGB sensor	TIFFs (MS) + RGB JPEG	Effective for UAV and proximal use; limited detail for very close imagery	Compact; well-calibrated; NDVI/NDRE ready	Lower resolution per band; parallax between sensors; alignment needed
DJI Mavic 3M Multispectral	Green 560 ± 16 nm; Red 650 ± 16 nm; Red-edge 730 ± 16 nm; NIR 860 ± 26 nm	4 × 5 MP MS + 20 MP RGB	Separate MS images + RGB	Good candidate for the current pipeline; better RGB helps texture/feature detection	Built-in light sensor; newer high-res RGB; portable	Alignment still needed; per-band resolution lower than RGB; possible close-range misalignment
Headwall Nano-Hyperspec	~270 bands (400–1000 nm; 2–3 nm bandwidth)	Pushbroom, ~640 spatial pixels	Hyperspectral datacube	Suitable; requires motion control	Very high spectral detail; no parallax (single slit)	High data volume; calibration and motion correction needed; costly [10,19,20,21,28]
Headwall Co-Aligned HP™ (VNIR–SWIR)	VNIR: 400–1000 nm (340 bands, 6 nm FWHM); SWIR: 900–2500 nm (267 bands, 6 nm FWHM)	VNIR: 1020 px; SWIR: 640 px	Co-aligned hyperspectral datacube	Designed for UAV/remote sensing; less suited to handheld close-ups	Co-aligned optics minimise parallax; full VNIR–SWIR coverage	Heavy (~4 kg); high data volume; motion correction & calibration required; costly [10,19,20,21,28]

5. Conclusions

The study shows that UAV-mounted multispectral sensors can be effectively used for closer-range crop imaging and health assessment even when they have multiple camera systems in the rig. The work demonstrated how computer vision techniques can be used to process close-up MX images of potted plants from top- and side-view angles taken from a 1.5-m distance while discussing the challenges encountered during the process. Robust computer vision techniques like ORB feature detection and RANSAC-based alignment were used to answer these challenges to generate a perfect layer-stacked image that can be used for further image analysis. While utilising these alignment techniques, it is essential to make sure that the crop has a fine contrast in all the bands against the background, which can be either homogeneous or non-homogeneous. Further, the image should have some markers with sharp edges and proper contrast. These steps will ensure that the key points are numerous and well distributed across the image, including on the crop itself. Further, the crop must always be in the centre of the FOV, which will allow for better alignment. By ensuring that these steps are followed, the MX sensors can be effectively used to study crops using close-up MX images.