Next Article in Journal
Single Target SAR 3D Reconstruction Based on Deep Learning
Next Article in Special Issue
Underwater Holographic Sensor for Plankton Studies In Situ including Accompanying Measurements
Previous Article in Journal
Comparison of Heating Strategies on Soil Water Measurement Using Actively Heated Fiber Optics on Contrasting Textured Soils
Previous Article in Special Issue
Vicarious Radiometric Calibration of Ocean Color Bands for FY-3D/MERSI-II at Lake Qinghai, China
Communication

The Potential of Satellite Imagery for Surveying Whales

1
BioConsult SH GmbH & Co.KG, Schobüller Str. 36, 25813 Husum, Germany
2
British Antarctic Survey, High Cross, Madingley Road, Cambridge CB3 0ET, UK
3
Scott Polar Research Institute, Department of Geography, University of Cambridge, Cambridge CB2 1TN, UK
4
UK Antarctic Heritage Trust, High Cross, Madingley Road, Cambridge CB3 0ET, UK
5
HiDef, Aerial Surveying Limited, 17 Silvermills Court, Edinburgh EH3 5DG, UK
6
Department of Ecology & Evolution, Stony Brook University, New York, NY 11794, USA
*
Author to whom correspondence should be addressed.
Those authors contributed equally to this paper.
Academic Editors: Luís Miguel Valente Goncalves, Marcos Silva Martins, Rui A. Lima and Graça Minas
Sensors 2021, 21(3), 963; https://doi.org/10.3390/s21030963
Received: 31 December 2020 / Revised: 26 January 2021 / Accepted: 28 January 2021 / Published: 1 February 2021
(This article belongs to the Special Issue Marine Sensors: Recent Advances and Challenges)

Abstract

The emergence of very high-resolution (VHR) satellite imagery (less than 1 m spatial resolution) is creating new opportunities within the fields of ecology and conservation biology. The advancement of sub-meter resolution imagery has provided greater confidence in the detection and identification of features on the ground, broadening the realm of possible research questions. To date, VHR imagery studies have largely focused on terrestrial environments; however, there has been incremental progress in the last two decades for using this technology to detect cetaceans. With advances in computational power and sensor resolution, the feasibility of broad-scale VHR ocean surveys using VHR satellite imagery with automated detection and classification processes has increased. Initial attempts at automated surveys are showing promising results, but further development is necessary to ensure reliability. Here we discuss the future directions in which VHR satellite imagery might be used to address urgent questions in whale conservation. We highlight the current challenges to automated detection and to extending the use of this technology to all oceans and various whale species. To achieve basin-scale marine surveys, currently not feasible with any traditional surveying methods (including boat-based and aerial surveys), future research requires a collaborative effort between biology, computation science, and engineering to overcome the present challenges to this platform’s use.
Keywords: very high-resolution (VHR) satellite imagery; remote sensing; great whale species very high-resolution (VHR) satellite imagery; remote sensing; great whale species

1. Introduction

The study of wildlife using satellite imagery commenced with the Landsat-1 satellite (80 m spatial resolution), initially launched in 1972 to monitor land cover, and later extended for habitat surveys and to map the distribution range of some species [1,2,3]. Since then, there have been several technical developments, including improvements of optical sensors for an increased spatial resolution, with the development of very high-resolution (VHR) satellite. VHR satellites offer a sub-meter spatial resolution, allowing the detection of individual animals such as cattle [4], large African animals [5,6], polar bears (Ursus maritimus) [7], grey seals (Halichoerus grypus) [8], and albatrosses (Family: Diomedeidae) [9].
Monitoring whales using VHR satellite imagery was first attempted using the IKONOS-2 satellite in 2002; however, the 0.82 m spatial resolution provided by this satellite was not sufficient to make confident identifications [10]. A change in US legislation in August 2014 increased the maximum spatial resolution for commercial satellite operators to 0.25 m. Rather than coarse imagery with rudimentary outlines of large objects, the greater spatial resolution of 0.25 m imagery allows the capture of more detail where smaller features can be identified with greater confidence. A subsequent survey by Fretwell and colleagues [11] identified and counted southern right whales (Eubalaena australis) in an image captured by the WorldView-2 satellite (0.46 m spatial resolution). The available spatial resolution at the time, did not allow for the identification of whale-defining features—such as the fluke. The launch of the WorldView-3 satellite in 2016, offering the highest commercially available spatial resolution of 0.31 m, is capable of capturing whale-defining features [12], increasing the confidence in detections and making this satellite the most adapted for whale surveys.
Traditional cetacean surveys are challenging and expensive, often requiring dedicated research vessels or aircraft and are thus limited in their temporal and spatial scope e.g., [13,14]. Much of the ocean’s surface has not been surveyed for whales [15,16], leaving large knowledge gaps [17,18] that hinder efforts to make sound conservation decisions that could more firmly protect threatened or endangered cetacean populations [19]. While cetacean surveys will be challenging with almost any survey platforms, VHR satellite imagery has the potential to fill knowledge gaps in whale research, allowing access to remote or rarely or infrequently monitored regions, as well as surveys of large spatial scope.
Building on recent work using VHR satellite imagery, and with the development of accessible and inexpensive artificial intelligence (AI) tools, there is a potential to (1) fill data gaps on the distribution, abundance, density, and population trends of whales; (2) supplement field-collected data by helping better plan aerial and boat-based surveys; and (3) allow for continued monitoring of whales when person-dependent surveys are restricted—as we have experienced during a pandemic—building a mechanism for important conservation research to continue globally. However, this method is in its infancy, with further pilot studies and experimentation needed. Here we aim to provide an overview of the development of these methods so far, in order to provide advice on future directions to follow and the main challenges that remain.
Because of the growing interest in using VHR satellite imagery as a new platform to study marine mammals, it is important to facilitate the discussion between stakeholders regarding the best approaches to data collection and analysis. Therefore, in December 2019, we led an international workshop entitled: “Marine mammal surveys from satellite imagery: Applications, automation, and challenges” during the World Marine Mammal Conference in Barcelona. A group of 33 participants from academia, consultancy, government, and industry came together to discuss the most recent advancements in satellite technologies and image recognition for marine surveys and the future of this field of study.

2. Considerations and Challenges Inherent to Satellite Images

VHR satellite imagery can either be accessed through satellite tasking (i.e., image capture is requested at a specific time and place), or archives of previously captured images (Figure 1). Archival image libraries are limited in areas of open ocean, as the majority of tasking occurs when and where there is demand and is largely concentrated in terrestrial areas, e.g., [20,21]. When requesting an image to be captured, satellite providers require a time window of several weeks, due to prioritisation of governmental and military acquisitions. This is mostly true for coastal areas.
Currently all VHR satellites are commercially owned (e.g., Maxar Technologies, Airbus, and Earth-i) with the exception of one governmentally-owned satellite (Cartosat-3 for India). As a result, acquiring VHR satellite imagery comes at a cost. When considering the cost of an aerial or ship-based survey in a remote region, the cost of imagery may be competitive [22], but costs certainly can be a barrier to surveying large areas of several thousands of km2. Ultimately image costs limit the potential to develop the application of VHR satellite imagery to study whales. Funding is limited in the field of marine conservation, and the current costs for acquiring imagery restrict the potential for investment by NGOs and research institutes, and for inclusivity and the opportunity of developing nations to utilise this platform.
When images are captured, environmental conditions can affect the quality of the image and the likelihood of detecting whales. Only cloudless images with few or no white caps will be useful for whale research, as clouds conceal the surface of the ocean, including whales, and strong winds create white caps, a confounding feature, that can create confusion and limit the confident detection of whales [12]. It may be possible to overcome the challenges of cloud presence through the use of radar sensors, capable of capturing weather independent imagery. However, the use of radar is dependent upon future technological advances in spatial resolution. At present the spatial resolution of 1 m for radar sensors (e.g., the TerraSAR-X) is too coarse for the detection of whales. Imagery with a low swell should also be preferred, as swell is known to influence the capacity to confidently detect whales [23]. It is worth considering that sea conditions are likewise a confounding factor in all visual cetacean surveys, including boat-based and aerial surveys. The WorldView-3 satellite, similar to other VHR satellites, captures both high spatial-resolution panchromatic images (i.e., 0.31 m greyscale band) and high spectral-resolution multispectral images, which provides a range of eight colour bands for analysis at 1.24 m resolution. Combining the two images during the image pre-processing stage, known as pansharpening (Figure 1), results in a very high-resolution coloured image (i.e., eight colour bands at 0.31 m spatial resolution) that tends to be preferred when detecting whales in satellite imagery. The coastal blue band, one of the eight bands available with Worldview-3, penetrates deeper into the water column, offering new perspectives of identifying features below the surface. Fretwell and colleagues [11] utilised the coastal blue band, identifying more whale-like features, interpreted as sub-surface, than were identified using other bands, such as the panchromatic, and proved to be the most successful band when automatically detecting whale-like features using thresholding techniques. While the coastal blue band can be used to detect whales below the surface, it will likely only allow the observation of whales near the surface, as all light will be absorbed by seawater at a certain depth [24]. Despite its potential, there will be whales beyond the detectable range of coastal blue. This may not be a concern for estimating whale abundance using satellite imagery as detecting whales below the surface may not be necessary for the future use of satellite imagery to estimate abundance. The reason being, correction factors accounting for whales below the surface, similar to those used in abundance estimates derived from aerial and boat surveys [25,26], could be adapted to satellites [27].
Researchers have explored both manual [10,11,12,27] and automated [28,29] methods of analysis (Figure 1). Manually counting whales in satellite images is currently the most accurate method, although the most time-consuming, as it can require approximately 3 h and 20 min to scan 100 km2 [12], and be erroneous due to observer bias [30]. Developing automated approaches using image-recognition techniques, driven by deep learning algorithms appears to be the way forward, particularly for surveying large marine regions. The revolution in the number and accessibility of satellites and the advances of computational power have led to increased use and recognition of deep-learning methods for whale monitoring.
Automated approaches are inherently more efficient than manual scanning, simply because humans are not required during the detection stages freeing up researchers to focus on other tasks. However, effective automation requires careful construction of a dataset of training images, from which the model can learn the characteristics of a whale in various ocean images. The training dataset must include a wide array of example images of different species of whales, in different environmental conditions, and displaying different postures/behaviors, as well as features that could be mistaken for a whale (Figure 1). Given the current limits on the available data of labelled satellite images of whales and similarly labelled images of confounding features within satellite imagery (e.g., rocks, boats, and white caps), required to create such a training dataset, aerial imagery can be used as a substitute for satellite imagery, to get the process underway. These aerial images should be down-sampled to the spatial resolution of VHR satellite imagery [28]. Alternatively, freely available photo archives of aerial and satellite imagery, can be accessed from sources such as Google Earth [29]. However, the satellite images from such sources are formatted in a way that reduces the spectral resolution (i.e., only three colour bands (red, green, and blue), compared to eight colour bands for the WorldView-3), and geospatial information (i.e., a jpeg format instead of a tiff). Further progress will be made by repetitively re-training the model, by incorporating new images as the algorithm detects them.
With automated approaches it is expected that some whales will likely be missed (false negatives) or that non-whale features will be counted as whales (false positives). Ideally with the appropriate training dataset, an automated method would yield no false negatives and only a few false positives. In this scenario, all positive detections can be manually checked and the process can improve over time by incorporating new observations into the existing training data (Figure 1). Another option is to quantify the false negatives and positives and account for them in biological analysis. For the time being, automated processes will likely necessitate a semi-automated approach, assisted by manual review. However, the overall technique has the potential to dramatically improve how quickly we can assess the distribution and population of some large cetaceans, as long as we can differentiate species, which is currently not feasible as VHR satellites has only been used in areas of known homogenous species presence.

3. Future Directions

With the continued development of high-performance computing infrastructure and the planned launch of 10 new 0.30 m resolution satellites by Maxar Technologies (https://www.maxar.com/splash/it-takes-a-legion) and Airbus, by 2021 (https://www.airbus.com/space/earth-observation/portfolio.html), VHR satellite imagery is on its way to becoming a valuable tool to monitoring large whales, particularly in remote, understudied, and underfunded regions. Currently, we can reliably conduct presence/absence surveys through manual detection; however, continuing to develop automated detection systems will improve the efficiency. Throughout the development of the use of satellite imagery to monitor whales, we must ensure collaboration between various stakeholders (i.e., from industry, government, and academia), the creation of an open-source database of labelled whale images from satellites, and an appropriate analysis (coding) framework, as shown in Figure 1.This will provide consistent and powerful solutions for conservation. Future research should also focus on (1) building large and complex datasets to train automated systems; (2) pre-processing workflows to identify a suitable standard method; (3) species identification in order to conduct density, abundance, and population trend assessments; (4) testing the ability of models to detect whales across different sea conditions to allow transferability; and (5) understanding and accounting for factors limiting detectability such as weather conditions, including clouds and strong winds generating white caps [11,12].
Further experimentation and pilot studies will improve the reliability of this platform and help us understand how best to combine imagery methods with traditional field approaches to improve whale conservation. Future accessibility to VHR satellite imagery will dictate what can be achieved. To protect our world’s oceans, we need a global effort and we need to create opportunities for that to happen. Accessibility to VHR satellite imagery is critical to the development of expertise and to enhance conservation efforts.

Author Contributions

Conceptualisation, C.H., H.C.C., P.J.C., and A.B.; writing—original draft preparation, C.H., H.C.C., P.J.C., and A.B.; writing—review and editing, C.H., H.C.C., P.J.C., G.H., and A.B; visualisation A.B., H.C.C. and C.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nelson, H.K.; Work, E., Jr. Appraising Changes in Continental Migratory Bird Habitat; Progress Report; Bureau of Sport Fisheries and Wildlife Northern Prairie Wildlife Research Center, Jamestown, North Dakota and Goddard Space Flight Center: Greenbelt, MD, USA, 1973. [Google Scholar]
  2. Reeves, H.M.; Cooch, F.G.; Munro, R.E. Monitoring arctic habitat and goose production by satellite imagery. J. Wildl. Manag. 1976, 40, 532–541. [Google Scholar] [CrossRef]
  3. Löffler, E.; Margules, C. Wombats detected from space. Remote Sens. Environ. 1980, 9, 47–56. [Google Scholar] [CrossRef]
  4. Laliberte, A.S.; Ripple, W.J. Automated wildlife counts from remotely sensed imagery. Wildl. Soc. Bull. 2003, 31, 362–371. [Google Scholar]
  5. Yang, Z.; Wang, T.; Skidmore, A.K.; de Leeuw, J.; Said, M.Y.; Freer, J. Spotting East African mammals in open savannah from space. PLoS ONE 2014, 9, e115989. [Google Scholar] [CrossRef] [PubMed]
  6. Duporge, I.; Isupova, O.; Reece, S.; Macdonald, D.W.; Wang, T. Using very-high-resolution satellite imagery and deep learning to detect and count African elephants in heterogeneous landscapes. Remote Sens. Ecol. Conserv. 2020. [Google Scholar] [CrossRef]
  7. Stapleton, S.; LaRue, M.; Lecomte, N.; Atkinson, S.; Garshelis, D.; Porter, C.; Atwood, T. Polar bears from space: Assessing satellite imagery as a tool to track arctic wildlife. PLoS ONE 2014, 9, e101513. [Google Scholar] [CrossRef]
  8. Moxley, J.H.; Bogomolni, A.; Hammill, M.O.; Moore, K.M.T.; Polito, M.J.; Sette, L.; Sharp, W.B.; Waring, G.T.; Gilbert, J.R.; Halpin, P.N.; et al. Google haul out: Earth observation imagery and digital aerial surveys in coastal wildlife management and abundance estimation. BioScience 2017, 67, 760–768. [Google Scholar] [CrossRef]
  9. Fretwell, P.T.; Scofield, P.; Phillips, R.A. Using super-high resolution satellite imagery to census threatened albatrosses. Ibis 2017, 159, 481–490. [Google Scholar] [CrossRef]
  10. Abileah, R. Marine mammal census using space satellite imagery. US Navy J. Underw. Acoust. 2002, 52, 709–724. [Google Scholar]
  11. Fretwell, P.T.; Staniland, I.J.; Forcada, J. Whales from space: Counting southern right whales by satellite. PLoS ONE 2014, 9, e88655. [Google Scholar] [CrossRef]
  12. Cubaynes, H.C.; Fretwell, P.T.; Bamford, C.; Gerrish, L.; Jackson, J.A. Whales from space: Four mysticete species described using new VHR satellite imagery. Mar. Mammal Sci. 2019, 35, 466–491. [Google Scholar] [CrossRef]
  13. Viquerat, S.; Herr, H. Mid-summer abundance estimates of fin whales balaenoptera physalus around the south orkney islands and elephant island. Endanger. Species Res. 2017, 32, 515–524. [Google Scholar] [CrossRef]
  14. Hammond, P.S.; Lacey, C.; Gilles, A.; Viquerat, S.; Börjesson, P.; Herr, H.; MacLeod, K.; Ridoux, V.; Santos, M.B.; Scheidat, M.; et al. Estimates of Cetacean Abundance in European Atlantic Waters in Summer 2016 from the SCANS-III Aerial and Shipboard Surveys; Wageningen Marine Research: Yerseke, The Netherlands, 2017; p. 40. [Google Scholar]
  15. Kaschner, K.; Tittensor, D.P.; Ready, J.; Gerrodette, T.; Worm, B. Current and future patterns of global marine mammal biodiversity. PLoS ONE 2011, 6, e19653. [Google Scholar] [CrossRef]
  16. Kaschner, K.; Quick, N.J.; Jewell, R.; Williams, R.; Harris, C.M. Global coverage of cetacean line-transect surveys: Status quo, data gaps and future challenges. PLoS ONE 2012, 7, e44075. [Google Scholar] [CrossRef] [PubMed]
  17. Redfern, J.V.; Moore, T.J.; Fiedler, P.C.; de Vos, A.; Brownell, R.L., Jr.; Forney, K.A.; Becker, E.A.; Ballance, L.T. Predicting cetacean distributions in data-poor marine ecosystems. Divers. Distrib. 2017, 23, 394–408. [Google Scholar] [CrossRef]
  18. Mannocci, L.; Roberts, J.J.; Miller, D.L.; Halpin, P.N. Extrapolating cetacean densities to quantitatively assess human impacts on populations in the high seas. Conserv. Biol. 2017, 31, 601–614. [Google Scholar] [CrossRef]
  19. Reynolds, J.E., III; Marsh, H.; Ragen, T.J. Marine mammal conservation. Endanger. Species Res. 2009, 7, 23–28. [Google Scholar] [CrossRef]
  20. Brandt, M.; Tucker, C.J.; Kariryaa, A.; Rasmussen, K.; Abel, C.; Small, J.; Chave, J.; Rasmussen, L.V.; Hiernaux, P.; Diouf, A.A.; et al. An unexpectedly large count of trees in the West African Sahara and Sahel. Nature 2020, 587, 78–82. [Google Scholar] [CrossRef]
  21. Xue, Y.; Wang, T.; Skidmore, A.K. Automatic counting of large mammals from very high resolution panchromatic satellite imagery. Remote Sens. 2017, 9, 878. [Google Scholar] [CrossRef]
  22. LaRue, M.A.; Stapleton, S.; Anderson, M. Feasibility of using high-resolution satellite imagery to assess vertebrate wildlife populations. Conserv. Biol. 2017, 31, 213–220. [Google Scholar] [CrossRef]
  23. Leaper, R.; Fretwell, P. Results of a Pilot Study on the Use of Satellite Imagery to Detect Blue Whales off the South Coast of Sri Lanka. Paper SC/66a/HIM/2 Presented to the IWC Scientific Committee (unpublished). 2015, p. 9. Available online: https://archive.iwc.int/pages/search.php?search=!collection216&bc_from=themes (accessed on 30 December 2020).
  24. Jerlov, N.G. Marine Optics; Elsevier Science: Amsterdam, The Netherlands, 1976; pp. 157–161. [Google Scholar]
  25. Buckland, S.T.; Anderson, D.R.; Burnham, K.P.; Laake, J.L.; Borchers, D.L.; Thomas, L. Introduction to Distance Sampling Estimating Abundance of Biological Populations, 1st ed.; Oxford University Press: Oxford, UK, 2001. [Google Scholar]
  26. Teilmann, J.; Christiansen, C.T.; Kjellerup, S.; Dietz, R.; Nachman, G. Geographic, seasonal, and diurnal surface behavior of harbor porpoises. Mar. Mammal Sci. 2013, 29, 1–17. [Google Scholar] [CrossRef]
  27. Bamford, C.; Kelly, N.; Dalla Rosa, L.; Cade, D.; Fretwell, P.; Trathan, P.; Cubaynes, H.; Mesquita, A.; Gerrish, L.; Friedlaender, A.; et al. A comparison of baleen whale density estimates derived from overlapping satellite imagery and a shipborne survey. Sci. Rep. 2020, 10, 1–12. [Google Scholar] [CrossRef] [PubMed]
  28. Borowicz, A.; Le, H.; Humphries, G.; Nehls, G.; Höschle, C.; Kosarev, V.; Lynch, H.J. Aerial-trained deep learning networks for surveying cetaceans from satellite imagery. PLoS ONE 2019, 14. [Google Scholar] [CrossRef] [PubMed]
  29. Guirado, E.; Tabik, S.; Rivas, M.L.; Alcaraz-Segura, D.; Herrera, F. Whale counting in satellite and aerial images with deep learning. Sci. Rep. 2019, 9, 14259. [Google Scholar] [CrossRef] [PubMed]
  30. Bowler, E.; Fretwell, P.T.; French, G.; Mackiewicz, M. Using deep learning to count albatrosses from space: Assessing results in light of ground truth uncertainty. Remote Sens. 2020, 12, 2026. [Google Scholar] [CrossRef]
Figure 1. Processing steps to detect whales in very high-resolution (VHR) satellite imagery.
Figure 1. Processing steps to detect whales in very high-resolution (VHR) satellite imagery.
Sensors 21 00963 g001
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop