Drones and Deep Learning for Detecting Fish Carcasses During Fish Kills
Abstract
1. Introduction
2. Materials and Methods
2.1. Study Sites
2.2. Drone Image Acquisition
2.3. Drone Image Processing and Tiling
2.4. Drone Image Annotation
2.5. Model Architecture and Training
2.6. Loss Metrics
2.7. Evaluation Metrics
2.8. Fish Species Composition Estimates and Length Measurements
3. Results
4. Discussion
4.1. Monitoring Shored Carcasses
4.2. Monitoring Floating Carcasses
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- La, V.T.; Cooke, S.J. Advancing the science and practice of fish kill investigations. Rev. Fish. Sci. 2011, 19, 21–33. [Google Scholar] [CrossRef]
- Meyer, F.P.; Barclay, L.A. Field Manual for the Investigation of Fish Kills; U.S. Fish and Wildlife Service: Washington, DC, USA, 1990. [Google Scholar]
- Thronson, A.; Quigg, A. Fifty-five years of fish kills in Coastal Texas. Estuaries Coasts 2008, 31, 802–813. [Google Scholar] [CrossRef]
- Holmlund, C.M.; Hammer, M. Ecosystem services generated by fish populations. Ecol. Econ. 1999, 29, 253–268. [Google Scholar] [CrossRef]
- Vanni, M.J.; Luecke, C.; Kitchell, J.F.; Allen, Y.; Temte, J.; Magnuson, J.J. Effects on lower trophic levels of massive fish mortality. Nature 1990, 344, 333–335. [Google Scholar] [CrossRef]
- Tribune News Service. Algal blooms can wreak havoc on Florida economy. Tampa Bay Times, 21 January 2024. [Google Scholar]
- American Fisheries Society. Investigation and Valuation of Fish Kills; American Fisheries Society, Southern Division, Pollution Committee: Bethesda, MD, USA, 1992; ISBN 978-0-913235-81-2. [Google Scholar]
- Powers, S.P.; Blanchet, H. Investigate Plan for Fish and Invertebrate Kills in the Northern Gulf of Mexico. MC 252 NRDA Fish Technical Working Group; 2010. Available online: https://www.google.com.hk/url?sa=t&source=web&rct=j&opi=89978449&url=https://www.fws.gov/doiddata/dwh-ar-documents/824/DWH-AR0013020.pdf&ved=2ahUKEwjPr5qU4KyOAxUF-aACHZEbH7MQFnoECBUQAQ&usg=AOvVaw31_k2zc8q05PvjocqaWOmB (accessed on 1 July 2025).
- Virginia Department of Environmental Quality. Fish Kill Investigation Guidance Manual; Water Quality Standards and Biological Monitoring Programs: Richmond, Virginia, 2002.
- Biernacki, E. Fish Kills Caused by Pollution.; U.S. Environmental Protection Agency: Washington, DC, USA, 1979.
- Fernandez-Figueroa, E.G.; Wilson, A.E.; Rogers, S.R. Commercially available unoccupied aerial systems for monitoring harmful algal blooms: A comparative study. Limnol. Oceanogr. Methods 2022, 20, 146–158. [Google Scholar] [CrossRef]
- Li, J.Y.Q.; Duce, S.; Joyce, K.E.; Xiang, W. SeeCucumbers: Using Deep Learning and Drone Imagery to Detect Sea Cucumbers on Coral Reef Flats. Drones 2021, 5, 28. [Google Scholar] [CrossRef]
- Grigusova, P.; Larsen, A.; Achilles, S.; Klug, A.; Fischer, R.; Kraus, D.; Übernickel, K.; Paulino, L.; Pliscoff, P.; Brandl, R.; et al. Area-Wide Prediction of Vertebrate and Invertebrate Hole Density and Depth across a Climate Gradient in Chile Based on UAV and Machine Learning. Drones 2021, 5, 86. [Google Scholar] [CrossRef]
- Povlsen, P.; Bruhn, D.; Durdevic, P.; Arroyo, D.O.; Pertoldi, C. Using YOLO Object Detection to Identify Hare and Roe Deer in Thermal Aerial Video Footage—Possible Future Applications in Real-Time Automatic Drone Surveillance and Wildlife Monitoring. Drones 2024, 8, 2. [Google Scholar] [CrossRef]
- Zhang, H.; Tian, Z.; Liu, L.; Liang, H.; Feng, J.; Zeng, L. Real-time detection of dead fish for unmanned aquaculture by yolov8-based UAV. Aquaculture 2025, 595, 741551. [Google Scholar] [CrossRef]
- Inman, V.L.; Leggett, K.E.A. Hidden Hippos: Using Photogrammetry and Multiple Imputation to Determine the Age, Sex, and Body Condition of an Animal Often Partially Submerged. Drones 2022, 6, 409. [Google Scholar] [CrossRef]
- Morimura, N.; Itahara, A.; Brooks, J.; Mori, Y.; Piao, Y.; Hashimoto, H.; Mizumoto, I. A Drone Study of Sociality in the Finless Porpoise (Neophocaena asiaeorientalis) in the Ariake Sound, Japan. Drones 2023, 7, 422. [Google Scholar] [CrossRef]
- Federal Aviation Administration. Part 107—Small Unmanned Aircraft Systems; Federal Aviation Administration, Department of Transportation: Washington, DC, USA, 2023.
- Civil Aviation Safety Authority. Part 101 (Unmanned Aircraft and Rockets) Manual of Standards; Australian Department of Infrastructure, Transport, Regional Development, Communications, Sport and the Arts: Canberra, Australia, 2019.
- Nahirnick, N.K.; Reshitnyk, L.; Campbell, M.; Hessing-Lewis, M.; Costa, M.; Yakimishyn, J.; Lee, L. Mapping with confidence; delineating seagrass habitats using Unoccupied Aerial Systems (UAS). Remote Sens. Ecol. Conserv. 2019, 5, 121–135. [Google Scholar] [CrossRef]
- Hong, S.-J.; Han, Y.; Kim, S.-Y.; Lee, A.-Y.; Kim, G. Application of Deep-Learning Methods to Bird Detection Using Unmanned Aerial Vehicle Imagery. Sensors 2019, 19, 1651. [Google Scholar] [CrossRef] [PubMed]
- Beck, M.W.; Altieri, A.; Angelini, C.; Burke, M.C.; Chen, J.; Chin, D.W.; Gardiner, J.; Hu, C.; Hubbard, K.A.; Liu, Y.; et al. Initial estuarine response to inorganic nutrient inputs from a legacy mining facility adjacent to Tampa Bay, Florida. Mar. Pollut. Bull. 2022, 178, 113598. [Google Scholar] [CrossRef]
- Sarasota Bay Estuary Program Director’s Note: Red Tide and Fish Kills—Likely Effects of Hurricane Ian, and Scalability of Management Paradigms. Available online: https://sarasotabay.org/directors-note-red-tide-and-fish-kills-likely-effects-of-hurricane-ian-and-scalability-of-management-paradigms/ (accessed on 6 March 2024).
- Tkachenko, M.; Malyuk, M.; Holmanyuk, A.; Liubimov, N. Label Studio: Data Labeling Software 2022. Available online: https://github.com/heartexlabs/label-studio (accessed on 1 July 2025).
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 21–37. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Bangar, S. Resnet Architecture. Available online: https://medium.com/@siddheshb008/resnet-architecture-explained-47309ea9283d (accessed on 10 January 2025).
- Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv 2016, arXiv:1603.04467. [Google Scholar]
- Krizhevsky, A. Convolutional Deep Belief Networks on CIFAR-10, 2010. Available online: https://www.google.com.hk/url?sa=t&source=web&rct=j&opi=89978449&url=https://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf&ved=2ahUKEwjt8YK-4ayOAxU7zzgGHUq0BMgQFnoECBoQAQ&usg=AOvVaw0H-S5UBwzu01elgnKfroeM (accessed on 1 July 2025).
- Diwan, T.; Anirudh, G.; Tembhurne, J.V. Object detection using YOLO: Challenges, architectural successors, datasets and applications. Multimed. Tools Appl. 2023, 82, 9243–9275. [Google Scholar] [CrossRef]
- Lin, T.-Y.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C.L.; Dollár, P. Microsoft COCO: Common Objects in Context. arXiv 2015, arXiv:1405.0312. [Google Scholar]
- Cocodataset/Cocoapi. 2024. Available online: https://github.com/cocodataset/cocoapi (accessed on 1 July 2025).
- Terven, J.; Cordova-Esparza, D.-M. A Comprehensive Review of YOLO: From YOLOv1 to YOLOv8 and Beyond. arXiv 2023, arXiv:2304.00501. [Google Scholar]
- Kells, V.; Carpenter, K. A Field Guide to Coastal Fishes: From Maine to Texas. In Biological Sciences Faculty Books; Johns Hopkins University Press: Baltimore, MD, USA, 2011. [Google Scholar]
- Florida Fish and Wildlife Conservation Commission Species Profiles. Available online: https://myfwc.com/wildlifehabitats/profiles/ (accessed on 20 December 2022).
- Fernandez-Figueroa, E.G.; Mapes, S.A.; Rogers, S.R. Fish kill lessons and data needs: A spatiotemporal analysis of citizen fish kill reports in coastal SW Florida. Mar. Ecol. Prog. Ser. 2024, 742, 21–33. [Google Scholar] [CrossRef]
- Bah, M.D.; Hafiane, A.; Canals, R. Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images. Remote Sens. 2018, 10, 1690. [Google Scholar] [CrossRef]
- Lenzi, J.; Barnas, A.F.; ElSaid, A.A.; Desell, T.; Rockwell, R.F.; Ellis-Felege, S.N. Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys. Sci. Rep. 2023, 13, 947. [Google Scholar] [CrossRef] [PubMed]
- Zhang, H.; Liptrott, M.; Bessis, N.; Cheng, J. Real-Time Traffic Analysis using Deep Learning Techniques and UAV based Video. In Proceedings of the 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, China, 18–21 September 2019; pp. 1–5. [Google Scholar]
Flight ID | Site ID | Date | Start Time | Flight Time (min.sec) | Flight Dim (m) | Flight Alt (m) | GSD (cm/pixel) | Temp (°C) | Solar Altitude | Azimuth | Marked Tiles |
---|---|---|---|---|---|---|---|---|---|---|---|
1a | 1 | 13 July 2021 | 7:42 PM | 5.50 | 17 × 46 | 9.8 | 0.44 | 27 | 11.51 | 71.29 | 20/480 |
1b | 1 | 14 July 2021 | 8:01 AM | 5.23 | 17 × 46 | 9.8 | 0.44 | 26 | 14.99 | 73.13 | 17/437 |
2a | 2 | 14 July 2021 | 11:02 AM | 6.27 | 17 × 53 | 9.8 | 0.44 | 29 | 54.42 | 91.2 | 36/341 |
2b | 2 | 15 July 2021 | 4:18 PM | 6.32 | 17 × 53 | 9.8 | 0.44 | 32 | 53.03 | 269.22 | 34/286 |
3a | 3 | 14 July 2021 | 9:19 AM | 5.16 | 17 × 46 | 9.8 | 0.44 | 27 | 31.75 | 80.58 | 45/560 |
3b | 3 | 16 July 2021 | 7:51 AM | 5.22 | 17 × 46 | 9.8 | 0.44 | 28 | 12.71 | 72.41 | 35/234 |
3c | 3 | 10 December 2022 | 8:02 AM | 14.54 | 28 × 162 | 10.3 | 0.40 | 18 | 10.74 | 123.17 | 11/2328 |
Fish Group | Mean Length (cm) | SD | Count |
---|---|---|---|
American eel | 54.48 | 13.19 | 8 |
American shad/ Scaled sardine/ Atlantic thread herring | 10.16 | n.a. | 1 |
Black grouper | 71.94 | n.a. | 1 |
Grunt | 19.32 | 0.45 | 2 |
Mullet | 37.17 | 9.37 | 4 |
Pinfish | 12.03 | 4.25 | 147 |
Scrawled cowfish | 17.04 | 5.46 | 3 |
Sea robin | 12.76 | 3.95 | 3 |
Spotted seatrout | 49.83 | n.a. | 1 |
Striped burrfish | 14.91 | 1.95 | 4 |
Toadfish | 22.55 | 12.88 | 7 |
Metric | IoU Range | Object Area | Max Detections | AP | AR |
---|---|---|---|---|---|
Average Precision (AP) | 0.50:0.95 | All | 100 | 0.773 | 0.810 |
Average Precision (AP) | 0.50 | All | 100 | 0.979 | |
Average Precision (AP) | 0.75 | All | 100 | 0.951 | |
Average Precision (AP) | 0.50:0.95 | Small | 100 | 0.769 | 0.807 |
Average Precision (AP) | 0.50:0.95 | Medium | 100 | 0.822 | 0.851 |
Average Precision (AP) | 0.50:0.95 | Large | 100 | 0.900 | 0.900 |
Average Recall (AR) | 0.50:0.95 | All | 1 | 0.049 | |
Average Recall (AR) | 0.50:0.95 | All | 10 | 0.361 | |
Average Recall (AR) | 0.50:0.95 | All | 100 | 0.773 | 0.810 |
Average Recall (AR) | 0.50:0.95 | Small | 100 | 0.769 | 0.807 |
Average Recall (AR) | 0.50:0.95 | Medium | 100 | 0.822 | 0.851 |
Average Recall (AR) | 0.50:0.95 | Large | 100 | 0.900 | 0.900 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Fernandez-Figueroa, E.G.; Rogers, S.R.; Neupane, D. Drones and Deep Learning for Detecting Fish Carcasses During Fish Kills. Drones 2025, 9, 482. https://doi.org/10.3390/drones9070482
Fernandez-Figueroa EG, Rogers SR, Neupane D. Drones and Deep Learning for Detecting Fish Carcasses During Fish Kills. Drones. 2025; 9(7):482. https://doi.org/10.3390/drones9070482
Chicago/Turabian StyleFernandez-Figueroa, Edna G., Stephanie R. Rogers, and Dinesh Neupane. 2025. "Drones and Deep Learning for Detecting Fish Carcasses During Fish Kills" Drones 9, no. 7: 482. https://doi.org/10.3390/drones9070482
APA StyleFernandez-Figueroa, E. G., Rogers, S. R., & Neupane, D. (2025). Drones and Deep Learning for Detecting Fish Carcasses During Fish Kills. Drones, 9(7), 482. https://doi.org/10.3390/drones9070482