Content uploaded by Marc Chaumont
Author content
All content in this area was uploaded by Marc Chaumont on Apr 30, 2022
Content may be subject to copyright.
A preview of the PDF is not available
Putting eagle rays on the map by coupling aerial video-surveys and deep learning
Abstract
Reliable and efficient techniques are urgently needed to monitor elasmobranch populations that face increasing threats worldwide. Aerial video-surveys provide precise and verifiable observations for the rapid assessment of species distribution and abundance in coral reefs, but the manual processing of videos is a major bottleneck for timely conservation applications. In this study, we applied deep learning for the automated detection and mapping of vulnerable eagle rays from aerial videos. A light aircraft dedicated to touristic flights allowed us to collect 42 h of aerial video footage over a shallow coral lagoon in New Caledonia (Southwest Pacific). We extracted the videos at a rate of one image per second before annotating them, yielding 314 images with eagle rays. We then trained a convolutional neural network with 80% of the eagle ray images and evaluated its accuracy on the remaining 20% (independent data sets). Our deep learning model detected 92% of the annotated eagle rays in a diversity of habitats and acquisition conditions across the studied coral lagoon. Our study offers a potential breakthrough for the monitoring of ray populations in coral reef ecosystems by providing a fast and accurate alternative to the manual processing of aerial videos. Our deep learning approach can be extended to the detection of other elasmobranchs and applied to systematic aerial surveys to not only detect individuals but also estimate species density in coral reef habitats.
... Their application in wildlife management and research has expanded rapidly in the last decade [25,26]. In the marine realm, drones have been used to elucidate movements and estimate population abundance of species using shallow water marine habitats such as coral reef sites (e.g., [26,27]) and an increasing number of studies have utilized drones to assess habitat use and population parameters of sharks [28][29][30][31][32]; however, studies of rays remain sparce [11]. Baseline information on the distribution and abundance of rays using key shallow-water habitats should therefore help to fill this knowledge gap, supporting effective monitoring, defining high-use areas and informing adaptive management processes. ...
Drones are becoming increasingly valuable tools for studying species in marine environments. Here, a consumer-grade drone was used to elucidate the distribution and population abundance of two threatened dasyatid rays, Pastinachus ater and Urogymnus granulatus, in a remote marine protected area in the Republic of Seychelles. Over six weeks in March and April 2023, a total of 80 survey flights, covering an area of 3.2 km2, recorded 1262 P. ater and 822 U. granulatus. Findings revealed previously unresolved high-use areas for both species, which almost exclusively used sandy areas within the habitat and were found in greater abundances in areas closer to the shoreline. Spatial patterns in abundance were strongly correlated between species, with both often found in mixed-species groups. The site was shown to support large populations of both species with total population abundance estimates of 2524 (2029–3019 95% CI, 0.1 CV) for P. ater and 2136 (1732–2539 95% CI, 0.09 CV) for U. granulatus. This study highlights the applicability of drones in acquiring highly useful data for delineating critical habitats and informing the adaptive management of marine protected areas.
... Drone-based aerial surveys in the marine environment are perceived as being a relatively efficient and reliable method for detecting and identifying coastal fauna, and have been used to assess animal behaviour [11], abundance [12][13][14], population health [15,16], as well minimising the potential for human-wildlife conflict, such as human-shark interactions along coastal beaches [17,18]. Despite the utility, sightability errors that affect reliability of the detections and identifications of marine life, can be apparent and similar to that reported from aerial surveys using crewed aircraft [4,8,17]. ...
Monitoring marine fauna is essential for mitigating the effects of disturbances in the marine environment, as well as reducing the risk of negative interactions between humans and marine life. Drone-based aerial surveys have become popular for detecting and estimating the abundance of large marine fauna. However, sightability errors, which affect detection reliability, are still apparent. This study tested the utility of spectral filtering for improving the reliability of marine fauna detections from drone-based monitoring. A series of drone-based survey flights were conducted using three identical RGB (red-green-blue channel) cameras with treatments: (i) control (RGB), (ii) spectrally filtered with a narrow ‘green’ bandpass filter (transmission between 525 and 550 nm), and, (iii) spectrally filtered with a polarising filter. Video data from nine flights comprising dolphin groups were analysed using a machine learning approach, whereby ground-truth detections were manually created and compared to AI-generated detections. The results showed that spectral filtering decreased the reliability of detecting submerged fauna compared to standard unfiltered RGB cameras. Although the majority of visible contrast between a submerged marine animal and surrounding seawater (in our study, sites along coastal beaches in eastern Australia) is known to occur between 515–554 nm, isolating the colour input to an RGB sensor does not improve detection reliability due to a decrease in the signal to noise ratio, which affects the reliability of detections.
... Their main advantage is the ability to provide abundance and distribution estimates across large spatial scales for a wide range of megafauna species (Mannocci et al., 2014;Martin et al., 2016;Laran et al., 2017). Video-surveys conducted from drones or light aircraft (Kiszka et al., 2016;Mannocci et al., 2021;Desgarnier et al., 2022) have the potential to outperform traditional observerbased aerial surveys in terms of accuracy and precision in the derived abundance estimates (Colefax, Butcher & Kelaher, 2018;Kelaher et al., 2020a). While many studies have investigated the effect of MPAs on the abundance of megafauna species (Bond et al., 2012;Espinoza et al., 2014;Dwyer et al., 2020;Jabado et al., 2021;Flowers et al., 2022), few have simultaneously accounted for the effect of habitat, i.e. how the spatial heterogeneity of habitat influences species abundance (Osgood, McCord & Baum, 2019;Albano et al., 2021). ...
Overfishing and habitat degradation are major threats to marine megafauna worldwide. Marine protected areas (MPAs) are effective spatial conservation tools for reducing anthropogenic pressures on threatened species but their benefits for megafauna are still debated. While the effects of MPAs on species abundances are widely reported, few studies have simultaneously investigated the confounding effect of habitat.
This study aimed at disentangling the effects of coral reef habitat and spatial protection on megafauna densities in a shallow lagoon partly covered by a no‐take MPA in New Caledonia (South‐west Pacific).
Twenty replicates of aerial‐video surveys (representing 17 h of videos) were conducted during a 5‐month period to estimate and map the densities of five megafauna taxa (dugongs, sea turtles, sharks, Dasyatidae rays and Myliobatidae rays). A permutational multivariate analysis of variance was then applied to assess and disentangle the effects of coral reef habitat obtained from high‐resolution satellite imagery and spatial protection on megafauna taxa densities.
The analysis revealed a significant effect of protection for sharks and Myliobatidae, with observed densities respectively 9 and 3 times higher inside the MPA compared with outside. The results also highlighted a significant combined effect of habitat and protection for dugongs and Dasyatidae, as well as a significant effect of habitat alone for Dasyatidae. In contrast, no significant effect of habitat or protection was detected for sea turtles.
In conclusion, this study revealed positive effects of protection (alone or combined with habitat) for four of the five studied megafauna taxa, confirming the effectiveness of the current MPA. Future studies should be conducted over broader spatial and temporal scales to examine whether detected effects hold beyond the surveyed period and area.
... Moreover, one of the potential applications can be assembling object detection framework with semantic segmentation methods such as Mask R-CNN (Bharati and Pramanik, 2020), U-Net (Esser et al., 2018) to extract additional physical information such as diseases, body fat, height as well as various animal activities including eating, running, and resting which can be helpful in better understanding animal health and habits (Norouzzadeh et al., 2018). Nevertheless, the current deep-learning model outshines classical automated image analysis and various state-of-the-art approaches in wildlife animal detection indicating future improvements in performance and usability for the precise and accurate endangered animal survey which can be applied to various automated wildlife monitoring (Desgarnier et al., 2022;Hou et al., 2020;Chen et al., 2020;Mannocci et al., 2021;Arbieu et al., 2021) and different biological conservation purposes (Stern and Humphries, 2022). The current framework can also be extended for various fault detection/thermal imaging (Glowacz, 2021b,a,c), human activity recognition (Xiao et al., 2021b) etc. ...
... Moreover, one of the potential applications can be assembling object detection framework with semantic segmentation methods such as Mask R-CNN (Bharati and Pramanik, 2020), U-Net (Esser et al., 2018) to extract additional physical information such as diseases, body fat, height as well as various animal activities including eating, running, and resting which can be helpful in better understanding animal health and habits (Norouzzadeh et al., 2018). Nevertheless, the current deeplearning model outshines classical automated image analysis and various state-of-the-art approaches in wildlife animal detection indicating future improvements in performance and usability for the precise and accurate endangered animal survey which can be applied to various automated wildlife monitoring (Arbieu et al., 2021;Chen et al., 2020;Desgarnier et al., 2022;Hou et al., 2020;Mannocci et al., 2021) and different biological conservation purposes (Stern and Humphries, 2022). ...
Objective. With climatic instability, various ecological disturbances, and human actions threaten the existence of various endangered wildlife species. Therefore, an up-to-date accurate and detailed detection process plays an important role in protecting biodiversity losses, conservation, and ecosystem management. Current state-of-the-art wildlife detection models, however, often lack superior feature extraction capability in complex environments, limiting the development of accurate and reliable detection models. Method. To this end, we present WilDect-YOLO, a deep learning (DL)-based automated high-performance detection model for real-time endangered wildlife detection. In the model, we introduce a residual block in the CSPDarknet53 backbone for strong and discriminating deep spatial features extraction and integrate DenseNet blocks to improve in preserving critical feature information. To enhance receptive field representation, preserve fine-grain localized information, and improve feature fusion, a Spatial Pyramid Pooling (SPP) and modified Path Aggregation Network (PANet) have been implemented that results in superior detection under various challenging environments. Results. Evaluating the model performance in a custom endangered wildlife dataset considering high variability and complex backgrounds, WilDect-YOLO obtains a mean average precision (mAP) value of 96.89%, F1-score of 97.87%, and precision value of 97.18% at a detection rate of 59.20 FPS outperforming current state-of-the-art models. Significance. The present research provides an effective and efficient detection framework addressing the shortcoming of existing DL-based wildlife detection models by providing highly accurate species-level localized bounding box prediction. Current work constitutes a step toward a non-invasive, fully automated animal observation system in real-time in-field applications.
The scale and drivers of marine biodiversity loss are being revealed by the International Union for Conservation of Nature (IUCN) Red List assessment process. We present the first global reassessment of 1,199 species in Class Chondrichthyes—sharks, rays, and chimeras. The first global assessment (in 2014) concluded that one-quarter (24%) of species were threatened. Now, 391 (32.6%) species are threatened with extinction. When this percentage of threat is applied to Data Deficient species, more than one-third (37.5%) of chondrichthyans are estimated to be threatened, with much of this change resulting from new information. Three species are Critically Endangered (Possibly Extinct), representing possibly the first global marine fish extinctions due to overfishing. Consequently, the chondrichthyan extinction rate is potentially 25 extinctions per million species years, comparable to that of terrestrial vertebrates. Overfishing is the universal threat affecting all 391 threatened species and is the sole threat for 67.3% of species and interacts with three other threats for the remaining third: loss and degradation of habitat (31.2% of threatened species), climate change (10.2%), and pollution (6.9%). Species are disproportionately threatened in tropical and subtropical coastal waters. Science-based limits on fishing, effective marine protected areas, and approaches that reduce or eliminate fishing mortality are urgently needed to minimize mortality of threatened species and ensure sustainable catch and trade of others. Immediate action is essential to prevent further extinctions and protect the potential for food security and ecosystem functions provided by this iconic lineage of predators.
Background
The ability to automatically count animals is important to design appropriate environmental policies and to monitor their populations in relation to biodiversity and maintain balance among species. Out of all living mammals on Earth, 60% are livestock, 36% humans, and only 4% are animals that live in the wild. In a relatively short period, development of human civilization caused a loss of 83% of wildlife and 50% of plants. The rate of species extinction is accelerating. Traditional wildlife surveys provide rough population estimates. However, emerging technologies, such as aerial photography, allow to perform large-scale surveys in a short period of time with high accuracy. In this paper, we propose the use of computer vision, through deep learning (DL) architecture, together with aerial photography and density maps, to count the population of Steller sea lions and African elephants with high precision.
Results
We have trained two deep learning models, a basic UNet without any feature extractor (Model-1) and another with the EfficientNet-B5 feature extractor (Model-2). We measured the model’s prediction accuracy, using Root Mean Square Error (RMSE) for the predicted and actual animal count. The results showed an RMSE of 1.88 and 0.60 to count Steller sea lions and African elephants, respectively, regardless of complex background, different illumination conditions, heavy overlapping and occlusion of the animals.
Conclusions
Our proposed solution performed very well in the counting prediction problem, with relatively low training parameters and minimum annotation. The approach adopted, combining DL and density maps, provided better results than state-of-art deep learning models used for counting, indicating that the proposed method has the potential to be used more widely in large-scale wildlife surveying projects and initiatives.
Deep learning has become a key tool for the automated monitoring of animal populations with video surveys. However, obtaining large numbers of images to train such models is a major challenge for rare and elusive species because field video surveys provide few sightings. We designed a method that takes advantage of videos accumulated on social media for training deep‐learning models to detect rare megafauna species in the field. We trained convolutional neural networks (CNNs) with social media images and tested them on images collected from field surveys. We applied our method to aerial video surveys of dugongs (Dugong dugon) in New Caledonia (southwestern Pacific). CNNs trained with 1303 social media images yielded 25% false positives and 38% false negatives when tested on independent field video surveys. Incorporating a small number of images from New Caledonia (equivalent to 12% of social media images) in the training data set resulted in a nearly 50% decrease in false negatives. Our results highlight how and the extent to which images collected on social media can offer a solid basis for training deep‐learning models for rare megafauna detection and that the incorporation of a few images from the study site further boosts detection accuracy. Our method provides a new generation of deep‐learning models that can be used to rapidly and accurately process field video surveys for the monitoring of rare megafauna.
Abstract Machine learning algorithms are being increasingly used to process large volumes of wildlife imagery data from unmanned aerial vehicles (UAVs); however, suitable algorithms to monitor multiple species are required to enhance efficiency. Here, we developed a machine learning algorithm using a low‐cost computer. We trained a convolutional neural network and tested its performance in: (1) distinguishing focal organisms of three marine taxa (Australian fur seals, loggerhead sea turtles and Australasian gannets; body size ranges: 0.8–2.5 m, 0.6–1.0 m, and 0.8–0.9 m, respectively); and (2) simultaneously delineating the fine‐scale movement trajectories of multiple sea turtles at a fish cleaning station. For all species, the algorithm performed best at detecting individuals of similar body length, displaying consistent behaviour or occupying uniform habitat (proportion of individuals detected, or recall of 0.94, 0.79 and 0.75 for gannets, seals and turtles, respectively). For gannets, performance was impacted by spacing (huddling pairs with offspring) and behaviour (resting vs. flying shapes, overall precision: 0.74). For seals, accuracy was impacted by morphology (sexual dimorphism and pups), spacing (huddling and creches) and habitat complexity (seal sized boulders) (overall precision: 0.27). For sea turtles, performance was impacted by habitat complexity, position in water column, spacing, behaviour (interacting individuals) and turbidity (overall precision: 0.24); body size variation had no impact. For sea turtle trajectories, locations were estimated with a relative positioning error of
Extinctions on land are often inferred from sparse sightings over time, but this technique is ill-suited for wide-ranging species. We develop a space-for-time approach to track the spatial contraction and drivers of decline of sawfishes. These iconic and endangered shark-like rays were once found in warm, coastal waters of 90 nations and are now presumed extinct in more than half (n = 46). Using dynamic geography theory, we predict that sawfishes are gone from at least nine additional nations. Overfishing and habitat loss have reduced spatial occupancy, leading to local extinctions in 55 of the 90 nations, which equates to 58.7% of their historical distribution. Retention bans and habitat protections are urgently necessary to secure a future for sawfishes and similar species.
Available at: https://advances.sciencemag.org/content/7/7/eabb6026
Overfishing is the primary cause of marine defaunation, yet declines in and increasing extinction risks of individual species are difficult to measure, particularly for the largest predators found in the high seas. Here we calculate two well-established indicators to track progress towards Aichi Biodiversity Targets and Sustainable Development Goals: the Living Planet Index (a measure of changes in abundance aggregated from 57 abundance time-series datasets for 18 oceanic shark and ray species) and the Red List Index (a measure of change in extinction risk calculated for all 31 oceanic species of sharks and rays). We find that, since 1970, the global abundance of oceanic sharks and rays has declined by 71% owing to an 18-fold increase in relative fishing pressure. This depletion has increased the global extinction risk to the point at which three-quarters of the species comprising this functionally important assemblage are threatened with extinction. Strict prohibitions and precautionary science-based catch limits are urgently needed to avert population collapse, avoid the disruption of ecological functions and promote species recovery.
Due to well-documented declines in many shark populations there is increasing pressure to implement new management and rebuilding strategies at the national and international scale. Since 2009, fifteen coastal countries in the Atlantic, Indian and Pacific Oceans have opted to ban commercial shark fishing altogether, and have laws that prohibit the possession, trade or sale of sharks and shark products. These ‘shark sanctuaries’ collectively cover >3% of the world’s oceans, a similar coverage as all currently established marine protected areas combined. Despite their prominence, and an intense scientific debate about their usefulness, the condition of shark sanctuaries has not yet been empirically evaluated. Here, we report results from a global diver survey used to set baselines of shark populations, human use patterns, public awareness and threats in all 15 shark sanctuaries, and contrasted with observations from 23 non-sanctuary countries. Specific results varied by country, but there were some general trends: i) shark sanctuaries showed less pronounced shark population declines, fewer observations of sharks being sold on markets, and lower overall fishing threats compared to non-shark sanctuaries, ii) bycatch, ghost gear, marine debris and habitat destruction are significant threats that are often not addressed by sanctuary regulations and need to be resolved in other ways, and iii) participants in sanctuaries were more optimistic about the survival of shark populations in local waters, but also highlighted the need for further conservation efforts. These results suggest that shark sanctuaries, as seen through the lens of local experts, may be a helpful conservation tool but likely not sufficient in isolation. There is an urgent need for higher-resolution data on shark abundance, incidental catch, and markets to direct priority conservation needs and optimize the conservation benefits of existing and future shark sanctuaries.
Much research on object detection focuses on building better model architectures and detection algorithms. Changing the model architecture, however, comes at the cost of adding more complexity to inference, making models slower. Data augmentation, on the other hand, doesn’t add any inference complexity, but is insufficiently studied in object detection for two reasons. First it is more difficult to design plausible augmentation strategies for object detection than for classification, because one must handle the complexity of bounding boxes if geometric transformations are applied. Secondly, data augmentation attracts less research attention perhaps because it is believed to add less value and to transfer poorly compared to advances in network architectures.