Shark AI Project

Identifying shark species using drones.

 
Shark with hovering drone.

About the Shark AI Project

The Shark AI project is a collaboration between astronomers at Macquarie University and scientists at the NSW Department of Primary Industries. We are applying astronomical image-processing techniques and machine learning algorithms to video footage from shark-monitoring drones.

Drone-based shark surveillance is emerging as an effective, safe and affordable approach to managing human-shark conflict. However, the method is currently labour intensive and relies heavily on pilot skill to detect and identify marine life. We believe we can vastly improve the reliability of drone-searches using by using artificial intelligence to confidently identify shark species.

Negative shark headlines.

The Problem

Shark attacks are of increasing public concern in many areas around the globe, with attacks generating media headlines worldwide. In Australia, the issue is severely impacting coastal tourism (worth over 5-billion AUD) and associated communities. To make beaches safer, Governments resort to controversial management methods, such as nets and baited drumlines, that kill sharks and other highly valued marine life. Non-destructive approaches, such as spotting sharks from helicopters, are also used but they are costly, provide low-frequency coverage, and suffer from many false-positive identifications, resulting in unnecessary beach closures that contribute to a climate of fear.

Negative shark headlines.

The Solution

Machine learning techniques have revolutionised computer vision in the last 5 years. Given enough training data, artificial neural networks (ANNs) can learn the distinguishing features of different objects and accurately recognise them in images. Our tests have shown that ANNs can reliably detect sharks in video footage from shark-spotting drones. In good conditions modern networks can even distinguish between similar looking species - for example harmless guitarfish and dangerous bull sharks. Reliable species-level identification means fewer unnecessary beach closures, fewer marine animals killed and safer beaches in general.

The Astronomy Angle

When the sea is murky or disturbed, even expert human observers find it difficult to distinguish between dangerous sharks and benign fish. This is where astronomers can help: we are investigating how image-processing techniques used in astronomy can be used to extract the best possible images of the objects under the surface. The example below demonstrates a very simple image stacking technique, where a median image is formed from a number of frames, partially compensating for the motion of the water.

Stacking example.

Results

Here are a selection of short videos showing our CNN-based object detector in action. For those interested in the technical details, the object detector is RetinaNet, implemented in Keras/TensorFlow, with ResNet50 as the backbone network architecture. We get between 95 and 99 percent accuracy on most classes of shark, but are severely affected by class imbalance for some rarer species. The DPI is targeting these rare species in order to help correct for this issue.

Footage of a bull shark off Ballina. This example has been chosen deliberately to show when our detector fails - when the shark is obscured by waves or sand. Note that each frame is classified independently (no time-domain information is used), so this illustrates that our classifier is stable and robust to dynamic water movement.

Footage of a white shark, taken from an angle by the shark-spotting drone. The classifier has been trained on data taken with a range viewing angles and can reasonably infer the species of the shark at an obtuse angle like shown here.

The Project Team

The project is led by Dr Cormac Purcell, Dr Andrew Walsh and Dr Andrew Colefax (Sci-eye) in close collaboration with Dr Paul Butcher at the NSW Department of Primary Industries.

Dr Cormac Purcell

Dr Purcell has 18 years experience analysing data from arrays of radio telescopes. Now he is applying his skills to develop machine-learning and image-processing algorithms to analyse the drone footage for this project.
[Personal Page]

Dr Andrew Walsh

Dr Walsh is a python developer with an astrophysics background and experience in machine learning, management, mentoring and public speaking. He is currently analysing Earth observation data at Geoscience Australia and is leading the data-processing effort for the Shark AI project.
[Linkedin]

Dr Andrew Colefax

Dr Colefax is a leading expert in using drones to monitor shark behaviour. He recently completed a PhD focusing on the behaviour of great white sharks and the use of drones for shark surveillance, and improving their detection performance.
[Linkedin]

Dr Paul Butcher

Dr Paul Butcher is a senior research scientist with the Fisheries Conservation Technology Unit of the DPI. Since 2016, he is has been running trials of drone technology to better detect and deter sharks off the NSW coast.

The project team also has close links to the Marine Predator Group at Macquarie University and the Centre for Translational Data Science at the University of Sydney.