About the Shark AI Project
The Shark AI project is a collaboration between astronomers at Macquarie University and scientists at the NSW Department of Primary Industries. We are applying astronomical image-processing techniques and machine learning algorithms to video footage from shark-monitoring drones.
Drone-based shark surveillance is emerging as an effective, safe and affordable approach to managing human-shark conflict. However, the method is currently labour intensive and relies heavily on pilot skill to detect and identify marine life. We believe we can vastly improve the reliability of drone-searches using by using artificial intelligence to confidently identify shark species.
Shark attacks are of increasing public concern in many areas around the globe, with attacks generating media headlines worldwide. In Australia, the issue is severely impacting coastal tourism (worth over 5-billion AUD) and associated communities. To make beaches safer, Governments resort to controversial management methods, such as nets and baited drumlines, that kill sharks and other highly valued marine life. Non-destructive approaches, such as spotting sharks from helicopters, are also used but they are costly, provide low-frequency coverage, and suffer from many false-positive identifications, resulting in unnecessary beach closures that contribute to a climate of fear.
Machine learning techniques have revolutionised computer vision in the last 5 years. Given enough training data, artificial neural networks (ANNs) can learn the distinguishing features of different objects and accurately recognise them in images. Our tests have shown that ANNs can reliably detect sharks in video footage from shark-spotting drones. In good conditions modern networks can even distinguish between similar looking species - for example harmless guitarfish and dangerous bull sharks. Reliable species-level identification means fewer unnecessary beach closures, fewer marine animals killed and safer beaches in general.
The Astronomy Angle
When the sea is murky or disturbed, even expert human observers find it difficult to distinguish between dangerous sharks and benign fish. This is where astronomers can help: we are investigating how image-processing techniques used in astronomy can be used to extract the best possible images of the objects under the surface. The example below demonstrates a very simple image stacking technique, where a median image is formed from a number of frames, partially compensating for the motion of the water.
Here are a selection of short videos showing our CNN-based object detector in action. For those interested in the technical details, the object detector is RetinaNet, implemented in Keras/TensorFlow, with ResNet50 as the backbone network architecture. We get between 95 and 99 percent accuracy on most classes of shark, but are severely affected by class imbalance for some rarer species. The DPI is targeting these rare species in order to help correct for this issue.
Footage of a bull shark off Ballina. This example has been chosen deliberately to show when our detector fails - when the shark is obscured by waves or sand. Note that each frame is classified independently (no time-domain information is used), so this illustrates that our classifier is stable and robust to dynamic water movement.
Footage of a white shark, taken from an angle by the shark-spotting drone. The classifier has been trained on data taken with a range viewing angles and can reasonably infer the species of the shark at an obtuse angle like shown here.
The Project Team
The project is led by Dr Cormac Purcell, Dr Andrew Walsh and Dr Andrew Colefax (Sci-eye) in close collaboration with Dr Paul Butcher at the NSW Department of Primary Industries.
Dr Purcell has 18 years experience analysing data
from arrays of radio telescopes. Now he is applying his
skills to develop machine-learning and image-processing
algorithms to analyse the drone footage for this
Dr Walsh is a python developer with an astrophysics
background and experience in machine learning,
management, mentoring and public speaking. He is
currently analysing Earth observation data at Geoscience
Australia and is leading the data-processing effort for
the Shark AI project.
Dr Colefax is a leading expert in using drones to
monitor shark behaviour. He recently completed a PhD focusing
on the behaviour of great white sharks and the use of
drones for shark surveillance, and improving their
Dr Paul Butcher is a senior research scientist with the Fisheries Conservation Technology Unit of the DPI. Since 2016, he is has been running trials of drone technology to better detect and deter sharks off the NSW coast.
The project team also has close links to the Marine Predator Group at Macquarie University and the Centre for Translational Data Science at the University of Sydney.