Study on Fish Behavior Recognition
Overview
Abigail Grassick, a doctoral candidate in computational biology, has focused her research on analyzing fish communities dwelling on bommies (coral clumps) off the coast of Curaçao. Her initial approach utilized existing computer vision models to track and identify fish behaviors, such as feeding or predator evasion. However, these models proved inadequate, prompting Grassick and her team to create a dedicated dataset.
The WildFin Dataset
To address the limitations of existing models, Grassick and a team of researchers developed WildFin, comprising nine hours of annotated underwater video detailing various fish behaviors. The team labeled the footage frame-by-frame, aiming to assist computer scientists in building more advanced models. The footage was collected during extensive fieldwork and includes varied scenarios to challenge these models.
Research and Collaboration
Grassick presented her work, titled “WildFin: An In-the-Wild Dataset for Fish Behavioral Recognition”, at the European Conference on Computer Vision. Co-author Andrew Hein emphasized the need for more high-quality wildlife footage in training models, noting that current models rely on a small fraction of scientific imagery.
Importance of Improved Models
Better computer vision tools could revolutionize ecological research, enabling more efficient data collection and monitoring of ecosystems. Hein expressed optimism about utilizing citizen science footage to expand the data pool, allowing for broader ecological insights without the extensive resource expenditure typically required for field studies.
Methodology and Challenges
Grassick and her undergraduate team meticulously annotated the bombie videos to catalog fish behaviors like foraging and fighting. Collaborations with researchers from the University of Colorado, Boulder, ensured a robust dataset. Despite the concerted effort—1,400 hours in the field and 600 hours of annotation—the models struggled to classify fish behavior accurately.
Future Directions
The researchers highlight the interdisciplinary nature of the problem, calling for the collaboration of statisticians, machine learning experts, and ecologists to improve analysis methods. They also encourage other ecologists to contribute their field recordings to enrich the training dataset for future models.
Call to Action
The initiative seeks to revitalize unused ecological footage, suggesting that AI could play a role in converting raw data into a usable format. Grassick and her collaborators urge researchers to harness these remnants of past studies to propel the field of fish behavior recognition forward.
Supporting Contributors
The paper includes contributions from a diverse team, including undergraduates and doctoral students from various institutions, with support partially funded by the National Science Foundation and NVIDIA’s Academic Grant Program.
Author: Patricia Waldron, Cornell Ann S. Bowers College of Computing and Information Science.
