Search
Presentation Details
| Interactive Human-Machine Collaboration to Characterize Vocal Repertoires |
| Maddie Cusimano1, Benjamin Hoffman1, Mark Johnson2, V�ctor Moreno-Gonz�lez3, Eva Trapote3, Vittorio Baglione3, Daniela Canestrari3, Jen-Yu Liu1, Christian Rutz4. 1Earth Species Project, Berkeley, CA, United States; 2Aarhus University, Aarhus, Aarhus, Denmark; 3Universidad de Le�n, Le�n, Le�n, Spain; 4University of St. Andrews, St. Andrews, Fife, United Kingdom |
A key challenge in animal communication research is to characterize the vocal repertoires of groups and individuals. Manual analyses of vocalization datasets scale poorly to large corpora, can introduce confirmation bias, and are difficult to replicate. While unsupervised machine learning (ML) tools have been developed to address these shortcomings, they lack systematic procedures for model selection. This presents a challenge for validating and interpreting their results. We present a method for characterizing vocal repertoires via interactive human-ML collaboration. An unsupervised ML algorithm summarizes a corpus before presenting scorers with a sequence of binary perceptual choices. These choices encourage fine-grained judgments, limit confirmation bias, and leverage system-specific expertise. This workflow inherently incorporates validation, while preserving benefits of unsupervised approaches. We apply our method to the vocalizations of a population of cooperatively-breeding carrion crows in northern Spain. Additionally, we assess inter-rater reliability, reliability between expert and non-expert scorers, and sensitivity to ML hyperparameters. |