Promise and pitfalls: Variable performance of AI-assisted passive acoustic monitoring of birds and frogs using BirdNET and VicFrogNET

Effective conservation relies on accurate data on species distributions and their change over time. Recent advances in low-cost acoustic sensors and artificial intelligence (AI) have transformed ecoacoustics into a scalable tool for automated, large-scale wildlife monitoring. However, two of the major limitations current AI models face are: (1) tools like BirdNET are widely used despite limited understanding of what drives variation in detection accuracy across species; and (2) equivalent open-access models for non-avian taxa such as amphibians are largely absent. We address both gaps by presenting the first open-source, multi-species frog classifier for 16 native species from southeastern Australia (VicFrogNET). To evaluate its performance, we used more than 1.5 TB of real-world passive acoustic data collected from 60 freshwater wetlands spanning approximately 300 km and analysed these recordings with both VicFrogNET and the default BirdNET bird classifier. This dataset contained 820,740 bird detections (172 species) and 2419 frog detections (15 species), with >17,000 detections manually validated. Both classifiers exhibited a bimodal performance pattern, with species tending to be detected either with very high (>90%) or very low (