Watched over by vision-language models of loving grace: Monitoring primate trade in online spaces

Indonesia has amongst the highest nonhuman primate species richness, with trade a significant factor in the decline of species diversity and population numbers. Detecting online offers for sale requires advanced methods as traditional natural language processing techniques have proven limited in handling the variety and semantic nuance of textual content found online. Object detection models for image processing require large domain-specific datasets for improved accuracy. This can prove expensive, and even unrealistic for projects that do not have access to such a dataset or model.Large language models (LLM) based on the Transformer deep learning architecture promise to perform many of these tasks, without specific training. In response to prompts, the models can answer questions and synthesise new content. Recent improvements have made responses sound more human, and addressed many issues found in earlier iterations. Multi-modal models can now process images, video and audio. The models have been successfully applied to thematic analysis and interpretation. At the same time, their black box nature and continued occurrence of biased and inaccurate responses require mitigation through careful study design and human oversight.We will present our use of vision-language models (VLMs) to analyse posts collected from social media sites, querying whether the post is likely an instance of primate trade. The responses aid in detecting instances of primate trade among tens of thousands other irrelevant posts. We will present our post collection and processing workflow as well response validation and prompt refinement. Finally, we will discuss the results and future direction for our work.ABOUT THE AUTHORNoel Zeng works at the Centre for eResearch, Waipapa Taumata Rau University of Auckland. He worked on the Instrument Data Service and led consultancy projects with research groups in urban planning, archaeology and anthropology, supporting software development, data collection and analysis. He is interested in applying his software development and data analysis experience to collaborate with academics from across disciplines, especially social sciences and humanities. -----For more information about the eResearch NZ / eRangahau Aotearoa conference, visit: https://eresearchnz.co.nz/