Audiovisual Analysis of Journal Digital

Read full text
Authors: Mathias Johansson, Johan Malmstedt
Cited by: 0

Abstract

Automated extraction and analysis of audio-visual co-occurrences in video data using deep learning models. ## Overview This project analyzes temporal co-occurrences of audio and visual features in videos: - Audio extraction : Uses MIT's Audio Spectrogram Transformer (AST) to classify audio content - Visual extraction : Uses Moondream vision model for object detection in frames - Pairing analysis : Identifies audio-visual co-occurrences with temporal alignment - Temporal analysis : Tracks changes in audio-visual patterns across years ## Structure - src.zip : The source code for the library - results.zip : The .csv outputs - moon_results.tar.gz : Results from the Visual extraction pipeline - audio_results.tar.gz : Results forom the Audio extraction pipeline

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
2 January 2026
Document type
Dataset