Sonification of Visual Art for Accessibility: A Mathematical Pipeline from Painting Structure to Auditory Signature — Argira Station v1.0

Read full text
Authors: Jose Ranero García
Cited by: 4
References: 1

Abstract

This repository contains the first version of the Argira Sonification Pipeline: a system that converts digitized paintings into unique auditory signatures through mathematical analysis of their pictorial structure. PRIMARY APPLICATION — ACCESSIBILITY FOR BLIND AND VISUALLY IMPAIRED PEOPLE The system does not describe paintings verbally. It translates the mathematical structure of each painting — its texture, color, spatial composition, and proportions — directly into sound. A blind person can experience the aesthetic difference between Monet and Van Gogh through their auditory signatures, without any verbal mediation. This is the core mission of the project: genuine aesthetic access to visual art for blind and visually impaired people. All other applications are secondary to this purpose. WHAT THIS REPOSITORY CONTAINS - sonify_painting.py — complete Python pipeline (CC BY-NC 4.0) argira_sonification_proposal_v2.docx — full methodological proposal monet_amapolas.wav — auditory signature of Monet's Poppies (1873) vangogh_noche_estrellada.wav — auditory signature of Van Gogh's Starry Night (1889) monet_v2.wav — auditory signature v2 of Monet's Poppies (refined mapping) vangogh_v2.wav — auditory signature v2 of Van Gogh's Starry Night (refined mapping) - README.md — documentation and usage instructions MATHEMATICAL PIPELINE Each painting is analyzed along four dimensions: 1. Fractal texture complexity (box-counting dimension D ∈ [1.0, 2.0]) 2. Color distribution (HSV space: hue, saturation, brightness, dispersion) 3. Spatial composition (9-zone visual weight grid, center of mass) 4. Golden proportion (distance to golden point — same hypothesis as Argira iris project) These four measurements are mapped to acoustic parameters: - Base frequency ← dominant hue (color is the largest inter-author difference) - Harmonic richness ← color saturation - Timbre ← hue dispersion (dispersed palette = rougher sound) - Tempo ← spatial variance (dynamic composition = more rhythmic pulse) - Vibrato/modulation ← fractal dimension + hue dispersion - Decay ← brightness (dark paintings = longer, sustained sound) FIRST EXPERIMENTAL EVIDENCE (March 2026) Two canonical paintings were sonified and compared: - Monet Poppies: D=1.794, hue=0.157 (red-orange), base frequency 803 Hz - Van Gogh Starry Night: D=1.841, hue=0.474 (blue-violet), base frequency 273 Hz Frequency difference: 530 Hz. The signatures were perceptually distinguishable by human listeners on first comparison. This constitutes first-order evidence for Hypothesis 1: paintings by different authors produce mathematically and perceptually distinct auditory signatures. RESEARCH HYPOTHESES H1: Each painting has a unique mathematical signature translatable into a distinguishable auditory signature. H2: The golden angle (137.508°) present in classical pictorial composition can act as a structural axis of sonification. H3: Blind and visually impaired people can distinguish and recognize pictorial styles, authors, or series through their auditory signatures. SECONDARY APPLICATIONS While accessibility is the primary mission, the same pipeline supports: - Art education (sonic memory of styles and authors) - Gamified learning: "What painting is this?" — listen and identify - Citizen science: game players validate perceptual robustness of mappings - Museum interfaces: sonic layer added to existing collections - Research in computational aesthetics and auditory perception STATE OF THE ART AND DIFFERENTIATION Related work in painting sonification exists since ~2019 (Virginia Tech, EvoMUSART/Springer, Rönnberg/Samsung, Microsoft Oregon Project, BlindTouch Korea). However, no prior work was identified combining: (a) fractal box-counting dimension as the primary structural metric, (b) golden angle proportion as a compositional axis, (c) open reproducible dataset with generated audio, (d) primary focus on accessibility for blind/VI people. This combination defines the specific contribution of this project. CONNECTION WITH ARGIRA IRIS PROJECT This pipeline is adapted from fractal methods developed in the Argira iris project (DOI: 10.5281/zenodo.18818856). Both projects use: - Box-counting fractal analysis - Golden angle as structural hypothesis - Open dataset published in Zenodo - Python code under MIT license If the golden angle hypothesis holds in both biological tissue and human artistic creation, it points toward something deeper about natural organization and aesthetic Commercial use requires explicit written permission from the author. Contact: josecorreo1@outlook.es

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
2 March 2026