A Paradigm Inverted: From Sonic Capture to Instrumental Rematerialization in Mixed Music

This research introduces a method of mixed music composition that translates environmental sounds and electroacoustic material into real-time musical notation, enabling immediate performance on a Yamaha Disklavier. By algorithmically extracting features such as pitch contours, rhythmic gestures, and dynamic envelopes, non-instrumental sonic sources—particularly environmental soundscapes—are recontextualized as instrumental music through score-based mediation. The generated notation is transmitted directly as MIDI data to the piano, creating a feedback loop in which contingent sound events become instrumental action. The Disklavier is further hybridized with exciters attached to its soundboard, allowing sonic components resistant to transcription to be reproduced through the piano’s resonance. This dual configuration produces a layered sound world where notated and unnotated, instrumental and electronic, symbolic and algorithmic elements coexist and interact. The piano thus functions simultaneously as a score-performing instrument and as a resonant body for electroacoustic playback. By inverting the conventional paradigm of mixed music—where composer-produced electronic sound is projected through loudspeakers alongside acoustic instruments—this work instead channels field-recorded sound into the instrument itself, allowing it to generate notation and to resonate from within its physical body. The approach expands compositional strategies for engaging with sound environments, opens possibilities for hybrid instrument design and algorithmic practice, and reframes what it means to play, write, and listen within a system where machine, environment, and score converge into a unified compositional entity.