Image-Driven Granular Synthesizer

Granular synthesizer plug-in window with an imported image driving the grains

Image-Driven Granular Synthesizer

This image-driven granular synthesis algorithm allows users to manipulate source audio using imported image files. The project contributes to previous studies and designs of sonification by highlighting the effectiveness of utilizing edge detection and luminance in creating intuitive sonic changes. The audio-visual correspondences employed in this project could be expanded to many other syntheses and sample manipulation techniques. Translating visual information into sonic changes could yield many creative applications in music production, as it offers musicians a simultaneously optical and auditory production experience. This approach to audio manipulation also increases the unpredictability of the sound output, which could be appealing to experimental musicians seeking to control sounds with the visual structure of artworks that they enjoy, as opposed to precise parameters. We look forward to seeing creative implementations of our techniques in audio-visual artworks, music production tools, and interactive multimedia systems.

  • Advisor: Roger B. Dannenberg, Carnegie Mellon University
  • Music Technology Award, International Electronic Music Competition 2022

JUCE plug-in, 2022

Flow chart from image and source sound file through edge detection and luminance to granular synthesis
Algorithm logic
Three code listings: a C++ DSP library, a C wrapper and an Xlisp function
Code execution logic