Nov 11, 2013 - Marius Watz | Workshop at ITP NYU
GitHub: https://github.com/mariuswatz/ITP2013Parametric
Code examples using Processing and Minim, showing useful data techniques for dealing with sound as data input. Highlights:
- Opening line in or sound file inputs
- Accessing sound buffer
- Normalizing volume
- Using damping to control FFT behavior
Seen from a data perspective, digital sound is just another input, a time series of air pressure measurements representing acoustic information. But as an element of human perception sound is a rich carrier of emotional and spatial experience, a layered temporal signal with the potential to convey complex patterns that can deeply affect the receiving audience.
The notion of combining sound and image into a single integrated experience is not new. But such practices have been made radically easier by the introduction of digital media interfaces, which are capable of processing and synthesizing sounds and images as a unified signal.
In this workshop we will look at code-based strategies for using sound (both live and static) as an input for other forms of creative expression. Possible scenarios include the creation of audio-visual performance software and sound-responsive visuals, as well as the mapping of sound data into geometric forms for 3D printing.
We will cover the basics of how to access sound streams as a data input, as well as ways to process and analyze that data. To create a harmonious mapping between sound and image we must first extract useful parameters from the input signal, allowing us to map rhythm and tonal qualities to visual behavior. By applying simple data mangling techniques we can then further manipulate and modulate those parameters, providing not only improved control but also the opportunity for live improvisation.