Dan Ellis : Resources :

Examples of Audio Processing in "Processing"

Processing is a very neat programming environment developed to allow users to quickly try out and develop small pieces of code. It is based on Java, and has found a strong following for art-style applications. We have been using it as part of the Music Signal Processing class as a quick way to build audio processing examples that you can interact with in real time.

This page is a place where I can put some of the little ``sketches'' I have developed in Processing. They are based on the minim sound library included in Processing. You should also see the set of Processing examples using minim that are on the main Processing site.

AudioSocketExample defines a class (including a FIFO audio buffer) to connect line in to line out, for a live audio processing loop within Processing.
AudioSocket_BPF builds upon the AudioSocket loop by adding a bandpass filter that you can control in real time with the mouse (based on the minim example bandpassfilter). It also plots the resulting spectrum in real-time.

Playback_BPF loops an MP3 file, applies a bandpass filter under interactive mouse control, and plots the spectrum of what's being played (showing the effect of the filter). Note: you have to put a soundfile named "sound.mp3" in the sketch's directory for it to find and play. PlaySpectrum_filt does much the same thing, but is based on the improved version of LiveSpectrum with axis labels, and implements its own BPF rather than using minim's built-in one. (This code was written to illustrate how to implement arbitrary filters in minim). You can check it out live as the PlaySpectrum_filt_applet.

Oscilloscope just plots a short excerpt of the input waveform. Keyboard controls '+', '-' adjust height of trace, '<', '>' adjust time base. Attempts to 'trigger' to keep a stable waveform..

LiveSpectrum takes the FFT of the input audio and plots it, in dB, on the screen. (A slight modification to the minim forward FFT example).

LiveSpectrogram plots the live spectrum and several seconds of its history as a scrolling grayscale image (a spectrogram). The modified version LiveSpectrogram_window allows you to vary the length of the time window, for interactive control of time/frequency resolution.

SinePiano implements a crude piano using the bottom two rows of keys on the computer keyboard. Here, each pitch is rendered with a sine oscillator whose envelope decays smoothly.
KSPiano is another typing-piano, but this time the note waveforms are generated in the sketch itself using the Karplus-Strong plucked string algorithm.

plucked_string simulates a plucked string by calculating two traveling waves moving in opposite directions. You can choose the pluck point and initial displacement interactively with the mouse. plucked_string_damp adds an interactive damping feature: shift-click while the string is sounding to simulate "touching" the string at a particular point to sound the harmonics. Shift-drag the mouse closer to the bottom of the screen to damp more.

Acknowledgment

This material is based in part upon work supported by the National Science Foundation under Grant No. IIS-0713334. Any opinions, findings and conclusions or recomendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).

Last updated: $Date: 2010/01/23 01:30:23 $

Dan Ellis <dpwe@ee.columbia.edu>