Dan Ellis : Resources : Matlab :

Sinewave Speech Analysis/Synthesis in Matlab


Sinewave Speech is a curious phenomenon where a small number of sinusoids added together take on some of the characteristics of speech - which in most respects they do not resemble at all.  Using three sinusoids that track the frequency and amplitude of the first three speech formants, high intelligibility can be achieved.  This phenomenon has been extensively investigated by Robert Remez, Philip Rubin and others.  There is a much more detailed description at the web site of Haskin's Lab in New Haven CT, where much of the work was done.

The Haskins site includes several example analysis files that you can download.  These files contain, in a compact form, all the data you need to resynthesize the sinewave speech.  The Matlab routines below do this for you.

Sinewave analysis

I was developing some examples of LPC analysis for my speech and audio class, and to my surprise, crude translation of LPC pole positions does a pretty good job of extracting sinewave speech parameters. Thus, I am pleased to offer the following routines:

An example use is shown below:

>> [d,r] = wavread('mpgr1_sx419.wav');
>> [F,M] = swsmodel(d,r);
>> plot(F'); % show all the frequencies
>> dr = synthtrax(F,M,r);
>> % Listen to it
>> sound(dr,r)
>> % Compare to noise-excited reconstruction of LPC analysis
>> [a,g] = lpcfit(d);
>> dl = lpcsynth(a,g);
>> sound(dl,r);
>> % The LPC reconstruction is based on more or less the same information 
>> % as the sinewave replica, but it sounds less 'weird'
>> % Compare the spectrograms
>> subplot(311)
>> specgram(d,256,r);
>> title('Original');
>> subplot(312)
>> specgram(dr,256,r);
>> title('Sine wave replica');
>> subplot(313)
>> specgram(dl,256,r);
>> title('Noise-excited LPC reconstruction');
[image of spectrograms]


If you wish to reference this code in your publications, you can use the following citation:

    D. P. W. Ellis (2004) 
    "Sinewave Speech Analysis/Synthesis in Matlab", 
    Web resource, available: http://www.ee.columbia.edu/ln/labrosa/matlab/sws/ .

Last updated: $Date: 2016/04/17 23:33:41 $

Dan Ellis <dpwe@ee.columbia.edu>