We're working with speech recognition on some very noisy speech files that were apparently recorded by a mic some distance from the speaker in a noisy office environment.

One special tool we have at our disposal is a trained, noise-robust pitch tracker derived from the RATS data, which consisted of pairs of files, a clean version, and badly corrupted version that had been transmitted over a variety of noisy radio links. We used these to train our Subband Autoccorrelation Classification (SAcC) pitch tracker which does well in conditions much too noisy for conventional explicit pitch trackers. Informally, it seems that the RATS-trained pitch tracker does well on our noisy Babel data.

This could be used in several places (e.g., for improved voice activity detection), but one idea is to try to enhance speech by filtering out just the harmonics corresponding to the "known" pitch.

I've been thinking about how to do this ever since Avery Wang did it to separate vocals from accompaniment at CCRMA in 1996. Spectrogram masking is a bit too crude and messy; his technique was to "heterodyne" each harmonic down to dc, then to use a low-pass filter to extract the amplitude, then remodulate it back to its original position.

Of course, a comb filter picks out all the harmonics in a single stroke, but it needs everything to be at a constant pitch. Rather than trying to demodulate each harmonic, my idea is to use time-varying resampling ("varispeed") to pitch-shift the target using the known pitch at each time so that the target pitch becomes constant. Then you can filter, and, in theory, un-resample to get back the original signal, now with the gaps between the harmonics "dug out".

The problem was that I'd always gotten tied up trying to make sense of how to undo the time resampling, since it's a confusing operation. But for the ehist project on energy histogram equalization, I ended up implementing piecewise linear mappings that are exactly invertible, and which thus gave me the piece I needed for the pitch-flattening.

The figure below a noisy OP1_204 test fragment (I started with the Wiener-filtered output), along with the pitch track and prob(vx) from SAcC-RATS. I tweaked the prior on voicing to make it guess at a pitch even when the evidence was weak, and I smoothed the prob(vx) with a 200ms median filter.

The next pane is that signal with time-varying resampling to flatten the inferred pitch track to 100 Hz. Then we put it through a comb filter to remove noise in-between the harmonics (third pane).

The final pane shows this cleaned signal un-resampled, to get back the original timebase. I also cross-fade between the pitch-filtered version and the original unprocessed signal using the smoothed p(vx) as the weighting.

This all works, I think. Unfortunately, it doesn't sound that good (yet?). But I think it's the basis of a really nice story - if only we can get some kind of ASR gain out of it.

The Matlab code for this is at https://github.com/dpwe/pitchfilter . The figure below is created within the pitchfilter command, e.g.

>> [d,sr] = wavread('nr1ex_wiener-8k.wav');
>> y = pitchfilter(d,sr);
>> wavwrite(y, sr, 'nr1ex_w+pitchfilt.wav');