This page provides a more detailed discussion of the components
and function of MEAPsoft. For a quick illustration of how to get
going, see the
MEAPsoft Quick Start Walkthrough.
Feature Extractors
Feature extractors analyze the segments in a .seg file and
output a features file containing one or more values representing the
features found. Some feature extractors, like ChunkPower,
simply put out one number representing the total power in each
segment. Others, like AvgChroma, put out an array of values
for each segment. For a short description of each feature extractor,
hover above the feature extractor's name and a tooltip will pop up.
You can select as many feature extractors as you like, although best
results are usually obtained by selecting features extractors that
work well with the composers you'll feed the .feat file to.
The number box beside each feature extractor is for entering
weights that allow you to specify the relative importance of each
feature in the analysis.
"Meta feature extractors" are a special class of FEs that do higher
level analysis on the outputs of previously run "normal" feature
extractors. In order to use a meta feature extractor, you need to
select at least one normal feature extractor in addition to the meta
feature extractor. If "clear non-meta features" is selected, the
output features file will only contain the selected meta features.
The "Display extracted features" button is enabled after processing a
segments file. It pops up a window with a simple viewer that allows
you to inspect the values of the extracted features. This is useful if
you need to set value ranges in a Composer.
Available feature extractors:
- AvgChroma - 12-dimensional vector of energy distribution across
each semitone of the octave.
- AvgChromaScalar - Single value giving dominant semitone within the
octave.
- AvgChunkPower - Computes the average power in each chunk.
- AvgFreqSimple - Provides a frequency estimation for each segment of
sound.
- AvgMelSpec - Computes the mean spectrum of a chunk and converts it to
the perceptually weighted Mel frequency scale.
- AvgMFCC - Computes the mean MFCCs of a chunk, a commonly used feature
in speech recognition.
- AvgPitchSimple - Provides a pitch estimation for each segment of
sound.
- AvgSpecCentroid - Computes the average spectral center of mass of a
chunk's frames.
- AvgSpecFlatness - Provides a measure of the peakiness of the chunks
average spectrum.
- AvgSpec - Computes the mean spectrum or each chunk.
- ChunkLength - Simply returns the length of each segment.
- ChunkStartTime - Simply returns the start time of each segment. Good
for making backwards tracks!
- Likelihood - MetaFeatureExtractor. Returns the
likelihood of each chunk based on its current features. Lower
numbers mean a segment is more common, higher numbers mean it is
more distinct.
- SpectralStability - Tracks the stability of the spectral energy within
each chunk of sound. More spectrally stable chunks are more likely to
be pitched material.
Composers
A composer takes a feature file as an input, analyzes/sorts/modifys
the segments in that file, and then creates an Edit Decision List
(EDL) representing the order in which the segments from the original
source sound file (as well as others) should be arranged by the
Synthesizer. Composers can be very simple or very complex. For
example, "simple sort" simply sorts a features file by the first
feature in each chunk. You could use this with AvgPitchSimple
to generate a glissando where all of the pitches in the input sound
are arranged from low to high. More complex Composers, like "MashUp"
and "head bang" perform more sophisticated operations.
Each composer is described by a short text that appears when you
select it. The controls for that composer (if any) will appear below
the text.
Controls: Each composer's controls (if any) are different,
and should be described in that composer's explanatory text. The
following Universal Chunk Operations apply to all composers.
- reverse: reverse audio in each chunk.
- apply fade in/out: to avoid pops between segments you may
want to apply a quick fade in/out at the boundaries of each
chunk.
- cross fade: slightly shift each segment so that their
fade in/outs overlap. This generally results in smoother, less clicky
sound.
- fade length (ms): the duration of the fade in/out, in
milliseconds.
- apply gain value: adds a gain value to each
segment in the output EDL. The synthesizer will use the gain value to
scale the amplitude of the samples it writes to the output
soundfile.
- gain value: the gain value to add
to each segment.
The "Display composed features" button is like the "Display extracted
features" button above, but it works on the newly composed EDL
file. This is convenient for inspecting the results of the
composer. For instance, if you run the "simple sort" composer and
display the EDL file you will see that the jumpy colors in the .feat
file have been turned into a smooth fade in the sorted EDL.
Available composers:
- BlipComposer - BlipComposer inserts a blip at the beginning of each
chunk in the input features file. Especially useful for
understanding the output of the segmenter.
- EDLComposer - EDLComposer applies composer options (gain,
crossfade, etc.) to an existing .edl file. It is meant to be used to
generate output from the visualizers.
- HeadBangComposer - HeadBangComposer rocks it hard-core style. Finds
the most common chunk length L and lengths related by a factor of
2, i.e. L/2, L/4, L/8, L*2. These chunks are then shuffled to
create a new piece with a clear beat.
- HMMComposer - HMMComposer uses a features file to train
a simple statistical model of a song and uses it to randomly
generate a new sequence of chunks. This works best when used with
chunks created by the beat detector.
- IntraChunkShuffleComposer - IntraChunkShuffleComposer chops each
chunk up into small pieces and rearranges them. This keeps
the meta chunks intact but scrambles them on a local level.
- MashupComposer - MashupComposer attempts to match chunks in the
input features file using chunks from the chunk database
features file. The result is the source sound file created
from chunks in the chunk database.
- MeapaeMComposer - MeapaeMComposer makes palindromes by writing each
chunk of audio forward and then backward.
- NNComposer - NNComposer starts at the first chunk and proceeds
through the sound file from each chunk to its nearest neighbor,
according to the features in the input features file.
- RotComposer - RotComposer rotates the beats in each measure by a
selectable number of positions. You can set the number of
beats/measure, the number of positions to rotate, and the direction
of rotation.
- SortComposer - SortComposer sorts the features in ascending or
descending order. If there are multiple features, or more than one
value per feature, it sorts according to distance in Euclidean
space.
- ThresholdComposer - ThresholdComposer selects chunks with feature
values falling inside the top and bottom thresholds. It then
creates an output file composed exclusively of either the selected
chunks or the not-selected chunks. ThresholdComposer only really
makes sense for one-dimensional features like pitch and power.
- VQComposer - VQComposer trains a vector quantizer on the
chunks in the input file. It then uses it to quantize the chunks
in another file. For best results use the beat segmenter so each
chunk has roughly the same length.
