===============================================================================
MULTIPLE ALGORITHM SUMMARY REPORT README
===============================================================================
(C)2009 Phillip Snyder, Columbia University, LabROSA
All files written by Phillip Snyder except...
	report.css, bg_timeline2.gif - Peiszer
	SpryTabbedPanels.css, SpryTabbedPanels.js - DreamWeaver template
	unique_no_sort.m - http://www.mathworks.com/matlabcentral/fileexchange/15209
	xml parser package


-------------------------------------------------------------------------------
CONTENTS
-------------------------------------------------------------------------------
I.  Preparation
	1. Files and Structure
	2. Text Files
	3. Matlab Prep

II. Instructions
	1. formal_distance.m
	2. segsumm.m

-------------------------------------------------------------------------------
I.  PREPARATION
-------------------------------------------------------------------------------

1. Files and Structure
	Summary.zip includes all necessary files to run the multiple algorithm
evaluation system. However, you must provide the SegmXML data. The file
structure is shown below followed by a list of all the files with a short
description. The name of the 'summary' folder is not important, but the
relationship between its contents is.

File Structure
	summary
	  |-- html_files
	  |	 \ alg_template.html
	  |	 \ bg_timeline2.gif
	  |	 \ report.css
	  |	 \ song_template.html
	  |	 \ SpryTabbedPanels.css
	  |	 \ SpryTabbedPanels.js	
	  |
	  |-- html_results
	  |	|--20090730T162551
	  |		\ EP Published--20090730T152932.html
	  |		\ My Results--20090730T152932.html
	  |		\ songs--20090730T152932.html
	  |
	  |-- matlab_scripts
	  |	|-- xml
	  |	|	\ xmldata.c
	  |	|	\ xmldata.m
	  |	|	\ xmlstruct.m
	  |	 \ color_choose.m
	  |	 \ formal_distance.m
	  |	 \ getBounds.m
	  |	 \ main_summary.m
	  |	 \ make_alg.m
	  |	 \ make_song.m
	  |	 \ represent.m
	  |	 \ sec2ms.m
	  |	 \ segeval.m
	  |	 \ segsumm.m
	  |	 \ unique_no_sort.m
	  |
	   \ algorithms.txt
	   \ MainSummary--20090730T152932.html
	   \ readme.txt
	   \ songs.txt


List of Files
	- alg_template.html		Template for algorithm summary page
	- algorithms.txt		Input text file read by segsumm.m
	- bg_timeline2.gif		Peiszer's image for timeline
	- color_choose.m		Get hex color code for numbers 0 to 1
	- formal_distance.m		Calculate formal distance for 2 strings
	- getBounds.m			Get segment boundaries from SegmXML
	- main_summary.m		Writes the main summary page
	- make_alg.m			Writes the algorithm summary pages
	- make_song.m			Writes the song summary page
	- readme.txt			This file
	- report.css			Peiszer's file used to make timelines
	- represent.m			Finds canonical representation
	- sec2ms.m			Format seconds to min, sec, ms.
	- segeval.m			Scoring function
	- segsumm.m			Main function, scores & creates report
	- song_template.html		Template for song summary
	- songs.txt			Input text file read by segsumm.m
	- SpryTabbedPanels.css		Format file for tabbed panels
	- SpryTabbedPanels.js		Javascript that creates tabbed panels
	- unique_no_sort.m		Same as unique.m but without sorting
	- xmldata.c, xmldata.m,		XML package used to read SegmXML
	  xmlstruct.m

	- All files with '20090731T152932' are my example output. This is an
	  evaluation id, these HTML pages are the output and a new set will
	  be created each time segsumm.m is run.


2. Text Files
	There are two text files that must be modified before the segsumm.m
file can be executed, songs.txt and algorithms.txt. These files contain
information about the songs and algorithms that are to be summarized.
	Algorithms.txt contains the names of each algorithm and the absolute
directories where the respective SegmXML files are located. The first line must
be the ground truth information. Segsumm.m will extract both the main levels
and sub levels from the top line only, therefore this line must be the ground
truth. The format of algorithms.txt is simple. Each line contains the name of
an algorithm and the directory of its SegmXML files separated by a tab, ie:

groundtruth	'directory of ground truth SegmXML'
algorithm1	'directory of algorithm 1 SegmXML'
algorithm2 .... and so on ....

	Songs.txt contains the names of the songs that are going to be included
in the evaluation. The only requirements are that every line hold only one song
and that all the songs listed have a corresponding SegmXML file for each
algorithm specified in algorithms.txt. File extensions in songs.txt are not
important. Segsumm.m will strip the current file extension and append '.xml'
when it needs to find that tracks SegmXML file.
	It does not matter how many algorithms or songs are listed in these
files. Just keep in mind that the more information included in the input means
that segsumm.m will take longer to run and the output files will be larger.

3. Matlab Prep
	There are three quick, simple steps that must be performed in MATLAB
before running segsumm.m.
	1. Add the path for the MATLAB scripts folder in summary.zip.
			addpath('??/summary/matlab_scripts');
	2. Add the path for the xml scripts.
			addpath('??/summary/matlab_scripts/xml');
	3. Mex the xmldata.c file. May require current directory to be xml folder
			cd('??/summary/matlab_scripts/xml');
			mex xmldata.c

	Note in the above paths that ?? denote where the summary folder is in your
file system. After completing the preparation you are ready to run segsumm.m.


-------------------------------------------------------------------------------
I.  INSTRUCTIONS
-------------------------------------------------------------------------------

1. formal_distance.m
	This function is used within segsumm.m. The only reason I write about
it individually is because the formal distance concept can be confusing and it
can be useful to use this function by itself for help understanding it.
	Formal distance does not care if A maps to A like in edit distance;
it cares that all A's map to all B's or all C's, as long as A maps to the same
letter. The algorithm for formal distance requires all permutations of letter
mappings of a string to be compared to the other string using edit distance.
The smallest edit distance is the formal distance.
	The provided function requires two arguments but takes up to four. The
first two arguments are the strings in which formal distance is calculated.
This function is only intended to be used on strings of the same length. The
optional arguments are weights and permutations. Weights is a list that weighs
each position in the strings, it must be the same length as the strings. If no
weight is given, all positions are weighted equally with value 1. The fourth
argument, max permutations, puts a cap on the number of permutations that are
examined during mapping. Strings with a large number of different letters can
take a long time to evaluate, upwards of 20 minutes. Setting the maximum
to 5050 allows the algorithm to find a value that is very close to what would
be found if all permutations where carried out. Also, 5050 keeps running time
relatively short.

2. segsumm.m
	This is the main function that will carry out everything required to
generate the summary report. This function takes four arguments: WIN, PROX,
TOL, and MAX_PERMS. Also, segsumm.m reads from the two text files songs.txt
and algorithms.txt. The information is loaded, the algorithms are scored, and
the HTML report is created. Each run of segsumm.m is given an evaluation id
that is specific to the current time, the HTML output uses this id. Segsumm.m
returns a cell array that contains all the information used in scoring the
algorithms.

! IMPORTANT !
	Since segsumm.m reads from the two text files and due to the structure
of the HTML output, the function segsumm.m must be executed when the current
directory is 'summary'.









