User Tools
**This is an old revision of the document!** ----
====== LSTM Library Benchmarks ====== | Authors | Colin Raffel | | Affiliation | Columbia University | | Code | [[https://github.com/craffel/rnntools|rnntools github]] [[https://github.com/craffel/lstm_benchmarks|experiment github]] | I wrote a new Long-Short Term Memory implementation in Theano and benchmarked it against two existing libraries. ===== Experiment ===== I compared the following three libraries; * [[http://sourceforge.net/projects/currennt/|CURRENNT]], a CUDA-based library * [[http://sourceforge.net/projects/rnnl/|RNNLIB]], a single CPU, pure C++ library * [[https://github.com/craffel/rnntools|rnntools]], a Theano-based Python module which is an extension of [[https://github.com/benanne/nntools|nntools]] that I wrote during the hackathon I ran the CHiME noisy speech recognition experiment (included with CURRENNT). The parameterization used was the same as the default parameters supplied with CURRENNT. However, RNNLIB doesn't seem to be able to use normally-distributed weight initialization, so I also ran CURRENNT with a uniform weight distribution. However, CURRENNT can't randomly initialize biases, so the training dynamics are somewhat different. ===== Results ===== The average time per epoch follows: * CURRENNT: 32 seconds * RNNLIB: 740 seconds * rnntools: 813.75 seconds CURRENNT is the clear winner, probably because it processes many sequences in parallel. For more details, see the paper which describes its implementation (([[http://www.mmk.ei.tum.de/publ/pdf/14/14wen7.pdf]])). As mentioned above, the training dynamics were different for each system because the intialization was not the same. The validation error at each epoch is plotted below for CURRENNT (both uniform and normal initializations) and RNNLIB is shown below. rnntools isn't included because it didn't finish training in time. {{::errors.png|}}
