====== DeepDreamEffect ======

| **Authors **| Christian Dittmar |
| **Affiliation** | International Audio Laboratories Erlangen |
| **eMail** | [[christian.dittmar@audiolabs-erlangen.de]] |
| **code** | [[https://github.com/stefan-balke/hamr2015-deepdreameffect]] |

===== What did I do =====

I used Google's DeepDream processing as an audio effect. Therefore, I export music magnitude spectrogram as RGB channels of PNG images and apply so-called 'Gradient Ascent' with pre-trained networks to these images. Afterwards, I convert the resulting images to spectrograms again and resynthesize them using Griffin and Lim's method.

{{ :overview.png?nolink&800 |}}

Since the networks were trained on natural images, this makes no sense musically. However, it gives interesting results:

===== Example 1: Piano =====

Input signal {{ :shenua.wav |}}
Result using layer conv3 (MIT places network){{ :output_shenhua_layer3.wav |}}
Result using layer pool5 (MIT places network){{ :output_shenhua_layer5.wav |}}

===== Example 2: Ethno =====

Input signal {{ :olcay.wav |}}
Result using layer conv3 (MIT places network) {{ :output_olcay_layer3.wav |}}

===== Example 3: Breakbeat =====

Input signal (Different drums encoded as RGB) {{ :amenbrotherbreaknorm_mix.wav |}}
Result using layer conv3 (MIT places network) {{ :output_amen_layer3.wav |}}

===== Libraries Used =====

    Anaconda Python Package
    Caffe Deep Deep Learning Framework
    Pre-Trained Networks
    iPython Notebook
    MATLAB