HAMR@ISMIR 2015 Proceedings

====== Differences ====== This shows you the differences between two versions of the page.

--- deepdreameffect [2015/10/25 11:01]
dmr
+++ deepdreameffect [2015/11/05 04:54]
dmr
@@ Line 4: / Line 4: @@
 | **Affiliation** | International Audio Laboratories Erlangen |
 | **eMail** | [[christian.dittmar@audiolabs-erlangen.de]] |
+| **code** | [[http://nbviewer.ipython.org/github/stefan-balke/deepdreameffect/blob/master/dream.ipynb]] |
-I propose to use Google's DeepDream processing as an audio effect. Therefore, I export music magnitude spectrogram as RGB channels of PNG images and apply to 'Gradient Ascent' with pre-trained networks to these images. Since the networks were trained on natural images, this makes no sense musically. However, it gives interesting results:
+===== What did I do =====
+I used Google's DeepDream processing as an audio effect. Therefore, I export music magnitude spectrogram as RGB channels of PNG images and apply so-called 'Gradient Ascent' with pre-trained networks to these images. Afterwards, I convert the resulting images to spectrograms again and resynthesize them using Griffin and Lim's method.
+{{ :overview.png?nolink&800 |}}
+Since the networks were trained on natural images, this makes no sense musically. However, it gives interesting results:
+===== Example 1: Piano =====
+Input signal {{ :shenua.wav |}}
+Result using layer conv3 (MIT places network){{ :output_shenhua_layer3.wav |}}
+Result using layer pool5 (MIT places network){{ :output_shenhua_layer5.wav |}}
+===== Example 2: Ethno =====
+Input signal {{ :olcay.wav |}}
+Result using layer conv3 (MIT places network) {{ :output_olcay_layer3.wav |}}
+===== Example 3: Breakbeat =====
+Input signal (Different drums encoded as RGB) {{ :amenbrotherbreaknorm_mix.wav |}}
+Result using layer conv3 (MIT places network) {{ :output_amen_layer3.wav |}}
+===== Libraries Used =====
+    Anaconda Python Package
+    Caffe Deep Deep Learning Framework
+    Pre-Trained Networks
+    iPython Notebook
+    MATLAB

HAMR@ISMIR 2015 Proceedings

User Tools

Site Tools

Page Tools