Topic outline

  • The course Deep Learning with Audio will introduce the state of the art in deep learning models applied to sound and music, with hands-on exercises on recent artificial intelligence (AI) implementations such as DDSP, AI-Duet, GANSynth, NSynth, GANSpaceSynth and SampleRNN. We will provide code templates that integrate the functionality from these open source deep learning audio projects into Pure Data programming environment. Students will be able to run, modify, access, control, input, output these deep learning models through Pure Data examples. Students will gain an understanding of the differences in input, computational cost and sonic characteristics between the different models, which will help formulate a course project. 

    We will provide detailed setup instructions and automated scripts to make installation of the required tools as easy as possible (for Pure Data, Python, Conda, Magenta, PyExt). Students will also learn and practice preparing data sets and training deep learning models. During the 3 weeks course period, students will have 24/7 access to the Azure cloud computing VMs in Aalto University and will be able to train their deep leaning models for the course exercises as well as for their projects. The current module of installations and course exercises regarding the content generation only work in macOS and Linux, unfortunately Windows is not supported at the moment. You can contact https://takeout.aalto.fi and reserve a laptop computer for this course. 

    Students will further explore a particular model and incorporate it into their own project work. Deep Learning with Audio is a project-based course, we dedicate half of the contact hours for project work, the lecturer and the teaching assistants will support students by giving sufficient guidance, feedback and tutoring. At the end of the course, students will submit and present their projects.