Översikt

  • Allmänt

    • Contents: Machine learning with deep neural networks. Programming using PyTorch.

    After the course, the student understands the basic principles of deep learning: fully-connected, convolutional and recurrent neural networks; stochastic gradient descent and backpropagation; means to prevent overfitting. The student understands how to do supervised learning (classification and regression) and unsupervised learning with neural networks. The student knows modern neural architectures used for image classification, time series modeling and natural language processing. The student has experience on training deep learning models in PyTorch.

    • Assessment: Exercises, a project work and an exam.

    • Prerequisites: Basics of machine learning, basics of probability and statistics, good level of programming in Python. Recommended: matrix algebra.
    Basic terms of machine learning:
      • supervised and unsupervised learning
      • overfitting and underfitting
      • regularization
    Basic terms of probability theory:
      • sum, product rule, Bayes' rule
      • expectation, mean, variance, median
    • Course contents:
      • Introduction and history of deep learning
      • Optimization for training deep models
      • Regularization for deep learning
      • Convolutional networks
      • Recurrent neural networks
      • Unsupervised learning with deep autoencoders and generative adversarial networks


  • Material

    The recommended book is Goodfellow, Bengio and Courville, 2016. Deep Learning. http://www.deeplearningbook.org/

  • Uppgifter

    Assignments:

    1. Introduction to PyTorch, training a simple fully-connected network
    2. Backpropagation
    3. Regularization
    4. Convolutional networks
    5. Recurrent neural networks
    6. Deep autoencoders and generative adversarial networks
    Instructions
    • Login to jupyter.cs.aalto.fi with your aalto credentials.
    • Click on tab “Assignment”, fetch an assignment.
    • NB: If you need to re-fetch the assignment again, first delete the fetched assignment and then fetch again.
    • The assignment files are saved to /notebooks/deeplearn2019/assignment_name
    • The first three assignments are computationally light, so you can try to solve them on jupyterhub. Assignments 4, 5, 6 are computationally heavier, it is recommended to train the models on another computer. In that case, you need to download the assignment and data files to your computer and keep the same folder structure as in /notebooks/deeplearn2019/.
    • In order to copy the files between your computer and jupyterhub, you may access your folder on jupyterhub as a network drive. More information can be found here.
    • Fetch the assignment called "data". Download the data to your computer.
    • In your computer, install the required packages (for example, you will need PyTorch 1.0.0). You can create a virtual environment using the environment file posted below.
    • Solve the assignment on your computer.
      • If you want to use GPU, google colab is worth checking out. Paniikki also has computers with GPUs. NB: Notebooks that you fetch from jupyterhub contain meta-data that are used for grading. If you create a copy of the original notebook (for example, for running in google colab), it is safer to insert your code back to the original notebook and submit that.
      • Set course_data_dir to the right location on your computer (check a cell at the beginning of the notebook).
      • When training a model, set skip_training=False in one of the cells at the beginning of a notebook. Set skip_training=True during validation (either on your computer or on jupyterhub). You can toggle that variable on and off depending on your needs.
    • After you've inserted all the code, run all of the cells with skip_training=True. Make sure that all the cells work correctly (if you have a trained model on disk, it should be loaded by one of the cells).
    • Upload the notebook and the files created during training to the same folder from where you copied the data on jupyterhub (replace the original files). Do not forget trained models (.pth files) or other files (one assignment needs *.npz file).
    • Run all the cells again with skip_training=True on jupyterhub. This way you make sure that you did not forget to copy some of the files needed during evaluation. This is the version that will be manually evaluated if the autograding system fails.
    • NB: Validation is equivalent to running all the cells of your notebook. You will not know how many points you get by validating your solution.
    • There have been issues with timeout during validation. Now the default timeout is set to 120 seconds which should be enough. If this problem occurs during validation, simply run all your cells and make sure that you do not get any errors.
    • Go to Assignments tab and click “submit” for a solved assignment. You can submit multiple times. Only the last submission will be evaluated.
    • The autograding system is nbgrader, if you want to read the docs, please visit https://nbgrader.readthedocs.io/en/stable/.

  • Project work

  • För Aalto

    Tillgänglig om: Fältet User account contains (use: aalto.fi) innehåller aalto.fi