Deep learning course (Practical part)

Welcome to the main page for the practical part of the course INF265: Deep learning! This course is run by the University of Bergen (Norway) and I (Natacha Galmiche) was responsible for the practical part of the course in spring 2022 and 2023. The course is a part of the machine learning specialisation of the master program in informatics offered by the department of Informatics of the University of Bergen. This course assumes that the students have already taken an introductory course in machine learning, such as INF264: Introduction to machine learning. To accommodate to all students, this course (including the practical part) is entirely feasible on a CPU!

You can find in this repository all the resources used for the practical part of the course. In addition to the resources you find here, the theoretical part of the course was based on the online book "Dive into deep learning" and registered participants had access to Q&A sessions with the course instructor.

Course content (Practical part)

The course includes three mandatory and graded projects where students practice implementation and application of machine learning algorithms into real-world data. Project assignments are done in pairs and are available three weeks before the deadline.

In addition, there are optional weekly exercises with solution to help student practice what they have learned from the book and to prepare them for the mandatory projects.

In this repository, you can find all the exercises and projects as well as suggestion of solution for each of them.

1. Module 0: PyTorch tutorials
   1. PyTorch datasets and transforms
   2. Define, train and evaluate a basic Neural Network in Pytorch
   3. Define a custom deep Neural Network in Pytorch
   4. Computational graph
   5. Input and output shapes of convolutional neural network layers
2. Module 1: Deep Learning Basics
   1. Pre-trained neural networks
   2. Machine learning pipeline and MLP
   3. Analysing data and results
   4. PROJECT 1: Backpropagation and gradient descent
3. Module 2: Image data
   1. Convolutional layer (+ BONUS: CNN: visualization and interpretation)
   2. Residual networks
   3. PROJECT 2: Object localization and detection
4. Module 3: Sequence data
   1. Text data
   2. LSTM
   3. PROJECT 3: Sequence models (RNN and Attention layer)
5. Module 4: Generative models
   1. AE, VAE, GAN

Learning Outcomes (both practical and theoretical parts)

By the end of this course, students should have the following competences:

• Knowledge, the student should be able to

  • Describe the different key aspects of the back-propagation algorithm and and can implement back-propagation.
  • Categorize a given problem into one of the main machine learning tasks (classification, regression, clustering, generation, dimensionality reduction)

• Skills, the student should be able to

  • Build a robust deep learning pipeline, without data leakage and with a correct and well adapted model evaluation and selection.
  • Apply and evaluate deep learning architectures on real data sets, including text and image datasets.
  • Identify the types of architectures, optimisation techniques that are suitable for a given problem. In addition the student is able to justify this choice by referring to the inherent properties of the problem at hand and selected tools.
  • Interpret results and identify weak and strong points of a given deep learning model.

• General competence, the student should be able to

  • Keep a critical eye on the potential implications and issues of the use of deep learning in society.
  • Communicate their results and make sure those are reproducible.