Lars Ruthotto

CS 584 / MATH 789R - Numerical Methods for Deep Learning

Instructor:
Prof. Lars Ruthotto
Syllabus:
The syllabus can be downloaded here.
Material:
Homework and additional course material will be made available using Canvas.
Times:
First day of classes: January 16, 2018
Last day of classes: April 30, 2018
Recess: tba
Description:

This course provides students with the mathematical background needed to analyze and further develop numerical methods at the heart of deep learning. The course is simultaneously geared towards computer science students, who may be interested/experienced with deep learning and would like to strengthen their theoretical foundation, and mathematics students, who may have a background in numerical analysis and who are curious about the emerging technology of deep learning. The course primarily targets graduate students. However, exceptional advanced undergraduate students may be admitted on a case-by-case basis.

The ultimate goal is for students to implement a simple, mathematically sound deep learning code from scratch. This look under the hood will provide valuable insight into successful and well-tested software packages for deep learning tasks and understand their success (and failure). Students may use MATLAB, Julia, or Python.

Class time will be split between hands-on coding exercises, lectures providing the necessary theoretical background, and discussions about selected articles published in this emerging field. Classes will provide an overview of state-of-the-art methods for deep learning with a focus on deep residual networks, and also contain crash courses on some of the most important aspects of numerical analysis that are involved in deep learning, such as convex and nonlinear optimization, differential equations, and optimal control.

Prerequisites:
In order to succeed in this class, students need to have a solid background in multivariate calculus and linear algebra and some programming experience in MATLAB, Julia, or Python.
In addition, students are also expected to have experience or skills in either numerical analysis (optimization, partial differential equations) or machine learning (e.g., CS534, CS584, or similar).