Ali Ghodsi

Professor, University of Waterloo

[first name].[last name] [AT] uwaterloo.ca

Teaching

Fall 2026

STAT 940

Deep Learning

A graduate-level course on the foundations, architectures, optimization methods, and modern applications of deep learning.

Course at a glance

Overview

Course STAT 940
Term Fall 2026
Instructor Ali Ghodsi
Class meetings Tuesdays and Thursdays, 1:00-2:20 p.m.
Location MC 4063

This page is the public home for the course topics, slide releases, and assignments. Enrollment-specific information, submissions, deadlines, and grades are maintained in Waterloo Learn.

Course support

Office Hours and Course Support

Teaching assistant office hours

Assignment and coding support

Go to the TAs for: questions about assignments or coding.

Teaching assistants

Pingchu Zhang and Chenxin Lyu

In-person and online office hours rotate weekly. Open the schedule below for dates, times, and locations.

View office-hours schedule
Week starting In-person session: date, time, TA and room Online session: date, time and TA
Oct. 5 Tuesday, Oct. 6
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Oct. 8
9–10 a.m. · Chenxin
Oct. 19 Tuesday, Oct. 20
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Oct. 22
9–10 a.m. · Chenxin
Oct. 26 Monday, Oct. 26
9–10 a.m. · Chenxin · M3 4221
Thursday, Oct. 29
9–10 a.m. · Pingchu
Nov. 2 Tuesday, Nov. 3
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Nov. 5
9–10 a.m. · Chenxin
Nov. 9 Monday, Nov. 9
9–10 a.m. · Chenxin · M3 4221
Thursday, Nov. 12
9–10 a.m. · Pingchu
Nov. 16 Tuesday, Nov. 17
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Nov. 19
9–10 a.m. · Chenxin
Nov. 23 Monday, Nov. 23
9–10 a.m. · Chenxin · M3 4221
Thursday, Nov. 26
9–10 a.m. · Pingchu
Nov. 30 Tuesday, Dec. 1
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Dec. 3
9–10 a.m. · Chenxin
Dec. 7 Monday, Dec. 7
9–10 a.m. · Chenxin · M3 4221
Thursday, Dec. 10
9–10 a.m. · Pingchu
Dec. 14 Tuesday, Dec. 15
10–11 a.m. · Pingchu · Desk #3, M3 2101
Thursday, Dec. 17
9–10 a.m. · Chenxin
Dec. 21 Monday, Dec. 21
9–10 a.m. · Chenxin · M3 4221
Thursday, Dec. 24
9–10 a.m. · Pingchu

Online office hours are accessed through the Zoom widget in LEARN.

From the course outline

Course details

Required textbook

Elements of Deep Learning

Benyamin Ghojogh and Ali Ghodsi. Springer, 2026. ISBN 978-3-032-10738-1 (eBook).

View textbook
Assessment and evaluation
Component Weight
Assignments 45%
Paper presentation 10%
Group final project 45%
Total 100%
Tentative assessment dates
Assessment Release Due
Assignment 1 Monday, October 5 Monday, October 26, 11:59 p.m.
Assignment 2 Monday, October 26 Monday, November 16, 11:59 p.m.
Assignment 3 Monday, November 16 Monday, November 30, 11:59 p.m.
Paper presentation Eligible venues specified in the course outline Scheduled presentation date
Group final project To be announced Monday, December 21, 11:59 p.m.

All dates are tentative and may change. Changes will be announced.

Latest updates

Announcements

Course site

Welcome to STAT 940

This page will be updated throughout the term. Slides and assignments will be posted below as they are released.

Communication

Use Piazza for course discussion

Questions about lectures and course content should be posted on Piazza so that answers are available to the class.

Course schedule

Tentative lecture schedule

This tentative schedule follows the lecture sequence in the course materials; textbook references provide supporting readings.

Lecture content Textbook alignment Materials
Lecture 1 - Motivation, history, machine-learning foundations, and a neural-network overview Ch. 1, Secs. 1-5; Ch. 2, Secs. 2.1-2.7 Lecture 1 slides
Lecture 2 - Neural-network history, McCulloch-Pitts networks, the perceptron and linear separability, feedforward neural networks, and backpropagation Ch. 1, Sec. 5; Ch. 2, Secs. 2.2-2.7; Ch. 3, Secs. 2-3 Lecture 2 slides
Lecture 3 - Stochastic gradient descent, mini-batches, learning-rate tuning, momentum, optimizer selection, and first- versus second-order methods Ch. 3, Secs. 2-5; the modern optimizer survey is supplemental Lecture 3 slides
Lecture 4 - Model selection, empirical and true error, SURE, double descent, implicit regularization, and grokking Ch. 5, Secs. 2-5; Ch. 6, Sec. 5; optional SURE proof: Ch. 5, Sec. 13 Lecture 4 slides
Lecture 5 - Regularization: weight decay, augmentation and noise injection, manifold tangent classifier, early stopping, parameter sharing, label smoothing, bagging, and dropout Ch. 5, Secs. 6-10.3; manifold tangent classifier, parameter tying/sharing, and general label smoothing are supplemental Lecture 5 slides
Lecture 6 - Dropout for linear regression, batch normalization, optimization effects, and normalization alternatives Ch. 5, Secs. 10.3-11; normalization alternatives and modern BatchNorm optimization analysis are supplemental Lecture 6 slides
Lecture 7 - Convolution and cross-correlation, sparse interactions, parameter sharing, equivariance, pooling, CNN training, ResNet, and DenseNet Ch. 4, Secs. 1, 3.1.1-3.1.4, 3.2.1, 3.2.3-3.2.7, 4-4.2, and 5.6-5.7 Lecture 7 slides
Lecture 8 - RNNs and parameter sharing, backpropagation through time, vanishing/exploding gradients, echo-state networks, long delays and leaky units, LSTM, GRU, and gradient clipping Ch. 8, Secs. 1-3.4, 4.1, and 4.3.1 Lecture 8 slides
Lecture 9 - Sequence-to-sequence models, attention for machine translation, self-attention, queries/keys/values, attention scoring and matrix formulation, and image captioning Ch. 9, Secs. 1-2.5 Lecture 9 slides
Lecture 10 - Transformer encoder/decoder, multi-head attention, position-wise feedforward networks, residual connections and layer normalization, masked self-attention, cross-attention, word prediction, and sinusoidal positional encoding Ch. 9, Secs. 3.1-3.3.3; attention review: Secs. 2.4.2-2.4.3 Lecture 10 slides
Lecture 11 - A unified view of MLPs, CNNs, attention, RNNs, and state-space models; linear attention and KAN; BERT pretraining, fine-tuning, and sentence embeddings; GPT, T5, and major language-model families Ch. 9, Secs. 2.4 and 4.2.1; Ch. 10, Sec. 7; Ch. 11, Secs. 2, 3.1-3.2, 3.4, and 4-5. Unified framework, KAN, and newer model updates: lecture slides. Lecture 11 slides
Lecture 12 - LLM alignment and InstructGPT; supervised fine-tuning, reward modeling, reinforcement-learning foundations and policy gradients, RLHF with KL regularization, and direct preference optimization (DPO) Ch. 11, Secs. 7-9; reinforcement-learning foundations and policy gradients: Ch. 18, Secs. 2 and 9 Lecture 12 slides
Lecture 13 - Performer attention, random-feature kernel approximations and FAVOR+, normalization and linear-time computation; variational inference, KL divergence and ELBO derivation, and a brief VAE introduction Ch. 9, Sec. 4.2; Ch. 12, Secs. 1-2.1 Lecture 13 slides
Lecture 14 - GAN training and objective functions, optimal discriminator and Jensen-Shannon divergence, mode collapse and minibatch discrimination, conditional GANs, adversarial autoencoders, and image-generation applications Ch. 14, Secs. 2.1-2.4, 2.6-2.7, 3.1-3.3, 7, and 8.1 Lecture 14 slides
Lecture 15 - Diffusion models and DDPMs; forward noising and reverse generation, Gaussian posterior and noise prediction, training and sampling, VAE comparison, and ELBO decomposition Ch. 15, Secs. 1 and 2.1-2.7 Lecture 15 slides
Lecture 16 - Graph data and graph-, node-, and edge-level tasks; graph Laplacians and spectral graph cuts, graph Fourier transforms, spectral convolution, and ChebNet with Chebyshev polynomial filters and local recurrence Ch. 17, Secs. 1-3 Lecture 16 slides
Lecture 17 - GCNs, neighborhood aggregation, graph learning tasks, GATs, and the Transformer connection Ch. 17, Secs. 3-6.2 Coming soon
Lecture 18 - PAC/VC generalization theory, Keras implementation, and deep reinforcement learning Ch. 6, Secs. 2-3 and 5.1; Ch. 18, Secs. 1-10; coding context: Ch. 22, Secs. 2-3 Coming soon

Class examples

Examples, Code & Data

Datasets, code, and notebooks used for demonstrations and examples are collected here independently of the lecture schedule.

Variational inference · Variational autoencoders

Variational Inference and VAE Demo

Explore posterior approximation, the variational lower bound, and reparameterization. Train a small VAE and compare reconstruction with generation. Run variational_autoencoder_demo in MATLAB R2016b or later; no additional toolboxes are required.

Direct preference optimization · Preference learning

DPO Loss Example

Compute the DPO loss for a preferred and a less-preferred response. Change the current-model probabilities, reference-model probabilities, and beta in the script to compare scenarios. Run DPO in MATLAB.

Final presentations

Presentation groups & papers

Register your group of up to four students now. Choose a presentation paper later, or propose another paper for approval.

Presentation schedule

Each group has 25 minutes total: approximately 15–20 minutes of prepared material and 5–10 minutes for discussion and questions. The split is flexible; questions are welcome throughout.

Thursday, November 26, 2026
Tuesday, December 1, 2026
TimeGroupPaper
1:00–1:25 p.m.Group 8How much can language models memorize?
1:25–1:50 p.m.Group 4Compute-Optimal Quantization-Aware Training
1:50–2:15 p.m.Group 5Log-Linear Attention
Tuesday, December 8, 2026

A pending paper does not change a group’s scheduled presentation time.

Presentation rubric

The presentation is worth 10% of the final course grade. The weights below apply to the presentation mark.

CriterionWhat we assessWeight
Motivation & problemExplain why the work matters and clearly define the problem it addresses.20%
Main idea & technical understandingExplain how the approach works, with enough technical depth to demonstrate understanding. Establish the main ideas before finer details.35%
Findings & interpretationExplain the main results, what we learn from them, and relevant limitations.20%
Clarity & organizationTeach the essential ideas clearly and selectively. Plan approximately 15–20 minutes of prepared material within the 25-minute session.15%
Discussion & questionsAllow approximately 5–10 minutes for discussion. Invite questions throughout and respond thoughtfully, demonstrating understanding.10%

You are assessed on understanding and explanation, not on covering every detail. Discussion is assessed by how you facilitate and respond to questions, not by how many questions classmates ask.

Final project

Project groups & topics

Register your group of up to four students now. Choose a project later, or propose your own for approval. Find project descriptions, starter materials, report templates and the marking scheme on the project page.

Coursework

Assignments

Due Monday, October 26, 2026, at 11:59 p.m. Toronto time

STAT 940 — Assignment 1: Foundations, Optimization, Regularization, and Neural Architectures

Individual assignment · 100 marks

View full instructions

Download the assignment PDF and both supplied Python files linked above. Keep data.py and model.py with your Question 7 code and use relative paths. Read the assignment instructions carefully; the Deliverables box at the end of each question specifies exactly what to submit.

Submission instructions
  • Crowdmark: Upload your written solutions, tables, and figures to the matching assignment questions. Do not paste full code into Crowdmark.
  • Learn: Submit one ZIP file named WatIAM_A1.zip to the Assignment 1 Dropbox, replacing WatIAM with your Waterloo username. Combine the required files for Questions 1 and 7 in this single ZIP: Q1.py (or Q1.R/Q1.m), Q7.py, data.py, model.py, any helpers, Q7_results.csv, Q7_history.csv, Q7_test.csv, and one README covering commands, dependencies, and seeds. Do not submit separate Q1 and Q7 ZIPs or individual code files.

For Question 1, use basic array operations and do not use automatic differentiation. For Question 7, use Python/NumPy and the supplied engine; write your own experiment loop, comparisons, logging, selection, and diagnostics, and do not call ready-made training, search, or early-stopping routines.

Required Generative AI Use Survey

Generative AI use is permitted. After completing the assignment, open Learn and select Submit → Surveys → Assignment 1: Generative AI Use Survey. Everyone must complete the survey, including students who used no generative AI; in that case, answer No to all five questions. The survey is not anonymous so completion can be checked. Your survey responses do not affect your assignment grade. Open Learn.

Individual Verification of Understanding

You may be asked to complete a brief individual quiz or participate in an oral discussion about your assignment. This may include explaining your reasoning and solutions, answering related questions, or explaining and making small modifications to your submitted code. You must demonstrate your own understanding of all submitted work, including work developed with permitted generative AI assistance. These checks must be completed without generative AI or assistance from others. If you cannot demonstrate adequate understanding of your submitted work, your overall assignment mark may be reduced.

Late Submission Policy

Submissions received up to 24 hours after the deadline receive a 10% penalty. Submissions more than 24 hours late receive a grade of zero.

Due Monday, November 16, 2026, at 11:59 p.m. Toronto time

STAT 940 — Assignment 2: Attention, Transformers, Alignment and VAEs

Individual assignment · 100 marks

View full instructions

Download the assignment PDF and student starter ZIP. Extract the ZIP and keep the Q6 files together. The supplied data.py generates the data locally; no separate dataset download is required. The Deliverables box at the end of each question specifies what to submit.

Submission instructions
  • Crowdmark: Upload written answers, tables, figures and interpretations to the matching question numbers. Do not paste full source code into Crowdmark.
  • Learn: Submit one combined ZIP named WatIAM_A2.zip to the Assignment 2 Dropbox, replacing WatIAM with your Waterloo username. Include both Q3/ and Q6/, code and helpers, generated CSV tables and figures, and your own README.md with run commands, package versions and seeds. Use relative paths. Exclude environments, installed packages and trained-model checkpoints. Do not submit separate ZIPs for Q3 and Q6.

Use Python with NumPy and Matplotlib for Q3; Q6 also uses PyTorch. Complete the five TODOs in Q6/model.py and write Q6/check_loss.py. The supplied Q6/run_q6.py handles data generation, training, evaluation and plotting. Equivalent independent code is permitted if it follows the PDF's protocol.

Late Submission Policy

Submissions received up to 24 hours after the deadline receive a 10% penalty on the earned mark. Submissions more than 24 hours late receive a grade of zero.

Course tools

Piazza & Waterloo Learn

Piazza

Use Piazza for announcements, lecture questions, clarifications, and class discussion.

Go to Piazza
Waterloo Learn (D2L)

Use Learn for graded work, submissions, private course documents, deadlines, and grades.

Open Waterloo Learn

Sign-in may be required. Students should use the course-specific links available in their enrolled Piazza and Learn accounts.

How the course runs

Course information

Course materials

Public slides are posted with the tentative topics. Additional or restricted materials are shared through Learn.

Assessment

Public assignment instructions are posted on this page. Submission links, feedback, and official grades are provided through Learn.

Questions

Use Piazza for course-content questions. Use a private Learn message or email for personal matters.

Deadlines

Consult Learn and Piazza for authoritative dates, room information, schedule changes, and deadlines.