Ali Ghodsi

Professor, University of Waterloo

[first name].[last name] [AT] uwaterloo.ca

Teaching

Fall 2026

STAT 841 / CM 763

Statistical Learning - Classification

An in-depth graduate course on the statistical foundations, computational methods, and modern advances underlying classification and pattern recognition.

Course at a glance

Overview

Course STAT 841 / CM 763
Term Fall 2026
Instructor Ali Ghodsi
Class meetings 4:00-5:20 p.m. (meeting days to be confirmed)
Location MC 4042

This page is the public home for the course topics, slide releases, and assignments. Enrollment-specific information, submissions, deadlines, and grades are maintained in Waterloo Learn.

Course support

Office Hours and Course Support

Instructor office hours

Ali Ghodsi

Wednesdays, 1–2 p.m.

M3 4009

Come to the instructor for: questions about course content or the final project.

Teaching assistant office hours

Assignment and coding support

Go to the TAs for: questions about assignments or coding.

Nafisat Ibrahim
Office hours
Fridays, 9–10 a.m.
Location
M3 2101
Jerry Zhen
Office hours
Thursdays, 1–2 p.m.
Location
M3 2101

From the course outline

Course details

Textbooks and references

There is no mandatory textbook. The following references cover substantial portions of the course:

  • The Elements of Statistical Learning, Trevor Hastie, Robert Tibshirani, and Jerome Friedman.
  • Pattern Recognition and Machine Learning, Christopher M. Bishop.
  • Machine Learning: A Probabilistic Perspective, Kevin P. Murphy.
Assessment and evaluation
Component Description Weight
Assignments Four individual assignments at 12.5% each 50%
Paper presentation Small-group presentation of a selected research paper 10%
Group final project Research or applied classification project 40%
Total — 100%
Tentative assessment dates
Assessment Release Due
Assignment 1 Monday, September 28 Monday, October 19, 11:59 p.m.
Assignment 2 Monday, October 19 Monday, November 9, 11:59 p.m.
Assignment 3 Monday, November 9 Monday, November 23, 11:59 p.m.
Assignment 4 Monday, November 23 Monday, December 7, 11:59 p.m.
Paper presentation Paper-selection process to be announced Scheduled presentation date
Group final project To be announced Monday, December 21, 11:59 p.m.

All dates are tentative and may change. Changes will be announced.

Latest updates

Announcements

Course site

Welcome to STAT 841 / CM 763

This page will be updated throughout the term. Slides and assignments will be posted below as they are released.

Communication

Use Piazza for course discussion

Questions about lectures and course content should be posted on Piazza so that answers are available to the class.

Course schedule

Tentative lecture schedule

Lecture topics and ordering are tentative and may be adjusted during the term.

Lecture Coverage Materials
Lecture 1 Course introduction; classification problems; hypothesis class, loss, and search; Bayes classifier and Bayes error Lecture 1 slides
Lecture 2 Gaussian classifiers; LDA and QDA; covariance assumptions and decision boundaries Lecture 2 slides
Lecture 3 Principal component analysis (PCA); variance-maximizing projections; Lagrange multipliers; covariance eigendecomposition and SVD; encoding and reconstruction Lecture 3 slides
Lecture 4 PCA versus supervised class separation; Fisher's linear discriminant analysis for two and multiple classes; within- and between-class scatter; generalized eigenvalue formulation Lecture 4 slides
Lecture 5 Logistic regression; binomial class probabilities; maximum likelihood; score and Hessian; Newton-Raphson and iteratively reweighted least squares (IRLS) Lecture 5 slides
Lecture 6 Neural-network history from McCulloch-Pitts through AlexNet; perceptron model and geometric decision boundary; gradient-based perceptron learning; linear separability and convergence Lecture 6 slides
Lecture 7 Feedforward and deep neural networks; backpropagation via the chain rule; gradient-based weight updates and epochs; stochastic gradient descent, mini-batches, and learning-rate tuning Lecture 7 slides
Lecture 8 Model selection and cross-validation; regularization and weight decay; data augmentation, noise injection, early stopping, and dropout; Keras neural-network architecture, activation functions, layers, losses, and optimizers Lecture 8 slides
Lecture 9 Radial basis function (RBF) networks; network architecture, matrix formulation, and closed-form least-squares fitting Lecture 9 slides
Lecture 10 Model selection; empirical and true error; Stein's unbiased risk estimator (SURE); RBF model complexity, noise-variance estimation, and regularization Lecture 10 slides
Lecture 11 Support vector machines; maximum-margin geometry and hard-margin optimization; Lagrangian dual formulation, KKT conditions, and support vectors; kernel methods and feature spaces; MATLAB classification of handwritten digits with linear and Gaussian kernels Lecture 11 slides
Lecture 12 Soft-margin SVMs; slack variables and margin violations; primal and dual optimization, the Lagrangian, and KKT conditions; support-vector cases, kernel formulation, and computing the intercept and classifier Lecture 12 slides
Lecture 13 Convolutional neural networks: convolution and cross-correlation, sparse connectivity, parameter sharing, and translation equivariance; pooling and downsampling; training convolutional layers and using random filters; ResNet and DenseNet Lecture 13 slides
Lecture 14 Decision trees for regression and classification; recursive partitioning, splitting criteria, overfitting, and cost-complexity pruning; node impurity, ID3 and CART; random forests, bootstrap sampling, random feature selection, majority voting, and variance reduction Lecture 14 slides
Lecture 15 Bernoulli and Gaussian Naive Bayes; conditional independence and class scores; k-nearest neighbours, feature scaling, and dimensionality; Mahalanobis metric learning and large-margin nearest neighbours (LMNN); Siamese networks, contrastive and triplet losses, and supervised contrastive learning Lecture 15 slides
Lecture 16 Supervised PCA; maximum mean discrepancy (MMD) and the Hilbert-Schmidt independence criterion (HSIC); kernel centering and empirical HSIC; label kernels and the eigenvalue solution; comparison with ordinary PCA, prediction, metric-learning connections, and limitations of linear projections Lecture 16 slides
Lecture 17 Decision trees; recursive partitioning, splitting criteria, overfitting, and pruning Coming soon
Lecture 18 Boosting; weak learners, adaptive reweighting, and stage-wise additive models Coming soon
Lecture 19 Bagging, random forests, variance reduction, and ensemble comparison Coming soon
Lecture 20 Semi-supervised learning, selected emerging methods, and integration of the course's main ideas Coming soon

Class examples

Examples, Code & Data

Datasets, code, and notebooks used for demonstrations and examples are collected here independently of the lecture schedule.

Classification · Handwritten digits

Handwritten Digits 2 and 3

A 64 × 400 MATLAB matrix named X containing 400 vectorized 8 × 8 handwritten-digit images. Each column is one image: columns 1–200 are examples of the digit 2, and columns 201–400 are examples of the digit 3.

LDA and QDA · PCA and FDA · Logistic regression

Classification and Feature Extraction Demo

An interactive MATLAB demo reviewing LDA, QDA, PCA, FDA, and logistic regression using three synthetic classification datasets. It steps through feature extraction, decision boundaries, and model evaluation.

Feature maps · Kernel methods

Kernel Mapping Demo

Visualize a quadratic feature map, a separating plane in feature space, and the corresponding nonlinear boundary in input space. Compare explicit feature inner products with kernel evaluations. Run kernel_demo in MATLAB; no additional toolboxes are required.

Support vectors · Kernel methods

Hard-Margin SVM Demo

Compare linear, quadratic, and Gaussian kernels on the same data. See when a hard-margin separator exists and how the support vectors determine its boundary. Requires MATLAB with Optimization Toolbox.

Slack variables · Margin violations

Soft-Margin SVM Demo

Adjust the penalty for margin violations and see how the decision boundary changes on overlapping data. Select a point to inspect its slack and see whether it is a support vector. Requires MATLAB with Optimization Toolbox.

Regression and classification trees · Recursive partitioning

Decision Tree Demo

Explore split thresholds, constant predictions within regions, tree growth, overfitting, and impurity criteria. Run decision_tree_demo in MATLAB R2016b or later; no additional toolboxes are required.

Bootstrap sampling · Random feature selection · Majority voting

Random Forest Demo

Compare identical trees, bagging, and random forests. Inspect sampled observations and candidate features, then click a point to see individual tree votes and the majority prediction. Run random_forest_demo in MATLAB R2016b or later; no additional toolboxes are required.

Cost-complexity pruning · Region merging

Tree Pruning Demo

Merge regions by collapsing tree branches and see how fitted constants, fitting error, and the leaf-count penalty change. Adjust alpha to compare simpler and more detailed trees. Run pruning_demo in MATLAB R2016b or later; no additional toolboxes are required.

Final presentations

Presentation groups & papers

Register your group of up to four students now. Choose a presentation paper later, or propose another paper for approval.

Presentation schedule

Each group has 25 minutes total: approximately 15–20 minutes of prepared material and 5–10 minutes for discussion and questions. The split is flexible; questions are welcome throughout.

Tuesday, December 8, 2026
TimeGroupPaper
4:00–4:25 p.m.Group 15Conformal Prediction in Hierarchical Classification with Constrained Representation Complexity
4:25–4:50 p.m.UnassignedNo group scheduled
4:50–5:15 p.m.UnassignedNo group scheduled

A pending paper does not change a group’s scheduled presentation time.

Presentation rubric

The presentation is worth 10% of the final course grade. The weights below apply to the presentation mark.

CriterionWhat we assessWeight
Motivation & problemExplain why the work matters and clearly define the problem it addresses.20%
Main idea & technical understandingExplain how the approach works, with enough technical depth to demonstrate understanding. Establish the main ideas before finer details.35%
Findings & interpretationExplain the main results, what we learn from them, and relevant limitations.20%
Clarity & organizationTeach the essential ideas clearly and selectively. Plan approximately 15–20 minutes of prepared material within the 25-minute session.15%
Discussion & questionsAllow approximately 5–10 minutes for discussion. Invite questions throughout and respond thoughtfully, demonstrating understanding.10%

You are assessed on understanding and explanation, not on covering every detail. Discussion is assessed by how you facilitate and respond to questions, not by how many questions classmates ask.

Final project

Project groups & topics

Register your group of up to four students now. Choose a project later, or propose your own for approval. Find project descriptions, starter materials, report templates and the marking scheme on the project page.

Coursework

Assignments

Due Monday, October 19, 2026, at 11:59 p.m. Toronto time

STAT 841 / CM 763 — Assignment 1: Statistical Learning — Classification

Individual assignment · 100 marks

View full instructions

Download the assignment PDF and the data ZIP linked above. Extract the data ZIP before starting Question 6 and keep the four CSV filenames unchanged. Read the assignment instructions carefully; the Deliverables box at the end of each question specifies exactly what to submit.

Submission instructions
  • Crowdmark: Upload your written solutions, tables, and figures to the corresponding assignment questions. Keep full code in your Learn submission.
  • Learn: Submit one ZIP file named WatIAM_A1.zip to the Assignment 1 Dropbox in Learn, replacing WatIAM with your Waterloo username. Include Q3.py, Q5.py, and Q6.py (or the corresponding .R or .m files), any required helper files, Q6_cv_results.csv, Q6_predictions.csv, and a short README giving run commands, software/dependencies, and random seeds.

Use Python, R, or MATLAB and follow the programming requirements in the assignment PDF.

Required Generative AI Use Survey

Generative AI use is permitted. After completing the assignment, open Learn and select Submit → Surveys → Assignment 1: Generative AI Use Survey. Everyone must complete the survey, including students who used no generative AI; in that case, answer No to all five questions. The survey is not anonymous so completion can be checked. Your survey responses do not affect your assignment grade. Open Learn.

Individual Verification of Understanding

You may be asked to complete a brief individual quiz or participate in an oral discussion about your assignment. This may include explaining your reasoning and solutions, answering related questions, or explaining and making small modifications to your submitted code. You must demonstrate your own understanding of all submitted work, including work developed with permitted generative AI assistance. These checks must be completed without generative AI or assistance from others. If you cannot demonstrate adequate understanding of your submitted work, your overall assignment mark may be reduced.

Late Submission Policy

Submissions received up to 24 hours after the deadline receive a 10% penalty. Submissions more than 24 hours late receive a grade of zero.

Due Monday, November 9, 2026, at 11:59 p.m. Toronto time

STAT 841 / CM 763 - Assignment 2: RBF Networks, Risk Estimation, Kernels, and SVMs

Individual assignment · 100 marks

View full instructions

Download the assignment PDF and the data ZIP linked above. Extract the data ZIP before working on Questions 3, 5, and 6, and keep the supplied filenames and observation order unchanged. Read the assignment instructions carefully; the Deliverables box at the end of each question specifies what to submit.

Submission instructions
  • Crowdmark: Upload your written answers, tables, and figures to the matching assignment questions. Keep full code in your Learn submission.
  • Learn: Submit one ZIP file named WatIAM_A2.zip to the Assignment 2 Dropbox in Learn, replacing WatIAM with your Waterloo username. Include Q1.py, Q3.py, Q5.py, and Q6.py (or their .R or .m equivalents), all required helper and input files, Q1_first.csv, Q1_simulation.csv, Q3_results.csv, Q5_results.csv, Q6_results.csv, and a short README listing run commands, dependencies, random seeds, and solver settings. After extracting the ZIP, each main script must run without first running another question.

Use Python, R, or MATLAB. Implement the requested RBF fits, risk calculations, kernel evaluations, and prediction rules yourself. Generic constrained-optimization or quadratic-programming solvers are permitted in Questions 3 and 6; construct their objectives and constraints yourself. Follow the detailed programming requirements in the PDF. Questions 2 and 4 require written answers only.

Required Generative AI Use Survey

Generative AI use is permitted. After completing the assignment, open Learn and select Submit → Surveys → Assignment 2: Generative AI Use Survey. Everyone must complete the survey, including students who used no generative AI; in that case, answer No to all five questions. The survey is not anonymous so completion can be checked. Your survey responses do not affect your assignment grade. Open Learn.

Individual Verification of Understanding

You may be asked to complete a brief individual quiz or oral discussion about your assignment, including explaining your reasoning, answering related questions, or explaining and making small modifications to your code. You must demonstrate your own understanding of all submitted work, including work developed with permitted generative AI assistance. These checks must be completed without generative AI or assistance from others. If you cannot demonstrate adequate understanding, your overall assignment mark may be reduced.

Late Submission Policy

Submissions received up to 24 hours after the deadline receive a 10% penalty on the earned mark. Submissions more than 24 hours late receive a grade of zero.

Due Monday, November 23, 2026, at 11:59 p.m. Toronto time

STAT 841 / CM 763 - Assignment 3: Kernels, Convolutional Networks, and Tree Ensembles

Individual assignment · 100 marks

View full instructions

Download the assignment PDF and the data ZIP linked above. Extract the data ZIP before working on Questions 2, 4, and 5, and keep the supplied filenames and observation order unchanged. Read README.md and README_CNN.md for the data conventions. The Deliverables box at the end of each question specifies what to submit.

Submission instructions
  • Crowdmark: Upload your written answers, tables, and figures to the matching assignment questions. Keep full code in your Learn submission.
  • Learn: Submit one ZIP file named WatIAM_A3.zip to the Assignment 3 Dropbox in Learn, replacing WatIAM with your Waterloo username. Include Q2.py, Q4.py, and Q5.py (or their .R or .m equivalents), all required helper and input files, Q2_results.csv, Q4_results.csv, Q5_results.csv, and a short README listing run commands, dependencies, seeds, and model settings. Each script must run independently after the ZIP is extracted.

Use Python, R, or MATLAB. Neural-network libraries and automatic differentiation are permitted in Question 2. In Question 4, use a library base-tree learner, but implement bootstrap sampling and hard-vote aggregation yourself; a package such as R's ranger may be used for one tree at a time with internal resampling disabled. In Question 5, implement candidate selection, AdaBoost updates, and prediction yourself; do not use a ready-made boosting routine. Follow the detailed programming requirements in the PDF. Questions 1 and 3 require written answers only.

Required Generative AI Use Survey

Generative AI use is permitted. After completing the assignment, open Learn and select Submit → Surveys → Assignment 3: Generative AI Use Survey. Everyone must complete the survey, including students who used no generative AI. The survey is not anonymous so completion can be checked. Your survey responses do not affect your assignment grade. Open Learn.

Individual Verification of Understanding

You may be asked to complete a brief individual quiz or oral discussion about your assignment, including explaining your reasoning, answering related questions, or explaining and making small modifications to your code. You must demonstrate your own understanding of all submitted work, including work developed with permitted generative AI assistance. These checks must be completed without generative AI or assistance from others. If you cannot demonstrate adequate understanding, your overall assignment mark may be reduced.

Late Submission Policy

Submissions received up to 24 hours after the deadline receive a 10% penalty on the earned mark. Submissions more than 24 hours late receive a grade of zero.

Course tools

Piazza & Waterloo Learn

Piazza

Use Piazza for announcements, lecture questions, clarifications, and class discussion.

Go to Piazza
Waterloo Learn (D2L)

Use Learn for graded work, submissions, private course documents, deadlines, and grades.

Open Waterloo Learn

Sign-in may be required. Students should use the course-specific links available in their enrolled Piazza and Learn accounts.

How the course runs

Course information

Course materials

Public slides are posted with the tentative topics. Additional or restricted materials are shared through Learn.

Assessment

Public assignment instructions are posted on this page. Submission links, feedback, and official grades are provided through Learn.

Questions

For assignment and coding questions, attend TA office hours. For course-content or final-project questions, attend instructor office hours. Use a private Learn message or email for personal matters.

Deadlines

Consult Learn and Piazza for authoritative dates, room information, schedule changes, and deadlines.