Ali Ghodsi

Course resources

Ali Ghodsi

Deep Learning

Deep Learning, Motivation and course administration

Feedforward Neural Network, Backpropagation

Optimization

Regularization

Dropout, Batch Normalization

Convolutional Neural Networks (CNN)

Regularization (Layer norm, FRN, TRU), Keras

Recurrent neural network (RNN)

Attention mechanism, self-attention, S2S

Transformers

BERT and GPT

Deep Reinforcement Learning (Part 1)

Deep Reinforcement Learning (Part 2)

RLHF, ChatGPT, Alignment in LLMs

Variational Autoencoder, VAE, Performer

Generative Adversarial Networks (GAN), AAE

Diffusion Models, DDPMs

Graph Neural Network (Part 1)

Graph Neural Network (Part 2)

PAC Learnability in Deep Learning

Data Visualization & Unsupervised Learning

Principal Component Analysis

PCA (Ordinary, Dual, Kernel)

FDA

MDS, Isomap, LLE

LLE, Spectral Clustering

Spectral Clustering, Laplacian Eigenmap, MVU

MVU, Action Respecting Embedding, Supervised PCA

Supervised PCA

SPCA, Nystrom Approximation, NMF

NMF via R1D algorithm

Sum-Product Networks

Neural Networks, Autoencoders, Word2Vec

Word2Vec Skip-Gram

Autoencoders, Clustering, Mixture of Gaussians

t-SNE

Variational Autoencoders

Statistical Learning — Classification

Intro to classifiers, Bayesian classifiers, LDA and QDA

QDA, PCA

FDA

Logistic regression

Model selection, Neural Networks

Spectral Clustering, Laplacian Eigenmap, MVU

Back Propagation, RBF

Complexity control for RBF

Regularization, Hard Margin SVM

SVM, Kernel SVM

Soft Margin SVM

Metric Learning

SPCA, Naive Bayes, K-nearest neighbour

Convolutional Neural Networks

Random features, Tree

Tree, Boosting method

Boosting method

Bagging

Deep Learning

Sep 7: Introduction (no video)

Sep 12: Perceptron, FFNN, Backpropagation

Sep 14: Overfitting, Regularization

Sep 19: Weight Decay, Introduction to Keras

Sep 26: Regularization, Dropout

Sep 28: Batch Normalization, CNN

Oct 3: CNN

Oct 5: RNN

Oct 12 Part 1: Variational Autoencoder

Oct 12 Part 2: Variational Autoencoder

Oct 17: Sum Product Network

Oct 19: Deep Reinforcement Learning

Oct 24: Generative Adversarial Networks

Statistical Learning — Classification

Machine Learning, Introduction

Formal definition of classification, Linear discriminant analysis (LDA), Quadratic discriminant analysis (QDA)

QDA, Principal Component Analysis (PCA)

PCA, Fisher’s Discriminant Analysis (FDA)

Logistic Regression

Logistic Regression, Perceptron

Backpropagation