Week 1 · Day 1 · Monday 12 October

State-of-the-Art Deep Learning Architectures

The backbones the rest of the course builds on: the CNN architectures that won ImageNet, then Vision Transformers, hierarchical variants such as Swin, and how they compare with CNNs.

Topic Description & Objectives Resource
CNN Architectures The notable CNN architectures and the ImageNet challenge winners, loss functions and optimizers, and the limitations of CNNs Open
Vision Transformers Patch embeddings and the ViT architecture, distillation, CNNs versus ViTs, Swin and other hierarchical transformers, DETR and ConvNeXt Open
Lab 1: Group Equivariant CNNs Why inductive biases matter: from augmentation to group convolutions that build rotation equivariance into a CNN, on MNIST Notebook Colab
Lab 2: Fine-Tuning a Vision Transformer Fine-tune Google's ViT-Large on Animals-10 with Hugging Face, from dataset preparation and stratified splits to evaluation Notebook Colab