| Learning from Videos |
Video data and its challenges, video understanding models, motion and temporal dynamics, and recognising actions |
Open |
| Foundation Models |
What a foundation model is and how to adapt one, CLIP, Grounding DINO and DINOv2, SAM 1 to 3, SEEM, Depth Anything and SuperGlue, TabPFN, and InternVideo2 |
Open |
| Lab 1: Action Recognition with 3D CNNs |
Classify the actions in HMDB51 videos with a 3D convolutional network that learns across space and time |
Notebook Colab |
| Lab 2: Foundation Models |
Ready-made models from the Hugging Face Hub, with no training: CLIP zero-shot classification, DINOv2, YOLO-World open-vocabulary detection, Depth Anything V2 and Florence-2 |
Notebook Colab |
| Optional: Video Generation |
Sora, Text2LIVE, Genie 1 to 3, world models, and latent predictive models with V-JEPA and V-JEPA 2 |
Open |
| Optional: World Models |
Ha and Schmidhuber's world models, PlaNet and Dreamer, generative simulators versus JEPA-style prediction, and LeWorldModel |
Open |
| Optional: World Models Lab |
From the reinforcement learning loop to an agent that learns a model of its environment and plans inside it |
Notebook Colab |
