CSE 291A/DSC 291: Deep Learning Systems
Instructor: Hao Zhang, UC San Diego, Spring 2026
Announcements
Week 4 Announcements
- PA2 has been released. The deadline is May 5, 2026 at 11:59 PM.
- Next week we will have our first guest lecture on Zoom. Attendance is mandatory for everyone.
Week 1
- March 31
-
- 1 Introduction
- No slides or recording for Lecture 1
- Survey Beginning of Quarter Survey (Due: End of Week 2 - 4/12)
- Readings (Due 4/12)
- Required: 1.1 - MLSys : Intro
- Optional: 1.2 - DL Primer, 1.3 - DNN Architectures, 1.4 - Systems Challenges for AI
- April 2
-
- 2 Basics: Modern DL, computational graph, frameworks
- Slides • Recording • Scribe Note
Week 2
- April 7
-
- 3 Basics: autodiff, ML system architecture overview
- Slides • Recording • Scribe Note
- Readings (Due 4/14)
- April 9
-
- 4 Tensor formats, matmul, accelerators
- Slides • Recording • Scribe Note
Week 3
- April 14
-
- 5 Accelerators, GPUs
- Slides • Recording • Scribe Note
- Readings (Due 4/21)
- Required: 3.1 - GPU Glossary
- Optional: 3.2 - GPU Matmul Optimization, 3.3 - GPU Puzzles
- April 16
-
- 6 GPU Matmul
- Slides • Recording • Scribe Note
Week 4
- April 21
-
- 7 Triton, graph optimization
- Slides • Recording • Scribe Note
- Readings (Due 4/28)
- Required: 4.1 - TVM, 4.2 - Triton
- Optional: 4.3 - TASO, 4.4 - Ansor, 4.5 - PET
- April 23
-
- 8 Memory
- Slides • Recording • Scribe Note
Week 5
- April 28
-
- 9 Quantization
- Slides • Recording • Scribe Note
- Readings (Due 5/5)
- Required: 5.1 - Quantization Survey, 5.2 - Low-bit LLM Survey
- Optional: 5.3 - LLM.int8(), 5.4 - SmoothQuant, 5.5 - AWQ
- April 30
-
- 10 Guest lecture: Hector Liu (MBZUAI IFM)
- Slides • Recording • Scribe Note
Week 6
- May 5
-
- 11 Transformers, LLMs, serving
- Slides • Recording • Scribe Note
- Readings (Due 5/12)
- May 7
-
- 12 Guest Lecture: Woosuk Kwon (Inferact.ai and vLLM)
- Slides • Recording • Scribe Note
Week 7
- May 12
-
- 13 Guest Lecture: Danyang Zhuo (Duke University)
- Slides • Recording • Scribe Note
- Readings (Due 5/19)
- Required: 7.1 - Orca
- Optional: 7.2 - DistServe, 7.3 - SARATHI / Chunked Prefills
- May 14
-
- 14 LLM internals, scaling law, inference and serving
- Slides • Recording • Scribe Note
Week 8
- May 19
-
- 15 Flash attention, training optimization, collective communication
- Slides • Recording • Scribe Note
- Readings (Due 5/26)
- Required: 8.1 - FlashAttention Tutorial, 8.2 - FlashAttention, 8.3 - Megatron-LM
- Optional: 8.4 - Alpa, 8.5 - GShard, 8.6 - GPipe
- May 21
-
- 16 Guest Lecture (Dr. Zihao Ye)
- Slides • Recording • Scribe Note
Week 9
- May 26
-
- 17 Parallelization
- Slides • Recording • Scribe Note
- Readings (Due 6/2)
- Required: 9.1 - MoE, 9.2 - ZeRO
- Optional: 9.3 - Speculative Decoding, 9.4 - Gated DeltaNet, 9.5 - DeepSpeed Ulysses, 9.6 - Ring Attention,
- May 28
-
- 18 Guest lecture (TBD): linear attention and attention variants
- Slides • Recording • Scribe Note
Week 10
- June 2
-
- 19 Parallelization
- Slides • Recording • Scribe Note
- Readings (Optional)
- You made it to Week 10, there is no required reading for this week!
- June 4
-
- 20 Deepseek V4 review
- Slides • Recording • Scribe Note