Part 2: Language Modeling
Course Home · Part 0 · Part I · Part II
2.1 Statistical Language Modeling
Topic Slides · 51 slides
2.2 Transformer
Topic Slides · 94 slides
- 2.2.1 Background: Neural Machine Translation and RNNs
- 2.2.2 Background: Attention
- 2.2.3 Transformer Overview
- 2.2.4 Transformer Blocks
- 2.2.5 Self-Attention
- 2.2.6 Feed-Forward Networks
- 2.2.7 Positional Encoding
- 2.2.8 Masking the Future
- 2.2.9 Training Stability
- 2.2.10 Transformer Block Summary
- 2.2.11 Transformer Architecture
- 2.2.12 Position Representations
- 2.2.13 Context Length
- 2.2.14 Transferability of Modifications
- 2.2.15 Suggested Readings
2.3 Pretrained Language Models
Topic Slides · 76 slides
2.4 Language Understanding Tasks
Topic Slides · 80 slides
2.5 Large Language Models
Topic Slides · 99 slides
- 2.5.1 Pretraining
- 2.5.2 Supervised Instruction Fine-Tuning
- 2.5.3 Reinforcement Learning from Human Feedback
- 2.5.4 Decoding Strategies
- 2.5.5 Suggested Readings
多模态理解与生成见 Part 4: Intelligent System。