Table of Contents
Introduction
Aligning large language models (LLMs) is a cutting-edge AI technology that ensures these models behave according to human intentions and values. This process involves techniques like reinforcement learning (RL), supervised fine-tuning, contextual learning, and socio-technical alignment. The course syllabus covers topics from foundational theories of LLMs to practical applications in alignment.
Course Structure: follow mainstream LLMs development pathways, emphasizing pre-training, supervised fine-tuning, and reinforcement learning from human feedback (RLHF). It systematically examines key algorithms behind LLMs and covers widely used algorithms like DPO and others;
Hardware: include NVIDIA hardware architecture and programming tools used in modern AI systems, focusing on how these technologies accelerate AI computation, optimize performance, and enable efficient neural network training;
Safety and Value Alignment: understand the importance of safety and value alignment in LLMs and cover advanced topics like model evaluation and governance, supporting the practical deployment of LLMs.
Location and Time
- Location: 理教407
- Time: Tuesday 15:10-18:00, Week 1-16；每周二下午15:10-18:00，共16周
Schedule and Plan
| 课程内容 | Course Content | 日期 | Date |
|---|---|---|---|
| 大语言模型介绍 | Introduction to LLMs | 02/18 | 02/18 |
| 大模型基础架构 | Basic Architecture of LLMs | 02/25 | 02/25 |
| 大模型预训练 | Pre-training of LLMs | 03/04 | 03/04 |
| 大模型推理与思维链 | Inference and Chain of Thought in LLMs | 03/11 | 03/11 |
| 大模型推理与微调 | Inference and Fine-tuning of LLMs | 03/18 | 03/18 |
| 大模型高效微调法 | Efficient Fine-tuning Methods for LLMs | 03/25 | 03/25 |
| 强化学习精要 | Essentials of Reinforcement Learning | 04/01 | 04/01 |
| 策略优化方法 | Policy Optimization Methods | 04/08 | 04/08 |
| RLHF模型对齐方法 | RLHF Alignment Methods | 04/15 | 04/15 |
| 直接对齐方法 | Direct Alignment Methods | 04/22 | 04/22 |
| 具身多模态模型对齐 | Embodied Multimodal Model Alignment | 04/29 | 04/29 |
| Nvidia现代AI训练架构 | Nvidia Modern AI Training Architecture | 05/13 | 05/13 |
| GPU/CUDA编程实践 | GPU/CUDA Programming Practice | 05/20 | 05/20 |
| DPU编程实践 (I) | DPU Programming Practice (I) | 05/27 | 05/27 |
| DPU编程实践 (II) | DPU Programming Practice (II) | 06/03 | 06/03 |
Contact Info
- Instructors
- Yaodong Yang ([email protected])
- Yixin Zhu ([email protected])
- TAs (More TAs will be added later)
- Jiayi Zhou ([email protected])
- Jiaming Ji ([email protected])