Table of Contents
Aligning large language models (LLMs) is a cutting-edge AI technology that ensures these models behave according to human intentions and values. This process involves techniques like reinforcement learning (RL), supervised fine-tuning, contextual learning, and socio-technical alignment. The course syllabus covers topics from foundational theories of LLMs to practical applications in alignment.
Course Structure: follow mainstream LLMs development pathways, emphasizing pre-training, supervised fine-tuning, and reinforcement learning from human feedback (RLHF). It systematically examines key algorithms behind LLMs and covers widely used algorithms like DPO and others;
Hands-on Training: conclude with practical in GPU hardware, using frameworks like PyTorch and Transformers. Students will independently train vertical LLMs with self-generated data and model deployment;
Hardware: include NVIDIA hardware architecture and programming tools used in modern AI systems, focusing on how these technologies accelerate AI computation, optimize performance, and enable efficient neural network training;
Safety and Value Alignment: understand the importance of safety and value alignment in LLMs and cover advanced topics like model evaluation and governance, supporting the practical deployment of LLMs.
Contact Info
- Instructors
- Yaodong Yang ([email protected])
- Yixin Zhu ([email protected])
- Guest Lecturers
- Siteng Ma ([email protected]) on NVIDIA CUDA
- Yan Cui ([email protected]) on NVIDIA DPU
- TAs
- Jiaming Ji ([email protected])
- Yifan Zhong ([email protected])
- Jiayi Zhou ([email protected])
Acknowledgment
We extend our sincere gratitude to AliCloud and NVIDIA for their generous support of this course. AliCloud has provided essential GPU resources, enabling our students to engage in hands-on learning and experimentation with cutting-edge LLM technologies. NVIDIA’s contribution of DPU resources has further enhanced our ability to offer comprehensive training in advanced computing techniques.
These valuable contributions from industry leaders have significantly enriched our course content and practical learning experiences. We are deeply appreciative of their commitment to fostering education and innovation in the field of artificial intelligence.