共计 60 篇文章
2026
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism
LLM Systems 18: Memory Optimization in Distributed Training
LLM Systems 17: System for MoE Models
LLM Systems 16: Model Parallel Training
LLM Systems Assignment 4: Transformer CUDA Acceleration
DDP: PyTorch Distributed-Experiences on Accelerating Data Parallel Training
LightSeq2: Accelerated Training for Transformer-based Models on GPUs
LightSeq: A High Performance Inference Library for Transformers
LLM Systems 15: Distributed Data Parallel Training