PMPP Chapter 03: Multidimensional grids and data 多维 grid & block gridDim gridDim.x 范围:[1, 2^{31} -1] gridDim.y & gridDim.z 范围:[1, 2^{16} -1] blockDim 最多 1024 线程(blockDim.x blockDim.y blockDim.z <= 1024) blockIdx threadId 2026-09-19 ML System #CMU 11868 #LLM System #CUDA #PMPP
PMPP Chapter 02: Heterogeneous data parallel computing 以 RGB 图片 -> 灰度图片转换为例,说明 Data Parallel 如何运行并行计算: CUDA 编程步骤: 分配 device global memory 拷贝 host memory 内容到 device memory launch kernel 在 device 进行计算 从 device 拷贝计算结果到 host 释放 device 内存 注: 2026-09-19 ML System #CMU 11868 #LLM System #CUDA #PMPP
PMPP Chapter 01: Introduction CPU vs. GPU 2003 年摩尔定律失效后,单核心 CPU 的频率/性能增长有限,新一代 CPU 的单核性能往往提升很小。面对这一问题 ,半导体产业产生了两条解决路径: 多核(multicore)CPU 多线程(multi-thread)GPU 在浮点运算性能(FLOPs)上,GPU 已经远远超过 CPU。这是因为二者设计哲学的区别: CPU 设计目的是优化 2026-09-19 ML System #CMU 11868 #LLM System #CUDA #PMPP
LLM Systems 04: GPU Acceleration 0. Recap Basic GPU CUDA operations memory allocation data movement creating threads and running on SMs specifying number of threads and number of blocks in a grid referring to data in GP 2026-09-19 ML System #CMU 11868 #LLM System
LLM Systems 03: GPU Programming-2 This article is the learning note of CMU 11868 LLM Systems course Lecture 3. For more details, please refer to the original slides. 0. Recap GPU is composed of streaming processing units ( 2026-09-19 ML System #CMU 11868 #LLM System
LLM Systems 02: GPU Programming This article is the learning note of CMU 11868 LLM Systems course Lecture 2. For more details, please refer to the original slides. 0. Recap Autoregressive Language Model: \(P\left( x_{1..T} 2026-09-12 ML System #CMU 11868 #LLM System
LLM Systems 01: Introduction This article is the learning note of CMU 11868 LLM Systems course Lecture 1. For more details, please refer to the original slides. 0. Learning Objective of This Course Understand key techni 2026-09-12 ML System #CMU 11868 #LLM System
读论文:Distributed LLM Serving Scheduling 0. Background 不同工作负载 prompts/response token 数之比、共享前缀比例、共享一个前缀的 request 数量: 关键问题:实现最大 KV Cache 复用与 GPU 间负载均衡 Load balance ratio(CV) is: \[ CV = \frac{ \sqrt{ \frac{1}{n} \sum_{i=1}^{n}{\le 2026-03-06 ML System #LLM Serving Scheduler
从 AlphaGo 到 ChatGPT,AI 浪潮下一名普通 00 后的十年 毕业工作后,博客月更似乎已经成为奢望。日复一日的工作的确让人少了许多输入和输出的时间与精力。今日机缘巧合,突然想记录如今这一波 AI 浪潮下,一名普通 00 后视角下对一些标志性事件的个人印象和经历。拥抱 AI,也许是这个时代必须做出的选择。 2016 春:AlphaGo 虽然如今 AI 这个词已经随处可见,人脸识别、指纹识别和语音助手等早已融入我们的日常生活。但是在我的记忆中, 2025-11-16 一支芦苇 #AI
读论文:The Landscape of GPU-Centric Communication Title:The Landscape of GPU-Centric Communication arXiv:https://arxiv.org/abs/2409.09874 1. 术语与通信类型 节点内(intro-node)通信: API:程序员或库在何处进行通信 API 调用。 Data Path:指示谁参与了数据移动,并显示相应的数据路径。 节点间(i 2025-06-02 ML System #GPU 通信