Date: October 23, 2026
Speaker: Yuqi Mai, Ph.D. Student, Cornell Bowers
Title: MoonBright: A GPU Memory Allocator with Device-Side Page Table Materialization and Deferred TLB Coherence
Abstract: Modern GPU workloads increasingly rely on dynamic and fine-grained memory allocation, yet GPU memory management remains CPU-centric. In current GPU runtimes, allocation metadata updates, page-table construction, translation-state propagation, and TLB shootdowns are largely serialized through the host control path, introducing substantial latency. We present MoonBright, a GPU memory allocator that enables device-side page-table materialization and deferred TLB coherence on commodity GPUs. MoonBright keeps validation and allocation metadata on the host, but moves bulk page-table construction to the GPU, turning translation updates into data-parallel device-memory operations. To avoid costly TLB shootdowns on the common path, MoonBright assigns fresh virtual addresses to new mappings, ensuring that no stale same-address TLB entries can exist. Our evaluation shows that MoonBright reduces allocation latency, improves LLM inference performance, and mitigates allocator-level external fragmentation across diverse workloads. Unlike application-specific memory managers, MoonBright requires no GPU hardware modifications and runs on commodity NVIDIA and AMD GPUs with software-only changes.
Paper: https://www.usenix.org/system/files/osdi26-zhang-yangyu.pdf
Bio: Yuqi Mai is a Ph.D. student in Computer Science at Cornell Tech, advised by Prof. Alex Conway. His research focuses on memory systems, particularly virtual memory, address translation, and hardware-software co-design.
