I am a PhD candidate at the University of Illinois Urbana-Champaign, advised by Prof. Minjia Zhang. My research is on efficient systems for AI on emerging hardware (e.g., CPU–GPU superchips), in particular how LLM serving and training interact with the memory hierarchy, interconnects, and GPU kernels. I previously worked on memory and storage systems.
Wrapped up my research internship at IBM Research. The analytical cost model I built for the Spyre AI accelerator is merged into open-source torch-spyre.
Sep 2025
Our work on lifetime-aware tensor offloading for LLM training (TeraIO) was accepted to NeurIPS 2025.
Mar 2025
Presented SkyByte at NVMW 2025 and HPCA 2025 in Las Vegas.
Sep 2024
Our CXL-SSD work, SkyByte, was accepted to HPCA 2025.
A lifetime-aware tensor offloading framework that extends GPU memory with low-cost PCIe SSDs for multi-GPU LLM training. TeraIO profiles tensor lifetimes in the first training iterations, plans offloading and prefetching around them, and moves tensors directly between GPUs and SSDs via GPUDirect Storage. It improves training performance by 1.47x on average over ZeRO-Offload and ZeRO-Infinity, reaching 80.7% of the ideal performance with unlimited GPU memory.
@inproceedings{yuan2025teraio,title={Cost-Efficient {LLM} Training with Lifetime-Aware Tensor Offloading via {GPUDirect} Storage},author={Yuan, Ziqi and Zhang, Haoyang and Zhou, Yirui Eric and Mohan, Apoorve and Chung, I-Hsin and Seelam, Seetharami and Huang, Jian},booktitle={Conference on Neural Information Processing Systems (NeurIPS)},year={2025},}
A CXL-based SSD that co-designs the host OS and the SSD controller: opportunistic context switches hide long flash accesses, the SSD DRAM is organized as a cacheline-level write log plus a page-level data cache, and adaptive page migration promotes hot pages to host memory. SkyByte outperforms current CXL-SSDs by 6.11x and cuts flash I/O traffic by 23.08x on average, reaching 75% of the performance of an ideal system with unlimited host DRAM.
@inproceedings{zhang2025skybyte,title={{SkyByte}: Architecting an Efficient Memory-Semantic {CXL}-based {SSD} with {OS} and Hardware Co-design},author={Zhang, Haoyang and Xue, Yuqi and Zhou, Yirui Eric and Li, Shaobo and Huang, Jian},booktitle={IEEE International Symposium on High-Performance Computer Architecture (HPCA)},year={2025},}
A unified GPU memory and storage architecture that merges host memory, GPU memory, and flash into one address space. Because DNN workloads are highly predictable, G10 uses compiler analysis to characterize tensor behavior and schedules data migrations ahead of time, hiding transfer costs. G10 outperforms state-of-the-art GPU memory solutions by up to 1.75x and reaches 90.3% of the performance of an ideal system with unlimited GPU memory.
@inproceedings{zhang2023g10,title={{G10}: Enabling An Efficient Unified {GPU} Memory and Storage Architecture with Smart Tensor Migrations},author={Zhang, Haoyang and Zhou, Yirui Eric and Xue, Yuqi and Liu, Yiqi and Huang, Jian},booktitle={IEEE/ACM International Symposium on Microarchitecture (MICRO)},year={2023},}
Two more papers on ML systems are under review; details on request. See all publications.
IBM Research, Yorktown Heights, NY. Research Intern, Hybrid Cloud (mentor: David Grove). Built an analytical cost model as a compiler pass for the Spyre AI accelerator in open-source torch-spyre.
Jul--Aug 2021
T-Head Division, Alibaba Cloud, Hangzhou, China. Software R&D Intern. Optimized the JPEG library for RISC-V vector processors.
Education
2022--2028 (expected)
University of Illinois Urbana-Champaign. Ph.D., Computer Science. Advisor: Prof. Minjia Zhang.
2020--2022
University of Michigan. B.S.E., Computer Engineering (dual degree).
MICRO 2023 Travel Grant; Roger King Scholarship (2021), University Honors (2021), and Dean's List (2020, 2021), University of Michigan.
Teaching
UIUC: Graduate TA, Computer Architecture (CS 233), FA 2024, SP 2025, FA 2025. SJTU: TA, Computer Architecture (VE 470), SU 2022; Honors Physics (VP 160), SU 2020.
