I am a PhD candidate at the University of Illinois Urbana-Champaign, advised by Prof. Minjia Zhang. My research is on efficient systems for AI on emerging hardware (e.g., CPU–GPU superchips), in particular how LLM serving and training interact with the memory hierarchy, interconnects, and GPU kernels. I previously worked on memory and storage systems.
Our work on hidden CPU-side interference in LLM serving and training on superchips (SuperGuard) was accepted to HPCA 2027.
Aug 2026
Wrapped up my research internship at IBM Research. The analytical cost model I built for the Spyre AI accelerator is merged into open-source torch-spyre.
Sep 2025
Our work on lifetime-aware tensor offloading for LLM training (TeraIO) was accepted to NeurIPS 2025.
Mar 2025
Presented SkyByte at NVMW 2025 and HPCA 2025 in Las Vegas.
Sep 2024
Our CXL-SSD work, SkyByte, was accepted to HPCA 2025.
On tightly coupled CPU-GPU superchips, CPU-GPU co-execution workloads can leave the GPU idle even when GPU compute, GPU memory bandwidth, and the CPU-GPU interconnect are unsaturated. Through controlled experiments and microarchitectural analysis, this work traces these stalls to CPU-side resource contention that delays host-driven kernel dispatch. SuperGuard is a lightweight runtime that monitors dispatch efficiency and adjusts CPU-side resource allocation (MPAM-based LLC partitioning) to protect GPU progress. Across RAG-based inference, agentic pipelines, and LLM training with CPU offloading on NVIDIA GH200, it improves end-to-end performance by up to 2.1x and reaches 96% of contention-free performance on average.
@inproceedings{ting2027superguard,title={Where Did the {GPU} Time Go? Uncovering and Controlling Hidden Interference in {LLM} Serving and Training on Superchips},author={Ting, Hsu-Tzu and Zhang, Haoyang and Lian, Xinyu and Zhang, Minjia},booktitle={IEEE International Symposium on High-Performance Computer Architecture (HPCA)},year={2027},note={To appear},}
A lifetime-aware tensor offloading framework that extends GPU memory with low-cost PCIe SSDs for multi-GPU LLM training. TeraIO profiles tensor lifetimes in the first training iterations, plans offloading and prefetching around them, and moves tensors directly between GPUs and SSDs via GPUDirect Storage. It improves training performance by 1.47x on average over ZeRO-Offload and ZeRO-Infinity, reaching 80.7% of the ideal performance with unlimited GPU memory.
@inproceedings{yuan2025teraio,title={Cost-Efficient {LLM} Training with Lifetime-Aware Tensor Offloading via {GPUDirect} Storage},author={Yuan, Ziqi and Zhang, Haoyang and Zhou, Yirui Eric and Mohan, Apoorve and Chung, I-Hsin and Seelam, Seetharami and Huang, Jian},booktitle={Conference on Neural Information Processing Systems (NeurIPS)},year={2025},}
A CXL-based SSD that co-designs the host OS and the SSD controller: opportunistic context switches hide long flash accesses, the SSD DRAM is organized as a cacheline-level write log plus a page-level data cache, and adaptive page migration promotes hot pages to host memory. SkyByte outperforms current CXL-SSDs by 6.11x and cuts flash I/O traffic by 23.08x on average, reaching 75% of the performance of an ideal system with unlimited host DRAM.
@inproceedings{zhang2025skybyte,title={{SkyByte}: Architecting an Efficient Memory-Semantic {CXL}-based {SSD} with {OS} and Hardware Co-design},author={Zhang, Haoyang and Xue, Yuqi and Zhou, Yirui Eric and Li, Shaobo and Huang, Jian},booktitle={IEEE International Symposium on High-Performance Computer Architecture (HPCA)},year={2025},}
A unified GPU memory and storage architecture that merges host memory, GPU memory, and flash into one address space. Because DNN workloads are highly predictable, G10 uses compiler analysis to characterize tensor behavior and schedules data migrations ahead of time, hiding transfer costs. G10 outperforms state-of-the-art GPU memory solutions by up to 1.75x and reaches 90.3% of the performance of an ideal system with unlimited GPU memory.
@inproceedings{zhang2023g10,title={{G10}: Enabling An Efficient Unified {GPU} Memory and Storage Architecture with Smart Tensor Migrations},author={Zhang, Haoyang and Zhou, Yirui Eric and Xue, Yuqi and Liu, Yiqi and Huang, Jian},booktitle={IEEE/ACM International Symposium on Microarchitecture (MICRO)},year={2023},}
One more paper on ML systems is under review; details on request. See all publications.
IBM Research, Yorktown Heights, NY. Research Intern, Hybrid Cloud (mentor: David Grove). Built an analytical cost model as a compiler pass for the Spyre AI accelerator in open-source torch-spyre.
Jul--Aug 2021
T-Head Division, Alibaba Cloud, Hangzhou, China. Software R&D Intern. Optimized the JPEG library for RISC-V vector processors.
Education
2022--2028 (expected)
University of Illinois Urbana-Champaign. Ph.D., Computer Science. Advisor: Prof. Minjia Zhang.
2020--2022
University of Michigan. B.S.E., Computer Engineering (dual degree).
MICRO 2023 Travel Grant; Roger King Scholarship (2021), University Honors (2021), and Dean's List (2020, 2021), University of Michigan.
Teaching
UIUC: Graduate TA, Computer Architecture (CS 233), FA 2024, SP 2025, FA 2025. SJTU: TA, Computer Architecture (VE 470), SU 2022; Honors Physics (VP 160), SU 2020.
