Zixian Wang
I am a PhD student in Computer Science at the University of Illinois Urbana-Champaign (UIUC), advised by Prof. Minjia Zhang. My research focuses on high-performance, large-scale LLM training: scaling Mixture-of-Experts (MoE) models on supercomputers, ML systems, and GPU performance optimization.
Before UIUC, I received my B.S. in Computer Science from UC San Diego, where I worked with Prof. Leon Bergen, Prof. Ramamohan Paturi, and Prof. Taylor Berg-Kirkpatrick. I also co-led the UCSD team at the SC23 and SC24 Student Cluster Competitions and interned twice at AMD.
Research interests: Large-scale LLM training & inference systems · Mixture-of-Experts at scale · GPU kernel optimization · High-performance computing
News
- 2026 X-MoE-4D (first author) is accepted to SC26! Trillion-parameter MoE training on hierarchical HPC systems.
- 2026 The Surprising Soupability of Documents in State Space Models is accepted to COLM 2026!
- 2025 Quiet Feature Learning in Algorithmic Tasks is accepted to AAAI 2026 as an oral presentation!
- 2025 Started my PhD at UIUC with Prof. Minjia Zhang.
- 2025 Interned at AMD on the MLPerf team (Llama 3.1 405B inference, Llama 3 8B training).
Publications
* denotes equal contribution. Full list on Google Scholar.
X-MoE-4D: Breaking the Scaling Barrier for Trillion Parameter MoEs on Hierarchical HPC Systems
Zixian Wang, Liheng Jing, Emily Herron, Sajal Dash, Feiyi Wang, Minjia Zhang
SC 2026 The International Conference for High Performance Computing, Networking, Storage, and Analysis
The Surprising Soupability of Documents in State Space Models
Yasaman Jafari*, Zixian Wang*, Leon Bergen, Taylor Berg-Kirkpatrick
COLM 2026 Conference on Language Modeling · arXiv
Quiet Feature Learning in Algorithmic Tasks
Prudhviraj Naidu, Zixian Wang, Leon Bergen, Ramamohan Paturi
AAAI 2026 AAAI Conference on Artificial Intelligence · Oral · arXiv
Omniwise: Predicting GPU Kernels Performance with LLMs
Zixian Wang, C. Ramos, M. A. Awad, K. Lowery
Preprint arXiv:2506.20886, 2025 · arXiv
Preliminary Results of the MLPerf BERT Inference Benchmark on AMD Instinct GPUs
Zixian Wang, K. Vu, M. Hodak, A. Mehrotra, F. Gutierrez, K. Smith, G. Seo, et al.
PEARC 2024 Practice and Experience in Advanced Research Computing
Competitions & Awards
- 1st Place among U.S. & European Teams (4th Overall), SC24 Student Cluster Competition2024, as team co-lead. SC (Supercomputing) is the world's premier conference for high-performance computing.
- 1st Place in the MLPerf Benchmark among AMD GPU Teams, SC24 Student Cluster Competition2024, as the MLPerf benchmark lead. 2nd across all GPU teams. Built the first open-source multi-node optimized MIGraphX backend for MLPerf SDXL inference.
- 3rd Place Overall (1st among U.S. Teams), SC23 Student Cluster Competition2023, as team co-lead. Designed a 3-node cluster with 12 AMD Instinct MI210 GPUs and 6 AMD EPYC CPUs.
- 1st Place in the MLPerf Benchmark (Co-winner), SC23 Student Cluster Competition2023, as the MLPerf benchmark lead for UCSD. Shared with Peking University.
- Community Impact Award, SC23 Student Cluster Competition2023, with the UCSD team.
Experience
AMD, Performance Engineer Intern, MLPerf Team May 2025 – Aug 2025
- Core contributor to the MLPerf Inference v5.1 Llama 3.1 405B submission
- Core contributor to the MLPerf Training Llama 3 8B benchmark
AMD, Research Intern, RAD Profiling Team Jun 2024 – Sep 2024
- LLM-guided GPU software optimization research, leading to Omniwise
- Co-authored two invention disclosures; U.S. patent application in progress
Education
University of Illinois Urbana-Champaign, Ph.D. in Computer Science 2025 – Present
University of California, San Diego, B.S. in Computer Science 2021 – 2025