Zixian Wang

Zixian Wang

He/Him

CS PhD Student @ UIUC
ML Systems · LLM Training · HPC

Publications

You can also find my articles on my Google Scholar profile.

Conference Papers

X-MoE-4D: Breaking the Scaling Barrier for Trillion Parameter MoEs on Hierarchical HPC Systems

Published in SC 2026, 2026

Accepted to SC26. Scaling trillion-parameter Mixture-of-Experts training on hierarchical HPC systems.

Recommended citation: Z. Wang, L. Jing, E. Herron, S. Dash, F. Wang, M. Zhang. "X-MoE-4D: Breaking the Scaling Barrier for Trillion Parameter MoEs on Hierarchical HPC Systems." The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC26), 2026.

The Surprising Soupability of Documents in State Space Models

Published in COLM 2026, 2026

Accepted to COLM 2026. Encoding documents independently and pooling their states in Mamba2 models.

Recommended citation: Y. Jafari*, Z. Wang*, L. Bergen, T. Berg-Kirkpatrick. "The Surprising Soupability of Documents in State Space Models." Conference on Language Modeling (COLM), 2026.
Download Paper

Preliminary Results of the MLPerf BERT Inference Benchmark on AMD Instinct GPUs

Published in PEARC 2024, 2024

MLPerf BERT Inference Benchmark results on AMD Instinct GPUs

Recommended citation: Z. Wang, K. Vu, M. Hodak, A. Mehrotra, F. Gutierrez, K. Smith, G. Seo, et al. "Preliminary Results of the MLPerf BERT Inference Benchmark on AMD Instinct GPUs." Practice and Experience in Advanced Research Computing 2024: Human Powered Computing.