The Surprising Soupability of Documents in State Space Models
Published in COLM 2026, 2026
Accepted to COLM 2026. (* equal contribution)
This paper investigates the soupability of documents within state space models, revealing surprising properties about how these models handle document mixing.
Recommended citation: Y. Jafari*, Z. Wang*, L. Bergen, T. Berg-Kirkpatrick. "The Surprising Soupability of Documents in State Space Models." Conference on Language Modeling (COLM), 2026.
Download Paper