Language Modeling with Hyperspherical Flows
The S-FLM introduces a Flow Language Model operating on unit-norm embeddings in hyperspherical space. It generates sequences by moving these vectors toward a uniform prior using a learned velocity field defined by an ODE. This method avoids the computational cost of one-hot vectors, offering continuous flow language models that improve large-vocabulary reasoning and close gaps with discrete diffusion models.