Preprint reports faster text generation through parallel concept prediction

Researchers report that Next-Concept Modeling generated text 21× faster than the strongest diffusion language model baseline on LM1B while achieving better generative perplexity at matched parameter scales. The preprint describes a two-stage system that encodes text into a hierarchy of discrete codes. A transformer generates one level at a time, predicting all codes within each level in parallel. On OpenWebText, the system was competitive with the strongest diffusion baseline while running 10× faster. On XSum, it matched that baseline's ROUGE-L score while running 31× faster. These comparisons concern the tested diffusion baselines and tasks.