دانشمندای هوش مصنوعی به سرپرستی آزالیا میرحسینی و تیمش یه معماری تر و تمیز رو کردن به اسم «مدل‌های مفهوم بعدی» یا همون NCM که قراره دودمان مدل‌های زبانی دیفیوژن رو به باد بده

Introducing Next-Concept Models (NCMs), a new paradigm for language generation that is significantly faster and more data efficient than diffusion models. How it works: 1) First we learn a series of compressed representations of text from coarse to fine-grained, aka ‘concepts’. This step is done once, similar to training a tokenizer. 2) At inference, we generate text going from a coarse to fine hierarchy via these ‘concepts’. A transformer predicts each level of the hierarchy sequentially, but the codes within each level are generated in parallel. Compared to strong diffusion language models, NCM achieves: Up to 31× faster generation on downstream tasks like summarization and machine translation. 10-21× faster generation across TinyStories, LM1B, and OpenWebText, using 3-5× fewer training data. On reasoning tasks like Sudoku, NCM outperforms both autoregressive and diffusion baselines with fewer sampling steps. Lead authors: @KumbongHermann & @ishanskhare, together with a great team!