Most language models still generate text one token at a time. Autoregressive systems need as many sequential forward passes as there are tokens in the output. Diffusion language models (DLMs) can update many positions in parallel, but they typically require dozens or hundreds of iterative denoising steps and still process the full sequence length at every step. Both approaches become expensive when inference volume is high.
In early October 2026, researchers from Stanford’s Hazy Research group introduced Next-Concept Modeling (NCM), a two-stage framework that generates text from coarse concepts to fine details. According to the paper (arXiv:2610.14162), NCM matches or beats strong diffusion baselines on several tasks while running 10–31× faster and using 4–5× fewer training tokens. The code, weights, and datasets are publicly available.
This post explains what NCM is, how the coarse-to-fine hierarchy works, what the speed and quality numbers actually mean, and where the current limitations lie.
The Problem NCM Tries to Solve
Autoregressive models dominate production systems because they are simple, well-understood, and scale reliably. Their cost is linear in sequence length: a 2,000-token response requires roughly 2,000 sequential steps. Speculative decoding and other optimizations help, but the fundamental sequential bottleneck remains.
Diffusion language models remove the left-to-right constraint. They can refine an entire sequence in parallel. In practice, however, quality often degrades when the number of sampling steps is reduced, and each step still operates on the full sequence. Training can also be data-hungry. The result is that the theoretical parallelism of diffusion does not always translate into large wall-clock speedups at high quality.
NCM sits between these two regimes. It keeps the idea of progressive refinement but organizes the refinement into a short hierarchy of discrete levels rather than many full-sequence denoising steps. Early levels are short and cheap; later levels are longer but still generated in parallel within each level. The total number of forward passes is small (often on the order of 8–12), and the early passes process far fewer tokens than the final output length.
How Next-Concept Modeling Works
NCM is a two-stage system.
Stage 1: Multiscale residual VQ-VAE tokenizer.
Text is encoded into a hierarchy of discrete codes. Early levels of the hierarchy are short and capture broad structure (roughly analogous to a high-level sketch). Later levels are longer and add finer lexical and syntactic detail. The codes are residual: each level stores a correction relative to the reconstruction produced by coarser levels. A vector-quantized variational autoencoder (VQ-VAE) with nested dropout is used so that the model learns useful representations at every scale. On the datasets reported, token-level reconstruction accuracy is high—often above 99.95%, and 100% on Sudoku, where exact recovery is required.
Stage 2: Parallel level-wise generation.
A transformer generates the hierarchy from coarse to fine. Within each level it predicts all codes in parallel, so one forward pass produces an entire level. Generation therefore requires only one forward evaluation per level rather than one per token or one per full-sequence denoising step. Because early levels are short, the first few passes are inexpensive. The model can also use block-diagonal attention patterns that further reduce cost relative to full causal attention across the growing sequence.
The net effect is a short, fixed schedule of coarse-to-fine steps. A typical run might use 8–12 levels. The earliest levels sketch the overall content; successive levels refine wording, structure, and details until the final discrete codes can be decoded into ordinary text.
Here is a simplified conceptual illustration of the generation loop (not the actual model code):
# Conceptual sketch only — real implementation is in the released NCM repo
def generate_ncm(prompt, num_levels=12):
# Start with an empty or prompt-conditioned coarse representation
codes = [None] * num_levels
for level in range(num_levels):
# One forward pass predicts ALL codes at this level in parallel
codes[level] = model.predict_level(
previous_levels=codes[:level],
prompt=prompt,
level=level
)
# Early levels are short; later levels grow longer
# Decode the full hierarchy of discrete codes into text
return tokenizer.decode(codes)
The real system uses a frozen multiscale residual VQ tokenizer and a separately trained next-concept generator. Both are released.
Results: Speed and Quality
The paper evaluates NCM primarily against strong diffusion language model baselines at matched parameter scales (around 100 million parameters for the main comparisons). Measurements were taken on a single NVIDIA H100. Key reported figures include:
• LM1B (unconditional generation): NCM reaches a generative perplexity of 93.05 in 12 function evaluations versus 97.30 for the DUO diffusion baseline in 128 evaluations — approximately 21× faster with better perplexity.
• OpenWebText: Competitive quality at roughly 10× the speed of the strongest diffusion baseline.
• XSum (extreme summarization): Matches the strongest diffusion baseline’s ROUGE-L while running about 31× faster (8 steps versus 128).
• WMT14 German–English: Competitive or better than several non-autoregressive baselines, with substantially fewer function evaluations.
• Sudoku (reasoning): Outperforms both autoregressive and diffusion baselines across easy, medium, and hard difficulties while using fewer function evaluations (for example, 86.1% of hard boards solved in 12 steps versus lower rates for the compared models).
Training efficiency is also notable. On LM1B and OpenWebText, NCM uses roughly 4–5× fewer training tokens than the strong diffusion baselines even after accounting for tokenizer training.
These numbers are the authors’ own measurements. Independent large-scale replications have not yet appeared, and the comparisons are mainly against diffusion and flow-based models rather than large frontier autoregressive systems. Still, the consistent pattern across multiple tasks is that a short coarse-to-fine schedule can deliver large speedups without obvious quality collapse on the benchmarks tested.
Why the Hierarchical Approach Helps
The design draws inspiration from next-scale prediction methods that have worked well for images (for example, visual autoregressive modeling and residual discrete token maps). Adapting the idea to text is non-trivial: text has no natural spatial downsampling operator, and a single token error can change meaning. NCM addresses this with a learned multiscale residual VQ-VAE that forces early levels to capture global structure while later levels resolve precise word choices.
The hierarchy also changes the compute profile. Early, short levels are cheap, so the model can spend most of its budget on the levels that add the most detail. Parallel prediction within each level avoids the sequential token-by-token cost of autoregressive decoding. Compared with full-sequence diffusion, NCM avoids repeatedly processing the entire final length at every step.
The name “concept” is descriptive rather than literal. The discrete codes are learned representations, not human-readable outlines. Qualitative probes in the paper suggest that coarser levels tend to capture broader structure while finer levels fill in specifics, but the codes themselves are not guaranteed to be interpretable as natural-language concepts.
Limitations and Open Questions
NCM is an early research result, and several limitations are acknowledged by the authors and visible in the experimental design:
• Scale. Most results are at roughly 100M-parameter generator size. Preliminary scaling curves show that larger models improve generative perplexity, but it is not yet known how the speed and quality advantages hold at the multi-billion-parameter regimes used in production.
• Fixed hierarchy. Every example currently uses the same number of levels and level lengths. The authors note that adaptive computation (stopping early on easy examples or allocating more levels to hard ones) is a natural next direction.
• Separate tokenizer. The VQ-VAE is trained first and then frozen. Joint training of tokenizer and generator might produce better representations, but it has not been demonstrated. Reconstruction errors, while rare, can still matter in text.
• Comparison scope. Speed claims are primarily versus diffusion language models of similar size. Direct wall-clock comparisons against highly optimized large autoregressive models (or production APIs) are not the focus of the paper.
• Task coverage. Public results cover unconditional generation, summarization, translation, and Sudoku. There is no released conversational or instruction-tuned model, and multilingual performance (especially languages with different tokenization characteristics) remains untested at scale.
The paper is still a preprint. Code and checkpoints are available under an MIT license, which lowers the barrier to independent verification.
Practical Implications and What’s Available Today
Anyone can clone the HazyResearch/ncm repository, install the dependencies, and run the released tokenizer + generator checkpoints on the supported tasks (XSum, WMT14, Sudoku, LM1B, OpenWebText, TinyStories). The models are not general-purpose chat models; they are task-specific research artifacts. They are useful for understanding the method, reproducing the reported numbers, and experimenting with hierarchical parallel generation.
For practitioners, NCM is most relevant as evidence that non-autoregressive, coarse-to-fine schedules can be competitive on both speed and quality for certain workloads. Whether the same ideas can be scaled, combined with larger models, or adapted to open-ended generation remains an open research question. The public release makes it straightforward to test those questions.
Conclusion
Next-Concept Modeling shows that generating text level by level—from short coarse codes to longer fine codes, with full parallelism inside each level—can produce large speedups relative to diffusion baselines while maintaining competitive quality on the tasks evaluated. The 10–31× figures come from controlled comparisons at moderate model sizes and should be read in that context. The approach is still early, the scaling picture is incomplete, and production chat performance has not been demonstrated.
What NCM does demonstrate is that the design space between pure autoregressive decoding and full-sequence diffusion is larger than it first appears. A short hierarchy of discrete residual codes, generated in parallel within each level, is one concrete point in that space that already delivers measurable efficiency gains.
Key takeaways
• NCM encodes text as a hierarchy of discrete residual codes (coarse to fine) with a multiscale VQ-VAE, then generates each level in parallel.
• On the reported benchmarks it is 10–31× faster than strong diffusion baselines at matched scale, with comparable or better quality and 4–5× fewer training tokens.
• Sudoku results suggest the hierarchical structure can help structured reasoning tasks.
• Limitations include moderate model scale, fixed schedules, a frozen tokenizer, and comparisons mainly against diffusion models.
• Code, weights, and datasets are public for anyone who wants to experiment.
If you work on inference efficiency or non-autoregressive generation, the NCM paper and repository are worth a close look. The next interesting results will be larger-scale runs, joint tokenizer-generator training, and adaptive depth—directions the authors themselves highlight.
Please let us know in the comments, what do you think about NCM.