# Preprint demonstrates generation of language-model weights for held-out data mixtures

_Published Friday, November 6, 2026 at 1:43 PM EST · AI, Science · Latest · Tier 2 — Notable_

Researchers report generating complete pretrained language-model weights for training-data mixtures withheld from their generator's training, in a new arXiv preprint. They found minimal changes in downstream test-set perplexity and general capability benchmark scores compared with models in their reference collection.

The team trained 100 GPT-2 models at each of three sizes: 51M, 124M and 355M parameters. Training mixtures varied across web text, books, mathematics, coding and French. A flow-matching model generated weights in a compressed representation, conditioned on those mixtures.

The researchers provide source code and artifacts. Their demonstrated generalization concerns held-out mixtures within that data-mixture space.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2611.07103)

---
Canonical: https://techandbusiness.org/newswire/cSLq2TlV5Xtjjgl6RScgv6
Published: 2026-11-06T18:43:17.566Z
Story chronology: 2026-11-06T05:00:00.000Z
Retrieved: 2026-11-06T22:50:40.788Z
Publisher: Tech & Business (techandbusiness.org)