# Preprint adapts language models to read graph structure directly

_Published Monday, October 5, 2026 at 11:08 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report that their Graph Transformer Language Model lets a pretrained language model process relationships between graph nodes directly, rather than compressing each node's text before passing it to a separate graph model. The preprint adds graph-aware attention biases with structure-related parameters equal to 0.015% of the base model.

In a test requiring retrieval of information within a graph, accuracy stayed flat from 1k to 64k tokens and 4x beyond the training length, while an identically trained flat-text baseline collapsed. Training updates the structural parameters and a small adapter; the model reduces exactly to the pretrained model when no graph is present.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2605.10247)

---
Canonical: https://techandbusiness.org/newswire/3ytbw6X0FcM3QAin-MwyuB
Published: 2026-10-05T15:08:31.415Z
Story chronology: 2026-10-05T04:00:00.000Z
Retrieved: 2026-10-05T16:50:40.537Z
Publisher: Tech & Business (techandbusiness.org)