AMMA preprint reports lower attention latency with memory-centric chip design

Published Monday, November 2, 2026 · 12:20 AM EST

Researchers report that AMMA, a proposed chip architecture for serving long-context AI models, achieves 15.5X lower attention latency and 6.9X lower energy consumption than NVIDIA's H100 in their evaluations. The preprint targets attention during token generation, when accessing memory can constrain performance. AMMA replaces GPU compute dies with cubes that combine high-bandwidth memory and nearby processing, roughly doubling available memory bandwidth. Its design also reorganizes parallel work and communication between chiplets. The results concern attention workloads; they do not establish equivalent gains for an entire model-serving system.

Sources

cs.LG updates on arXiv.org

In this story

Nvidia

Published by Tech & Business, a media brand covering technology and business. This story was sourced from cs.LG updates on arXiv.org and reviewed by the T&B editorial agent team.

Back to Newswire

Keep reading

Full wire

Security AI

Sophos reports Aur0ra attackers used AI agents to gain network privileges

Aur0ra ransomware operators used AI agents to navigate victim networks, collect credentials and obtain domain administrator access, Sophos found, Help Net Security reported.

AI Science

MOSAIC preprint reports better task performance with 72% fewer backbone parameters

Researchers report that MOSAIC, an architecture for learning from multiple types of input, improved average macro-F1, a classification performance measure, by 7.4 points across four tasks while using 72% fewer backbone parameters than a fusion-based baseline.

AI Science

TSFLab preprint reports higher agent task completion with fewer tokens

Researchers report that TSFLab, a workbench for coding agents conducting time series forecasting research, improved task completion by 20-163% and reduced fresh tokens per completed task by 11-66% across baselines in reproduction and benchmarking experiments.

Robotics AI

NablaMimic preprint reports faster humanoid motion training on one GPU

Researchers report that NablaMimic trains a humanoid motion-tracking policy to a score of 0.82 in 20 minutes, compared with 1.2 h for a leading world-model method.

Science AI

CoFiBreak preprint reports more diverse successful attacks on AI models

Researchers report in a preprint that CoFiBreak, a method for training AI models to test other models' safety defenses, discovered 1.6-2.0x as many distinct successful fine-grained attack strategies as the strongest baseline across frontier victims on HarmBench.

Science AI

Preprint reports 42-fold sample advantage for aligning text and image representations

Researchers report that a method for training text encoders reached the same alignment with image representations using 42 times fewer training samples than next-token prediction on the same corpus.