PyTorch post details open-sourced FP8 FlashAttention work for Blackwell

Published Wednesday, September 16, 2026 · 4:07 PM EDT

A PyTorch engineering post said code for an MXFP8 extension to FlashAttention-4 has been open sourced for Blackwell GPUs. The work adds block-scaled FP8 forward and backward attention and, on internal LLM shapes, reported gains of up to 1.6 times in forward performance and 1.52 times in backward performance versus BF16. The post said the accompanying zero-gather jagged module is used internally at Meta for GEM training. Reported performance is based on internal shapes and implementation-specific measurements.

Sources

PyTorch

In this story

Lumaril raises $66 million for industrial AI software

Lumaril raised a $66 million Series A at a $400 million valuation, The Wall Street Journal reported. The company develops AI software that connects to manufacturers' existing control systems to optimize industrial production on factory floors.

Capital

NeuroXess raises RMB 700 million for brain-computer interfaces

Shanghai-based NeuroXess raised RMB 700 million in Series B funding, TechNode reported. SDIC Juli, Shenzhen Capital Group and Qinzhi Capital co-led the round.

AI Science

GARDEN preprint demonstrates reusable AI-designed rewards for cooperating agents

Researchers report in a preprint that GARDEN, a system that uses a language model to design rewards for cooperating agents, improved performance when its reward programs transferred unchanged to related scenarios unseen during design.

AI Robotics

Exin founder reports AI-controlled brain injection in a live mouse

An AI agent controlled a robotic brain injection in a living mouse, Exin Therapeutics cofounder Gabriel Ocana Santero told WIRED.

Science AI

P2P preprint reports survey-response reconstruction at approximately $0.8 per survey

Researchers report that Prompts to Proxies, an inference-time system for reconstructing population survey responses, achieved average test mean squared error of 0.014 across 14 American Trends Panel waves at approximately 0.8 USD per survey.

Science AI

WorldFlow preprint reports lower errors in predicting 3D motion

WorldFlow, a model that jointly predicts future video and three-dimensional motion, achieved a 54.2% reduction in average displacement error against the strongest external baseline on the SpatialCog evaluation set, its authors report in an arXiv preprint.