PyTorch post details open-sourced FP8 FlashAttention work for Blackwell

A PyTorch engineering post said code for an MXFP8 extension to FlashAttention-4 has been open sourced for Blackwell GPUs. The work adds block-scaled FP8 forward and backward attention and, on internal LLM shapes, reported gains of up to 1.6 times in forward performance and 1.52 times in backward performance versus BF16. The post said the accompanying zero-gather jagged module is used internally at Meta for GEM training. Reported performance is based on internal shapes and implementation-specific measurements.