Four-lab round-robin shows experimental drift can break AI catalyst models

Researchers convened four U.S. laboratories to test the same experimental carbon monoxide-producing catalyst and found that small differences in protocol can produce incompatible data for AI models, according to a SLAC National Accelerator Laboratory writeup on phys.org. The teams published results in Nature Catalysis. They ran round-robin tests with agreed protocols and the same rhodium-based catalyst aimed at a key step in turning carbon dioxide into fuels. Each group reported different amounts of carbon monoxide and methane, an undesirable side product, so a model could not learn from the four mismatched outcomes. After further standardization across SLAC, Pennsylvania State University, Stanford University, and the University of California, Santa Barbara, results became more consistent. One of the largest sources of variability was how hard the mixture was shaken or stirred. First author Selin Bac of UC Santa Barbara said the findings are a reminder to exercise caution about what information feeds a machine-learning model. Senior author Adam Hoffman of SLAC said the work is meant as a guide for designing experiments intended for machine learning models. The paper is DOI: 10.1038/s41929-026-01559-y.