FTW preprint matches agent-training baselines without repeated group trials

Researchers report in a preprint that Follow the Winners, an algorithm for training language-model agents, matches GRPO and PPO on Sokoban and Search-R1 baselines while substituting stored samples in CPU memory for a value model or repeated groups of trials. The method ranks samples from a replay buffer by their returns and filters them to guide training. It targets environments such as live services and security sandboxes, where repeating agent trajectories can be impractical. The researchers identify a bounded preference for riskier outcomes as a tradeoff of this filtering approach, which FTW controls.