Flo reports improved health-model scores after open-model fine-tuning

Flo Health improved the share of model responses meeting both its medical-safety and usefulness criteria from 16% to 47% through fine-tuning, Databricks said in a customer case study. Medical-safety scores rose from 34% to approximately 61%, while usefulness increased from 42% to 76%. Flo used 26 AI judges and more than 15,000 synthetic questions to score candidate responses and train open models on 32 NVIDIA H100 GPUs. Clinical reviewers adjudicated disagreements between AI judges and initial human assessments. An initial training approach improved usefulness while leaving medical safety relatively flat. Flo then introduced graded safety scoring to distinguish stronger responses. The final combined score still left more than half of responses short of both criteria.