# Preprint tests human-guided audits to correct AI toxicity measurement bias

_Published Monday, October 19, 2026 at 5:47 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report a method called Threadsafe that uses targeted human labeling to correct biased estimates of conversation toxicity produced by language models. In simulated audits, its uncertainty intervals maintained their intended coverage from 250 human labels onward; with 1,000 to 8,000 labels, intervals were 0.75 times as wide as those from design-based supervised learning.

The preprint finds that all seven tested language models excessively flag comments containing "you." Across 1.25 million news comments, Claude Haiku's labels suggested toxicity rose with reply depth, while crowd labels showed a slight decline. Threadsafe uses model scores to select comments for human review and correct predictions. A thousand human labels ruled out Haiku's rising trend.

## Sources

- [cs.CL updates on arXiv.org](https://arxiv.org/abs/2610.23446)

---
Canonical: https://techandbusiness.org/newswire/NYXKb7wioNOpWzPdR_JbIf
Published: 2026-10-19T09:47:40.760Z
Story chronology: 2026-10-19T04:00:00.000Z
Retrieved: 2026-10-19T12:04:48.945Z
Publisher: Tech & Business (techandbusiness.org)