Anthropic, OpenAI, Google & Microsoft Back Unified AI Robots.txt Standard
In a rare joint accord, OpenAI, Anthropic, Google, and Microsoft have backed a unified robots.txt standard to let site owners opt out of model training without killing search discovery.
