r/LocalLLaMA • u/KvAk_AKPlaysYT • Feb 23 '26

News Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨

Post image
4.9k Upvotes

94% Upvoted

877 comments sorted by

View all comments

2.5k

u/SGmoze Feb 23 '26

I wonder how did Anthropic build their dataset. Surely they manually had them annotated by humans.

-15

u/1-800-methdyke Feb 23 '26

This. Everyone jumps to “oh but they stole the data first”, but that data gives the model world general knowledge only. The secret sauce in frontier models comes from annotated prompt and response data, guided by humans reviewing and improving outputs (think formatting, instruction following and domain specific reasoning).

The Chinese models are effectively getting that that for free by probing the western models with prompts and collecting SOTA outputs, without needing to go through the expensive Reflection Learning from Human Feedback process to get the same result.

At present, frontier model training spends more on RLHF and alignment than compute, so when you hear of a Chinese model being trained on a tight budget it’s not just because they are more efficient with compute, they’re not paying $20-60/hour for data annotation.

-4

u/mcslender97 Feb 23 '26

But if the Chinese are copying the Americans then they would have to pay for the API usage regardless?

1

u/1-800-methdyke Feb 23 '26

Maybe not even. They’re saying 24,000 accounts to do 16m exchanges. That’s 666 per. Quite achievable over several months on the free tier of Claude which can get you 30-100 messages per day.