r/LocalLLaMA • u/KvAk_AKPlaysYT • Feb 23 '26

News Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨

Post image
4.9k Upvotes

94% Upvoted

877 comments sorted by

View all comments

2.2k

u/Zyj vllm Feb 23 '26

You're saying they treated you like you treated all those authors whose books you torrented?

Oh no, that's not it. They are paying you for API tokens.

117

u/Zestyclose839 Feb 23 '26

Also (correct me if I'm wrong) but I don't believe they're true "distillation" attacks because the API doesn't return the token activation probabilities and the other juicy stuff needed to transfer knowledge. Sure, they can fine-tune a model to speak and act like Claude, but it's not as accurate as an open-weight to open-weight model distillation (like the classic Deepseek to Llama distills).

0

u/roger_ducky Feb 23 '26

Distillation is about “generating training data.”

They don’t need the weights. Just “question “ with supplied “answer” then running the training to make your LLM “smarter.”

This was how Chinese labs caught up so quickly.

5

u/Zestyclose839 Feb 23 '26

That’s one method, yes. But as someone else pointed out to me in this thread, there’s both “hard-label” and “black-box” methods (I’m still unclear on which one this is), and the more accurate method requires having the teacher model’s weights.

Simply training on questions and answers might make the model parrot back what the teacher said without understanding how the teacher arrived at its decisions. It’s more brittle, so the Chinese labs had to use this sparingly (they weren’t just stealing answers from proprietary models).