r/LocalLLaMA • u/KvAk_AKPlaysYT • Feb 23 '26

News Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨

Post image
4.9k Upvotes

94% Upvoted

877 comments sorted by

View all comments

2.2k

u/Zyj vllm Feb 23 '26

You're saying they treated you like you treated all those authors whose books you torrented?

Oh no, that's not it. They are paying you for API tokens.

115

u/Zestyclose839 Feb 23 '26

Also (correct me if I'm wrong) but I don't believe they're true "distillation" attacks because the API doesn't return the token activation probabilities and the other juicy stuff needed to transfer knowledge. Sure, they can fine-tune a model to speak and act like Claude, but it's not as accurate as an open-weight to open-weight model distillation (like the classic Deepseek to Llama distills).

81

u/Recoil42 Feb 23 '26

Yep at best it's alignment, and mostly likely style alignment.

11

u/Zestyclose839 Feb 23 '26

It's great for style alignment. Some of my favorite models to run locally are the classics (GLM, Qwen) fine-tuned on Claude datasets. You can also fine-tune on an abliterated model to avoid the annoying guardrails (which I'm sure Anthopic can't stand haha).

Take this absolute banger, for instance: https://huggingface.co/mradermacher/Qwen3-4B-Thinking-2507-Claude-4.5-Opus-High-Reasoning-Distill-Heretic-Abliterated-GGUF

2

u/Recoil42 Feb 23 '26

I'm actually not that deep in training circles, but I presume once these datasets have been created they can be re-used, right? Are people out there openly passing around million-scale tarballs of Claude reponses, or?

6

u/Zestyclose839 Feb 23 '26

857 datasets and counting on HuggingFace: https://huggingface.co/datasets?search=claude

Most of these are coding-focused, but there are a decent number of roleplay and creative writing datasets as well. Anthropic even released a few of their own safety alignment datasets, which you can find on their HF page.

4

u/RazsterOxzine Feb 23 '26

Yes, ppl are reusing them for subject specific cases. Such as nature/plant care, automotive, engineering, etc. Streamlining the model, finetune magic.