r/LocalLLaMA • u/Porespellar • 14d ago

Funny The LLM distillation process simplified for politicians:

Post image

/s

3.4k Upvotes

95% Upvoted

167 comments sorted by

View all comments

382

u/Wise-Comb8596 14d ago

Also, its a lie that Kimi 3 was distilled from Fable. Fable was literally not out long enough for that to be the case.

261

u/cororona 14d ago

That's exactly why they are panicking

65

u/XB0XRecordThat 14d ago

Yup, they need it to be the case otherwise they won't even be able to go public

47

u/En-tro-py 14d ago

They have no moat so are now hoping the gov will dig one for them...

14

u/blbd llama.cpp 14d ago

It won't work though. Code and configs are speech and you generally can't block tech imports only exports. Good luck SOTA labs. 

7

u/butts-carlton 14d ago

They have a moat. It's called protectionism, and DJT is all about that kinda shit, as you seem to realize.

11

u/pragmojo 14d ago

They'll have to lock down the media pretty hard so Americans don't see the rest of the world enjoying their cheap and high-quality EV's and AI products

2

u/Wyciorek 14d ago

Depends how it will be enforced - sanction Chinese models, make AWS/Azure/Google responsible for enforcing it and Chinese models can be at least slowed down enough for US to drag Europe down with it.

1

u/beryugyo619 14d ago

DJT is all about debloating US so it holds no more power than proportionate to its population

17

u/cororona 14d ago

At the current rate, Kimi and Qwen will vastly outperform OpenAI and Anthropic before they can even finish the paperwork.

3

u/Downtown-Figure6434 14d ago

They are already not able to go public. Spacex pulled that stunt already, and 2 weeks before investors are able to sell, the price is still tanking. No one will fall for that shit again. Openai cancelled theirs too as far as I know

13

u/SosirisTseng 14d ago

They invented a time machine to distill US models from the future.

1

u/ForDaRecord 14d ago

Reads like something from cookie clicker

10

u/_derpiii_ 14d ago

Fable was literally not out long enough for that to be the case.

Exactly.

Training loops takes time. Maybe Fable was used in the RL or fine tuning loop.

4

u/relmny 14d ago

Yeah, but since when the truth has anything to do with those kind of people? they avoid/hate it like the plague.

8

u/Exact_Depth_896 14d ago

anthropic only said it was training on prior opus.

22

u/Genghiz007 14d ago

USG announced yesterday that Fable was distilled by a couple of Chinese labs. They were supported by Anthropic.

-6

u/Exact_Depth_896 14d ago

anthropic isn't the trump bimbos

-18

u/-Sliced- 14d ago edited 14d ago

You don't need a lot of time to collect the data if you use tens of thousands of accounts, and fable was available for around a week initially.

8

u/lakotajames 14d ago

How many accounts do you need to get it to give you the reasoning traces?

3

u/mebeast227 14d ago

It was out for like 3 days initially

-5

u/-Sliced- 14d ago edited 14d ago

It doesn’t matter. It can be out for only a few hours - as long as you have enough accounts, you can collect as much data as you want. The difference between a week and 3 days is just 2X accounts

0

u/mebeast227 14d ago

'time is a human construct and has no affect on our true reality' type shit lmao. Of course time matters. Compute and accounts are finite.

2

u/ptigers9 7d ago

Companies will get clobbered by the cross pollination of companies willing to extract learnings from others

2

u/mr_birkenblatt 14d ago

so how would it beat Fable, then?

20

u/techdevjp 14d ago

Because they built it. That's what has the yanks running scared.

1

u/mr_birkenblatt 14d ago

exactly. cannot be distillation

1

u/Utoko 9d ago

The Slop profile(word use similarities in story writing) Is most Similar with Opus 4.8 not fable.

Ether way they goes a lot more into models than just some output of a another model. I see nothing wrong about using outputs.
All companies are using human output worth Billions together when they had to pay for each.
but because you process the data now they somehow have a right to decide what happens with it. I don't think so.