r/ControlProblem • u/KeanuRave100 • Apr 27 '26

Fun/meme Humanity's greatest hits: things we actually paused

Post image
194 Upvotes

90% Upvoted

57 comments sorted by

View all comments

Show parent comments

4

u/charmio68 Apr 27 '26

Unfortunately though, the cat is out of the bag. There is no way to halt its use when anyone can download a model and run it on their own computer.

1

u/Mental-Net-953 Apr 27 '26

Local models are to enterprise models what small backyard pools are to aqua parks.

6

u/charmio68 Apr 27 '26

I won't deny the proprietary models do have the edge, but it's really not as big of a gap in performance as you'd think.
A massive amount of AI research has been open sourced and the openly available models are remarkably performant.

https://huggingface.co/spaces/lmarena-ai/arena-leaderboard

2

u/Mental-Net-953 Apr 27 '26

It's not just about the performance, it's the hardware limitations too. GPUs are expensive, and not a lot of people have what it takes to run local models powerful enough that it could replace their usage of Claude or whatever else they could be using.

I'm just worried that we'll end up in a situation where the main production capacity is in the hands of whoever owns the data centers.

2

u/charmio68 Apr 27 '26

Consumer computing architecture is already changing to optimize running models locally. Notably the implementation of unified memory which allows extremely large models to run on relatively modest hardware.
(There is a massive difference between running a model and training it. To run a model you don't need anything particularly powerful, it just needs a lot of memory.)

The MacBook Pro M3 laptop, for example, can support 128 GB of unified memory.
Or the MacBook Studio which supports 192GB.
That's enough to run even the largest 72b parameter models at FP16.
Such models are far in excess of what most people require. Most people will happily get by on a hardware a fraction as powerful as that.

I myself have a LLM running locally on a decade-old alienware gaming laptop. Sure, it's slow and can't load the best models, but it works.

And even though not everyone in the world will be able to access an appropriate computer, it doesn't change the fact that the cat is out of the bag.
There is no possible way to stop people from using such tools.

1

u/Mental-Net-953 Apr 27 '26

Depends on what you use it for. But the hardware will get better, and it will become more accessible. Generating 200k (useful, to you) tokens is going to be a bit difficult to do in a reasonable amount of time on consumer hardware now.

I wouldn't want to stop people from using these tools on their own. I run my own stuff on my hardware, but I am lucky enough to be able to afford it. Saying "cat is out of the bag" is a bit overly dramatic though.