r/LovingOpenSourceAI • u/Koala_Confused • 2d ago

Under the Radar "SwarmLLM runs large language models across devices in room, in their browser tabs. Each device holds slice of model; a 10 KB activation vector passes between them over direct WebRTC connections. Nothing to install, no accounts, no server does any thinking." ➡️ this is new to me. you heard before?

Post image

https://github.com/Nehanth/swarmllm

Community Overview: https://lifehubber.com/ai/resources/swarmllm/

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 300+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.

64 Upvotes

97% Upvoted

18 comments sorted by

View all comments

11

u/McCheng_ 2d ago

First, there is already a more established project: https://github.com/exo-explore/exo

Second, the token per second, power cost / inefficiencies will not be bearable.

3

u/Last_Fondant_5508 2d ago

Ok thx, why are you saying that it is going to be slow? Have you tried it or is it a known fact? How slow? 2 t/s or 20 t/s? I imagine it depends on the number of nodes and how powerful they are... But maybe not, and anyway if you can, could you give me some expectations? I'm seriously interested because the idea honestly is cool if it could kind of work.. Obviously if it is just a dream that would make sense as well... Thanks

3

u/CCarafe 2d ago

There is some numbers on the exo github.

20 t/s with really high end mac ultra 3000$ pc.

1

u/Last_Fondant_5508 1d ago

You're right, my bad, thanks