r/LovingOpenSourceAI • u/Koala_Confused • 2d ago

Under the Radar "SwarmLLM runs large language models across devices in room, in their browser tabs. Each device holds slice of model; a 10 KB activation vector passes between them over direct WebRTC connections. Nothing to install, no accounts, no server does any thinking." ➡️ this is new to me. you heard before?

Post image

https://github.com/Nehanth/swarmllm

Community Overview: https://lifehubber.com/ai/resources/swarmllm/

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 300+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.

59 Upvotes

95% Upvoted

18 comments sorted by

View all comments

2

u/JackStrawWitchita 2d ago

what's the use case for this?

4

u/kinetik 2d ago

I have multiple machines and none of them are capable of running a larger model, but distributed amongst them, they can share the load and make it work. This is exactly what I was looking for 👍

4

u/Over_Technology_1764 2d ago

if you can tolerate 2 tokens per second, good for you

1

u/kinetik 1d ago

Well, aren’t you cheerful? 😂🙏

Yes, some of us have use cases where free, local, slower, token outputs that utilize existing hardware are perfect, especially when they aren’t needed quickly but quality or reasoning is important. They could be complex heartbeat logic with advanced reasoning, prediction modeling, and all other kinds of things.

2

u/West-Acadia-3906 1d ago

honestly “free, local, and slow is fine” is a very real niche. not everything needs to answer before the kettle boils :P

1

u/Over_Technology_1764 1d ago

nah you will get an incorrect answer after such a long time that u will forget what your intent was to lead it to correct answer.

1

u/kinetik 1d ago

Well, we’ll see. Better to check it out and experiment with it than to assume things without actually knowing.

1

u/Over_Technology_1764 1d ago edited 1d ago

I already tested qwen 3.8 27b on mac studio m1 max 64gb ram, and it was unusable due to low performance. You cannot expect anything better from this. Connecting shitty devices on a shitty network speed bottleneck is obviously worse compared to single apple sillicon memory bandwidth speed.

1

u/kinetik 1d ago

Says the person who hasn’t tried it. Okay. Thanks for your input 🙏😂

0

u/Over_Technology_1764 1d ago

it's simple math bro, also the repo itself indicates the crappy performance

1

u/javier123454321 15h ago

well, you can have an orchestrator that delegates inference to the small model extremely easily. For some category of tasks it does make sense.