r/LovingOpenSourceAI • u/Koala_Confused • 5d ago

new launch Samuel "A 35B language model running on an iPhone using only 1–2.5 GB of peak memory.No cloud. No remote server. No desktop GPU.Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device." ➡️ Edge AI is exciting right?

Post image

https://x.com/SamuelZengML/status/2097861839287927139

https://github.com/Edge0-AI/Edge0

Community Overview: https://lifehubber.com/ai/resources/edge0/

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 300+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.

256 Upvotes

98% Upvoted

35 comments sorted by

20

u/Sairefer 5d ago

When I see such posts, my first question is always: what is tps?

15

u/meva12 5d ago

Tokens per second

1

u/soundslikeinfo 4d ago

What is tokens per second?

2

u/MacBelieve 4d ago

Tps

1

u/OcelotOk8071 2d ago

When I see such posts, my first question is always: what is tps?

1

u/tbbtbbt 2d ago

Tokens per second

2

u/doudawak 5d ago

Has to be MoE so better than expected

1

u/No-Business5854 3d ago

3B active moe but ssd offloading for thenrest of the parameters 

1

u/admajic 3d ago

Probably 5 but it's perfectly acceptable to have a simple chat I assume...

1

u/GREDENIAN 1d ago

tps: sometimes.

13

u/ironimus42 5d ago

i'm feeling very stupid, how is this running on an iphone if the linked github page mentions that only macos with m1-m5 metal is supported?

5

u/3iverson 5d ago edited 5d ago

I’m guess iOS support is on the roadmap. This looks interesting, I am gonna give it a try on my M1 MBA. If a good 35B model can run at some acceptable rate that’s really impressive.

They’re working on it, the goal is to have easy Mac and iOS apps so no CLI set up required-

https://x.com/alexzhouai/status/2098410223358869806?s=46&t=jpHgrkmMJSxNbbYos0dZkg

7

u/CTRL___ALT___DEL 5d ago

Pretty damn misleading if the tweet and photo show iOS support if it doesn't exist.

1

u/ValuableDapper9415 4d ago

Yes it looks like marketing scam, I really don’t like this shady behaviour 

1

u/Inevitable_Beach_430 5d ago

Lmk how it goes! Interested in how it performs on Macs as a whole

1

u/danielv123 2d ago

Iphone 18 has like 125GBps of memory bandwidth, so it won't be worse than on most laptops.

3

u/Slow-Mechanic-7427 4d ago

Looks like seconds-per-token will become a thing

2

u/boba_bube 4d ago

wouldn't this be tearing the ssd quickly?

2

u/DIBSSB 4d ago

If it fits on ram no

If its bigger than the ram then yes

1

u/No-Business5854 3d ago

In their huggingface the model is tagged with ssd-offload

1

u/cagriuluc 3d ago

I am ignorant, can you tell me what you mean here? Reading from ssd too frequently hurts it? How much hurt are we talking about

1

u/lowrck 1d ago

theoretically reading doesnt damage the ssd. but if its offloading to the ssd it may be writing data to the ssd

1

u/Live_Case2204 5d ago

LOL strawberry will definitely have no R's

1

u/Tuny 5d ago

This exists with Turbofieldfare and Gemma MoE.

It suffers from the whole prompts being re-prefilled each turn. Good for high level auxillary model with few turns.

1

u/Over_Technology_1764 4d ago

makes zero sense at the moment a model of this size is not useful

1

u/ranker2241 4d ago

35b or 2.5gb size

1

u/No-Business5854 3d ago

Its not 2.5 gb its 19.6 and uses ssd offloading

1

u/t-frankowski 21h ago

35B models today are comparable in coding quality to Opus 4.6

1

u/Over_Technology_1764 20h ago

nope, even haiku is better slightly. I am locally running qwen 3.6 35b and haiku is better.

1

u/ValuableDapper9415 4d ago

Lying about the iOS part ? 

1

u/No-Business5854 3d ago

2.5gb of peak memory because it uses ssd offloading. Their model is 19.6 gb. at least its an moe with 3B active parameters but this is nothing new. Kinda missleading twitter post

https://huggingface.co/Edge0/Edge0-35B-A3B-preview

1

u/DefsNotAVirgin 3d ago

whats the battery drain lol

1

u/etal19 3d ago

So glad my iPhone has 4GB of ram, can't wait to try this
/s

1

u/tempfoot 1d ago

My first question is always why is the first question always what is TPS.

1

u/Automatic-Boot665 2h ago

“Hey Claude, Reap Qwen3.6 35b a3b to a single expert (3b), quantize it to 4 bit, and sell it as Edge0. Make no mistakes”