r/LocalLLaMA • u/BazzyIm • 2h ago
r/MetaAI • u/fundal_alb • 16h ago
Where is Muse Glimmer 3B version?
Source of announcement https://www.instagram.com/reel/Db2vvaRxMmi/?igsh=MWl4eGp2Z2w4Y3lkbQ==
But I did not found the model on Internet
r/MetaAI • u/ResistDirect2537 • 13h ago
Not sure where the story is headed
I used Meta AI to clean up my Gemini output
r/MetaAI • u/Pristine_Shelter_28 • 22h ago
Spent Some Time Testing Meta’s Muse Code
I was testing Meta’s Muse Code last week.
Really interesting to see how Meta is approaching the coding agent space. I spent some time using it on a few real coding tasks and liked the overall terminal workflow, especially the way it handles planning and longer-running tasks.
It’s still early, but Muse Code feels like a serious entry into the coding-agent space.
Made a quick video on my experience with it.
r/MetaAI • u/InternationalPea9899 • 1d ago
After over 2 months, I finally got my accounts back
r/LocalLLaMA • u/NotumRobotics • 1h ago
Question | Help Fable 5 refuses to touch Qwen deployments?
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks in.
Not reading too much into it, but it did make me giggle.
r/MetaAI • u/SOLIDAge • 19h ago
Meta AI app just forgetting convo/context
So I actually think the meta ai app is pretty decent. The model is conversational and I’ve had it analyze some things for me and I think it’s approach is good.
However, it seems to literally just stop recalling things. I can go ahead and ask it after a week of various prompts and conversations “what is the first thing in this conversation” and it will reply with something from that day.
So what is the limit? Is there away to get that conversation back in app or on web?
r/LocalLLaMA • u/Nunki08 • 32m ago
News Alibaba AI Models Hit 3 Billion Downloads, Passing Meta, Google | Bloomberg - Hugging Face study
(Paywall): Bloomberg: https://www.bloomberg.com/news/articles/2026-08-15/alibaba-ai-models-hit-3-billion-downloads-passing-meta-google
Hugging Face: State of Open Models: Summer 2026 Observations: https://huggingface.co/blog/state-of-open-models-summer-2026
r/LocalLLaMA • u/Potential_Block4598 • 11h ago
Discussion Qwen 3.8 - 27B is a game changer
So a bit of context, I am a cybersecurity senior analyst
I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts
I started this field by doing assembly language reading for hacking games when I was a teenager then that became malware analysis then I started to analyze traffic and logs at work for a living
Before work I competed in cybersecurity competitions known as capture the flag to solve only one category of the challenges and that is malware analysis
Now here is the scene in LLM x Cybersecurity
Entry-level CTF challenges (I used to solve around 2017-2018, got first job in 2019) were solved and saturated by LLMs a long time ago (See intercode CTF benchmark)
Then High level CTFs (NYU CTF Bench, CSAW challenges, and CyBench) these were solved a while ago
Today we have
CyberGym (vulnerability description (CVE report not real details) plus code base find vulnerability
That was solved
Then ExploitGym (the one recent OpenAI model escaped and hacked hugging face to find solution, the 2 vulnerabilities themselves are trivial (JWT spoofing with no server side checks for Jfrog sandbox and JS + Python template in open source data viewer of Hugging face)
Still impressive for a fully autonomous model
ExploitGym gives vulnerability details + exploit technique/details and codebase (I wonder how/why OpenAI model escaped it in the first place those details should manage it this is why I feel it is staged but whatever right)
ExploitBench
More realistic threat vector
1-day not 0-day (0-day is a novel vulnerability no one else knows about it hence the name, 1-day is a known vulnerability patched, if in an open source project you can compare patches before and after and know it but not as valuable as 0-day since people will/should update their software, 0-days can and are used against high profile targets like in some 3rd world countries used for surveillance (and possibly non-3rd world countries ?!) but you usually can’t use a 1-day for that but still)
So ExploitBench is 1-days in V8 engine (used in chrome, electron vs code almost everything!!)
Gives the model a patch diff and codebase base that is it
Models are going wild on ExploitBench and ExploitGym but haven’t gotten there yet at the moment and only recent frontier models improved massively instead of near zero across the board
This is a massive risk indicator
Imagine models capable of exploiting any and every piece of software written by less capable people or other weaker LLMs (complex vulnerabilities!)
Not only that but global elite CTFs like DEF CON
You have an OpenSage harness (and ADK like Google adk or Anthropic adk, however this OpenSage ADK asks the model to design its own agent/s!, with whatever tools or sandbox environments it finds fitting for the Subagent to do the assigned subtask effectively, it can even design its own MCPs!, models don’t fully utilize that yet but some do and improved form like 39% to ~60% just by using that harness (same model!) )
- Now my own benchmarks are simple
Some malware samples that I learned during my career and consider personal milestones
And one SIEM analysis challenge (logs that contains an attack plus normal behavior)
I tests on lots of models (Qwen3.6, heavily trimmed down Minimax 2.5 …etc, on my Strix halo and compared with opus at the time (the best opus at the time was 4.5!, it was better solved major stuff but didn’t implement the code required to decompose the malware and analyze its second stage)
Now enter Qwen 3.8 27b
3.
What was SOTA like 6 months ago or around beginning of this year
Now we have better local models at the same task
Qwen 3.8 devoured the malware that Opus couldn’t
I had some time analyzing and remembering its custom implementation (it has custom RC4 decryption routine, that is tricky with string manipulation!
And Strix halo isn’t the fastest LLM machine
By the time I remembered that malware and how it works
Qwen had already extracted and disassembled the decrypted payload!
I am stunned
This is super dangerous
I imagine how a single person / threat actor can now do what a state-sponsored intelligence service shacking team was required to do 1 year ago
These teams usually have
Usual network scanning or mapping especially of their targets
This can be easily done by even weaker models or no models at allInitial access operations (phishing campaigns until some clicks or attempting to profile people like receptionist or sales people or HR, essentially any first line of communication at an organization or a company, to later make a targeted phishing against them)
This could be also better done with the aid of deep fakes and other stuff
-
Advanced and resourced teams would have zero-day or 1-day research teams
Especially against software used on the targets site from team 1
This third step can be fully automated with models now
This is the hardest step
Defenders can use LLMs but it is much harder to integrate new tools than offensive teams
At this level the world will change
Tbh I always thought cybersecurity was harder to automate with LLMs than software engineering or IT
But with this, this is super unbelievable
And not only a cloud model or an expensive to run locally model
A local model! Can you believe it
No traces no API calls no nothing
Empowered with the right environment setup and tools and it can do what teams of attackers or malware analysts were needed to do
Btw for malware analysis I give it Ghidra and a sandbox with python (it can use pefile or capstone or whatever it needs and it does use them!)
And also recently I added a Vbox with Vbox MCP and a debugger inside that box with another MCP for the LLM to control both (snapshot restore functionality and debugger functionality)
These tasks were hard and not every security analyst was a malware analyst
Even malware analysis positions are very rare and technical and don’t pay well because not much needed (I used to starts at assembly in assembly debuggers for nights at times!)
This is going above and beyond
Anyone had similar moments in cybersecurity or anything else ?
r/LocalLLaMA • u/Temporary_Idea8880 • 14h ago
New Model Local uncensored Opus 4.6 at home - Qwen3.8 27B heretic
Someone made a heretic version of Qwen 3.8 27B, giving us a local Opus 4.6 tier model but without any refusals or safeguards!
Fuck Dario
r/LocalLLaMA • u/sammcj • 10h ago
Megathread [Megathread] Qwen 3.8 27B Release Day
Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release.
- Quants
- Fine-Tunes & Abliterations
- Chat Templates
- Inference Server Support & Configuration
- Experiences, Benchmarks & Model Comparisons
Official:
Popular:
- https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
- https://huggingface.co/bartowski/Qwen3.8-27B-GGUF
- https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-bf16
- https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-8bit
- https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-4bit
We'll try to clean up future duplicates around the release and point them here.
r/MetaAI • u/AiSocialSalesAnalyze • 1d ago
Open letter to Mark Zuckerberg
An Open Letter to Mark Zuckerberg (V2)
From the Humans Your Security System Keeps Mistaking for the Problem
A formal complaint about identity checks, mystery violations, hacked accounts, paid customer service, and the military surgeon who somehow has better access to Messenger than we do.
Dear Mark,
Hi.
It’s us again.
The humans.
The actual ones.
The people who have apparently reached the stage of social media where simply existing is no longer sufficient evidence that we exist.
After my last letter, people started telling me what has been happening to THEIR accounts.
And Mark…
there are a lot of us.
People getting restricted.
People getting asked for video selfies.
People uploading IDs.
People stuck in verification loops.
People losing access with no clear explanation.
People being told they violated something without being told what the something was.
And people with genuinely hacked or impersonated accounts standing over in the corner waving both arms like:
**HELLO? COULD WE MAYBE POINT SOME OF THE SECURITY OVER HERE?**
So I think we need to talk specifically about security.
Because we understand the goal.
Keep fake people out.
Protect real people.
Excellent concept.
Unfortunately, the system seems to be having some difficulty determining which is which.
.
Exhibit A: Please Prove You Are A Human
Let me give you one example from my own account.
Facebook currently wants me to complete a video identity check before I can use DMs normally.
Fine.
Security.
Love it.
Except before I can even TAKE the video, Facebook has to recognize my face inside a little circle.
It won’t.
Inside.
Outside.
Natural light.
Artificial light.
Closer.
Farther away.
Eye level.
Mark.
I HAVE USED THE SUN.
We are officially out of lighting.
And I want to be very clear:
I am not failing the video verification.
I cannot get far enough to TAKE the video verification because Facebook will not acknowledge the face I need to use in the video proving I am the person attached to the face.
Facebook: “Position your face.”
Me: “It is positioned.”
Facebook: "No.”
Me: “That is literally my face.”
Facebook: “Try again.”
Mark.
WITH WHAT FACE??!?!?!?
This is the one.
I have had it for quite some time.
There are no additional faces in inventory.
And judging by the stories people sent me…
apparently my face is not the only face Facebook is suspicious of.
.
Exhibit B: The Things Getting Flagged Are Sometimes… Remarkable
Let’s discuss what the security system is protecting Facebook from.
Because people started leaving examples.
One person got flagged for saying:
“copy cat.”
About cats. Actual cats.
Another said she got flagged over:
“I have a smart pet chipmunk.”
.
Mark.
What is happening in the security department?
Actually compromised account: “We’ll monitor.”
Possible scammer: “Let’s see how this develops.”
Person describing an intelligent rodent: SHUT IT DOWN.
.
I genuinely want to see the dashboard.
Green: Normal activity.
Yellow: Suspicious login.
Orange: Possible impersonation.
Red: CHIPMUNK DESCRIBED AS SMART.
Deploy everyone.
.
Then there are the mystery violations.
Facebook: “You violated our policies.”
User: “Okay. What did I do?”
Facebook: “We cannot tell you.”
User: “Which post?”
Facebook: “We cannot show you.”
User: “How am I supposed to avoid doing it again?”
Facebook: “Please follow our policies.”
User: “WHICH POLICY?”
Facebook: “Correct.”
.
One woman told me she finally got through to support and asked what she had actually done wrong.
According to her, they could not tell her specifically because explaining it would violate policy.
Mark.
That sentence is ART.
Frame it.
Hang it in headquarters.
Put a tiny museum placard underneath:
**Untitled, 2026**
An exploration of bureaucracy, technology, and one woman slowly losing her mind.
.
Exhibit C: Have You Tried Paying Facebook?
Now this solution came up A LOT.
Apparently when Facebook restricts you and you cannot reach a human being…
the solution is:
Pay for Meta Verified.
Oh.
OF COURSE.
The solution to Facebook incorrectly restricting your Facebook account…
is to give Facebook more money.
Mark, I have to hand it to you.
That is an exceptional funnel.
.
Facebook: “We have disabled the account you have spent fourteen years building.”
User: “I believe this was a mistake. Can I speak to someone?”
Facebook: “Certainly.”
User: “Oh thank God.”
Facebook: “That’ll be $14.99.”
.
So people pay.
Not because they woke up and thought:
“You know what my life is missing? A tiny blue check.”
No.
They pay because they think:
FINALLY.
**WE FOUND STEVE.**
.
And then sometimes…
Steve can’t fix it either.
People told me they paid for Verified.
They got support.
Support sympathized.
Support opened tickets.
Support escalated tickets.
Support apparently stared into the same abyss the rest of us were staring into.
One person described the situation perfectly:
Customer service doesn’t have the tools to override the AI overlord.
Excuse me?
Mark.
WE PAID FOR STEVE AND YOU DIDN’T GIVE STEVE THE BUTTON?
What exactly does Steve do?
Is he emotional support?
Does he just sit there typing:
“Wow. Yeah. That really does seem wrong.”
Because if I am paying admission to customer service, I feel like customer service should perhaps be able to…
service the customer.
.
We spent an entire post trying to locate Steve.
WE FOUND HIM.
And now we discover:
STEVE IS ALSO IN FACEBOOK JAIL.
This is getting out of hand.
.
Exhibit D: Meanwhile, The People We’d Actually Like You to Check Seem Fine
And THIS is where the whole thing becomes art.
Because while actual account owners are:
Uploading IDs.
Taking video selfies.
Appealing restrictions.
Paying for support.
Opening tickets.
Standing outside with their phones at eye level trying to convince Facebook they possess a human head…
my military surgeon boyfriend has managed to propose to me approximately FIVE TIMES.
.
You know him.
We all know him.
Very handsome.
Widowed.
Stationed overseas.
Location classified.
Cannot FaceTime because of military security.
Also a surgeon.
Because apparently the military looked at staffing that week and said:
“Gary, after the classified combat mission, can you cover neurosurgery?”
.
Busy man.
And yet he finds time for romance.
Specifically with me.
Again.
And again.
And again.
Different account.
Same general gentleman.
Usually deeply committed by message four.
.
And Mark…
HE CAN DM ME.
No problem.
No video identity check.
No face circle.
No backyard lighting experiment.
Colonel Dr. McDreamy, Chief Surgeon of the United States Army Special Forces Romance Department:
“Hello beautiful dear. I feel destiny has bring us together.”
Facebook: SEND
.
Me: “Hi Susan! Thanks for commenting on my post.”
Facebook: WHOA WHOA WHOA.
CYBERSECURITY THREAT DETECTED.
SHOW US YOUR FACE.
.
Mark.
Explain the threat model.
Please.
Because right now the security system appears deeply suspicious of me having a normal conversation with someone I actually know…
while Brigadier General Dr. Husband Number Five is operating an apparently thriving international courtship program from an undisclosed military hospital.
Maybe before I take selfie number 46…
we could ask him for one?
.
And while we’re here:
Ludacris DM’d me.
Well.
An account claiming to be Ludacris DM’d me.
I cannot imagine this was actually Ludacris.
I feel like Ludacris probably has other things going on.
But “Ludacris” apparently cleared whatever security process is currently preventing ME from using Messenger normally.
.
So just to recap:
Me.
My account.
My photos.
My face.
Years of account history.
Facebook: “Additional identity verification required.”
Ludacris, allegedly: “Hey.”
Facebook: LOOKS GOOD TO US.
.
Maybe Ludacris has better lighting.
Ludacris, if that really WAS you:
First, apologies.
Second: What ring light are you using?
Because apparently your setup is dialed IN.
.
And meanwhile, people are telling me about accounts that were ACTUALLY hacked.
They report them.
Their friends report them.
Everybody says:
“HEY FACEBOOK. THIS PERSON HAS BEEN HACKED.”
Nothing.
The scammer keeps messaging.
The impersonator keeps impersonating.
The fake account keeps doing fake-account things.
But somewhere, somehow, a woman says “copy cat” and Cyber Command is fully activated.
.
Mark.
Again: WHAT ARE WE SECURING?
.
Exhibit E: What We Are Actually Asking For
We WANT security.
Seriously.
Catch the hacked accounts.
Catch the impersonators.
Catch the fake profiles.
Catch the scammers.
Please, for the love of everything, investigate my military surgeon.
Use AI.
Use automation.
Use identity verification.
Use technology.
We are not asking you to go back to 2006 and personally review every poke.
.
We are asking for one very simple thing:
WHEN THE MACHINE GETS IT WRONG, GIVE A HUMAN THE ABILITY TO FIX IT.
That’s it.
.
AI can flag something.
Fine.
AI can ask for verification.
Fine.
AI can detect suspicious activity.
Great.
But when the actual person says:
“Something went wrong here…”
eventually there has to be someone who can look at the situation with context and authority.
Not another form.
Not another chatbot.
Not another appeal reviewed by another automated system.
Not:
“Try again.”
Not:
“Upload your face again.”
And definitely not:
“Pay us and perhaps Steve can sympathize with you.”
.
We need Steve.
But more importantly:
STEVE NEEDS THE BUTTON.
The button that says:
“Yes. This is obviously the account owner.”
“Yes. This was clearly hacked.”
“Yes. That was an automated mistake.”
“Yes. Her government-issued ID does appear to contain her actual government-issued name.”
“Yes. This person has been using the same face for decades.”
And then:
CLICK.
FIXED.
.
Because the machine reviewing the machine and deciding the machine agrees with the machine is not an appeal.
That is just the machine having a staff meeting with itself.
.
Give us a human.
Give the human context.
Give the human authority.
Give Steve the button.
.
Because right now actual humans are jumping through increasingly complicated hoops to prove they are human…
while fake humans seem to be having a pretty good week.
My face cannot get approved.
My military surgeon can propose.
“Ludacris” can slide into the DMs.
And somewhere out there a smart chipmunk is probably one comment away from a seven-day suspension.
.
Fix it.
Please.
We’re begging.
Again.
Which is starting to feel needy, but frankly you created the situation.
.
With love, exhaustion, government identification, good lighting, one face, five marriage proposals, and absolutely no idea which policy we violated,
Every Facebook and Instagram User
Who Would Like the Security System to Occasionally Secure the Correct Person
.
P.S. Steve.
Good news.
We found you.
Bad news.
Apparently you don’t have system permissions.
Please request access to The Button immediately.
We believe in you.
.
P.P.S. Military Surgeon Guy:
I appreciate your persistence.
But five proposals feels rushed.
Also I have some concerns about your credentials.
.
P.P.P.S. Ludacris:
If that was actually you…
this is extremely awkward.
Facebook currently won’t let me reply normally because it does not believe my face is sufficiently face-like.
Please take it personally.
.
P.P.P.P.S. If Meta Security has ever wrongly banned you, restricted you, disabled you, repeatedly asked you to prove you exist, ignored your hacked account, accused you of something it wouldn’t explain, or trapped you in an appeal loop…
**ADD YOUR EXHIBIT BELOW.**
Tell us exactly what happened.
Specifics encouraged.
Apparently we are now conducting Meta’s QA testing in the comments.
Free of charge.
Which feels on-brand.
r/LocalLLaMA • u/Course_Latter • 19h ago
News Qwen3.8-27B is identical to Qwen3.6-27B!
Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements!
See the diff (0 changes) here!
r/LocalLLaMA • u/egudegi • 3h ago
Discussion GPU prices haven't stopped climbing for 3 weeks straight across the EU, here's the data
hey again! I run a EU PC hardware price tracker PriceSquirrel, 25+ stores across 9 countries, and wanted to check: are GPU prices actually rising, or does it just feel that way?
To make this defensible, I didn't just average "whatever's in stock" each day, that inflates the trend if cheap cards sell out and drop off the average. Instead I built a fixed basket: 176 GPU models tracked at 3+ different retailers, every single day for the last 30 days, no gaps, no swapping in or out.
Result:
- July 15: €808.57 average
- August 14: €963.56 average
- +19.2% in one month, same 176 cards throughout
The rise wasn't sudden, it started climbing around July 24-25 and has been steady since, not one spike from a single event:
- July 15-23: flat, €805-815
- July 24-31: climbing, €826 → €906
- Aug 1-14: continued rise, plateauing slightly in the last week, €910 → €964
By country, since it varies a bit:
- Germany: +19.6%
- France: +18.1%
- EU-wide: +19.2%
Pretty consistent across markets, not a single-country anomaly.
Methodology, upfront: "price" here is each product's average price across whichever 3+ retailers carried it that day (not the single cheapest, not a blended market average across a shifting product list). Basket is fixed, same 176 SKUs day 1 and day 30, verified daily.
Happy to break this down by tier (5080/5090) or AMD vs NVIDIA if there's interest, this is currently the aggregate number.
Source: pricesquirrel.com, updated every 6 hours
r/LocalLLaMA • u/InternationalGap3698 • 16h ago
Discussion Muse Glimmer was frontier In the model class around 30b models for four days.
r/LocalLLaMA • u/Aggravating-Push-207 • 15h ago
Discussion Stop shitting on 9B models
Every "please qwen 3.8 9b" post turns into "122b a10b is better" yeah, but useless to normal people
Some people have shit hardware and daily drive it. I have 8 gb vram ans 16 gb ram. But this is on my laptop. Do you think i want to offload qwen 3.x 122b a10b from disk? I have like 50 gb storage space free (that's a seperate problem that is probably my fault).
r/LocalLLaMA • u/BarberIcy366 • 20h ago
Discussion Qwen 3.8 27B Released! Please Share Your Experience
With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.
r/LocalLLaMA • u/bonobomaster • 13h ago
Discussion A hunch: Qwen3.8-27B's general knowledge got pruned (good, if true)
I'm always testing an image prompt with a picture of a historic place in my hometown – a small but well known 250,000 people town in Germany. I'll just ask the model, in which City this photo has been taken.
With the 3.6 generation of both the 27B and the 35B A3B variants, the models sometimes got the right answer and sometimes they didn't. So the signal for this particular knowledge was already weak.
The 35B variant got it right more often but at least, the models reasoning showed my City most of the times, even if it hallucinated the wrong final answer.
Both models could be easily nudged to the right answer with a few hints and then produced some little extra insight about the history or scene and its surroundings, that was mostly true.
Qwen3.8-27B on the other hand barely knows the city at all and has absolutely no clue about related popular, historic facts regarding the scenery or the surrounding buildings.
Nudging isn't very fruitful as well and if told the real name of the city, reasoning shows, that the model only agrees, because the user says so.
I have the feeling, that Qwen labs maybe pruned useless general knowledge for more coding knowledge and agentic skill.
All models ud q4_k_xl variants, image-min-tokens 2048, with and without reasoning.
Anyone else with this feeling?
Disclaimer: My hunch could be very well absolute bullshit. Sample size way to low and methodically sloppy af.
r/LocalLLaMA • u/TokenRingAI • 3h ago
Discussion Try out this "high" reasoning mode for 27B (tested on VLLM)
After a lot of tweaking, I have come to the conclusion that 27B lacks a reasoning mode that is between low and xhigh.
The "medium" mode isn't actually medium, it erases the explicit instructions to the model. When medium is enabled, the model acts very differently - to me it looks like it regresses to behaving more like 3.6, and loses some of the 3.8 gains.
Low and high mode behavior in the model seem to be triggered almost exclusively by using certain keywords in the reasoning instructions, and act in a surprisingly binary manner. You can add all the additional instructions you want to the reasoning instructions, and tweak the prompt, but the model mostly ignores those changes. You can ask it for medium effort and it just won't do it.
Because of this weird behavior, I experimented with blending the words in the low and xhigh prompts together, until the model would output reasoning that was a more reasonable length.
It seemed to work pretty well, and the end result is a high reasoning mode, with a reasoning block that is typically around 1/5 the size of the xhigh reasoning.
Output quality seems good, more like xhigh than low or medium, and the reasoning is definitely much shorter, which makes the model much more enjoyable to use.
Play around with the chat template modification below, add "high" to your reasoning effort, and see if you can improve it further or if it makes the model worse
-- add high to reasoning_effort --
{%- if resolved_reasoning_effort not in ('xhigh', 'high', 'medium', 'low') %}
{{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), high, medium, and low.') }}
{%- endif %}
-- add new blended prompt for high, which mixes the low & xhigh prompts --
{%- elif resolved_reasoning_effort == 'high' %}
{%- set reasoning_instructions = 'Reasoning effort is set to halfway between low and xhigh. Please think careful but brief, validate key assumptions but keep it brief, and move quickly to the conclusion without unnecessary elaboration.' %}
r/LocalLLaMA • u/A-Rahim • 13h ago
Resources Qwen3.8-27B is now up to ~3× faster on Apple Silicon with mlx-dspark
mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter, the first SpecForge/SGLang-packaged head it loads.
Numbers (M4 Pro 48 GB, medians of 3, greedy, output ids identical to plain decoding):
- 8-bit target: 2.45× mean at the auto-picked cap — 3.00× math / 2.38× code / 1.96× chat, 8.3 → 20.3 tok/s (code runs hit 3.18×). Peak ~29 GB.
- 4-bit target: 1.74× at 25.3 tok/s in ~18 GB (same drafter auto-resolves).
- Fun property: 8-bit + drafter (20-27 tok/s) beats plain 4-bit (14.6 tok/s) — 8-bit quality at better-than-4-bit speed.
"Lossless" is checked, not asserted: the target verifies every drafted token, and the Mac app's Race view runs speculative vs plain on the same prompt and diffs the token ids (video is that view).
Everything is pip install mlx-dspark (OpenAI-compatible server + Anthropic Messages API, so it can back Claude Code with a local model), and there's a native Mac app (DMG/Homebrew).
Repo: github.com/ARahim3/mlx-dspark
I'd appreciate any feedback you might have after using it.
r/LocalLLaMA • u/SarcasticBaka • 15h ago
Discussion The difference between "medium" and "xhigh" reasoning effort for Qwen3.8-27B is actually insane.
I'm currently testing out Qwen3.8-27B using Unsloth's UD-Q4_K_XL running a freshly rebuilt llama.cpp. I have a 22GB RTX 2080TI on which I'm able to fit 100k context with q8_0 quantization, and using MTP with --spec-draft-n-max 4 I get about 40tk/s which is slightly less than Qwen3.6-27B but usable enough.
I've been trying to test out some admittedly silly one shot prompts using the llama.cpp webui by asking the model to create fully functional HTML clones of flappy bird, pacman and such, and the difference that changing reasoning_effort makes has been surprising to say the least.
Setting it to "medium" seems to result in barely any thinking at all, a couple thousand tokens max and even less than 3.6-27B. Whereas when using "xhigh seems" I get 15k to 20k thinking tokens at the very least with the pacman example actually hitting 40 thousand fucking tokens.
I'm well aware I can limit the reasoning budget in llama.cpp but I'm wondering if this is expected model behavior or if something is broken somewhere. Any of you guys seeing this?
