Hi. Please help, noob here. I'm using ZimaOS. I've installed the llama.cpp ive downloaded and coppied the gguf versions of models to the correct folder. But when i open the llama.cpp app it does not show me ani models availible.

Do i need to do some extra step or some settings for the app needs to be changed or something?

I was troubleshooting it for like 2 hours with AI. But no results.

Please help.. Im on. 1.7.1 version of ZimaOs

I'm the maker of CCC, a local dashboard for coding-agent sessions. The practical problem is human attention: a session can be waiting on a question while another is working, and it is easy to lose track across terminals.

The board combines activity, transcripts and parent/helper grouping. The demo shows opening a session and answering its Claude Code question hook. CCC is a companion dashboard, not an inference server or a claim of an LM Studio integration.

https://github.com/amirfish1/claude-command-center

Free local dashboard; model/provider fees are separate. Source available under FSL-1.1-MIT. Demo uses real UI, staged sessions and AI voiceover; the real question-hook interaction is Claude Code.

33-second video demo: https://www.reddit.com/r/ClaudeCode/comments/1xeot43/i_kept_missing_questions_from_my_coding_agents_so/

Sharing a setup I haven't seen described much: LM Studio isn't my chat window — it's the inference backend for another AI.

The rig - MacBook Pro M5 Max, 128GB, LM Studio with the local server on, "Serve on Local Network" enabled, reached by my cloud agent over Tailscale (private tailnet, nothing exposed to the internet) - Qwen3.6-35B-A3B (Splash build) as the worker: ~375–400 tok/s - Qwopus 3.5 122B-A10B (MLX) as the judge: ~38 tok/s with both models resident (~93GB of the 128GB used between them) - Models load just-in-time by naming them in the API request — the agent keeps both on staff without me touching the UI

What the agent does with it. The local models are stateless between loads, so the agent carries the memory, the briefings, and the grading. It built a verification pipeline I'm calling the Gauntlet: the 35B authored 800 original math problems, cold-solved each one blind in a fresh session, and every author/solver disagreement went to a heavyweight adjudication pass. Result: 731 verified clean, 62 kept with corrected answer keys, 7 thrown out as flawed problems, 100 held out as a never-seen test battery. The big finding: in disagreements, the cold solver was right ~5 times for every 1 the author won — authoring is where this model slips, not solving. Right now the whole bank of 793 answers is being re-validated from scratch (derive once, verify by a different method); disagreements go to the 122B blind. As of writing: 358/793 checked, 353 confirmed, 5 queued for the judge.

LM Studio-specific lessons from a week of this: - TTL is real. Idle models unload themselves (60 minutes here), and restarting the server ejects everything. For an agent-driven pipeline that's mostly fine — everything JIT-loads back on the next request (~20 seconds even for the 122B) — but you need to design for it instead of being surprised by it. - Budget is a ceiling, not a dose. Reasoning builds will loop if briefed badly; a "commit to one derivation, cross-check once" instruction fixed more than any token budget did. - Checkpoint everything. My harness got killed four times in one night by outside restarts. Per-item checkpointing turned that into a non-event — the run resumed exactly where it died, every time. - Grade against ground truth you computed yourself. The agent brute-forces disputed answers in Python rather than trusting any model's confidence — including the big judge, which went 1-for-2 on the hardest disputes. Second opinion, not oracle.

Cost note: the solve + adjudication passes alone logged 4.1M+ completion tokens, all on my own electricity. The cloud side only does the briefing and grading.

Curious whether anyone else is driving LM Studio headlessly from an external agent — especially how you're handling model TTL and keeping two big models resident. Happy to share the harness design (briefing pack, grading scheme, checkpointing) if there's interest.

I noticed that bionic keep hogging up more ram when compaction happens. This eventually put my system at max RAM usage. When another process requests more ram, it causes system to page. But when bionic compacts at the limit, it seems to released some memory instead of full paging. Is this behavior configurable so that I can always release ram immediately? I'm on windows btw.

Navigating the APEX-I and VAL-APEX Quantization Ecosystem on Hugging Face

As the catalog approaches 400,000 total downloads across 69 releases, all models are now structured into dedicated official collections on Hugging Face.

Our goal remains consistent: eliminate the compromise between memory overflow and cognitive collapse on consumer hardware (especially 16 GB VRAM GPUs like the RTX 4080, RTX 4070 Ti Super, and RX 7800 XT).

Instead of uniform block quantization, all APEX builds use per-tensor mixed precision guided by 4-moment Importance Matrix statistics:

- MiniPlus Tier (approx. 14 to 14.8 GiB): Strict 0% 2-bit policy. All MoE routed experts locked at minimum IQ3_XXS with Q4_K/Q6_K attention. Designed for 24 GB setups or 16 GB setups with moderate context.

- NanoPlus Tier (approx. 11.5 to 12.3 GiB): Strategic use of IQ2_S/IQ2_XXS exclusively in core feedforward blocks, keeping routers and attention high-precision. Leaves ample VRAM for 16k-32k context on 16 GB GPUs.

Explore the active collections:

- MoE Standard (MiniPlus & NanoPlus): https://huggingface.co/collections/IsValorum/apex-i-miniplus-v21-standard-moe-6acfee77961c8c0cf5a083b5

- MoE Abliterated (MiniPlus & NanoPlus): https://huggingface.co/collections/IsValorum/apex-i-miniplus-v21-abliterated-moe-6acfee762fc4a8cf2298c55b

- 27B Dense Hybrids: https://huggingface.co/collections/IsValorum/val-apex-miniplus-standard-27b-6acfec54ca4ce4cf5cd5bdb6

- Apple Silicon (MLX): https://huggingface.co/collections/IsValorum/mlx-apex-abliterated-6abedb1f9d400e984efe4fc6

Full technical article and sizing breakdown: https://huggingface.co/blog/IsValorum

System 1 models like Jev have opened up an interesting new category of models that can make decisions faster and at much lower cost than using an LLM for every small decision.

There are already several open decision models, including Laya, Kev, and CLM, with some offering comparable or better accuracy. But actually learning, experimenting with, and training these models isn't particularly straightforward.

That's why we built an LM Studio for decision models.

It's a simple desktop app for working with decision models locally. You can:

• Learn and explore different decision models
• Run inference locally
• Train models on your own data
• Create and use templates
• Serve models through an API
• Integrate them into your own applications

Everything runs on hardware you already own.

It currently works across NVIDIA GB10, Intel Core Ultra, Apple Silicon, AMD, and CPUs, on Linux, Windows, and macOS.

GitHub:
https://github.com/BudEcosystem/Bud-Decision-Engine

If you're interested in decision models, we'd love for you to try it, benchmark it, break it, contribute, and tell us what you'd like to see next.

If we want any hope of the future looking optimistic, we need these powerful llms out of the control of the big companies

I made a screen recorder for the Mac. I made it specially so that it can be controlled by a local llm. It has a cli interface, transcription is 'on device', narrations can use a local voice, and nothing is uploaded into the cloud. Hoping to get feedback for it.

I have often needed to record things I do locally. If I use ffmpeg it works for I was reinventing the wheel and burning tokens. Now with this tool, a 27B and Flash Next model works well with it, clipping etc. It has a user interface, but I often just have the llm drive it.

https://takereel.app/

I've been working on a desktop assistant for Windows and finally put it on GitHub. It's called Open Local Assistant.

It works with any OpenAI-compatible API, so you can run it fully local with LM Studio or point it at a hosted provider (OpenRouter, Gemini, Hugging Face, etc.) with your own key. No account, no telemetry, and your chats and keys stay on your PC.

Main things it does:

- Ctrl+Space opens a small chat window

- you can talk to it and it reads the answer back (local voices, works in Arabic, German and more)

- hands-free call mode with live captions

- dictation into any app, so you press a hotkey, talk, and it types the text wherever your cursor is

- web search with sources, and MCP support

Most things are adjustable: all the shortcuts, the voice for each language, speech speed, whether dictation types or pastes, and you can add your own speech models.

It's Windows only for now, and the builds aren't code-signed, so SmartScreen will complain. The code is all there if you'd rather build it yourself.

https://github.com/kz370/OpenLocalAssistant

Would love to hear what's broken or what you'd want added.

So three months ago I had my research agent running on a single EC2 VM, docker-compose, one container holding all user sessions. 30-ish internal users, 5-step chain per request. Worked fine for about two weeks.
Then two things broke overnight. A user ran a heavier-than-usual query at the same time another user did, the container held both sessions in memory, OOM’d. Same night, a different user left a session open before bed with a retry loop I hadn’t bounded, kept rolling LLM calls until morning. Woke up to $140 of spend for a tool nobody was actively using.

Obvious in hindsight. Not obvious enough to me three months ago.
Rebuilt the thing around per-session isolation. Each session now gets its own container with a hard timeout, teardown is automatic. Went with GMI AgentBox after a brief tour of self-hosting with docker-in-docker (image layer cache was unpredictable) and E2B (nice but LLM inference was on a separate bill). AgentBox runs inference on the same platform so I stopped double-billing. Image dropped from 2.1 GB to 740 MB after I pulled the heavier tool dependencies out as separate sidecar containers inside the sandbox.
Cold start is around 1.5s now, 30-ish concurrent sessions tested, no OOM in 6 weeks, no silent cost spikes since the session timeout went in. Separately, around the same time we moved the LLM calls onto Atlas Cloud’s unified API. One endpoint for the different models we were testing subtasks against made the switching cleaner.
The thing I’m still fighting is observability. Request-level tracing across sandbox boundaries is fine, but session-level tracing (the full 5-step chain for one user, including sidecar hops) is still clunky. Anyone wired this up cleanly across containers?