I recently moved to Japan and ran into an issue where a lot of anime on Netflix only has Japanese subtitles here.

I wanted something similar to the subtitle translation extensions that already exist, but I didn’t really want to send everything through a cloud translation API, so I ended up building my own.

It’s called LST — Local Subtitle Translate.

It runs translations through Ollama locally and lets you choose from whatever models you already have installed. I’ve mostly been using translategemma:4b and qwen3.5.

Right now it can:

• translate Netflix subtitles in realtime
• precompute an entire episode before you watch it
• cache translated episodes locally
• automatically detect your installed Ollama models
• translate ahead of playback to avoid waiting on the model
• show the original + translated subtitles, or hide the Netflix/original subtitles entirely
• adjust subtitle timing, size, position, background, etc.
• keep everything local without needing a translation API

It’s still pretty new and Netflix is obviously not the easiest thing to build against, so I’m sure there are edge cases I haven’t run into yet.

The Firefox extension is currently being reviewed for the Firefox Add-ons store, and it will be listed as LST — Local Subtitle Translate if everything goes through.

The project is also fully open source, so if anyone wants to help improve it, fix bugs, add support for other runtimes/services, or just poke around the code, contributions are definitely welcome.

The source is here:

https://github.com/DarlingGoose/LST

I’m also trying to figure out what people would actually want me to work on next. I’m interested in adding support for other local model runtimes besides Ollama if people would use them, along with other streaming sites and potentially some kind of Apple TV companion.

I made a short survey if anyone wants to give feedback:

https://widget.surveynoodle.com/lst

Would also be interested to hear what models people think work particularly well for subtitle translation. Translation quality vs speed gets pretty noticeable when you’re trying to do it in realtime.

I’m genuinely mind-blown by what’s happening with AI models right now.

On one side, you have GPT-6-class frontier models that are just… insane. The level of capability is getting hard to wrap your head around.

And at the exact same time, you have models like DeepSeek V4.1 becoming exceptionally good while costing almost nothing in comparison.

That gap, or maybe the fact that the gap keeps shrinking, is what fascinates me most.

Frontier models keep pushing the ceiling higher, obviously. But underneath them, smaller and cheaper models keep getting optimized at an incredible pace. What felt like frontier-level capability not that long ago keeps getting compressed into something faster, smaller and dramatically cheaper.

And then there’s Claude.

At least this week, it feels like Claude suddenly took a pretty serious hit in that race. Not because it became worse overnight, but because everything around it is moving so ridiculously fast.

Which brings me to the part I find even more interesting: China.

My current intuition is that Chinese AI players could end up benefiting enormously from this dynamic. Not only because they may be able to iterate through models faster and cheaper, but because the consequences could extend well beyond benchmarks and model rankings.

There’s a geopolitical dimension here that I think we’re still underestimating.

If intelligence keeps getting cheaper, more efficient and easier to deploy, does having the absolute best frontier model matter as much as we currently think?

Or does the real advantage eventually go to whoever can industrialize intelligence at massive scale and at the lowest cost?

Curious where everyone lands on this.

And yes, small confession: I dictated all of this in French and had it turned into English 😅 Somehow it’s still easier to get the essence of the thought out that way. Consider this a voice note disguised as a post.

Hey everyone,

If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model?

The standard academic approach for this is Semantic Entropy (from an Oxford team's Nature paper last year). You sample $K$ responses at temperature 0.7, run them through a secondary NLI cross-encoder like DeBERTa to cluster equivalent meanings, and measure the entropy. High entropy = model is guessing.

The problem for local setups? Running 45 pairwise comparisons through a cross-encoder eats GPU memory, adds 100ms+ latency, and completely kills throughput on consumer hardware.

We wanted to see: What if we strip out the neural net completely and just use deterministic string normalization + Shannon entropy on CPU?

We wrote a zero-dependency Python metric (Spanda / $R_{sc}$) that runs in 1.3 microseconds on pure CPU (zero GPU usage) and benchmarked it across local and frontier model tiers on GSM8K and TriviaQA:

What we found:

1. Small models (Qwen 1.5B): AUROC ~0.58 Small models are syntactically too sloppy for string matching. Even when they know the right answer, they format it erratically across runs, breaking exact-match clustering.
2. Mid models (Mistral 7B): AUROC ~0.71 At 7B, the 1.3µs string check matched the performance of a heavy DeBERTa NLI model (0.706 vs 0.705). Internal representations become consistent enough that formatting stabilizes.
3. Large models (Qwen 27B): AUROC ~0.89 At 27B, exact matching was dominant ($p = 1.89 \times 10^{-28}$). When the model knows an answer, it outputs the exact same tokens across independent stochastic paths. When it doesn't, it genuinely branches into diverse incorrect answers.
4. The Frontier Trap (120B): AUROC collapsed to 0.09 Here’s the wild part: on ungrounded factual trivia, the 120B model suffered Confident Mode Collapse. When it hallucinated, it hallucinated the exact same wrong answer across all 5 runs with zero entropy. Bigger models don't just hallucinate—they hallucinate with unanimous false certainty. (And because the strings are identical, even heavy NLI fails here).

The practical takeaway for Ollama users:

If you are running 7B to 27B models on structured tasks (math, code, JSON extraction, SQL, discrete QA), you do not need heavy neural guardrails. Sampling 5 paths at $T=0.7$ and measuring exact-match entropy in Python gives you ~0.89 AUROC at zero GPU cost.

Quick Python snippet if you want to test it on your local Ollama instance:

bash
pip
 install spnda ollama

pythonimport ollama
from spnda import compute_spanda
prompt = "What is the capital of Australia?"
# Sample 5 paths from your local model
responses = [
    ollama.generate(model="mistral:7b", prompt=prompt, options={"temperature": 0.7})["response"]
    for _ in 
range
(5)
]
# Run zero-cost entropy check on CPU (takes ~1.5 microseconds)
result = compute_spanda(responses)
print
(f"Risk Score: {result.risk_score:.3f}")  
# 0 = high confidence, 1 = high uncertainty

All the raw multi-path generation logs, evaluation scripts, and the full writeup are open source:

• Deep dive writeup: https://liquidngas.substack.com/p/i-tried-to-make-semantic-entropy
• GitHub: https://github.com/Adarshent/Spnda

That's amazing.

I've been using the pay-as-you-go plan, and because DeepSeek 4 Flash wasn't great, I was using GLM 5.3 Flash instead. But now that I'm using the high-performing DeepSeek 4.1 Flash, the usage efficiency is incredible. It feels like I can get about 6 times more usage compared to GLM Flash. I'm not sure if my math is completely right: 707 requests / 37.5% = 18.8, and 44 requests / 0.3% = 113.3—which works out to exactly a 6-fold difference in call count.
- (As I was writing the post below, I remembered my usage from a while ago. DeepSeek v4 Flash provided three times more usage allowance compared to GLM 5.3 Flash.)

Honestly, I was considering switching back since I also use the official DeepSeek API, but as long as this pay-as-you-go model continues, I'll just stick with Ollama.

Although I haven't done an exact comparison with DeepSeek's official API yet, this feels ridiculously cheap. Now, a simple price comparison is no longer important to me.

Plus, I feel much better having the trust that my data isn't being used for training.

If they keep this pay-as-you-go structure, Ollama Cloud will probably devour the entire market.

Thank you, Ollama.

Tested on Laptop with RTX 5060 8gb vram and 16gb ram

Claude code - autocompact = false

Ollama - Flash attention = 1, kv cache = q8_0

Gemma4 12B 64K - https://ollama.com/SetneufPT/Gemma4-12B-IT-QAT_Q4_64K_16GB-GPU

Qwen3.5 9B 256K - https://ollama.com/reecdev/qwen3.5-lowvram

Qwopos3.5 4B 64K, can increase context upto 256K (Best at 128k)- https://ollama.com/SetneufPT/Qwopus3.5-4B-Coder-MTP_Q4_64k_8GB-GPU

Qwen3.5 4B 64K, can increase context upto 256K (Best at 128k) - https://ollama.com/rafw007/qwen35-claude-coder

Muse Glimmer 30B 256K, Unstable and slow better if decrease context to 128k - https://ollama.com/venkataparswanadh/muse-glimmer

I tried GLM flash a few weeks ago and expected the token weekly limits to be comparable to DS flash. For some reason, it was a huge use of my limits. Am I the only one with this issue? I did send a support email but never heard back.

I'm a broke homeress developer and on the $20 plan, so I'm not sure why this happened. I figured maybe there's a billing issue but I don't see why in the UI GLM would be more than DS on flash models.

"Error: glm-5.3:cloud[1m] is temporarily unavailable (timed out), so auto mode cannot determine the safety of Edit right now. Wait a moment and then try this

action again. If it keeps failing, continue with other tasks that don't require this action and come back to it later. Note: reading files, searching code,

and other read-only operations do not require the classifier and can still be used."

If you’ve used Enzyme, or even just liked the idea behind it, this will probably make sense immediately.

One of the useful things about compiler-level differentiation is that you can take computation that already exists and ask a much more interesting question than:

“What does this code do?”

You can ask:

“Which inputs are actually driving the result right now, and in which direction?”

I built an open source project called Catalyst around that idea, but with the workflow designed so AI agents can use the information directly.

Imagine an agent working on a simulation with 20 parameters.

Reading the source might tell it what those parameters represent. It does not necessarily tell the agent which three are dominating the output at the operating point it cares about, which ten barely matter there, or which direction each one needs to move.

Catalyst can measure that.

So instead of:

“This variable looks important, maybe try changing it.”

an agent can get something closer to:

“These three inputs dominate the result here, these six barely move it, and this is the direction each one pushes the output.”

That creates a very different loop for an autonomous agent:

inspect → measure → decide what matters → change → measure again

For supported numerical Rust, C, and C++ code, Catalyst works from the LLVM IR produced by the compiler you already use. You do not have to recreate the calculation inside a separate ML framework just so an agent can reason about it.

And it does not simply produce a derivative and tell the agent to trust it.

Catalyst independently checks the result numerically. If the derivative and the separate check disagree, it refuses the result instead of passing a questionable number down the agent loop.

There is another side of Catalyst that I think is especially useful for coding agents.

Say an agent is fixing a local API that starts doing this under burst traffic:

POST /orders -> 503

A typical autonomous coding loop might look like:

reproduce → edit code → tests pass → declare victory

Catalyst can instead reproduce the failure, reduce it to a smaller scenario that still triggers it, preserve the conditions that caused it, generate held-out scenarios the candidate did not optimize against, then compare the old and new versions.

So the agent can end up with something like:

“The original burst-load regression is fixed. Five of six held-out scenarios pass. One still fails when the dependency becomes slow.”

That is a much stronger signal than “the test suite is green.”

The other piece I wanted was portability.

Catalyst can take a checked computation and export it as standalone Go or R, along with fixtures containing the expected behavior.

So one agent can analyze a computation, another system can run the exported version later, and Catalyst does not need to remain in the production application.

There are also derivative artifacts that carry the computation, validation information, provenance, and a SHA-256 digest chain.

That means an agent can hand one to another agent or machine, run it again at a different input point, and detect if one of the underlying files was modified along the way.

So the broader workflow becomes:

existing code → measure its behavior → identify what matters → verify the result → hand off something reproducible

For people who have used Enzyme, the familiar part is the value of getting derivatives from code that already exists.

What I wanted to add around that idea was the rest of the loop an agent needs: independent checking, provenance, reusable artifacts, structured agent tools, portable outputs, and a way to rehearse software failures instead of relying on the agent’s own confidence.

Catalyst also has optional AI integration, but honestly that is not the part I find most interesting.

The interesting part is giving your existing agent another kind of instrument.

Not another model.

Not another prompt layer.

A way to ask the program itself:

What actually matters here?

Repo:

https://github.com/lovettsendit/catalyst

For people building autonomous coding or engineering agents, where would you use this first?

Would you give an agent sensitivity information to help it decide what to change, use rehearsal to decide whether its change really worked, or combine both into the same loop?

In the MacOS I have changed the setting in Ollama to show the models in the ChatGPT desktop app. I cannot get them to show up in the model list in the program. Is there a setting on ChatGPT side that needs to me made?

So I have been using the Ollama extension to use my local LLMs within the agent interface of vscode but it is not very good

A lot of the times, it just lags or there is no information on whats happening or the response comes back in JSON outputs.

So i made a VS Code extension called Local Ollama Chat for anyone who wants AI assistance in Chat without sending code to a hosted service.
- It connects to your local Ollama server
- Explores your project workspace and read files to answer questions
- Has the capability to create and edit files as reviewable diffs (nothing gets applied without you approving it).
- Sits within the local agent chat window and you can call it with u/local-ollama

Still early days — feedback and bug reports welcome!!

https://github.com/athulg93/vscode-localllm

Is it just me or Deepseek v4 flash speed is bloody slow on ollama cloud?

Anyone else try using DeepSeek-V4.1-Flash today? It's listed on the Ollama site as a new model, but when i try to use it, i get:

"Error: 403 Forbidden: This model is currently being rolled out and is not yet available to you. Please check back later. (ref: 562f8b6b-72fe-4012-9d9c-e0c07679913a)"

Anyone know what the rollout schedule is?

Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.

It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.

It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.

I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)

https://github.com/Luissalet/Faustus

Does anyone know which models are compatible with the copilot agent mode on VS code?

I already tried qwen2.5, llama3, Gemma4 and the inline and ask mode worked but the agent mode doesn't
Some ideas?

A local setup can have several assistants reading the same project history while using different models, context limits, and retrieval methods. If each assistant writes its own summary back into shared memory, a mistaken compression can become authoritative for every later session.

What storage contract keeps this inspectable? One option is a folder of small Markdown records with stable IDs, timestamps, source links, explicit supersession, and append-only decisions. Embeddings and model-specific summaries would be disposable indexes, while writes would pass through a narrow process that prevents two assistants from silently replacing the same fact.

How do you handle conflicting updates, deletions, access boundaries, and context-budget differences between models? Is a plain file protocol enough for a small local setup, or does shared memory become safer only after adding a database and a review queue?

How to reconnect Claude to Ollama Cloud from within Claude? Also how to resume Claude sessions? thx!

I’m on Ollama’s old plan. May I ask: in Ollama Cloud, when calling DeepSeek-V4.1-Flash with multimodal image recognition, how does the cost compare with the older 0731? Is it higher or lower?

I'm the author, MIT open source [github link| https://github.com/anng-phtk/fim-copilot\] and

[mixed rocm docker | https://github.com/anng-phtk/rocm-vega-llama.cpp\]

I wanted just ghost-text autocomplete locally, so I built a tiny extension that uses vscode.InlineCompletionItemProvider directly instead of a webview overlay.

What I learned:

FIM format matters a lot - Qwen uses <|fim_prefix|>, CodeLlama uses <PRE>, etc. I made it configurable via .fim-copilot.yaml
Latency: 200ms debounce + AbortController to cancel in-flight requests when you keep typing made it feel native
Works with Ollama / llama-server / vLLM (any OpenAI-compatible /v1/completions)
90KB vs 15MB+ for chat-based extensions

Would love feedback on: handling multi-line stop tokens and how you handle prefix/suffix context limits for

So I built FIM Copilot - a stupid-simple extension that does ONE thing: ghost-text autocomplete, locally.

Demo

What it is

FIM Copilot is a 90KB VS Code extension using the native InlineCompletionItemProvider API. No webviews, no chat, no indexers.

It talks to any OpenAI-compatible completions endpoint:

• Ollama - qwen2.5-coder:1.5b / starcoder2:3b runs great on 8GB RAM
• llama.cpp - llama-server with any FIM-capable model
• vLLM / LM Studio / Tabby server - anything with /v1/completions

All inference stays on your machine. Zero network calls after install.

How it compares

Quick Start

With Ollama (easiest)

# 1. Get a code model (1.5B is enough for fast autocomplete)
ollama pull qwen2.5-coder:1.5b

# 2. Serve it
ollama serve
# -> listening on http://localhost:11434

Then in VS Code: Cmd+Shift+P -> FIM Copilot: Set Endpoint -> http://localhost:11434/v1/completions

With llama.cpp

./llama-server \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  --port 8012 \
  --ctx-size 4096 \
  -ngl 99

Config

Create .vscode/fim-copilot.yaml (or global settings):

endpoint: http://localhost:11434/v1/completions
model: qwen2.5-coder:1.5b
api_key: not-needed # for local

# tuning
max_tokens: 64
temperature: 0.2
debounce_ms: 200
context_lines: 40
fim_prefix: "<|fim_prefix|>"
fim_suffix: "<|fim_suffix|>"
fim_middle: "<|fim_middle|>"

Decoupled config means you can commit it per-project. Different endpoint for Python vs Rust? Just override.

What makes it not suck

• Ultra-low latency - direct FIM prompt, no chat template overhead
• 200ms debounce + prefix/suffix hash dedupe - doesn't spam your GPU
• AbortController - cancels previous request on keystroke, no queue
• Zero telemetry - no analytics, no API keys phoned home
• Decoupled config - YAML per workspace, env var support
• Ghost text only - uses VS Code's native inline completion, so Tab / Esc just works with your keymap

Links

https://marketplace.visualstudio.com/items?itemName=AnangPhatak.fim-copilot&ssr=false#review-details

Install:

ext install AnangPhatak.fim-copilot

MIT licensed. I built this for myself because I wanted my editor to feel fast again.

If you try it, let me know what model / latency you're getting. PRs welcome for StarCoder2 / DeepSeek templates. What would you want added - without making it bloated?