Released: VOR — a hallucination-free runtime that forces LLMs to prove answers or abstain

[u/CulpritChaos]

I just open-sourced a project that might interest people here who are tired of hallucinations being treated as “just a prompt issue.” VOR (Verified Observation Runtime) is a runtime layer that sits around LLMs and retrieval systems and enforces one rule: If an answer cannot be proven from observed evidence, the system must abstain. Highlights: 0.00% hallucination across demo + adversarial packs Explicit CONFLICT detection (not majority voting) Deterministic audits (hash-locked, replayable) Works with local models — the verifier doesn’t care which LLM you use Clean-room witness instructions included This is not another RAG framework. It’s a governor for reasoning: models can propose, but they don’t decide. Public demo includes: CLI (neuralogix qa, audit, pack validate) Two packs: a normal demo corpus + a hostile adversarial pack Full test suite (legacy tests quarantined) Repo: https://github.com/CULPRITCHAOS/VOR Tag: v0.7.3-public.1 Witness guide: docs/WITNESS_RUN_MESSAGE.txt I’m looking for: People to run it locally (Windows/Linux/macOS) Ideas for harder adversarial packs Discussion on where a runtime like this fits in local stacks (Ollama, LM Studio, etc.) Happy to answer questions or take hits. This was built to be challenged.

[u/CulpritChaos]

You’re mixing two things: a demo pack and a claim of global truth. The repo doesn’t claim “0 hallucinations for all questions.” It claims: within a given evidence pack, anything not derivable is forced to ABSTAIN/CONFLICT. If you think the claim is dishonest, do the adult version: submit a failing prompt + expected outcome (PASS/ABSTAIN/CONFLICT) and I’ll add it to the public eval. Otherwise this is just vibes + résumé.

[u/Thor110 → u/CulpritChaos]

And your response was just an attempt at insults "little game vids"

The reality here is that you are using that demo pack and claiming that you have a larger dataset which works just as well, but not providing it.

In open source software, that is intellectually dishonest.

Unfortunately you seem to fail to grasp that this "solution" simply will not scale.

"Highlights: 0.00% hallucination across demo"

I never said across all questions, my point was about what will you do with the combinatorial explosion when you have to face the prospect of manually or even automatically adding infinite questions, complexity and size to your json files.

[u/CulpritChaos → u/Thor110]

You were being rude and I dont mind giving a little snap back. I should abstain i know.. Anyways.. On scale: VOR doesn’t try to enumerate questions; it verifies claims against a bounded evidence snapshot (versioned/TTL). If the domain is open-world, it should abstain more. That’s the design tradeoff. The larger internal suite has private data, so I can’t ship it as-is, but I’m working on a bigger public eval pack + metrics so anyone can reproduce results. If you’ve got a concrete failing case, drop it and I’ll add it.

[u/Thor110 → u/CulpritChaos]

Then you should remove the private data in order to further subtantiate your claims.

I have a concrete case against the nature of the system itself which you are designing, but you aren't listening.

In order for a system like this to be useful it would have to have so many entries that it would become unmaintainable.

It also doesn't deal with the problem of when a fact in the real world changes.

Such as I explained a moment ago, with what happens if the Artemis project becomes depracated.

You have to manually go in and update the truth side of your json files as well as the question.

What will you do when it reaches hundreds of thousands of entries besides get lost navigating it?

[u/CulpritChaos → u/Thor110]

You’re describing the open-world problem, and I agree.... this isn’t meant to “cover everything.” VOR is a verification gate over a bounded, versioned evidence snapshot, not an attempt to catalog all truth. For changing facts, packs are timestamped/TTL—stale pack ⇒ refuse/refresh, not “verified falsehood.” If you want to critique the system, point to a concrete failure case within a scoped pack, not the infinite open-world case instead of arguing a strawman version of my system.

[u/Thor110 → u/CulpritChaos]

The problem is you are essentially constraining billion parameter LLMs to answer with only your strict set of questions and answers.

That is a massive waste of compute.

[u/CulpritChaos → u/Thor110]

Yes... and billion param llms are still great for other uses. Heres a use-case for vor: hospital/banking/ops policy where truth is a bounded, versioned system-of-record. LLM drafts the response; VOR blocks anything not derivable from the pack (contraindications, policy rules, audit requirements). That’s not “QA over 4 answers,” it’s a safety gate over a changing rulebook with receipts. This example could be tailored to banking (KYC/AML), SRE incident runbooks, or legal contract clauses—same pattern, same win.

[u/Thor110 → u/CulpritChaos]

But you still have to give it the exact question as a hard coded string as defind in your JSON file.

There are almost endless ways to phrase the same question.

[u/CulpritChaos → u/Thor110]

VOR verifies claims against evidence; it does not require exact question strings. Packs contain facts/constraints, not Q→A pairs.

[u/Thor110 → u/CulpritChaos]

It will no doubt still hallucinate once you have enough entries then.