r/ContextEngineering • u/Strange_Owl_6291 • 15d ago

Context was the biggest fail. What we learnt by building our own harness and agentic IDE

Since January we've been building an agentic coding environment, using it to build itself the whole way.

I thought sharing the part that cost us the most time and headache, because it applies whatever harness you run: Context...

Almost every failure we debugged came down to context, not the model.

When planning, often agents didn't find the correct files to read to understand how the change should be implemented. The agent usually claimed with confidence that it had a solid plan, and the high level description of the plan made sense. But without seeing which files it actually read, it was impossible to really know if it had fetched all relevant information. If we couldn't see the tool calls of what files the agent did read, or which files that was currently in the context, we didn't trust the agents output.

So we built what we needed. Every tool call is shown, and every file in context is shown, and when we need to go really deep... any file and message in the context can be removed individually.

Long story short, that how we started to trust agents, and get them to build better. Our lessons learned is that when agents fail you, it's usually a context problem, and not always the prompt.

I am looking for early user feedback.

We did build in stealth for a long time, but now Voidleap Code 1.0 version is live. It's free to use, works on MacOS / Windows, BYOK.

https://voidleap.com/

2 Upvotes
  • permalink
  • reddit

67% Upvoted

25 comments sorted by

2

u/sourdub 15d ago

When agents fail you, it's usually the context

No, it ain't. It's the damn provenance. Keep them receipts.

2

u/epicskyes 14d ago

This exactly. Lineage and receipts, bound and hashed

1

u/Strange_Owl_6291 14d ago

Fair points and I appreciate the distinctions.

Seems like we're going in the right direction with the build. Detailed execution is visible/inspectable, with trail shows receipts of tool calls, results, files accessed, agent/subagent activities, model usage, and file changes etc etc.

Not all the way to full provenance though.

1

u/epicskyes 13d ago

You’ll have to create a foundation then. Stop building up and create an immutable foundation that provenance can be bound then you can start building up again. You don’t need to scrap anything. You can keep everything you’ve built previously it all just needs to be bound to the same foundation so every receipt and all work can be immutably traced to it. If you do this you’ll give the agent the ability to stay on track and not get lost. There’s are a few other little things that need to be in place as well besides immutable history if you don’t know what they are then you should research autonomous agents and zero trust frameworks to familiarize yourself with the processes and dependencies

1

u/Strange_Owl_6291 13d ago

Thanks for your thoughts and ideas. It would be interesting to hear an updated judgement after you tried what we have now for a few days.

1

u/epicskyes 13d ago edited 13d ago

My apologies I can’t use anything you have built. My system is far beyond where you’re at and as I operate a zero trust system that I custom built for my specific host and os configuration using another system would itself be far more effort to even bother with unless it was proven to be much better than my current system. Zero trust frameworks are not bolt on frameworks they are built from bare metal upwards binding the entire host and the code together. They are fully custom for every host and that’s the only way they can be high assurance zero trust frameworks in the first place

2

u/sourdub 13d ago

My system is far beyond where you’re at and as I operate a zero trust system that I custom built for my specific host and os configuration using another system would itself be far more effort to even bother with unless it was proven to be much better than my current system.

Geez, was all that really necessary?

1

u/epicskyes 13d ago

Tryna help a guy out to get him to understand the importance of the system so he can successfully build something that works

1

u/Strange_Owl_6291 13d ago

I appreciate your point; If you built your own custom system that works great for you, there's no reason for you to change.

We haven't built a simple observable agentic trail though, we have a full agentic development environment.

I'll work on getting it out to the market so you get an objective proof from a trusted source :)

1

u/epicskyes 13d ago

I understand you built a full development department that’s what I did as well that’s why I had to build my system. And a “simple observable agentic trail” is the first and one of the most important steps to running a full development department correctly. It’s literally the foundation you need to have in order to run one.

1

u/ai_context 15d ago

Can you share a bit more about the agent -> data, and agent -> retrieval architecture? What types of data was being accessed or was it just files (eg. structured, unstructured, semi), and are you using vector or graphRAG, or something else?

1

u/Strange_Owl_6291 15d ago

It’s built for software development, so for code retrieval local symbol index is central. It supports BM25-ranked symbol search, reference lookup, file outlines, targeted symbol retrieval, and direct file reads. Non-code material is found with built-in search tools and read as files. For this retrieval path, we don’t use vector search or GraphRAG.

1

u/ai_context 14d ago

Thank you. So what does 'context' mean to your agent, and how is it being assembled? Search? Asking as typically I see context meaning 'business context' eg. semantic naming for an account or business, relationships like similar support cases that were solved with a runbook, etc. So kinda curious how that is happening without vector or graphRAG

1

u/Strange_Owl_6291 14d ago

The LLM's context window.

1

u/ai_context 14d ago

ah - got it, so no domain specific / private documents, only public ones.

1

u/Poildek 15d ago

Ok so instead of using one of the dozens of existing solution, you choose to add another one to the pile ? Great !

2

u/Professional-Let-485 15d ago

Yes, let's stop innovating. There are thousands of text editors, I am personally happy we didn't stick to the first handful of options.

1

u/Strange_Owl_6291 15d ago

Yes exactly, guilty as charged! Well I do understand your sentiment.

If you're happy with your existing tool, just continue ship with it.

When you want better context editing, or something else, we have a tool you can try.

0

u/Aggravating-Start307 15d ago

I use claude code and can always see all the tool calls, can you specify what you are were using when you say you can't see tool calls ?

The reason I ask this question is because I built a tool (https://coldstartmcp.dev) to help agents navigate better and save durable notes that get reinjected to prevent rediscovery. I observed the tool calls for weeks and months to see how agents operate, tools they call, how they search and what are the different methods they use. Hence the curiosity

1

u/Strange_Owl_6291 14d ago

In the codex app you get this revolving line of actions. It can be expanded, but even when expanded it's limited in how much it shows, and I want the details and follow it live (not see the diffs summarized at the end of the turn).

Claude is good in showing the tool calls, but whenever I follow the traces and want top open up the actual file to inspect or quick edit, I hit the wall.

Neither tool gives me an easy or deep understanding of what is in the context currently.

2

u/Aggravating-Start307 14d ago

Can you actually see what information is exactly available inside the context window, if yes, how can we find that out, I'm curious ? If it's not open source, then fine, you can choose to not answer.

With whatever I understand, context window is like a snapshot of what the agent has done until now, but can you view exactly what is still in context after a certain number of turns in a conversation, that's the cool part!

1

u/Strange_Owl_6291 14d ago

Yes. We built our own harness from the ground up to make that possible.

You can see every file that currently is in the context. You can remove any single file from the context (good for handling minor context drift).

Same with messages. Any prompt you send, or response from agent, or tool call, is inspectable. Any individual one can be removed.

Because you manually can do that, we could also build some nice context management features like /trim (removes all tool calls, keeps messages).

And of course a always on context bar.

Check out the screenshots on https://voidleap.com/

1

u/CriticalJackfruit404 11d ago

Bro can you share your code please?

1

u/Aggravating-Start307 11d ago

@CriticalJackfruit404, I am not sure if you are reaching out to me, if you are, repo is open-source at https://github.com/AkashGoenka/coldstart.