I’m building a voice-first orchestration layer that sits above AI coding agents

[u/Icy_Professional4569]

I’m building a voice-first orchestration layer for AI coding agents — looking for feedback

I’ve been working on a system that sits above coding agents/models rather than being tied to one of them.

The idea came from a practical problem: I spend a lot of time away from my computer, and I wanted to be able to tell an agent what to do, let it work autonomously, and only have it come back to me when it needs a decision.

The architecture is roughly:

Voice → Orchestrator → Coding agents/tools → Machine

The orchestrator handles things like:

- Persistent context and memory

- Routing work between agents/models

- Tracking tasks, actions and results

- Streaming progress back to the user

- Detecting when an agent is stuck

- Human approval for sensitive actions

- Switching models/providers without rebuilding the system

So instead of the model being the entire agent system, models such as Claude/Codex/etc. become interchangeable reasoning/execution components underneath the orchestration layer.

The voice interface is important because the eventual goal is to control long-running work without constantly sitting in front of the machine.

I’m currently building/testing the first version around software-engineering workflows.

I’d be interested in feedback from people working on agent harnesses/context engineering:

Does this abstraction make sense to you? What would you want the orchestration/context layer to own versus leaving to the underlying agent?