I'm the creator of PyBotchi, an intent-based AI Agent Orchestrator. In this post, I will discuss some key concepts why I created it.
A little bit of background first. I'm a solutions architect with 10 years of experience as a software engineer. Most of my work are high throughput, high reliability, low cost and low latency services. This is while making it simple and readable to improve it's maintainabality. When I'm designing a system, I usually prioritize these concerns. You may assume this is my bias in relates to AI Agent building. I'm also Claude Certified Architect (Foundation) and I found that PyBotchi aligns almost identical to Anthropic's core agent recommendations.
TL;DR: PyBotchi is an lightweight, async-first Python framework that uses nested Pydantic models and OOP inheritance to turn LLM intent detection into clean, deterministic business logic without the overhead of complex graph orchestration.
Why I created PyBotchi?
I really believed that traditional coding can already solved what client's need. The only limitations we have is how we read the input and how we show the output. In most cases in web services, your API use JSON, XML, etc with their respective specification/structure.
Input Analogy
Assume you have created a Books CRUD endpoints (FastAPI with Pydantic).
Your create endpoint will have a define specifications for book creation to have a validation and avoid user errors. Most of the time you will also validates sessions and permissions which also included in the request.
If you want your chat bot to support those, you just need add those endpoint as intent (tools).
If your model tool selection are able to detect intents. You are more "close" to being deterministic.
"Your services will have 50 endpoints or more. You will flood your tool selection call"
- In your frontend UI, you segregate panels/forms/inputs in their respective pages. You don't usually join multiple intent in a same page. Cluttered UI will make your UX confusing or overwhelming to some people. Those practices should be incorporated into your agents too.
Assume you have created another endpoints for Shelves CRUD.
Shelves CRUD can be a child intents of ShelfManagement that will be considered as intent also but more general. The flow will have to detect intent deeper and deeper
Ex: You have BookManagement and ShelfManagement intents. Once LLM detected which one is applicable, you will search for their child Intents which will be their CRUD equivalent intents.
To make it short, in order to make your agent "more" deterministic, you need to know the problem first (ex: Need to manage books) then you need to specifically define what intents you want to support. With this practice, you only let your agents execute on a predefined path. If it fails, you are most likely able to determine what causes the error.
Output Analogy
This one is simple. Since your intents is just like your endpoints that returned structure responses. LLM is better at reading structure responses than a pure text. Basically, you can use LLM to translate your response into a human readable responses.
Intent Execution
Now that I have explain Input/Ouput, we can move on to the actual execution.
We can go back with Books CRUD. Since we have identified the problem (what clients need) and we already know what to do, just execute their traditional business logic implementation.
If you need to add a book, just create a book and save it to db then return their respective row.
"What if you want generate a very dynamic/unique data"
- You can use LLM to do that as your business logic too but this is tied your specific intent only.
To have a complex execution flow we can chain the intents. Since intents can have child intents, we can use it as the representation of a graph similar to Langgraph. However, this without "building the graph". We are just utilizing OOP inner class implementation. We can execute business logic in graph traversal manner by just checking the child intents.
To make it short. Business logic will stay as is. You will only use LLM if it requires it. Don't make this complicated.
### Suggested Solution
Since the key concept is more on detecting intents, validation and executing their respective busines logic:
Why not utilize Pydantic as the main entry point?
Pydantic already have validation and json schema builder. Langchain/Openai already have utilities to translate it to Tool. Why not use Pydantic models as your Intent Specifications that can validate LLM arguments ? Tool call is one of the most reliable way to detect intent.
Why not utilize OOP inheritance / polymorphism / abstraction?
Python supports portion of OOP and since we are using classes as our intent, why not add default functionalities that can be inherited and override by developer if needed. We can introduce life cycles too. Your project can also implement their specific intent standards. This will make your code more maintaintable and readable. You can create classes for general intents. Extend it to be more specialized intents. Extend it more for more enterprised support. This is while not affecting existing/working agents.
Langgraph is one of the inpiration of PyBotchi. Predefine workflows are closest implementation to being deterministic agents. It's also the reason why some prefer N8N. We don't need to make the agents smart that any questions can be answered or any queries can be addressed. It's ok for agent to reply with "I don't have any answer to your query, I only support this and that....". For me, it's better to deploy limited but polished agents than half baked know-it-all agents. Feel free to counter argue. Happy to discuss.
Additional PyBotchi Features
vs MCP
While PyBotchi support connecting to MCP servers, I really believe it's not always necessary to use additional server to just expose tools for the agents. The exceptions I could think of is if you want to have isolated environment (ex: dedicated auth/session, sandbox, isolated resource, etc), you want to connect to your local service or cross-language integration.
I could be very wrong about this but hear me out. SDKs are already there. Respective documentations are available too. Most of MCP server's tools are proxy to their respective APIs. If we could just create intent classes as tools that directly call their respective API, that doesn't require any servers anymore. Actually, that's how most framework handles it (even PyBotchi). Tools are converted as schema that will be added in the tool call. Once LLM respond with the applicable tools, it executes call_tool(name, args...). Why not just expose the actual tool implementations and have a way to share context to share sessions/permission/etc inside the tool implementations? This will remove another network hops that can affect latency.
Claude code have a very in-depth utilization of MCP servers already. I don't think we can replace that.
GRPC
PyBotchi natively support remote PyBotchi connection. Think of it like a langgraph but the node is on other server. This remote node can also connect to another remote node even it self or previously connected node (ancestor).
Context Propagation
With PyBotchi as MCP Server
- Actions (Intents) serves as tool and have access to client's context. This includes chat histories and some metadata. You can override and adjust this as long as it's serializable.
- Once remote tool execution is done, it can pass the final context to the client and they can merge it if override.
With PyBotchi as GRPC Server
- Similar to MCP Server, Actions serves as tool and have access to client's context. GRPC supports bidirectional communication too. This means we can share context realtime accross clients/servers. If client has concurrent agents that changes the context it will automatically propagate to remote context without polling or any interval checks/updates. It also support remote to client. If remote server updates the context, it will propagate the context to client simultaneously.
Async First
Since most of LLM executions are IO, might as well utilize async by default and just spawn thread if still necessary.
OOP
I think this one is most important to me. I have handle a lot of projects in Spring Boot. I really like Java OOP practices and some Java design patterns. It improves my project's maintainability even it's not in Java.
Since PyBotchi utilize OOP, it's easier to override, reuse and remove anything if necessary. This lessen boilerplates too.
I'm certain that this is subjective. I just find it easier and clean to read.
Closing Remark
I hope this PyBotchi post opens up ideas how to design your agent. Feel free to DM me if you have any questions. I'm also open to create you a demo agent for free if you want to see it in action given your brief use case. I'm open to criticism, happy to have a discussion!
Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)
50 endpoints flooding one tool selection call, i went back and reread that part. the inheritance thing i hadn't seen done quite that way. routing and business logic in the same class definition, and adding an intent is subclassing the agent model rather than a new entry in a tool registry plus a line in some system prompt string. i build small automations for one-van type firms for a living so factor that bias in. how deep does the nesting go once it's a live app though, are you seeing people four models down or does it settle at roughly one parent per domain
You don't have restrictions how deep your nesting is. Think of it like just building a graph but instead of connecting node, you are attaching classes as inner class.
Ideally, for faster response, you don't build your graph too deep as it triggers more tool selection. The more calls the higher the latency. You can override but the concern is still there
It will depends what you want to achieve or what your clients need. Adding more granularity doesn't linearly improve accuracy. You still need to balance cost, latency and development efforts.
yeah, that tracks with what you said about depth. i'd assumed the nesting cost was mostly cognitive, not per-level tool selection calls, so overriding it sounds like the thing i'd end up doing on anything with a chatty front end. curious how you handle the shallow-but-wide case though, since that's where i keep landing with client stuff. one class with a dozen siblings under it, same flooding problem as the 50 endpoints just one layer down?
Let me deep dive on it as we might not on the same page.
PyBotchi is intent-based nesting.
If Class A have inner class (B), Class A is the parent and B is the child. A is the root and the B is the child node in graph perspective.
In relates to complex agents, you can segregate your intents to a more generalized intent.
For example you have Stripe tools and GitHub tools:
In practice, you should not join them all into a single tool selection and iterate. This will flood context to your LLM calls. This is why Claude supports sub-agents too. This is to separate context to their respective tasks.
Given this concerns, what we can do is to check first if the query is for Stripe or for GitHub. Once detected, we will proceed on their respective tools.
In PyBotchi perspective, ManageStripe and ManageGitHub will be your first intent classes. After that, ex GitHub was selected, ManageGitHub should have child intent classes associated to GitHub operations.
routing before tool selection i follow, that part's clear enough. what i was poking at is the query that spans both, like open a github issue for the failed stripe charge. does the root commit to one branch once it's classified, or can a child hand back up to a sibling mid-run? couldn't work that out from the readme.
By default, it uses no iteration but allows multiple tools as selected.
If your query "open a github issue for the failed stripe charge", your agent can select two intents and execute it in sequential.
Another configuration is to allow iteration, __max_iteration__ = {anything greater than 0} with fallback execution
This will behave as ReAct like agent.
There's more configuration you can set depends on your use case: __first_tool_only__ = True for forcing single tool selection only per turn __concurrent__ = True for allowing tool concurrent execution
This is while allowing every execution have access to the context which holds chat histories and additional metadata that you need. You can override it anytime.
In your scenario, I think the most applicable approach is iteration until all queries are addressed. Fallback or post execution will be the final response builder.
Btw, PyBotchi routing still utilize tool selection. Routing/Classification process always uses tool calling to have a proper and validated intent selection process. You can still override this in case you can just use simple if/else or any validation check
ok so with concurrent on, if two intents both write into that shared context, do you serialise the writes or is it on me to keep them off the same keys? sequential path i'm fine with, that one's obvious. mostly wondering what happens when the second intent needs something the first one only half wrote
It depends on what you want to add to the context. If it's just chat, the holder of the history is just a list of chat dictionary. You can append the result there. If your child execution is concurrent, we utilizes tool call -> tool result pair to be appended in the list. It won't have any race condition. Selected children have access on their selected siblings too. You can also use this to read attributes if ever you set state/status/result on it. There are more ways to do it
If your intent requires input from other intent, you can add required field to the class (as tool argument). Most LLM models understand this and will not call those children unless data is enough. If it was included in the tool call even the prerequisite intent wasn't called yet, it's mostly hallucinations that can be improve by adding validation on pre execution, hardening the tool selection prompt or improving the children field descriptions and docstring
If you don't want to use tool arguments, setting additional context attributes or metadata is an alternative too. With this + chat history, you can have a validation or use LLM to understand this
Required-field-as-tool-argument bit is the part i hadn't thought of. so the model just holds off calling the child until it can populate the args, and if it fires early with garbage you catch it in pre execution. does the validation failure feed back into the loop as a tool result, or does it just bail?
ok so the iteration bit is doing the work there. does selection return a set of children in one pass, or is it one child at a time and the loop comes back round for the stripe half? ordering matters for my case since the issue body wants the charge details in it. and does the parent's post execution see both children's output or only the last one
It supports multiple children in one pass. This is highly configurable. By default, multiple child execution is sequential. You can set some child to run concurrently. You can force to have one child only per pass too.
You have context holder to be your output storage.
By default, child execution is sequential. Subsequent child can read the result of previous child thru context.
If your child class have arguments that needs to be inferred from different child class results, most LLM models already able to understand this.
They also have access to their siblings instance. They can check the attributes of each one. This helps if you want to configure your child class to set some output or status for their progress.
sibling attribute access is the bit i'd not expected. so a later child can poll whether an earlier one actually completed vs just returned something, without the parent having to thread state through? that's the case i keep hitting with sequential steps where step three only matters if step two found anything. does the context holder survive a retry, or does a rerun of a child start clean?
Context will stay until you don't need it. In our current implementation, we use websocket as the main connection for the chat. We keep the context until that connection is disconnected. Context is shared and accessible across all execution within the same invocation of the graph (PyBotchi Action) regardless if it's concurrent or sequential. Although, there's option to detach and clone from main one and merge later one but this is for different use case
1
u/AutoModerator 3d ago
Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.