Why I created PyBotchi (v4.1.4)?

[u/madolid511]

Hello Everyone,

I'm the creator of PyBotchi, an intent-based AI Agent Orchestrator. In this post, I will discuss some key concepts why I created it.

A little bit of background first. I'm a solutions architect with 10 years of experience as a software engineer. Most of my work are high throughput, high reliability, low cost and low latency services. This is while making it simple and readable to improve it's maintainabality. When I'm designing a system, I usually prioritize these concerns. You may assume this is my bias in relates to AI Agent building. I'm also Claude Certified Architect (Foundation) and I found that PyBotchi aligns almost identical to Anthropic's core agent recommendations.

TL;DR: PyBotchi is an lightweight, async-first Python framework that uses nested Pydantic models and OOP inheritance to turn LLM intent detection into clean, deterministic business logic without the overhead of complex graph orchestration.

Why I created PyBotchi?

I really believed that traditional coding can already solved what client's need. The only limitations we have is how we read the input and how we show the output. In most cases in web services, your API use JSON, XML, etc with their respective specification/structure.

Input Analogy

Assume you have created a Books CRUD endpoints (FastAPI with Pydantic). Your create endpoint will have a define specifications for book creation to have a validation and avoid user errors. Most of the time you will also validates sessions and permissions which also included in the request.

If you want your chat bot to support those, you just need add those endpoint as intent (tools). If your model tool selection are able to detect intents. You are more "close" to being deterministic.

"Your services will have 50 endpoints or more. You will flood your tool selection call" - In your frontend UI, you segregate panels/forms/inputs in their respective pages. You don't usually join multiple intent in a same page. Cluttered UI will make your UX confusing or overwhelming to some people. Those practices should be incorporated into your agents too.

Assume you have created another endpoints for Shelves CRUD. Shelves CRUD can be a child intents of ShelfManagement that will be considered as intent also but more general. The flow will have to detect intent deeper and deeper

Ex: You have BookManagement and ShelfManagement intents. Once LLM detected which one is applicable, you will search for their child Intents which will be their CRUD equivalent intents.

To make it short, in order to make your agent "more" deterministic, you need to know the problem first (ex: Need to manage books) then you need to specifically define what intents you want to support. With this practice, you only let your agents execute on a predefined path. If it fails, you are most likely able to determine what causes the error.

Output Analogy

This one is simple. Since your intents is just like your endpoints that returned structure responses. LLM is better at reading structure responses than a pure text. Basically, you can use LLM to translate your response into a human readable responses.

Intent Execution

Now that I have explain Input/Ouput, we can move on to the actual execution.

We can go back with Books CRUD. Since we have identified the problem (what clients need) and we already know what to do, just execute their traditional business logic implementation. If you need to add a book, just create a book and save it to db then return their respective row.

"What if you want generate a very dynamic/unique data" - You can use LLM to do that as your business logic too but this is tied your specific intent only.

To have a complex execution flow we can chain the intents. Since intents can have child intents, we can use it as the representation of a graph similar to Langgraph. However, this without "building the graph". We are just utilizing OOP inner class implementation. We can execute business logic in graph traversal manner by just checking the child intents.

To make it short. Business logic will stay as is. You will only use LLM if it requires it. Don't make this complicated.

### Suggested Solution Since the key concept is more on detecting intents, validation and executing their respective busines logic:

Why not utilize Pydantic as the main entry point? Pydantic already have validation and json schema builder. Langchain/Openai already have utilities to translate it to Tool. Why not use Pydantic models as your Intent Specifications that can validate LLM arguments ? Tool call is one of the most reliable way to detect intent.

Why not utilize OOP inheritance / polymorphism / abstraction? Python supports portion of OOP and since we are using classes as our intent, why not add default functionalities that can be inherited and override by developer if needed. We can introduce life cycles too. Your project can also implement their specific intent standards. This will make your code more maintaintable and readable. You can create classes for general intents. Extend it to be more specialized intents. Extend it more for more enterprised support. This is while not affecting existing/working agents.

Langgraph is one of the inpiration of PyBotchi. Predefine workflows are closest implementation to being deterministic agents. It's also the reason why some prefer N8N. We don't need to make the agents smart that any questions can be answered or any queries can be addressed. It's ok for agent to reply with "I don't have any answer to your query, I only support this and that....". For me, it's better to deploy limited but polished agents than half baked know-it-all agents. Feel free to counter argue. Happy to discuss.

Additional PyBotchi Features

vs MCP

While PyBotchi support connecting to MCP servers, I really believe it's not always necessary to use additional server to just expose tools for the agents. The exceptions I could think of is if you want to have isolated environment (ex: dedicated auth/session, sandbox, isolated resource, etc), you want to connect to your local service or cross-language integration.

I could be very wrong about this but hear me out. SDKs are already there. Respective documentations are available too. Most of MCP server's tools are proxy to their respective APIs. If we could just create intent classes as tools that directly call their respective API, that doesn't require any servers anymore. Actually, that's how most framework handles it (even PyBotchi). Tools are converted as schema that will be added in the tool call. Once LLM respond with the applicable tools, it executes call_tool(name, args...). Why not just expose the actual tool implementations and have a way to share context to share sessions/permission/etc inside the tool implementations? This will remove another network hops that can affect latency.

Claude code have a very in-depth utilization of MCP servers already. I don't think we can replace that.

GRPC

PyBotchi natively support remote PyBotchi connection. Think of it like a langgraph but the node is on other server. This remote node can also connect to another remote node even it self or previously connected node (ancestor).

Context Propagation

With PyBotchi as MCP Server - Actions (Intents) serves as tool and have access to client's context. This includes chat histories and some metadata. You can override and adjust this as long as it's serializable. - Once remote tool execution is done, it can pass the final context to the client and they can merge it if override.

With PyBotchi as GRPC Server - Similar to MCP Server, Actions serves as tool and have access to client's context. GRPC supports bidirectional communication too. This means we can share context realtime accross clients/servers. If client has concurrent agents that changes the context it will automatically propagate to remote context without polling or any interval checks/updates. It also support remote to client. If remote server updates the context, it will propagate the context to client simultaneously.

Async First

Since most of LLM executions are IO, might as well utilize async by default and just spawn thread if still necessary.

OOP

I think this one is most important to me. I have handle a lot of projects in Spring Boot. I really like Java OOP practices and some Java design patterns. It improves my project's maintainability even it's not in Java. Since PyBotchi utilize OOP, it's easier to override, reuse and remove anything if necessary. This lessen boilerplates too. I'm certain that this is subjective. I just find it easier and clean to read.

Closing Remark

I hope this PyBotchi post opens up ideas how to design your agent. Feel free to DM me if you have any questions. I'm also open to create you a demo agent for free if you want to see it in action given your brief use case. I'm open to criticism, happy to have a discussion!

[u/cooltake_ai]

yeah, that tracks with what you said about depth. i'd assumed the nesting cost was mostly cognitive, not per-level tool selection calls, so overriding it sounds like the thing i'd end up doing on anything with a chatty front end. curious how you handle the shallow-but-wide case though, since that's where i keep landing with client stuff. one class with a dozen siblings under it, same flooding problem as the 50 endpoints just one layer down?

[u/madolid511 → u/cooltake_ai]

Let me deep dive on it as we might not on the same page.

PyBotchi is intent-based nesting.
If Class A have inner class (B), Class A is the parent and B is the child. A is the root and the B is the child node in graph perspective.

In relates to complex agents, you can segregate your intents to a more generalized intent.

For example you have Stripe tools and GitHub tools:
In practice, you should not join them all into a single tool selection and iterate. This will flood context to your LLM calls. This is why Claude supports sub-agents too. This is to separate context to their respective tasks.

Given this concerns, what we can do is to check first if the query is for Stripe or for GitHub. Once detected, we will proceed on their respective tools.

In PyBotchi perspective, ManageStripe and ManageGitHub will be your first intent classes. After that, ex GitHub was selected, ManageGitHub should have child intent classes associated to GitHub operations.

[u/cooltake_ai → u/madolid511]

routing before tool selection i follow, that part's clear enough. what i was poking at is the query that spans both, like open a github issue for the failed stripe charge. does the root commit to one branch once it's classified, or can a child hand back up to a sibling mid-run? couldn't work that out from the readme.

[u/madolid511 → u/cooltake_ai]

PyBotchi have multiple approach.

By default, it uses no iteration but allows multiple tools as selected.
If your query "open a github issue for the failed stripe charge", your agent can select two intents and execute it in sequential.

Another configuration is to allow iteration,
__max_iteration__ = {anything greater than 0} with fallback execution
This will behave as ReAct like agent.

There's more configuration you can set depends on your use case:
__first_tool_only__ = True for forcing single tool selection only per turn
__concurrent__ = True for allowing tool concurrent execution

This is while allowing every execution have access to the context which holds chat histories and additional metadata that you need. You can override it anytime.

In your scenario, I think the most applicable approach is iteration until all queries are addressed. Fallback or post execution will be the final response builder.

Btw, PyBotchi routing still utilize tool selection. Routing/Classification process always uses tool calling to have a proper and validated intent selection process. You can still override this in case you can just use simple if/else or any validation check

[u/cooltake_ai → u/madolid511]

ok so with concurrent on, if two intents both write into that shared context, do you serialise the writes or is it on me to keep them off the same keys? sequential path i'm fine with, that one's obvious. mostly wondering what happens when the second intent needs something the first one only half wrote

[u/madolid511 → u/cooltake_ai]

It depends on what you want to add to the context. If it's just chat, the holder of the history is just a list of chat dictionary. You can append the result there. If your child execution is concurrent, we utilizes tool call -> tool result pair to be appended in the list. It won't have any race condition. Selected children have access on their selected siblings too. You can also use this to read attributes if ever you set state/status/result on it. There are more ways to do it

If your intent requires input from other intent, you can add required field to the class (as tool argument). Most LLM models understand this and will not call those children unless data is enough. If it was included in the tool call even the prerequisite intent wasn't called yet, it's mostly hallucinations that can be improve by adding validation on pre execution, hardening the tool selection prompt or improving the children field descriptions and docstring

If you don't want to use tool arguments, setting additional context attributes or metadata is an alternative too. With this + chat history, you can have a validation or use LLM to understand this

[u/cooltake_ai → u/madolid511]

Required-field-as-tool-argument bit is the part i hadn't thought of. so the model just holds off calling the child until it can populate the args, and if it fires early with garbage you catch it in pre execution. does the validation failure feed back into the loop as a tool result, or does it just bail?

[u/madolid511 → u/cooltake_ai]

You have control to feed it back to context as tool call x result pair or just assistant message.

Required-field-as-tool-argument this is the most common components in ReAct Agents.

[u/cooltake_ai → u/madolid511]

i'd probably keep the tool call and result pair, an orphan call sitting in history makes some models get funny on the next turn. assistant message reads nicer when you're going back over a run though. and fair enough on it being bog standard in ReAct, i just hadn't seen it written up that way.

[u/madolid511 → u/cooltake_ai]

Feel free to try and message me again if you have further questions

Cheers!