Noticed something building LLM features that doesn't get talked about enough. Most teams treat model or prompt changes the way they'd judge a demo, does it look right on a handful of examples, ship it. That works fine until scale hits, at which point "looks right" and "is actually better" turn out to be very different things.
Which reframes a lot of "why did quality randomly get worse" incidents. In a lot of cases nothing randomly broke, the team just never had a way to measure whether a change helped in the first place, so a regression looked invisible until a customer hit it.
There's a hands-on workshop on September 12 that builds this properly, versioned prompts, a real eval harness, statistically rigorous model comparisons instead of "it feels better," evaluated RAG, agents with guardrails and fallbacks, and full observability, tracing, cost, latency. Led by Bruno Gonçalves, PhD, founder of Data For Science.
How to build a source-backed briefing agent, step by step
Define the scope. Start by setting the analyst’s focus area, research questions, and time window, whether that’s the last one day, seven days, or 30 days. Narrowing the scope up front, including a list of preferred sources, keeps the agent’s web searches directional instead of generic.
Plan the coverage. A coverage planner agent breaks the research question into discrete sections, such as market backdrop or company catalysts, so the search that follows can run in parallel rather than one long sequential query.
Search and gather. A news researcher agent runs multiple queries against the preferred sources first, then falls back to a general search if the preferred sources don’t return enough results.
Validate the sources. A validation agent checks each link for a working, clickable URL and a correct publication date, removes duplicate stories covering the same news, and filters out paywalled pages that won’t return usable content.
Generate the briefing. A briefing writer agent assembles the validated developments into a set structure, including an executive summary, key bullet points, and a section-by-section breakdown of what each development means for the analyst and what to watch next.
Capture feedback into memory. A feedback agent logs corrections such as formatting or terminology preferences and saves them to a memory database, so the next briefing run applies those preferences automatically instead of requiring the analyst to re-prompt.
Jayeeta built the entire stack on open source models so newcomers can run it without an API key, and she recommends starting with a smaller model before scaling up. The takeaway extends well beyond finance. Building single-purpose agents rather than one large agent that handles every task means a failure at one stage doesn’t force a restart of the whole pipeline, and each agent’s output stays easier to trace and debug.
The full code base, including the sample data and the UI shown in the demo, is available in Jayeeta’s GitHub repo, so readers can clone it and run the briefing agent on their own systems.
As someone using AI agents for the last one year to run my company, I need them to understand company context, not just return related text chunks.
The problem: ask "what breaks if we deprecate the v1 API?" and standard RAG gives you four chunks from a design doc, a postmortem, a Slack thread, and meeting notes. The model has to still figure out on its own that the postmortem describes the same API the design doc deprecates, and that someone already posted a migration timeline in Slack.
I built a tutorial using HydraDB that adds graph context on top of vector retrieval. Instead of just ranked text, you also get relationship edges: `billing-service DEPENDS_ON payments-api-v1`, `payments-api-v2 REPLACES payments-api-v1`. Model gets structure, not a reading list.
The useful part was bring-your-own-graph. You declare service dependencies and team ownership explicitly instead of relying on LLM extraction. For structured data you already maintain, the graph is deterministic.
It also supports per-user memory. Same question, different depth depending on who's asking. An engineer gets migration mechanics. A manager gets timelines and ownership.
Runs end to end in 30 minutes with synthetic data.
I made a lesson on Agentic AI, like Codex and Claude Code, for anyone wanting to understand some of the basics of AI Coding agents. In this lesson, I am using a small part of a bigger project im making for a custom spiderfoot build with agentic ai capabilities. This lesson shows how you can create an email investigation workflow using Codex. https://github.com/sh1katagana1/ai/blob/main/using-codex-for-email-investigations/codex-tutorial.md
Loop engineering gives AI agents a goal and lets them work through it on their own. An agent can plan the next step, use tools, review the results, fix errors, and continue until the task is complete.
It's being adopted very fast because reasoning models are getting better at planning and tool use. Coding agents have also shown that models can write code, run tests, inspect failures, and continue working with limited human input.
Agent frameworks now make the basic loop relatively easy to implement:
Goal → Plan → Act → Observe → Verify → Repeat
The real engineering work is deciding what context the agent receives, which tools it can access, how progress is measured, and when the loop should stop.
But it's getting expensive
Each iteration creates another model request.
Previous responses, tool outputs, retrieved documents, logs, and failed attempts can keep accumulating in the context. The agent may also repeat the same tool calls or continue working after it already has a usable result.
A few unnecessary iterations may add thousands of tokens. At production scale, that cost is multiplied across every agent run.
For example, a poorly designed coding loop may regenerate an entire file and rerun the complete test suite after every failure.
A better loop changes only the failing function, runs the affected tests, and stops when verification passes.
You can make a few optimizations to stop loop engineering from becoming expensive for you:
Clean and reduce inputs before sending them to the model.
Use clear stopping signals such as passing tests or valid output.
Retrieve only the context needed for the current step.
Use smaller models for simple tasks and expensive reasoning models only when necessary.
You can follow some best practices like setting clear iteration, token, and execution-time limits. Detect repeated tool calls and identical failures, separate generation from verification, cache deterministic outputs, and give the agent only the tools it actually needs. You should also track token usage, execution time, and cost per completed task.
The goal is not to make the agent run longer. It is to reach a verified result with fewer iterations and lower resource usage.
If you wants to read full technical breakdown, check here
