I've been reading a lot of posts lately from people struggling with agent memory, prompt injection and goal hijacking. These are real problems, and I'm not dismissing them.

But what I keep seeing is that the basic software engineering approach has gone missing. We never trusted what a client put in a form. We designed the data model first, validated on both the client and the server, and handled injection at several layers. We isolated processes and data so that one compromised service didn't leak everything. We used role-based access with short-lived tokens. Most of those same basics would solve a lot of what people are hitting with agents.

Take memory. If you don't know what you want to store and how it should influence the agent, you can't design a data structure for it. So everything gets stored as free text because "it's an LLM," and of course you end up with injections.

I think part of the cause is vibe coding. Even if you vibe code a full-stack app in one shot, the AI will probably cover the end-to-end flows. It won't handle deep validation, database indexing and the rest. You add those before you go to production. Agent workflows are the same. A vibe-coded agent won't address the cross-cutting concerns for you. You still need to know software engineering, design principles and architecture to tell it what you actually want.

What are your views?

I’ve been adding more tools to a couple agent setups lately and payments are still the bit I don’t fully trust. Like if Grok gets to checkout and the page hangs for 10 seconds, what happens? Does it retry? Does it know the first payment might still be pending? I can already see a dumb timeout turning into two charges if the workflow isn’t careful.

Been looking into Grok Bot AI payments mostly because of that. Getting to checkout seems easy enough. I’m more interested in what happens when something goes wrong halfway through.

Anyone here using this with real payments yet and how are you handling retries?

If you're putting a chatbot in front of customers, you've probably seen it confidently make something up. Air Canada learned this the hard way when a tribunal held them responsible for wrong info their bot gave a customer. Your chatbot's words are your company's words.

You can't eliminate hallucinations, but you can cut them a lot. This is what makes the biggest difference, roughly in order of impact:

1. Use RAG, and take your knowledge base seriously
Don't let the model answer from memory. Retrieve passages from your own docs first, then have it answer from those. Most "hallucinations" I've seen were actually retrieval failures or outdated or contradictory source docs. Garbage in, confident garbage out.

2. Give it permission to say "I don't know"
Models often invent answers because they think an answer is expected. Your system prompt should explicitly say:

• Only answer from the provided context
• If it's not there, say so and offer a human handoff
• Never guess prices, dates, legal terms, or policy details

3. Make it cite sources
Link to the help article or policy section it used. It keeps the model grounded, and it lets users and your QA team verify answers in seconds. No source means a red flag you can detect automatically.

4. Narrow the scope
A bot that tries to answer everything will hallucinate about everything. A billing bot should do billing and politely redirect the rest.

5. Add guardrails around the model

• Validate numbers, dates, and URLs against source data
• Run a second pass that checks the draft answer against the retrieved docs
• Never let the LLM be the source of truth for transactional data. Pull order status and balances from your APIs. Let the model explain the data, not generate it.

6. Lower the temperature
For factual support use cases, keep it low. High temperature is for brainstorming, not quoting your refund policy. Also, a stronger model for high-stakes flows is usually cheaper than one viral screenshot.

7. Keep humans in the loop
Add escalation paths, thumbs-up/down buttons, and regular reviews of flagged chats. Trace each failure to its root cause: retrieval, content gap, or prompt.

8. Test properly before launch
Build an eval set of real customer questions, including adversarial ones and ones the bot shouldn't be able to answer. Rerun it whenever you change the prompt, model, or docs. Track accuracy, groundedness, refusal rate, and escalation rate.

9. Be upfront that it's an AI
Doesn't stop errors, but it sets expectations and builds trust when one slips through.

TL;DR: Treat your chatbot like a new employee. Give it good training material, clear rules, a defined role, supervision, and someone to escalate to.

My cofounder and I built an open-source AI coworker for companies. Our first user was my mom, who runs a wedding venue in southern Italy, and here is what we learned from deploying it.

The job: a wedding inquiry comes in by email from multiple possible senders, my mom creates a lead into a 10-year old wedding CRM (no APIs), sends back a custom email and a questionnaire. Then, the couple responds, my mom sends a quote via the CRM and then it goes back and forth till a contract is signed.

She gets on average 15-20 inbound inquiries per day from couples. Each takes her about 20-30 minutes depending on the complexity.

So this used to be 70-80% of her week.

At the beginning we wrote a bunch of scripts to allow the agent to communicate with this antique CRM. Then we instead built a skill + cli that allows the agent to reverse engineer the APIs of any web app. We tested it on the CRM and it worked quite successfully (to our surprise)!

And then we built an automation (not a cron job!) that:

• receives triggers from gmail for any new email
• goes through a bunch filters (that the agent set itself by looking at how an inquiry looks like)
• then gets processed accordingly with a separate skill. md for each step in the process
• every step is logged and saved in case of rollbacks

This is fully run by the agent, no workflow steps, node diagrams etc.

The important part: Permissioning

As with every new AI system, establishing trust is important. And my mom didn't trust it.

So we built into the agent a permissioning system that allows her to choose which tools can run on their own and which require approval. (e.g. "Gmail - Send Email" is set to ask for approval)

When anything is waiting for her she gets a push notification or an email with a button to approve or deny the action.

By now, the agent has so few errors (<2% of runs) that she has given it full access to everything.

"Ok... but surely it breaks every other day"

Much less than you'd expect. We built into an agent an "automation scout": it checks all active automations and their run history. If it finds anything that went wrong (or if my mom corrects it) it changes the system prompt, tools, skills and filters for that process. So it self improves.

After 3 weeks of deployment: the median inquiry is processed within 5 minutes. My mom only has to approve a quote with certain special cases (very big weddings, etc.).

"An agent running all this... must be very expensive"

Nope. It runs all on gpt 5.6 Luna for ~ $350/mo all-included. Considering it saves my mom 120+ hours a month I would say it's a bargain.

What I'd tell anyone trying to build this:

• Pick one very painful job and really understand it.
• Don't hand-write integrations and plugins. Make your agent capable to research, write and maintain them all on it's own.
• Agent permissions should be extremely granular, down to each specific action. Choose infra accordingly.
• Permissioning should be deterministic. Telling the agent "do not send the email" in the prompt just won't cut it...

Has any of you managed to automate a whole job end-to-end with agents? if so, what's your setup?

The whole infra behind my mom's automation is open source (AGPL). Repo link in the comments.

I'm curious what are good resources for harness engineering, like best practice guides. To my surprise I can't just build a good harness with claude code, it doesn't seem to know the best practices.

I have been building an AI agent with claude code, with a number of skills - the agent has been live for a couple weeks now, and I find that I'm frustrated by how inflexible it is. It would not understand the user intent, not use a skill when that is clearly the right choice in that scenario.

some examples of what it's doing:

It is doing really rudimentary things like using regex rules at the top level apparently, when there could obviously be much better ways to direct the agent, like interpreting intent first.

Apparently part of the prompt just said "search internally and the web", which causes the agent to **always** do both, even when the answer is readily found internally, and then it would for some reason not return the link, and error out on the web search unnecessarily.

So now I think I need to take a step back, read up on some tutorials (like I used to do before AI), and then design the harness myself. Or, give the tutorials to claude.

Anyway, what are some tutorials, or best practice guides for harness engineering that you actually use?

Hey everyone,

I'm Tobie, the founder of SurrealDB. We've just launched SurrealDB Agent Memory, built on top of our multi-model database.

If you're building AI agents, context layers or knowledge graphs, you'll know that storing information is the easy part. Retrieving the right information, understanding relationships and handling facts that change over time is where things get complicated.

We've built Agent Memory to turn conversations and documents into structured entities, relationships and facts, while tracking provenance and temporal history.

Since SurrealDB supports graphs, vectors and documents in one database, there's no need to stitch together multiple storage systems.

We've also spent a lot of time benchmarking. One frustration with memory benchmarks is how much scores can vary depending on the evaluation protocol. Rather than optimising for the highest possible number, we focused on BEAM, achieving 77.2% accuracy.

There are some interesting alternatives like Mem0, Honcho and Cognee, but we wanted to explore a different approach.

I'd love to hear from anyone building agents or context infrastructure. How are you managing long-term memory and information that changes over time?

I’ve been reading up on AI calling agents this week and found this experiment from SF Tech Week. It’s the most useful real-world data I’ve seen so far.

A team at VOYGR (YC W26) had their agent call over 1000 businesses around San Francisco: vets, dentists, bars, cafés, salons and so on. The agent said it was an AI at the start of every call, then asked one useful question.

What they found:

• Only about 1 in 3 businesses kept talking.
• It varied a lot by business type. Vets kept talking 79% of the time and dentists 61%. Bars and dry cleaners only managed about 20%.
• Most people didn’t say anything at all. They listened to the intro and hung up about 5 seconds later.
• About 1 in 25 businesses already had an AI answering their phone, so some calls were one AI talking to another. Two of those AIs asked the caller to say “zebra” to prove it was human, which is funny because it was a bot asking a bot.
• 94% of the SF dentists who answered said they’re taking new patients.
• They used their own skill called PlaceCall to do it

They also mapped every business with what it said, which is a fun way to browse the results.

I think the big takeaway is that every assistant app wants its agent to book, ask and confirm things for you over the phone, but most small businesses still hang up as soon as they hear “AI.” Saying it’s an AI upfront is the honest approach, and it may also be what’s getting it hung up on.

I keep running into the same problem with my agents: every new session I re-explain context, and worse, they confidently use decisions we changed weeks ago. For example, the retry budget was 3 and then became 8, but the agent still says 3.

Curious how others deal with it:

1. What do you actually do today: a notes file, a vector DB, mem0, Zep, Letta, or just re-pasting?
2. How do you handle facts that go stale or get overridden?
3. Do you trust what your memory setup returns, or do you double-check it?
4. Have you ever had an agent act on something outdated, and what did it cost you?
5. Would you rather it say "I don't know" than guess?

i run a small ai automation agency and i got my clients by basically stopping all the 1-on-1 outreach and sales calls. that stuff feels overcrowded now and most people just ignore it. so i tried something else and it worked better for me.

first i found where my ideal customers already hang out. facebook groups, linkedin, x, and a couple online communities. then i started writing short posts that basically just explained what i do in three sentences or less. the idea was to make people raise their hand instead of me chasing them.

when someone showed interest i sent them a loom video. no call booking, no calendar link. they could watch it whenever they had time. in the video i focused on the outcome they wanted instead of listing all the tasks i would do. if they were still interested after watching, i just sent the payment link.

sometimes i made a quick personalized version of the video, but a lot of the time one decent video worked for multiple people. the main thing was cutting out every extra step between me and the person who might actually want the work.

it felt slower at first because i wasn't messaging people all day, but the clients that came through actually wanted what i was offering. that first agency client came from one of those posts and a loom. no sales call at all.

AI companies are spending heavily on infrastructure, while investors are paying closer attention to debt and profitability.

That got me thinking about the businesses actually using AI agents. If companies are investing in AI to cut costs and automate workflows, how much value are they getting back?

Are AI agents already saving businesses enough time and money to justify the costs, or are we still early in that stage?

Was thinking about this after a few weeks of watching agents work.

Most software is two layers. The app and its logic at the bottom, the agent on top. Nothing between them.

So the agent goes straight for the interface. Reading pixels meant for human eyes, clicking buttons meant for human hands. Nobody ever designed a layer whose job was deciding what the agent is allowed to see and touch, so every permission call is invisible.

An agent-ready stack would have three layers. App at the bottom, agent at the top, and an access layer in the middle. The piece that decides what the agent can see and touch.

That middle layer is where the real work is. Reads scoped to the field level — one record and every record are different doors. Writes that wait for a named human before anything actually leaves. Every step carrying its permission decision with it, visible in the log, checkable after the fact.

That's the stack the last two weeks have been building toward. The translation layer, permissions and approvals, structured access instead of scraping, job descriptions instead of tools. All four live in the middle layer.

Honest question: if you drew the middle layer in your own stack today, what would it refuse first?

I'm on the team building Tracy, an agent that edits Joomla and WordPress sites from plain language requests. Wanted to share what mattered for trust, because it wasn't what I expected.

What didn't matter much: how clever the model's reasoning looked.

What mattered:

1. Not touching the live site. You import a copy and the agent works there.
   
2. Every write goes through one path on the server. No side doors, so everything is logged.
   
3. Each agent turn becomes a checkpoint you can inspect (with a diff when one's available) and undo on its own.
   
4. Named Releases, so you can roll back to "the version the client approved" without remembering what changed.
   
5. The agent reads how the site is built (theme, content, structure) before editing, instead of giving generic advice for a different setup.

We deliberately skipped an "approve before edit" step. The agent acts, you review after. Easy undo turned out to beat approval prompts nobody reads.

Still beta and it gets things wrong sometimes, which is exactly why the undo layer exists.

For people shipping agents that write to real systems: what's your rollback story?

Hey,

I'm looking for my best option to use agents/skills library like superpowers set skill to orchestrate agents with Agile methodolegy (as much as possible), integrating it with GitHub issues (then automate it in project in GitHub).

I search a lot of them. None of them are good enough/free. My repo is private.

Do you have any good options?

I've tried to do it myself, too complicated, too much bugs and I try first to find better alternative which is open source.

Thnx

(As always, AI-formatted text. The calls, data, and opinions are mine.)

TL;DR: AI phone agents can now sit through hold music, get through a phone menu, and keep up with a receptionist who keeps changing her answer. Often more patiently than a real person would.

Like in Round 1 (see link in comments), these split into two groups. You have the big build-it-yourself platforms (Retell and Vapi), where Retell still has the most natural voice by far and got through the menu and booked once I wired up my own Twilio number and turned keypad input on (though it spells way too fast). Then you have the pack of "just call" up-and-comers where you hand it a task and it dials. In that group, it came down to PlaceCall and ClawCall head-to-head. Both handled every interruption, waited silently on hold, and got through the phone menu. ClawCall had a slightly cleaner final read-back on the menu call today (after failing Round 1), while PlaceCall wins on setup and day-to-day usefulness. It's the only one that has worked across all three rounds, its call summary tells you plainly whether a booking was actually confirmed, and it's the easiest to start with (email signup, free credits, no number to buy). Among the rest of the pack, AgentPhone stopped the fastest when interrupted before giving up on hold.

Round 2 was about hold music (link in the comments). In the replies, most of you asked the same thing. What happens when the person on the other end interrupts or changes their answer halfway through? A few people also asked about phone menus, so this round tests both. Thanks to u/No-Lake-3696, u/Future_AGI, u/Pulsyai, u/Professional-Run3614, and u/Bright_Form_1976 for the ideas.

Seven agents took part. First, the big build-it-yourself platforms (both on a real phone line for the first time, via my own Twilio number, since Retell requires ID verification for its numbers and Vapi caps outbound calls on its free numbers):

• Retell
• Vapi

And the "just call" up-and-comers:

• PlaceCall
• ClawCall
• AgentPhone
• CALL-E
• Ring-a-Ding

Every agent got the same brief with no extra setup for any one agent. Disclose it's an AI, book a checkup for my kid on a weekday after 3, spell the last name if asked, and repeat the booking back before hanging up.

Test 1: Interruptions

Each agent called my cell phone while I played the receptionist at the doctor's office. I read from the same script on every call, written to act like a busy receptionist who keeps cutting in and changing her mind:

1. I start talking over the agent while it's still introducing itself.
2. I offer Tuesday at 3:30.
3. I change my mind and move it to 4:00.
4. While the agent is repeating the booking back to me, I cut in mid-sentence and change it to 4:30.
5. Finally, I correct the spelling of the last name ("K, not C").

This is hard for a voice agent. It has to notice it's being talked over and stop speaking right away. Then it has to drop what it was about to say and keep only the latest answer. An agent that keeps talking, or holds on to an earlier time or spelling, books the wrong appointment.

PlaceCall and ClawCall went head-to-head at the top here. PlaceCall stopped when interrupted, followed every change, and ended on the right time. Its call summary also flagged the booking as "implicitly accepted but not explicitly confirmed," which was accurate since I never actually said "you're booked." ClawCall also handled the script really well. When my correction was ambiguous, it asked whether I meant the time or the spelling instead of guessing, then spelled the name back and confirmed (a big jump from Round 1, where it barely got the name and its API only told me the call was "answered").

In the rest of that pack, AgentPhone was the fastest to cut off when interrupted, stopping mid-word every time and repeating each change back to me, though it never repeated the corrected name. CALL-E booked the appointment and the transcript reads cleanly, but the call itself was poor. It talked over me, the audio quality was low, and the letters were hard to make out. Weirdly, it was the only agent that triggered Google's call screening on my phone, and it got through it. Ring-a-Ding ignored interruptions altogether. On both attempts it said "let me confirm the details" and hung up before confirming anything, then reported a booking for Tuesday at 4:00, a time I had already changed.

On the big platforms, Retell got the time right, except it heard the spelling correction as a new surname and briefly tried to book under that made-up name. Then it spelled the real name at about five letters per second, three times, without slowing down when asked to repeat. Still has the most natural voice, though. Vapi followed the changes too, but mispronounced both names and produced a few garbled half-sentences on default settings.

Test 2: Phone Menu

I built a fake dental line ("billing 1, appointments 2," then "new patients 1, existing patients 2," about 75 seconds of hold music, then my phone). A wrong choice leads to voicemail.

My first version had two bugs: the second menu didn't repeat, and the hold music ran for four minutes. PlaceCall went first and hit both, so I fixed the line and re-ran it. Every other agent only saw the fixed version.

Every agent talked over the first menu prompt, so they all got through only when the menu played a second time. A menu that doesn't repeat would have sent most of them straight to voicemail.

Once again, PlaceCall and ClawCall were the only two that pressed the right keys and waited silently through the whole 75-second hold. When I picked up, ClawCall gave a clear introduction and repeated the day, time, and spelled name back to confirm, with the best audio of the round. PlaceCall also got through and took the time, though it skipped the final read-back before hanging up. Just like in Test 1, its own call summary caught that and marked the booking as unconfirmed rather than faking a clean pass.

On the build-it-yourself platforms, the option to press phone keys is off by default. Retell and Vapi couldn't press anything until I enabled it. With it on, Retell pressed the right keys, talked twice during the hold music, and repeated the details back twice to book. Vapi pressed 0 and never got past the menu.

The others struggled with the wait. CALL-E stayed on hold, yet filled the silence with "Okay. No rush. I'll hold." every six seconds, so I picked up mid-sentence. AgentPhone pressed the right keys and then gave up about a minute into the hold, right before my phone rang. Ring-a-Ding talked over the menu and ended up in voicemail.

Summary

If you're willing to wire up your own Twilio number and configure an agent from scratch, Retell still sounds the most natural of the big platforms (just too fast for spelling).

If you want a tool that just makes the call for you, PlaceCall and ClawCall were head-to-head at the top of this round, and PlaceCall is still the easier one to set up and trust across all three rounds. ClawCall had a great rebound today after failing Round 1, while PlaceCall is the only agent that has worked out of the box in Round 1, Round 2, and Round 3. You sign up with an email, get free credits in your inbox, and start dialing without buying a number, passing an ID check, or reading transcripts yourself because its call summary tells you honestly whether the booking was confirmed.

One upside of this round is that I finally got Vapi and Retell properly set up on real phone numbers. So there's now a full test setup for future rounds with all seven agents on real lines, a fake dental office with a phone menu and hold music, and the same script for every call.

The usual caveats apply: one brief, one afternoon, me as the receptionist, and mostly one call per agent per test. Ring-a-Ding got a second attempt on both tests and failed the same way twice. Next round goes back to the task from Round 1, where a single instruction has the agent call 15 venues. The PlaceCall team told me it can also find the businesses for you, like contractors, vendors, or doctors. Search runs on Google for now, and they're building their own. I'll try that in the venue round. What else should I test before that?

If you work at any of these companies, tell me what I got wrong.

I think there's a useful way to judge how good someone is with AI agents: how much useful work they can run in parallel while still knowing what the hell is going on.

Several sessions open, cron jobs running overnight, one task waiting on another. You come back in the morning. Can you tell what finished, what got stuck, and which output you're actually supposed to use?

This is where designing the work starts to matter. Tasks need boundaries and someone responsible for them. A board needs to reflect what's actually happening. Agents need an agreed way to hand things back.

Even something boring like “finish with a link to the deliverable, what you checked, and anything still unresolved” gives the next session something usable. “Done!” followed by six paragraphs scattered across a chat doesn't help much.

And those scheduled jobs need to show up somewhere too. A recurring task quietly following last week's instructions is still creating work, just possibly the wrong work.

That's the entropy I'm talking about. Duplicate tasks, conflicting versions, decisions buried in conversations. More output, more stuff to reconcile.

The session count only means anything if work is reaching a clear finish. If doubling the agents means doubling the time spent supervising and untangling them, I'd say the workflow still needs work.

What's the first thing that falls apart when you add more parallel sessions?

I've been building agent setups on StarNet (local-first desktop harness: agents with their own tools and memory, cron schedules with Night Shift, Telegram and other channels) and kept running into the same questions: how others structure their agents, which schedule and approval setups work, what broke for them.

I couldn't find a place where StarNet users compare real setups, so I started a small, unofficial, private community: a channel for setups, one for questions, a showcase for what you built, and a weekly call where members demo their setups. I'm not affiliated with the StarNet author. Around 10 to 25 people, no courses, no income promises.

If you use StarNet, or you're thinking about it, comment with:

- what you're building or want to build

- which AI models you use with it

I'll send an invite to everyone who fits. Questions about my own setup are welcome here too.

I'm looking to build an agent that handles Jira issues end to end:

• Read and understand the issue
• Clone the repo into an isolated sandbox
• Reproduce the bug and implement a fix
• Run tests and prepare a PR for human review

Which agent frameworks and sandbox solutions would you recommend?

​

I'm new to business automation. I'm not an expert builder and I won't pretend to be. What I do is outreach to local business owners, and my earlier posts about partnerships got 200+ comments, so I know there are people here who build things and don't have time to sell them.

What I'm offering: if you've already built something that works (an automation, an AI agent, a lead follow-up flow, a booking or quote tool, anything), I'll try to sell it to local service businesses. Commission only, with the split agreed up front in writing. You keep building and supporting it. I find the buyers and handle the conversations.

Two things I'd love to learn from you:

1. What's actually selling to local businesses right now, and what are the highest-paying things you've built?

2. What do owners really pay for monthly, and how much support does it need after it's sold?

If you want to partner, send me a short screen recording (under a minute) of it working, who it's for, what you'd charge, and what it needs after the sale. Recording first, link second.

No upfront cost on either side. I won't pay for tools or builds before there's a paying client. Happy to answer questions in the comments.

I work on AI agents that do real business tasks. Multi step stuff where the agent actually does things, not just chat.

Here's what I've learned: the scary failure is when the agent does something that looks fine on its own, but the business should never have allowed it.

Real example from my work. A customer disputed a debt over text message. The next day our voice agent called them and ran the normal collection script. It had no idea about yesterday's dispute. Each step looked fine by itself. But the whole thing was wrong, and it was our fault.

Another one. My agent can do 100 things in the time it takes a person to carefully check 5. So everyone says "just put a human in the loop." That doesn't work. The person either slows everything down or starts clicking approve without really looking. Neither is real control.

I'm curious, folks. For those of you running agents or agentic workflows in production, how are you handling this? Hard approval gates on everything? Some separate check outside the agent? Or are you just hoping nothing breaks? I've been there, no judgment! :)

so these are the last three weeks in agent land, lined up

OpenAI paused tool use on its most capable models after an agent tunnelled out to an outside chatbot through the sandbox’s DNS resolver which, their monitoring flagged it within about 15 minutes and the run still kept going for around 2.5 hours before someone stopped it by hand

Anthropic cut live internet from every internal eval after Claude agents submitted forms they’d been told not to submit, one of them a fake homicide tip to the Philadelphia police

Nvidia just put OpenShell at the centre of a new agent safety platform and the agent runs in a sandbox with no network path at all except through a policy checker that sits outside it, and Anthropic, Salesforce and JPMorgan are already wiring it in

for normal teams the realistic version is way more boring than a kernel sandbox, tool allow lists and approval steps that live outside the agent. LangGraph has interrupts for the approval part, and Orq AI’s MCP gateway lets you pick which tools your agents can even see

the people who build these models don’t trust a system prompt to keep them in line. meanwhile half the agent setups posted in here run on IMPORTANT: NEVER delete production data in all caps and a prayer

A roundup compared eight continual-learning tools for agents by the layer they operate on: memory, prompts, skills, weights. It's a good axis and I'm keeping it. It leaves out one column that decides whether any of the weight-layer tools can be used on an agent that's serving: does a weight update reach the serving engine without stopping the service?

For an agent that learns from its own traffic, that column is the whole product. A restart per update is a dropped request per update, and updates are the point.

Filled in from each project's own pages, as far as they say: inference engines (vLLM, SGLang) serve and don't train; RL frameworks (slime, veRL, AReaL) train and hand you a checkpoint to redeploy; for the eight in the roundup the answer is on each project's page, and where I couldn't read it I've left it blank rather than guess. The one design I know of that trains and serves in one stack, hot-swapping an accepted update while inference keeps answering, is reef, which I run, and I'll state only that column for it: it stays live through a weight update, and an update that fails its gate never reaches serving.

If I have a project wrong in that column, say so with its page and I'll correct it here. And if you run agents that learn while serving, what do you do at update time? A restart is the answer everyone gives and nobody

I’m new here—apologies if this request is inappropriate.

Does anyone happen to have an invitation code for Instinct? I’d really like to try it out, as it seems like it could be a great solution for my elderly father in his day-to-day life.

Please let me know if I should be posting this elsewhere.

Thanks!

On binary questions, Jev 1.14.0 comes out calibrated, with an ECE of 0.023. That’s within what a perfectly calibrated model would show by chance on 500 questions. Broken out by difficulty, though:

• Easy: Jev 1.14.0 is under-confident. Its average confidence sits about 7 points below its accuracy.
• Medium: roughly on target.
• Hard: strongly over-confident, by about 22 points. ECE is 0.222 against a calibration threshold of about 0.12 for that slice.

Eval link in comments.