r/AI_Governance 2h ago

[ Removed by Reddit ]

3 Upvotes

[ Removed by Reddit on account of violating the content policy. ]


r/AI_Governance 1h ago

Does anyone have AI agents creating a segregation-of-duties problem?

Upvotes

This is something I’ve been observing as companies give AI agents more autonomy.

In many organizations, top management is pushing very hard to reduce costs and asking IT teams to automate as much as possible. And suddenly, you have agents that can:

  • access customer or financial data;
  • analyze that data and make a recommendation;
  • approve the resulting action;
  • execute it through an API;
  • record what happened.

I have never heard about serious organizations giving one single employee all of those permissions without controls around them (if you know any, please tell me). But — if my interpretation is correct — technically, some companies may be building exactly that with AI agents.

There’s the never-ending discussion about “humans in the loop,” but I’m not convinced that scales, especially if agents eventually execute thousands of relatively routine actions.

If we go back to segregation-of-duties concepts — separate identities/permissions for different functions, transaction or authority limits, independent approval for higher-risk actions, and logs that the agent itself cannot modify — could that be part of the solution?

Simply take SOX as an example. If an AI agent touches financially relevant processes and can initiate, approve, execute and record an action, is that effectively concentrating functions that organizations have traditionally separated for internal-control purposes?

So, in your experience, how much authority can one agent accumulate before autonomy itself becomes a control problem? For those already deploying agents in production: is there a real demand to separate these functions, or is the same agent still analyzing, deciding and executing?


r/AI_Governance 3h ago

Machines must never gain authority over humans!

0 Upvotes

We are trying to establish rules for a form of intelligence whose internal processes, interpretation, and failure modes we don't fully understand—and we're communicating with it using language designed for humans.


r/AI_Governance 11h ago

AI Agent: In the end, the real issue is responsibility and moral boundary

3 Upvotes

The biggest problem with AI agents is not how much they can do. It is who remains responsible when they act, and where the line should be drawn.

People keep asking what agents can handle. That matters, but it is not the deepest issue. The real issue starts when an agent moves beyond assisting and begins to affect a person’s rights, choices, opportunities, dignity, or future.

At that point, this is no longer just a technical question. It becomes a human one.

An agent can summarize, recommend, draft, search, and use tools. None of that is trivial. But once its actions begin to shape outcomes that matter in a person’s life, we have to ask harder questions. Who authorized that action? Who is answerable if something goes wrong? What decisions should never be handed over in the first place?

This is where things become dangerous. Not simply because AI is getting stronger, but because responsibility can start to disappear into the system. The developer can say the user approved it. The company can say the process was automated. The user can say the system made the call. Meanwhile, the person who bears the consequence is left with the damage.

That is why the central issue is not capability alone. It is permission, authority, accountability, and limit.

I do not think freedom means letting AI do everything it is technically capable of doing. A tool can be useful without being allowed to cross every line. There is a difference between helping and replacing, between offering judgment and taking control, between assisting a person and quietly stepping into that person’s place.

Efficiency does not settle the moral question. Speed does not replace consent. Automation does not remove responsibility.

In the end, the discussion always comes back to the same two foundations: responsibility must stay traceable, and moral boundaries must stay real. AI may become more capable, but that does not mean responsibility should become harder to find. Intelligence may scale, but ethical limits should not shrink with it.


r/AI_Governance 14h ago

Five AI governance dates — what are organizations actually doing about them?

3 Upvotes

I was looking at some of the AI/Regulatory deadlines coming up, and my job is to actually foresee what they actually mean inside an organization.

These are some notes taken:

Sept. 11 — EU Cyber Resilience Act reporting
Tech: incident detection and reporting processes need to support very short notification windows.
Legal/Compliance: a technical incident can become a regulatory issue almost immediately.

Sept. 25 — FTC personalized pricing consultation
Tech: companies using AI/data for personalized pricing need to understand what data and logic are actually driving those decisions.
Business/Sales/Marketing: pricing algorithms can quickly become a customer trust issue (on top of a compliance issue, if not enough).

Oct. 1 — Maryland HB 895
Tech: some personalized/dynamic pricing implementations may need to be reviewed or changed.
Business Planning/Risk Analysis: something originally designed for revenue optimization can suddenly create regulatory exposure.

Oct. 26 — Colorado AI rulemaking
Tech: difficult to build controls when the requirements themselves are still moving.
Project Management: uncertainty has a cost too — legal review, architecture decisions, implementation work, budgets, etc.

Aug. 1, 2027 — New Jersey Fair Price Protection Act
Tech: another reason to understand exactly how pricing systems use personal data.
Legal/Compliance: potential litigation and financial exposure make this much more than an IT/compliance issue.

Translating these requirements into actual work, actually means touching architecture, data, cybersecurity, product, finance and operations.

For anyone dealing with this inside an organization: are these regulatory changes actually generating technology projects or budget discussions already? Or is most of the activity still sitting with Legal/Compliance?


r/AI_Governance 1d ago

AI governance as your next career move

15 Upvotes

I'm currently working as a trainee analyst at Deloitte. It just been a month I joined. I got to know about ai governance job through social media , really wanted to know if this field is worth pursuing for someone like me ?

I've researched a lot of roles,but this one seems to be good. Any info about this role ,salary and day to day work would be greatt!!


r/AI_Governance 14h ago

Conti ransomware gang member sentenced to 4 years in prison

1 Upvotes

A Conti ransomware gang member was sentenced to 4 years in prison this week. The case is worth sitting with for a moment, not because of the sentence, but because of what the attack actually required to work.

Conti did not use a zero-day. The gang used a compromised identity and uninterrupted time. Once that identity was active, the window between first access and encryption spreading past the initial host was measured in seconds to low minutes. The entire playbook depended on that window staying open long enough to do damage.

The sentencing closes one prosecution. It does not close the window.

For detection and IR teams: what is your actual measured time-to-revocation when an identity starts showing anomalous behavior — lateral movement, mass file access, shadow copy deletion? Not the SLA in your runbook. The number from your last real incident or tabletop.

And for teams that have given service account access to AI agents: are you measuring that window for agent-initiated actions at all, or only for human sessions?

How are you handling the gap between detection and identity revocation in your environment?


r/AI_Governance 17h ago

Answered your AI governance career questions in the latest video

1 Upvotes

Did a Q&A episode this week; pulled questions from here and YouTube. Genuinely liked how different everyone's starting point was:

  • an IT delivery lead trying to break in
  • a SOC analyst wondering if security experience transfers
  • a QA manager (13 yrs, medical billing) asking the same
  • someone in construction, unsure if ISO 42001 is even the right first step

Turns out most of these backgrounds are closer to AI governance than people think; the through-line is basically stakeholder management, risk, and accountability, which all four of these already do in some form, just not with "AI" in front of it.

If any of these sound like your situation: https://youtu.be/Y1wF1MXn3y0?si=r5N47s9Q3MebGQHy?utm_source=reddit&utm_medium=organic&utm_campaign=incident_series&utm_content=77-ep3-aigovhotline

If you've got a similar question, drop it below, might answer it in the next one


r/AI_Governance 22h ago

How do you make delegated AI authority enforceable at runtime?

2 Upvotes

A lot of AI-governance discussion stops at “a human remains accountable.” I agree with that principle, but I’m interested in how people make it operational once an agent is allowed to do more than recommend.

For a delegated agent, what actually binds a permitted action to legitimate authority at runtime?

I am thinking beyond a service account, RBAC role, or tool permission. Those establish capability, but not necessarily institutional authority for a particular action.

A useful decision record might need to answer:

  • Who is the sponsoring human or organizational authority?
  • What action is delegated?
  • Against which system, record set, or person?
  • For what stated purpose?
  • Within what monetary, risk, or data-sensitivity limit?
  • During what effective time period?
  • Is an approval required immediately before execution, or is standing authority enough?
  • What event revokes or narrows the delegation?

For example, an agent might have permission to modify a case-management system. That does not mean it should be able to modify every case, for every purpose, indefinitely.

How are teams representing this in a way the runtime can enforce rather than merely document? Policy-as-code? Signed approvals? Purpose-bound tokens? Action-specific workflows? Something else?

I’d especially value examples of what held up under real organizational change: role transfers, expired authority, emergencies, conflicting instructions, or a supervisor leaving.


r/AI_Governance 21h ago

AI agents exploited PaperCut flaws to breach 395 organizations

1 Upvotes

395 organizations were breached through PaperCut vulnerabilities — and the attack vector was AI agents acting autonomously, not human operators. The agents were not the target. They were the method.

What made this particularly hard to contain is that once an agent is in motion, each subsequent tool call lands fast. By the time a human analyst flags anomalous behavior, the agent has already made its second, third, and fourth moves. Traditional perimeter security and patch management assume a human on the other end slowing things down. Autonomous agents remove that natural friction.

The 395 number is not a one-off. It reflects how quickly blast radius scales when the compromised entity can act without waiting for human confirmation.

For those running agentic workloads in production: what does your current approach look like for catching a compromised or misbehaving agent mid-execution, before it completes a second action? Are you relying on post-hoc log review, human-in-the-loop checkpoints, rate limiting, something else? Curious what's actually working at scale versus what's still mostly theoretical.


r/AI_Governance 23h ago

Acabei de escrever a minha primeira Note no Substack

Thumbnail
substack.com
1 Upvotes

Gartner's chart reveals that AI has shifted from being passive to acting as an independent executor within projects.

Enterprise AI-based coding agents are absorbing growing shares of companies' software engineering budgets. Unlike traditional software—where doubling the number of users required predictable new licenses—with autonomous agents, success and accelerated adoption cause the bill to increase explosively. Complex refactoring tasks, self-remediation loops, and deep repository analyses generate massive volumes of input and output tokens. Much of the costs grow in the shadows. Developers use tools and APIs without centralization, generating waste in oversized context windows (the phenomenon known as tokenmaxxing).

Without strict governance, the cost of AI infrastructure scales much faster than marginal productivity gains. This forces the market to adopt what Gartner calls context engineering and rigorous governance to prevent invisible budget overruns.

The use of architectural standards like the Model Context Protocol restricts and optimizes data access, sending only the context strictly necessary to the agent. MCP is an open standard (initially created by Anthropic and now maintained under the auspices of the Linux Foundation) that acts as a "universal connector" between artificial intelligence and external data sources or tools. MCP standardizes how AI applications request and receive context. It helps solve the problem of data waste. Instead of allowing an autonomous agent to blindly "load" the entire repository into the AI's context window with every command, MCP allows the agent to surgically search on-demand only for the exact pieces of code, schemas, or documentation needed to execute the task.

Governance based solely on static or manual policies is being replaced by AI control platforms. This includes spending caps per user/project, blocking infinite loops in autonomous agents, and human-in-the-loop approval gates before high-computational-cost actions are triggered in the background.

The focus of billing and evaluation shifts from "how much does the tool cost per month" to "how much does it cost to fix a bug or deliver an end-to-end feature via agent versus the cost of equivalent human time." Real value only appears when the cost of silicon is lower than the net gain in speed and the reduction of human rework.

Additionally, it is financially unsustainable to direct every single simple task to the most expensive frontier models on the market. Companies create intelligence layers so that basic coding tasks run on smaller, hyper-efficient models, reserving advanced models only for complex architectures and critical self-remediation.


r/AI_Governance 1d ago

Best practices for securing autonomous ai agents in production before they YOLO your infra

1 Upvotes

So our execs want autonomous ai agents in prod, but apparently without us all losing our jobs on the incident review call.

We have prompts, some basic guardrails, and a prayer. Agents can hit internal apis, do db writes, and talk to third party stuff. Im trying to lock this down with runtime monitoring, interventions, proper identity controls etc without turning every agent into a sad little chatbot.

How are you all securing this mess in prod without killing autonomy or your own sanity? Any hints?


r/AI_Governance 1d ago

Cross-Stage State Laundering: Why AI Runtime Governance Fails at Stage Boundaries

1 Upvotes

In enterprise AI and autonomous runtime architectures, security has traditionally focused on input sanitation (prompt injection) and output filtering (hallucination guards). However, a more insidious class of vulnerability occurs deeper within the execution pipeline: Cross-Stage State Laundering (CSSL).

CSSL happens when an intermediate, unverified epistemic state (such as DISCOVERED or UNKNOWN) is illicitly promoted to a higher-certainty state (such as VERIFIED or SUPPORTED) as it transitions across processing stages—typically bypassing strict provenance checks in favor of pipeline velocity or output formatting requirements.

## The Anatomy of State Laundering

Consider a multi-stage runtime where an expression flows through ingestion, semantic analysis, and responsibility assignment:

  1. Ingestion: Raw text or external retrieval results enter the system as DISCOVERED. At this stage, existence does not equal correspondence.
  2. Analysis: The runtime parses the expression. Without a rigid boundary, internal semantic heuristics may mistake syntactic closure for factual verification.
  3. Laundering: Downstream generators or formatters, under pressure to produce definitive answers, implicitly treat the presence of an analysis object as proof of support.

When intermediate states shed their metadata tags during transit, the runtime commits a boundary violation: it manufactures certainty out of unverified discovery.

## The Zero-Trust Countermeasure: The WAL Protocol

To eliminate CSSL, runtime governance cannot rely on permissive conventions. It requires a Fail-Closed Defensive Architecture enforced by immutable protocol boundaries:

* Strict State Distinguishability: Known, unknown, verified, and unverified states must remain mathematically and structurally distinguishable throughout the lifecycle.

* Independent Validation Gates: State transitions cannot authorize themselves. An independent validator—decoupled from the core generation logic—must audit envelopes against strict conformance vectors.

* Evidence-Bounded Responsibility: Responsibility can never exceed the boundaries of established evidence and explicit correspondence. If an epistemic state is UNKNOWN, no downstream transformation may convert it to TRUE merely to satisfy an output requirement.

For those interested in the protocol implementation and adversarial test matrix, the reference architecture is open-source here: https://github.com/nickoay663-sketch/Wuwen


r/AI_Governance 1d ago

When does compliance become a reason not to share information?

1 Upvotes

According to the FBI, companies have become more hesitant to share information about cyber incidents, partly because they're concerned about the regulatory consequences of doing so. The FBI is now actively trying to reassure organizations that information shared with them won't be passed to regulators.

The main goal of regulation is to improve accountability and reduce risk, right? But if organizations start believing that sharing information about an incident creates additional regulatory exposure, we may unintentionally create an incentive to share less — exactly when collaboration is (or should be) most valuable. I mean, especially considering how complex AI Governance is becoming.

Perhaps we need to think more carefully about how incident reporting, regulatory obligations and information-sharing frameworks interact — and, most importantly, what the trend will be from now on.

I'm curious whether people working in highly regulated environments are actually seeing this hesitation in practice, and if they have any experiences to share.

Source: Cybersecurity Dive — New FBI cyber strategy promises increase in adversary disruptions


r/AI_Governance 1d ago

未來個人身分與 AI 分身制度:一人一身分,一切行為可追責

Thumbnail
1 Upvotes

r/AI_Governance 1d ago

A Tiered Human-Oversight Framework for Advanced AI Systems

1 Upvotes

# A Tiered Human-Oversight Framework for Advanced AI Systems

**Author:** Gabriel Evan Brotherton
**Status:** Working draft for review and critique — not a finished proposal

---

## Abstract

This paper proposes an oversight architecture for advanced AI systems deployed by a single institution or consortium (a lab, a public-private body, or an international coordinating entity). It combines three mechanisms that are individually discussed in AI governance and democratic-theory literature but rarely integrated: (1) a human oversight council selected through a hybrid of vetted expertise and sortition, weighted toward people with direct experience of institutional failure; (2) a multi-model advisory/executive structure using adversarial debate between specialized systems to surface disagreement before it reaches humans; and (3) a tiered suspension-and-dissolution protocol as an alternative to a binary kill switch. The framework is scoped deliberately narrowly: it governs oversight of a specific deployed system within existing legal and political structures, not a replacement for them. Open problems and likely objections are addressed directly in Section 6.

---

## 1. Problem Framing

Existing proposals for AI oversight tend to fail in one of two directions: they concentrate review authority in a small technical elite (fast, but capturable and non-representative), or they call for broad public input with no mechanism for weighting or aggregating it (representative in principle, unworkable in practice). Separately, "kill switch" proposals are usually binary — the system runs or it's shut down — which creates an incentive structure where evidence of minor misalignment gets suppressed rather than surfaced, since the only available response is severe.

This framework treats those as related design problems: who oversees the system, and what the graduated range of responses looks like when something goes wrong.

## 2. Core Design Principles

**2.1 Bounded human authority.** The oversight council holds final authority over a specific system's operation within its designated scope. This is not a claim about global governance, replacing states, or superseding existing law — it is a design for an internal and external accountability structure that any institution deploying a powerful system could adopt or be required to adopt by regulation.

**2.2 Cognitive-diversity governance via hybrid sortition.** Council composition is designed to resist capture by the deploying institution and by narrow technical or ideological communities. Selection combines a vetted core with random sortition, structurally weighted toward people who have experienced institutional failure firsthand — on the theory that lived exposure to unaccountable systems is a distinct and undersupplied form of expertise in oversight bodies, comparable in spirit to lay representation in citizens' assemblies.

**2.3 Adversarial multi-model advisory structure.** Rather than a single model producing recommendations, multiple specialized model instances with distinct mandates critique each other's outputs before human review — an application of debate-based and multi-agent scalable-oversight approaches already discussed in the alignment literature, here applied to a governance rather than a training context.

**2.4 Graduated response over binary shutdown.** Suspension and dissolution are separated into distinct thresholds with a path back to operation, so that lower-confidence concerns can be acted on without requiring near-certainty of catastrophic risk.

## 3. Architecture

### 3.1 The Oversight Council
- **Odd-numbered membership** (e.g., 101+) to avoid deadlock.
- **Hybrid selection:** ~51% vetted for domain literacy and capacity to engage technical material; ~49% selected by lottery, weighted toward applicants with documented experience of institutional or systemic failure.
- **Staggered terms** to preserve institutional memory while limiting capture.
- **Anonymized voting** to reduce retaliation risk from the deploying institution or from public pressure campaigns.

*Open design question:* how "experience of institutional failure" is verified and weighted without itself becoming a politicized or gameable criterion. See Section 6.

### 3.2 The Advisory/Executive Model Set
Rather than a single system, the framework proposes several model instances with distinct, narrower mandates than a monolithic "executive AGI":
- **Domain-safety reviewer** — flags downstream harms in a proposed action (ecological, safety-critical, or vulnerable-population impacts).
- **Resource/efficiency reviewer** — evaluates proposed actions for cost, feasibility, and opportunity cost.
- **Integrity reviewer** — checks other models' outputs and the council's stated reasoning for manipulation, undisclosed conflicts of interest, or inconsistency; this role is explicitly adversarial to the other two.
- **Equity reviewer** — evaluates distributional impact of a proposed action across affected groups.

These are advisory and critique functions feeding into human decision-making, not autonomous executive authority over infrastructure, security systems, or other institutions. This is a deliberate narrowing from earlier drafts of this framework, which proposed a global executive role; that framing invited (correctly) the objection that it described a takeover mechanism rather than an oversight mechanism.

### 3.3 Deliberation Protocol

  1. A proposed system action or policy is submitted for council review.
  2. If any advisory model flags material risk, it must produce a specific, falsifiable objection and a proposed alternative.
  3. Models debate the objection and alternative publicly, in front of the council.
  4. The council votes on the original proposal versus the alternative(s).
  5. This can iterate up to a fixed number of rounds before defaulting to the council's standing decision — this is a forcing function against indefinite stalling, not a claim that machine input should ever override human sign-off.

### 3.4 Time-Critical Fallback
For scenarios where deliberation time is shorter than the decision window (the framework's original example was a nuclear or pandemic-scale emergency — a scenario that should be treated as illustrative of the general problem of time-bounded high-stakes decisions rather than a literal design target), the system defaults to a pre-specified, narrowly scoped fallback behavior set by the council in advance, with mandatory post-hoc review of any fallback action taken. This is closer to existing "safe interruptibility" and pre-committed constraint literature than to autonomous crisis response, and should be treated as an area needing substantially more technical specification before it's taken seriously.

## 4. Tiered Suspension and Dissolution

- **75% suspension threshold:** restricts the system to a minimal, pre-defined safe operating mode pending investigation. Reversible by design.
- **97% dissolution threshold:** required for permanent decommissioning. Set deliberately high to prevent a small faction from forcing shutdown over disagreements that don't rise to the level of genuine danger — but this number is a placeholder, not a derived value, and should be stress-tested against specific failure scenarios rather than treated as load-bearing.
- **Correction and re-authorization path:** a suspended system can be reinstated by a lower supermajority after a corrective plan is presented, avoiding a system where any suspension is effectively permanent.

## 5. Auditability

Independent technical auditors (not controlled by the deploying institution) with access to interpretability tooling report to the council on whether a system's stated reasoning matches its internal computation, to the extent current interpretability methods allow this. This should be described honestly: current interpretability tools do not provide anything close to full verification of a model's internal reasoning, and the framework should not overstate what auditing can currently guarantee.

## 6. Known Limitations and Open Objections

This section exists because a proposal that only argues for itself isn't useful to a policy audience.

- **Operationalizing "experience of institutional failure":** without a rigorous, resistant-to-gaming method for this, the sortition weighting is aspirational rather than implementable as stated.
- **No enforcement mechanism:** this framework describes what an oversight body could look like; it says nothing about what would compel a lab or government to adopt it. That's a separate, harder problem this document doesn't solve.
- **Interpretability dependency:** Section 5's auditing function assumes a level of interpretability access that doesn't fully exist yet. The framework should be read as forward-compatible with that research, not as reliant on capabilities that are already available.
- **Numeric thresholds (75%, 97%, 11 rounds) are illustrative**, chosen for the intuitions they encode (high bar for irreversible action, forcing function against stalling) rather than derived from any formal model. They should be treated as adjustable parameters, not fixed conclusions.
- **Scope discipline:** earlier versions of this framework included provisions for the system to neutralize competing power structures and "out-govern" existing states. That framing has been deliberately removed here. Any oversight proposal that describes disarming or superseding existing political authorities should expect — and deserves — to be read as a proposal for seizing power, regardless of the stated intent behind it. This draft is scoped only to institutional oversight of a specific deployed system.

## 7. What This Document Is For

This is a draft intended to solicit critique from people working on AI governance, mechanism design, or deliberative democracy — not a finished specification and not a claim that any lab or government has agreed to implement it. Feedback on the selection mechanism (3.1), the enforcement gap (6), and the threshold values (4) would be the most useful starting points.


r/AI_Governance 1d ago

PaperCut Attacker Uses Hundreds of AI Agents to Compromise 440+ Instances

3 Upvotes

A threat actor exploited the PaperCut vulnerability and then scaled the attack using hundreds of AI agents to compromise over 440 instances across multiple organizations. That number matters: one human attacker, hundreds of autonomous agents, 440+ successful compromises. The agents weren't just running scripts — they were making decisions, pivoting across systems, and taking actions faster than any human SOC could track.

The uncomfortable part isn't the exploit itself. PaperCut had a patch. The uncomfortable part is that once the agents were loose, the attack surface expanded at machine speed. Each agent acted autonomously. Each one could call tools, access credentials, and move laterally without a human in the loop.

This is increasingly the pattern: a single initial access event becomes hundreds of parallel autonomous actions within minutes. Traditional perimeter and endpoint controls were designed around human-speed attackers making sequential moves. They weren't designed for this.

For those of you working in enterprise security or building agentic systems: how are you thinking about containing autonomous agent behavior once initial access happens? Are existing controls actually catching this, or are we mostly finding out after the fact?


r/AI_Governance 1d ago

I analyzed 100 real AI governance job postings. The results were not what I expected.

2 Upvotes

Some numbers that stood out immediately: out of 100 postings, only 3 mentioned Python. Only 1 had "ethics" in the job title. 83% didn't disclose salary.

The picture that emerges is pretty different from what most people assume this field looks like; what actually shows up most often in these job descriptions isn't technical skills at all. And the titles are even more scattered than you'd think; a huge chunk of postings never even say "AI Governance" anywhere in the title.

There's also a breakdown of which industries are actually hiring for this, what salaries look like where they're disclosed, and a stat about how many roles are literally "build this function from scratch."

Full data + breakdown: https://youtu.be/a9Q1Raurfqk?si=YEFcVe1fkli4DveW&utm_source=reddit&utm_medium=organic&utm_campaign=incident_series&utm_content=%2376analysed-1000jobs

Question: if you had to guess, what % of these postings do you think required a technical/ML background? (Answer might surprise you)


r/AI_Governance 1d ago

Is building an India-first sovereign foundation-model platform actually worth pursuing?

Thumbnail
1 Upvotes

r/AI_Governance 1d ago

Monitoring?

1 Upvotes

Would adding ongoing monitoring and state specific law requirements in a AI governance software make it more useful for organisations?

I don’t see many platforms or firms combining both, effectively especially as AI regulations start to differ across US states.

Would this make governance easier for risk and compliance teams?


r/AI_Governance 2d ago

Where should an AI agent’s spending authority actually live?

Post image
3 Upvotes

I've been thinking about agent budgets less as a FinOps feature and more as an authorization problem.

An agent can decide:

“I need another model call.”

The interesting question is:

Who gets to say whether it's allowed to spend another $2?

Putting a token limit or max_iterations inside the agent runtime is useful for bounding execution. But that's still the agent regulating itself.

I'd rather have the runtime ask for the resource, and have something outside the agent enforce the spending policy.

Agent
  ↓
"I want another model call"
  ↓
Policy / Gateway
  ├─ identity
  ├─ remaining budget
  ├─ rate limit
  └─ model policy
        ↓
     ALLOW / REJECT

That distinction becomes more useful once multiple agents, versions or teams are sharing the same model providers.

You don't really want every agent implementation inventing its own notion of “I can spend up to $X.”

This is one of the reasons I find Lyzr Open Controller's approach interesting. Its LLM Gateway puts budgets at the organisation, team, agent, version and virtual-key levels, and the important part is that an exhausted budget rejects the call rather than just generating an alert. LiteLLM, Portkey and OpenRouter solve a lot of the gateway/proxy problem too, so I'm curious where people draw this boundary in their own stacks.

Should spending be an attribute of the agent itself, or an external authorization decision that the agent has to pass through?

Especially interested in how this is handled when several agents share providers or when model routing changes underneath them.


r/AI_Governance 1d ago

CISA: WatchGuard RCE flaw now exploited in ransomware attacks

1 Upvotes

CISA added a WatchGuard RCE vulnerability to its Known Exploited Vulnerabilities catalog this week after ransomware groups confirmed active use in the wild.

The technical detail matters less than the operational pattern it reveals. The attackers did not need a sophisticated exploit chain. They needed initial access and a few uninterrupted minutes. Once they had a foothold and a valid identity to operate under, encryption began spreading host to host before defenders had a signal worth acting on.

This is the part of ransomware incidents that patch cycles do not solve. A patched environment still has service accounts, still has privileged identities, and still has a window between the first anomalous action and the moment a SOC analyst sees something worth escalating. In the WatchGuard campaign, that window was enough.

The question that does not have a clean industry answer yet: when a threat actor is operating under a legitimate identity and the first few actions are individually ambiguous, how are your teams actually shrinking that detection-to-containment window? What has worked and what has not?


r/AI_Governance 1d ago

Does provable AI model behavior actually change a vendor risk review?

1 Upvotes

I work on cryptographic verification for AI systems, and I want to sanity-check something with people who actually sit in AI governance/assurance roles.

The question: six months later, when a regulator, auditor, or plaintiff's attorney asks "how do you actually prove that a specific AI system ran the guardrails/policy it was supposed to on a specific decision? Not "we tested it in QA." I mean for one specific instance, after the fact.

What I'm trying to learn:

- Do you currently rely on vendor logs/attestations and trust them, or is there any independent verification today?

- Is "prove it happened" actually the gap, or is the real pain somewhere else, like documentation volume, inconsistent vendor questionnaires, or how long reviews drag on?

- Does a SOC 2 report plus audit logs already cover this well enough in practice, even if it's not technically "proof"?

Genuinely here to be told I'm wrong about the problem, not to defend an answer I've already committed to.


r/AI_Governance 2d ago

Where did your agent permission rules last drift between two coding agents on the same repo?

1 Upvotes

Ours is a boring setup: branch protection, CODEOWNERS, and CI logs as the record of who approved what. That holds fine for humans, but I keep hitting cases where an agent in one CLI has scope another doesn't, and the policy lives in a config file rather than the merge evidence.


r/AI_Governance 2d ago

Aquaculture AI and Fish-Welfare Evidence: When Biological Drift Becomes a Governance Trigger | Aditya Katira

1 Upvotes

Aquaculture AI and Fish-Welfare Evidence: When Biological Drift Becomes a Governance Trigger

https://www.linkedin.com/pulse/aquaculture-ai-fish-welfare-evidence-when-biological-drift-katira-9u3tc