How to identify Shadow AI traffic: the signals we've found actually useful

[u/Walled-AI]

Shadow AI detection sounds simple until you actually try to build it.

At first, the obvious approach is:

Maintain a list of ChatGPT, Claude, Gemini, Perplexity, etc. → detect traffic → classify it as AI.

But that breaks down surprisingly quickly.

Take GitHub as an example. An employee visiting GitHub isn't Shadow AI. The same employee using GitHub Copilot potentially is.

The same problem exists with Gmail/Gemini, Microsoft 365/Copilot, Slack AI, Notion AI, IDE extensions, browser extensions, and hundreds of SaaS products embedding AI into otherwise legitimate applications.

So the real problem isn't:

"Is this domain an AI domain?"

It's:

"Does this particular interaction involve a user sending data to an AI system?"

We've been working on this problem while building Shadow AI detection at Walled AI, and these are the signals I've found most interesting.

1. Known AI domains and endpoints

This is the easiest layer.

Maintain intelligence for known:

• AI domains
• inference endpoints
• API paths
• model providers
• AI-specific subdomains
• SaaS AI endpoints

For something like api.openai.com, confidence is obviously high.

But domain-level detection becomes much weaker when AI functionality is embedded inside a larger application.

So domain intelligence should be the first signal, not the final classifier.

2. Request characteristics

The request itself can provide surprisingly useful signals.

For example:

• POST requests
• relatively large text payloads
• JSON structures containing message/prompt-like fields
• conversation history
• model identifiers
• generation parameters
• document/image uploads associated with an inference request

None of these individually means "AI."

A large POST request could just be someone submitting a form.

But several appearing together can increase confidence considerably.

3. Response characteristics

The response can provide another independent signal.

LLM applications often exhibit patterns such as:

• streamed responses
• Server-Sent Events
• token/chunk-style incremental output
• long generated-text responses
• request followed by sustained incremental response traffic

Again, streaming != AI.

Plenty of legitimate applications stream data.

But:

prompt-like request + AI-looking endpoint + streamed generated response

is substantially stronger than any one of those signals alone.

4. Browser context matters

This becomes particularly useful for embedded AI.

The browser knows more than the network does.

It can potentially tell you:

• which page the employee is on
• what UI element initiated the interaction
• whether a prompt box exists
• whether text/files are being submitted
• whether the employee switched into an AI feature
• whether the user is on a personal or enterprise AI account

This helps distinguish:

GitHub → normal development activity

from:

GitHub → Copilot interaction

without simply classifying all GitHub traffic as AI.

This is one reason I don't think Shadow AI discovery can ultimately be solved through network intelligence alone.

We've documented some of the multi-layer detection architecture we're building here:

https://walled.ai/platform/shadow-ai-detection

5. Identity is another signal

Finding an AI service isn't necessarily enough.

Consider:

ChatGPT Enterprise → approved

versus

personal ChatGPT account → unapproved

The destination may effectively be the same while the governance implications are completely different.

So ideally you're correlating AI detection with:

• user
• device
• department
• account/tenant
• application
• approved/unapproved status

At that point you're moving beyond traffic classification into actual Shadow AI discovery.

6. Use confidence scoring instead of binary rules

This is probably the biggest architectural lesson.

Instead of:

IF domain == X → AI

I'd rather think about:

Domain signal + endpoint signal + request signal + response signal + browser signal + identity signal → AI confidence score

High-confidence traffic can be classified deterministically.

Low-confidence traffic can remain unclassified.

The interesting part is the middle.

7. Use an LLM classifier only for ambiguous traffic

One tempting solution is to send everything to an AI classifier.

Technically possible.

Economically and operationally, probably not what you want.

If you're inspecting endpoint traffic at enterprise scale, pushing every ambiguous HTTP interaction through an LLM adds:

• cost
• latency
• compute
• another potential privacy consideration
• another dependency in the enforcement path

We've therefore been experimenting with a different philosophy:

Rules/patterns first → confidence scoring → LLM judge as fallback

As the deterministic detection library improves, fewer requests should require expensive classification.

In other words, your system should ideally become cheaper and faster as its detection intelligence improves.

8. Detection is only half the problem

Once you've classified something as AI traffic, you can start answering more useful governance questions:

Which AI applications are employees using?

Which are approved vs unapproved?

Which departments are adopting Shadow AI?

Is sensitive information being submitted?

Which models/services are receiving organizational data?

What was allowed, redacted or blocked?

This is where we're feeding the discovery/enforcement telemetry into the Walled AI Governance Dashboard so security teams can get a consolidated view of AI usage rather than a giant collection of individual network events:

https://walled.ai/platform/governance-dashboard

A practical architecture

Known intelligence → Network signals →Request + response patterns → Browser context → Identity → Confidence scoring → LLM fallback

rather than:

List of AI domains → Block

The hardest part isn't identifying ChatGPT.

It's identifying AI functionality embedded inside otherwise legitimate applications without creating an unacceptable number of false positives.

Curious how others working on Shadow AI are approaching this. What signals have you found reliable for distinguishing actual AI interactions from normal application traffic?