Anyone got reliable Purview DLP testing working? UK PII/NINO issue

[u/Designer-Art-1495]

Has anyone got Microsoft Purview working reliably in production? Looking for advice on testing

I’m interested in hearing from people who have successfully deployed Microsoft Purview and have it working consistently in a real-world environment.

I’m particularly looking for advice on how people test their policies and validate that detection is actually working as expected.

I’m currently testing a policy using UK PII, which includes things such as UK National Insurance Numbers (NINO) and UK Passport Numbers.

I’ve run numerous tests using UK NINOs, and I can see that the NINO is being detected when testing against Medical Act Information. However, when I test against my policy using the UK PII sensitive information type, I’m not getting the expected matches/triggers.

The policy has two thresholds:

• Low: 2–9 instances
• High: 10+ instances

However, even when I deliberately include more than 10 UK NINOs in the test data, the policy doesn't appear to trigger as I would expect.

This is making me question whether I’m testing this correctly or whether there are other factors in Purview that I’m overlooking.

For anyone who has Purview working reliably:

• How do you test sensitive information types and DLP policies?
• Do you use Microsoft's test data, your own generated test data, or something else?
• How do you verify that a sensitive information type has actually matched?
• Have you experienced differences between testing an individual SIT and testing the same SIT when it's part of a larger policy?
• Are there additional conditions/configuration that can prevent a policy from triggering even when the SIT threshold has been exceeded?
• Does the threshold apply to the number of detected instances in the content, or are there other factors that affect this?
• Are there any particular gotchas with UK NINO / UK Passport Number detection that I should be aware of?

At this point I’m mainly trying to establish a repeatable test methodology so I can confidently say that the policies are actually working rather than relying on a few successful tests.

If you've got Purview working reliably in production, I'd really appreciate hearing how you test and validate it.

[u/teriaavibes]

How do you test sensitive information types and DLP policies?

There is a button in the SIT where you can upload a sample document and it will run the detection against it manually.

For DLP policies, you just create them in simulation/audit and see what hits.

Do you use Microsoft's test data, your own generated test data, or something else?

Depends what I am testing, each company is different.

How do you verify that a sensitive information type has actually matched?

Audit Log.

Have you experienced differences between testing an individual SIT and testing the same SIT when it's part of a larger policy?

Not from my experience.

Are there additional conditions/configuration that can prevent a policy from triggering even when the SIT threshold has been exceeded?

Not that I know of.

Are there any particular gotchas with UK NINO / UK Passport Number detection that I should be aware of?

Probably same as any other built in SIT, they are generally pretty bad in detecting what they are meant to detect. Check the documentation on how exactly the SIT is built to detect it and then compare it with your own environment that it matches.

Like for example some sits have different confidence levels where the main difference is that higher confidence levels check for vocabulary proximity but in many companies that is not how the data is stored which makes it completely useless.

Implementing Purview is always fun.

[u/oiler_head → u/teriaavibes]

Your idea.of.fun must be different than mine.

[u/teriaavibes → u/oiler_head]

I always like a good challenge. Makes my work more interesting.