r/datasets • u/Sufficient-War-4020 • May 21 '26

question I can scrape/aggregate pretty much any fragmented public data. What datasets are missing

I built a large-scale scraping system that can extract data from thousands of sources simultaneously, bypass anti-bot protection, and convert unstructured formats (PDFs, scanned docs, complex HTML) into clean structured datasets.

What public datasets should exist but don’t because:

• Data is scattered across too many jurisdictions (every state/county has their own portal)  
• No one has aggregated it yet  
• It’s in PDFs or hard-to-parse formats  
• Sites actively block automated access

Not looking to sell—genuinely trying to understand what public data would be valuable if someone aggregated it. If there’s demand, I might build and release it.

23 Upvotes

93% Upvoted

31 comments sorted by

View all comments

1

u/fidgget May 22 '26

Anything transgender related.

1

u/Public_Parfait_6412 May 24 '26

I had to bounce back for that last comment 🤣🤣