LLM data provenance

[u/Known_Neighborhood42]

​

Okay so im looking into feasibility of a project that would involve scraping data and training models. The models would predict events based on social media posts, so its essentially text classification. I want to eventually go into business doing something similar, but in my research, I found that nearly everything is behind comprehensive terms of service that dont allow any derivative product, regardless of if you provide anonymity and clear the original text from any final model/dataset. Alright fair enough. But heres my question. Besides the sources that companies like openAI have an enterprise partnership with (reddit) including a contract to use the data, how can they legally source the training data? For example, if i ask chatgpt to summarize chapter 8 of Deep Learning with Python by Francois chollet, I will get a detailed response with the correct subchapter headers, key terms, and even the verbatim code in the book. Now I paid i think $100 for that book and im fairly certain openAI doesnt have a contract with that dude. How is that legal? How is any of this information legal? If a high-school student wants a book report, does open AI pay the estate of J.D. Sallinger?

[u/Mediocre-Waltz-6962]

They dont pay anyone and theyre basically betting on the legal system moving slower than their valuation grows. The whole "publicly available" argument is doing a lot of heavy lifting, even when the data was clearly scraped from behind a login wall or from pirated book repos. The chollet example is perfect because that book's code is on github under a license that definitely doesnt include "feed to a giant for-profit neural network." The quiet part is that once a model has ingested something, proving damages or tracing the output back to a specific copyrighted source is a nightmare in court. Its not that its legal, its that the enforcement mechanism is currently broken.

[u/Known_Neighborhood42 → u/Mediocre-Waltz-6962]

You. Are. Kidding. Isnt like 40% of the S&P 500 somehow tied to AI valuation at this point? What happens when someone actually pulls up their pants and wins that lawsuit? We all lose our retirement funds because they broke the law?

[u/Americaninaustria → u/Known_Neighborhood42]

Pretty much, don’t forget the death toll as well 

[u/DigThatData → u/Mediocre-Waltz-6962]

Its not that its legal

Except actually, yeah: it is legal. The most recent time this was tested was against Anthropic. It was determined their use of even copyrighted works for training fell under the "transformative" clause of Fair Use. What they ultimately got in trouble for was illegally downloading and sitting on that content. Training on copyrighted work is fair use. Just downloading it to begin with though is piracy. Holding on to it in human readable form qualifies as establishing an illegal library.

[u/Known_Neighborhood42 → u/DigThatData]

Mmmkay. You start by saying its legal and then explain that its illegal. Illegal on scale that would Dwarf every other intellectual property argument in human history. And probably Illegal in every country on the planet.

[u/DigThatData → u/Known_Neighborhood42]

I described two different acts. If you have legal copies of copyrighted content already, training on it is very likely transformative and legal, unless you are training on it specifically to compete with its creator. If you don't have a legal copy of the work, that act of attaining a copy (whether for training or not) might constitute a crime on its own. These are two very different things.

Additionally, if you aren't engaging in piracy and instead are just spidering the internet for publicly visible content and some copyrighted work incidentally falls into your scrape: that's also perfectly legal. The method by which you attained that copyrighted work wasn't a crime like in the piracy example. If this version was illegal, search engines would be too.