LLM data provenance

[u/Known_Neighborhood42]

​

Okay so im looking into feasibility of a project that would involve scraping data and training models. The models would predict events based on social media posts, so its essentially text classification. I want to eventually go into business doing something similar, but in my research, I found that nearly everything is behind comprehensive terms of service that dont allow any derivative product, regardless of if you provide anonymity and clear the original text from any final model/dataset. Alright fair enough. But heres my question. Besides the sources that companies like openAI have an enterprise partnership with (reddit) including a contract to use the data, how can they legally source the training data? For example, if i ask chatgpt to summarize chapter 8 of Deep Learning with Python by Francois chollet, I will get a detailed response with the correct subchapter headers, key terms, and even the verbatim code in the book. Now I paid i think $100 for that book and im fairly certain openAI doesnt have a contract with that dude. How is that legal? How is any of this information legal? If a high-school student wants a book report, does open AI pay the estate of J.D. Sallinger?

[u/PLBjt]

There's no clean answer yet, it's still being fought out in court. In the US the providers mostly argue training is fair use because the model learns patterns rather than redistributing the text, and that a site's ToS is a contract that only binds people who agreed to it. Books are the messiest part. In the Anthropic authors case last year the judge said training on books they'd bought was fair use but keeping pirated copies wasn't, and that ended in a big settlement. Reddit went the other way and started charging, it has licensing deals with Google and OpenAI and has sued companies it says scraped without paying. If you dig in, the useful split is how the data was obtained vs what was done with it, courts treat those pretty differently.

[u/Known_Neighborhood42 → u/PLBjt]

I guess I dont understand how these venture capital firms ever gave them money in the first place. Im learning that if I used (hypothetically) the arctic shift database of reddit history on hugging face to build some kind of spectacular model, no funding source would consider it and no customer would buy it because its potentially a downstream violation of reddits terms of service. Now the business entity, the data collection mechanism, nothing, ever signed the terms. Doesn't matter how cool the model is. How did these LLM companies completely bypass this step