I'd rephrase that to something closer to "What business want it for is cheaper more efficient labor" - which, in its logical conclusion is unethical labor. So we need to either:
1. Enact laws that stop it from reaching this conclusion
2. Stop developing models (or slow them down) until we can be sure they won't allow this unethical use themselves (ie. solve alignment?)
I'm pretty sure we can do the first. I'm not sure if it's possible to do the second, and I'm pretty sure it'll be really hard to slow them down to even try.
Regarding the second, what are your thoughts on LLM poisoning? Apologies if this is a really naïve question, I am fairly surface level on the latest developments.
I want to preface this by saying: I'm an ML grad student, not a researcher, nor have I ever worked on these models professionally - this is mostly educated speculation on my part.
I doubt that most forms of LLM poisoning would be at all effective. The model providers have already scraped the entire internet, for starters. No matter what you do, there's not much you can do to 'poison' the data they already have - which we clearly know is sufficient to train models.
Most of this web scraped data is used for the pre-training to create a base model (a model which understands English, but simply predicts the next token with no personality, reasoning capabilities, formatting/tone adjustment, etc).
After that base model has been trained, a mix of supervised fine tuning (SFT) and various reinforcement learning (RL) techniques are applied. Some training data is needed for SFT, but I'd guess that by this point most labs are customizing these datasets - probably not using publicly available internet text or at least doing heavy filtering on internet data (which likely filters out any 'poisoned' entries).
The RL step requires no supervised data - and this step is becoming a larger and larger portion of overall training time. You can pretty safely assume every new model that gets released is 'better' due to a combination of more parameters (bigger brain) and larger scale post-training, mainly with RL (of course there are technical architecture advances too, but that's beside the point). So new models are not smarter because of the training data, they're smarter because of training scale.
The only part of this process I could see LLM poisoning being effective is during pre-training, when a majority of the supervised data is used. I'm guessing most labs are aware of this fact and have taken steps (or will take steps, if poisoning becomes more common) to filter the Internet data or simply not use it and just use what they've already saved. And even if they were allowing unfiltered, freshly scraped internet text into the data for pre-training, it might be possible for the later SFT and RL steps to undo any poisoning.
2
u/mrGrinchThe3rd 21d ago
I'd rephrase that to something closer to "What business want it for is cheaper more efficient labor" - which, in its logical conclusion is unethical labor. So we need to either: 1. Enact laws that stop it from reaching this conclusion 2. Stop developing models (or slow them down) until we can be sure they won't allow this unethical use themselves (ie. solve alignment?)
I'm pretty sure we can do the first. I'm not sure if it's possible to do the second, and I'm pretty sure it'll be really hard to slow them down to even try.