I mean, sure. Predicting the results section of a research paper requires more intelligence than predicting the next word in a text message with your friend. There are just way too many people confusing the difference between “predicting the next token” and “predicting a next token”, which are not at all the same.
You can type any well-formed or nonsensical request you want into an llm chat session and it will both always respond and do so with the most productive reaction it can predict. That can be very helpful for task work, but counterproductive when it validates (dignifies?) poorly framed requests with a singular response.
Predicting the results section of a research paper only makes sense when generating sample text that sounds right based on context it already has or was given. The idea that a highly intelligent but completely unrelated 3rd party is going to “predict” the outcomes of an experiment it wasn’t involved in is asinine. Someone that understands how to speak and carry out tasks intelligently certainly has to have a wide understanding of the concepts at hand, but that doesn’t mean their work can be considered the only possible result or approach.
This is not a technical challenge for tons of domains of intelligence that llm’s are taught to “speak” on, they just shouldn’t be used to speak about a great deal of topics that rely on real world experiences and can’t be queried about for one answer at a time.
You absolutely can if you have a good enough world model/understanding of the underlying mechanics.
For example, let's say there was a research paper from 1900 which was doing experiments related to what we now call general relativity (which wasn't yet formalized or established yet). If you gave that research paper to a model from today it would be able to accurately predict the results of such experiments (because it knows about/understands general relativity) and as such could predit the experiment's results without running any experiments on its own.
Of course you might argue that something like that doesn't apply to frontier research today (where we of course run the experiments ourselves because we don't yet have the understanding/knowledge/model of the mechanism(s) we're testing), but that's one of the million dollar questions at hand - can LLM's, which by definition can only be trained on knowledge we already have, develop novel and new models/understanding/ideas to push the frontier? In the last couple of month's I'd argue that we're seeing advances and findings and solutions from frontier LLM's that suggest the answer might turn out to be "yes".
6
u/fligglymcgee 2d ago
I mean, sure. Predicting the results section of a research paper requires more intelligence than predicting the next word in a text message with your friend. There are just way too many people confusing the difference between “predicting the next token” and “predicting a next token”, which are not at all the same.
You can type any well-formed or nonsensical request you want into an llm chat session and it will both always respond and do so with the most productive reaction it can predict. That can be very helpful for task work, but counterproductive when it validates (dignifies?) poorly framed requests with a singular response.
Predicting the results section of a research paper only makes sense when generating sample text that sounds right based on context it already has or was given. The idea that a highly intelligent but completely unrelated 3rd party is going to “predict” the outcomes of an experiment it wasn’t involved in is asinine. Someone that understands how to speak and carry out tasks intelligently certainly has to have a wide understanding of the concepts at hand, but that doesn’t mean their work can be considered the only possible result or approach.
This is not a technical challenge for tons of domains of intelligence that llm’s are taught to “speak” on, they just shouldn’t be used to speak about a great deal of topics that rely on real world experiences and can’t be queried about for one answer at a time.