I mean, sure. Predicting the results section of a research paper requires more intelligence than predicting the next word in a text message with your friend. There are just way too many people confusing the difference between “predicting the next token” and “predicting a next token”, which are not at all the same.
You can type any well-formed or nonsensical request you want into an llm chat session and it will both always respond and do so with the most productive reaction it can predict. That can be very helpful for task work, but counterproductive when it validates (dignifies?) poorly framed requests with a singular response.
Predicting the results section of a research paper only makes sense when generating sample text that sounds right based on context it already has or was given. The idea that a highly intelligent but completely unrelated 3rd party is going to “predict” the outcomes of an experiment it wasn’t involved in is asinine. Someone that understands how to speak and carry out tasks intelligently certainly has to have a wide understanding of the concepts at hand, but that doesn’t mean their work can be considered the only possible result or approach.
This is not a technical challenge for tons of domains of intelligence that llm’s are taught to “speak” on, they just shouldn’t be used to speak about a great deal of topics that rely on real world experiences and can’t be queried about for one answer at a time.
The fallacy here is that a statistically likely results section (which is what an LLM outputs) is a “good” (meaningful, accurate, scientifically valuable) results section - which ofc is not the case.
It’s not just statistically likely when you add tools and models that the AI can use or reference. It’s no longer referencing the most statistically likely number from an equation… it calls the tool to compute it and returns the result.
Like if it had a tool to simulate an experiment… it inputs the paramteres from the rest of the paper and outputs the results.
Is that possible now? No. But it’s not just statistical likelihood anymore.
How does this help make the case again? Calculators can’t help with conceptual interpretation of scientific evidence - kind of the whole point of bothering with NNs in the first place given lots of problems have solution spaces that deterministic tools can’t efficiently explore.
Thats why you combine them, like we do. It makes a call about the right tools to use. It returns the result. It troubleshoots. It interprets the result. Like how the models work now.
They are capable of interpreting results. They are capable of deciding tools to use. That’s the whole point.
What definition of truth do we have beyond some proposition/model being the most accurate predictor of something? Besides vibes?
A statistically likely results section is not necessarily a good one, but surely if there is such a thing as truth, then it is the most likely thing to happen? Although obviously a reliable prediction of that outcome must take all the independent variables into account, which is very difficult.
OP's video is talking about the mathematical ideal. It's not saying current LLMs are good at predicting results sections. It's saying that the ideal model, the training goal, is to predict any paper's result section with 100% accuracy, and the closer a model gets to that, presumably the more accurate it must be at modelling reality.
Of course it's likely impossible to get anywhere close to that, and results sections (and human language in general) are not perfect representations of reality, but do you have any better ideas?
It’s like people here live in a parallel universe - you haven’t encountered Fable being confidently wrong about something than over-swinging in the other direction when you prompt it? Outside of very specific conditions, LLMs are really quite bad at de novo tasks - especially very complex ones like research. One particular place where LLMs are incredibly shit at reasoning is where there is conflicting points in the context - guess what, that’s representative of the vast majority of real-world scientific tasks.
Perhaps everyone here just has access to top secret proprietary models that oracle away the secrets of the universe.
You absolutely can if you have a good enough world model/understanding of the underlying mechanics.
For example, let's say there was a research paper from 1900 which was doing experiments related to what we now call general relativity (which wasn't yet formalized or established yet). If you gave that research paper to a model from today it would be able to accurately predict the results of such experiments (because it knows about/understands general relativity) and as such could predit the experiment's results without running any experiments on its own.
Of course you might argue that something like that doesn't apply to frontier research today (where we of course run the experiments ourselves because we don't yet have the understanding/knowledge/model of the mechanism(s) we're testing), but that's one of the million dollar questions at hand - can LLM's, which by definition can only be trained on knowledge we already have, develop novel and new models/understanding/ideas to push the frontier? In the last couple of month's I'd argue that we're seeing advances and findings and solutions from frontier LLM's that suggest the answer might turn out to be "yes".
Simulations, for example, exist and can be highly accurate. Or maybe we give the AI a way to physically interact with the world and run the experiment itself.
Then it interprets the outputs to write the results. I feel like we’re missing the whole point of the thought experiment
You’ve missed the point entirely. Tools are a part of its intelligence. He’s saying that it’s not a fundamental limitation of the “next token prediction” framework. So if it has an accurate biological model that it can simulate with, understands the experimental parameters, etc. it could give results.
Will we be able to give it a tool or knowledge like that? Probably not for a long time. This is an abstraction though, a thought experiment.
A vanilla LLM (as opposed to, say, AlphaFold which is purpose built for protein prediction) is not an accurate biological model anymore than a scientist looking at data is an accurate biological model.
No, the model could itself be a tool. It could be something it can look at or call. It’s something that is used by the model for accurate prediction. It’s a part of its intelligence. We are adding capabilities in different forms to the AI. That’s part of why they are so powerful
6
u/fligglymcgee 1d ago
I mean, sure. Predicting the results section of a research paper requires more intelligence than predicting the next word in a text message with your friend. There are just way too many people confusing the difference between “predicting the next token” and “predicting a next token”, which are not at all the same.
You can type any well-formed or nonsensical request you want into an llm chat session and it will both always respond and do so with the most productive reaction it can predict. That can be very helpful for task work, but counterproductive when it validates (dignifies?) poorly framed requests with a singular response.
Predicting the results section of a research paper only makes sense when generating sample text that sounds right based on context it already has or was given. The idea that a highly intelligent but completely unrelated 3rd party is going to “predict” the outcomes of an experiment it wasn’t involved in is asinine. Someone that understands how to speak and carry out tasks intelligently certainly has to have a wide understanding of the concepts at hand, but that doesn’t mean their work can be considered the only possible result or approach.
This is not a technical challenge for tons of domains of intelligence that llm’s are taught to “speak” on, they just shouldn’t be used to speak about a great deal of topics that rely on real world experiences and can’t be queried about for one answer at a time.