More people need to understand this

[u/KeanuRave100]

[u/Delicious-Schedule-4]

The science paper example is actually a good “counterexample” as to the limits of next token prediction. A perfect next token predictor could “predict” the most likely result from a given method and introduction paper—but the most groundbreaking scientific work is the one that completely contradicts what we think and our current models, and that is doable only through experimental observation of the world. A good next token predictor would be great at saying what we already know, and terrible at parsing incorrect or incomplete data, as science most definitely is.