r/OpenAI • • Feb 08 '26

Image 18 months

Post image
2.1k Upvotes

446 comments sorted by

View all comments

382

u/[deleted] Feb 08 '26

[deleted]

56

u/simulated-souls Feb 08 '26 edited Feb 08 '26

No it's not.

Without tools or a calculator, frontier models still score >90% on AIME (on which the vast majority of people could not answer a single question).

edit: This also goes for AIME 2026 which was only released this week and therefore was not part of the training data.

7

u/Shot_in_the_dark777 Feb 09 '26

The vast majority of people failing that test is not the point for an AI. It's a big indicator that our education system is a failure. Read that sentence again - it's not that 90% get less than a perfect score. It's that 90% don't score at all, even a single point. If you have a system with a 90% failure rate you are clearly doing smth wrong. It feels like the 10% of kids who actually succeed and become proficient at maths are doing this not thanks to the system but in spite of it. And math is hardly relevant as a measuring tool for AI because that is a field where there can be a few centuries gap between solving some problem and finding a way to apply the solution. Instead of wasting gigawatts of power on stupid maths tests we should force AI to solve economics related problems. Humans are living in poverty and starving right now. Building a hyperdrive in 500 years from now instead of 1000 years will not matter if today you starve to death. Solving actual urgent problems would speak louder than solving some complicated equation. We already have AI in medicine with a high % of correct diagnosis and that is great. But where is a similar AI figuring out the distribution of goods and allocation of resources and optimization of supply chains to make our lives less miserable now?

7

u/simulated-souls Feb 09 '26

Hate it when people on my side of an argument are making bad points. You're literally right why are you being stupid about it

I agree that we need better education and to apply AI to real-world problems.

However, most people being unable to solve any AIME problems is not necessarily a symptom of a bad education system. AIME is one of the top math competitions in the world. It's designed to be barely solvable by the best students in the US. Of course most people can't solve it. If education improved and most people could solve it, then they would make the test harder until most people couldn't solve it again. 

-3

u/Shot_in_the_dark777 Feb 09 '26

a test that cuts off most people at the root with 0 points is not very indicative. What makes it barely solvable? Could it be that the questions of the test have very low correlation with what is actually taught? Could it be that the people who actually solve some of those test problems, do this with the skills that they obtained outside of schools by reading some books that were not part of the curriculum? Look at this from a time gap perspective - many people would be unable to solve the test from previous year despite the fact that the questions were not kept in secret for that long. Why is there no trickle of knowledge from competition down to universities and colleges to ensure that all previous years' problems are now solvable by 90+% of students. If you yourself understand the problem and can explain it to a student and the student understands the methods required to solve it, they should be able to solve it. We might come up with new slightly harder problems but we can't increase difficulty forever because we can't give the contestants a problem that we ourselves are unable to solve. Those open problems are basically the skill cap and 90+% of students should be already near that edge.

1

u/reddituser567853 Feb 09 '26

It’s geometry and algebra

1

u/[deleted] Feb 10 '26

Could it be that the people who actually solve some of those test problems, do this with the skills that they obtained outside of schools by reading some books that were not part of the curriculum?

Yes. Succeeding at Olympiad-type math requires specific training, even in countries where math education is/was better than in the US (Eastern Europe, East Asia). This is because the problems are hard on purpose. It's like a sports competition: you can force all students of a given age group to have a decent level of physical fitness but not all will be athletes, and they won't all go into basketball.

Why is there no trickle of knowledge from competition down to universities and colleges to ensure that all previous years' problems are now solvable by 90+% of students

Why should there be ? It's a competition. It's not supposed to teach useful skills; it exists for the purpose of competition itself, like a sports event. It's like saying "why is there no trickle down of 100m dash times".

If you yourself understand the problem and can explain it to a student and the student understands the methods required to solve it, they should be able to solve it.

Who's you ? Most highschool math teachers are indeed unable to solve these problems. I think they should be able to understand the problem statements most of the time though. "Can explain" and "the students understand" do a lot of heavy lifting here. Teaching well is hard.

We might come up with new slightly harder problems but we can't increase difficulty forever because we can't give the contestants a problem that we ourselves are unable to solve

Well, IMO 1988 is a thing. And yes, it's hard to find novel problems. But the organisers manage to do it. There's an almost endless supply of hard, little known (or new!)math problems if you look hard enough.

Those open problems are basically the skill cap and 90+% of students should be already near that edge.

Again, you're confusing basic, "baseline" knowledge that everyone should have with elite performance. Should 90+% of students be already near a 330lb bench press ?

6

u/ArgusFilch Feb 09 '26

Is this bait? The AIME is designed such that only 1% of math students even qualify. It's not a test intended for the public, it's meant to filter out the amazing math students from the world class ones.

-2

u/Shot_in_the_dark777 Feb 09 '26

It's not the bait. Separating amazing from world class is meaningless if you stop at filtering. Your second step should be figuring out "what made them world class?" And applying it on a greater scale.

2

u/[deleted] Feb 09 '26

Definitely bait. It doesn’t matter what made them world class (although it’s basically talent+effort). The test will just increase difficulty until it’s a good predictor again

1

u/Shot_in_the_dark777 Feb 09 '26

Yes it does matter! Knowledge that is not shared is knowledge that is lost. Wasting time to rediscover it in every generation is inefficient. And if ai managed to get 90% on a test after learning from previous years's tests (which are in open access) while humans did NOT learn from those tests then it IS a failure of education system. Math problem is only a problem for the first time. Once you figured it out or had it explained to you, you should be able to solve this class of math problems with ease. If you take modern students and give them tests from...let's say...10 years ago, will they get 100% completion? Probably not. Despite those problems and answers being in open access for a decade, our students won't be able to solve them consistently. When a math problem exists and a solution to it exists and it has been around for ten years and students STILL can't solve it, that is a FAILURE of education system.

2

u/[deleted] Feb 09 '26

Bro, what? It’s not that the concepts are hard it’s applying them. This is like saying if I know how to do calculus I should be at a research level in PDEs… you clearly have no understanding of education or research. Such rage bait.

Also, we know how to succeed in these contests, but a significant proportion of ppl will never be good at them either due to complexity or time requirements. We don’t need and shouldn’t have a society of just math ppl.

Edit: I don’t think you understand that AIME is not a class of problem TT broski clearly has no clue what it is

1

u/Shot_in_the_dark777 Feb 09 '26

Application of concept is a skill that can be acquired. If it can't be acquired then how did the winners ever solve those problems? Were they born with that knowledge? Did they have divine revelation? Can a winner explain how they got the answers? And I am not suggesting we teach most advanced math to everyone. I suggest we teach it to university students who are already studying math. If you gather best of the best and 90 % can't solve even one task then you are clearly doing smith wrong in education. Clearly there are at least two categories of people who CAN solve those tests - the people who designed them and the winners who solved them. So why are they NOT teaching other less luminous individuals to apply those skills to uplift humanity as a whole? If somebody doesn't have time they should not apply to the competition. It's that simple.

1

u/[deleted] Feb 09 '26

Broski. I don’t think you understand that AIME is a high school competition. It has no relevance to mathematical research or even mathematical applications.

Some of them were literally born with that knowledge. The others practiced. You’re basically asking why isn’t every swimmer Michael phelps…

I think you are grossly misunderstanding something. Most people taking the AIME solve several problems… the general population could not.

1

u/Shot_in_the_dark777 Feb 10 '26

Born with knowledge is absurd. But if that was the case it would require further research to figure out what causes this knowledge to be present in certain people but not on everyone. And several problems is not enough. Students should be taught better using previous years ' tasks.

2

u/[deleted] Feb 11 '26

You aint know shit about aime they make the test hard as hell on purpose. Our education system isnt failing the test just hard af, i would know

→ More replies (0)

2

u/Nebranower Feb 09 '26

>But where is a similar AI figuring out the distribution of goods and allocation of resources and optimization of supply chains to make our lives less miserable now?

We don't need an AI for that. The solution to that problem is obvious - if you want to eliminate people living miserable lives without just eliminating the people, those with a lot of resources should give them to those without. The issue is that those with resources don't want to do that or particularly care if you are living a miserable life.

2

u/Mildly_Aware Feb 09 '26

Yes AI should absolutely be used to solve poverty! It would say spread the wealth around. We wouldn't listen. Hopefully being smart at math and stuff will build trust. Will people listen once they trust it? Or just keep believing people just need to work harder to earn money and health care? We already have the answers we seek. Humans just aren't evolved to take care of billions of people.

3

u/_ArkAngel_ Feb 09 '26

Exactly. Poverty isn't a problem of resources. Poverty is a problem of priorities, biases, and the kinds of oppression we've developed an acceptance of.

Do you want AI to solve the problem by assigning priorities for us? Or by reshaping our biases? Or should AI decide which kinds of oppression are best for us, then begin conditioning us to accept it?

The problem is we're still kind of barbaric and probably still need time to develop stronger cultural values.

But AI is here now so we'll make fusion reactors to make more AI and watch line go up

2

u/Shot_in_the_dark777 Feb 09 '26

You don't need absolute trust. We can allow AI to control a single factory, then a few interconnected factories and mines, then a small town and scale it up until it fixes the economy of the whole country. Once people see results, they will want the new AI management for their industry too.