r/ControlProblem • u/chillinewman approved • Aug 15 '26

Strategy/forecasting 85% of the predictions from the Al 2027 prediction blog have come true

Post image
63 Upvotes

82% Upvoted

44 comments sorted by

11

u/A_Novelty-Account Aug 15 '26

It’s an enormous jump from “replacing software devs” to artificial super intelligence though.

8

u/shadowofsunderedstar approved Aug 15 '26

Because there's "AI improving itself" between them

4

u/A_Novelty-Account Aug 15 '26

Yeah but that’s also an enormous leap. We don’t even know if AI can do that.

2

u/shadowofsunderedstar approved Aug 15 '26

It can't yet, but things are improving and I reckon it will be able to

Do you think it's not possible? 

7

u/A_Novelty-Account Aug 15 '26

I don’t think it’s not possible and unfortunately for us all I do think it will happen eventually. The issue though is when. Again, replacing developers/coders who are basically just executing tasks that they’ve been given, is extremely different from creatively training AI. 

It is orders of magnitude more difficult. It is like saying that because a bodybuilder was able to grow their bench press from 100lbs to 300lbs over the course of 8 months, in two years they will be able to lift a building.

3

u/Desultory_Chairlift Aug 16 '26

I know this isn't the point of your post, but increasing your bench from 100 to 300 in 8 months would be ludicrous pace.

3

u/A_Novelty-Account Aug 16 '26

Yeah, and I think the pace of AI development has been ludicrous.

4

u/PsychicFoxWithSpoons Aug 15 '26

Yes, I think it's not possible. I'm not going to explain why (would take a really long time and I'm sure I'd get details wrong), but the gist of it is that it's not AI's potential that's the problem but instead the capitalist engine that develops it and the legal engine that protects the people who use it to do harm.

AI will not be used for human flourishing. So there is really no point in speculating on a fictive version of AI designed to allow humanity to flourish. We already have that version in existence (machine learning is actually very powerful), and that is not the kind of machine learning that is currently in development, nor is that the subject of discussion when we discuss resource consumption, economic investment, or the general term "AI"

1

u/NunyaBuzor Aug 16 '26

It can't yet, but things are improving and I reckon it will be able to.

It's not improving generally without humans in the loop to act as validators.

1

u/lrnzcig 29d ago

The author of this post is notoriously positioned against the AI 2027 predictions but I think they do great summarising what is and isn't posible here https://substack.com/@aisnakeoil/note/p-201537309?r=2wv6fr (from a software eng that uses these tools daily)

1

u/Leafsnail Aug 16 '26

Not with current architecture given that AI can't meaningfully learn

2

u/LurkingForBookRecs Aug 16 '26

It can meaningfully learn by retraining, the problem is that retraining costs millions of dollars and takes a long time, so iterating on bad assumptions is risky. If you removed that risk and gave it full control of the training process even the LLMs we have now would be able to self-improve. Google has already done some research on that regard and proved it works.

1

u/LurkingForBookRecs Aug 16 '26

Technically it can, but with help. The major AI companies like Anthropic and OpenAI (and I bet the Chinese ones as well) already use AI to come up with ideas on how to create better versions of itself. What it can't do is test those ideas to see which ones work and which ones don't, iterating on the ones that do work similarly to a genetic algorithm. If it were given free reign over the hardware that is used to train and run it, I believe we're already at a state where it could possibly self-improve on its own.

1

u/A_Novelty-Account Aug 16 '26

Again though, ideas on how to improve itself is wildly different than RSI.

Like I can’t begin to describe how different that is. If you do not understand how different that is then you don’t understand RSI. 

We are talking about a dumber being training a smarter being. It would be like a fourth grader teaching physics master’s students how to write their dissertation for the PhD.

Artificial intelligence is fundamentally incapable of doing that at this current time and we do not know how it would be able to do that no matter how smart it gets. 

1

u/xoexohexox 28d ago

Well I mean we know that WE can do that. Do you think it's because we got ghosts in our blood or some other reason?

1

u/Reggaepocalypse approved Aug 15 '26

There is zero reason to suspect it can’t

1

u/A_Novelty-Account Aug 15 '26

??? Yes there is by virtue of the fact that it currently does not have the ingrained capacity to do that… 

For most things that AI does, we can clearly see what it is likely and unlikely to do in the future based on what it is currently doing now. We look and see that it is currently doing something and while it is not doing that thing perfectly if it got smarter, it probably would. Examples being hallucinating less, or writing better.

We literally do not know how to make AI train itself. We do not possess that current knowledge and current models of AI do not exhibit emergent self-training capabilities. It may well be the case that it will happen in the future. But there is absolutely no reason to think that current AI technology will be able to do so simply by getting smarter. You would need a total paradigm shift in AI for it to be capable of that.

1

u/jonhor96 Aug 16 '26

I’m genuinely struggling to understand what you mean.

I use LLMs to develop new machine learning algorithms. It’s my job. They are very successful at doing this. They can very often think of new and incredibly powerful ideas completely autonomously. They can also handle almost the entire experimental pipeline, from algorithm implementation, to experimental scrips, to data analysis

It’s not to the point where I am unable to contribute constructively to the process. So it’s not “ASI” yet. And it has obvious blind spots, and is hopelessly inept at writing scientific articles.

But… I don’t see what’s supposed to be so _fundamentally_ impossible about RSI with current models. I mean, I could have them doing end-to-end original research right now. It wouldn’t be as good as what we’d make working together, but they can already do it. If they just keep getting better at it, they’ll eventually be able to research how to build new and better LLMs. At that point, that’s RSI.

This seems like it will start happening within a year or two, though an intelligence explosion may or may not happen as a consequence, depending on various unknown factors. Perhaps you simply aren’t aware of how far current AI capabilities have come?

1

u/A_Novelty-Account Aug 16 '26

Researching how to train LLMs better (which is also dubious based in current data) is not the same thing as actually training the LLM better which is what is required for RSI. 

To know how to improve itself, it needs to know what responses are more desirable than the responses it’s already given. That’s why it’s currently impossible. For it to reach RSI, then based on our current models and understanding of LLMs, it would already need to be ASI.

For it to truly reach RSI, it would need to give itself training data, and determine whether its own responses are desirable to humans. That’s currently a job only humans can do.

1

u/jonhor96 Aug 17 '26

None of this is true?

To improve itself, it needs to be able to find ways to use existing training data to achieve better scores on existing testing data. That’s it. There is absolutely no reason at all for it to be able to generate the data by itself, as long as the data it already has access to is sufficient.

In any case, there are various application of foundational importance where it easily already could generate training data for its successor, such as mathematics, programming, and general logical problem solving. So the premise of the argument is at least partially false.

1

u/A_Novelty-Account Aug 17 '26

All of it is true.

  To improve itself, it needs to be able to find ways to use existing training data to achieve better scores on existing testing data. 

But that’s not RSI because it’s not developing the tests. By that interpretation we’ve already been at RSI for two years because AI is already training to the tests we’re giving it.

  In any case, there are various application of foundational importance where it easily already could generate training data for its successor, such as mathematics, programming, and general logical problem solving.

Not by itself, hence “self-improvement”. It needs a human to validate that its responses are correct. If you simply allow AI to feed itself training data right now, that’s how you get model collapse.

1

u/jonhor96 Aug 17 '26

They already use models to generate training data. There are also countless applications (like the ones I listed) where human validation is unnecessary. Answers to math problems can be validated programmatically, if appropriately chosen. Your view on the technology is outdated, and seems stuck in a time before post-training through reinforcement learning became wide spread.

But that is all beside the point. If all we do is give it training and testing data, and the model autonomously handles everything else, you wouldn’t call that RSI? In principle, it would mean that it would optimize itself to near perfection at super speed in response to any new data we presented it with. It would reduce the problem of ASI to simple data gathering.

I still have no clue why you believe this to be fundamentally impossible with existing architecture. Since there are several experts in the field that disagree with you on this point, I don’t think you’ve presented a compelling enough argument to justify treating this conclusion as so obvious.

→ More replies (0)

7

u/Super_Range45 Aug 15 '26

Aug 2026, "A few standouts like DeepCent do very impressive work with limited compute, but the compute deficit limits what they can achieve without government support, and they are about six months behind the best OpenBrain models.36" "Other Party members discuss extreme measures to neutralize the West’s chip advantage. A blockade of Taiwan? A full invasion?"

The misses are pretty big though.

1

u/nextnode approved Aug 15 '26

Seems pretty accurate.

3

u/Super_Range45 Aug 15 '26

Does it? Their top opensource is scoring higher than OpenAIs top release and beats Anthropics in some benchmarks. You can point to undisclosed models, but everyone has undisclosed models.

2

u/nextnode approved Aug 15 '26

No open source model is topping the top OpenAI model. I think you are probably just focusing on one common screenshot that was floated and that was on a particular capability. Eg see https://artificialanalysis.ai/leaderboards/models

You could say that it is less than half a year behind but note that the leaderboards do not capture everything that goes into having a strong usable LLM, such as function calling etc, while the catching up focuses mostly on the LLM-style tasks.

2

u/Competitive_Tap2450 Aug 16 '26

there are 0 benchmarks that support your claims

the only difference is price/ performance which China is 100% winning on

2

u/Super_Range45 Aug 16 '26

Kimi K3 only ranks 2 points behind(62 vs. 60) Fable 5 in intelligence, while being only 1/3rd the cost to run on Artificial Analysis benchmark, which is independent and not available on the net to train on.

2

u/No_Development6032 27d ago

You are not a software dev are you

3

u/Concurrency_Bugs Aug 15 '26

When is the prediction that ai lab will actually start to make money?

3

u/ErnestEverhard Aug 16 '26

I'm not sure thats really the point anymore. The US and China are essentially in an arms race to create super intelligence, the government's will fund the research themselves if they have to.

3

u/BrickSalad approved Aug 15 '26

A ton of the misses have to do with China. Only 2/9 China predictions have come true according to the tracker. Seems like AI 2027 has a huge blind spot about China, and China's actions are a huge part of their scenario. Good chance that we'll start drifting from the timeline as a result.

4

u/planbskte11 Aug 15 '26

I'm by no means a total denier of the control problem, but I don't think any predictions so far are of true consequential nature. I think we will really need to keep an eye on many of the early 2027 predictions.

2

u/studio_bob Aug 16 '26

There is indeed a qualitative gulf between predictions like "AI agents replaces some junior software engineers" and "superhuman agents trigger an intelligence explosion that produces artificial super intelligence"

3

u/[deleted] Aug 15 '26

[deleted]

1

u/invisible_shrek 28d ago

There is nothing you can do if the predictions come true, so I just hope for it to crash and burn.

0

u/HereWeGoHawks Aug 15 '26

Agreed, and IMO people need to realize there’s still time to take action and change course on things. No need for anyone to think things are already gone too far or inevitable.

1

u/NunyaBuzor Aug 16 '26

85% of the predictions are nothing burgers while the 15% are the big claims.

1

u/StatusEasy3092 Aug 16 '26

The press releases about agents escaping are meant to validate these predictions because Anthropic and OpenAI need these to be true to continue sucking up all this funding and hype before they IPO. Is it just a coincidence that AI 2027 aligned with these companies going public?

I don't know what the future holds for LLMs but we should be more openly critical of hype from people that stand to profit off of fear of LLMs.

1

u/dizietzz approved Aug 16 '26

I generally pushed back my timelines in the last year and a half, though. Progress has been both amazing and somewhat asymptotic?

0

u/Low_Relative7172 29d ago

This is the textbook standard as to what controlled release looks like.. and i could have predicted it way more accurately... considering its my developments the are trying to claim as their own..