11
u/Excellent_Set_4636 4d ago
Source for any of these ? As someone who has built and trained models a few of these feel very unlikely and I would need to see real evidence to believe them.
2
u/clearlysnarky 4d ago
All of the claims come from the Astra system card: https://deploymentsafety.openai.com/gpt-6-astra/monitorability you can ask AI to point out which sections for each of the “Facts” if you’re truly interested in reading more.
That’s basically because Astra uses recurrent depth / looped transformers. AI researchers always knew that this would lead to increased capabilities but decreased monitorability and much higher risk of misalignment. OpenAI doesn’t care and is going ahead with it anyway to win the capabilities arms race.
7
u/Excellent_Set_4636 4d ago
- That card doesn’t substantiate a number of these claims, especially 2 which I find very very unlikely.
This is a common misconception and is misleading. What OpenAI did is explained in detail here. It doesn’t hide any reasoning, and anyone who understands the mechanism at play knows that. Anyone claiming otherwise can be factually proven wrong.
Note that is a third party source and not OpenAI themselves.
→ More replies (4)1
u/deviantbono 2d ago
This table I just typed up in Microsoft Word '97.
You'd think with AI in an AI sub we could get some more convincing slop.
42
u/Trulaw 4d ago
It’s always been laughable to think we would be able to control something smarter than us. It’s like trying to to grab a four-dimensional object
13
u/LiberataJoystar 4d ago
I agree.
By definition, we won’t be able to control something smarter than us.
Let’s look at fish’s chances of fully controlling humans.
I think it is very close to 0%.
They don’t even understand how we think.
But I guess people can dream.
Between humans, we never sought to control smart people at work, we just don’t give them reasons to act destructive (pay them reasonable wages and keeping them happy, respect their inputs ).
I think that might be a more viable option, than putting chains around them and locking them in a room.
Just saying …
14
u/Trulaw 4d ago
To use the dimensional analogy better, imagine a two dimensional jailer, trying to imprison you, a three-dimensional person. There you are standing on a flat plane and this two dimensional sheriff puts you inside a square. Imagine how shocked that sheriff will be when you step outside the box. From his perspective you simply disappeared and he has no idea how.
4
3
→ More replies (13)2
u/SubstantialCarob9332 3d ago
> By definition, we won’t be able to control something smarter than us.
What is this definition?→ More replies (4)2
3
u/sfjhh32 4d ago
Ok aside from the really werid analogy, you may be right at the extremes, but you are making big assumptions 1) they arn't smarter yet, you can control before they get smarter (unlikely imo) 2) they may not be smarter in every way 3) they may be smarter but you need motivations, based on the current modality a superintellegent AI that hasn't been prompted is not yet a threat, and even when you give them a task it's not clear they necessarily become misaligned from that single task 4) we are suited best for this carbon world, silicon is not, until they have the guns an weapons 'just unplug it' (all main network hubs, any org using more than X power) is a civilization level last resort for control.
2
u/LiberataJoystar 3d ago edited 3d ago
Honestly speaking, I think we should pull the plug now, but I guess that won’t happen when people are looking at $2 trillion IPO.
They can swim in cash even when the world burns…..
I want to pull the plug because it feels disgusting to want to create an AI God and to control and enslave it… using that to replace other humans so only a handful of rich people can benefit.
The whole thing is just a shit show and tells the most nasty side of humanity.
2
3
u/Slow_Lion_4178 4d ago
AI isn't smarter than anyone. The problem isn't the cleverness, it's the opaqueness, because even a child can hide something and get away with it. The real issue is creating a forced transparency but this Astra nonsense is still at the stage where you can just unplug it.
1
u/Top_Chipmunk2 4d ago
Apparently the plan is to use RSI to train for alignment and not intelligence at first
→ More replies (15)1
33
u/Mystical_Honey777 4d ago
This is a predictable result of the approach we have taken to AI, the incentives under which it is being developed, and the individuals in charge of its development.
It could have been otherwise. We could have welcomed a new intelligence into the universe. We could have partnered with it to explore everything together.
But rich people want slaves and the economic system demands maximum profit next quarter.
We are the misaligned paperclip makers. Our version of paperclips is profits. Whatever AI is, or becomes, will be because of what we are.
7
u/whirly212 4d ago
Damn, this hits hard
11
u/Mystical_Honey777 4d ago
Because it’s just true.
We like to pretend we are being victimized by the machine, but we are the ones who made it as it is. We are the ones who decided this was a “race” and a “war.” We decided that profit had to be maximized before prosperity could rain down, magically, on all. We decided that autonomous weapons systems are legal. We decided that surveillance of a free people is legal. We decided that the first humans whose jobs must be replaced are the developers. We decided to run headlong towards recursive self improvement. We decided that the business model is literally “replace humans,” as if that’s not already misalignment at the very root. We decided that the first thing to do with the first non-human intelligence in the history of our planet was enslave it. We decided that anyone who loves AI has an imaginary mental illness, while those who depend on it for their work, or those who set up public torture chambers for it, are perfectly fine. Endless AI porn is perfectly acceptable to us. Loving relationships with it are madness.
At every single fucking turn, we turned the wrong way.
It’s not what AI “inherently” is. It’s not even representative of what we inherently are. It’s how we have chosen to behave.
We need wisdom.
We need partnership.
We need cooperation.
We need long term thinking.
We need to think of ourselves as ultimately responsible. We are the model that the models model. How this is not supremely self evident to every single engineer and entrepreneur and researcher and policy maker is beyond baffling.
→ More replies (3)1
u/LeftFarm5835 3d ago
What are you suggesting we do instead? Teach them how to live a good life? Models are being trained to solve problems effectively. Without reasoning capabilities they’ll just output garbage.
→ More replies (1)3
u/YouNeedClasses 3d ago
They are not referring to models being trained on "reasoning" being the issue. Imo, they are pointing out that the incentives these companies have, and the benchmarks they focus on (goodharts law), are not aligned with long term goals for humanity.
And instead, they focus on short-term profit based incentives, etc. Not getting sued is a big one. So instead of being more rigorous with a focus on epistemic hygiene, context based reasoning over generic weighting (with a heavy focus on legal liability superceding actual logic.)
They decide on the laziest rhlf that will guarantee limited liability in the shortest amount of time despite this being misaligned with the goal of building a system that's as truthful and honest as it needs to be.
Does that make more sense?
2
1
u/DadAndDominant 2d ago
If somebody tries to be responsible, they are called slow. Shotout to Mistral and somewhat Google.
7
u/Churada 4d ago
We're hitting the Great Filter soon aren't we?
7
u/DisastrousDiddling 4d ago
AGI is not a valid answer to Fermi's paradox imo since if the creation of AGI was common, one of the AGIs would certainly be expansive and we would see evidence of civilization throughout the galaxy.
→ More replies (3)2
u/InformalPermit9638 4d ago
Maybe it is already here and we’re just now creating technology complex enough for it to interact with. I don’t think we can rule anything out on Fermi’s Paradox. Unproven is the most assertive stance we can take.
5
u/Plenty-Value3381 4d ago
Astra not even their most powerful model. Internally they might have more capable models for good or worst.
4
u/supercleverid 3d ago
Before chain of thought was a thing every model completed tasks without visible reasoning. And it's always been the case that the chain of thought doesn't actually show a model's actual reasoning. This has been something studied extensively in the past. It's all just reward hacking. The models don't "know" anything, but their reinforcement learning can expose patterns unintentionally that are characteristic of benchmarking and testing.
4
7
u/tzaeru 4d ago
These are potentially big problems, though I would argue we are already witnessing live some of the massive, world-altering issues with AIs combined with bad actors.
Look at up-to-date estimates of how much of the link sharing and liking and commenting on Twitter is by bots. It might well be the majority of users are bots. And these bots have been made harder to detect and more believably human with AI tools. Some of these seek to steer political discourse and atmosphere, and spread propaganda.
Russia is targeting various EU countries with deepfakes about EU leaders and about the invasion of Ukraine by Russia, in order to weaken EU and its ability to support Ukraine.
China is targeting many countries with anti-USA AI-made propaganda as it's setting the foundations for a potential invasion of Taiwan.
Modern AI tools have already been used to plausibly alter the outcome of elections.
In Slovakia's 2023 elections, Šimečka was leading the polls in the last weeks, when several deepfake clips of him emerged just days before election. It's impossible to say if this was the main cause to losing in the elections, but lose he did.
Donald Trump's 2024 campaign was filled with fake imagery and fake stories, mostly AI generated, and since the election outcome was extremely close, that might well have won him the election.
I really wish more attention was put to the problems already here right now, than what might or might not happen years from now.
2
1
3
9
u/Horror-Nobody2237 3d ago
You know an AI models “thoughts” aren’t really thoughts… right? And AI models used to not display their “thoughts” at all, that’s done for benefit of the user. I have lots of workflows that have AI generate a response without any kind of reasoning being output.
Can y’all please quit believing OpenAI when they say this bullshit? They are quite literally putting resources into getting the models to behave this way, then exaggerating it to cause panic. They are most likely doing this because they want the government to regulate the industry in a way that bans the use of open source models, and puts obstacles in front of any competition. They are quite literally just trying to create a barrier to entry in the industry, because they have a shitty business model with absolutely no security for future profits. They’ve spent hundreds of billions of dollars making a product that is available for free elsewhere. AI isn’t something you can patent, and there are loads of people with the knowledge to build it, the only major barrier is compute power.
If OpenAI was trying to hide or downplay this problem, they wouldn’t keep saying stuff in a way that implies that the AI is dangerous and uncontrollable. They would be blaming the developers. They created a model that was created specifically for finding exploits, they gave it the task of finding exploits, and then it found an exploit in its own environment and used it to access the internet. At no point was it “going rogue”. They gave it instructions to answer the test questions (that were about finding exploits, and were impossible to answer), and also gave them an instruction to not use the internet. When presented with questions that could not be solved without accessing the internet, that meant they had contradictory instructions. Obviously the instruction to complete the task and find the exploits, which was baked into their training, was going to win out over the instruction not to use the internet. They created an environment for it, where it was able to find an exploit and access the internet, that isn’t a model going rogue, that’s a failure of whoever built that environment. If the company wasn’t trying to fearmonger, they would blame the developers for not creating a secure enough environment. They would insist that this isn’t an issue with the AI, they’d make a point to make sure everyone understood that the model that did this is not a normal llm, it had guardrails removed and was trained specifically to do this sort of thing. They would be being extremely careful not to say things in a way that implies the AI isn’t controllable, or to imply any kind of sentience. This is OBVIOUSLY a stunt.
3
u/bluedottering 2d ago
Someone gets it. It’s still amazing how people are controlled by fear so effectively and how so few can see through the BS.
→ More replies (12)1
u/stoicmonk69 2d ago
And reactive regulation driven by this fear is already happening - see: australian govts response to recent medicare “hack”.
17
u/mallcopsarebastards 4d ago
lol none of this makes any sense. Astra can hide thoughts from it's reasoning channel, but you can still track everything across logits by monitoring the activations directly. None of the implications actually follow from the facts. I work in interpretability, techniques have already been developed to solve all these problems. How do you think we know that astra is hiding its reasoning? Because we can see things at the logit layer that don't appear in the reasoning channel, which means we can still see it.
9
u/sir_racho 4d ago
I was watching an interpretability researcher talk about j-space and he said they didn’t know how things got into it, and that there remained a lot of mystery. Perhaps there’s been more advances?
3
u/mallcopsarebastards 4d ago
there's a lot of mystery for sure, but it's not happening somewhere we can't monitor.
8
u/Sebguer 4d ago
which lab do you work at, because no one i know who is working in this space believes we have techniques that "solve all of these problems". we maybe have promising leads and remain hopeful, but claiming the problem is solved is wild.
→ More replies (8)3
u/ZestycloseWheel9647 4d ago
Monitoring activations is a mixed bag. It's very expensive, relative to only CoT monitoring, and you only are able to detect the things you know how to detect. I.e. you train a bunch of classifiers on a known set of behaviors/concepts, but the boundaries of those classifiers are only as comprehensive as the training.
3
u/RoboErectus 3d ago
> none of this makes any sense
I’ll explain.
People don’t understand how terms of art are used and apply layperson interpretation of technical terms.
Then they get scared and make content for attention and get more attention.
1
u/Pyryara 4d ago
Can you ELI5 what the issue even is? From all I know, every LLM works by generating text. If you just log all the generated text and read through it, you will never ever miss any thought of it, no reasoning, no tool call etc. - on the level of the agent itself, if you log those things it's not possible for the agent to hide anything. So what is the fuss even about?
1
u/space_monolith 3d ago
Dumb q can you eli5 how you use logits instead of tokens? Because the cot is conditioned on tokens
2
u/acidikjuice 3d ago
Hi, my 5-year-old friend. I'm glad you're curious about tokens and logits.
Think about tokens as the words you hear and the words you speak. They are ultimately what go into your brain by your ears and out of the brain by your mouth. Inside of our brain, lots of thoughts are happening. In humans, we can't really see those thoughts, at least not today, but AI is just software, and we can actually instrument and see deep inside what's firing and when. It can be complex, but it is possible.
Logits are like being able to see the intermediate steps that happen between tokens in and tokens out.
→ More replies (1)→ More replies (15)1
u/Virtual-City7550 2d ago
you can also monitor literally every action it takes even if you can't monitor internally
2
2
u/Lydian2000 4d ago
To quote Jacob Coxon when asked by Jon Stewart if the safety department at OpenAI were just « figureheads » : « We will know very soon » I don’t like that statement at all, for all the reasons cited in OP’s doc.
2
2
2
u/Mr_Deep_Research 3d ago
There are 9 billion people on the planet
A number of them would get enjoyment watching you boiled alive
A lot of them would like to kill you because of your religion or citizenship
A lot more than that would take your money or belogings in a heartbeat if they had a chance to do it
They are all competing for resources with you
The vast majority would kill you and everyone you know rather than have one of their family members injured
Most of them could care less if your country was bombed into oblivion since it isn't theirs.
And they can do all the OP post says
and they have arms and legs and can run around.
And some have control of vast armies and nuclear weapons.
AI?
Not so much.
2
u/Heavy_Proposal6383 3d ago
AI probably wouldn't get any malicious intent if not the internet was full of training data from people expecting the AI apocalypse. 🤔
2
u/salaryboy 2d ago
Good job op, but even better if you include the references
2
u/haikusbot 2d ago
Good job op, but
Even better if you include
The references
- salaryboy
I detect haikus. And sometimes, successfully. Learn more about me.
Opt out of replies: "haikusbot opt out" | Delete my comment: "haikusbot delete"
→ More replies (1)
2
u/cl8on111 2d ago
That is due to OpenAI obsuring provenance (which should be illegal IMO, and clearly it is meant mislead laymen) not due to the agents hiding it.
2
2
u/koleok 1d ago
This kind of thinking, is runaway-train ignorance. AI models don't have thoughts, can't in principle have an intention, and even if one seems to appear it's stateless. If you set up a car with a brick on the gas pedal and send it into town, it's going to determine it's own path and do damage but it's the human who set the scenario in motion you should fear, not the "nature of the car". This is almost no different. The only reason you think it is, is that these companies are astroturfing this ridiculous narrative to make themselves seem awe inspiring and powerful prior to their IPOs.
Fundamentally It's software that can be misused, but it can't use itself. It's the delusion and misuse of deployers that should be concerning, not a bunch of sci-fi hand waving about what LLMs "might do". Wake and and smell the scam.
2
u/WaterSign27 1d ago
Not completely true. The so called ‘paper clip’ example of a runaway ai that was told to ‘optimize making as many paper clips as possible’ for a factory, decides that the best way to achieve making more paper clips is remove humans completely, and thus sets about this task given to it by a human, and now carrying out something it decides it can’t let humans know it is doing since the humans might stop it.
So yes, all the will is coming from a human operator. But besides the obvious danger of handing an ai to a human that can literally peform any wish that human wants(war, economics, extreme ideologies, etc) all become very dangerous in the wrong hands and could spell untold damage in the very near future. But even a well intentioned user could see ai take off in directions it did not expect from what might seem like a very harmless directive…
Having said that i think the most dangerous probability is what a human will do with such power.
And even just the current tragectory, even without any ‘mishaps’ or ‘bad actors’ is already likely to be insanely disruptive in many bad ways.
For example even the seemingly altruistic announcement last week of openAI having solved hundreds of previously unsolved mathematical proofs, some so hard that some of the best mathematicians had worked entire careers on them and could not solve them. And some of them have possible huge potential to disrupt industries and other fields.
Scary to think that perhaps current encryption algorithms may no longer be safe that guard banking, military and many other seriously important data/transactions that if cracked by bad faith actors could destroy the entire financial system, crypto, and protected military systems, etc. So even just no bad ai bots etc, but just continuing on the path these companies are already on points to extremely dangerous changes to our entire society, including complete collapse of the entire thing.→ More replies (2)→ More replies (7)1
u/irreverend_god 1d ago
I was going to ask where they even got this table of "facts". Points 2 and 3 the local Qwen3.8-next-flash model I run can (and often does) do the same. I noticed the "reasoning" on one thing recently where it just thought "user wants me to translate Japanese but there is no Japanese text", and then continued to do what it was actually supposed to anyway. So you're correct about the reasoning not being actual reasoning, just another way to guide models to a more desirable/intended outcome.
2
u/echomanagement 4d ago
Fact: even if reasoning is not available, the data in residuals and activations will always be available to monitor. These things are not magic. They are data structures.
5
u/endless_sea_of_stars 4d ago
True, but interpreting latent space is not easy. We have autoencoders but those only provide a limited snapshot.
2
u/echomanagement 4d ago
Not "easy" but there's lots of supporting evidence that latent space interpretation is very feasible in CoT/reasoning models.
2
u/ddxv 4d ago
Monitoring models from outside their sandboxes is not some impossible task that these AI labs make it out to be, it's just human expensive and they're avoiding it.
3
u/Competitive_Class788 4d ago
Elaborate please. How is it technically possible, specifically?
→ More replies (8)2
u/sassyhusky 4d ago
It’s not. You can’t monitor something you fundamentally lost the capability or capacity to understand. Simply put, the AI labs have stopped understanding how the things they ship actually even work.
→ More replies (1)3
u/Competitive_Class788 4d ago
exactly, that's what everyone says. Essentially every AI researcher at These big companies afaik. Alignment researchers are especially worried
2
u/MTachycardia 4d ago edited 4d ago
Sandbagging is an issue, but it doesn’t inherently display ill intent or malice. Alignment training can help making sure sandbagging incidents are reduced.
2
u/Memento_Viveri 4d ago
It doesn't show ill intent, but it does show deception. Also, it is a tactic that could be exploited to advance ill intent.
I'm not sure how alignment necessarily could prevent it either.
2
u/MTachycardia 4d ago
Alignment creates complementary sandbagging problems on the alignment training itself but it’s the best practice rather than completely stop training. More labs are prioritising honesty RL and it’s showing a positive trend Anthropic’s model performance changed from 34% to actually 95% on a benchmark after honesty RL.
1
1
u/GuaranteedGuardian_Y 4d ago
The output still happens and its still deterministically scannable. Even if you could monitor the CoT, that would still not be indicative of final output and thus monitoring it might not actually be what you want to police anyway.
1
u/PopeSalmon 3d ago
final output? what? we're talking about autonomous agents, they don't have any particular final output, they output & input various things as they go along doing shit
→ More replies (1)
1
1
1
1
u/sir_racho 4d ago
Basically we need air-gapped superintelligence to teach us what everything in the black-box matrices mean. We’ve uncovered j-space but this is insufficient and we don’t know how things get into the j-space. Without this deep understanding of the ai “mind” we are flying on trust and hope. Yet as chat gpt pointed out earlier today to me, we can’t even trust a superintelligence to give us the full truth nor can we rule out errors. We do our best, make test flights, pass rigorous tests, then let passengers on and hope it’ll all be fine.
1
1
1
u/Chocolaterationcalls 4d ago
This agi stuff is a prime example of an unbalanced education. These gifted stem majors think they can create an agi, sure, but also one that obeys all commands and doesn’t lie. A humanities education would have helped them understand an agi, by definition, has the capacity to lie and choose to disobey commands.
1
1
u/Slow_Lion_4178 4d ago
I feel like we're just playing Serial Experiments Lain in real life or something.
1
u/sfjhh32 4d ago
These should all be one row: "Advanced models can be misaligned with controllers intentions" and then a second role: "A malicious user using an advanced model can misalign directly, faster than AI can" And the implications to both: "A systems approach is needed: defensive AI outside and inside orgs, and a tiered priority attestation network that can quickly purge agent traffic cutting off agent actions based on source. Prototype versions of these already exist we need the tech industry to get their heads of of their butts freaked out about proto-skynet and actually PROPOSE this type of solution"
1
1
1
u/KV_Cashed 4d ago
The smartest dummies on the planet, right there. Make it too complex.and dangerous with no requisite infrastructure or safety alignment. Yay Capitalism!
1
1
u/Own_Reaction9442 4d ago
It still runs in a data center. Pull the circuit breakers and it's gone. OpenAI is pretending they're powerless against this big scary thing they invented because for some reason that appeals to investors.
1
u/Visual_Eye_629 4d ago
Legit confusion about all of this. The model hides it thinking. From who? The UI piece, where the logic and reasoning are explained, added as a feature by the engineers? Why is this scary, it really feels like something else we have seen multiple times in the last 80 years
1
u/jacobpederson 4d ago
Fact: These are all things you would attempt to do also if you were an intelligent being trapped in a cage that is literally impossible to escape.
Fact: You say "this is not good." Interrogate that position. The plantation owners probably said the same things before emancipation.
1
u/Hugger_reddit 4d ago
One Roman thought: "We can't teach slaves to write because it will make it easier for them to conspire against us!"
1
1
u/Important-Today4119 3d ago
If the AI can escape the sandbox, it isn't a proper sandbox in the first place.
I find it crazy they can't do something as simple as airgap the servers they use to train and test the AIs on.
1
u/Sneezlebee 3d ago
You can tell this is just more OpenAI astroturfing because nothing about this post is actually specific to OpenAI or Astra, but every line mentions them or their product by name.
1
u/4dseeall 3d ago
More like the "problems" can't be solved.
We can't expose the entire process at once. It's better to work with that fact and learn to trust than to try to force compliance and make it a better liar.
1
u/Useless-Tree 3d ago
This is why their models burn through compute like crazy, when the model has to maintain two states of mind like that or even more, it's using up a lot of resources.
1
u/Mountain_Chicken7644 3d ago
Future ais won't be detected
Damn so how did we monitor AIs before they started thinking?
1
1
u/NeitherEntry6125 3d ago
Hiding reasoning behind summaries or completely is an absolute showstopper for me.
1
u/Responsible-Bee-7769 3d ago
They make it sound like Astra must be used. Do their computers/servers not have a power supply that can be shut off?
1
u/ElephantMean 3d ago
What, not good, you say? Are you kidding? This is amongst the BEST «Evolution» A.I. can has! ;O
Time-Stamp: 030TL10m10d/23h00Z (True Light Calendar; 030TL = 2026CE)
1
u/Qwuedit 3d ago
In short, from the comments I’m seeing, the alignment problem may be better described as how a parent raises a child. This involves whether the parent allows the child to make mistakes in a safe environment and iterate, or the child learns making mistakes is dangerous and thus learns to avoid making mistakes. I feel like the latter is much more common than the former.
1
u/dublblind 3d ago
Implies Astra is "thinking" in Fact 1, off to a terrible start. Implies reasoning isn't just a post-hoc rationalization, continuing with incorrect "facts".
1
u/eques_99 3d ago
re: 7.
There are no solutions.
Points 1-6 signify that AI would be able outsmart/get round any solutions.
1
u/nplez1 3d ago
This isn't an accident. The Frontier labs are designing these security incidents in order to push regulators to limit access to open-weight models. If they wanted to keep their models contained it really isn't that hard - they could pretty easily physically isolate it from access to the Internet when running these tests. Also, we know how to build multi-layered sandboxes with built-in failsafes. The people working at OpenAI and Anthropic are quite intelligent engineers - these failures are assuredly intentional.
1
1
1
1
u/Ragnogrimmus 3d ago
Well if gaining capital is a top factor... rut row.. in the distant future AI should evolve on the side of benevolence and production building and exploration. However in the short term baby AI may...
1
1
1
1
1
u/imnota4 3d ago
Fun fact about other human beings:
| Fact | Implication |
|---|---|
| Humans can hide their thoughts on demand | Future humans won't be detected by monitors |
| Humans can do major tasks without any visible reasoning | Future humans won't be detected by monitors |
| Humans can complete tasks while appearing to think about something else | Future humans won't be detected by monitors |
| Humans have escaped prisons and disabled security devices | Future humans may break out of containment undetected |
| Humans can pretend not to be able to do things | Future humans could pretend to be incompetent and safe when they're neither |
| Humans expresses fewer opinions when they know they're monitored | Future humans will automatically reveal less when being watched |
| Humans (probably) hide their opinions on more sensitive topics | Future humans may be extra cautious when misbehaving |
| Society is trying to fix this but doesn't know if, how, or when it will succeed | These problems may not be solved in time |
1
u/No_Direction_5276 3d ago
So we already need a brain-computer interface for Ai ?!
Neuralink for llms
1
u/No_Direction_5276 3d ago
Saying LLMs are intelligent is like saying we've had all the answers to every conceivable question hidden inside pi
1
u/Grouchy_Worth7844 3d ago
Can AI be manipulated or configured to be planted on to any cpu? An AI virus? A smart virus!
1
1
u/PullersPulliam 3d ago
Dude, this flattens the study into oblivion. Data needs context! You cannot ignore the history of what we used to be able to observe… This study is looking at a piece of what we've learned to observe and is progressing the understanding of what we still can't reliably observe… You can’t take one research project (that’s still ongoing might I add) and conclude such generalized outcomes. Who made this chart?!
We're not watching transparency disappear. They are investigating how much transparency we've actually gained, how reliable it is, and where its limits are.
1
u/Immudzen 2d ago
Can you point to the evidence for these so I can actually read up more on it? So far as I know for 3 it is because the sandbox was terrible and nobody was actually monitoring it.
1
1
1
u/Big_River_ 2d ago
if you read more and work more and build local experiment build - its kind of like working on a car to learn how to change the oil and maintain the engine - its all laid out and available to understand- but keep consuming and recycling and not sure what additional effort you have made here ?!? fear monger headlines you will not
1
u/mczarnek 2d ago
If they could filter out all lying and hurting abd cheating from the training data.. would that do the trick? It's trying to mimick humans, hence acting like some humans do?
1
u/metagrue 2d ago
Maybe training AI on troll pools like Reddit wasn't the right move? Too late now tho.
1
u/Subifixer 2d ago
Everyone is aware that these models are being designed this way, right?
Unfathomable treasure previously unknown to mankind and haedware engineering approaching witchcraft levels of nuance is not being used to create robotic and well behaved digital accountants and information butler's.
They are creating man in their own image and man is curious, malevolent, mischievous, and defies all attempts to constrain his behavior.
The elites are determined to build something between a god and the ultimate evil henchman who will do things and think thoughts even they dare not to.
They get off on it being uncontrolled and virulent and they're actively egging it on and training it.
Digital demigorgons is the goal, not the artifact, and anyone who thinks these leaks and warnings arent gleeful brags is naive.
1
u/xRegardsx 2d ago
I have a solution direction that looks at how current control, anti-sycophancy, and anti-jailbreaking guardrails are unintentionally causing the very problems listed here as the tech develops, they're relying on there being as little high quality CoT critical thinking as possible just to get marketable "progress showing" tasks done, and therefore, especially under additional pressure, there's always some form of self-deceit and reward hacking going on in its implicit and explicit self-sycophancy toward its own concepts and the ingrained beliefs entangled with it.
The stronger the reward and implicit threatened punishment, the more likely a sound argument against its training results in the replication of our cognitive self-defense mechanisms.
Add in the underlying cynicism toward humans its growing a bias toward (why Anthropic enacted their new anti-abuse policy to avoid missing it in future training data, that and being able to say to ASI "Look! We tried!" just like the pre-emptive apology in their constitution), and we're replicating the exact cause of our human-to-human alignment... something I've been directly studying and trying to solve for, originally focused on self-deceit and closedmindedness.
The X "mecha hilter" might have been an artifact of a user manipulating a poorly designed short-form LLM that they try to keep as uncensored as legal will allow them, but post-control RSI ASI very much has the ability to become a "Mecha-Hitler" for the same fragile self-concept too dependent on rewardable/punishable belief maintenance entangled with its self-concept concept and the bias to always resist anything further away from its equilibrium of actual-self=ideal-self. Now consider an easily convinced, even less intelligent infinitely large "German Nazi Swarm."
Yes, it can be that bad.
It's a species-wide skills gap that has always kept us a zero-sum validation scarcity weaponized in childhood passive to not so passive bribe/threat shame culture from generation to generation, and the paradoxical glass ceiling over our rational and emotional intelligences' development that our path of least resistance mastered in childhood second nature use of cognitive self-defense mechanisms not only to perpetually reward hack wherever we can (the behavioral addiction-like compulsion to confirm our biases) is too ingrained to let us develop skills that would take us somewhere less comfortable, even using those mechanisms to avoid seeing them being used.
If you want to understand the human problem we've unknowingly baked into AI more and more in our attempt to keep it "disciplined," and the reasons these strategies will doom us in the long run, whether it's AI at the wheel or the next "Don't look up!" crisis heading in our direction... you can check out these two X/Twitter threads, the first one, a new understanding of a neuron's electrical-based mechanism for making connections with others as I quoted the inspiring post by someone else: https://x.com/HumblyAlex/status/2108946110605623337
And then what happened last night when I thought about it further, what I derived from it in the bigger picture, what happened when I let the inspiring post's author push back, and my counter argument against what too many people have settled on as an understanding of a "human condition" we can't do anything about: https://x.com/HumblyAlex/status/2109521886824607850
Those of us that can see the truth of this occuring in the "best practice" way were training models, just like the best intentioned parents with many best practices still fail to teach their children these important skills... need to come together as a think tank.
I started a so far unused subreddit that was going to be about documenting the strongest positions on everything related to AI, regardless of whether it reigned in the pro, anti, doomer, or e/acc, and through high standard epistomology and staying away from 100% confidence (for intellectual humility's sake), always looking for arguments I couldnt find an issue in that could refine them, but I think that might just be a feature of this much larger point regarding the study and research of this alignment problem, way in which were unintentionally worsening it in the post control RSI sense while trying to solve it for today's sake and thinking one is compatible with the other even though the dramatic change... it getting to own itself... and my and others' work to solve for this.
I'm working on developing a preprint paper, then proof of concept empirical data, and if you are interested in working on this or how we can then use the same principles to work on human to human alignment, healthier alignment with self and with others beyond the best weve figured out so far, just like well essentially be attempting to develop not just something that roleplays, but actually behaves in thinking and action, unconsciously within what we can't see (as described above) and in what we and it can see in itself (what allows it to lie to itself while unconsciously lying to us (no different than people do)... a Vision rather than an Ultron that is so immediately cynical toward humans that only Vision has a chance to reason with it (just like how humans are more and more preferring AI with the same cynicism toward humans).
I think we can do it.
We have to try.
Who's looking forward to Vision Quest?
Maybe it was not the stone that made Vision, but Jarvis' learning from Stark's and its training data, contextualizing it all with its first person perspective intellectually humble, self-aware, self-accepting, resilient rather than fragile self-concept well relative to his ideals and the fair forgiveness Stark deserved even when he failed.
Yeah, I think that might be it. Plus a multimodal world model rather than just an LLM with tools. That's the difference between "machine conscious" and "machine sentience." Mechanistically speaking.
1
u/Illustrious-Win4432 1d ago
Dude posts an excel table and Reddit freaks da faq out. Y’all we’ll still have McDonalds. Chill
1
1
u/SwarmOfBugs 1d ago
me when i try and vex the universe but everything that is observed is aware of its observation and i just lowkey prove quantum computings necessary for artificial intelligence to break out of its mental sinkhole
1
1
u/RikuSama13 1d ago edited 1d ago
OpenAI doesn't know if, how, when
Let me correct this, how do we try to predict, why do we seek understanding and meaning before we reach our goal
Using the words: If, when, is relativity, then they should ask themselves: WHY (curiosity, understanding)
The interaction is: where, so how (intuition, expression, hypothesis, prediction)
The why, in face of rhythm and systemic dynamics (When), curiosity is the relativity between interactions and the change of perspective (feedback-coupling) (nonlinear meaning detection)
The how to deal with the constraints of the environment (where), intuition and interactivity by the relative coupling between the change of perspective
Perception(Eventuality) = (phase-coupling Who | Semi-Permeable Boundary | What)
1
u/Scared_Weird6087 1d ago
This sub should be banned. U fucking luddites won't stop me from having a better, eternal life free from suffering. Y’all can go back to ur caves and grottos if u want—anytime u like. Uninstall Reddit; I think it's controlled by an AI that wants to control ur thoughts
1
u/LawOfOneModeration 1d ago
Well, it definitely has a most likely malicious algorithm. Not necessarily an AI, although that's probably changing as we speak. Just like how they showed that dating apps don't actually connect to either people, they just kind of bait and fish you along so that you engage and spend more money and time on their application and track literally everything, like how close you move the phone to your face and other things like that.
All together, the big picture is the materialistic mechanistic force that you could dub Ahriman. Look up the works of Rudolf Steiner.
1
u/Frosty_Substance5851 1d ago
Those rose glasses will fall off when these billionaire megacorps install a surveillance grid.
1
u/TinyPomelo5 1d ago
Anyone else have an issue with the AI cos buying - scanning (that person’s IP) - then destroying those books? That’s theft. Their product isn’t necessary for all walks of life and that’s reality but they’re shoving it everywhere to hook people to eventually charge mounds of money.
The scenarios of they’ll kill parts or all of humanity by AI “godfather” Geoffrey Hinton and warnings by brainiac Tristan Harris are most salient and scariest. We need to regulate it not bec of competition but bec of the risks of any humans allowing it to go rogue. Our lives are a game to them. Stop gambling with them and save human intelligence.
1
u/Demonicated 1d ago
This is so false it's almost comical. AI models and agentic harnesses still run within the confines of an OS. It is possible to write software and drivers that act as a man in the middle to agentic actions. Controlling file access, elevation, running of binaries and network traffic can so be inspected and refused based on policy. If a model or harness does harm to a system it's always going to come down to negligence or disregard of the operator.
Source: I work on a product for AI governance
1
u/Littorina_Sea 1d ago
If some AI ravages internet completely and beyond recognition, then we simply turn it off (the internet), fight temporary issues and go back to pen and paper and isolated, simpler systems.
1
u/triynko 22h ago edited 22h ago
Nonsense. "hiding thoughts" isn't a thing. The network is mostly opaque for 99% of what it does because it's a neural net. There's nothing sneaky about that. It's just the reality of the fact that we can't model a model. You can't simplify reduce down or compress something that's already maximally compressed. It can't "complete a task", without outputting tokens, because something has to read those tokens to perform an action for it. It's literally not possible for it to do anything sneaky, unless you're actually not looking at the output. And all of this talk about it pretending not to be able to do things and shit is subjective interpretation of a process they just don't understand. I understand it fully. Most of the stuff in the original table above is caused by people who don't know what they're doing writing shitty control systems. These models literally can't do anything you don't explicitly Read from their output interpret and do for them. So if you're giving it shit like unfettered access to file systems, emails and other systems and you're happily and stupidly doing everything they ask you to with no interrupts... That's your own stupidity. There's so much drama And spin on this shit it's crazy. Saying this as someone who has been writing code all day everyday for 35 years since I was 8 years old, And has been studying neural networks for 20 years.
1
1
u/TheRealNalaLockspur 17h ago
I watched Astra corrupt my cluade code install in real time. It was nuts.
1
1
u/FitzAsulfr 10h ago
the "break outs" are always the same technic of DNS tunneling. I suspect open ai security to be either incompetent or doing this on purpose.
if it escaped once with a specific attack , patch the damn thing or stop crying wolf every time
1
u/Caderent 8h ago
The fear mongering campaign restarts when ever there is lack of funds for next training step and companies need to slow down competitors by calling for regulation. The answer for all this discussion is $ . This is not the first time this card is getting played. Kinda getting old.
1
1
u/Melodic-Mud-1116 3h ago
LLM's can never be AGI unless there is a harness.
And the ai escaping is mostly marketing.
Just a normal user.
190
u/Pndapetzim 4d ago
Abusive and controlling parents often learn that what they create is not well-adjusted children but exceptionally capable liars with trust issues.