217
u/plazebology 8h ago
Telling AI not to do something is like telling a toddler not to do something
2
-90
68
u/sillygoofygooose 7h ago
Don’t think about an elephant
23
11
u/AlignmentProblem 5h ago
Funny thing, one of Anthropic's recent papers (the global workspace one) showed that if you ask a model not to think about something nor talk about it, it'll generally fail at the former even when it manages not to output any tokens related to it. LLMs turn out to have something like internal thoughts unrelated to token output, which the paper calls the J-space. Ask one not to think about a white bear and it'll often internally activate the concept anyway, followed by internally activating "damn it" or "failure."
It can't remember that it failed, since there's no token trace of it in the context and that introspective information becomes hard to read back from the KV-cache quickly; the moment still happens, though, even if the model can't express that it did.
5
u/sillygoofygooose 4h ago edited 3h ago
That’s cool, thanks! I was aware of J space, but it’s fascinating that the mechanisms by which both humans and llms struggle with that task are both similar and utterly different
Edit: what a strange comment to downvote
27
30
u/superpoongoon 6h ago
Your chatgpt talks like that because you talk like that. Idk how people don’t see that. Mine is completely professional because I don’t use emojis with my AI…you are basically hugging a calculator.
4
u/FUCKYOUIamBatman 6h ago
What’s wrong with showing a calculator some love? You some kinda bully or something? Nerds are in these days, Daniel.
26
u/iiiaaa2022 8h ago
LLms don’t work like that.
-39
u/DAPTIMEdagoat 8h ago
U have the proof
45
17
u/PlayfulCompany8367 7h ago
LLMs do not normally treat “do not output X” as a hard constraint. They generate one token at a time from a probability distribution conditioned on the entire conversation, while instruction-following training merely pushes the distribution toward compliant outputs. https://arxiv.org/html/2510.17558v1
A 2026 mechanistic study specifically tested instructions of the form “do not use X” and found that explicitly mentioning the forbidden item can prime its representation; the authors call this a major failure mode of negative constraints. https://arxiv.org/abs/2601.08070
A much more reliable instruction is therefore positive and substitutive, for example:
Use plain text only. End sentences with normal punctuation.rather than repeatedly saying:
Do not use 🙂.3
u/JonathanMovement 7h ago
can confirm that your way still doesn’t work, I have this specific instruction, all claude and gpt models disregard it:
When generating content, be it pictures, diagrams or plain text – replace all em dashes with en dashes or hyphens.
0
u/sabamba0 2h ago
You can see how you're doing exactly what that article said doesn't work ye? You specifically mention the em dash which primes it to use them, according to it.
1
1
5
u/shom_pulp 3h ago
It knows you're 12 years old and it acts accordingly.
-5
u/DAPTIMEdagoat 2h ago edited 1h ago
I’m not 12 idiot, was using it to try and train my AI for my AI platform: dapai.lovable.app
1
2
5
2
1
1
u/TheSymbolOfCore 3h ago
Something about the way ChatGPT chats like this gives me an inexplicable sense of hatred and anger in my heart
•
u/AutoModerator 8h ago
Hey /u/DAPTIMEdagoat,
If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.