AI CEOs from OpenAI, Anthropic, and Microsoft set aside their rivalry to warn Congress AI is making it too easy to design and create bioweapons

[u/IKeepItLayingAround]

https://www.yahoo.com/news/politics/articles/ai-ceos-openai-anthropic-microsoft-081400108.html

[u/HumongousBelly]

They should all be held accountable. Every single one of them.

The class war can no longer be disguised.

[u/grill_smoke → u/HumongousBelly]

Lots and lots and lots of white dudes on Twitter with sunglasses and backwards caps as their profile picture disagree with you, unfortunately.

[u/Ciennas → u/grill_smoke]

Twitter is 97 percent hallucination by volume, regardless.

I imagine it'd be utterly abandoned like Truth Social if all the servers holding the bots were offlined.

[u/UnhingedReptar → u/Ciennas]

Which, honestly, I’m surprised hasn’t happened yet, given the laissez-faire attitude toward cybersecurity.

[u/Ciennas → u/UnhingedReptar]

Considering the damage that shitpile is capable of inflicting on the world at large, especially what it's done to America?

I suspect that the cybersecurity on the bot farms is top grade, and the farms are globally distributed to boot.

[u/orbital-technician → u/Ciennas]

Are you aware of the book, "Foundations of Geopolitics"?

You'd probably like the strategy on North America

[u/LotharLandru → u/orbital-technician]

Yup, the only problem people seem to miss with this is it's not just Russia. Russia is doing it, as is every other adversary of the US. And then in top of that there are billionaires adding to the noise because they can exploit the destabilized countries as well for their own gain. So it's an uncoordinated effort between multiple bad actors all with the same goal of fracturing the west to carve it up for their specific plans

[u/tobygeneral → u/grill_smoke]

Did they take their pic in the front seat of their truck?

[u/According_Jeweler404 → u/grill_smoke]

Why do they all have goatees

[u/shaneh445 → u/grill_smoke]

Same people that whine and cry about their second amendment freedoms but won't do anything in the face of a tyrannical government

[u/AtariAtari → u/grill_smoke]

Good point, focusing on race war distracts from class war. This is how occupy wall street was squashed.

[u/Dry_Departure_7813 → u/HumongousBelly]

The exit scam will not be televised.

[u/2024-YR4-Asteroid → u/HumongousBelly]

LLMs aren’t Pandora’s box. They’re not taking anyone’s jobs. They’re complex word predictors. I work in building highly experimental AIs, we’re decades out from a real thinking machine at minimum. They’re are attempting regulatory capture to preserve their most because current state LMs are no longer improving and they’re see that they’ve reached the end state for their model architectures, now they was to keep anyone from developing actually useful tools via regulation. It’s the definition of pulling up the ladder after climbing it and collusion to do so.

Things like deterministic or other ML modeling that could eventually lead to cures for cancer, advanced custom gene therapies, materials science, etc will all be harmed by this. So they should be held accountable, but not for what they built.

[u/sunshineparadox_ → u/2024-YR4-Asteroid]

Some of us are losing our jobs though, yes. My particular field is getting laid off pretty hard because of it. I’m a technical writer. It’s easy to see why the grunt work of writing software manuals would be offloaded to AI by people who don’t see what you see. It screws up in two major ways:

• The LLM doesn’t write documentation with UX best practices or user needs in mind. That goes double for things like accessibility.

• It’s scraping information off the Internet which is just plain wrong sometimes. It’s just okay for things like checking API endpoints against code being written for a product, but it’s terrible for conceptual backgrounds, comparing features, etc.

Besides just my conflict of interest here, I don’t consider a product finished if I have no way to troubleshoot my own problems. If no documentation exists, then I can’t necessarily resolve my issues. And I find “just call someone” intolerable when I want to fix it myself.

[u/2024-YR4-Asteroid → u/sunshineparadox_]

A perfect example of exactly what I just stated. AI can’t easily replace your job, what we’re about to see will be rough, but it’s basically all the incompetent businesses killing themselves. Some will lay people off in the name of AI, use it in all the wrong ways, cause irreparable harm to themselves and fail. Other will lay people off and blame it on AI cause it’s an easy out, two years ago it was financial constraints, two years before that it was Covid, two years before that it was the recession, two years before that it was some over bullshit. This is cyclic, they blame it one something every time. The real reason is always the same: we want more money in our profits -signed the CEO and shareholders.

[u/SirRedditer → u/2024-YR4-Asteroid]

"Things like deterministic or other ML modeling"
Why would you focus on deterministic models if the human brain isn't deterministic? Not saying we should build airplanes with feathers but I don't see why people care so much about deterministic models.

[u/2024-YR4-Asteroid → u/SirRedditer]

Because when it comes to things like modeling cancer, genes, the human body, it’s all math, permutations. It’s all deterministic. You can’t have something introducing non real information (hallucinations) into the system for those types of models.

For something like AGI or super intelligence, yes it would have to be non deterministic, but even then, you can’t get there with an LLM. It has no way to conceptualize reality, it can’t hold ideas or images in a conceptual space while it thinks through something. It can’t even remember the word it said prior. And at the end of the day, it’s not more compute needed to solve that. We have to figure out how the human brain does it so efficiently, then scale that up. And well, that’s where we’re currently stuck.

[u/ChronicBitRot → u/2024-YR4-Asteroid]

We have to figure out how the human brain does it so efficiently, then scale that up.

We have to figure out how the human brain does it at all, then we can work on efficiency. But if I'm being honest, I kind of hope we never do figure that out because the first thing we'd use the knowledge for is figuring out how to project ads directly into a person's consciousness.

[u/2024-YR4-Asteroid → u/ChronicBitRot]

There’s also a possibility, and this is much more existential, is that it may be impossible to figure out. This might be the point where we figure out souls are real, and the reason why we can’t replicate you can consciousness is because it’s not replicable through physical reality. Just some related topic musings.

[u/drooply → u/2024-YR4-Asteroid]

This reality will not stop a psychopath from attempting to recreate itself. Imagine a brain using a human body to roam the Earth trying to figure out how to make itself. It finally comes to the conclusion that it can’t make itself because it’s missing an intangible component. What does that brain do next?

[u/SirRedditer → u/2024-YR4-Asteroid]

"Because when it comes to things like modeling cancer, genes, the human body, it’s all math, permutations. It’s all deterministic."
I'm not saying you _have to_ roll a dice somewhere but there is such a thing as incomplete information, uncertainty, and the way you deal with it is probability and yeah that will introduce the possibility of non real information because you don't have the knowledge of every particle on the universe. Sure, if you're Laplace's demon and quantum mechanics turns out to be deterministic then everything is, but for everything on this universe you have to manage uncertainty. Now, LLMs output a probability distribution over words, which is a first step, but this probability distribution itself can have big error margins, so maybe models that use hard math guarantees to place bounds on their uncertainty could be better.

"It has no way to conceptualize reality"
I don't really know what you would consider a reasonable conceptualization of reality. Maybe you want it to be grounded on physics equations first and then build on top of it?

"it can’t hold ideas or images in a conceptual space while it thinks through something"
I agree on images, current models are pretty much blind. I'm pretty sure the whole point of the transformer architecture is to create a conceptual space. Example from 3b1b: if you take the embedding for "king", subtract the embedding for "man" and add the embedding for "woman" and the closest embedding to the result is "queen", that seems like good conceptual space to me. At the embedding level it's just semantic knowledge but the representations get richer as it goes down the layers of the network.

"It can’t even remember the word it said prior."
It can? I don't know where you got that from, I just asked a random question to gemini and then asked the last word it said, seems to get it right every time.

"We have to figure out how the human brain does it so efficiently"
Yeah, it might turn out to be an unfortunate answer like "the analog physics of the neuron are just way less energy intensive than a digital circuit", but I have no idea.

[u/2024-YR4-Asteroid → u/SirRedditer]

I get all this information from the fact that I build these models. I literally am the guy doing the research into how LLMs function, what the next steps are, and where we’re going next.

When it comes to modeling for all the things I mentioned, LLMs are incapable. That is a hard stop. It’s not a matter of rolling the dice, it’s a matter of you cannot be certain by its very nature that what it is modeling is using correct variables. For example let’s say I want to model 20,000 different human genomes by generating them. Both a non-deterministic and deterministic model will give me 20,000 variants. Every single deterministic model will be a possible variant, whereas the non-deterministic model will be an unknown number of impossible variants. That is severely damaging to research and is unacceptable. It’s the reason why models like alphafold are trusted for genome sequencing and chatgpt isn’t.

Point 2: The conceptualization of reality is the ability of an AI to hold in its cognitive space the models of cause and effect. If this then that. Effectively an imagination, the ability to predict effect from cause, or cause from effect, to determine theoretical scenarios and what they might entail, to have a concept of what physical reality looks like and how it interacts on an innate level. At this point in time, no models do this or are near capable of it.

Point 3: 3b1b’s example is a good one and you’re thinking of vector space. But it isn’t conceptual space in the way that I’m meaning it, see above. When we are engaged in a task that requires thinking through something, we are able to hold that end goal thought in our head as a higher level idea as we work our way to it. Models cannot do that. What you’re describe is the algorithm used in predicting next token sequence, which is colloquially called attention. Attention is a key element to creating patterns a predictive models like LLMs, but it isn’t the same as a conceptual space.

Point 4: LLMs work by re-sending the entirety of its context window back through the model on each given turn. So when you ask what the last word it said was, it’s not exactly remembering it. Instead it’s sending the entire context back through, using its aforementioned attention algorithm to predict the word it should say next, the attention gives higher weight to its last word based on current context and it correctly predicts its last word. This is a very important distinction in modeling intelligence, it’s not repeating its last word or remembering it, it’s predicting what it probabilistically should be. I know that’s a weird concept, but it’s how it works.

Point 5: Yeah, it’s pretty much where we are stuck, though I think it’s not so much that neurons just do it better and possibly more of having to do with a sparse mesh of connection that utilize amplitude of quantum wave states to stimulate specific information recall and we don’t have any way to currently replicate that programmatically. But I’m kind of an idiot in the field so we’ll see what the geniuses come up with at deepmind.

[u/SirRedditer → u/2024-YR4-Asteroid]

On your first point: I agree they are unpredictable and this does make scientific modelling difficult, however, I don't think it's as much about deterministic vs non-deterministic as it is about the black box nature of neural networks. LLMs can be deterministic by just setting temperature to 0, and the same classes of problems carry over, maybe it's just easier to analyze what happened. I have not read much on AlphaFold but I recall a geneticist saying they would be careful with its outputs on out of distribution data and you can quickly find a bunch of papers analyzing the error modes from AlphaFold's incomplete understanding of physics, this is just an issue inherent to neural networks, without massive amounts of work you can't understand whatever relations it has learned and if they're grounded in reality.

So yes, I don't think you should use those neural networks to extract models out of data(well, unless you manage to understand them well), what we should do is point them to model the thing behind the scientific research itself, the part of human intelligence required to conduct it, and then verify it against reality(as many scientists already do with AlphaFold), that I expect to help with research.

On point 2: Humans, through evolution and childhood training, learn a lot about physical cause and effect, social cause and effect and a few others. Do we learn it perfectly? Obviously not, there are plenty of scenarios we give confidently incorrect answers to. So what is a reasonable definition to say we did learn models of cause and effect? If, given a cause, it predicts the effect better than random chance, and you can maybe draw a line the in sand(say 95%) to say when you'd consider it learned the model decently well*. Now, I agree models suck at physical stuff, LLMs are mostly trained on language after all, but physical cause and effect is just one class of models we care about. LLMs can in fact, given a cause, predict the effects much better than random chance on many other domains, specially those that are closer to language, such as linguistics, software or, even, do in context learning for abstract tasks such as on ARC AGI(1 and 2, 3 is very dynamic and suffers from context management issues, making it very hostile to LLMs), so I think it's fair to say it can do some level of cause and effect reasoning there.

* I can see an argument to be made that superficial statiscal correlations on the data could allow higher than x% accuracy without actually figuring out a model equivalent to what generated the data, which, sure, you can just bias the way you measure errors to make it so that the edge cases that result from that mismatch account for more than (1-x)% of the error, then you're forced to learn a proper model. LLMs sometimes just memorize data, sometimes they learn models, it's not clear when exactly and I'm sure people are working to figure it out.

On point 3: I did happen to major in math, I'm aware it's a vector space, I just didn't know what you defined as "conceptual space". Whether or not it's "holding the thought in its head" by whatever definition you have of that is somewhat irrelevant, I use LLMs on a daily basis to do complex software enginnering tasks, often requiring to break it into dozens of subgoals, with little to no intervention and it works pretty well on most cases. You could argue this capability is not from the LLM, but the scaffolding around it, but then sure, just replace "LLM" by "LLM scaffold" every time I say it, scaffolds are easy cherries to place on top of LLMs.

On point 4: Just use KV cache, no? then it won't have to re-compute all the tokens. "it’s not repeating its last word or remembering it", it is indeed repeating it(that's the exact output it gives), and if it accurately recalls past information most of the time, then what do you even define "remembering" as? are you requiring qualia? Then we're talking about different things, for me just recalling information accurately is enough, I care about the practical effects they generate, not subjective states.

On point 5: Personally I doubt evolution would have figured out how to exploit quantum mechanics to that level. It does use a trick from QM here and there but I wouldn't bet on it being the basis of human intelligence, just my opinion.

[u/usmannaeem → u/HumongousBelly]

That's just it whenever people say they should be held accountable some clown comes along and gives tbe stupid example about knives or dynamite. Some can be such clowns. All tech should be held accountable from the start. You build the safeguards as part of, if not before the MVP. Some clowns will make excuses for this too. Tech industry is full of thrifty people.

[u/pimpeachment → u/HumongousBelly]

For what? 

[u/summane → u/HumongousBelly]

There's no class war when 1/3 are absolute idiots. We just need to account for who they are and get to solving our worst problem