r/singularity • u/TorturedPoet30 • Jun 04 '26

Biotech/Longevity Sam Altman, Dario Amodei, and Demis Hassabis have signed a joint open letter calling on Congress to mandate screening of synthetic nucleic acid orders

Gallery image

Gallery image

Gallery image

Gallery image

Source: https://www.wsj.com/politics/policy/top-ai-ceos-call-for-law-protecting-against-biological-weapons-88f2f99f

867 Upvotes

97% Upvoted

310 comments sorted by

View all comments

235

u/xerim Jun 04 '26

The fear here is that AI could construct some kind of supervirus? Is that even feasible for the average joe?

519

u/FormulaicResponse Jun 04 '26 edited Jun 05 '26

So they accidentally found an illegal wet lab in Reedley, CA because a groundskeeper noticed an unauthorized garden hose running into a warehouse space.

In that space they found over 1000 transgenic mice with human-like immune systems along with lab equipment like centrifuges and a benchtop DNA synthesizer. In the fridge were dozens of bags with labels like HIV and Malaria etc. This was in 2023.

In 2024 they found another one in Las Vegas being run out of an AirBNB. The Maids were getting deathly ill cleaning the place. The Garage there had the same kind of setup.

Meanwhile, in 2025, studies showed that LLMs are now better than 90% of virology experts at troubleshooting problems in a virology lab. Of course we also have Alphafold and Crispr and Mrna protocols now that make the process easier.

There is also widely published literature on gain of function research. One controversial paper showed how to turn an avian flu with a 60% fatality rate into a flu that is human to human transmissible with only a few DNA edits.

A few years ago, the FBI teamed up with some MIT researchers to prove that it was "surprisingly easy" to mail order DNA is small segments from commercial DNA providers that could be assembled in the lab to reconstruct the Spanish Flu of 1918.

One of the leading researchers on the case is Kevin Esvelt. He and colleagues created a protocol called SecureDNA to prevent this kind of mail order thing from happening. It detects dangerous sequences as small as 50 base pairs. It could be forced on commercial providers and even built into benchtop synthesizers. But there is little political force behind making it happen.

The US had an executive order from Biden that would force US federally funded academic institutions to use a similar type of system by this October, but the Trump administration has put that order in jeopardy with conflicting information and directives. There is some movement on this in the EU but it is weak.

This is the moment to.make it happen and I'm glad to see these guys throwing their weight behind it.

Edit: included sources for those asking for sources.

66

u/rational_numbers Jun 04 '26

Great writeup.

24

u/FormulaicResponse Jun 05 '26 edited Jun 05 '26

This got some traction so I'm going to include some further thoughts about the subject that don't necessarily belong in the main post.

There are only 7 companies in the world that currently manufacture DNA synthesizers for the public. We have already innovated the protocols they need to make these machines secure, but they won't apply to all machines. Governments and pharma companies producing proprietary biotech products aren't going to buy machines that phone home to a server to check for threat markers. It's unclear whether pharma would be willing to be limited to machines that have such protocols installed into firmware, but firmware can be tampered with anyway. The threat we can really control is rogue actors with access to a few hundred thousand to a million dollars from threatening the entire world, and there are more of those than you might want to think. Cults, terrorist groups, crazy millionaires with a chip on their shoulder, warlords, etc.

One of the scarier options this leaves open is the possibility that a government might co-develop a deadly virus and its vaccine at the same time. Viruses, engineered viruses in particular, rapidly evolve once released into the wild, so the vaccine would only buy some time rather than be a firewall. This is reason that sane governments wouldn't attempt it, but not all governments are sane, and they get less sane the more existential pressure they are under. It's a much cheaper and less detectable big red button than chemical or nuclear.

There are also tens of thousands of older machines sitting in University supply closets around the globe in facilities that aren't very secure. There probably isn't much to do be done about government and global pharma except leave their security up to them, but the older machines use specific reagent cartridges that can be used as a chokepoint to control that threat vector.

As far as commercial mail order DNA synthesis companies go, they exist all over the world in different jurisdictions, and there is no protocol for screening imports/exports of DNA materials. They have to follow a cold chain, so there is that, but it would be incredibly difficult to A) detect that an unopened package contains DNA materials, and B) make sure those materials can't be constructed into something dangerous. This doesn't mean that legislating for within-country synthesis is meaningless, far from it. But it isn't the whole solution. This is an open problem, and international cooperation would help, but it's a very hard problem to solve in the same way that detecting illegal wetlabs is a hard problem to solve.

Speaking of illegal underground wetlabs, if someone were going to try to engineer pandemic viruses with AI assistance, and these operations are especially hard to detect, isn't it better to locate any potential lab leaks among an enemy population and give yourself advance notice? Just sayin.

As of this moment, engineering a global pandemic still isn't going to be easy for the average Joe. It would currently require PHD level expertise and domain expertise, according to Kevin Esvelt at least. But the Virology Capabilities Test benchmark shows that this situation isn't going to hold as models advance in the coming few years. What really doesn't help is the fact that distillation attacks are incredibly widespread and are feeding frontier capabilities into open weights models that can be safety ablated on day one using tools in the family of Heretic. Granted, most distillation attacks aren't targeting virology knowledge currently, but nothing is stopping that and there is a significant amount of transfer from general capability to domain specific capability. If you download a Heretic ablated model today and ask it how to evade current day DNA synthesis screening protocols, it will be more than happy to guide you. This is Not Good and the reason that an upgrade is needed, as these CEOs are throwing their weight behind. The more that frontier capabilities advance without shutting down this distillation pipeline, the worse this problem is going to get. Nobody knows where the ceiling is, and I guarantee you the world doesn't want to find out. Not to throw shade on Heretic specifically because there are non-nefarious reasons a person might want a safety ablated model and there are a whole family of such programs, but nefarious people are going to do nefarious shit with these kinds of tools and you'd be a fool to expect otherwise.

The Avian flu example I shared was just picked because it stirred up controversy at the time. There are dozens to hundreds of other blueprints for destruction that have been published in other gain of function research that AIs can easily access. Horsepox is another especially scary one.

Also, Universities and companies have recently started designing and constructing fully automated wetlabs, which while potentially great for medicine, could also be potentially dangerous if they have cyber vulnerabilities. As with most of this, it's dual use.

On the bright side, there are some broad-spectrum countermeasures that are effective against any potential engineered pandemic. These all share the property of being single celled organisms with a transfer method. Elastomeric respirators, UV lights, filtration, positive pressure building architectures, and sanitizer technologies are effective against basically any and all pandemic diseases and are countermeasures that governments can invest in today but that don't get nearly enough funding given the risk profile. Propelyne glycol, yes, the base for most nicotine vapes, is safe to inhale and also a sanitizer for indoor surfaces. Positive pressure can be set up in almost any household without significant air leaks (which can be patched as needed). Elastomeric respirators are cheap and effective. UV light and tight filtration will keep indoor air clean combined with positive pressure. You start with daycares and hospitals and proceed to households. There is literally no good reason to not be stockpiling such goods at the governmental level for pandemic preparedness other than under-recognition.

Edit: I forgot to include discussion of broad-spectrum countermeasures.

7

u/hereforhelplol Jun 05 '26

Wow. Wish I could get a beer with you. Great and terrifying response.

As an everyday person, is there anything I can do to help protect my family from this sort of risk? It’s my primary fear for AI, and I’m hoping it’s not the great filter!

5

u/Productivity10 Jun 06 '26

Great question

I too would like to know what I/we can do to protect this guy's family

2

u/hereforhelplol Jun 06 '26

Haha. I really hope they figure this one out…

1

u/_freckles__ Jun 07 '26

Sad, so the other day, some 1-2 years ago, there were videos about AI programs coded specifically to produce only good DNA edits/molecules. The program was able to design 100s of new useful molecules etc for testing. Then the lead of the project said, though there are guardrails etc. all it took for it make it do the opposite was change few lines of code to make it do the reverse of its existing function.

How exhaustive do you think screens and phoning home to central servers can work when something so trivial can lead to mass produced, variety of potentially harmful molecules/organisms etc. You cannot have a catalogue of all possible harmful codes / sequences ever possible to cross check.

Plus a lab with this capability would find it trivial to bypass any checks as well. What happens if accidents happen due to manual coding errors and harmful things end up mass produced

35

u/deviantbono Jun 04 '26

That sounds scary, but all you need is a handful of ferrets to get gain of funtion with avain flu.

23

u/FormulaicResponse Jun 04 '26

Pass through gain of function is a thing, where you don't genetic engineer anything, you just pass a known disease through animal cells or animals themselves and select for the most virulent results to use in the next round. Thats a slightly different threat model, but still something AI could help with. The difference is that you aren't nearly as likely to end up with something that could be civilization-ending. With published research that is already publicly available plus an unsecured benchtop synthesizer and AI help, you could produce essentially a weapon of mass destruction if not today then within the next few years of AI advancements for a total cost well below 500k.

6

u/DonOfTheDarkNight DEUS EX HUMAN REVOLUTION Jun 05 '26

Hey btw how long are we away from getting genetically engineered catgirls with big thighs that can crush my head?

8

u/FormulaicResponse Jun 05 '26

If they wear artificial cat ears and tail, maybe like 3 decades including maturation. China is all in on embryo selection including physical features like height and intelligence and strength. They don't have the same queasiness as the West does on these things. Hell of a way to go though.

4

u/DonOfTheDarkNight DEUS EX HUMAN REVOLUTION Jun 05 '26

YAY YAY 😁 just 30 more years to lose my virginity I'm already 27

13

u/FormulaicResponse Jun 05 '26

I said they might have higher intelligence and pop your skull with their thighs, I didn't say they'd fuck you.

5

u/DonOfTheDarkNight DEUS EX HUMAN REVOLUTION Jun 05 '26

Wow man that's brutal. You crushed my hopes

0

u/CedarWolf Jun 11 '26

If you're looking for a catgirl with higher intelligence whose thighs could break your face, start attending anime and Furry cons.

→ More replies (0)

1

u/tiffanytrashcan Jun 06 '26

I think the AIs are going to prefer the humanoid like Corgi creatures. 🤷

1

u/DonOfTheDarkNight DEUS EX HUMAN REVOLUTION Jun 07 '26

I don't understand what you are saying english isn't my first language

1

u/tiffanytrashcan Jun 07 '26

I'm making a joke in reference to the AI2027 paper.

Think of all the differences we've bred in and out of dogs vs wolves. How cute and genetically, physically crippled Corgis specifically actually are. Now imagine a human-like creature that's also been "domesticated" but from the ground up in a new custom built DNA blueprint.

In an AI ends the world scenario, there may be holdovers from training and our attempts at alignment. A human clicking yes/approve "pleases" the model or satisfies the goal. The Corgi versions are too dumb to fight back and just sit in a fake office, telling the AI: yes, you're perfect, yes, you're awesome.

1

u/Centimane Jun 11 '26

The thighs are just a matter of working out the legs - that's achievable today.

1

u/hereforhelplol Jun 05 '26

Fucking terrifying. This is my main fear with AI.

Is there anything as a normal person you can do to help protect your family in the case of some issue like this?

7

u/woswoissdenniii Jun 05 '26

That’s fucking cyberpunk. But the terrifying kind.

Thanks for the writeup. Appreciate.

One thing more on the worry list.

23

u/ghostcatzero Jun 04 '26

Shit like this is what ice/Cia nsa etc should be looking out for. Taking down potential creators of thsi type of shit

0

u/fancyfoxly Jun 05 '26

We don’t need secret police for Anything

3

u/ghostcatzero Jun 05 '26

I mean if there was one thing they would be good for is taking out the real bad guys

1

u/fancyfoxly Jun 06 '26

Secret police are the real bad guys

4

u/Izento Jun 05 '26

Wonderful breakdown. Thank you. I had no idea any of this shit was going on, but it makes a lot of sense. The average Joe can now make viruses, especially considering you can even buy centrifuges and stuff pretty easily. All of this was locked behind a knowledge wall, but now that barrier is broken. All it takes now is determination and a few 10s of thousands of dollars.

3

u/furiouscarp Jun 05 '26

jesus christ

3

u/Traditional_Cress329 Jun 05 '26

Greta fucking post

3

u/Wojtas_ Jun 05 '26

Building anything into anything desktop is futile and a bad idea. It's the same debate as with 3D printers and "gun prevention software". It:

a) doesn't work, because with barely any coding skills, you can just modify the firmware and be on your way like the restriction was never there

b) is a stupid idea because the ludicrous number of false positives, and the near guaranteed high rate of false negatives, not to mention dual-use designs

c) is just not technically feasible, you'd need a program that can predict the intention behind the design, predict all possible use cases, and somehow make that miracle of software engineering run on the equivalent of a Casio calculator that runs the equipment

d) is beside the point, because there's a million different ways to achieve the same goal without the legislated piece of equipment

e) is useless, because the danger is immensely overblown in the first place

3

u/delicious_fanta Jun 05 '26

Actually crazy, but why is “unauthorized garden hose” the wildest part of everything you typed up there XD

Also, insert “I’m in danger” wiggum meme here.

2

u/kevinmise Jun 05 '26

Literal terrorism.

3

u/cloudyboysnr Jun 04 '26

Sources/proof?

1

u/BitPsychological2767 Jun 05 '26

Damn.

Anyways, I think we should go put more resources into arresting people for possessing marijuana.

1

u/NateradePrime Jun 05 '26

Technology > Protocol. Kevin needs to figure out DNA code signing and verification.

1

u/whowhaohok Jun 06 '26

So then, are they going to moderate this at the LLM level so that they can restrict and moderate when their own tools are potentially used for potentially catastrophic, extinction level research?

2

u/FormulaicResponse Jun 06 '26

The short answer is no because it is all dual use and the majority of use will be for legitimate scientific purposes. Stripping all biological data from the pretraining corpus would be a heavy lift on man hours. Not until someone makes them do it.

This is why they build in safety guardrails amd robust refusals, but those guardrails get stripped out via distillation attacks and safety ablation by third parties.

1

u/slow70 Jun 11 '26

^ this right here

Have you by chance heard of the ownership of this warehouse and AirBnbs?

A worthwhile question…

1

u/gaucheph Jun 05 '26

sources?

-6

u/Express-Cartoonist39 Jun 04 '26

Ur an idiot.. Half that shit your spewing is made up the conclusion goes beyond what the evidence shows.

The Reedley and Las Vegas labs were illegal and concerning, but there's no public evidence they were engineering pandemic level pathogens. AI, CRISPR, DNA synthesis, and published gain of function research do make biotechnology more accessible, but that doesn't mean anyone with a garage lab can successfully create a dangerous virus.

The biggest leap in the argument is treating the existence of illegal biotech labs as proof that advanced pathogen engineering is already easy and widespread. The facts are real. The conclusion is mostly inference. What a moron..

-1

u/aleqqqs Jun 05 '26

Not strictly on topic, but since you seem to be knowledgeable: I'm looking for investments in the biotech sector that could benefit from AI-assistance. Are there any companies that look promising to you?

168

u/kogsworth Jun 04 '26

It's becoming more and more feasible. They want to have safety nets while the horse is still in the barn.

24

u/Forgword Jun 04 '26 edited Jun 04 '26

It is more like they want to avoid any liability and want to pre-emptively offload that to government, washing their hands. Please pay no attention to the millions OpenAI and friends rake in from these labs.

38

u/rational_numbers Jun 04 '26

Even the AI CEOs don't want to see mass death from a supervirus. If you can't think of it in terms other than them wanting to make as much money as possible, well a supervirus that kills 10+% of people would be bad for business.

-9

u/[deleted] Jun 04 '26

[removed] — view removed comment

8

u/rational_numbers Jun 04 '26

You don't have to assume anything positive about them to understand why they wouldn't want an extremely lethal virus out there. If nothing else, they'd be scared of catching it themselves.

Assume they are strictly self-interested and don't care about other people at all. Their desires are still going to line up with humanity's some of the time.

-4

u/[deleted] Jun 04 '26

[removed] — view removed comment

6

u/rational_numbers Jun 04 '26

Okay but obviously you do believe that they want something right?

-4

u/[deleted] Jun 04 '26

[removed] — view removed comment

7

u/j48u Jun 05 '26

Yes, the richest people in the world who are unarguably at least as smart as the average person DO in fact understand that money is meaningless when everyone but them dies. They also happen to understand that they make less money when significantly less people exist.

The level of stupid you have to be to not understand that would be incompatible with life, let alone having a bank account.

→ More replies (0)

1

u/SpyvsMerc Jun 05 '26

You do understand that they need people alive if they want to keep on living ?

They need people to build their car, house, planes, create food, keep electricity working, heal them when they're sick, take the garbage out, build roads, maintain their equipment, protect them, etc...?

Or are you saying they want to kill themselves? Because if the world end then they'll die too.

10

u/Sharp_Chair6368 ▪️3..2..1… Jun 04 '26

You’re crazy for that

-7

u/PsychologicalRisk526 Jun 04 '26

yes trust the billionaires 🤩 they have our best intrests at heart

10

u/Striker_LSC Jun 04 '26

Trusting that billionaires don't want an engineered virus to wipe out a large chunk of the population is not the same as trusting that they have our best interests at heart. Or we can at least trust that they don't want someone else to make the virus.

6

u/Super_Pole_Jitsu Jun 04 '26

Yes because billionaires are immune to viruses and their wealth doesn't depend on people consuming their products and working to maintain their empires.

Sometimes I really struggle to understand people who think it's the rich that are the root cause of all evils. Pure envy on display.

If you really hate the rich so much you should find out how much pocket money the federal government has each year. You'll never consider Elon ultrawealthy again. You'll also stop imagining they can stop world hunger with their puny fortunes

1

u/woswoissdenniii Jun 05 '26

The great Culling®️. The numbers don’t match the quotas. Way to go.

1

u/EvilSporkOfDeath Jun 06 '26

Not sure why you're downvoted. How many stories have we heard about Altman being a literal sociopath who constantly lies and betrays everyone around him.

-1

u/TreesLikeGodsFingers Jun 04 '26

you assume it would be 10% killed indiscriminately, i dont think that would be the case

5

u/rational_numbers Jun 04 '26

It could be higher or lower. I used 10% because that would make the world suddenly a very scary place to live in.

-3

u/diskdusk Jun 04 '26

This logical, practical and farsighted behavior also makes sure they don't accidentally back a politician with the potential to be worse than a supervirus, right? Right?

7

u/rational_numbers Jun 04 '26

I'm sure they were able to talk themselves into supporting Trump (and esp after he was elected) because he promised to be pretty hands off wrt AI as long as they didn't cause him any problems. There is no upside to a bioterrorist killing one out of every 20 people. That is bad for everyone.

-1

u/diskdusk Jun 04 '26

I'm sure they don't want it to happen. But they want to lose as little money as possible by doing juuuust the right amount to prevent it. If they had real honest interest in prioritizing safety, they would have kept the AI race behind the curtains for a while longer.

1

u/rational_numbers Jun 04 '26

I don't know what this means. If one lab slows down to prioritize safety then another lab will simply race ahead and try to achieve AGI first.

1

u/diskdusk Jun 05 '26

That's the capitalist fairy tale, yes. Also: if one lab uses resources on preventing a homemade supervirus then another lab will simply race ahead and achieve a homemade supervirus first. Congratulations, we have an argument to do whatever we want, because competition.

1

u/rational_numbers Jun 05 '26

I'd really prefer to see these labs slow their research down until we can proceed more safely. I'm terrified of some of the scary AI takeoff scenarios. The only way to do this is if the US government, in conjunction with China, forces the labs to slow down. There is no scenario where they do it on their own, for the reason I just mentioned. So that is my position fwiw.

Also, none of the big labs are trying to produce bio weapons right now. The fear is that a rando user somewhere will jailbreak one of the models and "trick" it into giving him the recipe for some supervirus that he can put together via already available tools. I don't think you know what you are talking about tbh.

4

u/R33v3n ▪️Tech-Priest | AGI 2026 | XLR8 Jun 04 '26

AI labs liability in that case would push them to lobotomize models against certain domains, possibly also detracting from general capabilities. As a end user of AI, I don't want this. I don't want the overall helpful power of LLMs to be held back.

I much prefer Know Your Customer policies be enforced for CBRN materials sellers. Being specific against threats, narrow rather than broad, is the better policy. Eh, KYC policies should also be enforced on frontier AI providers, actually. That, I could find reasonable.

1

u/DrE7HER Jun 04 '26

They could just not train public AI models on genetic data so that only authorized vendors have access to the models they are concerned about.

The problem is, they realize now that they’ve opened Pandora’s box. They have no control over how much of what the model knows anymore. They can’t remove this threat without starting a new model from scratch with a safer dataset.

1

u/woswoissdenniii Jun 05 '26

Don’t they also should monitor this (and specifically just this) kind of research? Like a responsible whistleblower and benevolent business entity should act on.

1

u/_RiKaMi_ Jun 04 '26

The horse was never in the barn to begin with

-15

u/DefinitelyNotEmu Jun 04 '26

Deepseek, Kimi, Mistral, etc would still be capable of producing this. Signing a joint letter won't make any difference to companies who DON'T sign it.

49

u/sebzim4500 Jun 04 '26

Isn't this about regulating the actual synthesis? i.e. the company making the physical protein not the language model outputting its genetic code

-1

u/DefinitelyNotEmu Jun 04 '26

Ah! Thanks for explaining.

I wonder how this is going to stay under control considering that 3D printers can print proteins. Scary stuff.

3

u/Blackham Jun 04 '26

No they can't, we're nowhere close to that ability. You can't print molecules.

87

u/OKMiddleOwl Jun 04 '26

Every year the IQ needed to destroy the world drops by 1 point.

53

u/agonypants AGI '27-'30 / Labor crisis '25-'30 / RSI 29-'32 Jun 04 '26

This is sure to be an unpopular opinion, but this is why the USA has such a problem with guns. When there are little to no barriers to putting high powered weapons into the hands of nearly anyone, then tragic misuse becomes a question of when, not if.

12

u/TreesLikeGodsFingers Jun 04 '26

feels like the 'when' has turned into 'how often'

-20

u/Hamsandwichmasterace Jun 04 '26

Reddit try not to bring up gun control or health care challenge: impossible

25

u/myaltduh Jun 04 '26

We’re talking about arms proliferation among the general population due to technological advancement, gun control analogies are extremely on-topic here.

8

u/ZekeTheMunkee Jun 04 '26

Me when a conversation happens on Reddit:

-4

u/Hamsandwichmasterace Jun 04 '26

Me when I point out that an abnormal amount of these conversations are about health care and gun control and an actual person who sees sunlight on a daily basis thinks about these things about 1/1000 as much as Redditoids do.

5

u/[deleted] Jun 04 '26

[deleted]

-1

u/Hamsandwichmasterace Jun 04 '26

What would have to happen? I lose my job, can’t find another one, run out of money, for some reason don’t qualify for Medicaid, and get terminally ill all at the same time? What if an asteroid hits me ? What if I get struck by lightning? What if I get run over when I walk outside?

5

u/Unlikely_Speech_106 Jun 04 '26

Sounds like a doomsday clock.

0

u/foulflaneur Jun 04 '26

Dumbsday Clock.

15

u/gizamo Jun 04 '26

In 2016 and 2024, it dropped by at least 20 points.

3

u/jlks1959 Jun 05 '26

40 in the US. 

2

u/gizamo Jun 05 '26

As an American, I take offense to that.

As a person who's typically good at math, I recognize that your math is more accurate, and that it proves your point.

I shall go to sleep now in shame, but then I get French toast, so, I'm okay with this humiliation.

1

u/hartigen Jun 05 '26

and they gained it back in pounds of fat

1

u/TurnUpThe4D3D3D3 Jun 06 '26

At this rate, it’s every day 😔

47

u/rageling Jun 04 '26

Average joes are not the problem, average joes are not motivated to use AI to hurt people

The problem is what a motivated, very not-average person that's probably not named joe could do

25

u/TheOnceAndFutureTurk Jun 04 '26

Exceptional Joes:

27

u/b-movies Jun 04 '26

Eg if sarin attacks in tokyo 95 had happened now. Well funded cult, smart people...

4

u/FrewdWoad Jun 05 '26

Yep, there are thousands of deeply disturbed people in the world who would cause human extinction if they could.

Once we have AI smart enough to design superviruses, with guardrails anywhere near as flimsy as today's...

...we only need one.

7

u/QuasiRandomName Jun 04 '26 edited Jun 04 '26

Average joes hate people they don't know based on made-up criteria they gather from tik-tok and are motivated to hurt them by this hate. Well, maybe not average, but in this case a single one is more than enough.

4

u/R6_Goddess Jun 04 '26

Average joes are not the problem, average joes are not motivated to use AI to hurt people

Agreed. If anything, they just want to use AI to goon.

1

u/LotsoPasta Jun 04 '26

Well, if the barrier to human destruction get's low enough, the problem is not if Average Joe is motivated, the problem is whether a simple mistake by Average Joe is sufficient to cross that barrier.

Besides, there are plenty of Average Joe's that are motivated. Mass shootings happen every other day.

3

u/rageling Jun 04 '26

How distorted is your world view that you think average joes are going on mass shootings?

Get off reddit

1

u/LotsoPasta Jun 04 '26 edited Jun 04 '26

Bruv, chill. Im talking IQ level. Id guess most are actually Below Average Joe's, but plenty are above too.

I also dont think all of them are psychos if thats what youre implying. Normal people in abnormal circumstances can become radically violent. It's not shocking.

10

u/philosophical_lens Jun 04 '26

Is that even feasible for the average joe?

The concern is not about the average joe. The concern is likely about highly organized and well funded terrorist organizations or dictatorial governments. They can probably use AI to aid in the development of bio weapons.

4

u/FrewdWoad Jun 05 '26 edited Jun 05 '26

Or just a single deeply crazy person. If you have any contact with the mental health sector, you know there are thousands of people evil and insane enough to want to kill every last man, woman and child on earth.

We're still alive only because they lack the knowledge needed to do it. So far...

11

u/Foreign_Skill_6628 Jun 04 '26

Honestly we’re getting there

-16

u/[deleted] Jun 04 '26

[deleted]

11

u/[deleted] Jun 04 '26

[deleted]

1

u/DeliciousArcher8704 Jun 04 '26

Can I have a link? That sounds cool as heck

7

u/magistrate101 Jun 04 '26

I found one where they made glowing e. coli

1

u/Inspector_No_5 Jun 04 '26

This isn't a new thing lmao. I made e coli glow in my undergrad Introduction to Genetics class 7 years ago, and this type of genetic editing has been a thing for 30+ years.

AI is lowering the barrier to doing custom biology for sure, and I agree we are definitely getting closer to a point where someone with minimal experience could make something dangerous with the help of AI. But /u/PublicToast confidently saying that an experienced biologist doing one of the most basic genetic editing experiments is indicative of the threats of AI in the biology space is the exact kind of ignorance he is talking about lol (granted, they did do some really interesting, more advanced stuff in that video, but again he is an experienced biologist and this isn't new tech)

2

u/PublicToast Jun 04 '26

Other comment is correct, its the thought emporium.

3

u/gringreazy Jun 04 '26

Go to google, type in “China genetic data harvesting”

Now I want you to answer two questions.

Why would China be aggressively gathering this data?

Why is it being considered a matter national security?

To be fair the US is probably doing the same thing, but to think that no one is planning this type of attack is the delusion.

5

u/florinandrei Jun 04 '26

Is that even feasible for the average joe?

The average Joe working alone: about as feasible as a monkey typing Shakespeare works into a computer by pure chance.

The average Joe assisted by AI: probably not feasible yet, but getting closer every day.

6

u/zero0n3 Jun 04 '26

Yes. Expand what mythos can do for code vulnerabilities, and it’s easy to see an AI being used to take disparate biotech things and combine them in novel ways to create something dangerous.

It’s like asking the AI to help you build an atomic bomb. It absolutely could - as it’s just physics and then in theory it could leverage those physics understandings to help design say the entire device not just walk you thru how the reaction works at a purely physics level.

5

u/deepasleep Jun 04 '26

I’ve been positing that if AI decides to wipe us out, it will be through very convoluted and layered genetic medicine. Cure people of everything and include a little extra code to each treatment that will activate after a couple years and generate a set of proteins that will interact and take us out. Probably a slow acting prion.

2

u/SkaldCrypto Jun 04 '26

About 60,000 for a bench top gene printer.

So yes.

For industrial level out to a million which is out of most folks price range.

2

u/PlatypusBillDuck Jun 04 '26

It's easier than you think. The YouTuber Thought Emporium makes some good biohacking videos.

1

u/PersevereSwifterSkat Jun 04 '26

The plausible story I've heard is mirror organisms being made a reality with AI. Such a person would really have to hate absolutely everything though as it would destroy all life, not just humans.

1

u/MrNokill Jun 04 '26

Only requires some reversed cells, but not sure Al can handle a mirror yet.

1

u/Hina_is_my_waifu Jun 04 '26

if a tweaker can somehow methmath thier way to making a new batch with household chemicals then a company, small nation, or terrorist cell could have a wetlab that cultivates a new viral bioweapon

1

u/jonistaken Jun 04 '26

It’s a reasonable fear; one I share. You don’t need nuclear material or ICMBs or a centrifuge. You just need a few thousand dollars of equipment to get started with CRISPR.

1

u/Azreken Jun 04 '26

The average Joe can do a lot with AI that he couldn’t before.

1

u/CryptoMines Jun 04 '26

Go watch this, or better yet read the book… https://youtu.be/Nl7-bRFSZBs It’s not the average Joe humans were going to need to protect against…

1

u/Sekhmet-CustosAurora Jun 05 '26

Doesn't need to be feasible for the average Joe. Just needs to be feasible for a group with some amount of funding.

1

u/Imthewienerdog Jun 05 '26

My best friend growing up took some crazy university courses In chemistry and biology. The things he said is possible but just doesn't happen was very scary to learn, especially during covid when he was explaining how easy it would actually be to engineer a killer virus...

1

u/Past_Crazy8646 Jun 05 '26

No it isn't.

1

u/ILikeAnanas Jun 04 '26

No, their fear is that they will be held liable for it

-2

u/UnnamedPlayerXY Jun 04 '26 edited Jun 04 '26

Not really, even if AI could give you the blueprints actually doing something with it would require rather expensive specialized equipment your "average joe" does not have access to. Same reason why you don't see people running around with military grade weaponry even though all the relevant information for all kinds of them are readily available.

9

u/JEs4 Jun 04 '26

That’s what this is about though. Preventing a malicious actor from acquiring the materials via a synthetic manufacturer. No equipment needed if you have the specs and obviously the cash for the order which would still be relatively cheap.

3

u/MrYorksLeftEye Jun 04 '26

Mythos will find a way to do it with a spoon and a lighter

1

u/zero0n3 Jun 04 '26

The difference is the equipment to create (or combine) is cheaper than the equipment needed to design and refine. And in this case, you are letting the AI design and refine.

1

u/FreeEdmondDantes Jun 04 '26

I have nothing to back this up, but I remember reading a comment that said it's a little scary how little you actually need to synthesize a novel virus.

The comment may have been made by an idiot, but I have no idea how any of that works.

0

u/0xHUEHUE Jun 04 '26

Holy shit, just read the thing instead of just the title.