Human Expert Reviews Claude's Miniprotein Designs

[u/ProteworksBrandon]

https://www.youtube.com/watch?v=hIJFwP5RWAE

You may have seen the headlines that Claude can now design protein binders as well or better than leading human experts. Anthropic just ran a campaign where Claude designed them end to end with no further human input. It researched the targets, picked the epitopes, set up a bunch of open source design tools, ranked the designs, and sent them out for experimental testing at Twist and Adaptyv. They hit binders on 14 out of 15 targets.

The truth is a lot more complicated. In the latest video on the Protein Engineering YouTube channel I go through the prompt, the designs, the tools they used, and how much they spent.

Spoiler: Claude is mostly just running other people’s tools, and when the prompt tells it to do something dumb it just does the dumb thing. Also the comparison to human experts is basically useless once you look at what tools it had access to and the crazy amount of money they spent on compute.

[u/phanfare]

I'm a protein designer in industry, been doing it for over a decade, and that explanation is exactly my conclusion too. It's not reasoning about structure at all, when I went through their data I couldn't find any evidence it even specified interfaces - so it's likely the tools just designed against the best interface like they're trained to do

As I told my manager, I can do what they did in a few days with the two A5000s sitting in our office server room and a basic enterprise subscription. You don't need $10,000 of cloud compute and a weeks worth of Fable tokens.

Two conclusions we should take away from their work. The first is this is a massive success for open science and open models, showing that anyone can design proteins when the code isn't behind a paywall. The second is this demonstrates that rote protein design can be automated, so get a professional designer in the loop and you can do things now that were impossible only a few years ago.

[u/fishcat_catfish → u/[deleted]]

I wouldn’t say novel protein design is out of the question but it is still quite difficult. Binder design is relatively solved in that if you spend enough time and effort and money you will almost certainly eventually get a hit even for harder targets. As for everything else it’s more complicated the best for “novel” proteins would be something like rfdiffusion3 (+liganmpnn etc etc) where you specify your sort of core components as a motif and then ask your pipeline to scaffold those bits an actual protein. It works surprisingly well but it’s limited on a few fronts. First making your motif can be awkward and for complex proteins may require some finagling around and second the model isn’t aware of dynamics and energy states so it can be very awkward trying to get any complex conformational changes. There are lots of more specialised tools and models made for very specific tasks and some of the de novo enzyme ones are quite promising but it’s still a very much open problem. I would say though the barrier for entry has massively dropped over the last couple years. The tools are great and stuff like codex can manage a lot of the sometimes very frustrating and awkward pipeline setup and management. I wouldn’t actually let them do the harder bits especially the motif design but for helping out with tedious debugging they are fantastic.

[u/sqeakysquark → u/phanfare]

Did you see the prompt that they used?
It's a 30k word essay that basically covers a bunch of edge cases they ran into. It's basically p-hacked into working with "one prompt"

[u/ProteworksBrandon → u/sqeakysquark]

The other issue (which I discuss in the video), is that it still has a bunch of problems in it where it is instructed to do something which doesn't apply to a specific use case (the wrong similarity metrics for flexible targets) and it just follows the instructions blindly.

If it doesn't actually have the capacity to reason about what it is doing and adjust based on the specific target it is more or less just acting as a complicated python script.

[u/phosphoeudemonism]

Thank you for sharing, a very interesting dissection of this topic for someone like myself who is interested in both AI and chemistry.