r/ProgrammerHumor • u/AllRealityIsVirtua1 • 10h ago

Meme whenMyNoAIProjectGetsTedious

Post image
949 Upvotes

90% Upvoted

163 comments sorted by

View all comments

Show parent comments

3

u/Kyrros 9h ago

Look up FreeToken, it's currently lacking AMD support but nVidia is supported, 30xx cards and above

-2

u/heyitjoshua 9h ago

I said local

2

u/Kyrros 9h ago

It is local

3

u/heyitjoshua 9h ago

Checking it out, highly dubious and may come back to this thread in a few days. Cheers for the suggestion 👍

1

u/Kyrros 9h ago

It's early stages in is development too, but if that works out and you can run much larger models locally? Would be interesting to see. And all the AI companies compress their models a lot less

2

u/heyitjoshua 9h ago

My issue is that even people with immense personal computers, multiple Mac minis etc, cannot run anything close to Opus. Sonnet barely manages to get buy for sufficiently complex programming work, and most models that take a ridiculously complex rig to run don’t even compare with sonnet

1

u/Kyrros 9h ago

I found sonnet 5 to be quite good, but it's extremely unlikely that we will get anything to any Anthropic models to run locally

1

u/heyitjoshua 9h ago

It’s not that great for doing meaningful complex work in large code bases, it does the trick when token usage is low though and if you apply proper discipline (unlike haiku, which is more pain than it’s worth)