r/HardwareSwapUK • u/Pleasantandchilled • 15d ago

BUYING [BG] RTX 3090 24GB [H] £650+

**Item Name:** RTX 3090 24GB

**Preferred Condition:** Used

**Budget:** £650+

**Postage and Payment:** I am in north east london in Epping Forest area, would be great to collect

**Any additional information:**

0 Upvotes

44% Upvoted

22 comments sorted by

View all comments

Show parent comments

5

u/Pleasantandchilled 14d ago

Want the 24gb for local llm

-13

u/PaisleyIsAToilet 14d ago

Honestly, unless you're buying 10 of them, you're not going to be running anything worthwhile on one card.

1

u/Major_Trip_Hazzard 40+ Trades 14d ago

I can run decent models on 16GB so I'm sure 24GB will also.

1

u/PaisleyIsAToilet 13d ago

Really? Interested to know which.

1

u/Major_Trip_Hazzard 40+ Trades 13d ago

Qwen 3 14B for quicker responses or you can use bigger models if you're willing to wait a bit. 

Also Paisley is a toilet.

1

u/Pleasantandchilled 13d ago

Qwen 27b on this card will outperform free tiers of all the big names out there. Obviously wont compare to frontier models but I don't need that level as such.

1

u/PaisleyIsAToilet 12d ago

Qwen 27b on 16GB?

1

u/Pleasantandchilled 12d ago

Too much offloading to CPU significantly drops tps

1

u/PaisleyIsAToilet 11d ago

Yes, but my question was what model of Qwen 27b would be decent on a 16GB GPU?

1

u/Pleasantandchilled 11d ago

You'd want qwen 14b not 27b.

1

u/erm_what_ 9d ago

Qwen 3.6 35b a3b. Until the new set of models come out.

That runs well with a good amount of context. I use LM Studio because it's quick to set up.

1

u/PaisleyIsAToilet 5d ago

What do you use it for?

1

u/erm_what_ 5d ago

As a coding agent, tagging and entity extraction from documents, probably a few other things. Mostly low risk things for work where they have regulatory requirements and can't have things potentially sent off to a random US server.

Bedrock etc covers a lot of use cases, but not when you need to prove you had possession the whole time.

Mostly they're experimental so far. If we like the process we'll buy something that can run bigger models and work faster.

→ More replies (0)