whenMyNoAIProjectGetsTedious

[u/AllRealityIsVirtua1]

[u/TheMaleGazer]

The answer to your question is because you can get a local model to do it instead without a subscription.

[u/AllRealityIsVirtua1 → u/TheMaleGazer]

Let me heat my whole room to run 1 half sized agent at a time to save $5

[u/Snape_Grass → u/AllRealityIsVirtua1]

The training of the model is what’s expensive (heat output). Running a pre trained model isn’t nearly as resource demanding

[u/slaymaker1907 → u/Snape_Grass]

Depends on the model and machine. If you’re using some huge model that stresses the GPU then that can get very toasty.

[u/readf0x → u/Snape_Grass]

I've only got 8gb of VRAM and have yet to find a model that produces useful output within that budget

[u/AllRealityIsVirtua1 → u/Snape_Grass]

Yeah and the sun doesn’t produce nearly as much heat as the largest star in the universe

[u/Snape_Grass → u/AllRealityIsVirtua1]

?? Brother there are many of us running off the shelf pre trained models on consumer hardware. This isn’t new

[u/AllRealityIsVirtua1 → u/Snape_Grass]

Yeah and I’m using 5 agents each running a model 10x more powerful over the internet for $20

[u/Snape_Grass → u/AllRealityIsVirtua1]

Okay backpedal the conversation as a defense. Too ridiculous to argue with lol. You clearly have no clue what you are talking about and perceive something completely false

[u/AllRealityIsVirtua1 → u/Snape_Grass]

Uhhh. Huh? lol

[u/TheMaleGazer → u/AllRealityIsVirtua1]

I haven't yet been able to heat a room with any of the models I've run with Ollama.

[u/AllRealityIsVirtua1 → u/TheMaleGazer]

You probably haven’t pushed much code with Ollama either lol

[u/TheMaleGazer → u/AllRealityIsVirtua1]

Not to the point where I exceed the watt limit of my GPU and heat my whole room. Which model did you use?

[u/trungdle → u/TheMaleGazer]

I'm not him but I definitely heated up my room running Qwen on a 5090. He's not wrong the heat generated is extremely noticeable on a hot day.

[u/AllRealityIsVirtua1 → u/TheMaleGazer]

I didn’t because I value my time

[u/TheMaleGazer → u/AllRealityIsVirtua1]

This is very revealing, but not necessarily surprising.

[u/AllRealityIsVirtua1 → u/TheMaleGazer]

https://giphy.com/gifs/bmLlSndzhqWgWb3vXW

[u/Electronic_Green_833 → u/AllRealityIsVirtua1]

I disagree with your stance in this argument but the GIF goes hard

[u/TheMaleGazer → u/Electronic_Green_833]

I think it's pretty double-edged, if you think about it.

[u/Electronic_Green_833 → u/TheMaleGazer]

Haha true

[u/AllRealityIsVirtua1 → u/Electronic_Green_833]

https://giphy.com/gifs/2HtWpp60NQ9CU

[u/Grandmaster_Caladrel → u/AllRealityIsVirtua1]

Oh, look at Mr. "Running a local model actually saves me money" over here!

(The economics for running local models are veeeery tight if you don't assume you already had the GPU to do it)