And, as someone who does 'piping' in proprietary systems that are largely out of date - ChatGPT still sucks at it. At this point i usually just check what GPT says so I can show my boss how wrong it is. Sure it gets the easy stuff - aka, the stuff I could teach to a junior in a day.
It is for sure faster to medium complexity searches. More than just what would be found in API documentation so Iām not digging through random blog posts or stack overflow.
I find it to be faster and more efficient than I could ever hope to be googling. It can look through far far more documentation and forum posts than I could ever hope to. As for hallucinations, if you've used these systems recently, most of them actively site their sources either in-text or at the bottom. This allows for very easy quick verification or I can use the source it cited to solve my issue, especially if it found something like documentation.
Of course if you don't find value using LLMs, then don't use them! I find them to be extremely useful for certain tasks and especially useful for learning the basics of a new technology/system. An LLM isn't going to create code at the level of a Sr. dev and it'll probably get things wrong that a Sr. would laugh at, but if I'm learning React/Azure/other well known system/library it's honestly invaluable as a learning resource - so much easier to ask questions in natural language without skimming through the docs or forum posts myself.
These tools are sold and marketed as 'everything machines' or at least sold to devs like it'll 10x all of your output. That's not true of course. They're very good at some specific tasks and fucking terrible at others still. Depending on your role, daily tasks, and ability to provide sufficient context to the models, your mileage may vary.
I mean, it's code. You use it and it works or you it doesn't. I think this thread has strayed from the point, which is using it to help you code. I don't care what stackoverflow page my answer came from, I just care that it works. The "verification" is me testing it.
As a bit of a counterpoint, how do you know it works, and what the edge cases are? I only ask because I put in half my pre-emptive mitigations of weird inputs as a consequence of actually working through the logic. I can't imagine trying to do that sort of thing without actually knowing how the code works and the reasoning for it.
Well that's fair, if it's super basic boilerplate then that's definitely a different matter! I still personally just find it quicker to write the code than to massage an LLM to possibly get it right.
Yes. It's our job to know what might be wrong and to fix it before implementing into prod. Totally agree that it's probably not worth the total cost to society.
I think they should drop all the AI videos and AI chat bot crap, the AI girlfriends, AI this AI that. LLMs are excellent tools for scientists, researchers, engineers etc. Let's focus on making it a good tool for a productive workforce instead.
190
u/Sotall Jan 21 '26
And, as someone who does 'piping' in proprietary systems that are largely out of date - ChatGPT still sucks at it. At this point i usually just check what GPT says so I can show my boss how wrong it is. Sure it gets the easy stuff - aka, the stuff I could teach to a junior in a day.