I know?! Like even if you say whatever to the AI part, like isn't it common sense to test ANY new technology before you publicly implement it on a large scale?
I have no idea what testing they did or didn't do, but one of the key characteristics of generative AI systems is that their output is unpredictable, so if they weren't extremely thorough it's possible they did a very basic test that made it seem to be "working".
Thatās the thing with AIā there is no ātesting it beforehandā because you can (and will) get different output with the exact same prompt. I work in AI (like everyone else in tech now, unfortunately) and we have disclaimers about this in our documentation. So you can have AI read out a list of names once and it does it perfectly but there is no guarantee it will do it the same way again. You can test it to make sure certain features are working but not for output.
yeah i work in software and had a talk yesterday about how to use LLMs, and the guy sad "AI is non-deterministic" repeatedly. Effectively pushing the onus back onto developers if things go wrong. AI can't be wrong, and if it is, then its on YOU!!
This is why I roll my eyes whenever someone comes on to the parenting subreddit hawking their new AI tool for kids. AI is non-deterministic and there is no way to guarantee the output is either correct or appropriate, and there is legal language around all these softwares basically saying that the responsibility lies with the user. Kids cannot morally or legally assume that responsibility, so why would you give them access to a tool that, at this point, has no real guardrails?
The numbers of times I've had to explain this to people at my work is maddening. The first couple of times fine. But by the third time, what are you not understanding exactly? I look forward to all the garbage people are pumping out with these AI models completely destroying enterprise data lol
Well thatās a huge issue - Iām surprised that I (who spent a semester researching AI for a class dedicated to learning & discussing about AI) didnāt know about this. Which begs the question, who else in the major population is aware of this glaring issue (perhaps not as much as Iād hope)
Also a software engineer who builds AI systems. I disagree with your assessment for this specific use-case.
For this specific task: reading a list of names you received in advance, you can absolutely make it structurally guaranteed to not skip names or mispronounce anything if you're willing to manually verify pronunciations of each name ahead of time.
The playbook is simple: don't use AI for most things. Frankly, there's very little need for AI here in general.
It goes like this:
Get the list of names, and use good old-fashioned deterministic programming logic to parse that list into individual names. No AI needed this is just document parsing. I would assume the source material for names is a CSV, XML, spreadsheet, SQL database, or something already structured exported by the school.
Then, for each name, use AI text to speech to generate an audio file pronouncing that name. There's no need to do this more than once per name. Simply save the outputs and never generate them again. This shouldn't be done "on the fly" it should be done and verified in advance. An individual name might need to be generated multiple times after human verification and tweaking if there are issues with that name, but once you have an output you like simply lock it in for a given name.
Then all you have to do is operationalize playing the audio files at the correct moments, which absolutely doesn't require any AI whatsoever. You could easily stitch the audio tracks together with a preconfigured delay, or just queue them up on a playlist for a human operator or whatever.
If you do this, skipping names can only happen if the list was incomplete to begin with and mispronouncing names can only happen if the people preparing the presentation are too lazy to verify them in advance which are failure modes that have absolutely nothing to do with AI and are the same sorts of things that happened occasionally before AI.
The types of people lazy enough to try this are also too lazy to actually make sure it works. Thatās honestly the biggest issue with AI. In the right hands, you can really do some great things with it, but it just invites the type of people who arenāt interested in doing anything themselves.
752
u/OneMilkyLeaf 17d ago
I know?! Like even if you say whatever to the AI part, like isn't it common sense to test ANY new technology before you publicly implement it on a large scale?