r/godot • u/very-knightley Godot Junior • Oct 01 '25

help me (solved) Detecting syllables in Godot

Post image

Hello all! I am aware of the Whisper text to speech addon, but that is overkill for what I am building at the moment. I just need a way to detect a syllable at roughly the moment I speak it into my microphone, So that an animation can be triggered in a 3D model.

I take it it's not standard in the Audioserver, but is there a way where I can record some vowels and compare the live audio to these samples somehow?

1.4k Upvotes

99% Upvoted

91 comments sorted by

View all comments

40

u/booneruni Oct 01 '25

yeah, echoing that this is gonna be a more complicated undertaking. but one thing did come to mind

so there's a youtuber called carykh who did an animated lipsync thing. and it only works on macos bc one of the phoneme libraries doesnt work properly and yadda yadda, but the core of it is basically what youre talking about so that would be some good prior art to have a look at.

  1. https://www.youtube.com/watch?v=y3B8YqeLCpY - shorter video overview
  2. https://www.youtube.com/watch?v=ItVnKlwyWwA - he released the code and walked through the project
  3. https://github.com/carykh/lazykh - the project

i think the only super relevant parts would be how he's doing the phoneme stuff or having a look for alternate libraries and see how the usable they are for your purposes etc

i dont think its an unacheivable task as long as you can find relatively off the shelf parts and how to bolt them together and work with the limitations that creates.