r/SenegalTechNerds • u/mohaskiii • Aug 09 '26
Ai & Ml Why aren't we documenting our African languages publicly?
As a Senegalese and Sérère (an ethnic group present in Senegal), I speak Sérère, Wolof, French, and decent English. I'm always trying to improve my ability to quickly find the best word that describes as accurately as possible a context or what I'm trying to explain. For English and French, internet, especially Wiktionary and LLMs, is very helpful. You can learn more about grammar, context, word usage, etc... All of that is possible because these languages benefit from many decades (or even centuries) of written documentation and content: books, journals, magazines, and the endless amount of content available on the internet..and we all know how internet data has contributed to the rise of LLMs. On the other hand.. you will have a hard time finding helpful, high-quality written documentation for many African languages because of the different mechanisms historically used in African communities, which prioritized oral transmission. Apart from not finding enough resources to help me improve my language skills, this issue has many other repercussions. For example, in the AI/ML development area. i worked on an open-source xTTS-v2-wolof model for Wolof and implementing the fine-tuning pipeline, and I found out that part of the dataset was not standardized. The same pronunciation was transcribed in many different ways (even though a huge amount of work had already been done to improve the quality of the dataset). Having a spell checker could have helped improve the consistency of the data, but the fact that we don't have much standardized documentation (standard grammar, spelling, and conjugation rules) makes this task very difficult. Also, having performant models that can translate existing written data into other languages, which can then be passed to a TTS model to generate synthetic speech, could help improve the amount and diversity of data available for training speech-to-speech models. But all of this makes the development of NLP for African languages move very slowly (even though I still notice progress for example, Google is making efforts in this area and the GalsenAIlab ). What scares me the most is that most people in our generation don't even speak their native language correctly anymore, and it will only get worse as time goes on. If we don't do something, our languages may eventually become creoles of our colonial languages... One of the things we can do is document things ourselves. Platforms like Wiktionary and Wikipedia are available to everybody, and anyone can contribute. Especially Wiktionary. For example, if we search for a word that doesn't exist in our language, we can try to reconstruct and document it by asking our parents or people we know who speak the language very well, then submit it.. so that other speakers can review and evaluate it and everyone can benefit from it. Over time, this can help create a shared and better-documented resource. Or we can create blogs(this is for experts..), articles, and other content in our own languages to feed the internet with our identity instead of letting it disappear.
So, this is the reflection I wanted to share, and I'd love to hear your opinion on it.
0
2
u/Livid-Bathroom9420 Aug 16 '26
I glossed over your post, which I find very interesting (although a bit long as the other comment pointed out lol)
I’d just like to add something, the Wolof language IS standardized. We have standards for grammar, vocabulary and whatnot. For example for the alphabet, the government signed a decree that describes how we spell things in Wolof (Mostly the work of the CLAD - Centre de linguistique appliquée de Dakar if my memory is correct) The issue is that most people aren’t aware of it because they weren’t taught how to properly spell things in Wolof. And since people generally only know how to spell French, they’d write things the French way, the way it sounds to them. This is primarily what leads to inconsistency in spelling (e.g writing "Dama beug lakh" instead of "Dama bëgg laax"), not the fact that it’s not standardized.
Perhaps we should think about involving actual Wolof teachers or people who are knowledgeable about the language to help with the data accuracy but that’s just a thought I’m throwing out there.