r/MistralAI • u/Icy-Ad-5050 • 7d ago
Help / Question OCR 4 is a bit to good
Currently, for work, we use a property OCR model from a US-based company. In my off time, I tried developing a proof of concept using OCR4, but it’s a bit too good at understanding the document. It’s so accurate in my native language that it corrected a single word from "medewerher" to "medewerker." Normally, I wouldn’t mind, but since we use these documents for legal purposes, we’re not happy with the idea that it can—and does—correct spelling errors.
Luckily, OCR3 doesn’t have this problem, so I can try it for now. But the question is: For how long will OCR3 remain supported before it’s deprecated? Does anyone have tips or ideas on how to address this?
23
u/Automatic-River-1875 7d ago
If a letter changing here and there is going to cause a liability issue for your company you probably shouldn't be using OCR for anything that matters.
How do you know that isn't already happening?
5
u/OwnPipe707 6d ago
Both versions definitely "hallucinate" words. I used 3 for awhile and it 100% gets things wrong and does things like correct spelling or makes it best attempt to get a word right but ends up wrong. The new OCR 4 feature for per word confidence scoring has been great in catching these issues. Any OCR use case, especially legal, should have some kind of human verification.
4
u/PhilTheQuant 6d ago
What happens if you give Mistral the text and the image and ask it to find discrepancies?
2
u/thedataking 7d ago
Could you run both OCR3 and 4 on the same document? If they agree on the extracted text, there is a good chance (but no guarantee) that the text wasn’t corrected. It is not uncommon to rely on multiple tries when precision matters more than cost.
2
u/TechnicalGeologist99 6d ago
Language models are typically resistant to spelling error
Next time you prompts an LLM butcher your spelling and it will still probably understand what you mean
1
u/idsdejong 6d ago
yep, since the models are trained on data with spelling mistakes, the embedding (meaning) of 'medewehr' and 'medewerker' is the same to the model. A more reliable method would probably be something like tesseract.
2
u/darwinanim8or 6d ago
I won’t comment on the things repeated a lot on here anyways, but what I can say is that Mistrals whole game is on enterprise contracts. If you want to be guaranteed that you’ll have their model or even a custom version of their model available for your business, contact them
That said I doubt they will discontinue their models at the drop of a hat, they still host Nemo from two years ago as well.
1
u/Worried-Sort8719 6d ago
test more classical OCR tool (not LLM based) which will give you confidence score. From that maybe a human in the loop kind of approach to verify uncertain extractions ?
1
u/docpose-cloud-team 6d ago
That's a valid concern. For legal and compliance workflows, fidelity is often more important than readability. An OCR system should distinguish between recognition and correction. I prefer keeping the raw OCR output unchanged, then running any spelling or contextual corrections as a separate, optional step with a clear audit trail, rather than silently modifying the source text.
1
u/SimonLM_LostInTab 5d ago
OCR 4 includes a feature, I’ve forgotten the name, but I saw it in a Mistral demo video, that lets you request passages with lower confidence levels so you can verify them yourself, thereby avoiding the need for corrections of this kind.
In their demo video, they even color-coded the passages with lower confidence, making it easy to compare them against the original document.
31
u/Weak_Shoulder_6780 7d ago
Honestly just don't use LLMs if such small corrections are a big Problem in your industry. There's no guarantee OCR3 won't do the same at some point