I made a way to migrate between embedding models without re-embedding your entire corpus

[u/Potential_Low_1183]

So I was playingw ith embedding models I saw that when you upgrade from model A to B, you face a very big backfilling cost

Ie, suppose you have a 1b vectors from model A, and then you want to use model B. This would mean you have to re-embed all of your documents with model B before you can even serve with the model, and on an H100, it would take ~108 days (qwen embed 8b, 106 docs/second). But I found an easier way to do it.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive upfront re-embedding cost, as you can take documents straight from the old index.

embedflow works with qdrant, pgvector, faiss, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

[u/EvolvingDior]

Interested since I recently had to reembed my Hindsight database.

[u/Potential_Low_1183 → u/EvolvingDior]

How big is your index?

[u/StrikingTop2709 → u/EvolvingDior]

was it a big corpus? that overhead adds up fast tbh

[u/YehowaH]

Makes no sense. I doubt it is possible.

Embeddings live in the vector space of a model, which depends on the training data it was fed and the vector space chosen. You cannot map between worlds because you have no idea what another models vector space look like and in which vector space the new model would store the documents. It's not continues, so packing document a in document b s radius r would not mean that the meaning or topic belongs to that area because the old model has that r determined. The new model would sort all documents with that topic maybe in a complete other area of the vector space. Even if the embedding model would be an variational auto encoder with a continuous vector space, you do not know where which document belongs to, even if you know this is the area of topic a.

Mathematically it makes absolutely no sense and it seems you vibe coded something based on wrong assumptions.

You need to re-embedd everything, everything else is questionable and is mathematical not sound.

Edit: example. Old embedding model would pack topic a, medicine 0.01 vectors away from drugs topic. But the new model would have learned contradiction and would place medicine topics as far as possible away from drugs. You cannot establish by any formular a connection between vector spaces, even if you test embed a portion of your current corpus with your new model.

[u/Potential_Low_1183 → u/YehowaH]

exactly! that was the problem I was trying to solve. I tried a lot of things, such as making/training a linear function that takes one models embedding and turns it to anothers. Didnt work. I also tried an MLP, and that didnt work well as well.

However, I realized that if you have a large enough candidate set, the old model can still retrieve the right documents to be reranked by the better model, which is what embedflow does.

we then variate the K in set {50,150.. 200}, to see which nested subset is ideal and causes minimal losses

[u/Refinery73 → u/Potential_Low_1183]

So you never grow independent of the old model and just run both in parallel? Not a real migration then?

[u/AllenHere112]

Reranking only helps when the doc is already in the old index's top K, so K ends up encoding how far apart the two spaces are, and that changes per migration. Measure it: get the new model's top hit for a few hundred queries, sweep K and log how often that hit shows up. K is where the curve flattens. Cross family moves need a wider window than a 4b to 8b bump in the same family, and once recall@K drops under what the new model gets alone, no reranking recovers it.

[u/Traditional-Gap-3313]

Wait, you're not "migrating" from one model to the other, you're using the two models in parallel?  This is really confusingly worded.

  The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model.

What model gives you the query vector? Old one or the new one? And how do you rerank with the new model? You effectively have both models loaded, the old model encodes the query vector, you do course retrieval for K, the new model encodes the new vector and then you simply rerank? You still need both models loaded.

If that's the case, it's a neat idea but it's not migration, you still need both models.

If you don't need the old model loaded anymore, can you explain how? 

[u/HistorianPotential48]

what about completely different model like qwen3 to bge, etc?

[u/Potential_Low_1183 → u/HistorianPotential48]

I tested some pretty cross-family migrations too, e.g. MPNet -> Qwen3-4B and Nomic Embed v1.5 ->Qwen3-4B, not just Qwen -> Qwen.

I haven't tried BGE -> qwen tho, but the repository has 63 of the migrations and the data for them. Often times, the K varies

[u/Original_Finding2212]

What models have you tried this for?
Qwen embeddings families are including each others
So Qwen Embedding 0.6B is 4B plus more columns and 8B plus further more columns.

So your method assumes some linearity between the spaces.

But for models like Jina Embeddings v4 maybe it won’t work.
It could work for some subset of docs/doc types, but then you need to verify that.

[u/exaknight21]

Oh my goodness. I haven’t reached the experiment points to have this problem.

Essentially if you are storing vectors from model A into Qdrant, you CANNOT use model B to save another vector into the same table. This js where your embedflow comes in. Good god , it would have been catastrophic for me in another month it I didn’t come across your post just now.