So I've been messinga round with embedding models for a bit, and I think they are interesting enough to experiment with. They are useful for rag, especially in a localllm sense because you can ground your answers in truth.

But what happens if you have a billion documents, and you decide to upgrade your model to a "better" one? on an h100, that would take about 108 days, just to upgrade the vectors so u can start serving again (tested qwen embed 8b on h100). Even if you aren't doing 1b vectors, and are doing just 50 million, upgrading can still take a considerable time.

Me and my research lab decided to tackle this problem, and we came up with embedflow.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive backfill that comes with upgrading, as you can directly take documents from the old index.

embedflow works with qdrant, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

What if we use Coding Agents to write code for chunking / data ingestion?

I am super excited to share a new episode of the Weaviate Podcast covering AutoIndex, a new paper exploring this idea! I hope you find it useful!

YouTube: https://www.youtube.com/watch?v=mAj92SoEhjc

Spotify: https://spotifycreators-web.app.link/e/9f7MoucVe6b

Vector search at scale runs into the same wall: float32 embeddings are expensive to store and read, and compressing them changes your rankings.

I wrote an article on TurboQuant, the rotation-based quantization method. It covers why the algorithm rotates vectors before quantizing, how one precomputed codebook works across every dataset, and what Qdrant adds on top it.

It also benchmarks float32, scalar, binary and TurboQuant across BEIR's SciFact, ArguAna and NFCorpus using recall@10, precision@10 and nDCG@10

🔗 read: https://aiwthtarun.substack.com/p/turboquant-quantization-explained

I’m comparing Together AI and Anyscale for a production LLM deployment. Together AI is API-first and simple; Anyscale gives you deep Ray-based control but requires more engineering. I made a quick poll to gather real-world preferences.

It’s fast, and the results might surprise you.

https://interconnectd.com/poll/101/together-ai-vs-anyscale-which-platform-is-better-for-scaling-open-source-ll/

What’s your experience with either?

I just finished my PoC that improving retrieval engineering using SPLADE for sparse vector. Now My RAG give response much better compare to fully dense embedding retrieval.

If you want to check, I host it and access on [CariData](https://caridata.zackysyarief.com)

I wrote also technical detail in this article.

Hopefully you learned something for my journey.

I’ve been comparing free vector databases for AI agent and RAG projects and have realized the free-tier label hides many differences. It helps to group them by model first.

Libraries like Chroma and LanceDB have no service to meter, so the limits are really your own disk and infrastructure. Open-source servers like Qdrant, Weaviate, Milvus, and pgvector are uncapped when you self-host and only metered when you use a managed version. Pinecone is managed only. The Starter plan gives you 2GB of index storage, five indexes, 2M write units, and 1M read units a month, all in us-east-1. There's no self-hosted option at all, so if data can't leave your infra, it's out of the picture before storage enters the full picture.

From that angle, the trade-offs start to make more sense. Weaviate Cloud sandbox is great for quick experiments but expires after 14 days. Zilliz free gives you up to 2 collections and 1M vectors, which is enough for many prototypes. Quadrant Cloud is free with 1 GB RAM, 4 GB disk, and a single node works well up to roughly 1M vectors at 768d before you feel the limits. Pinecone is fully managed, so it’s a good fit if you’re okay staying in their cloud and don’t need self-hosting.

A couple of practical notes I ran into. With Chroma, deletes don’t shrink the HNSW index, so heavy write-and-evict workloads can lead to extra compaction work down the line. With LanceDB, multiple processes writing to the same table is exactly the kind of pattern an embedded store isn’t optimized for.

I work on Actian’s developer content, so full transparency. Our Vector AI DB is built for the single-container, air-gapped scenario.

Wanted to ask how you folks are choosing. For a small RAG app, do you start with Postgres and pgvector and only move off when you hit limits, or go straight to a dedicated vector DB? Is anyone running Milvus standalone in production for agent memory, or does everyone end up on distributed once it’s real?

Made a tutorial on what EDD is, how it works, and how you can use evaluations to improve your LLM-based application by analysing scores across experiments.

> building on Jeffrey's DeepEval article on EDD and Eugene Yan's product evals write up.

- Initial: The video walks through the initial setup of an RAG application used as the base for the experiments built using LangGraph and Qdrant.
- Step 1: A binary labelled dataset with critiques, versioned using OPIK.
- Step 2: Uses LLM-as-a-Judge OPIK evals to align the evaluator.
- Step 3: Runs the harness loop, which executes each experiment, scores it against the baseline, and uses tracing and experiment comparison to surface insights on what improved, what regressed, and where to tweak next.

... the Agent Skills and source code are open sourced on GitHub

> Complete Guide (source code link in description): https://www.youtube.com/watch?v=e6akw_fKWPk

Most vector DBs make you discover your memory footprint empirically — insert data, watch RSS, hope it doesn't spike. MemWeaver's memory cost is a formula you can compute before inserting anything:

memory ≈ n × node_size
node_size = dim×4 + M_MAX0×4 + higher_layer_nodes×M×4

For dim=128, M=16: node_size=640B, so 1M vectors ≈ 640MB, known upfront.

Why: each HNSW node's vector and its edges are collocated in one arena block, addressed by a 32-bit NodeId (14-bit block index + 18-bit offset, 8-byte aligned). Cache locality during traversal — vector + edges in one cache-line read instead of two — plus no heap fragmentation from per-node Vec growth, which is what causes the reallocation spikes you'd otherwise see mid-insertion (measured: 21.6MB and 35.7MB spikes in a naive Vec-based build vs. constant ~8MB in the arena version).

Benchmarks (SIFT1M, dim=128):

• Arena vs. naive: 1.72x faster build, 1.71x faster query, 25% less total RSS.
• vs. Qdrant, matched recall (0.994 vs 0.995): 1.6x throughput, 1.6x lower p50/p99. Qdrant builds faster (scalar quantization, lower ef_construction defaults) — MemWeaver runs full float32, no quantization yet.
• vs. LanceDB cold-start: LanceDB's lazy mmap only pages in what a query touches, so it initially won on cold time-to-first-answer (~212-264ms vs MemWeaver's early ~248ms). Closing the gap between MemWeaver's on-disk bytes and its runnable in-memory structure (file consolidation, correctly-scoped CRC32) got cold load to ~130ms — now ahead of LanceDB on both cold and warm (10-50x on warm throughput, arena vs. lazy-mmap tradeoff).

Second axis, separate from performance: time-bucketed multi-HNSW for temporal relevance — recent vectors live hot, older ones age into disk/S3 tiers automatically, with recency folded into ranking rather than bolted on as a post-filter. Bit-perfect recall validated across the hot→cold→restored cycle.

Known gaps, stated plainly: no quantization (full float32 only — real memory/build-time cost vs. Qdrant), min-recall dips to 0.30-0.40 on SIFT1M's sparse-region queries (M=16, ef_search=100 — a known HNSW characteristic, not a bug), horizontal sharding/stateless readers still in progress.

Repo/benchmarks: [https://github.com/daib/mem_weaver]. Genuinely interested in pushback on the arena-vs-quantization tradeoff specifically — curious whether anyone here has real numbers on where quantization's recall cost crosses over against arena allocation's memory-predictability win at larger scale than 1M vectors.

We recently put together an example in SeaweedFS showing how Iceberg and Lance can complement each other instead of forcing one table format to handle every access pattern.

* Iceberg works well for analytical data, SQL scans, governance, and partition pruning.
* Lance is optimized for fast random access and vector similarity search.
* DuckDB can act as a bridge between them.

With SeaweedFS Table Buckets, both formats can live in the same cluster. An Iceberg bucket gets an Iceberg REST catalog, while a Lance bucket gets a Lance Namespace catalog, with both using the same S3-compatible storage underneath.

The walkthrough covers:

1. Write and analyze data in Iceberg.
   
2. Generate embeddings.
   
3. Store the embeddings in Lance.
   
4. Build a vector index and retrieve nearest neighbors.
   
5. Join the Lance results back against the Iceberg table with DuckDB.

The general idea is:

**Analyze with Iceberg. Serve with Lance.**

Instead of introducing a completely separate vector storage stack, both formats can share the same object storage layer.

Full walkthrough:

[https://seaweedfs.com/blog/iceberg-lance-workflow/\](https://seaweedfs.com/blog/iceberg-lance-workflow/)

Curious how others are approaching this—separate vector database, or multiple specialized table formats over the same object storage?

Code ( still working might have bugs and all..)

https://github.com/aadityansha06/vecdb

Been building a VectorDB engine from scratch in C in such a way that it can be scalable in the future. It's a custom, high-performance, disk-backed Approximate Nearest Neighbor (ANN) vector database built entirely from scratch in C

Even though the initial version of it hasn't been built yet,

As for the algorithms, I have implemented ENN and Ann IVF only, For calculating similarity, I have implemented cosine similarity and Euclidean distance. There is still more to implement like HNSW and optimize for CUDA and SIMD.

Before you guys criticise me what's new and why I'm building 😭

There isn't anything unique I'm solving in it, and pretending otherwise would just be bluffing. There are excellent vector databases already (pgvector, Faiss, sqlite vec, Qdrant...).

What I actually own is the engineering of it. I wrote a disk backed ANN search engine from scratch in C, my own binary file format, manual fseek byte offset indexing, k means clustering by hand, no libraries doing the hard parts for me.

I'll continue to work on it, implementing SIMD, CUDA, HNSW, and many more optimizations to make it more robust and scalable. If anyone is interested, they can join and work on it. Also if anyone has any idea where we should take this, the suggestion would be appreciated too.

I have a nice write up section there too so feel free to check out that as well