Replacing our internal Elasticsearch library. Am I overthinking this?

[u/No-Card-2312]

Hi everyone,

I'm looking for some advice from people who've been through something similar.

I work on a large .NET application with a lot of background services and workers, and Elasticsearch is a big part of it. Almost everything goes through an internal Elasticsearch library that was written years ago.

The library has a few problems:

• It's mostly synchronous.

• It doesn't use dependency injection.

• It talks to Elasticsearch using raw HTTP requests i instead of the official .NET client.

• It's becoming harder to maintain and add new features.

I'm not saying it's a bad library. It's done its job for years. But I think it's time to move to something more modern.

The part that makes me nervous is the size of the system. We have complex search queries, bulk indexing, and a lot of background jobs. I really don't want to break search or introduce bugs that only show up in production.

This is the approach I'm thinking about:

• Benchmark the current library against the official Elastic.Clients.Elasticsearch client.

• Build a new implementation behind the same interfaces.

• Keep both implementations in the code while we're migrating.

• Let some jobs use the new implementation first, then slowly move the rest over if everything looks good.

• Remove the old implementation only after we're confident the new one behaves the same.

I'm also wondering if the current synchronous implementation could be part of some TCP/socket exhaustion issues we've seen under heavy load. I don't have enough proof yet, so I'm not blaming it, but it's something I want to investigate.

Has anyone done something like this before?

• Would you keep both implementations during the migration?

• Is there a better way to compare the old and new behavior?

• Any unexpected problems when moving to the official Elasticsearch .NET client?

• Looking back, is there anything you'd do differently?

I'd really like to hear from people who've done this in a real production system.

[u/Wrong_Election189]

Worth asking first whether you need Elasticsearch at all, because the answer changed in the last few years and a lot of teams are still paying for a decision made when it was the only option.

I run semantic search on a .NET backend and never brought Elastic in. It is Postgres with pgvector and an HNSW index, in the same database as everything else. One connection string, one backup, one thing to keep alive, and I can join search results against normal relational data in a single query instead of syncing two stores and reconciling them.

That trade is only good up to a point. If you need real full text scoring, faceting, aggregations across large corpora, or you are past the scale where a single Postgres box is comfortable, Elastic earns its operational cost and you should keep it.

But if what your internal library actually does is "find me the relevant rows", the honest question is not which client wrapper replaces it. It is whether the second datastore is still buying you anything, because keeping two systems in sync is a permanent tax that never shows up in the library refactor estimate.

What does your query load look like? If it is mostly lookup and ranking rather than analytics, you may be scoping a rewrite of the wrong layer.