The dangerous migration is not from keyword search to semantic search. It is from explainable keyword behavior to an untested hybrid system with no fallback. Users do not care that embeddings are modern if exact order numbers, product names, error codes, and compliance filters regress on launch day.

Short Version

Migrate from Elasticsearch or OpenSearch keyword search to semantic search as a controlled relevance migration, not a feature toggle. Keep keyword search running, add embeddings through dual indexing, backfill safely, evaluate relevance, run A/B tests, preserve fallback, and cut over with rollback.

The right target is usually hybrid search, not pure semantic search. BM25 remains valuable for identifiers, exact terms, names, and field-aware search. Vectors add semantic recall. Reranking and fusion decide the final order.

Elasticsearch and OpenSearch feature compatibility, vector syntax, licensing, and managed-service behavior differ by version and provider — and the divergence has a specific origin. Elastic changed Elasticsearch and Kibana’s license away from Apache 2.0 in January 2021 (to SSPL, later Elastic License v2), and AWS responded in April 2021 by forking Elasticsearch and Kibana 7.10.2 under the Apache 2.0 license as OpenSearch. Every Elasticsearch feature released after that fork point (including most of Elastic’s own vector search work) is not automatically present in OpenSearch, and vice versa — the two codebases have diverged independently since. Confirm feature-by-feature compatibility for the specific versions in play rather than assuming shared ancestry implies shared syntax.

Situation

Many teams already have a mature keyword search index. It has analyzers, synonyms, dashboards, relevance tweaks, index templates, aliases, and incident history. Then semantic search arrives through RAG, AI search, or product-discovery requirements.

The business wants better answers without losing the known behavior users rely on.

The Problem

Existing search systems contain hidden contracts:

  • Exact identifiers must work.
  • Filters must preserve authorization.
  • Sort and boost rules encode product policy.
  • Synonyms and analyzers reflect years of tuning.
  • Dashboards and alerts assume existing query paths.
  • Customer support knows current failure modes.

Semantic search can improve recall, but it can also break those contracts. A vector result that is conceptually close but unavailable, unauthorized, obsolete, or missing the exact requested identifier is not an improvement.

The migration question is: how do you add semantic recall without destroying keyword reliability?

Core Technical Explanation

A safe migration keeps old and new retrieval paths alive long enough to compare them.

flowchart TD
    Source[source records] --> Current[current keyword index]
    Source --> Embed[embedding generation]
    Embed --> Candidate[new hybrid index]
    Current --> Eval[offline relevance evaluation]
    Candidate --> Eval
    Eval --> AB[A B test or shadow traffic]
    AB --> Cutover[alias cutover]
    Cutover --> Fallback[keyword fallback and rollback]

The migration has five workstreams.

Schema. Add vector fields and model metadata to a new index or versioned index template. Avoid mutating the only production index in place unless rollback is trivial.

Embedding generation. Generate embeddings from stable text fields. Store model version, dimension, source hash, and generated time.

Backfill. Index embeddings in controlled batches. Monitor indexing pressure, refresh behavior, merge pressure, and search latency.

Evaluation. Compare current keyword results, semantic results, and hybrid results against known query sets.

Cutover. Use aliases, feature flags, or route-level controls so traffic can move gradually and roll back quickly.

In Practice

Start by classifying queries:

  • Exact lookup: order ID, SKU, ticket number, error code.
  • Navigational: known title, known product, known document.
  • Exploratory: broad product or knowledge discovery.
  • Troubleshooting: long natural-language problem statement.
  • Regulated: queries where tenant, region, or entitlement filters dominate.

Do not tune all query classes with one average metric. Exact lookup may stay mostly BM25. Troubleshooting may benefit heavily from vectors. Product discovery may need hybrid plus business reranking.

Use shadow evaluation before user-facing A/B tests. For each real query, run the old keyword path and the new hybrid path, log candidate IDs, and compare offline. Once the new path passes basic safety checks, expose it to a small traffic slice.

Keep fallback boring. If vector search, embedding generation, reranking, or fusion fails, the system should be able to serve keyword results.

Where It Breaks

Failure modeMigration symptomMitigation
Pure semantic replacementExact identifiers regressKeep BM25 path and hybrid fusion
No dual indexRollback requires risky in-place editsBuild versioned index and alias switch
Backfill unthrottledCluster latency and merge pressureBatch and throttle indexing
No relevance baselineTeam argues from anecdotesMaintain query sets and offline reports
Filters rewritten incorrectlySecurity or availability regressionsShare filter logic and run negative tests
A/B test lacks guardrailsBad results reach too much trafficStart with shadow mode and small cohorts
Reranker opaqueCannot explain result changesLog candidate stages and features

Security and Tenancy Notes

The migration must preserve existing authorization behavior. If keyword search enforces tenant filters, region restrictions, deleted state, or compliance rules, the semantic path must enforce the same constraints.

Dual indexing can create data leakage if old and new indexes differ in redaction, deletion, or entitlement fields. Every migration should include negative tests: documents a user must not retrieve, tenants they must not cross, and deleted records that must not reappear.

If embeddings are generated outside the search cluster, review data handling and retention — this is a contractual question for whoever owns the data-processing agreement with that embedding provider, not a cluster configuration setting.

Cost Notes

Migration temporarily costs more than steady state. You may run two indexes, store both old and new fields, backfill embeddings, execute shadow queries, run A/B infrastructure, and keep rollback capacity.

Budget for:

  • Embedding generation.
  • Extra index storage.
  • Additional replicas or nodes during backfill.
  • Snapshot storage for both index versions.
  • Query CPU for shadow traffic.
  • Reranking or model calls.

The cost is justified if it prevents a relevance incident during cutover.

Observability Notes

Track migration-specific signals:

  • Backfill progress and failure rate.
  • Embedding lag and model-version distribution.
  • Keyword versus hybrid result overlap.
  • Exact-lookup regression rate.
  • Zero-result rate.
  • Click-through or success metrics by cohort.
  • Query latency by old path and new path.
  • Filter mismatch count.
  • Rollback trigger metrics.

Keep a dashboard for launch day that compares old and new behavior side by side.

Backup, Restore, and DR Notes

Before cutover, snapshot both the old and new index versions. More importantly, make sure both can be rebuilt from source systems.

If the hybrid index is corrupted, mapped incorrectly, or built with the wrong embeddings, restore may not be enough. You may need to rebuild. The runbook should include replacement index creation, backfill replay, relevance validation, alias switch, and rollback.

Decision Checklist

  • What existing keyword behavior must never regress?
  • Which query classes benefit from semantic search?
  • Is the target pure vector, hybrid, or hybrid plus reranking?
  • Are vector fields added in a new versioned index?
  • Is embedding generation idempotent and versioned?
  • Can the old keyword path serve as fallback?
  • Is there a protected relevance query set?
  • Are tenant and entitlement filters shared by both paths?
  • Can traffic be shifted gradually?
  • Can rollback happen without reindexing under pressure?

What to Do Next

Problem: Teams replace years of keyword tuning with a vector query in one release, and exact order numbers, product names, error codes, and compliance filters regress on launch day.

Solution: Treat the migration like a database cutover — dual indexing, shadow evaluation before user-facing A/B tests, a protected relevance query set per query class, and an alias-based rollback path — targeting hybrid search rather than pure semantic replacement.

Proof: The old keyword path and new hybrid path are compared offline on the same query set before any user sees the new path, and a negative test confirms no authorization regression between the two indexes.

Action: This week, confirm the migration plan can be stopped halfway (via alias switch) without user-visible damage — if rollback requires reindexing under pressure, that’s the first gap to close.

Sources to Verify