The mistake is treating natural language search as a vector database selection problem. The harder production question is whether the workload needs exact matching, semantic discovery, hard filters, ranked results, tool execution, relationship reasoning, or all of them in one controlled retrieval pipeline.

Situation

Product teams are adding natural language interfaces to search surfaces that already had years of operational complexity: marketplace listings, travel inventory, retail catalogs, support content, internal knowledge bases, order systems, incident runbooks, and policy repositories.

The request sounds simple: “Let users ask in natural language.” The architecture is not simple.

A user might type “quiet family hotel near the beach with breakfast under 250.” Another might ask “policy for MRI prior authorization after appeal denial.” An engineer might ask “replica lag after failover in the payments database.” These queries mix intent, exact terms, filters, freshness, ranking, and sometimes task execution.

The retrieval layer has to know which parts are soft meaning and which parts are hard constraints. Price, availability, tenant access, deletion state, eligibility, and booking inventory are not semantic preferences. They are source-of-truth facts.

The Problem

Most failed AI search designs start with the wrong first question:

“Which vector database should we use?”

That skips the real architecture review. Vector similarity is one retrieval technique. It is not a full search product, a ranking system, an authorization model, a freshness model, or a workflow engine.

Pure vector search is strong when wording differs but meaning is close. It is weak when the exact token matters: addresses, SKUs, drug names, legal clauses, policy IDs, error codes, hotel names, dates, prices, and acronyms.

Pure keyword search has the opposite failure mode. BM25 and full-text search are strong for exact terms, rare words, and fielded relevance. They struggle when the user’s language and the document’s language differ.

RAG adds another boundary. It retrieves context and asks an LLM to answer. That is useful for grounded Q&A, but it does not make bad retrieval good. If the right document never enters the context window, the model is answering from incomplete evidence.

Agents add planning and tools, but they increase cost, latency, and safety exposure. GraphRAG adds entity and relationship reasoning, but it adds ingestion cost, graph construction, summarization, and freshness complexity.

The architecture question is not “Which tool is best?” It is:

What retrieval problem is the system actually solving?

The Retrieval Decision Stack

Treat natural language search as a stack of retrieval responsibilities:

flowchart TD
    Q[User request] --> U[Query understanding]
    U --> L[Lexical retrieval]
    U --> V[Semantic retrieval]
    U --> F[Hard filters]
    L --> C[Candidate pool]
    V --> C
    F --> C
    C --> R[Ranking and fusion]
    R --> RR[Reranking]
    RR --> O{Needs action}
    O -->|No| S[Search results]
    O -->|Yes| A[Agent workflow]
    C --> G{Needs relationships}
    G -->|Yes| KG[Graph retrieval]

The baseline architecture for serious search systems is usually hybrid retrieval:

BM25 or full-text search for exact terms, vector retrieval for semantic intent, metadata filters for hard constraints, fusion for candidate combination, business ranking for product rules, reranking for final quality, and optional LLM generation only when an answer or explanation is useful.

That stack separates concerns:

LayerJob
Lexical searchExact terms, identifiers, phrase relevance, field boosts
Vector searchSemantic similarity, paraphrases, discovery queries
Metadata filtersTenant, access, price, location, inventory, lifecycle state
FusionCombine candidates without comparing raw scores blindly
Business rankingFreshness, quality, popularity, compliance, availability
RerankingReorder a bounded candidate set with richer features
RAG answerExplain, summarize, compare, cite
Agent workflowPlan, call tools, retry, validate, take action
Graph retrievalTraverse entities, relationships, communities, ownership, events

In Practice

The documented pattern across modern retrieval systems is convergence toward hybrid search rather than pure vector search.

OpenSearch documents hybrid search as combining keyword and semantic search through a search pipeline that normalizes and combines scores. That is a search-engine view of the problem: lexical and semantic signals produce different score distributions, so production systems need an explicit combination stage.

PostgreSQL and pgvector show the database-centered path. PostgreSQL full-text search provides tsvector, tsquery, and ranking functions. pgvector adds vector types and ANN indexes. The documented pattern is not “replace SQL with vectors.” It is “combine relational filters, full-text ranking, and vector similarity when the workload is still database-shaped.”

Weaviate, Qdrant, and Pinecone show the retrieval-service path. Each supports hybrid retrieval, but with different operating models: object and BM25F fusion, sparse plus dense vectors, or managed dense and sparse records. The production decision is therefore not just feature support. It is which team can operate, tune, secure, and recover the system.

Microsoft’s GraphRAG documentation describes graph extraction, community detection, community reports, local search, and global search over private corpora. That is a different architecture class. It is useful when relationships and corpus-level synthesis matter. It is not a better replacement for ordinary product or document search.

The practical decision rule is:

Use the cheapest architecture that preserves the user’s contract.

For a small internal runbook assistant, PostgreSQL full-text search plus pgvector may be enough. For a consumer marketplace, a search serving layer with lexical relevance, vector fields, facets, geo filters, ranking, and analytics is usually more natural. For a workflow assistant, add agents after retrieval quality and tool boundaries are understood. For relationship-heavy intelligence, add graph retrieval when entity links are the product.

Where It Breaks

Bad framingProduction failureBetter question
”Use embeddings for search”Exact names, IDs, prices, and filters failWhich fields require exact matching?
”Use RAG for answers”LLM answers from incomplete contextDid retrieval find the right evidence?
”Use agents for search”Latency and cost rise without better recallIs the user asking for an action?
”Use GraphRAG because it is advanced”Expensive graph pipeline serves simple lookupAre relationships the core value?
”Keep everything in one database”OLTP, search, ranking, and AI traffic competeHas search become a product feature?
”Move to a new vector service”Source-of-truth drift and rebuild gaps appearIs the ingestion path replayable?

What to Do Next

  • Problem: Classify query classes before selecting infrastructure: exact lookup, semantic discovery, filtered search, comparison, task execution, or relationship analysis.
  • Solution: Start with hybrid retrieval as the default production baseline, then add RAG, agents, or graph retrieval only when the workload requires them.
  • Proof: Build a query evaluation set with exact-token queries, natural-language queries, strict-filter queries, empty-result cases, and tenant-negative tests.
  • Action: Write the architecture decision record around source of truth, hard filters, ranking, observability, rebuild path, degraded mode, and ownership.

The right decision framework is not vendor-first. It is workload-first.

If the workload is database-shaped, stay close to the database. If search is a product, use a search platform. If retrieval is the service, evaluate vector-native systems. If the user wants an outcome, add agents. If relationships are the value, use graph retrieval.

Natural language search is not one architecture. It is a set of retrieval contracts that need to be made explicit before the first embedding index is created.

Sources