Hybrid Search

The Problem with Vector-Only Search

Vector search is elegant. Embed your query, find similar vectors, retrieve relevant documents. It captures semantic meaning beautifully.

But it has blind spots.

Ask a vector search system "What is the API rate limit for GPT-4?" and it might return documents about "API usage patterns" or "model pricing" - semantically related, but missing the exact information you need.

The phrase "rate limit" has a specific meaning. Sometimes you need exact matches, not semantic similarity.

•••

The Case for Hybrid Search

Hybrid search combines two retrieval methods:

Method	Strength	Weakness
Vector (Semantic)	Understands meaning, handles synonyms	Misses exact terms, keyword-specific queries
Keyword (BM25)	Precise term matching, handles rare words	No semantic understanding, misses synonyms

The magic happens when you combine them.

•••

Our Benchmark Results

We tested on 10,000 queries across our document corpus:

Search Method	Precision@5	Recall@10	MRR	Latency
Vector only	0.71	0.78	0.68	45ms
BM25 only	0.64	0.72	0.61	12ms
Hybrid (RRF)	0.82	0.89	0.79	52ms
Hybrid (Weighted)	0.84	0.87	0.81	55ms

Key findings:

•Hybrid search improved precision by 18-23% over vector-only
•The latency overhead is minimal (7-10ms)
•Different fusion methods work better for different query types

•••

Implementation: The Complete System

Step 1: Dual Indexing

First, index your documents in both systems:

Typescript

interface Document {
  id: string;
  content: string;
  metadata: Record<string, any>;
}

class HybridIndexer {
  private vectorStore: VectorStore;
  private keywordIndex: KeywordIndex;
  private embedder: Embedder;
  
  constructor(config: HybridConfig) {
    this.vectorStore = new VectorStore(config.vector);
    this.keywordIndex = new KeywordIndex(config.keyword);
    this.embedder = new Embedder(config.embedding);
  }
  
  async indexDocument(doc: Document): Promise<void> {
    // Generate embedding for vector store
    const embedding = await this.embedder.embed(doc.content);
    
    // Index in vector store
    await this.vectorStore.upsert({
      id: doc.id,
      vector: embedding,
      metadata: doc.metadata
    });
    
    // Index in keyword store (BM25)
    await this.keywordIndex.index({
      id: doc.id,
      content: this.preprocessForBM25(doc.content),
      metadata: doc.metadata
    });
  }
  
  private preprocessForBM25(text: string): string {
    return text
      .toLowerCase()
      .replace(/[^a-z0-9\s]/g, ' ')  // Remove special chars
      .replace(/\s+/g, ' ')           // Normalize whitespace
      .trim();
  }
  
  async indexBatch(docs: Document[]): Promise<void> {
    // Batch embedding for efficiency
    const embeddings = await this.embedder.embedBatch(
      docs.map(d => d.content)
    );
    
    // Parallel indexing
    await Promise.all([
      this.vectorStore.upsertBatch(
        docs.map((doc, i) => ({
          id: doc.id,
          vector: embeddings[i],
          metadata: doc.metadata
        }))
      ),
      this.keywordIndex.indexBatch(
        docs.map(doc => ({
          id: doc.id,
          content: this.preprocessForBM25(doc.content),
          metadata: doc.metadata
        }))
      )
    ]);
  }
}

Step 2: BM25 Implementation

BM25 (Best Matching 25) is the gold standard for keyword search:

Typescript

interface BM25Config {
  k1: number;  // Term frequency saturation (typically 1.2-2.0)
  b: number;   // Length normalization (typically 0.75)
}

class BM25Index {
  private k1: number;
  private b: number;
  private documents: Map<string, TokenizedDoc>;
  private idf: Map<string, number>;
  private avgDocLength: number;
  
  constructor(config: BM25Config = { k1: 1.5, b: 0.75 }) {
    this.k1 = config.k1;
    this.b = config.b;
    this.documents = new Map();
    this.idf = new Map();
    this.avgDocLength = 0;
  }
  
  index(doc: { id: string; content: string }): void {
    const tokens = this.tokenize(doc.content);
    const termFreqs = this.computeTermFrequencies(tokens);
    
    this.documents.set(doc.id, {
      id: doc.id,
      tokens,
      termFreqs,
      length: tokens.length
    });
    
    this.updateStatistics();
  }
  
  private tokenize(text: string): string[] {
    return text
      .toLowerCase()
      .split(/\s+/)
      .filter(t => t.length > 1);
  }
  
  private computeTermFrequencies(tokens: string[]): Map<string, number> {
    const freqs = new Map<string, number>();
    for (const token of tokens) {
      freqs.set(token, (freqs.get(token) || 0) + 1);
    }
    return freqs;
  }
  
  private updateStatistics(): void {
    const N = this.documents.size;
    let totalLength = 0;
    const docFreqs = new Map<string, number>();
    
    this.documents.forEach(doc => {
      totalLength += doc.length;
      const seenTerms = new Set<string>();
      
      doc.tokens.forEach(token => {
        if (!seenTerms.has(token)) {
          docFreqs.set(token, (docFreqs.get(token) || 0) + 1);
          seenTerms.add(token);
        }
      });
    });
    
    this.avgDocLength = totalLength / N;
    
    // Compute IDF for each term
    docFreqs.forEach((df, term) => {
      // IDF with smoothing
      this.idf.set(term, Math.log((N - df + 0.5) / (df + 0.5) + 1));
    });
  }
  
  search(query: string, topK: number = 10): SearchResult[] {
    const queryTokens = this.tokenize(query);
    const scores: Map<string, number> = new Map();
    
    this.documents.forEach((doc, docId) => {
      let score = 0;
      
      for (const token of queryTokens) {
        const tf = doc.termFreqs.get(token) || 0;
        const idf = this.idf.get(token) || 0;
        
        if (tf > 0) {
          // BM25 scoring formula
          const numerator = tf * (this.k1 + 1);
          const denominator = tf + this.k1 * (
            1 - this.b + this.b * (doc.length / this.avgDocLength)
          );
          score += idf * (numerator / denominator);
        }
      }
      
      if (score > 0) {
        scores.set(docId, score);
      }
    });
    
    // Sort and return top K
    return Array.from(scores.entries())
      .sort((a, b) => b[1] - a[1])
      .slice(0, topK)
      .map(([id, score]) => ({ id, score }));
  }
}

Step 3: Hybrid Search with Fusion

Now combine the results. We use two fusion strategies:

Reciprocal Rank Fusion (RRF)

RRF is simple and effective. It doesn't require score normalization:

Typescript

interface FusionConfig {
  k: number;  // RRF constant (typically 60)
  vectorWeight: number;
  keywordWeight: number;
}

class HybridSearcher {
  private vectorStore: VectorStore;
  private bm25Index: BM25Index;
  private embedder: Embedder;
  private config: FusionConfig;
  
  constructor(
    vectorStore: VectorStore,
    bm25Index: BM25Index,
    embedder: Embedder,
    config: FusionConfig = { k: 60, vectorWeight: 0.5, keywordWeight: 0.5 }
  ) {
    this.vectorStore = vectorStore;
    this.bm25Index = bm25Index;
    this.embedder = embedder;
    this.config = config;
  }
  
  async search(query: string, topK: number = 10): Promise<HybridResult[]> {
    // Run both searches in parallel
    const [vectorResults, keywordResults] = await Promise.all([
      this.vectorSearch(query, topK * 2),
      this.keywordSearch(query, topK * 2)
    ]);
    
    // Fuse results using RRF
    return this.reciprocalRankFusion(
      vectorResults,
      keywordResults,
      topK
    );
  }
  
  private async vectorSearch(
    query: string, 
    topK: number
  ): Promise<SearchResult[]> {
    const embedding = await this.embedder.embed(query);
    return this.vectorStore.search({
      vector: embedding,
      topK
    });
  }
  
  private keywordSearch(query: string, topK: number): SearchResult[] {
    return this.bm25Index.search(query, topK);
  }
  
  private reciprocalRankFusion(
    vectorResults: SearchResult[],
    keywordResults: SearchResult[],
    topK: number
  ): HybridResult[] {
    const scores = new Map<string, FusionScore>();
    const k = this.config.k;
    
    // Score vector results
    vectorResults.forEach((result, rank) => {
      const existing = scores.get(result.id) || { 
        id: result.id, 
        vectorScore: 0, 
        keywordScore: 0,
        vectorRank: -1,
        keywordRank: -1
      };
      existing.vectorScore = 1 / (k + rank + 1);
      existing.vectorRank = rank + 1;
      scores.set(result.id, existing);
    });
    
    // Score keyword results
    keywordResults.forEach((result, rank) => {
      const existing = scores.get(result.id) || {
        id: result.id,
        vectorScore: 0,
        keywordScore: 0,
        vectorRank: -1,
        keywordRank: -1
      };
      existing.keywordScore = 1 / (k + rank + 1);
      existing.keywordRank = rank + 1;
      scores.set(result.id, existing);
    });
    
    // Combine scores with weights
    const results: HybridResult[] = Array.from(scores.values()).map(s => ({
      id: s.id,
      score: (
        this.config.vectorWeight * s.vectorScore +
        this.config.keywordWeight * s.keywordScore
      ),
      vectorRank: s.vectorRank,
      keywordRank: s.keywordRank,
      inBoth: s.vectorRank > 0 && s.keywordRank > 0
    }));
    
    // Sort by combined score
    return results
      .sort((a, b) => b.score - a.score)
      .slice(0, topK);
  }
}

Weighted Score Fusion

For more control, normalize and weight the raw scores:

Typescript

private weightedScoreFusion(
  vectorResults: SearchResult[],
  keywordResults: SearchResult[],
  topK: number
): HybridResult[] {
  // Normalize vector scores (already 0-1 for cosine similarity)
  const vectorMax = Math.max(...vectorResults.map(r => r.score));
  const normalizedVector = vectorResults.map(r => ({
    ...r,
    score: r.score / vectorMax
  }));
  
  // Normalize BM25 scores (can vary widely)
  const keywordMax = Math.max(...keywordResults.map(r => r.score));
  const normalizedKeyword = keywordResults.map(r => ({
    ...r,
    score: r.score / keywordMax
  }));
  
  // Combine into single map
  const scores = new Map<string, number>();
  
  normalizedVector.forEach(r => {
    scores.set(r.id, this.config.vectorWeight * r.score);
  });
  
  normalizedKeyword.forEach(r => {
    const existing = scores.get(r.id) || 0;
    scores.set(r.id, existing + this.config.keywordWeight * r.score);
  });
  
  return Array.from(scores.entries())
    .map(([id, score]) => ({ id, score }))
    .sort((a, b) => b.score - a.score)
    .slice(0, topK);
}

•••

Query-Adaptive Weighting

Different queries benefit from different weights. We built a classifier:

Typescript

class AdaptiveHybridSearcher extends HybridSearcher {
  private queryClassifier: QueryClassifier;
  
  async search(query: string, topK: number = 10): Promise<HybridResult[]> {
    // Classify the query type
    const queryType = await this.queryClassifier.classify(query);
    
    // Adjust weights based on query type
    const weights = this.getWeightsForQueryType(queryType);
    
    // Override config for this search
    const originalConfig = this.config;
    this.config = { ...this.config, ...weights };
    
    const results = await super.search(query, topK);
    
    // Restore original config
    this.config = originalConfig;
    
    return results;
  }
  
  private getWeightsForQueryType(type: QueryType): Partial<FusionConfig> {
    switch (type) {
      case 'exact_match':
        // "What is the API key format?"
        return { vectorWeight: 0.3, keywordWeight: 0.7 };
        
      case 'conceptual':
        // "How does authentication work?"
        return { vectorWeight: 0.7, keywordWeight: 0.3 };
        
      case 'technical_term':
        // "OAuth2 PKCE flow"
        return { vectorWeight: 0.4, keywordWeight: 0.6 };
        
      case 'natural_language':
        // "I need help setting up my account"
        return { vectorWeight: 0.8, keywordWeight: 0.2 };
        
      default:
        return { vectorWeight: 0.5, keywordWeight: 0.5 };
    }
  }
}

class QueryClassifier {
  async classify(query: string): Promise<QueryType> {
    // Simple heuristics (can be replaced with ML model)
    const lowercaseQuery = query.toLowerCase();
    
    // Check for exact match indicators
    if (lowercaseQuery.includes('what is the') || 
        lowercaseQuery.includes('error code') ||
        query.match(/["'][^"']+["']/)) {
      return 'exact_match';
    }
    
    // Check for technical terms
    const technicalPatterns = /\b(api|oauth|jwt|http|sdk|config)\b/i;
    if (technicalPatterns.test(query)) {
      return 'technical_term';
    }
    
    // Check for conceptual questions
    if (lowercaseQuery.startsWith('how') || 
        lowercaseQuery.startsWith('why') ||
        lowercaseQuery.includes('explain')) {
      return 'conceptual';
    }
    
    // Default to natural language
    return 'natural_language';
  }
}

Adaptive Weighting Results

Query Type	Vector Weight	Keyword Weight	Precision Improvement
Exact match	0.3	0.7	+31% vs balanced
Conceptual	0.7	0.3	+18% vs balanced
Technical term	0.4	0.6	+22% vs balanced
Natural language	0.8	0.2	+12% vs balanced

•••

Production Considerations

Caching Strategy

Typescript

class CachedHybridSearcher {
  private searcher: HybridSearcher;
  private cache: LRUCache<string, HybridResult[]>;
  
  constructor(searcher: HybridSearcher, cacheSize: number = 10000) {
    this.searcher = searcher;
    this.cache = new LRUCache({ max: cacheSize, ttl: 1000 * 60 * 60 });
  }
  
  async search(query: string, topK: number = 10): Promise<HybridResult[]> {
    const cacheKey = this.buildCacheKey(query, topK);
    
    const cached = this.cache.get(cacheKey);
    if (cached) {
      metrics.cacheHit('hybrid_search');
      return cached;
    }
    
    const results = await this.searcher.search(query, topK);
    this.cache.set(cacheKey, results);
    
    return results;
  }
  
  private buildCacheKey(query: string, topK: number): string {
    // Normalize query for better cache hits
    const normalized = query.toLowerCase().trim().replace(/\s+/g, ' ');
    return normalized + ':' + topK;
  }
}

Index Synchronization

Keep both indexes in sync:

Typescript

class SynchronizedIndexer {
  private indexer: HybridIndexer;
  private pendingUpdates: Map<string, Document>;
  private syncInterval: number;
  
  constructor(indexer: HybridIndexer, syncIntervalMs: number = 5000) {
    this.indexer = indexer;
    this.pendingUpdates = new Map();
    this.syncInterval = syncIntervalMs;
    
    // Periodic sync
    setInterval(() => this.flush(), this.syncInterval);
  }
  
  async queueDocument(doc: Document): Promise<void> {
    this.pendingUpdates.set(doc.id, doc);
    
    // Immediate sync if queue is large
    if (this.pendingUpdates.size >= 100) {
      await this.flush();
    }
  }
  
  async flush(): Promise<void> {
    if (this.pendingUpdates.size === 0) return;
    
    const docs = Array.from(this.pendingUpdates.values());
    this.pendingUpdates.clear();
    
    try {
      await this.indexer.indexBatch(docs);
      metrics.documentsIndexed(docs.length);
    } catch (error) {
      // Re-queue failed documents
      docs.forEach(doc => this.pendingUpdates.set(doc.id, doc));
      throw error;
    }
  }
}

•••

When to Use Each Approach

Scenario	Recommendation
FAQ/Support docs	Hybrid with keyword bias (0.6)
Technical documentation	Hybrid balanced (0.5)
Legal/contract search	Keyword-heavy (0.7)
Conversational search	Vector-heavy (0.7)
Code search	Hybrid with keyword bias (0.6)
Research papers	Hybrid balanced (0.5)

•••

Key Takeaways

1Vector search alone misses exact matches - Important for technical queries, error codes, specific terms.

2BM25 alone misses semantic meaning - Important for natural language, synonyms, conceptual questions.

3Hybrid search gets the best of both - 18-23% precision improvement in our benchmarks.

4Adaptive weighting matters - Different query types need different balances.

5RRF is simple and effective - Start with RRF before trying more complex fusion methods.

6The latency overhead is minimal - 7-10ms extra for significant quality gains.

Hybrid search isn't about choosing between vector and keyword - it's about using both where they excel.

Hybrid Search: Why Vector-Only RAG Is Leaving Money on the Table

The Problem with Vector-Only Search

The Case for Hybrid Search

Our Benchmark Results

Implementation: The Complete System

Step 1: Dual Indexing

Step 2: BM25 Implementation

Step 3: Hybrid Search with Fusion

Reciprocal Rank Fusion (RRF)

Weighted Score Fusion

Query-Adaptive Weighting

Adaptive Weighting Results

Production Considerations

Caching Strategy

Index Synchronization

When to Use Each Approach

Key Takeaways

Found this helpful?

Related articles

A2A Production Patterns: Why Multi-Agent Systems Need a Protocol

httpx2: What Breaks in anthropic 1.0.0 and openai 3

EU AI Act Watermarking: What Engineers Must Do Now

📚 Continue Learning

Deploy ML to Production Workshop