Semantic Search

Semantic search understands the meaning behind queries, not just exact keyword matches. Skirre supports hybrid search that combines traditional BM25 keyword scoring with embedding-based semantic similarity.

Overview

Traditional keyword search matches documents based on the exact terms in the query. A search for "comfortable running shoes" won't find a document that says "ergonomic jogging sneakers" even though they mean the same thing.

Semantic search solves this by converting both queries and documents into vector embeddings, which are numerical representations of meaning. Documents that are semantically similar to the query score high, even without shared keywords.

Skirre's hybrid approach combines both: keyword matches ensure precision (exact terms rank high), while semantic matching improves recall (conceptually related documents also surface).

How It Works

  1. Indexing: When semantic search is enabled and documents are added, Skirre computes a vector embedding for each document's searchable text fields using a BGE-small embedding model.
  2. Querying: When a search query arrives, Skirre computes an embedding for the query text.
  3. Scoring: Each document receives two scores: a BM25 keyword score and a cosine similarity score between the query and document embeddings.
  4. Blending: The final ranking score is a weighted combination: score = (keyword_weight * bm25) + (semantic_weight * cosine_sim)

Enable semantic search in the index settings. Once enabled, Skirre begins computing embeddings for new documents. Existing documents are re-embedded automatically.

text
await idx.updateSettings({
  semantic: {
    enabled: true,
    keyword_weight: 0.6,
    semantic_weight: 0.4,
  },
});
text
curl -X PUT https://search.example.com/v1/indices/products/settings \
  -H "Authorization: Bearer sk_search_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "semantic": {
      "enabled": true,
      "keyword_weight": 0.6,
      "semantic_weight": 0.4
    }
  }'

Configuration

ParameterTypeDescription
enabled optionalbooleanEnable or disable semantic/hybrid search.
Default: false
keyword_weight optionalnumberWeight for the BM25 keyword score (0.0–1.0).
Default: 0.6
semantic_weight optionalnumberWeight for embedding cosine similarity (0.0–1.0).
Default: 0.4
Embedding model
Skirre uses the BGE-small embedding model, which runs locally on your Skirre instance. No external API calls are made, so your data stays on your infrastructure.

Tuning Weights

The default 60/40 split (keyword/semantic) works well for most use cases. Adjust based on your content and user behavior:

  • More keyword weight (0.8/0.2): When exact terminology matters, like technical docs, part numbers, or legal text.
  • Balanced (0.5/0.5): General e-commerce, content search where users phrase queries in varied ways.
  • More semantic weight (0.3/0.7): When users describe what they want in natural language rather than using exact product terms.
Use ranking scores to tune
Enable show_ranking_score in your search requests to see the blended score for each result. This helps you understand how the weights affect ranking.

When to Use Semantic Search

Good candidates

  • Product catalogs with varied descriptions ("comfortable shoes" vs "ergonomic footwear")
  • Content search where users use natural language queries
  • Multi-language content where meaning matters more than exact terms
  • Knowledge bases where similar concepts have different terminology

Consider keyword-only when

  • Users search by exact identifiers (SKUs, part numbers, codes)
  • The vocabulary is constrained and well-defined
  • Latency is critical and embedding computation adds overhead
  • Your index is very small (under 100 documents), where keyword search is typically sufficient