AI-narrated article. Prose is preserved with pronunciation adjustments; code and image cues refer to the written article.
HNSW in Production: A Fast Graph Is Not a Retrieval Contract
This is the AI-narrated article. The prose follows the written post; pronunciation is adjusted, and code and images are announced separately.
Connect HNSW graph structure to build policy, query recall, selective filters and safe index deployment, with a complete corpus acceptance check.
TL;DR: HNSW accelerates nearest-neighbor retrieval through a hierarchy of proximity graphs, but a successful query does not guarantee the exact nearest answers. Separate connectivity and construction effort from query effort, then evaluate recall against an exact baseline on representative queries. Keep metric, embedding revision, corpus snapshot and serving policy together. This article provides a complete pinned Python acceptance check, explains save/reload configuration, and diagnoses filtered result shortfalls without weakening authorization. Production decisions also need relevance judgments, freshness and latency budgets; an unfiltered recall average cannot validate every tenant or operational scenario.
A fast index can still violate the retrieval contract
A runbook search service returns quickly, but the incident responder cannot find the relevant document. Increasing replicas helps availability; it does not repair a missing neighbor. HNSW makes this operational tension explicit: a query can succeed technically while returning an incomplete approximation of the nearest vectors.
Treat retrieval as a contract with separate dimensions: distance metric, eligible corpus, result count, recall, latency, freshness and authorization. A global recall average can conceal poor results for a small tenant or a newly introduced embedding model. The platform needs a way to distinguish index quality from query policy before scaling the serving tier.
The graph gives search a route, not an answer guarantee
The original HNSW paper describes a hierarchy of proximity graphs. Upper layers contain fewer vertices and provide coarse navigation; search descends toward the densely populated base layer, where it explores candidates for the final result. Nodes can participate in multiple layers. Connections are selected to support navigation rather than simply storing every node's closest neighbors.
This is neither a balanced search tree nor an exact spatial partition. Graph traversal can miss the exact nearest vector. Its usefulness depends on connectivity, insertion choices, vector distribution and the search budget. Do not translate the paper's experiments into a latency promise for your embedding corpus.
Image. Schematic HNSW hierarchy with upper-layer navigation and a wider base-layer search
The diagram is schematic: highlighted routes illustrate navigation, not a trace from a measured index. Candidate exploration at the base layer is broader than one greedy path.
Separate build decisions from request decisions
The hnswlib parameter guide distinguishes three controls. M shapes graph connectivity and memory overhead. ef_construction determines construction candidate effort. Query ef controls the search candidate list and must be at least the requested result count k.
Table. Columns: Decision, Owner, Consequence.
Row 1. Decision: Connectivity and construction effort; Owner: Index pipeline; Consequence: Rebuild cost, graph quality and memory.
Row 2. Decision: Query effort and result count; Owner: Serving policy; Consequence: Recall versus request cost.
Row 3. Decision: Metric and embedding revision; Owner: Retrieval contract; Consequence: Meaning of distance and compatible vectors.
Row 4. Decision: Filter and eligible corpus; Owner: Product/security boundary; Consequence: Which neighbors are valid answers.
End of table.
Increasing query effort cannot be assumed to repair an inadequately built graph. Increasing connectivity is also not a free serving adjustment: it changes the index that must be built and deployed. Evaluate these decisions independently rather than changing every parameter during an incident.
Build an acceptance check around your own vectors
This example uses Python 3.11.9, NumPy 1.26.4 and hnswlib 0.8.0. Use a disposable virtual environment with a suitable C++ build toolchain if installation builds hnswlib from source. Provide embeddings.npy as an N-by-D matrix and queries.npy as a Q-by-D matrix from the same embedding revision. Labels are row offsets; production document IDs need an explicit mapping.
Use non-sensitive evaluation data. Choose a small, representative held-out query set, rather than copying corpus rows as queries, because exact comparison scans every corpus vector for every query. This script validates finite inputs, builds one index, compares approximate results with exhaustive squared-L2 ranking, and tests save/reload behavior. It does not evaluate semantic relevance or filtered retrieval.
Code example. Refer to the written article for the complete code.
Save the complete program as check_hnsw.py:
Code example. Refer to the written article for the complete code.
The pinned API documentation specifies squared-L2 distances and requires resetting query ef after binary index loading. That setting belongs in serving configuration. The script's exact baseline uses float64 arithmetic on float32 inputs; ties and near-ties can affect set overlap. Recall here means the fraction of exact top-k labels recovered, averaged across queries. It is not a relevance score or a production SLO.
No output or benchmark is preclaimed: run this on your selected corpus and retain its result with the embedding/index configuration. Repeat with separately built candidates before accepting a new graph. The exact baseline allocates corpus-sized temporary arrays, so cap the evaluation corpus and query count to available memory.
Diagnose filtered shortfalls without leaking documents
Consider an illustrative multi-tenant search incident: an unfiltered query returns ten documents, while an authorized tenant query returns only two despite many eligible documents. Before adjusting graph parameters, check where filtering occurs and compare with exact search restricted to that tenant.
In pgvector 0.8.1, approximate-index filtering occurs after scanning. Iterative scans can continue searching within configured limits. That behavior is implementation-specific; hnswlib's callback filter has different mechanics. More global candidates do not guarantee enough eligible neighbors.
Recovery options include increasing query effort within a latency budget, bounded iterative scanning where supported, or exact retrieval over a selective eligible subset. Tenant-specific indexes or partitions trade isolation and recall control for index count and operational complexity. Never remove the authorization predicate to make the result count look healthy. Record returned-count shortfalls separately from nearest-neighbor recall.
Ship the index as a versioned serving artifact
Record embedding revision, dimension, metric, corpus snapshot, build parameters and query policy together. Deploy a new index behind a canary and retain the previous compatible artifact. Re-embedding changes the distance space; replacing only the query encoder can invalidate the retrieval contract without producing an exception.
For hnswlib, marking a label deleted does not automatically create reusable capacity: replacement must be enabled explicitly. Separate mutation from querying; the documented concurrency guarantees do not permit overlapping those operations indiscriminately. A service should control this through an index lifecycle, not leave it to individual request handlers.
The temporary index in this example is removed automatically. Deactivate the environment after review; remove only the disposable environment and evaluation copies you created, preserving your source corpus. A successful HNSW rollout is an accepted retrieval contract with a rollback path, not merely a green process and a loaded graph.
Frequently asked questions
Does a larger ef guarantee exact results? No. It increases search effort; validate recall against an exact baseline on representative queries.
Should I use cosine or L2? Use the metric defined by your embedding/retrieval contract. This example checks squared L2 and does not silently normalize vectors.
Can recall replace relevance evaluation? No. Neighbor recall measures approximation error; relevance requires separate judgments about whether retrieved documents answer the user's task.