Method#
| Software | PostgreSQL 18.6 and pgvector 0.8.7 from the PGDG apt repository, Ubuntu 24.04; FAISS 1.15.1 (faiss-cpu) for the comparison |
|---|---|
| Hardware | 4 vCPU Intel Xeon @ 2.10GHz (cloud VM), 15 GB RAM |
| Postgres settings | shared_buffers = 4GB, maintenance_work_mem = 4GB, max_parallel_maintenance_workers = 3, work_mem = 64MB, jit = off; everything else default |
| Data | DBpedia entity abstracts embedded with OpenAI text-embedding-3-large at 1,536 dims (first 500,000 rows of Qdrant's 1M dataset) and at 3,072 dims (100K dataset, same articles) |
| Queries | 1,000 random articles from a shard never loaded into any table (for 3,072 dims: 1,000 rows held out of the 100K) |
| Ground truth | Exact top-100 by cosine distance in NumPy; verified identical to pgvector's own exact search |
| Metric | Recall@10 = |returned ∩ true top-10| / 10, averaged over queries |
| Latency | Wall-clock per query from Python (psycopg 3, prepared statements, binary vectors) over a Unix socket; 50 warm-up queries; one query in flight; p50/p95/p99 over 1,000 queries |
| Load | Binary COPY, then VACUUM ANALYZE, then CREATE INDEX |
| Build parallelism | max_parallel_maintenance_workers = 3 was set, but the table's parallel_workers option was not, so 100K-row builds ran as a single process (vectors live in TOAST, leaving the heap below the 8 MB threshold). The build chart shows the effect of setting it. |
Read these as shapes, not promises. Numbers come from one 4-vCPU cloud VM, single runs for builds, warm caches, one client. Absolute latency on your hardware will differ; the trade-offs — how recall moves with ef_search, what memory does to build time — transfer.
HNSW#
Recall@10 vs ef_search (HNSW, m = 16, ef_construction = 64)
100,000 × 1,536-dim OpenAI embeddings, 1,000 held-out queries. pgvector's default ef_search is 40.
| ef_search | Recall@10 | p50 latency | p95 latency | QPS (1 client) |
|---|---|---|---|---|
| 10 | 0.815 | 1.4 ms | 2.5 ms | 625 |
| 20 | 0.897 | 1.9 ms | 3.2 ms | 482 |
| 40 | 0.947 | 2.9 ms | 5.3 ms | 304 |
| 64 | 0.970 | 3.9 ms | 5.9 ms | 246 |
| 100 | 0.983 | 5.3 ms | 8.4 ms | 181 |
| 200 | 0.990 | 9.1 ms | 14 ms | 106 |
| 400 | 0.995 | 16 ms | 24 ms | 61 |
Effect of m on the recall/latency curve (ef_construction = 64)
Points are ef_search 10 → 400. Higher m lifts the recall ceiling at the same latency.
- m = 8
- m = 16
- m = 32
| m | ef_search | Recall@10 | p50 |
|---|---|---|---|
| 8 | 10 | 0.677 | 1.1 ms |
| 8 | 20 | 0.811 | 1.6 ms |
| 8 | 40 | 0.896 | 2.1 ms |
| 8 | 64 | 0.928 | 2.8 ms |
| 8 | 100 | 0.953 | 3.8 ms |
| 8 | 200 | 0.976 | 5.8 ms |
| 8 | 400 | 0.987 | 10 ms |
| 16 | 10 | 0.815 | 1.4 ms |
| 16 | 20 | 0.897 | 1.9 ms |
| 16 | 40 | 0.947 | 2.9 ms |
| 16 | 64 | 0.970 | 3.9 ms |
| 16 | 100 | 0.983 | 5.3 ms |
| 16 | 200 | 0.990 | 9.1 ms |
| 16 | 400 | 0.995 | 16 ms |
| 32 | 10 | 0.891 | 1.8 ms |
| 32 | 20 | 0.946 | 2.6 ms |
| 32 | 40 | 0.975 | 3.9 ms |
| 32 | 64 | 0.986 | 5.4 ms |
| 32 | 100 | 0.991 | 7.2 ms |
| 32 | 200 | 0.995 | 13 ms |
| 32 | 400 | 0.997 | 23 ms |
Effect of ef_construction on the recall/latency curve (m = 16)
Points are ef_search 10 → 400. A better-built graph gives more recall per millisecond.
- ef_construction = 32
- ef_construction = 64
- ef_construction = 128
- ef_construction = 256
| ef_construction | ef_search | Recall@10 | p50 |
|---|---|---|---|
| 32 | 10 | 0.782 | 1.4 ms |
| 32 | 20 | 0.874 | 1.9 ms |
| 32 | 40 | 0.925 | 2.7 ms |
| 32 | 64 | 0.949 | 3.7 ms |
| 32 | 100 | 0.966 | 4.9 ms |
| 32 | 200 | 0.979 | 8.1 ms |
| 32 | 400 | 0.988 | 14 ms |
| 64 | 10 | 0.815 | 1.4 ms |
| 64 | 20 | 0.897 | 1.9 ms |
| 64 | 40 | 0.947 | 2.9 ms |
| 64 | 64 | 0.970 | 3.9 ms |
| 64 | 100 | 0.983 | 5.3 ms |
| 64 | 200 | 0.990 | 9.1 ms |
| 64 | 400 | 0.995 | 16 ms |
| 128 | 10 | 0.819 | 1.4 ms |
| 128 | 20 | 0.906 | 2.0 ms |
| 128 | 40 | 0.960 | 2.9 ms |
| 128 | 64 | 0.979 | 4.0 ms |
| 128 | 100 | 0.989 | 5.4 ms |
| 128 | 200 | 0.996 | 9.4 ms |
| 128 | 400 | 0.998 | 17 ms |
| 256 | 10 | 0.818 | 1.5 ms |
| 256 | 20 | 0.910 | 2.0 ms |
| 256 | 40 | 0.964 | 3.0 ms |
| 256 | 64 | 0.983 | 4.3 ms |
| 256 | 100 | 0.991 | 5.9 ms |
| 256 | 200 | 0.997 | 10 ms |
| 256 | 400 | 0.998 | 19 ms |
HNSW build time by m and ef_construction
100,000 × 1,536 dims, maintenance_work_mem 4 GB. Single-process builds (no parallel workers launched — see the HNSW guide).
| m | ef_construction | Build time | Index size |
|---|---|---|---|
| 8 | 64 | 2.0 min | 781 MB |
| 16 | 32 | 2.0 min | 781 MB |
| 16 | 64 | 3.4 min | 781 MB |
| 32 | 64 | 5.6 min | 781 MB |
| 16 | 128 | 5.9 min | 781 MB |
| 16 | 256 | 10.0 min | 781 MB |
Analysis and recommendations: HNSW guide.
Scaling to 500K rows#
HNSW recall as the table grows (m 16, ef_construction 64)
Same 1,000 queries against 100K, 250K and 500K rows. Recall drops slowly with size at a fixed ef_search.
- ef_search 40
- ef_search 100
- ef_search 200
| Rows | ef_search | Recall@10 | p50 | Build time | Index size |
|---|---|---|---|---|---|
| 100,000 | 40 | 0.948 | 2.8 ms | 3.3 min | 781 MB |
| 100,000 | 100 | 0.982 | 5.4 ms | 3.3 min | 781 MB |
| 100,000 | 200 | 0.990 | 8.9 ms | 3.3 min | 781 MB |
| 250,000 | 40 | 0.938 | 3.1 ms | 2.6 min | 1.91 GB |
| 250,000 | 100 | 0.974 | 5.7 ms | 2.6 min | 1.91 GB |
| 250,000 | 200 | 0.986 | 9.7 ms | 2.6 min | 1.91 GB |
| 500,000 | 40 | 0.931 | 4.1 ms | 5.6 min | 3.81 GB |
| 500,000 | 100 | 0.968 | 6.2 ms | 5.6 min | 3.81 GB |
| 500,000 | 200 | 0.982 | 11 ms | 5.6 min | 3.81 GB |
Exact scan vs HNSW: median query latency
Exact = no index, sequential scan (at 100K rows Postgres planned no parallel workers — the same TOAST effect as builds). HNSW at ef_search 40.
- Exact scan
- HNSW
| Rows | Exact p50 | HNSW p50 (ef 40) | Speed-up |
|---|---|---|---|
| 100,000 | 447 ms | 2.8 ms | 160× |
| 250,000 | 680 ms | 3.1 ms | 217× |
| 500,000 | 1016 ms | 4.1 ms | 246× |
Build settings#
HNSW build time: memory, parallel workers and load order
Same 100,000 × 1,536-dim vectors, same index (m 16, ef_construction 64). Workers launched were counted from pg_stat_activity.
| Setup | Build time | Parallel workers |
|---|---|---|
| Index first, then load 100K rows | 13.8 min | 0 workers (row-by-row inserts) |
| 64 MB maintenance_work_mem (default) | 10.4 min | 0 workers launched |
| 256 MB maintenance_work_mem | 8.3 min | 0 workers launched |
| 4 GB, max_parallel_maintenance_workers = 3 | 3.4 min | 0 workers launched |
| 4 GB + ALTER TABLE SET (parallel_workers = 1) | 1.8 min | 1 worker launched |
| 4 GB + ALTER TABLE SET (parallel_workers = 3) | 56 s | 3 workers launched |
IVFFlat#
IVFFlat recall@10 vs probes (100K rows)
pgvector's default is probes = 1. More lists means smaller lists, so each probe covers less of the table.
- 100 lists
- 316 lists
- 1000 lists
| lists | probes | Recall@10 | p50 | p95 |
|---|---|---|---|---|
| 100 | 1 | 0.619 | 2.7 ms | 7.1 ms |
| 100 | 2 | 0.763 | 4.3 ms | 9.1 ms |
| 100 | 4 | 0.850 | 7.9 ms | 14 ms |
| 100 | 8 | 0.921 | 16 ms | 27 ms |
| 100 | 16 | 0.965 | 28 ms | 41 ms |
| 100 | 32 | 0.985 | 54 ms | 75 ms |
| 100 | 64 | 0.998 | 104 ms | 131 ms |
| 316 | 1 | 0.550 | 1.4 ms | 2.4 ms |
| 316 | 2 | 0.674 | 1.9 ms | 3.2 ms |
| 316 | 4 | 0.778 | 3.1 ms | 4.9 ms |
| 316 | 8 | 0.857 | 5.5 ms | 8.8 ms |
| 316 | 16 | 0.912 | 9.9 ms | 15 ms |
| 316 | 32 | 0.953 | 19 ms | 26 ms |
| 316 | 64 | 0.978 | 35 ms | 49 ms |
| 1000 | 1 | 0.485 | 1.4 ms | 2.0 ms |
| 1000 | 2 | 0.613 | 1.5 ms | 2.0 ms |
| 1000 | 4 | 0.716 | 2.0 ms | 3.2 ms |
| 1000 | 8 | 0.797 | 2.7 ms | 4.1 ms |
| 1000 | 16 | 0.860 | 4.4 ms | 6.9 ms |
| 1000 | 32 | 0.910 | 7.4 ms | 12 ms |
| 1000 | 64 | 0.947 | 13 ms | 18 ms |
IVFFlat recall vs latency at 500K rows
Points are probes 1 → 64. lists = rows/1000 (500) and √rows (707) are the usual rules of thumb.
- 500 lists
- 707 lists
- 2000 lists
| lists | probes | Recall@10 | p50 |
|---|---|---|---|
| 500 | 1 | 0.553 | 5.0 ms |
| 500 | 2 | 0.686 | 6.8 ms |
| 500 | 4 | 0.780 | 10 ms |
| 500 | 8 | 0.858 | 16 ms |
| 500 | 16 | 0.913 | 30 ms |
| 500 | 32 | 0.951 | 56 ms |
| 500 | 64 | 0.976 | 111 ms |
| 707 | 1 | 0.561 | 3.5 ms |
| 707 | 2 | 0.676 | 4.6 ms |
| 707 | 4 | 0.771 | 7.9 ms |
| 707 | 8 | 0.843 | 14 ms |
| 707 | 16 | 0.902 | 23 ms |
| 707 | 32 | 0.941 | 43 ms |
| 707 | 64 | 0.970 | 82 ms |
| 2000 | 1 | 0.499 | 3.8 ms |
| 2000 | 2 | 0.623 | 4.1 ms |
| 2000 | 4 | 0.722 | 5.9 ms |
| 2000 | 8 | 0.797 | 8.0 ms |
| 2000 | 16 | 0.856 | 13 ms |
| 2000 | 32 | 0.904 | 19 ms |
| 2000 | 64 | 0.942 | 32 ms |
Build time and index size: IVFFlat vs HNSW (100K rows)
Same table, maintenance_work_mem 4 GB, single-process builds.
| Index | Build time | Size |
|---|---|---|
| IVFFlat 100 lists | 9.8 s | 782 MB |
| IVFFlat 316 lists | 23 s | 784 MB |
| IVFFlat 1000 lists | 2.1 min | 789 MB |
| HNSW m16/efc64 | 3.4 min | 781 MB |
Analysis: IVFFlat guide.
halfvec and binary quantization#
Recall vs latency: vector, halfvec and binary quantization
100K × 1,536 dims, HNSW m 16 / ef_construction 64. Binary points vary the re-rank candidate count (20 → 640).
- vector (float32)
- halfvec (float16)
- binary + re-rank
| Variant | Setting | Recall@10 | p50 |
|---|---|---|---|
| vector | ef 10 | 0.815 | 1.4 ms |
| vector | ef 20 | 0.897 | 1.9 ms |
| vector | ef 40 | 0.947 | 2.9 ms |
| vector | ef 64 | 0.970 | 3.9 ms |
| vector | ef 100 | 0.983 | 5.3 ms |
| vector | ef 200 | 0.990 | 9.1 ms |
| vector | ef 400 | 0.995 | 16 ms |
| halfvec | ef 10 | 0.819 | 1.4 ms |
| halfvec | ef 20 | 0.896 | 1.8 ms |
| halfvec | ef 40 | 0.949 | 2.5 ms |
| halfvec | ef 64 | 0.968 | 3.5 ms |
| halfvec | ef 100 | 0.982 | 4.8 ms |
| halfvec | ef 200 | 0.991 | 7.8 ms |
| halfvec | ef 400 | 0.995 | 14 ms |
| binary + re-rank | 20 candidates | 0.854 | 1.8 ms |
| binary + re-rank | 40 candidates | 0.918 | 2.3 ms |
| binary + re-rank | 80 candidates | 0.966 | 3.5 ms |
| binary + re-rank | 160 candidates | 0.985 | 5.7 ms |
| binary + re-rank | 320 candidates | 0.993 | 10 ms |
| binary + re-rank | 640 candidates | 0.997 | 20 ms |
HNSW index size by representation
100,000 × 1,536 dims. Binary quantization indexes 1 bit per dimension; the full vectors stay in the table for re-ranking.
| Representation | Table size | Index size | Index build |
|---|---|---|---|
| vector(1536) | 795 MB | 781 MB | 3.4 min |
| halfvec(1536) | 400 MB | 391 MB | 2.9 min |
| bit(1536) via binary_quantize | (uses vector table) | 48 MB | 39 s |
Binary quantization: recall vs re-rank candidates
Fetch N candidates by Hamming distance from the bit index, re-order them by exact cosine distance, keep 10.
- 1,536 dims
- 3,072 dims
| Dims | Candidates | Recall@10 | p50 |
|---|---|---|---|
| 1536 | 20 | 0.854 | 1.8 ms |
| 1536 | 40 | 0.918 | 2.3 ms |
| 1536 | 80 | 0.966 | 3.5 ms |
| 1536 | 160 | 0.985 | 5.7 ms |
| 1536 | 320 | 0.993 | 10 ms |
| 1536 | 640 | 0.997 | 20 ms |
| 3072 | 40 | 0.930 | 2.9 ms |
| 3072 | 80 | 0.970 | 4.9 ms |
| 3072 | 160 | 0.986 | 8.0 ms |
| 3072 | 320 | 0.993 | 16 ms |
Shortening text-embedding-3-large: top-10 overlap with full 3,072 dims
Exact search on truncated, re-normalised vectors vs exact search on the full vectors. 99,000 rows, 1,000 queries.
| Dims | Top-10 overlap | vector bytes | halfvec bytes |
|---|---|---|---|
| 256 | 58.4% | 1,032 | 520 |
| 512 | 73.2% | 2,056 | 1,032 |
| 768 | 80.0% | 3,080 | 1,544 |
| 1024 | 84.1% | 4,104 | 2,056 |
| 1536 | 89.0% | 6,152 | 3,080 |
| 2048 | 91.9% | 8,200 | 4,104 |
| 3072 | 100.0% | 12,296 | 6,152 |
Analysis: quantization guide.
Filtered search#
Rows returned for LIMIT 10 with a WHERE filter (HNSW index)
100K rows, ef_search 40, no B-tree on the filter column. Without iterative scans, selective filters starve the result.
- iterative_scan = off
- iterative_scan = relaxed_order
| Rows matching | Mode | Avg rows | Recall@10 | p50 |
|---|---|---|---|---|
| 50% | off | 10.00 | 0.939 | 2.9 ms |
| 50% | relaxed_order | 10.00 | 0.939 | 2.9 ms |
| 10% | off | 3.99 | 0.398 | 2.7 ms |
| 10% | relaxed_order | 10.00 | 0.946 | 6.3 ms |
| 1% | off | 0.39 | 0.039 | 2.7 ms |
| 1% | relaxed_order | 10.00 | 0.967 | 37 ms |
| 0.1% | off | 0.03 | 0.003 | 2.6 ms |
| 0.1% | relaxed_order | 10.00 | 0.791 | 69 ms |
Recall@10 with filters: four strategies
100K rows, HNSW ef_search 40, LIMIT 10. The planner's choice switched to a B-tree + exact sort at 10% and below.
- HNSW, iterative_scan off
- HNSW, relaxed_order
- HNSW, strict_order
- Planner's choice (B-tree present)
| Rows matching | Strategy | Recall@10 | Avg rows | p50 | p95 |
|---|---|---|---|---|---|
| 50% | HNSW, iterative_scan off | 0.939 | 10.00 | 2.9 ms | 4.3 ms |
| 50% | HNSW, relaxed_order | 0.939 | 10.00 | 2.9 ms | 4.4 ms |
| 50% | HNSW, strict_order | 0.939 | 10.00 | 2.9 ms | 5.5 ms |
| 50% | Planner's choice (B-tree present) | 0.939 | 10.00 | 2.9 ms | 4.1 ms |
| 10% | HNSW, iterative_scan off | 0.398 | 3.99 | 2.7 ms | 4.1 ms |
| 10% | HNSW, relaxed_order | 0.946 | 10.00 | 6.3 ms | 9.7 ms |
| 10% | HNSW, strict_order | 0.899 | 10.00 | 6.2 ms | 10 ms |
| 10% | Planner's choice (B-tree present) | 1.000 | 10.00 | 58 ms | 77 ms |
| 1% | HNSW, iterative_scan off | 0.039 | 0.39 | 2.7 ms | 4.1 ms |
| 1% | HNSW, relaxed_order | 0.967 | 10.00 | 37 ms | 59 ms |
| 1% | HNSW, strict_order | 0.901 | 10.00 | 36 ms | 61 ms |
| 1% | Planner's choice (B-tree present) | 1.000 | 10.00 | 6.3 ms | 12 ms |
| 0.1% | HNSW, iterative_scan off | 0.003 | 0.03 | 2.6 ms | 4.0 ms |
| 0.1% | HNSW, relaxed_order | 0.791 | 10.00 | 69 ms | 88 ms |
| 0.1% | HNSW, strict_order | 0.773 | 10.00 | 70 ms | 86 ms |
| 0.1% | Planner's choice (B-tree present) | 1.000 | 10.00 | 0.82 ms | 1.1 ms |
Analysis: filtering guide.
pgvector vs FAISS#
pgvector HNSW vs FAISS HNSW, same data and parameters
100K × 1,536 dims, M 16, efConstruction 64; points are ef_search 10 → 400. pgvector latency includes SQL parsing, planning and the client round trip; FAISS is an in-process library call.
- pgvector 0.8.7 (SQL, local socket)
- FAISS 1.15.1 (in-process)
| ef_search | pgvector recall | pgvector p50 | FAISS recall | FAISS p50 |
|---|---|---|---|---|
| 10 | 0.815 | 1.4 ms | 0.765 | 0.25 ms |
| 20 | 0.897 | 1.9 ms | 0.871 | 0.35 ms |
| 40 | 0.947 | 2.9 ms | 0.937 | 0.60 ms |
| 64 | 0.970 | 3.9 ms | 0.964 | 0.85 ms |
| 100 | 0.983 | 5.3 ms | 0.977 | 1.1 ms |
| 200 | 0.990 | 9.1 ms | 0.989 | 2.0 ms |
| 400 | 0.995 | 16 ms | 0.994 | 3.7 ms |
Analysis: comparison.
Raw data and scripts#
Everything on this page is generated from these files. Each JSON file includes the environment it was measured in.
| Experiment | Results | Script |
|---|---|---|
| HNSW parameter sweep (100K) | hnsw_100k.json | e1_hnsw.py, e1b_topup.py |
| Scaling 100K → 500K | scale.json | e2_scale.py |
| IVFFlat lists / probes | ivfflat.json | e3_ivfflat.py |
| halfvec and binary quantization | quantization.json | e4_quant.py |
| 3,072 dimensions and shortening | dims3072.json | e5_3072.py |
| Filtering and iterative scans | filtering.json | e6_filter.py, e6b_forced.py |
| FAISS comparison | faiss.json | e7_faiss.py |
| Build memory and parallelism | build.json | e8_build.py, e8b_repeat.py, e9_scale_build.py |
| Hybrid search (full-text + vector) | hybrid.json | h1_hybrid.py, h2_queries.py |
| Home-page embedding map | embedding-map.json | m1_map.py |
| Shared helpers, data preparation | — | common.py, prep.py, e0_gt.py |
To reproduce: download the two Hugging Face datasets, run prep.py and e0_gt.py, create a database called bench with CREATE EXTENSION vector, then run each experiment script. Found a flaw in the method? Tell us.