Gaps: where HMS loses or ties¶
Generated with the table from the committed results; a gap stays here until a measured change closes it. Factor is HMS advantage (below 1 = HMS worse). Hypotheses are unmeasured unless a file is cited.
| Status | Dataset | Hardware class | Metric | Comparator | HMS | Comparator value | Factor [interval] | Hypothesis | Code path |
|---|---|---|---|---|---|---|---|---|---|
| LOSS | sift-128-euclidean (1,000,000 vectors, 10,000 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Build time | hnswlib | 312.6 s | 182.5 | 0.58x [0.58, 0.59] | graph construction cost: parallel build with a cheaper construction search (quantized_search) and a smaller build_ef, at a recall cost to be measured; for text, a lexical-only ingest path that skips dense-slot encoding | src/core/qgraph (Graph::build_owned, VGraph::from_graph_with) |
| LOSS | sift-128-euclidean (1,000,000 vectors, 10,000 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, build | hnswlib | 2968 MiB | 1525 | 0.51x [0.51, 0.52] | stream the build instead of holding the f32 matrix plus graph (ledger 1A target: peak build RSS <= 2x final index bytes) | src/core/qgraph/vertex.rs (VGraph search/refine), src/bin/public-bench.rs (ann-qgraph) |
| LOSS | sift-128-euclidean (1,000,000 vectors, 10,000 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | RSS, serving | hnswlib | 2204 MiB | 1019 | 0.46x [0.46, 0.81] | persist the index memory-mapped and drop the build-time allocations; for text the serving V40 RSS campaign in the ledger (2A) is the open lever | src/core/qgraph/vertex.rs (index arrays), build-time allocations in Graph::build_owned |
| LOSS | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Ingest/build time | SQLite FTS5 | 2.156 s | 1.181 | 0.55x [0.54, 0.55] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | On-disk size | Apache Lucene 9.12.3 (JVM) | 20.56 MiB | 12.17 | 0.59x (no CI) | compact postings and file-backed text (ledger 2B/2C) and removing preallocated arena slack | src/core/storage.rs (arena preallocation), src/core/documents/postings.rs |
| LOSS | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Cold start to first answer | tantivy | 0.8406 s | 0.0249 | 0.03x [0.03, 0.04] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Install size | SQLite FTS5 | 2.491 MiB | 0 | 0.00x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | nDCG@10 | SQLite FTS5 | 0.2654 nDCG | 0.2678 | 0.99x [0.69, 1.42] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | recall@100 | SQLite FTS5 | 0.5422 recall | 0.5546 | 0.98x [0.80, 1.20] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, build | SQLite FTS5 | 2480 MiB | 2480 | 1.00x (no CI) | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:fiqa (57,600 documents, 500 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, serving | SQLite FTS5 | 2480 MiB | 2480 | 1.00x (no CI) | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Ingest/build time | SQLite FTS5 | 2.515 s | 2.061 | 0.82x [0.79, 0.85] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Peak RSS, build | SQLite FTS5 | 132.8 MiB | 120.5 | 0.91x [0.91, 0.91] | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Peak RSS, serving | SQLite FTS5 | 65.91 MiB | 25.83 | 0.39x [0.38, 0.41] | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Cold start to first answer | tantivy | 0.6196 s | 0.07182 | 0.12x [0.10, 0.13] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Delete one document (durable) | Apache Lucene 9.12.3 (JVM) | 3.162 ms | 1.482 | 0.47x (no CI) | batching commits (memorize_batch) or a group commit; the per-op figure is one durable commit | src/core/engine/mod.rs commit (one durable transaction per call), src/core/durable_file.rs |
| LOSS | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | Install size | Whoosh | 2.376 MiB | 1.973 | 0.83x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | nDCG@10 | Apache Lucene 9.12.3 (JVM) | 0.4561 nDCG | 0.4627 | 0.99x [0.59, 1.71] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | recall@100 | SQLite FTS5 | 0.8203 recall | 0.8392 | 0.98x [0.78, 1.24] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | QPS (single thread) | tantivy | 2225 QPS | 2399 | 0.93x [0.83, 1.15] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | p50 latency | tantivy | 425 us | 399.6 | 0.94x [0.85, 1.12] | none identified; the median is within noise of the leader | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | modal-x86_64-8vcpu-32g | p99 latency | tantivy | 785.9 us | 733.1 | 0.93x [0.60, 1.19] | tail queries are the long ones; MaxScore-style early termination for long queries (the ledger's impact-index work, category 1C) is the candidate | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Ingest/build time | SQLite FTS5 | 1.214 s | 0.8816 | 0.73x [0.68, 0.75] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Cold start to first answer | tantivy | 0.3017 s | 0.02264 | 0.08x [0.06, 0.08] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Install size | SQLite FTS5 | 2.491 MiB | 0 | 0.00x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | nDCG@10 | Apache Lucene 9.12.3 (JVM) | 0.4561 nDCG | 0.4627 | 0.99x [0.59, 1.71] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | recall@100 | SQLite FTS5 | 0.8203 recall | 0.8392 | 0.98x [0.78, 1.24] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, build | SQLite FTS5 | 2480 MiB | 2480 | 1.00x (no CI) | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | memory (4,559 documents, 100 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, serving | SQLite FTS5 | 2480 MiB | 2480 | 1.00x (no CI) | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Ingest/build time | SQLite FTS5 | 0.2308 s | 0.1354 | 0.59x [0.57, 0.62] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | On-disk size | Apache Lucene 9.12.3 (JVM) | 2.063 MiB | 1.487 | 0.72x (no CI) | compact postings and file-backed text (ledger 2B/2C) and removing preallocated arena slack | src/core/storage.rs (arena preallocation), src/core/documents/postings.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Cold start to first answer | tantivy | 0.0889 s | 0.02018 | 0.23x [0.21, 0.31] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Install size | SQLite FTS5 | 2.491 MiB | 0 | 0.00x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | nDCG@10 | SQLite FTS5 | 0.2916 nDCG | 0.2914 | 1.00x [0.70, 1.44] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | recall@100 | Apache Lucene 9.12.3 (JVM) | 0.2274 recall | 0.2276 | 1.00x [0.66, 1.50] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, build | SQLite FTS5 | 747.8 MiB | 747.8 | 1.00x (no CI) | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, serving | SQLite FTS5 | 747.8 MiB | 747.8 | 1.00x (no CI) | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | Ingest/build time | SQLite FTS5 | 0.5922 s | 0.2858 | 0.48x [0.47, 0.49] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | Peak RSS, build | SQLite FTS5 | 57.3 MiB | 33.82 | 0.59x [0.58, 0.60] | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | Peak RSS, serving | SQLite FTS5 | 56.79 MiB | 29.93 | 0.53x [0.50, 0.54] | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | On-disk size | Apache Lucene 9.12.3 (JVM) | 2.063 MiB | 1.487 | 0.72x (no CI) | compact postings and file-backed text (ledger 2B/2C) and removing preallocated arena slack | src/core/storage.rs (arena preallocation), src/core/documents/postings.rs |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | Cold start to first answer | tantivy | 0.1754 s | 0.06822 | 0.39x [0.35, 0.43] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | Install size | Whoosh | 2.376 MiB | 1.973 | 0.83x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | nDCG@10 | SQLite FTS5 | 0.2916 nDCG | 0.2914 | 1.00x [0.70, 1.44] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | recall@100 | Apache Lucene 9.12.3 (JVM) | 0.2274 recall | 0.2276 | 1.00x [0.66, 1.50] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | QPS (single thread) | tantivy | 9532 QPS | 7105 | 1.34x [0.92, 1.69] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:nfcorpus (3,633 documents, 324 queries) | modal-x86_64-8vcpu-32g | p99 latency | tantivy | 298.2 us | 300.3 | 1.01x [0.43, 1.33] | tail queries are the long ones; MaxScore-style early termination for long queries (the ledger's impact-index work, category 1C) is the candidate | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Ingest/build time | SQLite FTS5 | 0.312 s | 0.1964 | 0.63x [0.61, 0.71] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | On-disk size | Apache Lucene 9.12.3 (JVM) | 2.91 MiB | 1.987 | 0.68x (no CI) | compact postings and file-backed text (ledger 2B/2C) and removing preallocated arena slack | src/core/storage.rs (arena preallocation), src/core/documents/postings.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Cold start to first answer | tantivy | 0.1174 s | 0.02462 | 0.21x [0.20, 0.22] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Install size | SQLite FTS5 | 2.491 MiB | 0 | 0.00x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | nDCG@10 | Apache Lucene 9.12.3 (JVM) | 0.7027 nDCG | 0.7022 | 1.00x [0.90, 1.12] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | recall@100 | SQLite FTS5 | 0.9264 recall | 0.9322 | 0.99x [0.94, 1.05] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, build | SQLite FTS5 | 73.18 MiB | 73.18 | 1.00x (no CI) | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:scifact (5,183 documents, 809 queries) | linux-x86_64-intel-ultra-9-185h-22c-27g | Peak RSS, serving | tantivy | 195.8 MiB | 173.7 | 0.89x [0.26, 2.75] | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | QPS (single thread) | tantivy | 2711 QPS | 2879 | 0.94x [0.62, 1.00] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | p50 latency | tantivy | 374 us | 337.6 | 0.90x [0.69, 0.96] | none identified; the median is within noise of the leader | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | Ingest/build time | SQLite FTS5 | 1.419 s | 0.4272 | 0.30x [0.28, 0.32] | unprofiled hypothesis: per-document text encoding in documents::prepare (src/core/encoding.rs) dominates ingest; profile it, then a lexical-only ingest path or a cheaper encoder is the lever | src/core/documents.rs prepare + src/core/engine/document_api.rs memorize_documents |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | Peak RSS, build | SQLite FTS5 | 65.45 MiB | 39.21 | 0.60x [0.59, 0.61] | unprofiled hypothesis: the arena and index are built in memory before commit; streaming commits per batch | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | Peak RSS, serving | SQLite FTS5 | 159.7 MiB | 54.49 | 0.34x [0.34, 0.35] | the whole store is resident after open; memory-mapped postings and file-backed text (ledger 2A/2B) are the open levers | src/core/storage.rs (arena), src/core/documents/index.rs, postings.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | On-disk size | Apache Lucene 9.12.3 (JVM) | 2.91 MiB | 1.987 | 0.68x (no CI) | compact postings and file-backed text (ledger 2B/2C) and removing preallocated arena slack | src/core/storage.rs (arena preallocation), src/core/documents/postings.rs |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | Cold start to first answer | SQLite FTS5 | 0.2393 s | 0.07661 | 0.32x [0.27, 0.37] | the store is read into memory at open (open_s in the raw file); lazy or memory-mapped loading of the arena and postings is the lever | src/core/storage.rs and src/core/engine/mod.rs (open: arena and document map load) |
| LOSS | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | Install size | Whoosh | 2.376 MiB | 1.973 | 0.83x (no CI) | strip symbols, split optional features out of the default binary | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | nDCG@10 | Apache Lucene 9.12.3 (JVM) | 0.7027 nDCG | 0.7022 | 1.00x [0.90, 1.12] | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs | |
| TIE | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | recall@100 | SQLite FTS5 | 0.9264 recall | 0.9322 | 0.99x [0.94, 1.05] | none: a rounding-level difference in analyzer details (stemmer, stop words), not a lever | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |
| TIE | beir:scifact (5,183 documents, 809 queries) | modal-x86_64-8vcpu-32g | p99 latency | tantivy | 526.3 us | 520 | 0.99x [0.43, 1.18] | tail queries are the long ones; MaxScore-style early termination for long queries (the ledger's impact-index work, category 1C) is the candidate | src/core/documents.rs (prepare, search), src/core/documents/search.rs, src/core/engine/document_api.rs |