Local embeddings¶

Semantic search with no API key, no service and no manual model setup. A pure-Rust CPU engine runs a sentence encoder (BERT, RoBERTa or XLM-R family) with int8 or f16 weights; the tokenizer ships inside the model file; nothing touches the network at inference.

Everything here is behind cargo features that are off by default: the default build, its dependencies and its serving memory are unchanged (see Default build).

The Python wheel and the Node package are built with both.

Use¶

Rust:

use holographic_memory::core::embed::{Embedder, EmbedOptions, SemanticIndex};
use std::sync::Arc;

let embedder = Arc::new(Embedder::new(&EmbedOptions::default())?);   // default model
let mut index = SemanticIndex::new(embedder);
index.add(&["a", "b"], &["The cat sat on the mat.", "Quarterly revenue grew."])?;
let hits = index.search("feline on a rug", 3)?;                     // [{id, score, text}]

Python (pip install holographic-vsa, or holographic-vsa[embed] for offline model packs):

import holographic_memory as hm
index = hm.SemanticIndex()                       # default model
index.add(["a", "b"], ["The cat sat on the mat.", "Quarterly revenue grew."])
index.search("feline on a rug", k=3)             # [(id, score, text), ...]
hm.Embedder(threads=4, max_length=256).embed_query("what is HMS?")   # list[float]

Node (npm i holographic-memory; npm i holographic-memory-models adds offline packs):

const { LocalEmbedder, SemanticIndex } = require('holographic-memory')
const embedder = await LocalEmbedder.load()
const index = SemanticIndex.create(embedder)
await index.add(['a', 'b'], ['The cat sat on the mat.', 'Quarterly revenue grew.'])
await index.search('feline on a rug', 3)

In the document store¶

One line selects the embedder for ingest and search of a store:

let config = HmsConfig { text_encoder: TextEncoder::LocalEmbedV1, ..HmsConfig::default() };
let core = HmsCore::new(4096, Some(path), Some(config))?;
core.memorize_documents(docs)?;             // chunks are embedded in batches
core.search_documents("feline on a rug", &SearchOptions::default())?;   // hybrid: BM25 + cosine

local-embed-v1 loads the embedder from HmsConfig::local_embed (EmbedOptions, defaults when absent), fills embedding_space from the model, embeds chunks that arrive without embeddings, embeds text queries, and encodes encode_text as a sparse sketch of the embedding. The store schema records model, upstream revision, quantization and dimension, so a store opened with a different model is rejected. Existing encoders (multiscale-words-v1, multiscale-capped-union-v1) and their stores are untouched; to add dense vectors to a lexical store, install an embedder with HmsCore::set_text_embedder (the store's embedding_space must equal Embedder::space()). Callers that pass their own embeddings / embedding are never overridden.

Options¶

All fields of EmbedOptions (camelCase in JSON / Node, keyword arguments in Python) are optional.

Determinism: output is bit-identical for any threads, any batchSize and any batch composition on one build (no padding; each output element has one fixed reduction order). Across CPU architectures the last bits can differ because aarch64 and FMA-enabled x86-64 builds use fused multiply-add. Memory: weights plus at most maxBatchTokens of activations (roughly 25 KB per token for 384-wide models); inputs are cut to maxLength * 32 bytes before tokenization.

Model packs, hashes and air-gapped installs¶

A pack is one file (<id>-int8.hmsemb: header, tokenizer, quantized tensors). The registry in src/core/embed/registry.rs pins, per model id, the upstream commit, the sha256 and size of each upstream file, and the sha256 of the converted pack. Resolution order for a registry model:

1. HMS_EMBED_PACK (path of the pack file)
2. modelDir option / HMS_EMBED_MODEL_DIR (directory containing the pack)
3. the cache (HMS_EMBED_CACHE, else the platform cache directory)
4. local-embed-download builds only, and never when offline: download the pinned upstream files from https://huggingface.co (override the host with HMS_EMBED_MIRROR), verify every file's size and sha256, convert to a pack, verify the pack's sha256, then rename into the cache.

Every load re-hashes exactly the bytes it parses; a mismatch fails before any weight is used (Sha256Mismatch). A damaged cache entry is replaced by a fresh verified download when allowed; a file you named (path, env, model directory) is never modified. An unreachable model fails with the exact variables to set. Air-gapped:

hms-embed fetch default --out ./packs            # on a connected machine
export HMS_EMBED_MODEL_DIR=./packs HMS_EMBED_OFFLINE=1   # on the offline one

pip install holographic-vsa[embed] installs holographic-vsa-models, a wheel carrying the packs (the Python Embedder finds it automatically); holographic-memory-models does the same for Node (index.js points HMS_EMBED_MODEL_DIR at it). Both are built by bindings/embed-models/python/build_packs.sh from the pinned upstream files, so their contents are the registry's, hash for hash. hms-embed pack converts any local BERT-family Hugging Face directory.

Why not bundle weights in the Rust crate: crates.io caps a crate at 10 MB and the smallest good int8 pack is about 25 MB. Python wheels (100 MB limit) and npm can carry them, so they do.

Model selection and results¶

See SELECTION.md (written from benchmarks/results/embedding_selection_v1.json) for the candidates, the splits, the engine-versus-reference agreement, and the retrieval gain, cost and throughput measured against the lexical default (benchmarks/results/embedding_store_v1.json). Reproduce with benchmarks/embedding/ (select_model.py, run_engine_eval.py, agreement.py, run_store_eval.py).

Default build¶

default = []. Without local-embed* no new dependency is compiled (cargo tree identical), no embedder code is linked, and the only changes to default code paths are one enum variant (TextEncoder::LocalEmbedV1, rejected by config validation unless the feature is on) and one branch in document preparation that default encoders never take.