Semantic vectors

Semantic retrieval is live on Bun with SQLite: sqlite-vec stores and searches the vectors, and any OpenAI-compatible endpoint produces them. Titen never ships a model.

Turn it on

Vector settings come from the environment, not from flags: one is a credential, and the rest are deployment facts that belong in a unit file.

[Service]
Environment=TITEN_EMBED_BASE_URL=http://127.0.0.1:11434/v1
Environment=TITEN_EMBED_MODEL=bge-m3
Environment=TITEN_EMBED_DIMS=1024
Environment=TITEN_VEC_DB_PATH=/var/lib/titen/vectors.db

All three required values must be present or the capability is not built at all and /readyz reports vector: "disabled". Partial configuration never half-enables retrieval. The prebuilt extension is per-platform: a platform without one degrades to FTS instead of refusing to start.

The index is an index

Vectors live in a separate database file from canonical records, and that is the load-bearing invariant rather than tidiness. Lose vectors.db and no evidence is lost — a fresh index file is created on the next start. Lose the canonical database and nothing brings the evidence back.

Only retrievable claims get embedded

The semantic unit is a retrievable claim — active or disputed — not every raw chat turn. Claims are compact, typed, temporal and linked to evidence; embedding raw turns inflates the index and raises the odds that stray instruction-shaped text dominates similarity.

An observation append and a consolidation each queue a row in index_outbox. The drain embeds one batch, writes the index, and only then marks rows done, so a model outage leaves them queued instead of dropping them. Non-claim rows are retired immediately so the queue cannot grow without bound, and so is a claim that was superseded, revoked or expired since it was queued — it is no longer retrievable.

curl -sX POST -H "authorization: Bearer $TITEN_API_KEY" \
  'http://127.0.0.1:8787/v1/index/drain?limit=50'

The response reports drained, indexed, skipped, remaining, plus the model and dimensions it used. On a deployment with an in-process maintenance timer you never call this — see background repair.

Hybrid ranking, and why normalization matters

The index contributes recall, not only re-ranking: it runs even when lexical search matched nothing, because the memory worth retrieving is often phrased nothing like the question. A semantic score then competes with the lexical one as relevance — max(lexical, semantic) — rather than becoming a sixth weighted term. With no vector capability the value is zero and ranking is arithmetically identical to lexical-only.

Both signals are normalized inside the candidate set before they compete, because real embedding models return cosines in a narrow absolute band and it is the spread that carries the signal. Against embeddinggemma, three claims spanned 0.437 to 0.463; passing those raw left an exactly-matching claim ranked last, because the confidence multiplier swung the composite far more than a 0.026 difference could.

What degrades, and how you see it

Compile never fails because a model failed. It degrades and says so in the same response:

"degraded": { "semantic": false, "vector": "used", "model": "enabled" }

vector is "used" when the index answered, "error" when the call failed, and "disabled" when no capability is configured.

/readyz reports the store and the model as separate facts, because a deployment can have an endpoint with no working index or an index with no reachable endpoint, and a single merged flag would hide both.

Next