@memhtml/index

View as Markdown Open in ChatGPT Open in Claude Open in Claude Code Open in Cursor

@memhtml/index is the SQLite layer under memhtml: the schema and its migrations, the indexer that projects the git tree into index.db, four-arm retrieval over that index, the state plane attached beside it as state.db, and the persistence half of the trace plane. It runs on Node’s own node:sqlite and on effect, and it imports three workspace packages: @memhtml/contracts for paths, types, edge classes and errors, @memhtml/domain for the ranking math, and @memhtml/html for the memory file parser. Its manifest also lists @memhtml/llm, but no source file imports it, because an embedder reaches this package as a port typed by its shape. The CLI composes the database, indexer, retrieval and recorder services from this package, the MCP server and the CLI both call its parseFacetFilters, and the eval package runs the real indexer and retrieval over a generated corpus.

makeDatabase opens one connection for a scope and closes it when the scope ends. It turns on WAL mode, synchronous = NORMAL and foreign keys, waits up to five seconds for the write lock, and registers the SQL function vector_distance_cos, which calls @memhtml/domain’s cosineDistance rather than a second copy of that arithmetic. It then applies each pending .sql file in the migrations directory it is given (MIGRATIONS_DIR for the shipped schema) in filename order, each file in one transaction with its ledger row. Given a state path, it calls attachState, which attaches state.db under the schema name STATE_SCHEMA and runs runStateMigrations over the state migrations directory it is given (STATE_MIGRATIONS_DIR for the shipped one) against a separate ledger, so deleting index.db never marks a state migration as unapplied.

DatabaseService is the tag for the DatabaseShape that makeDatabase returns: run, get and all for one statement, writeAll for a list of Write values bound with SqlValue parameters, script for a whole SQL script, the two migration counts, and hasState. writeAll and script each run in one transaction, so one call commits whole or not at all. A call that meets SQLITE_BUSY is retried with jittered exponential backoff on a schedule capped at 20 seconds, and a failure reaches the caller as a StorageFailure that names only the operation, so SQL parameters and memory text stay out of the error.

The SQL lives in migrations/ for index.db and in state-migrations/ for state.db, and schema-const.ts names the identifiers both sides use. files holds one row per memory file. Its child tables for tags, entities, facets and citations cascade on update as well as on delete, because an archive move changes the primary key. chunks and embeddings key on the content hash, not on the path. CHECK constraints keep each relation in edges inside its class. The partial unique index files_content_hash_active refuses a second active, non-task row with the same content hash, so within one index.db the database itself enforces dedup. FTS_INDEX_NAME is files_fts, an external-content FTS5 table that triggers keep in step with files, over the one column FTS_COLUMN, so bm25 ranks one text and fusion does the weighting. INDEX_STATE_ID is the key of the one-row index_state table. MEMORY_TABLES lists, children first, the tables a rebuild empties, and a rebuild deletes rows from those tables only, so the trace tables in TRACE_TABLES and the state plane keep theirs. STATE_TABLES names access and edge_corroboration. CHUNK_MAX_CHARS (1,800), SNIPPET_MAX_CHARS (700) and WRITE_BATCH_SIZE (500 statements per transaction) are the size limits.

IndexStateRow is the schema of the index_state row: head_sha, the commit the index describes, then embed_model and embed_dim, the vector space as <model-id>@<dim> and as a number, then two timestamps. readIndexState selects the schema’s own columns by name and decodes them. It returns undefined when there is no row and turns a malformed row into a StorageFailure. A row whose head_sha is NULL belongs to a rebuild that has not finished.

GitPort is the indexer’s view of git, behind the IndexGit tag: revParseHead, lsTreeR, catFileBatch, diffNameStatus, statusPorcelainV2, hashObject and readWorkingFile, which return TreeEntry, DiffEntry and StatusEntry rows. It is declared here, so the indexer depends on a shape and not on @memhtml/store, and its tag differs from the store’s memhtml/Git because the two shapes differ. makeGitPort builds the port from the store’s git client, which StoreGitShape, StoreTreeEntry, StoreChangedPath and StoreStatusEntry describe, plus the file reader and failure builder in GitAdapterDeps. It fails on a HEAD with no commit, decodes blobs as UTF-8, and drops tree entries that are not blobs, such as submodules. It maps a copy to an add, never to a rename, because the source of a copy still exists. It drops ignored and unmerged status entries and keeps the source path of a staged rename, so the indexer can retire the old row in the same pass. Every rejection, a defect included, is logged with its git operation and returned as the caller’s typed failure, so the returned failure carries none of git’s own error output.

makeIndexer takes IndexerDeps, which are the database, the git port, the configured embedWatermark and embedDim, an optional EmbedPort, an optional embedPersistEvery slice size and a clock, and returns the IndexerShape behind the Indexer tag.

rebuild reads the tree at HEAD under TREE_PREFIXES with one ls-tree and one cat-file --batch, and keeps each path isIndexablePath accepts: a .html file under projects, areas, resources or archive whose name is not in GENERATED_NAMES. It changes the database only after those reads. In one transaction it clears head_sha, copies the vectors in the configured space to a temporary table, and empties the MEMORY_TABLES. It then projects every file, puts back each saved vector whose chunk id came back, and records the commit. A rebuild that stops after that transaction commits and before it records the commit leaves head_sha NULL, and update refuses that state with IndexStale. A file that fails to parse is listed as skipped in the RebuildReport, and the rest of the tree is still indexed.

update diffs the recorded commit against HEAD with git diff --name-status -M and then adds the uncommitted changes from git status --porcelain=v2. Committed changes apply first, so a path ends at its working-tree content. A rename that stays inside the indexed tree is an UPDATE of the row’s path that the child tables’ ON UPDATE CASCADE carries along, so a git mv keeps the file’s chunks and vectors. When there is no index_state row at all, update runs a full rebuild instead. Its writes go in batches of WRITE_BATCH_SIZE, each its own transaction, and the recorded commit moves only after the last batch, so after a failed update the next one diffs from the same commit. UpdateReport counts the files added, modified, removed, renamed and dirty.

The stored vector space must equal embedWatermark. When it differs, update, backfill, embedMissing and any rebuild that cannot write vectors fail with EmbedModelMismatch. A rebuild with embed: true and an embedder bound is the one way past that check: it keeps only vectors already in the configured space, and it records that space before it writes any vector. A rebuild that cannot write vectors, over a store that holds some, fails with RebuildNoEmbedRefused unless force is set. embedMissing embeds the chunks that lack a vector in the configured space, either across all of chunks or only for the ids in EmbedMissingOptions, which is what update passes. It saves each slice, 960 chunks by default, before it embeds the next, so a model failure stops the pass, keeps what landed, and returns the count instead of failing. backfill runs the full scan, or only counts under dryRun, and returns a BackfillReport of what is left. embedderBound says whether an embedder is bound, and vectorCoverage reads the coverage ratio described below.

projectFile turns one parsed MemoryDoc into a FileProjection: the file’s chunks and the Write list for its rows. It is pure, and rebuild and update both call it. It deletes the file’s tags, entities, facets, citations and own authored edges, and only those of its chunks whose content hash is not the new one, so an edit to the head alone keeps the vectors. It then upserts the files row on path from FILE_COLUMNS, the one list behind the column list and the update clause. archived comes from the path, not from memhtml-status, because eviction is the move into archive/. Inbound edges are left alone, because each is another file’s assertion.

ftsTextFor joins the title, gist and body with newlines into the FTS column. disclosureTextFor is the text recall may quote: the <mark> claim, the <summary> headlines, the facets as name: value, and the citations, collected from the whole article. No other text goes in, so ordinary paragraphs and the prose of a <details> body or an <aside> stay out, though a facet or citation inside one is still included. entityRowsFor adds a concept: row for each <dfn> and a lang: row for each data-lang, keeps memhtml-entity names as authored apart from trimming, and files an entity it cannot split into type:name under the type unknown. authoredEdgesFor turns head links into edge rows, strips the leading slash from each href, and drops a link to the file itself. workspaceOf is the directory under projects/, and wordCountOf counts whitespace-separated words.

chunkText splits an article’s text into Chunk values of at most CHUNK_MAX_CHARS UTF-16 code units, with no overlap. An article that fits is one chunk, so its vector describes the whole article. A longer one packs whole sentences greedily and cuts a sentence that is too long, so no text is dropped. chunkIdFor is the sha256 of <content hash>:<ordinal>, so an identical body gets the same chunk ids at any path, and a git mv finds its vectors already stored.

sanitizeFtsQuery, ftsQueryForms and hasFtsTerms turn a caller’s text into a MATCH string that FTS5 accepts. An apostrophe, a type:name reference, a hyphenated word or a bare AND each fails an FTS5 statement outright. So the sanitizer keeps only lowercased runs of Unicode letters and digits, keeps double-quoted spans as phrases when the query’s quotes are balanced, and drops every other character instead of escaping it. FtsQueryForms holds all, the terms joined with AND, and any, the terms joined with OR, and sanitizeFtsQuery returns any. Both are empty when no term survives, and retrieval then drops the lexical arm, because an empty MATCH is an error too.

RANK_ARMS is the registry of RankArm entries that buildRrfSql folds into one statement. fts ranks files_fts by bm25, with weight 1.0. vector ranks each file by the cosine distance of its nearest chunk, with weight 1.0. recency ranks by coalesce(event_at, updated_at), newest first, with weight 0.5. salience reads state.access and ranks by an exponential decay over the hours since the last access, or since updated_at for a file never accessed, plus the log of one plus the access count, plus any positive outcome score, with weight 0.4. Each arm returns (path, rank), and the fold sums weight / (rank + 60) per path and breaks ties by path.

activeArms drops an arm whose input is missing, whether a query vector, the state plane or a query term, as the RrfOptions flags say. PARAM_QUERY, PARAM_ARM_LIMIT, PARAM_FINAL_LIMIT and PARAM_QUERY_VECTOR are the numbered slots ?1 to ?4, so dropping the vector arm moves no other value. Salience leaves out tasks, SALIENCE_EXCLUDED_TYPE, and the person files under SALIENCE_EXCLUDED_PREFIX, resources/people/, and such a row keeps its ranks in the other arms when the scope admits it. buildSnippetSql fetches the chosen files’ chunks, and truncateSnippet cuts one to SNIPPET_MAX_CHARS, ellipsis included.

assembleScope turns a SearchScope into an AssembledScope: one ArmHoles filter string that every arm receives, so a scope cannot reach three arms and miss the fourth, plus the values it binds from ?5 on. By default the filter keeps active rows and leaves out tasks, the type EXCLUDED_BY_DEFAULT names, and naming task in memoryTypes opts in. workspace must match exactly, tags widen the set to any listed tag, and entity is one type:name reference compared with ASCII case folded in SQL. facets are FacetFilter pairs that must all hold across names and widen within one name, which facetConditions builds as one EXISTS per name. includeArchived drops the archived filter, and asOf replaces it with a validity window at that instant, compared as strings. parseFacetFilter and parseFacetFilters read the name=value form, split it at the first FACET_SEPARATOR, and drop a spec with an empty side.

makeRetrieval takes RetrievalDeps, which are the database, an optional QueryEmbedPort, the configured watermark and a coverage floor, and returns the RetrievalShape behind the Retrieval tag. search takes a SearchInput and reads vector coverage first. When coverage is below the floor, no embedder is bound, or the embedder fails, the vector arm is left out and the result says degraded: true. The lexical arm binds the all form when some file in scope holds every term, and the any form otherwise. Each arm’s candidate limit is DEFAULT_ARM_LIMIT (40), re-exported from @memhtml/domain, and the fused list is MMR_POOL_FACTOR times the hit limit, which is DEFAULT_SEARCH_LIMIT (10) unless the caller sets one. A step in polarity.ts then moves a candidate whose negation disagrees with the query below any candidate that agrees with the query and is near-identical to it in vector space, and MMR picks the final hits.

Each SearchHit carries its best chunk as a snippet, its entities as sorted type:name references, and supersededBy, read from an authored supersedes edge. The SearchResult also reports vectorCoverage, the arms that ran, and entityScope. When a narrowing scope finds nothing, scopeEmpty is true, and without asOf the archivedMatches count and its ArchivedMatch rows say what the same scope finds among archived files.

recall takes a RecallInput, runs the same fusion and polarity step over DEFAULT_SEARCH_LIMIT times MMR_POOL_FACTOR candidates without MMR, and folds them into a RecallPack. foldDisclosure walks the candidates in rank order and quotes each DisclosureCandidate’s disclosure text as a DisclosedEntry while it fits the character budget. A candidate that does not fit becomes an IndexLine with its path and claim, and the fold goes on, so a later, shorter memory can still be quoted. MAX_PER_ENTITY (2) caps the full quotes per entity name, and a capped memory still gets its index line. Arcs fold under their own ARC_BODY_BUDGET (9,000) and ordinary memories under the caller’s budgetChars or MEMORY_BODY_BUDGET (16,000), so one synthesis cannot crowd out the memories behind it, and budgetFor picks between the two by type. Each DisclosureFold reports its quotes, its index lines, the characters spent, and whether anything was cut.

readVectorCoverage counts, in one statement, the chunks and the vectors in the configured space, or in the stored space when none is named, and returns a VectorCoverage. An index with no chunks has coverage 1. Retrieval leaves out the vector arm below VECTOR_COVERAGE_FLOOR (0.95) unless RetrievalDeps sets another floor, because when few files carry vectors, a vector rank on top of recency and salience can outscore an exact lexical match. VectorCoverageLow is a typed failure for coverage under a given floor, and VECTOR_COVERAGE_HARD_FLOOR (0.5) is a second, lower floor. A VectorCoverageLow reason names the counts and VECTOR_COVERAGE_REMEDY, plus VECTOR_COVERAGE_NO_VECTORS_REMEDY when the counted space holds no vector. formatCoverage prints a ratio as a percentage with at most one decimal.

reinforce bumps the state.access rows for a list of paths with one conditional upsert per REINFORCE_PATH_BATCH (500) paths. A path is bumped only when it has no recorded access or its last access is at least the cooldown old, 900 seconds by default from @memhtml/domain. SQLite decides that inside the statement and returns the rows it changed, so the ReinforceResult split into bumped and cooledDown holds for every process that calls reinforce on one store. Every bump raises the access count, and a bump whose signal is not neutral also raises the reinforcement count and moves the outcome score. A path’s first row takes the signal’s value, and after that each such bump moves the score by an EWMA with weight OUTCOME_EWMA_ALPHA (0.3), clamped between -1 and 1. Without a state plane it bumps nothing.

The state plane cannot be rebuilt from git, so the CLI’s state export keeps its access rows in a committed file, .memhtml/state/access.jsonl, through the sidecar functions here. accessRows reads every state.access row as an AccessRow, ordered by path. toSidecarEntry turns a row into a SidecarEntry, with the outcome score rounded by round4 to SIDECAR_PRECISION (4) decimals. renderSidecar writes one JSON object per line, so an unchanged table renders the same bytes. parseSidecar reads the file back one line at a time and counts, rather than fails on, a line it cannot parse, so a truncated file still restores the rows it holds. The file carries state.access only.

persistScanned writes one scanned transcript file, a ScannedFileLike, which is this package’s structural copy of the scan result @memhtml/traces produces. A skip writes nothing. A rescan upserts the session’s traces row and replaces its trace_prompts. For a tail, readStoredExtract reads the stored session, when there is one, and the injected TailMerger merges the tail into it before the same write. The row, its prompts and the file’s watermark commit in one transaction, and the PersistOutcome says which path ran. SessionExtractLike, PromptRowLike and WatermarkLike are declared by structure, because @memhtml/traces depends on this package and not the other way round. readWatermark and writeWatermark convert between the scanner’s epoch milliseconds and the table’s ISO text. Retrieval never reads a trace table.

makeIndexRecorder returns the IndexRecorderShape behind the IndexRecorder tag. recordLink adds a SessionLink, whose kind is a LinkKind from LINK_KINDS, to memory_session_links, and an exact repeat adds nothing. activePathForHash is the dedup lookup the CLI hands to the store, with the same memory_type <> 'task' exclusion as the unique index it mirrors. activeFramesFor answers a whole set of frame keys in one query with the active non-task claims on each key, as FrameMatch rows. activeNearestFor reads the active memories’ first-chunk vectors once and returns, for each query vector, at most k NearMatch rows at or above a cosine floor, best first.

The package publishes one import path, @memhtml/index, and its index.ts re-exports a chosen set of names, so the reference below is that set. A name a module exports that index.ts does not re-export is left off the page: all of polarity.ts, and helpers such as rrfParams, buildFtsProbeSql, isBusyCause, toDiffEntry and toStatusEntry.

Classes

EmbedModelMismatch

The stored vector space disagrees with the configured one.

A hard failure rather than a silent reindex. A half-migrated vector space degrades every cosine while every test still passes, because each vector is well-formed. memhtml index rebuild --embed is the one path allowed past this guard: on a model change its truncate carries no vector across (the stash in truncateForRebuild keeps only rows in the CONFIGURED space, and none are), and it records the configured model before any vector is written, so it migrates the whole space rather than mixing two.

Constructors

Constructor

1
new EmbedModelMismatch(stored, configured): EmbedModelMismatch;

Parameters

stored

string

configured

string

Returns

EmbedModelMismatch

Properties

_tag

1
readonly _tag: "EmbedModelMismatch" = "EmbedModelMismatch";

configured

1
readonly configured: string;

stored

1
readonly stored: string;

IndexStale

The index is a partial state update cannot diff from: a rebuild emptied the tables and did not finish repopulating them.

A rebuild clears index_state.head_sha in the same transaction as its truncate and writes the commit back only after every projection landed and the preserved vectors were re-inserted, so an EXISTING row whose head_sha is NULL is exactly a rebuild that died inside that window. That is a different state from no row at all, which is a store nothing has indexed yet and which update answers with a full rebuild. Here the tables are half-empty and there is no watermark to diff from, so a diff would report unchanged: true over rows the tree still holds. The CLI maps this tag to ERR_INDEX_STALE. The vector stash lives in the temp schema, so a process that dies in the window takes it along; the rebuild that recovers the window re-embeds, which costs model calls and loses nothing.

Constructors

Constructor

1
new IndexStale(reason): IndexStale;

Parameters

reason

string

Returns

IndexStale

Properties

_tag

1
readonly _tag: "IndexStale" = "IndexStale";

reason

1
readonly reason: string;

RebuildNoEmbedRefused

rebuild --no-embed asked over a store that carries vectors in the configured space, without --force.

--no-embed is the harness flag: every credential-free test run and every fresh install without a Bedrock credential passes it. It was invoked against a live store by accident, and ten hours of incremental updates later that store held 183 embeddings under 9,332 chunks, because an update embeds only its own batch’s chunks and nothing revisits the rest. A store that carries vectors is a store somebody embedded on purpose, so the bare call refuses and names the count. The vectors are preserved by the rebuild either way (see truncateForRebuild); what --no-embed costs on a live store is that every new or changed chunk stays unembedded, and --force is the operator saying that is understood. The CLI maps this tag to ERR_REBUILD_NO_EMBED_REFUSED.

Constructors

Constructor

1
new RebuildNoEmbedRefused(
2
   embeddings,
3
   model,
4
   because
5
): RebuildNoEmbedRefused;

Parameters

embeddings

number

How many vectors the store carries, all in model.

model

string

The configured space, which the guard already proved is the stored one.

because

"flag" | "no-embedder"

Which of the two no-embed spellings was seen: the --no-embed flag, or --embed with no embedder configured (MEMHTML_EMBED=off, the harness’s own variable). Both leave every new or changed chunk without a vector, and the prose names the one the operator can change.

Returns

RebuildNoEmbedRefused

Properties

_tag

1
readonly _tag: "RebuildNoEmbedRefused" = "RebuildNoEmbedRefused";

because

1
readonly because: "flag" | "no-embedder";

embeddings

1
readonly embeddings: number;

How many vectors the store carries, all in model.

model

1
readonly model: string;

The configured space, which the guard already proved is the stored one.

VectorCoverageLow

The vector plane covers too little of the corpus for a vector-driven pass to run.

Raised below VECTOR_COVERAGE_HARD_FLOOR, on the same channel as EmbedModelMismatch and IndexStale: a typed failure a caller reports by _tag and reason, so the report line names the ratio, the counts and the remedy. Defined here beside the reader because the ratio’s meaning is this module’s.

Constructors

Constructor

1
new VectorCoverageLow(coverage, floor): VectorCoverageLow;

Parameters

coverage

VectorCoverage

floor

number

Returns

VectorCoverageLow

Properties

_tag

1
readonly _tag: "VectorCoverageLow" = "VectorCoverageLow";

coverage

1
readonly coverage: VectorCoverage;

floor

1
readonly floor: number;

reason

1
readonly reason: string;

Interfaces

AccessRow

One path’s durable access bookkeeping. Absent from the result means it was not accessed.

Properties

access_count

1
readonly access_count: number;

last_accessed_at

1
readonly last_accessed_at: string | null;

last_reinforced_at

1
readonly last_reinforced_at: string | null;

outcome_score

1
readonly outcome_score: number;

path

1
readonly path: string;

reinforcement_count

1
readonly reinforcement_count: number;

updated_at

1
readonly updated_at: string;

ArchivedMatch

One archived memory the scope of an empty search still matches.

Properties

path

1
readonly path: string;

supersededBy

1
readonly supersededBy: string | null;

ArmHoles

The filter hole an arm’s SQL template carries. Assembled, never bound.

Properties

fileFilter

1
readonly fileFilter: string;

Additional AND conditions on the files row, covering the archived flag, the memory-type IN-list, the workspace equality, the tag overlap, and the entity reference. Columns are written against the literal token {alias}, which each arm replaces with the alias its own files row goes by. One filter string reaching every arm is what stops a scope from applying to three arms and not the fourth. Empty string when the search is unscoped, making the assembled SQL byte-identical to the unfiltered form.

AssembledScope

An assembled filter, holding the SQL fragment and the values that bind to its placeholders.

Properties

holes

1
readonly holes: ArmHoles;

params

1
readonly params: readonly (string | number)[];

Values for ?5 onward, in placeholder order.

BackfillReport

A backfill pass’s outcome, what memhtml index embed answers.

embeddingsRemaining is the honest half: the chunks still without a vector in the configured space after this pass. It is non-zero when no embedder is configured (MEMHTML_EMBED=off), when a slice failed partway, or under dryRun, and it is what an operator compares against chunks to see the size of the gap. The incident this exists for read 183 embeddings against 9,332 chunks.

Properties

chunks

1
readonly chunks: number;

embeddings

1
readonly embeddings: number;

embeddingsRemaining

1
readonly embeddingsRemaining: number;

embeddingsWritten

1
readonly embeddingsWritten: number;

headSha

1
readonly headSha: string | null;

The recorded watermark, or null while an interrupted rebuild’s window is open.

Chunk

One chunk of a memory’s article text.

Properties

charCount

1
readonly charCount: number;

chunkId

1
readonly chunkId: string;

sha256(content_hash + ":" + ordinal). Corpus-wide, so an identical body reuses the vector.

ordinal

1
readonly ordinal: number;

0-based position within THIS file’s chunk sequence.

text

1
readonly text: string;

DatabaseShape

Properties

all

1
readonly all: <A>(sql, params?) => Effect<readonly A[], StorageFailure>;

Type Parameters

A

A

Parameters

sql

string

params?

readonly SqlValue[]

Returns

Effect<readonly A[], StorageFailure>

get

1
readonly get: <A>(sql, params?) => Effect<A | undefined, StorageFailure>;

Type Parameters

A

A

Parameters

sql

string

params?

readonly SqlValue[]

Returns

Effect<A | undefined, StorageFailure>

hasState

1
readonly hasState: boolean;

True when state.access and state.edge_corroboration are reachable from this connection.

migrationsApplied

1
readonly migrationsApplied: number;

run

1
readonly run: (sql, params?) => Effect<void, StorageFailure>;

Parameters

sql

string

params?

readonly SqlValue[]

Returns

Effect<void, StorageFailure>

script

1
readonly script: (sql) => Effect<void, StorageFailure>;

Apply a whole SQL script — many statements, no parameters — in ONE transaction.

This is the migration primitive, exposed because a caller applying migration files must use the same one the runner does. Whether a DROP TABLE’s cascade is contained by that transaction is exactly the kind of fact a per-statement loop would answer differently.

Parameters

sql

string

Returns

Effect<void, StorageFailure>

stateMigrationsApplied

1
readonly stateMigrationsApplied: number;

Migrations recorded in the ATTACHed state plane’s own ledger, or 0 when no state database is attached. Two counters because the planes are versioned independently: index.db is deleted and rebuilt, state.db is not.

writeAll

1
readonly writeAll: (writes) => Effect<void, StorageFailure>;

Applies every write atomically: all commit, or none do.

Parameters

writes

readonly Write[]

Returns

Effect<void, StorageFailure>

DiffEntry

One git diff --name-status -M row.

R carries fromPath. A rename is the archive move (probed: reported as R100), and handling it as a rename rather than a delete-plus-add is what preserves chunks/embeddings. Those key on content_hash, so a git mv costs zero Bedrock calls.

Properties

fromPath?

1
readonly fromPath?: string;

path

1
readonly path: string;

status

1
readonly status: "A" | "M" | "D" | "R";

DisclosedEntry

A fully quoted memory.

Properties

body

1
readonly body: string;

gist

1
readonly gist: string;

memoryType

1
readonly memoryType: string;

path

1
readonly path: string;

title

1
readonly title: string;

DisclosureCandidate

A candidate for disclosure, already ranked.

Properties

disclosureText

1
readonly disclosureText: string;

Article text WITHOUT <details> bodies, so Tier 1 + Tier 2 only. The indexer supplies this separately from body_text so the fold cannot accidentally quote Tier 3.

entityNames

1
readonly entityNames: readonly string[];

Entity names this memory claims, for the per-entity cap.

gist

1
readonly gist: string;

The <mark> claim. The Tier-1 line, and what an overflow entry shows.

memoryType

1
readonly memoryType: string;

path

1
readonly path: string;

title

1
readonly title: string;

DisclosureFold

What one budget produced.

Properties

disclosed

1
readonly disclosed: readonly DisclosedEntry[];

indexLines

1
readonly indexLines: readonly IndexLine[];

spentChars

1
readonly spentChars: number;

Characters of quoted body actually spent. Never exceeds the budget.

truncated

1
readonly truncated: boolean;

True when at least one candidate became an index line instead of a quote.

EmbedMissingOptions

How wide IndexerShape.embedMissing looks for work.

Omitting candidateChunkIds, or omitting the options object entirely, asks for the FULL scan over chunks, which is the only form that can find a chunk whose vector belongs to a superseded model. memhtml index rebuild --embed must therefore never pass a list. The model migration IS the whole-store e.model <> ? disjunct, and scoping it to one batch’s ids would leave every other chunk in the old vector space while reporting success.

Passing a list asks a narrower question, “which of THESE chunks lack a current vector”, which is what an incremental pass actually needs, and what keeps its cost independent of store size.

Properties

candidateChunkIds?

1
readonly candidateChunkIds?: readonly string[];

Restrict the pending scan to these chunk ids. An empty array is a real answer rather than a missing one. It means the caller projected no chunks, so there is no work and no reason to query.

EmbedPort

What the embedder must provide. Structurally @memhtml/llm’s EmbeddingsShape, minus the query half.

Properties

embed

1
readonly embed: (texts) => Effect<readonly Float32Array<ArrayBufferLike>[], ModelUnavailable>;

Parameters

texts

readonly string[]

Returns

Effect<readonly Float32Array<ArrayBufferLike>[], ModelUnavailable>

FacetFilter

One <dl> facet predicate: a <dt> name and one <dd> value beneath it.

A pair rather than a name=value string, because the SQL binds the halves separately and a predicate builder that had to re-split its own input would carry a second copy of the split rule. parseFacetFilters owns the wire spelling, once, for both doors.

Properties

name

1
readonly name: string;

The <dt> text, as the corpus spells it. Matched exactly.

value

1
readonly value: string;

One <dd> text under that name. Matched exactly, as TEXT.

FileProjection

Everything one file contributes to the index.

Properties

chunks

1
readonly chunks: readonly Chunk[];

contentHash

1
readonly contentHash: string;

path

1
readonly path: string;

writes

1
readonly writes: readonly Write[];

FrameMatch

One live claim already occupying a frame key.

gist travels with path because the conflict is only reportable WITH it. Two rows sharing a frame key state the same relation and differ in their VALUES, so a caller handed paths alone would have to re-read every candidate file to say what the disagreement is. One query carries both.

Properties

gist

1
readonly gist: string;

path

1
readonly path: string;

FtsQueryForms

The two MATCH forms one query sanitizes to. Both are "" when the query holds no indexable term, and both are the same string when it holds exactly one term.

all joins the terms with AND and matches only a file holding every one of them. any joins them with OR and is the form that finds anything at all when no file holds every word.

The lexical arm runs all first and falls back to any only when all matches nothing in its scope, and the reason is the FUSION rather than bm25. Measured on the discrimination gate’s fixture (seed 20260802, 304 files, 36 probes): bm25 ranks the all-terms file first under both forms, with the same scores at the top, but under any the arm hands the fold 40 candidates where all hands it 3 or 4. RRF’s 1/(rank + 60) is nearly flat across 40 positions (1/61 against 1/100), so the arm stops acting as a filter, recency and salience decide among the 40, and the target’s lexical lead is compressed away: the any-of form alone holds MRR at 1.0 with zero inversions but drops corpus MRR from 1.0 to 0.28 (0.20 on the lexical floor). All-then-any keeps every number at the all-terms value and still answers the sentence that matches on one word.

Properties

all

1
readonly all: string;

Every term required. oscar AND wants AND talk.

any

1
readonly any: string;

Any term suffices, bm25 ranks. oscar OR wants OR talk.

GitAdapterDeps

How the adapter reads a working-tree file and how it reports a refusal.

Properties

fail

1
readonly fail: (operation) => Effect<never, never, never>;

Builds the port’s typed failure from an operation name. Injected so this module names no error class of its own and the port’s error type stays the caller’s decision.

Parameters

operation

string

Returns

Effect<never, never, never>

git

1
readonly git: StoreGitShape;

readFile

1
readonly readFile: (path) => Effect<string, unknown>;

Reads a repo-relative path as UTF-8 text. The CLI supplies one rooted at MEMHTML_ROOT.

MUST report a missing or unreadable file through its ERROR channel, so Effect.tryPromise and not Effect.promise. A defect propagates past Effect.catch and takes down the fiber, so an absent path would crash the indexer instead of becoming the counted skip it is. An agent listing a path it just archived is the normal case.

Parameters

path

string

Returns

Effect<string, unknown>

GitPort

Properties

catFileBatch

1
readonly catFileBatch: (shas) => Effect<ReadonlyMap<string, string>, StorageFailure>;

Blob contents by sha, streamed through one git cat-file --batch rather than N reads. One subprocess for the whole tree, and it works on a bare or detached checkout where the files are not on disk at all.

Parameters

shas

readonly string[]

Returns

Effect<ReadonlyMap<string, string>, StorageFailure>

diffNameStatus

1
readonly diffNameStatus: (from, to) => Effect<readonly DiffEntry[], StorageFailure>;

Parameters

from

string

to

string

Returns

Effect<readonly DiffEntry[], StorageFailure>

hashObject

1
readonly hashObject: (path) => Effect<string, StorageFailure>;

The blob sha a working-tree file WOULD have. The dirty path’s change key.

Parameters

path

string

Returns

Effect<string, StorageFailure>

lsTreeR

1
readonly lsTreeR: (ref, pathPrefixes) => Effect<readonly TreeEntry[], StorageFailure>;

Every blob under the given path prefixes at ref, in one subprocess.

Parameters

ref

string

pathPrefixes

readonly string[]

Returns

Effect<readonly TreeEntry[], StorageFailure>

readWorkingFile

1
readonly readWorkingFile: (path) => Effect<string, StorageFailure>;

A working-tree file’s bytes as text. Used only for uncommitted paths.

Parameters

path

string

Returns

Effect<string, StorageFailure>

revParseHead

1
readonly revParseHead: () => Effect<string, StorageFailure>;

The commit the working tree is on.

Returns

Effect<string, StorageFailure>

statusPorcelainV2

1
readonly statusPorcelainV2: () => Effect<readonly StatusEntry[], StorageFailure>;

Returns

Effect<readonly StatusEntry[], StorageFailure>

IndexerDeps

How an indexer is built. embedWatermark is @memhtml/llm’s EMBED_WATERMARK, never re-derived.

Properties

db

1
readonly db: DatabaseShape;

embedDim

1
readonly embedDim: number;

embeddings?

1
readonly embeddings?: EmbedPort;

embedPersistEvery?

1
readonly embedPersistEvery?: number;

How many pending chunks are embedded and PERSISTED per embedMissing slice. Defaults to EMBED_PERSIST_SLICE. Injectable so a test can force several slices over a small corpus; production callers leave it alone.

embedWatermark

1
readonly embedWatermark: string;

<model-id>@<dim>. Passed in so this package never names a model or a dimension.

git

1
readonly git: GitPort;

now

1
readonly now: () => string;

Wall-clock, injected. A fixed instant makes indexed_at assertable.

Returns

string

IndexerShape

Properties

backfill

1
readonly backfill: (opts) => Effect<BackfillReport,
2
  | StorageFailure
3
| EmbedModelMismatch>;

Fill every chunk that has no vector in the configured space, over the WHOLE chunks table, in persisted slices, and report the gap that is left. The recovery from a sparse vector plane that does not empty the index first. dryRun reports the gap and writes nothing.

Parameters

opts

dryRun

boolean

Returns

Effect<BackfillReport, | StorageFailure | EmbedModelMismatch>

embedderBound

1
readonly embedderBound: boolean;

True when a document embedder is bound, so embed: true on rebuild and update writes vectors.

What doctor reads to tell a deliberate lexical-only store (no embedder, zero vectors, healthy) from a store whose vector plane is IN USE and sparse. The indexer is the one holder of the embedder on that side of the composition root, so the fact is published here rather than threaded through a second dependency.

embedMissing

1
readonly embedMissing: (options?) => Effect<number,
2
  | StorageFailure
3
| EmbedModelMismatch>;

Fill vectors for chunks that have none, or whose vector belongs to another model.

Called with no argument this scans the whole chunks table. See EmbedMissingOptions for when that is required and when a candidate list is the cheaper, equivalent question.

Parameters

options?

EmbedMissingOptions

Returns

Effect<number, | StorageFailure | EmbedModelMismatch>

rebuild

1
readonly rebuild: (opts) => Effect<RebuildReport,
2
  | StorageFailure
3
  | EmbedModelMismatch
4
| RebuildNoEmbedRefused>;

Reproject the whole tree at HEAD. Vectors in the configured space survive it (see truncateForRebuild). embed: false over a store that carries vectors is refused with RebuildNoEmbedRefused unless force is set.

Parameters

opts

embed

boolean

force?

boolean

Returns

Effect<RebuildReport, | StorageFailure | EmbedModelMismatch | RebuildNoEmbedRefused>

update

1
readonly update: (opts) => Effect<UpdateReport,
2
  | StorageFailure
3
  | EmbedModelMismatch
4
| IndexStale>;

Parameters

opts

embed

boolean

Returns

Effect<UpdateReport, | StorageFailure | EmbedModelMismatch | IndexStale>

vectorCoverage

1
readonly vectorCoverage: () => Effect<VectorCoverage, StorageFailure>;

Vector coverage for the configured space (vector-coverage.ts): the share of chunks carrying a vector under this indexer’s embedWatermark. The same reader search and doctor use.

Returns

Effect<VectorCoverage, StorageFailure>

IndexLine

A memory that did not fit. Carries its claim and its path, so the agent can drill down.

Properties

gist

1
readonly gist: string;

memoryType

1
readonly memoryType: string;

path

1
readonly path: string;

title

1
readonly title: string;

IndexRecorderShape

The write-path recorder. The store calls this after a successful write, correction, read, or reinforcement, so the link exists in BOTH planes. It is file-borne as memhtml-session/memhtml-prompt/ memhtml-turn (survives a rebuild) and indexed here (queryable in both directions).

A service rather than a bare function because the store must not depend on this package. The CLI composes the two, and the store holds only the shape.

Properties

activeFramesFor

1
readonly activeFramesFor: (keys) => Effect<ReadonlyMap<string, readonly FrameMatch[]>, StorageFailure>;

The live claims occupying each of the given frame keys, the conflict assist’s substrate.

BATCH by signature rather than by convenience: one query per key-set, never one per key. A per-op lookup is the quadratic-write-cost shape the fleet has already paid for once, and a batch write of N memories asking N questions of a table this size is that mistake with a different predicate. A caller holding one key passes a one-element array.

Parameters

keys

readonly string[]

Returns

Effect<ReadonlyMap<string, readonly FrameMatch[]>, StorageFailure>

activeNearestFor

1
readonly activeNearestFor: (queries, options) => Effect<readonly (readonly NearMatch[])[], StorageFailure>;

The live memories embedding-near each query vector, the near-duplicate assist’s substrate.

BATCH by signature for activeFramesFor’s reason: the corpus vectors are read and decoded ONCE for the whole query set, never once per op. The answer is positional — element i holds the matches for queries[i], at or above floor, similarity descending, at most k — because the queries have no natural key the way frame keys do.

Parameters

queries

readonly Float32Array<ArrayBufferLike>[]

options

excludeTypes?

readonly string[]

Memory types to exclude, e.g. arc, since a synthesis is not a near-duplicate of its members.

floor

number

Cosine similarity at or above which a memory is reported.

k

number

Matches kept per query, best first.

Returns

Effect<readonly (readonly NearMatch[])[], StorageFailure>

activePathForHash

1
readonly activePathForHash: (contentHash) => Effect<string | null, StorageFailure>;

The content-hash dedup lookup the store’s write path gates on.

Parameters

contentHash

string

Returns

Effect<string | null, StorageFailure>

recordLink

1
readonly recordLink: (link) => Effect<void, StorageFailure>;

Parameters

link

SessionLink

Returns

Effect<void, StorageFailure>

IndexStateRow

The parsed shape IndexStateRow decodes to, named so a signature can refer to it.

Extends

IndexStateRowShape

Properties

embed_dim

1
readonly embed_dim: number = Schema.Int;

Inherited from

IndexStateRowShape.embed_dim

embed_model

1
readonly embed_model: string = Schema.String;

Inherited from

IndexStateRowShape.embed_model

head_sha

1
readonly head_sha: string | null;

Inherited from

IndexStateRowShape.head_sha

id

1
readonly id: number = Schema.Int;

Inherited from

IndexStateRowShape.id

rebuilt_at

1
readonly rebuilt_at: string = Schema.String;

Inherited from

IndexStateRowShape.rebuilt_at

updated_at

1
readonly updated_at: string = Schema.String;

Inherited from

IndexStateRowShape.updated_at

NearMatch

One live memory embedding-near a query vector.

gist travels with path for FrameMatch’s reason: the near-duplicate is only reportable WITH the claim it restates, or every caller would re-read each candidate file to say what the repetition is. similarity travels too because the floor is a policy of the CALLER — a consumer choosing between “note it” and “act on it” needs the measured value, not just that it passed.

Properties

gist

1
readonly gist: string;

path

1
readonly path: string;

similarity

1
readonly similarity: number;

PersistOutcome

What one file’s persistence produced.

Properties

action

1
readonly action: "skip" | "tail" | "rescan";

merged

1
readonly merged: boolean;

True when the tail merger ran. False for a skip and for a rescan.

promptsWritten

1
readonly promptsWritten: number;

sessionId

1
readonly sessionId: string | null;

PromptRowLike

One prompt row, structurally @memhtml/traces’s PromptRow.

Properties

agentId

1
readonly agentId: string | null;

at

1
readonly at: string;

ordinal

1
readonly ordinal: number;

0-based position among the distinct prompts of THIS session, first-appearance order.

promptId

1
readonly promptId: string;

textHead

1
readonly textHead: string;

turnUuid

1
readonly turnUuid: string;

QueryEmbedPort

What the query embedder must provide. Structurally @memhtml/llm’s embedQuery.

Properties

embedQuery

1
readonly embedQuery: (text) => Effect<Float32Array<ArrayBufferLike>, ModelUnavailable>;

Parameters

text

string

Returns

Effect<Float32Array<ArrayBufferLike>, ModelUnavailable>

RankArm

One ranking arm, holding a weight, a SQL template, and whether it needs the query vector.

Properties

name

1
readonly name: string;

needsEmbedding

1
readonly needsEmbedding: boolean;

True when the arm references ?4; dropped from the fold when no query vector is available.

needsQueryTerms

1
readonly needsQueryTerms: boolean;

True when the arm MATCHes ?1, and dropped when the query holds no indexable term. A query of only punctuation is a HARD driver error rather than an empty result (probed 2026-08-02, -x and !!! ??? both fail the statement), so the arm has to leave the fold rather than run and return nothing.

needsState

1
readonly needsState: boolean;

True when the arm reads the ATTACHed state plane; dropped when no state database is attached.

sql

1
readonly sql: (holes) => string;

The arm’s ranked-list SQL, returning exactly (path, rank) with rank 1-based.

Parameters

holes

ArmHoles

Returns

string

weight

1
readonly weight: number;

Unitless multiplier on this arm’s 1/(rank + k) contributions.

RebuildReport

A full rebuild’s outcome.

Properties

chunksIndexed

1
readonly chunksIndexed: number;

edgesIndexed

1
readonly edgesIndexed: number;

embeddingsPreserved

1
readonly embeddingsPreserved: number;

Vectors carried across the rebuild rather than re-embedded. Chunk ids are content-addressed, so a vector in the configured space is still the right vector for a chunk that came back with the same id; see truncateForRebuild. Zero on a model change, where nothing is carried.

embeddingsWritten

1
readonly embeddingsWritten: number;

filesIndexed

1
readonly filesIndexed: number;

headSha

1
readonly headSha: string;

skipped

1
readonly skipped: readonly object[];

Files git offered that failed to parse. Counted, never fatal. One bad file is not a bad tree.

RecallInput

A recall request. budgetChars bounds the quoted bodies, not the index lines.

Extends

SearchScope

Properties

asOf?

1
readonly asOf?: string;

Point-in-time view. Takes an ISO instant, and the candidate set becomes “what was believed valid at this moment”, with archived files INCLUDED, filtered by the validity window coalesce(valid_from, event_at, created_at) <= asOf AND (valid_until IS NULL OR valid_until > asOf).

The coalesce order is the stamping rule’s own (@memhtml/store’s validFromOf). An explicit valid-from beats the event time, which beats the write time. All three columns compare lexicographically as strings by design (0008_tasks.sql), so the predicate is string comparison, never a per-row parse. Applied HERE, at the one filter every arm receives, so the temporal lens cannot hold for three arms and leak in the fourth. When absent the assembled filter is byte-identical to today’s, and a pin test enforces that.

Inherited from

SearchScope.asOf

budgetChars?

1
readonly budgetChars?: number;

entity?

1
readonly entity?: string;

STRICT equality on ONE entity reference, in type:name form, so person:sasha and not sasha.

Strict about the TYPE half being required, and case-INSENSITIVE about spelling: a caller should not have to reproduce an author’s capitalization to find a memory. The fold is lower() applied to BOTH sides inside SQL, never once in TypeScript and once in SQL — JS toLowerCase is full-Unicode and SQLite’s lower() is ASCII-only, so splitting the fold across the two would make place:CAFÉ match from one door and miss from the other. One function, one side of the seam.

ASCII-only is the honest bound, and it is not the whole story: entity-resolution rewrites each file’s memhtml-entity meta to its NFC- and whitespace-normalized form, so full canonicalization is durable in the tree rather than re-derived per query. This fold covers the window before that phase has run.

The same spelling memory_list takes, and the same predicate, because the two are one vocabulary. A caller that learned person:sasha from a listing must be able to hand that exact string to a search. file_entities is keyed on (type, name), so the bare name is ambiguous. person:sasha and concept:sasha are two entities, and accepting the bare name here would make a scope that narrows to whichever of them the corpus happens to hold.

Singular, unlike tags, and that asymmetry is deliberate. This scope exists so a caller can chain a hop off a hit’s own entities list, which is one reference at a time. A list would raise the question of whether it broadens or narrows before anyone has asked for either.

Inherited from

SearchScope.entity

1
readonly facets?: readonly FacetFilter[];

<dl> facet predicates: AND across distinct NAMES, OR within one name.

The extension axis. memhtml’s element and <meta> vocabularies are closed, so a consumer that needs to model its own document kinds, states, or tiers writes them as <dt>/<dd> pairs and narrows on them here. Nothing about those names reaches this package: the predicate is over two TEXT columns, so one corpus can carry doc-type and another severity with no code between them.

The composition rule is stated because a silent choice here is a semantic contract. Two values under ONE name broaden (doc-type=runbook, doc-type=guide is either), matching how tags broadens. Two DIFFERENT names narrow (doc-type=runbook, tier=1 is both), matching how a workspace narrows a tag scope. A caller that read it the other way would get a superset or an empty set and no signal which.

value is matched as TEXT, and exactly, with no case fold. Two reasons, and the second is measurable. A facet is a consumer’s own machine-written vocabulary rather than a name a human types from memory, which is what entity’s fold exists for. And the fold would cost the index: probed 2026-08-26 on node 24’s node:sqlite, ff.name = ? AND ff.value IN (?) seeks sqlite_autoindex_file_facets_1 on (path=? AND name=? AND value=?), while wrapping either column in lower() degrades the same probe to (path=?) — every facet row of every candidate file read and filtered, returning identical rows.

file_facets.numeric_value is NOT reachable from here, and that is the contract rather than a gap: the column is UNITLESS (0001_files.sql:91-94), because the unit lives in the human phrasing beside it. A >= on an unlabelled number is a comparison whose meaning the corpus never stated, so the caller owning the unit is the only honest arrangement, and it owns it by matching the text it authored.

Inherited from

SearchScope.facets

includeArchived?

1
readonly includeArchived?: boolean;

Archived files are excluded unless asked for. Eviction is a git mv, so they still exist.

Inherited from

SearchScope.includeArchived

memoryTypes?

1
readonly memoryTypes?: readonly (
2
  | "arc"
3
  | "episodic"
4
  | "semantic"
5
  | "procedural"
6
  | "agent_insight"
7
  | "user_preference"
8
  | "error_pattern"
9
  | "verdict"
10
  | "precedent"
11
  | "task")[];

ANY-of. Absent or empty means every type EXCEPT task, and naming task opts a query in.

The asymmetry is deliberate and it is the one place task is treated apart from other types. A task is intermediate working state rather than a remembered fact. A corpus with fifty open to-do items would put fifty of them in front of every recall, crowding out the knowledge an agent asked for with a list it can read by ls-ing a directory. Tasks are CRUDL-able without retrieval by design, so retrieval is where they stay out of the way.

Inherited from

SearchScope.memoryTypes

query

1
readonly query: string;

tags?

1
readonly tags?: readonly string[];

ANY-of overlap. Each tag BROADENS the result set.

Inherited from

SearchScope.tags

workspace?

1
readonly workspace?: string;

STRICT equality. A workspace-scoped query never sees a NULL-workspace page. A memory with no workspace is unplaced rather than “in every workspace”, and returning it would make a project-scoped recall quietly global.

Inherited from

SearchScope.workspace

RecallPack

The recall pack. Arcs and ordinary memories fold separately, each under its own envelope.

Properties

arcs

1
readonly arcs: DisclosureFold;

degraded

1
readonly degraded: boolean;

memories

1
readonly memories: DisclosureFold;

spentChars

1
readonly spentChars: number;

truncated

1
readonly truncated: boolean;

vectorCoverage

1
readonly vectorCoverage: number;

The same ratio search reports, for the same reason: it names which degradation this is.

ReinforceResult

Which paths were bumped and which were still cooling down.

Properties

bumped

1
readonly bumped: readonly string[];

cooledDown

1
readonly cooledDown: readonly string[];

RetrievalDeps

Properties

db

1
readonly db: DatabaseShape;

embeddings?

1
readonly embeddings?: QueryEmbedPort;

embedWatermark?

1
readonly embedWatermark?: string;

The configured vector space, @memhtml/llm’s EMBED_WATERMARK, so coverage counts only vectors the query vector is comparable with. Absent falls back to the stored watermark, which is the same string whenever the indexer has been allowed to write (readVectorCoverage).

vectorCoverageFloor?

1
readonly vectorCoverageFloor?: number;

Coverage below which the vector arm is dropped. Defaults to VECTOR_COVERAGE_FLOOR; the CLI passes MEMHTML_VECTOR_COVERAGE_FLOOR and a test moves it to prove the gate is this comparison and nothing else.

RetrievalShape

Properties

recall

1
readonly recall: (input) => Effect<RecallPack, StorageFailure>;

Parameters

input

RecallInput

Returns

Effect<RecallPack, StorageFailure>

search

1
readonly search: (input) => Effect<SearchResult, StorageFailure>;

Parameters

input

SearchInput

Returns

Effect<SearchResult, StorageFailure>

RrfOptions

What buildRrfSql folds over.

Properties

arms?

1
readonly arms?: readonly RankArm[];

hasQueryTerms?

1
readonly hasQueryTerms?: boolean;

False when the sanitized query holds no indexable term, dropping needsQueryTerms arms. Defaults to true, so a caller that forgets it gets the full fold rather than a silently narrowed one. The failure then surfaces as a driver error in a test rather than as a missing arm in production.

hasQueryVector

1
readonly hasQueryVector: boolean;

False when the embedder failed or the query has no vector, dropping needsEmbedding arms.

hasState

1
readonly hasState: boolean;

False when no state database is attached, dropping needsState arms.

holes

1
readonly holes: ArmHoles;

ScannedFileLike

One scanned file, structurally @memhtml/traces’s ScannedFile.

Properties

action

1
readonly action: "skip" | "tail" | "rescan";

agentCount

1
readonly agentCount: number;

extract

1
readonly extract: SessionExtractLike | null;

file

1
readonly file: object;

filePath

1
readonly filePath: string;

kind

1
readonly kind: "session" | "subagent";

watermark

1
readonly watermark: WatermarkLike;

SearchHit

One ranked hit.

Properties

confidence

1
readonly confidence: number;

entities

1
readonly entities: readonly string[];

Every entity this memory carries, in type:name form, so person:sasha and never bare sasha.

The form is a CONTRACT with the entity scope rather than a display choice. A value taken from this array must be usable verbatim as the next search’s entity, which makes a two-hop chain two tool calls instead of a guess about how to spell the reference. file_entities is keyed on (type, name), so a bare name would be ambiguous and the second hop would scope to whichever type the corpus happened to hold.

Empty array when the memory names no entity, never absent. A caller reading an absent key cannot tell “no entities” from “this server does not report them”.

gist

1
readonly gist: string;

memoryType

1
readonly memoryType: string;

path

1
readonly path: string;

score

1
readonly score: number;

The fused RRF score, unitless and comparable only within one result set.

snippet

1
readonly snippet: string;

The text of this file’s best-matching chunk for THIS query, truncated to SNIPPET_MAX_CHARS with a … marker when cut. “Best” is nearest-to-the-query-vector when the vector arm ran. On the degraded path it is the file’s ordinal-0 chunk, the article’s opening text, which for the one-fact-per-file common case is the whole article. Empty string only when the file has no chunk at all (an empty article), never absent.

supersededBy

1
readonly supersededBy: string | null;

The path of the memory that superseded this one, or null when nothing has. Non-null only for an archived hit, which reaches a result set through asOf or includeArchived, so a point-in-time answer reads as history. The hit was believed then, and THIS is what replaced it. Present-and-nullable in every result, so a caller can tell “not superseded” from “this build does not report supersession”.

Derived from the edges table rather than the memhtml-superseded-by head meta. The loser is the dst_path of the winner’s authored supersedes edge, which edges_dst indexes. The meta reaches no SQL column at all, so reading it would cost a file open per hit.

title

1
readonly title: string;

updatedAt

1
readonly updatedAt: string;

SearchInput

The query and its scope. Mirrors memory_search’s parameters.

Extends

SearchScope

Properties

asOf?

1
readonly asOf?: string;

Inherited from

SearchScope.asOf

entity?

1
readonly entity?: string;

STRICT equality on ONE entity reference, in type:name form, so person:sasha and not sasha.

Inherited from

SearchScope.entity

1
readonly facets?: readonly FacetFilter[];

<dl> facet predicates: AND across distinct NAMES, OR within one name.

Inherited from

SearchScope.facets

includeArchived?

1
readonly includeArchived?: boolean;

Archived files are excluded unless asked for. Eviction is a git mv, so they still exist.

Inherited from

SearchScope.includeArchived

limit?

1
readonly limit?: number;

memoryTypes?

1
readonly memoryTypes?: readonly (
2
  | "arc"
3
  | "episodic"
4
  | "semantic"
5
  | "procedural"
6
  | "agent_insight"
7
  | "user_preference"
8
  | "error_pattern"
9
  | "verdict"
10
  | "precedent"
11
  | "task")[];

ANY-of. Absent or empty means every type EXCEPT task, and naming task opts a query in.

Inherited from

SearchScope.memoryTypes

query

1
readonly query: string;

tags?

1
readonly tags?: readonly string[];

ANY-of overlap. Each tag BROADENS the result set.

Inherited from

SearchScope.tags

workspace?

1
readonly workspace?: string;

Inherited from

SearchScope.workspace

SearchResult

A search’s outcome.

Properties

archived

1
readonly archived: readonly ArchivedMatch[];

Up to limit of those archived paths, sorted, each with the path of the memory that superseded it or null when nothing did. Empty unless scopeEmpty is true. supersededBy is derived from the supersedes edge the way a hit’s is, so compress’s canonical and correct’s replacement both resolve.

archivedMatches

1
readonly archivedMatches: number;

How many ARCHIVED memories the same scope matches, computed only when scopeEmpty is true and 0 otherwise.

The pointer behind an empty scope. Eviction and compress are a git mv into archive/, and the default scope excludes archived rows, so a correct facet over a real record (day=2026-09-02 on a journal compress folded last night) answers nothing and, without this number, says nothing about why. With it an agent can tell “never existed” from “archived”: retry with includeArchived, or follow archived[].supersededBy to what replaced it (issue #130). A scope match rather than a ranked one: the question is whether the scope’s address still resolves, not how the query would rank what it points at. Only the scope’s own axes are re-applied; the archived flag is what flips. Zero as well under asOf, whose lens already admits archived rows, so the flag is not what emptied that search.

arms

1
readonly arms: readonly string[];

The arms that actually contributed, for the operator envelope.

degraded

1
readonly degraded: boolean;

True when the vector arm did not fire, so the result came from the lexical floor. Three causes: no embedder is bound, the embedder failed, or vectorCoverage is below the floor and the arm was dropped on purpose (a sparse plane ranks the embedded few above every lexical match, issue #141). Reported rather than silent, because an agent comparing two searches needs to know one of them was ranked by fewer signals. vectorCoverage tells the third cause from the first two.

entityScope

1
readonly entityScope: string | null;

The entity reference this search was scoped to, or null when it was not scoped by entity.

Echoed back so an empty result is ATTRIBUTABLE. A scope that matches nothing never widens here, since there is no fallback to widen to. The failure mode left is a caller who cannot tell a corpus with no answer from a scope with a typo in it, and this field is what tells them.

hits

1
readonly hits: readonly SearchHit[];

scopeEmpty

1
readonly scopeEmpty: boolean;

True when a scope was named, it narrowed the query, and nothing survived it.

A BOOLEAN in every case rather than a field that appears only when it fires. The CLI’s --dense drops null-valued keys (apps/cli/src/envelope.ts:139), so a marker that were null when absent would vanish from exactly the output an agent pastes into a context window. false costs five bytes and is unambiguous.

Never true for an unscoped empty result. An empty corpus is not an over-narrow scope, and conflating them would make the marker mean “no hits”, which hits.length already says.

vectorCoverage

1
readonly vectorCoverage: number;

The share of the index’s chunks carrying a vector in the configured space, 0 to 1, read once per call. Present on every result so a caller can tell a coverage degradation (this is low) from an embedder outage (this is high and degraded is still true). 1 on an index with no chunks.

SearchScope

The caller’s scope. Every field absent means “the whole wiki, minus the tasks”.

Extended by

Properties

asOf?

1
readonly asOf?: string;

entity?

1
readonly entity?: string;

STRICT equality on ONE entity reference, in type:name form, so person:sasha and not sasha.

1
readonly facets?: readonly FacetFilter[];

<dl> facet predicates: AND across distinct NAMES, OR within one name.

includeArchived?

1
readonly includeArchived?: boolean;

Archived files are excluded unless asked for. Eviction is a git mv, so they still exist.

memoryTypes?

1
readonly memoryTypes?: readonly (
2
  | "arc"
3
  | "episodic"
4
  | "semantic"
5
  | "procedural"
6
  | "agent_insight"
7
  | "user_preference"
8
  | "error_pattern"
9
  | "verdict"
10
  | "precedent"
11
  | "task")[];

ANY-of. Absent or empty means every type EXCEPT task, and naming task opts a query in.

tags?

1
readonly tags?: readonly string[];

ANY-of overlap. Each tag BROADENS the result set.

workspace?

1
readonly workspace?: string;

SessionExtractLike

One session’s extract, structurally @memhtml/traces’s SessionExtract.

Properties

agentIds

1
readonly agentIds: readonly string[];

aiTitle

1
readonly aiTitle: string | null;

cwd

1
readonly cwd: string | null;

endedAt

1
readonly endedAt: string | null;

entrypoint

1
readonly entrypoint: string | null;

filePath

1
readonly filePath: string;

firstPrompt

1
readonly firstPrompt: string;

gitBranch

1
readonly gitBranch: string | null;

model

1
readonly model: string | null;

promptCount

1
readonly promptCount: number;

prompts

1
readonly prompts: readonly PromptRowLike[];

sessionId

1
readonly sessionId: string | null;

slug

1
readonly slug: string;

startedAt

1
readonly startedAt: string | null;

turnCount

1
readonly turnCount: number;

version

1
readonly version: string | null;

SessionLink

One memory_session_links row.

Properties

at

1
readonly at: string;

linkKind

1
readonly linkKind: "wrote" | "read" | "corrected" | "reinforced";

path

1
readonly path: string;

promptId?

1
readonly promptId?: string;

sessionId

1
readonly sessionId: string;

turnUuid?

1
readonly turnUuid?: string;

SidecarEntry

One sidecar line’s fields, in the order they are written.

Properties

accessCount

1
readonly accessCount: number;

lastAccessedAt

1
readonly lastAccessedAt: string | null;

lastReinforcedAt

1
readonly lastReinforcedAt: string | null;

outcomeScore

1
readonly outcomeScore: number;

path

1
readonly path: string;

reinforcementCount

1
readonly reinforcementCount: number;

updatedAt

1
readonly updatedAt: string;

StatusEntry

One git status --porcelain=v2 row: an uncommitted change the working tree carries.

Properties

deleted

1
readonly deleted: boolean;

True when the path is gone from the working tree.

fromPath?

1
readonly fromPath?: string;

The source of a staged rename, carried so the indexer can retire the source row. A staged git mv is one status entry whose path is the destination; without the source, the old row stays active and the destination’s projection collides with it on files_content_hash_active.

path

1
readonly path: string;

StoreChangedPath

Properties

fromPath

1
readonly fromPath: string | null;

kind

1
readonly kind: "added" | "modified" | "deleted" | "renamed" | "copied" | "typechanged";

path

1
readonly path: string;

StoreGitShape

The subset of @memhtml/store’s GitShape the indexer consumes. Declared, not imported.

Properties

catFileBatch

1
readonly catFileBatch: (shas) => Effect<ReadonlyMap<string, Uint8Array<ArrayBufferLike>>, unknown>;

Parameters

shas

readonly string[]

Returns

Effect<ReadonlyMap<string, Uint8Array<ArrayBufferLike>>, unknown>

diffNameStatus

1
readonly diffNameStatus: (from, to) => Effect<readonly StoreChangedPath[], unknown>;

Parameters

from

string

to

string

Returns

Effect<readonly StoreChangedPath[], unknown>

hashObject

1
readonly hashObject: (path) => Effect<string, unknown>;

Parameters

path

string

Returns

Effect<string, unknown>

lsTreeR

1
readonly lsTreeR: (commitish, pathspecs?) => Effect<readonly StoreTreeEntry[], unknown>;

Parameters

commitish

string

pathspecs?

readonly string[]

Returns

Effect<readonly StoreTreeEntry[], unknown>

revParseHead

1
readonly revParseHead: () => Effect<string | null, unknown>;

Returns

Effect<string | null, unknown>

statusPorcelainV2

1
readonly statusPorcelainV2: () => Effect<readonly StoreStatusEntry[], unknown>;

Returns

Effect<readonly StoreStatusEntry[], unknown>

StoreStatusEntry

Properties

fromPath

1
readonly fromPath: string | null;

The source of a staged rename (kind: "renamed"), null for every other kind.

kind

1
readonly kind: "renamed" | "changed" | "unmerged" | "untracked" | "ignored";

path

1
readonly path: string;

xy

1
readonly xy: string;

StoreTreeEntry

Properties

objectType

1
readonly objectType: string;

path

1
readonly path: string;

sha

1
readonly sha: string;

TreeEntry

One git ls-tree -r row.

Properties

blobSha

1
readonly blobSha: string;

The blob’s own sha. Equal to git hash-object <file>, which makes it the free change key.

path

1
readonly path: string;

Repo-root-relative, no leading slash — the git-tree form, and the files.path primary key.

UpdateReport

An incremental pass’s outcome. unchanged: true means the watermark and the tree already agreed.

Properties

added

1
readonly added: number;

dirty

1
readonly dirty: number;

embeddingsWritten

1
readonly embeddingsWritten: number;

headSha

1
readonly headSha: string;

modified

1
readonly modified: number;

removed

1
readonly removed: number;

renamed

1
readonly renamed: number;

skipped

1
readonly skipped: readonly object[];

unchanged

1
readonly unchanged: boolean;

VectorCoverage

The measured ratio and the two counts behind it.

Properties

chunks

1
readonly chunks: number;

Every chunk in the index, embedded or not.

coverage

1
readonly coverage: number;

embeddings / chunks, or 1 when there are no chunks.

embeddings

1
readonly embeddings: number;

Chunks carrying a vector in the counted space. Never more than chunks.

model

1
readonly model: string | null;

The space that was counted: the configured watermark when the caller named one, else the stored index_state.embed_model, else null for an index nothing has ever rebuilt.

WatermarkLike

A stat-based watermark, structurally @memhtml/traces’s Watermark. mtimeMs is epoch MILLISECONDS.

Properties

byteOff

1
readonly byteOff: number;

0-based byte offset, one past the last byte consumed, and the start of the next read.

mtimeMs

1
readonly mtimeMs: number;

size

1
readonly size: number;

Write

Properties

params

1
readonly params: readonly SqlValue[];

sql

1
readonly sql: string;

Type Aliases

LinkKind

1
type LinkKind = typeof LINK_KINDS[number];

SqlValue

1
type SqlValue = string | number | null | Uint8Array;

TailMerger

1
type TailMerger = (stored, tail) => SessionExtractLike;

@memhtml/traces’s mergeTailExtract, as a requirement.

Parameters

Variables

ARC_BODY_BUDGET

1
const ARC_BODY_BUDGET: 9000 = 9_000;

An arc synthesizes many memories, so it gets a bigger envelope than an ordinary memory.

CHUNK_MAX_CHARS

1
const CHUNK_MAX_CHARS: 1800 = 1_800;

Character ceiling for one chunk. Below it an entry’s whole article is chunk 0, which is the overwhelmingly common case, since the format is one fact per file.

DatabaseService

1
const DatabaseService: Service<DatabaseShape, DatabaseShape>;

DEFAULT_ARM_LIMIT

1
const DEFAULT_ARM_LIMIT: 40 = 40;

How many candidates each arm contributes before fusion. The retrieval SQL binds it as every arm’s LIMIT, and the v2 head caps its lexical and vector arms at it, so both stores fuse the same number of candidates per arm.

DEFAULT_SEARCH_LIMIT

1
const DEFAULT_SEARCH_LIMIT: 10 = 10;

How many hits search returns when the caller names no limit.

EXCLUDED_BY_DEFAULT

1
const EXCLUDED_BY_DEFAULT: "task" = "task";

The one type an unscoped query does not see.

Named as a constant so the SQL predicate, the dedupeLookup exclusion in traces-persist.ts, and the tests all read one value. Three copies of the string 'task' would let the retrieval default and the dedup carve-out drift apart, and each would look correct alone.

1
const FACET_SEPARATOR: "=" = "=";

What separates a facet’s name from its value in the name=value wire form.

FILE_COLUMNS

1
const FILE_COLUMNS: readonly ["path", "blob_sha", "content_hash", "memory_type", "title", "body_text", "gist", "fts_text", "disclosure_text", "para", "workspace", "confidence", "importance", "archived", "origin_path", "word_count", "created_at", "updated_at", "event_at", "archived_at", "valid_from", "valid_until", "reprieves", "needs_revision", "author", "session_id", "prompt_id", "turn_uuid", "indexed_at", "task_status", "due_at", "frame_key"];

The files columns in bind order. This list drives the insert’s column list and the upsert’s assignment clause. The placeholder count comes from the value array in projectFile, so a new column needs its value there too, at the same position. A count mismatch fails the statement, and project.test.ts checks both counts against this list. A value at the wrong position binds every subsequent value to the wrong column and every CHECK still passes.

FTS_COLUMN

1
const FTS_COLUMN: "fts_text" = "fts_text";

The one column files_fts covers.

ONE column so that a single MATCH finds a term wherever it lives. The indexer denormalizes title, gist, and body into fts_text, and the arm MATCHes that. A multi-column FTS5 table would make bm25() weight the columns against each other, which is a ranking decision the RRF fusion already owns. The lexical arm’s job is to contribute one relevance order, and not to pre-blend fields.

FTS_INDEX_NAME

1
const FTS_INDEX_NAME: "files_fts" = "files_fts";

The lexical index, an external-content FTS5 table over files, maintained by triggers.

It is a TABLE, not an index, which is what makes it MATCHable and bm25()-rankable. Nothing drops or recreates it around a bulk load: FTS5 writes are linear, so the bracket buys nothing and opens a window where a crash leaves the store with no lexical index. See indexer.ts’s applyProjectionWrites for the measurements.

GENERATED_NAMES

1
const GENERATED_NAMES: ReadonlyArray<string>;

Files the indexer refuses to consider, by name.

index.html and sitemap.xml are GENERATED by memhtml publish from the corpus. Indexing them would feed every directory listing back into retrieval as a memory whose body is the titles of other memories, and the corpus would rank its own table of contents above its content.

INDEX_STATE_ID

1
const INDEX_STATE_ID: 1 = 1;

The index_state singleton’s primary key. The table holds exactly one row by CHECK.

Indexer

1
const Indexer: Service<IndexerShape, IndexerShape>;

IndexGit

1
const IndexGit: Service<GitPort, GitPort>;

The tag is memhtml/IndexGit. @memhtml/store already publishes memhtml/Git for its own GitShape, and two different shapes under one tag would let a layer satisfy the wrong requirement silently. makeGitPort in git-adapter.ts is the bridge between them.

IndexRecorder

1
const IndexRecorder: Service<IndexRecorderShape, IndexRecorderShape>;

IndexStateRow

1
const IndexStateRow: Struct<{
2
  embed_dim: Int;
3
  embed_model: String;
4
  head_sha: NullOr<String>;
5
  id: Int;
6
  rebuilt_at: String;
7
  updated_at: String;
8
}>;

Every column of index_state, in the order 0007_watermark.sql declares them.

head_sha is the only nullable one, and the three states it distinguishes are not two. NO ROW is a store nothing has indexed. A row carrying NULL is a rebuild in flight or one that died: the rebuild clears the column in the same transaction as its truncate and writes the commit back only after every projection landed, so update reads NULL as IndexStale rather than as a watermark to diff from. A commit is an index that describes that commit. embed_model carries <model-id>@<dim> (@memhtml/llm’s EMBED_WATERMARK), and embed_dim restates the dimension as a number so a mismatch is comparable without parsing the watermark string.

embed_dim is Int rather than Number because the two guards divide the work. The column’s own CHECK (embed_dim > 0) owns the RANGE, and SQLite enforces it, so no row can carry a non-positive dimension. SQLite does not enforce the TYPE. Probed 2026-08-12, INTEGER affinity stores '12.5' as the real 12.5 and accepts it, and a fractional vector dimension is meaningless. Int is the half the database cannot state.

LINK_KINDS

1
const LINK_KINDS: readonly ["wrote", "read", "corrected", "reinforced"];

The kinds of link a memory can have to a session.

MAX_PER_ENTITY

1
const MAX_PER_ENTITY: 2 = 2;

At most two full quotes per ENTITY NAME, not per path.

Per-path would be no cap at all. Twelve memories about one service are twelve paths, and they would fill the budget with one entity’s history while every other entity the query touched gets an index line. The cap is what makes recall breadth-first over entities.

MEMORY_BODY_BUDGET

1
const MEMORY_BODY_BUDGET: 16000 = 16_000;

The shared envelope for ordinary memories.

MEMORY_TABLES

1
const MEMORY_TABLES: readonly ["file_citations", "file_facets", "file_entities", "file_tags", "embeddings", "chunks", "edges", "files"];

Tables a rebuild empties, in delete order. Children come before parents, so the statements are correct even with foreign keys enforced rather than relying on cascade.

MIGRATIONS_DIR

1
const MIGRATIONS_DIR: string;

Where the rebuildable index’s migrations live, applied in filename order.

MMR_POOL_FACTOR

1
const MMR_POOL_FACTOR: 3 = 3;

Fused candidates fetched before MMR, as a multiple of the final limit. Diversification can only reorder what it was given, so a pool the size of the limit makes MMR a no-op.

OUTCOME_EWMA_ALPHA

1
const OUTCOME_EWMA_ALPHA: 0.3 = 0.3;

The EWMA weight a new outcome signal carries. The remainder keeps the prior score.

The value the SQL below BINDS, and the twin of @memhtml/domain’s DEFAULT_EWMA_ALPHA, which is the reference implementation the arithmetic here is checked against. SQL cannot read that constant, so this one is declared here and tests/reinforce.test.ts pins the pair to agree — a fork of either side fails a suite rather than drifting the reference model off the shipped arithmetic.

PARAM_ARM_LIMIT

1
const PARAM_ARM_LIMIT: 2 = 2;

PARAM_FINAL_LIMIT

1
const PARAM_FINAL_LIMIT: 3 = 3;

PARAM_QUERY

1
const PARAM_QUERY: 1 = 1;

The bound-parameter positions, named so a reader never counts question marks.

PARAM_QUERY_VECTOR

1
const PARAM_QUERY_VECTOR: 4 = 4;

RANK_ARMS

1
const RANK_ARMS: ReadonlyArray<RankArm>;

The registry, in fold order. Order is presentation only. RRF’s sum commutes.

REINFORCE_PATH_BATCH

1
const REINFORCE_PATH_BATCH: 500 = 500;

Paths bound into one upsert.

SQLite’s bound-variable ceiling is a BUILD property rather than a language one, at 999 in older builds and 32766 since 3.32, and this package must not assume which one the driver shipped with. Five shared values plus this many paths stays under either, so the split is a correctness guard rather than a tuning knob: one extra statement per 500 paths against a batch that would otherwise be refused outright.

Retrieval

1
const Retrieval: Service<RetrievalShape, RetrievalShape>;

SALIENCE_EXCLUDED_PREFIX

1
const SALIENCE_EXCLUDED_PREFIX: "resources/people/" = "resources/people/";

The one path prefix salience does not rank.

There is no person memory type. A person file is a semantic record curation writes at an explicit path under resources/people/ (placement never routes there), so the prefix IS the discriminator. A reference record is reached by entity key, and decay is wrong for identity. A colleague unmentioned for six months is not less themselves. Memories ABOUT a person live elsewhere and keep their salience, which is the signal that answers “which five of fifty sasha-memories do we consult”.

SALIENCE_EXCLUDED_TYPE

1
const SALIENCE_EXCLUDED_TYPE: "task" = "task";

The one memory type salience does not rank.

A task is reached by task_status and due_at, nominal predicates and not a relevance contest. Salience over working state would reward STALENESS, so the stuck task re-read during every triage would outrank the fresh urgent one. Named as a constant so the predicate and the tests read one.

SIDECAR_PRECISION

1
const SIDECAR_PRECISION: 4 = 4;

Decimal places every float in the sidecar carries. Matches the domain’s fixed-point grid.

SNIPPET_MAX_CHARS

1
const SNIPPET_MAX_CHARS: 700 = 700;

Character ceiling for a search hit’s snippet, the text of the file’s best-matching chunk. Roughly three gists wide, enough to show WHY the chunk matched without turning a ten-hit result into a recall pack, which has its own budgeted door. A snippet cut at this ceiling ends in ….

STATE_MIGRATIONS_DIR

1
const STATE_MIGRATIONS_DIR: string;

The state plane’s own migration ledger. A separate directory because these statements are applied to the ATTACHed state database, which has its own schema_migrations table. The two planes have independent lifetimes, and index.db is deleted and rebuilt without touching state.db.

STATE_SCHEMA

1
const STATE_SCHEMA: "state" = "state";

The schema name state.db is ATTACHed under. Every cross-plane query qualifies with it.

STATE_TABLES

1
const STATE_TABLES: readonly ["access", "edge_corroboration"];

The state plane’s tables, qualified at the call site with STATE_SCHEMA.

TRACE_TABLES

1
const TRACE_TABLES: readonly ["trace_prompts", "traces", "trace_watermarks"];

The trace plane’s tables. Never touched by a memory rebuild, and never named in retrieval SQL.

TREE_PREFIXES

1
const TREE_PREFIXES: ReadonlyArray<string>;

The buckets a rebuild reads. Passed to ls-tree so .memhtml/ and the repo’s own docs stay out.

VECTOR_COVERAGE_FLOOR

1
const VECTOR_COVERAGE_FLOOR: 0.95 = 0.95;

Below this, the vector plane is treated as absent: search and recall drop the vector arm and report degraded: true, and doctor reports vectorCoverageLow and healthy: false. The CLI reads MEMHTML_VECTOR_COVERAGE_FLOOR over this default.

0.95 rather than 1.0 because coverage is never exactly complete on a live store: a write lands, its chunks are projected, and its vectors follow in the same update a moment later, so a reader between the two sees a handful of unembedded chunks on a healthy index. Five percent is far above that gap and far below the 2 percent the incident measured.

VECTOR_COVERAGE_HARD_FLOOR

1
const VECTOR_COVERAGE_HARD_FLOOR: 0.5 = 0.5;

Below this, a curation pass over vectors describes a sample rather than the corpus: half the corpus without vectors means dedup, mining and conflict detection compare a sample against itself and call the rest unique, with a green report. Not configurable: the soft floor is a quality knob, this is the point past which vector arithmetic stops describing the corpus.

VECTOR_COVERAGE_NO_VECTORS_REMEDY

1
const VECTOR_COVERAGE_NO_VECTORS_REMEDY: "or set `MEMHTML_EMBED=off` for a store that is not meant to embed" = "or set `MEMHTML_EMBED=off` for a store that is not meant to embed";

The clause a ZERO-vector plane adds to the remedy. An embedder is bound (or the store would be the lexical-only configuration) and it has written nothing, which is the shape of a default MEMHTML_EMBED=on with no credential: every embed call fails softly, every search is degraded, and the remedies above cannot succeed either. Naming the opt-out is what turns that refusal into a decision the operator can make.

VECTOR_COVERAGE_REMEDY

1
const VECTOR_COVERAGE_REMEDY: "run `memhtml index embed` to backfill the missing vectors, or `memhtml index rebuild --embed`" = "run `memhtml index embed` to backfill the missing vectors, or `memhtml index rebuild --embed`";

What an operator does about low coverage. Named once so three surfaces say the same thing.

memhtml index embed is the incremental backfill (issue #142): it embeds only the chunks without a vector and is safe to rerun, so it is named first. memhtml index rebuild --embed reprojects the whole tree and is the path when the index itself is suspect. A store that has no vectors at all gets a third clause from VectorCoverageLow: MEMHTML_EMBED=off, for the operator who never meant to embed.

WRITE_BATCH_SIZE

1
const WRITE_BATCH_SIZE: 500 = 500;

Rows per writeAll batch.

A bound rather than a tuning knob. One batch is one transaction, so this caps how much work a single failure discards and how long one write holds the WAL write lock against a concurrent reader. Whole-store passes are batched for that reason rather than sent as one transaction. There is no per-row cost here, because FTS5 does not have one.

Functions

accessRows()

1
function accessRows(db): Effect<readonly AccessRow[], StorageFailure>;

The whole state.access table, path-ordered.

Ordered in SQL instead of sorted afterwards so the sidecar is byte-stable. Two exports over an unchanged plane produce an identical file and therefore no commit.

Parameters

db

DatabaseShape

Returns

Effect<readonly AccessRow[], StorageFailure>

activeArms()

1
function activeArms(options): readonly RankArm[];

The arms that will actually fire, in registry order.

Parameters

options

RrfOptions

Returns

readonly RankArm[]

assembleScope()

1
function assembleScope(scope?): AssembledScope;

Assemble a scope into the arm hole plus its bound values.

The {alias} token stands in for whichever alias the arm gives its files row; each arm substitutes it. Emitting a fixed alias here would make the fragment usable by one arm only.

An empty scope produces AND {alias}.archived = 0 plus the default task exclusion. Every arm receives both, so a task cannot enter one arm’s candidate set and be missing from another’s.

Parameters

scope?

SearchScope = {}

Returns

AssembledScope

attachState()

1
function attachState(
2
   db,
3
   statePath,
4
   migrationsDir
5
): Effect<number, StorageFailure, never>;

ATTACH the state plane onto an existing connection under the state schema name and apply its own migrations.

One connection carrying both planes is what lets the salience retrieval arm LEFT JOIN state.access in the same statement as main.files, with no application-side join over two result sets.

Attaching is not idempotent, because a second ATTACH ... AS state fails with “database state is already in use”. So this is called exactly once per connection, by makeDatabase.

Parameters

db

DatabaseSync

statePath

string

migrationsDir

string

Returns

Effect<number, StorageFailure, never>

authoredEdgesFor()

1
function authoredEdgesFor(
2
   doc,
3
   path,
4
   contentHash,
5
   createdAt
6
): readonly Write[];

Authored edges from the head’s <link rel="memhtml-*"> elements.

href is the document-reference form, repo-root-relative WITH a leading slash, and the edges table stores the git-tree form, so normalizePath strips it. Storing the slashed form would make every edge’s dst_path fail to join files.path, and the join returning nothing looks exactly like a corpus with no edges.

A self-loop is dropped rather than inserted. The table’s CHECK would reject the whole batch, and one hand-authored file pointing at itself must not fail the indexing of every file beside it.

Parameters

arc memories take ARC_BODY_BUDGET; everything else takes MEMORY_BODY_BUDGET.

Parameters

memoryType

string

Returns

number

buildRrfSql()

1
function buildRrfSql(options): string | undefined;

The one fused statement. Each active arm becomes a CTE, their weighted reciprocal ranks are UNION ALLed, then summed per path.

Ties break on path ASC so the ordering is total and two runs over an unchanged corpus produce the same list, which is what the discrimination gate compares against.

Returns undefined when no arm is active. A caller must treat that as an empty result rather than assemble SELECT ... FROM (), which is why this is not a string.

Parameters

options

RrfOptions

Returns

string | undefined

buildSnippetSql()

1
function buildSnippetSql(options): string | undefined;

The snippet fetch, covering every chunk of the SELECTED paths, with its distance to the query vector when one exists. ONE statement after the fused ranking, never a change to the fused CTE. The ranking already chose the paths, and re-scoring the ≤limit winners’ chunks is brute force over a handful of rows.

Two forms, mirroring buildRrfSql’s degradation:

With a query vector, ?1 is the vector and paths bind from ?2. The LEFT JOIN keeps a chunk whose embedding is missing, with dist NULL. A sparse vector plane is a legal state, produced by --no-embed or a Bedrock outage mid-index. The caller treats NULL as “worst”, so such a file still gets its ordinal-0 text rather than vanishing from the snippet map. The CASE guard is what keeps vector_distance_cos from ever seeing a NULL blob.
Without one, paths bind from ?1 and only the ordinal-0 chunk comes back, the file’s opening text, which on this corpus is almost always the whole article (CHUNK_MAX_CHARS).

The winner-per-path fold happens in the caller rather than in SQL. PARTITION BY windows are unprobed on this driver, and a JS Map over ≤limit×chunks rows is the cheaper thing to be sure of.

Returns undefined for zero paths, the same contract as buildRrfSql. The caller must treat that as an empty result rather than assemble IN ().

Parameters

options

hasQueryVector

boolean

pathCount

number

Returns

string | undefined

chunkIdFor()

1
function chunkIdFor(contentHash, ordinal): string;

The chunk id for a (content_hash, ordinal) pair.

The colon separator is what keeps the mapping injective: without it, hash …ab at ordinal 1 and hash …ab1 at ordinal "" would be indistinguishable inputs. content_hash already carries its own sha256: prefix, so the digest input is unambiguous end to end.

Parameters

contentHash

string

ordinal

number

Returns

string

chunkText()

1
function chunkText(
2
   text,
3
   contentHash,
4
   maxChars?
5
): readonly Chunk[];

Split article text into chunks of at most CHUNK_MAX_CHARS characters, no overlap.

An entry short enough to be one chunk is the overwhelmingly common case, because the format is one fact per file. The fast path returns a single chunk 0 whose text is the whole article, and the embedding is then a function of the article rather than of an arbitrary window.

Longer text splits on sentence-ish boundaries, greedily packing whole sentences into each chunk. A sentence longer than the ceiling is hard-cut rather than dropped, so the function is total and no text is ever lost from the index.

Parameters

text

string

contentHash

string

maxChars?

number = CHUNK_MAX_CHARS

Returns

readonly Chunk[]

disclosureTextFor()

1
function disclosureTextFor(doc): string;

The recall disclosure body, meaning what memory_recall may QUOTE rather than what it may search.

body_text is the search surface and includes everything. Disclosure is narrower, and each of the two exclusions has a reason:

<details> bodies never appear. That is Tier 3, the “how this was learned” provenance, and it reaches an agent only through memory_read. Spending a shared character budget on one memory’s backstory starves the claims of memories the agent has not seen at all.
<aside> texts never appear. An aside is a scope caveat. A disclosure line has no room to say “this is the exception”, so quoting one presents the exception as the rule.

Composed from the doc’s separated extraction fields rather than by re-deriving them from the markup. @memhtml/html reports summaryTexts, facets, and citations apart from bodyText so a consumer can build a narrower view without a second parser. A second parser here would be a consumer reimplementing producer semantics, which is the mistake the fleet has paid for repeatedly.

The composition is claim-first and structured: the <mark> claim, then each <summary> headline, then the <dl> facets as name: value, then the citations. That is the memory’s substance in the form the format already gives it, and every part of it sits outside a <details> body and outside an <aside>.

Parameters

doc

MemoryDoc

Returns

string

entityRowsFor()

1
function entityRowsFor(doc): readonly object[];

The entity rows a doc claims. Its memhtml-entity metas, one concept:<term> row per <dfn>, and one lang:<value> row per <code data-lang>.

Promoting defined terms is what makes a semantic memory that DEFINES a term findable by that term without the author also writing a memhtml-entity meta. The <dfn> already said it, and asking for it twice is how the two drift apart. data-lang promotes on the same reasoning. The fence info string already named the language, so memhtml list --entity lang:ts finds every memory carrying TypeScript with no new query machinery and no restatement.

An entity with no type: separator is stored under the unknown type rather than dropped. The name is still a real handle a query can use, and dropping it would silently lose a hand-authored file’s only entity.

Names are stored AS AUTHORED, deliberately. entity-resolution rewrites a file’s memhtml-entity meta to its normalized form and finds that work by reading these rows back — so a projection that normalized here would hide every unnormalized meta from the one pass whose job is to fix it, and the phase would report namesNormalized: 0 over a corpus full of them. Casing is folded at COMPARE time instead (assembleScope, listMemories), where it costs a query nothing and blinds nobody.

Parameters

doc

MemoryDoc

Returns

readonly object[]

facetConditions()

1
function facetConditions(
2
   facets,
3
   alias,
4
   placeholder
5
): readonly string[];

The facet predicates for one filter list: one EXISTS per distinct NAME, values OR-ed inside it.

Shared by assembleScope and listMemories, parameterized on the alias and on how the caller emits a placeholder, because those are the only two things that differ between the four-arm filter (a {alias} token and numbered ?5-upward parameters) and the listing (a real alias and anonymous ?). Two hand-written copies would agree on the day they were written and disagree the first time the composition rule moved, and a caller cannot tell a subset from a superset by reading the rows back.

The grouping is what implements AND-across-names / OR-within-name: one EXISTS per name is a conjunction because the conditions are joined by AND, and value IN (…) inside it is the disjunction. A single EXISTS over (name, value) IN (…) would make every pair OR, so doc-type=runbook tier=1 would return every runbook plus every tier-1 memory.

Duplicate (name, value) pairs collapse. A repeated flag is a caller typing the same narrowing twice, and binding it twice would put two identical values in one IN list, which changes no rows and makes the assembled SQL depend on how many times someone pressed up-arrow.

Parameters

readonly FacetFilter[]

alias

string

placeholder

(value) => string

Returns

readonly string[]

foldDisclosure()

1
function foldDisclosure(
2
   candidates,
3
   budgetChars,
4
   maxPerEntity?
5
): DisclosureFold;

Fold ranked candidates into quotes and index lines under one character budget.

Rank order is authoritative. A candidate is never promoted past a better-ranked one to make it fit. A candidate that does not fit becomes an index line and the fold CONTINUES, so a later, shorter candidate can still be quoted. The budget is a character budget and not a position cut-off. Without that, one long memory in the middle of the list would silently truncate every shorter one after it.

maxPerEntity applies to full quotes only. A capped memory still gets its index line, so the cap narrows depth rather than dropping the memory.

Parameters

candidates

readonly DisclosureCandidate[]

budgetChars

number

maxPerEntity?

number = MAX_PER_ENTITY

Returns

DisclosureFold

formatCoverage()

1
function formatCoverage(coverage): string;

0.4 as 40% and 0.949 as 94.9%, for log lines and failure reasons. One decimal rather than a whole percent, so a ratio just under the floor never prints as the floor in the same sentence that says it is below it.

Parameters

coverage

number

Returns

string

ftsQueryForms()

1
function ftsQueryForms(query): FtsQueryForms;

Parameters

query

string

Returns

FtsQueryForms

ftsTextFor()

1
function ftsTextFor(doc): string;

1
function makeDatabase(
2
   databasePath,
3
   migrationsDir,
4
   state?
5
): Effect<DatabaseShape, StorageFailure, Scope>;

A scoped connection with foreign_keys on and every pending migration applied. Exported so tests drive the real driver against ":memory:". A fake would verify the shape of these calls and not the driver’s own constraint enforcement.

state attaches the durable plane over the same connection. Omit it for a caller that only reads the rebuildable index. The salience arm and every state.* write then have nothing to bind to, which is why DatabaseShape.hasState is on the service and the arm registry consults it rather than assuming.

Parameters

databasePath

string

migrationsDir

string

state?

migrationsDir

string

path

string

Returns

Effect<DatabaseShape, StorageFailure, Scope>

makeGitPort()

1
function makeGitPort(deps): GitPort;

Map a store GitShape onto the indexer’s port.

1
function parseFacetFilter(spec): FacetFilter | undefined;

One name=value spec as a FacetFilter, or undefined when it is not one.

The split is at the FIRST separator, so a VALUE may contain = (query=a=b is a=b under query) and a NAME may not. That asymmetry is the useful direction: a <dd> holds arbitrary authored text while a <dt> is a key a consumer chooses, and it can choose one without an =.

Both halves are trimmed and an empty half is a rejection, so =x, x=, and x yield nothing. Silently dropping a malformed spec is the same discipline SearchScope.tags follows for a blank tag: a scope value that cannot narrow anything must not narrow everything.

Parameters

spec

string

Returns

FacetFilter | undefined

parseFacetFilters()

1
function parseFacetFilters(specs): readonly FacetFilter[];

Every parseable name=value spec, in order. The one wire-form decode both doors call.

Parameters

specs

readonly string[]

Returns

readonly FacetFilter[]

parseSidecar()

1
function parseSidecar(contents): object;

Parse a sidecar back into entries, for memhtml state import.

Defensive per line: an unparseable line is skipped and counted instead of failing the import. The sidecar is the only durable copy of this plane, so a file truncated by an interrupted write must restore every row it does hold. Refusing the whole file would turn a partial loss into a total one.

Parameters

string

Returns

object

entries

1
readonly entries: readonly SidecarEntry[];

skipped

1
readonly skipped: number;

persistScanned()

1
function persistScanned(
2
   db,
3
   scanned,
4
   mergeTail,
5
   indexedAt
6
): Effect<PersistOutcome, StorageFailure>;

Persist one scanned file, meaning the traces row, its trace_prompts, and its watermark.

The three actions are genuinely different writes:

skip: the file was not opened, so there is no extract. Nothing is written at all, including the watermark. The stored one already describes this exact file, and rewriting it would move scanned_at on a file nobody read.
rescan: the extract describes the whole file and REPLACES the stored row outright, prompts delete-and-inserted. Merging here would fold the file into a stale copy of itself.
tail: the extract describes only the appended slice. The stored row is read back and mergeTail combines them. Writing the tail’s extract directly would reset first_prompt to a mid-conversation prompt, move started_at forward, and collide every prompt at ordinal 0.

A session with no id is dropped. traces.session_id is the primary key, and a file-history-* -only file has no session to be about.

file_path on the row is the MAIN transcript’s. A session’s sidecars are separate scanned files upserting into one row, and letting a sidecar claim the row’s file_path would point the citation at a subagent’s slice of the conversation.

Parameters

db

DatabaseShape

scanned

ScannedFileLike

mergeTail

TailMerger

indexedAt

string

Returns

Effect<PersistOutcome, StorageFailure>

projectFile()

1
function projectFile(input): FileProjection;

Project one file onto its complete row set.

archived is read from the path’s PARA bucket, not from memhtml-status. The path is the state, because eviction IS the git mv into archive/<YYYY>/. A file whose head says active while sitting under archive/ is stale metadata and the tree is right. Trusting the meta instead would let a mis-stamped file re-enter retrieval and break the partial unique index’s dedup guarantee.

Parameters

input

readIndexState()

1
function readIndexState(db): Effect<IndexStateRow | undefined, StorageFailure>;

Read the watermark row, or undefined before the first rebuild.

A malformed row becomes a StorageFailure rather than a schema error crossing the port. Callers already handle that channel, and a row this table cannot produce is a storage problem from every caller’s point of view. Whether an absent row is an error is the CALLER’s policy. The indexer declines to write against a missing watermark, while the two report paths render it as “not yet indexed”, so this returns undefined and decides nothing.

Parameters

db

DatabaseShape

Returns

Effect<IndexStateRow | undefined, StorageFailure>

readStoredExtract()

1
function readStoredExtract(db, sessionId): Effect<SessionExtractLike | null, StorageFailure>;

Reconstruct a stored session’s extract from its rows, for the tail merge.

Only the fields the merge reads are reconstructed. counters is not, because it is scan bookkeeping rather than session content and the row does not carry it. That is why the merger is typed on SessionExtractLike. The persisted row is genuinely a subset of what a fresh parse yields, and pretending otherwise would mean inventing counter values the database never saw.

Parameters

db

DatabaseShape

sessionId

string

Returns

Effect<SessionExtractLike | null, StorageFailure>

readVectorCoverage()

1
function readVectorCoverage(db, embedWatermark?): Effect<VectorCoverage, StorageFailure>;

Read the coverage in one statement.

embedWatermark is the configured space, @memhtml/llm’s EMBED_WATERMARK in production. A caller that does not know it (retrieval built without one, in a test) gets the STORED watermark, which is the same string whenever the indexer has been allowed to write, because the indexer refuses to write across a mismatch. The coalesce keeps that fallback inside the one statement rather than as a second query whose answer could straddle a rebuild.

Parameters

db

DatabaseShape

embedWatermark?

string

Returns

Effect<VectorCoverage, StorageFailure>

readWatermark()

1
function readWatermark(db): (filePath) => Effect<WatermarkLike | null, StorageFailure>;

The trace_watermarks reader @memhtml/traces’s scanTraceRoot takes as its callback.

Parameters

db

DatabaseShape

Returns

(filePath) => Effect<WatermarkLike | null, StorageFailure>

reinforce()

1
function reinforce(
2
   db,
3
   paths,
4
   signal,
5
   at,
6
   cooldownSeconds?
7
): Effect<ReinforceResult, StorageFailure>;

Bump the access bookkeeping for every path past its cooldown.

RETURNING is what makes the split authoritative rather than inferred. The conditional upsert decides in the database, at the instant of the write, and reports which rows it actually touched. Reading last_accessed_at first and deciding in TypeScript would race a concurrent reinforce and report a bump that never happened.

BATCHED by signature: one statement per REINFORCE_PATH_BATCH paths, never one per path. A multi-row upsert evaluates its DO UPDATE … WHERE per conflicting row, so each path is still decided on its own stamp, and RETURNING names exactly the rows that moved. A per-path statement is the store-scaled per-op shape this package refuses everywhere else (see the indexer’s pending scan): a recall reinforcing forty hits would pay forty round trips to answer one question.

at is passed in rather than read from the clock so a caller can pin the instant. The cooldown boundary test needs to name a time exactly REINFORCE_COOLDOWN_S after the stored stamp.

reinforcement_count increments and outcome_score moves only on a non-neutral signal. Being read is evidence of relevance and not of correctness, so a plain retrieval bumps access without claiming the memory was right.

Parameters

db

DatabaseShape

paths

readonly string[]

signal

"positive" | "negative" | "neutral"

at

string

cooldownSeconds?

number = REINFORCE_COOLDOWN_S

Returns

Effect<ReinforceResult, StorageFailure>

renderSidecar()

1
function renderSidecar(rows): string;

Effect<number, StorageFailure, never>

sanitizeFtsQuery()

1
function sanitizeFtsQuery(query): string;

The MATCH-safe form of a query, or "" when the query holds no indexable term.

string

Returns

Write