@memhtml/head

@memhtml/head: the corpus as a value, one version per commit held in persistent effect structures. A version is loaded from git, advanced by the paths two commits differ on, and overlaid with a session’s pending ops. It is searched by reciprocal-rank fusion over a lexical arm and an optional vector arm, with recency breaking exact ties, and the head gate compares two versions. The design is docs/design.md §16.

HeadVersion is one version: the read-only view interface @memhtml/contracts defines, answered from the version’s Indexes, plus its commit sha and its parent version (a pointer, so a caller can walk back). skippedFiles lists the candidate files left out because they failed to parse or their blob was missing from the batch read, each a SkippedFile with its path and reason, recorded rather than thrown, and skipped is their count. isCandidatePath decides which tree paths are parsed: a path ending in .html, not under a top-level .memhtml/, whose file name is not index.html. loadHead builds the version at a commit from a full tree read: one ls-tree and one batched cat-file read through @memhtml/store’s git service, folding insertRecord over the parsed records in tree order. advanceHead builds the version at a later commit from its parent. It lists the paths the two trees differ on with one git diff-tree -r -z --name-only --no-renames, so a rename contributes both of its paths. It removes the changed candidates absent at the new commit and re-parses the rest unless the blob sha equals the parent record’s, and each unchanged path keeps the parent’s record object. A changed file that fails to parse leaves the version and joins skippedFiles, and a target equal to the parent’s sha returns the parent itself. versionFromRecords builds a version from records already parsed, with no parent and skipped 0, by the same insertRecord fold.

recordFrom turns one file into the flat record type @memhtml/contracts defines, and the versions and overlays in this package parse through it. It runs the @memhtml/html parser and fails with the parser’s own error. To the parsed fields it adds the git blob sha of the text it is given, the article’s content hash, the claim’s frame key, and the archive flag from isArchivedPath, which is true for a path under archive/. blobShaOf computes the blob sha locally, sha1 over blob <byteLength>\0 followed by the UTF-8 bytes, because an overlay holds files git has not stored yet. It counts UTF-8 bytes rather than string length, so multibyte content agrees with git.

Indexes is the record map and the indexes derived from it, as one value of persistent effect maps and sets, so a change to one record builds a new value that shares the untouched structure. Its fields are records by path, byContentHash (content hash to an active path holding it), byFrameKey (frame key to the active paths whose claim shares it), inbound (href, as authored, to the paths linking to it), byEntity (entity to the paths naming it), and lexical. emptyIndexes is the empty value, and insertRecord and removeRecord are its two writers. insertRecord replaces any record at the same path, and an archived record enters only records, inbound and byEntity. A content hash goes to the first active record inserted while no path holds it; removing another record that shares it leaves it held, and removing the holder leaves it unmapped even while a twin stays active. viewOf puts the read surface over one Indexes value and a sha, with byFrameKey, inbound and byEntity returning paths sorted ascending. indexesOf returns the indexes behind a view: it reuses them for a view built here and rebuilds them by insertRecord over its records otherwise, which is how searchHead and withOverlay accept a caller’s own view.

LexicalIndex is the lexical arm’s persistent index over active records: per-token postings with each path’s term frequency, and per-path token counts with their total, so BM25 reads term frequency and average length at query time without re-tokenizing a body. A record is indexed by its title, claim and body text. tokenize lowercases and splits on runs of characters other than Unicode letters, numbers and underscore. It drops empty tokens and the 57 STOP_WORDS, a closed list of English function words kept short because a longer one starts dropping content words a memory corpus is searched by.

withOverlay is a session’s view: the base view with an ordered list of overlay ops (the op type @memhtml/contracts defines) applied through insertRecord and removeRecord. It keeps the base’s sha, because an overlay has no commit of its own, and fails on the first op that does not parse or resolve. put parses a file and inserts it at its path. archive removes the old path and inserts the given html, parsed at the destination path, as an archived record. link and unlink add or cut an edge in the file’s head, and label and unlabel a memhtml-entity meta, then re-parse the file; a link or label naming an edge or entity the file already carries changes nothing. All four fail on a path the view does not hold, link and unlink also on a rel outside the edge vocabulary, label on a malformed entity, and unlink and unlabel on an edge or entity the file does not carry. move re-parses the record’s current html, including earlier ops’ head edits, at its new path, and fails unless the old path is held and not archived.

searchHead ranks the active records of a view and returns SearchHit rows. It is synchronous and does not embed, so the caller owns any vectors. A SearchHeadInput carries the query, a limit (default DEFAULT_LIMIT, 10), an optional now, an optional admit predicate, and optionally vectors, a HeadVectors whose get looks a vector up by content hash, with a queryVector in the same space. admit narrows the records each arm ranks before its cap, while BM25’s statistics stay those of all active records. The lexical arm is BM25 (k1 of 1.2, b of 0.75) over the lexical index, counting each distinct query token once, and keeps its best LEXICAL_ARM_LIMIT (40). The vector arm runs only when both vectors and queryVector are set: it scores each admitted active record that has a vector by cosine against the query vector and keeps its best VECTOR_ARM_LIMIT (40). Within an arm, records with equal scores share the best rank of their run (1, 1, 3). The arms are fused by reciprocal-rank fusion at k = 60, with LEXICAL_WEIGHT and VECTOR_WEIGHT both 1.0, and without a vector arm the fold is the lexical arm alone.

Recency only breaks exact ties. Hits are ordered by fused score, then newest first by the record’s event time, or its update time when it has none (compared as strings, and clamped to now when the input sets it), then by path ascending. A score difference is not overturned and no record is added, so a query no arm matches returns nothing. Each SearchHit carries its path, fused score, claim, and arms, the SearchArm values behind its place: lexical and vector for the arms that ranked it, and recency when its fused score equals another candidate’s, which is when recency or path decided its position.

fillVectors is the one step in the package that calls an embedder, which keeps searchHead free of one. vectorTextOf is the text a record embeds: its article text, trimmed and cut to the first VECTOR_TEXT_MAX_CHARS (1,800) UTF-16 code units, the same number as @memhtml/index’s chunk ceiling. The title is left out because it lives in the file’s head, outside what the content hash digests, so a vector that read it could go stale under a key that does not change. A VectorCache holds the vectors of one VectorSpace (a modelId and a dimension) by content hash, so an unchanged article keeps its vector across versions, moves and head-only edits; emptyVectorCache starts one. fillVectors embeds, through a DocumentEmbedder, the active records whose content hash the cache lacks. DocumentEmbedder is the half of @memhtml/llm’s embeddings service a fill needs, typed structurally so this package does not import @memhtml/llm. A fill makes at most one embed call, with one text per distinct content hash, and skips archived records and records whose trimmed article text is empty. It returns a FillResult: the cache with the new vectors added beside the ones it held (vectors for hashes the view no longer holds stay), the FillCounts, and remaining, the distinct texts a limit held back for a later fill. An embedder that answers with the wrong number of vectors or a vector of the wrong width ends the fill as a defect instead of storing anything.

gateHeads is the head gate: it asks whether a landed version still finds what its base found, over two versions of the same corpus rather than a fixture. probesFor takes the records active in both versions as HeadGateProbe rows (a path and a query), so a record the landing adds or archives is not probed. It orders them by the first eight hex digits of sha1 over <seed>:<path>, with path breaking ties, and keeps the first count, so the sample is a fixed function of the seed and the paths. probeQueryFor gives each probe its query from the landed version’s record: the title, else the first sentence of the claim, else the claim. scoreProbes runs searchHead at one version for each probe with a limit of k and no vectors, so the gate needs no embedder. scoreProbes returns the ranks, the MRR (rounded to four places, 0 with no probes) and two inversion kinds: fell-out when the target is not in the top k, and superseded-outranks when a record named by one of the target’s supersedes links ranks above the target in the top k, or appears there while the target does not.

gateHeads scores the same probes at both versions. It passes when the landed MRR reaches the floor and the landed version has no more inversions than the base. The floor is the base MRR minus the tolerance, clamped at zero and rounded to four places; the tolerance allows for a landing that archives a near-duplicate and so shifts BM25’s document frequencies slightly. HeadGateOptions overrides probes, k, tolerance and seed, which default to HEAD_GATE_PROBES (200), HEAD_GATE_K (10), HEAD_GATE_TOLERANCE (0.02) and HEAD_GATE_SEED (1). A HeadGateReport carries both MRRs, the floor, the landed inversions beside the base’s count, and one HeadGateResult per probe. describeHeadGateFailure renders a report as one line: the probe count, then the MRR shortfall and the inversion growth (with up to five inversions) when they occurred, or passed. HeadGateFailed wraps a report as a value tagged HeadGateFailed whose reason is that line, which the CLI maps to ERR_DISCRIMINATION_FAILED.

Modules

The package publishes one import path, @memhtml/head, whose index.ts re-exports a chosen set of names from the other eight modules in the table below, so the reference on this page is the whole exported surface. Names a module defines that index.ts does not re-export are left off the page, which is why the five the table names as not exported appear unlinked where a signature or doc comment mentions them.

Module	What it holds
gate.ts	The defaults (HEAD_GATE_PROBES, HEAD_GATE_K, HEAD_GATE_TOLERANCE, HEAD_GATE_SEED), the shapes HeadGateOptions, HeadGateProbe, HeadGateInversion, HeadGateResult and HeadGateReport, probeQueryFor, probesFor, scoreProbes, gateHeads, describeHeadGateFailure and HeadGateFailed, plus Scored, the result type scoreProbes returns, which is not exported.
indexes.ts	Indexes, emptyIndexes, insertRecord, removeRecord, viewOf and indexesOf, plus IndexedView, the view type viewOf returns and HeadVersion extends, which is not exported from @memhtml/head.
lexical.ts	LexicalIndex, STOP_WORDS and tokenize, plus addToLexical and removeFromLexical, the lexical writers insertRecord and removeRecord call, which are not exported from @memhtml/head.
overlay.ts	withOverlay.
record.ts	recordFrom, blobShaOf and isArchivedPath.
search.ts	The shapes SearchArm, SearchHit, HeadVectors and SearchHeadInput, the constants LEXICAL_WEIGHT, VECTOR_WEIGHT, LEXICAL_ARM_LIMIT, VECTOR_ARM_LIMIT and DEFAULT_LIMIT, and searchHead, plus armRanks, the shared-rank step both arms use, which is not exported.
vectors.ts	VECTOR_TEXT_MAX_CHARS, vectorTextOf, the shapes VectorSpace, VectorCache, DocumentEmbedder, FillCounts and FillResult, emptyVectorCache and fillVectors.
version.ts	HeadVersion, SkippedFile, isCandidatePath, loadHead, versionFromRecords and advanceHead.
index.ts	The re-exports above, type-only for the interfaces and type aliases.

Classes

HeadGateFailed

A failed head gate, as a tagged error the CLI maps to ERR_DISCRIMINATION_FAILED, the code every refused landing carries. The whole report rides along, because a refusal an operator cannot read is a refusal they override.

Constructors

Constructor

new HeadGateFailed(report): HeadGateFailed;

Parameters

report

HeadGateReport

Returns

HeadGateFailed

Properties

_tag

readonly _tag: "HeadGateFailed" = "HeadGateFailed";

report

readonly report: HeadGateReport;

Accessors

reason

Get Signature

get reason(): string;

Returns

string

Interfaces

DocumentEmbedder

The half of @memhtml/llm’s EmbeddingsShape a fill needs, structurally, so this package never imports it.

Type Parameters

E

E

Properties

embed

readonly embed: (texts) => Effect<readonly Float32Array<ArrayBufferLike>[], E>;

Parameters

texts

readonly string[]

Returns

Effect<readonly Float32Array<ArrayBufferLike>[], E>

---

FillCounts

What one fill did, counted over the view’s active records.

Properties

embedded

readonly embedded: number;

Distinct texts sent to embed; two active records sharing a content hash cost one.

empty

readonly empty: number;

Active records with no text to embed (an empty article), which the arm never ranks.

records

readonly records: number;

Active records in the view.

reused

readonly reused: number;

Active records whose content hash already had a vector.

---

FillResult

Properties

cache

readonly cache: VectorCache;

counts

readonly counts: FillCounts;

remaining

readonly remaining: number;

Distinct texts still missing after this fill: 0 unless limit held some back, which the next fill picks up.

---

HeadGateOptions

@memhtml/head: the corpus as a value. One immutable version per commit, held as persistent effect structures, advanced by delta, read through HeadView, searched by RRF over a lexical and (given vectors) a vector arm, with recency breaking exact ties. See docs/design.md §16.

Properties

k?

readonly k?: number;

The top-k a target must appear in.

probes?

readonly probes?: number;

How many probes to sample. Fewer when the comparable set is smaller.

seed?

readonly seed?: number;

tolerance?

readonly tolerance?: number;

How far below the base MRR the landed MRR may sit and still pass.

---

HeadGateProbe

@memhtml/head: the corpus as a value. One immutable version per commit, held as persistent effect structures, advanced by delta, read through HeadView, searched by RRF over a lexical and (given vectors) a vector arm, with recency breaking exact ties. See docs/design.md §16.

Extended by

• HeadGateResult

Properties

path

readonly path: string;

query

readonly query: string;

---

HeadGateReport

@memhtml/head: the corpus as a value. One immutable version per commit, held as persistent effect structures, advanced by delta, read through HeadView, searched by RRF over a lexical and (given vectors) a vector arm, with recency breaking exact ties. See docs/design.md §16.

Properties

added

readonly added: number;

Active records at the landed version that were not active at the base.

base

readonly base: string | null;

floor

readonly floor: number;

mrrBefore - tolerance, clamped at zero. The landed MRR must reach it.

inversions

readonly inversions: readonly HeadGateInversion[];

inversionsBefore

readonly inversionsBefore: number;

k

readonly k: number;

landed

readonly landed: string | null;

mode

readonly mode: "head";

mrr

readonly mrr: number;

Mean reciprocal rank of the probes at the landed version, four decimals; 0 with no probes.

mrrBefore

readonly mrrBefore: number;

The same probes at the base version.

passed

readonly passed: boolean;

probes

readonly probes: number;

removed

readonly removed: number;

Active records at the base that are not active at the landed version.

results

readonly results: readonly HeadGateResult[];

seed

readonly seed: number;

tolerance

readonly tolerance: number;

---

HeadGateResult

@memhtml/head: the corpus as a value. One immutable version per commit, held as persistent effect structures, advanced by delta, read through HeadView, searched by RRF over a lexical and (given vectors) a vector arm, with recency breaking exact ties. See docs/design.md §16.

Extends

• HeadGateProbe

Properties

path

readonly path: string;

Inherited from

HeadGateProbe.path

query

readonly query: string;

Inherited from

HeadGateProbe.query

rankAfter

readonly rankAfter: number | null;

rankBefore

readonly rankBefore: number | null;

---

HeadVectors

Vectors by content hash, the key vectors.ts gives each one. A ReadonlyMap fits; so does any lookup a long-lived head process holds, which is why the input is this narrow.

Methods

get()

get(contentHash): ArrayLike<number> | undefined;

Parameters

contentHash

string

Returns

ArrayLike<number> | undefined

---

HeadVersion

One immutable version of the corpus, built from one commit.

loadHead builds the first version from a full tree read; advanceHead builds every later one from its parent plus the paths the two commits differ on, sharing every untouched record and index node with the parent. parent is kept so a caller can walk back; it is a pointer, not a copy, so a chain of versions costs the deltas and nothing more.

Extends

• IndexedView

Properties

[INDEXES]

readonly [INDEXES]: Indexes;

Inherited from

IndexedView.[INDEXES]

lexical

readonly lexical: LexicalIndex;

Inherited from

IndexedView.lexical

parent

readonly parent: HeadVersion | null;

sha

readonly sha: string;

The commit this view was built from; null for a view with no git ancestry (tests, overlays on nothing).

Overrides

IndexedView.sha

size

readonly size: number;

Inherited from

IndexedView.size

skipped

readonly skipped: number;

Files that were candidates and failed parseMemory. Skipped, counted, never thrown.

skippedFiles

readonly skippedFiles: readonly SkippedFile[];

Methods

byContentHash()

byContentHash(hash): string | undefined;

Active (non-archived) record holding this content hash, if any.

Parameters

hash

string

Returns

string | undefined

Inherited from

IndexedView.byContentHash

byEntity()

byEntity(entity): readonly string[];

Parameters

entity

string

Returns

readonly string[]

Inherited from

IndexedView.byEntity

byFrameKey()

byFrameKey(key): readonly string[];

Active records whose claim shares this frame key.

Parameters

key

string

Returns

readonly string[]

Inherited from

IndexedView.byFrameKey

get()

get(path): MemoryRecord | undefined;

Parameters

path

string

Returns

MemoryRecord | undefined

Inherited from

IndexedView.get

inbound()

inbound(href): readonly string[];

Paths whose links point at this href (root-relative form).

Parameters

href

string

Returns

readonly string[]

Inherited from

IndexedView.inbound

paths()

paths(): Iterable<string>;

Returns

Iterable<string>

Inherited from

IndexedView.paths

records()

records(): Iterable<MemoryRecord>;

Returns

Iterable<MemoryRecord>

Inherited from

IndexedView.records

---

Indexes

The persistent record map and every index derived from it, as one value.

Each field is an effect HashMap or HashSet, so a change to one record builds a new Indexes that shares every untouched node with the old one. insertRecord and removeRecord are the only two writers; every version and every overlay is a fold of those two over a parent.

Properties

byContentHash

readonly byContentHash: HashMap<string, string>;

content hash -> the ACTIVE path holding it. Archived records never enter.

byEntity

readonly byEntity: HashMap<string, HashSet<string>>;

entity (type:name, as authored) -> paths naming it. All records.

byFrameKey

readonly byFrameKey: HashMap<string, HashSet<string>>;

frame key -> ACTIVE paths whose claim shares it.

inbound

readonly inbound: HashMap<string, HashSet<string>>;

href (root-relative, as authored) -> paths whose links point at it. All records.

lexical

readonly lexical: LexicalIndex;

Active records only, see lexical.ts.

records

readonly records: HashMap<string, MemoryRecord>;

---

LexicalIndex

The lexical index: token to the paths that carry it, with each path’s term frequency, plus the per-document lengths BM25 normalizes by.

Every field is a persistent effect structure, so a version built from its parent by removeFromLexical and addToLexical shares every bucket the change did not touch. The bucket for one token is HashMap<path, tf> rather than a bare path set because BM25 needs the term frequency at query time and re-tokenizing a candidate per query would cost the whole body.

Properties

lengths

readonly lengths: HashMap<string, number>;

path -> token count after stop-word removal. Its size is the document count.

postings

readonly postings: HashMap<string, HashMap<string, number>>;

token -> path -> term frequency. Active records only.

totalLength

readonly totalLength: number;

Sum of every length, so the average length is O(1) at query time.

---

SearchHeadInput

Properties

admit?

readonly admit?: (record) => boolean;

Which active records the search is over. Applied inside each arm, before the arm’s cap, so a scope narrows the candidates every arm ranks rather than the fused list after it (v1’s assembleScope reaches every arm the same way). BM25’s statistics stay those of every active record, as FTS5’s do under a v1 scope. Absent admits every active record.

Parameters

record

MemoryRecord

Returns

boolean

limit?

readonly limit?: number;

now?

readonly now?: string;

query

readonly query: string;

queryVector?

readonly queryVector?: ArrayLike<number>;

The query in the same vector space as vectors.

vectors?

readonly vectors?: HeadVectors;

Vectors by content hash. The vector arm runs only when this and queryVector are both set.

---

SearchHit

Properties

arms

readonly arms: readonly SearchArm[];

The arms that ranked the hit, and recency when its fused score equals another candidate’s, which is when recency (or path after it) decided its place.

claim

readonly claim: string;

path

readonly path: string;

score

readonly score: number;

---

SkippedFile

Properties

path

readonly path: string;

reason

readonly reason: string;

---

VectorCache

Vectors of one space, by content hash.

Properties

space

readonly space: VectorSpace;

vectors

readonly vectors: ReadonlyMap<string, Float32Array<ArrayBufferLike>>;

---

VectorSpace

One embedder’s vector space. Two caches with different spaces hold incomparable vectors.

Properties

dimension

readonly dimension: number;

modelId

readonly modelId: string;

Type Aliases

HeadGateInversion

type HeadGateInversion =
  | {
  kind: "fell-out";
  path: string;
  query: string;
}
  | {
  by: string;
  byRank: number;
  kind: "superseded-outranks";
  path: string;
  query: string;
  rank: number | null;
};

@memhtml/head: the corpus as a value. One immutable version per commit, held as persistent effect structures, advanced by delta, read through HeadView, searched by RRF over a lexical and (given vectors) a vector arm, with recency breaking exact ties. See docs/design.md §16.

Union Members

Type Literal

{
  kind: "fell-out";
  path: string;
  query: string;
}

---

Type Literal

{
  by: string;
  byRank: number;
  kind: "superseded-outranks";
  path: string;
  query: string;
  rank: number | null;
}

by

readonly by: string;

The superseded record that ranked above its superseder.

byRank

readonly byRank: number;

kind

readonly kind: "superseded-outranks";

path

readonly path: string;

query

readonly query: string;

rank

readonly rank: number | null;

The target’s rank, null when it was not in the top k.

---

SearchArm

type SearchArm = "lexical" | "recency" | "vector";

Retrieval over one view: a lexical arm and, when the caller passes vectors and a query vector, a vector arm, fused with reciprocal rank fusion at k = 60, weights 1.0 and 1.0, over active records only, with recency as the tie-break.

The lexical arm is BM25 (Robertson/Sparck Jones idf, k1 = 1.2, b = 0.75) over the title, the claim, and the body, read from the version’s persistent lexical index, and keeps the best LEXICAL_ARM_LIMIT. The vector arm orders the active records that have a vector by cosine against the query vector, brute force, and keeps the best VECTOR_ARM_LIMIT. In both arms records with equal scores share one rank (see armRanks), so two records an arm cannot tell apart get the same fused share.

The rule: recency decides only between hits whose fused score is exactly equal, newest first by eventAt ?? updatedAt (clamped to now), and path ascending decides after that. It never overturns a score difference and never adds a record, so a query no arm matches returns nothing. A byte-identical newer twin therefore outranks the older copy, and a clearly better older match stays above a newer, weaker one.

Why not a weighted recency arm. Until 2026-09-28 recency was an RRF arm at weight 0.5 that ranked every lexical and vector candidate newest first. That made it a second full ranking rather than a tie-break, and a curation landing that made the surviving records newer made older targets lose to newer, weaker matches. On a 2026-09-28 rehearsal of a large collapse on a copy of a production store (the head gate’s 200 title probes, seed 1, top 10) the weighted arm scored MRR 0.5572 at the base and 0.4384 landed, so the gate refused the landing; this rule scores 0.9449 and 0.9485 without the vector arm and 0.9492 and 0.9522 with it, and on a 2026-09-23 copy of the same store it raised the gate probes from 0.4557 to 0.9975 without the vector arm and from 0.8321 to 0.9905 with it. Smaller weights land near the same numbers (0.05 scored 0.9374 and 0.9410 without the vector arm), but without shared ranks no weight small enough to be a tie-break lets a newer twin win, and letting recency reorder hits within 0.0003 of each other dropped two-arm MRR at the base to 0.6769 (docs/design.md §16, “Recency as the tie-break”).

The function stays pure: it never embeds. The caller owns the vectors (keyed by content hash, see vectors.ts) and the query vector, so a search with neither is the lexical search exactly.

Variables

DEFAULT_LIMIT

const DEFAULT_LIMIT: 10 = 10;

---

emptyIndexes

const emptyIndexes: Indexes;

---

HEAD_GATE_K

const HEAD_GATE_K: 10 = 10;

---

HEAD_GATE_PROBES

const HEAD_GATE_PROBES: 200 = 200;

The head gate: does the version being landed still find what the base version found?

The gate is stated over two versions of the same corpus, never over a fixture. Probes are a seeded sample of the records active in BOTH versions, each queried by its own title (or the first sentence of its claim when the title is blank), and searchHead is scored at the base and at the landed version with the same probes, so the two numbers differ only by what the landing changed. A record the landing adds is not probed: it has no rank at the base to compare against, and it becomes a probe at the next landing, when it is part of the base. A record the landing archives leaves the probe set for the same reason, and the probe of the record it was folded into is what measures whether the fold helped.

Two inversion kinds, both counted at both versions: a probe whose target is not in the top k (fell-out), and a record the target supersedes (a supersedes link from the target) ranking above the target (superseded-outranks). The gate passes when the landed MRR is at least the base MRR minus tolerance and the inversion count did not grow. The tolerance exists because a landing that archives a near-duplicate legitimately reshuffles BM25’s document frequencies by a hair; a drop past it is a landing that made the corpus harder to search.

Everything here is deterministic and credential-free: the sample is fixed by seed, both arms of searchHead are deterministic, and no embedder is involved.

---

HEAD_GATE_SEED

const HEAD_GATE_SEED: 1 = 1;

---

HEAD_GATE_TOLERANCE

const HEAD_GATE_TOLERANCE: 0.02 = 0.02;

---

LEXICAL_ARM_LIMIT

const LEXICAL_ARM_LIMIT: 40 = DEFAULT_ARM_LIMIT;

How many records the lexical arm ranks: v1’s DEFAULT_ARM_LIMIT, for the vector arm’s reason. BM25 scores every record that shares one token with the query, so uncapped the arm voted for hundreds of records on the live clone, and each weak match collected RRF mass that a record found by meaning alone could not: a paraphrase whose target was second in the vector arm ranked it 31st overall (docs/design.md §16, “The vector arm”).

---

LEXICAL_WEIGHT

const LEXICAL_WEIGHT: 1 = 1.0;

---

STOP_WORDS

const STOP_WORDS: ReadonlySet<string>;

English function words dropped before indexing and before scoring a query. A short closed list: a longer one starts dropping content words (system, time) that a memory corpus searches by.

---

VECTOR_ARM_LIMIT

const VECTOR_ARM_LIMIT: 40 = DEFAULT_ARM_LIMIT;

How many records the vector arm ranks: v1’s DEFAULT_ARM_LIMIT. Cosine gives every embedded record a score, so an uncapped arm would put a contribution on the whole corpus, most of it the noise far down a similarity list, and arms would name vector on every hit. The cap keeps the arm’s vote to the records it is confident about, as v1’s LIMIT does.

---

VECTOR_TEXT_MAX_CHARS

const VECTOR_TEXT_MAX_CHARS: 1800 = 1_800;

The longest text one record embeds, in characters: v1’s CHUNK_MAX_CHARS (@memhtml/index). Below it v1 embeds a record’s whole article as chunk 0, which is the common case in a one-fact-per-file corpus, so for those records the two stores embed the same text. v2 keeps one vector per record rather than v1’s chunks, so an article past the ceiling is cut, and the cut keeps the opening, where the format puts the claim.

---

VECTOR_WEIGHT

const VECTOR_WEIGHT: 1 = 1.0;

The vector arm’s weight: v1’s (vectorArm in @memhtml/index retrieval-sql.ts), so a read that moves from v1 to v2 weighs a semantic match the way it did. v1 fuses four arms at 1.0 (lexical), 1.0 (vector), 0.5 (recency), and 0.4 (salience); v2 keeps lexical and vector at v1’s weights and has no salience arm, and its recency is a tie-break rather than an arm.

Functions

advanceHead()

function advanceHead(
   git,
   from,
   to
): Effect<HeadVersion, InvalidMemory | GitFailure>;

The version at to, built from from plus the paths the two trees differ on.

Every changed path is removed from the parent’s indexes and, when it still exists at to as a candidate, re-parsed and re-inserted. A changed path whose blob sha equals the parent record’s (changed and changed back inside the range) keeps the parent’s record object. Nothing else is touched, which is the structural-sharing claim the tests check by reference.

Parameters

git

GitShape

from

HeadVersion

to

string

Returns

Effect<HeadVersion, InvalidMemory | GitFailure>

---

blobShaOf()

function blobShaOf(html): string;

The git blob sha of html: sha1 over "blob <byteLength>\0" followed by the UTF-8 bytes.

Computed locally rather than asked of git, because a session’s overlay holds files git has not seen yet and their records need the sha a later hash-object -w will produce. The byte length is the UTF-8 length and not html.length, which is what makes multibyte content agree with git.

Parameters

html

string

Returns

string

---

describeHeadGateFailure()

function describeHeadGateFailure(report): string;

The one-line account of a failed report, in the same shape the eval gate’s failure reads.

Parameters

report

HeadGateReport

Returns

string

---

emptyVectorCache()

function emptyVectorCache(space): VectorCache;

An empty cache in space, for a store that has none yet.

Parameters

space

VectorSpace

Returns

VectorCache

---

fillVectors()

function fillVectors<E>(input): Effect<FillResult, E>;

Embed every active record of view whose content hash cache lacks, in one embed call (which chunks at the model’s request ceiling and bounds its own concurrency), and return the cache with the new vectors added beside every vector it already held. limit bounds the texts one call sends, taking them in the view’s record order, and remaining counts the ones it held back, so a caller filling in the background pays for one bounded batch at a time.

Archived records are not embedded, because the arm ranks active records only. Entries for hashes the view no longer holds are kept: the cache serves every version of the store, and a record the next branch or the next rebase brings back reuses its vector rather than paying for it twice.

A vector of the wrong width is refused as a defect rather than stored, because one would make the cache disagree with its own space and every cosine against it meaningless. @memhtml/llm already refuses such a response as a ModelUnavailable, so this only fires for an embedder that is not that one.

Type Parameters

E

E

Parameters

input

cache

VectorCache

embedder

DocumentEmbedder<E>

limit?

number

view

HeadView

Returns

Effect<FillResult, E>

---

gateHeads()

function gateHeads(
   base,
   landed,
   options?
): HeadGateReport;

Run the gate over two versions. Pure; never throws for any pair of views.

Parameters

base

HeadView

landed

HeadView

options?

HeadGateOptions = {}

Returns

HeadGateReport

---

indexesOf()

function indexesOf(view): Indexes;

The indexes behind any HeadView: reused when the view was built here, rebuilt from records() when a caller supplied its own (a test’s Map-backed view).

Parameters

view

HeadView

Returns

Indexes

---

insertRecord()

function insertRecord(indexes, record): Indexes;

Put record at its path, replacing whatever was there. Archived records enter records, inbound, and byEntity only: the active-only indexes exist to answer “is this claim already live”, and an archived file is by definition not.

Parameters

indexes

Indexes

record

MemoryRecord

Returns

Indexes

---

isArchivedPath()

function isArchivedPath(path): boolean;

True when path sits under the archive bucket, which is what archived means on a record.

Parameters

path

string

Returns

boolean

---

isCandidatePath()

function isCandidatePath(path): boolean;

Whether a tree path is a memory file the head should parse.

The generated index.html is a corpus artifact and never a memory. Anything under .memhtml/ is state. Only .html blobs qualify, so .gitkeep, .gitattributes, and sitemap.xml are neither records nor skips: a census over the head counts eligible paths, not tree entries.

Parameters

path

string

Returns

boolean

---

loadHead()

function loadHead(git, sha): Effect<HeadVersion, InvalidMemory | GitFailure>;

Build the version at sha from a full tree read: one ls-tree, one cat-file --batch.

The bulk build folds insertRecord over the parsed records, the same writer advanceHead uses, so a version built from scratch and one built by advancing agree node for node.

Parameters

git

GitShape

sha

string

Returns

Effect<HeadVersion, InvalidMemory | GitFailure>

---

probeQueryFor()

function probeQueryFor(record): string;

The query a record is probed by: its title, else the first sentence of its claim, else the claim.

Parameters

record

MemoryRecord

Returns

string

---

probesFor()

function probesFor(
   base,
   landed,
   options
): readonly HeadGateProbe[];

The probes: records active in both versions, ordered by a seeded digest of their path so the sample is a fixed function of (seed, paths) and not of insertion order, the first count of them.

Parameters

base

HeadView

landed

HeadView

options

count

number

seed

number

Returns

readonly HeadGateProbe[]

---

recordFrom()

function recordFrom(input): Effect<MemoryRecord, InvalidMemory>;

Parse one file into its record. Fails with the parser’s own InvalidMemory.

Parameters

input

html

string

path

string

Returns

Effect<MemoryRecord, InvalidMemory>

---

removeRecord()

function removeRecord(indexes, path): Indexes;

Drop the record at path from every index. A path with no record is a no-op.

Parameters

indexes

Indexes

path

string

Returns

Indexes

---

scoreProbes()

function scoreProbes(
   view,
   probes,
   k
): Scored;

Score every probe at one version.

Parameters

view

HeadView

probes

readonly HeadGateProbe[]

k

number

Returns

Scored

---

searchHead()

function searchHead(view, input): readonly SearchHit[];

Parameters

view

HeadView

input

SearchHeadInput

Returns

readonly SearchHit[]

---

tokenize()

function tokenize(text): readonly string[];

Lowercase and split on runs of non-word characters, then drop stop words and empties.

The split class is Unicode-aware (\p{L} and \p{N} plus underscore) rather than the ASCII \W, so café stays one token instead of losing its last letter. Deterministic and total.

Parameters

text

string

Returns

readonly string[]

---

vectorTextOf()

function vectorTextOf(record): string;

The text a record embeds: its article’s text (bodyText, which opens with the claim), trimmed and cut at VECTOR_TEXT_MAX_CHARS.

The title is left out on purpose. It lives in <head>, outside what the content hash digests, so a vector that read it could go stale under a key that never changes: a retitled record would keep the old title’s vector. BM25 still reads the title, so a title match is not lost, only not double-counted. The article is also what v1 embeds per chunk.

Parameters

record

MemoryRecord

Returns

string

---

versionFromRecords()

function versionFromRecords(records, sha): HeadVersion;

A version built from records already parsed, for a caller holding a snapshot rather than a tree.

The fold is the same insertRecord the git load uses, so the version agrees node for node with one built from the commit, and advanceHead can carry it forward sharing every untouched node. A snapshot holds records only, so skipped is zero here: it says nothing about what the load that produced the snapshot left out, and a caller reporting the skip count of a snapshot-built version must say so.

Parameters

records

Iterable<MemoryRecord>

sha

string

Returns

HeadVersion

---

viewOf()

function viewOf(indexes, sha): IndexedView;

The read surface over one Indexes value.

Parameters

indexes

Indexes

sha

string | null

Returns

IndexedView

---

withOverlay()

function withOverlay(base, ops): Effect<HeadView, InvalidMemory>;

The base with ops applied in order. Fails on the first op that does not parse or resolve.

Parameters

base

HeadView

ops

readonly OverlayOp[]

Returns

Effect<HeadView, InvalidMemory>