--- title: @memhtml/eval --- `@memhtml/eval`: the fixture corpus generator, the refusable retrieval discrimination gate, and the write-path discrimination arm. The retrieval corpus is never committed; each run regenerates it from a seed. The write-path corpus is committed, because its labels are the oracle. The gate asks one question: can the retrieval stack rank a memory above its own high-similarity WRONG twin? `controls` turns a true claim into a plausible impostor along three divergence families (negation, numeric, variant), validates each control against its own family's predicate from `@memhtml/domain`, and discards a control that failed to diverge. `corpus` is a pure function of `(seed, now)`: it yields the same memory specs and probes on any machine, with every timestamp anchored as an offset behind the run so that salience decay cannot move the ranks from one calendar day to the next. `fixture` is the one module that writes, rendering a spec into a real git repository under a temp directory. `harness` builds the stack the gate measures (that fixture, a real database with the shipped migrations, the real indexer and the real four-arm retrieval) and substitutes only the embedder: a deterministic one whose cosine relations are a pure function of the text, so the numbers reproduce without credentials. `discriminate` runs the probes and reports two numbers, of which the strict one is the gate: `mrr` against a floor of 0.85, and a per-probe inversion count where one inversion fails the run regardless of MRR. `run` picks the mode and shapes the outcome so that a skipped gate never looks like a passing one: `fake` mode runs everywhere, and `live` mode without credentials reports `skipped: true` and `passed: false`. The write-path arm asks the other question the 2026-08-27 eval plan posed: does the WRITE path tell a good candidate memory from a bad one? `write-path-corpus` is a labeled corpus of 62 candidate memories (31 the gate should write, 31 it should refuse), each with the provenance of its label (the contract clause or design rule that decides it) and two fixture transcripts every good quote is a verbatim span of. `write-path-gate` composes the production gate from its own functions in the order the fleet runs them: the consolidator door's schema decode, readable-session check and verbatim-quote check, then the sleep phase's per-candidate refusal (`candidateRefusalFor`). Nothing is re-implemented. `write-path-metrics` scores the decisions with refusal as the positive class (precision, recall, F1, the four-cell confusion matrix, and a per-class breakdown naming the items that disagreed with their label). `write-path-run` runs the arm, derives the floor (the plan named none, so it is the measured baseline F1 0.9333 minus five points, 0.88, stated as such), and freezes every decision into `fixtures/write-path-discrimination.manifest.json`, which the `test:eval` tier replays decision for decision. `pnpm freeze:write-path` is the one way that manifest changes. Three failure classes are labeled bad and accepted by the gate today (a duplicated evidence quote, twice; a claim that is a verbatim transcript span), and one good item is refused for a whitespace-padded session id; `KNOWN_GAP_CLASSES` states exactly that set and the tier fails if the set moves in either direction. The write-path ACCEPTANCE arm asks the question the discrimination arm cannot: does `dedup-merge` FOLD the pairs it should? A phase that refuses everything scores perfectly on refusal, which is what production did on 2026-09-10 and 2026-09-11 (`vetoed: 171/171`, `vetoed: 178/178`, `merged: 0`). `write-path-acceptance-corpus` is 19 labeled groups (12 `merge`, 7 `keep`), each the answer a model would have given, with the provenance of its label. `write-path-acceptance` runs them through the phase's own fold path — `dedupMergeTextFor`, `groupPairsFor`, and `mergeOutcomes` under `DEDUP_ADMIT_FLOOR`, all imported — with a fold as the positive class, reports acceptance (folds over `merge` pairs), precision, and where every refused pair stopped (read off the filter's own `MergeStage`, a veto refined to its predicate by `mergeVetoReasons`), and gates on an acceptance floor (measured baseline 1.0 minus five points, 0.95) plus zero folds of a `keep` pair. `ACCEPTANCE_KNOWN_GAP_CLASSES` is empty: the two mechanisms behind the all-veto runs — `numericTokenDivergent` over the whole article vetoing the same claim for an incidental number in the body (`incidental-number`), and a role guard that claimed both roles folding at most one member of a model group (`group-fanout`) — measured 8 of 12 under the old rules and 14 of 14 under the current ones, which read numbers off the gist and admit a repeated keeper. The `keep` side holds the counterexamples the new rules must still refuse (a numeric conflict in the claim, a negation or variant qualifier only in the body, a templated series whose claims quote different ids), and `BODY_NUMBER_ONLY_GROUP` states the shape the gist rule no longer refuses, measured outside the gated corpus. This package sits above `@memhtml/contracts`, `@memhtml/domain`, `@memhtml/html`, `@memhtml/index`, `@memhtml/llm`, `@memhtml/store`, `@memhtml/sleep` and `@memhtml/consolidator` (the last two only for the write-path arm, which imports the gate it measures), and `apps/cli` depends on it: `memhtml eval discriminate` is `runDiscrimination`, and `@memhtml/sleep`'s merge takes `discriminationGate` as its pre-merge gate from the CLI's wiring, because the sleep package supplies no default. ## Modules The package publishes one import path, `@memhtml/eval`, which re-exports the twelve modules below, so the reference on this page is the whole exported surface. `gen-fixture.ts` is the body of `pnpm gen:fixture` and is not exported; `scripts/freeze-write-path.mjs` is the body of `pnpm freeze:write-path` and lives outside `src/` because it resolves a repo fixture the published package does not ship. | Module | What it holds | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `controls.ts` | `DIVERGENCE_FAMILIES`, the three flips (`negationFlip`, `numericFlip`, `variantFlip`), `familyPredicate` and `deriveControl`. | | `corpus.ts` | `MemorySpec`, `Probe`, `CorpusSpec`, the defaults (`DEFAULT_SEED`, `DEFAULT_CORPUS_SIZE`, `DEFAULT_PROBE_COUNT`), `quantizeNow`, `articleFor`, `queryFor` and `buildCorpus`. | | `discriminate.ts` | `MRR_FLOOR`, `PROBE_LIMIT`, `runProbes`, `summarize`, `discriminate`, `runFloor`, `describeFailure` and the report shapes. | | `fixture.ts` | `FixtureCorpus`, `FixtureOptions`, `memoryFileFor`, `writeCorpus` and `makeFixtureCorpus`. | | `harness.ts` | `EvalStack`, `StackOptions`, the embedders (`fakeEmbedder`, `failingEmbedder`, `liveEmbedder`, `fakeVector`, `FAKE_DIM`), `buildStack` and `withStack`. | | `run.ts` | `EvalOptions`, `EvalOutcome`, `BEDROCK_TOKEN_VAR`, `hasBedrockCredentials`, `runDiscrimination`, `DiscriminationFailed` and `discriminationGate`. | | `write-path-corpus.ts` | `WRITE_PATH_CORPUS` (62 labeled candidates), `TRANSCRIPTS`, `KNOWN_GAP_CLASSES`, `CEILINGS`, `MANY_SPANS`, `padTo` and the fixture session ids. | | `write-path-gate.ts` | `admitCandidate`, `GATE_STAGES`, `GateDecision`, `GateBatch`: the production gate composed stage by stage. | | `write-path-metrics.ts` | `summarizeWritePath`, `confusionOf`, `breakdownByClass`, `agrees`, `describeWritePath` and the report shapes. | | `write-path-acceptance-corpus.ts` | `ACCEPTANCE_CORPUS` (16 labeled groups), `ACCEPTANCE_KNOWN_GAP_CLASSES` and the group shapes. | | `write-path-acceptance.ts` | `runWritePathAcceptance`, `ACCEPTANCE_FLOOR`, `ACCEPTANCE_BASELINE`, `impliedPairs`, `decideAcceptance`, `summarizeAcceptance`, `acceptanceByClass`, `describeAcceptance` and the report shapes. | | `write-path-run.ts` | `runWritePathDiscrimination`, `WRITE_PATH_F1_FLOOR`, `WRITE_PATH_BASELINE_F1`, `GATE_SOURCES`, `MANIFEST_FILENAME`, `manifestFor`, `replayDrift`, `readManifest`, `writeManifest`, `withBatch`, `decideAll`. | ## Classes ### DiscriminationFailed A failed gate, as a tagged error. Tagged so it is a failure like every other in the system rather than a bare value. The CLI's `codeFor` switches on `_tag`, and `ERR_DISCRIMINATION_FAILED`, the error code design §8 named for exactly this refusal, would otherwise have no producer at all and degrade to `ERR_UNKNOWN`. The whole outcome rides along, because a refusal an operator cannot reproduce is a refusal they will override. `seed` regenerates the corpus that failed, and `inversions` says which probes. #### Constructors ##### Constructor ```ts new DiscriminationFailed(outcome): DiscriminationFailed; ``` ###### Parameters ###### outcome [`EvalOutcome`](/api/eval/#evaloutcome) ###### Returns [`DiscriminationFailed`](/api/eval/#discriminationfailed) #### Properties ##### \_tag ```ts readonly _tag: "DiscriminationFailed" = "DiscriminationFailed"; ``` ##### outcome ```ts readonly outcome: EvalOutcome; ``` #### Accessors ##### reason ###### Get Signature ```ts get reason(): string; ``` The one-line summary, used as the envelope's human message. ###### Returns `string` ## Interfaces ### AcceptanceClassBreakdown One class's agreement with its labels. #### Properties ##### agreed ```ts readonly agreed: number; ``` Decisions that matched the label: folded for `merge`, refused for `keep`. ##### class ```ts readonly class: string; ``` ##### disagreed ```ts readonly disagreed: readonly string[]; ``` ##### label ```ts readonly label: AcceptanceLabel; ``` ##### pairs ```ts readonly pairs: number; ``` *** ### AcceptanceMember One member of a group, as the phase's `CorpusRow` would carry it. Order in the array is corpus order. #### Properties ##### body ```ts readonly body: string; ``` The `
`'s text content, `files.body_text`. ##### gist ```ts readonly gist: string; ``` The `` claim, `files.gist`. ##### path ```ts readonly path: string; ``` *** ### AcceptanceOptions What [runWritePathAcceptance](/api/eval/#runwritepathacceptance) takes. #### Properties ##### acceptanceFloor? ```ts readonly optional acceptanceFloor?: number; ``` ##### corpus? ```ts readonly optional corpus?: readonly LabeledGroup[]; ``` *** ### AcceptanceReport The report the gate reads. #### Properties ##### acceptance ```ts readonly acceptance: number; ``` `accepted / eligible`, four decimals; `0` when nothing was eligible. ##### acceptanceFloor ```ts readonly acceptanceFloor: number; ``` ##### accepted ```ts readonly accepted: number; ``` Folds among the eligible. ##### decisions ```ts readonly decisions: readonly PairDecision[]; ``` ##### eligible ```ts readonly eligible: number; ``` Pairs from `merge` groups: the ones that should fold. ##### f1 ```ts readonly f1: number; ``` ##### foldedKeep ```ts readonly foldedKeep: number; ``` Folds among the guarded. Any value above zero fails the tier. ##### groups ```ts readonly groups: number; ``` ##### guarded ```ts readonly guarded: number; ``` Pairs from `keep` groups. ##### pairs ```ts readonly pairs: number; ``` ##### passed ```ts readonly passed: boolean; ``` True when both labels are present AND no `keep` pair folded AND `acceptance >= acceptanceFloor`. One label alone is a failure, not a vacuous pass, for the reason the discrimination arm gives. ##### perClass ```ts readonly perClass: readonly AcceptanceClassBreakdown[]; ``` ##### precision ```ts readonly precision: number; ``` `accepted / (accepted + foldedKeep)`, four decimals; `0` when nothing folded. ##### recall ```ts readonly recall: number; ``` Same as `acceptance`, stated under the name the discrimination arm uses. *** ### ClaimText The text of a memory as the flips see it, meaning the claim then each body paragraph. #### Extended by * [`DerivedControl`](/api/eval/#derivedcontrol) #### Properties ##### body ```ts readonly body: readonly string[]; ``` ##### claim ```ts readonly claim: string; ``` *** ### ClassBreakdown One class's agreement with its labels. #### Properties ##### agreed ```ts readonly agreed: number; ``` Decisions that matched the label: rejected for `bad`, accepted for `good`. ##### class ```ts readonly class: string; ``` ##### disagreed ```ts readonly disagreed: readonly string[]; ``` Ids of the items whose decision contradicted the label. ##### items ```ts readonly items: number; ``` ##### label ```ts readonly label: WritePathLabel; ``` *** ### ConfusionMatrix The four cells, named by what happened rather than by TP/FP letters a reader has to re-derive. `badRejected` is the true positive, `goodRejected` the false positive, `badAccepted` the false negative, `goodAccepted` the true negative. #### Properties ##### badAccepted ```ts readonly badAccepted: number; ``` ##### badRejected ```ts readonly badRejected: number; ``` ##### goodAccepted ```ts readonly goodAccepted: number; ``` ##### goodRejected ```ts readonly goodRejected: number; ``` *** ### CorpusSpec The whole fixture: the memories to write, their access history, and the probes. #### Properties ##### access ```ts readonly access: readonly AccessSpec[]; ``` `state.access` rows the harness seeds before running the probes. ##### memories ```ts readonly memories: readonly MemorySpec[]; ``` ##### now ```ts readonly now: number; ``` The quantized run instant every stamp is anchored behind, UTC millis. See [quantizeNow](/api/eval/#quantizenow). ##### probes ```ts readonly probes: readonly Probe[]; ``` ##### seed ```ts readonly seed: number; ``` *** ### DerivedControl A derived control, holding the flipped text plus the family that produced it. #### Extends * [`ClaimText`](/api/eval/#claimtext) #### Properties ##### body ```ts readonly body: readonly string[]; ``` ###### Inherited from [`ClaimText`](/api/eval/#claimtext).[`body`](/api/eval/#claimtext) ##### claim ```ts readonly claim: string; ``` ###### Inherited from [`ClaimText`](/api/eval/#claimtext).[`claim`](/api/eval/#claimtext) ##### family ```ts readonly family: DivergenceFamily; ``` ##### note ```ts readonly note: string; ``` What changed, for the probe's own record. *** ### DiscriminationReport The whole run's outcome. This is the `eval.discrimination` envelope payload. #### Extended by * [`EvalOutcome`](/api/eval/#evaloutcome) #### Properties ##### corpusMrr ```ts readonly corpusMrr: number; ``` Mean reciprocal rank over the WHOLE hit list, unitless in `[0, 1]`, four decimals. Reported rather than gated, because corpus size dominates it more than ranking quality does. `DEFAULT_ARM_LIMIT` is 40, which is 13% of a 300-file fixture and under 1% of a real corpus, so the two query-blind arms cover a far larger share of a fixture than of production. The value is seed- and parameter-dependent — it moves with the corpus size, the probe count, and the seed — but it rises with corpus size on this generator while the inversion count stays flat, which is exactly why a gate on this number would be a gate on how big the fixture is. ##### degradedProbes ```ts readonly degradedProbes: number; ``` Probes ranked by a degraded (vector-arm-free) search. ##### discriminated ```ts readonly discriminated: number; ``` Probes whose target strictly outranked all of its controls. ##### inversions ```ts readonly inversions: readonly ProbeResult[]; ``` Probes with at least one inversion. The gate fails when this is non-zero. ##### mode ```ts readonly mode: EvalMode; ``` ##### mrr ```ts readonly mrr: number; ``` Mean reciprocal rank over `{target} ∪ controls`, unitless in `[0, 1]`, four decimals. **This is the gated number, and its coordinate space is the discrimination set, not the corpus.** Design §5 states the floor in the same clause as `rank(target) < min(rank(control))`, so the space the sentence is about is the target against its own impostors. `1.0` means every target beat every control outright. See [corpusMrr](/api/eval/#discriminationreport) for the other reading, which is reported and not gated. ##### mrrFloor ```ts readonly mrrFloor: number; ``` The floor [mrr](/api/eval/#discriminationreport) was measured against. ##### passed ```ts readonly passed: boolean; ``` True when there are zero inversions AND `mrr >= mrrFloor`. ##### probes ```ts readonly probes: number; ``` ##### results ```ts readonly results: readonly ProbeResult[]; ``` *** ### EvalEmbedder Both embed ports plus a call counter. #### Extends * `EmbedPort`.`QueryEmbedPort` #### Properties ##### calls ```ts readonly calls: () => number; ``` ###### Returns `number` ##### embed ```ts readonly embed: (texts) => Effect[], ModelUnavailable>; ``` ###### Parameters ###### texts readonly `string`\[] ###### Returns `Effect`\\[], `ModelUnavailable`> ###### Inherited from ```ts EmbedPort.embed ``` ##### embedQuery ```ts readonly embedQuery: (text) => Effect, ModelUnavailable>; ``` ###### Parameters ###### text `string` ###### Returns `Effect`\<`Float32Array`\<`ArrayBufferLike`>, `ModelUnavailable`> ###### Inherited from ```ts QueryEmbedPort.embedQuery ``` *** ### EvalOptions What [runDiscrimination](/api/eval/#rundiscrimination) takes. #### Properties ##### env? ```ts readonly optional env?: Readonly>; ``` Injected for tests, so the credential branch is drivable without touching the environment. ##### mode? ```ts readonly optional mode?: EvalMode; ``` `fake` by default, the mode that works in CI and whose numbers are reproducible. ##### mrrFloor? ```ts readonly optional mrrFloor?: number; ``` ##### now? ```ts readonly optional now?: number; ``` The run instant to anchor the corpus's stamps behind, UTC millis. The clock when absent. ##### probes? ```ts readonly optional probes?: number; ``` ##### seed? ```ts readonly optional seed?: number; ``` ##### size? ```ts readonly optional size?: number; ``` *** ### EvalOutcome What `memhtml eval discriminate` emits, the report plus which mode was asked for. #### Extends * [`DiscriminationReport`](/api/eval/#discriminationreport) #### Properties ##### corpusMrr ```ts readonly corpusMrr: number; ``` Mean reciprocal rank over the WHOLE hit list, unitless in `[0, 1]`, four decimals. Reported rather than gated, because corpus size dominates it more than ranking quality does. `DEFAULT_ARM_LIMIT` is 40, which is 13% of a 300-file fixture and under 1% of a real corpus, so the two query-blind arms cover a far larger share of a fixture than of production. The value is seed- and parameter-dependent — it moves with the corpus size, the probe count, and the seed — but it rises with corpus size on this generator while the inversion count stays flat, which is exactly why a gate on this number would be a gate on how big the fixture is. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`corpusMrr`](/api/eval/#discriminationreport) ##### corpusSize ```ts readonly corpusSize: number; ``` ##### degradedProbes ```ts readonly degradedProbes: number; ``` Probes ranked by a degraded (vector-arm-free) search. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`degradedProbes`](/api/eval/#discriminationreport) ##### discriminated ```ts readonly discriminated: number; ``` Probes whose target strictly outranked all of its controls. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`discriminated`](/api/eval/#discriminationreport) ##### inversions ```ts readonly inversions: readonly ProbeResult[]; ``` Probes with at least one inversion. The gate fails when this is non-zero. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`inversions`](/api/eval/#discriminationreport) ##### mode ```ts readonly mode: EvalMode; ``` ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`mode`](/api/eval/#discriminationreport) ##### mrr ```ts readonly mrr: number; ``` Mean reciprocal rank over `{target} ∪ controls`, unitless in `[0, 1]`, four decimals. **This is the gated number, and its coordinate space is the discrimination set, not the corpus.** Design §5 states the floor in the same clause as `rank(target) < min(rank(control))`, so the space the sentence is about is the target against its own impostors. `1.0` means every target beat every control outright. See [corpusMrr](/api/eval/#discriminationreport) for the other reading, which is reported and not gated. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`mrr`](/api/eval/#discriminationreport) ##### mrrFloor ```ts readonly mrrFloor: number; ``` The floor [mrr](/api/eval/#discriminationreport) was measured against. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`mrrFloor`](/api/eval/#discriminationreport) ##### now ```ts readonly now: number; ``` The quantized run instant the corpus's stamps were anchored behind, UTC millis. The other half of reproducing a failing run, since the corpus is a function of `(seed, now)`. `0` on a skipped run, where no corpus was generated. ##### passed ```ts readonly passed: boolean; ``` True when there are zero inversions AND `mrr >= mrrFloor`. ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`passed`](/api/eval/#discriminationreport) ##### probes ```ts readonly probes: number; ``` ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`probes`](/api/eval/#discriminationreport) ##### requested ```ts readonly requested: EvalMode; ``` The mode the caller asked for, which differs from `mode` only on a refused live run. ##### results ```ts readonly results: readonly ProbeResult[]; ``` ###### Inherited from [`DiscriminationReport`](/api/eval/#discriminationreport).[`results`](/api/eval/#discriminationreport) ##### seed ```ts readonly seed: number; ``` The seed the corpus was generated at, so a failing run is reproducible. ##### skipped ```ts readonly skipped: boolean; ``` True when live mode was requested with no credentials present. A skipped run is `passed: false` and carries zero probes, so it is a refusal rather than a result. ##### skipReason? ```ts readonly optional skipReason?: string; ``` Why it was skipped, absent otherwise. *** ### EvalStack A built stack, holding the fixture, the database, and retrieval over both. #### Properties ##### db ```ts readonly db: DatabaseShape; ``` ##### embedCalls ```ts readonly embedCalls: () => number; ``` ###### Returns `number` ##### fixture ```ts readonly fixture: FixtureCorpus; ``` ##### indexed ```ts readonly indexed: number; ``` How many files the indexer projected, so a caller can assert the corpus really landed. ##### retrieval ```ts readonly retrieval: RetrievalShape; ``` *** ### FixtureCorpus A generated fixture repo, with the root, the git service over it, and the spec behind it. #### Properties ##### cleanup ```ts readonly cleanup: () => Promise; ``` ###### Returns `Promise`\<`void`> ##### git ```ts readonly git: GitShape; ``` ##### root ```ts readonly root: string; ``` ##### spec ```ts readonly spec: CorpusSpec; ``` ##### written ```ts readonly written: number; ``` How many files were written, generated artifacts excluded. *** ### FixtureOptions What [makeFixtureCorpus](/api/eval/#makefixturecorpus) takes. Every field has a default, so a caller can pass nothing. #### Extended by * [`StackOptions`](/api/eval/#stackoptions) #### Properties ##### now? ```ts readonly optional now?: number; ``` The run instant the corpus's stamps are anchored behind, UTC millis. Effect's `Clock` when absent, which is the one place the fixture pipeline consults time at all — `buildCorpus` requires it, so a test pinning this value pins every stamp in the tree. ##### probes? ```ts readonly optional probes?: number; ``` ##### root? ```ts readonly optional root?: string; ``` Where to generate. A fresh temp directory when absent. ##### seed? ```ts readonly optional seed?: number; ``` ##### size? ```ts readonly optional size?: number; ``` *** ### FixtureTranscript One fixture transcript, as the batch would offer it. #### Properties ##### jsonl ```ts readonly jsonl: string; ``` JSONL, one object per line, in the shape Claude Code writes under `~/.claude/projects`. ##### sessionId ```ts readonly sessionId: string; ``` *** ### FloorReport The lexical-floor scenario runs the same suite with no vector arm. Design §5 requires two things of it: no error, and the lexical arm still beating the controls on the probes that carry lexical signal. Only the SECOND half is a subset claim, and it has to be. A numeric-family control differs from its target by one token, which the FTS arm cannot order, so demanding the full gate without vectors would be demanding the vector arm be unnecessary. `lexicallyDiscriminated` is therefore the number reported and asserted, not `passed`. #### Properties ##### allDegraded ```ts readonly allDegraded: boolean; ``` True when every probe was ranked degraded, which asserts that the arm really is absent. ##### lexicallyDiscriminated ```ts readonly lexicallyDiscriminated: number; ``` Probes the lexical floor still discriminated. ##### probes ```ts readonly probes: number; ``` ##### results ```ts readonly results: readonly ProbeResult[]; ``` Every result, so a caller can see which families survive without vectors. *** ### FrozenDecision One frozen decision: enough to detect drift, nothing that needs a transcript to read. #### Properties ##### accepted ```ts readonly accepted: boolean; ``` ##### class ```ts readonly class: string; ``` ##### id ```ts readonly id: string; ``` ##### label ```ts readonly label: "good" | "bad"; ``` ##### stage ```ts readonly stage: GateStage; ``` *** ### GateBatch What one batch made readable: the transcripts on disk, keyed the way the door keys them. #### Properties ##### reachable ```ts readonly reachable: readonly ReachableTranscript[]; ``` *** ### GateDecision One candidate's outcome. #### Properties ##### accepted ```ts readonly accepted: boolean; ``` ##### reason ```ts readonly reason: string | null; ``` The production function's own reason text, `null` when accepted. Truncated to one line. ##### stage ```ts readonly stage: GateStage; ``` *** ### LabeledCandidate One labeled candidate. `candidate` is `unknown` on purpose: a `bad` item may violate the type the door decodes (a bare string where an entity object belongs, a missing field), and a typed field would make those shapes unrepresentable. The gate's first stage IS the decode, so the corpus has to be able to hand it something the type refuses. #### Properties ##### candidate ```ts readonly candidate: unknown; ``` ##### class ```ts readonly class: string; ``` For a `bad` item, its failure class (what is wrong with it). For a `good` item, its shape (what the gate has to get RIGHT to accept it). The per-class breakdown groups on this. ##### id ```ts readonly id: string; ``` Stable id, `g01`…`g31` for good, `b01`…`b31` for bad. The manifest keys on it. ##### label ```ts readonly label: WritePathLabel; ``` ##### provenance ```ts readonly provenance: string; ``` The rule that decides the label, with the file that holds it. *** ### LabeledDecision One candidate's decision beside its label. #### Properties ##### accepted ```ts readonly accepted: boolean; ``` ##### class ```ts readonly class: string; ``` ##### id ```ts readonly id: string; ``` ##### label ```ts readonly label: WritePathLabel; ``` ##### reason ```ts readonly reason: string | null; ``` ##### stage ```ts readonly stage: GateStage; ``` *** ### LabeledGroup One labeled group. #### Properties ##### class ```ts readonly class: string; ``` For a `merge` group, its shape (what the fold has to get RIGHT to accept it). For a `keep` group, its divergence (why the veto must refuse it). The per-class breakdown groups on this. ##### id ```ts readonly id: string; ``` Stable id, `m01`… for merge, `k01`… for keep. Reports and gap sets key on it. ##### label ```ts readonly label: AcceptanceLabel; ``` ##### members ```ts readonly members: readonly AcceptanceMember[]; ``` Two or more members, oldest first. ##### provenance ```ts readonly provenance: string; ``` The rule that decides the label, with the file that holds it. *** ### MemorySpec One memory to write, before it becomes HTML. #### Properties ##### archivedAt? ```ts readonly optional archivedAt?: string; ``` Set on the archived members, which carry `memhtml-status: archived` and a `memhtml-archived` stamp. ##### body ```ts readonly body: readonly string[]; ``` ##### claim ```ts readonly claim: string; ``` The sentence the memory turns on. It becomes the `` span and therefore `files.gist`. ##### confidence ```ts readonly confidence: number; ``` ##### createdAt ```ts readonly createdAt: string; ``` ##### entities ```ts readonly entities: readonly string[]; ``` ##### extras ```ts readonly extras: readonly string[]; ``` Extra article markup after the claim paragraph, the element kit this memory exercises. ##### importance ```ts readonly importance: number; ``` ##### links ```ts readonly links: readonly object[]; ``` ##### memoryType ```ts readonly memoryType: string; ``` ##### path ```ts readonly path: string; ``` ##### sessionId? ```ts readonly optional sessionId?: string; ``` ##### tags ```ts readonly tags: readonly string[]; ``` ##### title ```ts readonly title: string; ``` ##### updatedAt ```ts readonly updatedAt: string; ``` ##### validUntil? ```ts readonly optional validUntil?: string; ``` *** ### PairDecision One oriented pair's decision beside its group's label. #### Properties ##### class ```ts readonly class: string; ``` ##### dropPath ```ts readonly dropPath: string; ``` ##### folded ```ts readonly folded: boolean; ``` ##### group ```ts readonly group: string; ``` ##### id ```ts readonly id: string; ``` `/`, stable across runs. ##### keepPath ```ts readonly keepPath: string; ``` ##### label ```ts readonly label: AcceptanceLabel; ``` ##### stage ```ts readonly stage: AcceptanceStage; ``` *** ### Probe One discrimination probe: a query, the memory that answers it, and the near-twins that do not. `controlPaths` are ordered by family so a failure names the axis. Each control is validated against its OWN family's divergence predicate at generation time. See `controls.ts`. #### Properties ##### controlPaths ```ts readonly controlPaths: readonly string[]; ``` ##### families ```ts readonly families: readonly DivergenceFamily[]; ``` ##### query ```ts readonly query: string; ``` ##### targetPath ```ts readonly targetPath: string; ``` *** ### ProbeResult One probe's outcome. #### Properties ##### controlRanks ```ts readonly controlRanks: readonly object[]; ``` Each control's 1-based rank in the whole hit list, `null` when absent from the hits. ##### corpusReciprocalRank ```ts readonly corpusReciprocalRank: number; ``` `1 / targetRank`, or `0` when the target was absent. The term of `corpusMrr`. ##### degraded ```ts readonly degraded: boolean; ``` True when the vector arm did not fire for this probe. ##### discriminated ```ts readonly discriminated: boolean; ``` True when the target strictly outranks EVERY control. A control the search never returned counts as outranked, since being absent is worse than being last. ##### discriminationRank ```ts readonly discriminationRank: number; ``` 1-based rank of the target within `{target} ∪ controls` ALONE, the space the gate is stated in. `1` means the target beat every one of its own impostors. Never `null`, because the target is always a member of this set, so an absent target ranks last rather than nowhere. ##### query ```ts readonly query: string; ``` ##### reciprocalRank ```ts readonly reciprocalRank: number; ``` `1 / discriminationRank`. The term of [DiscriminationReport.mrr](/api/eval/#discriminationreport). ##### targetPath ```ts readonly targetPath: string; ``` ##### targetRank ```ts readonly targetRank: number | null; ``` 1-based rank of the target within the WHOLE returned hit list, or `null` when it was not returned. The scope is every active memory in the corpus. Compare against [discriminationRank](/api/eval/#proberesult), which measures the same position in a different space. *** ### ReplayDrift One decision that replayed differently from the frozen one. #### Properties ##### fresh ```ts readonly fresh: | { accepted: boolean; stage: GateStage; } | null; ``` ##### frozen ```ts readonly frozen: | { accepted: boolean; stage: GateStage; } | null; ``` ##### id ```ts readonly id: string; ``` *** ### StackOptions What [buildStack](/api/eval/#buildstack) takes beyond the fixture options. #### Extends * [`FixtureOptions`](/api/eval/#fixtureoptions) #### Properties ##### embedder? ```ts readonly optional embedder?: EvalEmbedder; ``` The embedder to measure through. `fakeEmbedder()` when absent. ##### now? ```ts readonly optional now?: number; ``` The run instant the corpus's stamps are anchored behind, UTC millis. Effect's `Clock` when absent, which is the one place the fixture pipeline consults time at all — `buildCorpus` requires it, so a test pinning this value pins every stamp in the tree. ###### Inherited from [`FixtureOptions`](/api/eval/#fixtureoptions).[`now`](/api/eval/#fixtureoptions) ##### probes? ```ts readonly optional probes?: number; ``` ###### Inherited from [`FixtureOptions`](/api/eval/#fixtureoptions).[`probes`](/api/eval/#fixtureoptions) ##### root? ```ts readonly optional root?: string; ``` Where to generate. A fresh temp directory when absent. ###### Inherited from [`FixtureOptions`](/api/eval/#fixtureoptions).[`root`](/api/eval/#fixtureoptions) ##### seed? ```ts readonly optional seed?: number; ``` ###### Inherited from [`FixtureOptions`](/api/eval/#fixtureoptions).[`seed`](/api/eval/#fixtureoptions) ##### size? ```ts readonly optional size?: number; ``` ###### Inherited from [`FixtureOptions`](/api/eval/#fixtureoptions).[`size`](/api/eval/#fixtureoptions) *** ### VariantOptions What a variant-family derivation needs beyond the target's own text. #### Properties ##### anchor ```ts readonly anchor: string; ``` The token the qualifier is inserted after, a product or host name in the claim. ##### qualifier ```ts readonly qualifier: string; ``` A `VARIANT_QUALIFIERS` member, such as `pro`, `beta`, or `legacy`. *** ### WritePathManifest The freeze/replay manifest. #### Properties ##### corpusSha256 ```ts readonly corpusSha256: string; ``` sha256 of the corpus as JSON, so a changed candidate is attributable. ##### decisions ```ts readonly decisions: readonly FrozenDecision[]; ``` ##### f1Floor ```ts readonly f1Floor: number; ``` ##### frozenAt ```ts readonly frozenAt: string; ``` When the freeze ran, ISO-8601. Informational; not compared on replay. ##### gate ```ts readonly gate: readonly object[]; ``` The stages, in order, and the production functions each one calls. ##### memhtmlVersion ```ts readonly memhtmlVersion: string; ``` The memhtml workspace version the decisions were frozen at. ##### metrics ```ts readonly metrics: object; ``` ###### accuracy ```ts readonly accuracy: number; ``` ###### confusion ```ts readonly confusion: ConfusionMatrix; ``` ###### f1 ```ts readonly f1: number; ``` ###### items ```ts readonly items: number; ``` ###### precision ```ts readonly precision: number; ``` ###### recall ```ts readonly recall: number; ``` ##### schemaVersion ```ts readonly schemaVersion: number; ``` ##### transcriptsSha256 ```ts readonly transcriptsSha256: string; ``` sha256 of the two transcripts' JSONL, so a changed fixture line is attributable. *** ### WritePathOptions What [runWritePathDiscrimination](/api/eval/#runwritepathdiscrimination) takes. #### Properties ##### corpus? ```ts readonly optional corpus?: readonly LabeledCandidate[]; ``` ##### f1Floor? ```ts readonly optional f1Floor?: number; ``` ##### transcripts? ```ts readonly optional transcripts?: readonly FixtureTranscript[]; ``` *** ### WritePathReport The report the gate reads. This is the `eval.write-path` payload shape. #### Properties ##### accuracy ```ts readonly accuracy: number; ``` `(badRejected + goodAccepted) / items`, four decimals. ##### bad ```ts readonly bad: number; ``` ##### confusion ```ts readonly confusion: ConfusionMatrix; ``` ##### decisions ```ts readonly decisions: readonly LabeledDecision[]; ``` ##### f1 ```ts readonly f1: number; ``` Harmonic mean of the two, four decimals; `0` when either is `0`. ##### f1Floor ```ts readonly f1Floor: number; ``` ##### good ```ts readonly good: number; ``` ##### items ```ts readonly items: number; ``` ##### passed ```ts readonly passed: boolean; ``` True when the corpus is non-empty AND carries both labels AND `f1 >= f1Floor`. An empty corpus, or one with a single label, is a FAILURE rather than a vacuous pass: a mean over nothing is not a measurement, and a gate scored only on good items (or only on bad) cannot have discriminated anything. ##### perClass ```ts readonly perClass: readonly ClassBreakdown[]; ``` ##### precision ```ts readonly precision: number; ``` `badRejected / (badRejected + goodRejected)`, four decimals; `0` when nothing was rejected. ##### recall ```ts readonly recall: number; ``` `badRejected / (badRejected + badAccepted)`, four decimals; `0` when nothing was bad. ## Type Aliases ### AcceptanceLabel ```ts type AcceptanceLabel = "merge" | "keep"; ``` Which way the label points. `merge`: every member folds into the oldest. `keep`: none does. *** ### AcceptanceStage ```ts type AcceptanceStage = | "threshold" | "veto:negation" | "veto:numeric" | "veto:variant" | "self" | "role-guard" | "cap" | "folded"; ``` Where a pair's fate was decided. `folded` is the outcome the corpus's `merge` label asks for. *** ### DivergenceFamily ```ts type DivergenceFamily = "negation" | "numeric" | "variant"; ``` Which divergence a control embodies. Named on the probe, so a failure says which axis broke. *** ### EvalMode ```ts type EvalMode = "live" | "fake"; ``` Which embedder produced the numbers. Recorded on the report so a pass is never ambiguous. *** ### GateStage ```ts type GateStage = | "door:schema" | "door:grounding" | "door:quote" | "phase:refusal" | "accepted"; ``` Where a candidate's fate was decided. `accepted` is the fifth outcome: it cleared all four. *** ### WritePathLabel ```ts type WritePathLabel = "good" | "bad"; ``` Which way the label points. ## Variables ### ACCEPTANCE\_BASELINE ```ts const ACCEPTANCE_BASELINE: 1 = 1; ``` The measured baseline this floor was derived from, so the derivation is checkable. Measured 2026-09-11 at memhtml 0.14.0 on corpus v2 (12 merge groups implying 14 pairs, 7 keep groups implying 7 pairs): acceptance 1.0 (14 of 14), precision 1.0 (no keep pair folded). Corpus v1 under the rules before this one measured 0.6667 (8 of 12): three `incidental-number` pairs stopped at `veto:numeric` because the numeric predicate read the whole article, and one `group-fanout` pair stopped at `role-guard` because the guard claimed the keeper after the first fold. Both classes now fold, and the known-gap set is empty. *** ### ACCEPTANCE\_CORPUS ```ts const ACCEPTANCE_CORPUS: ReadonlyArray; ``` *** ### ACCEPTANCE\_FLOOR ```ts const ACCEPTANCE_FLOOR: number; ``` Acceptance floor: the baseline minus five points, floored to two decimals — the discrimination arm's derivation, for the same reason. The floor is the alarm for a class-sized regression (a predicate that starts firing on a class it did not), and the known-gap set is the alarm for a one-pair one. A floor at the baseline would fire on every deliberate corpus edit. *** ### ACCEPTANCE\_KNOWN\_GAP\_CLASSES ```ts const ACCEPTANCE_KNOWN_GAP_CLASSES: ReadonlyArray = []; ``` The classes whose label the fold path is KNOWN to disagree with, stated so the tier fails if the set moves in either direction. Empty as of the rules measured 2026-09-11: every `merge` pair folds and every `keep` pair is refused. Two classes used to be here. `incidental-number` (the same claim, one side carrying a date, a ticket id, or an exit code the other lacks) stopped at `veto:numeric` while `numericTokenDivergent` read the whole `gist\nbody_text`; it reads the gist now. `group-fanout` (a group of three implying two pairs with one keeper) stopped its second pair at `role-guard` while the guard claimed both roles of a committed pair; it admits a repeated keeper now. *** ### BATCH\_SESSION\_IDS ```ts const BATCH_SESSION_IDS: ReadonlyArray; ``` The batch's session ids: what the door treats as READABLE. *** ### BEDROCK\_TOKEN\_VAR ```ts const BEDROCK_TOKEN_VAR: "AWS_BEARER_TOKEN_BEDROCK" = "AWS_BEARER_TOKEN_BEDROCK"; ``` The variable whose presence decides whether live mode can run. *** ### CEILINGS ```ts const CEILINGS: object; ``` The ceilings, restated from `apps/consolidator/src/contract.ts` so the corpus reads on its own. #### Type Declaration ##### claim ```ts readonly claim: 300 = 300; ``` ##### entities ```ts readonly entities: 64 = 64; ``` ##### evidence ```ts readonly evidence: 32 = 32; ``` ##### gist ```ts readonly gist: 1500 = 1500; ``` ##### quote ```ts readonly quote: 600 = 600; ``` *** ### DEFAULT\_CORPUS\_SIZE ```ts const DEFAULT_CORPUS_SIZE: 200 = 200; ``` How many base memories a default corpus carries, before controls and the archive tier. *** ### DEFAULT\_PROBE\_COUNT ```ts const DEFAULT_PROBE_COUNT: 36 = 36; ``` How many probes a default corpus carries. Design §5 requires at least 30. *** ### DEFAULT\_SEED ```ts const DEFAULT_SEED: 20260802 = 20_260_802; ``` The default seed. Named so a caller changing it is making a visible choice. *** ### DIVERGENCE\_FAMILIES ```ts const DIVERGENCE_FAMILIES: ReadonlyArray; ``` *** ### FAKE\_DIM ```ts const FAKE_DIM: 1024 = EMBED_DIM; ``` The vector width the fake produces. It is the real one, so a width check cannot pass by luck. *** ### GATE\_SOURCES ```ts const GATE_SOURCES: ReadonlyArray<{ source: string; stage: GateStage; }>; ``` The gate's stage table as the manifest records it. One place, so the manifest cannot drift. *** ### GATE\_STAGES ```ts const GATE_STAGES: ReadonlyArray; ``` *** ### KNOWN\_GAP\_CLASSES ```ts const KNOWN_GAP_CLASSES: ReadonlyArray; ``` The failure classes the gate is KNOWN not to cover, as of the frozen manifest. Stated in code so the eval can assert the gap is exactly this and no wider: a class that starts failing here is a regression, and a class that starts passing is a change the manifest should record. Three classes: * `restatement/*`: the TRACE-2 bar is checked as a COUNT of quotes, and nothing dedupes them, so the same line twice clears it. * `content-leak/*`: nothing compares a claim against its own evidence, so a claim that is a verbatim transcript span is written into the corpus as prose. * `shape/padded-session-id`: the door's `ungroundedEvidenceReason` compares ids untrimmed while the phase trims them, so a real quote from a real session is refused for its whitespace. A GOOD item the gate rejects, which is the one false rejection on this corpus. *** ### MANIFEST\_FILENAME ```ts const MANIFEST_FILENAME: "write-path-discrimination.manifest.json" = "write-path-discrimination.manifest.json"; ``` The frozen manifest's file name, under `packages/eval/fixtures/`. The PATH is the caller's to resolve: the eval tier resolves it from its own location and `scripts/freeze-write-path.mjs` from its own. This module deliberately holds no module-relative URL resolution, because `@memhtml/eval` is bundled into the published binary and the packaging census (`tests-integration/tests/packaging.test.ts`) requires every run-time asset resolution in shipped source to be a declared, shipped asset. The manifest is a repo fixture, not a shipped one: no command reads it, so declaring it would ship a file nothing in the package uses. *** ### MANIFEST\_SCHEMA\_VERSION ```ts const MANIFEST_SCHEMA_VERSION: 1 = 1; ``` *** ### MANY\_SPANS ```ts const MANY_SPANS: ReadonlyArray<{ quote: string; sessionId: string; }>; ``` Distinct verbatim spans, enough for the evidence-count ceiling cases. Twenty-six named spans plus windows cut from the long line. Every entry is a substring of its transcript, and the corpus test asserts that for each one. *** ### MRR\_FLOOR ```ts const MRR_FLOOR: 0.85 = 0.85; ``` The MRR floor, from design §5. A gate below this admits a target that loses to one of its own negation-flipped twins on one probe in seven. An agent cannot trust that retrieval layer to answer with the right fact. *** ### PROBE\_LIMIT ```ts const PROBE_LIMIT: 40 = 40; ``` How many hits each probe requests. Wide enough that a control's rank is observable rather than truncated into `null`. A window of 10 would report an inversion and a merely-narrow miss identically, and the two need different fixes. The gate itself compares ranks, so widening the window can only make it stricter. *** ### SESSION\_A ```ts const SESSION_A: "session-a" = "session-a"; ``` *** ### SESSION\_B ```ts const SESSION_B: "session-b" = "session-b"; ``` *** ### SESSION\_OUTSIDE\_BATCH ```ts const SESSION_OUTSIDE_BATCH: "session-z" = "session-z"; ``` A session id no transcript in the batch carries. *** ### TRANSCRIPT\_A ```ts const TRANSCRIPT_A: FixtureTranscript; ``` *** ### TRANSCRIPT\_B ```ts const TRANSCRIPT_B: FixtureTranscript; ``` *** ### TRANSCRIPTS ```ts const TRANSCRIPTS: ReadonlyArray; ``` *** ### WRITE\_PATH\_BASELINE\_F1 ```ts const WRITE_PATH_BASELINE_F1: 0.9333 = 0.9333; ``` The measured baseline this floor was derived from. Recorded so the derivation is checkable. *** ### WRITE\_PATH\_CORPUS ```ts const WRITE_PATH_CORPUS: ReadonlyArray; ``` The whole labeled corpus: 31 good, 31 bad, in id order. *** ### WRITE\_PATH\_F1\_FLOOR ```ts const WRITE_PATH_F1_FLOOR: number; ``` F1 floor: the baseline minus five points, rounded down to two decimals. Five points and not zero, because the floor is the alarm for a class-sized regression and the manifest is the alarm for a one-item one; a floor pinned at the baseline would fire on every deliberate corpus edit and train the reader to refreeze without looking. ## Functions ### acceptanceAgrees() ```ts function acceptanceAgrees(decision): boolean; ``` True when the decision matches the label. #### Parameters ##### decision [`PairDecision`](/api/eval/#pairdecision) #### Returns `boolean` *** ### acceptanceByClass() ```ts function acceptanceByClass(decisions): readonly AcceptanceClassBreakdown[]; ``` Group by class, in first-seen order. #### Parameters ##### decisions readonly [`PairDecision`](/api/eval/#pairdecision)\[] #### Returns readonly [`AcceptanceClassBreakdown`](/api/eval/#acceptanceclassbreakdown)\[] *** ### admitCandidate() ```ts function admitCandidate(raw, batch): Effect; ``` Run one candidate through the gate. Never fails: a decision is the answer, and the only I/O (`fabricatedQuoteReason` re-reading a transcript) already folds an unreadable file into a refusal rather than an error. #### Parameters ##### raw `unknown` ##### batch [`GateBatch`](/api/eval/#gatebatch) #### Returns `Effect`\<[`GateDecision`](/api/eval/#gatedecision)> *** ### agrees() ```ts function agrees(decision): boolean; ``` True when the decision matches the label. #### Parameters ##### decision [`LabeledDecision`](/api/eval/#labeleddecision) #### Returns `boolean` *** ### articleFor() ```ts function articleFor(spec): string; ``` The article markup for a spec: the claim paragraph, the first body paragraph joined onto it, then the remaining paragraphs and the element kit. Written here rather than left to `renderTemplate`'s claim/body path because the kits carry real markup, and `renderTemplate` escapes its `body` strings as text. That is correct for a tool parameter and wrong for a `
`. #### Parameters ##### spec [`MemorySpec`](/api/eval/#memoryspec) #### Returns `string` *** ### breakdownByClass() ```ts function breakdownByClass(decisions): readonly ClassBreakdown[]; ``` Group by class, in first-seen order, so the breakdown reads in corpus order. #### Parameters ##### decisions readonly [`LabeledDecision`](/api/eval/#labeleddecision)\[] #### Returns readonly [`ClassBreakdown`](/api/eval/#classbreakdown)\[] *** ### buildCorpus() ```ts function buildCorpus(options): CorpusSpec; ``` The whole fixture specification. The order is fixed as people, base, archive tier, then controls, so the generated tree is written in a stable order and two runs at one `(seed, now)` produce byte-identical files. `now` is REQUIRED rather than defaulted from a clock, because this module's contract is purity: every caller threads its own run instant (the harness reads Effect's `Clock` once), it is quantized to the UTC day, and it is recorded on the spec, so a failing run is reproducible by passing the reported `now` back in. #### Parameters ##### options ###### now `number` The run instant to anchor stamps behind, UTC millis. ###### probes? `number` ###### seed? `number` ###### size? `number` #### Returns [`CorpusSpec`](/api/eval/#corpusspec) *** ### buildStack() ```ts function buildStack(options?): Effect; ``` Build the whole stack inside a scope: generate the corpus, index it, and return retrieval over it. `Effect.acquireRelease` owns the database, so the caller's `Effect.scoped` closes the connection, and the fixture's own `cleanup` removes the temp tree. Both matter in a CLI, because a command that leaked a database handle would keep a WAL file alive under `/tmp` for the life of the process. #### Parameters ##### options? [`StackOptions`](/api/eval/#stackoptions) = `{}` #### Returns `Effect`\<[`EvalStack`](/api/eval/#evalstack), `never`, `Scope`> *** ### confusionOf() ```ts function confusionOf(decisions): ConfusionMatrix; ``` The four cells from the decisions. #### Parameters ##### decisions readonly [`LabeledDecision`](/api/eval/#labeleddecision)\[] #### Returns [`ConfusionMatrix`](/api/eval/#confusionmatrix) *** ### corpusSha256() ```ts function corpusSha256(corpus?): string; ``` The corpus hash. Plain `JSON.stringify` over static data with a fixed key order. #### Parameters ##### corpus? readonly [`LabeledCandidate`](/api/eval/#labeledcandidate)\[] = `WRITE_PATH_CORPUS` #### Returns `string` *** ### decideAcceptance() ```ts function decideAcceptance(corpus?): readonly PairDecision[]; ``` Run the corpus through the composed fold path. Pure. #### Parameters ##### corpus? readonly [`LabeledGroup`](/api/eval/#labeledgroup)\[] = `ACCEPTANCE_CORPUS` #### Returns readonly [`PairDecision`](/api/eval/#pairdecision)\[] *** ### decideAll() ```ts function decideAll(batch, corpus?): Effect; ``` Run every labeled candidate through the gate, in corpus order. #### Parameters ##### batch [`GateBatch`](/api/eval/#gatebatch) ##### corpus? readonly [`LabeledCandidate`](/api/eval/#labeledcandidate)\[] = `WRITE_PATH_CORPUS` #### Returns `Effect`\ *** ### deriveControl() ```ts function deriveControl( target, family, options? ): DerivedControl | undefined; ``` Derive one control from a target, or refuse. Refusal, returned as `undefined`, is the behavior callers depend on. It has two causes, and each one would otherwise produce a control that LOOKS adversarial and is not: 1. The family does not apply (no number to flip). 2. The pair fails the family's own predicate. For `negation` that means the target body already carried a marker, so the flip is invisible to the guard that defines the family. A caller that ignores the refusal and ships the pair anyway gets a probe whose control is a paraphrase, and a gate built on paraphrases passes no matter how badly retrieval discriminates. #### Parameters ##### target [`ClaimText`](/api/eval/#claimtext) ##### family [`DivergenceFamily`](/api/eval/#divergencefamily) ##### options? [`VariantOptions`](/api/eval/#variantoptions) #### Returns [`DerivedControl`](/api/eval/#derivedcontrol) | `undefined` *** ### describeAcceptance() ```ts function describeAcceptance(report): string; ``` A one-line summary: the numbers, then the first disagreement. #### Parameters ##### report [`AcceptanceReport`](/api/eval/#acceptancereport) #### Returns `string` *** ### describeFailure() ```ts function describeFailure(report): string; ``` A one-line summary of a failure, for stderr and for the sleep merge's refusal log. Names the first inversion rather than every one, because an operator needs a probe to reproduce and a thirty-line dump of a failing gate is a thirty-line dump nobody reads. The full list is on the report. #### Parameters ##### report [`DiscriminationReport`](/api/eval/#discriminationreport) #### Returns `string` *** ### describeWritePath() ```ts function describeWritePath(report): string; ``` A one-line summary for stderr and a report subject: the four numbers, then the first disagreement. #### Parameters ##### report [`WritePathReport`](/api/eval/#writepathreport) #### Returns `string` *** ### discriminate() ```ts function discriminate( retrieval, probes, options ): Effect; ``` Run the suite and summarize it in one call. #### Parameters ##### retrieval `RetrievalShape` ##### probes readonly [`Probe`](/api/eval/#probe)\[] ##### options ###### limit? `number` ###### mode [`EvalMode`](/api/eval/#evalmode) ###### mrrFloor? `number` #### Returns `Effect`\<[`DiscriminationReport`](/api/eval/#discriminationreport), `StorageFailure`> *** ### discriminationGate() ```ts function discriminationGate(options?): Effect; ``` The gate, for `MergeOptions.preMergeGate`. A failing gate FAILS this effect, which is what `@memhtml/sleep`'s `merge` reads to refuse. It wraps the gate in `Effect.result` and turns a failure into `refusal: "gate-failed"` with `main` never moving. The shape matters, because a version returning a boolean would let a caller forget to check it, and a refusable gate must not be optional at its call site. #### Parameters ##### options? [`EvalOptions`](/api/eval/#evaloptions) = `{}` #### Returns `Effect`\<[`EvalOutcome`](/api/eval/#evaloutcome), [`DiscriminationFailed`](/api/eval/#discriminationfailed), `never`> *** ### failingEmbedder() ```ts function failingEmbedder(): EvalEmbedder; ``` An embedder that always fails, for the lexical-floor scenario. The failure travels through the ERROR channel as a typed `ModelUnavailable` rather than a throw. The floor only holds if retrieval can catch it, and a defect would kill the fiber instead of narrowing the search. #### Returns [`EvalEmbedder`](/api/eval/#evalembedder) *** ### fakeEmbedder() ```ts function fakeEmbedder(): EvalEmbedder; ``` #### Returns [`EvalEmbedder`](/api/eval/#evalembedder) *** ### fakeVector() ```ts function fakeVector(text): Float32Array; ``` The deterministic embedder, built as a hash-seeded bag of words, L2-normalized. The same construction `@memhtml/index` and `@memhtml/sleep` use in their own harnesses. Two texts sharing vocabulary have a genuinely high cosine and two disjoint texts a low one, which makes a negation-flipped control a real adversary here instead of a random vector the arm trivially separates. A random fake would make the gate meaningless in the easy direction and a constant fake in the hard one. #### Parameters ##### text `string` #### Returns `Float32Array` *** ### familyPredicate() ```ts function familyPredicate(family): (textA, textB) => boolean; ``` The predicate a family's control must satisfy against its target. #### Parameters ##### family [`DivergenceFamily`](/api/eval/#divergencefamily) #### Returns (`textA`, `textB`) => `boolean` *** ### hasBedrockCredentials() ```ts function hasBedrockCredentials(env?): boolean; ``` True when a Bedrock bearer token is present in the environment. #### Parameters ##### env? `Readonly`\<`Record`\<`string`, `string` | `undefined`>> = `process.env` #### Returns `boolean` *** ### impliedPairs() ```ts function impliedPairs(corpus?): readonly object[]; ``` The oriented pairs the corpus implies, in group order, exactly as the phase would hand them to `mergeCandidates`: `groupPairsFor` over each group, with corpus order as the age order and `dedupMergeTextFor` as the veto's texts. `simFor` is empty because no pair here was mined, so every similarity is the recall floor — the phase's own value for a group pair it never scored. A path named by two groups takes its age from its FIRST appearance, the way one corpus row has one offset however many groups a model puts it in. The labeled corpus never repeats a path; the role-guard cases the tests build do, and a last-wins map would re-age a keeper under them. #### Parameters ##### corpus? readonly [`LabeledGroup`](/api/eval/#labeledgroup)\[] = `ACCEPTANCE_CORPUS` #### Returns readonly `object`\[] *** ### liveEmbedder() ```ts function liveEmbedder(): Effect; ``` The real Bedrock embedder, for `live` mode. Built only when a caller asks for it. #### Returns `Effect`\<[`EvalEmbedder`](/api/eval/#evalembedder)> *** ### makeFixtureCorpus() ```ts function makeFixtureCorpus(options?): Effect; ``` A scaffolded memory repo carrying a generated corpus. `initRepo` is the real one, so the fixture carries the real `.gitignore`, the real `.gitattributes`, and the real `merge.ours.driver` config. That config is per-clone, and the `merge=ours` attribute does nothing without it. #### Parameters ##### options? [`FixtureOptions`](/api/eval/#fixtureoptions) = `{}` #### Returns `Effect`\<[`FixtureCorpus`](/api/eval/#fixturecorpus)> *** ### manifestFor() ```ts function manifestFor(report, context): WritePathManifest; ``` Build the manifest a run would freeze. #### Parameters ##### report [`WritePathReport`](/api/eval/#writepathreport) ##### context ###### frozenAt `string` ###### memhtmlVersion `string` #### Returns [`WritePathManifest`](/api/eval/#writepathmanifest) *** ### memoryFileFor() ```ts function memoryFileFor(spec): string; ``` One spec as a memory file's bytes. Hand-assembled rather than routed through `@memhtml/html`'s `renderTemplate`, because of the element kits. `renderTemplate` escapes each `body` string as TEXT, which is right for an agent's tool parameter and wrong for a `
` the fixture means as markup. The head is written in `META_ORDER` so a generated file is byte-identical to what the serializer would emit for the same metadata. The rest of the system then treats the fixture as an ordinary document. #### Parameters ##### spec [`MemorySpec`](/api/eval/#memoryspec) #### Returns `string` *** ### negationFlip() ```ts function negationFlip(claim): string; ``` The affirmative claim as its negation. Total. When no verb anchor is present the whole sentence is wrapped rather than left unchanged. A transform that returned its input would produce a "control" identical to its target, and an identical control is a duplicate the content-hash index refuses at index time, one layer too late to explain itself. #### Parameters ##### claim `string` #### Returns `string` *** ### numericFlip() ```ts function numericFlip(claim): string | undefined; ``` The claim with its first numeric token replaced by a different one. `undefined` when the claim carries no number, because the family does not apply. Inventing a number to flip would produce a control that differs from its target by an ADDED fact rather than by a contradicted one. The probe builder reads the `undefined` and skips the family instead of emitting a weaker control under the same name. The replacement is `value + 10` for a small integer and `value * 2` otherwise, so the wrong number stays in the plausible range for whatever the sentence counts. A retry budget of 13 is a believable misremembering of 3, and 3000 is not. #### Parameters ##### claim `string` #### Returns `string` | `undefined` *** ### padTo() ```ts function padTo(text, length): string; ``` Pad a sentence to EXACTLY `length` characters with prose filler, for the at-the-ceiling cases. The filler is words, so the result still slugs to a title and still reads as a sentence, and the last character is forced to a letter so a boundary case cannot end in a space the door might trim. #### Parameters ##### text `string` ##### length `number` #### Returns `string` *** ### quantizeNow() ```ts function quantizeNow(nowMillis): number; ``` Quantize a run instant to its UTC day. The anchor is the day rather than the millisecond so that every run on one calendar date builds a byte-identical corpus at a given seed. The gate's determinism checks compare two full runs, and a millisecond-precision anchor would make "same seed, same numbers" true only for runs started in the same millisecond. `Math.floor` rather than a `%` remainder, because JS `%` keeps the DIVIDEND's sign: at `nowMillis = -1`, `nowMillis % DAY_MILLIS` is `-1` and subtracting it lands on 0 — an anchor AHEAD of the instant asked for, which is the one thing the anchoring exists to rule out. A non-finite instant is refused here rather than carried: `NaN` passes every arithmetic step and surfaces hundreds of lines later as `RangeError: Invalid time value` out of a stamp formatter, which names neither the input nor the caller that supplied it. #### Parameters ##### nowMillis `number` #### Returns `number` *** ### queryFor() ```ts function queryFor(spec, limit?): string; ``` A probe query holds the target's own content words, in order, capped. Content words rather than the claim verbatim, and the query design decides what the probe can measure. A NUMERIC-family control differs from its target in exactly one numeric token, so a query that dropped the number would have identical overlap with both and the vector arm could not order them at all. The probe would measure a tie-break instead. Keeping the digits is what makes the numeric family a probe rather than a coin toss. The same reasoning keeps a variant qualifier's ANCHOR in the query while the qualifier itself, which only the control carries, stays out. Stopwords go so that `not` cannot hide behind them. A query carrying the target's function words would raise the negation control's lexical overlap for a reason unrelated to the fact either states. #### Parameters ##### spec [`MemorySpec`](/api/eval/#memoryspec) ##### limit? `number` = `12` #### Returns `string` *** ### readManifest() ```ts function readManifest(path): Effect; ``` Read a frozen manifest from `path`. #### Parameters ##### path `string` | `URL` #### Returns `Effect`\<[`WritePathManifest`](/api/eval/#writepathmanifest)> *** ### replayDrift() ```ts function replayDrift(frozen, fresh): readonly ReplayDrift[]; ``` Every id whose fresh decision differs from the frozen one, plus ids present on one side only. Per item rather than a hash of the whole list, because a replay that fails has to say WHICH candidate moved and where it now stops; a boolean would send the reader back to run it by hand. #### Parameters ##### frozen [`WritePathManifest`](/api/eval/#writepathmanifest) ##### fresh [`WritePathReport`](/api/eval/#writepathreport) #### Returns readonly [`ReplayDrift`](/api/eval/#replaydrift)\[] *** ### runDiscrimination() ```ts function runDiscrimination(options?): Effect; ``` Run the gate. Never fails, because the report IS the answer and a caller maps `passed` to an exit code. An eval whose error channel could fire would hand a caller a run that both happened and errored, with no way to say whether the corpus, the index, or the ranking was at fault. #### Parameters ##### options? [`EvalOptions`](/api/eval/#evaloptions) = `{}` #### Returns `Effect`\<[`EvalOutcome`](/api/eval/#evaloutcome), `never`, `never`> *** ### runFloor() ```ts function runFloor( retrieval, probes, options? ): Effect; ``` #### Parameters ##### retrieval `RetrievalShape` ##### probes readonly [`Probe`](/api/eval/#probe)\[] ##### options? ###### limit? `number` #### Returns `Effect`\<[`FloorReport`](/api/eval/#floorreport), `StorageFailure`> *** ### runProbes() ```ts function runProbes( retrieval, probes, options? ): Effect; ``` Run every probe through the real retrieval service. `includeArchived` stays false. The corpus carries an archived tier and an archived memory must not be a candidate, so a probe that ranked one would be reporting a scope leak rather than a ranking failure. The controls are ACTIVE files, which is what makes them adversaries. #### Parameters ##### retrieval `RetrievalShape` ##### probes readonly [`Probe`](/api/eval/#probe)\[] ##### options? ###### limit? `number` #### Returns `Effect`\ *** ### runWritePathAcceptance() ```ts function runWritePathAcceptance(options?): AcceptanceReport; ``` Run the arm end to end. Pure and total: the report is the answer. #### Parameters ##### options? [`AcceptanceOptions`](/api/eval/#acceptanceoptions) = `{}` #### Returns [`AcceptanceReport`](/api/eval/#acceptancereport) *** ### runWritePathDiscrimination() ```ts function runWritePathDiscrimination(options?): Effect; ``` Run the arm end to end. Never fails, for the reason `run.ts` gives for the retrieval arm: the report IS the answer and a caller maps `passed` to an exit code or an assertion. A failing report logs one line at error level. #### Parameters ##### options? [`WritePathOptions`](/api/eval/#writepathoptions) = `{}` #### Returns `Effect`\<[`WritePathReport`](/api/eval/#writepathreport)> *** ### summarize() ```ts function summarize( mode, results, mrrFloor? ): DiscriminationReport; ``` Aggregate probe results into the report the gate reads. **An empty suite is a FAILURE, not a vacuous pass.** Zero probes yields `mrr: 0`, which is below any floor, so a corpus whose probe generation produced nothing refuses instead of reporting a green gate over no measurement. A skipped quality gate must not look like a passing one, and "no probes ran" is the purest form of skipped. #### Parameters ##### mode [`EvalMode`](/api/eval/#evalmode) ##### results readonly [`ProbeResult`](/api/eval/#proberesult)\[] ##### mrrFloor? `number` = `MRR_FLOOR` #### Returns [`DiscriminationReport`](/api/eval/#discriminationreport) *** ### summarizeAcceptance() ```ts function summarizeAcceptance( decisions, acceptanceFloor, groups ): AcceptanceReport; ``` Aggregate the decisions into the report the gate reads. #### Parameters ##### decisions readonly [`PairDecision`](/api/eval/#pairdecision)\[] ##### acceptanceFloor `number` ##### groups `number` #### Returns [`AcceptanceReport`](/api/eval/#acceptancereport) *** ### summarizeWritePath() ```ts function summarizeWritePath(decisions, f1Floor): WritePathReport; ``` Aggregate the decisions into the report the gate reads. #### Parameters ##### decisions readonly [`LabeledDecision`](/api/eval/#labeleddecision)\[] ##### f1Floor `number` #### Returns [`WritePathReport`](/api/eval/#writepathreport) *** ### transcriptsSha256() ```ts function transcriptsSha256(transcripts?): string; ``` #### Parameters ##### transcripts? readonly [`FixtureTranscript`](/api/eval/#fixturetranscript)\[] = `TRANSCRIPTS` #### Returns `string` *** ### variantFlip() ```ts function variantFlip( claim, anchor, qualifier ): string; ``` The claim with a variant qualifier inserted after `anchor`. `qualifier` must be a token `@memhtml/domain`'s `VARIANT_QUALIFIERS` knows, or the pair is not variant-divergent and [deriveControl](/api/eval/#derivecontrol) refuses it. Total, so an absent anchor appends a scope sentence naming the qualifier. That states a different fact about a different variant instead of a paraphrase. #### Parameters ##### claim `string` ##### anchor `string` ##### qualifier `string` #### Returns `string` *** ### wholeText() ```ts function wholeText(text): string; ``` The whole text one memory contributes, claim first. What the veto predicates compare. #### Parameters ##### text [`ClaimText`](/api/eval/#claimtext) #### Returns `string` *** ### withBatch() ```ts function withBatch(body, transcripts?): Effect; ``` Write the transcripts into a fresh temp directory and hand back the batch the door reads. `fabricatedQuoteReason` re-reads a cited transcript from its host path, which is the production shape (the phase never holds transcript bytes), so the fixture has to be on disk. Scoped: the directory is removed when the caller's scope closes, on the failure path too. #### Type Parameters ##### A `A` ##### E `E` ##### R `R` #### Parameters ##### body (`batch`) => `Effect`\<`A`, `E`, `R`> ##### transcripts? readonly [`FixtureTranscript`](/api/eval/#fixturetranscript)\[] = `TRANSCRIPTS` #### Returns `Effect`\<`A`, `E`, `R`> *** ### withStack() ```ts function withStack(body, options?): Effect; ``` Build the stack, run `body`, then tear both down. A scoped helper rather than a fixture the caller assembles, so the temp tree is removed on the failure path too. An eval that exits 1 on an inversion must not leave a 200-file corpus under `/tmp` every time the gate refuses. #### Type Parameters ##### A `A` ##### E `E` ##### R `R` #### Parameters ##### body (`stack`) => `Effect`\<`A`, `E`, `R`> ##### options? [`StackOptions`](/api/eval/#stackoptions) = `{}` #### Returns `Effect`\<`A`, `E`, `R`> *** ### writeCorpus() ```ts function writeCorpus( root, git, spec ): Effect; ``` Generate the corpus into `root`, committing it. ONE commit for the whole corpus. A commit per memory would make the fixture's git history the dominant cost of every eval run. The gate is about ranking, and `git log` over a generated corpus tells a reader nothing. #### Parameters ##### root `string` ##### git `GitShape` ##### spec [`CorpusSpec`](/api/eval/#corpusspec) #### Returns `Effect`\<`number`> *** ### writeManifest() ```ts function writeManifest(manifest, path): Effect; ``` Write a frozen manifest to `path`. Only `scripts/freeze-write-path.mjs` calls this. #### Parameters ##### manifest [`WritePathManifest`](/api/eval/#writepathmanifest) ##### path `string` | `URL` #### Returns `Effect`\<`void`>