bot/eval/golden.ts
openaliro/openaliro
Module

golden.ts

@file The golden set, and the gate that keeps its labels honest.

bot/eval/golden.ts2 documented symbols

Overview

@file The golden set, and the gate that keeps its labels honest. Gold labels key on an expect substring, never on a bare line number. That is not a style choice: the design brief this eval was written against cited mk/cdk.mk:261-262, and commit 3737673 grew that file by 71 lines and moved every one of those anchors a uniform +20. A golden set keyed on line numbers would have silently measured every baseline against the wrong lines. Same mechanism as scripts/check-citations.ts, one level stricter: an expect that matches more than one line in its file is reported too, because an ambiguous anchor makes recall look better than it is.

used by chunk-kconfig.ts corpus.ts deepwiki.ts independent.ts retrieve.ts scope.ts score-deepwiki.ts stage0.ts stage1-probe.ts

API

Fexport function resolveAnchor(anchor: GoldAnchor): ResolvedAnchor

bot/eval/golden.ts:59

Every 1-indexed line in file containing expect.

called by main, validate  ·  calls lines

Fexport function headSha(): string

bot/eval/golden.ts:77

The SHA the whole eval is pinned to, for the report header. Shelled out rather than read from .git/HEAD, because in a git worktree .git is a file.

called by main, main, main
Undocumented (3)

lines, loadGolden, main