a8caf27c14
Five weeks of forge evidence per week rather than a rolling window, so a finding survives under one key and "closed after three weeks" is computable. Rules run deterministically; agent.py hands the packet to whoever writes the prose and never calls a model itself. Model choice fails closed — no default, no substitute, and the exact named model must be served or the pass refuses to run.
212 lines
11 KiB
Markdown
212 lines
11 KiB
Markdown
# TASK-79 — project state: what became true, and what stopped being true
|
|
|
|
**Status:** first slice built and exercised 2026-08-18 over five weeks,
|
|
2026-W30 to 2026-W34. Internal; nothing publishes.
|
|
**Repos:** `SouveraineOS` (the tools and the store). Reads the forge and the
|
|
local clones; writes nothing outside this repo.
|
|
**Connects to:** `saf/state.md` (the hand-written spine this reads and never
|
|
writes), `docs/tasks/README.md` (the declared state it diffs),
|
|
`~/Projects/Gitea Pipeline/TASK-02-the-ledger-pipeline.md` D4/D5 (the narrator
|
|
and the quality pass — same seam, different question),
|
|
`~/Projects/Web/CaseyTunturi.com/scripts/ledger/collect.py` (the ancestor).
|
|
|
|
## The question this answers, and the one it does not
|
|
|
|
The ledger on caseytunturi.com answers **what happened** — day-bucketed, 30-day
|
|
rolling, overwritten in place. It is a record of a man at work and it is good at
|
|
that. It cannot answer what *became true*, because nothing carries across the
|
|
window: every week is an island, and a 31-day-old finding is simply gone.
|
|
|
|
This answers **what is true now, what stopped being true, and how long it has
|
|
been unresolved.** Same forge, different shape: evidence is kept per week and
|
|
never overwritten, and a finding survives from one week to the next under a
|
|
stable key. Statements the five-week run can already make:
|
|
|
|
- *task 8 left the index, held W30 to W32, closed after two weeks.*
|
|
- *task 15, Idle coordinator, has been Blocked or gated for four weeks.*
|
|
- *souveraine#1 has been unmerged for 38 days and is the oldest decision open.*
|
|
- *souveraine CI was regressed in W33 and mixed in W34* — the finding turned.
|
|
|
|
## Why here and not in the ledger's repo
|
|
|
|
The ledger's output repo is the public site. Its CI builds and deploys on every
|
|
push, so an analytical artifact living there is one careless import away from
|
|
being published. SouveraineOS already owns project state — `saf/state.md` is the
|
|
resume anchor and `docs/tasks/README.md` is the work index — and it deploys
|
|
nowhere. The forge answers 403 to anonymous requests, so the store is behind a
|
|
login even though the repo is not marked private.
|
|
|
|
Promotion to either public surface stays a separate, deliberate act:
|
|
`caseytunturi.com` for engineering history, `souveraineai.com` for current
|
|
capability. One evidence base, two editorial surfaces, neither automatic.
|
|
|
|
## Shape
|
|
|
|
```
|
|
forge API + local clones
|
|
↓ collect.py deterministic, no model
|
|
project-state/evidence/<week>.json
|
|
↓ synthesize.py rules first, model second
|
|
project-state/state.json durable, carried forward
|
|
project-state/history/<week>.json
|
|
project-state/reports/<week>.md prose, disposable
|
|
```
|
|
|
|
Nothing schedules it. The tools name their own gaps instead:
|
|
|
|
```
|
|
collect.py --gaps [--since 2026-W30] weeks holding no evidence
|
|
collect.py --backfill --since 2026-W30 collect every one of them, in order
|
|
synthesize.py --pending weeks collected but not passed over
|
|
synthesize.py --catch-up pass over them oldest first
|
|
```
|
|
|
|
Order matters — carry-forward is sequential, so `--catch-up` is the entry point
|
|
and `--week` warns when it would skip ahead of the state.
|
|
|
|
The deterministic/model split is the point. A model is never asked to discover a
|
|
fact code can read; it gets the computed findings and looks only for what
|
|
crosses between them. Its findings are marked `by: model`, live in their own
|
|
section of the report, and are dropped unless every evidence id they cite exists
|
|
in that week's evidence and spans two kinds of it.
|
|
|
|
The prose is regenerable — `--report-only` rewrites any week's report from the
|
|
stored state and evidence, so a future model can rewrite history without the
|
|
original report ever having been authoritative.
|
|
|
|
## Evidence, measured 2026-08-18
|
|
|
|
| | W30 | W31 | W32 | W33 | W34 (partial) |
|
|
|---|---|---|---|---|---|
|
|
| Commits | 226 | 287 | 201 | 260 | 43 |
|
|
| CI runs | 98 | 133 | 143 | 185 | 25 |
|
|
| Declared changes | 255 | 56 | 18 | 37 | 116 |
|
|
|
|
16 repos listed, ~95 API calls a week, five weeks collected in 20 s, zero disk.
|
|
|
|
Three things the collector inherits from the ledger because they were paid for
|
|
once already: the `limit=50` page clamp (Gitea ignores a larger limit and hid 24
|
|
of 74 repos for months), the branch-tip prefilter (a branch whose tip predates
|
|
the window cannot hold work inside it), and per-repo SHA dedup.
|
|
|
|
Two it does not:
|
|
|
|
- **The Actions API.** `/repos/{repo}/actions/runs` ignores `limit` and `page`
|
|
and returns the whole history, so it is fetched once and sliced. Paging it
|
|
loops on the same rows. `02-the-ledger.md` lists CI state as an unbuilt later
|
|
increment; this is that increment, joined to commits by `head_sha`.
|
|
- **Redaction.** Evidence is stored raw because it never leaves the forge.
|
|
`--redact` applies the ledger's patterns for anything promoted outward.
|
|
|
|
## Dimensions
|
|
|
|
The SAF's own sections, plus the two it has no page for:
|
|
|
|
`memory` `runtime` `reflection` `identity` `surface` `sensorium` `release`
|
|
`reliability` `federation` `adoption`
|
|
|
|
A commit is placed by scoring its changed paths and its subject against a
|
|
keyword table, weighted alike — paths alone file `compaction: target the active
|
|
conversation` under the server module it edits, and the subject is the author
|
|
saying which subsystem he thinks he moved. On W33: 198 by paths, 29 by subject,
|
|
22 by repository default, 11 placed nothing and are reported as `unassigned`
|
|
rather than guessed at. Every row keeps its `basis` and its paths, so any
|
|
placement can be argued with.
|
|
|
|
One dimension per commit is a simplification. A crash fix in the shell is both
|
|
`surface` and `reliability` and counts once.
|
|
|
|
## State vocabulary, and the resolution rule
|
|
|
|
`milestone_reached` `regressed` `blocked` `decision_required` `advanced`
|
|
`unchanged` — assigned by rule:
|
|
|
|
- a task's section in the index (Live / In progress / Blocked or gated / Open)
|
|
gives its standing state; a move backwards is `regressed`, and leaving the
|
|
index entirely is `milestone_reached`;
|
|
- `decision_required` comes from Casey's own phrasing in the row — *verdict
|
|
owed*, *decisions are Casey's*, *scope and deliverable unanswered*;
|
|
- a repository whose last run of the week failed is `regressed`;
|
|
- a pull unmerged past fourteen days is `decision_required`.
|
|
|
|
**A finding the week says nothing about stays open.** Only positive evidence
|
|
closes one — a task gone from the index, a pull merged. The first two-week run
|
|
resolved four CI findings because the second week had no runs for those repos,
|
|
which is the difference between a state document and a diff.
|
|
|
|
Findings that describe *one week* — a CI week, a dimension's activity, a release
|
|
cut — do not carry at all; they lapse with their week. Findings that describe a
|
|
*thing* — a task, a pull, an issue — carry until something closes them.
|
|
|
|
Task findings track the index snapshot, not only its diff. Tracking the diff
|
|
alone made every restatement a permanent finding and the task itself invisible;
|
|
the index is the state, the diff is the movement, and `weeks_in_section` is how
|
|
"this remains the oldest blocker" gets computed.
|
|
|
|
## Provenance
|
|
|
|
Each finding carries `provenance` — deterministic rule name or model, endpoint
|
|
locality resolved from the URL rather than asserted, timestamp, and the week its
|
|
evidence came from — plus a `human` block (`verdict`, `note`, `edited`) that is
|
|
empty and exists so a promote/reject pass has somewhere to write.
|
|
|
|
`synthesize.py --explain <id>` prints a finding and every evidence row it cites,
|
|
in full. That is the trace path, and it is one command.
|
|
|
|
## The analyst, measured
|
|
|
|
`ANALYST_URL` / `ANALYST_KEY` / `ANALYST_MODEL`, any OpenAI-compatible endpoint.
|
|
No URL means no model pass and a complete deterministic report — the same "never
|
|
a failed run" contract the ledger's narrator has.
|
|
|
|
Against Fable on llama-swap, 2026-08-18:
|
|
|
|
- **Warm, it answers in 6 s.** The 14 s empty first call recorded in TASK-02 D4
|
|
was **not** llama-swap loading the model. Fable is a reasoning model: it fills
|
|
`reasoning_content` first and leaves `content` empty when the budget runs out
|
|
before it concludes. At `max_tokens: 250` — what `collect.py`'s `narrate()`
|
|
sets — it can never answer. **The ledger's narrator has that bug now.**
|
|
- The full week as a brief is 39k characters; Fable spent 3000 tokens reasoning
|
|
over it, returned nothing, and took 323 s. Sampling ten commit subjects per
|
|
dimension cuts the brief to 20k and the run to ~200 s.
|
|
- Reasoning length varies run to run and can still eat the budget, so a
|
|
truncated answer is salvaged object by object — every complete finding before
|
|
the cut is kept.
|
|
- **Quality is the open problem.** On W33 it returned 21 findings, of which ~15
|
|
were commentary on the deterministic findings rather than on the project. A
|
|
tightened charter and a gate requiring two kinds of evidence cut that to four.
|
|
Of those four: one good (CI runs that neither pass nor fail nearly equal the
|
|
passes, which the counts support), one arguable, two wrong — one cites a
|
|
success run as evidence of failure, another cites an unrelated doc commit.
|
|
A 9B is not a good analyst for this. The seam works; the model is Casey's call.
|
|
|
|
## Acceptance
|
|
|
|
- [x] Weeks collected deterministically, every row carrying a stable id.
|
|
- [x] A durable state file carrying a finding across weeks under one key, with
|
|
`first_seen`, `weeks_open`, `weeks_in_section`, and resolution only on
|
|
positive evidence.
|
|
- [x] Carry-forward exercised over five real weeks, not designed.
|
|
- [x] A human-readable report where each finding names its evidence.
|
|
- [x] One command traces a finding back to every row it rests on.
|
|
- [x] The tools name their own unfilled weeks rather than assuming a schedule.
|
|
- [ ] An analyst whose findings survive review, or the pass explicitly parked.
|
|
- [ ] A cadence decision: on demand, or a Monday job on the same runner.
|
|
- [ ] A human verdict written into a finding — the seam exists, nothing uses it.
|
|
|
|
## Open, and Casey's to answer
|
|
|
|
- **Which model holds the analyst.** Fable is honest and slow and wrong about
|
|
half the time on this. The substrate's own config already reaches bifrost,
|
|
deepseek, zai and claude; the deterministic half needs none of them.
|
|
- **Where a promoted finding goes.** The seam exists; the editorial call does
|
|
not.
|
|
- **Whether the store is committed here.** It is project state, it belongs with
|
|
`saf/state.md`, and it is behind the forge login — but it is a machine-written
|
|
tree in a repository whose other files are hand-written.
|
|
|
|
## Noticed while building this, not part of it
|
|
|
|
`docs/tasks/README.md` publishes `10.10.20.123`, `172.16.42.1` and an ssh key
|
|
path into this repo. Behind the forge login, so not the leak TASK-02 D8 closed —
|
|
same class, and it sits in the file this task diffs every week.
|