STATE.md: federation test passed — two instance branches, both agents speaking on glm-5.2, identities split.
5.6 KiB
Provisioning gaps — what got hand-filled on 2026-07-16
Every item here is a manual intervention performed during the first live federation test (laptop + Pixel 3, agent Souveraine). Each one is a gap the real setup flow — the node commission ceremony FEDERATION.md parks — must close. This file is the evidence log: if we don't record what we did by hand, the ceremony gets designed from memory and misses them.
Ordering is by how badly the gap bit, not by build order.
1. Agent provider config drift — silent until b4dc39e
What happened: Souveraine's agent.json carried provider: claude with an
empty key while its model said glm-5.2. A model sweep updates model; nothing
polices provider. Every turn 401'd against Anthropic's OpenAI-compat endpoint
and, before the error-event fix, rendered as the agent going mute.
What the system must do: a preflight at agent load (or config write): resolve the agent's provider chain and fail loudly if the key is empty or the provider unreachable. "Can this agent speak?" is a checkable invariant, not a runtime surprise.
2. Memfs remote naming — gitea vs origin
What happened: memory sync requires an origin remote. Existing memfs
checkouts carried the same URL under the name gitea. Fixed by aliasing on
both machines by hand.
What the system must do: commission-time provisioning names the shared
remote origin, and/or sync accepts a configured remote name. Existing
checkouts need a migration nudge, not a silent error.
3. Cloned machine identity — the shared-seed trap
What happened: the phone's ~/.souveraine was cloned from the laptop, so
both machines held the same seed-id (6344f24e…) for months. Left alone, both
would have pushed the same instance/ branch and the federation test would
have proven nothing. Split by hand: laptop machine init --migrate-from,
phone machine init --fresh.
What the system must do: commission ceremony detects a seed that already exists in the federation (registry or remote branch collision) and forces the fresh-mint path. A cloned filesystem must not silently become a cloned identity.
4. Agent server is a hand-started process
What happened: on both machines souveraine server runs as a
setsid/nohup user process logging to /tmp. The phone hard-reset and the
server simply didn't exist afterward; restart was manual over SSH. Its logs
die with /tmp.
What the system must do: a user systemd unit (souveraine-server.service)
with journal logging and restart policy, installed by packaging, enabled at
commission. The machined precedent already exists — mirror it.
5. No crash forensics on the phone
What happened: the phone hard-reset mid-souveraine chat (14:11). No
shutdown record, no coredumps, journal stops mid-write, /sys/fs/pstore is
empty. The reset is unattributable — thermal, watchdog, TZ/remoteproc wedge —
because nothing was configured to remember it.
What the system must do: wire ramoops/pstore (Android configures it on this exact hardware — the reference answer, per the Android-is-the-reference rule). Belongs in Pixel3Arch kernel/DT work, not the ceremony, but the ceremony should verify pstore is live on phone-class nodes.
6. machined socket ACL doesn't survive restart
What happened: usermod -aG souveraine lands at next login; until then a
manual setfacl on /run/souveraine is needed after every machined restart.
Done by hand twice today.
What the system must do: packaging owns it — tmpfiles.d or group-writable socket creation in machined itself, so a restart never strands the user session.
7. Instance label ↔ glyph mapping doesn't exist agent-side
What happened: sync reports branches as machine-pubkey prefixes
(instance/69bf24d7b185). The laptop agent, asked whether the peer branch was
the phone, correctly refused to confirm — it has no stored association between
glyphs (▽●△△ / ▽■◑★) and instance labels, and said recording Casey's
assertion isn't verification. It was right.
What the system must do: the memfs_commit payload and sync output
should carry the glyph alongside the pubkey; the in-memfs system/instances.md
(which the phone agent wrote unprompted after reading the briefing) is the
durable half. Commission stamps the mapping; the agent never has to take it on
faith.
8. Binary deploys are scp + pkill by hand
What happened: deploy = scp from archdev, sudo install, then kill and
restart the server — including the recurring pkill-matches-own-ssh-cmdline
footgun, hit twice this session.
What the system must do: packaging (PKGBUILD or a deploy script in-repo)
plus the systemd unit from §4 makes deploy = install + systemctl restart.
9. New system files aren't surfaced to the agent
What happened: a briefing file written directly into the phone memfs was invisible to the primary until the subconscious caught the false "no such file" claim and corrected it mid-thread ([intrusive] surfacing — the architecture working as designed, but as a backstop, not a flow).
What the system must do: memfs writes that arrive from outside the agent's own tool path (human edits, sync merges) should generate the same felt signal as local compaction — this is FEDERATION.md's receive-side doctrine; the local case (files appearing on disk between turns) needs it too.
Resolved during the same session, recorded for completeness: hosted_agents
announce read the wrong tree (server/agents/ vs agents/) — fixed with a
regression test in da410b1; silent turn-death — fixed end-to-end error
events in 33068d6/b4dc39e.