Watch
1
0
Fork
You've already forked SouveraineOS
0
SouveraineOS/docs/tasks/27-gitea-pipeline-audit.md
2026-08-10 17:00:23 -04:00

12 KiB
Raw Blame History

TASK 27 — Gitea pipeline audit

Status: audited 2026-07-25 against the live Gitea API. Findings below are measured, not assumed. Size: one session for the audit itself; the fixes are separate.

The goal is the one Casey stated: everything built in ~/Projects ships as a signed package through Gitea, so the phone updates with pacman -Syu and its integrity is provable. TASK-25 owns the design. This task is the audit — what is actually wired today, and where a change silently fails to reach the device.

The gap that matters most: not everything is a package

Three delivery mechanisms are in play and only one of them is auditable.

What How it reaches the phone Auditable
souveraine, -secrets, -machined, -sessiond, upower-souveraine CI → signed pkg → pacman -Syu yes
rootfs-overlay/, overlays/ (units, scripts, hypr config) copied at provision time by provision-rootfs.sh no
kernel, hyprgrass, squeekboard, stevia, firefox-arm, culver, player hand-carried pacman -U, or hyprpm, or nothing no

The overlay is not a package. A fix committed to rootfs-overlay/ reaches a running phone only via a reprovision — which costs Casey his live system, so it effectively never happens. Two fixes are stuck behind this right now: blueline-hexagonrpcd-sdsp.service (Restart=always + the readiness gate) and blueline-proximity-lock. Both are committed in Pixel3Arch 557c8e1 and neither is on the device.

Deciding what owns the overlay — a blueline-rootfs package, or per-component packages that each own their own units — is the first real decision here.

Pipeline census, measured from the Gitea API 2026-07-25

61 repos on gitea.wiuf.net (= 10.10.20.120:4455; same box, three URL forms in use — ssh://…:2222, https://gitea.wiuf.net, http://<token>@10.10.20.120:4455). Four of them carry a .gitea/workflows/. One produces a package.

repo runs green publishes
souveraine 48 yes, #4448 signed pacman repo → edge release
souveraine-updater 1 green on its first run, 07-25 signed pkg → edge, both arches
culver 17 first green ever: #17, 07-25 nothing — CI has no packaging step
Pixel3Arch 11 never nothing
linux-blueline, pocketboot, souveraine-player, stevia, tuie, profile-engine, … no CI nothing

The Gitea package registry is empty of Arch packages — two redflag containers from June, nothing else. Distribution is entirely release assets on souveraine's rolling edge tag, which is why the registry is not the thing to audit.

Pixel3Arch/pkgs/ holds 15 PKGBUILDs — hyprgrass, stevia, squeekboard, q6voiced, tqftpserv, usb-signaller, pulseaudio-q6, hexagonrpc, paru, pachub and more. Not one is built by CI. Every one is hand-built on archdev and hand-carried per pkgs/README.md. Installed that way they are pacman-owned but frozen: -Syu has no repo to upgrade them from, so ownership records the version without ever advancing it.

The runner is a single host-mode gitea-runner.service user unit on archdev (PID 132, Linger=yes, enabled — that part is sound). Host mode means every job mutates one shared box. That is precisely how the poisoned sysroot in finding 6 happened, and nothing isolates the next one.

Findings, measured 2026-07-25

  1. There is no primary branch on souveraine. ci.yml triggers on [primary, public]; the API shows only main, public, experimental/session-trust-phase1, archive/main-mit-0.1.0. So half the trigger is dead and public is the de facto release branch. Worse, the working branch experimental/session-trust-phase1 is not in the trigger list at all — a push there runs no CI and publishes nothing, silently. main is the repo's default branch and has not moved since 2026-05-22, so a fresh clone gets two-month-old code. (culver is consistent by contrast: default primary, CI triggers on primary.)

    Half-fixed 2026-07-25: primary now exists at 9d8bce2 and the local checkout tracks it, so the documented model — dev on primary, promote to public, publish gated to public — is real and the trigger no longer names a ghost. Still open: main remains the default branch and still points at May's 1cf98ef5. Either move it or retire it.

  2. kernel.yml has never once passed. 11 runs, 11 failures, none since Jul 14 — eleven days with nobody retrying. kernel-latest is a 404. The actual failure in the last run (#11, job 1790) is not a credential problem at all — the credential fixes worked and the build got as far as olddefconfig, which then tripped the drift guard at kernel/PKGBUILD:53:

    -CONFIG_RUST_IS_AVAILABLE=y
    -# CONFIG_RUST is not set
    ERROR: olddefconfig changed kernel/config-blueline.aarch64
    

    The guard is right to exist and wrong as written: CONFIG_RUST_IS_AVAILABLE is derived from the build host's rustc, not from our intent. Baking a host-probed symbol into a canonical config makes the check fail whenever the runner's toolchain differs from whoever last regenerated it. Fix by stripping host-probed symbols before the diff, not by chasing the config.

    Remaining blockers behind it, unchanged:

    • pkgver is static 7.1.1, so -Syu can never see an upgrade
    • a bare .g<sha> sorts as hex and is non-monotonic — use 7.1.1.r<count>.g<sha>
    • kernel/config-blueline.aarch64 was moved to archive/ while PKGBUILD:53 still copies it from $startdir. Restored and verified present 2026-07-25; a stale duplicate remains at archive/kernel/config-blueline.aarch64 and should go.
  3. The edge release clobbers other producers. Fixed 2026-07-25, souveraine 044c373. packaging/arch/publish-edge.sh replaces the delete-and-recreate: release and tag are never deleted, the live souveraine-<arch>.db (and .files) is fetched and merged with repo-add under a shared archdev flock, and only the producer's own superseded builds are removed — matched by %NAME% in the live db, so a producer cannot delete another's asset.

    Two guards, since the failure is otherwise silent: the merged db must retain every foreign entry (else abort, archive untouched), and the db is read back over pacman's own download URL and diffed. A db that is published but unfetchable is a hard failure, never a fall-through to building a fresh one — that fallback was in the first draft and would have reintroduced the bug on any transient 500.

    Databases are fetched by releases/download/<tag>/<name>, not the asset-id API, which returns attachment metadata rather than bytes on some Gitea versions.

    Rehearsed against a throwaway tag before adoption, and since proven live: souveraine-updater published into edge on 2026-07-25 and both souveraine and upower-souveraine survived in both per-arch databases.

  4. Unowned files are still the standing risk. 5,624 under /usr /etc /opt /boot. /var/cache/hyprpm was one of them, and that is how a routine hyprutils 0.13.1 → 0.14.0 upgrade killed every touch gesture with nothing reporting it. hyprgrass is now a package (Pixel3Arch pkgs/hyprgrass, 557c8e1) — but no pipeline delivers it, because finding 2 blocks the same last mile the kernel needs.

  5. no-ai-attribution fails on two old commits (a7e909d, bc6ee12) carrying co-author trailers. Fixing means a history rewrite. Casey's call, still pending.

5b. culver had never once been green — 16 red runs from its first on 2026-07-19. Green as of #17, 2026-07-25. The whole blockage was accumulated clippy lint, in a chain: culver-coreculver-telephony + culver-contactsculver-matrix → the cxx-qt app crate. -D warnings stops at the first failing crate, so each fix only exposed the next and nobody had walked it. Four commits: 1e3ea12, b28b15f, ab559bc, 8dc8553.

None of it was a real defect, but the rot is worth recording because a working pipeline would have caught each item the day it was written:

  • three unwrap_or_else(|| Utc::now()) fallbacks sitting under comments that said "never now" — the dead branch contradicted its own docstring; now DateTime::UNIX_EPOCH
  • a DiffMirror.account field kept alive by an _account() accessor that was itself dead
  • the app crate had never been linted at all: four pub use qobject::X as QX re-exports referenced nowhere (dead by construction — nothing can import from a binary crate), four matching unused imports, beginInsertRows/endInsertRows declared on two models that never insert (chat.rs genuinely uses its pair, which is why clippy spared it), a pending: VecDeque<ModelUpdate> queue documented as "drained on first apply" that nothing drained, and an uncalled set_header.

One #[allow] was used, on handle_command's 8 arguments, with the reason in a comment. Everything else is a real fix.

The structural point outlasts the fix: culver's CI has no packaging step. It runs test, clippy and build, then stops. Green culver still ships nothing. Same for any repo whose CI ends at cargo build.

  1. Build-host integrity is unaudited, and it lies. The archdev aarch64 sysroot was half-updated: extracting packages unprivileged silently skips every file that already exists root-owned, so hyprland.pc read Version: 0.56.0 while the headers were still 0.55's layout. A cross build against that sysroot compiles happily against a mixed ABI. Anything cross-built on archdev in the past few weeks is suspect and worth rebuilding. pkgs/hyprgrass/build-cross.sh now refreshes as root and asserts SONAME == NEEDED; that assertion, or an equivalent, belongs in every cross build. A signed package built from a poisoned sysroot is still a broken package — signing proves origin, not correctness.

  2. The update path only exists at home. The phone's repo is Server = http://10.10.20.120:4455/Fimeg/souveraine/releases/download/edge — a LAN address over plain HTTP, reached through a XferCommand wrapper that feeds a token from /etc/pacman.d/souveraine-gitea.netrc because pacman will not forward credentials in a Server URL. Package signatures make HTTP acceptable for integrity; they do not make the host reachable. Off the LAN, pacman -Syu simply fails. A daily-driver phone that can only update in one building is a constraint worth naming now rather than discovering on a trip.

  3. Published ≠ installed, and nothing closes the loop. edge currently carries souveraine 0.1.r220.g9d8bce2a08cd for both arches, signed, with a valid db. The phone is on r219. Nothing pulls, notifies, or reports the drift — the last mile is a human remembering. That is TASK-28's subject and it starts here, not at the packaging.

Acceptance

  • Every repo in ~/Projects that ships anything to the phone has a CI job ending in repo-add --include-sigs --sign into souveraine-{arch}.
  • No CI trigger names a branch that does not exist.
  • pacman -Syu on the phone installs kernel, hyprgrass, squeekboard, stevia, souveraine and the overlay content. Nothing is hand-carried.
  • A reprovision restores repo wiring from rootfs-overlay/ with no manual steps.
  • Unowned files under /usr /etc /opt /boot trend to zero, and the count is reported by a job rather than discovered during an outage.
  • Cross builds fail loudly on an ABI mismatch instead of shipping one.

Connects to

TASK-25 (the design and the archive shape), the absorbed proximity-authority work (blueline-proximity-lock as an unowned second authority — now demoted to a reporter, still unpackaged), PAF/build.md, souveraine/packaging/arch/.