delivery plumbing that got us there:
- acks clear on result-recorded, not command lifecycle status (no more 34-deep recycling)
- timeouts, cancels, dropped acks/receipts, failed actions all land in history instead of dying on stdout
- one shared closure-cleared predicate so auto-confirm and manual approve can't drift
override waives the vuln call only and gets journaled; signing and hash verification stay non-negotiable.
- Replace unbounded goroutine fan-out with bounded pool (8 concurrent)
so 300-package dnf scans no longer timeout every request against api.osv.dev
- Drop in-memory osvDedup sync.Map; gate on persisted supply_chain_checked_at
so the dedup survives restart and failed checks retry naturally
- On query failure, record the error without checked_at so the package stays
a candidate for the next cycle (ETHOS: errors are history, assume failure)
- Shared RunOSVChecks in services/ used by both scan path and startup backfill
- Add FreshSupplyChainPackages query for persist-driven freshness lookup
- Bump version to 0.2.3.0
Add server/internal/orchestrator — a stateless, DB-driven service that advances
the package update lifecycle instead of leaving every transition to an operator
button press. It holds no state of its own: each 60s sweep re-reads from the DB,
so a restart reconciles on the next tick, and every advance rides the
LIFECYCLE-001 guarded transition (idempotent by construction).
- Auto-approval is policy-gated by policy.auto_approve_max_severity (default
off). Eligible pending packages are approved and their dry-run enqueued
(-> checking_dependencies). It never advances to installing on its own, never
runs when allow_dry_runs=false, never approves packages carrying
supply_chain_vulns, and an unrecognized policy value fails safe to disabled.
- Stuck-state recovery: checking_dependencies past 30m re-enqueues the dry-run
up to twice then fails the package; installing past 60m fails it;
pending_dependencies past 24h logs a stale count without changing state.
- Synchronous fast-path: ReportUpdates fires OnPackagesDiscovered() so
auto-approve runs at scan time, serialized with the timer via TryLock.
Supporting changes: exported TransitionByID/TransitionByPackage and
GetPackagesInStatus/BumpRetryCounter on UpdateQueries; GetPolicyString on the
settings service; EnqueueDryRun extracted from the manual dry-run endpoint and
shared with the orchestrator. Unit tests cover policy gating, requeue-then-fail,
and the install timeout.
Route every current_package_state status change through one transitionStatus
path: read the observed status, validate against PackageStatusTransitions,
run a status-guarded UPDATE, record terminal history. Replaces ten raw-SQL
transition functions whose WHERE guards validated nothing and silently
no-op'd on an illegal state. ApproveUpdate, the Reject/Install/Set* family,
BulkApprove and UpdatePackageStatus now share the core; illegal moves return
a named from->to error instead of a silent miss, and concurrent callers are
caught by the guarded row count.
Migration 047 renames the terminal success state updated -> installed in
current_package_state and update_version_history, realigning both CHECK
constraints with the Go PackageStatus/HistoryStatus constants.
UpdateStats updated_updates -> installed_updates to match.
UpdateCurrentStateInTx documents its reconcile CASE as the SQL twin of
models.ReconcileFromScan so the two stay in lockstep.
Dashboard: vulnerable-package count surfaced in AttentionPanel, plus a
Vulnerable quick-filter on the Updates view.
SEC-001: Registration tokens stored as SHA-256 hashes. Migration 046 adds
token_hash column, backfills from plaintext, drops token column. All queries
use hash. Token plaintext shown once at creation (reveal panel in UI), never
retrievable again. Follows the refresh-token pattern.
SEC-005: README "no sanitization" claims corrected — code correctly sanitizes
against log injection (ANSI stripping, control char replacement, truncation).
Wording updated to match reality.
SEC-008: Command creation with idempotency_key uses ON CONFLICT DO NOTHING
instead of blind insert. Prevents duplicate command execution.
Trust model: Ed25519 key rotation documented — signing_keys table supports
multiple concurrent active keys with a sliding window for zero-downtime
rotation. OSV.dev ecosystem coverage updated (apt, dnf added).
A retried command carries the same action/result as its original, so the
history read as a fresh attempt. The lineage already lived in
agent_commands.retried_from_id — it just was not projected.
GetAllUnifiedHistory now selects is_retry + retried_from_id (both UNION
halves; logs are always false/null); UnifiedHistoryItem carries them; the
handler prefixes the narrative with "Retry — ". ChatTimeline composes its
own command sentences (narrative is only a log fallback), so it gets the
same prefix guarded by entry.is_retry, matching the is_retry/
retried_from_id convention LiveOperations already consumes.
Also drop a leftover heartbeat console.log debug block in Agents.tsx.
Make the capability gate runnable as installed. The helper is now a
first-class signed artifact distributed through the same pipeline as the
agent binary: built in the server image, signed at startup, listed in the
signed release manifest, served over GET /api/v1/helper/:arch with
X-Content-Signature, and Ed25519-verified + provisioned at install time
(binary root:root 0755, keyring, replay-guard dir, agent_id).
dnf discovery runs unprivileged: SandboxOpts redirect log/cache to an
agent-writable temp dir, so the agent holds zero dnf sudo (only the helper
invocation line). Removed the dead dnf discovery sudoers grants.
dnf artifact resolution: dnf5 pulls the matching .src.rpm from COPR-style
repos alongside the binary, which made singleRPMInDir refuse as ambiguous,
dropping the closure to empty and failing the mint closed ("no resolved
closure stored"). Filter source rpms before the ambiguity check so it pins
the one install artifact.
Drop the Fedora "updates" repo from the dnf security-severity heuristic;
it is not security-specific. Delete orphaned installer/sudoers.go (no
callers; emitted a contradictory unit). Migration 036: remove embedded
BEGIN/COMMIT that closed the runner's own transaction early.
- NeedsSupplyChainCheck: unchanged, OSV.dev eligibility (npm/pypi only)
- CanServerFetchArtifact: server can download from public registries (npm/pypi)
- NeedsCapabilityGate: ecosystems that route through capability tokens (dnf, apt, npm, pypi)
- computeAndStorePackageHash now uses CanServerFetchArtifact
- usesCapabilityExecution now uses NeedsCapabilityGate
Previously, ApproveUpdate logged mint_skipped for dnf/apt because
computeAndStorePackageHash returns empty (server cannot download those
artifacts). But the agent already resolved and reported the full closure
with per-artifact hashes via ReportDependencies → pinReportedClosure.
Now when artifactHash is empty, the handler calls mintResolvedClosure
which reads the stored closure from the dry-run phase. Server-fetched
ecosystems (npm/pypi) use the existing artifactHash path unchanged.
Bind /renew to the registered machine so a stolen refresh token can't mint
tokens from another host. Rotate the refresh token on every renewal; replaying
a consumed token whose successor is also consumed revokes the family. Accept-
previous-once grace covers agent crash-before-save. Typed auth errors so the
polling loop renews on 401 and treats refresh/machine failures as terminal.
No unsigned binary path: build refuses when signing is disabled, downloads
return 404 when no signed package resolves. Signed release manifest endpoint,
installer verifies manifest signature and pins binary hash, Windows token
mandatory, Rust helper verify-binary.
Extract the agent polling loop from main.go into internal/agent/loop.go
so the Windows service and the CLI agent share one code path. The loop
now reads jitter cap and backoff curve from PollingConfig (struct with
merge + file/env defaults) instead of hardcoding 30s/10s/300s. Machine
ID resolution uses the canonical system.GetMachineID() in both the
registration and runtime paths, removing the inline 'unknown-' fallback.
Stuck command retries are parameterized (maxRetries arg) rather than
hardcoded to < 5.
Restructure settings into a uniform hub-of-cards pattern: extract inline
Account Settings into /settings/general, un-orphan SecuritySettings with
working /settings/security/:tab routes. Add fleet-wide polling resilience
tuning (jitter_max_seconds, backoff_base_seconds, backoff_max_seconds)
as operational settings — stored in security_settings, delivered over
GET /api/v1/agents/:id/config, merged into the agent's local config at
runtime with a 15-minute refresh cadence. Frontend includes AgentPolling
page, hook, hub card, and route.
Lands the long-dropped in-flight work plus two slices of the pinning-mirror direction.
Registry-gap closure (in-flight, was repeatedly dropped):
- Agent resolves canonical artifact hashes from its own signed repo metadata
(dnf download + rpm header; apt-cache policy+show) — server no longer serves a
placeholder dnf URL and says so honestly.
- Server pins the agent-reported closure and mints the capability token at the
dependency-confirmation boundary; receipt updates package status.
Slice 1 — package detail pane:
- GET /updates/:id/fleet (cross-agent view). Detail pane gains Supply Chain card
(pinned sha256, published/age, age-gate verdict, resolved closure) and Affected
Agents card (per-host version delta + status, click-to-pivot).
Package-centric Updates list:
- ListAggregatedPackages rollup (GET /packages): one row per package across the
fleet — agent/version counts, max severity, vuln + hash-pin rollups, status
breakdown. List view rewritten to package rows that drill into the fleet view.
Slice 2 — version timeline catalog:
- migration 043 package_versions; idempotent upsert populated at scan, enriched at
approval (OSV posture, publish date, hash) and at closure pin (per-artifact hash).
- GET /updates/:id/versions + Version Timeline card.
UI: description overflow fix, shared table density px-6->px-4, status label cleanup.
Version: 0.2.0.7 across versions.go, docker-compose, Makefile (Makefile was stale at
0.2.0.3/0.2.0).
Server:
- ApproveUpdate() now calls computeAndStorePackageHash() to download artifact,
compute SHA256, and store in DB
- GET /dashboard/updates/verify-hash endpoint for agents to fetch hashes
Database:
- Migration 040: added expected_sha256 VARCHAR(64) to current_package_state table
Agent:
- HandleInstallUpdates() fetches expected hash from server before install
- DNFInstaller.VerifyHash() downloads and verifies package hash
- APT/Docker/Winget/WindowsUpdate: hash verification stubs (fail-open)
- LRU cache (100 entries) to reduce server load
Security:
- Hash verification happens BEFORE package manager install
- Mismatch blocks installation with error logged
- Fail-open: hash fetch failure doesn't block, but verification failure does
- AgentUpdatesEnhanced: ['active-commands'] → ['activeCommands'] (hyphenated key
never matched the camelCase query key, so invalidation was silently dead)
- useUpdates (install + approve): invalidate ['dashboard-stats'] and
['activeCommands'] on success so Dashboard and Live Operations react without
waiting for their independent poll cycles
- Updates.tsx handleConfirmDependencies: replace window.location.reload() with
targeted queryClient.invalidateQueries calls (BUG-017 pattern)
- LiveOperations: updateId = cmd.params?.update_id || cmd.id so "View Update
Details" navigates to the correct update package, not the command record
- useUpdates query: add refetchInterval: 30000 / staleTime: 15000 so agent-side
completions surface without window-focus or manual refresh
- updates.go ReportLog: emit system_event (agent_update/failed) when an
update_agent or verify_command command returns result=failed, using
RenderUpdateLog for operator-facing narrative
ReadWritePaths was missing the install target dir, so self-upgrade cp'd
against a ProtectSystem=strict read-only mount even via sudo. Added
${INSTALL_DIR} to the template and /usr/local/bin to the Go installer
constant.
queueSystemHeartbeat now writes heartbeat_source=system to agent metadata,
mirroring the pattern in TriggerHeartbeat / triggerSystemHeartbeat, so
GetHeartbeatStatus returns the correct source and the dashboard renders
the blue indicator for system-initiated heartbeats.
Three structural pieces that only make sense as a unit: the dispatch
chokepoint queries the policy table, and the event renderer is consumed by
the same handlers the policy gates guard.
B. Auto-heartbeat at the dispatch chokepoint (agents.go, models/command.go)
- models.RequiresRapidPolling(commandType): central classification (no
per-handler opt-in for rapid-polling commands).
- signAndCreateCommand auto-queues enable_heartbeat (Source=system)
ahead of any rapid-polling command, unless the agent is already in an
active heartbeat window.
- TimeoutService.reconcileAgentUpdates: effectiveUpdateTimeout() reads
operational.update_stuck_minutes live (no restart needed).
C. Event renderer (services/event_renderer.go NEW)
- RenderSystemEvent / RenderUpdateLog: single source of operator-facing
verbiage.
- Narrative field on SystemEvent / UpdateLog / UnifiedHistoryItem
(JSON-only, not persisted). Populated in agent_events.go GetAgentEvents
and updates.go GetAllLogs.
- ChatTimeline.tsx consults narrative only at the prior fallback line —
real stdout / package extraction branches untouched.
D. Policy table (migration 038 + security_settings_service helpers)
- policy.allow_dry_runs (default true): updates.go::InstallUpdate
returns 403 when false.
- policy.require_nonce (default true): agent_updates.go::UpdateAgent
skips nonce validation when false (logged at INFO).
- policy.auto_heartbeat_enabled (default true): agents.go gates the
auto-heartbeat side-effect.
- operational.update_stuck_minutes (default 5): TimeoutService reads
live for reconcile threshold.
- GetPolicyBool / GetOperationalInt on SecuritySettingsService.
Forward-only (no policy.allow_downgrade) is ETHOS §2 doctrine, not a knob.
UpdateAgent and BulkUpdateAgents were passing req.AgentID (zero UUID, never
populated in this code path) to the rollback / command-creation calls instead
of the agentIDUUID parsed from the URL path. The Source field was also set to
"web_ui" / "web_ui_bulk" which violates the agent_commands.source CHECK
constraint (allowed: manual/system), producing a 500 on every dashboard
update attempt.
- Replace req.AgentID with agentIDUUID at all rollback / command / log sites
- Change Source to "manual" in both single and bulk update commands
- Bump server AgentVersion + ConfigVersion from 0.2.0.3 to 0.2.0.4
- Remove stale hardcoded AgentVersion="0.1.16" from Windows service;
use version.Version (injected via ldflags at build time) instead
- Delete orphaned polling-loop body that survived the CRITICAL-007
refactor (the dead code between lines 178-360)
- CRITICAL-008: Fix sudoers templates in sudoers.go + linux.sh.tmpl to match
actual agent DNF/APT commands. Remove stale dnf refresh subcommand from
security.go AllowedCommands; align both templates with agent's real flags.
- CRITICAL-004+005 Phase 1: Remove demo-mode lie from windows.go installUpdates().
Failed installs now return error instead of false success. Add stderr checks
to wuauclt path. Hardcode GetPendingUpdates() replaced with error noting
go-ole COM API will be the real implementation.
- CRITICAL-006: Replace 15 lines of fmt.Printf debug noise in getWindowsCPUInfo()
with 5 structured log.Printf lines (ETHOS [TAG] format). Remove intermediate
parse-progress lines that had no diagnostic value.
- CRITICAL-007: Extract shared RunPollingLoop exported from agent package so
the Windows service (service/windows.go) calls the same loop as the console
agent instead of maintaining a forked copy with 5 duplicated handler functions.
Delete the 5 dead forked handlers (~500 lines). Add StopCh field for clean
service shutdown.
- P3-BUG-004: Delete dead deriveKeyFromNonce() and decryptAES256GCM() from
agent_update.go (~35 lines). Remove now-unused crypto/aes and crypto/cipher
imports.
Also included: pre-existing uncommitted work on dispatch, agent_updates,
AgentUpdatesModal, and reboot handler.
Server / agent install pipeline:
- Restore dry_run_update, confirm_dependencies, install_updates,
enable_heartbeat, disable_heartbeat handlers on the agent side (lost in
the TD-001 god-function refactor at 9da5134e); wire them through
handlers/dispatch.go so the cross-platform agent loop dispatches them
alongside scans and update_agent.
- Wire JWT renewal into the polling loop on 401 (RenewToken existed in the
client but was dead code in loop.go).
- Self-update path now shells through sudo for cp/chmod/systemctl restart,
matching the redflag-agent user's hardened systemd unit.
Server build orchestrator:
- BuildAndSignAgent now reuses the existing signed package row when the
on-disk binary's checksum matches the stored one. Previously the server
re-signed and inserted 4 fresh rows on every boot, leaving dozens of
duplicate agent_update_packages entries.
- CreateUpdatePackage is now ON CONFLICT (version, platform, architecture)
DO UPDATE so a fresh build of the same version replaces in place.
- New migration 037: dedupes existing rows (keep newest per tuple) and
enforces UNIQUE (version, platform, architecture).
- Drop dead verification.go endpoint stub — architecturally broken in a
pull-only polling model.
Dashboard:
- AgentUpdatesModal filters packages to the selected agents' os_type and
os_architecture, dedupes by (version, platform, arch), and renders
platform/arch together so 32/64-bit differentiate visually. Drops the
platform dropdown (now agent-driven).
Install script template:
- Fix server_public_key + initial_binary.sig ownership so the agent user
can overwrite them; convert hex key to raw 32 bytes inline.
- Add sudoers entries for the agent's self-update cp/chmod/systemctl path.
Downloads handler resolves ?version=latest to AgentVersion so install
scripts pull a signed package instead of a 404.
Version bumped to 0.2.0.3 across versions.go, docker-compose, Makefile,
downloads.go, security min_agent_version.
sanitize.go: ANSI strip, control-char scrub, per-field 4KB cap, total 16KB
cap, JSON validation. SecurityLogger.writeToFile now routes all string fields
through SanitizeForLog before writing.
Migration 036 converts every TIMESTAMP column to TIMESTAMPTZ across 20+ tables,
interpreting existing values as UTC. Server handlers, queries, services, and
agent files converted from time.Now() to time.Now().UTC() consistently.
Monotonic-paired time.Now() sites preserved where needed.
The general-case "any Git remote" adapter. source_ref is a clone URL —
HTTPS or git:// — anything go-git can resolve. Fetch performs the
equivalent of `git ls-remote --tags <url>` against the remote, filters
to tag refs (skipping the "^{}" peel suffix for annotated tags), and
picks the highest by CompareVersions (which is why the rc10/rc2 fix
matters: this adapter actually relies on numeric-aware suffix sort).
Uses go-git's in-memory storage so nothing hits disk and there's no
runtime dependency on the git binary in the server image. ListContext
respects the syncer's per-row 20-second timeout via the passed ctx.
Anonymous-only. Private repos belong on the dedicated github / gitea /
gitlab / bitbucket adapters which honor their token env vars; the
git_tags adapter is for the long tail (kernel.org, savannah, any
non-platform Git remote).
Dep additions: github.com/go-git/go-git/v5 plus its tree. The go.mod
hygiene upgrades to golang.org/x/* came along with `go get`; no API
changes affect existing code.
PublishedAt is left nil — ls-remote returns refs and SHAs, not tag
dates, and a follow-up fetch-by-tag would defeat the "no disk, no
clone" design. Operators who want the date should track via the
hosted-platform adapters.
Three more ReleaseSource adapters, all parallel in shape to GitHub:
- gitea — /api/v1/repos/{owner}/{repo}/releases/latest, host from
REDFLAG_GITEA_HOST (mandatory; empty = adapter still
registers but every Fetch surfaces the misconfig on
tracked_software.last_error), token from REDFLAG_GITEA_TOKEN.
- gitlab — /api/v4/projects/{url-encoded path}/releases/permalink/latest,
host from REDFLAG_GITLAB_HOST (defaults to gitlab.com),
token via PRIVATE-TOKEN header from REDFLAG_GITLAB_TOKEN.
source_ref supports nested namespaces (group/subgroup/project).
- bitbucket — Bitbucket Cloud has no "latest release" endpoint, so we
list /2.0/repositories/{ws}/{repo}/refs/tags?sort=-name and
pick the highest via the new CompareVersions (lex-sort from
Bitbucket isn't semver-aware; this closes the loop on the
previous commit). Token from REDFLAG_BITBUCKET_TOKEN.
owner/repo splitting goes through the shared splitOwnerRepo helper
introduced with the github adapter.
Registered in cmd/server/main.go alongside the other adapters.
Adds the third ReleaseSource adapter, dispatched by source="github".
source_ref is "owner/repo" (e.g. "kubernetes/kubernetes"). Calls
GET /repos/{owner}/{repo}/releases/latest and maps tag_name +
published_at + html_url onto the normalized Release shape.
Auth via REDFLAG_GITHUB_TOKEN env var, optional. Unauthenticated cap
is 60 req/hr per source IP across the syncer; authenticated cap is
5000/hr. The 403-with-X-RateLimit-Remaining=0 case surfaces a
distinct error pointing operators at the token, instead of returning
a generic "github said no."
splitOwnerRepo helper will also be used by the gitea/bitbucket
adapters in the next commit (they share the "owner/repo" ref shape),
which is why it's package-level rather than inlined.
Registered in cmd/server/main.go upstream compose block.
The scheduler created agent commands with an empty Signature field and
handed them to CreateCommand, while admin-initiated commands went
through SigningService.SignCommand first. Result: scheduled scans /
update checks reached agents unsigned and were either dropped at
verify time or — worse — slipped through if signing enforcement was
lax. ETHOS §2 ("Security is Non-Negotiable") doesn't permit either.
Wire signingService through NewScheduler. In the worker's processJob,
hard-fail if the service is missing or disabled rather than silently
emitting an unsigned command, and sign the command before CreateCommand.
Reconciliation note in vanguards-memories/system/vanguard-state.md
(line 51) already marked this DONE for 2026-05-23; this commit makes
the working tree match the memory.
The suffix tail of CompareVersions was a plain string compare, so any
multi-digit prerelease counter sorted wrong: "-rc10" lex-compared less
than "-rc2" because '1' < '2'. Drift severity for a project doing rc
iterations would flap unpredictably and git-tag adapters (next commit)
would pick the wrong "highest" tag.
Replace the lex tail with compareSuffix: strip '+build' metadata per
SemVer §10, then walk both sides chunk-by-chunk where each chunk is a
maximal run of digits or non-digits. Digit chunks compare numerically;
non-digit chunks compare lexically; mixed chunks let the digit run sort
lesser (SemVer §11.4.3, also matches rpmvercmp). Shorter prefix wins
when common chunks are equal (SemVer §11.4.4).
Still hand-rolled rather than golang.org/x/mod/semver — that library
rejects "15.4" outright and Repology/endoflife return such versions.
Test additions cover the rc10/rc2 case, dot-separated SemVer
prereleases, build-metadata stripping, alpha<beta<rc lex ordering,
the "more identifiers wins" rule, postgres-style "15rc1" suffixes,
and date-based majors like "20231130-1.fc40".
IsWithinMaintenanceWindow returned false when the table was empty,
blocking every install with 403. Now: no windows configured = unrestricted.
Windows configured = only allow inside them (unchanged behavior).
Six files: compiled-in AgentVersion/ConfigVersion, LATEST_AGENT_VERSION env
default, downloads config_template agent_version, security min_agent_version,
and Makefile ldflags for native agent builds. docker-compose.yml was already
fixed. The Dockerfile uses the ARG BUILD_VERSION so it picks up the compose
default.
