ReadWritePaths was missing the install target dir, so self-upgrade cp'd
against a ProtectSystem=strict read-only mount even via sudo. Added
${INSTALL_DIR} to the template and /usr/local/bin to the Go installer
constant.
queueSystemHeartbeat now writes heartbeat_source=system to agent metadata,
mirroring the pattern in TriggerHeartbeat / triggerSystemHeartbeat, so
GetHeartbeatStatus returns the correct source and the dashboard renders
the blue indicator for system-initiated heartbeats.
Three structural pieces that only make sense as a unit: the dispatch
chokepoint queries the policy table, and the event renderer is consumed by
the same handlers the policy gates guard.
B. Auto-heartbeat at the dispatch chokepoint (agents.go, models/command.go)
- models.RequiresRapidPolling(commandType): central classification (no
per-handler opt-in for rapid-polling commands).
- signAndCreateCommand auto-queues enable_heartbeat (Source=system)
ahead of any rapid-polling command, unless the agent is already in an
active heartbeat window.
- TimeoutService.reconcileAgentUpdates: effectiveUpdateTimeout() reads
operational.update_stuck_minutes live (no restart needed).
C. Event renderer (services/event_renderer.go NEW)
- RenderSystemEvent / RenderUpdateLog: single source of operator-facing
verbiage.
- Narrative field on SystemEvent / UpdateLog / UnifiedHistoryItem
(JSON-only, not persisted). Populated in agent_events.go GetAgentEvents
and updates.go GetAllLogs.
- ChatTimeline.tsx consults narrative only at the prior fallback line —
real stdout / package extraction branches untouched.
D. Policy table (migration 038 + security_settings_service helpers)
- policy.allow_dry_runs (default true): updates.go::InstallUpdate
returns 403 when false.
- policy.require_nonce (default true): agent_updates.go::UpdateAgent
skips nonce validation when false (logged at INFO).
- policy.auto_heartbeat_enabled (default true): agents.go gates the
auto-heartbeat side-effect.
- operational.update_stuck_minutes (default 5): TimeoutService reads
live for reconcile threshold.
- GetPolicyBool / GetOperationalInt on SecuritySettingsService.
Forward-only (no policy.allow_downgrade) is ETHOS §2 doctrine, not a knob.
UpdateAgent and BulkUpdateAgents were passing req.AgentID (zero UUID, never
populated in this code path) to the rollback / command-creation calls instead
of the agentIDUUID parsed from the URL path. The Source field was also set to
"web_ui" / "web_ui_bulk" which violates the agent_commands.source CHECK
constraint (allowed: manual/system), producing a 500 on every dashboard
update attempt.
- Replace req.AgentID with agentIDUUID at all rollback / command / log sites
- Change Source to "manual" in both single and bulk update commands
Leftover from 8d441a5b RunPollingLoop extraction, which deleted the
forked Windows handler block but left the import. Blocked the Windows
agent cross-compile in docker build.
- Bump server AgentVersion + ConfigVersion from 0.2.0.3 to 0.2.0.4
- Remove stale hardcoded AgentVersion="0.1.16" from Windows service;
use version.Version (injected via ldflags at build time) instead
- Delete orphaned polling-loop body that survived the CRITICAL-007
refactor (the dead code between lines 178-360)
- CRITICAL-008: Fix sudoers templates in sudoers.go + linux.sh.tmpl to match
actual agent DNF/APT commands. Remove stale dnf refresh subcommand from
security.go AllowedCommands; align both templates with agent's real flags.
- CRITICAL-004+005 Phase 1: Remove demo-mode lie from windows.go installUpdates().
Failed installs now return error instead of false success. Add stderr checks
to wuauclt path. Hardcode GetPendingUpdates() replaced with error noting
go-ole COM API will be the real implementation.
- CRITICAL-006: Replace 15 lines of fmt.Printf debug noise in getWindowsCPUInfo()
with 5 structured log.Printf lines (ETHOS [TAG] format). Remove intermediate
parse-progress lines that had no diagnostic value.
- CRITICAL-007: Extract shared RunPollingLoop exported from agent package so
the Windows service (service/windows.go) calls the same loop as the console
agent instead of maintaining a forked copy with 5 duplicated handler functions.
Delete the 5 dead forked handlers (~500 lines). Add StopCh field for clean
service shutdown.
- P3-BUG-004: Delete dead deriveKeyFromNonce() and decryptAES256GCM() from
agent_update.go (~35 lines). Remove now-unused crypto/aes and crypto/cipher
imports.
Also included: pre-existing uncommitted work on dispatch, agent_updates,
AgentUpdatesModal, and reboot handler.
Server / agent install pipeline:
- Restore dry_run_update, confirm_dependencies, install_updates,
enable_heartbeat, disable_heartbeat handlers on the agent side (lost in
the TD-001 god-function refactor at 9da5134e); wire them through
handlers/dispatch.go so the cross-platform agent loop dispatches them
alongside scans and update_agent.
- Wire JWT renewal into the polling loop on 401 (RenewToken existed in the
client but was dead code in loop.go).
- Self-update path now shells through sudo for cp/chmod/systemctl restart,
matching the redflag-agent user's hardened systemd unit.
Server build orchestrator:
- BuildAndSignAgent now reuses the existing signed package row when the
on-disk binary's checksum matches the stored one. Previously the server
re-signed and inserted 4 fresh rows on every boot, leaving dozens of
duplicate agent_update_packages entries.
- CreateUpdatePackage is now ON CONFLICT (version, platform, architecture)
DO UPDATE so a fresh build of the same version replaces in place.
- New migration 037: dedupes existing rows (keep newest per tuple) and
enforces UNIQUE (version, platform, architecture).
- Drop dead verification.go endpoint stub — architecturally broken in a
pull-only polling model.
Dashboard:
- AgentUpdatesModal filters packages to the selected agents' os_type and
os_architecture, dedupes by (version, platform, arch), and renders
platform/arch together so 32/64-bit differentiate visually. Drops the
platform dropdown (now agent-driven).
Install script template:
- Fix server_public_key + initial_binary.sig ownership so the agent user
can overwrite them; convert hex key to raw 32 bytes inline.
- Add sudoers entries for the agent's self-update cp/chmod/systemctl path.
Downloads handler resolves ?version=latest to AgentVersion so install
scripts pull a signed package instead of a 404.
Version bumped to 0.2.0.3 across versions.go, docker-compose, Makefile,
downloads.go, security min_agent_version.
Offline agent status is already prominent in the main agents list and stats
panel. Duplicating it in the attention feed added noise without urgency signal.
Removed WifiOff import and offline severity rank.
generateEnvContent was appending REDFLAG_SIGNING_PRIVATE_KEY after
server-side createSharedEnvContentForDisplay had already embedded it,
producing two copies in the setup output.
dispatch.go::DispatchCrossPlatformCommand centralizes all scan + update_agent
routing. loop.go and service/windows.go now call through it instead of
maintaining duplicate switches. Windows service switch retains only
windows-specific cases (collect_specs, dry_run_update).
sanitize.go: ANSI strip, control-char scrub, per-field 4KB cap, total 16KB
cap, JSON validation. SecurityLogger.writeToFile now routes all string fields
through SanitizeForLog before writing.
Migration 036 converts every TIMESTAMP column to TIMESTAMPTZ across 20+ tables,
interpreting existing values as UTC. Server handlers, queries, services, and
agent files converted from time.Now() to time.Now().UTC() consistently.
Monotonic-paired time.Now() sites preserved where needed.
Surface the gitea/gitlab/bitbucket/git adapters in the
UpstreamTracking page so operators can actually pick them from the
form. Three changes:
- UpstreamSource union in types/index.ts gains 'gitea' | 'gitlab' |
'bitbucket' | 'git'. The /admin/upstream API already returns these
in its "sources" array (handler reads Registry.Names()), so the
dropdown picks them up automatically once the union allows.
- New sourceRefHint(source) returns per-source placeholder + help
text so the source_ref field documents itself instead of always
saying "e.g. postgresql". Each entry names the required env vars
so operators don't 404 their way to discovering REDFLAG_GITHUB_TOKEN.
- New sourceHref(s) replaces the hard-coded ternary that only knew
repology and endoflife. Sources without a stable public URL pattern
(gitea/anitya/npm/pypi, where host varies per deployment) return
null and skip the "Source" link rather than guessing wrong.
COMMON_SEEDS chips deliberately unchanged — those are the "one-click
to a working dashboard" path for new operators, all on endoflife
because EOL coverage is what justifies the chip. The form is the path
for everything else.
The general-case "any Git remote" adapter. source_ref is a clone URL —
HTTPS or git:// — anything go-git can resolve. Fetch performs the
equivalent of `git ls-remote --tags <url>` against the remote, filters
to tag refs (skipping the "^{}" peel suffix for annotated tags), and
picks the highest by CompareVersions (which is why the rc10/rc2 fix
matters: this adapter actually relies on numeric-aware suffix sort).
Uses go-git's in-memory storage so nothing hits disk and there's no
runtime dependency on the git binary in the server image. ListContext
respects the syncer's per-row 20-second timeout via the passed ctx.
Anonymous-only. Private repos belong on the dedicated github / gitea /
gitlab / bitbucket adapters which honor their token env vars; the
git_tags adapter is for the long tail (kernel.org, savannah, any
non-platform Git remote).
Dep additions: github.com/go-git/go-git/v5 plus its tree. The go.mod
hygiene upgrades to golang.org/x/* came along with `go get`; no API
changes affect existing code.
PublishedAt is left nil — ls-remote returns refs and SHAs, not tag
dates, and a follow-up fetch-by-tag would defeat the "no disk, no
clone" design. Operators who want the date should track via the
hosted-platform adapters.
Three more ReleaseSource adapters, all parallel in shape to GitHub:
- gitea — /api/v1/repos/{owner}/{repo}/releases/latest, host from
REDFLAG_GITEA_HOST (mandatory; empty = adapter still
registers but every Fetch surfaces the misconfig on
tracked_software.last_error), token from REDFLAG_GITEA_TOKEN.
- gitlab — /api/v4/projects/{url-encoded path}/releases/permalink/latest,
host from REDFLAG_GITLAB_HOST (defaults to gitlab.com),
token via PRIVATE-TOKEN header from REDFLAG_GITLAB_TOKEN.
source_ref supports nested namespaces (group/subgroup/project).
- bitbucket — Bitbucket Cloud has no "latest release" endpoint, so we
list /2.0/repositories/{ws}/{repo}/refs/tags?sort=-name and
pick the highest via the new CompareVersions (lex-sort from
Bitbucket isn't semver-aware; this closes the loop on the
previous commit). Token from REDFLAG_BITBUCKET_TOKEN.
owner/repo splitting goes through the shared splitOwnerRepo helper
introduced with the github adapter.
Registered in cmd/server/main.go alongside the other adapters.
Adds the third ReleaseSource adapter, dispatched by source="github".
source_ref is "owner/repo" (e.g. "kubernetes/kubernetes"). Calls
GET /repos/{owner}/{repo}/releases/latest and maps tag_name +
published_at + html_url onto the normalized Release shape.
Auth via REDFLAG_GITHUB_TOKEN env var, optional. Unauthenticated cap
is 60 req/hr per source IP across the syncer; authenticated cap is
5000/hr. The 403-with-X-RateLimit-Remaining=0 case surfaces a
distinct error pointing operators at the token, instead of returning
a generic "github said no."
splitOwnerRepo helper will also be used by the gitea/bitbucket
adapters in the next commit (they share the "owner/repo" ref shape),
which is why it's package-level rather than inlined.
Registered in cmd/server/main.go upstream compose block.
The scheduler created agent commands with an empty Signature field and
handed them to CreateCommand, while admin-initiated commands went
through SigningService.SignCommand first. Result: scheduled scans /
update checks reached agents unsigned and were either dropped at
verify time or — worse — slipped through if signing enforcement was
lax. ETHOS §2 ("Security is Non-Negotiable") doesn't permit either.
Wire signingService through NewScheduler. In the worker's processJob,
hard-fail if the service is missing or disabled rather than silently
emitting an unsigned command, and sign the command before CreateCommand.
Reconciliation note in vanguards-memories/system/vanguard-state.md
(line 51) already marked this DONE for 2026-05-23; this commit makes
the working tree match the memory.
The suffix tail of CompareVersions was a plain string compare, so any
multi-digit prerelease counter sorted wrong: "-rc10" lex-compared less
than "-rc2" because '1' < '2'. Drift severity for a project doing rc
iterations would flap unpredictably and git-tag adapters (next commit)
would pick the wrong "highest" tag.
Replace the lex tail with compareSuffix: strip '+build' metadata per
SemVer §10, then walk both sides chunk-by-chunk where each chunk is a
maximal run of digits or non-digits. Digit chunks compare numerically;
non-digit chunks compare lexically; mixed chunks let the digit run sort
lesser (SemVer §11.4.3, also matches rpmvercmp). Shorter prefix wins
when common chunks are equal (SemVer §11.4.4).
Still hand-rolled rather than golang.org/x/mod/semver — that library
rejects "15.4" outright and Repology/endoflife return such versions.
Test additions cover the rc10/rc2 case, dot-separated SemVer
prereleases, build-metadata stripping, alpha<beta<rc lex ordering,
the "more identifiers wins" rule, postgres-style "15rc1" suffixes,
and date-based majors like "20231130-1.fc40".
IsWithinMaintenanceWindow returned false when the table was empty,
blocking every install with 403. Now: no windows configured = unrestricted.
Windows configured = only allow inside them (unchanged behavior).
Six files: compiled-in AgentVersion/ConfigVersion, LATEST_AGENT_VERSION env
default, downloads config_template agent_version, security min_agent_version,
and Makefile ldflags for native agent builds. docker-compose.yml was already
fixed. The Dockerfile uses the ARG BUILD_VERSION so it picks up the compose
default.
The tagged release was building as 0.2.0 because docker-compose.yml defaulted
to the old version. New installs and rebuilds now pick up the correct version
from the file.
Migration 020 created the security_settings table with updated_at /
updated_by but no created_at / created_by columns. Query code
(database/queries/security_settings.go) SELECTs and INSERTs both
create-side fields, causing "failed to initialize default security
settings" at server startup — the dashboard's security panel then
shows hardcoded defaults instead of DB-backed values.
Adds both columns with NOT NULL + DEFAULT NOW() on created_at and a
nullable FK to users on created_by, matching the updated_by shape.
No backfill scaffolding (no live clients per release stance).
Closes AUDIT_TASKS.md §4.
Single-approve at /updates/:id/approve has always run the OSV.dev
ecosystem check for npm/PyPI packages, but the bulk endpoint
/updates/approve called BulkApproveUpdates directly — a single DB
write with no vulnerability lookup. So selecting N items in the UI
silently bypassed a check the README advertises.
ApproveUpdates now mirrors the single-approve loop: GetUpdateByID,
NeedsSupplyChainCheck, CheckOSVVulnerabilities, then
ApproveUpdateWithVulns when the OSV query returns CVEs (preserving
supply_chain_vulns / supply_chain_checked_at in metadata) or plain
ApproveUpdate when clean. Per-package warnings are aggregated and
returned to the caller. Fail-open semantics from the single-approve
path carry through — OSV unreachable does not block approval.
Also removed server/internal/api/handlers/update_handler.go.
UnifiedUpdateHandler was a parallel implementation of every
UpdateHandler method but NewUnifiedUpdateHandler was never called
from main.go or anywhere else. The file was confusing on grep and
masked which approve path was actually wired.
The agent runs apt/dnf/winget/windows on their own schedules (per-scanner
agent_subsystems rows still exist server-side) but the UI now collapses
them into a single "System Update Scanner" row. Chips on the description
line (APT, DNF, WINGET, Windows Update) light up from
agent.metadata.available_scanners — the real signal from
syncAvailableScanners — rather than the old os_type string heuristic
that sniffed "fedora" / "debian" out of the platform field.
The row is synthesized client-side from the per-scanner backers; no
'updates' DB row needed (the zombie kill landed in 7fb61a36). Toggle,
auto-run, interval, and Scan on the unified row cascade to every backing
per-scanner subsystem that actually exists for the agent.
Widened AgentSubsystem.subsystem type to string — the narrow union was
stale (DB has apt/dnf/winget/windows/storage/system/docker, plus the
synthetic 'updates' aggregate the UI produces).
Removed the os-string heuristic helper and its console.log noise.
The legacy "updates" virtual subsystem was deprecated (scheduler.go:159
skips it), but five paths kept re-creating the row in agent_subsystems:
- agent migration detection flagged missing "updates" subsystem as a
missing security feature, prompting the executor to re-add it to the
local config on every startup migration
- server install-config template, scanner-timeout list, and intervals
map all kept "updates" alive in the config artifact sent to agents
Removed at all five sites. Scheduler skip logic, per-scanner mapping
helper (subsystems.go:236), and update-report data path remain — they
are not subsystem-row creators.
Historical migration 024_disable_updates_subsystem left intact.
Registration retries hit the (agent_id, subsystem) unique constraint,
which errored out the INSERT and poisoned the entire transaction. The
handler treated this as non-fatal, but PostgreSQL doesn't allow any
further statements in an aborted tx.
Adds ON CONFLICT DO NOTHING + treats sql.ErrNoRows as success.
validator.go references MigrationDetection and AgentFileInventory from
the parent migration package but was missing the import, causing the agent
build to fail since the aggregator-agent rename in c8b0ee10.
Migration 033 adds the 'received' status to agent_commands so the server can
distinguish "agent confirmed receipt" from "sent but may be lost in flight."
Stuck-command re-issuance now excludes received commands — the TimeoutService
handles the longer timeout for those (default 30m) vs the per-poll re-issuer
(sent/pending at 5m).
The agent side: disk-persists executed command IDs to survive restart (closes
the in-memory-only dedup gap), reports received_command_ids on each check-in so
the server transitions sent→received before issuing new work, and authenticates
binary downloads with JWT+X-Machine-ID (was unauthenticated http.Get — would
401 in production).
TimeoutService extended with reconcileAgentUpdates: clears is_updating when
current_version matches updating_to_version (success), or after a 15m threshold
(timeout, with system_event) so the dashboard never shows "updating" forever.
isVersionUpgrade replaced with utils.IsNewerVersion (no panic on 2-part
versions, no false-reject on 4-part).
MarkCommand* failures elevated from [WARNING] to [ERROR] + should_retry
response hint so agents know to re-deliver results (silent drops were ETHOS #1
violations).
Fixes: build broken on public since eac8a012 (command.go accidentally emptied).
