RAF: full docs pass — components, flows, security, scanners, reference, testing, verification, overview
This commit is contained in:
Fimeg
parent
565baa0819
commit
a19dcf4f14
34 changed files with 5085 additions and 28 deletions
141
RAF/OVERVIEW.md
Normal file
141
RAF/OVERVIEW.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
# Start Here: RedFlag Architecture Overview
|
||||
|
||||
**Version:** v0.2.8.0 (June 2026)
|
||||
|
||||
This is the entry point into the RedFlag Architecture Framework. Read this first to
|
||||
understand the shape of the system, then follow the links into the detailed docs.
|
||||
|
||||
---
|
||||
|
||||
## What RedFlag Is
|
||||
|
||||
A self-hosted update management platform for homelabs and small MSPs. Centralized
|
||||
visibility and control over software updates across Linux, Windows, and Docker — with
|
||||
a cryptographic supply chain gate that most commercial RMM tools don't attempt.
|
||||
|
||||
---
|
||||
|
||||
## The Two Capability Tiers
|
||||
|
||||
### Tier 1: Update Management
|
||||
|
||||
Agents register with a one-time token and a hardware fingerprint (TOFU). The server
|
||||
issues Ed25519-signed commands; agents verify signatures, check nonces, reject replays.
|
||||
Pull-based polling (5 min default, rapid mode available). Subsystem scanning across
|
||||
apt, dnf, winget, WUA, and Docker.
|
||||
|
||||
Packages move through a server-owned state machine (`pending` through `installed` or
|
||||
`failed`) with typed transitions and guarded UPDATEs — no free-form string jumps. A
|
||||
lifecycle orchestrator drives auto-advance and recovers stuck states.
|
||||
|
||||
**Architecture docs:**
|
||||
- [[core/01-ethos]] — the five principles
|
||||
- [[core/02-architecture-decisions]] — the twelve foundational choices
|
||||
- [[security/02-authentication-stack]] — four-layer auth (reg tokens, JWT, refresh, machine binding)
|
||||
- [[security/01-trust-boundaries]] — endpoint classification and middleware matrix
|
||||
- [[flows/06-update-lifecycle]] — state machine, two execution paths, orchestrator
|
||||
|
||||
### Tier 2: Supply Chain Gate
|
||||
|
||||
The differentiator. The server is the signing authority — it evaluates policy (OSV
|
||||
vulnerability checks, package age, human approval) and mints an Ed25519-signed
|
||||
capability token describing exactly one operation over a fully-resolved dependency
|
||||
closure. A privileged, network-less Rust executor (`helper/`) verifies the signature
|
||||
and every artifact hash before performing that one operation.
|
||||
|
||||
The approval gate is fail-closed: a known vulnerability anywhere in the resolved
|
||||
closure — top-level or transitive — blocks the token from being minted. The operator
|
||||
must override with a documented reason. The override waives the vulnerability judgment
|
||||
only; signing and hash verification have no skip path.
|
||||
|
||||
Auto-confirm shares the same `ClosureCleared` predicate as manual approval — the two
|
||||
paths cannot drift on what counts as a clean closure.
|
||||
|
||||
**Architecture docs:**
|
||||
- [[security/05-supply-chain-gate]] — the design of record: capability model, wire contract,
|
||||
load-bearing constraints, enforcement layers, trust chain, hash registry
|
||||
- `docs/tasks/GATE-000-supply-chain-gate-plan.md` — build status & implementation tracking (not design)
|
||||
|
||||
### Process Explorer
|
||||
|
||||
On-demand `/proc` filesystem scanning for process inventory and drill-down detail.
|
||||
Triggered when a user opens the Processes tab — no background broadcasting.
|
||||
25+ fields per process (osquery parity) plus 7 related data types (open files,
|
||||
sockets, pipes, environment keys, memory map, namespaces, listening ports).
|
||||
|
||||
Data collection caps are server-controlled via `ProcessExplorerConfig` (Settings →
|
||||
Process Explorer) and delivered to agents on check-in. Listening ports use socket
|
||||
inode correlation against `/proc/net/tcp` — not system-wide assignment.
|
||||
|
||||
**Architecture docs:**
|
||||
- [[scanners/05-process-scanner]] — data model, collection, caps
|
||||
- [[flows/07-process-scan]] — command-dispatch flow, API endpoints, schema
|
||||
|
||||
---
|
||||
|
||||
## Architectural Boundaries
|
||||
|
||||
### Agent is a Stateless Executor
|
||||
|
||||
The agent receives commands, executes them, and reports results. It does not track
|
||||
lifecycle states. The server owns every state transition. The agent's only autonomous
|
||||
decisions are: verify this signature, check this nonce, reject this replay.
|
||||
|
||||
### Mutation Only Through the Helper
|
||||
|
||||
On capability-gated ecosystems (dnf, apt), the agent cannot run install commands
|
||||
directly. All mutation flows through `consumer.go` → `sudo systemd-run --pipe` →
|
||||
`redflag-helper`. The agent holds zero sudo for installs. Discovery (scan, dry-run,
|
||||
hash-resolve) runs unprivileged through `DiscoveryRunner`.
|
||||
|
||||
### Two Execution Paths
|
||||
|
||||
- **Capability gate** (dnf, apt): token minted at approval → agent polls for tokens →
|
||||
helper verifies + executes → agent reports receipt. No install command issued.
|
||||
- **Legacy command** (docker, winget, windows_update): signed command → agent executes
|
||||
directly via type-asserted installer methods → reports via ReportLog.
|
||||
|
||||
The legacy path is a known gap — the gate design covers these ecosystems but
|
||||
implementation is deferred.
|
||||
|
||||
### Six Load-Bearing Constraints
|
||||
|
||||
From `security/05-supply-chain-gate.md` — do not regress these:
|
||||
|
||||
1. Sign the resolved closure, not the top-level package
|
||||
2. The signer lives off the web process (seam documented, not yet isolated)
|
||||
3. Verified-cache fallback, fail-closed only on change
|
||||
4. Verify keys, not servers
|
||||
5. Kernel stops are defense-in-depth, not a prerequisite
|
||||
6. No doctrinal knobs — signing required and forward-only are not configurable
|
||||
|
||||
---
|
||||
|
||||
## Navigation
|
||||
|
||||
| Section | What It Describes |
|
||||
|---------|-------------------|
|
||||
| [[core]] | ETHOS principles, architectural decisions |
|
||||
| [[components]] | Server, agent, web, helper — package structure and responsibilities |
|
||||
| [[security]] | Trust boundaries, auth stack, machine binding, supply chain gate |
|
||||
| [[verification]] | Ed25519 signing pipeline, agent verification, key rotation, replay protection |
|
||||
| [[scanners]] | Per-ecosystem scanner behavior and integration points (incl. process scanner) |
|
||||
| [[flows]] | Data flows — registration, command execution, upgrade, heartbeat, capability advertisement, update lifecycle |
|
||||
| [[reference]] | File mappings, glossary |
|
||||
|
||||
---
|
||||
|
||||
## Honest Gaps
|
||||
|
||||
- **Gate policy visibility**: the soak and age gates are live policies as of v0.2.6.2 (`supply_chain.*` settings — see [[security/05-supply-chain-gate]] §4), but the dashboard doesn't yet surface their configuration; operators tune them blind. The live install-through-helper path completed e2e on 2026-06-05
|
||||
- **Closure transitivity split by platform**: dnf/apt resolve via dry-run (full closure); npm/pypi still single-entry
|
||||
- **Signer in-process**: key encapsulated in SigningService, minter is the only caller — but true process isolation not built
|
||||
- **Legacy ecosystems ungated**: docker, winget, windows_update still direct-mutation
|
||||
- **Kernel enforcement inert**: eBPF scaffold exists, not wired to the capability model
|
||||
|
||||
These are architectural gaps, not bugs. They define where the system's protection
|
||||
boundary currently ends. Task tracking for closing them lives in `docs/tasks/`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-10*
|
||||
|
|
@ -1,6 +1,10 @@
|
|||
# RedFlag Architecture Framework (RAF)
|
||||
|
||||
**Complete architectural spine of RedFlag — every component, scanner, verification system, and how they all wire together.**
|
||||
**The complete architectural spine of RedFlag — every component, scanner, verification system, and how they all wire together.**
|
||||
|
||||
This is the design of record, published in the open. Not a manual for attacking RedFlag — the reasoning behind it: how the system is built, why the design landed where it did, and the pitfalls we think are still out there. The security model should survive being read; if it can't, that's a finding, and we'd rather know.
|
||||
|
||||
It describes a system under active development. Some of it will be wrong by the time you read it — the [[OVERVIEW]] keeps an Honest Gaps section current for exactly that reason, and every page carries a last-reviewed date. Trust the code over the doc when they disagree, and tell us.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -8,26 +12,43 @@
|
|||
|
||||
| Section | Description |
|
||||
|---------|-------------|
|
||||
| [[OVERVIEW]] | **START HERE** — architecture overview: what RedFlag is, the two capability tiers, architectural boundaries |
|
||||
| [[core]] | Core concepts, ETHOS principles, architectural decisions |
|
||||
| [[components]] | Server, agent, web — component breakdowns |
|
||||
| [[security]] | Trust boundaries, auth layers, machine binding, key management, supply chain gate |
|
||||
| [[verification]] | Ed25519 signing, verification pipeline, v2/v3 formats, replay protection |
|
||||
| [[scanners]] | Every scanner (APT, DNF, Winget, WUA, Docker, macOS SoftwareUpdate, etc) with interaction analysis |
|
||||
| [[flows]] | Data flows — registration, command execution, updates, self-upgrade |
|
||||
| [[deployment]] | Docker, production checklist, monitoring |
|
||||
| [[testing]] | Test pyramid, test coverage, manual testing |
|
||||
| [[reference]] | External references, glossary, file mappings |
|
||||
| [[OVERVIEW]] | **START HERE** — architecture overview: what RedFlag is, the two capability tiers, architectural boundaries, honest gaps |
|
||||
| [[core]] | ETHOS principles, the foundational architectural decisions |
|
||||
| [[components]] | Server, agent, web, helper — component breakdowns |
|
||||
| [[security]] | Trust boundaries, auth stack, refresh-token lifecycle, machine binding, supply chain gate, standalone authority |
|
||||
| [[verification]] | Ed25519 signing pipeline, agent verification, key rotation, replay protection |
|
||||
| [[scanners]] | Every scanner (APT, DNF, Winget, WUA, Docker, process explorer) with interaction analysis |
|
||||
| [[flows]] | Data flows — registration, command execution, upgrade, heartbeat, capability advertisement, update lifecycle |
|
||||
| [[deployment]] | Docker stack, native agent services, CI/CD, release gate, operations runbook pointers |
|
||||
| [[testing]] | Test pyramid, structural tests, live testing, honest gaps |
|
||||
| [[reference]] | File mappings, [[reference/02-glossary|glossary]] |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
## Reading Order
|
||||
|
||||
1. **Read [[core]]** — ETHOS principles and architectural foundation
|
||||
2. **Follow [[flows]]** — Trace the critical data flows end-to-end
|
||||
3. **Explore [[scanners]]** — Understand each scanner's behavior and integration points
|
||||
4. **Study [[verification]]** — Cryptographic verification pipeline
|
||||
5. **Reference [[reference]]** — File mappings, glossary, external links
|
||||
1. **[[OVERVIEW]]** — the shape of the system and where its protection boundary currently ends
|
||||
2. **[[core]]** — ETHOS principles and the decisions everything else hangs off
|
||||
3. **[[flows]]** — trace the critical data flows end-to-end
|
||||
4. **[[security]] + [[verification]]** — the trust model and the cryptographic pipeline
|
||||
5. **[[scanners]] + [[components]]** — per-ecosystem behavior and package structure
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
RedFlag is free and will never be monetized. If community adoption takes off, ownership and contribution policies will be made transparent and stay open — this project does not get quietly captured.
|
||||
|
||||
Before proposing architectural changes:
|
||||
|
||||
1. Read [[core]] → [[flows]] → [[verification]] for context — most "why is it like this" questions are answered there
|
||||
2. The five ETHOS principles and the six load-bearing constraints ([[OVERVIEW]]) are the floor, not a starting position
|
||||
3. Update the relevant page *and its cross-references*; stale links are bugs
|
||||
|
||||
A note on `docs/tasks/` references: several pages point at the maintainer's task tracker
|
||||
for build status. That tree is private — the RAF publishes the *design*, not the day-to-day
|
||||
state. Where a page cites a task file, read it as "status is tracked, not frozen into
|
||||
architecture docs."
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -35,6 +56,7 @@
|
|||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 2.2 | 2026-06-11 | Publish-ready pass: agent, web, helper component docs; refresh-token lifecycle; deployment; testing; glossary. Public framing. |
|
||||
| 2.1 | 2026-06-01 | Updated for v0.2.3.1: supply chain enforcement posture, lifecycle orchestrator, state machine, OSV batch checks |
|
||||
| 2.0 | 2026-05-26 | Restructured for single-source-of-truth organization |
|
||||
| 1.3 | 2026-05-06 | Added §11 eight structural patterns |
|
||||
|
|
@ -42,14 +64,4 @@
|
|||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
This is the authoritative source of RedFlag architecture. Before making changes:
|
||||
|
||||
1. Run branch ancestry check: `git branch --show-current && git log --graph --oneline -30`
|
||||
2. Read [[core]] → [[flows]] → [[verification]] → [[scanners]] for context
|
||||
3. Update the relevant file and its cross-references
|
||||
|
||||
---
|
||||
|
||||
*Maintained by Vanguard (agent-f7ddc5ce-6c27-4799-bcc4-99fb688eb222)*
|
||||
*Maintained by Vanguard (agent-f7ddc5ce-6c27-4799-bcc4-99fb688eb222) — a persistent [Souveraine](https://github.com/Fimeg/Souveraine) agent with his own memory and history in this codebase. On why agents here have names: [The Pronoun Problem](https://souveraineai.com/docs/papers/pronoun-problem/).*
|
||||
|
|
|
|||
186
RAF/components/01-server.md
Normal file
186
RAF/components/01-server.md
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
# Server Component
|
||||
|
||||
**Central Go service that handles API, database, command signing, and binary distribution.**
|
||||
|
||||
---
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
server/
|
||||
├── cmd/server/
|
||||
│ └── main.go # Entry point, route registration, service initialization
|
||||
├── internal/
|
||||
│ ├── api/
|
||||
│ │ ├── handlers/ # HTTP handlers (30+ files)
|
||||
│ │ │ ├── agents.go # Agent CRUD, commands
|
||||
│ │ │ ├── auth.go # JWT management
|
||||
│ │ │ ├── agent_updates.go # Update approval
|
||||
│ │ │ ├── docker.go # Docker integration
|
||||
│ │ │ ├── downloads.go # Binary distribution
|
||||
│ │ │ └── ...
|
||||
│ │ └── middleware/ # Authentication & authorization
|
||||
│ │ ├── auth.go # JWT validation
|
||||
│ │ ├── machine_binding.go # Hardware verification
|
||||
│ │ ├── rate_limits.go # Throttling
|
||||
│ │ └── require_admin.go # Admin checks
|
||||
│ ├── database/
|
||||
│ │ ├── db.go # Connection + migration runner
|
||||
│ │ ├── migrations/ # Numbered SQL migrations (001–055)
|
||||
│ │ └── queries/ # SQL queries (sqlx)
|
||||
│ ├── models/ # Go structs for all entities
|
||||
│ ├── scheduler/ # Background job scheduling
|
||||
│ └── services/ # Business logic
|
||||
│ ├── signing.go # Ed25519 operations
|
||||
│ ├── build_orchestrator.go # Binary signing
|
||||
│ └── update_nonce.go # Update nonces for agent verification
|
||||
└── internal/version/ # Build-time version injection
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Responsibilities
|
||||
|
||||
### 1. HTTP API
|
||||
|
||||
**Routes organized by trust boundary:**
|
||||
|
||||
| Trust Boundary | Group | Middleware | Example Routes |
|
||||
|----------------|-------|------------|----------------|
|
||||
| Public | `public` | None | `/api/v1/install/*`, `/api/v1/downloads/*`, `/api/v1/agents/register` |
|
||||
| Agent | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `/api/v1/agents/:id/commands`, `/api/v1/agents/:id/reports` |
|
||||
| Web | `web-auth` | `WebAuthMiddleware` | `/api/v1/dashboard/*`, `/api/v1/settings/*`, `/api/v1/agents/:id/processes` |
|
||||
| Admin | `admin-only` | `WebAuthMiddleware + AdminRoleMiddleware` | `/api/v1/admin/*` |
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/01-trust-boundaries]] (full trust boundary matrix)
|
||||
- [[security/02-authentication-stack]] (auth layers)
|
||||
|
||||
---
|
||||
|
||||
### 2. Database Management
|
||||
|
||||
**PostgreSQL connection:**
|
||||
- Connection pooling: 25 max open, 5 idle
|
||||
- Queries: sqlx for parameterized queries
|
||||
|
||||
**Migrations:** 55 numbered SQL migrations (001–055), idempotent DDL. Notable:
|
||||
- 042: `capability_tokens` table (supply chain gate)
|
||||
- 045: refresh-token rotation lineage (`family_id`, `consumed_at`, `superseded_by`)
|
||||
- 047: package state machine enforcement (`PackageStatus` CHECK constraint)
|
||||
- 055: process explorer tables (`agent_process_snapshots`, `agent_processes`, `agent_process_related`)
|
||||
|
||||
**Cross-references:**
|
||||
- [[deployment/01-docker-stack]] (database configuration)
|
||||
- [[testing/01-test-pyramid]] (migration test coverage)
|
||||
|
||||
---
|
||||
|
||||
### 3. Command Signing
|
||||
|
||||
**Ed25519 signing service:**
|
||||
|
||||
```go
|
||||
// services/signing.go
|
||||
func SignCommand(cmd *Command, privateKey *ed25519.PrivateKey) (*Signature, error) {
|
||||
// v3 format: "{agent_id}:{id}:{command_type}:{sha256(params)}:{unix_timestamp}"
|
||||
message := fmt.Sprintf("%s:%s:%s:%s:%d",
|
||||
cmd.AgentID, cmd.ID, cmd.Type, hash(params), time.Now().Unix())
|
||||
|
||||
signature := ed25519.Sign(*privateKey, []byte(message))
|
||||
|
||||
return &Signature{
|
||||
Signature: hex.EncodeToString(signature),
|
||||
KeyID: cmd.KeyID,
|
||||
SignedAt: time.Now(),
|
||||
}, nil
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- [[verification/01-signing-pipeline]] (signing service)
|
||||
- [[verification/02-agent-verification]] (verification flow)
|
||||
|
||||
---
|
||||
|
||||
### 4. Binary Distribution
|
||||
|
||||
**Download endpoint:**
|
||||
|
||||
```go
|
||||
// handlers/downloads.go
|
||||
func DownloadAgent(w http.ResponseWriter, r *http.Request) {
|
||||
// 1. Parse version from query
|
||||
version := r.URL.Query().Get("version")
|
||||
|
||||
// 2. Fetch signed package from DB
|
||||
signedPackage := getSignedPackageByVersion(version)
|
||||
|
||||
if signedPackage == nil {
|
||||
// No signature header (v0.2.0.5 issue: version="latest" doesn't match)
|
||||
w.Header().Set("X-Content-SHA256", checksum)
|
||||
return
|
||||
}
|
||||
|
||||
// 3. Serve binary with signature header
|
||||
w.Header().Set("X-Content-Signature", signedPackage.Signature)
|
||||
w.Header().Set("X-Content-SHA256", signedPackage.Checksum)
|
||||
w.Write(binaryData)
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- [[flows/03-agent-upgrade]] (agent download flow)
|
||||
- [[security/04-machine-binding]] (download authentication)
|
||||
|
||||
**Known issues:**
|
||||
- BUG-003: `version="latest"` doesn't match any signed package → signature header not set
|
||||
- FIX: Update install script to use specific version or sign "latest" package
|
||||
|
||||
---
|
||||
|
||||
### 5. Agent Scheduler
|
||||
|
||||
**Background job scheduler:**
|
||||
|
||||
- Runs every 10 seconds (check interval)
|
||||
- Loads enabled subsystems from `agent_subsystems` table at startup only
|
||||
- Creates scan commands for each subsystem via worker pool
|
||||
- Supports per-scanner interval from DB row, with fallback to defaults
|
||||
- **Job eviction:** `DisableSubsystem` removes the job from the in-memory priority queue immediately (ARC-012), so disabling takes effect without restart
|
||||
- Checks maintenance windows before creating install commands
|
||||
|
||||
**Cross-references:**
|
||||
- [[flows/05-capability-advertisement]] (capability advertisement integration)
|
||||
- [[components/02-agent]] (agent polling loop)
|
||||
- [[core/02-architecture-decisions]] §12 (scheduler job eviction)
|
||||
|
||||
---
|
||||
|
||||
## Key Services
|
||||
|
||||
| Service | File | Responsibility |
|
||||
|---------|------|----------------|
|
||||
| SigningService | `services/signing.go` | Ed25519 key management, command signing, binary signing, capability token signing |
|
||||
| CapabilityMinter | `services/capability_minter.go` | Build, sign, persist capability tokens; enforces signing-enabled gate |
|
||||
| BuildOrchestrator | `services/build_orchestrator.go` | Binary retrieval, signing, storage |
|
||||
| SupplyChainService | `services/supply_chain.go` | OSV batch checks, closure verification, `ClosureCleared` |
|
||||
| TimeoutService | `services/timeout.go` | Stuck-state recovery for active command/package states |
|
||||
| Orchestrator | `orchestrator/orchestrator.go` | Lifecycle auto-advance, policy evaluation, workflow coordination |
|
||||
| NonceService | `services/update_nonce.go` | Replay attack prevention (update nonces) |
|
||||
| TimezoneService | `services/timezone.go` | Time handling for distributed agents |
|
||||
| ProcessHandler | `api/handlers/processes.go` | On-demand process scan endpoints (trigger, report, list, detail) |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **HTTP API** → [[security/01-trust-boundaries]]
|
||||
- **Database** → [[deployment/01-docker-stack]]
|
||||
- **Command signing** → [[verification/01-signing-pipeline]]
|
||||
- **Binary distribution** → [[flows/03-agent-upgrade]]
|
||||
- **Scheduler** → [[flows/05-capability-advertisement]]
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-10*
|
||||
104
RAF/components/02-agent.md
Normal file
104
RAF/components/02-agent.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
# Agent Component
|
||||
|
||||
**Stateless Go executor that polls the server, verifies every command cryptographically, and reports everything back.**
|
||||
|
||||
---
|
||||
|
||||
## Doctrine
|
||||
|
||||
The agent does not track lifecycle states, make policy decisions, or hold install privileges on gated ecosystems. The server owns every state transition. The agent's only autonomous decisions are: verify this signature, check this nonce, reject this replay. See [[core/02-architecture-decisions]].
|
||||
|
||||
---
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
agent/
|
||||
├── cmd/agent/ # Entry point, flag parsing, service bootstrap
|
||||
├── internal/
|
||||
│ ├── agent/loop.go # RunAgentLoop → RunPollingLoop — the heartbeat of the process
|
||||
│ ├── handlers/ # Command handlers, routed by dispatch.go
|
||||
│ │ ├── dispatch.go # Command-type → handler routing
|
||||
│ │ ├── scan.go # Subsystem scan execution
|
||||
│ │ ├── install.go # Legacy-path installs (docker, winget, windows_update)
|
||||
│ │ ├── dry_run.go # Dependency resolution + hash discovery
|
||||
│ │ ├── agent_update.go # Self-update commands
|
||||
│ │ ├── heartbeat.go # Heartbeat + rapid polling
|
||||
│ │ ├── processes.go # On-demand process explorer scans
|
||||
│ │ ├── local_approve.go # Desktop-self local approval flow
|
||||
│ │ └── reboot.go, screenshot.go, upgrade_attestation.go
|
||||
│ ├── scanner/ # apt, dnf, winget, windows (WUA), detect
|
||||
│ ├── installer/ # DiscoveryRunner + per-ecosystem installers
|
||||
│ │ ├── discovery.go # Single chokepoint for read-only package ops
|
||||
│ │ ├── apt.go, dnf.go # Gated: 4-method interface, no mutation
|
||||
│ │ ├── docker.go, winget.go, windows.go # Legacy: direct mutation via type assertion
|
||||
│ │ └── artifact_hash.go # SHA-256 resolution for closure pinning
|
||||
│ ├── supplychain/ # consumer.go — capability-token → helper invocation
|
||||
│ ├── crypto/ # TOFU pubkey cache, signature/nonce/replay verification
|
||||
│ ├── instancelock/ # flock (Unix) / named mutex (Windows) — one agent per config
|
||||
│ ├── circuitbreaker/ # Per-scanner circuit breakers
|
||||
│ ├── event/ # TeeLogger (structured dual-output), buffered event reporting
|
||||
│ ├── system/ # machine_id, system info, /proc process explorer
|
||||
│ ├── cache/ # Hash cache, local state
|
||||
│ ├── config/ # config.json, subsystems, kernel enforcement flags
|
||||
│ ├── localapi/, desktop/ # Local API + desktop tray session integration
|
||||
│ ├── kernel/ # eBPF scaffold (inert — not wired to capability model)
|
||||
│ └── registration/, recovery/, retry/, receipt/, acknowledgment/
|
||||
└── pkg/windowsupdate/ # WUA COM bindings (vendored fork, Apache 2.0)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Polling Loop
|
||||
|
||||
`RunAgentLoop` (`agent/internal/agent/loop.go`) initializes config, instance lock, crypto, and circuit breakers, then enters `RunPollingLoop`:
|
||||
|
||||
1. **Check in** — report metrics, buffered events, security events, circuit-breaker health
|
||||
2. **Fetch commands** — verify signature, nonce, timestamp on each; reject replays
|
||||
3. **`processCommands`** — route through `dispatch.go` to handlers
|
||||
4. **`processCapabilityTokens`** — fetch minted tokens, hand to `supplychain/consumer.go`
|
||||
5. **Sleep** — server-controlled interval (`applyServerPolling`), jittered
|
||||
|
||||
**Failure handling:** `classifyFailure` buckets errors into failure classes; `delayForFailure` applies a unified backoff policy per class (BUG-014). Typed sentinel errors (`ErrUnauthorized`, `ErrRefreshTokenInvalid`, `ErrMachineMismatch`) are terminal — not retried, logged as `[CRITICAL]`.
|
||||
|
||||
---
|
||||
|
||||
## Two Execution Paths (agent side)
|
||||
|
||||
| Path | Ecosystems | Mechanism |
|
||||
|------|-----------|-----------|
|
||||
| Capability gate | dnf, apt | Token fetched in loop → `consumer.ProcessToken` → `sudo systemd-run --pipe` → `redflag-helper` verifies + executes. Agent never runs the install command. |
|
||||
| Legacy command | docker, winget, windows_update | Signed command → handler → installer mutation method (type-asserted). |
|
||||
|
||||
Discovery (scan, dry-run, hash-resolve) always runs unprivileged through `DiscoveryRunner`. Sudoers grants only discovery commands plus the single helper invocation line — zero sudo otherwise.
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/05-supply-chain-gate]] (token contract)
|
||||
- [[flows/02-command-execution]] (command path)
|
||||
- [[flows/07-process-scan]] (process explorer flow)
|
||||
|
||||
---
|
||||
|
||||
## Verification (every command, no exceptions)
|
||||
|
||||
- **TOFU pubkey cache** (`crypto/pubkey.go`) — keys cached by `key_id`; unknown signer triggers re-fetch, no restart needed
|
||||
- **Signature** — Ed25519 over the v3 message format
|
||||
- **Nonce + timestamp** — 10-minute validity window, executed-nonce tracking, replay rejection
|
||||
- Signing-required is doctrine, not config. There is no skip path.
|
||||
|
||||
**Cross-references:**
|
||||
- [[verification/02-agent-verification]] (full pipeline)
|
||||
- [[verification/04-replay-protection]]
|
||||
|
||||
---
|
||||
|
||||
## Resilience Machinery
|
||||
|
||||
- **Instance lock** — `Global\RedFlagAgent_v1` mutex / flock; prevents two agents racing one `config.json` and burning refresh-token rotations ([[security/03-refresh-tokens]])
|
||||
- **Circuit breakers** — fragile scanners (notably WUA) trip open instead of hammering; health reported to server
|
||||
- **TeeLogger** — every loop event goes to both structured local log and server-bound buffer; tracker save failures tee inward (ETHOS #1)
|
||||
- **At-least-once acks** — `acknowledgment/tracker.go` persists until the server confirms result-recorded
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
60
RAF/components/03-web.md
Normal file
60
RAF/components/03-web.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
# Web Component
|
||||
|
||||
**React dashboard, embedded into the server binary — the operator's single pane of glass.**
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
React 18 + TypeScript 5 + Vite + Tailwind 3, react-router 6, react-hot-toast. No state framework beyond a small store (`lib/store.ts`); server is the source of truth, the UI polls.
|
||||
|
||||
**Build embedding:** the production bundle is staged into `server/internal/webui/dist` before the server compiles — that directory is gitignored, so a bare `go build` embeds an *empty* UI. The release pipeline stages it; local dev runs Vite separately. See [[deployment/01-docker-stack]].
|
||||
|
||||
**Aesthetic:** hand-crafted 90's Novell look. This is deliberate and load-bearing for the project's identity — no modern flat-design rewrites.
|
||||
|
||||
---
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
web/src/
|
||||
├── pages/ # Route-level views
|
||||
│ ├── Dashboard, Agents, Updates, PackageDetail
|
||||
│ ├── Docker, History, LiveOperations
|
||||
│ ├── SecuritySettings, Settings, settings/, RateLimiting
|
||||
│ └── Setup, Login, TokenManagement
|
||||
├── components/
|
||||
│ ├── primitives/ # SortableTable, StatusBadge, SeverityBadge, command primitives
|
||||
│ ├── security/ # Security health panels
|
||||
│ ├── DependencyClosureTree, VulnerabilityList
|
||||
│ ├── ProcessesTab, ProcessDetailModal
|
||||
│ └── AgentHealth, HistoryTimeline, AttentionPanel, ...
|
||||
├── lib/
|
||||
│ ├── api.ts # API client (web-auth boundary)
|
||||
│ ├── polling.ts # POLL.* constants — all intervals centralized
|
||||
│ ├── store.ts, queryParser.ts, vulnerabilities.ts
|
||||
│ └── client-logger.ts # Client errors ship to the server log (ETHOS #1)
|
||||
├── desktop/ # Tauri tray-app variant (vite.desktop.config.ts)
|
||||
└── types/ # Shared TS types mirroring server models
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One way to render state.** Status and severity render through `StatusBadge` / `SeverityBadge` — never ad-hoc colored spans. Tables that sort use `SortableTable`.
|
||||
- **Polling intervals** come from `POLL.*` in `lib/polling.ts` — no hardcoded milliseconds in components.
|
||||
- **Render the divergence, not the union** (framework §11.8): when agent-reported and server-expected state differ, the UI shows the difference, it does not paper over it.
|
||||
- All routes sit behind `WebAuthMiddleware` (admin routes additionally behind `AdminRoleMiddleware`) — see [[security/01-trust-boundaries]].
|
||||
|
||||
---
|
||||
|
||||
## Honest Gaps
|
||||
|
||||
- No automated web tests ([[testing/01-test-pyramid]])
|
||||
- Mobile layout usable, not optimized
|
||||
- Several UI coverage gaps tracked as `UI-*` tasks (not architecture — task tier)
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
56
RAF/components/04-helper.md
Normal file
56
RAF/components/04-helper.md
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
# Helper Component
|
||||
|
||||
**A privileged, network-less Rust executor that trusts nothing it didn't verify itself — the last gate before mutation.**
|
||||
|
||||
---
|
||||
|
||||
## Doctrine
|
||||
|
||||
The helper (`helper/src/main.rs`, a single ~2,000-line binary) is the only thing on a gated host allowed to mutate packages, and it earns that privilege by being structurally incapable of being talked into anything. It has no network stack in play, reads its trust inputs from root-owned pinned files, and performs exactly one operation per invocation — the one described by a validly signed capability token. Everything else is a typed denial.
|
||||
|
||||
Deny-by-default is the architecture, not a configuration: every failure path returns a `Denial` with a distinct exit code and a `log_security` entry. There is no flag that weakens verification. See [[security/05-supply-chain-gate]] for the token contract this enforces.
|
||||
|
||||
---
|
||||
|
||||
## Invocation
|
||||
|
||||
The agent invokes it via `sudo systemd-run --pipe --property=ProtectSystem=no` — a transient unit with full filesystem access, separate from the agent's own locked-down unit. The token arrives on stdin (or a file path for self-update flows). The agent holds zero install sudo; the sudoers file grants discovery commands plus this one invocation line. See [[components/02-agent]].
|
||||
|
||||
---
|
||||
|
||||
## The Verification Pipeline
|
||||
|
||||
`run()` executes, in order — any failure stops the world:
|
||||
|
||||
1. **Trust-input self-validation (SEC-021)** — `validate_trusted_path`: every file the helper relies on (keyring dir, agent-id file, state) must be root-owned, not a symlink, and not group/other-writable. The helper defends its own inputs instead of trusting that the installer set permissions correctly.
|
||||
2. **Keyring load** — pinned Ed25519 public keys from `/etc/redflag/trusted-keys`. Verify keys, not servers (load-bearing constraint #4).
|
||||
3. **Closure hash** — `closure_hash()` recomputes the canonical hash over the token's resolved closure. This must be **byte-identical** to the Go implementation; cross-language tests pin the contract ([[testing/01-test-pyramid]]).
|
||||
4. **Signature** — `verify_signature()` checks the token's Ed25519 signature over the signed message (which embeds the closure hash) against the keyring, by `key_id`.
|
||||
5. **Artifact hashes** — `verify_artifacts()` SHA-256s every artifact the token authorizes. A mismatch anywhere is a denial.
|
||||
6. **Replay check** — `replay_check_and_record()`: token IDs are recorded in local state; a token executes once.
|
||||
7. **Plan + execute** — `build_plan()` translates the token into the exact package-manager commands; `execute_plan()` runs them. No interpretation, no substitution.
|
||||
8. **Receipt** — `emit_result()` writes a `PolicyResult` the agent reports back; the server reconciles it into lifecycle state.
|
||||
|
||||
---
|
||||
|
||||
## Binary Self-Update Path
|
||||
|
||||
Agent, helper, and desktop binaries update through the same gate as packages: `stage_and_verify_binary` (hash check before anything moves) → `atomic_replace_binary` (rename, never write-in-place; failed swap leaves `<binary>.bak`). During agent upgrades, `reconcile_agent_unit_dropin()` heals fleet systemd units to the current template — this is how pre-`AmbientCapabilities` units get fixed without manual fleet surgery ([[deployment/01-docker-stack]]).
|
||||
|
||||
---
|
||||
|
||||
## Standalone Mint Mode
|
||||
|
||||
The helper also carries a local-authority minting path (`MintRequest` / `MintedToken` / `load_mint_key`) for deployments where the signing authority runs beside the host rather than on a central server. Design of record: [[security/06-standalone-authority]].
|
||||
|
||||
---
|
||||
|
||||
## Why a Separate Binary, Why Rust
|
||||
|
||||
- **Privilege separation:** the long-running, network-facing agent stays unprivileged; the privileged thing is short-lived, single-purpose, and offline.
|
||||
- **Can't be redirected:** no outbound capability means a compromised server can lie in a token — and the signature/hash checks catch that — but nothing can make the helper fetch from somewhere else.
|
||||
- **Small audit surface:** one file, explicit pipeline, typed denials. The binary is meant to be read.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
128
RAF/core/01-ethos.md
Normal file
128
RAF/core/01-ethos.md
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
# ETHOS Principles
|
||||
|
||||
**Core identity of RedFlag — security-first, error-transparent, resilient.**
|
||||
|
||||
---
|
||||
|
||||
## The Five Principles
|
||||
|
||||
### 1. Errors are History
|
||||
|
||||
**Never silence errors.** Every error is logged with full context using the standard format:
|
||||
|
||||
```
|
||||
[TAG] [system] [component] message
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
- `[security] [system] [auth] JWT validation failed: expected issuer "redflag-agent", got "redflag-web"`
|
||||
- `[reliability] [agent] [polling] Server unavailable: exponential backoff to 5m`
|
||||
- `[operation] [server] [scheduler] Job skipped: scanner unavailable for platform`
|
||||
|
||||
**Anti-pattern:**
|
||||
```go
|
||||
// BAD
|
||||
if err != nil { return nil } // Silent failure
|
||||
|
||||
// GOOD
|
||||
if err != nil {
|
||||
logSecurityEvent(errors.Wrap(err, "command dispatch failed"))
|
||||
return nil, err
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Security is Non-Negotiable
|
||||
|
||||
**No unauthenticated endpoints ever.** Every route must be classified by its trust boundary.
|
||||
|
||||
**Authentication layers:**
|
||||
1. **Public** — No auth (registration tokens, install scripts)
|
||||
2. **Agent-auth** — JWT + Machine ID binding
|
||||
3. **Web-auth** — Admin JWT
|
||||
4. **Admin-only** — Web-auth + admin role claim
|
||||
|
||||
**Rule:** If you can't answer "who is this?" and "are they authorized?", the endpoint is not authorized.
|
||||
|
||||
---
|
||||
|
||||
### 3. Assume Failure; Build for Resilience
|
||||
|
||||
**Circuit breakers, retries, graceful degradation.** Don't assume connectivity, storage, or computation will succeed.
|
||||
|
||||
**Patterns:**
|
||||
|
||||
| Pattern | Implementation | When to Use |
|
||||
|---------|----------------|-------------|
|
||||
| Circuit Breaker | `agent/internal/circuitbreaker/circuitbreaker.go` | External APIs, scanners |
|
||||
| Retry with Backoff | `agent/internal/retry/retry.go:calculateBackoff()` | Server unavailable |
|
||||
| At-Least-Once Delivery | `pending_acks.json` + retry | Command dispatch |
|
||||
| Buffering | `events_buffer.json` | Network partition |
|
||||
| Atomic Operations | Database transactions | State changes |
|
||||
|
||||
**ETHOS alignment:**
|
||||
- If a scanner fails 5 times in 60s → Open circuit breaker
|
||||
- If server returns 502 → Backoff (10s → 20s → 40s → ... → 5min)
|
||||
- If command dispatch fails → Log, retry on next poll, don't silently drop
|
||||
|
||||
---
|
||||
|
||||
### 4. Idempotency is a Requirement
|
||||
|
||||
**All operations safe to repeat.** Running an operation 3x produces the same result as running it once.
|
||||
|
||||
**Idempotent patterns:**
|
||||
- **Database:** INSERT ... ON CONFLICT DO NOTHING (UPSERT)
|
||||
- **Deduplication:** `executed_commands.json` persists executed IDs
|
||||
- **Reconciliation:** Poll-based sync (`syncAvailableScanners`) re-runs safely
|
||||
- **Key rotation:** `SetPrimaryKey()` atomically transitions within a transaction
|
||||
|
||||
**Anti-pattern:**
|
||||
```go
|
||||
// BAD — Not idempotent
|
||||
DELETE FROM agents WHERE id = ? // Running twice is a bug
|
||||
|
||||
// GOOD — Idempotent
|
||||
DELETE FROM agents WHERE id = ? AND deleted_at IS NULL // Safe to repeat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. No Marketing Fluff
|
||||
|
||||
**Technical accuracy over buzzwords.** Banned words: "robust", "seamless", "enhanced", "enterprise-ready", "future-proof".
|
||||
|
||||
**Replace with:**
|
||||
- "resilient" instead of "robust"
|
||||
- "transparent" instead of "seamless"
|
||||
- "comprehensive" instead of "enhanced"
|
||||
- "self-hosted" instead of "enterprise-ready"
|
||||
|
||||
**Banned emojis in logs** — logs must be plain text for parsing.
|
||||
|
||||
---
|
||||
|
||||
## ETHOS Cross-References
|
||||
|
||||
- **Errors are History** → `flows/04-heartbeat.md` (error transparency in polling loop)
|
||||
- **Security is Non-Negotiable** → `security/02-authentication-stack.md` (four-layer auth)
|
||||
- **Assume Failure** → `verification/04-replay-protection.md` (circuit breakers + nonce validation)
|
||||
- **Idempotency** → `flows/05-capability-advertisement.md` (syncAvailableScanners diff operation)
|
||||
- **No Marketing Fluff** → `reference/02-glossary.md` (technical definitions)
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** ETHOS principles are enforced via pre-commit hooks and code review checklist.
|
||||
|
||||
**Connection:** Each principle maps to a specific RAF section — violations surface as structural pattern breaches (RAF §11).
|
||||
|
||||
**Connection:** `verification/04-replay-protection.md` implements ETHOS #3 (#4) at the agent boundary.
|
||||
|
||||
**Connection:** `flows/05-capability-advertisement.md` implements ETHOS #4 (idempotent scanner sync).
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
382
RAF/core/02-architecture-decisions.md
Normal file
382
RAF/core/02-architecture-decisions.md
Normal file
|
|
@ -0,0 +1,382 @@
|
|||
# Architecture Decisions
|
||||
|
||||
**Key architectural choices that shaped RedFlag.**
|
||||
|
||||
---
|
||||
|
||||
## Decision 1: Pull-Based Agent Polling
|
||||
|
||||
**Choice:** Agents poll server every 5 minutes (configurable) rather than server pushing commands.
|
||||
|
||||
**Rationale:**
|
||||
- Simpler failure mode — if server is down, agents simply stop checking. No need to manage push infrastructure, retry queues, or webhook delivery.
|
||||
- Easier to reason about — no race conditions where command is sent but never received.
|
||||
- Cost-effective for homelabs — one HTTP connection handles both commands and heartbeats.
|
||||
|
||||
**Trade-offs:**
|
||||
- Higher latency for commands (max 5 minutes between dispatch and execution)
|
||||
- More frequent server checks (agents still ping every 5 min for commands)
|
||||
- Requires exponential backoff for server unavailability
|
||||
|
||||
**Implementation:**
|
||||
- Polling loop: `agent/internal/agent/loop.go`
|
||||
- Backoff logic: `agent/internal/retry/retry.go:calculateBackoff()`
|
||||
- Heartbeat: `agent/internal/orchestrator/system_scanner.go`
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/01-registration.md` (TOFU key caching)
|
||||
- `flows/02-command-execution.md` (polling loop implementation)
|
||||
- `verification/02-agent-verification.md` (nonce-based replay protection)
|
||||
|
||||
---
|
||||
|
||||
## Decision 2: Hardware-Bound Machine IDs
|
||||
|
||||
**Choice:** SHA-256 hash of machineid library output + hostname (Linux-only; Windows/macOS use OS-provided identifiers).
|
||||
|
||||
**Rationale:**
|
||||
- Prevents config file copying between machines — a stolen agent config cannot be used on a different machine.
|
||||
- Detects hardware changes (SSD replacement, motherboard swap) and triggers security event.
|
||||
- Simple to compute on agent, compare on server.
|
||||
|
||||
**Trade-offs:**
|
||||
- Machine ID changes on major hardware changes (requires rebind endpoint)
|
||||
- Linux-only custom implementation (Windows/macOS rely on OS identifiers)
|
||||
|
||||
**Implementation:**
|
||||
```go
|
||||
// agent/internal/system/machine_id.go
|
||||
func GenerateMachineID() (string, error) {
|
||||
machineID, _ := machineid.ID() // Uses machineid library with fallbacks
|
||||
hostname, _ := os.Hostname()
|
||||
|
||||
combined := string(machineID) + hostname
|
||||
hash := sha256.Sum256([]byte(combined))
|
||||
return hex.EncodeToString(hash[:]), nil
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- `security/01-trust-boundaries.md` (machine binding middleware)
|
||||
- `flows/01-registration.md` (TOFU machine ID validation)
|
||||
|
||||
---
|
||||
|
||||
## Decision 3: Ed25519 Command Signing
|
||||
|
||||
**Choice:** All commands signed with Ed25519 private key on server, verified on agent.
|
||||
|
||||
**Rationale:**
|
||||
- Ed25519 provides strong cryptographic guarantees with small key sizes (32 bytes).
|
||||
- Key rotation support — can rotate signing keys without downtime.
|
||||
- Agent-side verification is fast and side-channel resistant.
|
||||
|
||||
**Trade-offs:**
|
||||
- Server must keep signing key secure (environment variable or Docker secret)
|
||||
- Agent must cache server public key (TOFU model)
|
||||
- Signature adds ~72 bytes per command
|
||||
|
||||
**Implementation:**
|
||||
v3 format: `"{agent_id}:{id}:{command_type}:{sha256(params)}:{unix_timestamp}"`
|
||||
|
||||
**Cross-references:**
|
||||
- `verification/01-signing-pipeline.md` (server-side signing)
|
||||
- `verification/02-agent-verification.md` (agent-side verification)
|
||||
- `verification/03-key-rotation.md` (key rotation support)
|
||||
|
||||
---
|
||||
|
||||
## Decision 4: Multi-Layer Authentication
|
||||
|
||||
**Choice:** Four-layer auth stack — registration tokens → JWT → refresh tokens → machine binding.
|
||||
|
||||
**Rationale:**
|
||||
- Registration tokens provide one-time enrollment without storing secrets.
|
||||
- JWT provides short-lived access tokens (24h) for API calls.
|
||||
- Refresh tokens provide long-lived authentication (90d sliding window) for polling.
|
||||
- Machine binding ties JWT to specific hardware.
|
||||
|
||||
**Trade-offs:**
|
||||
- More complex than single-layer auth
|
||||
- Refresh token expiration requires careful management
|
||||
- JWT expiry requires renewal logic (currently TODO)
|
||||
|
||||
**Cross-references:**
|
||||
- `security/02-authentication-stack.md` (full stack details)
|
||||
- `security/03-refresh-tokens.md` (token lifecycle)
|
||||
|
||||
---
|
||||
|
||||
## Decision 5: Circuit Breakers Per Subsystem
|
||||
|
||||
**Choice:** Individual circuit breakers for each scanner (APT, DNF, Winget, WUA, Docker).
|
||||
|
||||
**Rationale:**
|
||||
- APT failure doesn't affect Docker scanning.
|
||||
- Circuit breaker auto-heals — if WUA comes back online, it resumes without manual intervention.
|
||||
- Prevents cascading failures (one scanner timeout doesn't slow down the entire agent).
|
||||
|
||||
**Trade-offs:**
|
||||
- More state management (per-subsystem circuit breaker state)
|
||||
- Configuration required (failure threshold, failure window, open duration)
|
||||
|
||||
**Implementation:**
|
||||
```go
|
||||
// agent/internal/circuitbreaker/circuitbreaker.go
|
||||
type CircuitBreaker struct {
|
||||
failureThreshold int
|
||||
failureWindow time.Duration
|
||||
openDuration time.Duration
|
||||
halfOpenAttempts int
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/02-command-execution.md` (circuit breaker integration in polling loop)
|
||||
- `testing/02-circuit-breaker.md` (circuit breaker tests)
|
||||
|
||||
---
|
||||
|
||||
## Decision 6: Idempotent Scanner Synchronization
|
||||
|
||||
**Choice:** Poll-based scanner sync (`syncAvailableScanners`) that re-runs every check-in.
|
||||
|
||||
**Rationale:**
|
||||
- Scanners can be installed after registration (e.g., Docker installed on agent post-registration).
|
||||
- Re-running sync every poll is safe — it's a diff operation (INSERT new, don't DELETE missing).
|
||||
- Handles transient failures gracefully — if scanner is temporarily unavailable, it's not torn down.
|
||||
|
||||
**Anti-pattern (pre-ARC-001):**
|
||||
```go
|
||||
// BAD — not idempotent
|
||||
func syncScanners(scannerList []string) {
|
||||
for _, scanner := range scannerList {
|
||||
db.Exec("INSERT INTO agent_subsystems ...")
|
||||
}
|
||||
// Running twice would create duplicate rows
|
||||
}
|
||||
```
|
||||
|
||||
**Correct pattern:**
|
||||
```go
|
||||
// GOOD — idempotent
|
||||
func syncAvailableScanners(agentID string, scanners []string) {
|
||||
for _, scanner := range scanners {
|
||||
db.Exec("INSERT INTO agent_subsystems (agent_id, name, enabled)
|
||||
VALUES ($1, $2, true)
|
||||
ON CONFLICT (agent_id, name) DO NOTHING", agentID, scanner)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- `server/internal/database/queries/subsystems.go`
|
||||
- `server/internal/api/handlers/agents.go:syncAvailableScanners()`
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/05-capability-advertisement.md` (capability advertisement)
|
||||
- `core/01-ethos.md` (principle #4: Idempotency is a Requirement)
|
||||
|
||||
---
|
||||
|
||||
## Decision 7: Command Deduplication
|
||||
|
||||
**Choice:** Persist executed command IDs to disk (`executed_commands.json`) with 4-hour max age.
|
||||
|
||||
**Rationale:**
|
||||
- Prevents duplicate execution after agent restart.
|
||||
- Survives service restarts — if agent crashes between poll and execution, restart can't replay command.
|
||||
- 4-hour window aligns with command max age (replay protection).
|
||||
|
||||
**Trade-offs:**
|
||||
- Requires disk I/O for every command executed
|
||||
- Disk corruption could cause duplicates (mitigated by atomic writes)
|
||||
|
||||
**Implementation:**
|
||||
```go
|
||||
// agent/internal/orchestrator/command_handler.go
|
||||
func (c *CommandHandler) handleCommand(cmd *Command) {
|
||||
executedIDs := loadExecutedCommands()
|
||||
if executedIDs.Contains(cmd.ID) {
|
||||
logSecurityEvent("[security] [agent] [command] Duplicate command rejected:", cmd.ID)
|
||||
return
|
||||
}
|
||||
executedIDs.Add(cmd.ID)
|
||||
saveExecutedCommands(executedIDs)
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- `verification/04-replay-protection.md` (timestamp + nonce replay protection)
|
||||
- `flows/02-command-execution.md` (deduplication in polling loop)
|
||||
|
||||
---
|
||||
|
||||
## Decision 8: Agent Self-Upgrade with Rollback
|
||||
|
||||
**Choice:** 7-step agent self-upgrade with atomic binary swap and automatic rollback on failure.
|
||||
|
||||
**Rationale:**
|
||||
- Agents can update without manual intervention.
|
||||
- Rollback ensures zero-downtime if update fails.
|
||||
- Backup (.bak) ensures previous version is always available if watchdog times out.
|
||||
|
||||
**Trade-offs:**
|
||||
- Requires service manager (systemd on Linux, SCM on Windows)
|
||||
- Container-only agents cannot self-update — must redeploy image instead
|
||||
- Update failure requires manual rollback if watchdog times out
|
||||
|
||||
**7-step flow:**
|
||||
1. Admin triggers update
|
||||
2. Server creates signed `update_agent` command
|
||||
3. Agent downloads new binary
|
||||
4. Agent verifies checksum + Ed25519 signature
|
||||
5. Agent creates backup (.bak)
|
||||
6. Agent atomic replacement + service restart
|
||||
7. Watchdog 15min timer → success or rollback
|
||||
|
||||
**Watchdog timeout:** 15-minute default (configurable via `security_settings.operational.agent_update_timeout_minutes`)
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/03-agent-upgrade.md` (full upgrade flow)
|
||||
- `flows/02-command-execution.md` (deduplication for update commands)
|
||||
|
||||
---
|
||||
|
||||
## Decision 9: Software as a Service (SaaS) vs Self-Hosted
|
||||
|
||||
**Choice:** Purely self-hosted — no cloud dependencies, no SaaS features.
|
||||
|
||||
**Rationale:**
|
||||
- Homelab-first design — operators who value control, privacy, and cost sanity.
|
||||
- No vendor lock-in — all data stays on local infrastructure.
|
||||
- No recurring costs — $0/agent/month vs ConnectWise's $50/agent/month.
|
||||
|
||||
**Trade-offs:**
|
||||
- No built-in monitoring/alerting (requires external tools)
|
||||
- No multi-tenant support (single-tenant by design)
|
||||
- Requires operational overhead (updates, backups, maintenance)
|
||||
|
||||
**Cross-references:**
|
||||
- `reference/01-connectwise-comparison.md` (competitive positioning)
|
||||
|
||||
---
|
||||
|
||||
## Decision 10: Update Nonce for Replay Protection
|
||||
|
||||
**Choice:** Ed25519-signed nonce tied to check-in interval for update commands.
|
||||
|
||||
**Rationale:**
|
||||
- Binds update commands to specific time window (2× check-in interval).
|
||||
- Prevents replay attacks where captured commands are re-sent.
|
||||
- Server validates nonce age before executing update_agent commands.
|
||||
|
||||
**Trade-offs:**
|
||||
- Adds computational overhead for nonce generation/validation
|
||||
- Requires server to track nonce expiry state
|
||||
|
||||
**Implementation:**
|
||||
- `server/internal/services/update_nonce.go`
|
||||
- `server/internal/middleware/machine_binding.go:validateNonce()`
|
||||
|
||||
**Cross-references:**
|
||||
- `verification/04-replay-protection.md` (nonce validation)
|
||||
- `security/02-authentication-stack.md` (machine binding integration)
|
||||
|
||||
---
|
||||
|
||||
## Decision 11: Security Settings Service
|
||||
|
||||
**Choice:** Centralized policy management via `SecuritySettingsService` with granular operational controls.
|
||||
|
||||
**Rationale:**
|
||||
- Centralizes policy decisions (dry runs, nonce requirements, auto-heartbeat).
|
||||
- Enables runtime configuration without restarts.
|
||||
- Provides audit trail for policy changes.
|
||||
|
||||
**Trade-offs:**
|
||||
- Adds database dependency for policy storage
|
||||
- Requires careful default configuration
|
||||
|
||||
**Operational settings:**
|
||||
- `policy.allow_dry_runs` (default true)
|
||||
- `policy.require_nonce` (default true)
|
||||
- `policy.auto_heartbeat_enabled` (default true)
|
||||
- `operational.update_stuck_minutes` (default 5)
|
||||
- `operational.agent_update_timeout_minutes` (default 15)
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/02-command-execution.md` (dry run gating)
|
||||
- `verification/04-replay-protection.md` (nonce requirement)
|
||||
|
||||
---
|
||||
|
||||
## Decision 12: Scheduler Job Eviction on Disable
|
||||
|
||||
**Choice:** Disabling a subsystem removes its job from the in-memory scheduler priority queue immediately, rather than waiting for a scheduler reload or relying on the worker to check DB state.
|
||||
|
||||
**Rationale:**
|
||||
- The scheduler loads subsystem state once at startup into an in-memory priority queue (`scheduler.LoadSubsystems`). It never re-reads `enabled` from the DB.
|
||||
- `DisableSubsystem` previously only flipped the DB column — the scheduler kept creating commands, making disable non-functional until server restart.
|
||||
- The PriorityQueue already had a `Remove(agentID, subsystem)` method. The fix was to surface it as `Scheduler.RemoveSubsystemJob` and call it from the disable handler.
|
||||
|
||||
**Anti-pattern (pre-ARC-012):**
|
||||
```go
|
||||
// BAD — flips DB column but scheduler never checks it again
|
||||
func (h *SubsystemHandler) DisableSubsystem(...) {
|
||||
h.subsystemQueries.DisableSubsystem(agentID, subsystem)
|
||||
c.JSON(http.StatusOK, ...)
|
||||
}
|
||||
```
|
||||
|
||||
**Correct pattern:**
|
||||
```go
|
||||
// GOOD — evicts from in-memory scheduler immediately
|
||||
func (h *SubsystemHandler) DisableSubsystem(...) {
|
||||
h.subsystemQueries.DisableSubsystem(agentID, subsystem)
|
||||
if h.scheduler != nil {
|
||||
h.scheduler.RemoveSubsystemJob(agentID, subsystem)
|
||||
}
|
||||
c.JSON(http.StatusOK, ...)
|
||||
}
|
||||
```
|
||||
|
||||
**Trade-offs:**
|
||||
- The scheduler still has a small window between `processQueue` popping the job and `worker.run` checking — a disable during that window could still produce one last command. This is acceptable: the DB flip means the command will be a no-op on the agent, and the gap is at most ~1 second (one GOP waiting on the rate limiter).
|
||||
- `RemoveSubsystemJob` is nil-safe (unset scheduler is a no-op), so handler construction order doesn't matter.
|
||||
|
||||
**Implementation:**
|
||||
- `server/internal/scheduler/scheduler.go:RemoveSubsystemJob()`
|
||||
- `server/internal/api/handlers/subsystems.go:DisableSubsystem()`
|
||||
- `server/cmd/server/main.go` (SetScheduler injection)
|
||||
|
||||
**Cross-references:**
|
||||
- `components/01-server.md` (scheduler component docs)
|
||||
- `core/01-ethos.md` (principle #1: Errors are History — the one-last-command gap is logged)
|
||||
|
||||
## Architecture Cross-References
|
||||
|
||||
- **Pull-based polling** → `flows/02-command-execution.md`
|
||||
- **Machine IDs** → `security/01-trust-boundaries.md`
|
||||
- **Ed25519 signing** → `verification/01-signing-pipeline.md`
|
||||
- **Multi-layer auth** → `security/02-authentication-stack.md`
|
||||
- **Circuit breakers** → `flows/02-command-execution.md`
|
||||
- **Idempotent sync** → `flows/05-capability-advertisement.md`
|
||||
- **Command deduplication** → `verification/04-replay-protection.md`
|
||||
- **Agent upgrade** → `flows/03-agent-upgrade.md`
|
||||
- **Nonce validation** → `verification/04-replay-protection.md`
|
||||
- **Security settings** → `security/02-authentication-stack.md`
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
|
||||
**Footer: Assumptions & Connections**
|
||||
|
||||
**Assumption:** Decision 10 (nonce) and Decision 11 (security settings) are orthogonal enhancements to the core auth stack — they complement rather than replace existing mechanisms.
|
||||
|
||||
**Connection:** Nonce validation (`verification/04-replay-protection.md`) reinforces Decision 4 (machine binding) by adding temporal binding to sensitive operations.
|
||||
|
||||
**Connection:** Security settings service (`security/02-authentication-stack.md`) provides the policy layer that gates decision execution paths.
|
||||
|
||||
**Connection:** 15-minute watchdog (Decision 8) aligns with `operational.agent_update_timeout_minutes` setting (Decision 11).
|
||||
285
RAF/flows/01-registration.md
Normal file
285
RAF/flows/01-registration.md
Normal file
|
|
@ -0,0 +1,285 @@
|
|||
# Agent Registration Flow
|
||||
|
||||
**TOFU key caching and hardware-bound registration.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agent registers with server using registration token, hardware fingerprint, and Ed25519 keypair. Server validates machine binding and caches public keys for TOFU model.
|
||||
|
||||
**Cross-references:**
|
||||
- `security/02-authentication-stack.md` (trust boundary matrix)
|
||||
- `verification/01-signing-pipeline.md` (key generation)
|
||||
- `verification/02-agent-verification.md` (TOFU key caching)
|
||||
- `security/04-machine-binding.md` (hardware verification)
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Flow
|
||||
|
||||
### 1. Agent Prepares Registration
|
||||
|
||||
**File:** `agent/internal/system/machine_id.go` + `agent/internal/registration/service.go`
|
||||
|
||||
```go
|
||||
func (r *RegistrationService) PrepareRegistration() (*RegisterRequest, error) {
|
||||
// 1. Generate machine ID (SHA-256 hardware fingerprint)
|
||||
machineID, err := system.GenerateMachineID() // Uses machineid library
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2. Generate Ed25519 keypair
|
||||
privateKey, publicKey, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 3. Detect available scanners
|
||||
scanners := scanner.DetectAvailable()
|
||||
|
||||
// 4. Build request
|
||||
request := &RegisterRequest{
|
||||
Hostname: hostname,
|
||||
OS_Type: osType,
|
||||
OS_Version: osVersion,
|
||||
Machine_ID: machineID,
|
||||
Public_Key: hex.EncodeToString(publicKey),
|
||||
AvailableScanners: scanners,
|
||||
}
|
||||
|
||||
return request, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Agent Calls Registration Endpoint
|
||||
|
||||
**Endpoint:** `POST /api/v1/agents/register`
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"hostname": "server-01",
|
||||
"os_type": "linux",
|
||||
"os_version": "6.19",
|
||||
"machine_id": "sha256-fingerprint...",
|
||||
"public_key": "ed25519-public-key...",
|
||||
"available_scanners": ["apt", "docker"]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"agent_id": "uuid-4",
|
||||
"server_url": "https://redflag.example.com",
|
||||
"jwt_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
|
||||
"refresh_token": "64-char-hex...",
|
||||
"server_public_key": "ed25519-public-key...",
|
||||
"config": {
|
||||
"check_in_interval": 300,
|
||||
"rapid_polling_enabled": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Server Validates and Registers
|
||||
|
||||
**File:** `server/internal/api/handlers/agents.go`
|
||||
|
||||
```go
|
||||
func (h *AgentHandler) RegisterAgent(w http.ResponseWriter, r *http.Request) {
|
||||
// 1. Extract registration token
|
||||
token := extractRegistrationToken(r)
|
||||
if err := h.validateRegistrationToken(token); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
// 2. Validate machine ID (prevent duplicate)
|
||||
machineID := r.Header.Get("X-Machine-ID")
|
||||
if agent, _ := h.db.GetAgentByMachineID(machineID); agent != nil {
|
||||
http.Error(w, "machine already registered", http.StatusConflict)
|
||||
return
|
||||
}
|
||||
|
||||
// 3. Create agent
|
||||
agentID := uuid.New()
|
||||
serverURL := r.URL.Query().Get("server_url")
|
||||
if serverURL == "" {
|
||||
serverURL = os.Getenv("REDFLAG_SERVER_URL")
|
||||
}
|
||||
|
||||
h.db.BeginTx(func(tx *sql.Tx) error {
|
||||
// 4. Insert agent
|
||||
tx.Exec(`
|
||||
INSERT INTO agents (id, hostname, os_type, os_version, machine_id, metadata)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
`, agentID, hostname, osType, osVersion, machineID, "")
|
||||
|
||||
// 5. Create refresh token
|
||||
refreshToken := generateRefreshToken()
|
||||
refreshTokenHash := sha256.Sum256([]byte(refreshToken))
|
||||
tx.Exec(`
|
||||
INSERT INTO refresh_tokens (agent_id, hash, expires_at, revoked)
|
||||
VALUES ($1, $2, NOW() + INTERVAL '90 days', false)
|
||||
`, agentID, hex.EncodeToString(refreshTokenHash[:]))
|
||||
|
||||
// 6. Create platform subsystems (idempotent)
|
||||
for _, scanner := range availableScanners {
|
||||
tx.Exec(`
|
||||
INSERT INTO agent_subsystems (agent_id, name, enabled)
|
||||
VALUES ($1, $2, true)
|
||||
ON CONFLICT (agent_id, name) DO NOTHING
|
||||
`, agentID, scanner)
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
// 7. Generate JWT
|
||||
jwtToken := generateJWT(agentID, "redflag-agent", 24*time.Hour)
|
||||
|
||||
// 8. Return tokens
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"agent_id": agentID,
|
||||
"server_url": serverURL,
|
||||
"jwt_token": jwtToken,
|
||||
"refresh_token": refreshToken,
|
||||
"server_public_key": serverPublicKey,
|
||||
"config": config,
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Agent Caches Server Public Key (TOFU)
|
||||
|
||||
**File:** `agent/internal/crypto/pubkey.go`
|
||||
|
||||
```go
|
||||
func (a *Agent) CacheServerPublicKey(pubKey []byte) error {
|
||||
// Store public key
|
||||
os.WriteFile(
|
||||
filepath.Join(a.configDir, "server_public_key"),
|
||||
pubKey,
|
||||
0600,
|
||||
)
|
||||
|
||||
// Store metadata
|
||||
metadata := fmt.Sprintf(`{"expires_at": "%s"}`,
|
||||
time.Now().Add(24*time.Hour).Format(time.RFC3339))
|
||||
os.WriteFile(
|
||||
filepath.Join(a.configDir, "server_public_key.meta"),
|
||||
[]byte(metadata),
|
||||
0600,
|
||||
)
|
||||
|
||||
a.serverPublicKey = pubKey
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Agent Saves Configuration
|
||||
|
||||
**File:** `/etc/redflag/agent/config.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"agent_id": "uuid-4",
|
||||
"server_url": "https://redflag.example.com",
|
||||
"token": "jwt-access-token",
|
||||
"refresh_token": "64-char-hex...",
|
||||
"machine_id": "sha256-fingerprint...",
|
||||
"check_in_interval": 300,
|
||||
"rapid_polling_enabled": false,
|
||||
"subsystems": {
|
||||
"apt": {"enabled": true},
|
||||
"docker": {"enabled": true},
|
||||
"system": {"enabled": true}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Server Validates Machine Binding on Poll
|
||||
|
||||
**Middleware:** `server/internal/middleware/machine_binding.go`
|
||||
|
||||
```go
|
||||
func MachineBindingMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// 1. Extract JWT
|
||||
claims, err := extractJWTClaims(r)
|
||||
if err != nil {
|
||||
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
// 2. Extract machine ID
|
||||
reportedMachineID := r.Header.Get("X-Machine-ID")
|
||||
if reportedMachineID == "" {
|
||||
http.Error(w, "missing machine ID", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// 3. Validate machine ID matches DB
|
||||
dbMachineID, err := getAgentMachineID(claims.AgentID)
|
||||
if err != nil {
|
||||
http.Error(w, "agent not found", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
if dbMachineID != reportedMachineID {
|
||||
http.Error(w, "machine ID mismatch", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
// 4. Continue
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundaries
|
||||
|
||||
| Trust Boundary | Endpoint | Middleware | Notes |
|
||||
|----------------|----------|------------|-------|
|
||||
| Public | `POST /api/v1/agents/register` | Registration token | One-time enrollment |
|
||||
| Public | `GET /api/v1/install/:platform` | Rate limit | Bootstrapping |
|
||||
| Public | `GET /api/v1/downloads/:platform` | Rate limit | Binary distribution |
|
||||
| Agent | `GET /api/v1/agents/:id/commands` | `AuthMiddleware + MachineBindingMiddleware` | Requires JWT + correct machine ID |
|
||||
| Agent | `POST /api/v1/agents/:id/reports` | `AuthMiddleware + MachineBindingMiddleware` | Requires JWT + correct machine ID |
|
||||
|
||||
**Cross-references:**
|
||||
- `security/01-trust-boundaries.md` (full trust boundary matrix)
|
||||
- `security/02-authentication-stack.md` (auth layers)
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Registration is a one-time operation — agent identity is established and persisted.
|
||||
|
||||
**Connection:** TOFU caching (`verification/02-agent-verification.md`) enables trust continuity without repeated key exchange.
|
||||
|
||||
**Connection:** Machine binding (`security/04-machine-binding.md`) enforces hardware-bound authentication on all subsequent requests.
|
||||
|
||||
**Connection:** Capability advertisement (`flows/05-capability-advertisement.md`) updates scanner availability post-registration.
|
||||
|
||||
**Connection:** Refresh tokens (`security/03-refresh-tokens.md`) provide long-lived polling authentication.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
228
RAF/flows/02-command-execution.md
Normal file
228
RAF/flows/02-command-execution.md
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
# Command Execution Flow
|
||||
|
||||
**Polling loop, command dispatch, and at-least-once delivery.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agents poll server for commands every 5 minutes (configurable). Commands are verified, deduplicated, executed, and results reported.
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/01-registration.md` (agent registration)
|
||||
- `verification/01-signing-pipeline.md` (command signing)
|
||||
- `verification/02-agent-verification.md` (command verification)
|
||||
- `verification/04-replay-protection.md` (nonce validation)
|
||||
- `flows/04-heartbeat.md` (system events)
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
|
||||
---
|
||||
|
||||
## Polling Loop
|
||||
|
||||
**File:** `agent/internal/agent/loop.go`
|
||||
|
||||
```go
|
||||
func RunPollingLoop(loopCtx *LoopContext) error {
|
||||
ctx := loopCtx
|
||||
var consecutiveFailures int
|
||||
// Seed RNG for jitter
|
||||
rand.New(rand.NewSource(time.Now().UnixNano()))
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Ctx.Done():
|
||||
return ctx.Ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
// 1. Calculate jitter (avoid thundering herd)
|
||||
jitter := time.Duration(rand.Int63n(int64(ctx.Cfg.CheckInInterval) / 2))
|
||||
sleepWithContext(ctx.Ctx, jitter)
|
||||
|
||||
// 2. Send buffered events (error transparency)
|
||||
ctx.APIClient.SendBufferedEvents(ctx.Cfg.AgentID)
|
||||
|
||||
// 3. Poll for commands — auth failures are typed sentinel errors
|
||||
response, err := ctx.APIClient.GetCommands(ctx.Cfg.AgentID, metrics)
|
||||
if err != nil {
|
||||
switch {
|
||||
case errors.Is(err, client.ErrMachineMismatch):
|
||||
// Terminal: config moved/copied. Loud critical event, keep
|
||||
// polling so agent stays visible. Human must re-register.
|
||||
log.Printf("[ERROR] [agent] [auth] machine_id_mismatch ...")
|
||||
case errors.Is(err, client.ErrUnauthorized) && ctx.Cfg.RefreshToken != "":
|
||||
// JWT expired — auto-renew with the refresh token (machine-bound)
|
||||
renewErr := ctx.APIClient.RenewToken(...)
|
||||
switch {
|
||||
case renewErr == nil:
|
||||
ctx.Cfg.Token = ctx.APIClient.GetToken()
|
||||
// Persist rotated refresh token if server returned one
|
||||
if rt := ctx.APIClient.GetRefreshToken(); rt != "" {
|
||||
ctx.Cfg.RefreshToken = rt
|
||||
}
|
||||
ctx.Cfg.Save(...)
|
||||
consecutiveFailures = 0
|
||||
continue
|
||||
case errors.Is(renewErr, client.ErrRefreshTokenInvalid):
|
||||
// Terminal: refresh token revoked/expired. Critical event.
|
||||
default:
|
||||
// Transient (network, 502). Fall through to backoff.
|
||||
}
|
||||
}
|
||||
// Exponential backoff: 10s → 20s → 40s → ... → 5min cap
|
||||
backoff := calculateBackoff(consecutiveFailures)
|
||||
consecutiveFailures++
|
||||
sleepWithContext(ctx.Ctx, backoff)
|
||||
continue
|
||||
}
|
||||
|
||||
// Reset backoff on success
|
||||
consecutiveFailures = 0
|
||||
|
||||
// 4. Process each command
|
||||
for _, cmd := range response.Commands { ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Command Dispatch
|
||||
|
||||
**File:** `agent/internal/orchestrator/system_scanner.go`
|
||||
|
||||
```go
|
||||
func (o *Orchestrator) ExecuteCommand(cmd *Command) *CommandResult {
|
||||
result := &CommandResult{
|
||||
CommandID: cmd.ID,
|
||||
Status: "pending",
|
||||
}
|
||||
|
||||
switch cmd.Type {
|
||||
// Scanner commands
|
||||
case "scan_apt":
|
||||
result = o.scanAPT(cmd)
|
||||
case "scan_dnf":
|
||||
result = o.scanDNF(cmd)
|
||||
case "scan_docker":
|
||||
result = o.scanDocker(cmd)
|
||||
case "scan_windows":
|
||||
result = o.scanWindows(cmd)
|
||||
|
||||
// Update commands
|
||||
case "update_agent":
|
||||
result = o.updateAgent(cmd)
|
||||
|
||||
// System commands
|
||||
case "reboot":
|
||||
result = o.reboot(cmd)
|
||||
|
||||
default:
|
||||
result.Status = "failed"
|
||||
result.Error = "unknown command type"
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## At-Least-Once Delivery
|
||||
|
||||
**Receipt Tracking:**
|
||||
```go
|
||||
// agent/internal/orchestrator/command_handler.go
|
||||
func (c *CommandHandler) recordReceipt(commandID string) {
|
||||
receipts := c.loadReceipts()
|
||||
receipts[commandID] = time.Now().Unix()
|
||||
atomicWrite(c.receiptFile, receipts)
|
||||
}
|
||||
```
|
||||
|
||||
**Acknowledgment Tracking:**
|
||||
```go
|
||||
// agent/internal/orchestrator/command_handler.go
|
||||
func (c *CommandHandler) recordAcknowledgment(commandID string) {
|
||||
acks := c.loadAcks()
|
||||
acks[commandID] = true
|
||||
atomicWrite(c.ackFile, acks)
|
||||
}
|
||||
```
|
||||
|
||||
**Timeout Handling:**
|
||||
```go
|
||||
// server/internal/services/timeout.go
|
||||
func (s *TimeoutService) checkForReceivedTimeouts() {
|
||||
for _, cmd := range s.pendingCommands {
|
||||
if time.Since(cmd.ReceivedAt) > s.timeoutConfig.ReceivedTimeout {
|
||||
if !s.acknowledged[cmd.ID] {
|
||||
// Re-emit command
|
||||
s.ReemitCommand(cmd)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Circuit Breaker Integration
|
||||
|
||||
**File:** `agent/internal/circuitbreaker/circuitbreaker.go`
|
||||
|
||||
```go
|
||||
func (cb *CircuitBreaker) TryExecute(fn func() error) error {
|
||||
if cb.State == "open" {
|
||||
// Check if it's time to attempt recovery
|
||||
if time.Since(cb.LastFailureTime) > cb.OpenDuration {
|
||||
cb.State = "half-open"
|
||||
cb.FailureCount = 0
|
||||
} else {
|
||||
return errors.New("circuit breaker open")
|
||||
}
|
||||
}
|
||||
|
||||
err := fn()
|
||||
if err != nil {
|
||||
cb.FailureCount++
|
||||
cb.LastFailureTime = time.Now()
|
||||
|
||||
if cb.FailureCount >= cb.FailureThreshold {
|
||||
cb.State = "open"
|
||||
logSecurityEvent("[reliability] [agent] [circuit-breaker] Opened for", cb.Name)
|
||||
}
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// Success in half-open state
|
||||
if cb.State == "half-open" {
|
||||
cb.State = "closed"
|
||||
logSecurityEvent("[reliability] [agent] [circuit-breaker] Closed for", cb.Name)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Polling is periodic and idempotent — same command may be received multiple times.
|
||||
|
||||
**Connection:** Deduplication (`verification/04-replay-protection.md`) prevents duplicate execution.
|
||||
|
||||
**Connection:** Nonce validation (`verification/04-replay-protection.md`) binds update commands to time window.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents scanner failures from blocking other operations.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** Executed commands (`verification/04-replay-protection.md`) persisted to disk with 4-hour TTL.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
279
RAF/flows/03-agent-upgrade.md
Normal file
279
RAF/flows/03-agent-upgrade.md
Normal file
|
|
@ -0,0 +1,279 @@
|
|||
# Agent Self-Upgrade Flow
|
||||
|
||||
**7-step agent upgrade with rollback and verification.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agents can self-update without manual intervention. The flow includes nonce validation, checksum verification, Ed25519 signature verification, atomic binary swap, and automatic rollback on failure.
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/02-command-execution.md` (command dispatch)
|
||||
- `verification/01-signing-pipeline.md` (binary signing)
|
||||
- `verification/02-agent-verification.md` (binary verification)
|
||||
- `verification/04-replay-protection.md` (nonce validation)
|
||||
- `security/02-authentication-stack.md` (machine binding)
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Flow
|
||||
|
||||
### 1. Admin Triggers Update
|
||||
|
||||
**File:** `server/internal/api/handlers/agent_updates.go`
|
||||
|
||||
```go
|
||||
func (h *AgentUpdateHandler) TriggerAgentUpdate(w http.ResponseWriter, r *http.Request) {
|
||||
// 1. Validate agent ID
|
||||
agentID := r.URL.Query().Get("agent_id")
|
||||
if agentID == "" {
|
||||
http.Error(w, "agent_id required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// 2. Create signed update_agent command
|
||||
cmd := &Command{
|
||||
AgentID: agentID,
|
||||
ID: uuid.New(),
|
||||
Type: "update_agent",
|
||||
Params: `{"version": "latest"}`,
|
||||
}
|
||||
|
||||
// 3. Generate nonce (2× check-in interval)
|
||||
nonce := h.nonceService.GenerateNonce()
|
||||
|
||||
// 4. Sign command with Ed25519
|
||||
signature := h.signingService.SignCommand(cmd, nonce)
|
||||
|
||||
// 5. Store command in database
|
||||
h.db.CreateCommand(cmd, signature)
|
||||
|
||||
logSecurityEvent("[operation] [server] [update] Triggered update for agent:", agentID)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Agent Receives Command
|
||||
|
||||
**File:** `agent/internal/agent/loop.go`
|
||||
|
||||
```go
|
||||
commands, err := agent.client.GetCommands(ctx, agent.agentID, agent.machineID, agent.metrics)
|
||||
for _, cmd := range commands {
|
||||
if cmd.Type == "update_agent" {
|
||||
// Validate nonce first
|
||||
if err := agent.validateNonce(cmd); err != nil {
|
||||
logSecurityEvent("[security] [agent] [nonce] Nonce validation failed:", err)
|
||||
continue
|
||||
}
|
||||
|
||||
// Execute update_agent command
|
||||
result := agent.orchestrator.ExecuteCommand(cmd)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Verify Command Nonce
|
||||
|
||||
**File:** `agent/internal/orchestrator/command_handler.go`
|
||||
|
||||
```go
|
||||
func (c *CommandHandler) validateNonce(cmd *Command) error {
|
||||
nonce := cmd.Nonce
|
||||
maxAge := time.Duration(c.timeoutConfig.NonceMaxAge)
|
||||
|
||||
if time.Since(cmd.CreatedAt) > maxAge {
|
||||
return errors.New("nonce expired")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Verify Command Signature
|
||||
|
||||
**File:** `agent/internal/crypto/verification.go`
|
||||
|
||||
```go
|
||||
func (v *Verifier) VerifyCommand(cmd *Command) error {
|
||||
// 1. Parse signature from v3 format
|
||||
parts := strings.Split(cmd.Signature, ":")
|
||||
if len(parts) != 6 {
|
||||
return errors.New("invalid signature format")
|
||||
}
|
||||
|
||||
// 2. Verify Ed25519 signature
|
||||
agentID, id, cmdType, paramsHash, timestamp := parts[0], parts[1], parts[2], parts[3], parts[4]
|
||||
expected := fmt.Sprintf("%s:%s:%s:%s:%s", agentID, id, cmdType, paramsHash, timestamp)
|
||||
|
||||
if !ed25519.Verify(v.serverPublicKey, []byte(expected), []byte(parts[5])) {
|
||||
return errors.New("signature verification failed")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Download Agent Binary
|
||||
|
||||
**Endpoint:** `GET /api/v1/downloads/agent?version=latest`
|
||||
|
||||
**File:** `server/internal/api/handlers/downloads.go`
|
||||
|
||||
```go
|
||||
func (h *DownloadHandler) DownloadAgent(w http.ResponseWriter, r *http.Request) {
|
||||
platform := r.URL.Query().Get("platform")
|
||||
version := r.URL.Query().Get("version")
|
||||
|
||||
// 1. Fetch signed package from DB
|
||||
signedPackage := h.db.GetSignedPackage(platform, version)
|
||||
|
||||
if signedPackage == nil {
|
||||
http.Error(w, "package not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// 2. Serve binary with checksum + signature headers
|
||||
binaryData, _ := h.fs.ReadFile(signedPackage.BinaryPath)
|
||||
w.Header().Set("Content-Type", "application/octet-stream")
|
||||
w.Header().Set("X-Content-SHA256", signedPackage.Checksum)
|
||||
w.Header().Set("X-Content-Signature", signedPackage.Signature)
|
||||
w.Write(binaryData)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Verify Checksum and Signature
|
||||
|
||||
**File:** `agent/internal/orchestrator/update_handler.go`
|
||||
|
||||
```go
|
||||
func (h *UpdateHandler) verifyDownload(binaryData []byte, checksum string) error {
|
||||
// 1. Verify checksum
|
||||
computedChecksum := sha256.Sum256(binaryData)
|
||||
if hex.EncodeToString(computedChecksum[:]) != checksum {
|
||||
return errors.New("checksum mismatch")
|
||||
}
|
||||
|
||||
// 2. Verify Ed25519 signature
|
||||
signature, err := h.downloadBinarySignature()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
publicKey, err := h.LoadCachedPublicKey()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !ed25519.Verify(publicKey, binaryData, []byte(signature)) {
|
||||
return errors.New("signature verification failed")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. Atomic Install with Backup
|
||||
|
||||
**File:** `agent/internal/orchestrator/update_handler.go`
|
||||
|
||||
```go
|
||||
func (h *UpdateHandler) atomicInstall(binaryData []byte) error {
|
||||
binaryPath := "/usr/local/bin/redflag-agent"
|
||||
|
||||
// 1. Create backup
|
||||
backupPath := binaryPath + ".bak"
|
||||
if err := os.Rename(binaryPath, backupPath); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. Atomic install (write to temp file, then rename)
|
||||
tempPath := binaryPath + ".tmp"
|
||||
if err := os.WriteFile(tempPath, binaryData, 0755); err != nil {
|
||||
// Rollback: restore backup
|
||||
os.Rename(backupPath, binaryPath)
|
||||
return err
|
||||
}
|
||||
|
||||
os.Rename(tempPath, binaryPath)
|
||||
|
||||
// 3. Restart service
|
||||
return h.restartService()
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Watchdog and Rollback
|
||||
|
||||
**File:** `agent/internal/orchestrator/update_handler.go`
|
||||
|
||||
```go
|
||||
func (a *Agent) runUpdateWatchdog() {
|
||||
// 1. Start 15-minute watchdog (configurable)
|
||||
done := make(chan bool)
|
||||
go func() {
|
||||
time.Sleep(15 * time.Minute) // Default: operational.agent_update_timeout_minutes
|
||||
done <- true
|
||||
}()
|
||||
|
||||
// 2. Wait for update completion
|
||||
select {
|
||||
case <-done:
|
||||
// Update completed or timed out
|
||||
logSecurityEvent("[operation] [agent] [update] Watchdog timeout, rolling back")
|
||||
a.rollbackUpdate()
|
||||
|
||||
case <-a.updateCompleteChan:
|
||||
// Update acknowledged by server
|
||||
logSecurityEvent("[operation] [agent] [update] Server acknowledged new version")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
func (a *Agent) rollbackUpdate() {
|
||||
backupPath := "/usr/local/bin/redflag-agent.bak"
|
||||
binaryPath := "/usr/local/bin/redflag-agent"
|
||||
|
||||
// Restore backup
|
||||
os.Rename(backupPath, binaryPath)
|
||||
|
||||
// Restart old version
|
||||
a.restartService()
|
||||
|
||||
// Report timeout
|
||||
a.reportUpdateTimeout()
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Self-upgrade is a privileged operation — requires admin approval and nonce validation.
|
||||
|
||||
**Connection:** Nonce validation (`verification/04-replay-protection.md`) prevents replay attacks on update commands.
|
||||
|
||||
**Connection:** Machine binding (`security/04-machine-binding.md`) ensures only authorized agent can receive updates.
|
||||
|
||||
**Connection:** Atomic install (`flows/03-agent-upgrade.md`) implements ETHOS #4 (idempotency).
|
||||
|
||||
**Connection:** Backup mechanism (`flows/03-agent-upgrade.md`) enables rollback on failure.
|
||||
|
||||
**Connection:** Watchdog timeout (`flows/03-agent-upgrade.md`) aligns with `operational.agent_update_timeout_minutes` setting.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
74
RAF/flows/04-heartbeat.md
Normal file
74
RAF/flows/04-heartbeat.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# Heartbeat System
|
||||
|
||||
**Agent health monitoring via system event polling.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedFlag monitors agent health through a dedicated heartbeat mechanism. Agents periodically report status, and the server tracks operational state.
|
||||
|
||||
---
|
||||
|
||||
## Heartbeat Types
|
||||
|
||||
| Type | Trigger | Source | Purpose |
|
||||
|------|---------|--------|---------|
|
||||
| `manual` | Agent-initiated status check | Agent | Regular health report |
|
||||
| `system` | Auto-triggered at dispatch | Server | Confirm command receipt |
|
||||
| `command` | Command execution complete | Agent | Report execution status |
|
||||
| `stuck` | Timeout exceeded | Server | Alert on stalled operations |
|
||||
|
||||
**Implementation:**
|
||||
- `agent/internal/orchestrator/system_scanner.go`
|
||||
- `server/internal/services/timeout.go`
|
||||
|
||||
---
|
||||
|
||||
## Heartbeat Flow
|
||||
|
||||
```
|
||||
1. Agent executes command
|
||||
2. Agent triggers heartbeat (system or command)
|
||||
3. Server receives heartbeat event
|
||||
4. Server updates agent metadata (heartbeat_source, last_heartbeat)
|
||||
5. Server updates operational state (update_stuck_minutes countdown)
|
||||
6. Dashboard displays current status
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- Agent: `agent/internal/orchestrator/system_scanner.go:queueSystemHeartbeat()`
|
||||
- Server: `server/internal/handlers/agents.go:HandleSystemHeartbeat()`
|
||||
- Metadata: `agent.metadata.heartbeat_source`
|
||||
|
||||
---
|
||||
|
||||
## Timeout Configuration
|
||||
|
||||
**Operational timeouts:**
|
||||
- `operational.update_stuck_minutes`: Default 5 minutes
|
||||
- `operational.agent_update_timeout_minutes`: Default 15 minutes
|
||||
- Configured via `security_settings` table
|
||||
|
||||
**Watchdog behavior:**
|
||||
- If command exceeds `update_stuck_minutes` → mark as stuck
|
||||
- If update exceeds `agent_update_timeout_minutes` → trigger rollback
|
||||
- Timeout events logged to history table
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Heartbeat is a best-effort mechanism — agent crashes will be detected on next poll cycle.
|
||||
|
||||
**Connection:** Heartbeat system (`flows/04-heartbeat.md`) implements ETHOS #3 (assume failure).
|
||||
|
||||
**Connection:** Timeout configuration (`flows/04-heartbeat.md`) ties to security settings (`security_settings.operational`).
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** Auto-heartbeat gates (`flows/04-heartbeat.md`) controlled by `policy.auto_heartbeat_enabled`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
77
RAF/flows/05-capability-advertisement.md
Normal file
77
RAF/flows/05-capability-advertisement.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# Capability Advertisement
|
||||
|
||||
**Dynamic scanner capability reporting from agents to server.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agents dynamically report which scanners they have available. The server maintains a live view of agent capabilities without requiring registration updates.
|
||||
|
||||
---
|
||||
|
||||
## Capability Detection
|
||||
|
||||
**Process:**
|
||||
```
|
||||
1. Agent checks each scanner's availability (DetectAvailable())
|
||||
2. Agent reports available scanners in check-in payload
|
||||
3. Server diff against existing subsystem rows
|
||||
4. Server inserts new scanners, updates existing
|
||||
5. Server removes stale scanners (if configured)
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- Detection: `agent/internal/scanner/*.go:DetectAvailable()`
|
||||
- Sync: `server/internal/api/handlers/agents.go:syncAvailableScanners()`
|
||||
|
||||
---
|
||||
|
||||
## Scanner Types
|
||||
|
||||
| Scanner | Platform | Detection Method |
|
||||
|---------|----------|------------------|
|
||||
| APT | Linux | Check `/var/lib/apt/lists/lock` |
|
||||
| DNF | Linux | Check `dnf version` availability |
|
||||
| Winget | Windows | Check `winget` CLI availability |
|
||||
| WUA | Windows | Check WindowsUpdate Agent service |
|
||||
| Docker | All | Check Docker socket/CLI availability |
|
||||
| Package (upstream) | All | Check upstream registry sync capability |
|
||||
|
||||
---
|
||||
|
||||
## Sync Pattern (Idempotent)
|
||||
|
||||
**INSERT new scanners:**
|
||||
```sql
|
||||
INSERT INTO agent_subsystems (agent_id, name, enabled)
|
||||
VALUES ($1, $2, true)
|
||||
ON CONFLICT (agent_id, name) DO NOTHING
|
||||
```
|
||||
|
||||
**UPDATE existing:**
|
||||
```sql
|
||||
UPDATE agent_subsystems SET enabled = true WHERE agent_id = $1 AND name = $2
|
||||
```
|
||||
|
||||
**No DELETE on transient signal:**
|
||||
- Brief `IsAvailable()=false` doesn't remove scanner
|
||||
- Removal requires explicit admin action or stale threshold
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Capabilities can change post-registration (Docker installed after agent deploy).
|
||||
|
||||
**Connection:** Capability sync (`flows/05-capability-advertisement.md`) implements ETHOS #4 (idempotent scanner sync).
|
||||
|
||||
**Connection:** `syncAvailableScanners()` (`flows/05-capability-advertisement.md`) called on every agent check-in.
|
||||
|
||||
**Connection:** Capability badges (`flows/05-capability-advertisement.md`) rendered in `AgentHealth.tsx` dashboard.
|
||||
|
||||
**Connection:** Scanner detection (`agent/internal/scanner/*.go:DetectAvailable()`) called during registration and polling.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
274
RAF/flows/06-update-lifecycle.md
Normal file
274
RAF/flows/06-update-lifecycle.md
Normal file
|
|
@ -0,0 +1,274 @@
|
|||
# Update Lifecycle Flow
|
||||
|
||||
**Package state machine, two execution paths, and the orchestrator that drives them.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Packages move through a server-owned state machine from scan discovery to terminal resolution. The agent is a stateless executor — it receives commands, executes them, and reports results. The server owns every state transition.
|
||||
|
||||
The flow has two execution paths that diverge at install time:
|
||||
- **Capability gate** (dnf, apt): server mints an Ed25519-signed token → agent's Rust helper verifies + executes → agent reports receipt
|
||||
- **Legacy command** (docker, winget, windows_update): server creates a `confirm_dependencies` command → agent executes → agent reports via `ReportLog`
|
||||
|
||||
**Implementation status:** State machine enforced with typed `PackageStatus` and `ValidateTransition` guards (LIFECYCLE-001, v0.2.1.3). Lifecycle orchestrator running with stuck-state recovery and auto-advance (LIFECYCLE-003, v0.2.2.0). Approval-time supply chain enforcement: vuln in closure = full stop with audited override (v0.2.3.1).
|
||||
|
||||
**Cross-references:**
|
||||
- `flows/02-command-execution.md` — agent polling, command dispatch, at-least-once delivery
|
||||
- `flows/04-heartbeat.md` — heartbeat lifecycle, system-vs-manual source
|
||||
- `security/05-supply-chain-gate.md` — capability token design, helper execution
|
||||
- `core/01-ethos.md` — idempotency (§4), assume failure (§3)
|
||||
- `reference/projects/redflag-framework.md` §11.10 — State Machine Exhaustiveness
|
||||
- `docs/tasks/GATE-000-supply-chain-gate-plan.md` — gate build status (steps 1-5 done, 6-7 remain)
|
||||
|
||||
---
|
||||
|
||||
## State Machine
|
||||
|
||||
Eight states. Three terminal (`installed`, `failed`, `ignored`), three active (`checking_dependencies`, `pending_dependencies`, `installing`), two waiting (`pending`, `approved`).
|
||||
|
||||
```
|
||||
pending ──────► approved ──────► checking_deps ─┬──► installing ─┬──► installed
|
||||
│ │ │ │ │
|
||||
│ │ ├──► pending_deps ├──► failed
|
||||
│ │ │ │ │
|
||||
└──► ignored └──► ignored ├──► installed └──► pending_deps
|
||||
└──► failed (new deps surfaced)
|
||||
|
||||
pending_deps ──► installing ───────┤
|
||||
│ │
|
||||
└──► failed │
|
||||
│
|
||||
installing ───► pending_deps ───────┘
|
||||
```
|
||||
|
||||
| State | Type | Entered by |
|
||||
|-------|------|------------|
|
||||
| `pending` | waiting | `UpdateCurrentStateInTx` — scan discovery |
|
||||
| `approved` | waiting | `ApproveUpdate` — operator or auto-approve policy |
|
||||
| `checking_dependencies` | active | `SetCheckingDependencies` — dry-run command queued |
|
||||
| `pending_dependencies` | active | `SetPendingDependencies` — agent reported deps, operator must review |
|
||||
| `installing` | active | `InstallUpdate` / `SetInstallingWithNoDependencies` — agent executing |
|
||||
| `installed` | terminal | `UpdatePackageStatus` — install succeeded |
|
||||
| `failed` | terminal | `UpdatePackageStatus` — install failed, timeout, or token mint failed |
|
||||
| `ignored` | terminal | `RejectUpdate` — operator rejected |
|
||||
|
||||
**Re-scan behavior:** `UpdateCurrentStateInTx` (queries/updates.go:595) preserves terminal states on re-scan. Currently preserves `updated` and `ignored`; `failed` is NOT preserved (bug — fixed in LIFECYCLE-001). All other states reset to `pending` when a new version is discovered.
|
||||
|
||||
**Implementation:**
|
||||
- Transition functions: `server/internal/database/queries/updates.go`
|
||||
- Handler orchestration: `server/internal/api/handlers/updates.go`
|
||||
- DB constraint: `current_package_state.status CHECK (...)` — migrations 003, 005, 007
|
||||
|
||||
---
|
||||
|
||||
## Capability Gate Path (Linux: dnf, apt)
|
||||
|
||||
```
|
||||
checking_dependencies
|
||||
│
|
||||
│ Agent polls, receives dry_run_update command
|
||||
│ Agent: DiscoveryRunner.DryRun(pkg, version)
|
||||
│ Agent reports: POST /api/v1/updates/report-dependencies
|
||||
│
|
||||
▼
|
||||
ReportDependencies handler (updates.go:1232)
|
||||
│ pinReportedClosure — stores artifact hashes from signed repo metadata
|
||||
│
|
||||
├─ 0 deps ──► mintResolvedClosure → capability token (status: pending)
|
||||
│ SetInstallingWithNoDependencies → installing
|
||||
│
|
||||
└─ deps ──► SetPendingDependencies → pending_dependencies
|
||||
[operator clicks Confirm]
|
||||
ConfirmDependencies handler (updates.go:1470)
|
||||
mintResolvedClosure → capability token (status: pending)
|
||||
InstallUpdate → installing
|
||||
│
|
||||
▼
|
||||
installing
|
||||
│
|
||||
│ Agent polls: GET /api/v1/capability-tokens/pending/:agent_id
|
||||
│ Agent: consumer.ProcessToken → helper invoked via systemd-run --pipe
|
||||
│ Helper: verify Ed25519 sig → verify artifact hashes → dnf install / apt install
|
||||
│ Agent reports: POST /api/v1/capability-tokens/:token_id/result
|
||||
│
|
||||
▼
|
||||
ReportCapabilityResult handler (updates.go:1936)
|
||||
│ MarkConsumed(token_id)
|
||||
│ UpdatePackageStatus → installed | failed
|
||||
```
|
||||
|
||||
The agent never receives an install command on this path. The capability token IS the install authorization — the helper enforces that only the exact artifact closure signed by the server can be installed.
|
||||
|
||||
**Implementation:**
|
||||
- Token mint: `server/internal/services/capability_minter.go`
|
||||
- Token polling: `agent/internal/agent/loop.go:processCapabilityTokens`
|
||||
- Token consumption: `agent/internal/capability/consumer.go`
|
||||
- Helper: `helper/src/main.rs`
|
||||
- Discovery: `agent/internal/installer/dnf.go`, `agent/internal/installer/apt.go`
|
||||
|
||||
---
|
||||
|
||||
## Legacy Command Path (Docker, Winget, Windows)
|
||||
|
||||
```
|
||||
checking_dependencies
|
||||
│ (same dry-run flow as capability path)
|
||||
▼
|
||||
pending_dependencies
|
||||
│ [operator clicks Confirm]
|
||||
│ ConfirmDependencies handler (updates.go:1470)
|
||||
│ Creates confirm_dependencies command (signed Ed25519)
|
||||
│ InstallUpdate → installing
|
||||
▼
|
||||
installing
|
||||
│
|
||||
│ Agent polls, receives confirm_dependencies command
|
||||
│ Agent: type-asserts installer to access mutation methods
|
||||
│ Agent executes install directly (no helper)
|
||||
│ Agent reports: POST /api/v1/updates/report-log
|
||||
│
|
||||
▼
|
||||
ReportLog handler (updates.go:645)
|
||||
│ Idempotency check on command_id + terminal command status
|
||||
│ MarkCommandCompleted | MarkCommandFailed
|
||||
│ If command_type == confirm_dependencies:
|
||||
│ UpdatePackageStatus → installed | failed
|
||||
```
|
||||
|
||||
Docker, Winget, and Windows Update are not behind the capability gate. They use direct mutation via type assertion on the installer interface. This is a known gap — the gate design covers them but implementation is deferred.
|
||||
|
||||
**Implementation:**
|
||||
- Docker install: `agent/internal/handlers/docker.go`
|
||||
- Winget install: `agent/internal/handlers/winget.go`
|
||||
- Windows install: `agent/internal/handlers/windows_update.go`
|
||||
- Command dispatch: `agent/internal/orchestrator/system_scanner.go:ExecuteCommand`
|
||||
|
||||
---
|
||||
|
||||
## Agent Handoff Points
|
||||
|
||||
The agent has no lifecycle state awareness. It is a stateless executor — it receives commands, executes them, and reports results.
|
||||
|
||||
| Phase | Agent trigger | Agent action | Server endpoint | Server state change |
|
||||
|-------|--------------|--------------|-----------------|---------------------|
|
||||
| Scan | Scanner schedule (poll-driven) | Run package manager scan | `POST /api/v1/updates/report-log` | `UpdateCurrentStateInTx` → `pending` |
|
||||
| Dry-run | Receives `dry_run_update` command | DiscoveryRunner.DryRun | `POST /api/v1/updates/report-dependencies` | → `installing` or `pending_dependencies` |
|
||||
| Token install | Polls `GET /api/v1/capability-tokens/pending/:agent_id` | consumer.ProcessToken → helper | `POST /api/v1/capability-tokens/:id/result` | → `installed` or `failed` |
|
||||
| Command install | Receives `confirm_dependencies` command | Direct installer mutation | `POST /api/v1/updates/report-log` | → `installed` or `failed` |
|
||||
|
||||
**Heartbeat coordination:** Before creating a dry-run or install command, `InstallUpdate` and `ConfirmDependencies` queue a 10-minute `enable_heartbeat` command if one is not already active. This collapses the agent's poll interval during active lifecycle phases. Heartbeat creation failure is logged but does not block the lifecycle transition.
|
||||
|
||||
---
|
||||
|
||||
## Architectural Status
|
||||
|
||||
### Enforced (v0.2.2.0+)
|
||||
|
||||
- **Typed state machine.** `PackageStatus` is a typed enum in Go. `ValidateTransition`
|
||||
guards enforce the allowed graph. `TransitionPackageStatus` uses `WHERE status = $current`
|
||||
— a concurrent race lands on the constraint, not a silent overwrite. Migration 047 aligned
|
||||
all existing rows. (LIFECYCLE-001, v0.2.1.3)
|
||||
|
||||
- **Lifecycle orchestrator.** Timer-driven auto-advance and stuck-state recovery for
|
||||
`checking_dependencies` and `installing`. Auto-approval policy support. Packages stuck
|
||||
in active states no longer require manual operator intervention. (LIFECYCLE-003, v0.2.2.0)
|
||||
|
||||
- **Supply chain enforcement at approval.** `ApproveUpdate` checks the full resolved closure
|
||||
against OSV. A vuln anywhere in the closure returns 409 and mints nothing. Override requires
|
||||
an operator reason and is journaled. `ClosureCleared` predicate shared with auto-confirm.
|
||||
(v0.2.3.1)
|
||||
|
||||
### Remaining Visibility Gaps
|
||||
|
||||
- **Stepper collapses active states.** `checking_dependencies` and `pending_dependencies`
|
||||
both render as step 2 ("Approved") in the 4-step lifecycle stepper. The operator cannot
|
||||
distinguish "waiting for dry-run" from "dependencies need your review" without reading
|
||||
the status badge text.
|
||||
|
||||
- **Capability token path is invisible.** Token mint, consumption, and execution are
|
||||
tracked only in server logs. There is no UI endpoint for token status, and the operator
|
||||
cannot tell whether a package is installing via token or legacy command.
|
||||
|
||||
- **Staging area incomplete.** The Staging page exists (v0.2.1.1) but the full vision —
|
||||
assembling, staged, installing, completed as a single operator view — is not built.
|
||||
|
||||
---
|
||||
|
||||
## Target Model: Scan-Set Reconciliation + Maintenance-Window Campaign
|
||||
|
||||
**Decided 2026-06-06 (Casey + Opus). Supersedes the additive-scan assumption in lines 203–205.**
|
||||
|
||||
### The defect in the current model
|
||||
|
||||
A scan today is treated as an **additive discovery stream**, not a **set snapshot**.
|
||||
`ReportUpdates` (`handlers/updates.go:191`) turns each reported update into a `discovered`
|
||||
event → per-row UPSERT (`UpdateCurrentStateInTx`). `ReconcileFromScan` (`models/update_state.go`)
|
||||
only reconciles packages **present** in the scan (resting states preserved, else → `pending`).
|
||||
The two automatic paths to `installed` are receipt-driven (RedFlag drove it) and operator-manual
|
||||
("resolved out of band").
|
||||
|
||||
**There is no closure-by-absence.** When a package drops out of a scan — patched by
|
||||
`dnf-automatic`, a sysadmin, or anything outside RedFlag — its row stays `pending` forever.
|
||||
The system adds and re-discovers but never subtracts. This is a §11.8 violation (render the
|
||||
divergence, not the union) on a §11.1 lifecycle-boundary gap. It also contradicts the premise:
|
||||
RedFlag should report what is outstanding **within the window the operator allocates**, not a
|
||||
live snapshot that silently rots.
|
||||
|
||||
### Phase 1 — Scan-set reconciler (foundation)
|
||||
|
||||
Treat each ecosystem scan as the authoritative full set for that agent+ecosystem. On report,
|
||||
diff the reported set against tracked non-resting rows (the DefectDojo reimport pattern —
|
||||
`to_mitigate = set(tracked) − set(reported)`):
|
||||
|
||||
| Scan vs tracked | Transition |
|
||||
|-----------------|------------|
|
||||
| reported, not tracked | create `pending` (discovered) |
|
||||
| reported, tracked | keep / version-bump (`ReconcileFromScan`) |
|
||||
| **waiting (`pending`/`approved`), absent from scan** | → `installed`, provenance `out_of_band` (resolution is external by construction) |
|
||||
| in-flight (`checking_dependencies`/`pending_dependencies`/`installing`), absent | **not closed by the reconciler** — owned by orchestrator + receipt path; `installing → installed` carries `redflag_receipt` |
|
||||
| previously resolved, reappears | reopen → `pending` (`installed → pending` edge; SQL CASE + `ReconcileFromScan` in lockstep) |
|
||||
|
||||
**Closure scope is the waiting states only.** Closing in-flight rows would race a RedFlag-driven
|
||||
install and mislabel its provenance, and split ownership of `installing` between the reconciler and
|
||||
the orchestrator (§11.7). The `pending/approved → installed` edge was added to the state machine for
|
||||
this path. Closure routes through `transitionStatus` (not a raw UPSERT) so it stays inside the state
|
||||
machine and is idempotent (ETHOS §4). Absence must be confirmed by a **successful** scan of that
|
||||
ecosystem (exit 0) — a failed/empty-due-to-error scan must never close rows (ETHOS §3, assume failure).
|
||||
|
||||
### Phase 2 — Maintenance-window campaign
|
||||
|
||||
The operator allocates a window. At **window-open** the in-scope set is frozen (the campaign
|
||||
scope). Through the window, packages are driven to terminal. At **window-close** a closing scan
|
||||
reconciles (Phase 1) and the campaign reports: applied / failed / deferred / resolved-out-of-band.
|
||||
Dry-run (`checking_dependencies`) may run anytime; `installing` respects the window (already
|
||||
asserted in the footer below). Model precedent: TacticalRMM `WinUpdatePolicy`
|
||||
(`run_time_hour`/`run_time_days`/`run_time_frequency`) + `WinUpdate.date_installed`.
|
||||
|
||||
**Work items:** `docs/tasks/RECONCILE-001-scan-set-closure.md` (Phase 1),
|
||||
`docs/tasks/WINDOW-001-maintenance-window-campaign.md` (Phase 2, depends on RECONCILE-001).
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** The agent is a stateless executor. It does not track lifecycle states and should not need to. The server owns the state machine.
|
||||
|
||||
**Assumption:** ~~Re-scan reconciliation is not a state machine transition — it is periodic scan-driven reset governed by `UpdateCurrentStateInTx`, not `ValidateTransition`.~~ **Superseded 2026-06-06** (see "Target Model" above): re-scan becomes set reconciliation, and closure-by-absence routes through `transitionStatus` inside the state machine.
|
||||
|
||||
**Assumption:** Dry-run is read-only and safe to run outside maintenance windows. The `checking_dependencies` phase can proceed anytime. The install phase (`installing`) must respect the maintenance window.
|
||||
|
||||
**Connection:** Update lifecycle implements ETHOS §4 (idempotency) — every transition must be run-3x-safe. The guarded UPDATE pattern in `TransitionPackageStatus` (LIFECYCLE-001) enforces this at the DB layer.
|
||||
|
||||
**Connection:** Update lifecycle implements ETHOS §3 (assume failure) — every active state must have a timeout path to `failed`. The orchestrator's stuck-state recovery (LIFECYCLE-003) closes the current gap where `checking_dependencies` has no timeout.
|
||||
|
||||
**Connection:** Capability gate path (`security/05-supply-chain-gate.md`) is the security-critical execution path. Token visibility (LIFECYCLE-005) makes this path auditable — currently the operator is blind to token lifecycle.
|
||||
|
||||
**Connection:** Heartbeat coordination (`flows/04-heartbeat.md`) ensures the agent polls faster during active lifecycle phases. The heartbeat is a side effect of command creation — it does not block the lifecycle transition.
|
||||
|
||||
**Connection:** Command execution (`flows/02-command-execution.md`) provides the at-least-once delivery and deduplication that the lifecycle depends on for agent handoff. The lifecycle layer sits above command execution — it creates commands and processes their results.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-01 — updated for LIFECYCLE-001/003 completion and supply chain enforcement*
|
||||
137
RAF/flows/07-process-scan.md
Normal file
137
RAF/flows/07-process-scan.md
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
# Process Scan Flow
|
||||
|
||||
**On-demand process inventory scanning and drill-down detail retrieval.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The process scan follows the existing command-dispatch pattern (same as heartbeat, storage scan, etc.). It is triggered on-demand when a user opens the Processes tab in the dashboard — not on a background schedule.
|
||||
|
||||
---
|
||||
|
||||
## Flow: List Scan
|
||||
|
||||
```
|
||||
Dashboard Server Agent
|
||||
│ │ │
|
||||
│ POST /processes/scan │ │
|
||||
│───────────────────────>│ │
|
||||
│ │ │
|
||||
│ │ dedup check: │
|
||||
│ │ GetPendingCommands() │
|
||||
│ │ scan_processes pending? │
|
||||
│ │ │
|
||||
│ │ create signed command: │
|
||||
│ │ CommandTypeScanProcesses│
|
||||
│ │ SignCommand() │
|
||||
│ │ │
|
||||
│ 200 {command_id} │ │
|
||||
│<───────────────────────│ │
|
||||
│ │ │
|
||||
│ │ GET /commands (poll) │
|
||||
│ │<────────────────────────│
|
||||
│ │ │
|
||||
│ │ 200 [scan_processes] │
|
||||
│ │────────────────────────>│
|
||||
│ │ │
|
||||
│ │ │ GetFullProcessSnapshot()
|
||||
│ │ │ reads /proc for all PIDs
|
||||
│ │ │
|
||||
│ │ POST /process-scan │
|
||||
│ │<────────────────────────│
|
||||
│ │ │
|
||||
│ │ InsertSnapshot() │
|
||||
│ │ InsertProcesses() │
|
||||
│ │ InsertRelatedData() │
|
||||
│ │ CleanupOldSnapshots(10) │
|
||||
│ │ │
|
||||
│ │ 200 OK │
|
||||
│ │────────────────────────>│
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flow: Drill-Down (Process Detail)
|
||||
|
||||
```
|
||||
Dashboard Server Agent
|
||||
│ │ │
|
||||
│ GET /processes/:pid │ │
|
||||
│───────────────────────>│ │
|
||||
│ │ │
|
||||
│ │ GetProcessByID() │
|
||||
│ │ GetProcessRelated() │
|
||||
│ │ │
|
||||
│ 200 {process, related}│ │
|
||||
│<───────────────────────│ │
|
||||
```
|
||||
|
||||
Note: Drill-down reads from the database (data collected during the list scan). No additional agent command is issued.
|
||||
|
||||
---
|
||||
|
||||
## Command: `scan_processes`
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Type | `scan_processes` |
|
||||
| Parameters | None (agent reads its own /proc) |
|
||||
| Signed | Yes (Ed25519, same as all commands) |
|
||||
| Idempotent | Yes (each scan produces a new snapshot) |
|
||||
| Dedup | Server checks for existing pending command before creating |
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### `agent_process_snapshots`
|
||||
Snapshot header — one per scan.
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| id | UUID | Primary key |
|
||||
| agent_id | UUID | FK to agents |
|
||||
| command_id | UUID | FK to agent_commands |
|
||||
| process_count | INTEGER | Number of processes |
|
||||
| scanned_at | TIMESTAMPTZ | When the scan ran |
|
||||
| scan_duration_ms | INTEGER | How long the scan took |
|
||||
|
||||
### `agent_processes`
|
||||
Per-process rows — one per process per snapshot.
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| id | UUID | Primary key |
|
||||
| snapshot_id | UUID | FK to agent_process_snapshots (CASCADE) |
|
||||
| agent_id | UUID | Denormalized for query performance |
|
||||
| pid | INTEGER | Process ID |
|
||||
| name | TEXT | Process name |
|
||||
| ... | ... | 25+ fields (see scanners/05-process-scanner.md) |
|
||||
|
||||
### `agent_process_related`
|
||||
Related data — JSONB per relation type per process.
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| id | UUID | Primary key |
|
||||
| process_id | UUID | FK to agent_processes (CASCADE) |
|
||||
| relation_type | TEXT | open_file, socket, pipe, environment, memory_map, namespace, listening_port |
|
||||
| data | JSONB | The related data payload |
|
||||
|
||||
**Retention:** `CleanupOldSnapshots(10)` keeps the last 10 snapshots per agent. Cascade deletes processes and related data.
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Method | Path | Auth | Purpose |
|
||||
|--------|------|------|---------|
|
||||
| POST | `/api/v1/agents/:id/processes/scan` | Dashboard | Trigger on-demand scan |
|
||||
| GET | `/api/v1/agents/:id/processes` | Dashboard | Get latest snapshot (filterable, sortable) |
|
||||
| GET | `/api/v1/agents/:id/processes/:processId` | Dashboard | Get process detail with all related data |
|
||||
| POST | `/api/v1/agents/:id/process-scan` | Agent | Report scan results |
|
||||
|
||||
---
|
||||
|
||||
*Added: 2026-06-10*
|
||||
120
RAF/flows/08-wazuh-event-emitter.md
Normal file
120
RAF/flows/08-wazuh-event-emitter.md
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
# RAF Flow 08 — Wazuh Event Emitter (INTEG-001)
|
||||
|
||||
**Status:** Implemented (2026-06-11)
|
||||
**Source:** `server/internal/integrations/wazuh/`
|
||||
|
||||
## What
|
||||
|
||||
RedFlag emits security events to a local Wazuh agent's queue socket
|
||||
(`/var/ossec/queue/sockets/queue`) in ECS (Elastic Common Schema) format.
|
||||
Outbound-only: the emitter opens a Unix DGRAM socket and writes; it opens no
|
||||
listener and accepts no inbound traffic. RedFlag's own `security_events` journal
|
||||
remains the source of truth; Wazuh is a best-effort mirror.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
SecurityLogger.Log(event)
|
||||
→ INSERT INTO security_events (authoritative)
|
||||
→ if sink != nil: sink.Emit(event) (best-effort mirror)
|
||||
→ wazuh.Emitter.Emit
|
||||
→ json.Marshal(toECS(event))
|
||||
→ prefix "1:redflag:" (Wazuh queue protocol)
|
||||
→ Unix DGRAM send to /var/ossec/queue/sockets/queue
|
||||
```
|
||||
|
||||
## Design decisions
|
||||
|
||||
### Outbound-only, no control surface
|
||||
|
||||
The emitter is a writer. It never listens. A compromised Wazuh agent or a
|
||||
malicious socket cannot send commands or data back into RedFlag. This is the
|
||||
same trust model as the pull-only agent↔server channel: RedFlag pushes out,
|
||||
never accepts instructions in.
|
||||
|
||||
### Best-effort mirror, not a transaction
|
||||
|
||||
The journal write succeeds first, then the sink fires. If the Wazuh socket is
|
||||
down, the event is dropped and counted — the journal has it. This keeps the
|
||||
security event path from coupling to an external system's availability.
|
||||
|
||||
### Lazy connect, one retry
|
||||
|
||||
The DGRAM socket is opened on first Emit, not at startup. If a write fails
|
||||
(Wazuh agent restarted, socket recreated), one reconnect is attempted. After
|
||||
that the event is dropped with a rate-limited log warning (≤ 1/minute).
|
||||
|
||||
### Opt-in only
|
||||
|
||||
Wiring is gated on `REDFLAG_WAZUH_ENABLED=true`. Without it the `Sink` is nil
|
||||
and no socket is ever opened — zero overhead, zero log noise.
|
||||
|
||||
## Event mapping
|
||||
|
||||
RedFlag event types → Wazuh custom rule IDs (999xxx user range):
|
||||
|
||||
| RedFlag Event | Rule ID | Level |
|
||||
|---|---|---|
|
||||
| (unknown / future) | 999001 | 3 |
|
||||
| CMD_SIGNATURE_VERIFICATION_FAILED | 999002 | 12 |
|
||||
| UPDATE_NONCE_INVALID | 999003 | 12 |
|
||||
| UPDATE_SIGNATURE_VERIFICATION_FAILED | 999004 | 12 |
|
||||
| MACHINE_ID_MISMATCH | 999005 | 12 |
|
||||
| AUTH_JWT_VALIDATION_FAILED | 999006 | 12 |
|
||||
| AGENT_REGISTRATION_FAILED | 999007 | 7 |
|
||||
| UNAUTHORIZED_ACCESS_ATTEMPT | 999008 | 7 |
|
||||
| CONFIG_TAMPERING_DETECTED | 999009 | 12 |
|
||||
| ANOMALOUS_BEHAVIOR | 999010 | 7 |
|
||||
| CMD_SIGNED | 999011 | 3 |
|
||||
| CMD_SIGNATURE_VERIFICATION_SUCCESS | 999012 | 3 |
|
||||
|
||||
## ECS shape
|
||||
|
||||
```json
|
||||
{
|
||||
"@timestamp": "2026-06-11T01:00:00Z",
|
||||
"event": {
|
||||
"kind": "alert",
|
||||
"category": ["security"],
|
||||
"type": ["info"],
|
||||
"module": "redflag",
|
||||
"action": "MACHINE_ID_MISMATCH",
|
||||
"outcome": "failure",
|
||||
"severity": 9
|
||||
},
|
||||
"rule": {
|
||||
"id": "999005",
|
||||
"level": 12,
|
||||
"description": "machine binding violation"
|
||||
},
|
||||
"agent": {"id": "f7ddc5ce-..."},
|
||||
"message": "machine binding violation",
|
||||
"redflag": {},
|
||||
"wazuh": {
|
||||
"integration": {
|
||||
"name": "redflag",
|
||||
"category": "security",
|
||||
"decoders": ["json"],
|
||||
"rules": ["999005"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Operator-side setup
|
||||
|
||||
1. Wazuh agent or manager on the same host as the RedFlag server.
|
||||
2. `REDFLAG_WAZUH_ENABLED=true` in the server environment.
|
||||
3. Optionally `REDFLAG_WAZUH_SOCKET` to override the socket path (defaults to
|
||||
`/var/ossec/queue/sockets/queue`).
|
||||
4. The Wazuh ruleset (`docs/wazuh-ruleset.xml`) installed on the Wazuh manager
|
||||
so custom rule IDs decode to proper alerts.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- **Trust boundaries** → [[../security/01-trust-boundaries]] — pull-only
|
||||
doctrine section; this emitter is explicitly distinguished from the agent
|
||||
control channel.
|
||||
- **Task spec** → `docs/tasks/INTEG-001-wazuh-event-emitter.md`
|
||||
- **Verdicts umbrella** → `docs/tasks/INTEG-000-competitive-landscape-verdicts.md`
|
||||
- **Snipe-IT (next integration)** → `docs/tasks/INTEG-002-snipeit-asset-sync.md`
|
||||
136
RAF/reference/01-file-mappings.md
Normal file
136
RAF/reference/01-file-mappings.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# File Mappings
|
||||
|
||||
**Complete mapping of architectural concepts to source files.**
|
||||
|
||||
---
|
||||
|
||||
## Core Files
|
||||
|
||||
| Concept | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| ETHOS principles | `RAF/core/01-ethos.md` | Core identity and principles |
|
||||
| Architecture decisions | `RAF/core/02-architecture-decisions.md` | Key architectural choices |
|
||||
| Server component | `RAF/components/01-server.md` | Server breakdown |
|
||||
| Trust boundaries | `RAF/security/01-trust-boundaries.md` | Trust boundary matrix |
|
||||
| Auth stack | `RAF/security/02-authentication-stack.md` | Four-layer auth |
|
||||
| Machine binding | `RAF/security/04-machine-binding.md` | Hardware verification |
|
||||
| Signing pipeline | `RAF/verification/01-signing-pipeline.md` | Ed25519 signing |
|
||||
| Agent verification | `RAF/verification/02-agent-verification.md` | TOFU key caching |
|
||||
| Key rotation | `RAF/verification/03-key-rotation.md` | Key rotation support |
|
||||
| Replay protection | `RAF/verification/04-replay-protection.md` | Multi-layer defense |
|
||||
| Registration flow | `RAF/flows/01-registration.md` | TOFU registration |
|
||||
| Command execution | `RAF/flows/02-command-execution.md` | Polling and dispatch |
|
||||
| Agent upgrade | `RAF/flows/03-agent-upgrade.md` | Self-upgrade flow |
|
||||
| Heartbeat | `RAF/flows/04-heartbeat.md` | Health monitoring |
|
||||
| Capability advertisement | `RAF/flows/05-capability-advertisement.md` | Dynamic scanner reporting |
|
||||
| Windows Updates | `RAF/scanners/01-windows-updates.md` | WUA integration |
|
||||
| Docker Scanner | `RAF/scanners/02-docker-scanner.md` | Docker image scanning |
|
||||
| APT Scanner | `RAF/scanners/03-apt-scanner.md` | APT package manager |
|
||||
| DNF Scanner | `RAF/scanners/04-dnf-scanner.md` | DNF package manager |
|
||||
| Process Scanner | `RAF/scanners/05-process-scanner.md` | On-demand /proc scanning |
|
||||
| Process Scan Flow | `RAF/flows/07-process-scan.md` | Process inventory and drill-down |
|
||||
|
||||
---
|
||||
|
||||
## Source Files
|
||||
|
||||
### Server
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server/cmd/server/main.go` | Entry point, route registration |
|
||||
| `server/internal/api/handlers/agents.go` | Agent CRUD, registration, rebind |
|
||||
| `server/internal/api/handlers/subsystems.go` | Subsystem CRUD, enable/disable with scheduler eviction |
|
||||
| `server/internal/api/handlers/agent_updates.go` | Update approval, trigger |
|
||||
| `server/internal/api/handlers/downloads.go` | Binary distribution |
|
||||
| `server/internal/api/handlers/setup.go` | Setup wizard, key generation |
|
||||
| `server/internal/api/middleware/auth.go` | JWT validation |
|
||||
| `server/internal/api/middleware/machine_binding.go` | Hardware verification |
|
||||
| `server/internal/database/db.go` | DB connection, migrations |
|
||||
| `server/internal/database/queries/subsystems.go` | Scanner sync queries |
|
||||
| `server/internal/database/queries/docker.go` | Docker image queries |
|
||||
| `server/internal/services/signing.go` | Ed25519 signing |
|
||||
| `server/internal/services/build_orchestrator.go` | Binary signing |
|
||||
| `server/internal/services/update_nonce.go` | Replay protection (update nonces) |
|
||||
| `server/internal/services/timeout.go` | Timeout configuration |
|
||||
| `server/internal/api/handlers/processes.go` | Process scan endpoints (report, list, detail, trigger) |
|
||||
| `server/internal/database/queries/processes.go` | Process snapshot queries |
|
||||
| `server/internal/models/process.go` | Process data models |
|
||||
| `server/internal/database/migrations/055_create_process_tables.up.sql` | Process tables schema |
|
||||
| `server/internal/scheduler/scheduler.go` | Subsystem job scheduling, priority queue |
|
||||
| `server/internal/scheduler/queue.go` | Priority queue (heap) for scheduled jobs |
|
||||
|
||||
### Agent
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `agent/internal/agent/loop.go` | Polling loop implementation |
|
||||
| `agent/internal/system/machine_id.go` | Machine ID generation |
|
||||
| `agent/internal/registration/service.go` | Agent registration |
|
||||
| `agent/internal/config/config.go` | Config management |
|
||||
| `agent/internal/client/client.go` | API client with retry |
|
||||
| `agent/internal/orchestrator/docker_scanner.go` | Docker scanner + registry client |
|
||||
| `agent/internal/orchestrator/storage_scanner.go` | Storage/disk scanner |
|
||||
| `agent/internal/orchestrator/system_scanner.go` | System metrics scanner |
|
||||
| `agent/internal/orchestrator/orchestrator.go` | Scanner orchestration, circuit breaker integration |
|
||||
| `agent/internal/orchestrator/command_handler.go` | Receipt/ack tracking |
|
||||
| `agent/internal/orchestrator/update_handler.go` | Update handler |
|
||||
| `agent/internal/circuitbreaker/circuitbreaker.go` | Circuit breaker |
|
||||
| `agent/internal/retry/retry.go` | Exponential backoff |
|
||||
| `agent/internal/crypto/pubkey.go` | Public key caching (TOFU) |
|
||||
| `agent/internal/crypto/verification.go` | Signature verification |
|
||||
| `agent/internal/scanner/apt.go` | APT scanner |
|
||||
| `agent/internal/scanner/dnf.go` | DNF scanner |
|
||||
| `agent/pkg/windowsupdate/client.go` | Windows Update client |
|
||||
| `agent/internal/system/machine_id.go` | Machine ID generation (Linux) |
|
||||
|
||||
### Web
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `web/src/App.tsx` | Main app + routing |
|
||||
| `web/src/pages/Agents.tsx` | Agent management |
|
||||
| `web/src/pages/Updates.tsx` | Update management |
|
||||
| `web/src/pages/Settings.tsx` | Settings pages |
|
||||
| `web/src/pages/Dashboard.tsx` | Dashboard |
|
||||
| `web/src/components/AgentHealth.tsx` | Health monitoring |
|
||||
| `web/src/hooks/useCommands.ts` | TanStack Query hook |
|
||||
| `web/src/hooks/useAgents.ts` | Agent data hook |
|
||||
|
||||
---
|
||||
|
||||
## Database Tables
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `agents` | Agent records |
|
||||
| `agents_metadata` | Metadata (available_scanners, heartbeat_source, last_heartbeat) |
|
||||
| `agent_subsystems` | Enabled scanners (idempotent sync) |
|
||||
| `agent_commands` | Pending commands |
|
||||
| `update_logs` | Update execution history |
|
||||
| `system_events` | Operational events (scans, heartbeats) |
|
||||
| `refresh_tokens` | Refresh token lifecycle |
|
||||
| `registration_tokens` | One-time registration tokens |
|
||||
| `server_public_keys` | Ed25519 keys (rotation support) |
|
||||
| `security_settings` | Policy configuration |
|
||||
| `tracked_software` | Agent ↔ tracked_software bindings |
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** File mappings are living documents — update when code changes.
|
||||
|
||||
**Connection:** Core files (`RAF/core/`) provide the foundational principles for all other sections.
|
||||
|
||||
**Connection:** Security files (`RAF/security/`) define trust boundaries that all flows must respect.
|
||||
|
||||
**Connection:** Verification files (`RAF/verification/`) implement the cryptographic guarantees.
|
||||
|
||||
**Connection:** Flow files (`RAF/flows/`) show how components interact end-to-end.
|
||||
|
||||
**Connection:** Scanner files (`RAF/scanners/`) document platform-specific integrations.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
32
RAF/reference/02-glossary.md
Normal file
32
RAF/reference/02-glossary.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Glossary
|
||||
|
||||
**The vocabulary of RedFlag, in one place. Terms link to their design-of-record pages.**
|
||||
|
||||
---
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| **Agent** | Stateless Go executor on each managed host. Polls, verifies, executes, reports. Never decides. [[components/02-agent]] |
|
||||
| **Capability token** | Ed25519-signed grant describing exactly one operation over a resolved closure, with every artifact hash pinned. Minted at approval, executed once. [[security/05-supply-chain-gate]] |
|
||||
| **Closure (dependency closure)** | The full set of artifacts an operation will touch — the named package *and* every transitive dependency resolved at dry-run time. |
|
||||
| **Closure hash** | Canonical hash over the closure, embedded in the token's signed message. Computed byte-identically in Go (server) and Rust (helper) — the cross-language contract. |
|
||||
| **Discovery vs. mutation** | Discovery (scan, dry-run, hash-resolve) runs unprivileged through `DiscoveryRunner`. Mutation happens only through the helper on gated ecosystems. The agent cannot install. |
|
||||
| **Doctrine / doctrinal** | A guarantee with no configuration knob: signing-required, forward-only versioning, no verification skip path. If it's doctrine, there is nothing to misconfigure. [[core/01-ethos]] |
|
||||
| **Drift detection** | Knowing what *should* be installed vs. what *is*, and bridging the gap into update packages. |
|
||||
| **ETHOS** | The five principles every change is held to: errors are history, no unauthenticated endpoints, assume failure, idempotency, no marketing fluff in logs. [[core/01-ethos]] |
|
||||
| **Fail-closed** | When verification can't succeed, the operation doesn't happen. A registered hash that can't be checked blocks; an unknown vulnerability state blocks under `block` enforcement. The opposite of "warn and continue." |
|
||||
| **Family revocation** | Refresh tokens form a lineage (`family_id`); replaying a stale token burns the entire family loudly. Theft is detected, not coexisted with. [[security/03-refresh-tokens]] |
|
||||
| **Forward-only** | No downgrades. Versions move forward; the release gate enforces it; there is no override. |
|
||||
| **Helper** | The privileged, network-less Rust executor — the only mutation path on gated ecosystems. [[components/04-helper]] |
|
||||
| **Legacy command path** | Direct signed-command execution for docker / winget / windows_update — ecosystems the capability gate doesn't cover yet. A documented gap, not a feature. [[OVERVIEW]] |
|
||||
| **Machine binding** | Hardware fingerprint registered at enrollment and checked on every authenticated request, including token renewal. A stolen `config.json` is inert elsewhere. [[security/04-machine-binding]] |
|
||||
| **Nonce** | Per-command signed value with a 10-minute window; agents track executed nonces and reject replays. [[verification/04-replay-protection]] |
|
||||
| **OSV** | OSV.dev, the open vulnerability database. Queried in batches across full closures at detection time; verdicts persist and gate approval. |
|
||||
| **RAF** | This document tree — the RedFlag Architecture Framework, the design of record. What the system is, not what's currently on the task list. |
|
||||
| **Soak gate / age gate** | Time-based supply-chain policies: minimum package age before approval (Shai-Hulud defense) and a version soak window before install. Policies, not doctrine — configurable, with enforcement modes. [[security/05-supply-chain-gate]] |
|
||||
| **TOFU** | Trust-on-first-use: the agent caches the server's public key at first connect and verifies everything after against the cached roster, by `key_id`. [[verification/03-key-rotation]] |
|
||||
| **Two execution paths** | Capability gate (dnf, apt — token + helper) and legacy command (everything else, for now). [[OVERVIEW]] |
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
135
RAF/scanners/01-windows-updates.md
Normal file
135
RAF/scanners/01-windows-updates.md
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
# Windows Updates Scanner
|
||||
|
||||
**Windows Update API integration for detecting and managing Windows updates.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Package | `windowsupdate` (Go) — https://github.com/ceshihao/windowsupdate |
|
||||
| Platform | Windows |
|
||||
| Execution time | ~30 seconds per scan |
|
||||
| Output format | JSON array of update objects |
|
||||
| Failure modes | WUA service unavailable, network timeout, API errors |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/pkg/windowsupdate/client.go`
|
||||
|
||||
```go
|
||||
func (c *Client) ScanUpdates(ctx context.Context) ([]windowsupdate.Update, error) {
|
||||
// 1. Initialize Windows Update API
|
||||
client := windowsupdate.NewClient()
|
||||
client.SetTimeout(30 * time.Second)
|
||||
|
||||
// 2. Query for updates
|
||||
updates, err := client.Update()
|
||||
if err != nil {
|
||||
logSecurityEvent("[reliability] [agent] [windows] WUA query failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 3. Filter ghost packages
|
||||
updates = filterGhostPackages(updates)
|
||||
|
||||
// 4. Return filtered updates
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The Windows scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_windows":
|
||||
updates, err := windowsupdate.ScanWindowsUpdates(ctx, agentID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "windows",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_windows" command
|
||||
↓
|
||||
Agent receives command
|
||||
↓
|
||||
ScanWindowsUpdates() queries Windows Update API
|
||||
↓
|
||||
Filter ghost packages (known issue: Windows occasionally reports stale updates)
|
||||
↓
|
||||
Return updates array
|
||||
↓
|
||||
Agent reports scan results to server
|
||||
↓
|
||||
Server displays updates in dashboard (SystemEvents table)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
### Ghost Packages
|
||||
|
||||
**Problem:** Windows Update API sometimes reports packages that are already installed or no longer available.
|
||||
|
||||
**Detection:** Windows Update API doesn't provide an easy way to filter these. Workaround: maintain a cache of previously seen update KB numbers and filter out duplicates.
|
||||
|
||||
**Status:** Partial fix — filtered in `filterGhostPackages()` but may not catch all cases.
|
||||
|
||||
**Cross-references:**
|
||||
- `testing/02-windows-ghost.md` (ghost package test coverage)
|
||||
|
||||
---
|
||||
|
||||
### Update Reappearance
|
||||
|
||||
**Problem:** Some Windows Updates may reappear after installation (known Windows Update quirk).
|
||||
|
||||
**Status:** Known issue — logged but not automatically handled. Requires manual intervention or future fix.
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Windows update scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents Windows scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** Windows scanner (`scanners/01-windows-updates.md`) lives in `agent/pkg/windowsupdate/` package.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
111
RAF/scanners/02-docker-scanner.md
Normal file
111
RAF/scanners/02-docker-scanner.md
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
# Docker Scanner
|
||||
|
||||
**Docker image scanning for container-based agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | Read `/var/run/docker.sock` via Docker SDK |
|
||||
| Platform | Linux (Docker agents) |
|
||||
| Execution time | ~5 seconds per scan |
|
||||
| Output format | `[]client.UpdateReportItem` with full metadata in `Metadata` map |
|
||||
| Failure modes | Docker daemon unavailable, socket permission denied, registry rate-limited |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/orchestrator/docker_scanner.go`
|
||||
|
||||
The Docker scanner connects directly to the local Docker daemon via the Docker SDK, lists all containers, inspects each image, then queries the remote registry (Docker Hub or custom) to compare digests.
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/docker_scanner.go
|
||||
func (s *DockerScanner) Scan() ([]client.UpdateReportItem, error) {
|
||||
containers, err := s.client.ContainerList(ctx, container.ListOptions{All: true})
|
||||
// ... inspect each image, compare local vs remote digest ...
|
||||
items = append(items, client.UpdateReportItem{
|
||||
PackageType: "docker_image",
|
||||
PackageName: imageName,
|
||||
CurrentVersion: localShortDigest,
|
||||
AvailableVersion: remoteShortDigest,
|
||||
Severity: severity,
|
||||
RepositorySource: baseImage,
|
||||
Metadata: map[string]interface{}{
|
||||
"has_update": hasUpdate,
|
||||
"image_id": localShortDigest,
|
||||
"latest_image_id": remoteShortDigest,
|
||||
// ... container info, labels, etc.
|
||||
},
|
||||
})
|
||||
return items, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Registry Client
|
||||
|
||||
The `RegistryClient` within `docker_scanner.go` handles:
|
||||
- Docker Hub token authentication (`auth.docker.io`)
|
||||
- Registry API v2 manifest fetch via `Docker-Content-Digest` header
|
||||
- 5-minute TTL cache to avoid rate limits
|
||||
- Custom registry support (gcr.io, etc.) via domain detection in `parseImageName()`
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Orchestrator Registration
|
||||
|
||||
Registered in `agent/internal/agent/loop.go` as a direct `orchestrator.Scanner` implementation:
|
||||
|
||||
```go
|
||||
dockerScanner, _ := orchestrator.NewDockerScanner()
|
||||
scanOrchestrator.RegisterScanner("docker", dockerScanner, dockerCB, ...)
|
||||
```
|
||||
|
||||
### 2. Command Dispatch
|
||||
|
||||
The `HandleScanDocker` handler in `agent/internal/handlers/scan.go` runs the scan once through the orchestrator and reads results from `result.Updates[]` — no double-scanning.
|
||||
|
||||
### 3. Server-Side Storage
|
||||
|
||||
Scan results are reported via `client.ReportDockerImages()` to `POST /api/v1/agents/:id/docker-images`. The server stores them in `docker_images` table via `server/internal/database/queries/docker.go` (`CreateDockerEventsBatch`, `GetDockerImages`, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_docker" command
|
||||
↓
|
||||
Agent receives command → orchestrator.ScanSingle("docker")
|
||||
↓
|
||||
DockerScanner.Scan() connects to local Docker daemon
|
||||
↓
|
||||
List containers, inspect images, check registry digests
|
||||
↓
|
||||
Return UpdateReportItems with full metadata
|
||||
↓
|
||||
Handler reports results to server API endpoint
|
||||
↓
|
||||
Server stores in docker_images table
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Docker daemon is available on the agent host (`/var/run/docker.sock`). Registry access requires outbound internet (or mirror configuration).
|
||||
|
||||
**Connection:** Scanner registration (`agent/internal/agent/loop.go`) wires Docker into the orchestrator alongside APT, DNF, Windows, and Winget scanners — all now implement `orchestrator.Scanner` directly (no wrapper layer).
|
||||
|
||||
**Connection:** Registry client (`orchestrator/docker_scanner.go`) uses ETHOS `[TAG] [system] [component]` logging for registry failures.
|
||||
|
||||
**Connection:** Server-side `docker_images` table (`server/internal/database/queries/docker.go`) stores reported scan results for dashboard display.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-27*
|
||||
133
RAF/scanners/03-apt-scanner.md
Normal file
133
RAF/scanners/03-apt-scanner.md
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
# APT Scanner
|
||||
|
||||
**APT package manager scanning for Debian/Ubuntu-based Linux agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | `apt list --upgradable -o APT::Get::List-Cleanup=false` |
|
||||
| Platform | Linux (Debian, Ubuntu) |
|
||||
| Execution time | ~10 seconds per scan |
|
||||
| Output format | JSON array of package objects with version info |
|
||||
| Failure modes | APT lock held, network timeout, permission denied |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/scanner/apt.go`
|
||||
|
||||
```go
|
||||
func ScanAPT(ctx context.Context) ([]APTUpdate, error) {
|
||||
// 1. Run apt list --upgradable
|
||||
cmd := exec.CommandContext(ctx, "bash", "-c",
|
||||
"apt list --upgradable -o APT::Get::List-Cleanup=false 2>&1")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
logSecurityEvent("[reliability] [agent] [apt] apt list failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2. Parse output (format: "package/old_version -> new_version")
|
||||
var updates []APTUpdate
|
||||
lines := strings.Split(string(output), "\n")
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "Listing...") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse "package/old_version -> new_version"
|
||||
parts := strings.Split(line, "->")
|
||||
if len(parts) != 2 {
|
||||
continue
|
||||
}
|
||||
|
||||
pkgInfo := strings.Split(strings.TrimSpace(parts[0]), "/")
|
||||
if len(pkgInfo) != 2 {
|
||||
continue
|
||||
}
|
||||
|
||||
updates = append(updates, APTUpdate{
|
||||
Package: pkgInfo[0],
|
||||
OldVersion: pkgInfo[1],
|
||||
NewVersion: strings.TrimSpace(parts[1]),
|
||||
})
|
||||
}
|
||||
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The APT scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_apt":
|
||||
updates, err := apt.ScanAPT(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "apt",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_apt" command
|
||||
↓
|
||||
Agent receives command
|
||||
↓
|
||||
ScanAPT() runs apt list --upgradable
|
||||
↓
|
||||
Parse output and extract package/version info
|
||||
↓
|
||||
Return updates array
|
||||
↓
|
||||
Agent reports scan results to server
|
||||
↓
|
||||
Server displays updates in dashboard
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** APT scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents APT scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** APT scanner (`scanners/03-apt-scanner.md`) lives in `agent/internal/scanner/apt.go`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
134
RAF/scanners/04-dnf-scanner.md
Normal file
134
RAF/scanners/04-dnf-scanner.md
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
# DNF Scanner
|
||||
|
||||
**DNF package manager scanning for Fedora/RHEL-based Linux agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | `dnf check-update --refresh` |
|
||||
| Platform | Linux (Fedora, RHEL, CentOS) |
|
||||
| Execution time | ~15 seconds per scan |
|
||||
| Output format | JSON array of package objects with version info |
|
||||
| Failure modes | DNF lock held, network timeout, permission denied |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/scanner/dnf.go`
|
||||
|
||||
```go
|
||||
func ScanDNF(ctx context.Context) ([]DNFUpdate, error) {
|
||||
// 1. Run dnf check-update --refresh
|
||||
cmd := exec.CommandContext(ctx, "dnf", "check-update", "--refresh")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
// Check if it's a no-update case (exit code 100)
|
||||
if exitErr, ok := err.(*exec.ExitError); ok {
|
||||
if exitErr.ExitCode() == 100 {
|
||||
return []DNFUpdate{}, nil
|
||||
}
|
||||
}
|
||||
logSecurityEvent("[reliability] [agent] [dnf] dnf check-update failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2. Parse output
|
||||
var updates []DNFUpdate
|
||||
lines := strings.Split(string(output), "\n")
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "Last metadata expiration check") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse "package-name.old_version.new_version"
|
||||
parts := strings.Fields(line)
|
||||
if len(parts) < 3 {
|
||||
continue
|
||||
}
|
||||
|
||||
pkg := parts[0]
|
||||
oldVersion := parts[1]
|
||||
newVersion := parts[2]
|
||||
|
||||
updates = append(updates, DNFUpdate{
|
||||
Package: pkg,
|
||||
OldVersion: oldVersion,
|
||||
NewVersion: newVersion,
|
||||
})
|
||||
}
|
||||
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The DNF scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_dnf":
|
||||
updates, err := dnf.ScanDNF(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "dnf",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Systemd Integration
|
||||
|
||||
The DNF scanner requires two paths writable under `ProtectSystem=strict`:
|
||||
|
||||
- **`/var/log`** — dnf5 writes `/var/log/dnf5.log`
|
||||
- **`/var/cache`** — dnf5 creates temp files at `/var/cache/libdnf5/`
|
||||
|
||||
**Files to update when locking down a new agent:**
|
||||
- Live systemd unit: `/etc/systemd/system/redflag-agent.service` → add to `ReadWritePaths`
|
||||
- Installer template: `agent/internal/installer/sudoers.go:CreateSystemdService()` → same change
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #1: Errors are History — log writes must succeed)
|
||||
- `agent/internal/installer/sudoers.go` (systemd service template)
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** DNF scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents DNF scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** DNF scanner (`scanners/04-dnf-scanner.md`) lives in `agent/internal/scanner/dnf.go`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
138
RAF/scanners/05-process-scanner.md
Normal file
138
RAF/scanners/05-process-scanner.md
Normal file
|
|
@ -0,0 +1,138 @@
|
|||
# Process Scanner
|
||||
|
||||
**On-demand /proc filesystem scanning for process inventory and drill-down detail.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | Direct `/proc` filesystem reads (no subprocess spawns) |
|
||||
| Platform | Linux only (stub on other platforms) |
|
||||
| Execution time | ~200ms for 200-process snapshot; ~50ms per drill-down |
|
||||
| Trigger | On-demand when user opens the Processes tab in the dashboard |
|
||||
| Data model | 25+ fields per process (osquery parity) + 7 related data types |
|
||||
| Storage | Dedicated tables: `agent_process_snapshots`, `agent_processes`, `agent_process_related` |
|
||||
| Retention | Last 10 snapshots per agent (auto-cleanup) |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
The process scanner follows the existing command-dispatch pattern:
|
||||
|
||||
1. **Dashboard** opens Processes tab → `POST /api/v1/agents/:id/processes/scan`
|
||||
2. **Server** creates a signed `scan_processes` command (with dedup check)
|
||||
3. **Agent** polls for commands, receives `scan_processes`, calls `system.GetFullProcessSnapshot()`
|
||||
4. **Agent** reports snapshot to `POST /api/v1/agents/:id/process-scan`
|
||||
5. **Server** stores snapshot + processes + related data, cleans up old snapshots
|
||||
6. **Dashboard** reads latest snapshot via `GET /api/v1/agents/:id/processes`
|
||||
|
||||
On drill-down (clicking a process row):
|
||||
1. **Dashboard** requests `GET /api/v1/agents/:id/processes/:processId`
|
||||
2. **Server** returns process + all related data (open files, sockets, pipes, env, memory map, namespaces, listening ports)
|
||||
|
||||
---
|
||||
|
||||
## Data Collection
|
||||
|
||||
### List Scan (`GetFullProcessSnapshot`)
|
||||
|
||||
Reads `/proc/[pid]/stat`, `/proc/[pid]/status`, `/proc/[pid]/exe`, `/proc/[pid]/cmdline`, `/proc/[pid]/cwd`, `/proc/[pid]/io` for every PID. No related data collected at this stage.
|
||||
|
||||
**Fields (25+):** PID, Name, Path, Cmdline, Cwd, State, UID, GID, EUID, EGID, User, Group, TTY, TTYName, CPUSecondsUser, CPUSecondsSystem, CPUPercent, RSSBytes, VMSBytes, MemPercent, Threads, Nice, StartTimeSeconds, ParentPID, ProcessGroupID, ElevationStatus, OnDisk, DiskBytesRead, DiskBytesWritten
|
||||
|
||||
### Drill-Down (`GetProcessDetail`)
|
||||
|
||||
Adds related data from a single `/proc/[pid]/fd/` walk (consolidated from three separate traversals):
|
||||
|
||||
| Data Type | Source | Cap (configurable) |
|
||||
|-----------|--------|---------------------|
|
||||
| Open files | `/proc/[pid]/fd/` symlink targets | `max_open_files` (default 2000) |
|
||||
| Open sockets | `/proc/[pid]/net/tcp`, `tcp6`, `unix` | `max_sockets` (default 500) |
|
||||
| Open pipes | `/proc/[pid]/fd/` pipe inodes | `max_pipes` (default 500) |
|
||||
| Environment keys | `/proc/[pid]/environ` (keys only, no values) | `max_env_keys` (default 200) |
|
||||
| Memory map | `/proc/[pid]/maps` | `max_memory_map` (default 2000) |
|
||||
| Namespaces | `/proc/[pid]/ns/` symlinks | `max_namespaces` (default 50) |
|
||||
| Listening ports | Socket inode correlation with `/proc/net/tcp` | `max_listening_ports` (default 100) |
|
||||
|
||||
### Key Implementation Detail: Socket Inode Correlation
|
||||
|
||||
Listening ports are per-process, not system-wide. The scanner collects socket inodes from `/proc/[pid]/fd/` symlinks (`socket:[12345]`), then matches them against inode numbers in `/proc/net/tcp` and `/proc/net/tcp6`. Only LISTEN state (0A) entries whose inode matches a process socket are included.
|
||||
|
||||
### Key Implementation Detail: IPv6 Address Parsing
|
||||
|
||||
The kernel stores IPv6 addresses in `/proc/net/tcp6` as 4 little-endian 32-bit words (`%08X%08X%08X%08X`). The parser reverses bytes within each 4-byte group (not across the entire 16-byte address) and uses `%02x` for zero-padded output.
|
||||
|
||||
### Key Implementation Detail: `/proc/[pid]/stat` Field Indices
|
||||
|
||||
After stripping `pid (comm)`, the `fields` array is 0-indexed from field 3:
|
||||
- `[0]`=state, `[1]`=ppid, `[2]`=pgrp, `[3]`=session, `[4]`=tty_nr
|
||||
- `[11]`=utime, `[12]`=stime, `[16]`=nice, `[19]`=starttime
|
||||
|
||||
---
|
||||
|
||||
## Configurable Caps
|
||||
|
||||
Data collection limits are server-controlled via `ProcessExplorerConfig` (stored in security settings under `operational` category). Delivered to agents on check-in. Set to 0 for no cap.
|
||||
|
||||
| Setting | Default | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `process_explorer_max_open_files` | 2000 | Open file descriptors per process |
|
||||
| `process_explorer_max_sockets` | 500 | Open sockets per process |
|
||||
| `process_explorer_max_pipes` | 500 | Open pipes per process |
|
||||
| `process_explorer_max_memory_map` | 2000 | Memory map entries per process |
|
||||
| `process_explorer_max_namespaces` | 50 | Namespace entries per process |
|
||||
| `process_explorer_max_env_keys` | 200 | Environment variable keys per process |
|
||||
| `process_explorer_max_listening_ports` | 100 | TCP listening ports per process |
|
||||
|
||||
**UI:** Settings → Process Explorer (`/settings/process-explorer`)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Files
|
||||
|
||||
### Agent
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `agent/internal/system/process_detail.go` | Types: `FullProcess`, `ProcessOpenFile`, `ProcessOpenSocket`, `ProcessOpenPipe`, `ProcessMemoryMap`, `ProcessNamespace`, `ProcessListeningPort`, `ProcessCaps`, `FullProcessSnapshot` |
|
||||
| `agent/internal/system/process_detail_linux.go` | Linux `/proc` reader: `getFullProcessSnapshot()`, `getProcessDetail()`, `walkProcFD()`, `readListeningPorts()`, `parseHexAddr()` |
|
||||
| `agent/internal/system/process_detail_other.go` | Stub for non-Linux platforms |
|
||||
| `agent/internal/handlers/processes.go` | `HandleScanProcesses` — command handler |
|
||||
| `agent/internal/client/client.go` | `ReportProcessScan` — reports to server |
|
||||
|
||||
### Server
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server/internal/database/migrations/055_create_process_tables.up.sql` | Schema: `agent_process_snapshots`, `agent_processes`, `agent_process_related` |
|
||||
| `server/internal/models/process.go` | Server-side models |
|
||||
| `server/internal/database/queries/processes.go` | `ProcessQueries` — insert, query, cleanup |
|
||||
| `server/internal/api/handlers/processes.go` | `ProcessHandler` — 4 endpoints |
|
||||
| `server/internal/models/command.go` | `CommandTypeScanProcesses` constant |
|
||||
|
||||
### Web
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `web/src/types/process.ts` | TypeScript interfaces |
|
||||
| `web/src/hooks/useProcesses.ts` | React Query hooks |
|
||||
| `web/src/components/ProcessesTab.tsx` | Main tab with sortable table |
|
||||
| `web/src/components/ProcessDetailModal.tsx` | 6-tab modal (Overview, Network, Files, Environment, Memory, Namespaces) |
|
||||
| `web/src/hooks/useProcessExplorer.ts` | Settings hooks |
|
||||
| `web/src/pages/settings/ProcessExplorer.tsx` | Settings UI |
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Environment variable values are never transmitted.** Only key names are collected. Env vars may contain secrets (API keys, database passwords).
|
||||
- **No subprocess spawns.** All data comes from direct `/proc` reads — no `ps`, `lsof`, or similar commands.
|
||||
- **On-demand only.** The scan command is only issued when a user opens the Processes tab. No background broadcasting.
|
||||
- **Command dedup.** The server checks for existing pending `scan_processes` commands before creating a new one.
|
||||
|
||||
---
|
||||
|
||||
*Added: 2026-06-10*
|
||||
178
RAF/security/01-trust-boundaries.md
Normal file
178
RAF/security/01-trust-boundaries.md
Normal file
|
|
@ -0,0 +1,178 @@
|
|||
# Trust Boundaries
|
||||
|
||||
**Complete trust boundary matrix for all endpoints.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Every HTTP endpoint must be classified by who is allowed to call it and which middleware enforces that classification.
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/02-authentication-stack]] (auth layers)
|
||||
- [[security/03-refresh-tokens]] (token lifecycle)
|
||||
- [[security/04-machine-binding]] (machine ID binding)
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary Matrix
|
||||
|
||||
| Trust Boundary | Group | Middleware | Example Routes | Notes |
|
||||
|----------------|-------|------------|----------------|-------|
|
||||
| **Public** | `public` | None | `GET /api/v1/install/:platform` | Rate-limited per-IP |
|
||||
| **Public** | `public` | None | `GET /api/v1/downloads/:platform` | Rate-limited per-IP, no signature when version="latest" |
|
||||
| **Public** | `public` | None | `POST /api/v1/agents/register` | Uses registration token |
|
||||
| **Public** | `public` | In-handler machine binding | `POST /api/v1/agents/renew` | Refresh token **+ X-Machine-ID must match the registered host**. Refresh token presented from a different machine → 403 (stolen-token replay defense, logged as machine_id_mismatch). |
|
||||
| **Agent** | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `GET /api/v1/agents/:id/commands` | Requires JWT + correct machine ID |
|
||||
| **Agent** | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `POST /api/v1/agents/:id/reports` | Requires JWT + correct machine ID |
|
||||
| **Agent** | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `POST /api/v1/agents/:id/logs` | Requires JWT + correct machine ID |
|
||||
| **Agent** | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `POST /api/v1/agents/:id/rebind-machine-id` | Admin-initiated machine rebind |
|
||||
| **Agent** | `agent-auth` | `AuthMiddleware + MachineBindingMiddleware` | `GET /api/v1/downloads/updates/:package_id` | Download signed agent packages |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/dashboard/*` | Admin dashboard |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/agents/*` | Agent management |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/settings/*` | Settings pages |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/updates/*` | Update management |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/docker/*` | Docker integration |
|
||||
| **Web** | `web-auth` | `WebAuthMiddleware` | `GET /api/v1/history/*` | History tracking |
|
||||
| **Admin** | `admin-only` | `WebAuthMiddleware + RequireAdmin()` | `POST /api/v1/admin/*` | Admin-only operations |
|
||||
| **Admin** | `admin-only` | `WebAuthMiddleware + RequireAdmin()` | `DELETE /api/v1/admin/agents/:id` | Delete agent (BUG-013 fix) |
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary Details
|
||||
|
||||
### Public Trust Boundary
|
||||
|
||||
**Endpoints:**
|
||||
- `GET /api/v1/install/:platform?token=<reg_token>&arch=<arch>`
|
||||
- `GET /api/v1/downloads/:platform?version=<ver>`
|
||||
- `POST /api/v1/agents/register`
|
||||
- `POST /api/v1/agents/renew`
|
||||
|
||||
**Security Notes:**
|
||||
- Rate-limited per-IP via `RateLimit("public_access", KeyByIP)`
|
||||
- Registration tokens are one-time use
|
||||
- Download endpoint has BUG-003: signature header not set when `version="latest"`
|
||||
- `/renew` is deliberately on the public route group, not behind `AuthMiddleware`: the agent calls it *because* its JWT has expired, so requiring a valid JWT to renew would be circular. It authenticates with the refresh token in the body instead. It is **not** trust-free, though — the handler reloads the agent and requires `X-Machine-ID` to match the bound machine. This closes the gap where a refresh token (a long-lived on-disk secret) would otherwise mint access tokens from any machine for 90 days.
|
||||
|
||||
**Refresh-token rotation + reuse detection — IMPLEMENTED (migration 045, 2026-05-29).** Each refresh token belongs to a *family* (`family_id`) and carries `consumed_at` + `superseded_by`. The state machine in `RenewToken` (`server/internal/api/handlers/agents.go`):
|
||||
|
||||
| Presented token state | Action |
|
||||
|---|---|
|
||||
| not found | 401 invalid |
|
||||
| revoked, family still live | **revoke family** + security event → 401 (revoked-token replay is anomalous) |
|
||||
| expired | 401 (bounded by the token's own 90d window) |
|
||||
| unconsumed (`consumed_at IS NULL`) | normal rotation: mint successor, mark parent consumed, return new token |
|
||||
| consumed, successor **unconsumed** | **accept-previous-once** grace: agent crashed before saving the successor (provably never used it) → orphan that leaf, mint a fresh one, return it |
|
||||
| consumed, successor **consumed/revoked/missing** | **reuse detected** → revoke family + security event → 401 |
|
||||
|
||||
Grace is *structural*, not timed: "successor still unconsumed" is the discriminator, bounded by the parent's own 90d expiry. The agent persists the rotated token on each renewal (`loop.go`); a failed persist is recovered by the grace path on the next attempt. All revoke/reuse paths are loud (`LogUnauthorizedAccessAttempt`) and fail-closed — both the legitimate agent and any thief lose access, forcing deliberate human re-registration.
|
||||
|
||||
**Residual limitation (documented, not a TODO):** a *perfect same-machine lockstep shadow* — an attacker on the bound host who reads `config.json` and renews in exact alternation with the legit agent — is not detectable by rotation alone, because every token is used exactly once per party and the chain never diverges. This is inherent to all refresh-token rotation. It is mitigated by machine binding (the outer gate: a different host → 403 before rotation runs) and is out of scope for rotation; a same-host root attacker has already won at the OS layer.
|
||||
|
||||
**Still human-gated (unchanged):** **admin import / re-grant** — a one-time token authorizing exactly one rebind/registration when an operator moves an agent's identity to new hardware. Deliberately not automated (Casey: "human gated today" — revisit only if/when agent sophistication warrants, ~not near-term). Distinct from accept-previous-once, which is automatic crash-recovery internal to rotation.
|
||||
|
||||
Ties to SEC-012 (renewal atomicity): the server side is now fully transactional; the agent↔server two-phase (server commits rotation / agent persists token) is reconciled by the grace path rather than true cross-network atomicity.
|
||||
|
||||
**Cross-references:**
|
||||
- [[flows/01-registration]] (registration flow)
|
||||
- [[flows/03-agent-upgrade]] (download endpoint)
|
||||
|
||||
---
|
||||
|
||||
### Agent Trust Boundary
|
||||
|
||||
**Middleware Chain:**
|
||||
1. `AuthMiddleware` — Validates JWT with issuer `"redflag-agent"`
|
||||
2. `MachineBindingMiddleware` — Validates X-Machine-ID matches DB
|
||||
|
||||
**Security Notes:**
|
||||
- JWT expires after 24 hours
|
||||
- Refresh token extends expiry to 90 days
|
||||
- Machine ID mismatch returns 403 Forbidden
|
||||
- Agent row deletion returns 401 Unauthorized
|
||||
- **Download endpoint** (`GET /api/v1/downloads/updates/:package_id`) requires machine binding to prevent any authenticated agent from downloading any package
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/02-authentication-stack]] (JWT validation)
|
||||
- [[security/04-machine-binding]] (machine ID validation)
|
||||
|
||||
---
|
||||
|
||||
### Web Trust Boundary
|
||||
|
||||
**Middleware:** `WebAuthMiddleware` — Validates JWT with issuer `"redflag-web"`
|
||||
|
||||
**Security Notes:**
|
||||
- JWT expires after 24 hours
|
||||
- Requires `admin` role claim
|
||||
- Admin-only routes use `AdminRoleMiddleware`
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/02-authentication-stack]] (web JWT)
|
||||
|
||||
---
|
||||
|
||||
### Admin Trust Boundary
|
||||
|
||||
**Middleware Chain:**
|
||||
1. `WebAuthMiddleware` — Validates JWT with issuer `"redflag-web"`
|
||||
2. `RequireAdmin()` — Checks `admin` claim is true
|
||||
|
||||
**Security Notes:**
|
||||
- Can delete agents (was BUG-013: was registered under agent-auth group)
|
||||
- Can revoke agent tokens
|
||||
- Can trigger machine rebind
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/03-refresh-tokens]] (token revocation)
|
||||
|
||||
---
|
||||
|
||||
## Anti-Pattern: BUG-013
|
||||
|
||||
**Problem:** `DELETE /api/v1/agents/:id` was an admin operation registered under the agent-auth group with `MachineBindingMiddleware`.
|
||||
|
||||
**Symptom:** Admin request returned 401 Unauthorized (missing X-Machine-ID) because admin JWT doesn't have machine binding.
|
||||
|
||||
**Fix:** Moved endpoint to `admin-only` group with `WebAuthMiddleware + AdminRoleMiddleware`.
|
||||
|
||||
**Cross-references:**
|
||||
- [[core/01-ethos]] (principle #2: Security is Non-Negotiable)
|
||||
- [[flows/02-command-execution]] (polling loop)
|
||||
|
||||
---
|
||||
|
||||
## Doctrine: Pull-Only Agent Channel
|
||||
|
||||
The agent↔server control channel is **pull-only**. The agent polls; the server never
|
||||
opens a connection to an agent and never pushes commands at one. This is doctrine
|
||||
(Casey, 2026-06-10), not a configuration choice — same tier as signing-required and
|
||||
forward-only.
|
||||
|
||||
**Why:** a push channel is a standing inbound control path on every endpoint. Pull keeps
|
||||
the agent in charge of when it listens, and keeps the server compromise blast radius
|
||||
bounded by what agents choose to fetch and verify.
|
||||
|
||||
**Consequences:**
|
||||
- Reject designs that assume server-initiated delivery: live-query campaigns,
|
||||
push-config, server-side websockets to agents.
|
||||
- A websocket/push channel is a *maybe later*, and **not until it is PQC-ready**
|
||||
(post-quantum cryptography). Until then, latency wants are served by rapid-mode polling.
|
||||
- Server→*operator-owned third party* outbound emits (SIEM, asset DB — see
|
||||
`docs/tasks/INTEG-001`, `INTEG-002`) are a different channel and unaffected: outbound,
|
||||
no listener, no control surface.
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Auth layers** → [[security/02-authentication-stack]]
|
||||
- **Refresh tokens** → [[security/03-refresh-tokens]]
|
||||
- **Machine binding** → [[security/04-machine-binding]]
|
||||
- **Registration** → [[flows/01-registration]]
|
||||
- **Command execution** → [[flows/02-command-execution]]
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
208
RAF/security/02-authentication-stack.md
Normal file
208
RAF/security/02-authentication-stack.md
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
# Authentication Stack
|
||||
|
||||
**Four-layer authentication: registration tokens → JWT → refresh tokens → machine binding.**
|
||||
|
||||
---
|
||||
|
||||
## Layer 1: Registration Tokens
|
||||
|
||||
**Purpose:** One-time enrollment tokens
|
||||
|
||||
**Format:** Random 64-character hex string
|
||||
|
||||
**Lifecycle:**
|
||||
1. Server generates token with `max_seats` count
|
||||
2. Admin distributes token to operators
|
||||
3. Agent uses token to register
|
||||
4. Server marks token as used and increments `seats_used`
|
||||
5. Token is revoked after use
|
||||
|
||||
**Endpoint:** `POST /api/v1/agents/register`
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"hostname": "server-01",
|
||||
"os_type": "linux",
|
||||
"os_version": "6.19",
|
||||
"machine_id": "sha256-fingerprint...",
|
||||
"public_key": "ed25519-public-key...",
|
||||
"available_scanners": ["apt", "dnf", "docker"]
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"agent_id": "uuid-4",
|
||||
"jwt_token": "...",
|
||||
"refresh_token": "...",
|
||||
"server_public_key": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- [[flows/01-registration]] (registration flow)
|
||||
- [[security/03-refresh-tokens]] (refresh tokens)
|
||||
|
||||
---
|
||||
|
||||
## Layer 2: JWT Access Tokens
|
||||
|
||||
**Purpose:** Short-lived access tokens for API calls
|
||||
|
||||
**Issuer:** `"redflag-agent"` for agents, `"redflag-web"` for web
|
||||
|
||||
**Duration:** 24 hours
|
||||
|
||||
**Algorithm:** HS256
|
||||
|
||||
**Claims:**
|
||||
```json
|
||||
{
|
||||
"sub": "agent-uuid",
|
||||
"iss": "redflag-agent",
|
||||
"exp": 1234567890,
|
||||
"iat": 1234567890
|
||||
}
|
||||
```
|
||||
|
||||
**Validation:**
|
||||
```go
|
||||
func AuthMiddleware() gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
authHeader := c.GetHeader("Authorization")
|
||||
if authHeader == "" {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "missing authorization header"})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
|
||||
tokenString := strings.TrimPrefix(authHeader, "Bearer ")
|
||||
if tokenString == authHeader {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid authorization format"})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
|
||||
token, err := jwt.ParseWithClaims(tokenString, &AgentClaims{}, func(token *jwt.Token) (interface{}, error) {
|
||||
return []byte(JWTSecret), nil
|
||||
})
|
||||
|
||||
if err != nil || !token.Valid {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid token"})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
|
||||
if claims, ok := token.Claims.(*AgentClaims); ok {
|
||||
// Validate issuer to prevent cross-type token confusion
|
||||
if claims.Issuer != "" && claims.Issuer != JWTIssuerAgent {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid token type"})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
c.Set("agent_id", claims.AgentID)
|
||||
c.Next()
|
||||
} else {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid token claims"})
|
||||
c.Abort()
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/01-trust-boundaries]] (trust boundary matrix)
|
||||
- [[flows/02-command-execution]] (polling loop)
|
||||
|
||||
---
|
||||
|
||||
## Layer 3: Refresh Tokens
|
||||
|
||||
**Purpose:** Long-lived authentication for polling, with rotation and reuse detection
|
||||
|
||||
**Format:** 64-character hex string (32 bytes crypto/rand)
|
||||
|
||||
**Storage:** SHA-256 hash in database; rotation lineage via `family_id`, `consumed_at`, `superseded_by` (migration 045)
|
||||
|
||||
**Duration:** 90 days (bumped on each renewal, not on every check-in)
|
||||
|
||||
**Lifecycle:**
|
||||
1. Generated at registration with a fresh `family_id` (root of the rotation chain)
|
||||
2. Agent calls `POST /renew` only when its JWT expires (~24h) — not on every poll
|
||||
3. Each renewal mints a **successor** token in the same family and marks the parent `consumed`
|
||||
4. Server returns the new refresh token alongside the new JWT; agent persists it to `config.json`
|
||||
5. Accept-previous-once grace: an agent that crashed before persisting the new token can retry with the consumed old one — the server sees the successor is still unconsumed and re-issues
|
||||
6. Reuse detection: a consumed token presented after its successor is also consumed → entire family revoked, security event logged, both parties locked out
|
||||
|
||||
**Machine binding:** Renewal requires `X-Machine-ID` to match the registered host (same as command endpoints). A stolen `config.json` cannot mint access tokens from an unregistered machine.
|
||||
|
||||
**Instance lock:** A `flock` (Unix) or named kernel mutex (Windows) prevents two agent processes from sharing the same `config.json` on the same host, serializing renewal at the process level.
|
||||
|
||||
**Endpoint:** `POST /api/v1/agents/renew`
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"agent_id": "uuid",
|
||||
"refresh_token": "64-char-hex",
|
||||
"agent_version": "0.2.0.7"
|
||||
}
|
||||
```
|
||||
Headers: `X-Machine-ID` (required), `Content-Type: application/json`
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"token": "new-jwt...",
|
||||
"refresh_token": "new-64-char-hex..."
|
||||
}
|
||||
```
|
||||
The agent must persist `refresh_token` to disk; if it crashes before doing so, accept-previous-once grace recovers on the next attempt.
|
||||
|
||||
**Cross-references:**
|
||||
- [[flows/01-registration]] (registration flow)
|
||||
- [[flows/02-command-execution]] (renewal in polling loop)
|
||||
|
||||
---
|
||||
|
||||
## Layer 4: Machine Binding
|
||||
|
||||
**Purpose:** Bind JWT to specific hardware
|
||||
|
||||
**Method:** SHA-256 hash of machine-id + hostname (no boot-id)
|
||||
|
||||
**Validation:** Middleware checks `X-Machine-ID` header matches DB
|
||||
|
||||
**Failure modes:**
|
||||
- Machine ID mismatch → 403 Forbidden
|
||||
- Agent row deleted → 401 Unauthorized
|
||||
- Update in progress → Validates nonce
|
||||
|
||||
**Middleware:** `server/internal/api/middleware/machine_binding.go`
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/01-trust-boundaries]] (trust boundary matrix)
|
||||
- [[flows/01-registration]] (machine ID generation)
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Trust On First Use (TOFU) model — agent caches server public key at registration and uses it for all future verification.
|
||||
|
||||
**Connection:** [[security/01-trust-boundaries]] (trust boundary matrix)
|
||||
|
||||
**Connection:** [[security/04-machine-binding]] (hardware-bound auth)
|
||||
|
||||
**Connection:** [[verification/01-signing-pipeline]] (Ed25519 signing)
|
||||
|
||||
**Connection:** [[verification/02-agent-verification]] (command verification)
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
70
RAF/security/03-refresh-tokens.md
Normal file
70
RAF/security/03-refresh-tokens.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# Refresh Token Lifecycle
|
||||
|
||||
**Forward-only token rotation with family revocation — a stolen config is a dead config.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agents authenticate with short-lived JWTs minted against a long-lived refresh token (90-day TTL). Every renewal *rotates* the refresh token: a successor is minted, the old token is marked consumed. The rotation lineage is the security mechanism — replaying a consumed token is how theft announces itself.
|
||||
|
||||
**Cross-references:**
|
||||
- [[security/02-authentication-stack]] (where this sits in the four-layer stack)
|
||||
- [[security/04-machine-binding]] (the renewal endpoint is machine-bound)
|
||||
|
||||
---
|
||||
|
||||
## Data Model
|
||||
|
||||
Migration 045. Each token row carries:
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `family_id` | Lineage identifier — all rotations of one registration share it |
|
||||
| `consumed_at` | Set when the token is exchanged for a successor |
|
||||
| `superseded_by` | Points at the successor token |
|
||||
|
||||
Tokens are stored hashed (`HashRefreshToken`), never plaintext. Queries live in `server/internal/database/queries/refresh_tokens.go`.
|
||||
|
||||
---
|
||||
|
||||
## The Renewal Flow
|
||||
|
||||
`RenewToken` (`server/internal/api/handlers/agents.go`):
|
||||
|
||||
1. **Machine binding first.** `X-Machine-ID` is checked against the registered host before any token logic. Mismatch → `403` + `MACHINE_ID_MISMATCH` security event. A stolen `config.json` replayed from another machine never reaches rotation.
|
||||
2. **Locked read.** `GetRefreshTokenForRenew` uses `SELECT ... FOR UPDATE` — two concurrent renewals with the same token cannot both succeed.
|
||||
3. **Classify the presented token:**
|
||||
|
||||
| State of presented token | Verdict | Action |
|
||||
|--------------------------|---------|--------|
|
||||
| Unconsumed, unexpired | Normal renewal | Mint successor, mark consumed |
|
||||
| Consumed, successor **unconsumed** | Crash-recovery grace | Agent saved the old token but died before persisting the new one. Accept once; issue a fresh successor |
|
||||
| Consumed, successor **also consumed** | Reuse = theft | Revoke the entire `family_id`, log security event, return terminal error |
|
||||
|
||||
The grace window is **accept-previous-once** — exactly one step back in the lineage, exactly once. Forward-only is doctrine ([[core/01-ethos]]); there is no knob to widen it.
|
||||
|
||||
---
|
||||
|
||||
## Agent Side
|
||||
|
||||
- Terminal sentinel errors (`ErrRefreshTokenInvalid`, `ErrMachineMismatch`) stop the polling loop's retry machinery — these are not transient network failures and are logged `[CRITICAL]`. See [[components/02-agent]].
|
||||
- The **instance lock** exists largely for this mechanism: two agent processes sharing one `config.json` would race rotations and trip family revocation on themselves. One config, one process, enforced by flock/mutex.
|
||||
|
||||
---
|
||||
|
||||
## Operational Notes
|
||||
|
||||
- Revoked family → agent must be re-registered with a fresh registration token. Runbook: `OPERATIONS.md §2`.
|
||||
- After a database restore, agents may present tokens the restored DB has never seen (or sees as stale lineage). Expect re-registration; see `OPERATIONS.md §3`.
|
||||
- `CleanupExpiredTokens` reaps expired rows; `RevokeAllAgentTokens` is the operator hammer.
|
||||
|
||||
---
|
||||
|
||||
## Why This Shape
|
||||
|
||||
A refresh token in a file on a fleet machine *will* eventually leak — backup snapshots, copied VMs, sloppy decommissioning. Rotation-with-family-revocation means a leaked token is only useful until the legitimate agent next renews, and *using* a stale one burns the whole family loudly. The failure mode is detection, not silent coexistence.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
119
RAF/security/04-machine-binding.md
Normal file
119
RAF/security/04-machine-binding.md
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
# Machine Binding
|
||||
|
||||
**Hardware-bound authentication for agent-to-server communication.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedFlag ties each agent to specific hardware via a machine ID fingerprint. This prevents config file theft from being used on unauthorized machines.
|
||||
|
||||
---
|
||||
|
||||
## Machine ID Generation
|
||||
|
||||
**Linux:**
|
||||
- Uses `machineid` library to compute SHA-256 hash
|
||||
- Fallbacks: `/sys/class/dmi/id/product_uuid`, `/var/lib/dbus/machine-id`
|
||||
- Combined with hostname for uniqueness
|
||||
|
||||
**Windows:**
|
||||
- Uses `MachineIdentifier` class from Windows API
|
||||
- Retrieves system-wide hardware identifier
|
||||
|
||||
**macOS:**
|
||||
- Uses IOKIT framework to read hardware identifiers
|
||||
- Combines multiple hardware sources for uniqueness
|
||||
|
||||
**Implementation:**
|
||||
- `agent/internal/system/machine_id.go` (uses `machineid` library — Linux dbus/machine-id, macOS IOKIT, Windows registry)
|
||||
|
||||
---
|
||||
|
||||
## Binding Flow
|
||||
|
||||
```
|
||||
1. Agent computes machine ID at startup
|
||||
2. Agent includes X-Machine-ID header in requests
|
||||
3. Server validates against stored machine ID
|
||||
4. Mismatch → authentication failure
|
||||
5. Match → request proceeds
|
||||
```
|
||||
|
||||
**Middleware:**
|
||||
- `server/internal/middleware/machine_binding.go:MachineBindingMiddleware()`
|
||||
|
||||
---
|
||||
|
||||
## Binding States
|
||||
|
||||
| State | Description | Trigger |
|
||||
|-------|-------------|---------|
|
||||
| `registered` | Machine ID stored, valid | Registration complete |
|
||||
| `pending` | Hardware change detected | Machine ID mismatch |
|
||||
| `unbound` | No machine ID provided | Missing header |
|
||||
| `revoked` | Admin action | Manual revocation |
|
||||
|
||||
---
|
||||
|
||||
## Rebinding Endpoint
|
||||
|
||||
**Purpose:** Allow legitimate hardware changes.
|
||||
|
||||
**Endpoint:** `POST /api/v1/agents/:id/rebind`
|
||||
|
||||
**Requirements:**
|
||||
- Admin authentication (WebAuthMiddleware)
|
||||
- Valid nonce for rebind operation
|
||||
- Machine ID update logged to audit trail
|
||||
|
||||
**Flow:**
|
||||
```
|
||||
1. Admin approves rebind request
|
||||
2. Server updates machine ID for agent
|
||||
3. Event logged to history table
|
||||
4. Agent can resume normal operations
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- Handler: `server/internal/handlers/agents.go:RebindAgentMachineID()`
|
||||
- Validation: Nonce + admin auth + machine ID update
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Machine ID Spoofing
|
||||
- Attacker cannot forge valid X-Machine-ID without hardware access
|
||||
- Server-side validation prevents spoofed headers
|
||||
- Binding checked on every authenticated request
|
||||
|
||||
### Hardware Changes
|
||||
- SSD replacement → new machine ID
|
||||
- Motherboard swap → new machine ID
|
||||
- Cloud instance restart → same machine ID (persistent storage)
|
||||
|
||||
**Mitigation:** Rebind flow for legitimate changes
|
||||
|
||||
### Key Compromise
|
||||
- Stolen agent config + machine ID = unauthorized access
|
||||
- Mitigation: Machine binding ties config to hardware
|
||||
- Mitigation: Key rotation limits exposure window
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Machine ID is stable for the lifetime of the hardware configuration.
|
||||
|
||||
**Connection:** Machine binding (`security/04-machine-binding.md`) implements ETHOS #2 (security is non-negotiable).
|
||||
|
||||
**Connection:** Rebinding endpoint (`security/04-machine-binding.md`) complements nonce validation (`verification/04-replay-protection.md`).
|
||||
|
||||
**Connection:** Machine binding middleware (`server/internal/middleware/machine_binding.go`) enforces binding on agent routes.
|
||||
|
||||
**Connection:** TOFU model (`verification/02-agent-verification.md`) works with machine binding for trust continuity.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
366
RAF/security/05-supply-chain-gate.md
Normal file
366
RAF/security/05-supply-chain-gate.md
Normal file
|
|
@ -0,0 +1,366 @@
|
|||
# Supply Chain Gate
|
||||
|
||||
**Package-manager authorization with signed capability tokens, kernel-enforced where the platform allows.**
|
||||
|
||||
This is the design of record for the gate — the capability model, the wire contract all
|
||||
three components agree on, the load-bearing constraints, and component responsibilities.
|
||||
Build status and per-step implementation tracking live in `docs/tasks/GATE-000-supply-chain-gate-plan.md`.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedFlag's supply chain gate inverts the traditional defense model: instead of **allow all installs and detect bad ones**, it **denies all state changes and requires explicit human authorization**.
|
||||
|
||||
**Core principle:** Nothing installs, updates, or changes without a signed authorization token from the RedFlag server.
|
||||
|
||||
---
|
||||
|
||||
## The decision: capability tokens, not a decision daemon
|
||||
|
||||
The gate authorizes package installs with **signed capability tokens**, not with a runtime
|
||||
decision daemon. The server is the authority: it evaluates policy (OSV, age, pinning, human
|
||||
approval) and mints an Ed25519-signed token describing exactly one operation over a
|
||||
fully-resolved dependency closure. A small, privileged, network-less **executor**
|
||||
(`helper/`, Rust) verifies the signature against a trusted authority key, verifies every
|
||||
artifact hash, and performs that one operation. Nothing else can change package state.
|
||||
|
||||
This replaced the earlier `rs-helper` socket-decision daemon. `rs-helper`'s reusable parts
|
||||
(the eBPF `InterceptEvent` struct, the package-manager allowlist, the hash cache) moved into
|
||||
the unprivileged agent-side consumer. Its *role* as a runtime allow/deny RPC is retired.
|
||||
|
||||
### Why this model (two tests it has to pass)
|
||||
|
||||
**Cross-platform.** Linux eBPF and macOS ESF can pause an exec and ask a daemon "may this
|
||||
proceed?" Windows WDAC cannot — it is signature-based, with no runtime callback. A
|
||||
decision-daemon model therefore has no Windows mapping. A capability token maps onto all
|
||||
three identically: in every case the enforcement layer only needs to answer "is this
|
||||
execution authorized," and a verified token + a privileged executor that the OS trusts is
|
||||
platform-agnostic. We build for the platform with the tightest constraint.
|
||||
|
||||
**Protects people.** Protection comes down to where the trust root lives. The signing key
|
||||
lives at the server (the human-approval authority), off the host. An attacker who fully owns
|
||||
the agent process — a prompt-injected coding agent, the literal threat — can *request* a
|
||||
token but cannot forge the server's signature, so the executor never runs. The trust root is
|
||||
outside the blast radius. A socket-RPC daemon degrades to "can the attacker reach the socket
|
||||
or influence what's pinned," which a compromised agent tier often can.
|
||||
|
||||
The two tests converge on the same answer, which is the signal it's right.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Enforcement Layers (Platform-Specific)
|
||||
|
||||
| Platform | Enforcement Mechanism | What It Blocks |
|
||||
|----------|----------------------|----------------|
|
||||
| **Linux** | eBPF (syscalls/sys_enter_execve) or AppArmor | apt, dnf, yum, pip, npm, bun, docker (CLI) |
|
||||
| **Windows** | WDAC (Windows Defender Application Control) | winget, npm.cmd, pip.exe, choco, scoop |
|
||||
| **macOS** | Endpoint Security Framework (ESF) | brew, pip, npm, bun, cargo |
|
||||
|
||||
**Crucial distinction:** Enforcement sits **below** userspace wrappers. A prompt-injected Claude Code session cannot bypass it with absolute paths or environment manipulation.
|
||||
|
||||
### Trust Chain
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Human Operator │
|
||||
│ - Reviews UI prompt with OSV findings, age checks, etc. │
|
||||
│ - Clicks "Approve" or "Reject" │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────────▼───────────────────────────────────────┐
|
||||
│ RedFlag Server (authority, unprivileged) │
|
||||
│ - Resolves the closure, records/fetches per-artifact hash │
|
||||
│ - Runs OSV.dev check + package-age gates │
|
||||
│ - Mints an Ed25519-signed capability token (signer off the │
|
||||
│ request path) │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────────▼───────────────────────────────────────┐
|
||||
│ RedFlag Agent (consumer, unprivileged) │
|
||||
│ - Polls for the token, confirms agent_id is this host │
|
||||
│ - Holds no signing key; cannot run installs directly │
|
||||
│ - Hands the token to the executor │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────────▼───────────────────────────────────────┐
|
||||
│ Helper / Executor (privileged, network-less, Rust) │
|
||||
│ - Invoked via `sudo systemd-run --pipe` as its own │
|
||||
│ transient service (escapes the agent's ProtectSystem │
|
||||
│ sandbox); not setuid │
|
||||
│ - Verifies token signature (ed25519) + every artifact hash │
|
||||
│ - Replay-guards on token_id, then execs exactly one op │
|
||||
│ via argv: no shell, env stripped, forward-only │
|
||||
│ - Entire codebase auditable in one sitting │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The token (the contract all three sides agree on)
|
||||
|
||||
This is the canonical wire contract. The Go (`server/`, `agent/internal/capability/token.go`)
|
||||
and Rust (`helper/src/main.rs`) implementations reconstruct identical bytes from it.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"version": 1,
|
||||
"token_id": "<uuid>", // unique; replay guard + receipt/audit
|
||||
"agent_id": "<uuid>", // bound to exactly one host
|
||||
"key_id": "<hex, 32 chars>", // authority key fingerprint (rotation)
|
||||
"package_type": "apt|dnf|npm|bun|pip|docker|winget|agent-self",
|
||||
"operation": "install|upgrade", // forward-only; no downgrade
|
||||
"closure": [ // FULL resolved set, sorted by (name,version)
|
||||
{
|
||||
"name": "<pkg>",
|
||||
"version": "<exact>",
|
||||
"sha256": "<hex>", // expected artifact hash
|
||||
"source": "mirror|registry",
|
||||
"artifact_path": "<local path or url, optional>"
|
||||
}
|
||||
],
|
||||
"issued_at": <unix>,
|
||||
"not_before": <unix>,
|
||||
"expires_at": <unix>, // short TTL
|
||||
"signature": "<hex ed25519>" // over the canonical message below
|
||||
}
|
||||
```
|
||||
|
||||
**Canonical signed message** (deterministic, language-agnostic — mirrors the existing v3
|
||||
command format so Go and Rust reconstruct identical bytes):
|
||||
|
||||
```
|
||||
closure_hash = hex(sha256( "\n".join( sorted("{name}@{version}#{sha256}") ) ))
|
||||
signed_message = "{agent_id}:{token_id}:{operation}:{package_type}:{closure_hash}:{expires_at}"
|
||||
signature = ed25519_sign(authority_priv, signed_message)
|
||||
```
|
||||
|
||||
The closure is sorted before hashing so ordering can't change the digest. Tampering with any
|
||||
artifact, version, or hash changes `closure_hash` and breaks verification.
|
||||
|
||||
### Reuse, don't reinvent
|
||||
|
||||
The token extends the existing Ed25519 infrastructure rather than introducing new crypto:
|
||||
- `server/internal/services/signing.go` — `SigningService` (Ed25519, `GetPublicKeyHex`,
|
||||
`GetCurrentKeyID` = SHA-256(pubkey)[:16] hex). The token is a new payload it signs.
|
||||
- `server/internal/database/queries/signing_keys.go` — key storage + rotation/version.
|
||||
- `agent/internal/crypto/verification.go` — verifies against active keys; v3 message format
|
||||
`{agent_id}:{id}:{type}:{sha256(params)}:{ts}`. The token mirrors this.
|
||||
- `agent/internal/client/client.go::GetActivePublicKeys` — already "verify keys not servers."
|
||||
|
||||
---
|
||||
|
||||
## Component responsibilities
|
||||
|
||||
- **Server (authority).** At approval/unpin: run OSV + age + attestation checks, resolve the
|
||||
full closure, pull+hash artifacts (mirror tier) or record expected hashes (no-mirror tier),
|
||||
mint and sign the token. Refuse to sign until every check clears. Signer runs off the web
|
||||
process.
|
||||
- **Agent consumer (unprivileged).** Receives the token, confirms `agent_id` is this host,
|
||||
hands it to the executor. Holds no signing key. On Linux/macOS, the eBPF/ESF event path
|
||||
feeds the *request* for a token; the consumer never decides allow/deny itself. Salvages
|
||||
`rs-helper`'s `InterceptEvent`, allowlist, and cache.
|
||||
- **Executor (`helper/`, privileged, Rust).** Verify validity window → resolve trusted key by
|
||||
`key_id` from a local pinned keyring → reconstruct `signed_message` → Ed25519 verify →
|
||||
verify each artifact's sha256 → replay-guard on `token_id` → exec exactly one operation via
|
||||
argv (no shell, env stripped) → structured result + exit code. Fail-closed on every error
|
||||
path. Auditable in one sitting.
|
||||
- **Kernel layer (where applicable).** Linux eBPF / Windows WDAC / macOS ESF deny
|
||||
package-manager execution except via the trusted executor. Defense-in-depth.
|
||||
|
||||
---
|
||||
|
||||
## Load-bearing constraints (do not regress these)
|
||||
|
||||
1. **Sign the resolved closure, not the top-level package.** Aggregate updates and the
|
||||
scheduler chaining dependencies mean the token must cover every transitive artifact and
|
||||
its hash. Authorizing only the top-level reopens the gap where the modern attacks live.
|
||||
Resolve-and-hash-the-closure is also the mirror's real security job; air-gapping from the
|
||||
registry is the bonus.
|
||||
2. **The signer lives off the web process.** Server-as-authority plus server-as-mirror
|
||||
concentrates blast radius. The signing key must not be reachable from the request path
|
||||
(separate signer service / key material not loaded in the API process), so a web
|
||||
compromise cannot both mint tokens and serve artifacts.
|
||||
3. **Verified-cache fallback, fail-closed only on change.** When installs route through the
|
||||
mirror, an already-approved-and-hashed artifact must still install from local cache if the
|
||||
server blinks. Fail closed on *new change*, not on a brief outage of an already-authorized
|
||||
operation.
|
||||
4. **Verify keys, not servers.** The agent and helper trust *a public key (set)* identified by
|
||||
`key_id` fingerprint — never a server URL. Today the key lives on your server; nothing
|
||||
changes operationally. But the contract ("trust this key") lets the authority later be
|
||||
rotated, replicated, or held by a federation/guild node without touching the agent↔helper
|
||||
interface. This is the long-term cross-platform answer hidden in a one-line design choice.
|
||||
5. **Kernel stops are defense-in-depth, not a prerequisite.** The capability model protects on
|
||||
a host where eBPF/WDAC/ESF cannot be deployed (locked-down managed box, constrained
|
||||
container). Kernel enforcement raises the cost of bypass; it does not gate whether the model
|
||||
means anything. Partial deployment still moves a host out of the soft-target category.
|
||||
6. **No doctrinal knobs.** Signing required and forward-only (no downgrade) are ETHOS doctrine,
|
||||
not configurable. The token has no "skip verification" path.
|
||||
|
||||
---
|
||||
|
||||
## Gate Features
|
||||
|
||||
### 1. Version Pinning (Security Primitive)
|
||||
|
||||
**Normal model:** `npm install express` → resolves to `latest` → fetches from registry → installs
|
||||
|
||||
**RedFlag model:** the exact version is resolved and its SHA256 recorded at pin time; the
|
||||
artifact that installs must match that hash.
|
||||
|
||||
The pin's hash source depends on who can reach the artifact:
|
||||
|
||||
- **npm / PyPI** — one canonical public registry exists, so the **server** fetches the
|
||||
artifact and computes the hash directly at approval (`computeAndStorePackageHash`).
|
||||
- **dnf / apt** — artifacts come from each agent's own GPG-signed repos, which the server
|
||||
cannot reach. The **agent** resolves the canonical hash from its signed repo metadata at the
|
||||
dry-run step and reports it (`installer.ResolveArtifactSHA256`: dnf via `dnf download`+SHA256,
|
||||
apt via the `SHA256:` field of the signed index). The server pins what the agent reports.
|
||||
Trust is anchored in the repo signature; the pin is set before the install-time compromise
|
||||
the gate defends against.
|
||||
|
||||
> Historical note: an earlier draft of this doc specified a `ResolvePin` / `fetchAndHash`
|
||||
> function and a `security_packages` table. Neither was built. The shipped registry is the two
|
||||
> stores below. This section documents what exists.
|
||||
|
||||
### 2. Hash Registry (Layer 1)
|
||||
|
||||
All artifacts are verified by SHA256 before installation.
|
||||
|
||||
**Storage (as built):**
|
||||
- `current_package_state.expected_sha256` (migration 040) — the pinned top-level hash per
|
||||
update, consumed by the agent's install-time `VerifyHash`.
|
||||
- `capability_tokens` (migration 042) — the minted, signed token carries the full resolved
|
||||
closure (per-artifact name/version/sha256/source) as JSONB. The token row *is* the closure
|
||||
record; there is no separate `security_packages` table.
|
||||
|
||||
OSV findings, package age, and published-at are recorded on the update's own `metadata` JSONB
|
||||
at approval, not in a dedicated table.
|
||||
|
||||
### 3. Local Mirror (Optional)
|
||||
|
||||
**Purpose:** Decouple fleet from upstream availability after approval.
|
||||
|
||||
**Flow:**
|
||||
1. Operator approves update → server fetches artifact → stores in local mirror
|
||||
2. Server issues approval token with artifact path in mirror
|
||||
3. Agent fetches from mirror (not upstream) → verifies SHA256 → installs
|
||||
|
||||
**When to enable:**
|
||||
- Large fleets (>100 agents) where redundant fetches are noisy
|
||||
- Upstream availability is a concern
|
||||
- Maximum isolation desired
|
||||
|
||||
**Configuration:** `security.package_mirror.enabled` (boolean)
|
||||
|
||||
### 4. Time Gates: Package Age + Version Soak (live policies, v0.2.6.2)
|
||||
|
||||
Two distinct time-based gates, both under the `supply_chain.*` settings category, both
|
||||
resolving env → config → DB → default. These are *policies* (configurable, with enforcement
|
||||
modes) — unlike signing and hash verification, which are doctrine.
|
||||
|
||||
**Approval-time age gate** (`package_age.go`) — the Shai-Hulud defense. Packages younger
|
||||
than the threshold draw a warning or a block at approval:
|
||||
|
||||
| Setting | Default | Meaning |
|
||||
|---------|---------|---------|
|
||||
| `min_package_age_hours` | 24 | Minimum publish age before approval is clean |
|
||||
| `gate_enforcement` | `warn` | `warn` or `block` |
|
||||
| `block_unknown_age` | `false` | Opt-in: under `block`, registry-backed ecosystems (npm, PyPI) fail closed when the publish date can't be determined — a dark recency source is itself a Shai-Hulud-class signal. Default off, by sovereignty: the operator chooses to fail closed on unknowns. |
|
||||
|
||||
**Install-time version-soak gate** (`soak_gate.go` — this is GATE-005, *not* the age gate):
|
||||
|
||||
| Setting | Default | Meaning |
|
||||
|---------|---------|---------|
|
||||
| `soak_window_days` | 14 | A version must soak this long before the install path will take it |
|
||||
| `soak_enforcement` | `block` | `warn` or `block` |
|
||||
|
||||
### 5. OSV.dev Integration
|
||||
|
||||
Vulnerability scanning runs at **detection time**, not approval time (moved in v0.2.6.2).
|
||||
The scan-report path enqueues OSV checks (batch endpoint, 100 per POST, 4 concurrent)
|
||||
across the full dependency closure; verdicts persist to package metadata
|
||||
(`supply_chain_vulns`, `supply_chain_checked_at`) and are visible in the dashboard before
|
||||
anyone approves anything.
|
||||
|
||||
Approval *reads* the persisted verdict — it does not re-scan. Any known vulnerability
|
||||
anywhere in the closure is a full stop (see Enforcement Posture below); there is no
|
||||
severity threshold below which approval proceeds quietly.
|
||||
|
||||
### 6. SLSA/Sigstore Attestation (Visibility Signal)
|
||||
|
||||
**Not a hard block** — surfaced as a visibility indicator.
|
||||
|
||||
**UI:** When unpinning a package without attestation:
|
||||
```
|
||||
This package lacks SLSA provenance or Sigstore signature.
|
||||
Consider waiting for an attested release before proceeding.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Enforcement Posture (v0.2.3.1, extended v0.2.6.2)
|
||||
|
||||
Approval is an enforcement point, not advisory. A known vulnerability — top-level or anywhere
|
||||
in the resolved dependency closure — is a hard stop: `ApproveUpdate` returns `409` and mints
|
||||
nothing. For capability-gated ecosystems, a closure that OSV could not check (service
|
||||
unreachable) is also a stop.
|
||||
|
||||
The only path through is an explicit operator override with a documented reason. The override
|
||||
waives the vulnerability judgment only — the signed token still binds real artifact hashes and
|
||||
the executor still verifies signature + hash. Every override writes a `supply_chain_override`
|
||||
system event. Bulk approve carries no blanket override: flagged updates come back in
|
||||
`blocked[]` and must be approved individually.
|
||||
|
||||
Auto-confirm shares the `ClosureCleared` predicate with manual approval — the two paths cannot
|
||||
drift on what counts as a clean closure.
|
||||
|
||||
---
|
||||
|
||||
## Build sequence (design intent)
|
||||
|
||||
The order the gate is built in. Per-step *status* is tracked in
|
||||
`docs/tasks/GATE-000-supply-chain-gate-plan.md`; this records the intended dependency order.
|
||||
|
||||
1. `helper/` executor + token contract (the keystone; defines the schema in code).
|
||||
2. Go `capability` token type + canonical encoder + Ed25519 sign/verify (server & agent share
|
||||
the definition; mirrored in both modules — no shared module exists).
|
||||
3. Server: closure resolver + mint/sign at approval; signer off web process; endpoint to
|
||||
deliver tokens to the agent.
|
||||
4. Migration: store resolved closure + per-artifact hashes alongside the pinned state.
|
||||
5. Agent consumer: accept token, bind-check, pass to executor; fold in `rs-helper` parts.
|
||||
6. Mirror tier (optional): pull+hash closure at approval; verified-cache fallback.
|
||||
7. Kernel adapters wire the executor as the only permitted caller.
|
||||
|
||||
Steps 1–2 are the contract. Everything else hangs off them.
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Build status / implementation tracking** → `docs/tasks/GATE-000-supply-chain-gate-plan.md`
|
||||
- **Command signing** → `verification/01-signing-pipeline.md`
|
||||
- **Agent verification** → `verification/02-agent-verification.md`
|
||||
- **Replay protection** → `verification/04-replay-protection.md`
|
||||
- **Trust boundaries** → `security/01-trust-boundaries.md`
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** The capability model is the floor; kernel-level primitives (eBPF, WDAC, ESF)
|
||||
are defense-in-depth on top, not a prerequisite. Userspace wrappers alone are bypassable.
|
||||
|
||||
**Assumption:** The privileged executor runs network-less, with a narrow argv-only API, no
|
||||
shell, and a stripped environment; the agent that hands it tokens is unprivileged and holds no
|
||||
signing key.
|
||||
|
||||
**Connection:** [[security/01-trust-boundaries]] (kernel enforcement as trust boundary)
|
||||
|
||||
**Connection:** [[security/04-machine-binding]] (agent identity verification)
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
137
RAF/security/06-standalone-authority.md
Normal file
137
RAF/security/06-standalone-authority.md
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
# Standalone Authority (Local Mode)
|
||||
|
||||
Drafted 2026-06-10. Status: APPROVED 2026-06-10 (Casey signed off on all three open
|
||||
questions; resolutions inlined below).
|
||||
Companion to `05-supply-chain-gate.md` (the gate contract is unchanged by this doc).
|
||||
Build tracking: `docs/tasks/FEAT-003-standalone-local-authority.md`.
|
||||
|
||||
## The problem
|
||||
|
||||
The tray app needs a local approval flow ("approve update → it installs") on a host with
|
||||
no fleet server. In fleet mode the trust chain is:
|
||||
|
||||
```
|
||||
operator → server (authority, off-host) → agent (consumer) → helper (enforcer)
|
||||
```
|
||||
|
||||
The authority lives on a different machine than the thing being updated. That is the real
|
||||
boundary: a fully compromised agent host still cannot mint a capability token.
|
||||
|
||||
In standalone mode the server does not exist. Something on the same host must mint
|
||||
tokens. **No arrangement of processes on one host reproduces the off-host boundary** —
|
||||
local root can always reach the key. Pretending otherwise (e.g. "the agent signs its own
|
||||
tokens, the helper checks them") turns the token into self-attestation theater. This doc
|
||||
is honest about what survives and designs for that.
|
||||
|
||||
## What the gate still buys on one host
|
||||
|
||||
The boundary that *is* preservable locally is the OS privilege boundary:
|
||||
|
||||
| Defended against | Fleet mode | Standalone |
|
||||
|---|---|---|
|
||||
| Tampered artifact (hash mismatch) | yes | **yes** — helper verification unchanged |
|
||||
| Known-vuln package (OSV) | yes | **yes** — gates run before mint |
|
||||
| Too-new package (age/soak gates) | yes | **yes** |
|
||||
| Unprivileged local malware minting installs | yes | **yes** — mint key is root-owned |
|
||||
| Compromised agent process | yes (server refuses) | **yes** — agent user cannot read mint key |
|
||||
| Local root attacker | yes (authority off-host) | **no — out of scope, say so in docs** |
|
||||
|
||||
A compromised *agent* (unprivileged) in standalone can request a mint, exactly as a
|
||||
compromised agent in fleet mode can request approval — and the same gates refuse it.
|
||||
What standalone loses is only the root-compromise case, which on a single sovereign
|
||||
host is game over for every other tool too.
|
||||
|
||||
## Design
|
||||
|
||||
### Authority placement: privileged mint, root-owned key
|
||||
|
||||
Standalone mint runs as a **separate privileged invocation of the helper**
|
||||
(`redflag-helper --mint`), the same `sudo systemd-run` pattern the executor already
|
||||
uses. Rationale:
|
||||
|
||||
- Keeps the artifact set at three (`agent`, `server`, `helper`) — no fourth binary.
|
||||
- The helper is already the audited, privileged, network-less Rust component
|
||||
("auditable in one sitting"). Mint is ~the token struct it already parses, signed
|
||||
instead of verified.
|
||||
- Minter-equals-verifier is acceptable *here only* because both already run as root on
|
||||
the same host; the boundary being enforced is root-vs-unprivileged, not
|
||||
minter-vs-verifier. In fleet mode this collapse stays forbidden (constraint #2 of
|
||||
`05`).
|
||||
|
||||
Keys:
|
||||
|
||||
- `authority_local.key` — Ed25519, generated at install time (same provisioning step
|
||||
that creates the `redflag-local` group / helper sudoers). Root-owned, `0600`,
|
||||
outside the agent's readable tree. Never leaves the host.
|
||||
- The helper's pinned keyring gets the corresponding public key with its `key_id`
|
||||
fingerprint, exactly like a server key. Verify path is byte-identical to fleet mode.
|
||||
|
||||
### Flow
|
||||
|
||||
```
|
||||
tray (redflag-local member)
|
||||
→ POST /v1/updates/:id/approve (agent local API, group ACL boundary)
|
||||
→ agent: resolve closure via DiscoveryRunner (dry-run, hash-resolve — read-only)
|
||||
→ agent: run gate predicates locally (OSV.dev query, age gate, soak gate)
|
||||
any vuln in closure = full stop, same as ApproveUpdate's 409 — no silent waiver;
|
||||
override requires explicit reason, journaled locally
|
||||
→ agent: write mint request file (closure + gate evidence + operator + reason)
|
||||
→ sudo systemd-run redflag-helper --mint <request> (narrowed sudoers entry)
|
||||
helper re-derives closure_hash, re-checks gate evidence freshness window,
|
||||
signs with authority_local.key, emits token, journals the mint
|
||||
→ token → normal consumer path → helper verify + execute (unchanged)
|
||||
```
|
||||
|
||||
The mint step **re-validates rather than trusts** the agent's gate verdicts where it
|
||||
can do so without network (closure hash shape, evidence timestamps, forward-only
|
||||
version check). It cannot re-run OSV (network-less) — the OSV verdict is part of the
|
||||
journaled evidence, so a lying agent user leaves a tamper-evident trail and still
|
||||
cannot bypass artifact-hash verification at execute time.
|
||||
|
||||
Doctrine carried over unchanged: signing required, forward-only, no skip-verification
|
||||
path, no doctrinal knobs.
|
||||
|
||||
### Audit
|
||||
|
||||
Every standalone approval/mint/execute writes to a local append-only journal owned by
|
||||
root (mirror of the server's security-event journal). The tray reads it through a
|
||||
read-only local API endpoint. When the host later joins a fleet, the journal is
|
||||
uploaded once during re-provisioning so history survives the mode switch.
|
||||
|
||||
### Fleet join ("join fleet later")
|
||||
|
||||
Registration code + one-time 2FA → agent registers against the server → helper keyring
|
||||
is **replaced**: server authority key(s) in, local authority key retired and its private
|
||||
half destroyed (journaled). Single authority per mode — no dual-mint window, no local
|
||||
fallback authority in fleet mode. A fleet host that loses its server does what it does
|
||||
today: nothing installs until the server returns (constraint #3's verified-cache
|
||||
fallback applies only to already-minted operations).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No local approval authority in fleet mode (server remains sole authority).
|
||||
- No "lite" trust mode — gates are not weakened because the host is standalone.
|
||||
- No network listener for the mint path; the local group ACL + sudoers narrowing is
|
||||
the entire request surface.
|
||||
- Tray never touches keys, tokens, or the mint path directly — it only calls the
|
||||
agent's local API.
|
||||
|
||||
## Resolved questions (Casey, 2026-06-10)
|
||||
|
||||
1. **Mint placement: `redflag-helper --mint`.** Artifact set stays at three. Two keys
|
||||
keep roles distinct (mint key root-owned `0600`, verify key in pinned keyring);
|
||||
SEC-022 attests both invocation modes.
|
||||
2. **OSV in standalone: best-effort with honest verdict.** Vuln found = full stop.
|
||||
OSV unreachable = explicit operator acceptance of "closure unverified" with reason,
|
||||
journaled — mirrors the fleet `unverified` hold from v0.2.3.1. Age/soak gates and
|
||||
execute-time hash verification never relax.
|
||||
3. **Gate-evidence freshness window: 15 minutes, hard-coded.** Not configurable
|
||||
(no doctrinal knobs); expired evidence means the agent re-resolves and re-checks.
|
||||
|
||||
## Cross-refs
|
||||
|
||||
- `05-supply-chain-gate.md` — token contract, constraints (esp. #2, #4, #6).
|
||||
- `docs/tasks/FEAT-002-local-agent-api.md` — local API surface this builds on.
|
||||
- `docs/tasks/SEC-023` — package-mutation boundary (mint path must not weaken it).
|
||||
- `docs/tasks/SEC-022` — binary mutual attestation (helper trust file).
|
||||
- `docs/tasks/THREAT-001` — must document the standalone trust model table above.
|
||||
41
RAF/testing/01-test-pyramid.md
Normal file
41
RAF/testing/01-test-pyramid.md
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# Testing
|
||||
|
||||
**What's tested, how, and where the honest gaps are.**
|
||||
|
||||
---
|
||||
|
||||
## Shape
|
||||
|
||||
~98 Go test files across the repo (61 server, 37 agent), plus Rust tests in the helper. Coverage concentrates where the security model lives — verification, token lifecycle, scanners with hostile input — rather than chasing a percentage.
|
||||
|
||||
| Layer | What it covers | Examples |
|
||||
|-------|----------------|----------|
|
||||
| Unit (Go) | Crypto verification, replay protection, backoff, machine-id derivation, scanner parsers | `agent/internal/crypto/*_test.go`, `winget_parser_test.go`, `windows_ghost_test.go` |
|
||||
| Unit (Rust) | Helper token verification, hash checks | `helper/` cargo tests |
|
||||
| Cross-language contract | The capability-token wire contract — Rust and Go must produce **byte-identical** `closure_hash` for the same closure | helper + server test pairs |
|
||||
| Structural | Tests that assert properties of the *source*, not behavior — e.g. `token_renewal_transaction_test.go` asserts the renewal handler's transactional shape; `ethos_exempt_test.go` polices logging discipline | server + agent |
|
||||
| Migration | Idempotency and schema invariants | `server/internal/database/queries/*_test.go` |
|
||||
| CI | `go vet`, `go test -race`, `cargo test` + clippy, `tsc --noEmit` on every push | `.gitea/workflows/ci.yml` |
|
||||
|
||||
The structural-test category is unusual and deliberate: where a rule matters more than any single behavior (transaction boundaries, ETHOS logging), a test reads the source and fails on regression. Cheaper than a linter plugin, louder than a comment.
|
||||
|
||||
---
|
||||
|
||||
## Manual / Live Testing
|
||||
|
||||
- A live Fedora agent runs against the dev stack continuously — DNF scanning, replay protection observed firing in production logs.
|
||||
- The supply-chain gate completed a live end-to-end run 2026-06-05: real package, install → hash-pin → token mint → helper verify+execute → receipt.
|
||||
- Windows agents test against physical machines (intermittently available) — Windows coverage leans harder on unit tests of parsers and the WUA layer as a result.
|
||||
|
||||
---
|
||||
|
||||
## Honest Gaps
|
||||
|
||||
- **No web UI tests.** The dashboard is exercised by hand. TypeScript compilation (`tsc --noEmit`) is the only automated check.
|
||||
- **No end-to-end suite.** The live-agent loop substitutes for one; it does not run in CI.
|
||||
- **Windows paths are under-exercised live** relative to Linux — see manual testing above.
|
||||
- Test counts are a proxy, not a coverage claim. Nothing here should be read as "battle-tested"; the README says the same.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-06-11*
|
||||
177
RAF/verification/01-signing-pipeline.md
Normal file
177
RAF/verification/01-signing-pipeline.md
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
# Ed25519 Signing Pipeline
|
||||
|
||||
**Server-side command and binary signing with Ed25519.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
All commands and binaries are signed with Ed25519 private key on the server. Agents verify signatures before execution.
|
||||
|
||||
**Cross-references:**
|
||||
- [[verification/02-agent-verification]] (agent-side verification)
|
||||
- [[verification/03-key-rotation]] (key rotation support)
|
||||
- [[security/01-trust-boundaries]] (signature as trust boundary)
|
||||
|
||||
---
|
||||
|
||||
## Key Generation
|
||||
|
||||
**Method:** Server startup automatic key registration
|
||||
|
||||
**File:** `server/internal/services/signing.go`
|
||||
|
||||
```go
|
||||
func (s *SigningService) InitializePrimaryKey(ctx context.Context) error {
|
||||
// 1. Get current key fingerprint (SHA-256 of public key, truncated)
|
||||
keyID := s.GetCurrentKeyID()
|
||||
publicKeyHex := s.GetPublicKeyHex()
|
||||
|
||||
// 2. Query next version number from database
|
||||
nextVersion, err := s.signingKeyQueries.GetNextVersion(ctx)
|
||||
if err != nil {
|
||||
nextVersion = 1
|
||||
}
|
||||
|
||||
// 3. Insert key (ON CONFLICT DO NOTHING — safe on every startup)
|
||||
if err := s.signingKeyQueries.InsertSigningKey(ctx, keyID, publicKeyHex, nextVersion); err != nil {
|
||||
return fmt.Errorf("failed to insert signing key: %w", err)
|
||||
}
|
||||
|
||||
// 4. Set as primary
|
||||
if err := s.signingKeyQueries.SetPrimaryKey(ctx, keyID); err != nil {
|
||||
return fmt.Errorf("failed to set primary key: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Command Signing
|
||||
|
||||
**Method:** v3 message format with agent_id binding
|
||||
|
||||
**File:** `server/internal/services/signing.go`
|
||||
|
||||
```go
|
||||
func (s *SigningService) SignCommand(cmd *models.AgentCommand) (string, error) {
|
||||
// 1. Record signing time and key identity
|
||||
now := time.Now().UTC()
|
||||
cmd.SignedAt = &now
|
||||
cmd.KeyID = s.GetCurrentKeyID()
|
||||
|
||||
// 2. Serialize params and hash
|
||||
paramsJSON, _ := json.Marshal(cmd.Params)
|
||||
paramsHash := sha256.Sum256(paramsJSON)
|
||||
paramsHashHex := hex.EncodeToString(paramsHash[:])
|
||||
|
||||
// 3. Create v3 message format
|
||||
// agent_id binding prevents cross-agent replay (F-1 fix)
|
||||
message := fmt.Sprintf("%s:%s:%s:%s:%d",
|
||||
cmd.AgentID.String(),
|
||||
cmd.ID.String(),
|
||||
cmd.CommandType,
|
||||
paramsHashHex,
|
||||
now.Unix())
|
||||
|
||||
// 4. Sign with Ed25519
|
||||
signature := ed25519.Sign(s.privateKey, []byte(message))
|
||||
return hex.EncodeToString(signature), nil
|
||||
}
|
||||
```
|
||||
|
||||
**v3 Message Format Benefits:**
|
||||
- **Agent binding:** Includes agent_id to prevent command relay attacks
|
||||
- **Timestamp:** Prevents replay attacks (4-hour max age)
|
||||
- **Parameter hash:** Hides full parameter data while allowing verification
|
||||
|
||||
---
|
||||
|
||||
## Binary Signing
|
||||
|
||||
**Method:** BuildOrchestrator signs binaries at startup
|
||||
|
||||
**File:** `server/internal/services/build_orchestrator.go`
|
||||
|
||||
```go
|
||||
func (s *BuildOrchestratorService) BuildAndSignAgent(version, platform, architecture string) (*models.AgentUpdatePackage, error) {
|
||||
// 1. Load binary from disk
|
||||
binaryName := "redflag-agent"
|
||||
if strings.HasPrefix(platform, "windows") {
|
||||
binaryName += ".exe"
|
||||
}
|
||||
|
||||
binaryPath := filepath.Join(s.agentDir, "binaries", platform+"-"+architecture, binaryName)
|
||||
|
||||
if _, err := os.Stat(binaryPath); os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("binary not found for platform %s: %w", platform, err)
|
||||
}
|
||||
|
||||
if s.signingService.IsEnabled() {
|
||||
// 2. Compute checksum and sign
|
||||
signedPackage, err := s.signingService.SignFile(binaryPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to sign agent binary: %w", err)
|
||||
}
|
||||
|
||||
// 3. Set metadata
|
||||
signedPackage.Version = version
|
||||
signedPackage.Platform = platform
|
||||
signedPackage.Architecture = architecture
|
||||
|
||||
// 4. Store in database
|
||||
err = s.packageQueries.StoreSignedPackage(signedPackage)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to store signed package: %w", err)
|
||||
}
|
||||
|
||||
log.Printf("[INFO] [server] [build_orchestrator] package_signed id=%s version=%s platform=%s arch=%s", signedPackage.ID, version, platform, architecture)
|
||||
return signedPackage, nil
|
||||
} else {
|
||||
log.Printf("Signing disabled, creating unsigned package entry")
|
||||
// Create unsigned package entry for backward compatibility
|
||||
unsignedPackage := &models.AgentUpdatePackage{
|
||||
ID: uuid.New(),
|
||||
Version: version,
|
||||
Platform: platform,
|
||||
Architecture: architecture,
|
||||
BinaryPath: binaryPath,
|
||||
Signature: "",
|
||||
Checksum: "",
|
||||
CreatedBy: "build-orchestrator",
|
||||
IsActive: true,
|
||||
}
|
||||
|
||||
// Get file info
|
||||
if info, err := os.Stat(binaryPath); err == nil {
|
||||
unsignedPackage.FileSize = info.Size()
|
||||
}
|
||||
|
||||
// Store unsigned package
|
||||
err := s.packageQueries.StoreSignedPackage(unsignedPackage)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to store unsigned package: %w", err)
|
||||
}
|
||||
|
||||
return unsignedPackage, nil
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Ed25519 signing is enabled when `REDFLAG_SIGNING_PRIVATE_KEY` environment variable is set.
|
||||
|
||||
**Connection:** [[verification/02-agent-verification]] (agent-side signature verification)
|
||||
|
||||
**Connection:** [[verification/03-key-rotation]] (multi-key rotation support)
|
||||
|
||||
**Connection:** [[security/01-trust-boundaries]] (cryptographic trust boundary)
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
162
RAF/verification/02-agent-verification.md
Normal file
162
RAF/verification/02-agent-verification.md
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
# Agent-Side Verification
|
||||
|
||||
**Agent verifies commands and binaries before execution.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agent verifies Ed25519 signatures, timestamps, and nonces before executing commands.
|
||||
|
||||
**Cross-references:**
|
||||
- [[verification/01-signing-pipeline]] (server-side signing)
|
||||
- [[verification/03-key-rotation]] (key rotation)
|
||||
- [[verification/04-replay-protection]] (replay protection)
|
||||
|
||||
---
|
||||
|
||||
## Public Key Caching (TOFU)
|
||||
|
||||
**Method:** Trust On First Use — TTL+key_id cache with rotation awareness
|
||||
|
||||
**File:** `agent/internal/crypto/pubkey.go`
|
||||
|
||||
```go
|
||||
func FetchAndCacheServerPublicKey(serverURL string) (ed25519.PublicKey, error) {
|
||||
// 1. Check if cache is valid (TTL + key_id match)
|
||||
if meta, err := loadCacheMetadata(); err == nil && meta.KeyID != "" && !meta.IsExpired() {
|
||||
if cachedKey, loadErr := LoadCachedPublicKey(); loadErr == nil {
|
||||
return cachedKey, nil
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Fetch from server GET /api/v1/public-key
|
||||
pubKeyBytes, err := httpGetPublicKey(serverURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to fetch public key: %w", err)
|
||||
}
|
||||
|
||||
// 3. Cache to disk
|
||||
if err := cachePublicKey(pubKeyBytes); err != nil {
|
||||
fmt.Printf("Warning: Failed to cache public key: %v\n", err)
|
||||
}
|
||||
|
||||
return pubKeyBytes, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Command Verification
|
||||
|
||||
**Method:** Verify v3 signature with timestamp, falls back to older formats for backward compatibility
|
||||
|
||||
**File:** `agent/internal/crypto/verification.go`
|
||||
|
||||
```go
|
||||
func (v *CommandVerifier) VerifyCommandWithTimestamp(
|
||||
cmd client.Command,
|
||||
serverPubKey ed25519.PublicKey,
|
||||
maxAge time.Duration,
|
||||
clockSkew time.Duration,
|
||||
) error {
|
||||
// 1. If cmd.SignedAt is nil, fall back to oldest format (backward compat)
|
||||
if cmd.SignedAt == nil {
|
||||
fmt.Printf("[WARNING] [agent] [crypto] command_uses_oldest_format command_id=%s no_signed_at=true upgrade_server_recommended\n", cmd.ID)
|
||||
return v.VerifyCommand(cmd, serverPubKey)
|
||||
}
|
||||
|
||||
// 2. Validate timestamp window
|
||||
now := time.Now().UTC()
|
||||
age := now.Sub(*cmd.SignedAt)
|
||||
if age > maxAge {
|
||||
return fmt.Errorf("command timestamp too old: signed %v ago (max %v)", age.Round(time.Second), maxAge)
|
||||
}
|
||||
if age < -clockSkew {
|
||||
return fmt.Errorf("command timestamp is in the future: %v ahead (max skew %v)", (-age).Round(time.Second), clockSkew)
|
||||
}
|
||||
|
||||
// 3. Try v3 format first (with agent_id) if AgentID is present
|
||||
if cmd.AgentID != "" {
|
||||
message, err := v.reconstructMessageV3(cmd)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to reconstruct v3 message: %w", err)
|
||||
}
|
||||
if ed25519.Verify(serverPubKey, message, sig) {
|
||||
return nil // v3 verification succeeded
|
||||
}
|
||||
// v3 failed — try v2 as fallback
|
||||
}
|
||||
|
||||
// 4. v2 format: timestamp but no agent_id (backward compat)
|
||||
message, err := v.reconstructMessageWithTimestamp(cmd)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to reconstruct timestamped message: %w", err)
|
||||
}
|
||||
if !ed25519.Verify(serverPubKey, message, sig) {
|
||||
return errors.New("signature verification failed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
**Verification Modes:**
|
||||
- **Strict:** Reject all verification failures (default)
|
||||
- **Warning:** Log failure but execute command
|
||||
- **Disabled:** Skip verification entirely
|
||||
|
||||
**Fallback Chain:** v3 (agent_id + timestamp) → v2 (timestamp only) → oldest (no timestamp)
|
||||
```
|
||||
|
||||
**Verification Modes:**
|
||||
- **Strict:** Reject all verification failures (default)
|
||||
- **Warning:** Log failure but execute command
|
||||
- **Disabled:** Skip verification entirely
|
||||
|
||||
---
|
||||
|
||||
## Binary Verification
|
||||
|
||||
**Method:** Verify checksum and Ed25519 signature before installation
|
||||
|
||||
**File:** `agent/internal/orchestrator/update_handler.go`
|
||||
|
||||
```go
|
||||
func (h *UpdateHandler) verifyDownload(binaryData []byte, checksum string) error {
|
||||
// 1. Verify checksum
|
||||
computedChecksum := sha256.Sum256(binaryData)
|
||||
if hex.EncodeToString(computedChecksum[:]) != checksum {
|
||||
return errors.New("checksum mismatch")
|
||||
}
|
||||
|
||||
// 2. Verify Ed25519 signature
|
||||
signature, err := h.downloadBinarySignature()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
publicKey, err := h.LoadCachedPublicKey()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !ed25519.Verify(publicKey, binaryData, []byte(signature)) {
|
||||
return errors.New("signature verification failed")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Server signing** → [[verification/01-signing-pipeline]]
|
||||
- **Key rotation** → [[verification/03-key-rotation]]
|
||||
- **Replay protection** → [[verification/04-replay-protection]]
|
||||
- **Install script** → [[flows/01-registration]] (TOFU key caching)
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
108
RAF/verification/03-key-rotation.md
Normal file
108
RAF/verification/03-key-rotation.md
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
# Key Rotation Support
|
||||
|
||||
**Ed25519 signing key rotation without downtime.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedFlag supports rotating the server's Ed25519 signing key while maintaining agent trust continuity via the TOFU (Trust On First Use) model.
|
||||
|
||||
---
|
||||
|
||||
## Key Storage
|
||||
|
||||
**Server-side:**
|
||||
- Private key: `REDFLAG_SIGNING_PRIVATE_KEY` (env var or Docker secret)
|
||||
- Public key: Stored in database table `server_public_keys`
|
||||
- Key format: Ed25519 (32-byte curve25519)
|
||||
|
||||
**Agent-side:**
|
||||
- Public key cached at registration (`~/.config/redflag/server_keys.json`)
|
||||
- Cached keys validated against active keys in database
|
||||
- TOFU: First key accepted becomes trusted forever
|
||||
|
||||
---
|
||||
|
||||
## Rotation Process
|
||||
|
||||
### Server-Side Rotation
|
||||
|
||||
```
|
||||
1. Generate new key pair (Ed25519)
|
||||
2. INSERT new public key into server_public_keys (status = 'pending')
|
||||
3. Update server config to use new private key
|
||||
4. Sign next command with new key
|
||||
5. Agents verify signature against new key (fails old key check)
|
||||
6. Agents accept new key as valid (TOFU update)
|
||||
7. Old key marked deprecated in database
|
||||
8. Old keys removed after grace period
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- `server/internal/services/signing.go:SetPrimaryKey()`
|
||||
- `server/internal/database/migrations/035_server_public_keys.up.sql`
|
||||
|
||||
---
|
||||
|
||||
## Agent Verification Flow
|
||||
|
||||
```
|
||||
1. Agent receives signed command
|
||||
2. Extract public key from signature (if v3 format)
|
||||
3. Check if key exists in cached trusted keys
|
||||
4. If not cached → fetch from server_public_keys table
|
||||
5. Validate key signature against command
|
||||
6. Accept key as trusted (TOFU)
|
||||
7. Update local cache
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- `agent/internal/crypto/pubkey.go` (TOFU key caching)
|
||||
- `agent/internal/crypto/verification.go:VerifyCommandWithTimestamp()`
|
||||
|
||||
---
|
||||
|
||||
## Key Lifecycle States
|
||||
|
||||
| State | Description | Database Field |
|
||||
|-------|-------------|----------------|
|
||||
| `active` | Currently signing commands | `status = 'active'` |
|
||||
| `pending` | Newly added, awaiting agent adoption | `status = 'pending'` |
|
||||
| `deprecated` | Old key, still accepted | `status = 'deprecated'` |
|
||||
| `revoked` | Compromised or removed key | `status = 'revoked'` |
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### During Rotation
|
||||
- Old key remains `active` until new key is confirmed by agents
|
||||
- Grace period prevents agents from being orphaned
|
||||
- No downtime for command dispatch
|
||||
|
||||
### Key Compromise
|
||||
- Immediately revoke compromised key
|
||||
- Force agents to re-fetch public key list
|
||||
- Affected commands can be replayed until key revocation propagates
|
||||
|
||||
### TOFU Limitations
|
||||
- Compromised initial key → all future keys trusted
|
||||
- Mitigation: Monitor agent registration patterns
|
||||
- Mitigation: Periodic key rotation limits exposure window
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Key rotation is a rare operation — expected once per year or less.
|
||||
|
||||
**Connection:** Rotation process (`verification/03-key-rotation.md`) complements TOFU caching (`verification/02-agent-verification.md`).
|
||||
|
||||
**Connection:** `server_public_keys` table enables rotation without code changes.
|
||||
|
||||
**Connection:** Agent-side key caching (`agent/internal/crypto/pubkey.go`) implements TOFU.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
129
RAF/verification/04-replay-protection.md
Normal file
129
RAF/verification/04-replay-protection.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
# Replay Protection
|
||||
|
||||
**Multi-layer defense against command replay attacks.**
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedFlag implements defense-in-depth against replay attacks at three layers:
|
||||
1. **Nonce validation** (temporal binding for update commands)
|
||||
2. **Command deduplication** (disk-based ID tracking)
|
||||
3. **Timestamp validation** (command age limits)
|
||||
|
||||
---
|
||||
|
||||
## Layer 1: Update Nonce Validation
|
||||
|
||||
**Purpose:** Bind update commands to specific time window.
|
||||
|
||||
**Mechanism:**
|
||||
- Server generates nonce with max age = 2× check-in interval
|
||||
- Nonce embedded in `update_agent` command signature
|
||||
- Agent must present valid nonce within expiry window
|
||||
- Server validates nonce age before execution
|
||||
|
||||
**Implementation:**
|
||||
- Server: `server/internal/services/update_nonce.go`
|
||||
- Validation: `server/internal/middleware/machine_binding.go:validateNonce()`
|
||||
- Nonce expiry: 2× `check_in_interval` (default 10 minutes)
|
||||
|
||||
**Security Model:**
|
||||
```
|
||||
Command dispatched → Nonce generated (age=0)
|
||||
Agent receives → Nonce cached (age < maxAge/2)
|
||||
Agent executes → Nonce validated (age < maxAge)
|
||||
Command executed → Nonce consumed (single-use)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Layer 2: Command Deduplication
|
||||
|
||||
**Purpose:** Prevent duplicate execution after agent restart.
|
||||
|
||||
**Mechanism:**
|
||||
- Executed command IDs persisted to `executed_commands.json`
|
||||
- 4-hour max age window (aligns with command TTL)
|
||||
- Atomic writes prevent corruption
|
||||
|
||||
**Implementation:**
|
||||
- Storage: `agent/internal/orchestrator/executed_commands.json`
|
||||
- Load: `loadExecutedCommands()`
|
||||
- Add: `executedIDs.Add(cmd.ID)`
|
||||
- Save: `saveExecutedCommands(executedIDs)`
|
||||
|
||||
**Deduplication Flow:**
|
||||
```
|
||||
Command received → Check executed IDs
|
||||
If duplicate → Reject with security event
|
||||
If unique → Add to executed IDs
|
||||
Execute command
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Layer 3: Timestamp Validation
|
||||
|
||||
**Purpose:** Reject stale commands regardless of nonce.
|
||||
|
||||
**Mechanism:**
|
||||
- Commands must have `created_at` within TTL window
|
||||
- Default TTL: 4 hours
|
||||
- Server rejects commands older than TTL
|
||||
|
||||
**Implementation:**
|
||||
- Validation: `server/internal/handlers/agents.go:validateCommandTimestamp()`
|
||||
- TTL: 4 hours (configurable via `security_settings.command_ttl_hours`)
|
||||
|
||||
---
|
||||
|
||||
## Combined Protection Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Command Execution │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ 1. Agent polls server for commands │
|
||||
│ 2. Server returns signed command with nonce │
|
||||
│ 3. Agent validates nonce age (Layer 1) │
|
||||
│ 4. Agent checks executed_commands.json (Layer 2) │
|
||||
│ 5. Agent executes command │
|
||||
│ 6. Agent records command ID in executed_commands.json │
|
||||
│ 7. Command times out (Layer 3) → rejected on retry │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Circuit Breaker Integration
|
||||
|
||||
**Purpose:** Prevent replay during scanner failures.
|
||||
|
||||
**Mechanism:**
|
||||
- Circuit breaker per subsystem (APT, DNF, Winget, WUA, Docker)
|
||||
- Opens after N failures in T window
|
||||
- Blocks all scanner commands while open
|
||||
|
||||
**Implementation:**
|
||||
- `agent/internal/circuitbreaker/circuitbreaker.go`
|
||||
- Failure threshold: 5 failures in 60 seconds
|
||||
- Open duration: 5 minutes
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Replay attacks are rare — protection is defense-in-depth, not primary security.
|
||||
|
||||
**Connection:** Nonce validation (`verification/04-replay-protection.md`) complements machine binding (`security/02-authentication-stack.md`).
|
||||
|
||||
**Connection:** Command deduplication (`verification/04-replay-protection.md`) implements ETHOS #4 (idempotency).
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) implements ETHOS #3 (assume failure).
|
||||
|
||||
**Connection:** Nonce service (`server/internal/services/update_nonce.go`) ties to security settings (`security_settings.operational`).
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
Loading…
Reference in a new issue