- Remove unhashed 'unknown-' fallback from registration (F-D1-1)
Registration aborts if GetMachineID() fails (no bad data)
- Add POST /admin/agents/:id/rebind-machine-id endpoint (F-D1-2)
Admin can update stored machine ID after hardware change
- Delete dead example_integration.go with wrong usage (F-D1-3)
- Remove redundant Windows machineid.ID() retry (F-D1-4)
- Replace fmt.Printf with log.Printf in client.go (F-D1-5)
Operator note: agents registered with 'unknown-' machine IDs
must be rebound before upgrading. See D1_Fix_Implementation.md.
All tests pass. No regressions.
Comprehensive audit of machine ID implementations across the
agent codebase. Identified 3 production call sites with 1 critical
divergence.
Key findings:
- F-D1-1 HIGH: Registration fallback "unknown-"+hostname is unhashed,
mismatches runtime SHA256 hash, causes permanent agent lockout
when GetMachineID() transiently fails then recovers
- F-D1-2 MEDIUM: No recovery path from machine ID mismatch
- F-D1-3 LOW: example_integration.go is dead code calling
machineid.ID() directly (bypasses canonical hashing)
- F-D1-4 LOW: Windows redundant machineid.ID() retry
- F-D1-5 LOW: client.go uses fmt.Printf for machine ID error
6 findings total. See docs/D1_MachineID_Audit.md for details.
- Delete install.sh.deprecated (dead code)
- Add TODO(DEV-031) for ghost update scanner-side prevention
- Add TODO(DEV-030) with specific missing service cycles
- ETHOS sweep: zero banned words, emojis, or fmt.Printf
- All tests pass, Linux builds clean
- Apply B-2 jitter and backoff fixes to Windows service (F-C1-5)
Proportional jitter and exponential backoff now in service polling loop
- Add known winget install location search for SYSTEM account (F-C1-1)
Checks PATH then system-wide WindowsApps locations
- Fix winget text parser for package names with spaces (F-C1-2)
Column-position parsing from header keywords replaces whitespace split
- Add ghost update post-install state verification (F-C1-3)
RebootRequired flag on InstallResult marks pending reboot
- Replace fmt.Printf with log.Printf in winget scanner (F-C1-6)
- Remove emoji from Windows service log messages (F-C1-7)
GOOS=linux build: PASS. All tests pass, no regressions.
Pre-fix test suite for 7 Windows-specific findings. All tests
are SHARED (no build tags) — they compile and run on Linux
using source file inspection and direct function calls.
Tests added:
- F-C1-1 HIGH: Winget PATH-only search (2 tests)
- F-C1-2 MEDIUM: Winget text parser spaces bug (4 tests)
- F-C1-3 HIGH: Ghost updates — no post-install verification (3 tests)
- F-C1-4 RESOLVED: Service auto-restart already configured (1 test)
- F-C1-5 HIGH: Duplicated polling loop missing B-2 fixes (5 tests)
- F-C1-6 LOW: Winget uses fmt.Printf (2 tests)
- F-C1-7 LOW: Service has emojis in logs (2 tests)
Current state: 8 FAIL, 11 PASS. All prior tests pass.
Comprehensive audit of Windows agent code: winget detection,
Windows Update ghost updates, service wrapper, HWID, and
vendored windowsupdate package.
Key findings:
- F-C1-1 HIGH: Winget not found as SYSTEM (PATH-only search)
- F-C1-3 HIGH: No post-install verification (ghost updates)
- F-C1-5 HIGH: Windows service has duplicated polling loop
missing B-2 fixes (jitter cap, exponential backoff)
- F-C1-2 MEDIUM: Fragile winget text parser
- F-C1-4 MEDIUM: No service auto-restart on crash
9 findings total. See docs/C1_Windows_Audit.md for details.
retry_count column and filter existed but counter was never
incremented. Stuck commands always had retry_count=0 and
always passed the WHERE retry_count < 5 filter, making
the cap ineffective.
Fix: Added RedeliverStuckCommandTx that sets
retry_count = retry_count + 1 on stuck->sent re-delivery.
GetCommands handler now uses MarkCommandSentTx for new
commands (retry_count stays 0) and RedeliverStuckCommandTx
for stuck re-delivery (retry_count increments).
All 77 tests pass. DEV-029 resolved.
- Wrap agent registration in DB transaction (F-B2-1/F-B2-8)
All 4 ops atomic, manual DeleteAgent rollback removed
- Use SELECT FOR UPDATE SKIP LOCKED for atomic command delivery (F-B2-2)
Concurrent requests get different commands, no duplicates
- Wrap token renewal in DB transaction (F-B2-9)
Validate + update expiry atomic
- Add rate limit to GET /agents/:id/commands (F-B2-4)
agent_checkin rate limiter applied
- Add retry_count column, cap stuck command retries at 5 (F-B2-10)
Migration 029, GetStuckCommands filters retry_count < 5
- Cap polling jitter at current interval (fixes rapid mode) (F-B2-5)
maxJitter = min(pollingInterval/2, 30s)
- Add exponential backoff with full jitter on reconnection (F-B2-7)
calculateBackoff: base=10s, cap=5min, reset on success
All tests pass. No regressions from A-series or B-1.
Pre-fix test suite documenting 7 data integrity and concurrency
bugs. Tests FAIL where they assert correct post-fix behavior,
PASS where they document current buggy state.
Tests added:
- F-B2-1/8 HIGH: Registration not transactional (3 tests)
- F-B2-2 MEDIUM: Command delivery race condition (3 tests)
- F-B2-9 MEDIUM: Token renewal not transactional (2 tests)
- F-B2-4 MEDIUM: No rate limit on GetCommands (3 tests)
- F-B2-5 LOW: Jitter negates rapid mode (2 tests)
- F-B2-10 LOW: No max retry for stuck commands (2 tests)
- F-B2-7 MEDIUM: No exponential backoff on reconnection (2 tests)
Current state: 7 FAIL, 10 PASS. No A/B-1 regressions.
See docs/B2_PreFix_Tests.md for full inventory.
- Fix migration 024 self-insert and bad column reference (F-B1-1, F-B1-2)
Uses existing enabled/auto_run columns instead of non-existent deprecated
- Abort server on migration failure instead of warning (F-B1-11)
main.go now calls log.Fatalf, prints [INFO] only on success
- Fix migration 018 scanner_config filename suffix (F-B1-3)
Renumbered to 027 with .up.sql suffix
- Remove GRANT to non-existent role in scanner_config (F-B1-4)
- Resolve duplicate migration numbers 009 and 012 (F-B1-13)
Renamed to 009b and 012b for unique lexical sorting
- Add IF NOT EXISTS to all non-idempotent migrations (F-B1-15)
Fixed: 011, 012, 017, 023, 023a
- Replace N+1 dashboard stats loop with GetAllUpdateStats (F-B1-6)
Single aggregate query replaces per-agent loop
- Add composite index on agent_commands(status, sent_at) (F-B1-5)
New migration 028 with partial index for timeout service
- Add background refresh token cleanup goroutine (F-B1-10)
24-hour ticker calls CleanupExpiredTokens
- ETHOS log format in migration runner (no emojis)
All 55 tests pass (41 server + 14 agent). No regressions.
See docs/B1_Fix_Implementation.md and DEV-025 through DEV-028.
Comprehensive audit of the custom migration runner, all 26 migrations,
query patterns, foreign keys, and schema state management.
Critical findings:
- F-B1-11 P0: Server starts with incomplete schema after migration
failure, prints [OK] — main.go swallows migration errors
- F-B1-1: Migration 024 self-inserts into schema_migrations
- F-B1-2: Migration 024 references non-existent deprecated column
- F-B1-3: Migration 018 scanner_config has wrong file extension
- F-B1-6: N+1 query in GetDashboardStats (1 query per agent)
15 findings total across P0/CRITICAL/HIGH/MEDIUM/LOW.
See docs/B1_Database_Audit.md for full analysis.
All pre-existing tests pass after dead code cleanup.
No regressions from A-1, A-2, or A-3 fix rounds.
41 tests pass, 1 skip (pre-existing). Zero new failures.
Ready to proceed to B-series database audit.
Fixes 9 auth middleware findings from the A-3 recon audit.
F-A3-11 CRITICAL: Removed JWT secret from WebAuthMiddleware log output.
Replaced emoji-prefixed fmt.Printf with ETHOS-compliant log.Printf.
No secret values in any log output.
F-A3-7 CRITICAL: Config download now requires WebAuthMiddleware.
GET /downloads/config/:agent_id is admin-only (agents never call it).
F-A3-6 HIGH: Update package download now requires AuthMiddleware.
GET /downloads/updates/:package_id requires valid agent JWT.
F-A3-10 HIGH: Scheduler stats changed from AuthMiddleware to
WebAuthMiddleware. Agent JWTs can no longer view scheduler internals.
F-A3-13 LOW: RequireAdmin() middleware implemented. 7 security settings
routes re-enabled (GET/PUT/POST under /security/settings).
security_settings.go.broken renamed to .go, API mismatches fixed.
F-A3-12 MEDIUM: JWT issuer claims added for token type separation.
Agent tokens: issuer=redflag-agent, Web tokens: issuer=redflag-web.
AuthMiddleware rejects tokens with wrong issuer.
Grace period: tokens with no issuer still accepted (backward compat).
F-A3-2 MEDIUM: /auth/verify now has WebAuthMiddleware applied.
Endpoint returns 200 with valid=true for valid admin tokens.
F-A3-9 MEDIUM: Agent self-unregister (DELETE /:id) now rate-limited
using the same agent_reports rate limiter as other agent routes.
F-A3-14 LOW: CORS origin configurable via REDFLAG_CORS_ORIGIN env var.
Defaults to http://localhost:3000 for development.
Added PATCH method and agent-specific headers to CORS config.
All 27 server tests pass. All 14 agent tests pass. No regressions.
See docs/A3_Fix_Implementation.md and docs/Deviations_Report.md
(DEV-020 through DEV-022).
Pre-fix test suite documenting 8 auth middleware bugs found during
the A-3 recon audit. Tests are written to FAIL where they assert
correct post-fix behavior, and PASS where they document current
buggy behavior. No bugs are fixed in this commit.
Tests added:
- F-A3-11 CRITICAL: WebAuthMiddleware leaks JWT secret to stdout
(3 tests: secret in output, emoji in output, ETHOS format)
- F-A3-7 CRITICAL: Config download requires no auth (2 tests)
- F-A3-6 HIGH: Update package download requires no auth (2 tests)
- F-A3-10 HIGH: Scheduler stats accepts agent JWT (2 tests)
- F-A3-12 MEDIUM: Cross-type JWT token confusion (2 tests)
- F-A3-2 MEDIUM: /auth/verify dead endpoint (2 tests)
- F-A3-13 LOW: RequireAdmin middleware missing (1 test + 1 build-tagged)
- F-A3-9 MEDIUM: Agent self-unregister no rate limit (2 tests)
Current state: 10 FAIL, 7 PASS, 1 SKIP (build-tagged), 1 unchanged
See docs/A3_PreFix_Tests.md for full inventory.
Complete RedFlag codebase with two major security audit implementations.
== A-1: Ed25519 Key Rotation Support ==
Server:
- SignCommand sets SignedAt timestamp and KeyID on every signature
- signing_keys database table (migration 020) for multi-key rotation
- InitializePrimaryKey registers active key at startup
- /api/v1/public-keys endpoint for rotation-aware agents
- SigningKeyQueries for key lifecycle management
Agent:
- Key-ID-aware verification via CheckKeyRotation
- FetchAndCacheAllActiveKeys for rotation pre-caching
- Cache metadata with TTL and staleness fallback
- SecurityLogger events for key rotation and command signing
== A-2: Replay Attack Fixes (F-1 through F-7) ==
F-5 CRITICAL - RetryCommand now signs via signAndCreateCommand
F-1 HIGH - v3 format: "{agent_id}:{cmd_id}:{type}:{hash}:{ts}"
F-7 HIGH - Migration 026: expires_at column with partial index
F-6 HIGH - GetPendingCommands/GetStuckCommands filter by expires_at
F-2 HIGH - Agent-side executedIDs dedup map with cleanup
F-4 HIGH - commandMaxAge reduced from 24h to 4h
F-3 CRITICAL - Old-format commands rejected after 48h via CreatedAt
Verification fixes: migration idempotency (ETHOS #4), log format
compliance (ETHOS #1), stale comments updated.
All 24 tests passing. Docker --no-cache build verified.
See docs/ for full audit reports and deviation log (DEV-001 to DEV-019).
- Line 833 in ChatTimeline still had scan_updates
- Updated to check platform-specific scans (apt, dnf, winget, windows)
- Verifying: no scan_updates references remain in frontend
- scan_updates removed following FINAL_Issue3 design
- Platform-specific scans only (Windows: scan_winget + scan_windows, Linux: scan_apt/scan_dnf)
- Updated command naming service to include platform-specific scans
- All subsystem handlers properly isolated per design spec
- Replace all .replace() calls with formatCommandAction
- Replace hard-coded scan conditionals with dynamic naming
- All command display now uses centralized service
Command naming service now fully integrated for ETHOS compliance.
- Create ETHOS-compliant command naming service
- Add imports to ChatTimeline
- Replace .replace() calls with formatCommandAction
- Partial scan conditional replacement
Status: Command naming service exists and is imported, full integration in progress
- Command factory with CreateWithIdempotency support
- SubsystemHandler uses factory for all scan commands
- Idempotency prevents duplicate commands from rapid clicks
- UI updates for AgentStorage and heartbeat
- Includes previous factory, queries, and main.go changes
Now all command creation goes through factory for consistent validation and UUID generation.
- Add commandFactory field to SubsystemHandler
- Update NewSubsystemHandler to accept factory
- Modify TriggerSubsystem to use CreateWithIdempotency
- Update main.go to pass factory to handler
Now all subsystem commands use idempotency keys, preventing duplicates from rapid clicks.
- Add CreateWithIdempotency method to command factory
- Add GetCommandByIdempotencyKey to command queries
- Update CreateCommand to handle idempotency_key field
- Fix system command list to match actual usage
This enables proper idempotency for rapid-click prevention.
- Add ID generation in signAndCreateCommand to prevent zero UUIDs
- Fix storage metrics metadata field (remove pq.Array)
- Add IdempotencyKey field to AgentCommand model
These issues were causing duplicate key violations and storage page errors.
Remove development and investigation files that shouldn't be in repo:
- Kate editor swap files (*.swp, *.kate-swp)
- Discord development folder (contains credentials)
- Development investigation scripts (db_investigation.sh, etc.)
- Configuration files (docker-compose.dev.yml)
Note: Files removed from git but kept locally (rm --cached)
Files are still present in working directory but won't be tracked
The TODO claimed update system was a "placeholder" but the code that
follows implements complete functionality:
- downloadUpdatePackage() downloads the package
- computeSHA256() verifies checksum
- verifyBinarySignature() with ed25519.Verify() checks signature
- installNewBinary() performs atomic installation
- Full rollback/cleanup with defer on failure
The comment was incorrect and misleading about the code quality.
BREAKING CHANGE: Storage and system scans no longer create entries in update_logs
**Problem**
- Storage scans were appearing on Updates page (mixed with package updates)
- System scans were appearing on Updates page (mixed with package updates)
- Duplicate "Scan All" entries from collective + individual logging
**Root Cause**
Scan handlers were calling both ReportLog() and dedicated endpoints:
- reportLogWithAck → POST /api/v1/agents/:id/logs → update_logs table
- This caused storage/system metrics to appear alongside package updates
**Fix**
Removed ALL ReportLog() calls from scan handlers:
1. handleScanUpdatesV2 (lines 44-46): Removed collective logging
2. handleScanStorage (lines 103-105): Use only ReportStorageMetrics
3. handleScanSystem (lines 189-191): Use only ReportMetrics
4. handleScanDocker (lines 269-271): Use only ReportDockerImages
**Verification**
- All 4 handlers have working dedicated endpoints (verified via subagent)
- Routes already registered: POST /storage-metrics, POST /metrics, etc.
- Frontend queries correct endpoints (verified)
- No data loss: dedicated endpoints store in proper tables
**Result**
- Storage scans → storage_metrics table → Storage page only ✅
- System scans → system reporting → System page only ✅
- Package updates → update_logs table → Updates page only ✅
- No duplicate "Scan All" entries ✅
**Files Changed**
- aggregator-agent/cmd/agent/subsystem_handlers.go: Removed 20 lines of ReportLog calls
- internal/api/handlers/agents.go: Command recovery enhancements
- internal/api/handlers/updates.go: Subsystem extraction logic
- internal/database/queries/commands.go: GetStuckCommands query
Problem: Version check middleware blocked old agents from checking in to receive
update commands, creating a deadlock where agents couldn't upgrade because they
were blocked from checking in.
Solution: Modified MachineBindingMiddleware to allow old agents checking in for
commands to proceed IF they have a pending update_agent command. This allows
agents to receive the update command even when below minimum version.
Changes:
- Added grace period logic in middleware for command endpoints
- Check if agent has pending update command before blocking
- If update pending, allow check-in and log it
- Added HasPendingUpdateCommand() to AgentQueries for checking pending updates
- Also added same method to CommandQueries for completeness
This prevents the version tracking deadlock while maintaining security for
agents without pending updates.
NOTE: Need to test that old agents can actually receive and execute update
commands when allowed through this path.
