projection: begin the exact-path public epoch
The public tree and its history contain only the listed paths. Earlier projection history remains preserved internally. Source-Sha: 913fde029b935671833254797f0f20f1eb9fabba Policy-Sha: 913fde029b935671833254797f0f20f1eb9fabba Tree-Digest: 180ae530c1058a2a5c89837bdce2d323ae83e669e38590ca72e75b8d92b7262f
This commit is contained in:
commit
67e26be2d9
838 changed files with 157302 additions and 0 deletions
135
RAF/scanners/01-windows-updates.md
Normal file
135
RAF/scanners/01-windows-updates.md
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
# Windows Updates Scanner
|
||||
|
||||
**Windows Update API integration for detecting and managing Windows updates.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Package | `windowsupdate` (Go) — https://github.com/ceshihao/windowsupdate |
|
||||
| Platform | Windows |
|
||||
| Execution time | ~30 seconds per scan |
|
||||
| Output format | JSON array of update objects |
|
||||
| Failure modes | WUA service unavailable, network timeout, API errors |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/pkg/windowsupdate/client.go`
|
||||
|
||||
```go
|
||||
func (c *Client) ScanUpdates(ctx context.Context) ([]windowsupdate.Update, error) {
|
||||
// 1. Initialize Windows Update API
|
||||
client := windowsupdate.NewClient()
|
||||
client.SetTimeout(30 * time.Second)
|
||||
|
||||
// 2. Query for updates
|
||||
updates, err := client.Update()
|
||||
if err != nil {
|
||||
logSecurityEvent("[reliability] [agent] [windows] WUA query failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 3. Filter ghost packages
|
||||
updates = filterGhostPackages(updates)
|
||||
|
||||
// 4. Return filtered updates
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The Windows scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_windows":
|
||||
updates, err := windowsupdate.ScanWindowsUpdates(ctx, agentID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "windows",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_windows" command
|
||||
↓
|
||||
Agent receives command
|
||||
↓
|
||||
ScanWindowsUpdates() queries Windows Update API
|
||||
↓
|
||||
Filter ghost packages (known issue: Windows occasionally reports stale updates)
|
||||
↓
|
||||
Return updates array
|
||||
↓
|
||||
Agent reports scan results to server
|
||||
↓
|
||||
Server displays updates in dashboard (SystemEvents table)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
### Ghost Packages
|
||||
|
||||
**Problem:** Windows Update API sometimes reports packages that are already installed or no longer available.
|
||||
|
||||
**Detection:** Windows Update API doesn't provide an easy way to filter these. Workaround: maintain a cache of previously seen update KB numbers and filter out duplicates.
|
||||
|
||||
**Status:** Partial fix — filtered in `filterGhostPackages()` but may not catch all cases.
|
||||
|
||||
**Cross-references:**
|
||||
- `testing/02-windows-ghost.md` (ghost package test coverage)
|
||||
|
||||
---
|
||||
|
||||
### Update Reappearance
|
||||
|
||||
**Problem:** Some Windows Updates may reappear after installation (known Windows Update quirk).
|
||||
|
||||
**Status:** Known issue — logged but not automatically handled. Requires manual intervention or future fix.
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Windows update scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents Windows scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** Windows scanner (`scanners/01-windows-updates.md`) lives in `agent/pkg/windowsupdate/` package.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
111
RAF/scanners/02-docker-scanner.md
Normal file
111
RAF/scanners/02-docker-scanner.md
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
# Docker Scanner
|
||||
|
||||
**Docker image scanning for container-based agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | Read `/var/run/docker.sock` via Docker SDK |
|
||||
| Platform | Linux (Docker agents) |
|
||||
| Execution time | ~5 seconds per scan |
|
||||
| Output format | `[]client.UpdateReportItem` with full metadata in `Metadata` map |
|
||||
| Failure modes | Docker daemon unavailable, socket permission denied, registry rate-limited |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/orchestrator/docker_scanner.go`
|
||||
|
||||
The Docker scanner connects directly to the local Docker daemon via the Docker SDK, lists all containers, inspects each image, then queries the remote registry (Docker Hub or custom) to compare digests.
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/docker_scanner.go
|
||||
func (s *DockerScanner) Scan() ([]client.UpdateReportItem, error) {
|
||||
containers, err := s.client.ContainerList(ctx, container.ListOptions{All: true})
|
||||
// ... inspect each image, compare local vs remote digest ...
|
||||
items = append(items, client.UpdateReportItem{
|
||||
PackageType: "docker_image",
|
||||
PackageName: imageName,
|
||||
CurrentVersion: localShortDigest,
|
||||
AvailableVersion: remoteShortDigest,
|
||||
Severity: severity,
|
||||
RepositorySource: baseImage,
|
||||
Metadata: map[string]interface{}{
|
||||
"has_update": hasUpdate,
|
||||
"image_id": localShortDigest,
|
||||
"latest_image_id": remoteShortDigest,
|
||||
// ... container info, labels, etc.
|
||||
},
|
||||
})
|
||||
return items, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Registry Client
|
||||
|
||||
The `RegistryClient` within `docker_scanner.go` handles:
|
||||
- Docker Hub token authentication (`auth.docker.io`)
|
||||
- Registry API v2 manifest fetch via `Docker-Content-Digest` header
|
||||
- 5-minute TTL cache to avoid rate limits
|
||||
- Custom registry support (gcr.io, etc.) via domain detection in `parseImageName()`
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Orchestrator Registration
|
||||
|
||||
Registered in `agent/internal/agent/loop.go` as a direct `orchestrator.Scanner` implementation:
|
||||
|
||||
```go
|
||||
dockerScanner, _ := orchestrator.NewDockerScanner()
|
||||
scanOrchestrator.RegisterScanner("docker", dockerScanner, dockerCB, ...)
|
||||
```
|
||||
|
||||
### 2. Command Dispatch
|
||||
|
||||
The `HandleScanDocker` handler in `agent/internal/handlers/scan.go` runs the scan once through the orchestrator and reads results from `result.Updates[]` — no double-scanning.
|
||||
|
||||
### 3. Server-Side Storage
|
||||
|
||||
Scan results are reported via `client.ReportDockerImages()` to `POST /api/v1/agents/:id/docker-images`. The server stores them in `docker_images` table via `server/internal/database/queries/docker.go` (`CreateDockerEventsBatch`, `GetDockerImages`, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_docker" command
|
||||
↓
|
||||
Agent receives command → orchestrator.ScanSingle("docker")
|
||||
↓
|
||||
DockerScanner.Scan() connects to local Docker daemon
|
||||
↓
|
||||
List containers, inspect images, check registry digests
|
||||
↓
|
||||
Return UpdateReportItems with full metadata
|
||||
↓
|
||||
Handler reports results to server API endpoint
|
||||
↓
|
||||
Server stores in docker_images table
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** Docker daemon is available on the agent host (`/var/run/docker.sock`). Registry access requires outbound internet (or mirror configuration).
|
||||
|
||||
**Connection:** Scanner registration (`agent/internal/agent/loop.go`) wires Docker into the orchestrator alongside APT, DNF, Windows, and Winget scanners — all now implement `orchestrator.Scanner` directly (no wrapper layer).
|
||||
|
||||
**Connection:** Registry client (`orchestrator/docker_scanner.go`) uses ETHOS `[TAG] [system] [component]` logging for registry failures.
|
||||
|
||||
**Connection:** Server-side `docker_images` table (`server/internal/database/queries/docker.go`) stores reported scan results for dashboard display.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-27*
|
||||
133
RAF/scanners/03-apt-scanner.md
Normal file
133
RAF/scanners/03-apt-scanner.md
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
# APT Scanner
|
||||
|
||||
**APT package manager scanning for Debian/Ubuntu-based Linux agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | `apt list --upgradable -o APT::Get::List-Cleanup=false` |
|
||||
| Platform | Linux (Debian, Ubuntu) |
|
||||
| Execution time | ~10 seconds per scan |
|
||||
| Output format | JSON array of package objects with version info |
|
||||
| Failure modes | APT lock held, network timeout, permission denied |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/scanner/apt.go`
|
||||
|
||||
```go
|
||||
func ScanAPT(ctx context.Context) ([]APTUpdate, error) {
|
||||
// 1. Run apt list --upgradable
|
||||
cmd := exec.CommandContext(ctx, "bash", "-c",
|
||||
"apt list --upgradable -o APT::Get::List-Cleanup=false 2>&1")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
logSecurityEvent("[reliability] [agent] [apt] apt list failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2. Parse output (format: "package/old_version -> new_version")
|
||||
var updates []APTUpdate
|
||||
lines := strings.Split(string(output), "\n")
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "Listing...") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse "package/old_version -> new_version"
|
||||
parts := strings.Split(line, "->")
|
||||
if len(parts) != 2 {
|
||||
continue
|
||||
}
|
||||
|
||||
pkgInfo := strings.Split(strings.TrimSpace(parts[0]), "/")
|
||||
if len(pkgInfo) != 2 {
|
||||
continue
|
||||
}
|
||||
|
||||
updates = append(updates, APTUpdate{
|
||||
Package: pkgInfo[0],
|
||||
OldVersion: pkgInfo[1],
|
||||
NewVersion: strings.TrimSpace(parts[1]),
|
||||
})
|
||||
}
|
||||
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The APT scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_apt":
|
||||
updates, err := apt.ScanAPT(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "apt",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Agent Poll → Server creates "scan_apt" command
|
||||
↓
|
||||
Agent receives command
|
||||
↓
|
||||
ScanAPT() runs apt list --upgradable
|
||||
↓
|
||||
Parse output and extract package/version info
|
||||
↓
|
||||
Return updates array
|
||||
↓
|
||||
Agent reports scan results to server
|
||||
↓
|
||||
Server displays updates in dashboard
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** APT scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents APT scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** APT scanner (`scanners/03-apt-scanner.md`) lives in `agent/internal/scanner/apt.go`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
134
RAF/scanners/04-dnf-scanner.md
Normal file
134
RAF/scanners/04-dnf-scanner.md
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
# DNF Scanner
|
||||
|
||||
**DNF package manager scanning for Fedora/RHEL-based Linux agents.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | `dnf check-update --refresh` |
|
||||
| Platform | Linux (Fedora, RHEL, CentOS) |
|
||||
| Execution time | ~15 seconds per scan |
|
||||
| Output format | JSON array of package objects with version info |
|
||||
| Failure modes | DNF lock held, network timeout, permission denied |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
**File:** `agent/internal/scanner/dnf.go`
|
||||
|
||||
```go
|
||||
func ScanDNF(ctx context.Context) ([]DNFUpdate, error) {
|
||||
// 1. Run dnf check-update --refresh
|
||||
cmd := exec.CommandContext(ctx, "dnf", "check-update", "--refresh")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
// Check if it's a no-update case (exit code 100)
|
||||
if exitErr, ok := err.(*exec.ExitError); ok {
|
||||
if exitErr.ExitCode() == 100 {
|
||||
return []DNFUpdate{}, nil
|
||||
}
|
||||
}
|
||||
logSecurityEvent("[reliability] [agent] [dnf] dnf check-update failed:", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2. Parse output
|
||||
var updates []DNFUpdate
|
||||
lines := strings.Split(string(output), "\n")
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "Last metadata expiration check") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse "package-name.old_version.new_version"
|
||||
parts := strings.Fields(line)
|
||||
if len(parts) < 3 {
|
||||
continue
|
||||
}
|
||||
|
||||
pkg := parts[0]
|
||||
oldVersion := parts[1]
|
||||
newVersion := parts[2]
|
||||
|
||||
updates = append(updates, DNFUpdate{
|
||||
Package: pkg,
|
||||
OldVersion: oldVersion,
|
||||
NewVersion: newVersion,
|
||||
})
|
||||
}
|
||||
|
||||
return updates, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Command Dispatch
|
||||
|
||||
The DNF scanner is invoked via command execution flow:
|
||||
|
||||
```go
|
||||
// agent/internal/orchestrator/system_scanner.go
|
||||
case "scan_dnf":
|
||||
updates, err := dnf.ScanDNF(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
report := &SystemEvent{
|
||||
AgentID: agentID,
|
||||
EventType: EventTypeAgentScan,
|
||||
ScanType: "dnf",
|
||||
Data: updates,
|
||||
}
|
||||
return reportSystemEvent(report)
|
||||
```
|
||||
|
||||
### 2. Circuit Breaker
|
||||
|
||||
- **Failure threshold:** 5 failures in 60 seconds
|
||||
- **Open duration:** 300 seconds (5 minutes)
|
||||
- **Half-open attempts:** 3 consecutive successes to recover
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #3: Assume Failure)
|
||||
- `verification/04-replay-protection.md` (circuit breaker integration)
|
||||
|
||||
---
|
||||
|
||||
## Systemd Integration
|
||||
|
||||
The DNF scanner requires two paths writable under `ProtectSystem=strict`:
|
||||
|
||||
- **`/var/log`** — dnf5 writes `/var/log/dnf5.log`
|
||||
- **`/var/cache`** — dnf5 creates temp files at `/var/cache/libdnf5/`
|
||||
|
||||
**Files to update when locking down a new agent:**
|
||||
- Live systemd unit: `/etc/systemd/system/redflag-agent.service` → add to `ReadWritePaths`
|
||||
- Installer template: `agent/internal/installer/sudoers.go:CreateSystemdService()` → same change
|
||||
|
||||
**Cross-references:**
|
||||
- `core/01-ethos.md` (principle #1: Errors are History — log writes must succeed)
|
||||
- `agent/internal/installer/sudoers.go` (systemd service template)
|
||||
|
||||
---
|
||||
|
||||
## Footer: Assumptions & Connections
|
||||
|
||||
**Assumption:** DNF scanning is a periodic operation that runs independently of command execution.
|
||||
|
||||
**Connection:** Circuit breaker (`verification/04-replay-protection.md`) prevents DNF scanner from blocking other subsystems.
|
||||
|
||||
**Connection:** System events (`flows/04-heartbeat.md`) published to history table for audit trail.
|
||||
|
||||
**Connection:** DNF scanner (`scanners/04-dnf-scanner.md`) lives in `agent/internal/scanner/dnf.go`.
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-05-26*
|
||||
197
RAF/scanners/05-process-scanner.md
Normal file
197
RAF/scanners/05-process-scanner.md
Normal file
|
|
@ -0,0 +1,197 @@
|
|||
# Process Scanner
|
||||
|
||||
**On-demand /proc filesystem scanning for process inventory and drill-down detail.**
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | Direct `/proc` filesystem reads (no subprocess spawns) |
|
||||
| Platform | Linux only (stub on other platforms) |
|
||||
| Execution time | ~200ms for 200-process snapshot; ~50ms per drill-down |
|
||||
| Trigger | On-demand when user opens the Processes tab in the dashboard |
|
||||
| Data model | 25+ fields per process (osquery parity) + 7 related data types |
|
||||
| Storage | Dedicated tables: `agent_process_snapshots`, `agent_processes`, `agent_process_related` |
|
||||
| Retention | Last 10 snapshots per agent (auto-cleanup) |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
The process scanner follows the existing command-dispatch pattern:
|
||||
|
||||
1. **Dashboard** opens Processes tab → `POST /api/v1/agents/:id/processes/scan`
|
||||
2. **Server** creates a signed `scan_processes` command (with dedup check)
|
||||
3. **Agent** polls for commands, receives `scan_processes`, calls `system.GetFullProcessSnapshot()`
|
||||
4. **Agent** reports snapshot to `POST /api/v1/agents/:id/process-scan`
|
||||
5. **Server** stores snapshot + processes + related data, cleans up old snapshots
|
||||
6. **Dashboard** reads latest snapshot via `GET /api/v1/agents/:id/processes`
|
||||
|
||||
On drill-down (clicking a process row):
|
||||
1. **Dashboard** requests `GET /api/v1/agents/:id/processes/:processId`
|
||||
2. **Server** returns process + all related data (open files, sockets, pipes, env, memory map, namespaces, listening ports)
|
||||
|
||||
---
|
||||
|
||||
## Data Collection
|
||||
|
||||
### List Scan (`GetFullProcessSnapshot`)
|
||||
|
||||
Reads `/proc/[pid]/stat`, `/proc/[pid]/status`, `/proc/[pid]/exe`, `/proc/[pid]/cmdline`, `/proc/[pid]/cwd`, `/proc/[pid]/cgroup`, `/proc/[pid]/io` for every PID. No related data collected at this stage.
|
||||
|
||||
**Fields (25+):** PID, Name, Path, Cmdline, Cwd, State, UID, GID, EUID, EGID, User, Group, TTY, TTYName, CPUSecondsUser, CPUSecondsSystem, CPUPercent, RSSBytes, VMSBytes, MemPercent, Threads, Nice, StartTimeSeconds, ParentPID, ProcessGroupID, ElevationStatus, OnDisk, DiskBytesRead, DiskBytesWritten, Cgroup, Unit, ContainerID, ContainerRuntime, CapabilitiesEffective
|
||||
|
||||
### Drill-Down (`GetProcessDetail`)
|
||||
|
||||
Adds related data from a single `/proc/[pid]/fd/` walk (consolidated from three separate traversals):
|
||||
|
||||
| Data Type | Source | Cap (configurable) |
|
||||
|-----------|--------|---------------------|
|
||||
| Open files | `/proc/[pid]/fd/` symlink targets | `max_open_files` (default 2000) |
|
||||
| Open sockets | `/proc/[pid]/net/tcp`, `tcp6`, `unix` | `max_sockets` (default 500) |
|
||||
| Open pipes | `/proc/[pid]/fd/` pipe inodes | `max_pipes` (default 500) |
|
||||
| Environment keys | `/proc/[pid]/environ` (keys only, no values) | `max_env_keys` (default 200) |
|
||||
| Memory map | `/proc/[pid]/maps` | `max_memory_map` (default 2000) |
|
||||
| Namespaces | `/proc/[pid]/ns/` symlinks | `max_namespaces` (default 50) |
|
||||
| Listening ports | Socket inode correlation with `/proc/net/tcp` | `max_listening_ports` (default 100) |
|
||||
| Capabilities | `CapEff` mask read during the list scan, decoded into names here | — |
|
||||
|
||||
### Key Implementation Detail: Ownership Attribution
|
||||
|
||||
`/proc/[pid]/cgroup` answers who owns a process, which is the join between the process
|
||||
list and the service, container, and package inventories. Without it a process list is a
|
||||
task manager: a name, a number, and no way to ask what put it there.
|
||||
|
||||
`parseCgroupOwner` produces four fields from that one file:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `cgroup` | the path the attribution was read from, kept so a surprising answer can be checked |
|
||||
| `unit` | innermost systemd unit — `nginx.service`, `app-firefox-4090.scope` |
|
||||
| `container_id` | full container ID as the kernel spells it |
|
||||
| `container_runtime` | `docker`, `podman`, `containerd`, `crio`, `lxc` |
|
||||
|
||||
Both cgroup generations are handled. v2 is the single `0::<path>` line. v1 has one line per
|
||||
controller and the paths can disagree, so the `name=systemd` hierarchy wins when present —
|
||||
it is the one that carries unit and container scopes.
|
||||
|
||||
Container detection covers both cgroup drivers, because the same runtime writes different
|
||||
paths depending on how it was configured: systemd driver gives `docker-<id>.scope`,
|
||||
`libpod-<id>.scope`, `cri-containerd-<id>.scope`; cgroupfs driver gives the bare ID under
|
||||
`/docker/` or `/kubepods/.../pod<uid>/`. LXC carries a name rather than a hash. A scope
|
||||
whose suffix is not hex of length 12 or 64 is a unit, not a container — `docker-notahash.scope`
|
||||
attributes as a unit.
|
||||
|
||||
**The Docker join is a prefix match, not equality.** `container_id` here is the full ID
|
||||
from the kernel; the Docker scanner reports `c.ID[:12]` in its inventory
|
||||
(`agent/internal/orchestrator/docker_scanner.go`). A consumer joining the two compares
|
||||
prefixes. Truncating the kernel's answer to match one scanner's display width would be
|
||||
fabricating uniformity, which the mutation protocol forbids for the same reason.
|
||||
|
||||
Kernel threads have cgroup `/` and stay unattributed, correctly — they have no unit, no
|
||||
container, and no package. On a 321-process Arch desktop that is 172 of them, and every
|
||||
userland process attributes.
|
||||
|
||||
### Key Implementation Detail: Effective Capabilities
|
||||
|
||||
`CapEff` in `/proc/[pid]/status` is the mask that decides what a process may actually do
|
||||
to the machine — mount filesystems, load modules, trace another process, rewrite the
|
||||
network stack. UID alone does not answer that: a non-root process can hold
|
||||
`CAP_NET_ADMIN`, and a root process in a container usually holds far less than the full
|
||||
set.
|
||||
|
||||
The mask is read during the list scan, because it costs nothing — the status file is
|
||||
already open. It is expanded into names only on drill-down: fully privileged processes
|
||||
hold all 41, and 41 strings on every row of a 300-process inventory is payload nobody
|
||||
reads. `decodeCapabilities` is a pure function over the hex mask, so it is fixture-tested
|
||||
without a machine.
|
||||
|
||||
A bit past `CAP_LAST_CAP` is reported as `CAP_<n>` rather than dropped. A newer kernel
|
||||
than this table should produce an unfamiliar name, not silence.
|
||||
|
||||
Verification against a known constant: Docker's default container mask, `a80425fb`,
|
||||
decodes to exactly the fourteen capabilities Docker documents itself as granting.
|
||||
|
||||
### Key Implementation Detail: Socket Inode Correlation
|
||||
|
||||
Listening ports are per-process, not system-wide. The scanner collects socket inodes from `/proc/[pid]/fd/` symlinks (`socket:[12345]`), then matches them against inode numbers in `/proc/net/tcp` and `/proc/net/tcp6`. Only LISTEN state (0A) entries whose inode matches a process socket are included.
|
||||
|
||||
### Key Implementation Detail: IPv6 Address Parsing
|
||||
|
||||
The kernel stores IPv6 addresses in `/proc/net/tcp6` as 4 little-endian 32-bit words (`%08X%08X%08X%08X`). The parser reverses bytes within each 4-byte group (not across the entire 16-byte address) and uses `%02x` for zero-padded output.
|
||||
|
||||
### Key Implementation Detail: `/proc/[pid]/stat` Field Indices
|
||||
|
||||
After stripping `pid (comm)`, the `fields` array is 0-indexed from field 3:
|
||||
- `[0]`=state, `[1]`=ppid, `[2]`=pgrp, `[3]`=session, `[4]`=tty_nr
|
||||
- `[11]`=utime, `[12]`=stime, `[16]`=nice, `[19]`=starttime
|
||||
|
||||
---
|
||||
|
||||
## Configurable Caps
|
||||
|
||||
Data collection limits are server-controlled via `ProcessExplorerConfig` (stored in security settings under `operational` category). Delivered to agents on check-in. Set to 0 for no cap.
|
||||
|
||||
| Setting | Default | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `process_explorer_max_open_files` | 2000 | Open file descriptors per process |
|
||||
| `process_explorer_max_sockets` | 500 | Open sockets per process |
|
||||
| `process_explorer_max_pipes` | 500 | Open pipes per process |
|
||||
| `process_explorer_max_memory_map` | 2000 | Memory map entries per process |
|
||||
| `process_explorer_max_namespaces` | 50 | Namespace entries per process |
|
||||
| `process_explorer_max_env_keys` | 200 | Environment variable keys per process |
|
||||
| `process_explorer_max_listening_ports` | 100 | TCP listening ports per process |
|
||||
|
||||
**UI:** Settings → Process Explorer (`/settings/process-explorer`)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Files
|
||||
|
||||
### Agent
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `agent/internal/system/process_detail.go` | Types: `FullProcess`, `ProcessOpenFile`, `ProcessOpenSocket`, `ProcessOpenPipe`, `ProcessMemoryMap`, `ProcessNamespace`, `ProcessListeningPort`, `ProcessCaps`, `FullProcessSnapshot` |
|
||||
| `agent/internal/system/process_owner.go` | `ProcessOwner` and `parseCgroupOwner` — cgroup attribution, platform-independent and fixture-tested |
|
||||
| `agent/internal/system/process_capabilities.go` | `decodeCapabilities` and the capability-bit table — platform-independent and fixture-tested |
|
||||
| `agent/internal/system/process_detail_linux.go` | Linux `/proc` reader: `getFullProcessSnapshot()`, `getProcessDetail()`, `walkProcFD()`, `readListeningPorts()`, `parseHexAddr()` |
|
||||
| `agent/internal/system/process_detail_other.go` | Stub for non-Linux platforms |
|
||||
| `agent/internal/handlers/processes.go` | `HandleScanProcesses` — command handler |
|
||||
| `agent/internal/client/client.go` | `ReportProcessScan` — reports to server |
|
||||
|
||||
### Server
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server/internal/database/migrations/055_create_process_tables.up.sql` | Schema: `agent_process_snapshots`, `agent_processes`, `agent_process_related` |
|
||||
| `server/internal/models/process.go` | Server-side models |
|
||||
| `server/internal/database/queries/processes.go` | `ProcessQueries` — insert, query, cleanup |
|
||||
| `server/internal/api/handlers/processes.go` | `ProcessHandler` — 4 endpoints |
|
||||
| `server/internal/models/command.go` | `CommandTypeScanProcesses` constant |
|
||||
|
||||
### Web
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `web/src/types/process.ts` | TypeScript interfaces |
|
||||
| `web/src/hooks/useProcesses.ts` | React Query hooks |
|
||||
| `web/src/components/ProcessesTab.tsx` | Main tab with sortable table |
|
||||
| `web/src/components/ProcessDetailModal.tsx` | 6-tab modal (Overview, Network, Files, Environment, Memory, Namespaces) |
|
||||
| `web/src/hooks/useProcessExplorer.ts` | Settings hooks |
|
||||
| `web/src/pages/settings/ProcessExplorer.tsx` | Settings UI |
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Environment variable values are never transmitted.** Only key names are collected. Env vars may contain secrets (API keys, database passwords).
|
||||
- **No subprocess spawns.** All data comes from direct `/proc` reads — no `ps`, `lsof`, or similar commands.
|
||||
- **On-demand only.** The scan command is only issued when a user opens the Processes tab. No background broadcasting.
|
||||
- **Command dedup.** The server checks for existing pending `scan_processes` commands before creating a new one.
|
||||
|
||||
---
|
||||
|
||||
*Added: 2026-06-10 · ownership attribution and effective capabilities 2026-08-31*
|
||||
98
RAF/scanners/06-pacman-scanner.md
Normal file
98
RAF/scanners/06-pacman-scanner.md
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
# Pacman Scanner and Resolver
|
||||
|
||||
**Arch update discovery stays unprivileged; approved mutation crosses the signed helper envelope.**
|
||||
|
||||
---
|
||||
|
||||
## Discovery
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Method | `checkupdates --color=never` |
|
||||
| Platform | Arch Linux and pacman derivatives |
|
||||
| Output | `pkgname oldver -> newver` |
|
||||
| Root required | No |
|
||||
| Requirements | `pacman`, `pacman-contrib`, `fakeroot` |
|
||||
|
||||
`agent/internal/scanner/pacman.go` runs pacman-contrib's `checkupdates` through
|
||||
`DiscoveryRunner`. `checkupdates` synchronizes a private database and compares it with the
|
||||
installed local database. The Agent has no `pacman -Sy` sudo grant and discovery never
|
||||
refreshes live sync state.
|
||||
|
||||
Epoch-bearing versions remain opaque pacman versions. The parser preserves a line such as
|
||||
`fakeroot 1:1.37.2-1 -> 1:1.37.2-2`; it does not split or reinterpret the epoch.
|
||||
|
||||
---
|
||||
|
||||
## Standalone closure resolution
|
||||
|
||||
`agent/internal/installer/pacman_resolver_linux.go` owns the approval-time resolver:
|
||||
|
||||
1. create an Agent-private operation directory, sync database, and cache;
|
||||
2. symlink only the installed `local` database into that private database;
|
||||
3. run `fakeroot pacman -Sy` against the private database with a cleared, C locale;
|
||||
4. resolve the requested exact `name=version` transaction and repository identities;
|
||||
5. download the transaction with `pacman -Sw` into the private cache;
|
||||
6. inspect each archive with locale-pinned `pacman -Qp`, require its adjacent detached
|
||||
signature, and hash both files;
|
||||
7. keep the cache alive through helper mint and execution, then remove it.
|
||||
|
||||
The resolver returns an error rather than guessing when the requested root is absent, the
|
||||
repository join is missing, an archive or signature is not regular, identity output is
|
||||
ambiguous, or the transaction is empty.
|
||||
|
||||
---
|
||||
|
||||
## Privileged path
|
||||
|
||||
The Agent encodes each package as `name@version` plus an exact JSON payload containing:
|
||||
|
||||
- repository;
|
||||
- whether this action is the one requested root;
|
||||
- archive cache path and SHA-256;
|
||||
- detached-signature cache path and SHA-256.
|
||||
|
||||
The standalone helper will not sign a pacman envelope without every detached signature.
|
||||
Before mint it copies Agent paths into a root-owned `0700` operation directory, verifies
|
||||
both hashes, checks archive identity with `pacman -Qp`, verifies the detached signature
|
||||
through the Arch keyring, and uses pacman's `vercmp` against installed state. Execution
|
||||
repeats those checks, atomically consumes the authorization, and invokes:
|
||||
|
||||
```text
|
||||
/usr/bin/pacman -U --noconfirm -- <staged archives...>
|
||||
```
|
||||
|
||||
Source and exchange files are opened without following the final symlink. Helper exchange
|
||||
directories are walked component-by-component with `openat(..., O_NOFOLLOW)` and result
|
||||
files are create-new, so an Agent-controlled path cannot redirect a root write through a
|
||||
swapped directory or pre-existing link.
|
||||
|
||||
There is no `--needed`: a successful receipt cannot conceal a package-manager skip. The
|
||||
helper requires exactly one requested root. `upgrade` requires that root to be installed at
|
||||
a strictly older version; `install` requires it to be absent. Equal versions remain valid
|
||||
dependencies, while every older target is refused.
|
||||
|
||||
---
|
||||
|
||||
## Limits
|
||||
|
||||
- AUR packages are not scanned or resolved.
|
||||
- Requested-root identity lives in the signed pacman payload rather than the common
|
||||
manifest, so other backends do not inherit pacman semantics accidentally.
|
||||
- RedFlag currently has no OSV ecosystem mapping for Arch. Local approval records
|
||||
`unsupported` and requires an explicit override reason.
|
||||
- Package signature verification proves the artifact against the local Arch keyring. It
|
||||
does not add reproducible-build or transparency-log evidence.
|
||||
|
||||
---
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [security/05-supply-chain-gate](../security/05-supply-chain-gate.md)
|
||||
- [security/06-standalone-authority](../security/06-standalone-authority.md)
|
||||
- [components/04-helper](../components/04-helper.md)
|
||||
- `protocol/README.md`
|
||||
|
||||
---
|
||||
|
||||
*Last reviewed: 2026-09-01*
|
||||
Loading…
Reference in a new issue