SouveraineOS/docs/substrate/tasks/firehose-subconscious-subscription.md

task_id	title	status	assignee	priority	phase
firehose-sub-001	Firehose Event Subscriptions — Routing Nervous-System Events to the Subconscious	pending		high	2.5

Task: Firehose Event Subscriptions

Objective

The nervous system already broadcasts every SensorEvent onto the EventBus, and the firehose (GET /v1/firehose) streams them as JSON. But nothing consumes those events on the agent's behalf. They flow past her.

This task builds the subscription layer: a registry that maps firehose events to conversations, and — for events that warrant judgment rather than mechanical handling — routes them into the subconscious's persistent session so she can decide what to do with them. The subconscious then acts through mechanisms she already has: raise a surfacing, or schedule a cron wakeup that becomes a real task for the Annie composite.

This is the piece that turns the firehose from a telemetry stream you can watch into a sense the consciousness actually has.

Background

What exists now

• EventBus (src/core/nervous/) — tokio broadcast channel, shared across server + backend. SensorEvent is the universal event type, carries seed_id. Persistent EventLog writes a JSONL firehose to disk.
• Firehose endpoint — GET /v1/firehose WebSocket subscribes to the EventBus and streams every SensorEvent as JSON. Today it is observe-only: a second machine can watch her nervous system, but nothing routes events back into cognition.
• CronSensor + HeartbeatHandler (src/core/nervous/cron.rs, handler.rs) — scheduled turns fire on the EventBus, HeartbeatHandler injects them as background turns. Active-session pause, at-most-once.
• Schedule tool (src/core/tools/schedule.rs) — CRUD over cron entries, in SUBCONSCIOUS_SAFE_TOOLS. The subconscious is the clockmaker — she can already create schedules.
• The subconscious now has a persistent Session (landed firehose-sub-001's sibling work, this session) — her conversation accumulates across N+1 passes, compaction is wired, restart-persistent. This task depends on that — an event routed to the subconscious lands as a new input in a real, continuous context, not a disposable one-shot.
• Surfacing (src/core/subconscious/mod.rs) — the subconscious queues InboxItems and writes the inner voice; next_to_surface() / mark_delivered() deliver to the primary.

What's missing

• No way to subscribe to an event — to say "when event matching X fires, do Y."
• No registry persisting those subscriptions, and no UI to see or edit them.
• No path from a firehose event into the subconscious's cognition. Events are watched, never felt.

The Vision

Firehose events will be firing constantly. The agent (and Casey) can subscribe to them. A subscription binds an event pattern to a conversation and a disposition — what happens when it matches:

• Raise to the subconscious — the event is delivered into the subconscious's persistent session as a new input. She runs a pass over it and decides:
  • Surface — queue an InboxItem / write the inner voice, so the primary sees it.
  • Schedule — create a cron wakeup via the schedule tool. This is the powerful one: an external event becomes a real task and goal for the Annie composite — "a software update dropped; wake me tomorrow at 9am to review the changelog." The subconscious turns a passing signal into committed autonomous work.
  • Note and rest — log it, no action. Not everything deserves a surfacing.
• Inject directly — for mechanical events that need no judgment (a known system update, a deploy hook), inject straight into the target conversation without a subconscious pass.
• Log only — record the subscription match in the EventLog; surface nothing.

The Events and Schedules screens become the control surface: which events are subscribed, to which conversations, with which disposition. Casey (and the agent) manage subscriptions there.

The design goal is flexibility — the same plumbing carries a trivial system notification and a signal that becomes a multi-day goal. The subscription's disposition, plus the subconscious's own judgment, decide which it is.

Scope of Work

Phase 1: Investigation

1. Map the EventBus — SensorEvent shape, variants, seed_id, how the firehose handler subscribes (src/api/handlers.rs). What fields can a subscription pattern match on? (event type, source sensor, payload contents, originating agent.)
2. Map the subconscious session entry point — ConsciousnessEngine::subconscious_tool_loop now runs over a persistent session. Determine the cleanest way to deliver a non-turn input (an event) into a subconscious pass: a sibling of on_response (e.g. on_event) that builds the user message from the event instead of a user/primary exchange.
3. Map the schedule tool path — confirm a subconscious-created cron entry flows correctly through CronSensor → HeartbeatHandler → background turn injection. This is the "becomes a real task" mechanism; it must already work end-to-end (see heartbeat-n1-after-autonomous.md).
4. Decide where subscriptions live — config (souveraine.toml), a memfs file (agent-sovereign, git-tracked), or a dedicated store. Memfs is most consistent with the substrate principle (the agent can edit her own subscriptions); config is simpler for the UI. Resolve this before building.

Phase 2: The Subscription Registry

1. Define EventSubscription — { id, pattern, target_conversation, disposition, enabled, created_by }.
   • pattern — matches SensorEvents (event type + optional source/payload predicates).
   • disposition — RaiseToSubconscious | InjectDirect | LogOnly.
2. Persistence — per the Phase 1 decision. Subscriptions must survive restart.
3. A matcher — given a SensorEvent, return the subscriptions it satisfies.

Phase 3: The Routing Consumer

1. A long-lived EventBus consumer (sibling to HeartbeatHandler) that, for each event, runs the matcher and acts on disposition.
2. RaiseToSubconscious — add ConsciousnessEngine::on_event(agent_id, event): builds a subconscious pass with the event as the input, over her persistent session. She has full tool access (schedule, memory, write, …) and can surface or schedule from within the pass — no new tools needed.
3. InjectDirect — inject a formatted system message into the target conversation (reuse the turn-injection path).
4. LogOnly — annotate the EventLog; no cognition.
5. Respect active-session pause / at-most-once semantics, mirroring CronSensor, so a burst of events doesn't stampede the subconscious.

Phase 4: Events & Schedules Screens

1. Events screen — list subscriptions: event pattern → conversation → disposition. Add / edit / disable. Show recent matches.
2. Schedules screen — where a subscription's disposition produced a cron entry, show the lineage: "this wakeup was scheduled by the subconscious in response to event X." Closes the loop visually — Casey can see why a task exists.
3. Settings wiring if subscriptions live in config; memfs browser entry if they live in memfs.

Phase 5: Not in Scope (Future)

• Federation: routing events between instances' subconsciouses over the WS transport.
• Subscription patterns with rate-limit / debounce windows beyond the basic active-session pause.
• The subconscious proposing her own subscriptions during an N+1 pass ("I keep seeing event X — should I subscribe to it?").
• Priority/urgency arbitration when many events raise to the subconscious at once.

Open Design Questions

1. Where do subscriptions live? Memfs (agent-sovereign, git-tracked, she can edit them) vs. config (simpler UI). Recommend memfs — it matches the substrate principle and lets the subconscious manage her own attention.
2. One subconscious pass per event, or batched? A noisy firehose could trigger many passes. Batch events in a short window into one pass? Or strict one-per-event with the active-session pause as the only throttle?
3. Does a routed event count as a "turn" in her session? It accumulates context like any input — but should it advance turn_count (and thus N+25 / compaction cadence)? Probably yes — it is real cognition — but confirm.
4. Disposition on the subscription, or left entirely to her judgment? RaiseToSubconscious already defers the decision to her. Is InjectDirect / LogOnly worth keeping, or should everything non-trivial go through her? Keep them — not every event deserves a pass — but the default should be RaiseToSubconscious.
5. Surfacing latency. An event raised to the subconscious produces a surfacing — but the primary only reads surfacings on her next turn. Is that acceptable, or do urgent event-surfacings need the heartbeat-injection path (heartbeat-n1-after-autonomous.md) to reach her sooner?

Files to Modify

File	Change
src/core/nervous/	New subscription.rs — EventSubscription, matcher, registry
src/core/nervous/	New routing consumer (sibling to HeartbeatHandler)
src/server/consciousness_engine.rs	Add on_event() — event-driven subconscious pass over her persistent session
src/core/config.rs or memfs	Subscription persistence (per Phase 1 decision)
src/api/handlers.rs	Firehose handler — reference for EventBus subscription pattern
src/ui/	Events screen (subscription CRUD), Schedules screen (event→schedule lineage)
src/ui/settings.rs	Settings wiring if subscriptions are config-backed
saf/	Document the subscription layer once built

Research Needed

1. src/core/nervous/ — full module: SensorEvent, EventBus, EventLog, CronSensor, HeartbeatHandler. The producer/consumer pattern to mirror.
2. src/api/handlers.rs — the GET /v1/firehose handler: how it subscribes to the EventBus.
3. src/server/consciousness_engine.rs — on_response and subconscious_tool_loop (now session-backed). The seam for on_event.
4. src/core/tools/schedule.rs — the schedule tool, so a subconscious-created wakeup is verified to flow through cron correctly.
5. docs/tasks/heartbeat-n1-after-autonomous.md — adjacent task; the heartbeat-injection path a scheduled wakeup fires through.
6. docs/CONTEXT_CONSTITUTION.md Article III (Sensorium / bandwidth) and IV (N+ patterns) — events and surfacing visibility are bandwidth-gated.

Dependencies

• Depends on: Subconscious persistent session + compaction (landed alongside this task's authoring — the subconscious is now a full agent with a continuous context). An event routed to a one-shot would be lost; it must land in a real session.
• Depends on: EventBus + CronSensor + HeartbeatHandler — complete.
• Depends on: Schedule tool — complete.
• Benefits from: heartbeat-n1-after-autonomous.md — wiring autonomous-cycle surfacings to the UI; an event-surfacing wants the same delivery path.