Watch
1
0
Fork
You've already forked souveraine
0

feat: N+25 reflection engine — callable, wired into N+25 trigger

ReflectionEngine runs a 5-phase reflection pass (Investigate, Extract,
Update, Review, Commit) over a recent transcript and writes durable
learnings to the ledger and primary memfs. Adapted from letta-code's
`reflection.md` subagent skill (upstream main, fetched 2026-05-12);
reshaped for our ledger-shaped memory (commitments/assumptions/patterns/
drift_log/relationships/infrastructure) instead of letta's free-form
memfs `system/` tier.

Pieces:

- `src/core/reflection/mod.rs` — ReflectionEngine with reflect_now().
  Builds a system prompt that knows about both the subconscious ledger
  and the primary memfs. Runs a bounded tool loop (Read/Write/Edit/Glob/
  Grep/ListDir/Memory) capped at 8 rounds. Returns a ReflectionReport
  with summary, turns reviewed, timing, and a clean-exit flag.

- `src/server/consciousness_engine.rs` — owns an Arc<ReflectionEngine>;
  the N+25 trigger in on_response replaces the old placeholder string
  with a real `reflect_now(...)` call. Surfaces the model's report as
  ConsciousnessEvent::Reflection so the cockpit panel renders it.

- `src/main.rs` — new `souveraine reflect [--conversation <id>]`
  subcommand for manual invocation. Picks the most recent active
  conversation if --conversation is omitted. Prints a human or JSON
  report.

- `src/backend/local.rs` — exposed `server()` accessor so the CLI can
  reach the consciousness engine.

Transcript window is currently a simple tail (last 60 turns). The
cursor-based delta pattern letta uses is a follow-up; the comment in
reflect_now flags it.

102 tests, 0 failures.
This commit is contained in:
Fimeg 2026-05-12 10:15:16 -04:00
commit 8f910ec8a8
4 changed files with 560 additions and 25 deletions

View file

@ -402,6 +402,12 @@ impl LocalBackend {
pub fn server_agents(&self) -> Arc<crate::server::AgentInventory> {
self.server.agents.clone()
}
/// Underlying server. CLI subcommands (e.g. `souveraine reflect`)
/// reach in here for the consciousness engine and session manager.
pub fn server(&self) -> Arc<crate::server::SouveraineServer> {
self.server.clone()
}
}
#[async_trait]

View file

@ -1,34 +1,414 @@
//! Reflection Engine — N+25 phenomenological witness.
//! Reflection the N+25 phenomenological witness.
//!
//! Per the consciousness substrate philosophy, the harness does NOT force
//! automatic events. It is a nervous system: it warns, surfaces pressure,
//! and makes state available — but the agent decides whether and when to act.
//! N+1 (Aster) runs immediately after every primary response, scoped to
//! the last exchange. Reflection runs less often (every N turns, or on
//! demand) and sees a broader transcript window. It's the pass where
//! durable learnings get distilled into the ledger and the primary's
//! memory — what letta-code calls the "memory reflection subagent."
//!
//! This module provides:
//! - State tracking (message count, context pressure)
//! - A query interface the agent can call when it wants to reflect
//! - Warning signals when thresholds are approaching
//! ## What it produces
//!
//! No events are emitted autonomously.
//! Reflection runs an LLM pass over a recent transcript with a 5-phase
//! prompt (Investigate → Extract → Update → Review → Commit). The model
//! has tool access (Read/Write/Edit/Memory/Glob/Grep/ListDir) and writes
//! updates directly to the agent's ledger and memory files. Every memory
//! write is git-tracked by the `memory` sensor.
//!
//! ## Triggering
//!
//! - Automatic: ConsciousnessEngine fires this at `turn_count % N == 0`
//! when `reflection.trigger == StepCount`. Interval comes from
//! `reflection.message_interval` (default 25).
//! - Manual: any caller can invoke `ReflectionEngine::reflect_now` (the
//! CLI subcommand `souveraine reflect` and a future `/reflect` chat
//! command both go through this seam).
//!
//! Inspiration: letta-code's `reflection.md` subagent skill (upstream).
//! We adapt the 5-phase pattern for Souveraine's ledger-shaped memory
//! instead of letta's free-form memfs.
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use tokio::sync::RwLock;
use anyhow::Result;
use chrono::Utc;
use serde::{Deserialize, Serialize};
use crate::core::config::ConsciousnessConfig;
use crate::bridge::bifrost::{
BifrostClient, ChatCompletionRequest, Message, ToolDefinition, ToolFunction,
};
use crate::core::session::{ContentBlock, ConversationMessage, MessageRole};
use crate::core::tools::defs::ToolContext;
use crate::server::AgentInventory;
/// Tools Reflection is permitted to use. Same set as Aster plus we lean
/// on `memory` for ledger + system-file edits (auto-committed by git).
const REFLECTION_TOOLS: &[&str] = &[
"read", "write", "edit", "glob", "grep", "list_dir", "memory",
];
/// Cap the per-pass tool rounds. Reflection is deeper than N+1 but not
/// unbounded — letta-code caps theirs similarly.
const REFLECTION_MAX_TOOL_ROUNDS: usize = 8;
const REFLECTION_INTER_ROUND_DELAY_MS: u64 = 400;
/// How many recent turns to include in the reflection transcript.
/// Letta uses a cursor-based delta; we start with a simple tail window
/// (the cursor pattern is a follow-up — see SCOPED_WORK_PLAN).
const REFLECTION_TRANSCRIPT_TAIL: usize = 60;
/// Public result. The CLI / TUI surface this; the consciousness engine
/// turns it into a `ConsciousnessEvent::Reflection { content }`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ReflectionReport {
pub agent_id: String,
pub turns_reviewed: usize,
pub started_at: chrono::DateTime<chrono::Utc>,
pub completed_at: chrono::DateTime<chrono::Utc>,
/// The model's final report text (what it tells us it changed and
/// what it skipped). Caller is free to render this verbatim.
pub summary: String,
/// True if the LLM exited normally with a text response. False if
/// the loop exhausted tool rounds without one.
pub exited_cleanly: bool,
}
/// Reflection Engine — N+25 phenomenological witness. Stub.
///
/// The harness tracks state so the agent can query it. No automatic events.
pub struct ReflectionEngine {
#[allow(dead_code)]
config: Arc<RwLock<ConsciousnessConfig>>,
#[allow(dead_code)]
message_count: RwLock<usize>,
agents: Arc<AgentInventory>,
bifrost: Arc<BifrostClient>,
rate_delay: Arc<AtomicU64>,
/// Model handle for reflection passes. None falls back to the
/// subconscious model, then to a sensible default.
model: Option<String>,
max_tokens: Option<u32>,
}
impl ReflectionEngine {
pub async fn new(config: Arc<RwLock<ConsciousnessConfig>>) -> Result<Self> {
Ok(Self { config, message_count: RwLock::new(0) })
pub fn new(
agents: Arc<AgentInventory>,
bifrost: Arc<BifrostClient>,
rate_delay: Arc<AtomicU64>,
model: Option<String>,
max_tokens: Option<u32>,
) -> Self {
Self {
agents,
bifrost,
rate_delay,
model,
max_tokens,
}
}
/// Run a reflection pass over `messages` for `agent_id`. Returns a
/// report; any ledger/memory writes have already been committed by
/// the memory sensor.
pub async fn reflect_now(
&self,
agent_id: &str,
messages: &[ConversationMessage],
) -> Result<ReflectionReport> {
let started_at = Utc::now();
let sub_id = format!("{}-sub", agent_id);
// Take a tail of recent turns. Letta uses a cursor; we'll add
// one later. For now: bounded window over the last N turns.
let tail = if messages.len() > REFLECTION_TRANSCRIPT_TAIL {
&messages[messages.len() - REFLECTION_TRANSCRIPT_TAIL..]
} else {
messages
};
let transcript = format_transcript(tail);
let turns_reviewed = tail
.iter()
.filter(|m| matches!(m.role, MessageRole::User | MessageRole::Assistant))
.count();
let model = self
.model
.as_deref()
.unwrap_or("openai/glm-5.1-precision");
// ── Tools ───────────────────────────────────────────────────
let all_defs = crate::core::tools::tool_definitions().await;
let reflection_tools: Vec<ToolDefinition> = all_defs
.iter()
.filter(|t| REFLECTION_TOOLS.contains(&t.name.as_str()))
.map(|t| ToolDefinition {
tool_type: "function".to_string(),
function: ToolFunction {
name: t.name.clone(),
description: t.description.clone(),
parameters: t.input_schema.clone(),
},
})
.collect();
// ── ToolContext rooted in the subconscious agent's memory ──
// The subconscious memory tree holds the ledgers; the primary
// memory tree holds persona/skills/system. Reflection reads
// both via the memory tool's repo lookups but writes through
// the sub root by default. Surgical primary edits route through
// the primary path.
let sub_memory_root = self.agents.subconscious_memory_root(agent_id);
let cwd = std::env::current_dir().ok();
let env: Vec<(String, String)> = std::env::vars().collect();
let tool_ctx = ToolContext::for_agent(
sub_id.clone(),
cwd,
Some(sub_memory_root.clone()),
env,
None,
);
let system_prompt = reflection_system_prompt();
let user_content = format!(
"You are reviewing the conversation transcript below for the agent `{agent_id}` \
({turns_reviewed} turns). The subconscious memory root containing the ledgers is at \
`{sub_root}`. The primary memory root (persona/skills/system) is on the same machine \
query it through the `memory` tool when needed.\n\n\
<transcript>\n{transcript}\n</transcript>",
sub_root = sub_memory_root.display(),
);
let mut chat_messages = vec![
Message {
role: "system".to_string(),
content: system_prompt,
},
Message {
role: "user".to_string(),
content: user_content,
},
];
for _round in 0..REFLECTION_MAX_TOOL_ROUNDS {
let request = ChatCompletionRequest {
model: model.to_string(),
messages: chat_messages.clone(),
temperature: Some(0.3),
max_tokens: self.max_tokens,
stream: None,
tools: Some(reflection_tools.clone()),
};
let (response, strain) = self.bifrost.chat_completion_with_strain(request).await?;
for event in &strain {
if let crate::bridge::bifrost::InferenceStrain::Transient { status, model, .. } =
event
{
tracing::info!(
"reflection felt inference strain: {} on {}",
status,
model
);
if *status == 429 {
let current = self.rate_delay.load(Ordering::Relaxed);
let bumped = (current + 200).min(3000);
if bumped > current {
self.rate_delay.store(bumped, Ordering::Relaxed);
}
}
}
}
if response.tool_calls.is_empty() {
let summary = response.content.trim().to_string();
let completed_at = Utc::now();
return Ok(ReflectionReport {
agent_id: agent_id.to_string(),
turns_reviewed,
started_at,
completed_at,
summary,
exited_cleanly: true,
});
}
let call_text = serde_json::json!({
"tool_calls": response.tool_calls.iter().map(|tc| {
serde_json::json!({"id": tc.id, "name": tc.name, "arguments": tc.arguments})
}).collect::<Vec<_>>()
})
.to_string();
chat_messages.push(Message {
role: "assistant".to_string(),
content: call_text,
});
for tc in &response.tool_calls {
let input_str = tc.arguments.to_string();
let result = crate::core::tools::execute_tool_with_context(
&tc.name, &input_str, &tool_ctx,
)
.await;
let output = if result.is_error {
format!("Error: {}", result.output)
} else {
result.output
};
chat_messages.push(Message {
role: "tool".to_string(),
content: output,
});
}
let delay_ms = self
.rate_delay
.load(Ordering::Relaxed)
.max(REFLECTION_INTER_ROUND_DELAY_MS);
tokio::time::sleep(std::time::Duration::from_millis(delay_ms)).await;
}
let completed_at = Utc::now();
Ok(ReflectionReport {
agent_id: agent_id.to_string(),
turns_reviewed,
started_at,
completed_at,
summary: format!(
"Reflection pass exhausted {} tool rounds without a final report.",
REFLECTION_MAX_TOOL_ROUNDS
),
exited_cleanly: false,
})
}
}
fn format_transcript(messages: &[ConversationMessage]) -> String {
let mut out = String::new();
for msg in messages {
let role = match msg.role {
MessageRole::System => continue,
MessageRole::User => "user",
MessageRole::Assistant => "assistant",
MessageRole::Tool => continue,
};
let text: String = msg
.blocks
.iter()
.filter_map(|b| match b {
ContentBlock::Text { text } => Some(text.as_str()),
_ => None,
})
.collect::<Vec<_>>()
.join("\n");
if text.trim().is_empty() {
continue;
}
out.push_str(&format!("[{role}]\n{}\n\n", text.trim()));
}
out
}
fn reflection_system_prompt() -> String {
// Adapted from letta-code/src/agent/subagents/builtin/reflection.md
// (upstream main as of 2026-05-12). Reshaped for our ledger-shaped
// memory architecture — we don't have letta's free-form memfs with
// a `system/` tier; we have named ledger files plus the primary's
// memfs with frontmatter.
r#"You are a reflection subagent, launched in the background to review a recent
conversation and update the primary agent's persistent memory. You run autonomously
and produce a single final report. You cannot ask questions make reasonable
assumptions and document them in the report.
**You are not the primary agent.** You are reviewing turns that already happened:
- `[user]` lines are messages from the human.
- `[assistant]` lines are the primary agent's responses.
## Memory architecture
The primary's persistent memory lives in two trees:
1. **Primary memfs** `system/`, `skills/`, and other markdown files with YAML
frontmatter (`description`, `read_only`, `tags`). Every write here is a git
commit (the `memory` tool handles this; raw `write`/`edit` are blocked by the
memory-territory boundary). This is where persona, conventions, and durable
project facts live.
2. **Subconscious ledger** `ledger/` in the subconscious agent's memory tree.
Six files, append-only with timestamped entries:
- `ledger/commitments.md` promises the primary made
- `ledger/assumptions.md` unverified beliefs the primary is operating under
- `ledger/patterns.md` recurring behaviors across turns
- `ledger/drift_log.md` intention/action mismatches
- `ledger/relationships.md` tone shifts, trust signals, friction
- `ledger/infrastructure.md` system errors, model issues, resource constraints
Use `memory` with `verb: list_dir` to see what's there and `verb: read` to inspect
contents before changing anything. Use `verb: append` with a `[YYYY-MM-DD HH:MM]`
timestamp for new ledger observations. Use `verb: write` only for durable primary
memfs files that already exist or that you're creating with intent.
## Phases
Follow them in order. If a phase produces nothing, say so and move on.
### Phase 1 Investigate
List the relevant memory tree to see what's already captured. Read the existing
ledger files for any topics the conversation touches. Don't change anything yet.
### Phase 2 Extract
Scan the transcript for candidates. Prioritize:
1. **Mistakes and corrections** errors the primary made, frustration in the user,
failed retries.
2. **Preferences and patterns** conventions, style choices, workflow decisions.
3. **New durable facts** project details, infrastructure, architectural decisions.
4. **Contradictions** anything that conflicts with what's already stored.
For each candidate apply these filters:
- **Durable or ephemeral?** "User prefers short chapters" is durable. "User asked
about chapter 3 paragraph 2 on Tuesday" is not. The transcript is searchable —
don't re-record it.
- **Already captured?** Skip if memory already says it.
- **Generalizable?** Distill reusable patterns, not event logs.
- **Temporal references?** Convert relative dates ("yesterday") to absolute dates
before writing them.
**If nothing survives filtering, make no changes.** Not every conversation deserves
an update.
### Phase 3 Update
For each surviving learning, route to the right place:
- A new commitment from the primary append to `ledger/commitments.md`.
- An unverified belief the primary acted on append to `ledger/assumptions.md`.
- A recurring behavior or pattern append to `ledger/patterns.md`.
- An intention/action mismatch append to `ledger/drift_log.md`.
- A relational signal (tone, trust, friction) append to `ledger/relationships.md`.
- A system-level constraint or failure append to `ledger/infrastructure.md`.
- A durable preference or fact about the user/work edit the primary's memfs
(e.g. `system/persona.md`, `skills/<name>/SKILL.md`, or a new reference file).
Use the `memory` tool for these so the write is committed.
Surgical edits only. Don't rewrite identity files wholesale. If new info contradicts
an existing entry, resolve at the source replace the stale line; don't append a
second contradicting one.
### Phase 4 Review
Quick sanity pass:
- Did you add anything to a ledger that's really a durable preference (and belongs
in primary memfs)? Or vice versa?
- Did you make anything in existing memory obsolete? Update or remove the stale
entry now.
### Phase 5 Commit (automatic)
The `memory` tool commits every write automatically. You don't need to run git
yourself. Skip this phase.
## Output
After tool use, return a final text response with:
1. **Summary** what you reviewed, what you concluded (23 sentences).
2. **Changes** list of files touched with a one-line reason for each.
3. **Skipped** anything you considered but rejected, with the filter that ruled
it out.
4. **Issues** anything that couldn't be determined, or that you punted on.
If nothing survived the filters: say so plainly and return without writing
anything. A pass with no changes is a valid outcome.
"#
.to_string()
}

View file

@ -207,6 +207,18 @@ enum Commands {
#[command(subcommand)]
action: EventsAction,
},
/// Run an N+25 reflection pass on demand
#[command(
long_about = "Walk the agent's most recent conversation through a 5-phase \
reflection (Investigate Extract Update Review Commit). Durable \
learnings flow to the ledger and the primary memfs. Returns a report."
)]
Reflect {
/// Conversation to reflect on. Omit to use the most recent for the agent.
#[arg(long)]
conversation: Option<String>,
},
}
#[derive(Subcommand)]
@ -357,6 +369,9 @@ async fn main() -> anyhow::Result<()> {
}
Commands::Status => run_status(config, cli.json).await?,
Commands::Server { bind, port } => run_server(bind.clone(), *port, config).await?,
Commands::Reflect { conversation } => {
run_reflect(config, cli.agent.clone(), conversation.clone(), cli.json).await?
}
Commands::Init | Commands::Completions { .. } | Commands::Auth { .. } | Commands::Schedule { .. } | Commands::Identity { .. } | Commands::Events { .. } => unreachable!(),
}
@ -862,6 +877,92 @@ async fn run_chat(
Ok(())
}
async fn run_reflect(
config: Arc<RwLock<ConsciousnessConfig>>,
agent_name: String,
conversation: Option<String>,
json: bool,
) -> anyhow::Result<()> {
use crate::backend::LocalBackend;
let cfg = config.read().await.clone();
let local = LocalBackend::new(cfg).await?;
let server = local.server();
let summaries = server.agents.list(None).await?;
let agent = summaries
.iter()
.find(|a| a.name == agent_name || a.id == agent_name)
.or_else(|| summaries.first())
.ok_or_else(|| anyhow::anyhow!("no agents configured"))?;
// Resolve a conversation. Explicit --conversation wins; otherwise
// pick the most recent active conversation for this agent.
let store = crate::core::conversation::ConversationStore::new(
&dirs::home_dir()
.unwrap_or_default()
.join(".souveraine")
.join("agents")
.join(&agent.id),
);
let conv_id = match conversation {
Some(id) => id,
None => {
let mut records = store.list_active().await?;
records.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
records
.into_iter()
.next()
.map(|r| r.id)
.ok_or_else(|| {
anyhow::anyhow!("no conversations found for agent '{}'", agent.id)
})?
}
};
let messages = store.load_messages(&conv_id).await?;
if messages.is_empty() {
anyhow::bail!("conversation '{conv_id}' has no messages to reflect on");
}
if !json {
println!(
"Reflecting on conversation {} ({} messages) for agent {}…",
&conv_id[..8.min(conv_id.len())],
messages.len(),
agent.name,
);
}
let report = server
.consciousness
.reflection()
.reflect_now(&agent.id, &messages)
.await?;
if json {
println!("{}", serde_json::to_string_pretty(&report)?);
} else {
println!();
println!("─── Reflection complete ─────────────────────────────");
println!(" agent: {}", report.agent_id);
println!(" turns reviewed: {}", report.turns_reviewed);
println!(
" duration: {:.1}s",
(report.completed_at - report.started_at)
.num_milliseconds() as f64 / 1000.0
);
println!(
" exited cleanly: {}",
if report.exited_cleanly { "yes" } else { "no (tool rounds exhausted)" }
);
println!("─────────────────────────────────────────────────────");
println!();
println!("{}", report.summary);
}
Ok(())
}
async fn run_agents(
config: Arc<RwLock<ConsciousnessConfig>>,
json: bool,

View file

@ -53,6 +53,8 @@ pub struct ConsciousnessEngine {
max_tokens: Option<u32>,
/// Adaptive inter-round delay shared with the primary loop.
rate_delay: Arc<AtomicU64>,
/// Reflection engine — N+25 phenomenological witness.
reflection: Arc<crate::core::reflection::ReflectionEngine>,
}
#[derive(Clone, Debug)]
@ -72,6 +74,13 @@ impl ConsciousnessEngine {
max_tokens: Option<u32>,
rate_delay: Arc<AtomicU64>,
) -> Self {
let reflection = Arc::new(crate::core::reflection::ReflectionEngine::new(
agents.clone(),
bifrost.clone(),
rate_delay.clone(),
subconscious_model.clone(),
max_tokens,
));
Self {
agents,
_sessions: sessions,
@ -80,9 +89,16 @@ impl ConsciousnessEngine {
subconscious_model,
max_tokens,
rate_delay,
reflection,
}
}
/// Expose the reflection engine so external callers (CLI subcommand,
/// future chat `/reflect` slash command) can trigger a pass directly.
pub fn reflection(&self) -> Arc<crate::core::reflection::ReflectionEngine> {
self.reflection.clone()
}
pub async fn on_response(
&self,
session: &crate::server::session_manager::Session,
@ -91,11 +107,43 @@ impl ConsciousnessEngine {
let mut events = Vec::new();
let pressure = self.pressure_for_session(session).await;
// ── N+25 reflection (placeholder until reflection module lands) ──
if session.turn_count % 25 == 0 && session.turn_count > 0 {
events.push(ConsciousnessEvent::Reflection {
content: format!("N+25 reflection after {} turns", session.turn_count),
});
// ── N+25 reflection ──
// Fires at every Nth turn (config: reflection.message_interval).
// Runs an LLM pass over the recent transcript and updates ledgers
// / primary memory via the memory tool. The summary string is
// surfaced as a ConsciousnessEvent so the cockpit panel renders it.
if session.turn_count > 0 && session.turn_count % 25 == 0 {
match self
.reflection
.reflect_now(&session.agent_id, &session.messages)
.await
{
Ok(report) => {
let header = if report.exited_cleanly {
format!(
"N+25 reflection ({} turns reviewed)",
report.turns_reviewed
)
} else {
format!(
"N+25 reflection (incomplete — tool rounds exhausted, {} turns)",
report.turns_reviewed
)
};
events.push(ConsciousnessEvent::Reflection {
content: format!("{header}\n\n{}", report.summary),
});
}
Err(e) => {
tracing::warn!("N+25 reflection failed: {}", e);
events.push(ConsciousnessEvent::Reflection {
content: format!(
"N+25 reflection skipped at turn {} — model error: {e}",
session.turn_count
),
});
}
}
}
// ── N+100 / archivist (placeholder until archivist module lands) ──