SouveraineOS/docs/substrate/research/model-selection-audit.md

title	date	status
Model Selection Audit Report	2026-05-07	complete

Model Selection Audit Report

Executive Summary

Audited jcode, claw-open, Letta Code, and Bifrost for model selection, routing, and /model command patterns. Key findings:

1. jcode has a production-grade /model command with provider abstraction, JSON output, and subscription filtering
2. claw-open uses Rust config structs for model configuration
3. Letta Code uses provider-based model routing with context limit tracking
4. Bifrost (primary inference system) exposes /v1/models endpoint that lists all available models dynamically

Critical Finding: Bifrost is an OpenAI-compatible API gateway at http://10.10.20.120:3360/v1 that neutralizes provider complexity. The Souveraine BifrostClient already has a list_models() method that fetches models from /v1/models.

Recommendation: Leverage Bifrost's /v1/models endpoint for dynamic model discovery, with Souveraine-specific enhancements for model physics (context limits, thresholds, task preferences).

---

jcode Implementation

File: src/cli/commands.rs - run_model_command()

Purpose: Lists available models for a provider with optional JSON output

Key Code:

pub async fn run_model_command(
    choice: &super::provider_init::ProviderChoice,
    model: Option<&str>,
    emit_json: bool,
    verbose: bool,
) -> Result<()> {
    let provider = super::provider_init::init_provider_quiet(choice, model).await?;

    if let Err(err) = provider.prefetch_models().await
        && !super::output::quiet_enabled()
    {
        eprintln!("Warning: failed to refresh dynamic model list: {}", err);
    }

    let routes = provider.model_routes();
    let filtered_routes = filter_cli_model_routes_for_choice(choice, &routes);
    let models = if filtered_routes.len() == routes.len() {
        collect_cli_model_names(&routes, provider.available_models_display())
    } else {
        collect_cli_model_names(&filtered_routes, Vec::new())
    };

    if models.is_empty() {
        anyhow::bail!(
            "No models found for provider '{}'. Check credentials or try a different --provider.",
            provider.name()
        );
    }

    if emit_json {
        let report = ModelListReport {
            provider: provider_label,
            selected_model: provider.model(),
            models,
            routes: filtered_routes
                .iter()
                .map(|route| ModelListRouteReport {
                    provider: cli_route_provider_display(&route.provider, &route.api_method),
                    model: route.model.clone(),
                    method: cli_api_method_display(&route.api_method).to_string(),
                    available: route.available,
                })
                .collect(),
        };
        println!("{}", serde_json::to_string_pretty(&report)?);
    } else {
        if verbose {
            println!(
                "Provider: {}",
                crate::provider_catalog::runtime_provider_display_name(provider.name())
            );
            println!("Selected model: {}", provider.model());
            println!("Available models: {}", models.len());
            println!();
        }
        for model in models {
            println!("{}", model);
        }
    }

    Ok(())
}

Patterns Identified:

• Provider choice abstraction: ProviderChoice enum handles different providers
• Model route filtering: filter_cli_model_routes_for_choice() filters by provider/subscription
• JSON output mode: Machine-readable output with ModelListReport struct
• Verbose vs quiet: Different output levels for different use cases
• Prefetching: provider.prefetch_models() refreshes dynamic model list
• Error handling: Clear error messages when no models found

File: src/provider/models.rs

Purpose: Model catalog management with caching and subscription filtering

Key Code:

/// Dynamic cache of model context window sizes, populated from API at startup.
static CONTEXT_LIMIT_CACHE: std::sync::LazyLock<RwLock<HashMap<String, usize>>> =
    std::sync::LazyLock::new(|| RwLock::new(HashMap::new()));

pub(crate) fn filtered_model_routes(routes: Vec<ModelRoute>) -> Vec<ModelRoute> {
    if !crate::subscription_catalog::is_runtime_mode_enabled() {
        return routes;
    }

    routes
        .into_iter()
        .filter(|route| crate::subscription_catalog::is_curated_model(&route.model))
        .collect()
}

pub(crate) fn ensure_model_allowed_for_subscription(model: &str) -> Result<()> {
    if crate::subscription_catalog::is_runtime_mode_enabled()
        && !crate::subscription_catalog::is_curated_model(model)
    {
        anyhow::bail!(
            "Model '{}' is not included in the current jcode subscription catalog",
            model
        );
    }
    Ok(())
}

Patterns Identified:

• Context limit caching: Static RwLock<HashMap> for model context windows
• Subscription filtering: Models filtered by subscription tier
• Validation: ensure_model_allowed_for_subscription() checks model availability
• Dynamic catalog: Models fetched from API at startup

File: src/provider/models_catalog.rs

Purpose: Fetch and cache model catalogs from providers

Key Patterns:

• Catalog caching: JSON files cache model lists (openai_model_catalog_cache.json)
• OAuth support: fetch_anthropic_model_catalog_oauth() for authenticated access
• Context limits: fetch_openai_context_limits() retrieves model capabilities

---

claw-open Implementation

File: rust/crates/runtime/src/config.rs

Purpose: Model configuration in Rust structs

Key Patterns:

• Config structs: Strongly-typed model configuration
• Per-model settings: Context limits, thresholds, preferences
• TOML serialization: Config loaded from TOML files

Example Pattern:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ModelConfig {
    pub provider: String,
    pub model: String,
    #[serde(default = "default_128k")]
    pub context_limit: usize,
    #[serde(default = "default_8k")]
    pub output_limit: usize,
    #[serde(default = "default_threshold_70")]
    pub archivist_threshold: f32,
    #[serde(default = "default_100")]
    pub archivist_interval: usize,
    #[serde(default)]
    pub preferred_for: Vec<TaskType>,
}

---

Letta Code Implementation

Provider-Based Model Routing

Key Patterns:

• Provider abstraction: Different providers (Bifrost, Ollama, vLLM)
• Model routing: Task-based model selection
• Context tracking: Token usage monitoring

---

Bifrost Implementation

File: src/bridge/bifrost.rs - list_models()

Purpose: Fetch available models from Bifrost's /v1/models endpoint

Key Code:

/// List available models from Bifrost
pub async fn list_models(&self) -> Result<Vec<String>> {
    let url = format!("{}/models", self.base_url);
    let resp = self.client
        .get(&url)
        .headers(self.auth_headers())
        .send()
        .await
        .with_context(|| "Failed to fetch Bifrost models")?;

    let body: serde_json::Value = resp.json().await?;
    let models = body["data"]
        .as_array()
        .map(|arr| {
            arr.iter()
                .filter_map(|m| m["id"].as_str().map(String::from))
                .collect::<Vec<_>>()
        })
        .unwrap_or_default();

    Ok(models)
}

Patterns Identified:

• OpenAI-compatible endpoint: /v1/models returns { "data": [{ "id": "model-name", ... }] }
• Bearer token auth: Uses Authorization: Bearer <api_key> header
• Virtual key support: Optional x-bf-vk header for provider-specific routing
• Dynamic discovery: Models fetched at runtime, not hardcoded

Bifrost Configuration:

// From Souveraine config.rs
pub struct BifrostConfig {
    pub base_url: String,      // "http://10.10.20.120:3360"
    pub api_key: String,       // Bearer token
    pub virtual_key: String,   // Optional provider key
    pub primary_model: String, // Default model
    pub models: HashMap<String, BifrostModelConfig>, // Per-model overrides
}

Example Bifrost Response:

{
  "data": [
    { "id": "fireworks/accounts/fireworks/routers/kimi-k2p5-turbo", ... },
    { "id": "fireworks/accounts/fireworks/routers/kimi-k2.6", ... },
    { "id": "anthropic/claude-sonnet-4-5-20250501", ... },
    { "id": "openai/deepseek-v4-pro", ... }
  ]
}

Why Bifrost Neutralizes Complexity

1. Unified API: OpenAI-compatible endpoint for all providers
2. Dynamic Discovery: /v1/models lists available models at runtime
3. Provider Abstraction: Bifrost handles routing to actual providers
4. Virtual Keys: Per-provider routing via x-bf-vk header
5. No Hardcoding: Models don't need to be hardcoded in clients

---

Recommendations for Souveraine

1. /model Command Structure - Enhanced for Bifrost

Implementation:

// src/cli/commands.rs
pub async fn run_model_command(
    model_name: Option<&str>,
    emit_json: bool,
    verbose: bool,
) -> Result<()> {
    let config = ConsciousnessConfig::load(&config_path).await?;
    
    // Create Bifrost client to fetch dynamic model list
    let bifrost = BifrostClient::new(
        &config.bifrost.base_url,
        &config.bifrost.api_key,
        &config.bifrost.virtual_key,
        &config.bifrost.primary_model,
    );
    
    // Fetch models from Bifrost
    let bifrost_models = bifrost.list_models().await.unwrap_or_default();
    
    // Merge with configured models (configured models override/add to Bifrost list)
    let mut all_models = bifrost_models.clone();
    for name in config.models.keys() {
        if !all_models.contains(name) {
            all_models.push(name.clone());
        }
    }
    
    if let Some(name) = model_name {
        // Set specific model
        if !all_models.contains(&name.to_string()) {
            anyhow::bail!("Model '{}' not found. Available: {:?}", name, all_models);
        }
        
        // Update config and save
        let mut new_config = config.clone();
        new_config.bifrost.primary_model = name.to_string();
        new_config.save(&config_path)?;
        
        println!("Selected model: {}", name);
        return Ok(());
    }
    
    // List all models
    if emit_json {
        let report = ModelListReport {
            selected: config.bifrost.primary_model.clone(),
            bifrost_discovered: bifrost_models,
            configured: config.models.keys().cloned().collect(),
            all_models: all_models.iter().map(|name| {
                let cfg = config.models.get(name);
                ModelReport {
                    name: name.clone(),
                    provider: cfg.map(|c| c.provider.clone()).unwrap_or_else(|| "bifrost".to_string()),
                    context_limit: cfg.map(|c| c.context_limit).unwrap_or(128000),
                    output_limit: cfg.map(|c| c.output_limit).unwrap_or(8192),
                    preferred_for: cfg.map(|c| c.preferred_for.clone()).unwrap_or_default(),
                    from_bifrost: bifrost_models.contains(name),
                }
            }).collect(),
        };
        println!("{}", serde_json::to_string_pretty(&report)?);
    } else {
        if verbose {
            println!("Selected: {}", config.bifrost.primary_model);
            println!("Bifrost discovered: {}", bifrost_models.len());
            println!("Configured models: {}", config.models.len());
            println!("Total available: {}", all_models.len());
            println!();
        }
        for model in all_models {
            let marker = if bifrost_models.contains(&model) { "⚡" } else { "⚙️" };
            println!("{} {}", marker, model);
        }
    }
    
    Ok(())
}

CLI Arguments:

// src/cli/args.rs
#[derive(Parser)]
pub struct ModelArgs {
    /// Model name to set (or list all if omitted)
    #[arg(short, long)]
    model: Option<String>,
    
    /// Output as JSON
    #[arg(long)]
    json: bool,
    
    /// Verbose output
    #[arg(short, long)]
    verbose: bool,
}

2. Model Configuration Format

Recommended TOML:

[bifrost]
base_url = "http://10.10.20.120:3360"
primary_model = "kimi-k2.5-turbo"
api_key = "..."
virtual_key = "..."

[models."kimi-k2.5-turbo"]
provider = "bifrost"
model = "fireworks/accounts/fireworks/routers/kimi-k2p5-turbo"
context_limit = 128000
output_limit = 8192
archivist_threshold = 0.7
archivist_interval = 100
preferred_for = ["conversation", "reflection"]

[models."deepseek-v4-pro"]
provider = "bifrost"
model = "openai/deepseek-v4-pro"
context_limit = 32768
output_limit = 4096
archivist_threshold = 0.6
archivist_interval = 50
preferred_for = ["synthesis"]

3. Model Router Implementation

Enhanced ModelRouter:

pub struct ModelRouter {
    configs: HashMap<String, ModelConfig>,
    current_usage: Arc<RwLock<TokenUsage>>,
    token_counter: TokenCounter,
    selected_model: String,  // Current selection
}

impl ModelRouter {
    /// Set the primary model
    pub fn set_model(&mut self, model_name: &str) -> Result<()> {
        if !self.configs.contains_key(model_name) {
            anyhow::bail!("Model '{}' not found", model_name);
        }
        self.selected_model = model_name.to_string();
        Ok(())
    }
    
    /// Get current model
    pub fn current_model(&self) -> &str {
        &self.selected_model
    }
    
    /// List all available models
    pub fn list_models(&self) -> Vec<String> {
        self.configs.keys().cloned().collect()
    }
    
    /// Get model info for display
    pub fn model_info(&self, name: &str) -> Option<ModelInfo> {
        self.configs.get(name).map(|cfg| ModelInfo {
            name: name.to_string(),
            provider: cfg.provider.clone(),
            context_limit: cfg.context_limit,
            output_limit: cfg.output_limit,
            preferred_for: cfg.preferred_for.clone(),
        })
    }
}

4. Context Limit Enforcement

Strategy:

• Track token usage per model
• Warn when approaching limit (80%)
• Trigger archivist when at threshold (70%)
• Hard fail only when absolutely necessary

Implementation:

pub enum ContextEnforcement {
    Warn,      // Log warning but continue
    Fail,      // Hard fail when limit exceeded
    AutoCompact, // Trigger archivist automatically
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ModelConfig {
    // ... existing fields ...
    #[serde(default = "default_enforcement")]
    pub enforcement: ContextEnforcement,
}

5. Provider Abstraction

Recommended Structure:

pub enum Provider {
    Bifrost { base_url: String, api_key: String },
    Ollama { base_url: String },
    vLLM { base_url: String },
    Remote { url: String },
}

impl Provider {
    pub async fn list_models(&self) -> Result<Vec<String>> {
        match self {
            Provider::Bifrost { .. } => self.fetch_bifrost_models().await,
            Provider::Ollama { .. } => self.fetch_ollama_models().await,
            // ...
        }
    }
    
    pub async fn complete(
        &self,
        model: &str,
        messages: &[Message],
    ) -> Result<CompletionResponse> {
        // Provider-specific implementation
    }
}

---

Implementation Priority

Phase 1: Model Router Enhancement (1-2 hours)

1. Add set_model() and current_model() methods
2. Add list_models() and model_info() methods
3. Wire into config loading/saving

Phase 2: /model Command (1 hour)

1. Create CLI command in src/cli/commands.rs
2. Add args in src/cli/args.rs
3. Support JSON and verbose output
4. Support setting model by name

Phase 3: Provider Abstraction (2-3 hours)

1. Create Provider enum
2. Implement provider-specific model listing
3. Wire into ModelRouter

Phase 4: Context Enforcement (1-2 hours)

1. Add enforcement modes to ModelConfig
2. Implement warning/fail/auto-compact logic
3. Wire into Bifrost client

---

Risks and Considerations

1. Breaking Changes: Changing model selection may affect existing conversations
2. Config Migration: Need to handle old config formats
3. Provider Compatibility: Different providers have different model naming conventions
4. Performance: Model listing should be cached, not fetched every time

---

Files to Create/Modify

New Files:

• src/cli/commands.rs - Add run_model_command()
• src/cli/args.rs - Add ModelArgs
• src/provider/mod.rs - Provider abstraction (if not exists)

Modified Files:

• src/core/config.rs - Add enforcement field to ModelConfig
• src/bridge/model_router.rs - Add set_model(), list_models() methods
• src/main.rs - Wire up /model command

---

Testing Strategy

1. Unit Tests: Test ModelRouter methods
2. Integration Tests: Test /model command with different providers
3. Config Tests: Test model switching and persistence
4. Error Tests: Test invalid model names, missing configs

---

References

• jcode: /home/casey/Projects/jcode/src/cli/commands.rs (run_model_command)
• jcode: /home/casey/Projects/jcode/src/provider/models.rs (model catalog)
• claw-open: /home/casey/Projects/claw-open/rust/crates/runtime/src/config.rs
• Souveraine: src/core/config.rs (existing model config)
• Souveraine: src/bridge/model_router.rs (existing router)