Skip to content

OpenAI-Compatible LLM Endpoint (OpenRouter) Implementation Plan — xylolabs-api

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Make the three LLM call sites POST to a configurable OpenAI-compatible endpoint (LLM_ENDPOINT, default: Gemini's compatibility URL) with LLM_API_KEY/LLM_MODEL overrides falling back to GEMINI_API_KEY/GEMINI_MODEL.

Architecture: Resolution happens once in AppConfig::from_env(); the resolved key/model land in the existing gemini_api_key/gemini_model fields so call sites and fixtures barely change. One new struct field (llm_endpoint). No behavior change when the new env vars are unset.

Tech Stack: Rust 2024 / Axum 0.8; no new dependencies.

Spec: docs/superpowers/specs/2026-07-22-openai-compatible-llm-endpoint-design.md

Global Constraints

  • Default endpoint (exact): https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
  • Fallback order: LLM_API_KEY → GEMINI_API_KEY; LLM_MODEL → GEMINI_MODEL → "gemini-3.6-flash"
  • llm_endpoint must be non-empty and start with http:// or https:// (checked in validate())
  • gemini_api_key stays [REDACTED] in Debug; llm_endpoint is printed as-is
  • All commits GPG-signed (-S), Conventional Commits + gitmoji, hash mined via ~/flash-shared/gitminer-cuda/mine_commit.sh 7 before push
  • After the feature commit: git pull --rebase, push, deploy via bash scripts/deploy.sh

Task 1: Config resolution + validation + tests

Files: - Modify: crates/xylolabs-server/src/config.rs (fields ~line 107-108, from_env ~line 390-392, Debug impl ~line 227-228, validate() ~line 811, sample_config() ~line 1126, tests module ~line 1219+) - Modify: crates/xylolabs-server/src/router.rs:1157 region (test fixture) - Modify: crates/xylolabs-server/tests/common/mod.rs:373 and :722 regions (fixtures) - Modify: crates/xylolabs-server/tests/api_ingest.rs:739 region (fixture)

Interfaces: - Produces: pub const DEFAULT_LLM_ENDPOINT: &str in config.rs (module level, next to the struct); pub llm_endpoint: String field on AppConfig. Tasks 2-3 rely on config.llm_endpoint and DEFAULT_LLM_ENDPOINT.

  • [ ] Step 1: Write the failing tests (in config.rs mod tests, following the existing with_env + minimal_required_env() pattern; add the five LLM/Gemini keys as explicit None/Some entries per test so nothing leaks from the environment, exactly like the retention tests do)
#[test]
fn llm_env_overrides_win_over_gemini_env() {
    let mut env: Vec<(&'static str, Option<&'static str>)> = minimal_required_env()
        .into_iter()
        .map(|(k, v)| (k, Some(v)))
        .collect();
    env.push(("GEMINI_API_KEY", Some("gemini-key")));
    env.push(("GEMINI_MODEL", Some("gemini-3.6-flash")));
    env.push(("LLM_API_KEY", Some("openrouter-key")));
    env.push(("LLM_MODEL", Some("google/gemini-3.6-flash")));
    env.push((
        "LLM_ENDPOINT",
        Some("https://openrouter.ai/api/v1/chat/completions"),
    ));
    let config = with_env(env, AppConfig::from_env).unwrap();
    assert_eq!(config.gemini_api_key.as_deref(), Some("openrouter-key"));
    assert_eq!(config.gemini_model, "google/gemini-3.6-flash");
    assert_eq!(
        config.llm_endpoint,
        "https://openrouter.ai/api/v1/chat/completions"
    );
}

#[test]
fn llm_env_unset_falls_back_to_gemini_env_and_default_endpoint() {
    let mut env: Vec<(&'static str, Option<&'static str>)> = minimal_required_env()
        .into_iter()
        .map(|(k, v)| (k, Some(v)))
        .collect();
    env.push(("GEMINI_API_KEY", Some("gemini-key")));
    env.push(("GEMINI_MODEL", Some("gemini-3.6-flash")));
    env.push(("LLM_API_KEY", None));
    env.push(("LLM_MODEL", None));
    env.push(("LLM_ENDPOINT", None));
    let config = with_env(env, AppConfig::from_env).unwrap();
    assert_eq!(config.gemini_api_key.as_deref(), Some("gemini-key"));
    assert_eq!(config.gemini_model, "gemini-3.6-flash");
    assert_eq!(config.llm_endpoint, DEFAULT_LLM_ENDPOINT);
}

#[test]
fn validate_rejects_non_http_llm_endpoint() {
    let mut config = sample_config();
    config.llm_endpoint = "ftp://openrouter.ai/api".to_string();
    let error = config.validate().unwrap_err().to_string();
    assert!(error.contains("LLM_ENDPOINT"), "unexpected error: {error}");
}
  • [ ] Step 2: Run tests to verify they fail

Run: cargo test -p xylolabs-server --lib config::tests::llm -- --nocapture Expected: compile error — llm_endpoint field and DEFAULT_LLM_ENDPOINT do not exist yet.

  • [ ] Step 3: Implement config changes

Module-level const (above pub struct AppConfig):

/// Default OpenAI-compatible chat/completions endpoint: Gemini's
/// compatibility layer. Point `LLM_ENDPOINT` at any other
/// OpenAI-compatible provider (e.g. OpenRouter:
/// `https://openrouter.ai/api/v1/chat/completions`) to switch.
pub const DEFAULT_LLM_ENDPOINT: &str =
    "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions";

Struct fields (replace the two existing lines; keep field names):

    /// Resolved LLM bearer key: `LLM_API_KEY` if set, else `GEMINI_API_KEY`.
    /// Field name kept for churn-avoidance — the value is whatever key the
    /// configured `llm_endpoint` expects, not necessarily a Gemini key.
    pub gemini_api_key: Option<String>,
    /// Resolved LLM model id: `LLM_MODEL` if set, else `GEMINI_MODEL`.
    /// Same field-name caveat as `gemini_api_key`.
    pub gemini_model: String,
    /// Full URL of the OpenAI-compatible `chat/completions` endpoint all
    /// three LLM call sites POST to. Defaults to Gemini's compat layer.
    pub llm_endpoint: String,

from_env (replace the two existing reads):

        let gemini_api_key = std::env::var("LLM_API_KEY")
            .ok()
            .or_else(|| std::env::var("GEMINI_API_KEY").ok());
        let gemini_model = std::env::var("LLM_MODEL")
            .or_else(|_| std::env::var("GEMINI_MODEL"))
            .unwrap_or_else(|_| "gemini-3.6-flash".to_string());
        let llm_endpoint = std::env::var("LLM_ENDPOINT")
            .unwrap_or_else(|_| DEFAULT_LLM_ENDPOINT.to_string());

…and add llm_endpoint, to the struct literal near gemini_api_key, / gemini_model, (~line 589-590).

Debug impl (after the gemini_model field line ~228):

            .field("llm_endpoint", &self.llm_endpoint)

validate() (directly after the gemini_model empty check ~line 813):

        let llm_endpoint = self.llm_endpoint.trim();
        if llm_endpoint.is_empty() {
            anyhow::bail!("LLM_ENDPOINT must not be empty");
        }
        if !llm_endpoint.starts_with("http://") && !llm_endpoint.starts_with("https://") {
            anyhow::bail!("LLM_ENDPOINT must be an http(s) URL");
        }

Fixtures — add one line right after gemini_model: "gemini-3.6-flash".to_string(), in each of the five struct literals:

            llm_endpoint: DEFAULT_LLM_ENDPOINT.to_string(),

Locations: config.rs sample_config() (~1127), router.rs (~1157), tests/common/mod.rs (~373 and ~722), tests/api_ingest.rs (~739). In the three tests/ files the const is xylolabs_server::config::DEFAULT_LLM_ENDPOINT — check the existing use lines and import it the same way AppConfig is imported there.

  • [ ] Step 4: Run tests to verify they pass

Run: cargo test -p xylolabs-server --lib config::tests -- --nocapture Expected: all config tests PASS (the three new ones plus every pre-existing one).

  • [ ] Step 5: Verify workspace still compiles

Run: cargo check Expected: zero errors (this catches any fixture missed in Step 3).

Task 2: Point the three call sites at config.llm_endpoint

Files: - Modify: crates/xylolabs-server/src/services/alert_llm.rs:9-10 (const), :160 (post) - Modify: crates/xylolabs-server/src/routes/daily_report.rs:23-24 (const), :507 (post) - Modify: crates/xylolabs-server/src/routes/facility_assistant.rs:41-42 (const), :1041 (post inside run_gemini_chat_loop), :1149 (fn signature summarize_tool_results_with_gemini), :1182 (post)

Interfaces: - Consumes: config.llm_endpoint: String from Task 1. - Produces: summarize_tool_results_with_gemini gains a llm_endpoint: &str parameter (first caller is inside run_gemini_chat_loop, which has state).

  • [ ] Step 1: alert_llm.rs — delete the GEMINI_ENDPOINT const (lines 9-10); change the request build at line 160 to:
    let response = http_client
        .post(&config.llm_endpoint)
        .bearer_auth(api_key)

(config: &AppConfig is already a parameter.) Also update the two "Gemini … not configured/unavailable" error strings in this file to say "LLM" (pure string rename only).

  • [ ] Step 2: daily_report.rs — delete the const (lines 23-24); at line 507:
            .post(&state.config.llm_endpoint)
  • [ ] Step 3: facility_assistant.rs — delete the const (lines 41-42). In run_gemini_chat_loop (~1041): .post(&state.config.llm_endpoint). Add a parameter to summarize_tool_results_with_gemini:
async fn summarize_tool_results_with_gemini(
    client: &Client,
    llm_endpoint: &str,
    api_key: &str,
    // …existing params unchanged

change its post (~1182) to .post(llm_endpoint), and update its single call site (inside run_gemini_chat_loop — grep summarize_tool_results_with_gemini() to pass &state.config.llm_endpoint.

  • [ ] Step 4: Compile + full test suite

Run: cargo check && cargo clippy -- -D warnings 2>&1 | tail -5 Expected: zero errors/warnings. Run: cargo test -p xylolabs-server 2>&1 | tail -15 Expected: all suites pass (integration tests need the dev DB up — if the environment lacks it, run at minimum cargo test -p xylolabs-server --lib and say so in the report).

Task 3: Registry description, env templates, docs

Files: - Modify: crates/xylolabs-server/src/config_manager.rs:465 (registry description) - Modify: .env.example:133-135 region - Modify: scripts/setup-server.sh:109-110 region - Modify: docs/DEPLOYMENT-GUIDE.md:57-58 region (env table) - Modify: docs/API.en.md:6524 and docs/API.ko.md:6500 (daily-report model note) - Modify: docs/KNOWLEDGE-BASE.md (§ daily report, ~line 807) and docs/KNOWLEDGE-BASE.ko.md (~line 537)

Interfaces: none (docs/config-registry only).

  • [ ] Step 1: config_manager.rs:465 — change the description string to:
            "Model override for the facility assistant, sent to the configured LLM endpoint",
  • [ ] Step 2: .env.example — extend the Gemini block:
# Gemini / SMS integrations (optional)
GEMINI_API_KEY=
GEMINI_MODEL=gemini-3.6-flash
# Optional OpenAI-compatible endpoint override (e.g. OpenRouter).
# When set, LLM_API_KEY/LLM_MODEL take precedence over GEMINI_API_KEY/GEMINI_MODEL.
# LLM_ENDPOINT=https://openrouter.ai/api/v1/chat/completions
# LLM_API_KEY=
# LLM_MODEL=google/gemini-3.6-flash

scripts/setup-server.sh gets the same three commented lines after its GEMINI_MODEL= line.

  • [ ] Step 3: docs — DEPLOYMENT-GUIDE env table: add three rows after GEMINI_API_KEY:
| `LLM_ENDPOINT` | `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` | OpenAI-compatible chat/completions URL all LLM calls POST to (set to OpenRouter to switch providers) |
| `LLM_API_KEY` | *(falls back to `GEMINI_API_KEY`)* | Bearer key for `LLM_ENDPOINT` |
| `LLM_MODEL` | *(falls back to `GEMINI_MODEL`)* | Model id sent to `LLM_ENDPOINT` (OpenRouter style: `google/gemini-3.6-flash`) |

API.en.md daily-report Model paragraph: append the sentence "Model resolution order: LLM_MODEL → GEMINI_MODEL; the endpoint and key come from LLM_ENDPOINT/LLM_API_KEY with Gemini fallbacks (see Deployment Guide)." API.ko.md: same sentence in Korean ("모델 결정 순서: LLM_MODEL → GEMINI_MODEL. endpoint와 키는 LLM_ENDPOINT/LLM_API_KEY가 우선하며 미설정 시 Gemini 값으로 폴백한다(배포 가이드 참조).") KNOWLEDGE-BASE EN+KO: one short bullet under the daily-report section describing the OpenRouter switch procedure (three env vars + container restart; deleting them reverts to Gemini).

  • [ ] Step 4: Commit + deploy
cargo check && cargo fmt --check
git add -A
git commit -S -m "feat(llm): ✨ configurable OpenAI-compatible endpoint (OpenRouter support)"
~/flash-shared/gitminer-cuda/mine_commit.sh 7
git pull --rebase && git push
bash scripts/deploy.sh

Post-deploy verification (per CLAUDE.md): app container Up (healthy); curl -s https://api.xylolabs.com/api/health → {"status":"ok"}; container env unchanged (GEMINI_MODEL=gemini-3.6-flash, no LLM_* set) — production keeps using Gemini identically.