Skip to content

Per-Device Storage Toggle — Design

Date: 2026-07-17 Status: Approved (brainstorming complete)

Problem

Bench/test units stream large volumes of audio and metadata into the batch ingest pipeline, filling S3 and the metadata_chunks table with data nobody needs. Operators need a way to temporarily stop persisting data from a specific device while keeping the device online, streaming, and visible in live dashboards.

Decision Summary

Question Decision
What stops being saved All bulk timeseries: S3 audio chunks, metadata_chunks inserts, derived noise-level samples, UWB survey edges. Device health reports and device_events are still saved.
Expiry semantics Manual only — stays off until an operator toggles it back on. No auto-expiry.
UI surface Admin console (admin.api.xylolabs.com) DevicesPage: one-click row toggle + state shown in the edit modal. FacilityAdmin role.
Mid-session effect Takes effect within seconds mid-session (proactive cache invalidation; 30 s TTL fallback). No device reconnect required.
Mechanism Dedicated devices.storage_disabled boolean column (approach A), mirroring the existing is_active end-to-end pattern.

1. Data Model & API

Migration

crates/xylolabs-db/migrations/<YYYYMMDDHHMMSS>_devices_storage_disabled.sql:

ALTER TABLE devices ADD COLUMN storage_disabled BOOLEAN NOT NULL DEFAULT false;
  • Version prefix must be strictly monotonic and greater than every existing migration (verify with ls crates/xylolabs-db/migrations | awk -F_ '{print $1}' | sort -c).
  • Template: 20260715230000_devices_usb_connected.sql (documented comment block), but NOT NULL DEFAULT false since this is an operator toggle, not nullable telemetry.

Rust types

  • Device (crates/xylolabs-core/src/models/device.rs): add storage_disabled: bool.
  • UpdateDeviceRequest (crates/xylolabs-core/src/dto/ingest.rs, ~line 173): add storage_disabled: Option<bool> beside is_active. Struct is #[serde(deny_unknown_fields)], so the field must be added here for PATCH to accept it.
  • DeviceResponse (dto/ingest.rs, ~line 275): add storage_disabled: bool.
  • DeviceDashboardItem (dto/device.rs, ~line 257): add storage_disabled: bool so the admin device list can render toggle state.

Repo

crates/xylolabs-db/src/repo/device.rs::update (line ~666, DEVICE_FACILITY_SYNC_SQL):

  • Add storage_disabled = COALESCE($N, storage_disabled) to the UPDATE head.
  • Append the bind after the current last param ($17) per the documented "append last to avoid renumbering" convention (comment at ~line 690).
  • All device-fetch queries that materialize Device/DeviceDashboardItem select the new column.

HTTP API

  • PATCH /api/v1/devices/{id} (existing endpoint, routes/devices.rs::update, FacilityAdmin): accepts storage_disabled: bool (optional field; omitted = unchanged).
  • Response bodies (DeviceResponse, dashboard list) include storage_disabled.
  • The flag change is recorded in the audit log (action e.g. device.storage_toggle, with old/new value), following the existing admin-action audit pattern.
  • No new endpoint, so no RBAC-coverage matrix change; the existing PATCH role gate (FacilityAdmin) applies.

2. Ingest Gate (server behavior)

Storage-flag cache

A small in-memory TTL cache owned by the ingest side (on IngestManager or AppState), mapping device_id (Uuid) → (storage_disabled: bool, fetched_at: Instant):

  • is_storage_disabled(device_id) -> bool: returns the cached value if younger than 30 s, otherwise re-reads the single indexed devices column and refreshes the entry.
  • Fail-open: if the DB read errors, treat the device as enabled (persist data). Losing production data is worse than storing extra bench data.
  • The PATCH handler proactively invalidates/updates the cache entry when the flag changes (same process), so the toggle takes effect at the very next flush. The 30 s TTL is the fallback (e.g. multi-instance future, missed invalidation).

Gated write paths

  1. Batch ingest flush — ingest/manager.rs::flush_samples (~line 2195): immediately after the existing samples.is_empty() guard, check the cache; when disabled, early-return Ok(()), skipping:
  2. the S3 audio chunk upload (storage.upload_bytes),
  3. the metadata_chunks insert (repo::metadata_chunk::create_with_id),
  4. the session byte-stats update (repo::ingest_session::update_stats).

The device receives normal success and keeps streaming. The live broadcast and latest-sample caches are updated in process_batch before flush, so real-time dashboards keep working while storage is off. 2. UWB survey edges — routes/uwb_ingest.rs::ingest_uwb_edges: after device resolution, when disabled, accept-and-drop — return the normal success response shape without calling insert_edges. 3. Noise-level worker — services/noise_level.rs: skip devices whose flag is set (no new source chunks exist anyway; the skip avoids deriving from chunks stored just before the toggle).

Explicitly NOT gated (kept as-is)

  • Device health reports (POST /devices/health → devices row + device_health_history).
  • device_events rows.
  • Continuous live-streaming archive (live/manager.rs::archive_lc3) — separate stream-keyed feature. Note: on-demand live-capture (capture_kind="live") sessions ARE ingest sessions, so they are gated automatically by (1).
  • Ingest session rows (tiny; still created so operators can see the device connected — stats simply stop advancing while off).

3. Admin UI (frontend/)

  • frontend/src/api/devices.ts: add storage_disabled to the Device interface and UpdateDeviceBody.
  • frontend/src/pages/DevicesPage.tsx:
  • One-click toggle switch per device row using the same mutation pattern as the existing collision-clear action (updateDevice(id, { storage_disabled: … })), with toast feedback and query invalidation.
  • A subdued SVG indicator ("storage off") on rows where disabled — no emoji.
  • The edit modal displays the current state read-only; the row switch is the single write control.
  • i18n: EN + KO strings for the toggle label, indicator, and toasts.
  • Mobile-first: the toggle must be usable at 375 px; inputs ≥16 px font-size.
  • The operator app (frontend-app/) is intentionally untouched (read-only device views today; toggling is a FacilityAdmin/admin-console action).

4. Edge Cases

  • Buffered samples at toggle time: whatever is in the session buffer is dropped at the next flush. Acceptable — the point is to stop persisting.
  • Timeline gaps: timeseries/timeline queries return no data for the off window. Expected; the admin UI indicator explains why.
  • Session stats: ingest_sessions rows exist with stalled byte counts while off. Cosmetic; acceptable.
  • Facility move / delete / soft-delete: unaffected; storage_disabled is orthogonal to is_active (soft-delete). Do NOT reuse is_active — it has facility-cascade soft-delete semantics and is not an ingest gate.
  • Auto-registered devices: new devices default to storage_disabled = false.

5. Testing & Validation

  • Integration tests (new api_*.rs or extension of the ingest/devices suites):
  • PATCH sets the flag (FacilityAdmin OK, plain User rejected 403).
  • With flag on: ingest flush persists no metadata_chunks row and no S3 object; session still accepted; toggle off → writes resume.
  • UWB edge ingest accept-and-drop while disabled.
  • DeviceResponse / dashboard list expose the field.
  • Build validation: cargo check + cargo clippy; npx tsc -b --noEmit and npx vite build in frontend/ (and frontend-app/ for the shared check).
  • Browser testing: Playwright at 375×812 / 768×1024 / 1280×800 on the DevicesPage; zero pageerror events.
  • Docs: device PATCH field + ingest behavior documented in docs/API.en.md and docs/API.ko.md.
  • Deploy: scripts/deploy.sh to api.xylolabs.com, then post-deploy verification per CLAUDE.md.