Skip to content

Sampling Mode Session Persistence Plan

Status: Implemented
Date: 2026-04-05
Scope: DB schema, server ingest, SDK, API, frontend, docs

1. Problem

Devices operate in two distinct modes:

Mode Behavior Example Sessions/day
Continuous Streams data non-stop Vibration monitor on rotating machinery 1–2
Sampling Measures periodically, sleeps between Battery sensor: 10s every 5min ~288

Current session timeout (ingest.session_timeout_secs, default 300s) auto-closes sessions after 5 minutes of inactivity. For sampling devices with intervals > 5min, every measurement cycle creates a new session. This causes:

  • Session explosion: Hundreds of sessions per device per day
  • Fragmented visualization: Each session is a disconnected island on the timeline
  • Difficult management: Browsing/filtering becomes unusable
  • Wasted overhead: Session creation/close cycle for every measurement burst

Goal

A single long-lived session per device that persists across sampling gaps, with continuous timeline visualization that naturally shows the gaps.

2. Design: Session Modes

2.1 Ingest Session Mode

Add a mode field to ingest sessions:

Mode Constant Timeout Behavior Session Lifetime
continuous SESSION_MODE_CONTINUOUS Close after session_timeout_secs idle (current behavior) Hours–days
sampling SESSION_MODE_SAMPLING Close after sampling_timeout_secs idle (much longer) Days–weeks

2.2 Sampling Configuration

Add optional sampling config to the session:

pub struct SamplingConfig {
    pub interval_secs: u32,     // Expected interval between measurements (e.g., 300 = 5min)
    pub duration_secs: u32,     // Expected measurement duration (e.g., 10)
}

This is informational metadata — it tells the server what to expect, but doesn't enforce timing. The server uses it for: - Timeout calculation: sampling_timeout_secs = interval_secs * 3 (miss 3 cycles before closing) - Visualization hints: Frontend knows where to expect gaps - Health monitoring: Detect if a sampling device misses its schedule

2.3 Timeout Logic

For continuous mode (default):
  timeout = config("ingest.session_timeout_secs", 300)  // 5 minutes

For sampling mode:
  timeout = max(
    config("ingest.sampling_timeout_secs", 3600),        // 1 hour minimum
    sampling_config.interval_secs * 3                     // 3× the expected interval
  )

Example: A device sampling 10s every 5min (interval=300s) would have timeout = max(3600, 300*3) = 3600s = 1 hour.

A device sampling 10s every 30min (interval=1800s) would have timeout = max(3600, 1800*3) = 5400s = 1.5 hours.

3. Schema Changes

3.1 Migration: Add mode and sampling config to ingest_sessions

ALTER TABLE ingest_sessions
  ADD COLUMN mode TEXT NOT NULL DEFAULT 'continuous'
    CHECK (mode IN ('continuous', 'sampling')),
  ADD COLUMN sampling_interval_secs INTEGER,
  ADD COLUMN sampling_duration_secs INTEGER;

COMMENT ON COLUMN ingest_sessions.mode IS
  'continuous = nonstop streaming; sampling = periodic bursts with idle gaps';
COMMENT ON COLUMN ingest_sessions.sampling_interval_secs IS
  'Expected seconds between measurement starts (sampling mode only)';
COMMENT ON COLUMN ingest_sessions.sampling_duration_secs IS
  'Expected seconds per measurement burst (sampling mode only)';

CREATE INDEX idx_ingest_sessions_mode ON ingest_sessions (mode) WHERE mode = 'sampling';

3.2 Migration: Add session timeout config

INSERT INTO system_config (key, value, category, value_type, description)
VALUES (
  'ingest.sampling_timeout_secs', '3600', 'ingest', 'integer',
  'Base idle timeout for sampling-mode sessions (seconds). Actual timeout is max(this, interval*3).'
)
ON CONFLICT (key) DO NOTHING;

4. API Changes

4.1 Create Session Request

Add optional fields to POST /api/v1/ingest/sessions:

{
  "device_uid": 42,
  "name": "sensor-batch-001",
  "mode": "sampling",
  "sampling_interval_secs": 300,
  "sampling_duration_secs": 10,
  "streams": [...]
}
Field Type Default Validation
mode string "continuous" Must be "continuous" or "sampling"
sampling_interval_secs u32? null Required if mode=sampling; 1–86400
sampling_duration_secs u32? null Required if mode=sampling; 1–sampling_interval_secs

4.2 Session Response

Add new fields to IngestSessionResponse:

{
  "id": "...",
  "mode": "sampling",
  "sampling_interval_secs": 300,
  "sampling_duration_secs": 10,
  ...
}

4.3 List Sessions

Add mode filter to GET /api/v1/metadata/sessions?mode=sampling:

WHERE ($mode IS NULL OR mode = $mode)

5. Server Changes

5.1 IngestManager — Session State

Add mode info to SessionState:

pub struct SessionState {
    // ... existing fields
    pub mode: IngestSessionMode,
    pub sampling_interval_secs: Option<u32>,
    pub timeout_ms: u64,  // Pre-calculated at registration time
}

5.2 IngestManager — Timeout Calculation

In register_session(), calculate timeout based on mode:

let timeout_ms = match mode {
    IngestSessionMode::Continuous => {
        config.get_i64("ingest.session_timeout_secs", 300).await as u64 * 1000
    }
    IngestSessionMode::Sampling => {
        let base = config.get_i64("ingest.sampling_timeout_secs", 3600).await as u64;
        let interval_based = sampling_interval_secs.unwrap_or(0) as u64 * 3;
        base.max(interval_based) * 1000
    }
};

5.3 IngestManager — Flusher Timeout Check

In run_flusher(), use per-session timeout instead of global:

// BEFORE (global timeout for all sessions):
let timeout_ms = self.config_manager.get_i64("ingest.session_timeout_secs", 300).await as u64 * 1000;

// AFTER (per-session timeout):
for (id, session) in sessions.iter() {
    let last_ms = session.last_activity_ms.load(Ordering::Acquire);
    if now_ms.saturating_sub(last_ms) > session.timeout_ms {
        stale_sessions.push(*id);
    }
}

5.4 Session Resume (Optional Enhancement)

For devices that power-cycle between sampling bursts and cannot maintain TCP/WebSocket:

Allow POST /api/v1/ingest/sessions/{id}/data to reactivate a closed sampling session. If the session was closed by auto-timeout (not explicit close), and the device sends new data within sampling_timeout_secs * 2, the server can re-open it:

// In receive_data():
if session.status == "closed" && session.mode == "sampling" {
    let closed_ago = now - session.closed_at;
    if closed_ago < Duration::from_secs(session.timeout_ms / 1000 * 2) {
        repo::ingest_session::reopen(&db, session.id).await?;
        state.ingest.register_session(session.id, ...);
    }
}

This is an optional quality-of-life feature. The primary mechanism (longer timeout) handles most cases.

6. SDK Changes

6.1 Config

Add to Config:

pub session_mode: SessionMode,              // Default: Continuous
pub sampling_interval_secs: Option<u32>,    // Required for Sampling mode
pub sampling_duration_secs: Option<u32>,    // Required for Sampling mode

6.2 Session Creation

The SDK sends mode and sampling config in the session creation request.

6.3 C SDK

Add to xylolabs_session_config_t:

typedef enum {
    XYLOLABS_SESSION_MODE_CONTINUOUS = 0,
    XYLOLABS_SESSION_MODE_SAMPLING   = 1,
} xylolabs_session_mode_t;

typedef struct {
    // ... existing fields
    xylolabs_session_mode_t mode;
    uint32_t sampling_interval_secs;  // 0 = not set
    uint32_t sampling_duration_secs;  // 0 = not set
} xylolabs_session_config_t;

7. Frontend Changes

7.1 Metadata Sessions List (Uploads Page)

  • Add mode badge (continuous/sampling) next to status badge
  • Add mode filter dropdown alongside status and device filters
  • For sampling sessions, show interval info (e.g., "10s / 5min")

7.2 Session Detail Page — Gap-Aware Visualization

For sampling-mode sessions, the timeline chart must handle gaps:

Current behavior: Charts show data points as a continuous line. Gaps between measurement bursts would appear as long flat lines connecting the last point of one burst to the first point of the next.

Required behavior: - Detect gaps larger than sampling_duration_secs * 2 between consecutive points - Break the line at gaps (null/NaN data points in chart series) - Optionally shade gap regions with a light background color - Show gap duration on hover

Implementation in StreamChart / AccelChart: 

// Insert null points at detected gaps to break the line
function insertGapBreaks(points: StreamDataPoint[], gapThresholdUs: number): StreamDataPoint[] {
  const result: StreamDataPoint[] = []
  for (let i = 0; i < points.length; i++) {
    if (i > 0 && points[i].timestampUs - points[i-1].timestampUs > gapThresholdUs) {
      // Insert a null point to break the line
      result.push({ timestampUs: points[i-1].timestampUs + 1, value: null })
    }
    result.push(points[i])
  }
  return result
}

7.3 Session Detail — Sampling Info Panel

Show sampling schedule info above the charts:

Mode: Sampling | Interval: 5 min | Duration: 10 sec | Active since: Apr 5, 12:00

7.4 Dashboard

  • Show sampling vs continuous session counts in the active sessions panel
  • Optionally group by mode

8. Visualization Timeline Design

8.1 Gap Detection

The frontend detects gaps by comparing consecutive timestamps:

gap_threshold_us = sampling_interval_secs * 1_000_000 * 0.5
  (half the interval — if a gap exceeds this, it's a measurement boundary)

For continuous mode: use a fixed threshold based on the stream's sample_rate_hz.

8.2 Time Range Selector

The existing time range selector ("All", "Last 1m", "5m", "15m", "1h") works for both modes. For sampling sessions, "All" may span days/weeks — the downsample parameter on the API already handles this.

8.3 Spectrogram / FFT

VibrationSpectrogram should only compute FFT within measurement bursts, not across gaps. Split the input values at gap boundaries and compute separate spectrograms per burst, placing them at the correct time offsets.

9. Implementation Phases

Phase 1: Schema + Server (Backend)

  • [ ] Add migration for mode, sampling_interval_secs, sampling_duration_secs
  • [ ] Add migration for ingest.sampling_timeout_secs config
  • [ ] Update CreateIngestSessionRequest DTO with mode/sampling fields
  • [ ] Update IngestSessionResponse DTO
  • [ ] Update IngestSession model
  • [ ] Update SessionState with per-session timeout
  • [ ] Update flusher timeout logic to use per-session timeout
  • [ ] Update list_sessions query to filter by mode
  • [ ] Add validation: sampling fields required when mode=sampling
  • [ ] Tests: sampling session survives long idle, continuous session times out as before

Phase 2: SDK

  • [ ] Add session mode to Rust SDK Config
  • [ ] Add session mode to C SDK config struct
  • [ ] Send mode/sampling fields in session creation
  • [ ] Update SDK README and examples

Phase 3: Frontend

  • [ ] Update MetadataSessionsPage with mode filter and badge
  • [ ] Update SessionDetailPage with sampling info panel
  • [ ] Implement gap-aware line chart rendering
  • [ ] Update spectrogram to handle gaps
  • [ ] Add i18n strings (EN + KO)

Phase 4: Documentation

  • [ ] Update XMBP-SPECIFICATION.md with session mode
  • [ ] Update KNOWLEDGE-BASE.md
  • [ ] Update AGENTS.md
  • [ ] Update README.md
  • [ ] Update platform guides

10. Backward Compatibility

  • Default mode is continuous — existing sessions and devices are unaffected
  • Existing SDK firmware (no mode field) creates continuous sessions as before
  • The migration adds columns with defaults; no data migration needed
  • API response adds new fields; clients ignoring them are unaffected

11. Configuration Summary

Config Key Default Description
ingest.session_timeout_secs 300 Continuous mode idle timeout (unchanged)
ingest.sampling_timeout_secs 3600 Sampling mode base idle timeout (new)

Per-session effective timeout: - Continuous: session_timeout_secs (5 min default) - Sampling: max(sampling_timeout_secs, interval * 3) (1 hour+ default)