E2E Test Results — Hardware Validation Session¶
Xylolabs API — End-to-End Hardware Test Results Test date: 2026-03-27
XAP and XMBP are patent-pending technologies of Xylolabs Inc.
Test Environment¶
| Item | Details |
|---|---|
| Host machine | Mac mini (Apple Silicon), macOS |
| Server | xylolabs-server (Rust/axum), release build, 0.0.0.0:3000 |
| Database | PostgreSQL 17 (Docker, tmpfs) |
| Object storage | MinIO (Docker, tmpfs) |
| Network | WiFi Wireless_2.4G (WPA2), same router bridge |
| Firmware | MicroPython v1.27.0 on all devices |
Devices Under Test¶
| # | Device | MCU | Architecture | Clock | RAM | WiFi | MAC |
|---|---|---|---|---|---|---|---|
| 1 | Raspberry Pi Pico 2 W | RP2350 A2 | Cortex-M33 | 150 MHz | 446 KB | CYW43439 | 2c:cf:67:bc:ca:b2 |
| 2 | M5StampS3A | ESP32-S3 v0.2 | Xtensa LX7 | 160 MHz | 222 KB | Native | 30:ed:a0:c9:4b:ec |
| 3 | Seeed XIAO ESP32S3 | ESP32-S3 v0.2 | Xtensa LX7 | 160 MHz | 222 KB | Native | 1c:db:d4:75:92:70 |
WiFi Connectivity Tests¶
Network Scan¶
All three devices performed active scans at test start; 7–13 networks were visible in range.
SSID Compatibility¶
| SSID | Pico 2 W | M5StampS3A | Seeed XIAO |
|---|---|---|---|
| "Legacy" | BADAUTH (-3) |
Silent fail | Silent fail |
Wireless_2.4G |
Connected | Connected | Connected |
The "Legacy" SSID operates in WPA2/WPA3 transitional mode, which is incompatible with the CYW43439 driver and the ESP32-S3 MicroPython network stack. Both result in a silent or error fail; Wireless_2.4G (pure WPA2) succeeded on all devices.
Connection Results¶
| Device | Connect time | RSSI | IP address |
|---|---|---|---|
| Pico 2 W | 9 s | -20 dBm | 172.30.86.26 |
| M5StampS3A | 3 s | -34 dBm | 172.30.86.25 |
| Seeed XIAO ESP32S3 | 6 s | -65 dBm | 172.30.86.27 |
WiFi E2E Pipeline¶
Each device executed the full pipeline in order:
- WiFi connect
GET /api/health→ HTTP 200POST /api/auth/login→ JWT tokenPOST /api/auth/api-keys→ API keyPOST /api/sessions→ HTTP 201- 10 × XMBP batch sends
POST /api/sessions/{id}/close→ HTTP 200
| Device | Batches sent | Session close |
|---|---|---|
| Pico 2 W | 10/10 OK | 200 OK |
| M5StampS3A | 10/10 OK | ETIMEDOUT (data flushed OK) |
| Seeed XIAO ESP32S3 | 10/10 OK | 200 OK |
The M5StampS3A session-close timeout is a client-side TCP timeout on the final request; server-side logs confirm the session was flushed and closed correctly before the socket reset.
Profiling Results¶
Memory¶
| Metric | Pico 2 W | M5StampS3A | Seeed XIAO ESP32S3 |
|---|---|---|---|
| Total RAM | 446 KB | 222 KB | 222 KB |
| Free (idle) | 431 KB (97%) | 209 KB (94%) | 209 KB (94%) |
| MicroPython overhead | 15 KB | 13 KB | 13 KB |
| Max single alloc | 256 KB | 128 KB | 128 KB |
| Post-test free | 430 KB | 163 KB | 163 KB |
| Memory leak | None | None | None |
The Pico 2 W has approximately twice the available heap of the ESP32-S3 devices. Post-test free on ESP32-S3 (163 KB vs 209 KB at idle) reflects accumulated MicroPython runtime state during the session, not a leak; the runtime footprint stabilises after the first few transmit cycles.
CPU Benchmark (MicroPython)¶
| Operation | Pico 2 W | M5StampS3A | Seeed XIAO ESP32S3 |
|---|---|---|---|
| Int 10K ops (us) | 42,376 | 62,030 | 62,067 |
| Int ops/s | 235,983 | 161,212 | 161,116 |
| Float 10K ops (us) | 125,983 | 140,916 | 140,904 |
| Float ops/s | 79,376 | 70,964 | 70,970 |
| Struct pack 1K (us) | 35,010 | 25,548 | 25,553 |
| Struct pack/s | 28,563 | 39,142 | 39,134 |
The Pico 2 W (Cortex-M33) leads on integer and float throughput. The ESP32-S3 (Xtensa LX7) outperforms on struct.pack throughput, which benefits XMBP batch encoding.
Network Latency (HTTP GET /api/health, 5 samples)¶
| Metric | Pico 2 W | M5StampS3A | Seeed XIAO ESP32S3 |
|---|---|---|---|
| Avg (us) | 15,226 | 38,091 | 51,693 |
| Min (us) | 8,903 | 33,383 | 45,153 |
| Max (us) | 22,146 | 44,074 | 60,957 |
Pico 2 W achieves the lowest latency, consistent with its stronger RSSI (-20 dBm). Seeed XIAO antenna placement (-65 dBm) adds ~13 ms average overhead versus M5StampS3A.
XMBP Throughput (20 batches, 4 streams × 10 samples per batch)¶
| Metric | Pico 2 W | M5StampS3A | Seeed XIAO ESP32S3 |
|---|---|---|---|
| Encode avg (us) | 3,315 | 3,160 | 3,242 |
| Encode max (us) | 3,410 | 3,498 | 3,396 |
| Send avg (us) | 14,577 | 38,114 | 42,311 |
| Send min (us) | 8,307 | 30,563 | 32,251 |
| Send max (us) | 28,560 | 56,503 | 56,873 |
| Total avg (us) | 17,893 | 41,275 | 45,554 |
| Bytes per batch | 514 | 514 | 514 |
| Batches/sec | 55.9 | 24.2 | 22.0 |
| Samples/sec | 2,236 | 969 | 878 |
Encode time is comparable across all three devices (3.2–3.3 ms). The dominant cost is network send time, which correlates directly with RSSI and TCP round-trip latency.
Resilience Tests¶
9 test cases executed on all 3 devices.
| TC | Description | Pico 2 W | M5StampS3A | Seeed XIAO |
|---|---|---|---|---|
| TC1 | Rapid-fire 50 batches | PASS (42.1/s) | PASS (17.6/s) | PASS (18.9/s) |
| TC2 | Large batch 4.8 KB | PASS (245 ms) | PASS (462 ms) | PASS (542 ms) |
| TC3 | Many concurrent streams | PASS | PASS | PASS |
| TC4 | WiFi disconnect/reconnect | PASS (3 s) | PASS (2 s) | PASS (2 s) |
| TC5 | 3 concurrent sessions | PASS (30/30) | PASS (30/30) | PASS (30/30) |
| TC6 | Malformed XMBP | PASS (400/400/400/200) | PASS | PASS |
| TC7 | Auth edge cases | PASS (401/401/201) | PASS | PASS |
| TC8 | Sequence gaps | 6/7 | 6/7 | 6/7 |
| TC9 | 30 s sustained 10 Hz | PASS (300/300) | PASS (300/300) | PASS (300/300) |
| Score | 8/9 | 8/9 | 8/9 |
TC8 (sequence gaps): one gap variant was not handled at the server as expected; this is a known server-side issue unrelated to device behaviour. All three devices produced identical results.
Burn-In Results¶
Burn-in tests were run using the host-side SDK simulation (Rust). All scenarios completed with zero failed batches and zero reconnects.
Scenario Summary¶
| Scenario | Duration | Audio | Sensors | Devices | Audio batches | Meta batches | Failed | Reconnects | Headroom |
|---|---|---|---|---|---|---|---|---|---|
| Standard | 60 s | 4ch @ 16kHz | 4 @ 100 Hz | 1 | 118 | 59 | 0 | 0 | 99.5% |
| Stress | 120 s | 4ch @ 96kHz | 26 @ 100 Hz | 1 | 239 | 119 | 0 | 0 | 99.1% |
| Endurance | 120 s | 2ch @ 16kHz | 4 @ 10 Hz | 1 | 237 | 119 | 0 | 0 | 99.6% |
| Multi-device | 60 s | 4ch @ 16kHz | 4 @ 100 Hz | 10 | 118 each | 59 each | 0 | 0 | 99.6–99.7% |
Per-Scenario Encode and Tick Performance¶
| Scenario | Encode avg (us) | Tick avg (us) | Tick max (us) |
|---|---|---|---|
| Standard | 10 | 41 | 19,845 |
| Stress | 6 | 9 | 213,690 |
| Endurance | 5 | 36 | 15,398 |
| Multi-device | 7–8 per device | 22–35 per device | — |
The Stress scenario tick max (213,690 us) reflects occasional OS scheduling preemption at 96kHz / 26-sensor load; it does not indicate a missed deadline in the SDK ring buffer.
Server-Side Verification¶
All sessions were verified against server-side logs and database state after the test run.
| Metric | Result |
|---|---|
| Devices registered | 10+ across all sessions |
| Metadata streams | 74+ streams across 13+ sessions |
| S3 storage | Zstd-compressed chunks flushed, 125–752 bytes per chunk |
| Session lifecycle | create → stream → flush → close confirmed in server logs |
| Server errors | Zero |
RISC-V (Hazard3) Supplementary Test¶
The Pico 2 W (RP2350) contains a second core implementing the Hazard3 RISC-V processor. A Pico SDK C benchmark was compiled for the rp2350-riscv target and flashed via picotool.
| Item | Result |
|---|---|
| Target | rp2350-riscv |
| Flash method | picotool, family rp2350-riscv |
| USB CDC serial | Not received |
| Root cause | Known TinyUSB / Hazard3 initialisation limitation |
| Core spec | RV32IMAC, no FPU, 150 MHz |
| Suitability | Sensor-only workloads, ADPCM encoding |
USB CDC output was not received on any attempt. This is a known limitation of the TinyUSB stack on the Hazard3 core; the binary flashed successfully and the device enumerated, but serial output was not produced. This does not affect the Cortex-M33 primary core results.
Known Issues¶
| # | Issue | Affected | Severity |
|---|---|---|---|
| 1 | WPA2/WPA3 transitional SSID incompatible with CYW43 and ESP32 MicroPython | All devices | Medium — use pure WPA2 SSID |
| 2 | USB CDC blocked on Hazard3 (TinyUSB init issue) | Pico 2 W RISC-V only | Low — does not affect primary core |
| 3 | Antenna placement reduces ESP32-S3 throughput 10–20% (-34 vs -65 dBm) | Seeed XIAO ESP32S3 | Low — physical placement dependent |
| 4 | Embedded Rust firmware examples have dependency version conflicts (embassy-rp, esp-hal) |
Firmware examples | Low — affects dev ergonomics only |