ESP32 Platform Guide

Xylolabs SDK integration for ESP32 microcontrollers.

---

Supported Targets

MCU	Core	Clock	RAM	Connectivity	Audio Capability
ESP32-S3	2x Xtensa LX7	240 MHz	512 KB + 8 MB PSRAM	WiFi 802.11b/g/n + BLE 5.0	4ch XAP @96kHz
ESP32-C3	RISC-V RV32IMC	160 MHz	400 KB	WiFi 802.11b/g/n + BLE 5.0	Sensors only

---

Hardware Setup

I2S Microphone Connection (ESP32-S3)

The SDK expects interleaved PCM from two stereo I2S MEMS microphones (e.g. two INMP441 or ICS-43434 pairs providing left/right channels on a shared bus).

ESP32-S3 GPIO          MEMS Mic (e.g. INMP441)
──────────────────     ───────────────────────
GPIO14 (I2S_BCK) ───── BCK
GPIO15 (I2S_WS)  ───── WS / LRCK
GPIO16 (I2S_DIN) ───── SD (data in)
3V3              ───── VDD
GND              ───── GND
GND              ───── L/R (left mic)
3V3              ───── L/R (right mic)

For 4-channel audio with two stereo mic pairs, use two I2S peripheral instances (I2S0 and I2S1) or a single bus with L/R select and interleave in firmware.

WiFi Configuration

The SDK connects over TCP/TLS. WiFi provisioning is handled outside the SDK core:

// Rust (Embassy) - Recommended
use esp_wifi::wifi::{WifiController, WifiStaDevice};
use embassy_net::{Stack, Config};

// Initialize WiFi STA mode before xylolabs init
let wifi = WifiController::new(&init, wifi, WifiStaDevice).unwrap();
wifi.set_configuration(&wifi::Configuration::Client(wifi::ClientConfiguration {
    ssid: "your_ssid".try_into().unwrap(),
    password: "your_password".try_into().unwrap(),
    ..Default::default()
})).unwrap();
wifi.start().await.unwrap();
wifi.connect().await.unwrap();
// Wait for DHCP, then proceed

Legacy C equivalent

// Initialize WiFi before calling xylolabs_init()
wifi_init_sta("your_ssid", "your_password");
// Wait for IP assignment, then proceed

For production deployments, use ESP-IDF's esp_wifi provisioning component or WiFiProvisioning over BLE.

LTE-M1 Modem via UART (Optional)

For deployments without WiFi coverage, attach a UART LTE modem:

ESP32-S3 GPIO          LTE Modem (BG770A)
──────────────────     ──────────────────────────
GPIO17 (UART1_TX) ──── RXD
GPIO18 (UART1_RX) ──── TXD
GPIO19 (GPIO)     ──── PWRKEY
GPIO20 (GPIO)     ──── RESET_N
5V / VBAT         ──── VCC
GND               ──── GND

Sensor Buses

Bus	GPIO (S3 example)	Typical devices
I2C0	GPIO8 (SDA), GPIO9 (SCL)	BME280, LIS3DH, MPU6050
SPI2	GPIO11/12/13 + CS	MAX31865, ADS1256
ADC1	GPIO1–GPIO10	Analog sensors

---

Rust Build (Recommended)

# Install ESP Rust toolchain
cargo install espup
espup install

# Build ESP32-S3 example
cd sdk/examples/esp32s3-wifi
cargo build --release --target xtensa-esp32s3-none-elf

# Flash
cargo espflash flash --release

See sdk/examples/ for all ESP32 examples.

---

Legacy C Build System

Option A: ESP-IDF (idf.py)

# Install ESP-IDF (v5.2+ recommended for S3 USB support)
. $IDF_PATH/export.sh

# Create project
idf.py create-project my_xylolabs_device
cd my_xylolabs_device

# Add SDK as a component
mkdir -p components
ln -s /path/to/sdk/c/esp32 components/xylolabs_esp32
ln -s /path/to/sdk/c/common components/xylolabs_common

In CMakeLists.txt (project root):

cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(my_xylolabs_device)

In main/CMakeLists.txt:

idf_component_register(
    SRCS "main.c" "platform_esp32.c"
    INCLUDE_DIRS "."
    REQUIRES xylolabs_esp32 xylolabs_common esp_wifi esp_netif nvs_flash
)

Option B: PlatformIO

In platformio.ini:

[env:esp32s3]
platform = espressif32
board = esp32-s3-devkitc-1
framework = espidf
build_flags =
    -DXYLOLABS_PLATFORM_ESP32
    -DXYLOLABS_PLATFORM_ESP32S3
lib_deps =
    symlink://path/to/sdk/c/esp32
    symlink://path/to/sdk/c/common

---

ESP32-S3 Advantages

XAP and XMBP are patent-pending technologies of Xylolabs Inc.

The ESP32-S3 is the recommended ESP32 variant for full audio + sensor deployments:

• PSRAM (8 MB): Enables large audio ring buffers beyond internal SRAM limits. Map XYLOLABS_AUDIO_RING_SIZE into PSRAM for extended buffering.
• Dual-core architecture: Run audio encoding on Core 0 and network I/O on Core 1 via FreeRTOS task pinning, eliminating audio glitches during TCP retransmissions.
• USB OTG: Serial console and firmware flashing without a UART adapter.
• Vector extensions: Accelerated DSP for XAP spectral operations.

PSRAM Audio Ring Buffer

// Rust (Embassy) - Recommended
use esp_hal::psram;

// Enable PSRAM in config (sdkconfig or cargo features)
// Allocate audio ring buffer in PSRAM via esp-alloc
const AUDIO_RING_SIZE: usize = 256 * 1024; // 256 KB in PSRAM

#[link_section = ".ext_ram.bss"]
static mut AUDIO_RING_MEM: [u8; AUDIO_RING_SIZE] = [0u8; AUDIO_RING_SIZE];

Legacy C equivalent

// In sdkconfig or idf.py menuconfig:
// Component config → ESP PSRAM → Support for external, SPI-connected RAM → Enable

// Override ring buffer size to use PSRAM
#define XYLOLABS_AUDIO_RING_SIZE  (256 * 1024)  // 256 KB in PSRAM

// Allocate in PSRAM (ESP-IDF attribute)
static uint8_t s_audio_ring_mem[XYLOLABS_AUDIO_RING_SIZE]
    __attribute__((section(".ext_ram.bss")));

---

ESP32-C3: Sensor-Only Node

The ESP32-C3 RISC-V core does not include a hardware FPU suitable for real-time XAP encoding at useful audio rates. Use C3 for sensor-only deployments:

// Rust (Embassy) - Recommended
// Config override for C3 — set in build.rs or Cargo.toml features
// No audio on C3 (RISC-V, no FPU)
const AUDIO_CHANNELS: usize = 0;
const XMBP_BUF_SIZE: usize = 4096;
const SENSOR_CHANNELS: usize = 4;
const MOTOR_CHANNELS: usize = 8;

Legacy C equivalent

// config override for C3
#define XYLOLABS_AUDIO_CHANNELS    0   // No audio
#define XYLOLABS_XMBP_BUF_SIZE    4096
#define XYLOLABS_SENSOR_CHANNELS   4
#define XYLOLABS_MOTOR_CHANNELS    8

The C3 is well-suited for: - Environmental monitoring (temperature, humidity, CO2) - Vibration threshold alerting (raw ADC, not XAP-encoded) - Low-power WiFi sensor nodes with deep sleep

---

FreeRTOS Task Architecture (ESP32-S3)

Pin audio and network tasks to separate cores to prevent network jitter from corrupting audio captures:

// Rust (Embassy) - Recommended
use esp_hal::i2s::I2s;
use embassy_executor::Spawner;

// Audio capture task (pinned to Core 0)
#[embassy_executor::task]
async fn audio_task(i2s: I2s<'static>) {
    let mut buf = [0i16; 256 * 4];
    loop {
        let samples = i2s.read_async(&mut buf).await.unwrap();
        xylolabs_audio_feed(&buf[..samples]);
    }
}

// Network + SDK tick task (pinned to Core 1)
#[embassy_executor::task]
async fn network_task(stack: Stack<'static>) {
    let mut client = XylolabsClient::new(stack);
    client.set_api_key("xk_your_key");
    client.set_device_id(1);
    client.session_create("esp32_node_1", &streams).await;

    loop {
        client.tick().await;
        Timer::after_millis(10).await;
    }
}

Legacy C equivalent

// Audio capture + encode task (Core 0)
void audio_task(void *arg) {
    while (1) {
        // DMA callback fills s_i2s_buf
        i2s_read(I2S_NUM_0, s_i2s_buf, sizeof(s_i2s_buf), &bytes_read, portMAX_DELAY);
        xylolabs_audio_feed((int16_t *)s_i2s_buf, bytes_read / sizeof(int16_t));
    }
}

// Network + SDK tick task (Core 1)
void network_task(void *arg) {
    xylolabs_init(&g_platform);
    xylolabs_set_api_key("xk_your_key");
    xylolabs_set_device_id(1);
    xylolabs_session_create("esp32_node_1", streams, stream_count);

    while (1) {
        xylolabs_tick();
        vTaskDelay(pdMS_TO_TICKS(10));
    }
}

void app_main(void) {
    // ...WiFi init...
    xTaskCreatePinnedToCore(audio_task,   "audio",   4096, NULL, 5, NULL, 0);
    xTaskCreatePinnedToCore(network_task, "network", 8192, NULL, 4, NULL, 1);
}

---

WiFi vs LTE-M1

Feature	WiFi (native)	LTE-M1 (UART modem)
Range	~50–150 m indoor	Global
Latency	~5–20 ms	~50–200 ms
Power	~80–160 mA active	~10–200 mA active
Setup	esp_wifi API	AT commands over UART
Use case	Factory floor with WiFi AP	Remote/outdoor sites

Both connectivity options use the same SDK transport callbacks. Only the platform implementation differs.

---

OTA Firmware Updates

Use ESP-IDF's esp_https_ota component for over-the-air updates:

// Rust (Embassy) - Recommended
use esp_hal::reset::software_reset;
use esp_ota::OtaUpdate;

async fn ota_update(stack: Stack<'static>) {
    let url = "https://update.yourserver.com/firmware/esp32s3/latest.bin";
    let mut ota = OtaUpdate::begin().unwrap();
    // Stream firmware from HTTPS and write to OTA partition
    let mut client = HttpClient::new(stack);
    let response = client.get(url).await.unwrap();
    ota.write_all(response.body()).await.unwrap();
    ota.finalize().unwrap();
    software_reset();
}

Legacy C equivalent

#include "esp_https_ota.h"

void ota_task(void *arg) {
    esp_http_client_config_t config = {
        .url = "https://update.yourserver.com/firmware/esp32s3/latest.bin",
        .cert_pem = server_cert_pem_start,
    };
    esp_err_t ret = esp_https_ota(&config);
    if (ret == ESP_OK) {
        esp_restart();
    }
    vTaskDelete(NULL);
}

Trigger OTA from the Xylolabs dashboard by sending a command over the SSE control channel.

---

Power Management

Light Sleep Between Ticks

// Rust (Embassy) - Recommended
// Embassy automatically enters light sleep between async task polls.
// WiFi stays connected; CPU wakes on timer or interrupt.
loop {
    client.tick().await;
    Timer::after_millis(10).await; // CPU sleeps between ticks
}

Legacy C equivalent

// Configure automatic light sleep (WiFi stays connected)
esp_pm_config_t pm_config = {
    .max_freq_mhz = 240,
    .min_freq_mhz = 40,
    .light_sleep_enable = true,
};
esp_pm_configure(&pm_config);

// SDK tick at 10ms interval; CPU sleeps between ticks automatically

Deep Sleep for Sensor-Only (C3)

// Rust (Embassy) - Recommended
use esp_hal::sleep::{TimerWakeupSource, enter_deep_sleep};
use fugit::ExtU64;

// Wake every 60 seconds, send sensor batch, sleep again
let wakeup = TimerWakeupSource::new(60u64.secs());
// ... collect sensors, client.tick().await, flush ...
enter_deep_sleep(&[&wakeup]);

Legacy C equivalent

// Wake every 60 seconds, send sensor batch, sleep again
esp_sleep_enable_timer_wakeup(60 * 1000000ULL);  // 60s in microseconds
// ... collect sensors, xylolabs_tick(), flush ...
esp_deep_sleep_start();

ULP Coprocessor

For C3 or S3 ultra-low-power scenarios, use the ULP (Ultra-Low-Power) coprocessor to sample slow sensors (temperature, humidity) while the main CPU is in deep sleep. Wake the main CPU only when a threshold is crossed or a batch interval expires.

---

Rust (Recommended for ESP32-S3)

Rust is the recommended language for new ESP32 projects. The Embassy-based Rust examples provide async WiFi, I2S audio capture, and sensor streaming with compile-time safety.

cd sdk/examples/esp32s3-audio  # or esp32c3-sensor
cargo build --release
# Flash via espflash: espflash flash target/xtensa-esp32s3-none-elf/release/esp32s3-audio

See sdk/examples/README.md for the full architecture overview and per-target build instructions.

Note: For ESP32-C3 sensor-only deployments, both Rust and C are well-supported. Choose based on team familiarity.

---

C Examples (alternative)

### C Example: ESP32-S3 Full Audio + Sensors ### Platform Implementation

// platform_esp32s3.c
#include "xylolabs/client.h"
#include "esp_wifi.h"
#include "esp_netif.h"
#include "lwip/sockets.h"

static int s_sock = -1;

static xylolabs_err_t esp32_tcp_connect(const char *host, uint16_t port) {
    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_port   = htons(port),
    };
    inet_pton(AF_INET, host, &addr.sin_addr);

    s_sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if (s_sock < 0) return XYLOLABS_ERR_NETWORK;

    if (connect(s_sock, (struct sockaddr *)&addr, sizeof(addr)) != 0) {
        close(s_sock);
        s_sock = -1;
        return XYLOLABS_ERR_NETWORK;
    }
    return XYLOLABS_OK;
}

static xylolabs_err_t esp32_tcp_send(const uint8_t *data, size_t len) {
    ssize_t sent = send(s_sock, data, len, 0);
    return (sent == (ssize_t)len) ? XYLOLABS_OK : XYLOLABS_ERR_NETWORK;
}

static uint64_t esp32_get_time_us(void) {
    return (uint64_t)esp_timer_get_time();
}

static void esp32_sleep_ms(uint32_t ms) {
    vTaskDelay(pdMS_TO_TICKS(ms));
}

const xylolabs_platform_t g_platform = {
    .tcp_connect      = esp32_tcp_connect,
    .tcp_send         = esp32_tcp_send,
    .tcp_recv         = esp32_tcp_recv,
    .tcp_close        = esp32_tcp_close,
    .tcp_is_connected = esp32_tcp_is_connected,
    .get_time_us      = esp32_get_time_us,
    .sleep_ms         = esp32_sleep_ms,
    .watchdog_feed    = esp32_watchdog_feed,
};

### Main Application

// Rust (Embassy) - Recommended
use xylolabs_sdk::{XylolabsClient, Config, StreamDef};
use xylolabs_hal_esp::EspPlatform;

#[embassy_executor::main]
async fn main(spawner: Spawner) {
    let peripherals = esp_hal::init(esp_hal::Config::default());
    let platform = EspPlatform::new(/* wifi config */);
    let config = Config::default();
    let mut client = XylolabsClient::<_, 65536, 16384>::new(platform, config);

    client.set_api_key("xk_your_key_here");
    client.set_device_id(2);

    let streams = [
        StreamDef::new(0, "vibration_x", ValueType::F32, "g", 100.0),
        StreamDef::new(1, "temperature", ValueType::F32, "celsius", 1.0),
        StreamDef::new(2, "humidity", ValueType::F32, "percent", 1.0),
    ];
    client.session_create("esp32_s3_node", &streams).unwrap();

    loop {
        // I2S capture + XAP encode handled by HAL
        client.meta_feed_f32(0, read_accel_x().await).unwrap();
        client.meta_feed_f32(1, read_temp().await).unwrap();
        client.meta_feed_f32(2, read_humidity().await).unwrap();
        client.tick().unwrap();
    }
}

Legacy C equivalent

// main.c
#include "xylolabs/client.h"
#include "platform_esp32s3.h"
#include "driver/i2s_std.h"
#include "driver/i2c.h"

static int16_t s_i2s_buf[256 * 4];  // 256 samples x 4ch interleaved
static i2s_chan_handle_t s_i2s_rx;

void app_main(void) {
    // 1. Initialize WiFi
    wifi_init_sta("my_ssid", "my_password");

    // 2. Configure I2S for 4ch mic capture
    i2s_chan_config_t chan_cfg = I2S_CHANNEL_DEFAULT_CONFIG(I2S_NUM_0, I2S_ROLE_MASTER);
    i2s_new_channel(&chan_cfg, NULL, &s_i2s_rx);

    i2s_std_config_t std_cfg = {
        .clk_cfg  = I2S_STD_CLK_DEFAULT_CONFIG(96000),
        .slot_cfg = I2S_STD_PHILIPS_SLOT_DEFAULT_CONFIG(I2S_DATA_BIT_WIDTH_16BIT, I2S_SLOT_MODE_STEREO),
        .gpio_cfg = { .bclk = GPIO_NUM_14, .ws = GPIO_NUM_15, .dout = I2S_GPIO_UNUSED, .din = GPIO_NUM_16 },
    };
    i2s_channel_init_std_mode(s_i2s_rx, &std_cfg);
    i2s_channel_enable(s_i2s_rx);

    // 3. Initialize SDK
    xylolabs_init(&g_platform);
    xylolabs_set_api_key("xk_your_key_here");
    xylolabs_set_device_id(2);

    xylolabs_stream_def_t streams[] = {
        { 0, "vibration_x", XMBP_VT_F32, "g", 100.0f },
        { 1, "temperature",  XMBP_VT_F32, "celsius", 1.0f },
        { 2, "humidity",     XMBP_VT_F32, "percent",  1.0f },
    };
    xylolabs_session_create("esp32_s3_node", streams, 3);

    // 4. Main loop (runs on Core 1 via task pinning)
    size_t bytes_read;
    while (1) {
        i2s_channel_read(s_i2s_rx, s_i2s_buf, sizeof(s_i2s_buf), &bytes_read, portMAX_DELAY);
        xylolabs_audio_feed(s_i2s_buf, bytes_read / sizeof(int16_t));

        xylolabs_meta_feed_f32(0, read_lis3dh_x());
        xylolabs_meta_feed_f32(1, read_bme280_temp());
        xylolabs_meta_feed_f32(2, read_bme280_humidity());

        xylolabs_tick();
    }
}

--- ## Troubleshooting | Symptom | Likely cause | Fix | |---------|-------------|-----| | Audio glitches / gaps | WiFi TX interrupting I2S DMA | Pin audio task to Core 0, network to Core 1 | | Heap exhaustion on S3 | Large buffers in internal RAM | Move audio ring to PSRAM with `__attribute__((section(".ext_ram.bss")))` | | TCP connect timeout | DNS resolution on STA interface | Use IP address or call `getaddrinfo()` before `tcp_connect` | | C3 hardfault on audio | FPU instruction on RISC-V | Set `XYLOLABS_AUDIO_CHANNELS=0` (C3 is sensors-only) | | OTA fails mid-update | Power loss during flash | Enable rollback partition in `sdkconfig`; test with `esp_ota_mark_app_valid_cancel_rollback()` | | High current in sleep | WiFi modem not suspended | Call `esp_wifi_stop()` before `esp_light_sleep_start()` |