XMBP — Xylolabs Metadata Binary Protocol Specification¶

PATENT PENDING — XMBP (Xylolabs Metadata Binary Protocol) is a proprietary technology of Xylolabs Inc. Patent applications have been filed. Unauthorized use, reproduction, or distribution is prohibited.

Table of Contents¶

Overview
Batch Wire Format
Stream Block Format
Sample Layout
Value Type Registry
Audio Codec Identifiers
Encoding API
Decoding API
Bulk Stream Writers
Feature Flags
Transport
Storage Format (XMCH)
Wire Format Examples
Size Calculation Reference
Error Handling
Related Documents
Changelog
Legal

1. Overview¶

XMBP (Xylolabs Metadata Binary Protocol) is a compact binary framing protocol for IoT sensor and motor telemetry data. It is the on-wire format used between Xylolabs SDK-equipped devices and the Xylolabs ingest server.

1.1 Design Goals¶

Zero-allocation encoding. The encoder writes directly into a caller-supplied byte buffer with no heap allocation, making it suitable for bare-metal microcontrollers with no memory allocator (no_std, no alloc).
Big-endian byte order. All multi-byte integers and IEEE 754 floats are transmitted in network byte order (big-endian, most significant byte first).
Microsecond timestamps. Every sample carries a u64 timestamp measured in microseconds since an application-defined epoch.
Multi-stream batching. A single XMBP batch can carry samples from multiple independent sensor streams, each with its own type and sample count.
Typed streams. Each stream block declares a single MetadataValueType, enabling decoders to parse samples without per-sample type tags.
Compact integer types. I16 and I8 value types minimize wire overhead for low-resolution sensors (e.g., temperature, humidity, digital I/O).

1.2 Protocol Version¶

Current version: 1 (XMBP_VERSION = 0x01).

1.3 Supported Platforms¶

Platform	MCU	Encoder	Decoder	Notes
RP2350	Cortex-M33	yes	no	`no_std`, no `alloc`
STM32F103	Cortex-M3	yes	no	20 KB RAM, ADPCM only
STM32F411	Cortex-M4F	yes	no	128 KB RAM, I2S + LC3
STM32WB55	Cortex-M4F	yes	no	BLE 5.3, 4ch ADC audio
STM32WBA55	Cortex-M33	yes	no	BLE 5.4, 4ch ADC audio @ 96 kHz
ESP32-S3	Xtensa LX7	yes	no	WiFi/BLE, 4ch audio @ 96 kHz
ESP32-C3	RISC-V	yes	no	WiFi/BLE, sensor-only
nRF52840	Cortex-M4F	yes	no	BLE 5.0, 2ch audio @ 48 kHz
nRF9160	Cortex-M33	yes	no	Integrated LTE-M1 modem
Server	x86_64 / aarch64	yes	yes	`alloc` / `std` features enabled

2. Batch Wire Format¶

An XMBP batch is a contiguous big-endian byte sequence. The batch begins with a fixed-length envelope header followed by an optional device ID field and a variable-length sequence of stream blocks.

2.1 Envelope Layout¶

Offset  Size  Field
------  ----  -----
0       4     magic           0x58 0x4D 0x42 0x50  (ASCII "XMBP")
4       1     version         0x01
5       1     flags           bit field (see 2.2)
6       2     batch_seq       u16 BE — monotonically increasing sequence number
[8]     [4]   device_id       u32 BE — present only when flags & 0x01 != 0
8 or 12 2     stream_count    u16 BE — number of stream blocks that follow

The minimum batch (no streams, no device ID) is 10 bytes.

With a device ID, the minimum batch is 14 bytes.

2.2 Flags Field¶

Bit	Constant	Hex	Meaning
0	`FLAG_HAS_DEVICE_ID`	0x01	A 4-byte `device_id` field follows immediately after `batch_seq`.
1	`FLAG_ZSTD_COMPRESSED`	0x02	Payload is zstd-compressed. Reserved; not yet active.
2-7	—	—	Reserved. Must be zero on write; must be ignored on read.

2.3 Field Reference¶

Field	Type	Size	Notes
magic	u32	4	Fixed value `0x584D4250`. Reject any batch with wrong magic.
version	u8	1	Currently `0x01`. Reject unknown versions.
flags	u8	1	Bit field. See table above.
batch_seq	u16 BE	2	Wraps at 65535. Used for gap detection on the server.
device_id	u32 BE	4	Optional. Present when `FLAG_HAS_DEVICE_ID` is set.
stream_count	u16 BE	2	Number of `StreamBlock` records that follow.

2.4 Byte-Level Diagram¶

Byte:  00 01 02 03  04  05  06 07  [08 09 0A 0B]  [08|0C] [09|0D]
       +-----------+---+---+------+[------------]+[------+-------]+---...
       | 58 4D 42  | 01 |   |      |[ device_id  ]| stream_count  |
       | 50 magic  |ver |flg|b_seq |[ u32 BE opt ]| u16 BE        | stream blocks...
       +-----------+---+---+------+[------------]+[------+-------]+---...

Byte offsets in brackets [ ] are conditional on FLAG_HAS_DEVICE_ID.

3. Stream Block Format¶

Immediately following stream_count, there are exactly stream_count stream blocks in sequence. Each stream block carries samples of a single value type.

Offset  Size  Field
------  ----  -----
0       2     stream_id     u16 BE — identifies the logical stream
2       1     value_type    u8    — MetadataValueType wire tag (see Section 5)
3       2     sample_count  u16 BE — number of samples in this block
5       var   samples       sample_count samples (see Section 4)

3.1 Field Reference¶

Field	Type	Size	Notes
stream_id	u16 BE	2	Matches the stream index from the ingest session definition.
value_type	u8	1	Wire tag from `MetadataValueType`. See Section 5.
sample_count	u16 BE	2	Number of samples. Maximum 65535 per block. Server caps at 256 stream blocks per batch.
samples	—	var	Concatenated samples. Layout per value type. See Section 4.

3.2 Reserved Stream IDs¶

Stream ID	Usage
0x0064	`AUDIO_STREAM_INDEX` (100) — audio data via Bytes type
0x0000-0x000F	Metadata streams (up to 16 per session)

4. Sample Layout¶

Every sample begins with an 8-byte microsecond timestamp (u64 BE), followed immediately by the encoded value. The value encoding depends on the stream's value_type.

4.1 Fixed-Size Types¶

Type	Timestamp	Value Encoding	Total per Sample
F64	u64 BE (8)	IEEE 754 double, BE (8)	16 bytes
F32	u64 BE (8)	IEEE 754 single, BE (4)	12 bytes
I64	u64 BE (8)	signed 64-bit int, BE (8)	16 bytes
I32	u64 BE (8)	signed 32-bit int, BE (4)	12 bytes
Bool	u64 BE (8)	u8: 0=false, 1=true (1)	9 bytes
I16	u64 BE (8)	signed 16-bit int, BE (2)	10 bytes
I8	u64 BE (8)	signed 8-bit int (1)	9 bytes

4.2 Variable-Size Types¶

For variable-length types, each sample is: timestamp(8) + length_prefix + payload.

Type	Length Prefix	Payload	Notes
String	u16 BE (2)	UTF-8 encoded bytes	Max 65535 bytes
Bytes	u16 BE (2)	Raw byte blob	Max 65535 bytes
F64Array	u16 BE (2)	N x IEEE 754 double BE (8 bytes each)	Count max 65535; decoder caps at 4096
F32Array	u16 BE (2)	N x IEEE 754 single BE (4 bytes each)	Count max 65535; decoder caps at 4096
I32Array	u16 BE (2)	N x signed 32-bit int BE (4 each)	Count max 65535; decoder caps at 4096
Json	u32 BE (4)	UTF-8 encoded JSON string	Max ~4 GiB (u32); use sparingly

Note: Json uses a 4-byte (u32) length prefix, not u16. All other variable types use a 2-byte (u16) prefix.

4.3 Sample Byte Diagrams¶

F32 sample (12 bytes):

+--+--+--+--+--+--+--+--+--+--+--+--+
| timestamp_us (u64 BE, 8 bytes)     | f32 BE (4B) |
+--+--+--+--+--+--+--+--+--+--+--+--+

I16 sample (10 bytes):

+--+--+--+--+--+--+--+--+--+--+
| timestamp_us (u64 BE, 8 bytes)     | i16 BE (2B) |
+--+--+--+--+--+--+--+--+--+--+

I8 sample (9 bytes):

+--+--+--+--+--+--+--+--+--+
| timestamp_us (u64 BE, 8 bytes)     |i8|
+--+--+--+--+--+--+--+--+--+

Bool sample (9 bytes):

+--+--+--+--+--+--+--+--+--+
| timestamp_us (u64 BE, 8 bytes)     |v |
+--+--+--+--+--+--+--+--+--+
  v = 0x00 (false) or 0x01 (true)

String sample (variable):

+--+--+--+--+--+--+--+--+--+--+-- ... --+
| timestamp_us (u64 BE, 8 bytes)     |len(u16)|  UTF-8 bytes (len bytes)  |
+--+--+--+--+--+--+--+--+--+--+-- ... --+

Json sample (variable):

+--+--+--+--+--+--+--+--+--+--+--+--+-- ... --+
| timestamp_us (u64 BE, 8 bytes)     | len(u32 BE, 4B) | UTF-8 JSON (len bytes) |
+--+--+--+--+--+--+--+--+--+--+--+--+-- ... --+

5. Value Type Registry¶

The MetadataValueType discriminant is transmitted as a single byte wire tag in each stream block header.

Wire Tag	Rust Variant	Display Name	Fixed Value Size	Length Prefix	Notes
0x00	`F64`	`f64`	8 bytes	—	IEEE 754 double precision, BE
0x01	`F32`	`f32`	4 bytes	—	IEEE 754 single precision, BE
0x02	`I64`	`i64`	8 bytes	—	Signed 64-bit integer, BE
0x03	`I32`	`i32`	4 bytes	—	Signed 32-bit integer, BE
0x04	`Bool`	`bool`	1 byte	—	0x00=false, 0x01=true
0x05	`String`	`string`	variable	u16 BE	UTF-8; max 65535 bytes
0x06	`Bytes`	`bytes`	variable	u16 BE	Raw blob; max 65535 bytes
0x07	`F64Array`	`f64_array`	variable	u16 BE	N x f64 BE
0x08	`F32Array`	`f32_array`	variable	u16 BE	N x f32 BE
0x09	`I32Array`	`i32_array`	variable	u16 BE	N x i32 BE
0x0A	`Json`	`json`	variable	u32 BE	UTF-8 JSON; length prefix is u32
0x0B	`I16`	`i16`	2 bytes	—	Signed 16-bit integer, BE
0x0C	`I8`	`i8`	1 byte	—	Signed 8-bit integer

Wire tags 0x0D through 0xFF are reserved. A decoder receiving an unknown tag must return ProtocolError::InvalidValueType.

5.1 Choosing a Value Type¶

Use Case	Recommended Type	Bytes/Sample	Notes
High-precision sensor (float)	F64	16	Temperature with sub-millidegree precision
Standard sensor (float)	F32	12	Temperature, humidity, pressure
Low-resolution sensor	I16	10	ADC counts, motor RPM, small integers
Digital I/O, status codes	I8	9	On/off, error codes, GPIO state
Boolean flags	Bool	9	Same wire size as I8, semantic clarity
Multi-axis accelerometer	F32Array	variable	3-axis or 6-axis IMU data per sample
Audio payload	Bytes	variable	Codec-wrapped PCM/ADPCM/XAP/Opus frames
Device config/events	Json	variable	Structured event payloads; use sparingly

6. Audio Codec Identifiers¶

When an XMBP Bytes stream carries encoded audio, the first byte of each sample's payload is a codec identifier. The remaining bytes are the codec-specific frame data.

Codec ID	Constant	Name	Description
0x01	`CODEC_ID_RAW_PCM`	Raw PCM	Uncompressed PCM audio samples
0x02	`CODEC_ID_ADPCM`	IMA-ADPCM	IMA Adaptive Differential PCM, 4:1 compression
0x03	`CODEC_ID_XAP`	XAP	Xylolabs Audio Protocol
0x04	`CODEC_ID_OPUS`	Opus	Opus codec (reserved for future use)

The XAP file extension is .xap (XAP_FILE_EXTENSION).

For detailed XAP framing, codec parameters, and performance characteristics, see XAP-SPECIFICATION.md and CODEC-ANALYSIS.md.

7. Encoding API¶

7.1 XmbpWriter¶

XmbpWriter is a zero-allocation, no_std-compatible encoder. It writes big-endian encoded data directly into a caller-provided &mut [u8] buffer. If at any point the buffer cannot accommodate the requested write, the writer enters an overflow state and all subsequent writes become silent no-ops.

pub struct XmbpWriter<'a> {
    buf: &'a mut [u8],
    pos: usize,
    overflow: bool,
}

Key methods:

Method	Description
`XmbpWriter::new(buf: &mut [u8]) -> Self`	Create writer over the given buffer.
`reset(&mut self)`	Reset position to zero and clear overflow flag.
`len(&self) -> usize`	Bytes written so far.
`is_empty(&self) -> bool`	Returns `true` if zero bytes written.
`is_ok(&self) -> bool`	Returns `true` when no overflow has occurred.
`write_batch_header(version: u8, flags: u8, batch_seq: u16)`	Write magic + version + flags + batch_seq (8 bytes).
`write_device_id(device_id: u32)`	Write optional device ID field (4 bytes).
`write_stream_count(count: u16)`	Write stream count (2 bytes).
`begin_stream(stream_id: u16, value_type: MetadataValueType, sample_count: u16)`	Write stream block header (5 bytes).

Sample writers (fixed-size):

Method	Sample Size	Notes
`write_sample_f64(timestamp_us: u64, value: f64)`	16 bytes
`write_sample_f32(timestamp_us: u64, value: f32)`	12 bytes
`write_sample_i64(timestamp_us: u64, value: i64)`	16 bytes
`write_sample_i32(timestamp_us: u64, value: i32)`	12 bytes
`write_sample_bool(timestamp_us: u64, value: bool)`	9 bytes
`write_sample_i16(timestamp_us: u64, value: i16)`	10 bytes	Added in v1.1
`write_sample_i8(timestamp_us: u64, value: i8)`	9 bytes	Added in v1.1

Sample writers (variable-size):

Method	Notes
`write_sample_string(timestamp_us: u64, s: &str)`	u16 length prefix
`write_sample_bytes(timestamp_us: u64, data: &[u8])`	u16 length prefix
`write_sample_f32_array(timestamp_us: u64, values: &[f32])`	u16 count prefix
`write_sample_f64_array(timestamp_us: u64, values: &[f64])`	u16 count prefix
`write_sample_i32_array(timestamp_us: u64, values: &[i32])`	u16 count prefix
`write_sample_json(timestamp_us: u64, json_str: &str)`	u32 length prefix

7.2 Encoding Example¶

use xylolabs_protocol::{XmbpWriter, MetadataValueType, constants};

let mut buf = [0u8; 256];
let mut w = XmbpWriter::new(&mut buf);

// Write batch header: version=1, no flags, sequence=0
w.write_batch_header(constants::XMBP_VERSION, 0, 0);
// Write stream count
w.write_stream_count(1);
// Begin stream 0: F32 type, 1 sample
w.begin_stream(0, MetadataValueType::F32, 1);
// Write sample: timestamp 1000 us, value 23.5
w.write_sample_f32(1000, 23.5);

assert!(w.is_ok());
let len = w.len();
drop(w);
let encoded = &buf[..len]; // 22 bytes total

7.3 Convenience Builder (Uniform F32 Streams)¶

use xylolabs_protocol::{XmbpWriter, constants};

let mut buf = [0u8; 512];
let mut w = XmbpWriter::new(&mut buf);

let timestamps = [1000u64, 2000, 3000];
let values = [1.0f32, 2.0, 3.0,  // stream 0
               4.0, 5.0, 6.0];   // stream 1

let size = w.build_f32_batch(
    0,      // batch_seq
    0xABCD, // device_id (pass 0 to omit)
    2,      // num_streams
    3,      // samples_per_stream
    &timestamps,
    &values,
    3,      // values_stride
);
assert!(size > 0);

values is laid out as values[stream * values_stride + sample]. Passing device_id = 0 omits the device ID field and clears the FLAG_HAS_DEVICE_ID flag.

8. Decoding API¶

8.1 XmbpReader¶

XmbpReader is a stateless decoder. It decodes a complete XMBP batch from a raw &[u8] slice into structured heap-allocated types. The decoder requires the alloc feature.

pub struct XmbpReader;

impl XmbpReader {
    pub fn decode(data: &[u8]) -> Result<XmbpBatch, ProtocolError>;
}

Decoded types:

Type	Fields
`XmbpBatch`	`version: u8`, `flags: u8`, `batch_seq: u16`, `device_id: Option<u32>`, `stream_blocks: Vec<StreamBlock>`
`StreamBlock`	`stream_id: u16`, `value_type: MetadataValueType`, `samples: Vec<Sample>`
`Sample`	`timestamp_us: u64`, `value: SampleValue`
`SampleValue`	Enum: `F64(f64)`, `F32(f32)`, `I64(i64)`, `I32(i32)`, `Bool(bool)`, `String(String)`, `Bytes(Vec<u8>)`, `F64Array(Vec<f64>)`, `F32Array(Vec<f32>)`, `I32Array(Vec<i32>)`, `Json(String)`, `I16(i16)`, `I8(i8)`

Decoder safety limits:

stream_count is capped at 256 to bound allocations from untrusted data.
Array element counts (F64Array, F32Array, I32Array) are capped at 4096 per sample.

8.2 Decoding Example¶

use xylolabs_protocol::{XmbpReader, SampleValue};

let batch = XmbpReader::decode(&encoded_bytes)?;

println!("batch seq={}, device={:?}", batch.batch_seq, batch.device_id);
for block in &batch.stream_blocks {
    println!("stream {} ({:?}): {} samples",
             block.stream_id, block.value_type, block.samples.len());
    for sample in &block.samples {
        match &sample.value {
            SampleValue::F32(v) => println!("  t={} us  val={}", sample.timestamp_us, v),
            SampleValue::I16(v) => println!("  t={} us  val={}", sample.timestamp_us, v),
            SampleValue::I8(v)  => println!("  t={} us  val={}", sample.timestamp_us, v),
            _ => {}
        }
    }
}

9. Bulk Stream Writers¶

For performance-critical paths, XmbpWriter provides bulk stream writers that perform a single overflow check for the entire stream block, avoiding per-sample bounds checking.

Method	Value Type	Sample Size	Notes
`write_stream_f64_bulk(stream_id, timestamps, values)`	F64	16
`write_stream_f32_bulk(stream_id, timestamps, values)`	F32	12
`write_stream_i64_bulk(stream_id, timestamps, values)`	I64	16
`write_stream_i32_bulk(stream_id, timestamps, values)`	I32	12
`write_stream_bool_bulk(stream_id, timestamps, values)`	Bool	9
`write_stream_i16_bulk(stream_id, timestamps, values)`	I16	10	Added in v1.1
`write_stream_i8_bulk(stream_id, timestamps, values)`	I8	9	Added in v1.1

Each method writes a complete stream block (header + all samples) in a single call. The timestamps and values slices must have equal length; the effective count is min(timestamps.len(), values.len(), u16::MAX).

Example (SDK client usage):

// The SDK client uses bulk writers for metadata batches:
writer.write_stream_f32_bulk(
    stream_id,
    &timestamps[..count],
    &values[..count],
);

// For I16 streams (e.g., ADC counts):
writer.write_stream_i16_bulk(
    stream_id,
    &timestamps[..count],
    &i16_values[..count],
);

10. Feature Flags¶

The xylolabs-protocol crate uses Cargo feature flags to control code size and dependencies.

Feature	Default	Effect
`alloc`	No	Enables the `decode` module: `XmbpReader`, `XmbpBatch`, `StreamBlock`, `Sample`, `SampleValue`
`std`	No	Implies `alloc`. Provides `std::error::Error` impl for `ProtocolError`.
`serde`	No	Derives `Serialize` / `Deserialize` on `MetadataValueType`.

With no features enabled, only XmbpWriter, MetadataValueType, ProtocolError, and constants are compiled. This is the configuration used for bare-metal firmware where heap allocation is unavailable.

11. Transport¶

11.1 API Key Validation (Ping)¶

Before starting an ingest session, firmware can validate its API key with a lightweight ping:

GET /api/v1/ping
Host: api.xylolabs.com
X-Api-Key: <device-api-key>

Returns 200 OK with {"pong": true, "api_key_name": "...", "facility_id": "..."} if the key is valid, or 401 if not. Use this to fail fast on misconfigured keys before allocating session resources.

11.2 Ingest Endpoint¶

XMBP batches are delivered via HTTP/1.1 POST requests to the Xylolabs ingest API:

POST /api/v1/ingest/sessions/{session_id}/data
Host: api.xylolabs.com
Content-Type: application/octet-stream
X-API-Key: <device-api-key>
Content-Length: <byte count>

<raw XMBP batch bytes>

session_id is a UUID identifying the active ingest session, obtained from the session creation endpoint.
The X-API-Key header carries a device-scoped API key for authentication.
The request body is the raw XMBP batch with no additional framing or encoding.

11.3 Batch Sequencing¶

The batch_seq field (u16, wraps at 65535) is a monotonically increasing counter maintained by the sender. The server uses it for gap detection: if consecutive received batches have non-sequential batch_seq values, the server records a gap event. The sender increments batch_seq by 1 for each transmitted batch.

11.4 Device ID¶

When FLAG_HAS_DEVICE_ID is set, the device_id field (u32) identifies the originating hardware unit within a multi-device deployment. The server associates the batch with the matching registered device record.

11.5 TLS Requirement¶

All production transport must use TLS. The SDK modem drivers establish TLS connections via: - AT+QSSLOPEN (Quectel BG770A on RP2350/STM32) - AT#XSOCKET with TLS flag (nRF9160 integrated modem) - Native TLS stack (ESP32 WiFi)

12. Storage Format (XMCH)¶

After ingest, the server persists samples in XMCH (Xylolabs Metadata Chunk) format. XMCH uses a column-oriented layout separating timestamps from values to improve compression.

12.1 XMCH Header (32 bytes, fixed)¶

Offset  Size  Field          Value / Notes
------  ----  -----          ------
0       4     magic          0x58 0x4D 0x43 0x48  (ASCII "XMCH")
4       1     version        0x01
5       1     value_type     MetadataValueType wire tag (same as XMBP)
6       2     reserved       0x0000
8       4     sample_count   u32 BE — number of samples in this chunk
12      8     start_ts_us    u64 BE — timestamp of first sample
20      8     end_ts_us      u64 BE — timestamp of last sample
28      4     data_bytes     u32 BE — byte count of the data section

Total header size: 32 bytes.

12.2 Data Section Layout¶

Immediately following the 32-byte header:

[timestamp column] [value column]

Timestamp column: sample_count x u64 BE — all timestamps in ascending order.
Value column: sample_count values encoded identically to the XMBP sample value encoding (same type-specific rules, same byte order), but without per-sample timestamp prefixes.

For variable-length types (String, Bytes, F64Array, F32Array, I32Array), each value in the value column uses a u32 BE length/count prefix (not u16 as in XMBP wire format).

For Json, the value column uses a u32 BE length prefix, identical to the XMBP wire format.

12.3 Compression¶

XMCH chunks may be stored compressed using zstd. Decompression produces the raw 32-byte header + data section layout described above. The server enforces a 64 MiB decompressed output limit to prevent decompression bomb attacks.

12.4 Empty Chunks¶

A chunk with sample_count = 0 contains only the 32-byte header with data_bytes = 0. The timestamp and data sections are absent.

13. Wire Format Examples¶

13.1 Minimal Batch (10 bytes)¶

A batch with zero streams, no device ID, and sequence number 42:

Offset  Hex   Description
------  ---   -----------
0       58    magic[0] 'X'
1       4D    magic[1] 'M'
2       42    magic[2] 'B'
3       50    magic[3] 'P'
4       01    version = 1
5       00    flags = 0 (no device ID)
6       00    batch_seq high byte
7       2A    batch_seq low byte  (= 42)
8       00    stream_count high byte
9       00    stream_count low byte (= 0)

Total: 10 bytes.

13.2 Single F32 Stream (22 bytes)¶

One stream, one F32 sample (timestamp=1000 us, value=23.5):

Offset  Hex            Description
------  ---            -----------
0-3     58 4D 42 50    magic "XMBP"
4       01             version
5       00             flags (no device ID)
6-7     00 00          batch_seq = 0
8-9     00 01          stream_count = 1
--- stream block 0 ---
10-11   00 00          stream_id = 0
12      01             value_type = F32 (tag 0x01)
13-14   00 01          sample_count = 1
--- sample 0 ---
15-22   00 00 00 00    timestamp_us = 1000 us (u64 BE)
        00 00 03 E8
23-26   41 BC 00 00    f32 BE = 23.5  (IEEE 754: 0x41BC0000)

Total: 27 bytes (header 10 + stream header 5 + sample 12).

13.3 Single I16 Stream (25 bytes)¶

One stream, one I16 sample (timestamp=500 us, value=1023):

Offset  Hex            Description
------  ---            -----------
0-3     58 4D 42 50    magic "XMBP"
4       01             version
5       00             flags
6-7     00 01          batch_seq = 1
8-9     00 01          stream_count = 1
--- stream block 0 ---
10-11   00 00          stream_id = 0
12      0B             value_type = I16 (tag 0x0B)
13-14   00 01          sample_count = 1
--- sample 0 ---
15-22   00 00 00 00    timestamp_us = 500 us (u64 BE)
        00 00 01 F4
23-24   03 FF          i16 BE = 1023

Total: 25 bytes (header 10 + stream header 5 + sample 10).

13.4 Multi-Stream Batch with Device ID¶

Two F32 streams (2 samples each), device_id = 0x0000ABCD:

Offset  Hex            Description
------  ---            -----------
0-3     58 4D 42 50    magic "XMBP"
4       01             version
5       01             flags = 0x01 (FLAG_HAS_DEVICE_ID)
6-7     00 05          batch_seq = 5
8-11    00 00 AB CD    device_id = 0x0000ABCD
12-13   00 02          stream_count = 2
--- stream block 0 (stream_id=0, F32, 2 samples) ---
14-15   00 00          stream_id = 0
16      01             value_type = F32
17-18   00 02          sample_count = 2
19-30   [sample 0: 8-byte ts + 4-byte f32]
31-42   [sample 1: 8-byte ts + 4-byte f32]
--- stream block 1 (stream_id=1, F32, 2 samples) ---
43-44   00 01          stream_id = 1
45      01             value_type = F32
46-47   00 02          sample_count = 2
48-59   [sample 0: 8-byte ts + 4-byte f32]
60-71   [sample 1: 8-byte ts + 4-byte f32]

Total: 14 (header + device_id) + 2 x (5 + 2 x 12) = 14 + 58 = 72 bytes.

13.5 Mixed-Type Batch (3 streams)¶

Three streams of different types, no device ID:

header (10 bytes):
  magic + ver=1 + flags=0 + batch_seq=0 + stream_count=3

stream 0: F64, 1 sample   -> header(5) + sample(16) = 21
stream 1: I32, 1 sample   -> header(5) + sample(12) = 17
stream 2: Bool, 1 sample  -> header(5) + sample(9)  = 14

Total: 10 + 21 + 17 + 14 = 62 bytes

14. Size Calculation Reference¶

14.1 Constants¶

Constant	Value	Description
`BATCH_HEADER_BASE`	8	magic(4) + version(1) + flags(1) + batch_seq(2)
`DEVICE_ID_SIZE`	4	Optional device_id field
`STREAM_COUNT_SIZE`	2	stream_count field
`STREAM_HEADER_SIZE`	5	stream_id(2) + value_type(1) + sample_count(2)
`SAMPLE_F64_SIZE`	16	timestamp(8) + f64(8)
`SAMPLE_F32_SIZE`	12	timestamp(8) + f32(4)
`SAMPLE_I64_SIZE`	16	timestamp(8) + i64(8)
`SAMPLE_I32_SIZE`	12	timestamp(8) + i32(4)
`SAMPLE_BOOL_SIZE`	9	timestamp(8) + bool(1)
`SAMPLE_I16_SIZE`	10	timestamp(8) + i16(2)
`SAMPLE_I8_SIZE`	9	timestamp(8) + i8(1)

14.2 Compile-Time `batch_size` Helper¶

use xylolabs_protocol::constants;

// Buffer for 4 streams x 100 F32 samples, with device ID
let required = constants::batch_size(4, 100, constants::SAMPLE_F32_SIZE, true);
// = 8 + 4 + 2 + 4 * (5 + 100 * 12) = 4834 bytes

// Buffer for 8 streams x 32 I16 samples, no device ID
let required = constants::batch_size(8, 32, constants::SAMPLE_I16_SIZE, false);
// = 8 + 2 + 8 * (5 + 32 * 10) = 2610 bytes

Formula:

batch_size = BATCH_HEADER_BASE
           + (has_device_id ? DEVICE_ID_SIZE : 0)
           + STREAM_COUNT_SIZE
           + num_streams * (STREAM_HEADER_SIZE + samples_per_stream * sample_size)

15. Error Handling¶

15.1 Encoder Errors¶

The encoder uses an overflow flag instead of returning Result. After any write overflows the buffer, is_ok() returns false and all subsequent writes are silent no-ops. Always check is_ok() after completing a batch.

15.2 Decoder Errors (`ProtocolError`)¶

Variant	Meaning
`BufferOverflow`	Allocation limit exceeded (stream/array count cap)
`InvalidMagic`	Magic bytes do not match `0x584D4250`
`UnsupportedVersion`	Version field is not `0x01`
`InvalidValueType(u8)`	Unknown value type wire tag
`UnexpectedEof`	Input ended before all expected bytes were read
`InvalidUtf8`	String or JSON field contains invalid UTF-8

Document	Description
XAP-SPECIFICATION.md	XAP (Xylolabs Audio Protocol) framing, codec parameters, and container format
CODEC-ANALYSIS.md	Comparative analysis of audio codecs supported by XMBP
API.en.md	Xylolabs REST API reference (ingest, sessions, devices)
PERFORMANCE-EVALUATION.md	Protocol throughput and latency benchmarks
SDK-GUIDE.md	Xylolabs SDK integration guide

Source code:

Path	Description
`crates/xylolabs-protocol/src/lib.rs`	Crate root, re-exports
`crates/xylolabs-protocol/src/constants.rs`	Protocol constants and size helpers
`crates/xylolabs-protocol/src/value_type.rs`	`MetadataValueType` enum
`crates/xylolabs-protocol/src/encode.rs`	`XmbpWriter` zero-alloc encoder
`crates/xylolabs-protocol/src/decode.rs`	`XmbpReader` decoder (requires alloc)
`crates/xylolabs-protocol/src/error.rs`	`ProtocolError` type

17. Changelog¶

Date	Version	Changes
2026-04-05	1.1	Added I16 (0x0B) and I8 (0x0C) value types. Added bulk stream writers (`write_stream_i16_bulk`, `write_stream_i8_bulk`). Added `SAMPLE_I16_SIZE` and `SAMPLE_I8_SIZE` constants. Added Section 9 (Bulk Stream Writers), Section 14 (Size Calculation Reference), Section 15 (Error Handling), Section 17 (Changelog). Added platform support table. Moved to `docs/protocol/`.
2026-03-31	1.0	Initial specification. 11 value types (F64 through Json). XmbpWriter, XmbpReader, XMCH storage format, transport protocol.

18. Legal¶

XMBP (Xylolabs Metadata Binary Protocol) is proprietary technology developed by Xylolabs Inc.

Unauthorized use, reproduction, modification, or distribution of this specification or any implementation thereof — in whole or in part — is strictly prohibited without prior written authorization from Xylolabs Inc.