Data model

Pre-launch / design spec

The tensor shapes and label schema below are the proposed contract for v0.1. They finalize with the dataset loader implementation.

Overview

Every EMMA (Electromagnetic Multi-task Model Assessment) task reads the same canonical record: raw, complex, multi-antenna I/Q (in-phase and quadrature) plus per-scene metadata, with per-task labels extracted from that metadata. The record is representation-agnostic at the input layer. A task that needs a spectrogram or a CSI (channel state information) tensor derives it from the raw I/Q inside its readout, never from disk. This keeps the physically meaningful signal, inter-antenna phase coherence, available to every task.

Place In The System

The data model is the contract between EMMADataset (which loads scenes) and the FrozenBackbone plus ReadoutHead (which consume them). rfgen produces the on-disk scene; LabelExtractor turns rfgen annotations into task labels; the dataset yields tensors the backbone can ingest. See Datasets and rfgen for how scenes are sourced.

Boundaries

  • Owned by EMMA: the loader (EMMADataset), the label extraction (LabelExtractor), and the array specification (ArraySpec).

  • Owned by rfgen: the I/Q samples themselves, the emitter definitions, the channel model, and the on-disk annotation format (SigMF, Signal Metadata Format).

Data Flow

A single record flows through four shapes:

rfgen scene (SigMF on disk)
   │
   ▼
EMMADataset ──► raw I/Q tensor + scene metadata
   │
   ▼
LabelExtractor ──► per-task label(s)
   │
   ▼
(FrozenBackbone ──► ReadoutHead) consumes (I/Q tensor, label)

The Canonical Record

Primary input: raw complex multi-antenna I/Q

The canonical input tensor has shape (num_rx, 2, N) as float32, where:

  • num_rx is the number of receiver antenna elements.

  • 2 is the in-phase (channel 0) and quadrature (channel 1) components, stored as real floating-point parts for autograd compatibility rather than as a native complex dtype.

  • N is the number of time samples.

Storing the complex baseband signal as two real floats (rather than a native complex dtype) keeps the tensor differentiable through standard libraries (torch, torchmetrics). This matches the convention in the glossary entry for I/Q.

Per-scene metadata

Each I/Q tensor is paired with a scene manifest carrying:

  • Environment: the channel environment identifier (for example E-URBAN, E-RURAL), which the OOD (out-of-distribution) protocol splits on.

  • Array: the receiver array geometry via ArraySpec, naming the array type (ULA, uniform linear array, or URA, uniform rectangular array), element count, and spacing.

  • Emitters: the list of emitters in the scene, their waveforms, and their directions.

  • SNR: the signal-to-noise ratio in decibels, an OOD axis for several tasks.

Per-task labels

LabelExtractor reads the scene metadata and emits the label a specific task trains on. One scene yields many labels: the same capture produces an angle-of-arrival (AoA) regression target for E-LOC-AOA, a line-of-sight flag for E-LOC-LOS, and a device identity for E-ID-FP. The label is a function of (scene, task), computed at load time, not stored per task on disk.

Why Derived Views Are Secondary

A spectrogram, a constellation diagram, and a CSI matrix are all functions of the raw I/Q. They are also lossy projections of it:

  • A magnitude spectrogram discards phase. Inter-antenna phase differences are exactly what direction-finding reads, so once the spectrogram is taken, AoA and beam prediction become physically unevaluable from the input.

  • A CSI tensor collapses the time-domain waveform into a frequency-domain channel description. The CSI camp (LWM, DeepMIMO) operates here natively; EMMA offers CSI only as a bridge task (E-CH-CSI), never as the primary input.

EMMA therefore stores and loads the raw multi-antenna I/Q, and lets a readout head derive whatever view it needs. A model that wants a spectrogram can compute one inside the head; a model that wants the raw phase can read it directly. The canonical record never makes that choice for the model.

Minimal Example

from emma.datasets import EMMADataset, LabelExtractor
from emma.enums import TaskID

dataset = EMMADataset.from_recipe("EMMA-LOC-v0.1", split="dev")
extractor = LabelExtractor.for_task(TaskID.E_LOC_AOA)

for scene in dataset:
    iq = scene.iq                 # torch.float32, shape (num_rx, 2, N)
    meta = scene.metadata         # environment, array, emitters, snr_db
    label = extractor(scene)      # AoA target in degrees for E-LOC-AOA

Design Notes

  • Phase coherence is the moat. The (num_rx, 2, N) shape preserves the fixed phase relationship between antennas. Any input representation that destroys it removes the one property no incumbent RF dataset natively provides.

  • One scene, many labels. Decoupling scenes from task labels is what lets one backbone be scored across many tasks without re-generating data. See Tasks.

  • SigMF on disk, tensors in memory. The interchange format with rfgen is SigMF; the in-memory contract is a float32 tensor. Neither requires the other to change.

References

  • Yang et al., “SUPERB: Speech processing Universal PERformance Benchmark,” Interspeech 2021, arXiv:2105.01051. The one-shared-input, many-readouts contract this data model serves.

  • Schmidt, “Multiple emitter location and signal parameter estimation,” IEEE Trans. Acoustics, Speech, Signal Processing 1986, DOI:10.1109/TASSP.1986.1164830. The MUSIC algorithm; grounds that AoA is physically a continuous angular quantity read from inter-antenna phase.

See Also