Datasets¶

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, schema fields, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

Dataset recipes, the core dataset abstraction, label extraction, the data release resolver, and the real-capture adapters. EMMA owns dataset recipes (pinned rfgen configurations), loaders, and label extraction. It does not own emitters, channels, propagation, antenna patterns, or any signal generation; those live in rfgen. See Datasets and rfgen for the boundary.

Three roles¶

Role	Examples
Data types	RfgenRecipe, ArraySpec
ABC	RealCaptureAdapter
Concrete classes	EMMADataset, LabelExtractor, DataRelease, ColosseumAdapter, PowderAdapter, CosmosAdapter

Class index¶

Class	Role	Purpose
RfgenRecipe	Data type	Frozen rfgen pin: commit, resolved config, seed ranges
ArraySpec	Data type	Receiver array geometry reference
EMMADataset	Concrete	Loads scenes as canonical multi-antenna I/Q plus labels
LabelExtractor	Concrete	Maps rfgen metadata to per-task labels
DataRelease	Concrete	Resolves a manifest to a set of scenes
RealCaptureAdapter	ABC	Normalizes a public testbed into the EMMA scene format
ColosseumAdapter	Concrete	Northeastern Colosseum testbed adapter
PowderAdapter	Concrete	POWDER (Platform for Open Wireless Data-driven Experimental Research) testbed adapter
CosmosAdapter	Concrete	COSMOS (Cloud-enhanced Open Software Defined Mobile Wireless Testbed) adapter

`class emma.datasets.RfgenRecipe`¶

The frozen rfgen pin carried by every DatasetRecipe. Pins the rfgen commit, the resolved configuration path, and the seed ranges. EMMA regenerates scenes through rfgen using this pin; it never generates I/Q (in-phase and quadrature) itself.

@dataclass(frozen=True, slots=True)
class RfgenRecipe:
    commit: str
    resolved_config: str
    seed_ranges: dict[Split, tuple[int, int]]

Fields¶

Field	Type	Purpose
`commit`	`str`	Frozen rfgen git SHA
`resolved_config`	`str`	Path to the fully composed rfgen `GenerationConfig`
`seed_ranges`	`dict[{ref}`Split `, tuple[int, int]]`	Inclusive seed range per split

Invariants¶

The triple (commit, resolved_config, seed) fully determines a scene bit-for-bit. Changing any field changes the scenes and requires a new content hash.
seed_ranges for Split.HOLDOUT is a frozen secret held by the external co-steward. It is never serialized into a public recipe.

`class emma.datasets.ArraySpec`¶

Typed reference to a rfgen receiver array configuration. Resolved from the arrays field on a SceneManifest at load time.

@dataclass(frozen=True, slots=True)
class ArraySpec:
    geometry: str
    num_elements: int
    spacing: float

Fields¶

Field	Type	Purpose
`geometry`	`str`	rfgen `ArrayGeometry` enum value, for example `ULA` (uniform linear array) or `URA` (uniform rectangular array)
`num_elements`	`int`	Number of antenna elements
`spacing`	`float`	Inter-element spacing in wavelengths

Notes¶

geometry is a rfgen ArrayGeometry enum member, not an EMMA enum. EMMA reads it at load time; it does not redefine array geometry types.

`class emma.datasets.LabelExtractor`¶

Maps rfgen scene metadata to per-task labels. Reads the rfgen metadata paths declared in a DatasetRecipe label_extraction map and produces the label tensor a ReadoutHead trains against.

class LabelExtractor:
    def __init__(self, recipe: DatasetRecipe) -> None: ...

    def extract(
        self, scene: SceneManifest, task_id: TaskID
    ) -> torch.Tensor | dict[str, torch.Tensor]: ...

`extract(scene, task_id)`¶

Extracts the label for one task from one scene’s manifest.

Parameter	Type	Purpose
`scene`	SceneManifest	The per-scene provenance record carrying realized emitter metadata
`task_id`	TaskID	The task whose label is requested

Returns. A torch.Tensor for a single-label task (regression angle, binary class), or a dict[str, torch.Tensor] for a multi-label task. The exact shape and dtype are pinned by the task’s TaskSpec.

Notes¶

The extractor reads rfgen metadata paths such as emitters[].aoa_deg; it does not recompute physical quantities. One recipe yields many labels off the same scene set.

`class emma.datasets.EMMADataset(torch.utils.data.Dataset)`¶

The core dataset abstraction. Loads raw multi-antenna I/Q from scenes resolved by a DataRelease, pairs each scene with per-task labels via LabelExtractor, and yields the canonical record a FrozenBackbone consumes.

class EMMADataset(torch.utils.data.Dataset):
    def __init__(
        self,
        release: DataRelease,
        split: Split,
        task_id: TaskID,
        label_extractor: LabelExtractor | None = None,
    ) -> None: ...

    def __len__(self) -> int: ...

    def __getitem__(self, index: int) -> tuple[torch.Tensor, torch.Tensor | dict[str, torch.Tensor]]: ...

Constructor parameters¶

Parameter	Type	Default	Purpose
`release`	DataRelease	required	The resolved data release providing scenes
`split`	Split	required	Which split to expose
`task_id`	TaskID	required	Which task’s labels to produce
`label_extractor`	LabelExtractor \| None	`None`	Override extractor; if `None`, built from the release recipe

Methods¶

`len()`¶

Returns the number of scenes in the requested split.

`getitem(index)`¶

Parameter	Type	Purpose
`index`	`int`	Scene index within the split

Returns. A tuple (iq, label) where iq is a torch.float32 tensor of shape (num_rx, 2, N) (the multi-antenna I/Q layout) and label is the per-task label produced by LabelExtractor.

Invariants¶

iq dtype is torch.float32, shape (num_rx, 2, N): channel 0 is in-phase, channel 1 is quadrature. Matches the rfgen multi-RX layout.
Requesting Split.HOLDOUT outside the scoring path raises SplitNotAvailable.

`class emma.datasets.DataRelease`¶

Resolves a DataReleaseManifest to a set of scenes. Materializes the scenes from the pinned rfgen recipe (for synthetic splits) or via the real-capture adapters (for real-capture subsets), verifies content hashes, and partitions scenes by split.

class DataRelease:
    def __init__(
        self,
        manifest: DataReleaseManifest,
        cache_dir: str | None = None,
    ) -> None: ...

    def scenes(self, split: Split) -> list[SceneManifest]: ...

    def recipe(self, dataset_id: str) -> DatasetRecipe: ...

    @property
    def content_hash(self) -> str: ...

Constructor parameters¶

Parameter	Type	Default	Purpose
`manifest`	DataReleaseManifest	required	The release manifest to resolve
`cache_dir`	`str \| None`	`None`	Local cache directory for regenerated scenes

Methods¶

`scenes(split)`¶

Returns the scene manifests assigned to the requested split. Raises SplitNotAvailable for Split.HOLDOUT outside the scoring path.

`recipe(dataset_id)`¶

Returns the DatasetRecipe for a dataset_id bundled in this release.

Invariants¶

Content hashes are verified on load. A scene whose hash does not match the manifest raises RecipeMismatchError.
Synthetic scenes are regenerated through rfgen from the pinned recipe; EMMA never generates I/Q itself.

ContentHash utility¶

The module exposes static content-hash helpers used by manifests and the release resolver:

class ContentHash:
    @staticmethod
    def scene_hash(commit: str, config: str, seed: int) -> str: ...

    @staticmethod
    def release_hash(scene_hashes: list[str]) -> str: ...

The digest construction is documented in Content hashing.

Real-capture adapters¶

Real-capture OOD (out-of-distribution) subsets are ingested through adapter implementations behind the RealCaptureAdapter ABC. Each adapter normalizes a public RF (radio-frequency) testbed into the EMMA scene format so the same frozen backbone and the same per-task metrics score both synthetic and real scenes. Real-capture data feeds the sim-to-real gap column only; it never enters the aggregate.

`class emma.datasets.RealCaptureAdapter(ABC)`¶

The extension contract for real-capture data sources. Subclass to add a testbed. One ABC per concept; no concrete-to-concrete inheritance.

class RealCaptureAdapter(ABC):
    @property
    @abstractmethod
    def name(self) -> str: ...

    @abstractmethod
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...

    @abstractmethod
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Abstract methods¶

Method	Returns	Purpose
`name`	`str`	Adapter identifier, for example `colosseum`
`list_scenes(split)`	`list[{ref}`SceneManifest `]`	Scenes available for the split
`load_iq(scene_id)`	`torch.Tensor`	Multi-antenna I/Q tensor for the scene

Notes¶

Adapter names are open plugin names (plain str), not closed enum members, because anyone can ship a new adapter.
The adapter populates the SceneManifest generator field with its name, distinguishing real captures from rfgen scenes.

`class emma.datasets.ColosseumAdapter(RealCaptureAdapter)`¶

Adapter for the Northeastern Colosseum testbed. Normalizes Colosseum capture records into EMMA scene manifests and multi-antenna I/Q tensors.

class ColosseumAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
`root`	`str`	required	Path to the local Colosseum dataset mirror
`band`	`str \| None`	`None`	Optional frequency-band filter

Notes¶

Colosseum is the Northeastern University RF emulation testbed; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.
Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

`class emma.datasets.PowderAdapter(RealCaptureAdapter)`¶

Adapter for the POWDER (Platform for Open Wireless Data-driven Experimental Research) testbed. Normalizes POWDER capture records into EMMA scene manifests.

class PowderAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
`root`	`str`	required	Path to the local POWDER dataset mirror
`band`	`str \| None`	`None`	Optional frequency-band filter

Notes¶

POWDER is the NSF PAWR city-scale testbed at the University of Utah; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.
Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

`class emma.datasets.CosmosAdapter(RealCaptureAdapter)`¶

Adapter for the COSMOS (Cloud-enhanced Open Software Defined Mobile Wireless Testbed) testbed. Normalizes COSMOS capture records into EMMA scene manifests.

class CosmosAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
`root`	`str`	required	Path to the local COSMOS dataset mirror
`band`	`str \| None`	`None`	Optional frequency-band filter

Notes¶

COSMOS is the NSF PAWR city-scale testbed led by Rutgers in West Harlem; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.
Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

Datasets¶

Three roles¶

Class index¶

class emma.datasets.RfgenRecipe¶

Fields¶

Invariants¶

class emma.datasets.ArraySpec¶

Fields¶

Notes¶

class emma.datasets.LabelExtractor¶

extract(scene, task_id)¶

Notes¶

class emma.datasets.EMMADataset(torch.utils.data.Dataset)¶

Constructor parameters¶

Methods¶

__len__()¶

__getitem__(index)¶

Invariants¶

class emma.datasets.DataRelease¶

Constructor parameters¶

Methods¶

scenes(split)¶

recipe(dataset_id)¶

Invariants¶

ContentHash utility¶

Real-capture adapters¶

class emma.datasets.RealCaptureAdapter(ABC)¶

Abstract methods¶

Notes¶

class emma.datasets.ColosseumAdapter(RealCaptureAdapter)¶

Notes¶

class emma.datasets.PowderAdapter(RealCaptureAdapter)¶

Notes¶

class emma.datasets.CosmosAdapter(RealCaptureAdapter)¶

Notes¶

See Also¶

`class emma.datasets.RfgenRecipe`¶

`class emma.datasets.ArraySpec`¶

`class emma.datasets.LabelExtractor`¶

`extract(scene, task_id)`¶

`class emma.datasets.EMMADataset(torch.utils.data.Dataset)`¶

`len()`¶

`getitem(index)`¶

`class emma.datasets.DataRelease`¶

`scenes(split)`¶

`recipe(dataset_id)`¶

`class emma.datasets.RealCaptureAdapter(ABC)`¶

`class emma.datasets.ColosseumAdapter(RealCaptureAdapter)`¶

`class emma.datasets.PowderAdapter(RealCaptureAdapter)`¶

`class emma.datasets.CosmosAdapter(RealCaptureAdapter)`¶