Datasets

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, schema fields, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

Dataset recipes, the core dataset abstraction, label extraction, the data release resolver, and the real-capture adapters. EMMA owns dataset recipes (pinned rfgen configurations), loaders, and label extraction. It does not own emitters, channels, propagation, antenna patterns, or any signal generation; those live in rfgen. See Datasets and rfgen for the boundary.

Three roles

Role	Examples
Data types	RfgenRecipe, ArraySpec
ABC	RealCaptureAdapter
Concrete classes	EMMADataset, LabelExtractor, DataRelease, ColosseumAdapter, PowderAdapter, CosmosAdapter

Class index

Class	Role	Purpose
RfgenRecipe	Data type	Frozen rfgen pin: commit, resolved config, seed ranges
ArraySpec	Data type	Receiver array geometry reference
EMMADataset	Concrete	Loads scenes as canonical multi-antenna I/Q plus labels
LabelExtractor	Concrete	Maps rfgen metadata to per-task labels
DataRelease	Concrete	Resolves a manifest to a set of scenes
RealCaptureAdapter	ABC	Normalizes a public testbed into the EMMA scene format
ColosseumAdapter	Concrete	Northeastern Colosseum testbed adapter
PowderAdapter	Concrete	POWDER (Platform for Open Wireless Data-driven Experimental Research) testbed adapter
CosmosAdapter	Concrete	COSMOS (Cloud-enhanced Open Software Defined Mobile Wireless Testbed) adapter

---

class emma.datasets.RfgenRecipe

The frozen rfgen pin carried by every DatasetRecipe. Pins the rfgen commit, the resolved configuration path, and the seed ranges. EMMA regenerates scenes through rfgen using this pin; it never generates I/Q (in-phase and quadrature) itself.

@dataclass(frozen=True, slots=True)
class RfgenRecipe:
    commit: str
    resolved_config: str
    seed_ranges: dict[Split, tuple[int, int]]

Fields

Field	Type	Purpose
commit	str	Frozen rfgen git SHA
resolved_config	str	Path to the fully composed rfgen GenerationConfig
seed_ranges	dict[{ref}Split , tuple[int, int]]	Inclusive seed range per split

Invariants

• The triple (commit, resolved_config, seed) fully determines a scene bit-for-bit. Changing any field changes the scenes and requires a new content hash.

• seed_ranges for Split.HOLDOUT is a frozen secret held by the external co-steward. It is never serialized into a public recipe.

---

class emma.datasets.ArraySpec

Typed reference to a rfgen receiver array configuration. Resolved from the arrays field on a SceneManifest at load time.

@dataclass(frozen=True, slots=True)
class ArraySpec:
    geometry: str
    num_elements: int
    spacing: float

Fields

Field	Type	Purpose
geometry	str	rfgen ArrayGeometry enum value, for example ULA (uniform linear array) or URA (uniform rectangular array)
num_elements	int	Number of antenna elements
spacing	float	Inter-element spacing in wavelengths

Notes

• geometry is a rfgen ArrayGeometry enum member, not an EMMA enum. EMMA reads it at load time; it does not redefine array geometry types.

---

class emma.datasets.LabelExtractor

Maps rfgen scene metadata to per-task labels. Reads the rfgen metadata paths declared in a DatasetRecipe label_extraction map and produces the label tensor a ReadoutHead trains against.

class LabelExtractor:
    def __init__(self, recipe: DatasetRecipe) -> None: ...

    def extract(
        self, scene: SceneManifest, task_id: TaskID
    ) -> torch.Tensor | dict[str, torch.Tensor]: ...

extract(scene, task_id)

Extracts the label for one task from one scene’s manifest.

Parameter	Type	Purpose
scene	SceneManifest	The per-scene provenance record carrying realized emitter metadata
task_id	TaskID	The task whose label is requested

Returns. A torch.Tensor for a single-label task (regression angle, binary class), or a dict[str, torch.Tensor] for a multi-label task. The exact shape and dtype are pinned by the task’s TaskSpec.

Notes

• The extractor reads rfgen metadata paths such as emitters[].aoa_deg; it does not recompute physical quantities. One recipe yields many labels off the same scene set.

---

class emma.datasets.EMMADataset(torch.utils.data.Dataset)

The core dataset abstraction. Loads raw multi-antenna I/Q from scenes resolved by a DataRelease, pairs each scene with per-task labels via LabelExtractor, and yields the canonical record a FrozenBackbone consumes.

class EMMADataset(torch.utils.data.Dataset):
    def __init__(
        self,
        release: DataRelease,
        split: Split,
        task_id: TaskID,
        label_extractor: LabelExtractor | None = None,
    ) -> None: ...

    def __len__(self) -> int: ...

    def __getitem__(self, index: int) -> tuple[torch.Tensor, torch.Tensor | dict[str, torch.Tensor]]: ...

Constructor parameters

Parameter	Type	Default	Purpose
release	DataRelease	required	The resolved data release providing scenes
split	Split	required	Which split to expose
task_id	TaskID	required	Which task’s labels to produce
label_extractor	LabelExtractor | None	None	Override extractor; if None, built from the release recipe

Methods

__len__()

Returns the number of scenes in the requested split.

__getitem__(index)

Parameter	Type	Purpose
index	int	Scene index within the split

Returns. A tuple (iq, label) where iq is a torch.float32 tensor of shape (num_rx, 2, N) (the multi-antenna I/Q layout) and label is the per-task label produced by LabelExtractor.

Invariants

• iq dtype is torch.float32, shape (num_rx, 2, N): channel 0 is in-phase, channel 1 is quadrature. Matches the rfgen multi-RX layout.

• Requesting Split.HOLDOUT outside the scoring path raises SplitNotAvailable.

---

class emma.datasets.DataRelease

Resolves a DataReleaseManifest to a set of scenes. Materializes the scenes from the pinned rfgen recipe (for synthetic splits) or via the real-capture adapters (for real-capture subsets), verifies content hashes, and partitions scenes by split.

class DataRelease:
    def __init__(
        self,
        manifest: DataReleaseManifest,
        cache_dir: str | None = None,
    ) -> None: ...

    def scenes(self, split: Split) -> list[SceneManifest]: ...

    def recipe(self, dataset_id: str) -> DatasetRecipe: ...

    @property
    def content_hash(self) -> str: ...

Constructor parameters

Parameter	Type	Default	Purpose
manifest	DataReleaseManifest	required	The release manifest to resolve
cache_dir	str | None	None	Local cache directory for regenerated scenes

Methods

scenes(split)

Returns the scene manifests assigned to the requested split. Raises SplitNotAvailable for Split.HOLDOUT outside the scoring path.

recipe(dataset_id)

Returns the DatasetRecipe for a dataset_id bundled in this release.

Invariants

• Content hashes are verified on load. A scene whose hash does not match the manifest raises RecipeMismatchError.

• Synthetic scenes are regenerated through rfgen from the pinned recipe; EMMA never generates I/Q itself.

ContentHash utility

The module exposes static content-hash helpers used by manifests and the release resolver:

class ContentHash:
    @staticmethod
    def scene_hash(commit: str, config: str, seed: int) -> str: ...

    @staticmethod
    def release_hash(scene_hashes: list[str]) -> str: ...

The digest construction is documented in Content hashing.

---

Real-capture adapters

Real-capture OOD (out-of-distribution) subsets are ingested through adapter implementations behind the RealCaptureAdapter ABC. Each adapter normalizes a public RF (radio-frequency) testbed into the EMMA scene format so the same frozen backbone and the same per-task metrics score both synthetic and real scenes. Real-capture data feeds the sim-to-real gap column only; it never enters the aggregate.

class emma.datasets.RealCaptureAdapter(ABC)

The extension contract for real-capture data sources. Subclass to add a testbed. One ABC per concept; no concrete-to-concrete inheritance.

class RealCaptureAdapter(ABC):
    @property
    @abstractmethod
    def name(self) -> str: ...

    @abstractmethod
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...

    @abstractmethod
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Abstract methods

Method	Returns	Purpose
name	str	Adapter identifier, for example colosseum
list_scenes(split)	list[{ref}SceneManifest ]	Scenes available for the split
load_iq(scene_id)	torch.Tensor	Multi-antenna I/Q tensor for the scene

Notes

• Adapter names are open plugin names (plain str), not closed enum members, because anyone can ship a new adapter.

• The adapter populates the SceneManifest generator field with its name, distinguishing real captures from rfgen scenes.

---

class emma.datasets.ColosseumAdapter(RealCaptureAdapter)

Adapter for the Northeastern Colosseum testbed. Normalizes Colosseum capture records into EMMA scene manifests and multi-antenna I/Q tensors.

class ColosseumAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
root	str	required	Path to the local Colosseum dataset mirror
band	str | None	None	Optional frequency-band filter

Notes

• Colosseum is the Northeastern University RF emulation testbed; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.

• Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

---

class emma.datasets.PowderAdapter(RealCaptureAdapter)

Adapter for the POWDER (Platform for Open Wireless Data-driven Experimental Research) testbed. Normalizes POWDER capture records into EMMA scene manifests.

class PowderAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
root	str	required	Path to the local POWDER dataset mirror
band	str | None	None	Optional frequency-band filter

Notes

• POWDER is the NSF PAWR city-scale testbed at the University of Utah; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.

• Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

---

class emma.datasets.CosmosAdapter(RealCaptureAdapter)

Adapter for the COSMOS (Cloud-enhanced Open Software Defined Mobile Wireless Testbed) testbed. Normalizes COSMOS capture records into EMMA scene manifests.

class CosmosAdapter(RealCaptureAdapter):
    def __init__(self, root: str, band: str | None = None) -> None: ...

    @property
    def name(self) -> str: ...
    def list_scenes(self, split: Split) -> list[SceneManifest]: ...
    def load_iq(self, scene_id: str) -> torch.Tensor: ...

Parameter	Type	Default	Purpose
root	str	required	Path to the local COSMOS dataset mirror
band	str | None	None	Optional frequency-band filter

Notes

• COSMOS is the NSF PAWR city-scale testbed led by Rutgers in West Harlem; the adapter consumes the published capture format and does not re-acquire or re-simulate scenes.

• Ingested scenes feed the sim-to-real gap column only and never enter the aggregate OOD average.

See Also

• Dataset recipe: the schema that pins rfgen output.

• Datasets and rfgen: the boundary between EMMA and rfgen.

• Content hashing: the reproducibility anchor for resolved scenes.

• Sim-to-real gap: how real-capture subsets feed the fidelity column.