Dataset recipe

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, schema fields, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

A dataset recipe is a frozen rfgen configuration plus a label-extraction map. It is the unit of content-hashed reproducibility: the artifact a dataset page documents, the input to EMMADataset, and the contract between rfgen (which owns signal generation) and the EMMA pipeline (which owns evaluation). A recipe pins rfgen output; it does not re-specify emitter, channel, or propagation physics. See Datasets and rfgen for the concept and the rfgen config groups a recipe pins.

The serialized form is the DatasetRecipe Pydantic model. Its rfgen field carries a RfgenRecipe (the pinned rfgen commit, resolved configuration path, and per-split seed ranges); its splits field partitions integer seeds into Split members; its label_extraction field tells LabelExtractor which rfgen metadata path maps to each task label.

Schema fields

Field

Type

Purpose

dataset_id

str

Canonical dataset identifier, for example EMMA-LOC-v0.1. Matches the registry row in Datasets.

task_id

TaskID

The primary task the recipe feeds. A recipe can carry labels for several tasks, but one is primary.

version

ReleaseVersion

The release the recipe ships in, for example V0_1.

rfgen

RfgenRecipe

The frozen rfgen pin: commit, resolved configuration path, and per-split seed ranges.

splits

dict[Split, SeedRange]

Map of split (TRAIN, DEV, HOLDOUT) to a seed range and scene count. The holdout range is never published.

label_extraction

dict[str, MetadataPath]

Map of label name to the rfgen metadata field that LabelExtractor reads to produce it. One recipe yields many labels.

target_domain

Domain

The domain the recipe targets, for example POSITIONING or DEFENSE.

real_capture_counterpart

str | None

Optional dataset id of the real-capture OOD (out-of-distribution) subset paired with this recipe, for example EMMA-REAL-OOD-v0.1.

Note

label_extraction keys are label names, not TaskID members. A label such as aoa_deg can feed one task (E-LOC-AOA) while los_flag feeds another (E-LOC-LOS) off the same scene set. The map records the rfgen metadata path; the task-to-label wiring lives in the task spec.

Minimal example

The recipe pins rfgen output and labels it; it does not specify RF (radio-frequency) physics. The rfgen.config value names a composed rfgen GenerationConfig that already pins the emitter_zoo, channel, scene, placement, label, storage, executor, and run groups. The recipe pins receiver array geometry (array_geometry) and element count (num_rx) directly under the rfgen block.

# dataset recipe (proposal) - pins rfgen output, does not specify RF physics
dataset_id: EMMA-LOC-v0.1
task_id: E_LOC_AOA
version: V0_1
rfgen:
  commit: a91f3c2           # frozen rfgen git SHA; placeholder until pinned
  config: scenes/v0.1/urban-mimo-ura.yaml
  array_geometry: URA        # rfgen ArrayGeometry enum member
  num_rx: 8
splits:
  TRAIN:   {start: 1, stop: 160000, count: 160000}
  DEV:     {start: 160001, stop: 200000, count: 40000}
  HOLDOUT: {start: 200001, stop: 240000, count: 40000}   # secret; never published
label_extraction:
  aoa_deg: {field: emitters[].aoa_deg, kind: regression}
  los_flag: {field: emitters[].los_flag, kind: binary}
target_domain: POSITIONING
real_capture_counterpart: EMMA-REAL-OOD-v0.1

The splits value at each Split key is a SeedRange (start, stop, count) matching the helper type in API / Schemas; the loader rejects a range whose realized scene count does not equal count. Receiver array geometry is a member of rfgen’s ArrayGeometry enum (ULA, uniform linear array, or URA, uniform rectangular array) pinned directly under the rfgen block as array_geometry; EMMA reads it into ArraySpec at load time. The emitters[].aoa_deg path is a rfgen metadata path resolved by LabelExtractor, not a field EMMA defines.

References

  • NVIDIA Sionna: Hoydis et al., “Sionna: An Open-Source Library for Next-Generation Physical Layer Research,” 2023, arXiv:2203.11854. The ray-tracing and link-level simulator rfgen builds on; the source of the metadata paths a recipe labels.

  • Yang et al., “SUPERB: Speech processing Universal PERformance Benchmark,” Interspeech 2021, arXiv:2105.01051. The shared-input, many-readouts contract a recipe serves.

See Also