Primitives

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, schema fields, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

Closed-set enumerations and the exception hierarchy used across every EMMA (Electromagnetic Multi-task Model Assessment) contract. Two modules live here: emma.enums (framework-owned closed choices as StrEnum types) and emma.errors (the root exception hierarchy). These pages appear first because later API pages reference enum members and error classes as primitive vocabulary.

Why StrEnum

Framework-owned closed choices (task pillars, OOD (out-of-distribution) axes, splits, environments, submission modes, metric directions, target domains) use StrEnum members. YAML and recipe configs use the string value; Pydantic v2 deserializes it at validation time. Plugin registry names (for example a custom real-capture adapter name) stay as plain str because they resolve through the registry, not a closed enum. See STYLE.md for the enum-vs-string rule.

emma.enums

Class index

Enum	Members	Purpose
TaskID	15	Canonical task identifier
Pillar	5	Top-level task family
ReleaseVersion	4	Benchmark release track
Maturity	5	Contract state label
Split	3	Train, development, holdout partition
Environment	3	Channel propagation setting
OODAxis	8	Controlled transfer dimension
SubmissionMode	2	Prediction versus sandboxed code
MetricDirection	2	Higher or lower is better
Domain	6	Target application domain

class emma.enums.TaskID(StrEnum)

Canonical task identifier. A task that is not in TaskID cannot be scored. Members carry the string value used in YAML; the member name is the Python symbol.

class TaskID(StrEnum):
    # Localization
    E_LOC_AOA  = "E-LOC-AOA"   # angle-of-arrival / direction-of-arrival
    E_LOC_LOS  = "E-LOC-LOS"   # LoS / NLoS classification
    E_LOC_POS  = "E-LOC-POS"   # 2D / 3D positioning

    # Identity
    E_ID_DRONE = "E-ID-DRONE"  # multi-antenna drone detection
    E_ID_FP    = "E-ID-FP"     # emitter / device fingerprinting (SEI)
    E_ID_RWAVE = "E-ID-RWAVE"  # radar waveform recognition
    E_ID_UAVDOP= "E-ID-UAVDOP" # UAV micro-Doppler classification
    E_ID_AMC   = "E-ID-AMC"    # automatic modulation classification (continuity)

    # Channel
    E_CH_BEAM  = "E-CH-BEAM"   # beam management
    E_CH_CSI   = "E-CH-CSI"    # channel estimation / CSI feedback

    # Scene understanding
    E_SC_SEP   = "E-SC-SEP"    # signal separation
    E_SC_ANOM  = "E-SC-ANOM"   # anomaly / novelty detection
    E_SC_SENSE = "E-SC-SENSE"  # spectrum sensing

    # Signal-to-text
    E_S2T_CAP  = "E-S2T-CAP"   # RF captioning
    E_S2T_QA   = "E-S2T-QA"    # RF question answering

Members

Member	Value	Pillar	Version
E_LOC_AOA	E-LOC-AOA	LOCALIZATION	v0.1
E_LOC_LOS	E-LOC-LOS	LOCALIZATION	v0.1
E_LOC_POS	E-LOC-POS	LOCALIZATION	v1
E_ID_DRONE	E-ID-DRONE	IDENTITY	v0.1
E_ID_FP	E-ID-FP	IDENTITY	v0.1
E_ID_RWAVE	E-ID-RWAVE	IDENTITY	v0.2
E_ID_UAVDOP	E-ID-UAVDOP	IDENTITY	v0.2
E_ID_AMC	E-ID-AMC	IDENTITY	v0.1
E_CH_BEAM	E-CH-BEAM	CHANNEL	v0.2
E_CH_CSI	E-CH-CSI	CHANNEL	v1
E_SC_SEP	E-SC-SEP	SCENE	v1
E_SC_ANOM	E-SC-ANOM	SCENE	v1
E_SC_SENSE	E-SC-SENSE	SCENE	v1
E_S2T_CAP	E-S2T-CAP	SIGNAL_TO_TEXT	v2
E_S2T_QA	E-S2T-QA	SIGNAL_TO_TEXT	v2

The per-task metric, OOD axis, readout head, and domain wiring live in Task reference. E_ID_AMC is a continuity column: reported but excluded from the aggregated OOD average.

class emma.enums.Pillar(StrEnum)

Top-level task family. EMMA scores five pillars on raw multi-antenna I/Q (in-phase and quadrature).

class Pillar(StrEnum):
    LOCALIZATION   = "localization"
    IDENTITY       = "identity"
    CHANNEL        = "channel"
    SCENE          = "scene"
    SIGNAL_TO_TEXT = "signal_to_text"

Member	Value	Covers
LOCALIZATION	localization	AoA (angle of arrival), LoS / NLoS (line-of-sight / non-line-of-sight), positioning
IDENTITY	identity	Drone detection, fingerprinting, waveform, micro-Doppler, AMC (automatic modulation classification)
CHANNEL	channel	Beam management, CSI (channel state information) feedback
SCENE	scene	Separation, anomaly detection, spectrum sensing
SIGNAL_TO_TEXT	signal_to_text	Captioning, question answering

class emma.enums.ReleaseVersion(StrEnum)

Benchmark release track. Each task, dataset recipe, and schema carries the version it ships in.

class ReleaseVersion(StrEnum):
    V0_1 = "v0.1"
    V0_2 = "v0.2"
    V1   = "v1"
    V2   = "v2"

Member	Value	Scope
V0_1	v0.1	Launch: four scored tasks plus AMC continuity and real-capture subset
V0_2	v0.2	Radar waveform, micro-Doppler, beam management
V1	v1	Positioning, CSI, signal separation, anomaly, sensing
V2	v2	Signal-to-text frontier: captioning and question answering

class emma.enums.Maturity(StrEnum)

Contract state label. Matches the maturity vocabulary in STYLE.md.

class Maturity(StrEnum):
    PLANNED           = "planned"
    PROPOSED_CONTRACT = "proposed-contract"
    IMPLEMENTED       = "implemented"
    VERIFIED          = "verified"
    DEPRECATED        = "deprecated"

Member	Value	Meaning
PLANNED	planned	Named roadmap item; no contract yet
PROPOSED_CONTRACT	proposed-contract	Intended API, schema, or recipe shape; may still change
IMPLEMENTED	implemented	Code exists; examples are runnable
VERIFIED	verified	Implementation has contract tests or golden validation
DEPRECATED	deprecated	Scheduled for removal; use the replacement instead

Until src/emma/ lands, every API surface is PROPOSED_CONTRACT.

class emma.enums.Split(StrEnum)

Data partition. The holdout split is never published with labels.

class Split(StrEnum):
    TRAIN   = "train"
    DEV     = "dev"
    HOLDOUT = "holdout"

Member	Value	Released	Purpose
TRAIN	train	public, re-derivable	Pretraining and fine-tuning
DEV	dev	public, re-derivable	Local evaluation and public leaderboard proxies
HOLDOUT	holdout	private, never released	Official scoring via prediction or sandboxed-code submission

class emma.enums.Environment(StrEnum)

Channel propagation setting. v0.1 ships with at least two synthetic environments so leave-one-environment-out is live from day one.

class Environment(StrEnum):
    E_URBAN   = "E-URBAN"
    E_RURAL   = "E-RURAL"
    E_REAL_CAP= "E-REAL-CAP"

Member	Value	rfgen channel model	Used for
E_URBAN	E-URBAN	Sionna UMa (urban macro) ray tracer	Train plus one OOD target
E_RURAL	E-RURAL	Sionna RMa (rural macro) or TDL (tapped delay line) near-LoS	Train plus one OOD target
E_REAL_CAP	E-REAL-CAP	None; ingested via RealCaptureAdapter	Sim-to-real gap column only

E_REAL_CAP is fidelity only: real-capture subsets (Colosseum, POWDER, COSMOS) feed the sim-to-real gap column, never the aggregate. See Environment reference.

class emma.enums.OODAxis(StrEnum)

Controlled transfer dimension. Each task carries one primary axis; several layer a secondary axis. The transfer score, not in-distribution accuracy, is the contract.

class OODAxis(StrEnum):
    ENVIRONMENT     = "environment"
    DEVICE          = "device"
    FREQUENCY_BAND  = "frequency_band"
    SNR_REGIME      = "snr_regime"
    DRONE_MODEL     = "drone_model"
    WAVEFORM_FAMILY = "waveform_family"
    EMITTER_MIX     = "emitter_mix"
    SCENE_TYPE      = "scene_type"

Member	Value	Transfer dimension	Example task
ENVIRONMENT	environment	Unseen channel environment	E_LOC_AOA, E_LOC_LOS, E_ID_DRONE
DEVICE	device	Unseen device or unit	E_ID_FP (leave-one-unit-out)
FREQUENCY_BAND	frequency_band	Unseen frequency band	E_LOC_LOS (secondary)
SNR_REGIME	snr_regime	Unseen SNR (signal-to-noise ratio) regime	E_ID_AMC
DRONE_MODEL	drone_model	Unseen drone model	E_ID_DRONE (secondary)
WAVEFORM_FAMILY	waveform_family	Unseen waveform family	E_ID_RWAVE
EMITTER_MIX	emitter_mix	Unseen emitter mix or scene density	E_SC_SEP
SCENE_TYPE	scene_type	Unseen scene type or scenario	E_S2T_CAP, E_CH_BEAM

The per-task axis wiring lives in Task reference; the evaluation mechanics live in OOD protocol.

class emma.enums.SubmissionMode(StrEnum)

Leaderboard submission mode. v0.1 accepts predictions only; v1 adds sandboxed code.

class SubmissionMode(StrEnum):
    PREDICTION      = "prediction"
    SANDBOXED_CODE  = "sandboxed_code"

Member	Value	Version	What the submitter sends
PREDICTION	prediction	v0.1	A PredictionBundle (Parquet per task); EMMA scores it on the secret holdout
SANDBOXED_CODE	sandboxed_code	v1	A sandboxed runner that materializes the holdout and scores in isolation

class emma.enums.MetricDirection(StrEnum)

Optimization sense for a metric. Consumed by OODAvg to decide whether to invert before normalization.

class MetricDirection(StrEnum):
    HIGHER = "higher"
    LOWER  = "lower"

Member	Value	Meaning
HIGHER	higher	Higher raw score is better (accuracy, AUC (area under the curve), SI-SDR (scale-invariant signal-to-distortion ratio))
LOWER	lower	Lower raw score is better (angular error, NMSE (normalized mean squared error), EER (equal error rate))

class emma.enums.Domain(StrEnum)

Target application domain. Each task and recipe declares the community it serves.

class Domain(StrEnum):
    DEFENSE      = "defense"
    SPECTRUM     = "spectrum"
    COMMS        = "comms"
    POSITIONING  = "positioning"
    RADAR_SAR    = "radar_sar"
    FRONTIER     = "frontier"

Member	Value	Community
DEFENSE	defense	Specific emitter identification, drone detection
SPECTRUM	spectrum	Spectrum sensing, anomaly detection, AMC
COMMS	comms	Beam management, CSI feedback
POSITIONING	positioning	AoA, LoS / NLoS, positioning
RADAR_SAR	radar_sar	Radar waveform recognition, micro-Doppler
FRONTIER	frontier	Signal-to-text captioning and question answering

emma.errors

The exception hierarchy for the benchmark. Every error EMMA raises is a subclass of EmmaError, so callers can catch all benchmark faults with a single except clause while discriminating on specific subclasses for task-gating, split-access, or submission-validation handling.

Every subclass carries a structured context: dict[str, object] attribute for telemetry. Values are JSON-serializable so log sinks emit them as-is.

Class index

Class	Parent	Purpose
EmmaError	Exception	Root exception; carries context
TaskNotReleased	EmmaError	Task is not available in the requested release
SplitNotAvailable	EmmaError	Split cannot be accessed (holdout is private)
RecipeMismatchError	EmmaError	Resolved scenes do not match the pinned recipe hash
PredictionSchemaError	EmmaError	Prediction bundle failed schema validation
SubmissionRejected	EmmaError	Leaderboard submission failed validation
HoldoutAccessError	EmmaError	Unauthorized attempt to read holdout labels

class emma.errors.EmmaError(Exception)

Root of the EMMA exception hierarchy. Every benchmark-raised exception subclasses this.

class EmmaError(Exception):
    def __init__(
        self,
        message: str = "",
        *,
        context: dict[str, object] | None = None,
    ) -> None: ...

Attributes

Attribute	Type	Purpose
context	dict[str, object]	Structured key/value attachments; JSON-serializable. Defaults to {} when no context kwarg is passed

Notes

• Subclasses inherit __init__ unchanged unless they override it.

• Callers can catch all benchmark faults with except EmmaError (where EmmaError is the root) while discriminating on subclasses for targeted handling.

class emma.errors.TaskNotReleased(EmmaError)

Raised when TaskRegistry is asked for a task that does not ship in the requested ReleaseVersion. The message names the task id and the version.

class emma.errors.SplitNotAvailable(EmmaError)

Raised when a caller requests a split that cannot be accessed. The canonical case is attempting to read Split.HOLDOUT labels outside the scoring path. The holdout labels never leave the scoring server.

class emma.errors.RecipeMismatchError(EmmaError)

Raised when DataRelease resolves scenes whose content hash does not match the DatasetRecipe pin. The context carries the expected and observed hashes so the mismatch is debuggable.

class emma.errors.PredictionSchemaError(EmmaError)

Raised by SubmissionValidator when a PredictionBundle fails schema validation: wrong columns, wrong dtype, missing scene_id, or out-of-range values.

class emma.errors.SubmissionRejected(EmmaError)

Raised when a leaderboard submission is rejected after validation. Causes include rate-limit violations flagged by AntiOverfit, provenance gaps, or a missing PreRegistration when one is required.

class emma.errors.HoldoutAccessError(EmmaError)

Raised on any unauthorized attempt to read holdout labels or to execute outside the sandboxed scoring path. Distinct from SplitNotAvailable, which covers the general case; this class flags a governance boundary violation recorded in the audit log.

See Also

• STYLE.md: the enum-versus-string rule, maturity labels, and version badges.

• Task reference: the per-task metric and OOD axis wiring.

• Environment reference: the environment and axis catalogs.

• Schemas: the Pydantic v2 models that consume these enums.