Task reference

Warning

Pre-implementation. This page describes proposed contracts. Class signatures, parameter types, schema fields, and behavior are subject to change before code lands. Once implementation exists, content here will be regenerated from docstrings or sourced from running tests.

EMMA (Electromagnetic Multi-task Model Assessment) scores one frozen backbone across five task pillars on raw multi-antenna I/Q (in-phase and quadrature). Every task carries a controlled OOD (out-of-distribution) axis; that transfer score, not in-distribution accuracy, is the contract. Localization is the flagship pillar: AoA (angle-of-arrival) physically requires the inter-antenna phase coherence of the array, which is the load-bearing justification for the raw multi-antenna I/Q input. The tables below are the normative per-task contracts; the concept page links here for the exact IDs, metrics, axes, domains, readout heads, and versions. Until src/emma/ lands, every row is proposed-contract.

Column legend: ID is the TaskID member; the Metric column links the metric class; OOD axis shows the OODAxis member, or the task-specific holdout where no enum member is formalized yet; Domain shows the Domain member; Readout head links the head ABC; Version is the release badge; the Maturity column states the contract state. Metric acronyms (AUC, EER, NMSE, SI-SDR, AUROC, BLEU (Bilingual Evaluation Understudy), METEOR (Metric for Evaluation of Translation with Explicit ORdering), CIDEr (Consensus-based Image Description Evaluation)) are expanded on first use and defined in the glossary.

Localization

The flagship. AoA / DoA (direction-of-arrival), LoS / NLoS (line-of-sight / non-line-of-sight) detection, and 2D / 3D positioning all serve the positioning community and all depend on array phase coherence.

ID	Task	Metric	OOD axis	Domain	Readout head	Version	Maturity
E-LOC-AOA	Angle-of-arrival / direction-of-arrival	MeanAngularError (continuous regression, degrees, lower is better)	LEAVE_ONE_ENVIRONMENT_OUT	POSITIONING	AngularRegressionHead	v0.1	proposed-contract
E-LOC-LOS	LoS / NLoS classification	BalancedAccuracy / AUC (area under the ROC curve)	LEAVE_ONE_ENVIRONMENT_OUT (secondary: LEAVE_ONE_BAND_OUT, unseen frequency band)	POSITIONING	ClassificationHead	v0.1	proposed-contract
E-LOC-POS	Positioning (2D / 3D)	Mean localization error (m), CDF@90% (metric class pending)	LEAVE_ONE_ENVIRONMENT_OUT (secondary: DEVICE, unseen device)	POSITIONING	AngularRegressionHead	v1	proposed-contract

Note

E-LOC-AOA is scored as continuous regression, not binned classification. The physically meaningful quantity is an angle; binning lets a model hide systematic angular bias behind class accuracy. See Tasks validation for the construct-validity argument.

Identity

Device and emitter identification, drone detection, radar waveform and micro-Doppler recognition, and the AMC (automatic modulation classification) continuity column.

ID	Task	Metric	OOD axis	Domain	Readout head	Version	Maturity
E-ID-DRONE	Multi-antenna RF (radio-frequency) drone detection	AUC	LEAVE_ONE_ENVIRONMENT_OUT (secondary: DRONE_MODEL, unseen drone model)	DEFENSE / SPECTRUM	ClassificationHead	v0.1	proposed-contract
E-ID-FP	Emitter / device fingerprinting (SEI: specific emitter identification)	Top1Accuracy / EER (equal error rate)	LEAVE_ONE_UNIT_OUT	DEFENSE	ClassificationHead	v0.1	proposed-contract
E-ID-RWAVE	Radar waveform recognition	PlainAccuracy	WAVEFORM_FAMILY (unseen waveform family)	RADAR_SAR / DEFENSE	ClassificationHead	v0.2	proposed-contract
E-ID-UAVDOP	UAV (unmanned aerial vehicle) micro-Doppler classification	PlainAccuracy	unseen platform + motion profile	RADAR_SAR / DEFENSE	ClassificationHead	v0.2	proposed-contract
E-ID-AMC	AMC (automatic modulation classification)	PlainAccuracy	SNR_REGIME (unseen SNR (signal-to-noise ratio) regime)	SPECTRUM	ClassificationHead	v0.1	proposed-contract

Note

E-ID-AMC is a continuity column, included so the AMC community can map from RadioML. It is saturated, single-antenna-solvable, and its publisher states the data has known errata and is not used in its products, so it is reported but excluded from the aggregated OOD-avg score.

Note

E-ID-DRONE is a near-monopoly for EMMA: every public drone-RF dataset (DroneRF, DroneDetect / RFUAV) is single-antenna, so multi-antenna drone AoA and localization are unevaluable in the existing literature.

Channel

Beam management and the CSI (channel state information) bridge task.

ID	Task	Metric	OOD axis	Domain	Readout head	Version	Maturity
E-CH-BEAM	Beam management (best-beam prediction)	TopKAccuracy	SCENE_TYPE (unseen scenario + array)	COMMS	BeamPredictionHead	v0.2	proposed-contract
E-CH-CSI	Channel estimation / CSI feedback	NMSE (normalized mean square error)	unseen scenario	COMMS	AngularRegressionHead (dedicated CSI head proposed)	v1	proposed-contract

Note

E-CH-CSI is a deliberate bridge to the CSI camp (DeepMIMO, LWM). EMMA does not compete head-on on CSI feedback; it offers an on-ramp for models that reason over channel tensors. A dedicated CSI readout head is proposed to replace the interim angular regression head at v1.

Scene understanding

Signal separation, anomaly detection, and spectrum sensing.

ID	Task	Metric	OOD axis	Domain	Readout head	Version	Maturity
E-SC-SEP	Signal separation / RF scene parsing	SISDR (scale-invariant SDR) / SDR (signal-to-distortion ratio)	EMITTER_MIX (unseen emitter mix)	SPECTRUM	ClassificationHead (dedicated separation head proposed)	v1	proposed-contract
E-SC-ANOM	Anomaly / novelty detection	AUROC (area under the ROC curve)	unseen normal baseline	SPECTRUM	ClassificationHead	v1	proposed-contract
E-SC-SENSE	Spectrum sensing (signal detection)	AUC	unseen band + noise floor	SPECTRUM	ClassificationHead	v1	proposed-contract

Signal-to-text

The frontier. Captioning and question-answering probe whether a backbone understands RF rather than merely classifies it. No community does this at scale today, so it ships at v2, after the inner ring (communications, spectrum, drone-RF) is proven.

ID	Task	Metric	OOD axis	Domain	Readout head	Version	Maturity
E-S2T-CAP	RF captioning	Bleu / Meteor / CIDER	SCENE_TYPE (unseen scene type)	FRONTIER	CaptionHead	v2	proposed-contract
E-S2T-QA	RF question-answering (QA)	ExactMatch / F1	SCENE_TYPE (unseen scene type)	FRONTIER	QAHead	v2	proposed-contract

Aggregated score

The headline leaderboard number is a normalized OOD-avg across the four v0.1 scored tasks (E-LOC-AOA, E-LOC-LOS, E-ID-DRONE, E-ID-FP), each computed under its OOD axis. E-ID-AMC is reported but excluded from the aggregate. See Aggregation for the normalization contract and Generalization for the protocol.

References

• Schmidt, “Multiple emitter location and signal parameter estimation,” IEEE Trans. Acoustics, Speech, Signal Processing 1986, DOI:10.1109/TASSP.1986.1164830. The MUSIC algorithm; grounds that AoA is a continuous angular quantity scored as regression rather than binned classification. (verify)

• Roy and Kailath, “ESPRIT: Estimation of signal parameters via rotational invariance techniques,” IEEE Trans. Acoustics, Speech, Signal Processing 1989, DOI:10.1109/29.32276. Direction-of-arrival estimation; the second classical array-signal anchor for E-LOC-AOA. (verify)

• Hanna and Hussain, “Robust Low-SNR Modulation Classification,” 2026, arXiv:2605.27673. Apparent model gaps on RadioML collapse under matched hyperparameter search; grounds the E-ID-AMC continuity-column exclusion. (verify)

• DeepSig, “RadioML datasets,” datasets page (deepsig.ai/datasets). The publisher’s public note that RML2016 has known errata and is not used in DeepSig products; grounds the E-ID-AMC continuity-column exclusion. (verify URL)

• “No Radio Left Behind,” 2019. RF device identification across capture sessions; the leave-one-unit-out motivation for E-ID-FP. (verify)

• Media Inhof et al. (DroneRF), 2019. The de-facto single-antenna drone-RF benchmark; grounds the E-ID-DRONE near-monopoly. (verify)

• 3GPP TR 38.843, “Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR.” Ships AI/ML use cases (CSI feedback, beam management, positioning) with no dataset or baseline; the bridge-task motivation for E-CH-CSI and E-CH-BEAM. (verify revision-year)

• Alkhateeb, “DeepMIMO,” 2019, arXiv:1902.06435; Djordjevic, Ali, and Alkhateeb, “LWM,” 2024, arXiv:2411.08872. The CSI camp the E-CH-CSI bridge targets. (verify)

• Le Roux, Wisdom, Erdogan, and Hershey, “SDR: Half-baked or Well Done?,” ICASSP 2019, arXiv:1811.02508. SI-SDR for E-SC-SEP. (verify)

• Papineni et al., “BLEU,” ACL 2002; Banerjee and Lavie, “METEOR,” ACL 2005 (verify); Vedantam et al., “CIDEr,” CVPR 2015 (verify). Captioning metrics for E-S2T-CAP.

• Rajpurkar et al., “SQuAD,” EMNLP 2016, arXiv:1606.05250. Exact-match and F1 for E-S2T-QA. (verify)

• Davis and Goadrich, “The relationship between Precision-Recall and ROC curves,” ICML 2006. AUC / AUROC grounding for E-LOC-LOS, E-ID-DRONE, E-SC-ANOM, E-SC-SENSE. (verify)

• Brodersen et al., “The balanced accuracy and its posterior distribution,” ICPR 2010. Balanced accuracy under class imbalance for E-LOC-LOS. (verify)

See Also

• Tasks: the narrative version of the five pillars.

• Generalization: the OOD protocol each task is scored under.

• Aggregation: the OOD-avg normalization contract.

• Metrics reference: the per-task metric contracts.

• Tasks: the readout-head and OOD-protocol API surfaces.

• Tasks validation: the six-lens validation of these contracts.