Tasks

Pre-launch / design spec

Task IDs, metrics, and release versions below are proposals. They are designed to be concrete enough to build against and critique, not commitments.

EMMA (Electromagnetic Multi-task Model Assessment) scores one electromagnetic foundation model across five task pillars on a unified raw multi-antenna I/Q (in-phase and quadrature) input. The flagship is localization - AoA (angle-of-arrival) / DoA (direction-of-arrival) is the task that physically requires the inter-antenna phase coherence of multi-antenna I/Q, so without it there is no reason to insist on the array. It is simultaneously legible to the communications, radar, spectrum, and RF (radio- frequency) drone communities, and it is unclaimed by every existing benchmark.

Every task carries a controlled out-of-distribution (OOD) axis - that is the score, not in-distribution accuracy. See Generalization for the protocol. The metric acronyms in the tables are: AUC (area under the receiver operating characteristic curve), EER (equal error rate), NMSE (normalized mean squared error), SI-SDR (scale-invariant signal-to-distortion ratio), AUROC (the same area as AUC, named explicitly to distinguish it from precision-recall variants, used for anomaly detection), BLEU (Bilingual Evaluation Understudy), METEOR (Metric for Evaluation of Translation with Explicit ORdering), CIDEr (Consensus-based Image Description Evaluation); see the glossary for full definitions. The exact task contracts are tabulated at Task reference.

Status badges: v0.1 ships at launch · v0.2 · v1 · v2.

---

Localization - the flagship

ID	Task	Output / primary metric	OOD axis	Status
E-LOC-AOA	Angle-of-arrival / direction-of-arrival	Continuous regression - mean angular error (°)	unseen channel environment	v0.1
E-LOC-LOS	Line-of-sight / non-line-of-sight (LoS / NLoS) classification	Accuracy / AUC	unseen environment, frequency band	v0.1
E-LOC-POS	Positioning (2D / 3D)	Mean localization error (m), CDF@90%	unseen environment, device	v1

E-LOC-AOA is evaluated as continuous regression (mean angular error), not binned classification - because the physically meaningful quantity is an angle, and binning lets a model hide systematic bias behind class accuracy.

Localization is the flagship for three reasons. First, it physically requires inter-antenna phase coherence: the moment the input becomes a magnitude spectrogram or a CSI (channel state information) tensor, AoA and beam prediction are unevaluable from the input. That makes it the load-bearing justification for EMMA’s raw multi-antenna I/Q contract. Second, it is legible across communities: communications cares about positioning, radar about direction-finding, and the drone-RF community about localizing emitters. Third, it is unclaimed: every public RF dataset is single-antenna (RadioML, TorchSig) or a derived representation (DeepMIMO), so array direction-finding is literally unevaluable in the existing literature.

Recipe pages: EMMA-LOC v0.1, EMMA-POS v1.

Identity

ID	Task	Output / primary metric	OOD axis	Status
E-ID-DRONE	Multi-antenna RF drone detection	AUC	unseen drone model, environment	v0.1
E-ID-FP	Emitter / device fingerprinting	Top-1 / EER	leave-one-unit-out (unseen device)	v0.1
E-ID-RWAVE	Radar waveform recognition	Accuracy	unseen waveform family	v0.2
E-ID-UAVDOP	UAV (unmanned aerial vehicle) micro-Doppler classification	Accuracy	unseen platform, motion profile	v0.2
E-ID-AMC	AMC (automatic modulation classification) (legacy / continuity)	Accuracy	unseen SNR (signal-to-noise ratio) regime	v0.1

E-ID-DRONE is a headline defense-relevant task and a near-monopoly for EMMA: every public drone-RF dataset is single-antenna, so drone AoA/localization is literally unevaluable in the existing literature.

E-ID-FP reframes device identification (RF fingerprinting, or SEI: specific emitter identification) as one readout head on a shared backbone, and is scored leave-one-unit-out - the field’s hardest unsolved problem, because existing datasets confound device identity with capture session and receiver front-end.

E-ID-AMC (modulation classification) is included only as a continuity column so the AMC community can map from RadioML. It is saturated, single- antenna-solvable, and its publisher states the data has known errata and is not used in its products - so it is never a headline metric.

Recipe pages: EMMA-DRONE v0.1, EMMA-FP v0.1, EMMA-AMC v0.1, EMMA-RWAVE v0.2, EMMA-UAVDOP v0.2.

Channel

ID	Task	Output / primary metric	OOD axis	Status
E-CH-BEAM	Beam management (best-beam prediction)	Top-k accuracy	unseen scenario, array	v0.2
E-CH-CSI	Channel estimation / CSI feedback (bridge task)	NMSE	unseen scenario	v1

E-CH-CSI is a deliberate bridge to the CSI camp - EMMA does not compete head-on on CSI feedback; it offers an on-ramp for models that reason over channel tensors.

Recipe pages: EMMA-BEAM v0.2, EMMA-CSI v1.

Scene understanding

ID	Task	Output / primary metric	OOD axis	Status
E-SC-SEP	Signal separation / RF scene parsing	SI-SDR / SDR	unseen emitter mix	v1
E-SC-ANOM	Anomaly / novelty detection	AUROC	unseen “normal” baseline	v1
E-SC-SENSE	Spectrum sensing (signal detection)	AUC	unseen band, noise floor	v1

Recipe pages: EMMA-SC-SEP v1, EMMA-SC-ANOM v1, EMMA-SC-SENSE v1.

Signal-to-text - the frontier

ID	Task	Output / primary metric	OOD axis	Status
E-S2T-CAP	RF captioning	BLEU / METEOR / CIDEr	unseen scene type	v2
E-S2T-QA	RF question-answering	Exact-match / F1	unseen scene type	v2

Signal-to-text is the probe that a backbone understands RF rather than merely classifies it. No community currently does it at scale, so it cannot carry adoption - it ships at v2, after the inner ring (comms, spectrum, drone-RF) is proven.

Recipe pages: EMMA-S2T-CAP v2, EMMA-S2T-QA v2.

---

Aggregated score

The headline number on the leaderboard is a normalized aggregate across a model’s v0.1 tasks (E-LOC-AOA, E-LOC-LOS, E-ID-DRONE, E-ID-FP), each computed under its OOD axis. Modulation classification (E-ID-AMC) is reported but excluded from the aggregate. See Aggregation for the normalization.

References

• Schmidt, “Multiple emitter location and signal parameter estimation,” IEEE Trans. Acoustics, Speech, Signal Processing 1986, DOI:10.1109/TASSP.1986.1164830. The MUSIC algorithm; grounds that AoA is physically a continuous angular quantity, scored as regression rather than binned classification.

• Hanna and Hussain, “Robust Low-SNR Modulation Classification,” 2026, arXiv:2605.27673. Apparent model gaps on RadioML collapse under matched hyperparameter search; grounds that AMC is saturated and excluded from the aggregate.

• DeepSig, “RadioML datasets,” datasets page (deepsig.ai/datasets). The publisher’s public note that RML2016 has known errata and is not used in DeepSig products; grounds the AMC continuity-column exclusion. (verify URL)

• “No Radio Left Behind,” 2019. RF device identification across capture sessions; the leave-one-unit-out motivation for E-ID-FP. (verify)

• Media Inhof et al. (DroneRF), 2019. The de-facto drone-RF benchmark; single SDR, so multi-antenna drone localization is unevaluable there. (verify)

• 3GPP TR 38.843, “Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR.” Ships AI/ML use cases (CSI feedback, beam management, positioning) with no dataset or baseline; the bridge-task motivation for E-CH-CSI and E-CH-BEAM. (verify revision-year)

• Debbah and the MBZUAI group, “RF-GPT,” 2026, arXiv:2602.14833. Spectrogram-based RF-language model; the signal-to-text frontier E-S2T-CAP and E-S2T-QA build toward. (verify)

See Also

• Task reference: the exact per-task contracts, metrics, and OOD axes.

• Generalization: the OOD protocol each task is scored under.

• Signal as a modality: why one backbone is read out across many tasks.

• Data model: the raw multi-antenna I/Q every task reads.

• Leaderboard: how the aggregated score is reported.