EMMA¶
Pre-launch / design spec
You are reading a pre-launch design spec. EMMA is under active design; the tasks, dataset splits, schemas, CLI, and harness described here are proposals and will change before v0.1. This site exists so the design can be previewed and shaped before anything ships.
Electromagnetic Multi-task Model Assessment: a generalization-first, multi-task evaluation of electromagnetic (signal) foundation models.
One pretrained backbone, scored on how well it transfers across tasks, on the raw phase-coherent multi-antenna I/Q (in-phase and quadrature) that localization physically requires.
The angle¶
Every foundation-model modality has a shared benchmark that lets one pretrained backbone be read out across many tasks: GLUE and SuperGLUE for language, ImageNet for vision, SUPERB for speech, and MLPerf for systems. EMMA is that benchmark for a new modality, electromagnetic signal. One frozen backbone, lightweight task-specific readouts, one aggregated score.
The thing no existing signal asset measures is generalization: how well a model holds up under unseen channels, environments, frequency bands, and devices. Leave-one-environment-out is a first-class protocol, not an appendix. In-distribution accuracy is what RadioML, MSTAR, and every saturated single-task dataset already measure; scoring only that would put EMMA behind on day one.
Get started¶
Install EMMA, point it at your backbone, and run your first evaluation.
What EMMA measures: the pillars, the metric per task, the out-of-distribution axis.
What the columns mean, the OOD-avg score, the sim-to-real gap, and why neutrality matters.
Design pillars¶
Every task is scored under controlled out-of-distribution axes (unseen channels, environments, frequency bands, devices), not just in-distribution accuracy.
Localization and beam tasks physically depend on inter-antenna phase: spectrograms destroy it, CSI (channel state information) discards it, and every existing dataset is single-antenna or derived. This moat is enforced by physics.
Synthetic scenes are paired with real-capture out-of-distribution subsets, and EMMA reports a sim-to-real gap alongside every synthetic score.
Superpose also builds signal foundation models, so the benchmark is governed by an independent steering committee, ring-fenced from the model team, with third-party re-scoring. Neutrality cannot be manufactured retroactively.
EMMA is stewarded by Superpose. Source on GitHub. For benchmark builders and contributors, see the [developer documentation](https://emma.docs.superpose.us/dev/index.html).