Roadmap

Pre-launch / design spec

Dates are indicative, not committed. The version sequence is the plan; the contents of each version will change before release.

EMMA (Electromagnetic Multi-task Model Assessment) ships in a deliberate order, adding one audience and one capability layer per release. The sequence is the contract: each version is a coherent scoring surface, not a grab-bag of tasks. Feature-level maturity is tracked in the maturity matrix; until src/emma/ lands, every contract is proposed-contract.

Version sequence

v0.1: the localization-led MVP (minimum viable product)

Localization-led suite, scored on OOD (out-of-distribution) transfer from day one.

  • Tasks: E-LOC-AOA (continuous regression), E-LOC-LOS, multi-antenna E-ID-DRONE, E-ID-FP (leave-one-unit-out); E-ID-AMC as a continuity column, reported but excluded from the aggregate.

  • Data: rfgen-generated public train and dev; at least two channel environments so leave-one-environment-out is live; a real-capture OOD validation subset (EMMA-REAL-OOD) carrying a sim-to-real gap column.

  • Leaderboard: results-only, prediction submission against a private held-out test set; a frozen, deliberately beatable open baseline on the board.

  • Harness: open-source emma CLI (command-line interface).

v0.2: radar and beam

Adds the radar and communications audiences.

  • E-ID-RWAVE (radar waveform recognition) and E-ID-UAVDOP (UAV, unmanned aerial vehicle, micro-Doppler classification), both synthesized via rfgen because no real radar dataset exposes raw I/Q.

  • E-CH-BEAM (beam management), pulling in the 3GPP (3rd Generation Partnership Project) audience.

v1: coverage and governance

Broadens coverage and formalizes neutral governance.

  • Sandboxed code submission, not just prediction submission.

  • An independent steering committee seated; broader task coverage: E-CH-CSI (the CSI (channel state information) bridge), E-SC-* scene tasks, SAR (synthetic aperture radar) ATR (automatic target recognition).

  • The first public “State of Signal Foundation Models” report.

v2: the frontier

Signal understanding tasks that no community does at scale today.

  • Signal-to-text tasks: E-S2T-CAP (RF, radio-frequency, captioning) and E-S2T-QA (RF question-answering).

  • Dynamic adversarial collection; a certification path.

Per-feature version table

The feature each release ships, the version it lands in, and its maturity. Maturity labels follow STYLE.md; the normative per-feature status lives in the maturity matrix.

Feature

Version

Maturity

E-LOC-AOA, E-LOC-LOS, E-ID-DRONE, E-ID-FP tasks

v0.1

proposed-contract

E-ID-AMC continuity column

v0.1

proposed-contract

rfgen recipes (LOC, DRONE, FP, AMC)

v0.1

proposed-contract

Real-capture OOD subset + sim-to-real gap

v0.1

proposed-contract

emma CLI (list-tasks, download, eval, submit, score, release)

v0.1

proposed-contract

Results-only leaderboard with private held-out

v0.1

proposed-contract

Operational ring-fence with named external co-steward

v0.1

proposed-contract

E-ID-RWAVE, E-ID-UAVDOP, E-CH-BEAM tasks

v0.2

proposed-contract

Sandboxed code submission

v1

planned

Independent steering committee

v1

planned

E-CH-CSI bridge task

v1

proposed-contract

E-SC-SEP, E-SC-ANOM, E-SC-SENSE scene tasks

v1

proposed-contract

SAR (synthetic aperture radar) ATR (automatic target recognition)

v1

planned

Independent non-profit operator (Linux Foundation / JDF vehicle)

v1+

planned

E-S2T-CAP, E-S2T-QA signal-to-text tasks

v2

proposed-contract

Dynamic adversarial collection, certification path

v2

planned

JDF is the Joint Development Foundation. Code submission and the steering committee are planned rather than proposed-contract because their contracts are not yet pinned.

Sequencing rationale

EMMA ships in a deliberate order: the model as a resource first (generator plus open baseline plus harness), the leaderboard second, formal neutral governance last. That is the order every benchmark that became a standard actually followed. ImageNet seeded baselines and governance followed adoption; MLPerf launched and MLCommons formalized neutral governance roughly two years later. Coverage and a non-empty board earn the trust that governance then protects.

Reversing the order fails for specific reasons. Shipping governance before a non-empty board produces rules with nothing to protect and signals that the operator prioritizes control over results. Shipping the leaderboard before the open baseline produces a board the operator sweeps, which is the credibility-killer the ring-fence exists to prevent.

References

  • Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” IJCV 2015, arXiv:1409.0575. Seeded with baselines; governance followed adoption.

  • Mattson et al., “MLPerf Training Benchmark,” MLSys 2020, arXiv:1910.01500. MLCommons neutral governance formalized roughly two years after MLPerf launched; the sequencing precedent for EMMA’s v1+ transition.

  • Srivastava et al., “Beyond the Imitation Game (BIG-bench),” 2023, arXiv:2206.04615. The tribe-contributes-a-task adoption mechanism the v1 task growth follows.

See Also

  • Maturity matrix: the normative per-feature status table.

  • Landscape: the gap each release fills.

  • Governance: the ring-fence maturation this sequence schedules.

  • Tasks: the pillar layout and version badges.

  • Open questions: the go and no-go signals that gate the sequence.