Regenerate a dataset

Note

The commands on this page are proposed until the CLI (command-line interface) ships. The signatures match API / CLI; once src/emma/ lands the steps below run as written.

Regenerate an EMMA (Electromagnetic Multi-task Model Assessment) data split locally from its dataset recipe. The generator is the dataset: nothing large downloads as a blob. emma download calls rfgen under the hood, using a frozen rfgen commit, a resolved configuration, and a seed range, so anyone can re-derive the same scenes bit-for-bit.

Goal

Produce a local copy of a dev split for one or more tasks, verify it matches the published content hash, and inspect one scene’s metadata.

Prerequisites

  • The emma CLI is installed (pip install emma-benchmark).

  • rfgen and its dependencies are available; emma download shells out to it.

  • You know which dataset id you want. The registry is in Datasets and scenes.

Steps

  1. Pick a dataset id from the registry. Each row pins the antenna layout, emitter mix, channel environment, OOD (out-of-distribution) axis, and the rfgen commit. For the v0.1 localization pillar, the id is EMMA-LOC-v0.1 and it feeds E-LOC-AOA and E-LOC-LOS.

    $ emma list-tasks --version v0.1

  2. Regenerate the dev split for the tasks you need. Pass the task IDs, not the dataset id; the CLI resolves the matching DatasetRecipe.

    $ emma download --split dev --tasks E-LOC-AOA,E-ID-DRONE --out ./scenes

    The --split argument accepts train or dev. It rejects holdout: the holdout is a frozen secret seed that only exists inside the scoring path (see Content hashing).

  3. Verify the content hash. The command prints a hash line per recipe; compare it against the hash recorded on the dataset’s recipe page. A matching hash means the regenerated scenes are bit-identical to the published split.

    hash : sha256:b7e3...f019

    SHA (Secure Hash Algorithm) 256 is the digest algorithm; see Content hashing for the canonical construction.

  4. Inspect one scene. Each scene carries a SceneManifest recording the generator commit, config, seed, antenna array, emitters, and channel. Read it to confirm the antenna geometry and label fields for your task.

    $ cat ./scenes/emma-scene-00017a/manifest.json

Expected result

./scenes/ holds the regenerated dev scenes for the requested tasks, each with a manifest whose content hash matches the published recipe. You can now point emma eval at the directory.

See Also