Skip to content

Preserve discrete action specs in CanonicalSpecWrapper - #378

Open
sylvesterkaczmarek wants to merge 3 commits into
google-deepmind:masterfrom
sylvesterkaczmarek:fix/294-canonical-discrete-actions
Open

sylvesterkaczmarek wants to merge 3 commits into
google-deepmind:masterfrom
sylvesterkaczmarek:fix/294-canonical-discrete-actions

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek sylvesterkaczmarek commented Oct 9, 2026 •

Copy link
Copy Markdown

Summary

Fixes #294.

CanonicalSpecWrapper currently treats every dm_env.specs.BoundedArray subclass as a continuous action. In particular, DiscreteArray inherits from BoundedArray but does not accept minimum or maximum in its constructor. Requesting the canonical action spec therefore raises TypeError. Its step() path also incorrectly rescales integer action indices as continuous values.

  • Canonicalize only actual BoundedArray action specs, not the DiscreteArray subclass.
  • Preserve discrete action specs and integer action indices unchanged, including with clip=True.
  • Continue canonicalizing and rescaling ordinary continuous bounded actions, including those inside nested specs.
  • Add six regression tests for discrete metadata, action forwarding, continuous scaling, mixed nested actions, custom continuous subclasses and compatibility with SinglePrecisionWrapper.

Verification

  • 6 focused tests passed with the real dm_env and dm-tree types.
  • Ruff, Pyink formatting, Python compilation and Git whitespace checks passed.
  • Local Python 3.12 environment cannot install the old optional dm-launchpad wheel, so the isolated test runner supplied an unused import-only Launchpad placeholder. The tests exercise the actual Acme wrapper implementation with real dm_env types; no full Acme or agent training suite was run.
  • This change is independent of open Acme PRs involving frame stacking or DQN epsilon settings.

@Narendra02053

Copy link
Copy Markdown

Hi,
One small test gap I noticed: the SinglePrecisionWrapper test currently wraps it inside CanonicalSpecWrapper, but the traceback in #294 shows the opposite order, with SinglePrecisionWrapper outside CanonicalSpecWrapper.

Could we add a test for that order too? It would exercise the exact action_spec() path from the original issue and check that the discrete action spec is preserved after the dtype conversion.

Would it be helpful if I implemented this test? I'm happy to work on it and open a small PR if that's okay with you.

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
@sylvesterkaczmarek

Copy link
Copy Markdown
Author

Good catch, Narendra. I added the reversed wrapper order from #294 in 5829631: SinglePrecisionWrapper(CanonicalSpecWrapper(env)) now has a regression that checks the discrete spec, int64-to-int32 conversion and unchanged action forwarding. All 7 focused tests pass. Thanks for taking a close look; no separate PR needed.

@Narendra02053

Copy link
Copy Markdown

Thanks for adding the test! Glad I could help. I'll keep an eye out for more edge cases in Acme.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missleading errors when using discrete action space with CanonicalSpecWrapper

2 participants