Skip to content

fix(metrics): stop spurious overwrite warnings from set_default_dimensions - #8403

Open
vishwakt wants to merge 1 commit into
aws-powertools:developfrom
vishwakt:fix/metrics-set-default-dimensions-warning
Open

fix(metrics): stop spurious overwrite warnings from set_default_dimensions#8403
vishwakt wants to merge 1 commit into
aws-powertools:developfrom
vishwakt:fix/metrics-set-default-dimensions-warning

Conversation

@vishwakt

Copy link
Copy Markdown
Contributor

Issue number: closes #8402

Summary

Changes

There are two causes, and the second makes this worse than reported: even with default dimensions set once at module level, the recommended pattern, the warning fires for every default dimension on every flush.

  1. Metrics.set_default_dimensions called self.provider.set_default_dimensions(**dimensions) and then re-added every dimension through add_dimension. The second pass always found the keys already registered, so the warning fired even on the very first call. The redundant loop is removed and the method now delegates to the provider. The docstring, which sat after the first statement and was not treated as a docstring by Python, moved to the top of the method.

  2. The provider re-registers default dimensions internally: clear_metrics re-adds them after every flush, and warm invocations repeat set_default_dimensions. The previous condition in AmazonCloudWatchEMFProvider.add_dimension warned for any name already present in dimension_set or default_dimensions, so these internal paths warned too. It now warns only when a dimension is overwritten with a different value, which matches the warning message ("The previous value will be overwritten") and the intent of feat(metrics): warn when overwriting dimension #5653, whose test covers overwriting with a different value.

User experience

Before: one warning per dimension on the first set_default_dimensions call, plus one per default dimension on every flush and on every warm re-registration.

After: no warnings when nothing is overwritten. Overwriting a dimension with a different value still warns once, including when done via set_default_dimensions.

Added five tests covering: no warning on first call, no warning on unchanged re-registration, exactly one warning when a default value changes, no warnings across log_metrics invocations, and no warning when re-adding a dimension with the same value. The existing test_add_dimension_overwrite_warning passes unchanged.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…sions

Metrics.set_default_dimensions called provider.set_default_dimensions
and then re-added every dimension through add_dimension, so the second
pass always found the keys already registered and warned even on the
first call. Remove the redundant loop and delegate to the provider.

The provider also re-registers default dimensions internally, in
clear_metrics after every flush and on repeated set_default_dimensions
calls, which triggered the same warning on every warm invocation. Warn
only when a dimension is overwritten with a different value, matching
the warning message and the intent of aws-powertools#5653.

Closes aws-powertools#8402
@sonarqubecloud

Copy link
Copy Markdown

@ericbn

ericbn commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

This change is still allowing the add_dimension method in the provider to be called twice. I see there's another redundant call: self.set_default_dimensions(**self.default_dimensions) in the clear_metrics method in the provider.

The change to hide the warning when the new value is the same as the old one is nice, but it's hiding the redundant call that's still happening. Not sure if I'd change the condition when the warning is emitted there.

Also, I think it might be worth investigating this whole code better. I don't get why Metrics keeps its own data and I don't understand the explanation in the comment in

# NOTE: We use class attrs to share metrics data across instances
# this allows customers to initialize Metrics() throughout their code base (and middlewares)
# and not get caught by accident with metrics data loss, or data deduplication
# e.g., m1 and m2 add metric ProductCreated, however m1 has 'version' dimension but m2 doesn't
# Result: ProductCreated is created twice as we now have 2 different EMF blobs

There's even a side effect here: since Metrics passes references of all its data structures to the provider, they're both (the Metrics and the provider) sharing the same instances of data. In this case mutating them from Metrics and from the provider is also redundant. And this does not happen if a Metrics is initialized with a provider instance given in the constructor by the user. So there's a lot of confusing complexity here, I'd say most of it possibly unintentional.

(Disclaimer: this was all reviewed and written without the use of AI)

@vishwakt

Copy link
Copy Markdown
Contributor Author

Good catch on clear_metrics, but that call isn't actually redundant. serialize_metric_set only reads dimension_set, never default_dimensions, so the re-registration in clear_metrics is what brings the defaults back after every flush. Drop it and default dimensions would only show up in the first invocation's output (test_log_persist_default_dimensions covers this).

That's also why I changed the warning condition instead of removing the call: the re-registration is doing real work, it just shouldn't be reported to the user as an overwrite. And the old condition was inaccurate on its own anyway, since it warned "the previous value will be overwritten" even when the value hadn't changed. Real overwrites still warn.

The cleaner design would be to merge default_dimensions at serialize time and never store them in dimension_set, but that touches MAX_DIMENSIONS accounting and add_dimension's override semantics, so I kept it out of this PR.

Agreed the Metrics vs provider state sharing is confusing. The class attributes are intentional (shared state across Metrics() instances, see the NOTE at metrics.py:77), but the default provider aliasing those dicts while a custom provider diverges does feel accidental. Probably worth its own issue rather than growing this PR.

@ericbn

ericbn commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Here's my take on this: #8404

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metrics size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: metrics.set_default_dimensions emits PowertoolsUserWarning: Dimension 'Key' has already been added. The previous value will be overwritten.

2 participants