Repository navigation
Preserve ModelManager metric_tag across RunInference side-input model updates - #40714
Conversation
|
R: @tvalentyn |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #40714 +/- ##
========================================
Coverage 80.04% 80.05%
========================================
Files 490 490
Lines 82916 83031 +115
========================================
+ Hits 66373 66467 +94
- Misses 16543 16564 +21
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| self.models = models | ||
| self.use_model_manager = not isinstance(models, list) | ||
| self.model_tag = model_tag | ||
| self.metric_tag = metric_tag or model_tag |
There was a problem hiding this comment.
could we clarify in the docstring the difference between these two concepts?
…te; document model_tag vs metric_tag
…note that model updates should only change weights
|
is this ready to merge? thanks! |
Yep thanks! |
Summary
When
RunInferenceupdates its model via side input (ModelMetadata),_RunInferenceDoFn._load_modelacquires the new model under the updatedmodel_idtag (self._cur_tag) soModelManagercaches and evicts the new model version separately from the previous version, while theBeamML_RunInferencetransform annotation remains fixed at construction time (self._model_tag/model_identifier).This PR passes
metric_tag=self._model_tagthrough_SharedModelWrappertoModelManager.acquire_model(..., metric_tag=...), and mapsmetric_tagto the active version tag inModelManagerandResourceEstimatorsonum_loaded_models_<metric_tag>andmemory_estimate_mb_<metric_tag>continue reporting under the originalmetric_tagacross runtime side-input model updates.Testing
test_run_inference_dofn_model_update_acquires_and_releases_matching_taginbase_test.pyto verifymetric_tagpreservation across model updates.test_model_manager_metrics_with_metric_taginmodel_manager_test.py.