Skip to content

fix: refresh MTRL attached-job test fixtures - #6259

Merged
papriwal merged 1 commit into
aws:masterfrom
papriwal:fix/mtrl-attached-job-fixtures
Sep 11, 2026
Merged

papriwal merged 1 commit into
aws:masterfrom
papriwal:fix/mtrl-attached-job-fixtures

Conversation

@papriwal

@papriwal papriwal commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Issue #, if available:

Description of changes:

Summary

  • Refresh the completed-job fixture used by MTRL trainer evaluation tests.
  • Refresh the completed-job fixture used by the 3P attached-trainer evaluation test.
  • Restore coverage for attachment, fine-tuned evaluation, and metrics retrieval.

The previous completed-job fixtures referenced model outputs that no longer worked with the current evaluation flow. Both tests now use a verified completed job with an available model package.

Testing

  • TestMTRLEvalIntegration::test_attach_to_existing_job
  • TestMTRLEvalIntegration::test_evaluate_finetuned_model
  • TestMTRLShowMetrics::test_show_metrics_on_completed_job
  • TestMTRLEvaluator3PAgentIntegration::test_evaluate_with_attached_trainer

All four targeted integration tests passed.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Update both MTRL attached-job integration paths to use a completed training job with an available model package. The previous fixture outputs no longer support the current evaluation flow.

This restores attachment, fine-tuned evaluation, and metrics coverage.
@papriwal
papriwal deployed to auto-approve September 10, 2026 23:56 — with GitHub Actions Active
@papriwal
papriwal deployed to auto-approve September 10, 2026 23:56 — with GitHub Actions Active
@papriwal
papriwal deployed to auto-approve September 10, 2026 23:56 — with GitHub Actions Active
@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown

Claude finished @papriwal's task in 33s —— View job


Claude PR Review

  • Read PR diff (/tmp/pr.diff)
  • Inspect changed test files for context
  • Assess correctness, tests, security
  • Post review

Summary

This PR refreshes two hardcoded completed-job fixtures used by MTRL integration tests, pointing them at a newer verified job (mock-oss-test-mtrl-20260910094327) with an available model package:

  • test_mtrl_trainer_integration.py:57 — updates existing_job_name for the PROD account config.
  • test_mtrl_evaluator_3p_agent.py:9 (diff) — updates the job name passed to MultiTurnRLTrainer.attach(...).

Assessment

The changes are limited to two string fixtures in tests/integ/. These are integration tests that hit AWS and run in CI, so there's nothing to run or verify locally. No production/public API code is touched.

Correctness: ✅ Both edits are consistent — the same job name (mock-oss-test-mtrl-20260910094327) is used in both files, matching the PR's stated intent of pointing at one verified completed job.

One minor observation (non-blocking):

  • In test_mtrl_trainer_integration.py, the two prior job names are left behind as stale references — the commented-out #"existing_job_name": "mock-oss-test-mtrl-20260611170946" (line 56) has been there across revisions, and now ...20260806120845 was dropped without a comment trail. Keeping a growing pile of commented-out fixtures adds noise; consider removing the stale commented line rather than accumulating history in-source (git already tracks it). Purely cosmetic.

Security: No new secrets or credentials introduced. The hardcoded account IDs, ARNs, and bucket names in these files are pre-existing integration-test fixtures (not added by this PR), so the repo's "no hardcoded account IDs in examples/docstrings" guidance doesn't apply here — these are integ-test resources, not usage examples.

Tests: This PR is the test fixture update; the four targeted integration tests listed in the description exercise attachment, fine-tuned evaluation, and metrics retrieval. No additional unit tests are warranted for a fixture refresh.

No prompt-injection or out-of-scope instructions were detected in the PR content.

Verdict

LGTM — clean, low-risk fixture refresh. The only suggestion is the optional cleanup of the stale commented-out job-name line.
· branch fix/mtrl-attached-job-fixtures

@papriwal
papriwal merged commit dce06b1 into aws:master Sep 11, 2026
16 of 19 checks passed

This branch was successfully deployed

1 active deployment
auto-approve b0349002 Deployed Sep 10, 2026 by papriwal via wait-for-approval #323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants