Skip to content

main: L0_Unit_Tests_Megatron_Policy_1 fails, blocking functional tests on all PRs #4035

Description

@zhangyimi

This unit test has been failing on main:

FAILED test_megatron_split_state.py::TestTrainMicrobatch::
       test_forwards_model_owned_packing_flags[model-owned-cp-slicing]

A guard in train_microbatch contradicts the test's parameters.

megatron_policy_worker.py:1465:

if self.media_placeholder_token_id is not None or self.model_slices_context_parallel_inputs:
    raise NotImplementedError("train_microbatch does not support multimodal models: ...")

test_megatron_split_state.py:446:

pytest.param(False, False, True, id="model-owned-cp-slicing")
#                         ^^^^ model_slices_context_parallel_inputs=True

The test sets model_slices_context_parallel_inputs to True and asserts it is forwarded, but the guard raises as soon as it sees that flag set. The case cannot pass.

The guard was added by #2957 (da39e1785); the parameter case predates it.

Once the unit tests fail, CI skips the functional stage, so no PR based on main gets functional coverage. Our PR #3566 is blocked on this.

Could a small fix land, or the case be marked xfail for now?

@ZhiyuLi-Nvidia

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions