Skip to content

[Bug]: --shuffle [seed] consumes the following test filter as the seed #43156

Closed

Description

Version

main (1.65.0-next, 6936b23)

Steps to reproduce

npx playwright test --shuffle tests/login.spec.ts

Verified the parsing with commander 15.0.0 (the version pinned in package.json):

const { Command } = require('commander');
const p = new Command();
p.command('test [test-filter...]').option('--shuffle [seed]').action((f, o) => console.log(f, o));
p.parse(['node', 'x', 'test', '--shuffle', 'tests/login.spec.ts']);
// => [] { shuffle: 'tests/login.spec.ts' }

Expected behavior

Only tests/login.spec.ts runs, in shuffled order, or at least an error/warning that the seed looks like a file path.

Actual behavior

The whole suite runs with seed "tests/login.spec.ts"; the filter is silently dropped. (npx playwright test tests/login.spec.ts --shuffle works as expected.)

Additional context

Option added in #42877 (packages/playwright/src/program.ts). Possible fixes: a separate --shuffle-seed <seed> option, or validating the seed (e.g. numeric only) and treating anything else as a filter. --only-changed [ref] has the same commander pattern.

Environment

Playwright main @ 6936b230a (1.65.0-next). Found by code review of recent commits; not OS/browser specific.

Activity

  1. pavelfeldman commented on Oct 6, 2026

    @pavelfeldman
    Member

    Does not look like a real issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions