Skip to content

Fix dropped dependencies when new extras are requested in a recompile - #145

Open
abrisco wants to merge 2 commits into
periareon:mainfrom
abrisco:external
Open

abrisco wants to merge 2 commits into
periareon:mainfrom
abrisco:external

Conversation

@abrisco

@abrisco abrisco commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

A solution file only records the extras that were active when it was compiled. SolutionRepository rebuilt each project's metadata from those annotations, but answered requires(<any other extra>) with silence instead of admitting it did not know. Recompiling against an existing snapshot and asking for an extra that was not already pinned therefore dropped that extra's dependencies.

Against this repo's own lockfile, before this change:

echo 'requests[socks]' > in.txt
req-compile --no-index --solution 3rdparty/requirements.linux_x86_64.txt in.txt
requests==2.32.3           # in.txt ([socks])
 ...                        # PySocks missing

With Bazel this means py_reqs_compiler writes a lockfile missing packages and py_reqs_solution_test passes on it, so nothing catches the error.

This change fixes this issue.

@abrisco
abrisco requested a review from sputt September 17, 2026 01:06
@sputt

sputt commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator
  1. On req_compile/repos/solution.py where _known_extras is updated from constraint_req.extras:

This can mark more extras as known than the reconstructed metadata actually describes. For a valid annotation like child==1 # parent[x1,x2], this records both x1 and x2, but _create_metadata_req() below uses next(iter(parse_requirement(name).extras)) and reconstructs child under only one of them.

A later recompile asking for the other extra can therefore accept the solution candidate and still silently drop child.

Please preserve all extras in the reconstructed marker (e.g. OR them), or only mark extras that were actually reconstructed, and add a regression covering this case.

  1. On the _known_extras initialization / load_from_file() path:

_known_extras is state for one loaded solution, but load_from_file() replaces self.solution without clearing this map.

If a SolutionRepository instance is reused to load a different solution, extras from the previous file can remain marked as known, causing get_candidates() to trust metadata that the new file does not actually describe.

Please reset _known_extras when starting a new load and add a reload regression.

@sputt sputt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted as a top level comment

@abrisco

abrisco commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

Updated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants