Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #736 +/- ##
==========================================
- Coverage 90.39% 90.02% -0.37%
==========================================
Files 59 60 +1
Lines 17408 17536 +128
==========================================
+ Hits 15736 15787 +51
- Misses 1672 1749 +77 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
6604d25 to
9be718d
Compare
9be718d to
b555718
Compare
|
Good idea to improve the error message there. It can be indeed hard to see why exactly the versions differ. I made a small test with the linux-blueprint by doing this: diff --git a/recipes/rootfs.yaml b/recipes/rootfs.yaml
index f70530f..2f649a8 100644
--- a/recipes/rootfs.yaml
+++ b/recipes/rootfs.yaml
@@ -30,6 +30,7 @@ depends:
use: [tools]
depends:
- graphics::xorg::mkfontscale
+ - libs::openssl-tgt
- core::rootfs-skel
- core::dbusSo I've explicitly depended on I'm afraid that is next to unreadable. Can we have only one header for the package and have all differences in a package next to each other. Also the SHA1 sum is not really helpful. I guess it's the variant-ID, is it? Unless there is a deeper reason, I wouldn't show it. The Is there a limit how deep the recursion goes? In the worst case, recursing the whole dependency tree is not viable. My gut feeling is that recursion should happen only if the package that is currently compared only differs on dependencies. In other words, like above, if there are already differences in variables the recursion should stop. There is no need to look deeper. Likewise, if the dependency differs, it should only be followed if the dependency is derived from the same recipe. Imagine the variants are caused by some conditional dependencies. The fact that there is a new dependency (or has been removed) is enough. But in principle, I really like the idea. I haven't looked at the implementation. I think we first have to get the output minimized... |
14a7cfb to
0b88b4a
Compare
|
Thanks for the input. The output now reuses the existing diff format and shows the diff of the first package which differs. For your example: ❯ bob ls
Parse error: Incompatible variants of package: /amd64-glibc-busybox-x11/rootfs/graphics::xorg::xserver/libs::openssl-tgt vs. /amd64-glibc-busybox-x11/rootfs/libs::openssl-tgt
Processing stack: amd64-glibc-busybox-x11/rootfs
This error is caused by 'libs::openssl-tgt' that is passed upwards via 'provideDeps' from multiple dependencies of 'rootfs'.
These dependencies constitute different variants of 'libs::openssl-tgt' and can therefore not be used in 'rootfs'.
Concrete differences:
/amd64-glibc-busybox-x11/rootfs/graphics::xorg::xserver/libs::openssl-tgt <> /amd64-glibc-busybox-x11/rootfs/libs::openssl-tgt
- buildTools.target-toolchain=devel::cross-toolchain-x86_64-linux-gnu
+ buildTools.target-toolchain=devel::host-compat-toolchain
- buildVars.AR=x86_64-cross-linux-gnu-ar
+ buildVars.AR=x86_64-bob_compat-linux-gnu-ar
- buildVars.AUTOCONF_HOST=x86_64-cross-linux-gnu
+ buildVars.AUTOCONF_HOST=x86_64-bob_compat-linux-gnu
...In my tree the issue I tried to understand was that the output of ❯ bob show --format diff //devel::host-compat-toolchain
- buildTools.target-toolchain=ada::tests::buildall-aarch64/devel::host-compat-toolchain/devel::compat::cross-toolchain
+ buildTools.target-toolchain=arm64-qemu-virt-multicore/devel::host-compat-toolchain/devel::compat::cross-toolchain
- depends[0]=ada::tests::buildall-aarch64/devel::host-compat-toolchain/devel::compat::binutils
+ depends[0]=arm64-qemu-virt-multicore/devel::host-compat-toolchain/devel::compat::binutils
- depends[1]=ada::tests::buildall-aarch64/devel::host-compat-toolchain/devel::compat::gcc-canadian-cross
+ depends[1]=arm64-qemu-virt-multicore/devel::host-compat-toolchain/devel::compat::gcc-canadian-cross
- meta.buildVariantId=10c4757bc31f84bed573beea37e9c1c42a5bd665
+ meta.buildVariantId=799eb3354454fca61ed94e1c336922c05b334cee
- meta.package=ada::tests::buildall-aarch64/devel::host-compat-toolchain
- meta.packageVariantId=f22518aee1a96f967150ecff62cc8aa9b64e954b
+ meta.package=arm64-qemu-virt-multicore/devel::host-compat-toolchain
+ meta.packageVariantId=a6c057918a8a4018c4f817b192c082b944ebda3f
- packageTools.target-toolchain=ada::tests::buildall-aarch64/devel::host-compat-toolchain/devel::compat::cross-toolchain
+ packageTools.target-toolchain=arm64-qemu-virt-multicore/devel::host-compat-toolchain/devel::compat::cross-toolchainwasn't helpful as it only confirmed both variants are using different tools. Now it shows: ❯ bob show --format diff --recursive //devel::host-compat-toolchain
ada::tests::buildall-aarch64/devel::host-compat-toolchain/perl::perl <> arm64-qemu-virt-multicore/devel::host-compat-toolchain/perl::perl
- meta.package=ada::tests::buildall-aarch64/devel::host-compat-toolchain/perl::perl
- meta.packageVariantId=8b902c0f85b4cf0a8363926d1d2d3d6344b34024
+ meta.package=arm64-qemu-virt-multicore/devel::host-compat-toolchain/perl::perl
+ meta.packageVariantId=31ef4d668c588653c9539f1d49e2269c6862567d
- packageVars.PERL_ARCH=x86_64-linux
+ packageVars.PERL_ARCH=aarch64-linuxpointing me to the package built with a cross-target-toolchain while inheriting Also improved the diff output by suppressing dependency differences if they have the same variantId. |
|
Thanks. This is already looking much better. I created a small project to test the feature: If I just do a test-recursive-diff$ bob ls
Parse error: Incompatible variants of package: /root/l1-b/l2 vs. /root/l1-a/l2
Processing stack: root
This error is caused by 'l2' that is passed upwards via 'provideDeps' from multiple dependencies of 'root'.
These dependencies constitute different variants of 'l2' and can therefore not be used in 'root'.
Concrete differences:
l3 <> l3
- meta.packageVariantId=975f054787b7bb6f0fe0e4ee33a8fee5f533214c
+ meta.packageVariantId=0fb28c327f32928bf155e6e25cbf153a1623247a
- packageVars.L3=second
+ packageVars.L3=firstIt shows correctly If you comment out the $ bob show --format diff --recursive root/l1-a/l2 root/l1-b/l2
root/l3 <> root/l1-b/l3
- meta.package=root/l3
+ meta.package=root/l1-b/l3
- meta.packageVariantId=0fb28c327f32928bf155e6e25cbf153a1623247a
+ meta.packageVariantId=975f054787b7bb6f0fe0e4ee33a8fee5f533214c
- packageVars.L3=first
+ packageVars.L3=secondWhat is missing IMHO in both cases, is how we got there. In principle, this is something like an exception trace. How about something like the following? $ bob show --format diff --recursive root/l1-a/l2 root/l1-b/l2
diff root/l1-a/l2 root/l1-b/l2
--- root/l1-a/l2
+++ root/l1-b/l2
- buildTools.asdf=root/l3
+ buildTools.asdf=root/l1-b/l3
- meta.buildVariantId=60d63115983d91f4ca3fcdbf0ad362625eb718b4
+ meta.buildVariantId=50d9b6029dc91e83b00079d06d5a072a7f941036
- meta.package=root/l1-a/l2
+ meta.package=root/l1-b/l2
- meta.packageVariantId=267007b2179c5f2a39512f9653faf17f049b5076
+ meta.packageVariantId=3738715b44e0beda5eb7275f1e2c1ee65ff6a94c
- packageTools.asdf=root/l3
+ packageTools.asdf=root/l1-b/l3
The above package diff was caused by 'buildTools.asdf':
diff root/l3 root/l1-b/l3
--- root/l3
+++ root/l1-b/l3
- meta.package=root/l3
+ meta.package=root/l1-b/l3
- meta.packageVariantId=0fb28c327f32928bf155e6e25cbf153a1623247a
+ meta.packageVariantId=975f054787b7bb6f0fe0e4ee33a8fee5f533214c
- packageVars.L3=first
+ packageVars.L3=second |
99be084 to
3b4200e
Compare
|
I really like the exception trace idea. With this the user can follow the error back to it's root. 👍 With multiple_host_toolchains.diff.gz to the linux-blueprint |
jkloetzke
left a comment
There was a problem hiding this comment.
Just some minor naming thing. Overall, I'm very pleased with the feature. 👍
| """Find the first package with direct differences in variant inputs.""" | ||
| visited = set() | ||
|
|
||
| def compare(a, b, recipeOnly=False): |
There was a problem hiding this comment.
What does recipeOnly mean exactly? And why do we have to filter the fingerprint parts of the variantId? I'm afraid I don't see it (yet).
There was a problem hiding this comment.
The idea was not to blame a tool if only the fingerprint script of the tool or it's deps differs. However, I'm unable to build a concrete example for this setup, so I guess this can be removed.
There was a problem hiding this comment.
I had a deeper look again and it does have its merit. In the variant-id calculation the fingerprint part of tools is ignored. If the "recipe" part of a tool does differ, the recursion needs to ignore the fingerprint parts.
I pushed some fixups that fully align the calcPackageDiffRecursive with CoreStep.getDigest(). Interestingly, Claude noted that tools that differ only in the path or libs are now correctly detected but were not visible in the package diff. The last commit fixes this.
|
I added a small fixup that defers the calculation of the packages until they are actually needed. Please have a look and squash if you think this is good enough. |
dd17066 to
e42042f
Compare
Share package inspection and diff rendering without changing their behavior. This prepares the helpers for use while reporting incompatible variants.
Trace differing dependencies to the first concrete package difference and show the comparisons that led there. Resolve provider references with their ambient context so diagnostic paths remain accurate.
Check that conflicting variants report their differences and retain provider paths through inherited tools.
Follow differing dependencies to the first concrete package difference and show the chain of package comparisons. Reuse the regular diff output and its field selection and common-value options.
Document how recursive comparisons follow dependencies and apply the selected diff options.
Exercise recursive comparisons, dependency equivalence and the supported diff display options.
e42042f to
c0e7885
Compare
The path and library paths of tools are part of the Variant-Id. Show them too, so that variant differences caused by them are visible in the (recursive) package diff. Tools are now shown as map with the providing package, path and libs instead of just the package path.
Debugging:
foo?"is tricky if the difference orients from a deeply nested package. Improve this by adding a recursive variant diff.