fix: enforce pip require-hashes and provide verified dependency hashes

[sriram-quad]

Update the benchmark installation script to enforce pip hash verification and supply cryptographically locked dependency manifest files.
Vulnerability: pip download/install detected in your repo affecting rpc://cloudhpc/hpc-toolkit-blueprints

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a security vulnerability related to insecure package installation by enforcing strict hash verification for Python dependencies. By migrating to a locked requirements manifest, the installation process is now protected against tampering, ensuring that only verified versions of packages are deployed in the hpc-toolkit-blueprints environment.

Highlights

• Security Hardening: Updated the benchmark installation script to enforce pip hash verification using the --require-hashes flag.
• Dependency Management: Introduced a cryptographically locked dependency manifest (requirements.txt) and its source (requirements.in) to ensure supply chain integrity.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request aims to enhance security by enforcing hash verification during dependency installation. It introduces a pinned requirements.txt with hashes for jinja2 and markupsafe, and updates install-hpl-dependencies.sh to use pip3 install --require-hashes. However, feedback highlights that this command will fail because the targeted /opt/ramble/requirements.txt is cloned from an upstream repository and lacks hash annotations. To resolve this and keep the benchmark script self-contained, it is recommended to write the required dependencies and their hashes inline using a heredoc.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces dependency hashing for Python packages in the HPL benchmark by adding a hashed requirements manifest and updating the installation script to enforce hash verification. However, a critical bug was identified in the installation script: using $(dirname "$0") to locate the requirements file will fail when the script is executed as a payload on compute nodes via srun. To resolve this and ensure the benchmark script remains self-contained, it is recommended to inline the requirements directly within the script using a heredoc.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request enhances security in the HPL dependencies installation script by defining Python requirements with cryptographic hashes inline and installing them using the --require-hashes flag. It also introduces requirements.in and requirements.txt files. However, the current changes are incomplete and will break Ramble because they only specify jinja2 and markupsafe, omitting other critical dependencies such as jsonschema and ruamel.yaml. To resolve this, all of Ramble's dependencies must be compiled with hashes and included in the script.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request implements a security fix in the HPL dependencies installation script by pinning Python requirements with cryptographic hashes and enforcing them via the --require-hashes flag during installation. It also adds the corresponding requirements.in and requirements.txt files. However, the generated requirements.txt is missing hashes for several wheels and platforms (such as jinja2 and ruamel-yaml-clib), which will cause installation failures on other Python versions or architectures. It is recommended to regenerate this file using a modern version of pip-compile with internet access to ensure all necessary hashes are included.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request enhances the security of the h4d-hpc-benchmarks community component by pinning Python dependencies with cryptographic hashes. It introduces a requirements.in file and an autogenerated requirements.txt. The install-hpl-dependencies.sh script has been updated to write these requirements inline via a heredoc and install them using pip3 with the --require-hashes flag to comply with vulnerability scanner policies. Additionally, the heredoc delimiter in the installation script was changed to PAYLOAD_EOF to prevent conflicts. I have no feedback to provide as the changes are correct and follow security best practices.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new requirements.in file for the community/h4d-hpc-benchmarks directory, defining a set of Python package dependencies including Google Cloud libraries, data science tools like pandas and matplotlib, and utility packages. I have no feedback to provide as there are no review comments to evaluate.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request updates the HPL dependency installation script to secure Python package installation by embedding a pinned requirements.txt with cryptographic hashes and using --require-hashes with pip3. It also introduces a requirements.in file and quotes various shell variables for robustness. However, two issues were identified in the review: first, the updated srun command contains critical errors, including running a non-existent script on a single node without sudo or partition parameters; second, the Spack repository should be pinned to a stable release tag (such as v0.22.0) rather than cloning the unstable default branch.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request enhances the security and robustness of the HPL dependency installation script by introducing a pinned requirements.txt with cryptographic hashes to satisfy vulnerability scanner policies, along with quoting variables and adding ShellCheck directives. It also adds a requirements.in file. The review feedback recommends pinning the Spack repository clone to a specific release tag to avoid breaking changes from upstream updates, and correcting a version mismatch for jsonschema between requirements.in and requirements.txt.

[sriram-quad]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds Python dependency requirements (requirements.in and requirements.txt) for the h4d-hpc-benchmarks community benchmark. However, a critical issue was identified in install-hpl-dependencies.sh, where a large portion of the installation logic and key variables (such as $PARTITION, $NODE_COUNT, and $INSTALL_SCRIPT) were accidentally deleted, which will cause the script to fail. It is recommended to restore the deleted installation logic and update the pip installation step to use the new requirements.txt with the --require-hashes flag.