Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/code_quality_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ jobs:
with:
fetch-depth: 0 # Fetch all history for pre-commit to work

- name: Set up uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

# python:3.9 image has no git; pre-commit requires it.
- name: Install git
run: |
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/functional-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ jobs:
steps:
- uses: actions/checkout@v5

- name: Set up uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

- name: Install xmllint
run: |
apt-get update
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/release-trusted-publisher.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,11 @@ jobs:
git tag ${{ steps.next_version.outputs.version }}
git push origin ${{ steps.next_version.outputs.version }}

- name: Install build dependencies
run: |
python -m pip install --upgrade pip
python -m pip install build
- name: Set up uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

- name: Build package
run: |
python -m build
run: uv build

- name: Upload to PyPI using Trusted Publisher
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/unit-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ jobs:
steps:
- uses: actions/checkout@v5

- name: Set up uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

- name: Install xmllint
run: |
apt-get update
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/update-plugin-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}

- name: Set up uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

- name: Install dependencies
run: ./dev-setup.sh

Expand Down
78 changes: 50 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,31 @@ Node Scraper is a tool which performs automated data collection and analysis for
system debug. For details on what data is collected and analyzed, see the [plugin reference table](docs/PLUGIN_DOC.md).

## Table of Contents
- [Installation](#installation)
- [Install from PyPI](#install-from-pypi)
- [Install from Source](#install-from-source)
- [CLI Usage](#cli-usage)
- [Execution Methods](#execution-methods)
- [Example: Remote Execution](#example-remote-execution)
- [Example: connection_config.json](#example-connection_configjson)
- [Subcommands](#subcommands)
- ['describe' subcommand](#describe-subcommand)
- ['run-plugins' sub command](#run-plugins-sub-command)
- ['gen-plugin-config' sub command](#gen-plugin-config-sub-command)
- ['compare-runs' subcommand](#compare-runs-subcommand)
- ['summary' sub command](#summary-sub-command)
- [Configs](#configs)
- [Global args](#global-args)
- [Plugin config: `--plugin-configs` command](#plugin-config---plugin-configs-command)
- [Reference config: `gen-reference-config` command](#reference-config-gen-reference-config-command)
- **Extending Node Scraper (integration & external plugins)** → See [EXTENDING.md](EXTENDING.md)
- **Full view of the plugins with the associated collectors & analyzers as well as the commands
invoked by collectors** -> See [docs/PLUGIN_DOC.md](docs/PLUGIN_DOC.md)
- [Node Scraper](#node-scraper)
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Install from PyPI](#install-from-pypi)
- [Install from Source](#install-from-source)
- [1. Install uv](#1-install-uv)
- [2. Virtual Environment (Optional)](#2-virtual-environment-optional)
- [3. Install from Source (Required)](#3-install-from-source-required)
- [4. Git Hooks (Optional)](#4-git-hooks-optional)
- [CLI Usage](#cli-usage)
- [Execution Methods](#execution-methods)
- [Example: Remote Execution](#example-remote-execution)
- [Example: connection\_config.json](#example-connection_configjson)
- [Subcommands](#subcommands)
- [**'describe' subcommand**](#describe-subcommand)
- [**'run-plugins' sub command**](#run-plugins-sub-command)
- [**'gen-plugin-config' sub command**](#gen-plugin-config-sub-command)
- [**'compare-runs' subcommand**](#compare-runs-subcommand)
- [**'show-redfish-oem-allowable' subcommand**](#show-redfish-oem-allowable-subcommand)
- [**RedfishEndpointPlugin**](#redfishendpointplugin)
- [**'summary' sub command**](#summary-sub-command)
- [Configs](#configs)
- [Global args](#global-args)
- [Plugin config: **'--plugin-configs' command**](#plugin-config---plugin-configs-command)
- [Reference config: **'gen-reference-config' command**](#reference-config-gen-reference-config-command)

## Installation
### Install from PyPI
Expand All @@ -32,6 +37,12 @@ Node Scraper is published on [PyPI](https://pypi.org/project/amd-node-scraper/)
pip install amd-node-scraper
```

With [uv](https://docs.astral.sh/uv/):

```sh
uv pip install amd-node-scraper
```

Use a virtual environment if you prefer. After installation, confirm the CLI is available:

```sh
Expand All @@ -40,29 +51,40 @@ node-scraper --help

### Install from Source
Node Scraper requires Python 3.9+ for installation. After cloning this repository,
call dev-setup.sh script with 'source'. This script creates an editable install of Node Scraper in
a python virtual environment and also configures the pre-commit hooks for the project.
run `dev-setup.sh` with `source`. This script uses [uv](https://docs.astral.sh/uv/) to create a
Python 3.9+ virtual environment, perform an editable install, and configure pre-commit hooks.

```sh
source dev-setup.sh
```

Alternatively, follow these manual steps:

### 1. Virtual Environment (Optional)
### 1. Install uv
```sh
curl -LsSf https://astral.sh/uv/install.sh | sh
```

### 2. Virtual Environment (Optional)
```sh
python3 -m venv venv
uv venv venv --python 3.9
source venv/bin/activate
```
On Debian/Ubuntu, you may need: `sudo apt install python3-venv`
On Debian/Ubuntu without uv, you can use `python3.9 -m venv venv` instead.

### 2. Install from Source (Required)
### 3. Install from Source (Required)
```sh
python3 -m pip install --editable .[dev] --upgrade
uv pip install -e ".[dev]"
```
This installs Node Scraper in editable mode with development dependencies. To verify: `node-scraper --help`

### 3. Git Hooks (Optional)
Equivalent using pip:

```sh
python3 -m pip install --editable .[dev] --upgrade
```

### 4. Git Hooks (Optional)
```sh
pre-commit install
```
Expand Down
22 changes: 17 additions & 5 deletions dev-setup.sh
Original file line number Diff line number Diff line change
@@ -1,16 +1,28 @@
#!/usr/bin/env bash

# Create venv if not already present
if [ ! -d "venv" ]; then
python3 -m venv venv
if ! which uv >/dev/null 2>&1; then
if ! which curl >/dev/null 2>&1; then
wget -qO- https://astral.sh/uv/install.sh | sh
else
curl -LsSf https://astral.sh/uv/install.sh | sh
fi
export PATH="$HOME/.local/bin:$PATH"
fi

# Use system certificate store so uv works behind corporate SSL proxies
export UV_SYSTEM_CERTS=1

# Create venv if not already present or if Python version is below 3.9
if [ ! -d "venv" ] || ! venv/bin/python -c 'import sys; exit(0 if sys.version_info >= (3, 9) else 1)' 2>/dev/null; then
uv venv venv --python python3 --no-python-downloads
fi

# Activate the desired venv
source venv/bin/activate

python3 -m pip install --editable .[dev] --upgrade
uv pip install -e ".[dev]"

# Only install pre-commit hooks if not in CI environment
if [ -z "$CI" ]; then
if [ -z "${CI:-}" ]; then
pre-commit install
fi
Loading