Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ aren't shipped as actions; they stay copy-paste
| [`python/lint`](./python/lint) | Lint + type-check a Python project with ruff (`check` + `format`) and ty, via `uv`. |
| [`python/test`](./python/test) | Install with `uv` and run pytest, optionally pinned to a Python version. |
| [`python/publish`](./python/publish) | Build with `uv` and publish to a package index (devpi by default), gated to `main` by default. |
| [`python/flyte-deploy`](./python/flyte-deploy) | Deploy a Flyte environment with the `flyte` CLI (secret from Vault); `--dry-run` on pull requests. |
| [`python/flyte-deploy`](./python/flyte-deploy) | Deploy a Flyte environment with the `flyte` CLI (client credentials as inputs); `--dry-run` on pull requests. |
| [`mise-setup`](./mise-setup) | Provision a repo's tools with mise into a per-repository dir (cached installs, prune of unreferenced versions), so later steps can call them. |
| [`pr-approve-and-merge`](./pr-approve-and-merge) | Approve PRs as github-actions[bot] and optionally merge them as a GitHub App, so the merge triggers push workflows. |

Expand Down
65 changes: 32 additions & 33 deletions python/flyte-deploy/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# python/flyte-deploy

Deploy a Flyte environment with the `flyte` CLI (run through `uv`). The deploy
client id, oauth scope and secret come from Vault; the action writes them into a
throwaway config file (merged on top of the repo's own, if it has one) so the
CLI authenticates with the client credentials grant instead of falling back to
its default browser (PKCE) flow. Everything else the CLI needs — endpoint, org,
project, domain, version — is passed as a command option, so a repo needs no
`flyte` config file in the checkout. `domain` takes a list, so one step can
client id, oauth scope and secret come from the inputs; the action writes them
into a throwaway config file (merged on top of the repo's own, if it has one) so
the CLI authenticates with the client credentials grant instead of falling back
to its default browser (PKCE) flow. Everything else the CLI needs — endpoint,
org, project, domain, version — is passed as a command option, so a repo needs
no `flyte` config file in the checkout. `domain` takes a list, so one step can
deploy the same version to several domains (`development,production`).

On a pull request the deploy runs with `--dry-run` (nothing is registered); on
Expand All @@ -21,30 +21,27 @@ always get `-<short-sha>` appended so they never collide with a main deploy.
- `actions/checkout@v7` before this step.
- The project's environment must provide the `flyte` package (`uv run` resolves
it from the checkout's `pyproject.toml`/lockfile).
- `permissions: id-token: write` on the calling job. The action fetches the
client secret from Vault (`vault1.dlh-k8s.com:8200`, JWT auth, role
`ci-actions`), which needs the job's OIDC token. A composite action cannot
declare permissions, so this is on the caller.
- The deploy client's credentials, passed from the org secrets
`FLYTE_DEPLOY_CLIENT_ID`, `FLYTE_DEPLOY_CLIENT_SECRET` and
`FLYTE_DEPLOY_CLIENT_SCOPE`.

## Inputs

| Input | Required | Default | Description |
| --------------------------- | -------- | -------------------------------------------- | ------------------------------------------------------------------------------------------- |
| `environment` | yes | — | Trailing args of `flyte deploy`, e.g. `my_pkg/workflows.py` or `--all my_pkg/workflows.py`. |
| `project` | yes | — | Flyte project to deploy into. |
| `domain` | no | `development` | Flyte domain(s), as a comma/space/newline separated list (`development,production`). |
| `endpoint` | no | `dns:///flyte.apps.dlh-k8s.com` | Flyte admin endpoint. |
| `org` | no | `flyte` | Flyte organization. |
| `version` | no | `''` | Version to deploy; empty means the commit sha. |
| `register-on-pr` | no | `false` | Register for real on pull requests; the short sha is always appended to the version. |
| `dry-run` | no | `auto` | `auto` (dry run on pull requests unless `register-on-pr`), `true`, or `false`. |
| `extra-args` | no | `''` | Extra args appended to `flyte deploy` before the environment. |
| `working-directory` | no | `.` | Directory to run the deploy from. |
| `vault-url` | no | `https://vault1.dlh-k8s.com:8200` | Vault address. |
| `vault-role` | no | `ci-actions` | Vault JWT role. |
| `vault-secret-clientid` | no | `kv/data/k8s/flyte/oauth deployClientId` | Vault secret holding the client id for oauth, as `<path> <key>`. |
| `vault-secret-clientsecret` | no | `kv/data/k8s/flyte/oauth deployClientSecret` | Vault secret holding the client secret for oauth, as `<path> <key>`. |
| `vault-secret-clientscope` | no | `kv/data/k8s/flyte/oauth deployClientScope` | Vault secret holding the oauth scope requested for the deploy, as `<path> <key>`. |
| Input | Required | Default | Description |
| ------------------- | -------- | ------------------------------- | ------------------------------------------------------------------------------------------- |
| `environment` | yes | — | Trailing args of `flyte deploy`, e.g. `my_pkg/workflows.py` or `--all my_pkg/workflows.py`. |
| `project` | yes | — | Flyte project to deploy into. |
| `domain` | no | `development` | Flyte domain(s), as a comma/space/newline separated list (`development,production`). |
| `endpoint` | no | `dns:///flyte.apps.dlh-k8s.com` | Flyte admin endpoint. |
| `org` | no | `flyte` | Flyte organization. |
| `version` | no | `''` | Version to deploy; empty means the commit sha. |
| `register-on-pr` | no | `false` | Register for real on pull requests; the short sha is always appended to the version. |
| `dry-run` | no | `auto` | `auto` (dry run on pull requests unless `register-on-pr`), `true`, or `false`. |
| `extra-args` | no | `''` | Extra args appended to `flyte deploy` before the environment. |
| `working-directory` | no | `.` | Directory to run the deploy from. |
| `client-id` | yes | — | OAuth client id of the deploy client. |
| `client-secret` | yes | — | OAuth client secret of the deploy client. |
| `client-scope` | no | `''` | OAuth scope for the client credentials grant; empty uses what the endpoint advertises. |

## Outputs

Expand All @@ -61,7 +58,6 @@ jobs:
deploy:
runs-on: self-hosted
permissions:
id-token: write # Vault JWT auth
contents: read
steps:
- uses: actions/checkout@v7
Expand All @@ -71,6 +67,9 @@ jobs:
project: flyte-otel-demo
domain: production
environment: --all flyte_otel_demo/hello_otel.py
client-id: ${{ secrets.FLYTE_DEPLOY_CLIENT_ID }}
client-secret: ${{ secrets.FLYTE_DEPLOY_CLIENT_SECRET }}
client-scope: ${{ secrets.FLYTE_DEPLOY_CLIENT_SCOPE }}
```

Run the same workflow on pull requests to get a `--dry-run` validation of the
Expand Down Expand Up @@ -99,7 +98,6 @@ jobs:
deploy:
runs-on: self-hosted
permissions:
id-token: write # Vault JWT auth
contents: read
steps:
- uses: actions/checkout@v7
Expand All @@ -117,6 +115,9 @@ jobs:
domain:
${{ github.event_name == 'pull_request' && 'development' ||
'development,production' }}
client-id: ${{ secrets.FLYTE_DEPLOY_CLIENT_ID }}
client-secret: ${{ secrets.FLYTE_DEPLOY_CLIENT_SECRET }}
client-scope: ${{ secrets.FLYTE_DEPLOY_CLIENT_SCOPE }}
```

On a pull request that is a `--dry-run` against `development` only; on `main`
Expand All @@ -132,7 +133,5 @@ the same version is registered to `development` and then `production`.
`admin.clientSecretEnvVar`; the secret itself stays in the environment and is
never written to disk. Env vars alone are not enough here — `Config.auto()`
only reads the environment once it has found a config file.
- The secret is fetched with `exportEnv: false` and handed only to the deploy
step, so it does not end up in the environment of the caller's later steps.
- Point a repo at different credentials by changing `vault-secret` (or the value
behind that Vault key), not by adding a repo secret.
- The credentials are handed only to the deploy step, so they do not end up in
the environment of the caller's later steps.
62 changes: 17 additions & 45 deletions python/flyte-deploy/action.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
name: "Flyte deploy"
description:
"Deploy a Flyte environment with the flyte CLI (via uv), authenticating with a
client secret read from Vault; pull requests deploy with --dry-run unless
register-on-pr is set."
client id and secret passed as inputs; pull requests deploy with --dry-run
unless register-on-pr is set."
author: "DataLabHell"

branding:
icon: "send"
color: "blue"

# Installs uv (via setup-uv), reads the deploy client secret from Vault and runs
# the flyte CLI. Everything the CLI needs is passed as a command option rather
# Installs uv (via setup-uv) and runs the flyte CLI with the client credentials
# from the inputs. Everything the CLI needs is passed as a command option rather
# than a config file, so a repo only has to say what it deploys. The caller owns
# the job (runner, checkout).
inputs:
Expand Down Expand Up @@ -65,29 +65,18 @@ inputs:
description: "Directory to run the deploy from."
required: false
default: "."
vault-url:
description: "Vault address the client secret is read from."
required: false
default: "https://vault1.dlh-k8s.com:8200"
vault-role:
description: "Vault JWT role used for the OIDC login."
required: false
default: "ci-actions"
vault-secret-clientsecret:
description:
'Vault secret holding the deploy client secret, as "<path> <key>".'
required: false
default: "kv/data/k8s/flyte/oauth deployClientSecret"
vault-secret-clientid:
description: 'Vault secret holding the deploy client id, as "<path> <key>".'
required: false
default: "kv/data/k8s/flyte/oauth deployClientId"
vault-secret-clientscope:
client-id:
description: "OAuth client id of the deploy client."
required: true
client-secret:
description: "OAuth client secret of the deploy client."
required: true
client-scope:
description:
'Vault secret holding the OAuth scope requested for the client credentials
grant, as "<path> <key>".'
"OAuth scope requested for the client credentials grant. Empty = whatever
the admin endpoint advertises."
required: false
default: "kv/data/k8s/flyte/oauth deployClientScope"
default: ""

outputs:
version:
Expand All @@ -106,23 +95,6 @@ runs:
- name: Install uv
uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3

# Authenticates with the job's OIDC token, so the caller needs
# "permissions: id-token: write" (see the README). exportEnv is off: the
# secret stays in this step's outputs instead of leaking into the
# environment of every later step in the caller's job.
- name: Get Flyte client secret
id: vault
uses: hashicorp/vault-action@892a26828f195e65540a40b4768ae4571f51ebfc # v4.0.0
with:
url: ${{ inputs.vault-url }}
method: jwt
role: ${{ inputs.vault-role }}
exportEnv: "false"
secrets: |
${{ inputs.vault-secret-clientid }} | FLYTE_CLIENT_ID;
${{ inputs.vault-secret-clientsecret }} | FLYTE_CLIENT_SECRET;
${{ inputs.vault-secret-clientscope }} | FLYTE_CLIENT_SCOPE

- name: Deploy to Flyte
id: deploy
shell: bash
Expand All @@ -141,6 +113,6 @@ runs:
# On pull requests GITHUB_SHA is the temporary merge commit of
# refs/pull/<n>/merge, not the commit that was pushed.
COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
FLYTE_CLIENT_ID: ${{ steps.vault.outputs.FLYTE_CLIENT_ID }}
FLYTE_CLIENT_SECRET: ${{ steps.vault.outputs.FLYTE_CLIENT_SECRET }}
FLYTE_CLIENT_SCOPE: ${{ steps.vault.outputs.FLYTE_CLIENT_SCOPE }}
FLYTE_CLIENT_ID: ${{ inputs.client-id }}
FLYTE_CLIENT_SECRET: ${{ inputs.client-secret }}
FLYTE_CLIENT_SCOPE: ${{ inputs.client-scope }}
7 changes: 3 additions & 4 deletions python/flyte-deploy/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ for required in ENVIRONMENT PROJECT DOMAIN ENDPOINT ORG; do
done

if [ -z "${FLYTE_CLIENT_SECRET:-}" ]; then
echo "::error::FLYTE_CLIENT_SECRET is empty — the Vault step returned nothing for vault-secret" >&2
echo "::error::input 'client-secret' is required" >&2
exit 1
fi

if [ -z "${FLYTE_CLIENT_ID:-}" ]; then
echo "::error::FLYTE_CLIENT_ID is empty — the Vault step returned nothing for vault-secret-clientid" >&2
echo "::error::input 'client-id' is required" >&2
exit 1
fi

Expand Down Expand Up @@ -64,8 +64,7 @@ config = config if isinstance(config, dict) else {}
admin = config.setdefault("admin", {})
admin["clientId"] = os.environ["FLYTE_CLIENT_ID"]
# Entra ID only issues a token for the Flyte API when the grant asks for that
# API's scope, so the scope is kept in Vault next to the id and secret. Empty
# means "use whatever the admin endpoint advertises".
# API's scope. Empty means "use whatever the admin endpoint advertises".
admin["clientSecretEnvVar"] = "FLYTE_CLIENT_SECRET"
admin["authType"] = "ClientSecret"
if os.environ["FLYTE_CLIENT_SCOPE"]:
Expand Down
Loading