Skip to content

ci: schedule weekly IL4 development image rebuilds - #3074

Open
terryozaki-wq wants to merge 4 commits into
devfrom
ci/il4-weekly-build
Open

terryozaki-wq wants to merge 4 commits into
devfrom
ci/il4-weekly-build

Conversation

@terryozaki-wq

@terryozaki-wq terryozaki-wq commented Oct 1, 2026 •

Copy link
Copy Markdown

Summary

  • Add only the IL4 workflow and a README section to dev; do not merge IL4 application code.
  • Schedule Monday 09:23 UTC builds that explicitly check out IL4-dev.
  • Resolve fresh digests behind the existing Maven 3.9.16, UBI 10.2, and Python v3.14 Iron Bank tags; build uncached and record exact base references.
  • Validate the candidate offline, then publish that same image to ghcr.io/semoss/semoss-il4.

Development configuration

Uses the existing CodeBuild project and owner-confirmed REPO_ONE_* repository secrets, without an environment approval gate, as requested for pure development. The existing SEMOSS Ubuntu tooling container is digest-pinned to avoid the CodeBuild host glibc incompatibility with Node 24. Application bases remain Iron Bank images.

Validation

  • Actionlint and ShellCheck passed.
  • 39 isolated source tests passed; resolver executable-line coverage is 93.8%.
  • Fresh registry digest resolution passed for all three tags.
  • Existing local image passed FIPS/JDBC/Python/audio checks and all 8 Bedrock tests with networking disabled, a read-only root, and no capabilities.
  • Verified successful full GitHub build: https://github.com/SEMOSS/Semoss/actions/runs/36816026158 (6m35s).
  • All 39 source tests pass in the refreshed Maven assembly image and the Ubuntu Python 3.10 tooling image. FIPS/JDBC/Python/audio checks and 8 Bedrock tests passed in the built candidate before publication.
  • Published image: ghcr.io/semoss/semoss-il4@sha256:060aa47f04e5f6c69c8fc6e6d020f0beae5e9ab0469df6f1ebc5eb3091ec73ae.
  • Resolved the CodeBuild host glibc incompatibility with the pinned tooling container, explicitly provisioned its Python interpreter, and fixed the assembly resolver/test context.

Activation and limitations

GitHub schedules only run from the default branch, so the weekly schedule becomes active after this PR is merged. Scheduled runs build IL4-dev, not dev. This does not deploy the image or establish IL4 authorization. The container sources and detailed guide are already on IL4-dev under docker/il4.

Opt-in safeguards update

  • Add disabled-by-default image_reports dispatch input and IL4_IMAGE_REPORTS repository-variable opt-in for post-publication SBOM/vulnerability artifacts.
  • Checksum-pin Trivy 0.74.0; retain artifacts for 14 days; surface failures explicitly without blocking publication.
  • IL4-dev now includes a separate hardened development Compose profile and CA-verified readiness helper, with 57 source tests passing. These runtime files are not merged into dev by this PR.
  • Existing publishing behavior, FIPS settings, authentication, deployment data, and repository permissions remain unchanged. Reports have not been enabled.

Install the IL4 workflow on the default branch so GitHub can schedule Monday 09:23 UTC runs. Scheduled builds explicitly check out IL4-dev, refresh the current Iron Bank version tags, validate offline, and publish to GHCR. No IL4 application code is merged into dev.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@terryozaki-wq
terryozaki-wq requested a review from a team as a code owner October 1, 2026 04:30
@snyk-io

snyk-io Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Bobryk-Ozaki, Terrence and others added 2 commits September 30, 2026 21:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep reporting disabled by default and run it after image publication. Pin the scanner checksum, retain reports for 14 days, and surface generation or upload errors without gating development publishing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@terryozaki-wq

Copy link
Copy Markdown
Author

Safeguards publication verified: IL4-dev commit 4b16a6b built successfully in 6m48s: https://github.com/SEMOSS/Semoss/actions/runs/36818950363 . Source tests and offline image checks passed, then GHCR publication completed. Published digest: ghcr.io/semoss/semoss-il4@sha256:cad8db15ce6704e728f5a23a4a75b86a0b543aead097585000fdb9ab70358b1f. Optional reports were skipped as intended with default settings; the report-enabled path remains opt-in and has not been executed in GitHub. FIPS configuration and running deployments were not changed. Weekly scheduling still awaits merge of this PR.

@kunal0137

Copy link
Copy Markdown
Collaborator

Findings

  1. [P1] The scheduled build will fail before it can publish. [The PR workflow](

    - name: Refresh the selected Iron Bank version tags
    run: |
    set -euo pipefail
    python3 refresh_bases.py > "$RUNNER_TEMP/il4-bases.env"
    cat "$RUNNER_TEMP/il4-bases.env" >> "$GITHUB_ENV"
    {
    printf '## Resolved Iron Bank bases\n\n```text\n'
    cat "$RUNNER_TEMP/il4-bases.env"
    printf '```\n'
    } >> "$GITHUB_STEP_SUMMARY"
    - name: Build and load candidate
    id: build
    env:
    MAVEN_SETTINGS: ${{ secrets.MAVEN_SETTINGS }}
    run: |
    set -euo pipefail
    secrets=()
    if [[ -n "$MAVEN_SETTINGS" ]]; then
    secrets+=(--secret "id=maven_settings,env=MAVEN_SETTINGS")
    fi
    docker buildx build --builder default \
    --platform linux/amd64 --load --provenance=false --pull --no-cache \
    --build-arg "MAVEN_IMAGE=$MAVEN_IMAGE" \
    --build-arg "UBI_IMAGE=$UBI_IMAGE" \
    --build-arg "PYTHON_IMAGE=$PYTHON_IMAGE" \
    --tag "$IMAGE_REF" \
    --label "org.opencontainers.image.source=${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}" \
    --label "org.opencontainers.image.revision=${SOURCE_SHA}" \
    --label "org.semoss.base.maven=${MAVEN_IMAGE}" \
    --label "org.semoss.base.ubi=${UBI_IMAGE}" \
    --label "org.semoss.base.python=${PYTHON_IMAGE}" \
    "${secrets[@]}" .
    ) checks out IL4-dev and goes straight from resolving base images to building the candidate. The current [IL4-dev Dockerfile](
    # EXPERIMENTAL: built from SEMOSS/Monolith dev HEAD (FIPS-compliant PBKDF2 password hashing,
    # commit 612f0c9, not yet in any published release). CI produces dev-build-artifacts/ with
    # build-dev-artifacts.Dockerfile before this Dockerfile ever runs (see il4-container.yml's
    # "Build dev-HEAD Monolith/semosshome artifacts" step); a local build must populate that
    # directory the same way first. semossweb remains pinned to the published 5.4.0 release.
    COPY dev-build-artifacts/ /build/dev-build-artifacts/
    # Maven output isn't byte reproducible between builds of the same commit (javadoc
    # timestamps, archive member ordering), so these 3 coordinates can't carry a fixed
    # sha256 in artifacts.lock.json the way an externally fetched release artifact can;
    # pin_dev_artifacts.py repins them to what was actually just built, immediately
    # before assemble.py's resolve() enforces an exact match on every coordinate.
    RUN python3 pin_dev_artifacts.py
    ) requires generated dev-build-artifacts/, which is absent from the branch. IL4-dev’s own workflow now creates those artifacts in two additional steps. GitHub runs scheduled workflows from the default branch, so the PR’s older workflow will be used for Monday runs. [GitHub Actions documentation](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#schedule) Port those preparation steps into this PR before merging.

  2. [P2] The README describes the wrong application baseline. [It says the build does not advance the “locked SEMOSS release”](

    Semoss/README.md

    Lines 76 to 78 in 25b3cb8

    Each run refreshes the digests behind the existing Iron Bank Maven `3.9.16`,
    UBI `10.2`, and Python `v3.14` tags and builds without cache. It does not
    automatically advance version tags or the locked SEMOSS release.
    ). Current IL4-dev instead [builds SEMOSS from that branch and Monolith from a pinned commit](
    FROM ${MAVEN_IMAGE} AS semoss-build
    USER 0
    WORKDIR /build
    COPY . .
    RUN --mount=type=secret,id=maven_settings,target=/root/.m2/settings.xml \
    mvn -B -ntp -P deploy clean install -DskipTests -Dgpg.skip=true
    FROM ${MAVEN_IMAGE} AS monolith-build
    USER 0
    WORKDIR /build
    COPY --from=semoss-build /root/.m2 /root/.m2
    COPY --from=monolith-src . .
    RUN --mount=type=secret,id=maven_settings,target=/root/.m2/settings.xml \
    mvn -B -ntp -P deploy,fips clean install -DskipTests -Dgpg.skip=true
    FROM scratch AS artifacts
    COPY --from=semoss-build /root/.m2/repository/org/semoss/semoss/0.0.1-SNAPSHOT/semoss-0.0.1-SNAPSHOT-semosshome.tar.gz /
    COPY --from=monolith-build /root/.m2/repository/org/semoss/monolith/0.0.1-SNAPSHOT/monolith-0.0.1-SNAPSHOT.war /
    COPY --from=monolith-build /root/.m2/repository/org/semoss/monolith/0.0.1-SNAPSHOT/monolith-0.0.1-SNAPSHOT-libraries.tar.gz /
    ). Update the provenance description so operators know which application sources the image contains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants