[sdojjy]

Scope

Address all seven open High alerts in the repository snapshot: #1, #3, #7, #19, #21, #25, #39.

Changes

• Replace vulnerable Flask/Werkzeug 2.0 pins in the optional Superset test profile with Flask >=3.1.3 and Werkzeug >=3.1.9. Use the compatible Superset 6.1 profile and modern Jinja2/MarkupSafe/WTForms requirements.
• Raise Black minimum to 26.3.1 (lock resolves 26.10.1).
• Correct the stale editable package name in Pipfile to tidbcloudlake-sqlalchemy and regenerate Pipfile.lock. Flask/Werkzeug are already absent from the current default/development lock; alerts build(deps): bump werkzeug from 2.0.3 to 3.1.6 in /tests #1/build(deps): bump flask from 2.0.3 to 3.1.3 in /tests #3/fix: initialize optional Amazon S3 parameters #7 refer to stale lockfile entries, not a currently installed default dependency.
• Make the existing integration workflow verify/sync the committed lockfile; add separate locked-unit and Superset compatibility/security jobs.
• Correct three pre-existing unit test assumptions: URL query ordering, initialized server version for view reflection, and current reflected column types/comments/query. Production dialect source is unchanged.

Validation

• pipenv verify; pipenv sync --dev passed.
• Locked SQLAlchemy 2.0 conformance profile: 10 unit tests passed (also verified against SQLAlchemy 2.1.6 locally).
• Superset 6.1.0 / SQLAlchemy 1.4.54 profile: 10 unit tests and 7 security/compatibility tests passed; all 154 installed packages passed dependency consistency checks.
• Actual Superset imports and Lake dialect discovery passed.
• Regression checks cover permanent-session cache Vary headers and multipart field limits. A Flask 2.2.4 negative control reproduced missing Vary: Cookie on permanent-session refresh; patched stack passes.
• New security tests are optional when Superset is not installed; the dedicated job installs the full profile and runs all seven tests.
• Git diff whitespace check and new test formatting passed.
• Alert package/manifest versions checked against GitHub first-patched versions; four active entries are patched and three stale entries are absent from the current lock.
• Live Lake integration is not claimed locally; the existing service-backed CI job remains required.

Compatibility: only the optional Superset TEST environment is moved to Superset 6.1/Python 3.11; the published library runtime requirements and dialect implementation are unchanged. Existing dependency-only PRs were not closed.

Not merged, published, or deployed. Dependabot closure requires merge and re-scan.

Integration infrastructure repair

The first full CI run failed before tests: the container bootstrap downloaded an HTTP 410 page as the MinIO executable. The suite performs no S3 object-storage I/O, so use the image documented filesystem backend instead and remove the unused MinIO port. Readiness now executes an authenticated SELECT 1 with bounded retries and checks the JSON result, rather than accepting any response from /v1/health. This changes only the isolated CI service, not production storage.

Conformance harness compatibility

The service-backed run then exposed an existing harness import of DeprecatedCompoundSelectTest, removed in SQLAlchemy 2.1. Constrain ONLY the Pipenv development/test environment to SQLAlchemy >=2.0,<2.1; leave the published library runtime requirement unchanged. Remove the obsolete jobs-less ci.yml placeholder (publishing already lives in release.yml) that produced invalid-workflow failures. Dedicated Superset tests still validate SQLAlchemy 1.4 compatibility.

Final CI result (head b94d464)

• Locked unit tests: PASS.
• Full Superset compatibility/security profile: PASS.
• Real service-backed conformance/integration suite: PASS — 697 passed, 688 skipped against Databend nightly v1.2.954 and SQLAlchemy 2.0.55.
• CLA: PASS.
• No production changes were made; this PR remains unmerged.