Skip to content

fix: align alpha scope and add stable release validation gates - #23

Merged
Jairogelpi merged 1 commit into
mainfrom
fix/stable-release-readiness
Sep 15, 2026
Merged

Jairogelpi merged 1 commit into
mainfrom
fix/stable-release-readiness

Conversation

@Jairogelpi

@Jairogelpi Jairogelpi commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

Why

The README described a broader product than the available validation supports, and the installed package reported conflicting Python and CLI versions.

What changed

  • Explicit alpha positioning and implemented boundaries replace the unsupported competitor capability table; historical benchmark results are distinguished from unproven tool-integration benefit.
  • New VALIDATION.md records source-linked evidence and limitations. RELEASE_CRITERIA.md defines prospective independent evaluation, held-out safety, agent benefit and pilot gates.
  • The external protocol specifies independent labels, dataset-level holdout, error accounting and paired real-model evaluation. Those evaluations remain pending; the current harness is explicitly not claimed to implement the full independent protocol.
  • CI covers Linux/Python 3.11 and 3.12 plus Windows/Python 3.12, including external smoke cases and clean-wheel checks outside the checkout.
  • FR-CLI-001: public version now agrees with distribution/CLI 0.3.0, protected by a regression test. CHANGELOG and traceability are updated.

Evidence

  • New version regression failed before the fix and passed afterward.
  • make verify: 216 tests passed, 96.01% branch-aware coverage; Ruff, strict mypy, 21 schemas and 119 traceability entries passed.
  • python -m build and python scripts/check_distribution.py passed locally, including isolated wheel install, version/CLI/MCP checks and 112 mutation pairs.
  • GitHub clean-wheel package jobs passed on all three OS/Python combinations; complete quality jobs still running at this update: https://github.com/Jairogelpi/answerable_data/actions/runs/34962566570
  • Remote tree exactly matches the locally tested tree 53efd9576b432f0425ee5bea8eded26706a9aeb6.

Risks and rollback

No public schema or verdict-rule change; no quality threshold lowered. Windows coverage can expose pre-existing portability defects. Prospective criteria are not completed validations and do not establish stable readiness. No release is published. Revert this commit to roll back the documentation, CI and version correction.

@Jairogelpi
Jairogelpi marked this pull request as ready for review September 15, 2026 11:22
Copilot AI lite review requested due to automatic review settings September 15, 2026 11:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Windows external-evaluation checks fail due to line endings, and FR-CLI-001 traceability should remain verified.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR narrows alpha scope, documents validation and release criteria, aligns version reporting, and expands CI/package verification.

Changes:

  • Adds validation evidence, limitations, and stable-release gates.
  • Aligns public, CLI, and distribution versions at 0.3.0.
  • Adds isolated wheel checks and Linux/Windows CI coverage.
File summaries
File Summary
tests/test_distribution_version.py Tests version consistency.
src/answerable/__init__.py Updates the public version to 0.3.0.
scripts/check_distribution.py Validates isolated wheel installation and interfaces.
requirements/traceability.yaml Updates FR-CLI-001 traceability.
README.md Clarifies alpha scope and limitations.
docs/VALIDATION.md Records validation evidence and boundaries.
docs/RELEASE_CRITERIA.md Defines prospective release gates.
CHANGELOG.md Documents release-readiness changes.
benchmarks/external/README.md Specifies the independent evaluation protocol.
.github/workflows/ci.yml Adds OS/Python matrices and distribution checks.
Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/ci.yml
Comment on lines +24 to +28
os: [ubuntu-latest]
python-version: ["3.11", "3.12"]
include:
- os: windows-latest
python-version: "3.12"
status: verified
implementation: ["src/answerable/cli.py", "src/answerable/__init__.py"]
tests: ["tests/test_interfaces.py", "tests/test_distribution_version.py"]
status: implemented
@Jairogelpi
Jairogelpi merged commit c432104 into main Sep 15, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants