Skip to content

test: add future experimentation opt-out capability cases #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leoromanovsky merged 6 commits into from
Oct 22, 2026
Merged

test: add future experimentation opt-out capability cases #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
db86208
test: add future experimentation opt-out capability cases
leoromanovsky Oct 21, 2026
1c831fe
test: keep future opt-out fixture changes compact
leoromanovsky Oct 21, 2026
b288762
Merge main into future opt-out fixtures
leoromanovsky Oct 21, 2026
72bdaa4
test: cover ordinary allocations at opt-out boundaries
leoromanovsky Oct 22, 2026
92d7373
docs: explain canonical contract and downstream conformance
leoromanovsky Oct 22, 2026
8aba254
docs: clarify the actual opt-out time and later cutoffs
leoromanovsky Oct 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 34 additions & 6 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,22 @@

Thank you for your interest in contributing to the FFE system test data repository!

## Changing the contract

Read [contract ownership](README.md#contract-ownership) before changing an
expectation. These fixtures define behavior for downstream implementations;
they are not snapshots of whichever SDK or backend happens to be deployed.

Describe the rule being added or corrected, include a concrete input and expected
result, and explain the effect on consumers. Validate against actual native
implementations where available and record the tested revisions and capability
coverage. A consumer mismatch calls for investigation; it does not by itself
justify changing the expected result. Follow [handling a mismatch](README.md#handling-a-mismatch).

Keep deployment prerequisites, release versions, and temporary consumer workarounds
in downstream changes. Contract documentation should remain valid as those
projects ship support.

## Getting Started

1. Fork the repository
Expand Down Expand Up @@ -76,12 +92,24 @@ merge by itself.

## Updating Downstream Repositories

After your changes are merged to this repository, update the submodule references in downstream repos. Each downstream repository should:

1. Keep `ffe-system-test-data` as a git submodule adjacent to the OpenFeature/FFE tests
2. Configure Dependabot with `package-ecosystem: "gitsubmodule"` so fixture updates are proposed automatically
3. Load `ufc-config.json` and loop over every `evaluation-cases/*.json` file in unit tests
4. Avoid copied fixture directories and programmatic-only shared evaluator cases
After your changes are merged to this repository, update the pinned fixture
revision in downstream repos. Each rules-based consumer should:

1. Pin an exact fixture revision, normally with a git submodule adjacent to the
OpenFeature/FFE tests. Generated snapshots must retain the source repository
and commit SHA and be refreshed from that source.
2. Automate fixture updates. For submodules, configure Dependabot with
`package-ecosystem: "gitsubmodule"`; for snapshots, automate regeneration from
the source revision.
3. Load `ufc-config.json` and loop over every `evaluation-cases/*.json` file in
unit tests.
4. Run the cases for every implemented capability as well as the baseline;
follow the [runner requirements](README.md#implement-the-rules-based-runner).
5. Preserve shared expectations unchanged. Avoid hand-maintained copies and
programmatic-only replacements for shared evaluator cases.

Precomputed-assignment consumers use the same pinning and provenance rules with
the [precomputed fixture runner](precomputed-assignments/README.md).

Current downstream consumers are `system-tests`, `dd-trace-go`, `dd-trace-java`, `dd-trace-js`, `dd-trace-py`, `dd-trace-rb`, `dd-trace-dotnet`, `dd-trace-php`, `libdatadog`, and `openfeature-js-client`.

Expand Down
Loading
Loading