Repository navigation
Harden manifest endpoints, hook paths, cleanup, linked dirs and regex graders - #98
Merged
Merged
Conversation
… graders - openai-compatible: require https except to loopback, constrain the key variable name, and print the host and variable before the first request to a non-default host. - discover: a plugin manifest's hooks path must resolve inside the plugin. - reachability: remove only the scratch directory the measurement created. - statement: refuse a subject holding a link that resolves outside it, so the digest and the producers read the same bytes. - fastjudge: stop a regex grader after a time bound and end the run with a clear error instead of hanging. Each change has a test that failed before it. Signed-off-by: Jothi Mani Rajendran <250249270+jothimani-rajendran@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Name only the host in the first-request notice. - Test the regex deadline with an injected slow worker instead of a backtracking pattern literal. Signed-off-by: Jothi Mani Rajendran <250249270+jothimani-rajendran@users.noreply.github.com>
- schema: apiKeyEnv gets a name pattern and a description of the rule; baseUrl says https is required except for a loopback host. - spec README: the openai-compatible entry states both rules and the first-request notice. - A test checks the schema itself rejects a key variable outside the convention. Signed-off-by: Jothi Mani Rajendran <250249270+jothimani-rajendran@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Input-hardening for values a run or producer takes from a manifest or a subject (plan item SR-2c).
openai-compatiblemodels: https required except to a loopback host;apiKeyEnvmust be named like a provider key (^[A-Z][A-Z0-9_]*_KEY$); a stderr notice naming the host before the first request to a non-default host. Enforced in the backend constructor and at manifest load.hookspath must resolve inside the plugin.digest_path), so digest and rows read the same bytes.verifyreports it as a clean error.RunError.Spec
spec/run/v0.1/README.mdandschema.json(and the packaged copysrc/context_report/data/run-v0.1.schema.json) state thebaseUrlandapiKeyEnvrules; the schema now rejects anapiKeyEnvoutside the convention.Tests
Each change has a test that failed before it (
tests/test_hardening.py,tests/efficacy/test_openai_endpoint_hardening.py).ruff checkandruff format --checkclean; CI green.Notes