Repository navigation
feat: usage instructions and architecture drivers, with plugin eval tests - #14
Merged
Merged
Conversation
| - **Freshness**: the acceptable staleness of results, from a record arriving at the source to it showing in the result (seconds, minutes, hours, daily). This drives the architecture. If it is "realtime", say why minutes would not do. | ||
| - **API latency SLA**: per-query latency target (for example p99 < 200 ms) and request rate, average and peak. | ||
| - **Workload**: data volume and event rates, sustained and peak, expected growth, and what happens above peak. | ||
| - **Budget**: what running the pipeline may cost (per month), or the infrastructure it must fit. |
Collaborator
There was a problem hiding this comment.
In the agent side, we probably need to update the need a cost-calculation skill and
| - **Access**: public, authenticated, per-tenant? Which field carries the tenant. | ||
| - **Access**: which claim scopes each query, mutation and subscription (see Results). | ||
|
|
||
| ## Test data and acceptance criteria |
Collaborator
There was a problem hiding this comment.
In agent side, I guess we need to have some kind of mapping to meet the requirements set in the ## Test data and acceptance criteria. What I mean by mapping, which config we need to use while building the SQRL so that agent can apply those ones. We have a PR that focus on the cost optimization for some configs but it doesn't cover completely. For example, what should agent do if API latecy SLA is set as (p99 < 200 ms)?
8 tasks
HasanAlpCaferoglu
force-pushed
the
feat/usage_instructions
branch
from
November 10, 2026 20:09
9259567 to
2d0afc2
Compare
10 of 11 tasks
…se were added because we need those answers during planning to come up with the best design.
HasanAlpCaferoglu
force-pushed
the
feat/usage_instructions
branch
from
November 12, 2026 21:00
628a292 to
4916332
Compare
HasanAlpCaferoglu
marked this pull request as ready for review
November 12, 2026 21:00
HasanAlpCaferoglu
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
Add usage instructions to
start, and architecture drivers torequirements. Add the first behavior tests for the skills.startrequirementscheck (section 4) and hand-off (section 5) before planning.requirements## Open Questionsin the file.requirements-template.md## Resultssection with authorization, and a## Hard constraintssection. The API surface that the user defines counts as a hard constraint.deployandpromotedeploytells the user that a new deployment does not stop or pause existing ones.promotesays what promotion changes for APIs and data lake views, and asks for a clear confirmation.Tests
tests/claude-plugin-evals/: 8claude plugin evalcases forrequirements,startandpatch. See tests/claude-plugin-evals/README.md.datasqrl-agent.shlets thepatchcase run without Docker. The stub is used only in a temporary copy of the plugin.Version: 0.3.8. Main already has 0.3.7 from #18.
Area
scripts/datasqrl-agent.sh)scripts/datasqrl-cloud.sh)How did you test it?
.github/check-plugin.shpython -m pytest testsclaude plugin evalonskill-specific-tests/: 7 cases, 3 runs each,--threshold 0.8postgres-is-open-question) failed in 1 of 3 runsrun-stub-launcher-tests.shonstub-launcher-tests/: 1 case, 3 runsCommands:
Checklist
feat:,fix:ordocs:.github/check-plugin.shpasses