docs(guardrails): note BYOG and LLM-as-judge are not enabled on every tenant yet - #1868
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a documentation-only availability note to clarify that BYOG’s SDK pieces are available, but the required Admin UI configuration is still rolling out (Enterprise ring in 26.10), reducing tenant-level confusion without removing the BYOG docs.
Changes:
- Adds an
!!! infoadmonition under the “Bring Your Own Guardrail (BYOG)” section describing feature-flag/rollout status. - Adds a
<!-- REMOVE WHEN: ... -->cleanup hook comment to ensure the note is removed once rollout completes.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
apetraru-uipath
added a commit
to UiPath/uipath-langchain-python
that referenced
this pull request
Aug 24, 2026
Matches the retitle in UiPath/uipath-python#1868 so both pages of the same site use one label. "Version Availability" is taken, and means a uipath PyPI version floor — not a platform ring, which is what 26.10 is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apetraru-uipath
added a commit
to UiPath/uipath-langchain-python
that referenced
this pull request
Aug 24, 2026
Matches UiPath/uipath-python#1868. One note here, not two: this page has a single "### LLM-as-judge" heading (decorator pattern), and UiPathLLMAsJudgeMiddleware only appears in the middleware table, whose bullet already deep-links to the core page's #llm-as-judge anchor — which now carries the same note. The note names both classes so either reader is covered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apetraru-uipath
changed the title
apetraru-uipath
force-pushed
the
docs/byog-availability-note
branch
from
August 24, 2026 12:10
3098e17 to
5f5609e
Compare
apetraru-uipath
added a commit
to UiPath/uipath-langchain-python
that referenced
this pull request
Aug 24, 2026
…ing in 26.10 Pairs with UiPath/uipath-python#1868. This repo's docs/ is symlinked into the SDK site as docs/langchain by that repo's publish-docs.yml, so the guardrails guidance here is published alongside the core page and needs the same notes. The BYOG and LLM-as-judge sections read as if the features were live. Their SDK halves are — UiPathByoGuardrailMiddleware, UiPathLLMAsJudgeMiddleware, ByoValidator and LLMAsJudgeValidator all ship on PyPI — but the platform side of each is still behind a feature flag and reaches the Enterprise ring in 26.10. Three notes, not four. BYOG has a section under both the middleware and decorator patterns and both get one: the decorator section has its own anchor that people deep-link to, so a bare cross-reference was not enough. LLM-as-judge has a single heading (decorator pattern) — UiPathLLMAsJudgeMiddleware appears only in the middleware table, whose bullet already deep-links to the core page's #llm-as-judge anchor, which now carries the same note. That note names both classes so either reader is covered. Also drops the vaguer "if the Guardrails Configurations page is not available, BYOG is not enabled on your tenant yet" from the Admin prerequisite paragraph — the note now says that two paragraphs above, with a version attached. Merging this fires a repository_dispatch that republishes the whole site, so land the uipath-python PR first or alongside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… tenant yet Both sections read as if the features were live. Their SDK halves are — ByoValidator and LLMAsJudgeValidator ship on PyPI — but the platform side of each is still behind a feature flag and is not enabled on every tenant. A customer already hit this on BYOG. Adds a "Platform Availability" note to each section rather than removing the docs: they are correct for their audience (coded-agent developers consuming the PyPI package), they just never said where the boundary is. Versioned docs were already ruled out as too costly to maintain. No release number in the notes, on review feedback — a date we publish here is a date we have to hit, and this repo is public. The notes say the feature is not enabled everywhere yet and point at the product release notes instead. That reference is deliberately not a link to core/release_notes.md: that page tracks PyPI package releases (uipath, uipath-langchain, uipath-runtime) and will never say when a guardrail lands on a tenant. For BYOG the note also gives a self-service check that needs no date at all — whether the Guardrails Configurations page is in your Admin panel. The two notes are worded differently on purpose. BYOG's gate is one admin step — configuring a guardrail connection from a guardrail connection template — with the rest of the feature live. For LLM-as-judge the whole guardrail is gated, so its note says it is unavailable regardless of which judge models the governance policy permits; without that clause the "must be a model your governance policy allows" line right below reads as if a policy change fixes it. Title is "Platform Availability", not "Version Availability". The latter is taken and means a uipath PyPI version floor (2.1.168, 2.2.12, 2.2.13, package on 2.14.7). This page renders both kinds, so they have to be tellable apart. The REMOVE WHEN comments are the cleanup hook, one per feature since the two flags may not clear together. They carry no date either — HTML comments still ship in the published page source. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apetraru-uipath
force-pushed
the
docs/byog-availability-note
branch
from
August 24, 2026 13:29
5f5609e to
502beda
Compare
apetraru-uipath
added a commit
to UiPath/uipath-langchain-python
that referenced
this pull request
Aug 24, 2026
… tenant yet Pairs with UiPath/uipath-python#1868. This repo's docs/ is symlinked into the SDK site as docs/langchain by that repo's publish-docs.yml, so the guardrails guidance here is published alongside the core page and needs the same notes. The BYOG and LLM-as-judge sections read as if the features were live. Their SDK halves are — UiPathByoGuardrailMiddleware, UiPathLLMAsJudgeMiddleware, ByoValidator and LLMAsJudgeValidator all ship on PyPI — but the platform side of each is still behind a feature flag and is not enabled on every tenant. No release number in the notes, on review feedback — a date we publish here is a date we have to hit, and this repo is public. The notes say the feature is not enabled everywhere yet and point at the product release notes instead. Three notes, not four. BYOG has a section under both the middleware and decorator patterns and both get one: the decorator section has its own anchor that people deep-link to, so a bare cross-reference was not enough. LLM-as-judge has a single heading (decorator pattern) — UiPathLLMAsJudgeMiddleware appears only in the middleware table, whose bullet already deep-links to the core page's #llm-as-judge anchor, which now carries the same note. That note names both classes so either reader is covered. Also drops the vaguer "if the Guardrails Configurations page is not available, BYOG is not enabled on your tenant yet" from the Admin prerequisite paragraph — the note now says that two paragraphs above. Merging this fires a repository_dispatch that republishes the whole site, so land the uipath-python PR first or alongside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apetraru-uipath
changed the title
valentinabojan
approved these changes
Aug 24, 2026
|
apetraru-uipath
added a commit
to UiPath/uipath-langchain-python
that referenced
this pull request
Aug 24, 2026
… tenant yet Pairs with UiPath/uipath-python#1868. This repo's docs/ is symlinked into the SDK site as docs/langchain by that repo's publish-docs.yml, so the guardrails guidance here is published alongside the core page and needs the same notes. The BYOG and LLM-as-judge sections read as if the features were live. Their SDK halves are — UiPathByoGuardrailMiddleware, UiPathLLMAsJudgeMiddleware, ByoValidator and LLMAsJudgeValidator all ship on PyPI — but the platform side of each is still behind a feature flag and is not enabled on every tenant. No release number in the notes, on review feedback — a date we publish here is a date we have to hit, and this repo is public. The notes say the feature is not enabled everywhere yet and point at the product release notes instead. Three notes, not four. BYOG has a section under both the middleware and decorator patterns and both get one: the decorator section has its own anchor that people deep-link to, so a bare cross-reference was not enough. LLM-as-judge has a single heading (decorator pattern) — UiPathLLMAsJudgeMiddleware appears only in the middleware table, whose bullet already deep-links to the core page's #llm-as-judge anchor, which now carries the same note. That note names both classes so either reader is covered. Also drops the vaguer "if the Guardrails Configurations page is not available, BYOG is not enabled on your tenant yet" from the Admin prerequisite paragraph — the note now says that two paragraphs above. Merging this fires a repository_dispatch that republishes the whole site, so land the uipath-python PR first or alongside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apetraru-uipath
added a commit
to UiPath/skills
that referenced
this pull request
Aug 25, 2026
…lity Three coded-guardrail references tell the agent to WebFetch core/guardrails/ and langchain/guardrails/ and treat them as a source of truth. Those pages just gained "Platform Availability" notes saying BYOG and LLM-as-judge are not enabled on every tenant (UiPath/uipath-python#1868, UiPath/uipath-langchain-python#1045). Nothing told the agent that prose is product-wide, not a statement about the tenant in front of it. The risk is a false stop: an agent refusing to author, or filing a review finding, for a validator that `uip agent guardrails list` reports as Available on this tenant. guardrails-recommend.md is the sharpest case — in Validate mode it already calls the SDK docs "authoritative, sufficient" and demotes the tenant list to a non-mandatory cross-check. So: a precedence line in each of the three, placed where each file scopes what to take from the fetched pages. Deliberately no availability note of the docs' kind in any skill — skills resolve availability at runtime against the live tenant, and a static "not available yet" line would go stale the moment the flag flips and could cause the very refusal this guards against. Verified: skills:validate OK (26 skills, both flavors). Evals on Sonnet 5 (coder-eval 0.10.2, the version tests/.coder-eval-version pins) over the 17 tasks covering these three files — 15/17 pass. The two failures, coded-byog-middleware (0.000) and coded-byog-decorator (0.231), reproduce identically on unmodified main with the same flags: same scores, same checker messages. Pre-existing, not caused by this change, and worth its own look since a prior recorded Sonnet run had both at 1.0. Not fixed here, found while checking: getGuardrailDefinitions drops "FeatureDisabled" outright, so a flagged-off validator yields no list entry at all rather than a bad status. The coded authoring rule keys on Status != "Available" and so never fires for it, while the low-code twin handles absence explicitly. No skill reads the "FeatureDisabled" status the catalog does expose. Needs its own issue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gabrielavaduva
pushed a commit
to UiPath/skills
that referenced
this pull request
Aug 25, 2026
…lity (#2787) Three coded-guardrail references tell the agent to WebFetch core/guardrails/ and langchain/guardrails/ and treat them as a source of truth. Those pages just gained "Platform Availability" notes saying BYOG and LLM-as-judge are not enabled on every tenant (UiPath/uipath-python#1868, UiPath/uipath-langchain-python#1045). Nothing told the agent that prose is product-wide, not a statement about the tenant in front of it. The risk is a false stop: an agent refusing to author, or filing a review finding, for a validator that `uip agent guardrails list` reports as Available on this tenant. guardrails-recommend.md is the sharpest case — in Validate mode it already calls the SDK docs "authoritative, sufficient" and demotes the tenant list to a non-mandatory cross-check. So: a precedence line in each of the three, placed where each file scopes what to take from the fetched pages. Deliberately no availability note of the docs' kind in any skill — skills resolve availability at runtime against the live tenant, and a static "not available yet" line would go stale the moment the flag flips and could cause the very refusal this guards against. Verified: skills:validate OK (26 skills, both flavors). Evals on Sonnet 5 (coder-eval 0.10.2, the version tests/.coder-eval-version pins) over the 17 tasks covering these three files — 15/17 pass. The two failures, coded-byog-middleware (0.000) and coded-byog-decorator (0.231), reproduce identically on unmodified main with the same flags: same scores, same checker messages. Pre-existing, not caused by this change, and worth its own look since a prior recorded Sonnet run had both at 1.0. Not fixed here, found while checking: getGuardrailDefinitions drops "FeatureDisabled" outright, so a flagged-off validator yields no list entry at all rather than a bad status. The coded authoring rule keys on Status != "Available" and so never fires for it, while the low-code twin handles absence explicitly. No skill reads the "FeatureDisabled" status the catalog does expose. Needs its own issue. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The BYOG and LLM-as-judge sections read as if both features are live. Their SDK halves are —
ByoValidatorandLLMAsJudgeValidatorship on PyPI — but the platform side of each is still behind a feature flag and isn't enabled on every tenant. A customer already hit this on BYOG.Adds a
Platform Availabilitynote to each section instead of removing the docs. The docs are correct for their audience (coded-agent developers consuming the PyPI package) — they just never said where the boundary is. Removing them would break the audience they serve; versioned docs were already ruled out as too costly to maintain.No release number, on review feedback. A date published here is a date we then have to hit, and this repo is public. The notes say the feature isn't enabled everywhere yet and point at the product release notes instead. That reference is deliberately not a link to
core/release_notes.md— that page tracks PyPI package releases (uipath,uipath-langchain,uipath-runtime) and will never say when a guardrail lands on a tenant; linking it would repeat the exact SDK-version/platform-rollout conflation these notes exist to prevent. For BYOG the note also gives a self-service check that needs no date at all: whether the Guardrails Configurations page is in your Admin panel.The two notes are worded differently on purpose. BYOG's gate is one admin step — configuring a guardrail connection from a guardrail connection template under Admin → AI Trust Layer → Guardrails Configurations — with the rest of the feature live. For LLM-as-judge the whole guardrail is gated, so its note says it's unavailable regardless of which judge models the governance policy permits. Without that clause, the "must be a model your governance policy allows" line directly below reads as if a policy change fixes it.
Title:
Platform Availability, notVersion Availability. The latter is already taken and means auipathPyPI version floor — all three existing uses say "available starting from uipath version 2.1.168 / 2.2.12 / 2.2.13", and the package is on 2.14.7. This page renders both kinds of note, so they need to be tellable apart. See the review thread.Blast radius: additive, docs only. Two markdown blocks. No SDK or behavior change.
Cleanup: the
<!-- REMOVE WHEN: ... -->comments are the hook —grep -rn "REMOVE WHEN" docs/finds them. One per feature, since the two flags may not clear together. They carry no date either; HTML comments still ship in the published page source.Paired change: UiPath/uipath-langchain-python#1045 carries the same notes, since
publish-docs.ymlsymlinks that repo'sdocs/into this site asdocs/langchain. Merging it fires arepository_dispatchthat republishes the whole site, so land this PR first or alongside.Verified: built the full site locally with the langchain docs symlinked exactly the way
publish-docs.ymldoes. Confirmed in the rendered HTML that all notes come out as styledadmonition infoblocks titledPlatform Availability, that no literal!!!leaks, that the#llm-as-judgeand#bring-your-own-guardrail-byoganchors still resolve, and that no release number appears anywhere in the built page.--strictwas not used — it aborts on a pre-existinggoogle_tag_manager_idconfig warning unrelated to this change. Build produced no new warnings.🤖 Generated with Claude Code